Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled

Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
2026-07-05 09:36:54 +00:00
parent 3216769225
commit bedb527145
21108 changed files with 6010766 additions and 0 deletions

View File

@@ -0,0 +1,301 @@
// Google tests cover api plugin behavior.
import { describe, expect, it } from "vitest";
import {
isGoogleGenerativeAiApi,
isGoogleVertexBaseUrl,
isGoogleVertexHostname,
normalizeGoogleApiBaseUrl,
normalizeGoogleGenerativeAiBaseUrl,
normalizeGoogleProviderConfig,
parseGeminiAuth,
resolveGoogleGenerativeAiHttpRequestConfig,
resolveGoogleGenerativeAiApiOrigin,
resolveGoogleGenerativeAiTransport,
shouldNormalizeGoogleGenerativeAiProviderConfig,
} from "./api.js";
describe("google generative ai helpers", () => {
it("detects the Google Generative AI transport id", () => {
expect(isGoogleGenerativeAiApi("google-generative-ai")).toBe(true);
expect(isGoogleGenerativeAiApi("google-gemini-cli")).toBe(false);
expect(isGoogleGenerativeAiApi(undefined)).toBe(false);
});
it("normalizes only explicit Google Generative AI baseUrls", () => {
expect(normalizeGoogleGenerativeAiBaseUrl("https://generativelanguage.googleapis.com")).toBe(
"https://generativelanguage.googleapis.com/v1beta",
);
expect(normalizeGoogleGenerativeAiBaseUrl("https://proxy.example.com/google/v1beta")).toBe(
"https://proxy.example.com/google/v1beta",
);
expect(normalizeGoogleGenerativeAiBaseUrl("https://aiplatform.googleapis.com")).toBe(
"https://aiplatform.googleapis.com",
);
expect(normalizeGoogleGenerativeAiBaseUrl("proxy/generativelanguage.googleapis.com")).toBe(
"proxy/generativelanguage.googleapis.com",
);
expect(normalizeGoogleGenerativeAiBaseUrl("generativelanguage.googleapis.com")).toBe(
"generativelanguage.googleapis.com",
);
expect(normalizeGoogleGenerativeAiBaseUrl("https://xgenerativelanguage.googleapis.com")).toBe(
"https://xgenerativelanguage.googleapis.com",
);
expect(normalizeGoogleGenerativeAiBaseUrl()).toBeUndefined();
});
it("keeps /openai on generic Google base URL normalization and strips it only for native Gemini callers", () => {
expect(
normalizeGoogleApiBaseUrl("https://generativelanguage.googleapis.com/v1beta/openai"),
).toBe("https://generativelanguage.googleapis.com/v1beta/openai");
expect(
normalizeGoogleGenerativeAiBaseUrl("https://generativelanguage.googleapis.com/v1beta/openai"),
).toBe("https://generativelanguage.googleapis.com/v1beta");
expect(
normalizeGoogleGenerativeAiBaseUrl(
"https://generativelanguage.googleapis.com/v1alpha/openai/",
),
).toBe("https://generativelanguage.googleapis.com/v1alpha");
});
it("normalizes Google provider configs by provider key, provider api, or model api", () => {
expect(
shouldNormalizeGoogleGenerativeAiProviderConfig("google", {
models: [{ api: "openai-completions" }],
}),
).toBe(true);
expect(
shouldNormalizeGoogleGenerativeAiProviderConfig("custom", {
api: "google-generative-ai",
models: [{ api: "openai-completions" }],
}),
).toBe(true);
expect(
shouldNormalizeGoogleGenerativeAiProviderConfig("custom", {
models: [{ api: "google-generative-ai" }],
}),
).toBe(true);
expect(
shouldNormalizeGoogleGenerativeAiProviderConfig("custom", {
api: "openai-completions",
models: [{ api: "openai-completions" }],
}),
).toBe(false);
expect(
shouldNormalizeGoogleGenerativeAiProviderConfig("google", {
api: "openai-completions",
models: [{ api: "openai-completions" }],
}),
).toBe(false);
expect(
shouldNormalizeGoogleGenerativeAiProviderConfig("google-vertex", {
baseUrl: "https://aiplatform.googleapis.com",
}),
).toBe(false);
});
it("detects native Google Vertex hosts by hostname only", () => {
expect(isGoogleVertexHostname("aiplatform.googleapis.com")).toBe(true);
expect(isGoogleVertexHostname("us-central1-aiplatform.googleapis.com")).toBe(true);
expect(isGoogleVertexHostname("generativelanguage.googleapis.com")).toBe(false);
expect(isGoogleVertexHostname("evil-aiplatform.googleapis.com.attacker.com")).toBe(false);
expect(
isGoogleVertexBaseUrl(
"https://generativelanguage.googleapis.com/v1beta/proxy/aiplatform.googleapis.com",
),
).toBe(false);
});
it("normalizes transport baseUrls only for Google Generative AI", () => {
expect(
resolveGoogleGenerativeAiTransport({
provider: "google",
api: undefined,
baseUrl: "https://generativelanguage.googleapis.com",
}),
).toEqual({
api: "google-generative-ai",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
});
expect(
resolveGoogleGenerativeAiTransport({
api: "google-generative-ai",
baseUrl: "https://generativelanguage.googleapis.com",
}),
).toEqual({
api: "google-generative-ai",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
});
expect(
resolveGoogleGenerativeAiTransport({
api: "openai-completions",
baseUrl: "https://generativelanguage.googleapis.com",
}),
).toEqual({
api: "openai-completions",
baseUrl: "https://generativelanguage.googleapis.com",
});
expect(
resolveGoogleGenerativeAiTransport({
provider: "google-vertex",
api: undefined,
baseUrl: "https://us-central1-aiplatform.googleapis.com",
}),
).toEqual({
api: "google-vertex",
baseUrl: "https://us-central1-aiplatform.googleapis.com",
});
expect(
resolveGoogleGenerativeAiTransport({
provider: "google-vertex",
api: "openai-completions",
baseUrl:
"https://aiplatform.googleapis.com/v1/projects/test/locations/us-central1/endpoints/openapi",
}),
).toEqual({
api: "openai-completions",
baseUrl:
"https://aiplatform.googleapis.com/v1/projects/test/locations/us-central1/endpoints/openapi",
});
});
it("normalizes google-vertex model ids without rewriting the OpenAI-compatible baseUrl", () => {
expect(
normalizeGoogleProviderConfig("google-vertex", {
api: "openai-completions",
baseUrl:
"https://aiplatform.googleapis.com/v1/projects/test/locations/us-central1/endpoints/openapi",
models: [
{
id: "gemini-3.1-flash-lite",
name: "Gemini Flash Lite",
input: ["text"],
reasoning: false,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1,
maxTokens: 1,
},
],
}),
).toEqual({
api: "openai-completions",
baseUrl:
"https://aiplatform.googleapis.com/v1/projects/test/locations/us-central1/endpoints/openapi",
models: [
{
contextWindow: 1,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
id: "gemini-3.1-flash-lite",
input: ["text"],
maxTokens: 1,
name: "Gemini Flash Lite",
reasoning: false,
},
],
});
});
it("derives the Gemini API origin without duplicating /v1beta", () => {
expect(resolveGoogleGenerativeAiApiOrigin()).toBe("https://generativelanguage.googleapis.com");
expect(resolveGoogleGenerativeAiApiOrigin("https://generativelanguage.googleapis.com")).toBe(
"https://generativelanguage.googleapis.com",
);
expect(
resolveGoogleGenerativeAiApiOrigin("https://generativelanguage.googleapis.com/v1beta"),
).toBe("https://generativelanguage.googleapis.com");
});
it("parses project-aware oauth auth payloads into bearer headers", () => {
expect(
parseGeminiAuth(JSON.stringify({ token: "oauth-token", projectId: "project-1" })),
).toEqual({
headers: {
Authorization: "Bearer oauth-token",
"Content-Type": "application/json",
},
});
});
it("falls back to API key headers for raw tokens", () => {
expect(parseGeminiAuth("api-key-123")).toEqual({
headers: {
"x-goog-api-key": "api-key-123",
"Content-Type": "application/json",
},
});
});
it("builds shared Google Generative AI HTTP request config", () => {
const oauthConfig = resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: JSON.stringify({ token: "oauth-token" }),
baseUrl: "https://generativelanguage.googleapis.com",
capability: "audio",
transport: "media-understanding",
});
expect(oauthConfig.baseUrl).toBe("https://generativelanguage.googleapis.com/v1beta");
expect(oauthConfig.allowPrivateNetwork).toBe(false);
expect(Object.fromEntries(new Headers(oauthConfig.headers).entries())).toEqual({
authorization: "Bearer oauth-token",
"content-type": "application/json",
});
const apiKeyConfig = resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: "api-key-123",
capability: "image",
transport: "http",
});
expect(apiKeyConfig.baseUrl).toBe("https://generativelanguage.googleapis.com/v1beta");
expect(apiKeyConfig.allowPrivateNetwork).toBe(false);
expect(Object.fromEntries(new Headers(apiKeyConfig.headers).entries())).toEqual({
"content-type": "application/json",
"x-goog-api-key": "api-key-123",
});
});
it("preserves explicit OpenAI-compatible Google endpoints during provider normalization", () => {
expect(
resolveGoogleGenerativeAiTransport({
api: "openai-completions",
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
}),
).toEqual({
api: "openai-completions",
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
});
});
it("strips URL credentials during Google base URL normalization", () => {
const normalized = normalizeGoogleApiBaseUrl(
"https://user:secret@generativelanguage.googleapis.com/v1beta/openai?x=1#frag",
);
expect(normalized).toBe("https://generativelanguage.googleapis.com/v1beta/openai");
});
it("rejects non-Google Gemini base URLs and honors explicit private-network opt-in", () => {
expect(() =>
resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: "api-key-123",
baseUrl: "https://proxy.example.com/v1beta",
capability: "image",
transport: "http",
}),
).toThrow("Google Generative AI baseUrl must use https://generativelanguage.googleapis.com");
expect(() =>
resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: "api-key-123",
baseUrl: "http://generativelanguage.googleapis.com/v1beta",
capability: "image",
transport: "http",
}),
).toThrow("Google Generative AI baseUrl must use https://generativelanguage.googleapis.com");
const config = resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: "api-key-123",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
capability: "image",
transport: "http",
request: { allowPrivateNetwork: true },
});
expect(config.allowPrivateNetwork).toBe(true);
});
});

94
extensions/google/api.ts Normal file
View File

@@ -0,0 +1,94 @@
// Google API module exposes the plugin public contract.
import {
resolveProviderHttpRequestConfig,
type ProviderRequestTransportOverrides,
} from "openclaw/plugin-sdk/provider-http";
import { parseGeminiAuth } from "./gemini-auth.js";
export { parseGeminiAuth };
export { applyGoogleGeminiModelDefault, GOOGLE_GEMINI_DEFAULT_MODEL } from "./onboard.js";
import {
DEFAULT_GOOGLE_API_BASE_URL,
normalizeGoogleGenerativeAiBaseUrl,
} from "./provider-policy.js";
export { normalizeAntigravityModelId, normalizeGoogleModelId } from "./model-id.js";
export {
createGoogleThinkingPayloadWrapper,
createGoogleThinkingStreamWrapper,
isGoogleGemini3FlashModel,
isGoogleGemini3ProModel,
isGoogleGemini3ThinkingLevelModel,
isGoogleThinkingRequiredModel,
resolveGoogleGemini3ThinkingLevel,
sanitizeGoogleThinkingPayload,
stripInvalidGoogleThinkingBudget,
type GoogleThinkingInputLevel,
type GoogleThinkingLevel,
} from "./thinking-api.js";
export {
buildGoogleGenerativeAiParams,
createGoogleGenerativeAiTransportStreamFn,
} from "./transport-stream.js";
export {
DEFAULT_GOOGLE_API_BASE_URL,
isGoogleGenerativeAiApi,
isGoogleVertexBaseUrl,
isGoogleVertexHostname,
normalizeGoogleApiBaseUrl,
normalizeGoogleGenerativeAiBaseUrl,
normalizeGoogleProviderConfig,
resolveGoogleGenerativeAiApiOrigin,
resolveGoogleGenerativeAiTransport,
shouldNormalizeGoogleGenerativeAiProviderConfig,
shouldNormalizeGoogleProviderConfig,
} from "./provider-policy.js";
export { buildGoogleGeminiCliProvider } from "./gemini-cli-provider.js";
export { buildGoogleProvider } from "./provider-registration.js";
type GoogleGenerativeAiRequestOverrides = ProviderRequestTransportOverrides & {
allowPrivateNetwork?: boolean;
};
function resolveTrustedGoogleGenerativeAiBaseUrl(baseUrl?: string): string {
const normalized =
normalizeGoogleGenerativeAiBaseUrl(baseUrl ?? DEFAULT_GOOGLE_API_BASE_URL) ??
DEFAULT_GOOGLE_API_BASE_URL;
let url: URL;
try {
url = new URL(normalized);
} catch {
throw new Error(
"Google Generative AI baseUrl must be a valid https URL on generativelanguage.googleapis.com",
);
}
if (
url.protocol !== "https:" ||
url.hostname.toLowerCase() !== "generativelanguage.googleapis.com"
) {
throw new Error(
"Google Generative AI baseUrl must use https://generativelanguage.googleapis.com",
);
}
return normalized;
}
export function resolveGoogleGenerativeAiHttpRequestConfig(params: {
apiKey: string;
baseUrl?: string;
headers?: Record<string, string>;
request?: GoogleGenerativeAiRequestOverrides;
capability: "image" | "audio" | "video";
transport: "http" | "media-understanding";
}) {
return resolveProviderHttpRequestConfig({
baseUrl: resolveTrustedGoogleGenerativeAiBaseUrl(params.baseUrl),
defaultBaseUrl: DEFAULT_GOOGLE_API_BASE_URL,
allowPrivateNetwork: params.request?.allowPrivateNetwork,
headers: params.headers,
request: params.request,
defaultHeaders: parseGeminiAuth(params.apiKey).headers,
provider: "google",
api: "google-generative-ai",
capability: params.capability,
transport: params.transport,
});
}

View File

@@ -0,0 +1,406 @@
import crypto from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import type { CliBackendPreparedExecution } from "openclaw/plugin-sdk/cli-backend";
import { isRecord } from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import {
GOOGLE_GEMINI_CLI_PROVIDER_ID,
resolveGeminiCliProfileHome as resolveGeminiCliProfileHomePath,
} from "./gemini-cli-auth-home.js";
const GEMINI_CLI_PROVIDER_ID = GOOGLE_GEMINI_CLI_PROVIDER_ID;
const GOOGLE_PROVIDER_ID = "google";
const VERCEL_AI_GATEWAY_PROVIDER_ID = "vercel-ai-gateway";
const GEMINI_CLI_CREDENTIALS_FILENAME = "gemini-credentials.json";
const GEMINI_CLI_GCA_AUTH_ENV = [
"GOOGLE_GENAI_USE_GCA",
"GOOGLE_CLOUD_ACCESS_TOKEN",
"GOOGLE_APPLICATION_CREDENTIALS",
"GEMINI_FORCE_ENCRYPTED_FILE_STORAGE",
"GEMINI_FORCE_FILE_STORAGE",
];
const GEMINI_CLI_API_KEY_AUTH_ENV = [
...GEMINI_CLI_GCA_AUTH_ENV,
"GOOGLE_GENAI_USE_VERTEXAI",
"GOOGLE_API_KEY",
"GOOGLE_CLOUD_PROJECT",
"GOOGLE_CLOUD_PROJECT_ID",
"GOOGLE_CLOUD_QUOTA_PROJECT",
"GOOGLE_CLOUD_LOCATION",
"GOOGLE_GEMINI_BASE_URL",
"GEMINI_CLI_CUSTOM_HEADERS",
"GEMINI_API_KEY_AUTH_MECHANISM",
];
const GEMINI_CLI_PROFILE_AUTH_ENV = [...GEMINI_CLI_API_KEY_AUTH_ENV, "GEMINI_API_KEY"];
const GEMINI_CLI_PROFILE_SETTINGS_ENV = ["GEMINI_CLI_SYSTEM_SETTINGS_PATH"];
type GeminiAuthProfileCredential = {
type: "api_key" | "oauth" | "token";
provider: string;
key?: string;
token?: string;
access?: string;
refresh?: string;
expires?: number;
idToken?: string;
projectId?: string;
};
type GeminiOAuthCredential = GeminiAuthProfileCredential & {
type: "oauth";
provider: typeof GEMINI_CLI_PROVIDER_ID;
access: string;
refresh: string;
expires: number;
};
type GeminiApiKeyCredential = GeminiAuthProfileCredential & {
type: "api_key";
provider: typeof GEMINI_CLI_PROVIDER_ID | typeof GOOGLE_PROVIDER_ID;
key: string;
};
type GeminiCliAuthHomeContext = {
agentDir?: string;
authProfileId?: string;
systemSettingsPath?: string;
};
type GeminiCliAuthSelectedType = "oauth-personal" | "gemini-api-key";
function normalizeString(value: string | undefined): string | undefined {
const trimmed = value?.trim();
return trimmed ? trimmed : undefined;
}
function throwUnsupportedGeminiCredential(credential: GeminiAuthProfileCredential): never {
if (credential.provider === VERCEL_AI_GATEWAY_PROVIDER_ID) {
throw new Error(
"Gemini CLI execution cannot use a vercel-ai-gateway auth profile. Use the OpenClaw vercel-ai-gateway provider instead.",
);
}
throw new Error("Gemini CLI execution requires a google-gemini-cli auth profile.");
}
function throwUnstageableSelectedGeminiProfile(
ctx: GeminiCliAuthHomeContext,
credential: GeminiAuthProfileCredential | undefined,
): never {
const authProfileId = normalizeString(ctx.authProfileId);
if (!authProfileId) {
throw new Error("Gemini CLI execution requires a selected auth profile.");
}
if (!credential) {
throw new Error(
"Gemini CLI auth profile was selected but no credential material was found. Re-authenticate with `openclaw models auth login --provider google-gemini-cli --force`.",
);
}
if (credential.provider !== GEMINI_CLI_PROVIDER_ID) {
throwUnsupportedGeminiCredential(credential);
}
throw new Error(
"Gemini CLI execution supports google-gemini-cli OAuth or API-key auth profiles. Re-authenticate with `openclaw models auth login --provider google-gemini-cli --force`.",
);
}
function requireGeminiOAuthCredential(
credential: GeminiAuthProfileCredential | undefined,
): GeminiOAuthCredential | null {
if (!credential) {
return null;
}
if (credential.type !== "oauth") {
return null;
}
if (credential.provider !== GEMINI_CLI_PROVIDER_ID) {
throwUnsupportedGeminiCredential(credential);
}
const access = normalizeString(credential.access);
const refresh = normalizeString(credential.refresh);
if (
!access ||
!refresh ||
typeof credential.expires !== "number" ||
!Number.isFinite(credential.expires)
) {
throw new Error(
"Gemini CLI OAuth profile is missing usable token material. Re-authenticate with `openclaw models auth login --provider google-gemini-cli --force`.",
);
}
return {
...credential,
type: "oauth",
provider: GEMINI_CLI_PROVIDER_ID,
access,
refresh,
expires: credential.expires,
idToken: normalizeString(credential.idToken),
projectId: normalizeString(credential.projectId),
};
}
function requireGeminiApiKeyCredential(
credential: GeminiAuthProfileCredential | undefined,
): GeminiApiKeyCredential | null {
if (!credential) {
return null;
}
if (credential.type !== "api_key") {
return null;
}
if (
credential.provider !== GEMINI_CLI_PROVIDER_ID &&
credential.provider !== GOOGLE_PROVIDER_ID
) {
throwUnsupportedGeminiCredential(credential);
}
const key = normalizeString(credential.key);
if (!key) {
throw new Error("Gemini CLI API-key profile is missing usable key material.");
}
return {
...credential,
type: "api_key",
provider: credential.provider,
key,
};
}
function resolveGeminiCliProfileHome(ctx: GeminiCliAuthHomeContext): {
home: string;
geminiDir: string;
} {
const agentDir = normalizeString(ctx.agentDir);
if (!agentDir) {
throw new Error("Gemini CLI auth profile execution requires an agent directory.");
}
const authProfileId = normalizeString(ctx.authProfileId);
if (!authProfileId) {
throw new Error("Gemini CLI auth profile execution requires a selected auth profile.");
}
const home = resolveGeminiCliProfileHomePath(agentDir, authProfileId);
return { home, geminiDir: path.join(home, ".gemini") };
}
function readGeminiAuthProfileCredential(
credential: unknown,
): GeminiAuthProfileCredential | undefined {
if (!isRecord(credential)) {
return undefined;
}
return credential as GeminiAuthProfileCredential;
}
async function readGeminiCliJsonObject(
filePath: string | undefined,
): Promise<Record<string, unknown>> {
const normalized = normalizeString(filePath);
if (!normalized) {
return {};
}
try {
const parsed = JSON.parse(await fs.readFile(normalized, "utf8")) as unknown;
if (!isRecord(parsed)) {
throw new Error(`Gemini CLI system settings must be a JSON object: ${normalized}`);
}
return { ...parsed };
} catch (error) {
if (
error &&
typeof error === "object" &&
"code" in error &&
(error as { code?: unknown }).code === "ENOENT"
) {
return {};
}
throw error;
}
}
function buildGeminiCliAuthSettings(
selectedType: GeminiCliAuthSelectedType,
): Record<string, unknown> {
return { security: { auth: { selectedType } } };
}
async function buildGeminiCliSystemSettings(
ctx: GeminiCliAuthHomeContext,
selectedType: GeminiCliAuthSelectedType,
): Promise<Record<string, unknown>> {
const base = await readGeminiCliJsonObject(ctx.systemSettingsPath);
const security = isRecord(base.security) ? { ...base.security } : {};
const auth = isRecord(security.auth) ? { ...security.auth } : {};
const enforcedType = normalizeString(
typeof auth.enforcedType === "string" ? auth.enforcedType : undefined,
);
if (enforcedType && enforcedType !== selectedType) {
throw new Error(
`Gemini CLI system settings enforce ${enforcedType} auth, but the selected OpenClaw profile requires ${selectedType}.`,
);
}
security.auth = { ...auth, selectedType };
return {
...base,
security,
};
}
async function writeGeminiCliJson(filePath: string, value: unknown): Promise<void> {
const tempPath = path.join(
path.dirname(filePath),
`.${path.basename(filePath)}.${process.pid}.${crypto.randomUUID()}.tmp`,
);
await fs.writeFile(tempPath, `${JSON.stringify(value, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});
await fs.chmod(tempPath, 0o600);
await fs.rename(tempPath, filePath);
await fs.chmod(filePath, 0o600);
}
async function prepareGeminiCliProfileHome(
ctx: GeminiCliAuthHomeContext,
selectedType: GeminiCliAuthSelectedType,
): Promise<{
home: string;
geminiDir: string;
systemSettingsPath: string;
beforeExecution: () => Promise<void>;
cleanup: () => Promise<void>;
}> {
const { home, geminiDir } = resolveGeminiCliProfileHome(ctx);
const settings = buildGeminiCliAuthSettings(selectedType);
const systemSettings = await buildGeminiCliSystemSettings(ctx, selectedType);
const systemSettingsDir = await fs.mkdtemp(
path.join(resolvePreferredOpenClawTmpDir(), "openclaw-gemini-cli-"),
);
await fs.chmod(systemSettingsDir, 0o700);
const systemSettingsPath = path.join(systemSettingsDir, "settings.json");
return {
home,
geminiDir,
systemSettingsPath,
beforeExecution: async () => {
await fs.mkdir(geminiDir, { recursive: true, mode: 0o700 });
await fs.chmod(home, 0o700);
await fs.chmod(geminiDir, 0o700);
await Promise.all([
writeGeminiCliJson(path.join(geminiDir, "settings.json"), settings),
writeGeminiCliJson(path.join(home, "settings.json"), settings),
writeGeminiCliJson(systemSettingsPath, systemSettings),
]);
},
cleanup: async () => {
await fs.rm(systemSettingsDir, { recursive: true, force: true });
},
};
}
async function clearGeminiCliCachedCredentials(geminiDir: string): Promise<void> {
// Gemini prefers its token store over oauth_creds.json. Rebuild that store
// from the selected OpenClaw profile each run so stale CLI auth cannot win.
await fs.rm(path.join(geminiDir, GEMINI_CLI_CREDENTIALS_FILENAME), { force: true });
}
function buildGeminiCliProjectEnv(projectId: string | undefined): Record<string, string> {
const normalized = normalizeString(projectId);
if (!normalized) {
return {};
}
return {
GOOGLE_CLOUD_PROJECT: normalized,
GOOGLE_CLOUD_PROJECT_ID: normalized,
GOOGLE_CLOUD_QUOTA_PROJECT: normalized,
};
}
async function prepareGeminiCliOAuthHome(
ctx: GeminiCliAuthHomeContext,
credential: GeminiAuthProfileCredential | undefined,
): Promise<CliBackendPreparedExecution | null> {
const oauth = requireGeminiOAuthCredential(credential);
if (!oauth) {
return null;
}
const profileHome = await prepareGeminiCliProfileHome(ctx, "oauth-personal");
const idToken = normalizeString(oauth.idToken);
const oauthCreds: Record<string, string | number> = {
access_token: oauth.access,
refresh_token: oauth.refresh,
expiry_date: oauth.expires,
token_type: "Bearer",
};
if (idToken) {
oauthCreds.id_token = idToken;
}
return {
env: {
GEMINI_CLI_HOME: profileHome.home,
GEMINI_CLI_SYSTEM_SETTINGS_PATH: profileHome.systemSettingsPath,
GEMINI_FORCE_FILE_STORAGE: "true",
...buildGeminiCliProjectEnv(oauth.projectId),
},
clearEnv: [...GEMINI_CLI_PROFILE_AUTH_ENV, ...GEMINI_CLI_PROFILE_SETTINGS_ENV],
beforeExecution: async () => {
await profileHome.beforeExecution();
await clearGeminiCliCachedCredentials(profileHome.geminiDir);
await writeGeminiCliJson(path.join(profileHome.geminiDir, "oauth_creds.json"), oauthCreds);
},
cleanup: profileHome.cleanup,
};
}
async function prepareGeminiCliApiKeyHome(
ctx: GeminiCliAuthHomeContext,
credential: GeminiAuthProfileCredential | undefined,
): Promise<CliBackendPreparedExecution | null> {
const apiKey = requireGeminiApiKeyCredential(credential);
if (!apiKey) {
return null;
}
const profileHome = await prepareGeminiCliProfileHome(ctx, "gemini-api-key");
return {
env: {
GEMINI_CLI_HOME: profileHome.home,
GEMINI_CLI_SYSTEM_SETTINGS_PATH: profileHome.systemSettingsPath,
GEMINI_FORCE_FILE_STORAGE: "true",
GEMINI_API_KEY: apiKey.key,
},
clearEnv: [...GEMINI_CLI_PROFILE_AUTH_ENV, ...GEMINI_CLI_PROFILE_SETTINGS_ENV],
beforeExecution: async () => {
await profileHome.beforeExecution();
await Promise.all([
fs.rm(path.join(profileHome.geminiDir, "oauth_creds.json"), { force: true }),
clearGeminiCliCachedCredentials(profileHome.geminiDir),
]);
},
cleanup: profileHome.cleanup,
};
}
export async function prepareGeminiCliAuthHome(
ctx: GeminiCliAuthHomeContext,
credential: unknown,
): Promise<CliBackendPreparedExecution | null> {
const authCredential = readGeminiAuthProfileCredential(credential);
const prepared =
(await prepareGeminiCliOAuthHome(ctx, authCredential)) ??
(await prepareGeminiCliApiKeyHome(ctx, authCredential));
if (prepared) {
return prepared;
}
if (normalizeString(ctx.authProfileId)) {
throwUnstageableSelectedGeminiProfile(ctx, authCredential);
}
return null;
}

View File

@@ -0,0 +1,129 @@
import type { CliBackendPlugin } from "openclaw/plugin-sdk/cli-backend";
import {
CLI_FRESH_WATCHDOG_DEFAULTS,
CLI_RESUME_WATCHDOG_DEFAULTS,
} from "openclaw/plugin-sdk/cli-backend";
const GEMINI_MODEL_ALIASES: Record<string, string> = {
pro: "gemini-3.1-pro-preview",
flash: "gemini-3.1-flash-preview",
"flash-lite": "gemini-3.1-flash-lite",
};
const GEMINI_CLI_DEFAULT_MODEL_REF = "google-gemini-cli/gemini-3-flash-preview";
type GeminiCliBackendConfig = CliBackendPlugin["config"];
type GeminiCliOutputMode = NonNullable<GeminiCliBackendConfig["output"]>;
function mapGeminiCliOutputFormat(value: string | undefined): GeminiCliOutputMode | undefined {
if (value === "stream-json") {
return "jsonl";
}
if (value === "json" || value === "text") {
return value;
}
return undefined;
}
function readGeminiCliOutputFormat(args: readonly string[] | undefined): GeminiCliOutputMode {
for (let index = 0; index < (args?.length ?? 0); index += 1) {
const arg = args?.[index];
if (arg === "--output-format" || arg === "-o") {
return mapGeminiCliOutputFormat(args?.[index + 1]) ?? "text";
}
const inline = arg?.startsWith("--output-format=")
? arg.slice("--output-format=".length)
: arg?.startsWith("-o=")
? arg.slice("-o=".length)
: undefined;
const mapped = mapGeminiCliOutputFormat(inline);
if (mapped) {
return mapped;
}
}
return "text";
}
function normalizeGeminiCliBackendConfig(config: GeminiCliBackendConfig): GeminiCliBackendConfig {
const output = readGeminiCliOutputFormat(config.args);
const resumeOutput = readGeminiCliOutputFormat(config.resumeArgs ?? config.args);
const usesStreamJson = output === "jsonl" || resumeOutput === "jsonl";
return {
...config,
output,
resumeOutput,
jsonlDialect: usesStreamJson ? "gemini-stream-json" : undefined,
};
}
export function buildGoogleGeminiCliBackend(): CliBackendPlugin {
return {
id: "google-gemini-cli",
modelProvider: "google",
liveTest: {
defaultModelRef: GEMINI_CLI_DEFAULT_MODEL_REF,
defaultImageProbe: true,
defaultMcpProbe: true,
docker: {
npmPackage: "@google/gemini-cli",
binaryName: "gemini",
},
},
bundleMcp: true,
bundleMcpMode: "gemini-system-settings",
nativeToolMode: "always-on",
authEpochMode: "profile-only",
normalizeConfig: normalizeGeminiCliBackendConfig,
prepareExecution: async (ctx) => {
const { prepareGeminiCliAuthHome } = await import("./cli-backend-auth.runtime.js");
return await prepareGeminiCliAuthHome(
{
agentDir: ctx.agentDir,
authProfileId: ctx.authProfileId,
systemSettingsPath:
(ctx as typeof ctx & { env?: Record<string, string> }).env
?.GEMINI_CLI_SYSTEM_SETTINGS_PATH ?? process.env.GEMINI_CLI_SYSTEM_SETTINGS_PATH,
},
(ctx as typeof ctx & { authCredential?: unknown }).authCredential,
);
},
config: {
command: "gemini",
args: [
"--skip-trust",
"--approval-mode",
"auto_edit",
"--output-format",
"stream-json",
"--prompt",
"{prompt}",
],
resumeArgs: [
"--skip-trust",
"--approval-mode",
"auto_edit",
"--resume",
"{sessionId}",
"--output-format",
"stream-json",
"--prompt",
"{prompt}",
],
output: "jsonl",
input: "arg",
jsonlDialect: "gemini-stream-json",
imageArg: "@",
imagePathScope: "workspace",
modelArg: "--model",
modelAliases: GEMINI_MODEL_ALIASES,
sessionMode: "existing",
sessionIdFields: ["session_id", "sessionId"],
reliability: {
watchdog: {
fresh: { ...CLI_FRESH_WATCHDOG_DEFAULTS },
resume: { ...CLI_RESUME_WATCHDOG_DEFAULTS },
},
},
serialize: true,
},
};
}

View File

@@ -0,0 +1,116 @@
// Google tests cover default model plugin behavior.
import type { OpenClawConfig } from "openclaw/plugin-sdk/provider-onboard";
import { describe, expect, it } from "vitest";
import { applyGoogleGeminiModelDefault, GOOGLE_GEMINI_DEFAULT_MODEL } from "./api.js";
describe("google default model", () => {
it("sets defaults when model is unset", () => {
const cfg: OpenClawConfig = { agents: { defaults: {} } };
const applied = applyGoogleGeminiModelDefault(cfg);
expect(applied.changed).toBe(true);
expect(applied.next.agents?.defaults?.model).toEqual({ primary: GOOGLE_GEMINI_DEFAULT_MODEL });
});
it("overrides existing models", () => {
const applied = applyGoogleGeminiModelDefault({
agents: { defaults: { model: { primary: "anthropic/claude-opus-4-6" } } },
} as OpenClawConfig);
expect(applied.changed).toBe(true);
expect(applied.next.agents?.defaults?.model).toEqual({ primary: GOOGLE_GEMINI_DEFAULT_MODEL });
});
it("normalizes retired Gemini model map keys when applying the default", () => {
const applied = applyGoogleGeminiModelDefault({
agents: {
defaults: {
model: {
primary: "google/gemini-3-pro-preview",
fallbacks: ["google/gemini-3-pro-preview"],
},
models: {
"google/gemini-3-pro-preview": { alias: "gemini" },
},
},
},
} as OpenClawConfig);
expect(applied.changed).toBe(true);
expect(applied.next.agents?.defaults?.model).toEqual({
primary: "google/gemini-3.1-pro-preview",
fallbacks: ["google/gemini-3.1-pro-preview"],
});
expect(applied.next.agents?.defaults?.models).toEqual({
"google/gemini-3.1-pro-preview": { alias: "gemini" },
});
});
it("normalizes retired Gemini model maps even when the primary is already current", () => {
const applied = applyGoogleGeminiModelDefault({
agents: {
defaults: {
model: {
primary: GOOGLE_GEMINI_DEFAULT_MODEL,
fallbacks: ["google/gemini-3-pro-preview"],
},
models: {
"google/gemini-3-pro-preview": { alias: "gemini" },
},
},
},
} as OpenClawConfig);
expect(applied.changed).toBe(true);
expect(applied.next.agents?.defaults?.model).toEqual({
primary: GOOGLE_GEMINI_DEFAULT_MODEL,
fallbacks: [GOOGLE_GEMINI_DEFAULT_MODEL],
});
expect(applied.next.agents?.defaults?.models).toEqual({
[GOOGLE_GEMINI_DEFAULT_MODEL]: { alias: "gemini" },
});
});
it("normalizes retired Gemini provider catalog rows when the primary is already current", () => {
const applied = applyGoogleGeminiModelDefault({
agents: {
defaults: {
model: {
primary: GOOGLE_GEMINI_DEFAULT_MODEL,
},
},
},
models: {
providers: {
google: {
api: "google-generative-ai",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
models: [
{
id: "google/gemini-3-pro-preview",
name: "Gemini 3 Pro",
contextWindow: 1_000_000,
maxTokens: 8192,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
reasoning: true,
},
],
},
},
},
} as OpenClawConfig);
expect(applied.changed).toBe(true);
expect(applied.next.models?.providers?.google?.models?.map((model) => model.id)).toEqual([
GOOGLE_GEMINI_DEFAULT_MODEL,
]);
});
it("no-ops when already on the target default", () => {
const cfg = {
agents: { defaults: { model: { primary: GOOGLE_GEMINI_DEFAULT_MODEL } } },
} as OpenClawConfig;
const applied = applyGoogleGeminiModelDefault(cfg);
expect(applied.changed).toBe(false);
expect(applied.next).toEqual(cfg);
});
});

View File

@@ -0,0 +1,19 @@
// Google API module exposes the plugin public contract.
import type { DoctorSessionRouteStateOwner } from "openclaw/plugin-sdk/runtime-doctor";
export const sessionRouteStateOwners: DoctorSessionRouteStateOwner[] = [
{
id: "google",
label: "Google",
providerIds: ["google", "google-antigravity", "google-gemini-cli", "google-vertex"],
runtimeIds: ["google-gemini-cli"],
cliSessionKeys: ["google-gemini-cli", "gemini-cli"],
authProfilePrefixes: [
"google:",
"google-antigravity:",
"google-gemini-cli:",
"google-vertex:",
"gemini-cli:",
],
},
];

View File

@@ -0,0 +1,246 @@
// Google tests cover embedding batch bounded JSON response reads.
import { afterEach, describe, expect, it, vi } from "vitest";
import { runGeminiEmbeddingBatches } from "./embedding-batch.js";
import type { GeminiEmbeddingClient } from "./embedding-provider.js";
// Pass-through so onResponse receives real Response objects (required by
// readProviderJsonResponse which needs a real .body ReadableStream).
vi.mock("openclaw/plugin-sdk/memory-core-host-engine-embeddings", async (importOriginal) => {
const actual =
await importOriginal<typeof import("openclaw/plugin-sdk/memory-core-host-engine-embeddings")>();
return {
...actual,
withRemoteHttpResponse: async <T>(params: {
url: string;
ssrfPolicy?: unknown;
init?: RequestInit;
onResponse: (response: Response) => Promise<T>;
}): Promise<T> => {
const response = await fetch(params.url, params.init);
return await params.onResponse(response);
},
};
});
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
function fetchInputUrl(input: RequestInfo | URL): string {
if (typeof input === "string") {
return input;
}
if (input instanceof URL) {
return input.href;
}
return input.url;
}
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { "Content-Type": "application/json" },
});
}
function makeGeminiClient(): GeminiEmbeddingClient {
return {
baseUrl: "https://gemini-compatible.example/v1beta",
model: "text-embedding-004",
modelPath: "models/text-embedding-004",
headers: { "x-goog-api-key": "test-key" },
apiKeys: ["test-key"],
ssrfPolicy: undefined,
};
}
type GeminiBatchRequest = Parameters<typeof runGeminiEmbeddingBatches>[0]["requests"][number];
function singleRequest(): GeminiBatchRequest[] {
return [
{
custom_id: "r0",
request: {
model: "models/text-embedding-004",
content: { parts: [{ text: "hello" }] },
taskType: "RETRIEVAL_DOCUMENT",
},
},
];
}
function makeOversizedResponse(): {
response: Response;
getReadCount: () => number;
wasCanceled: () => boolean;
} {
const chunkSize = 1024 * 1024;
const chunkCount = 20; // 20 MiB — over 16 MiB cap
let readCount = 0;
let canceled = false;
return {
response: new Response(
new ReadableStream<Uint8Array>({
pull(controller) {
if (readCount >= chunkCount) {
controller.close();
return;
}
readCount += 1;
controller.enqueue(new Uint8Array(chunkSize));
},
cancel() {
canceled = true;
},
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
),
getReadCount: () => readCount,
wasCanceled: () => canceled,
};
}
describe("Google embedding-batch bounded JSON reads", () => {
it("bounds oversized file-upload JSON response and cancels the stream", async () => {
const streamed = makeOversizedResponse();
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
if (fetchInputUrl(input).includes("/upload/")) {
return streamed.response;
}
return new Response("unexpected", { status: 500 });
}),
);
await expect(
runGeminiEmbeddingBatches({
gemini: makeGeminiClient(),
agentId: "main",
requests: singleRequest(),
wait: true,
concurrency: 1,
pollIntervalMs: 50,
timeoutMs: 5_000,
}),
).rejects.toThrow(/gemini\.batch-file-upload/);
expect(streamed.wasCanceled()).toBe(true);
expect(streamed.getReadCount()).toBeLessThan(20);
});
it("bounds oversized batch-create JSON response and cancels the stream", async () => {
const streamed = makeOversizedResponse();
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const url = fetchInputUrl(input);
if (url.includes("/upload/")) {
return jsonResponse({ name: "files/f-ok" });
}
if (url.includes(":asyncBatchEmbedContent")) {
return streamed.response;
}
return new Response("unexpected", { status: 500 });
}),
);
await expect(
runGeminiEmbeddingBatches({
gemini: makeGeminiClient(),
agentId: "main",
requests: singleRequest(),
wait: true,
concurrency: 1,
pollIntervalMs: 50,
timeoutMs: 5_000,
}),
).rejects.toThrow(/gemini\.batch-create/);
expect(streamed.wasCanceled()).toBe(true);
expect(streamed.getReadCount()).toBeLessThan(20);
});
it("bounds oversized batch-status poll JSON response and cancels the stream", async () => {
const streamed = makeOversizedResponse();
let statusCalled = false;
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const url = fetchInputUrl(input);
if (url.includes("/upload/")) {
return jsonResponse({ name: "files/f-ok" });
}
if (url.includes(":asyncBatchEmbedContent")) {
return jsonResponse({ name: "batches/b-0", state: "PENDING" });
}
if (url.includes("/batches/") && !statusCalled) {
statusCalled = true;
return streamed.response;
}
return new Response("unexpected", { status: 500 });
}),
);
await expect(
runGeminiEmbeddingBatches({
gemini: makeGeminiClient(),
agentId: "main",
requests: singleRequest(),
wait: true,
concurrency: 1,
pollIntervalMs: 50,
timeoutMs: 5_000,
}),
).rejects.toThrow(/gemini\.batch-status/);
expect(streamed.wasCanceled()).toBe(true);
expect(streamed.getReadCount()).toBeLessThan(20);
});
it("parses small responses on all three JSON paths correctly", async () => {
// Use a unit-length vector so sanitizeAndNormalizeEmbedding preserves values.
const outputLine = JSON.stringify({
key: "r0",
embedding: { values: [1, 0, 0] },
});
let statusCalled = false;
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const url = fetchInputUrl(input);
if (url.includes("/upload/")) {
return jsonResponse({ name: "files/f-ok" });
}
if (url.includes(":asyncBatchEmbedContent")) {
return jsonResponse({ name: "batches/b-0", state: "PENDING" });
}
if (url.includes("/batches/") && !statusCalled) {
statusCalled = true;
return jsonResponse({
name: "batches/b-0",
state: "SUCCEEDED",
outputConfig: { file: "files/out-0" },
});
}
if (url.includes(":download")) {
return new Response(outputLine, { status: 200 });
}
return new Response("unexpected", { status: 500 });
}),
);
const result = await runGeminiEmbeddingBatches({
gemini: makeGeminiClient(),
agentId: "main",
requests: singleRequest(),
wait: true,
concurrency: 1,
pollIntervalMs: 50,
timeoutMs: 5_000,
});
expect(result.get("r0")).toEqual([1, 0, 0]);
});
});

View File

@@ -0,0 +1,387 @@
// Google plugin module implements embedding batch behavior.
import crypto from "node:crypto";
import {
buildEmbeddingBatchGroupOptions,
runEmbeddingBatchGroups,
buildBatchHeaders,
debugEmbeddingsLog,
normalizeBatchBaseUrl,
sanitizeAndNormalizeEmbedding,
withRemoteHttpResponse,
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
import {
createProviderHttpError,
readProviderJsonResponse,
} from "openclaw/plugin-sdk/provider-http";
import { normalizeStringEntries } from "openclaw/plugin-sdk/string-coerce-runtime";
import type { GeminiEmbeddingClient, GeminiTextEmbeddingRequest } from "./embedding-provider.js";
type EmbeddingBatchExecutionParams = {
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
concurrency: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
};
type GeminiBatchRequest = {
custom_id: string;
request: GeminiTextEmbeddingRequest;
};
type GeminiBatchStatus = {
name?: string;
state?: string;
outputConfig?: { file?: string; fileId?: string };
metadata?: {
output?: {
responsesFile?: string;
};
};
error?: { message?: string };
};
type GeminiBatchOutputLine = {
key?: string;
custom_id?: string;
request_id?: string;
embedding?: { values?: number[] };
response?: {
embedding?: { values?: number[] };
error?: { message?: string };
};
error?: { message?: string };
};
const GEMINI_BATCH_MAX_REQUESTS = 50000;
function hashText(text: string): string {
return crypto.createHash("sha256").update(text).digest("hex");
}
function getGeminiUploadUrl(baseUrl: string): string {
if (baseUrl.includes("/v1beta")) {
return baseUrl.replace(/\/v1beta\/?$/, "/upload/v1beta");
}
return `${baseUrl.replace(/\/$/, "")}/upload`;
}
function buildGeminiUploadBody(params: { jsonl: string; displayName: string }): {
body: Blob;
contentType: string;
} {
const boundary = `openclaw-${hashText(params.displayName)}`;
const jsonPart = JSON.stringify({
file: {
displayName: params.displayName,
mimeType: "application/jsonl",
},
});
const delimiter = `--${boundary}\r\n`;
const closeDelimiter = `--${boundary}--\r\n`;
const parts = [
`${delimiter}Content-Type: application/json; charset=UTF-8\r\n\r\n${jsonPart}\r\n`,
`${delimiter}Content-Type: application/jsonl; charset=UTF-8\r\n\r\n${params.jsonl}\r\n`,
closeDelimiter,
];
const body = new Blob([parts.join("")], { type: "multipart/related" });
return {
body,
contentType: `multipart/related; boundary=${boundary}`,
};
}
async function submitGeminiBatch(params: {
gemini: GeminiEmbeddingClient;
requests: GeminiBatchRequest[];
agentId: string;
}): Promise<GeminiBatchStatus> {
const baseUrl = normalizeBatchBaseUrl(params.gemini);
const jsonl = params.requests
.map((request) =>
JSON.stringify({
key: request.custom_id,
request: request.request,
}),
)
.join("\n");
const displayName = `memory-embeddings-${hashText(String(Date.now()))}`;
const uploadPayload = buildGeminiUploadBody({ jsonl, displayName });
const uploadUrl = `${getGeminiUploadUrl(baseUrl)}/files?uploadType=multipart`;
debugEmbeddingsLog("memory embeddings: gemini batch upload", {
uploadUrl,
baseUrl,
requests: params.requests.length,
});
const filePayload = await withRemoteHttpResponse({
url: uploadUrl,
ssrfPolicy: params.gemini.ssrfPolicy,
init: {
method: "POST",
headers: {
...buildBatchHeaders(params.gemini, { json: false }),
"Content-Type": uploadPayload.contentType,
},
body: uploadPayload.body,
},
onResponse: async (fileRes) => {
if (!fileRes.ok) {
const text = await fileRes.text();
throw new Error(`gemini batch file upload failed: ${fileRes.status} ${text}`);
}
return readProviderJsonResponse<{ name?: string; file?: { name?: string } }>(
fileRes,
"gemini.batch-file-upload",
);
},
});
const fileId = filePayload.name ?? filePayload.file?.name;
if (!fileId) {
throw new Error("gemini batch file upload failed: missing file id");
}
const batchBody = {
batch: {
displayName: `memory-embeddings-${params.agentId}`,
inputConfig: {
file_name: fileId,
},
},
};
const batchEndpoint = `${baseUrl}/${params.gemini.modelPath}:asyncBatchEmbedContent`;
debugEmbeddingsLog("memory embeddings: gemini batch create", {
batchEndpoint,
fileId,
});
return await withRemoteHttpResponse({
url: batchEndpoint,
ssrfPolicy: params.gemini.ssrfPolicy,
init: {
method: "POST",
headers: buildBatchHeaders(params.gemini, { json: true }),
body: JSON.stringify(batchBody),
},
onResponse: async (batchRes) => {
if (batchRes.ok) {
return readProviderJsonResponse<GeminiBatchStatus>(batchRes, "gemini.batch-create");
}
const text = await batchRes.text();
if (batchRes.status === 404) {
throw new Error(
"gemini batch create failed: 404 (asyncBatchEmbedContent not available for this model/baseUrl). Disable remote.batch.enabled or switch providers.",
);
}
throw new Error(`gemini batch create failed: ${batchRes.status} ${text}`);
},
});
}
async function fetchGeminiBatchStatus(params: {
gemini: GeminiEmbeddingClient;
batchName: string;
}): Promise<GeminiBatchStatus> {
const baseUrl = normalizeBatchBaseUrl(params.gemini);
const name = params.batchName.startsWith("batches/")
? params.batchName
: `batches/${params.batchName}`;
const statusUrl = `${baseUrl}/${name}`;
debugEmbeddingsLog("memory embeddings: gemini batch status", { statusUrl });
return await withRemoteHttpResponse({
url: statusUrl,
ssrfPolicy: params.gemini.ssrfPolicy,
init: {
headers: buildBatchHeaders(params.gemini, { json: true }),
},
onResponse: async (res) => {
if (!res.ok) {
throw await createProviderHttpError(res, "gemini batch status failed");
}
return readProviderJsonResponse<GeminiBatchStatus>(res, "gemini.batch-status");
},
});
}
async function fetchGeminiFileContent(params: {
gemini: GeminiEmbeddingClient;
fileId: string;
}): Promise<string> {
const baseUrl = normalizeBatchBaseUrl(params.gemini);
const file = params.fileId.startsWith("files/") ? params.fileId : `files/${params.fileId}`;
const downloadUrl = `${baseUrl}/${file}:download`;
debugEmbeddingsLog("memory embeddings: gemini batch download", { downloadUrl });
return await withRemoteHttpResponse({
url: downloadUrl,
ssrfPolicy: params.gemini.ssrfPolicy,
init: {
headers: buildBatchHeaders(params.gemini, { json: true }),
},
onResponse: async (res) => {
if (!res.ok) {
throw await createProviderHttpError(res, "gemini batch file content failed");
}
return await res.text();
},
});
}
function parseGeminiBatchOutput(text: string): GeminiBatchOutputLine[] {
if (!text.trim()) {
return [];
}
return normalizeStringEntries(text.split("\n")).map(
(line) => JSON.parse(line) as GeminiBatchOutputLine,
);
}
async function waitForGeminiBatch(params: {
gemini: GeminiEmbeddingClient;
batchName: string;
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
initial?: GeminiBatchStatus;
}): Promise<{ outputFileId: string }> {
const start = Date.now();
let current: GeminiBatchStatus | undefined = params.initial;
while (true) {
const status =
current ??
(await fetchGeminiBatchStatus({
gemini: params.gemini,
batchName: params.batchName,
}));
const state = status.state ?? "UNKNOWN";
if (["SUCCEEDED", "COMPLETED", "DONE"].includes(state)) {
const outputFileId =
status.outputConfig?.file ??
status.outputConfig?.fileId ??
status.metadata?.output?.responsesFile;
if (!outputFileId) {
throw new Error(`gemini batch ${params.batchName} completed without output file`);
}
return { outputFileId };
}
if (["FAILED", "CANCELLED", "CANCELED", "EXPIRED"].includes(state)) {
const message = status.error?.message ?? "unknown error";
throw new Error(`gemini batch ${params.batchName} ${state}: ${message}`);
}
if (!params.wait) {
throw new Error(`gemini batch ${params.batchName} still ${state}; wait disabled`);
}
if (Date.now() - start > params.timeoutMs) {
throw new Error(`gemini batch ${params.batchName} timed out after ${params.timeoutMs}ms`);
}
params.debug?.(`gemini batch ${params.batchName} ${state}; waiting ${params.pollIntervalMs}ms`);
await new Promise((resolve) => {
setTimeout(resolve, params.pollIntervalMs);
});
current = undefined;
}
}
export async function runGeminiEmbeddingBatches(
params: {
gemini: GeminiEmbeddingClient;
agentId: string;
requests: GeminiBatchRequest[];
} & EmbeddingBatchExecutionParams,
): Promise<Map<string, number[]>> {
return await runEmbeddingBatchGroups({
...buildEmbeddingBatchGroupOptions(params, {
maxRequests: GEMINI_BATCH_MAX_REQUESTS,
debugLabel: "memory embeddings: gemini batch submit",
}),
runGroup: async ({ group, groupIndex, groups, byCustomId, pollIntervalMs, timeoutMs }) => {
const batchInfo = await submitGeminiBatch({
gemini: params.gemini,
requests: group,
agentId: params.agentId,
});
const batchName = batchInfo.name ?? "";
if (!batchName) {
throw new Error("gemini batch create failed: missing batch name");
}
params.debug?.("memory embeddings: gemini batch created", {
batchName,
state: batchInfo.state,
group: groupIndex + 1,
groups,
requests: group.length,
});
if (
!params.wait &&
batchInfo.state &&
!["SUCCEEDED", "COMPLETED", "DONE"].includes(batchInfo.state)
) {
throw new Error(
`gemini batch ${batchName} submitted; enable remote.batch.wait to await completion`,
);
}
const completed =
batchInfo.state && ["SUCCEEDED", "COMPLETED", "DONE"].includes(batchInfo.state)
? {
outputFileId:
batchInfo.outputConfig?.file ??
batchInfo.outputConfig?.fileId ??
batchInfo.metadata?.output?.responsesFile ??
"",
}
: await waitForGeminiBatch({
gemini: params.gemini,
batchName,
wait: params.wait,
pollIntervalMs,
timeoutMs,
debug: params.debug,
initial: batchInfo,
});
if (!completed.outputFileId) {
throw new Error(`gemini batch ${batchName} completed without output file`);
}
const content = await fetchGeminiFileContent({
gemini: params.gemini,
fileId: completed.outputFileId,
});
const outputLines = parseGeminiBatchOutput(content);
const errors: string[] = [];
const remaining = new Set(group.map((request) => request.custom_id));
for (const line of outputLines) {
const customId = line.key ?? line.custom_id ?? line.request_id;
if (!customId) {
continue;
}
remaining.delete(customId);
if (line.error?.message) {
errors.push(`${customId}: ${line.error.message}`);
continue;
}
if (line.response?.error?.message) {
errors.push(`${customId}: ${line.response.error.message}`);
continue;
}
const embedding = sanitizeAndNormalizeEmbedding(
line.embedding?.values ?? line.response?.embedding?.values ?? [],
);
if (embedding.length === 0) {
errors.push(`${customId}: empty embedding`);
continue;
}
byCustomId.set(customId, embedding);
}
if (errors.length > 0) {
throw new Error(`gemini batch ${batchName} failed: ${errors.join("; ")}`);
}
if (remaining.size > 0) {
throw new Error(`gemini batch ${batchName} missing ${remaining.size} embedding responses`);
}
},
});
}

View File

@@ -0,0 +1,265 @@
// Google tests cover embedding provider plugin behavior.
import { afterEach, describe, expect, it, vi } from "vitest";
vi.mock("openclaw/plugin-sdk/memory-core-host-engine-embeddings", async (importOriginal) => {
const actual =
await importOriginal<typeof import("openclaw/plugin-sdk/memory-core-host-engine-embeddings")>();
return {
...actual,
withRemoteHttpResponse: (async <T>(params: {
url: string;
init?: RequestInit;
onResponse: (response: Response) => Promise<T>;
}): Promise<T> => {
const response = await fetch(params.url, params.init);
return await params.onResponse(response);
}) satisfies typeof actual.withRemoteHttpResponse,
};
});
import {
buildGeminiEmbeddingRequest,
buildGeminiTextEmbeddingRequest,
createGeminiEmbeddingProvider,
DEFAULT_GEMINI_EMBEDDING_MODEL,
GEMINI_EMBEDDING_2_MODELS,
isGeminiEmbedding2Model,
normalizeGeminiModel,
resolveGeminiOutputDimensionality,
} from "./embedding-provider.js";
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
function installFetchMock(
handler: (input: RequestInfo | URL, init?: RequestInit) => unknown,
): ReturnType<typeof vi.fn> {
const fetchMock = vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => {
return new Response(JSON.stringify(handler(input, init)), {
status: 200,
headers: { "Content-Type": "application/json" },
});
});
vi.stubGlobal("fetch", fetchMock);
return fetchMock;
}
function fetchJsonBody(fetchMock: ReturnType<typeof vi.fn>, index: number): unknown {
const init = fetchMock.mock.calls[index]?.[1] as RequestInit | undefined;
const body = init?.body;
if (typeof body !== "string") {
throw new Error("Expected JSON string request body.");
}
return JSON.parse(body) as unknown;
}
function requireFirstFetchInput(fetchMock: ReturnType<typeof vi.fn>): RequestInfo | URL {
const [call] = fetchMock.mock.calls;
if (!call) {
throw new Error("expected Gemini embedding fetch call");
}
return call[0] as RequestInfo | URL;
}
describe("Gemini embedding request helpers", () => {
it("builds requests and resolves model settings", () => {
expect(
buildGeminiTextEmbeddingRequest({
text: "hello",
taskType: "RETRIEVAL_DOCUMENT",
modelPath: "models/gemini-embedding-2-preview",
outputDimensionality: 1536,
}),
).toEqual({
model: "models/gemini-embedding-2-preview",
content: { parts: [{ text: "hello" }] },
taskType: "RETRIEVAL_DOCUMENT",
outputDimensionality: 1536,
});
expect(
buildGeminiEmbeddingRequest({
input: {
text: "Image file: diagram.png",
parts: [
{ type: "text", text: "Image file: diagram.png" },
{ type: "inline-data", mimeType: "image/png", data: "abc123" },
],
},
taskType: "RETRIEVAL_DOCUMENT",
modelPath: "models/gemini-embedding-2-preview",
outputDimensionality: 1536,
}),
).toEqual({
model: "models/gemini-embedding-2-preview",
content: {
parts: [
{ text: "Image file: diagram.png" },
{ inlineData: { mimeType: "image/png", data: "abc123" } },
],
},
taskType: "RETRIEVAL_DOCUMENT",
outputDimensionality: 1536,
});
expect(GEMINI_EMBEDDING_2_MODELS.has("gemini-embedding-2-preview")).toBe(true);
expect(isGeminiEmbedding2Model("gemini-embedding-2-preview")).toBe(true);
expect(isGeminiEmbedding2Model("gemini-embedding-001")).toBe(false);
expect(isGeminiEmbedding2Model("text-embedding-004")).toBe(false);
expect(resolveGeminiOutputDimensionality("gemini-embedding-001")).toBeUndefined();
expect(resolveGeminiOutputDimensionality("text-embedding-004")).toBeUndefined();
expect(resolveGeminiOutputDimensionality("gemini-embedding-2-preview")).toBe(3072);
expect(resolveGeminiOutputDimensionality("gemini-embedding-2-preview", 768)).toBe(768);
expect(resolveGeminiOutputDimensionality("gemini-embedding-2-preview", 1536)).toBe(1536);
expect(resolveGeminiOutputDimensionality("gemini-embedding-2-preview", 3072)).toBe(3072);
expect(() => resolveGeminiOutputDimensionality("gemini-embedding-2-preview", 512)).toThrow(
/Invalid outputDimensionality 512/,
);
expect(() => resolveGeminiOutputDimensionality("gemini-embedding-2-preview", 1024)).toThrow(
/Valid values: 768, 1536, 3072/,
);
expect(normalizeGeminiModel("models/gemini-embedding-2-preview")).toBe(
"gemini-embedding-2-preview",
);
expect(normalizeGeminiModel("gemini/gemini-embedding-2-preview")).toBe(
"gemini-embedding-2-preview",
);
expect(normalizeGeminiModel("google/gemini-embedding-2-preview")).toBe(
"gemini-embedding-2-preview",
);
expect(normalizeGeminiModel("")).toBe(DEFAULT_GEMINI_EMBEDDING_MODEL);
});
});
describe("Gemini embedding provider", () => {
it("handles legacy and v2 request/response behavior", async () => {
const fetchMock = installFetchMock((input) => {
const url = input instanceof URL ? input.href : typeof input === "string" ? input : input.url;
return url.endsWith(":batchEmbedContents")
? {
embeddings: Array.from({ length: 2 }, () => ({
values: [0, 0, 5],
})),
}
: { embedding: { values: [3, 4, 0] } };
});
const { provider } = await createGeminiEmbeddingProvider({
config: {} as never,
provider: "gemini",
remote: { apiKey: "test-key" },
model: "gemini-embedding-2-preview",
outputDimensionality: 768,
taskType: "SEMANTIC_SIMILARITY",
fallback: "none",
});
await expect(provider.embedQuery(" ")).resolves.toStrictEqual([]);
await expect(provider.embedBatch([])).resolves.toStrictEqual([]);
await expect(provider.embedQuery("test query")).resolves.toEqual([0.6, 0.8, 0]);
const structuredBatch = await provider.embedBatchInputs?.([
{
text: "Image file: diagram.png",
parts: [
{ type: "text", text: "Image file: diagram.png" },
{ type: "inline-data", mimeType: "image/png", data: "img" },
],
},
{
text: "Audio file: note.wav",
parts: [
{ type: "text", text: "Audio file: note.wav" },
{ type: "inline-data", mimeType: "audio/wav", data: "aud" },
],
},
]);
expect(structuredBatch).toEqual([
[0, 0, 1],
[0, 0, 1],
]);
expect(requireFirstFetchInput(fetchMock)).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2-preview:embedContent",
);
expect(fetchJsonBody(fetchMock, 0)).toEqual({
outputDimensionality: 768,
taskType: "SEMANTIC_SIMILARITY",
content: { parts: [{ text: "test query" }] },
});
expect(fetchJsonBody(fetchMock, 1)).toEqual({
requests: [
{
model: "models/gemini-embedding-2-preview",
content: {
parts: [
{ text: "Image file: diagram.png" },
{ inlineData: { mimeType: "image/png", data: "img" } },
],
},
taskType: "SEMANTIC_SIMILARITY",
outputDimensionality: 768,
},
{
model: "models/gemini-embedding-2-preview",
content: {
parts: [
{ text: "Audio file: note.wav" },
{ inlineData: { mimeType: "audio/wav", data: "aud" } },
],
},
taskType: "SEMANTIC_SIMILARITY",
outputDimensionality: 768,
},
],
});
});
it("rejects non-object successful embedding responses", async () => {
installFetchMock(() => []);
const { provider } = await createGeminiEmbeddingProvider({
config: {} as never,
provider: "gemini",
remote: { apiKey: "test-key" },
model: "gemini-embedding-001",
fallback: "none",
});
await expect(provider.embedQuery("test query")).rejects.toThrow(
"gemini embeddings failed: malformed JSON response",
);
});
it("rejects wrong single embedding vector shapes", async () => {
installFetchMock(() => ({ embedding: { values: [1, "bad"] } }));
const { provider } = await createGeminiEmbeddingProvider({
config: {} as never,
provider: "gemini",
remote: { apiKey: "test-key" },
model: "gemini-embedding-001",
fallback: "none",
});
await expect(provider.embedQuery("test query")).rejects.toThrow(
"gemini embeddings failed: malformed JSON response",
);
});
it("rejects batch embedding count mismatches", async () => {
installFetchMock(() => ({ embeddings: [{ values: [1, 2] }] }));
const { provider } = await createGeminiEmbeddingProvider({
config: {} as never,
provider: "gemini",
remote: { apiKey: "test-key" },
model: "gemini-embedding-001",
fallback: "none",
});
await expect(provider.embedBatch(["one", "two"])).rejects.toThrow(
"gemini embeddings failed: malformed JSON response",
);
});
});

View File

@@ -0,0 +1,439 @@
// Google provider module implements model/runtime integration.
import {
buildRemoteBaseUrlPolicy,
debugEmbeddingsLog,
sanitizeAndNormalizeEmbedding,
withRemoteHttpResponse,
type EmbeddingInput,
type MemoryEmbeddingProvider,
type MemoryEmbeddingProviderCreateOptions,
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
import { resolveMemorySecretInputString } from "openclaw/plugin-sdk/memory-core-host-secret";
import {
collectProviderApiKeysForExecution,
executeWithApiKeyRotation,
requireApiKey,
resolveApiKeyForProvider,
} from "openclaw/plugin-sdk/provider-auth-runtime";
import {
createProviderHttpError,
providerOperationRetryConfig,
readProviderJsonObjectResponse,
} from "openclaw/plugin-sdk/provider-http";
import type { SsrFPolicy } from "openclaw/plugin-sdk/ssrf-runtime";
import {
asOptionalRecord as asRecord,
normalizeOptionalString,
} from "openclaw/plugin-sdk/string-coerce-runtime";
export type GeminiEmbeddingClient = {
baseUrl: string;
headers: Record<string, string>;
ssrfPolicy?: SsrFPolicy;
model: string;
modelPath: string;
apiKeys: string[];
outputDimensionality?: number;
};
export const DEFAULT_GEMINI_EMBEDDING_MODEL = "gemini-embedding-001";
const DEFAULT_GOOGLE_API_BASE_URL = "https://generativelanguage.googleapis.com/v1beta";
const GEMINI_MAX_INPUT_TOKENS: Record<string, number> = {
"text-embedding-004": 2048,
"gemini-embedding-001": 2048,
"gemini-embedding-2-preview": 8192,
};
function parseGeminiAuth(apiKey: string): { headers: Record<string, string> } {
if (apiKey.startsWith("{")) {
try {
const parsed = JSON.parse(apiKey) as { token?: string };
if (typeof parsed.token === "string" && parsed.token) {
return {
headers: {
Authorization: `Bearer ${parsed.token}`,
"Content-Type": "application/json",
},
};
}
} catch {
// Fall back to API-key auth below.
}
}
return {
headers: {
"x-goog-api-key": apiKey,
"Content-Type": "application/json",
},
};
}
type GeminiTaskType = NonNullable<MemoryEmbeddingProviderCreateOptions["taskType"]>;
// --- gemini-embedding-2-preview support ---
export const GEMINI_EMBEDDING_2_MODELS = new Set([
"gemini-embedding-2-preview",
// Add the GA model name here once released.
]);
const GEMINI_EMBEDDING_2_DEFAULT_DIMENSIONS = 3072;
const GEMINI_EMBEDDING_2_VALID_DIMENSIONS = [768, 1536, 3072] as const;
type GeminiTextPart = { text: string };
type GeminiInlinePart = {
inlineData: { mimeType: string; data: string };
};
type GeminiPart = GeminiTextPart | GeminiInlinePart;
type GeminiEmbeddingInputPart = NonNullable<EmbeddingInput["parts"]>[number];
type GeminiEmbeddingRequest = {
content: { parts: GeminiPart[] };
taskType: GeminiTaskType;
outputDimensionality?: number;
model?: string;
};
export type GeminiTextEmbeddingRequest = GeminiEmbeddingRequest;
function malformedGeminiEmbeddingResponse(): Error {
return new Error("gemini embeddings failed: malformed JSON response");
}
function readGeminiEmbeddingValues(value: unknown): number[] {
if (!Array.isArray(value)) {
throw malformedGeminiEmbeddingResponse();
}
for (const entry of value) {
if (typeof entry !== "number" || !Number.isFinite(entry)) {
throw malformedGeminiEmbeddingResponse();
}
}
return value;
}
function readGeminiSingleEmbedding(payload: Record<string, unknown>): number[] {
const embedding = asRecord(payload.embedding);
if (!embedding) {
throw malformedGeminiEmbeddingResponse();
}
return readGeminiEmbeddingValues(embedding.values);
}
function readGeminiBatchEmbeddings(
payload: Record<string, unknown>,
expectedCount: number,
): number[][] {
if (!Array.isArray(payload.embeddings) || payload.embeddings.length !== expectedCount) {
throw malformedGeminiEmbeddingResponse();
}
return payload.embeddings.map((entry) => {
const embedding = asRecord(entry);
if (!embedding) {
throw malformedGeminiEmbeddingResponse();
}
return readGeminiEmbeddingValues(embedding.values);
});
}
/** Builds the text-only Gemini embedding request shape used across direct and batch APIs. */
export function buildGeminiTextEmbeddingRequest(params: {
text: string;
taskType: GeminiTaskType;
outputDimensionality?: number;
modelPath?: string;
}): GeminiTextEmbeddingRequest {
return buildGeminiEmbeddingRequest({
input: { text: params.text },
taskType: params.taskType,
outputDimensionality: params.outputDimensionality,
modelPath: params.modelPath,
});
}
export function buildGeminiEmbeddingRequest(params: {
input: EmbeddingInput;
taskType: GeminiTaskType;
outputDimensionality?: number;
modelPath?: string;
}): GeminiEmbeddingRequest {
const request: GeminiEmbeddingRequest = {
content: {
parts: params.input.parts?.map((part: GeminiEmbeddingInputPart) =>
part.type === "text"
? ({ text: part.text } satisfies GeminiTextPart)
: ({
inlineData: { mimeType: part.mimeType, data: part.data },
} satisfies GeminiInlinePart),
) ?? [{ text: params.input.text }],
},
taskType: params.taskType,
};
if (params.modelPath) {
request.model = params.modelPath;
}
if (params.outputDimensionality != null) {
request.outputDimensionality = params.outputDimensionality;
}
return request;
}
/**
* Returns true if the given model name is a gemini-embedding-2 variant that
* supports `outputDimensionality` and extended task types.
*/
export function isGeminiEmbedding2Model(model: string): boolean {
return GEMINI_EMBEDDING_2_MODELS.has(model);
}
/**
* Validate and return the `outputDimensionality` for gemini-embedding-2 models.
* Returns `undefined` for older models (they don't support the param).
*/
export function resolveGeminiOutputDimensionality(
model: string,
requested?: number,
): number | undefined {
if (!isGeminiEmbedding2Model(model)) {
return undefined;
}
if (requested == null) {
return GEMINI_EMBEDDING_2_DEFAULT_DIMENSIONS;
}
const valid: readonly number[] = GEMINI_EMBEDDING_2_VALID_DIMENSIONS;
if (!valid.includes(requested)) {
throw new Error(
`Invalid outputDimensionality ${requested} for ${model}. Valid values: ${valid.join(", ")}`,
);
}
return requested;
}
function resolveRemoteApiKey(remoteApiKey: unknown): string | undefined {
const trimmed = resolveMemorySecretInputString({
value: remoteApiKey,
path: "agents.*.memorySearch.remote.apiKey",
});
if (!trimmed) {
return undefined;
}
if (trimmed === "GOOGLE_API_KEY" || trimmed === "GEMINI_API_KEY") {
return process.env[trimmed]?.trim();
}
return trimmed;
}
export function normalizeGeminiModel(model: string): string {
const trimmed = model.trim();
if (!trimmed) {
return DEFAULT_GEMINI_EMBEDDING_MODEL;
}
const withoutPrefix = trimmed.replace(/^models\//, "");
if (withoutPrefix.startsWith("gemini/")) {
return withoutPrefix.slice("gemini/".length);
}
if (withoutPrefix.startsWith("google/")) {
return withoutPrefix.slice("google/".length);
}
return withoutPrefix;
}
async function fetchGeminiEmbeddingPayload(params: {
client: GeminiEmbeddingClient;
endpoint: string;
body: unknown;
signal?: AbortSignal;
}): Promise<Record<string, unknown>> {
return await executeWithApiKeyRotation({
provider: "google",
apiKeys: params.client.apiKeys,
transientRetry: providerOperationRetryConfig("read"),
execute: async (apiKey) => {
const authHeaders = parseGeminiAuth(apiKey);
const headers = {
...authHeaders.headers,
...params.client.headers,
};
return await withRemoteHttpResponse({
url: params.endpoint,
ssrfPolicy: params.client.ssrfPolicy,
signal: params.signal,
init: {
method: "POST",
headers,
body: JSON.stringify(params.body),
},
onResponse: async (res) => {
if (!res.ok) {
throw await createProviderHttpError(res, "gemini embeddings failed");
}
return await readProviderJsonObjectResponse(res, "gemini embeddings failed");
},
});
},
});
}
function normalizeGeminiBaseUrl(raw: string): string {
const trimmed = raw.replace(/\/+$/, "");
const openAiIndex = trimmed.indexOf("/openai");
if (openAiIndex > -1) {
return normalizeGoogleApiBaseUrl(trimmed.slice(0, openAiIndex));
}
return normalizeGoogleApiBaseUrl(trimmed);
}
function buildGeminiModelPath(model: string): string {
return model.startsWith("models/") ? model : `models/${model}`;
}
function normalizeGoogleApiBaseUrl(baseUrl: string): string {
const trimmed = baseUrl.trim().replace(/\/+$/, "");
if (!trimmed) {
return DEFAULT_GOOGLE_API_BASE_URL;
}
try {
const url = new URL(trimmed);
url.hash = "";
url.search = "";
if (
url.origin.toLowerCase() === "https://generativelanguage.googleapis.com" &&
url.pathname.replace(/\/+$/, "") === ""
) {
url.pathname = "/v1beta";
}
return url.toString().replace(/\/+$/, "");
} catch {
return trimmed;
}
}
export async function createGeminiEmbeddingProvider(
options: MemoryEmbeddingProviderCreateOptions,
): Promise<{ provider: MemoryEmbeddingProvider; client: GeminiEmbeddingClient }> {
const client = await resolveGeminiEmbeddingClient(options);
const baseUrl = client.baseUrl.replace(/\/$/, "");
const embedUrl = `${baseUrl}/${client.modelPath}:embedContent`;
const batchUrl = `${baseUrl}/${client.modelPath}:batchEmbedContents`;
const isV2 = isGeminiEmbedding2Model(client.model);
const outputDimensionality = client.outputDimensionality;
const embedQuery = async (
text: string,
callOptions?: { signal?: AbortSignal },
): Promise<number[]> => {
if (!text.trim()) {
return [];
}
const payload = await fetchGeminiEmbeddingPayload({
client,
endpoint: embedUrl,
body: buildGeminiTextEmbeddingRequest({
text,
taskType: options.taskType ?? "RETRIEVAL_QUERY",
outputDimensionality: isV2 ? outputDimensionality : undefined,
}),
signal: callOptions?.signal,
});
return sanitizeAndNormalizeEmbedding(readGeminiSingleEmbedding(payload));
};
const embedBatchInputs = async (
inputs: EmbeddingInput[],
callOptions?: { signal?: AbortSignal },
): Promise<number[][]> => {
if (inputs.length === 0) {
return [];
}
const payload = await fetchGeminiEmbeddingPayload({
client,
endpoint: batchUrl,
body: {
requests: inputs.map((input) =>
buildGeminiEmbeddingRequest({
input,
modelPath: client.modelPath,
taskType: options.taskType ?? "RETRIEVAL_DOCUMENT",
outputDimensionality: isV2 ? outputDimensionality : undefined,
}),
),
},
signal: callOptions?.signal,
});
const embeddings = readGeminiBatchEmbeddings(payload, inputs.length);
return embeddings.map((values) => sanitizeAndNormalizeEmbedding(values));
};
const embedBatch = async (
texts: string[],
optionsLocal?: { signal?: AbortSignal },
): Promise<number[][]> => {
return await embedBatchInputs(
texts.map((text) => ({
text,
})),
optionsLocal,
);
};
return {
provider: {
id: "gemini",
model: client.model,
maxInputTokens: GEMINI_MAX_INPUT_TOKENS[client.model],
embedQuery,
embedBatch,
embedBatchInputs,
},
client,
};
}
async function resolveGeminiEmbeddingClient(
options: MemoryEmbeddingProviderCreateOptions,
): Promise<GeminiEmbeddingClient> {
const remote = options.remote;
const remoteApiKey = resolveRemoteApiKey(remote?.apiKey);
const remoteBaseUrl = remote?.baseUrl?.trim();
const apiKey = remoteApiKey
? remoteApiKey
: requireApiKey(
await resolveApiKeyForProvider({
provider: "google",
cfg: options.config,
agentDir: options.agentDir,
}),
"google",
);
const providerConfig = options.config.models?.providers?.google;
const rawBaseUrl =
remoteBaseUrl ||
normalizeOptionalString(providerConfig?.baseUrl) ||
DEFAULT_GOOGLE_API_BASE_URL;
const baseUrl = normalizeGeminiBaseUrl(rawBaseUrl);
const ssrfPolicy = buildRemoteBaseUrlPolicy(baseUrl);
const headerOverrides = Object.assign({}, providerConfig?.headers, remote?.headers);
const headers: Record<string, string> = {
...headerOverrides,
};
const apiKeys = collectProviderApiKeysForExecution({
provider: "google",
primaryApiKey: apiKey,
});
const model = normalizeGeminiModel(options.model);
const modelPath = buildGeminiModelPath(model);
const outputDimensionality = resolveGeminiOutputDimensionality(
model,
options.outputDimensionality,
);
debugEmbeddingsLog("memory embeddings: gemini client", {
rawBaseUrl,
baseUrl,
model,
modelPath,
outputDimensionality,
embedEndpoint: `${baseUrl}/${modelPath}:embedContent`,
batchEndpoint: `${baseUrl}/${modelPath}:batchEmbedContents`,
});
return { baseUrl, headers, ssrfPolicy, model, modelPath, apiKeys, outputDimensionality };
}

View File

@@ -0,0 +1,21 @@
// Google plugin module implements gemini auth behavior.
import { parseGoogleOauthApiKey } from "./oauth-token-shared.js";
export function parseGeminiAuth(apiKey: string): { headers: Record<string, string> } {
const parsed = apiKey.startsWith("{") ? parseGoogleOauthApiKey(apiKey) : null;
if (parsed?.token) {
return {
headers: {
Authorization: `Bearer ${parsed.token}`,
"Content-Type": "application/json",
},
};
}
return {
headers: {
"x-goog-api-key": apiKey,
"Content-Type": "application/json",
},
};
}

View File

@@ -0,0 +1,9 @@
import crypto from "node:crypto";
import path from "node:path";
export const GOOGLE_GEMINI_CLI_PROVIDER_ID = "google-gemini-cli";
export function resolveGeminiCliProfileHome(agentDir: string, profileId: string): string {
const profileHash = crypto.createHash("sha256").update(profileId).digest("hex").slice(0, 24);
return path.join(agentDir, `${GOOGLE_GEMINI_CLI_PROVIDER_ID}-home`, "profiles", profileHash);
}

View File

@@ -0,0 +1,149 @@
import { createLazyRuntimeModule } from "openclaw/plugin-sdk/lazy-runtime";
// Google provider module implements model/runtime integration.
import type {
OpenClawPluginApi,
ProviderAuthContext,
ProviderFetchUsageSnapshotContext,
} from "openclaw/plugin-sdk/plugin-entry";
import { buildOauthProviderAuthResult } from "openclaw/plugin-sdk/provider-auth-result";
import type { ProviderPlugin } from "openclaw/plugin-sdk/provider-model-shared";
import { fetchGeminiUsage } from "openclaw/plugin-sdk/provider-usage";
import { GOOGLE_GEMINI_CLI_PROVIDER_ID } from "./gemini-cli-auth-home.js";
import { formatGoogleOauthApiKey, parseGoogleUsageToken } from "./oauth-token-shared.js";
import { GOOGLE_GEMINI_PROVIDER_HOOKS } from "./provider-hooks.js";
import { isModernGoogleModel, resolveGoogleGeminiForwardCompatModel } from "./provider-models.js";
const PROVIDER_ID = GOOGLE_GEMINI_CLI_PROVIDER_ID;
const PROVIDER_LABEL = "Gemini CLI OAuth";
const DEFAULT_MODEL = "google/gemini-3.1-pro-preview";
const ENV_VARS = [
"OPENCLAW_GEMINI_OAUTH_CLIENT_ID",
"OPENCLAW_GEMINI_OAUTH_CLIENT_SECRET",
"GEMINI_CLI_OAUTH_CLIENT_ID",
"GEMINI_CLI_OAUTH_CLIENT_SECRET",
] as const;
const loadOauthRuntimeModule = createLazyRuntimeModule(() => import("./oauth.runtime.js"));
async function fetchGeminiCliUsage(ctx: ProviderFetchUsageSnapshotContext) {
return await fetchGeminiUsage(ctx.token, ctx.timeoutMs, ctx.fetchFn, PROVIDER_ID);
}
export function buildGoogleGeminiCliProvider(): ProviderPlugin {
return {
id: PROVIDER_ID,
label: PROVIDER_LABEL,
docsPath: "/providers/models",
aliases: ["gemini-cli"],
envVars: [...ENV_VARS],
auth: [
{
id: "oauth",
label: "Google OAuth",
hint: "PKCE + localhost callback",
kind: "oauth",
run: async (ctx: ProviderAuthContext) => {
await ctx.prompter.note(
[
"This is an unofficial integration and is not endorsed by Google.",
"Some users have reported account restrictions or suspensions after using third-party Gemini CLI and Antigravity OAuth clients.",
"Proceed only if you understand and accept this risk.",
].join("\n"),
"Google Gemini CLI caution",
);
const proceed = await ctx.prompter.confirm({
message: "Continue with Google Gemini CLI OAuth?",
initialValue: false,
});
if (!proceed) {
await ctx.prompter.note("Skipped Google Gemini CLI OAuth setup.", "Setup skipped");
return { profiles: [] };
}
const spin = ctx.prompter.progress("Starting Gemini CLI OAuth…");
try {
const { loginGeminiCliOAuth } = await loadOauthRuntimeModule();
const result = await loginGeminiCliOAuth({
isRemote: ctx.isRemote,
openUrl: ctx.openUrl,
log: (msg) => ctx.runtime.log(msg),
note: ctx.prompter.note,
prompt: async (message) => ctx.prompter.text({ message }),
progress: spin,
});
spin.stop("Gemini CLI OAuth complete");
return buildOauthProviderAuthResult({
providerId: PROVIDER_ID,
defaultModel: DEFAULT_MODEL,
access: result.access,
refresh: result.refresh,
expires: result.expires,
email: result.email,
configPatch: {
agents: {
defaults: {
models: {
[DEFAULT_MODEL]: { agentRuntime: { id: PROVIDER_ID } },
},
},
},
},
...(result.projectId ? { credentialExtra: { projectId: result.projectId } } : {}),
...(result.projectId
? {
notes: [
"If requests fail, set GOOGLE_CLOUD_PROJECT or GOOGLE_CLOUD_PROJECT_ID.",
],
}
: {}),
});
} catch (err) {
spin.stop("Gemini CLI OAuth failed");
await ctx.prompter.note(
"Trouble with OAuth? Ensure your Google account has Gemini CLI access.",
"OAuth help",
);
throw err;
}
},
},
],
wizard: {
setup: {
choiceId: "google-gemini-cli",
choiceLabel: "Gemini CLI OAuth",
choiceHint: "Google OAuth with project-aware token payload",
methodId: "oauth",
},
},
resolveDynamicModel: (ctx) =>
resolveGoogleGeminiForwardCompatModel({
providerId: PROVIDER_ID,
ctx,
}),
...GOOGLE_GEMINI_PROVIDER_HOOKS,
isModernModelRef: ({ modelId }) => isModernGoogleModel(modelId),
formatApiKey: (cred) => formatGoogleOauthApiKey(cred),
refreshOAuth: async (cred) => {
const { refreshGeminiCliOAuthToken } = await loadOauthRuntimeModule();
return await refreshGeminiCliOAuthToken(cred);
},
resolveUsageAuth: async (ctx) => {
const auth = await ctx.resolveOAuthToken();
if (!auth) {
return null;
}
return {
...auth,
token: parseGoogleUsageToken(auth.token),
};
},
fetchUsageSnapshot: async (ctx) => await fetchGeminiCliUsage(ctx),
};
}
export function registerGoogleGeminiCliProvider(api: OpenClawPluginApi) {
api.registerProvider(buildGoogleGeminiCliProvider());
}

View File

@@ -0,0 +1,122 @@
// Google provider module implements model/runtime integration.
import type { MusicGenerationProvider } from "openclaw/plugin-sdk/music-generation";
import { isProviderApiKeyConfigured } from "openclaw/plugin-sdk/provider-auth";
import type {
VideoGenerationProvider,
VideoGenerationProviderConfiguredContext,
} from "openclaw/plugin-sdk/video-generation";
export const DEFAULT_GOOGLE_MUSIC_MODEL = "lyria-3-clip-preview";
export const GOOGLE_PRO_MUSIC_MODEL = "lyria-3-pro-preview";
export const GOOGLE_MAX_INPUT_IMAGES = 10;
export const DEFAULT_GOOGLE_VIDEO_MODEL = "veo-3.1-fast-generate-preview";
export const GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS = [4, 6, 8] as const;
export const GOOGLE_VIDEO_MIN_DURATION_SECONDS = GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS[0];
export const GOOGLE_VIDEO_MAX_DURATION_SECONDS =
GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS[GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS.length - 1];
function isGoogleProviderConfigured(
ctx: { agentDir?: string } | VideoGenerationProviderConfiguredContext,
): boolean {
return isProviderApiKeyConfigured({
provider: "google",
agentDir: ctx.agentDir,
});
}
export function createGoogleMusicGenerationProviderMetadata(): Omit<
MusicGenerationProvider,
"generateMusic"
> {
return {
id: "google",
label: "Google",
defaultModel: DEFAULT_GOOGLE_MUSIC_MODEL,
models: [DEFAULT_GOOGLE_MUSIC_MODEL, GOOGLE_PRO_MUSIC_MODEL],
isConfigured: isGoogleProviderConfigured,
capabilities: {
generate: {
maxTracks: 1,
supportsLyrics: true,
supportsInstrumental: true,
supportsFormat: true,
supportedFormatsByModel: {
[DEFAULT_GOOGLE_MUSIC_MODEL]: ["mp3"],
[GOOGLE_PRO_MUSIC_MODEL]: ["mp3", "wav"],
},
},
edit: {
enabled: true,
maxTracks: 1,
maxInputImages: GOOGLE_MAX_INPUT_IMAGES,
supportsLyrics: true,
supportsInstrumental: true,
supportsFormat: true,
supportedFormatsByModel: {
[DEFAULT_GOOGLE_MUSIC_MODEL]: ["mp3"],
[GOOGLE_PRO_MUSIC_MODEL]: ["mp3", "wav"],
},
},
},
};
}
export function createGoogleVideoGenerationProviderMetadata(): Omit<
VideoGenerationProvider,
"generateVideo"
> {
return {
id: "google",
label: "Google",
defaultModel: DEFAULT_GOOGLE_VIDEO_MODEL,
models: [
DEFAULT_GOOGLE_VIDEO_MODEL,
"veo-3.1-generate-preview",
"veo-3.1-lite-generate-preview",
"veo-3.0-fast-generate-001",
"veo-3.0-generate-001",
"veo-2.0-generate-001",
],
isConfigured: isGoogleProviderConfigured,
capabilities: {
generate: {
maxVideos: 1,
maxDurationSeconds: GOOGLE_VIDEO_MAX_DURATION_SECONDS,
supportedDurationSeconds: [...GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS],
aspectRatios: ["16:9", "9:16"],
resolutions: ["720P", "1080P"],
supportsAspectRatio: true,
supportsResolution: true,
supportsSize: true,
supportsAudio: false,
},
imageToVideo: {
enabled: true,
maxVideos: 1,
maxInputImages: 1,
maxDurationSeconds: GOOGLE_VIDEO_MAX_DURATION_SECONDS,
supportedDurationSeconds: [...GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS],
aspectRatios: ["16:9", "9:16"],
resolutions: ["720P", "1080P"],
supportsAspectRatio: true,
supportsResolution: true,
supportsSize: true,
supportsAudio: false,
},
videoToVideo: {
enabled: true,
maxVideos: 1,
maxInputVideos: 1,
maxDurationSeconds: GOOGLE_VIDEO_MAX_DURATION_SECONDS,
supportedDurationSeconds: [...GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS],
aspectRatios: ["16:9", "9:16"],
resolutions: ["720P", "1080P"],
supportsAspectRatio: true,
supportsResolution: true,
supportsSize: true,
supportsAudio: false,
},
},
};
}

View File

@@ -0,0 +1,9 @@
// Google plugin module implements google genai runtime behavior.
import { GoogleGenAI } from "@google/genai";
export type GoogleGenAIClient = InstanceType<typeof GoogleGenAI>;
type GoogleGenAIOptions = ConstructorParameters<typeof GoogleGenAI>[0];
export function createGoogleGenAI(options: GoogleGenAIOptions): GoogleGenAIClient {
return new GoogleGenAI(options);
}

View File

@@ -0,0 +1,213 @@
import { resolveFfmpegBin } from "openclaw/plugin-sdk/media-runtime";
// Google tests cover google plugin behavior.
import {
registerProviderPlugin,
requireRegisteredProvider,
} from "openclaw/plugin-sdk/plugin-test-runtime";
import { normalizeTranscriptForMatch } from "openclaw/plugin-sdk/provider-test-contracts";
import { isLiveTestEnabled } from "openclaw/plugin-sdk/test-env";
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
import { createGeminiWebSearchProvider } from "./src/gemini-web-search-provider.js";
const GOOGLE_API_KEY =
process.env.GEMINI_API_KEY?.trim() ||
process.env.GOOGLE_API_KEY?.trim() ||
process.env.GEMINI_PROVIDER_API_KEY?.trim() ||
"";
const LIVE = isLiveTestEnabled() && GOOGLE_API_KEY.length > 0;
const describeLive = LIVE ? describe : describe.skip;
async function withGoogleApiEnvUnset<T>(fn: () => Promise<T>): Promise<T> {
const geminiApiKey = process.env.GEMINI_API_KEY;
const googleApiKey = process.env.GOOGLE_API_KEY;
delete process.env.GEMINI_API_KEY;
delete process.env.GOOGLE_API_KEY;
try {
return await fn();
} finally {
if (geminiApiKey === undefined) {
delete process.env.GEMINI_API_KEY;
} else {
process.env.GEMINI_API_KEY = geminiApiKey;
}
if (googleApiKey === undefined) {
delete process.env.GOOGLE_API_KEY;
} else {
process.env.GOOGLE_API_KEY = googleApiKey;
}
}
}
function isTransientGeminiSearchError(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
if (error.name === "AbortError") {
return true;
}
const message = error.message.toLowerCase();
return message.includes("timeout") || message.includes("aborted");
}
function hasTrustedFfmpegForLiveVoiceNote(): boolean {
try {
resolveFfmpegBin();
return true;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes("ffmpeg not found in trusted system directories")) {
console.warn("[google:live] skip voice-note transcode: ffmpeg unavailable");
return false;
}
throw error;
}
}
const registerGooglePlugin = () =>
registerProviderPlugin({
plugin,
id: "google",
name: "Google Provider",
});
describeLive("google plugin live", () => {
it("synthesizes speech through the registered provider", async () => {
const { speechProviders } = await registerGooglePlugin();
const provider = requireRegisteredProvider(speechProviders, "google");
const audioFile = await provider.synthesize({
text: "OpenClaw Google text to speech integration test OK.",
cfg: { plugins: { enabled: true } } as never,
providerConfig: { apiKey: GOOGLE_API_KEY },
target: "audio-file",
timeoutMs: 90_000,
});
expect(audioFile.outputFormat).toBe("wav");
expect(audioFile.fileExtension).toBe(".wav");
expect(audioFile.audioBuffer.byteLength).toBeGreaterThan(512);
}, 120_000);
it("transcodes speech to Opus for voice-note targets", async () => {
if (!hasTrustedFfmpegForLiveVoiceNote()) {
return;
}
const { speechProviders } = await registerGooglePlugin();
const provider = requireRegisteredProvider(speechProviders, "google");
const audioFile = await provider.synthesize({
text: "OpenClaw Google voice note integration test OK.",
cfg: { plugins: { enabled: true } } as never,
providerConfig: { apiKey: GOOGLE_API_KEY },
target: "voice-note",
timeoutMs: 90_000,
});
expect(audioFile.outputFormat).toBe("opus");
expect(audioFile.fileExtension).toBe(".opus");
expect(audioFile.voiceCompatible).toBe(true);
expect(audioFile.audioBuffer.byteLength).toBeGreaterThan(128);
}, 120_000);
it("transcribes synthesized speech through the media provider", async () => {
const { mediaProviders, speechProviders } = await registerGooglePlugin();
const speechProvider = requireRegisteredProvider(speechProviders, "google");
const mediaProvider = requireRegisteredProvider(mediaProviders, "google");
const phrase = "Testing Google audio transcription with pineapple.";
const audioFile = await speechProvider.synthesize({
text: phrase,
cfg: { plugins: { enabled: true } } as never,
providerConfig: { apiKey: GOOGLE_API_KEY },
target: "audio-file",
timeoutMs: 90_000,
});
const transcript = await mediaProvider.transcribeAudio?.({
buffer: audioFile.audioBuffer,
fileName: "google-live.wav",
mime: "audio/wav",
apiKey: GOOGLE_API_KEY,
timeoutMs: 90_000,
});
const normalized = normalizeTranscriptForMatch(transcript?.text ?? "");
expect(normalized).toContain("google");
expect(normalized).toContain("pineapple");
}, 180_000);
it("runs Gemini web search through the registered provider tool", async () => {
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool?.({
config: {},
searchConfig: { gemini: { apiKey: GOOGLE_API_KEY }, cacheTtlMinutes: 0, timeoutSeconds: 90 },
} as never);
let result: { provider?: string; content?: unknown; citations?: unknown } | undefined;
let lastError: unknown;
for (let attempt = 0; attempt < 2; attempt += 1) {
try {
result = await tool?.execute({ query: "OpenClaw GitHub", count: 1 });
lastError = undefined;
break;
} catch (error) {
lastError = error;
if (!isTransientGeminiSearchError(error) || attempt === 1) {
throw error;
}
}
}
if (lastError) {
throw toLintErrorObject(lastError, "Non-Error thrown");
}
expect(result?.provider).toBe("gemini");
expect(typeof result?.content).toBe("string");
expect((result!.content as string).length).toBeGreaterThan(20);
expect(Array.isArray(result?.citations)).toBe(true);
}, 120_000);
it("runs Gemini web search through the Google model provider config fallback", async () => {
await withGoogleApiEnvUnset(async () => {
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool?.({
config: {
models: {
providers: {
google: {
apiKey: GOOGLE_API_KEY,
},
},
},
},
searchConfig: { provider: "gemini", cacheTtlMinutes: 0, timeoutSeconds: 90 },
} as never);
const result = await tool?.execute({ query: "OpenClaw GitHub", count: 1 });
expect(process.env.GEMINI_API_KEY).toBeUndefined();
expect(process.env.GOOGLE_API_KEY).toBeUndefined();
expect(result?.provider).toBe("gemini");
expect(typeof result?.content).toBe("string");
expect((result!.content as string).length).toBeGreaterThan(20);
expect(Array.isArray(result?.citations)).toBe(true);
expect((result!.citations as unknown[]).length).toBeGreaterThan(0);
});
}, 120_000);
});
function toLintErrorObject(value: unknown, fallbackMessage: string): Error {
if (value instanceof Error) {
return value;
}
if (typeof value === "string") {
return new Error(value);
}
const error = new Error(fallbackMessage, { cause: value });
if ((typeof value === "object" && value !== null) || typeof value === "function") {
Object.assign(error, value);
}
return error;
}

View File

@@ -0,0 +1,572 @@
// Google tests cover image generation provider plugin behavior.
import * as providerAuthRuntime from "openclaw/plugin-sdk/provider-auth-runtime";
import * as providerHttp from "openclaw/plugin-sdk/provider-http";
import { mockPinnedHostnameResolution } from "openclaw/plugin-sdk/test-env";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { buildGoogleImageGenerationProvider } from "./image-generation-provider.js";
import { testing as geminiWebSearchTesting } from "./src/gemini-web-search-provider.js";
let ssrfMock: { mockRestore: () => void } | undefined;
function jsonResponse(payload: unknown): Response {
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "Content-Type": "application/json" },
});
}
function mockGoogleApiKeyAuth() {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-test-key",
source: "env",
mode: "api-key",
});
}
function installGoogleFetchMock(params?: {
data?: string;
mimeType?: string;
inlineDataKey?: "inlineData" | "inline_data";
}) {
const mimeType = params?.mimeType ?? "image/png";
const data = params?.data ?? "png-data";
const inlineDataKey = params?.inlineDataKey ?? "inlineData";
const fetchMock = vi.fn().mockResolvedValue(
jsonResponse({
candidates: [
{
content: {
parts: [
{
[inlineDataKey]: {
[inlineDataKey === "inlineData" ? "mimeType" : "mime_type"]: mimeType,
data: Buffer.from(data).toString("base64"),
},
},
],
},
},
],
}),
);
vi.stubGlobal("fetch", fetchMock);
return fetchMock;
}
function fetchRequest(fetchMock: ReturnType<typeof vi.fn>): {
body?: string;
headers?: HeadersInit;
method?: string;
url: string;
} {
const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit | undefined];
expect(typeof url).toBe("string");
if (!init) {
throw new Error("Expected fetch init");
}
return {
body: typeof init.body === "string" ? init.body : undefined,
headers: init.headers,
method: init.method,
url,
};
}
function postJsonRequestOptions(spy: unknown): {
allowPrivateNetwork?: boolean;
pinDns?: boolean;
ssrfPolicy?: { allowRfc2544BenchmarkRange?: boolean };
} {
const options = (spy as { mock?: { calls?: Array<[unknown]> } }).mock?.calls?.[0]?.[0];
if (!options) {
throw new Error("Expected postJsonRequest options");
}
return options as {
allowPrivateNetwork?: boolean;
pinDns?: boolean;
ssrfPolicy?: { allowRfc2544BenchmarkRange?: boolean };
};
}
describe("Google image-generation provider", () => {
beforeEach(() => {
ssrfMock = mockPinnedHostnameResolution();
});
afterEach(() => {
ssrfMock?.mockRestore();
ssrfMock = undefined;
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
it("generates image buffers from the Gemini generateContent API", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-test-key",
source: "env",
mode: "api-key",
});
const fetchMock = vi.fn().mockResolvedValue(
jsonResponse({
candidates: [
{
content: {
parts: [
{ text: "generated" },
{
inlineData: {
mimeType: "image/png",
data: Buffer.from("png-data").toString("base64"),
},
},
],
},
},
],
}),
);
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleImageGenerationProvider();
const result = await provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a cat",
cfg: {},
size: "1536x1024",
});
const request = fetchRequest(fetchMock);
expect(request.url).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-image-preview:generateContent",
);
expect(request.method).toBe("POST");
expect(JSON.parse(request.body ?? "")).toEqual({
contents: [
{
role: "user",
parts: [{ text: "draw a cat" }],
},
],
generationConfig: {
responseModalities: ["TEXT", "IMAGE"],
imageConfig: {
aspectRatio: "3:2",
imageSize: "2K",
},
},
});
expect(result).toEqual({
images: [
{
buffer: Buffer.from("png-data"),
mimeType: "image/png",
fileName: "image-1.png",
},
],
model: "gemini-3.1-flash-image-preview",
});
});
it("passes request SSRF policy to the provider HTTP helper", async () => {
mockGoogleApiKeyAuth();
const postJsonRequest = vi.spyOn(providerHttp, "postJsonRequest").mockResolvedValue({
response: new Response(
JSON.stringify({
candidates: [
{
content: {
parts: [
{
inlineData: {
mimeType: "image/png",
data: Buffer.from("png-data").toString("base64"),
},
},
],
},
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
),
finalUrl:
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-image-preview:generateContent",
release: async () => {},
});
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a cat",
cfg: {},
ssrfPolicy: { allowRfc2544BenchmarkRange: true },
});
expect(postJsonRequestOptions(postJsonRequest).ssrfPolicy).toEqual({
allowRfc2544BenchmarkRange: true,
});
});
it("wraps wrong-shape successful Gemini image responses", async () => {
mockGoogleApiKeyAuth();
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue(jsonResponse({ candidates: { content: { parts: [] } } })),
);
const provider = buildGoogleImageGenerationProvider();
await expect(
provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a cat",
cfg: {},
}),
).rejects.toThrow("Google image generation response malformed");
});
it("rejects invalid inline image data in successful Gemini responses", async () => {
mockGoogleApiKeyAuth();
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue(
jsonResponse({
candidates: [
{
content: {
parts: [{ inlineData: { mimeType: "image/png", data: "not-base64!" } }],
},
},
],
}),
),
);
const provider = buildGoogleImageGenerationProvider();
await expect(
provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a cat",
cfg: {},
}),
).rejects.toThrow("Google image generation response malformed");
});
it("accepts OAuth JSON auth and inline_data responses", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: JSON.stringify({ token: "oauth-token" }),
source: "profile",
mode: "token",
});
const fetchMock = vi.fn().mockResolvedValue(
jsonResponse({
candidates: [
{
content: {
parts: [
{
inline_data: {
mime_type: "image/jpeg",
data: Buffer.from("jpg-data").toString("base64"),
},
},
],
},
},
],
}),
);
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleImageGenerationProvider();
const result = await provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a dog",
cfg: {},
});
const request = fetchRequest(fetchMock);
expect(request.url.length).toBeGreaterThan(0);
expect(request.headers).toBeInstanceOf(Headers);
expect(new Headers(request.headers).get("authorization")).toBe("Bearer oauth-token");
expect(result).toEqual({
images: [
{
buffer: Buffer.from("jpg-data"),
mimeType: "image/jpeg",
fileName: "image-1.jpg",
},
],
model: "gemini-3.1-flash-image-preview",
});
});
it("accepts valid multi-image inline JSON responses above the generic provider JSON cap", async () => {
mockGoogleApiKeyAuth();
const imageBytes = Buffer.alloc(6 * 1024 * 1024, 1);
const imagePayload = imageBytes.toString("base64");
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue(
jsonResponse({
candidates: [
{
content: {
parts: Array.from({ length: 3 }, () => ({
inlineData: {
mimeType: "image/png",
data: imagePayload,
},
})),
},
},
],
}),
),
);
const provider = buildGoogleImageGenerationProvider();
const result = await provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a cat",
cfg: {},
});
expect(result.images).toHaveLength(3);
expect(result.images.map((image) => image.buffer.byteLength)).toEqual([
imageBytes.byteLength,
imageBytes.byteLength,
imageBytes.byteLength,
]);
});
it("still rejects oversized Google image JSON responses", async () => {
mockGoogleApiKeyAuth();
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue(
jsonResponse({
candidates: [
{
content: {
parts: [{ text: "x".repeat(35 * 1024 * 1024) }],
},
},
],
}),
),
);
const provider = buildGoogleImageGenerationProvider();
await expect(
provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a cat",
cfg: {},
}),
).rejects.toThrow("google.image-generation: JSON response exceeds");
});
it("sends reference images and explicit resolution for edit flows", async () => {
mockGoogleApiKeyAuth();
const fetchMock = installGoogleFetchMock();
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3-pro-image-preview",
prompt: "Change only the sky to a sunset.",
cfg: {},
resolution: "4K",
inputImages: [
{
buffer: Buffer.from("reference-bytes"),
mimeType: "image/png",
fileName: "reference.png",
},
],
});
const request = fetchRequest(fetchMock);
expect(request.url).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
);
expect(request.method).toBe("POST");
expect(JSON.parse(request.body ?? "")).toEqual({
contents: [
{
role: "user",
parts: [
{
inlineData: {
mimeType: "image/png",
data: Buffer.from("reference-bytes").toString("base64"),
},
},
{ text: "Change only the sky to a sunset." },
],
},
],
generationConfig: {
responseModalities: ["TEXT", "IMAGE"],
imageConfig: {
imageSize: "4K",
},
},
});
});
it("forwards explicit aspect ratio without forcing a default when size is omitted", async () => {
mockGoogleApiKeyAuth();
const fetchMock = installGoogleFetchMock();
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3-pro-image-preview",
prompt: "portrait photo",
cfg: {},
aspectRatio: "9:16",
});
const request = fetchRequest(fetchMock);
expect(request.url).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
);
expect(request.method).toBe("POST");
expect(JSON.parse(request.body ?? "")).toEqual({
contents: [
{
role: "user",
parts: [{ text: "portrait photo" }],
},
],
generationConfig: {
responseModalities: ["TEXT", "IMAGE"],
imageConfig: {
aspectRatio: "9:16",
},
},
});
});
it("disables DNS pinning for Google image generation requests", async () => {
mockGoogleApiKeyAuth();
installGoogleFetchMock();
const postJsonRequestSpy = vi.spyOn(providerHttp, "postJsonRequest");
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a fox",
cfg: {},
});
expect(postJsonRequestOptions(postJsonRequestSpy).pinDns).toBe(false);
});
it("honors configured private-network opt-in for Google image generation", async () => {
mockGoogleApiKeyAuth();
installGoogleFetchMock();
const postJsonRequestSpy = vi.spyOn(providerHttp, "postJsonRequest");
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3.1-flash-image-preview",
prompt: "draw a fox",
cfg: {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
request: { allowPrivateNetwork: true },
models: [],
},
},
},
},
});
expect(postJsonRequestOptions(postJsonRequestSpy).allowPrivateNetwork).toBe(true);
});
it("normalizes a configured bare Google host to the v1beta API root", async () => {
mockGoogleApiKeyAuth();
const fetchMock = installGoogleFetchMock();
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3-pro-image-preview",
prompt: "draw a cat",
cfg: {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com",
models: [],
},
},
},
},
});
const request = fetchRequest(fetchMock);
expect(request.url).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
);
expect(typeof request.method).toBe("string");
});
it("strips a configured /openai suffix before calling the native Gemini image API", async () => {
mockGoogleApiKeyAuth();
const fetchMock = installGoogleFetchMock();
const provider = buildGoogleImageGenerationProvider();
await provider.generateImage({
provider: "google",
model: "gemini-3-pro-image-preview",
prompt: "draw a fox",
cfg: {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
models: [],
},
},
},
},
});
const request = fetchRequest(fetchMock);
expect(request.url).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
);
expect(typeof request.method).toBe("string");
});
it("prefers scoped configured Gemini API keys over environment fallbacks", () => {
expect(
geminiWebSearchTesting.resolveGeminiApiKey({
apiKey: "gemini-secret",
}),
).toBe("gemini-secret");
});
it("falls back to the default Gemini model when unset or blank", () => {
expect(geminiWebSearchTesting.resolveGeminiModel()).toBe("gemini-2.5-flash");
expect(geminiWebSearchTesting.resolveGeminiModel({ model: " " })).toBe("gemini-2.5-flash");
expect(geminiWebSearchTesting.resolveGeminiModel({ model: "gemini-2.5-pro" })).toBe(
"gemini-2.5-pro",
);
});
});

View File

@@ -0,0 +1,294 @@
// Google provider module implements model/runtime integration.
import {
generatedImageAssetFromBase64,
resolveInlineImageJsonResponseMaxBytes,
type GeneratedImageAsset,
type ImageGenerationProvider,
} from "openclaw/plugin-sdk/image-generation";
import { MAX_IMAGE_BYTES } from "openclaw/plugin-sdk/media-runtime";
import { parseStrictPositiveInteger } from "openclaw/plugin-sdk/number-runtime";
import { isProviderApiKeyConfigured } from "openclaw/plugin-sdk/provider-auth";
import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runtime";
import {
assertOkOrThrowHttpError,
postJsonRequest,
readProviderJsonResponse,
sanitizeConfiguredModelProviderRequest,
} from "openclaw/plugin-sdk/provider-http";
import {
isRecord,
normalizeLowercaseStringOrEmpty,
normalizeOptionalString,
} from "openclaw/plugin-sdk/string-coerce-runtime";
import { normalizeGoogleModelId, resolveGoogleGenerativeAiHttpRequestConfig } from "./api.js";
const DEFAULT_GOOGLE_IMAGE_MODEL = "gemini-3.1-flash-image-preview";
const DEFAULT_IMAGE_TIMEOUT_MS = 180_000;
const DEFAULT_OUTPUT_MIME = "image/png";
const GOOGLE_MAX_IMAGE_RESULTS = 4;
const MB = 1024 * 1024;
const GOOGLE_SUPPORTED_SIZES = [
"1024x1024",
"1024x1536",
"1536x1024",
"1024x1792",
"1792x1024",
] as const;
const GOOGLE_SUPPORTED_ASPECT_RATIOS = [
"1:1",
"2:3",
"3:2",
"3:4",
"4:3",
"4:5",
"5:4",
"9:16",
"16:9",
"21:9",
] as const;
const GOOGLE_IMAGE_MALFORMED_RESPONSE = "Google image generation response malformed";
function normalizeGoogleImageModel(model: string | undefined): string {
const trimmed = model?.trim();
return normalizeGoogleModelId(trimmed || DEFAULT_GOOGLE_IMAGE_MODEL);
}
function resolveGeneratedImageMaxBytes(req: {
cfg: { agents?: { defaults?: { mediaMaxMb?: number } } };
}): number {
const configured = req.cfg.agents?.defaults?.mediaMaxMb;
if (typeof configured === "number" && Number.isFinite(configured) && configured > 0) {
return Math.floor(configured * MB);
}
return MAX_IMAGE_BYTES;
}
function mapSizeToImageConfig(
size: string | undefined,
): { aspectRatio?: string; imageSize?: "2K" | "4K" } | undefined {
const trimmed = size?.trim();
if (!trimmed) {
return undefined;
}
const normalized = normalizeLowercaseStringOrEmpty(trimmed);
const mapping = new Map<string, string>([
["1024x1024", "1:1"],
["1024x1536", "2:3"],
["1536x1024", "3:2"],
["1024x1792", "9:16"],
["1792x1024", "16:9"],
]);
const aspectRatio = mapping.get(normalized);
const [widthRaw, heightRaw] = normalized.split("x");
const width = parseStrictPositiveInteger(widthRaw);
const height = parseStrictPositiveInteger(heightRaw);
if (width === undefined || height === undefined) {
return undefined;
}
const longestEdge = Math.max(width, height);
const imageSize = longestEdge >= 3072 ? "4K" : longestEdge >= 1536 ? "2K" : undefined;
if (!aspectRatio && !imageSize) {
return undefined;
}
return {
...(aspectRatio ? { aspectRatio } : {}),
...(imageSize ? { imageSize } : {}),
};
}
function googleResponseParts(payload: unknown): unknown[] {
if (!isRecord(payload)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
const candidates = payload.candidates;
if (candidates === undefined || candidates === null) {
return [];
}
if (!Array.isArray(candidates)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
const parts: unknown[] = [];
for (const candidate of candidates) {
if (!isRecord(candidate)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
const content = candidate.content;
if (content === undefined || content === null) {
continue;
}
if (!isRecord(content)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
const candidateParts = content.parts;
if (candidateParts === undefined || candidateParts === null) {
continue;
}
if (!Array.isArray(candidateParts)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
parts.push(...candidateParts);
}
return parts;
}
function googleInlineDataFromPart(part: unknown): Record<string, unknown> | undefined {
if (!isRecord(part)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
const inline = part.inlineData ?? part.inline_data;
if (inline === undefined || inline === null) {
return undefined;
}
if (!isRecord(inline)) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
return inline;
}
export function buildGoogleImageGenerationProvider(): ImageGenerationProvider {
return {
id: "google",
label: "Google",
defaultModel: DEFAULT_GOOGLE_IMAGE_MODEL,
models: [DEFAULT_GOOGLE_IMAGE_MODEL, "gemini-3-pro-image-preview"],
isConfigured: ({ agentDir }) =>
isProviderApiKeyConfigured({
provider: "google",
agentDir,
}),
capabilities: {
generate: {
maxCount: GOOGLE_MAX_IMAGE_RESULTS,
supportsSize: true,
supportsAspectRatio: true,
supportsResolution: true,
},
edit: {
enabled: true,
maxCount: GOOGLE_MAX_IMAGE_RESULTS,
maxInputImages: 5,
supportsSize: true,
supportsAspectRatio: true,
supportsResolution: true,
},
geometry: {
sizes: [...GOOGLE_SUPPORTED_SIZES],
aspectRatios: [...GOOGLE_SUPPORTED_ASPECT_RATIOS],
resolutions: ["1K", "2K", "4K"],
},
},
async generateImage(req) {
const auth = await resolveApiKeyForProvider({
provider: "google",
cfg: req.cfg,
agentDir: req.agentDir,
store: req.authStore,
});
if (!auth.apiKey) {
throw new Error("Google API key missing");
}
const model = normalizeGoogleImageModel(req.model);
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: auth.apiKey,
baseUrl: req.cfg?.models?.providers?.google?.baseUrl,
request: sanitizeConfiguredModelProviderRequest(
req.cfg?.models?.providers?.google?.request,
),
capability: "image",
transport: "http",
});
const imageConfig = mapSizeToImageConfig(req.size);
const inputParts = (req.inputImages ?? []).map((image) => ({
inlineData: {
mimeType: image.mimeType,
data: image.buffer.toString("base64"),
},
}));
const resolvedImageConfig = {
...imageConfig,
...(req.aspectRatio?.trim() ? { aspectRatio: req.aspectRatio.trim() } : {}),
...(req.resolution ? { imageSize: req.resolution } : {}),
};
const { response: res, release } = await postJsonRequest({
url: `${baseUrl}/models/${model}:generateContent`,
headers,
body: {
contents: [
{
role: "user",
parts: [...inputParts, { text: req.prompt }],
},
],
generationConfig: {
responseModalities: ["TEXT", "IMAGE"],
...(Object.keys(resolvedImageConfig).length > 0
? { imageConfig: resolvedImageConfig }
: {}),
},
},
timeoutMs: req.timeoutMs ?? DEFAULT_IMAGE_TIMEOUT_MS,
fetchFn: fetch,
pinDns: false,
allowPrivateNetwork,
ssrfPolicy: req.ssrfPolicy,
dispatcherPolicy,
});
try {
await assertOkOrThrowHttpError(res, "Google image generation failed");
const payload = await readProviderJsonResponse(res, "google.image-generation", {
maxBytes: resolveInlineImageJsonResponseMaxBytes(
GOOGLE_MAX_IMAGE_RESULTS,
resolveGeneratedImageMaxBytes(req),
),
});
let imageIndex = 0;
const images: GeneratedImageAsset[] = [];
for (const part of googleResponseParts(payload)) {
const inline = googleInlineDataFromPart(part);
if (!inline) {
continue;
}
const data = normalizeOptionalString(inline.data);
if (!data) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
const image = generatedImageAssetFromBase64({
base64: data,
index: imageIndex,
mimeType:
normalizeOptionalString(inline.mimeType) ??
normalizeOptionalString(inline.mime_type) ??
DEFAULT_OUTPUT_MIME,
});
if (!image) {
throw new Error(GOOGLE_IMAGE_MALFORMED_RESPONSE);
}
imageIndex += 1;
images.push(image);
}
if (images.length === 0) {
throw new Error("Google image generation response missing image data");
}
return {
images,
model,
};
} finally {
await release();
}
},
};
}

View File

@@ -0,0 +1,447 @@
// Google tests cover index plugin behavior.
import { mkdtemp, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import type { Context, Model } from "openclaw/plugin-sdk/llm";
import type {
ProviderReplaySessionEntry,
ProviderSanitizeReplayHistoryContext,
} from "openclaw/plugin-sdk/plugin-entry";
import { createTestPluginApi } from "openclaw/plugin-sdk/plugin-test-api";
import {
registerProviderPlugin,
requireRegisteredProvider,
} from "openclaw/plugin-sdk/plugin-test-runtime";
import { createCapturedThinkingConfigStream } from "openclaw/plugin-sdk/provider-test-contracts";
import type { RealtimeVoiceProviderPlugin } from "openclaw/plugin-sdk/realtime-voice";
import { describe, expect, it, vi } from "vitest";
import { registerGoogleGeminiCliProvider } from "./gemini-cli-provider.js";
import googlePlugin from "./index.js";
import googleProviderDiscovery from "./provider-discovery.js";
import { registerGoogleProvider } from "./provider-registration.js";
const googleProviderPlugin = {
register(api: Parameters<typeof registerGoogleProvider>[0]) {
registerGoogleProvider(api);
registerGoogleGeminiCliProvider(api);
},
};
const refreshGeminiCliOAuthTokenMock = vi.hoisted(() => vi.fn());
vi.mock("./oauth.runtime.js", () => ({
refreshGeminiCliOAuthToken: refreshGeminiCliOAuthTokenMock,
}));
describe("google provider plugin hooks", () => {
it("owns replay policy and reasoning mode for the direct Gemini provider", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google");
const customEntries: ProviderReplaySessionEntry[] = [];
expect(
provider.buildReplayPolicy?.({
provider: "google",
modelApi: "google-generative-ai",
modelId: "gemini-3.1-pro-preview",
} as never),
).toEqual({
sanitizeMode: "full",
sanitizeToolCallIds: true,
toolCallIdMode: "strict",
sanitizeThoughtSignatures: {
allowBase64Only: true,
includeCamelCase: true,
},
repairToolUseResultPairing: true,
applyAssistantFirstOrderingFix: true,
validateGeminiTurns: true,
validateAnthropicTurns: false,
allowSyntheticToolResults: true,
});
expect(
provider.resolveReasoningOutputMode?.({
provider: "google",
modelApi: "google-generative-ai",
modelId: "gemini-3.1-pro-preview",
} as never),
).toBe("native");
expect(
provider.resolveReasoningOutputMode?.({
provider: "google",
modelId: "gemini-3.1-pro-preview",
} as never),
).toBe("native");
const sanitized = await Promise.resolve(
provider.sanitizeReplayHistory?.({
provider: "google",
modelApi: "google-generative-ai",
modelId: "gemini-3.1-pro-preview",
sessionId: "session-1",
messages: [
{
role: "assistant",
content: [{ type: "text", text: "hello" }],
},
],
sessionState: {
getCustomEntries: () => customEntries,
appendCustomEntry: (customType: string, data: unknown) => {
customEntries.push({ customType, data });
},
},
} as ProviderSanitizeReplayHistoryContext),
);
const bootstrapMessage = sanitized?.[0] as
| { role?: string; content?: unknown; timestamp?: unknown }
| undefined;
expect(bootstrapMessage?.role).toBe("user");
expect(bootstrapMessage?.content).toBe("(session bootstrap)");
expect(typeof bootstrapMessage?.timestamp).toBe("number");
expect(sanitized?.[1]).toEqual({
role: "assistant",
content: [{ type: "text", text: "hello" }],
});
expect(customEntries).toHaveLength(1);
expect(customEntries[0]?.customType).toBe("google-turn-ordering-bootstrap");
});
it("keeps google-gemini-cli on tagged reasoning mode", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const cliProvider = requireRegisteredProvider(providers, "google-gemini-cli");
expect(
cliProvider.resolveReasoningOutputMode?.({
provider: "google-gemini-cli",
modelApi: "google-gemini-cli",
modelId: "gemini-2.5-pro",
} as never),
).toBe("tagged");
});
it("keeps google-antigravity hook aliases on tagged reasoning mode", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google-antigravity");
expect(
provider.resolveReasoningOutputMode?.({
provider: "google-antigravity",
modelApi: "openai-completions",
modelId: "gemini-3-pro-low",
} as never),
).toBe("tagged");
});
it("keeps google-vertex hook aliases on native reasoning mode", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google-vertex");
expect(
provider.resolveReasoningOutputMode?.({
provider: "google-vertex",
modelApi: "google-vertex",
modelId: "gemini-3.1-pro-preview",
} as never),
).toBe("native");
expect(
provider.resolveReasoningOutputMode?.({
provider: "google-vertex",
modelId: "gemini-3.1-pro-preview",
} as never),
).toBe("native");
});
it("resolves Google Vertex ADC auth evidence to the config marker", async () => {
const tempDir = await mkdtemp(path.join(os.tmpdir(), "openclaw-google-vertex-config-key-"));
const credentialsPath = path.join(tempDir, "application_default_credentials.json");
await writeFile(
credentialsPath,
JSON.stringify({
type: "authorized_user",
client_id: "client-id",
client_secret: "client-secret",
refresh_token: "refresh-token",
}),
"utf8",
);
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google-vertex");
expect(
provider.resolveConfigApiKey?.({
provider: "google-vertex",
env: {
GOOGLE_APPLICATION_CREDENTIALS: credentialsPath,
GOOGLE_CLOUD_PROJECT: "vertex-project",
GOOGLE_CLOUD_LOCATION: "global",
},
}),
).toBe("gcp-vertex-credentials");
expect(
provider.resolveConfigApiKey?.({
provider: "google-vertex",
env: {
GOOGLE_APPLICATION_CREDENTIALS: credentialsPath,
GOOGLE_CLOUD_PROJECT: "",
GCLOUD_PROJECT: "vertex-project",
GOOGLE_CLOUD_LOCATION: "global",
},
}),
).toBe("gcp-vertex-credentials");
expect(
googleProviderDiscovery.resolveConfigApiKey?.({
provider: "google-vertex",
env: {
GOOGLE_APPLICATION_CREDENTIALS: credentialsPath,
GOOGLE_CLOUD_PROJECT: "vertex-project",
GOOGLE_CLOUD_LOCATION: "global",
},
}),
).toBe("gcp-vertex-credentials");
});
it("owns Gemini tool schema normalization for direct and CLI providers", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const providerIds = ["google", "google-gemini-cli"] as const;
for (const providerId of providerIds) {
const provider = requireRegisteredProvider(providers, providerId);
const [tool] =
provider.normalizeToolSchemas?.({
provider: providerId,
tools: [
{
name: "write_file",
description: "Write a file",
parameters: {
type: "object",
additionalProperties: false,
properties: {
path: { type: "string", pattern: "^src/" },
},
},
},
],
} as never) ?? [];
expect(tool).toEqual({
name: "write_file",
description: "Write a file",
parameters: {
type: "object",
properties: {
path: { type: "string" },
},
},
});
expect(tool?.parameters).not.toHaveProperty("additionalProperties");
expect(
(tool?.parameters as { properties?: { path?: Record<string, unknown> } })?.properties?.path,
).not.toHaveProperty("pattern");
expect(
provider.inspectToolSchemas?.({
provider: providerId,
tools: [tool],
} as never),
).toEqual([]);
}
});
it("wires google-thinking stream hooks for direct and Gemini CLI providers", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const googleProvider = requireRegisteredProvider(providers, "google");
const cliProvider = requireRegisteredProvider(providers, "google-gemini-cli");
const capturedStream = createCapturedThinkingConfigStream();
const runCase = (provider: typeof googleProvider, providerId: string) => {
const wrapped = provider.wrapStreamFn?.({
provider: providerId,
modelId: "gemini-3.1-pro-preview",
thinkingLevel: "high",
streamFn: capturedStream.streamFn,
} as never);
void wrapped?.(
{
api: "google-generative-ai",
provider: providerId,
id: "gemini-3.1-pro-preview",
} as Model<"google-generative-ai">,
{ messages: [] } as Context,
{},
);
const capturedPayload = capturedStream.getCapturedPayload();
expect(capturedPayload).toEqual({
config: {
thinkingConfig: {
thinkingLevel: "HIGH",
},
},
});
const thinkingConfig = (
(capturedPayload as Record<string, unknown>).config as Record<string, unknown>
).thinkingConfig as Record<string, unknown>;
expect(thinkingConfig).not.toHaveProperty("thinkingBudget");
};
runCase(googleProvider, "google");
runCase(cliProvider, "google-gemini-cli");
});
it("wires Vertex transport before request-time metadata ADC detection", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google");
expect(
provider.createStreamFn?.({
model: {
api: "google-vertex",
provider: "google",
id: "gemini-2.5-pro",
},
} as never),
).toEqual(expect.any(Function));
});
it("advertises adaptive thinking for Gemini dynamic thinking", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google");
if (!provider.resolveThinkingProfile) {
throw new Error("expected Google provider thinking profile resolver");
}
const resolveThinkingProfile = provider.resolveThinkingProfile;
const gemini3Profile = resolveThinkingProfile({
provider: "google",
modelId: "gemini-3.1-pro-preview",
} as never);
const gemini25Profile = resolveThinkingProfile({
provider: "google",
modelId: "gemini-2.5-flash",
} as never);
expect(gemini3Profile?.levels).toEqual([
{ id: "off" },
{ id: "low" },
{ id: "adaptive" },
{ id: "high" },
]);
expect(gemini25Profile?.levels).toEqual([
{ id: "off" },
{ id: "minimal" },
{ id: "low" },
{ id: "medium" },
{ id: "adaptive" },
{ id: "high" },
]);
});
it("shares Gemini replay and stream hooks across Google provider variants", async () => {
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const googleProvider = requireRegisteredProvider(providers, "google");
const cliProvider = requireRegisteredProvider(providers, "google-gemini-cli");
expect(googleProvider.buildReplayPolicy).toBe(cliProvider.buildReplayPolicy);
expect(googleProvider.wrapStreamFn).toBe(cliProvider.wrapStreamFn);
});
it("buffers early realtime audio while the lazy Google bridge loads", () => {
let realtimeProvider: RealtimeVoiceProviderPlugin | undefined;
googlePlugin.register(
createTestPluginApi({
registerRealtimeVoiceProvider(provider) {
realtimeProvider = provider;
},
}),
);
const bridge = realtimeProvider?.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio() {},
onClearAudio() {},
});
if (!bridge) {
throw new Error("expected Google realtime bridge");
}
expect(bridge.sendAudio(Buffer.alloc(160))).toBeUndefined();
expect(bridge.setMediaTimestamp(20)).toBeUndefined();
expect(bridge.sendUserMessage?.("hello")).toBeUndefined();
});
it("refreshes Gemini CLI OAuth through the provider-owned refresh hook", async () => {
refreshGeminiCliOAuthTokenMock.mockResolvedValueOnce({
type: "oauth",
provider: "google-gemini-cli",
access: "fresh-access",
refresh: "fresh-refresh",
expires: Date.now() + 60_000,
email: "user@example.com",
projectId: "project-1",
});
const { providers } = await registerProviderPlugin({
plugin: googleProviderPlugin,
id: "google",
name: "Google Provider",
});
const provider = requireRegisteredProvider(providers, "google-gemini-cli");
const credential = {
type: "oauth" as const,
provider: "google-gemini-cli",
access: "stale-access",
refresh: "stale-refresh",
expires: Date.now() - 60_000,
email: "user@example.com",
projectId: "project-1",
};
await expect(provider.refreshOAuth?.(credential)).resolves.toMatchObject({
access: "fresh-access",
refresh: "fresh-refresh",
email: "user@example.com",
projectId: "project-1",
});
expect(refreshGeminiCliOAuthTokenMock).toHaveBeenCalledWith(credential);
});
});

355
extensions/google/index.ts Normal file
View File

@@ -0,0 +1,355 @@
// Google plugin entrypoint registers its OpenClaw integration.
import type { ImageGenerationProvider } from "openclaw/plugin-sdk/image-generation";
import type { MediaUnderstandingProvider } from "openclaw/plugin-sdk/media-understanding";
import type { MusicGenerationProvider } from "openclaw/plugin-sdk/music-generation";
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
import type {
RealtimeVoiceBridge,
RealtimeVoiceBridgeCreateRequest,
RealtimeVoiceProviderConfig,
RealtimeVoiceProviderPlugin,
} from "openclaw/plugin-sdk/realtime-voice";
import { normalizeResolvedSecretInputString } from "openclaw/plugin-sdk/secret-input";
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
import type { VideoGenerationProvider } from "openclaw/plugin-sdk/video-generation";
import { buildGoogleGeminiCliBackend } from "./cli-backend.js";
import { registerGoogleGeminiCliProvider } from "./gemini-cli-provider.js";
import {
createGoogleMusicGenerationProviderMetadata,
createGoogleVideoGenerationProviderMetadata,
} from "./generation-provider-metadata.js";
import { geminiMemoryEmbeddingProviderAdapter } from "./memory-embedding-adapter.js";
import { registerGoogleProvider } from "./provider-registration.js";
import { buildGoogleSpeechProvider } from "./speech-provider.js";
import { createGeminiWebSearchProvider } from "./src/gemini-web-search-provider.js";
let googleImageGenerationProviderPromise: Promise<ImageGenerationProvider> | null = null;
let googleMediaUnderstandingProviderPromise: Promise<MediaUnderstandingProvider> | null = null;
let googleMusicGenerationProviderPromise: Promise<MusicGenerationProvider> | null = null;
let googleRealtimeVoiceProviderPromise: Promise<RealtimeVoiceProviderPlugin> | null = null;
let googleVideoGenerationProviderPromise: Promise<VideoGenerationProvider> | null = null;
type GoogleMediaUnderstandingProvider = Required<
Pick<
MediaUnderstandingProvider,
"describeImage" | "describeImages" | "transcribeAudio" | "describeVideo"
>
>;
async function loadGoogleImageGenerationProvider(): Promise<ImageGenerationProvider> {
if (!googleImageGenerationProviderPromise) {
googleImageGenerationProviderPromise = import("./image-generation-provider.js").then((mod) =>
mod.buildGoogleImageGenerationProvider(),
);
}
return await googleImageGenerationProviderPromise;
}
async function loadGoogleMediaUnderstandingProvider(): Promise<MediaUnderstandingProvider> {
if (!googleMediaUnderstandingProviderPromise) {
googleMediaUnderstandingProviderPromise = import("./media-understanding-provider.js").then(
(mod) => mod.googleMediaUnderstandingProvider,
);
}
return await googleMediaUnderstandingProviderPromise;
}
async function loadGoogleMusicGenerationProvider(): Promise<MusicGenerationProvider> {
if (!googleMusicGenerationProviderPromise) {
googleMusicGenerationProviderPromise = import("./music-generation-provider.js").then((mod) =>
mod.buildGoogleMusicGenerationProvider(),
);
}
return await googleMusicGenerationProviderPromise;
}
async function loadGoogleRealtimeVoiceProvider(): Promise<RealtimeVoiceProviderPlugin> {
if (!googleRealtimeVoiceProviderPromise) {
googleRealtimeVoiceProviderPromise = import("./realtime-voice-provider.js").then((mod) =>
mod.buildGoogleRealtimeVoiceProvider(),
);
}
return await googleRealtimeVoiceProviderPromise;
}
async function loadGoogleVideoGenerationProvider(): Promise<VideoGenerationProvider> {
if (!googleVideoGenerationProviderPromise) {
googleVideoGenerationProviderPromise = import("./video-generation-provider.js").then((mod) =>
mod.buildGoogleVideoGenerationProvider(),
);
}
return await googleVideoGenerationProviderPromise;
}
async function loadGoogleRequiredMediaUnderstandingProvider(): Promise<GoogleMediaUnderstandingProvider> {
const provider = await loadGoogleMediaUnderstandingProvider();
if (
!provider.describeImage ||
!provider.describeImages ||
!provider.transcribeAudio ||
!provider.describeVideo
) {
throw new Error("google media understanding provider missing required handlers");
}
return provider as GoogleMediaUnderstandingProvider;
}
function createLazyGoogleImageGenerationProvider(): ImageGenerationProvider {
return {
id: "google",
label: "Google",
defaultModel: "gemini-3.1-flash-image-preview",
models: ["gemini-3.1-flash-image-preview", "gemini-3-pro-image-preview"],
capabilities: {
generate: {
maxCount: 4,
supportsSize: true,
supportsAspectRatio: true,
supportsResolution: true,
},
edit: {
enabled: true,
maxCount: 4,
maxInputImages: 5,
supportsSize: true,
supportsAspectRatio: true,
supportsResolution: true,
},
geometry: {
sizes: ["1024x1024", "1024x1536", "1536x1024", "1024x1792", "1792x1024"],
aspectRatios: ["1:1", "2:3", "3:2", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"],
resolutions: ["1K", "2K", "4K"],
},
},
generateImage: async (req) => (await loadGoogleImageGenerationProvider()).generateImage(req),
};
}
function createLazyGoogleMediaUnderstandingProvider(): MediaUnderstandingProvider {
return {
id: "google",
capabilities: ["image", "audio", "video"],
defaultModels: {
image: "gemini-3-flash-preview",
audio: "gemini-3-flash-preview",
video: "gemini-3-flash-preview",
},
autoPriority: { image: 30, audio: 40, video: 10 },
nativeDocumentInputs: ["pdf"],
describeImage: async (...args) =>
await (await loadGoogleRequiredMediaUnderstandingProvider()).describeImage(...args),
describeImages: async (...args) =>
await (await loadGoogleRequiredMediaUnderstandingProvider()).describeImages(...args),
transcribeAudio: async (...args) =>
await (await loadGoogleRequiredMediaUnderstandingProvider()).transcribeAudio(...args),
describeVideo: async (...args) =>
await (await loadGoogleRequiredMediaUnderstandingProvider()).describeVideo(...args),
};
}
function createLazyGoogleMusicGenerationProvider(): MusicGenerationProvider {
return {
...createGoogleMusicGenerationProviderMetadata(),
generateMusic: async (...args) =>
await (await loadGoogleMusicGenerationProvider()).generateMusic(...args),
};
}
function resolveGoogleRealtimeProviderConfig(
rawConfig: RealtimeVoiceProviderConfig,
cfg?: { models?: { providers?: { google?: { apiKey?: unknown } } } },
): RealtimeVoiceProviderConfig {
const providers =
typeof rawConfig.providers === "object" &&
rawConfig.providers !== null &&
!Array.isArray(rawConfig.providers)
? (rawConfig.providers as Record<string, unknown>)
: undefined;
const nested = providers?.google;
const raw =
typeof nested === "object" && nested !== null && !Array.isArray(nested)
? (nested as Record<string, unknown>)
: typeof rawConfig.google === "object" &&
rawConfig.google !== null &&
!Array.isArray(rawConfig.google)
? (rawConfig.google as Record<string, unknown>)
: rawConfig;
return {
...raw,
...(raw.apiKey === undefined
? cfg?.models?.providers?.google?.apiKey === undefined
? {}
: {
apiKey: normalizeResolvedSecretInputString({
value: cfg.models.providers.google.apiKey,
path: "models.providers.google.apiKey",
}),
}
: {
apiKey: normalizeResolvedSecretInputString({
value: raw.apiKey,
path: "plugins.entries.voice-call.config.realtime.providers.google.apiKey",
}),
}),
};
}
function resolveGoogleRealtimeEnvApiKey(): string | undefined {
return (
normalizeOptionalString(process.env.GEMINI_API_KEY) ??
normalizeOptionalString(process.env.GOOGLE_API_KEY)
);
}
const GOOGLE_REALTIME_LAZY_MAX_PENDING_AUDIO_CHUNKS = 320;
function createLazyGoogleRealtimeVoiceBridge(
req: RealtimeVoiceBridgeCreateRequest,
): RealtimeVoiceBridge {
let bridge: RealtimeVoiceBridge | undefined;
let bridgePromise: Promise<RealtimeVoiceBridge> | undefined;
let closed = false;
let latestMediaTimestamp: number | undefined;
let pendingGreeting: string | undefined;
const pendingAudio: Buffer[] = [];
const pendingUserMessages: string[] = [];
const loadBridge = async () => {
if (!bridgePromise) {
bridgePromise = loadGoogleRealtimeVoiceProvider().then((provider) =>
provider.createBridge(req),
);
}
bridge = await bridgePromise;
return bridge;
};
const requireBridge = () => {
if (!bridge) {
throw new Error("Google realtime voice bridge is not connected");
}
return bridge;
};
const flushPending = (loadedBridge: RealtimeVoiceBridge) => {
if (typeof latestMediaTimestamp === "number") {
loadedBridge.setMediaTimestamp(latestMediaTimestamp);
}
for (const audio of pendingAudio.splice(0)) {
loadedBridge.sendAudio(audio);
}
for (const text of pendingUserMessages.splice(0)) {
loadedBridge.sendUserMessage?.(text);
}
if (pendingGreeting !== undefined) {
const greeting = pendingGreeting;
pendingGreeting = undefined;
loadedBridge.triggerGreeting?.(greeting);
}
};
return {
supportsToolResultContinuation: true,
connect: async () => {
const loadedBridge = await loadBridge();
if (closed) {
loadedBridge.close();
return;
}
await loadedBridge.connect();
flushPending(loadedBridge);
},
sendAudio: (audio) => {
if (bridge) {
bridge.sendAudio(audio);
return;
}
if (!closed) {
if (pendingAudio.length >= GOOGLE_REALTIME_LAZY_MAX_PENDING_AUDIO_CHUNKS) {
pendingAudio.shift();
}
pendingAudio.push(audio);
}
},
setMediaTimestamp: (ts) => {
latestMediaTimestamp = ts;
bridge?.setMediaTimestamp(ts);
},
sendUserMessage: (text) => {
if (bridge) {
bridge.sendUserMessage?.(text);
return;
}
if (!closed) {
pendingUserMessages.push(text);
}
},
triggerGreeting: (instructions) => {
if (bridge) {
bridge.triggerGreeting?.(instructions);
return;
}
if (!closed) {
pendingGreeting = instructions;
}
},
handleBargeIn: (options) => requireBridge().handleBargeIn?.(options),
submitToolResult: (callId, result, options) =>
requireBridge().submitToolResult(callId, result, options),
acknowledgeMark: () => requireBridge().acknowledgeMark(),
close: () => {
closed = true;
pendingAudio.length = 0;
pendingUserMessages.length = 0;
pendingGreeting = undefined;
bridge?.close();
},
isConnected: () => bridge?.isConnected() ?? false,
};
}
function createLazyGoogleRealtimeVoiceProvider(): RealtimeVoiceProviderPlugin {
return {
id: "google",
label: "Google Live Voice",
autoSelectOrder: 20,
resolveConfig: ({ cfg, rawConfig }) => resolveGoogleRealtimeProviderConfig(rawConfig, cfg),
isConfigured: ({ cfg, providerConfig }) =>
Boolean(
normalizeOptionalString(providerConfig.apiKey) ??
normalizeOptionalString(cfg?.models?.providers?.google?.apiKey) ??
resolveGoogleRealtimeEnvApiKey(),
),
createBridge: createLazyGoogleRealtimeVoiceBridge,
createBrowserSession: async (req) => {
const provider = await loadGoogleRealtimeVoiceProvider();
if (!provider.createBrowserSession) {
throw new Error("Google realtime voice browser sessions are unavailable");
}
return await provider.createBrowserSession(req);
},
};
}
function createLazyGoogleVideoGenerationProvider(): VideoGenerationProvider {
return {
...createGoogleVideoGenerationProviderMetadata(),
generateVideo: async (...args) =>
await (await loadGoogleVideoGenerationProvider()).generateVideo(...args),
};
}
export default definePluginEntry({
id: "google",
name: "Google Plugin",
description: "Bundled Google plugin",
register(api) {
api.registerCliBackend(buildGoogleGeminiCliBackend());
registerGoogleGeminiCliProvider(api);
registerGoogleProvider(api);
api.registerMemoryEmbeddingProvider(geminiMemoryEmbeddingProviderAdapter);
api.registerImageGenerationProvider(createLazyGoogleImageGenerationProvider());
api.registerMediaUnderstandingProvider(createLazyGoogleMediaUnderstandingProvider());
api.registerMusicGenerationProvider(createLazyGoogleMusicGenerationProvider());
api.registerRealtimeVoiceProvider(createLazyGoogleRealtimeVoiceProvider());
api.registerSpeechProvider(buildGoogleSpeechProvider());
api.registerVideoGenerationProvider(createLazyGoogleVideoGenerationProvider());
api.registerWebSearchProvider(createGeminiWebSearchProvider());
},
});

View File

@@ -0,0 +1,105 @@
// Google tests cover manifest plugin behavior.
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
type GoogleManifest = {
modelIdNormalization?: {
providers?: Record<
string,
{
aliases?: Record<string, string>;
}
>;
};
modelCatalog?: {
suppressions?: Array<{
provider?: string;
model?: string;
reason?: string;
}>;
};
};
const RETIRED_GEMINI_CHAT_MODELS = [
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-1.5-pro",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-live-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-2.5-flash-exp-native-audio-thinking-dialog",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-lite-preview-09-25",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-09-25",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-preview-native-audio-dialog",
"gemini-2.5-pro-exp-03-25",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-3-pro-preview",
"gemini-3.1-pro-preview-customtools",
"gemini-live-2.5-flash",
"gemini-live-2.5-flash-preview",
"gemini-live-2.5-flash-preview-native-audio",
] as const;
const GOOGLE_CHAT_PROVIDERS = ["google", "google-gemini-cli", "google-vertex"] as const;
function loadManifest(): GoogleManifest {
return JSON.parse(readFileSync(new URL("./openclaw.plugin.json", import.meta.url), "utf8"));
}
describe("google manifest model catalog", () => {
it("suppresses retired Gemini chat model identifiers for all Google chat providers", () => {
const manifest = loadManifest();
const suppressionRefs = new Set(
(manifest.modelCatalog?.suppressions ?? []).map(
(suppression) => `${suppression.provider}/${suppression.model}`,
),
);
for (const provider of GOOGLE_CHAT_PROVIDERS) {
for (const model of RETIRED_GEMINI_CHAT_MODELS) {
expect(suppressionRefs).toContain(`${provider}/${model}`);
}
}
});
it("does not suppress still-callable Google replacement models", () => {
const manifest = loadManifest();
const suppressionRefs = new Set(
(manifest.modelCatalog?.suppressions ?? []).map(
(suppression) => `${suppression.provider}/${suppression.model}`,
),
);
expect(suppressionRefs).not.toContain("google/gemini-2.0-flash");
expect(suppressionRefs).not.toContain("google/gemini-2.5-flash");
expect(suppressionRefs).not.toContain("google/gemini-2.5-flash-lite");
expect(suppressionRefs).not.toContain("google/gemini-2.5-pro");
expect(suppressionRefs).not.toContain("google/gemini-3.1-pro-preview");
});
it("normalizes retired Gemini 3 Pro aliases for all Google chat providers", () => {
const manifest = loadManifest();
for (const provider of GOOGLE_CHAT_PROVIDERS) {
const aliases = manifest.modelIdNormalization?.providers?.[provider]?.aliases;
expect(aliases?.["gemini-3-pro"]).toBe("gemini-3.1-pro-preview");
expect(aliases?.["gemini-3-pro-preview"]).toBe("gemini-3.1-pro-preview");
}
});
});

View File

@@ -0,0 +1,164 @@
// Google provider module implements model/runtime integration.
import {
describeImageWithModel,
describeImagesWithModel,
type AudioTranscriptionRequest,
type AudioTranscriptionResult,
type MediaUnderstandingProvider,
type VideoDescriptionRequest,
type VideoDescriptionResult,
} from "openclaw/plugin-sdk/media-understanding";
import {
assertOkOrThrowProviderError,
postJsonRequest,
readProviderJsonResponse,
type ProviderRequestTransportOverrides,
} from "openclaw/plugin-sdk/provider-http";
import {
DEFAULT_GOOGLE_API_BASE_URL,
normalizeGoogleModelId,
resolveGoogleGenerativeAiHttpRequestConfig,
} from "./runtime-api.js";
const DEFAULT_GOOGLE_AUDIO_MODEL = "gemini-3-flash-preview";
const DEFAULT_GOOGLE_VIDEO_MODEL = "gemini-3-flash-preview";
const DEFAULT_GOOGLE_AUDIO_PROMPT = "Transcribe the audio.";
const DEFAULT_GOOGLE_VIDEO_PROMPT = "Describe the video.";
async function generateGeminiInlineDataText(params: {
buffer: Buffer;
mime?: string;
apiKey: string;
baseUrl?: string;
headers?: Record<string, string>;
request?: ProviderRequestTransportOverrides;
model?: string;
prompt?: string;
timeoutMs: number;
fetchFn?: typeof fetch;
defaultBaseUrl: string;
defaultModel: string;
defaultPrompt: string;
defaultMime: string;
httpErrorLabel: string;
missingTextError: string;
}): Promise<{ text: string; model: string }> {
const fetchFn = params.fetchFn ?? fetch;
const model = (() => {
const trimmed = params.model?.trim();
if (!trimmed) {
return params.defaultModel;
}
return normalizeGoogleModelId(trimmed);
})();
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: params.apiKey,
baseUrl: params.baseUrl,
headers: params.headers,
request: params.request,
capability: params.defaultMime.startsWith("audio/") ? "audio" : "video",
transport: "media-understanding",
});
const resolvedBaseUrl = baseUrl ?? params.defaultBaseUrl;
const url = `${resolvedBaseUrl}/models/${model}:generateContent`;
const prompt = (() => {
const trimmed = params.prompt?.trim();
return trimmed || params.defaultPrompt;
})();
const body = {
contents: [
{
role: "user",
parts: [
{ text: prompt },
{
inline_data: {
mime_type: params.mime ?? params.defaultMime,
data: params.buffer.toString("base64"),
},
},
],
},
],
};
const { response: res, release } = await postJsonRequest({
url,
headers,
body,
timeoutMs: params.timeoutMs,
fetchFn,
allowPrivateNetwork,
dispatcherPolicy,
});
try {
await assertOkOrThrowProviderError(res, params.httpErrorLabel);
const payload = await readProviderJsonResponse<{
candidates?: Array<{
content?: { parts?: Array<{ text?: string }> };
}>;
}>(res, params.httpErrorLabel);
const parts = payload.candidates?.[0]?.content?.parts ?? [];
const text = parts
.map((part) => part?.text?.trim())
.filter(Boolean)
.join("\n");
if (!text) {
throw new Error(params.missingTextError);
}
return { text, model };
} finally {
await release();
}
}
export async function transcribeGeminiAudio(
params: AudioTranscriptionRequest,
): Promise<AudioTranscriptionResult> {
const { text, model } = await generateGeminiInlineDataText({
...params,
defaultBaseUrl: DEFAULT_GOOGLE_API_BASE_URL,
defaultModel: DEFAULT_GOOGLE_AUDIO_MODEL,
defaultPrompt: DEFAULT_GOOGLE_AUDIO_PROMPT,
defaultMime: "audio/wav",
httpErrorLabel: "Audio transcription failed",
missingTextError: "Audio transcription response missing text",
});
return { text, model };
}
export async function describeGeminiVideo(
params: VideoDescriptionRequest,
): Promise<VideoDescriptionResult> {
const { text, model } = await generateGeminiInlineDataText({
...params,
defaultBaseUrl: DEFAULT_GOOGLE_API_BASE_URL,
defaultModel: DEFAULT_GOOGLE_VIDEO_MODEL,
defaultPrompt: DEFAULT_GOOGLE_VIDEO_PROMPT,
defaultMime: "video/mp4",
httpErrorLabel: "Video description failed",
missingTextError: "Video description response missing text",
});
return { text, model };
}
export const googleMediaUnderstandingProvider: MediaUnderstandingProvider = {
id: "google",
capabilities: ["image", "audio", "video"],
defaultModels: {
image: DEFAULT_GOOGLE_VIDEO_MODEL,
audio: DEFAULT_GOOGLE_AUDIO_MODEL,
video: DEFAULT_GOOGLE_VIDEO_MODEL,
},
autoPriority: { image: 30, audio: 40, video: 10 },
nativeDocumentInputs: ["pdf"],
describeImage: describeImageWithModel,
describeImages: describeImagesWithModel,
transcribeAudio: transcribeGeminiAudio,
describeVideo: describeGeminiVideo,
};

View File

@@ -0,0 +1,226 @@
// Google tests cover media understanding provider.video plugin behavior.
import { createServer, type Server } from "node:http";
import {
createRequestCaptureJsonFetch,
installPinnedHostnameTestHooks,
withFetchPreconnect,
} from "openclaw/plugin-sdk/test-env";
import { describe, expect, it } from "vitest";
import { describeGeminiVideo, transcribeGeminiAudio } from "./media-understanding-provider.js";
import { resolveGoogleGenerativeAiHttpRequestConfig } from "./runtime-api.js";
installPinnedHostnameTestHooks();
const LOOPBACK_RESPONSE_BYTES = 18 * 1024 * 1024;
async function listenLoopbackServer(server: Server): Promise<number> {
return await new Promise((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => {
server.off("error", reject);
const address = server.address();
if (!address || typeof address === "string") {
reject(new Error("expected loopback TCP address"));
return;
}
resolve(address.port);
});
});
}
function createOversizedJsonServer(): { server: Server; closed: Promise<number> } {
let resolveClosed: (sentBytes: number) => void = () => {};
const closed = new Promise<number>((resolve) => {
resolveClosed = resolve;
});
const server = createServer((_req, res) => {
let sentBytes = 0;
const chunk = Buffer.alloc(64 * 1024, 0x20);
res.writeHead(200, { "content-type": "application/json" });
const timer = setInterval(() => {
if (sentBytes >= LOOPBACK_RESPONSE_BYTES) {
clearInterval(timer);
res.end();
return;
}
sentBytes += chunk.length;
res.write(chunk);
}, 1);
res.on("close", () => {
clearInterval(timer);
resolveClosed(sentBytes);
});
});
return { server, closed };
}
describe("describeGeminiVideo", () => {
it("respects case-insensitive x-goog-api-key overrides", async () => {
let seenKey: string | null = null;
const fetchFn = withFetchPreconnect(async (_input: RequestInfo | URL, init?: RequestInit) => {
const headers = new Headers(init?.headers);
seenKey = headers.get("x-goog-api-key");
return new Response(
JSON.stringify({
candidates: [{ content: { parts: [{ text: "video ok" }] } }],
}),
{ status: 200, headers: { "content-type": "application/json" } },
);
});
const result = await describeGeminiVideo({
buffer: Buffer.from("video"),
fileName: "clip.mp4",
apiKey: "test-key",
timeoutMs: 1000,
headers: { "X-Goog-Api-Key": "override" },
fetchFn,
});
expect(seenKey).toBe("override");
expect(result.text).toBe("video ok");
});
it("keeps private-network disabled for the default Google media endpoint", async () => {
expect(
resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: "test-key",
capability: "video",
transport: "media-understanding",
}).allowPrivateNetwork,
).toBe(false);
const fetchFn = withFetchPreconnect(async () => {
return new Response(
JSON.stringify({
candidates: [{ content: { parts: [{ text: "video ok" }] } }],
}),
{ status: 200, headers: { "content-type": "application/json" } },
);
});
await describeGeminiVideo({
buffer: Buffer.from("video"),
fileName: "clip.mp4",
apiKey: "test-key",
timeoutMs: 1000,
fetchFn,
});
});
it("builds the expected request payload", async () => {
const { fetchFn, getRequest } = createRequestCaptureJsonFetch({
candidates: [
{
content: {
parts: [{ text: "first" }, { text: " second " }, { text: "" }],
},
},
],
});
const result = await describeGeminiVideo({
buffer: Buffer.from("video-bytes"),
fileName: "clip.mp4",
apiKey: "test-key",
timeoutMs: 1500,
baseUrl: "https://generativelanguage.googleapis.com/v1beta/",
model: "gemini-3-pro",
headers: { "X-Other": "1" },
fetchFn,
});
const { url: seenUrl, init: seenInit } = getRequest();
expect(result.model).toBe("gemini-3.1-pro-preview");
expect(result.text).toBe("first\nsecond");
expect(seenUrl).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-pro-preview:generateContent",
);
expect(seenInit?.method).toBe("POST");
expect(seenInit?.signal).toBeInstanceOf(AbortSignal);
const headers = new Headers(seenInit?.headers);
expect(headers.get("x-goog-api-key")).toBe("test-key");
expect(headers.get("content-type")).toBe("application/json");
expect(headers.get("x-other")).toBe("1");
const bodyText =
typeof seenInit?.body === "string"
? seenInit.body
: Buffer.isBuffer(seenInit?.body)
? seenInit.body.toString("utf8")
: "";
const body = JSON.parse(bodyText);
expect(body.contents?.[0]?.parts?.[0]?.text).toBe("Describe the video.");
expect(body.contents?.[0]?.parts?.[1]?.inline_data?.mime_type).toBe("video/mp4");
expect(body.contents?.[0]?.parts?.[1]?.inline_data?.data).toBe(
Buffer.from("video-bytes").toString("base64"),
);
});
it("bounds oversized video JSON responses and closes the stream early", async () => {
const { server, closed } = createOversizedJsonServer();
const port = await listenLoopbackServer(server);
const fetchFn = withFetchPreconnect(async () =>
fetch(`http://127.0.0.1:${port}/google-video-json`),
);
try {
await expect(
describeGeminiVideo({
buffer: Buffer.from("video-bytes"),
fileName: "clip.mp4",
apiKey: "test-key",
timeoutMs: 1500,
fetchFn,
}),
).rejects.toThrow(/JSON response exceeds 16777216 bytes/u);
await expect(closed).resolves.toBeLessThan(LOOPBACK_RESPONSE_BYTES);
} finally {
server.close();
}
});
it("rejects non-Google video base URLs before sending authenticated requests", async () => {
await expect(
describeGeminiVideo({
buffer: Buffer.from("video-bytes"),
fileName: "clip.mp4",
apiKey: "test-key",
timeoutMs: 1500,
baseUrl: "https://example.com/v1beta/",
fetchFn: async () => {
throw new Error("fetch should not run");
},
}),
).rejects.toThrow(
"Google Generative AI baseUrl must use https://generativelanguage.googleapis.com",
);
});
it("formats Google audio transcription HTTP errors with provider details", async () => {
await expect(
transcribeGeminiAudio({
buffer: Buffer.from("audio-bytes"),
fileName: "clip.wav",
apiKey: "test-key",
timeoutMs: 1500,
fetchFn: async () =>
new Response(
JSON.stringify({
error: {
message: "Unsupported audio",
status: "INVALID_ARGUMENT",
},
}),
{
status: 400,
headers: { "x-request-id": "google_audio_req" },
},
),
}),
).rejects.toThrow(
"Audio transcription failed (400): Unsupported audio [code=INVALID_ARGUMENT] [request_id=google_audio_req]",
);
});
});

View File

@@ -0,0 +1,80 @@
// Google plugin module implements memory embedding adapter behavior.
import {
hasNonTextEmbeddingParts,
isMissingEmbeddingApiKeyError,
mapBatchEmbeddingsByIndex,
sanitizeEmbeddingCacheHeaders,
type MemoryEmbeddingProviderAdapter,
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
import { runGeminiEmbeddingBatches } from "./embedding-batch.js";
import {
buildGeminiEmbeddingRequest,
createGeminiEmbeddingProvider,
DEFAULT_GEMINI_EMBEDDING_MODEL,
} from "./embedding-provider.js";
function supportsGeminiMultimodalEmbeddings(model: string): boolean {
const normalized = model
.trim()
.replace(/^models\//, "")
.replace(/^(gemini|google)\//, "");
return normalized === "gemini-embedding-2-preview";
}
export const geminiMemoryEmbeddingProviderAdapter: MemoryEmbeddingProviderAdapter = {
id: "gemini",
defaultModel: DEFAULT_GEMINI_EMBEDDING_MODEL,
transport: "remote",
authProviderId: "google",
autoSelectPriority: 30,
allowExplicitWhenConfiguredAuto: true,
supportsMultimodalEmbeddings: ({ model }) => supportsGeminiMultimodalEmbeddings(model),
shouldContinueAutoSelection: isMissingEmbeddingApiKeyError,
create: async (options) => {
const { provider, client } = await createGeminiEmbeddingProvider({
...options,
provider: "gemini",
fallback: "none",
});
return {
provider,
runtime: {
id: "gemini",
cacheKeyData: {
provider: "gemini",
baseUrl: client.baseUrl,
model: client.model,
outputDimensionality: client.outputDimensionality,
headers: sanitizeEmbeddingCacheHeaders(client.headers, [
"authorization",
"x-goog-api-key",
]),
},
batchEmbed: async (batch) => {
if (batch.chunks.some((chunk) => hasNonTextEmbeddingParts(chunk.embeddingInput))) {
return null;
}
const byCustomId = await runGeminiEmbeddingBatches({
gemini: client,
agentId: batch.agentId,
requests: batch.chunks.map((chunk, index) => ({
custom_id: String(index),
request: buildGeminiEmbeddingRequest({
input: chunk.embeddingInput ?? { text: chunk.text },
taskType: "RETRIEVAL_DOCUMENT",
modelPath: client.modelPath,
outputDimensionality: client.outputDimensionality,
}),
})),
wait: batch.wait,
concurrency: batch.concurrency,
pollIntervalMs: batch.pollIntervalMs,
timeoutMs: batch.timeoutMs,
debug: batch.debug,
});
return mapBatchEmbeddingsByIndex(byCustomId, batch.chunks.length);
},
},
};
},
};

View File

@@ -0,0 +1,49 @@
// Google tests cover model id plugin behavior.
import { describe, expect, it } from "vitest";
import { normalizeAntigravityModelId, normalizeGoogleModelId } from "./api.js";
describe("google model id helpers", () => {
it.each(["gemini-3-pro", "gemini-3.1-pro", "gemini-3-1-pro"])(
"adds default -low suffix to bare antigravity pro id: %s",
(id) => {
expect(normalizeAntigravityModelId(id)).toBe(`${id}-low`);
},
);
it.each([
"gemini-3-pro-low",
"gemini-3-pro-high",
"gemini-3.1-flash",
"claude-opus-4-6-thinking",
])("keeps already-tiered and non-pro ids unchanged: %s", (id) => {
expect(normalizeAntigravityModelId(id)).toBe(id);
});
it("maps the deprecated 3.1 flash alias to the real preview model", () => {
expect(normalizeGoogleModelId("gemini-3.1-flash")).toBe("gemini-3-flash-preview");
expect(normalizeGoogleModelId("gemini-3.1-flash-preview")).toBe("gemini-3-flash-preview");
});
it("keeps bare Gemini 3.1 Pro as an alias for Google's preview-suffixed API id", () => {
expect(normalizeGoogleModelId("gemini-3-pro")).toBe("gemini-3.1-pro-preview");
expect(normalizeGoogleModelId("gemini-3-pro-preview")).toBe("gemini-3.1-pro-preview");
expect(normalizeGoogleModelId("gemini-3.1-pro")).toBe("gemini-3.1-pro-preview");
expect(normalizeGoogleModelId("gemini-3.1-pro-preview")).toBe("gemini-3.1-pro-preview");
});
it("normalizes provider-prefixed Gemini 3 Pro config ids", () => {
expect(normalizeGoogleModelId("google/gemini-3-pro-preview")).toBe(
"google/gemini-3.1-pro-preview",
);
});
it("keeps GA gemini-3.1-flash-lite unchanged and maps old preview name to GA", () => {
expect(normalizeGoogleModelId("gemini-3.1-flash-lite")).toBe("gemini-3.1-flash-lite");
expect(normalizeGoogleModelId("gemini-3.1-flash-lite-preview")).toBe("gemini-3.1-flash-lite");
});
it("maps the old Gemma 4 26B shorthand to Google's canonical API id", () => {
expect(normalizeGoogleModelId("gemma-4-26b")).toBe("gemma-4-26b-a4b-it");
expect(normalizeGoogleModelId("google/gemma-4-26b")).toBe("google/gemma-4-26b-a4b-it");
});
});

View File

@@ -0,0 +1,46 @@
// Google plugin module implements model id behavior.
const ANTIGRAVITY_BARE_PRO_IDS = new Set(["gemini-3-pro", "gemini-3.1-pro", "gemini-3-1-pro"]);
const GOOGLE_PROVIDER_PREFIX = "google/";
export function stripGoogleProviderPrefix(id: string): string {
return id.startsWith(GOOGLE_PROVIDER_PREFIX) ? id.slice(GOOGLE_PROVIDER_PREFIX.length) : id;
}
export function normalizeGoogleModelId(id: string): string {
if (id.startsWith(GOOGLE_PROVIDER_PREFIX)) {
const modelId = stripGoogleProviderPrefix(id);
const normalizedModelId = normalizeGoogleModelId(modelId);
return normalizedModelId === modelId ? id : `${GOOGLE_PROVIDER_PREFIX}${normalizedModelId}`;
}
if (id === "gemini-3-pro" || id === "gemini-3-pro-preview") {
return "gemini-3.1-pro-preview";
}
if (id === "gemini-3-flash") {
return "gemini-3-flash-preview";
}
// Google exposes Gemini 3.1 Pro in the Gemini API as the preview-suffixed id.
// Keep the bare form as a user convenience alias, not as a canonical API id.
if (id === "gemini-3.1-pro") {
return "gemini-3.1-pro-preview";
}
// Gemini 3.1 Flash Lite graduated to GA on 2026-05-07; the -preview
// endpoint is deprecated (shutdown 2026-05-25). Map old preview name
// to the stable GA id.
if (id === "gemini-3.1-flash-lite-preview") {
return "gemini-3.1-flash-lite";
}
if (id === "gemini-3.1-flash" || id === "gemini-3.1-flash-preview") {
return "gemini-3-flash-preview";
}
if (id === "gemma-4-26b") {
return "gemma-4-26b-a4b-it";
}
return id;
}
export function normalizeAntigravityModelId(id: string): string {
if (ANTIGRAVITY_BARE_PRO_IDS.has(id)) {
return `${id}-low`;
}
return id;
}

View File

@@ -0,0 +1,282 @@
// Google tests cover music generation provider plugin behavior.
import { afterAll, afterEach, describe, expect, it, vi } from "vitest";
const { createGoogleGenAIMock, generateContentMock } = vi.hoisted(() => {
const generateContentMockLocal = vi.fn();
const createGoogleGenAIMockLocal = vi.fn(() => {
return {
models: {
generateContent: generateContentMockLocal,
},
};
});
return {
createGoogleGenAIMock: createGoogleGenAIMockLocal,
generateContentMock: generateContentMockLocal,
};
});
vi.mock("./google-genai-runtime.js", () => ({
createGoogleGenAI: createGoogleGenAIMock,
}));
import * as providerAuthRuntime from "openclaw/plugin-sdk/provider-auth-runtime";
import { expectExplicitMusicGenerationCapabilities } from "openclaw/plugin-sdk/provider-test-contracts";
import { buildGoogleMusicGenerationProvider } from "./music-generation-provider.js";
type GoogleGenAIConfig = {
apiKey?: string;
httpOptions?: {
baseUrl?: string;
};
};
type GenerateContentRequest = {
model?: string;
config?: unknown;
};
function lastGoogleGenAIConfig(): GoogleGenAIConfig {
const calls = createGoogleGenAIMock.mock.calls as unknown[][];
const config = calls.at(-1)?.[0];
if (!config) {
throw new Error("Expected GoogleGenAI config");
}
return config as GoogleGenAIConfig;
}
function firstGenerateContentRequest(): GenerateContentRequest {
const calls = generateContentMock.mock.calls as unknown[][];
const request = calls[0]?.[0];
if (!request) {
throw new Error("Expected generateContent request");
}
return request as GenerateContentRequest;
}
describe("google music generation provider", () => {
afterEach(() => {
vi.restoreAllMocks();
generateContentMock.mockReset();
createGoogleGenAIMock.mockClear();
});
afterAll(() => {
vi.doUnmock("./google-genai-runtime.js");
vi.resetModules();
});
it("declares explicit mode capabilities", () => {
expectExplicitMusicGenerationCapabilities(buildGoogleMusicGenerationProvider());
});
it("submits generation and returns inline audio bytes plus lyrics", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateContentMock.mockResolvedValue({
candidates: [
{
content: {
parts: [
{ text: "wake the city up" },
{
inlineData: {
data: Buffer.from("mp3-bytes").toString("base64"),
mimeType: "audio/mpeg",
},
},
],
},
},
],
});
const provider = buildGoogleMusicGenerationProvider();
const result = await provider.generateMusic({
provider: "google",
model: "lyria-3-clip-preview",
prompt: "upbeat synthpop anthem",
cfg: {},
instrumental: true,
});
const generateRequest = firstGenerateContentRequest();
expect(generateRequest.model).toBe("lyria-3-clip-preview");
expect(generateRequest.config).toEqual({
responseModalities: ["AUDIO", "TEXT"],
});
expect(result.tracks).toHaveLength(1);
expect(result.tracks[0]?.mimeType).toBe("audio/mpeg");
expect(result.lyrics).toEqual(["wake the city up"]);
expect(lastGoogleGenAIConfig().apiKey).toBe("google-key");
});
it("strips /v1beta suffix from configured baseUrl before passing to GoogleGenAI SDK", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateContentMock.mockResolvedValue({
candidates: [
{
content: {
parts: [
{
inlineData: {
data: Buffer.from("mp3-bytes").toString("base64"),
mimeType: "audio/mpeg",
},
},
],
},
},
],
});
const provider = buildGoogleMusicGenerationProvider();
await provider.generateMusic({
provider: "google",
model: "lyria-3-clip-preview",
prompt: "ambient ocean",
cfg: {
models: {
providers: {
google: { baseUrl: "https://generativelanguage.googleapis.com/v1beta", models: [] },
},
},
},
instrumental: true,
});
expect(lastGoogleGenAIConfig().httpOptions?.baseUrl).toBe(
"https://generativelanguage.googleapis.com",
);
});
it("does NOT strip /v1beta when it appears mid-path (end-anchor proof)", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateContentMock.mockResolvedValue({
candidates: [
{
content: {
parts: [
{ inlineData: { data: Buffer.from("x").toString("base64"), mimeType: "audio/mpeg" } },
],
},
},
],
});
const provider = buildGoogleMusicGenerationProvider();
await provider.generateMusic({
provider: "google",
model: "lyria-3-clip-preview",
prompt: "test",
cfg: {
models: {
providers: { google: { baseUrl: "https://proxy.example.com/v1beta/route", models: [] } },
},
},
instrumental: true,
});
expect(lastGoogleGenAIConfig().httpOptions?.baseUrl).toBe(
"https://proxy.example.com/v1beta/route",
);
});
it("passes baseUrl unchanged when no /v1beta suffix is present", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateContentMock.mockResolvedValue({
candidates: [
{
content: {
parts: [
{ inlineData: { data: Buffer.from("x").toString("base64"), mimeType: "audio/mpeg" } },
],
},
},
],
});
const provider = buildGoogleMusicGenerationProvider();
await provider.generateMusic({
provider: "google",
model: "lyria-3-clip-preview",
prompt: "test",
cfg: {
models: {
providers: {
google: { baseUrl: "https://generativelanguage.googleapis.com", models: [] },
},
},
},
instrumental: true,
});
expect(lastGoogleGenAIConfig().httpOptions?.baseUrl).toBe(
"https://generativelanguage.googleapis.com",
);
});
it("does not set baseUrl when none is configured", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateContentMock.mockResolvedValue({
candidates: [
{
content: {
parts: [
{ inlineData: { data: Buffer.from("x").toString("base64"), mimeType: "audio/mpeg" } },
],
},
},
],
});
const provider = buildGoogleMusicGenerationProvider();
await provider.generateMusic({
provider: "google",
model: "lyria-3-clip-preview",
prompt: "test",
cfg: {},
instrumental: true,
});
expect(lastGoogleGenAIConfig().httpOptions?.baseUrl).toBeUndefined();
});
it("rejects unsupported wav output on clip model", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
const provider = buildGoogleMusicGenerationProvider();
await expect(
provider.generateMusic({
provider: "google",
model: "lyria-3-clip-preview",
prompt: "ambient ocean",
cfg: {},
format: "wav",
}),
).rejects.toThrow("supports mp3 output");
});
});

View File

@@ -0,0 +1,177 @@
// Google provider module implements model/runtime integration.
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
import type {
GeneratedMusicAsset,
MusicGenerationProvider,
MusicGenerationRequest,
} from "openclaw/plugin-sdk/music-generation";
import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runtime";
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolveGoogleGenerativeAiApiOrigin } from "./api.js";
import {
createGoogleMusicGenerationProviderMetadata,
DEFAULT_GOOGLE_MUSIC_MODEL,
GOOGLE_MAX_INPUT_IMAGES,
GOOGLE_PRO_MUSIC_MODEL,
} from "./generation-provider-metadata.js";
import { createGoogleGenAI } from "./google-genai-runtime.js";
const DEFAULT_TIMEOUT_MS = 180_000;
type GoogleInlineDataPart = {
mimeType?: string;
mime_type?: string;
data?: string;
};
type GoogleGenerateMusicResponse = {
candidates?: Array<{
content?: {
parts?: Array<{
text?: string;
inlineData?: GoogleInlineDataPart;
inline_data?: GoogleInlineDataPart;
}>;
};
}>;
};
function resolveConfiguredGoogleMusicBaseUrl(req: MusicGenerationRequest): string | undefined {
const configured = normalizeOptionalString(req.cfg?.models?.providers?.google?.baseUrl);
return configured ? resolveGoogleGenerativeAiApiOrigin(configured) : undefined;
}
function buildMusicPrompt(req: MusicGenerationRequest): string {
const parts = [req.prompt.trim()];
const lyrics = normalizeOptionalString(req.lyrics);
if (req.instrumental === true) {
parts.push("Instrumental only. No vocals, no sung lyrics, no spoken word.");
}
if (lyrics) {
parts.push(`Lyrics:\n${lyrics}`);
}
return parts.join("\n\n");
}
function resolveSupportedFormats(model: string): readonly string[] {
return model === GOOGLE_PRO_MUSIC_MODEL ? ["mp3", "wav"] : ["mp3"];
}
function resolveTrackFileName(params: { index: number; mimeType: string; model: string }): string {
const ext =
extensionForMime(params.mimeType)?.replace(/^\./u, "") ||
(params.model === GOOGLE_PRO_MUSIC_MODEL ? "wav" : "mp3");
return `track-${params.index + 1}.${ext}`;
}
function extractTracks(params: { payload: GoogleGenerateMusicResponse; model: string }): {
tracks: GeneratedMusicAsset[];
lyrics: string[];
} {
const lyrics: string[] = [];
const tracks: GeneratedMusicAsset[] = [];
for (const candidate of params.payload.candidates ?? []) {
for (const part of candidate.content?.parts ?? []) {
const text = normalizeOptionalString(part.text);
if (text) {
lyrics.push(text);
continue;
}
const inline = part.inlineData ?? part.inline_data;
const data = normalizeOptionalString(inline?.data);
if (!data) {
continue;
}
const mimeType =
normalizeOptionalString(inline?.mimeType) ||
normalizeOptionalString(inline?.mime_type) ||
"audio/mpeg";
tracks.push({
buffer: Buffer.from(data, "base64"),
mimeType,
fileName: resolveTrackFileName({
index: tracks.length,
mimeType,
model: params.model,
}),
});
}
}
return { tracks, lyrics };
}
export function buildGoogleMusicGenerationProvider(): MusicGenerationProvider {
return {
...createGoogleMusicGenerationProviderMetadata(),
async generateMusic(req) {
if ((req.inputImages?.length ?? 0) > GOOGLE_MAX_INPUT_IMAGES) {
throw new Error(
`Google music generation supports at most ${GOOGLE_MAX_INPUT_IMAGES} reference images.`,
);
}
const auth = await resolveApiKeyForProvider({
provider: "google",
cfg: req.cfg,
agentDir: req.agentDir,
store: req.authStore,
});
if (!auth.apiKey) {
throw new Error("Google API key missing");
}
const model = normalizeOptionalString(req.model) || DEFAULT_GOOGLE_MUSIC_MODEL;
if (req.format) {
const supportedFormats = resolveSupportedFormats(model);
if (!supportedFormats.includes(req.format)) {
throw new Error(
`Google music generation model ${model} supports ${supportedFormats.join(", ")} output.`,
);
}
}
const client = createGoogleGenAI({
apiKey: auth.apiKey,
httpOptions: {
...(resolveConfiguredGoogleMusicBaseUrl(req)
? { baseUrl: resolveConfiguredGoogleMusicBaseUrl(req) }
: {}),
timeout: req.timeoutMs ?? DEFAULT_TIMEOUT_MS,
},
});
const response = (await client.models.generateContent({
model,
contents: [
{ text: buildMusicPrompt(req) },
...(req.inputImages ?? []).map((image) => ({
inlineData: {
mimeType: normalizeOptionalString(image.mimeType) || "image/png",
data: image.buffer?.toString("base64") ?? "",
},
})),
],
config: {
responseModalities: ["AUDIO", "TEXT"],
},
})) as GoogleGenerateMusicResponse;
const { tracks, lyrics } = extractTracks({
payload: response,
model,
});
if (tracks.length === 0) {
throw new Error("Google music generation response missing audio data");
}
return {
tracks,
...(lyrics.length > 0 ? { lyrics } : {}),
model,
metadata: {
inputImageCount: req.inputImages?.length ?? 0,
instrumental: req.instrumental === true,
...(normalizeOptionalString(req.lyrics) ? { requestedLyrics: true } : {}),
...(req.format ? { requestedFormat: req.format } : {}),
},
};
},
};
}

View File

@@ -0,0 +1,40 @@
// Google tests cover oauth token shared plugin behavior.
import { describe, expect, it } from "vitest";
import {
formatGoogleOauthApiKey,
parseGoogleOauthApiKey,
parseGoogleUsageToken,
} from "./oauth-token-shared.js";
describe("google oauth token helpers", () => {
it("formats oauth credentials with project-aware payloads", () => {
expect(
formatGoogleOauthApiKey({
type: "oauth",
access: "token-123",
projectId: "project-abc",
}),
).toBe(JSON.stringify({ token: "token-123", projectId: "project-abc" }));
});
it("returns an empty string for non-oauth credentials", () => {
expect(formatGoogleOauthApiKey({ type: "token", access: "token-123" })).toBe("");
});
it("parses project-aware oauth payloads for usage auth", () => {
expect(parseGoogleUsageToken(JSON.stringify({ token: "usage-token" }))).toBe("usage-token");
});
it("parses structured oauth payload fields", () => {
expect(
parseGoogleOauthApiKey(JSON.stringify({ token: "usage-token", projectId: "proj-1" })),
).toEqual({
token: "usage-token",
projectId: "proj-1",
});
});
it("falls back to the raw token when the payload is not JSON", () => {
expect(parseGoogleUsageToken("raw-token")).toBe("raw-token");
});
});

View File

@@ -0,0 +1,43 @@
// Google plugin module implements oauth token shared behavior.
import { readStringValue } from "openclaw/plugin-sdk/string-coerce-runtime";
type GoogleOauthApiKeyCredential = {
type?: string;
access?: string;
projectId?: string;
};
export function parseGoogleOauthApiKey(apiKey: string): {
token?: string;
projectId?: string;
} | null {
try {
const parsed = JSON.parse(apiKey) as { token?: unknown; projectId?: unknown };
return {
token: readStringValue(parsed.token),
projectId: readStringValue(parsed.projectId),
};
} catch {
return null;
}
}
export function formatGoogleOauthApiKey(cred: GoogleOauthApiKeyCredential): string {
if (cred.type !== "oauth" || typeof cred.access !== "string" || !cred.access.trim()) {
return "";
}
return JSON.stringify({
token: cred.access,
projectId: cred.projectId,
});
}
export function parseGoogleUsageToken(apiKey: string): string {
const parsed = parseGoogleOauthApiKey(apiKey);
if (parsed?.token) {
return parsed.token;
}
// Keep the raw token when the stored credential is not a project-aware JSON payload.
return apiKey;
}

View File

@@ -0,0 +1,364 @@
// Google plugin module implements oauth.credentials behavior.
import { existsSync, readFileSync, readdirSync, realpathSync } from "node:fs";
import type { Dirent } from "node:fs";
import { delimiter, dirname, join } from "node:path";
import { lowercasePreservingWhitespace } from "openclaw/plugin-sdk/string-coerce-runtime";
import { CLIENT_ID_KEYS, CLIENT_SECRET_KEYS } from "./oauth.shared.js";
type CredentialFs = {
existsSync: (path: Parameters<typeof existsSync>[0]) => ReturnType<typeof existsSync>;
readFileSync: (path: Parameters<typeof readFileSync>[0], encoding: "utf8") => string;
realpathSync: (path: Parameters<typeof realpathSync>[0]) => string;
readdirSync: (
path: Parameters<typeof readdirSync>[0],
options: { withFileTypes: true },
) => Dirent[];
};
const defaultFs: CredentialFs = {
existsSync,
readFileSync,
realpathSync,
readdirSync,
};
let credentialFs: CredentialFs = defaultFs;
const GEMINI_CLI_TREE_SEARCH_DEPTH = 10;
type GeminiCliCredentialExtractDiagnostics = {
searchedPaths: string[];
recursiveSearchRoots: string[];
parseFailures: string[];
readErrors: string[];
};
function resolveEnv(keys: string[]): string | undefined {
for (const key of keys) {
const value = process.env[key]?.trim();
if (value) {
return value;
}
}
return undefined;
}
let cachedGeminiCliCredentials: { clientId: string; clientSecret: string } | null = null;
let geminiCliCredentialExtractError: string | null = null;
export function clearCredentialsCache(): void {
cachedGeminiCliCredentials = null;
geminiCliCredentialExtractError = null;
}
export function setOAuthCredentialsFsForTest(overrides?: Partial<CredentialFs>): void {
credentialFs = overrides ? { ...defaultFs, ...overrides } : defaultFs;
}
export function extractGeminiCliCredentials(): { clientId: string; clientSecret: string } | null {
if (cachedGeminiCliCredentials) {
return cachedGeminiCliCredentials;
}
geminiCliCredentialExtractError = null;
const diagnostics: GeminiCliCredentialExtractDiagnostics = {
searchedPaths: [],
recursiveSearchRoots: [],
parseFailures: [],
readErrors: [],
};
try {
const geminiPath = findInPath("gemini");
if (!geminiPath) {
geminiCliCredentialExtractError =
"Gemini CLI binary was not found in PATH during OAuth credential extraction.";
return null;
}
const resolvedPath = credentialFs.realpathSync(geminiPath);
const geminiCliDirs = resolveGeminiCliDirs(geminiPath, resolvedPath);
for (const geminiCliDir of geminiCliDirs) {
const directCredentials = readGeminiCliCredentialsFromKnownPaths(geminiCliDir, diagnostics);
if (directCredentials) {
cachedGeminiCliCredentials = directCredentials;
return directCredentials;
}
const bundledCredentials = readGeminiCliCredentialsFromBundle(geminiCliDir, diagnostics);
if (bundledCredentials) {
cachedGeminiCliCredentials = bundledCredentials;
return bundledCredentials;
}
diagnostics.recursiveSearchRoots.push(geminiCliDir);
const discoveredCredentials = findGeminiCliCredentialsInTree(
geminiCliDir,
GEMINI_CLI_TREE_SEARCH_DEPTH,
diagnostics,
);
if (discoveredCredentials) {
cachedGeminiCliCredentials = discoveredCredentials;
return discoveredCredentials;
}
}
geminiCliCredentialExtractError = formatGeminiCliCredentialExtractError({
geminiPath,
resolvedPath,
diagnostics,
});
} catch (error) {
geminiCliCredentialExtractError = `Unexpected error while extracting Gemini CLI OAuth credentials: ${formatError(error)}`;
}
return null;
}
function formatGeminiCliCredentialExtractError({
geminiPath,
resolvedPath,
diagnostics,
}: {
geminiPath: string;
resolvedPath: string;
diagnostics: GeminiCliCredentialExtractDiagnostics;
}): string {
const prefix = [
"Found Gemini CLI in PATH, but could not extract OAuth credentials.",
`geminiPath=${geminiPath}`,
`resolvedPath=${resolvedPath}`,
];
if (diagnostics.parseFailures.length > 0) {
return [
...prefix,
"Candidate credential files did not contain a parseable OAuth client id/secret.",
`candidates=${diagnostics.parseFailures.join(", ")}`,
].join(" ");
}
if (diagnostics.readErrors.length > 0) {
return [
...prefix,
"Unexpected errors occurred while reading candidate credential files/directories.",
`errors=${diagnostics.readErrors.join(", ")}`,
].join(" ");
}
return [
...prefix,
"Could not locate oauth2.js or bundled credential source.",
`searched=${diagnostics.searchedPaths.join(", ") || "(none)"}`,
`recursiveSearchRoots=${diagnostics.recursiveSearchRoots.join(", ") || "(none)"}`,
`recursiveSearchDepth=${GEMINI_CLI_TREE_SEARCH_DEPTH}`,
].join(" ");
}
function formatError(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function resolveGeminiCliDirs(geminiPath: string, resolvedPath: string): string[] {
const binDir = dirname(geminiPath);
const candidates = [
dirname(dirname(resolvedPath)),
join(dirname(resolvedPath), "node_modules", "@google", "gemini-cli"),
join(binDir, "node_modules", "@google", "gemini-cli"),
join(dirname(binDir), "node_modules", "@google", "gemini-cli"),
join(dirname(binDir), "lib", "node_modules", "@google", "gemini-cli"),
];
const deduped: string[] = [];
const seen = new Set<string>();
for (const candidate of candidates) {
for (const searchDir of resolveGeminiCliSearchDirs(candidate)) {
const key =
process.platform === "win32"
? lowercasePreservingWhitespace(searchDir.replace(/\\/g, "/"))
: searchDir;
if (seen.has(key)) {
continue;
}
seen.add(key);
deduped.push(searchDir);
}
}
return deduped;
}
function resolveGeminiCliSearchDirs(candidate: string): string[] {
const searchDirs = [
candidate,
join(candidate, "node_modules", "@google", "gemini-cli"),
join(candidate, "lib", "node_modules", "@google", "gemini-cli"),
];
return searchDirs.filter(looksLikeGeminiCliDir);
}
function looksLikeGeminiCliDir(candidate: string): boolean {
return (
credentialFs.existsSync(join(candidate, "package.json")) ||
credentialFs.existsSync(join(candidate, "node_modules", "@google", "gemini-cli-core"))
);
}
function findInPath(name: string): string | null {
const exts = process.platform === "win32" ? [".cmd", ".bat", ".exe", ""] : [""];
for (const dir of (process.env.PATH ?? "").split(delimiter)) {
for (const ext of exts) {
const path = join(dir, name + ext);
if (credentialFs.existsSync(path)) {
return path;
}
}
}
return null;
}
function readGeminiCliCredentialsFile(
path: string,
diagnostics: GeminiCliCredentialExtractDiagnostics,
): { clientId: string; clientSecret: string } | null {
try {
const credentials = parseGeminiCliCredentials(credentialFs.readFileSync(path, "utf8"));
if (!credentials) {
diagnostics.parseFailures.push(path);
}
return credentials;
} catch (error) {
diagnostics.readErrors.push(`${path}: ${formatError(error)}`);
return null;
}
}
function parseGeminiCliCredentials(
content: string,
): { clientId: string; clientSecret: string } | null {
const clientId =
content.match(/OAUTH_CLIENT_ID\s*=\s*["']([^"']+)["']/)?.[1] ??
content.match(/(\d+-[a-z0-9]+\.apps\.googleusercontent\.com)/)?.[1];
const clientSecret =
content.match(/OAUTH_CLIENT_SECRET\s*=\s*["']([^"']+)["']/)?.[1] ??
content.match(/(GOCSPX-[A-Za-z0-9_-]+)/)?.[1];
if (!clientId || !clientSecret) {
return null;
}
return { clientId, clientSecret };
}
function readGeminiCliCredentialsFromKnownPaths(
geminiCliDir: string,
diagnostics: GeminiCliCredentialExtractDiagnostics,
): { clientId: string; clientSecret: string } | null {
const searchPaths = [
join(
geminiCliDir,
"node_modules",
"@google",
"gemini-cli-core",
"dist",
"src",
"code_assist",
"oauth2.js",
),
join(
geminiCliDir,
"node_modules",
"@google",
"gemini-cli-core",
"dist",
"code_assist",
"oauth2.js",
),
];
diagnostics.searchedPaths.push(...searchPaths);
for (const path of searchPaths) {
if (!credentialFs.existsSync(path)) {
continue;
}
const credentials = readGeminiCliCredentialsFile(path, diagnostics);
if (credentials) {
return credentials;
}
}
return null;
}
function readGeminiCliCredentialsFromBundle(
geminiCliDir: string,
diagnostics: GeminiCliCredentialExtractDiagnostics,
): { clientId: string; clientSecret: string } | null {
const bundleDir = join(geminiCliDir, "bundle");
if (!credentialFs.existsSync(bundleDir)) {
return null;
}
try {
for (const entry of credentialFs.readdirSync(bundleDir, { withFileTypes: true })) {
if (!entry.isFile() || !entry.name.endsWith(".js")) {
continue;
}
const credentials = readGeminiCliCredentialsFile(join(bundleDir, entry.name), diagnostics);
if (credentials) {
return credentials;
}
}
} catch (error) {
diagnostics.readErrors.push(`${bundleDir}: ${formatError(error)}`);
// Preserve the read error for diagnostics and fall back to the recursive search.
}
return null;
}
function findGeminiCliCredentialsInTree(
dir: string,
depth: number,
diagnostics: GeminiCliCredentialExtractDiagnostics,
): { clientId: string; clientSecret: string } | null {
if (depth <= 0) {
return null;
}
try {
for (const entry of credentialFs.readdirSync(dir, { withFileTypes: true })) {
const path = join(dir, entry.name);
if (entry.isFile() && entry.name === "oauth2.js") {
const credentials = readGeminiCliCredentialsFile(path, diagnostics);
if (credentials) {
return credentials;
}
continue;
}
if (entry.isDirectory() && !entry.name.startsWith(".")) {
const found = findGeminiCliCredentialsInTree(path, depth - 1, diagnostics);
if (found) {
return found;
}
}
}
} catch (error) {
diagnostics.readErrors.push(`${dir}: ${formatError(error)}`);
}
return null;
}
export function resolveOAuthClientConfig(): { clientId: string; clientSecret?: string } {
const envClientId = resolveEnv(CLIENT_ID_KEYS);
const envClientSecret = resolveEnv(CLIENT_SECRET_KEYS);
if (envClientId) {
return { clientId: envClientId, clientSecret: envClientSecret };
}
const extracted = extractGeminiCliCredentials();
if (extracted) {
return extracted;
}
const detail = geminiCliCredentialExtractError
? ` Details: ${geminiCliCredentialExtractError}`
: "";
throw new Error(
`Gemini CLI not found. Install it first: brew install gemini-cli (or npm install -g @google/gemini-cli), or set GEMINI_CLI_OAUTH_CLIENT_ID.${detail}`,
);
}

View File

@@ -0,0 +1,62 @@
// Google plugin module implements oauth.flow behavior.
import { generateHexPkceVerifierChallenge } from "openclaw/plugin-sdk/provider-auth";
import {
generateOAuthState,
parseOAuthCallbackInput,
waitForLocalOAuthCallback,
} from "openclaw/plugin-sdk/provider-auth-runtime";
import { isWSL2Sync } from "openclaw/plugin-sdk/runtime-env";
import { resolveOAuthClientConfig } from "./oauth.credentials.js";
import { AUTH_URL, REDIRECT_URI, SCOPES } from "./oauth.shared.js";
export { generateOAuthState };
export function shouldUseManualOAuthFlow(isRemote: boolean): boolean {
return isRemote || isWSL2Sync();
}
export function generatePkce(): { verifier: string; challenge: string } {
return generateHexPkceVerifierChallenge();
}
export function buildAuthUrl(challenge: string, state: string): string {
const { clientId } = resolveOAuthClientConfig();
const params = new URLSearchParams({
client_id: clientId,
response_type: "code",
redirect_uri: REDIRECT_URI,
scope: SCOPES.join(" "),
code_challenge: challenge,
code_challenge_method: "S256",
state,
access_type: "offline",
prompt: "consent",
});
return `${AUTH_URL}?${params.toString()}`;
}
export function parseCallbackInput(
input: string,
): { code: string; state: string } | { error: string } {
return parseOAuthCallbackInput(input, {
missingState: "Missing 'state' parameter. Paste the full URL.",
invalidInput: "Paste the full redirect URL, not just the code.",
});
}
export async function waitForLocalCallback(params: {
expectedState: string;
timeoutMs: number;
onProgress?: (message: string) => void;
}): Promise<{ code: string; state: string }> {
return await waitForLocalOAuthCallback({
expectedState: params.expectedState,
timeoutMs: params.timeoutMs,
port: 8085,
callbackPath: "/oauth2callback",
redirectUri: REDIRECT_URI,
successTitle: "Gemini CLI OAuth complete",
progressMessage: `Waiting for OAuth callback on ${REDIRECT_URI}`,
onProgress: params.onProgress,
});
}

View File

@@ -0,0 +1,121 @@
// Google tests cover oauth.http proxy-mode selection for the Gemini CLI OAuth
// token-exchange/identity calls (issue openclaw#46184).
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { TOKEN_URL } from "./oauth.shared.js";
const fetchWithSsrFGuardMock = vi.fn();
vi.mock("openclaw/plugin-sdk/ssrf-runtime", async () => {
const actual = await vi.importActual<typeof import("openclaw/plugin-sdk/ssrf-runtime")>(
"openclaw/plugin-sdk/ssrf-runtime",
);
return {
...actual,
fetchWithSsrFGuard: (params: unknown) => fetchWithSsrFGuardMock(params),
};
});
const { fetchWithTimeout } = await import("./oauth.http.js");
const PROXY_ENV_KEYS = [
"HTTP_PROXY",
"HTTPS_PROXY",
"ALL_PROXY",
"NO_PROXY",
"http_proxy",
"https_proxy",
"all_proxy",
"no_proxy",
] as const;
const savedEnv = new Map<string, string | undefined>();
type ProxyEnvOverrides = {
HTTP_PROXY?: string;
HTTPS_PROXY?: string;
ALL_PROXY?: string;
NO_PROXY?: string;
};
function setProxyEnv(values: ProxyEnvOverrides): void {
for (const key of PROXY_ENV_KEYS) {
delete process.env[key];
}
if (values.HTTP_PROXY !== undefined) {
process.env.HTTP_PROXY = values.HTTP_PROXY;
}
if (values.HTTPS_PROXY !== undefined) {
process.env.HTTPS_PROXY = values.HTTPS_PROXY;
}
if (values.ALL_PROXY !== undefined) {
process.env.ALL_PROXY = values.ALL_PROXY;
}
if (values.NO_PROXY !== undefined) {
process.env.NO_PROXY = values.NO_PROXY;
}
}
function lastGuardedOptions(): Record<string, unknown> {
const call = fetchWithSsrFGuardMock.mock.calls.at(-1)?.[0];
if (!call || typeof call !== "object") {
throw new Error("Expected fetchWithSsrFGuard to be called");
}
return call as Record<string, unknown>;
}
describe("oauth.http fetchWithTimeout proxy selection", () => {
beforeEach(() => {
for (const key of PROXY_ENV_KEYS) {
savedEnv.set(key, process.env[key]);
}
fetchWithSsrFGuardMock.mockReset();
fetchWithSsrFGuardMock.mockResolvedValue({
response: new Response("{}", { status: 200 }),
finalUrl: TOKEN_URL,
release: async () => {},
});
});
afterEach(() => {
for (const [key, value] of savedEnv) {
if (value === undefined) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
savedEnv.clear();
});
it("routes the Google token exchange through the env proxy when configured", async () => {
setProxyEnv({ HTTPS_PROXY: "http://127.0.0.1:7897", HTTP_PROXY: "http://127.0.0.1:7897" });
await fetchWithTimeout(TOKEN_URL, { method: "POST", body: "grant_type=refresh_token" });
expect(lastGuardedOptions().mode).toBe("trusted_env_proxy");
});
it("keeps the strict default when no proxy is configured", async () => {
setProxyEnv({});
await fetchWithTimeout(TOKEN_URL, { method: "POST" });
expect(lastGuardedOptions().mode).toBeUndefined();
});
it("keeps the strict default when NO_PROXY bypasses the target host", async () => {
setProxyEnv({ HTTPS_PROXY: "http://127.0.0.1:7897", NO_PROXY: "googleapis.com" });
await fetchWithTimeout(TOKEN_URL, { method: "POST" });
expect(lastGuardedOptions().mode).toBeUndefined();
});
it("keeps the strict default for ALL_PROXY-only environments", async () => {
setProxyEnv({ ALL_PROXY: "http://127.0.0.1:7897" });
await fetchWithTimeout(TOKEN_URL, { method: "POST" });
expect(lastGuardedOptions().mode).toBeUndefined();
});
});

View File

@@ -0,0 +1,172 @@
// Google tests cover oauth.http body-byte-cap for the Gemini CLI OAuth
// token-exchange/identity calls.
import http from "node:http";
import type { AddressInfo } from "node:net";
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { TOKEN_URL } from "./oauth.shared.js";
const fetchWithSsrFGuardMock = vi.fn();
const releaseMock = vi.fn(async () => undefined);
vi.mock("openclaw/plugin-sdk/ssrf-runtime", async () => {
const actual = await vi.importActual<typeof import("openclaw/plugin-sdk/ssrf-runtime")>(
"openclaw/plugin-sdk/ssrf-runtime",
);
return {
...actual,
fetchWithSsrFGuard: (params: unknown) => fetchWithSsrFGuardMock(params),
};
});
const { fetchWithTimeout } = await import("./oauth.http.js");
describe("oauth.http fetchWithTimeout body byte cap", () => {
beforeEach(() => {
fetchWithSsrFGuardMock.mockReset();
releaseMock.mockClear();
});
afterEach(() => {
vi.restoreAllMocks();
});
it("caps oversized response body at 16 MiB with labeled overflow error", async () => {
// Build a Response with a body that exceeds the 16 MiB cap.
// 1 MiB chunks × 18 chunks = 18 MiB queued; the bounded reader reads
// up to the 16 MiB cap (16 chunks = 16777216 bytes) and one extra
// chunk before throwing on overflow, so the labeled `size` is the
// cap plus the trailing chunk: 16777216 + 1048576 = 17825792 bytes.
const CHUNK = 1024 * 1024;
let sent = 0;
const body = new ReadableStream({
pull(controller) {
if (sent < 18) {
controller.enqueue(new Uint8Array(CHUNK));
sent++;
} else {
controller.close();
}
},
});
fetchWithSsrFGuardMock.mockResolvedValue({
response: new Response(body, {
status: 200,
headers: { "content-type": "application/json" },
}),
finalUrl: TOKEN_URL,
release: releaseMock,
});
await expect(fetchWithTimeout(TOKEN_URL, { method: "POST" })).rejects.toThrow(
/google HTTP fetch: body exceeds 16777216 bytes \(got 17825792\)/,
);
expect(releaseMock).toHaveBeenCalledOnce();
});
it("returns a Response for normal-size bodies", async () => {
fetchWithSsrFGuardMock.mockResolvedValue({
response: new Response('{"access_token":"abc","expires_in":3600}', {
status: 200,
headers: { "content-type": "application/json" },
}),
finalUrl: TOKEN_URL,
release: releaseMock,
});
const res = await fetchWithTimeout(TOKEN_URL, { method: "POST" });
expect(res.status).toBe(200);
expect(await res.json()).toEqual({ access_token: "abc", expires_in: 3600 });
expect(releaseMock).toHaveBeenCalledOnce();
});
});
// Real-wire loopback proof. These tests bypass `fetchWithSsrFGuard` (which
// blocks 127.0.0.1 by design) and exercise `readResponseWithLimit` directly
// against a real `http.createServer` listener — the same helper that
// `fetchWithTimeout` calls inside its try/finally block. Captured vitest
// output for these two tests is the ClawSweeper "real behavior proof" required
// before merge.
describe("oauth.http bounded-read real wire proof (loopback http.createServer)", () => {
it("caps an oversized body streamed chunked over real wire", async () => {
const CHUNK = 1024 * 1024;
const MAX = 16 * 1024 * 1024;
const TOTAL = 18 * 1024 * 1024;
const server = http.createServer((req, res) => {
res.writeHead(200, { "content-type": "application/octet-stream" });
let sent = 0;
const tick = setInterval(() => {
if (sent < 18) {
res.write(Buffer.alloc(CHUNK));
sent++;
} else {
clearInterval(tick);
res.end();
}
}, 1);
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => resolve());
});
const port = (server.address() as AddressInfo).port;
let captured: Error | undefined;
try {
const response = await fetch(`http://127.0.0.1:${port}/`);
// Wire framing merges TCP packets, so the exact reported size varies by
// runtime. The stable invariant is that the cap fires after MAX.
try {
await readResponseWithLimit(response, MAX, {
onOverflow: ({ size, maxBytes }) =>
new Error(`real wire: body exceeds ${maxBytes} bytes (got ${size})`),
});
} catch (err) {
captured = err as Error;
}
expect(captured).toBeInstanceOf(Error);
const match = captured!.message.match(/real wire: body exceeds \d+ bytes \(got (\d+)\)/);
expect(match).not.toBeNull();
const got = Number(match![1]);
expect(got).toBeGreaterThan(MAX);
// Print to vitest stdout for PR-body real behavior proof capture.
console.log(
`[oauth.http loopback proof] oversized path: cap=${MAX} reported=${got} server_total=${TOTAL}`,
);
} finally {
await new Promise<void>((resolve) => {
server.close(() => resolve());
});
}
});
it("returns a Buffer for normal-size responses on real wire", async () => {
const bodyText = '{"access_token":"loopback","expires_in":3600}';
const server = http.createServer((req, res) => {
res.writeHead(200, { "content-type": "application/json" });
res.end(bodyText);
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => resolve());
});
const port = (server.address() as AddressInfo).port;
try {
const response = await fetch(`http://127.0.0.1:${port}/`);
const body = await readResponseWithLimit(response, 16 * 1024 * 1024, {
onOverflow: ({ size, maxBytes }) =>
new Error(`real wire: body exceeds ${maxBytes} bytes (got ${size})`),
});
expect(body.byteLength).toBe(Buffer.byteLength(bodyText, "utf8"));
expect(new TextDecoder("utf-8").decode(body)).toBe(bodyText);
console.log(
`[oauth.http loopback proof] normal path: cap=16777216 returned=${body.byteLength} body=${JSON.stringify(new TextDecoder("utf-8").decode(body))}`,
);
} finally {
await new Promise<void>((resolve) => {
server.close(() => resolve());
});
}
});
});

View File

@@ -0,0 +1,50 @@
// Google plugin module implements oauth.http behavior.
import {
shouldUseEnvHttpProxyForUrl,
withTrustedEnvProxyGuardedFetchMode,
} from "openclaw/plugin-sdk/fetch-runtime";
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import { DEFAULT_FETCH_TIMEOUT_MS } from "./oauth.shared.js";
const GOOGLE_OAUTH_BODY_MAX_BYTES = 16 * 1024 * 1024;
export async function fetchWithTimeout(
url: string,
init: RequestInit,
timeoutMs = DEFAULT_FETCH_TIMEOUT_MS,
): Promise<Response> {
const guardedOptions = { url, init, timeoutMs };
const { response, release } = await fetchWithSsrFGuard(
shouldUseEnvHttpProxyForUrl(url)
? withTrustedEnvProxyGuardedFetchMode(guardedOptions)
: guardedOptions,
);
try {
// 16 MiB cap. A hostile or broken Google OAuth endpoint (or any
// accounts.google.com mirror / enterprise proxy) cannot force the
// runtime to buffer an unbounded body before the caller sees it.
// Complements #97587, which caps at the call site — this is the
// shared entry-point cap.
const body = await readResponseWithLimit(response, GOOGLE_OAUTH_BODY_MAX_BYTES, {
onOverflow: ({ size, maxBytes }) =>
new Error(`google HTTP fetch: body exceeds ${maxBytes} bytes (got ${size})`),
});
// `readResponseWithLimit` returns a `Buffer` (Node Uint8Array view). The
// global `Response` constructor accepts `BufferSource` (Uint8Array /
// ArrayBuffer) as a body; cast through `BodyInit` because `Buffer.buffer`
// is typed as `ArrayBufferLike` (could be `ArrayBuffer` or
// `SharedArrayBuffer`), but the helper always returns a regular `Buffer`
// backed by an `ArrayBuffer` with no shared-memory paths. The same
// wrap-shape is used by the googlechat google-auth helper at
// extensions/googlechat/src/google-auth.runtime.ts:454.
const bodyBytes = new Uint8Array(body.buffer, body.byteOffset, body.byteLength);
return new Response(bodyBytes as unknown as BodyInit, {
status: response.status,
statusText: response.statusText,
headers: response.headers,
});
} finally {
await release();
}
}

View File

@@ -0,0 +1,70 @@
// Google tests cover oauth.local login plugin behavior.
import { beforeEach, describe, expect, it, vi } from "vitest";
const AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth?state=state-123";
const exchangeCodeForTokensMock = vi.hoisted(() =>
vi.fn(async () => ({
access: "access-token",
refresh: "refresh-token",
expires: 123,
})),
);
const waitForLocalCallbackMock = vi.hoisted(() =>
vi.fn(async () => ({ code: "oauth-code", state: "state-123" })),
);
vi.mock("./oauth.flow.js", () => ({
buildAuthUrl: () => AUTH_URL,
generateOAuthState: () => "state-123",
generatePkce: () => ({ challenge: "pkce-challenge", verifier: "pkce-verifier" }),
parseCallbackInput: vi.fn(),
shouldUseManualOAuthFlow: (isRemote: boolean) => isRemote,
waitForLocalCallback: waitForLocalCallbackMock,
}));
vi.mock("./oauth.token.js", () => ({
exchangeCodeForTokens: exchangeCodeForTokensMock,
}));
describe("loginGeminiCliOAuth local browser flow", () => {
beforeEach(() => {
exchangeCodeForTokensMock.mockClear();
waitForLocalCallbackMock.mockClear();
});
it("prints the auth URL before attempting best-effort browser launch", async () => {
const events: string[] = [];
const { loginGeminiCliOAuth } = await import("./oauth.js");
const openUrl = vi.fn(async () => {
events.push("open");
});
const log = vi.fn((message: string) => {
events.push(`log:${message}`);
});
const result = await loginGeminiCliOAuth({
isRemote: false,
openUrl,
log,
note: async () => {},
prompt: async () => "",
progress: { update: () => {}, stop: () => {} },
});
expect(result).toEqual({
access: "access-token",
refresh: "refresh-token",
expires: 123,
});
expect(log).toHaveBeenCalledWith(expect.stringContaining(AUTH_URL));
expect(openUrl).toHaveBeenCalledWith(AUTH_URL);
expect(events.findIndex((event) => event.startsWith("log:"))).toBeLessThan(
events.indexOf("open"),
);
expect(waitForLocalCallbackMock).toHaveBeenCalledWith(
expect.objectContaining({ expectedState: "state-123" }),
);
expect(exchangeCodeForTokensMock).toHaveBeenCalledWith("oauth-code", "pkce-verifier");
});
});

View File

@@ -0,0 +1,235 @@
// Google plugin module implements oauth.project behavior.
import { fetchWithTimeout } from "./oauth.http.js";
import {
CODE_ASSIST_ENDPOINT_PROD,
LOAD_CODE_ASSIST_ENDPOINTS,
TIER_FREE,
TIER_LEGACY,
TIER_STANDARD,
USERINFO_URL,
} from "./oauth.shared.js";
const LOAD_CODE_ASSIST_METADATA = {
ideType: "IDE_UNSPECIFIED",
platform: "PLATFORM_UNSPECIFIED",
pluginType: "GEMINI",
} as const;
async function getUserEmail(accessToken: string): Promise<string | undefined> {
try {
const response = await fetchWithTimeout(USERINFO_URL, {
headers: { Authorization: `Bearer ${accessToken}` },
});
if (response.ok) {
const data = (await response.json()) as { email?: string };
return data.email;
}
} catch {
// ignore
}
return undefined;
}
function isVpcScAffected(payload: unknown): boolean {
if (!payload || typeof payload !== "object") {
return false;
}
const error = (payload as { error?: unknown }).error;
if (!error || typeof error !== "object") {
return false;
}
const details = (error as { details?: unknown[] }).details;
if (!Array.isArray(details)) {
return false;
}
return details.some(
(item) =>
typeof item === "object" &&
item &&
(item as { reason?: string }).reason === "SECURITY_POLICY_VIOLATED",
);
}
function getDefaultTier(
allowedTiers?: Array<{ id?: string; isDefault?: boolean }>,
): { id?: string } | undefined {
if (!allowedTiers?.length) {
return { id: TIER_LEGACY };
}
return allowedTiers.find((tier) => tier.isDefault) ?? { id: TIER_LEGACY };
}
async function pollOperation(
endpoint: string,
operationName: string,
headers: Record<string, string>,
): Promise<{ done?: boolean; response?: { cloudaicompanionProject?: { id?: string } } }> {
for (let attempt = 0; attempt < 24; attempt += 1) {
await new Promise((resolve) => {
setTimeout(resolve, 5000);
});
const response = await fetchWithTimeout(`${endpoint}/v1internal/${operationName}`, {
headers,
});
if (!response.ok) {
continue;
}
const data = (await response.json()) as {
done?: boolean;
response?: { cloudaicompanionProject?: { id?: string } };
};
if (data.done) {
return data;
}
}
throw new Error("Operation polling timeout");
}
export async function resolveGoogleOAuthIdentity(accessToken: string): Promise<{
email?: string;
projectId?: string;
}> {
const email = await getUserEmail(accessToken);
const projectId = await discoverProject(accessToken);
return { email, projectId };
}
export async function resolveGooglePersonalOAuthIdentity(accessToken: string): Promise<{
email?: string;
projectId?: string;
}> {
return { email: await getUserEmail(accessToken) };
}
async function discoverProject(accessToken: string): Promise<string> {
const envProject = process.env.GOOGLE_CLOUD_PROJECT || process.env.GOOGLE_CLOUD_PROJECT_ID;
const headers = {
Authorization: `Bearer ${accessToken}`,
"Content-Type": "application/json",
"User-Agent": "google-api-nodejs-client/9.15.1",
"X-Goog-Api-Client": `gl-node/${process.versions.node}`,
"Client-Metadata": JSON.stringify(LOAD_CODE_ASSIST_METADATA),
};
const loadBody = {
...(envProject ? { cloudaicompanionProject: envProject } : {}),
metadata: {
...LOAD_CODE_ASSIST_METADATA,
...(envProject ? { duetProject: envProject } : {}),
},
};
let data: {
currentTier?: { id?: string };
cloudaicompanionProject?: string | { id?: string };
allowedTiers?: Array<{ id?: string; isDefault?: boolean }>;
} = {};
let activeEndpoint = CODE_ASSIST_ENDPOINT_PROD;
let loadError: Error | undefined;
for (const endpoint of LOAD_CODE_ASSIST_ENDPOINTS) {
try {
const response = await fetchWithTimeout(`${endpoint}/v1internal:loadCodeAssist`, {
method: "POST",
headers,
body: JSON.stringify(loadBody),
});
if (!response.ok) {
const errorPayload = await response.json().catch(() => null);
if (isVpcScAffected(errorPayload)) {
data = { currentTier: { id: TIER_STANDARD } };
activeEndpoint = endpoint;
loadError = undefined;
break;
}
loadError = new Error(`loadCodeAssist failed: ${response.status} ${response.statusText}`);
continue;
}
data = (await response.json()) as typeof data;
activeEndpoint = endpoint;
loadError = undefined;
break;
} catch (err) {
loadError = err instanceof Error ? err : new Error("loadCodeAssist failed", { cause: err });
}
}
const hasLoadCodeAssistData =
Boolean(data.currentTier) ||
Boolean(data.cloudaicompanionProject) ||
Boolean(data.allowedTiers?.length);
if (!hasLoadCodeAssistData && loadError) {
if (envProject) {
return envProject;
}
throw loadError;
}
if (data.currentTier) {
const project = data.cloudaicompanionProject;
if (typeof project === "string" && project) {
return project;
}
if (typeof project === "object" && project?.id) {
return project.id;
}
if (envProject) {
return envProject;
}
throw new Error(
"This account requires GOOGLE_CLOUD_PROJECT or GOOGLE_CLOUD_PROJECT_ID to be set.",
);
}
const tier = getDefaultTier(data.allowedTiers);
const tierId = tier?.id || TIER_FREE;
if (tierId !== TIER_FREE && !envProject) {
throw new Error(
"This account requires GOOGLE_CLOUD_PROJECT or GOOGLE_CLOUD_PROJECT_ID to be set.",
);
}
const onboardBody: Record<string, unknown> = {
tierId,
metadata: {
...LOAD_CODE_ASSIST_METADATA,
},
};
if (tierId !== TIER_FREE && envProject) {
onboardBody.cloudaicompanionProject = envProject;
(onboardBody.metadata as Record<string, unknown>).duetProject = envProject;
}
const onboardResponse = await fetchWithTimeout(`${activeEndpoint}/v1internal:onboardUser`, {
method: "POST",
headers,
body: JSON.stringify(onboardBody),
});
if (!onboardResponse.ok) {
throw new Error(`onboardUser failed: ${onboardResponse.status} ${onboardResponse.statusText}`);
}
let lro = (await onboardResponse.json()) as {
done?: boolean;
name?: string;
response?: { cloudaicompanionProject?: { id?: string } };
};
if (!lro.done && lro.name) {
lro = await pollOperation(activeEndpoint, lro.name, headers);
}
const projectId = lro.response?.cloudaicompanionProject?.id;
if (projectId) {
return projectId;
}
if (envProject) {
return envProject;
}
throw new Error(
"Could not discover or provision a Google Cloud project. Set GOOGLE_CLOUD_PROJECT or GOOGLE_CLOUD_PROJECT_ID.",
);
}

View File

@@ -0,0 +1,2 @@
// Google plugin module implements oauth behavior.
export { loginGeminiCliOAuth, refreshGeminiCliOAuthToken } from "./oauth.js";

View File

@@ -0,0 +1,73 @@
// Google plugin module implements oauth.settings behavior.
import { existsSync, readFileSync } from "node:fs";
import { homedir } from "node:os";
import { join } from "node:path";
import { isRecord, normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
type OAuthSettingsFs = {
existsSync: (path: Parameters<typeof existsSync>[0]) => ReturnType<typeof existsSync>;
readFileSync: (path: Parameters<typeof readFileSync>[0], encoding: "utf8") => string;
homedir: typeof homedir;
};
const defaultFs: OAuthSettingsFs = {
existsSync,
readFileSync,
homedir,
};
let oauthSettingsFs: OAuthSettingsFs = defaultFs;
type GeminiCliAuthSettings = {
security?: {
auth?: {
selectedType?: unknown;
enforcedType?: unknown;
};
};
selectedAuthType?: unknown;
enforcedAuthType?: unknown;
};
function readSettingsFile(): GeminiCliAuthSettings | null {
const settingsPath = join(oauthSettingsFs.homedir(), ".gemini", "settings.json");
if (!oauthSettingsFs.existsSync(settingsPath)) {
return null;
}
try {
const parsed = JSON.parse(oauthSettingsFs.readFileSync(settingsPath, "utf8")) as unknown;
return isRecord(parsed) ? (parsed as GeminiCliAuthSettings) : null;
} catch {
return null;
}
}
export function setOAuthSettingsFsForTest(overrides?: Partial<OAuthSettingsFs>): void {
oauthSettingsFs = overrides ? { ...defaultFs, ...overrides } : defaultFs;
}
export function resolveGeminiCliSelectedAuthType(): string | undefined {
const settings = readSettingsFile();
if (settings) {
const security = isRecord(settings.security) ? settings.security : undefined;
const auth = isRecord(security?.auth) ? security.auth : undefined;
const selectedAuthType =
normalizeOptionalString(auth?.selectedType) ??
normalizeOptionalString(auth?.enforcedType) ??
normalizeOptionalString(settings.selectedAuthType) ??
normalizeOptionalString(settings.enforcedAuthType);
if (selectedAuthType) {
return selectedAuthType;
}
}
if (process.env.GOOGLE_GENAI_USE_GCA === "true") {
return "oauth-personal";
}
return undefined;
}
export function isGeminiCliPersonalOAuth(): boolean {
return resolveGeminiCliSelectedAuthType() === "oauth-personal";
}

View File

@@ -0,0 +1,45 @@
// Google plugin module implements oauth.shared behavior.
export const CLIENT_ID_KEYS = ["OPENCLAW_GEMINI_OAUTH_CLIENT_ID", "GEMINI_CLI_OAUTH_CLIENT_ID"];
export const CLIENT_SECRET_KEYS = [
"OPENCLAW_GEMINI_OAUTH_CLIENT_SECRET",
"GEMINI_CLI_OAUTH_CLIENT_SECRET",
];
export const REDIRECT_URI = "http://localhost:8085/oauth2callback";
export const AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth";
export const TOKEN_URL = "https://oauth2.googleapis.com/token";
export const USERINFO_URL = "https://www.googleapis.com/oauth2/v1/userinfo?alt=json";
export const CODE_ASSIST_ENDPOINT_PROD = "https://cloudcode-pa.googleapis.com";
const CODE_ASSIST_ENDPOINT_DAILY = "https://daily-cloudcode-pa.sandbox.googleapis.com";
const CODE_ASSIST_ENDPOINT_AUTOPUSH = "https://autopush-cloudcode-pa.sandbox.googleapis.com";
export const LOAD_CODE_ASSIST_ENDPOINTS = [
CODE_ASSIST_ENDPOINT_PROD,
CODE_ASSIST_ENDPOINT_DAILY,
CODE_ASSIST_ENDPOINT_AUTOPUSH,
];
export const DEFAULT_FETCH_TIMEOUT_MS = 10_000;
export const SCOPES = [
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/userinfo.profile",
];
export const TIER_FREE = "free-tier";
export const TIER_LEGACY = "legacy-tier";
export const TIER_STANDARD = "standard-tier";
export type GeminiCliOAuthCredentials = {
access: string;
refresh: string;
expires: number;
email?: string;
projectId?: string;
};
export type GeminiCliOAuthContext = {
isRemote: boolean;
openUrl: (url: string) => Promise<void>;
log: (msg: string) => void;
note: (message: string, title?: string) => Promise<void>;
prompt: (message: string) => Promise<string>;
progress: { update: (msg: string) => void; stop: (msg?: string) => void };
};

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,161 @@
// Google plugin module implements oauth.token behavior.
import {
asDateTimestampMs,
resolveExpiresAtMsFromDurationSeconds,
} from "openclaw/plugin-sdk/number-runtime";
import { readResponseTextLimited } from "openclaw/plugin-sdk/provider-http";
import { resolveOAuthClientConfig } from "./oauth.credentials.js";
import { fetchWithTimeout } from "./oauth.http.js";
import { resolveGoogleOAuthIdentity, resolveGooglePersonalOAuthIdentity } from "./oauth.project.js";
import { isGeminiCliPersonalOAuth } from "./oauth.settings.js";
import { REDIRECT_URI, TOKEN_URL, type GeminiCliOAuthCredentials } from "./oauth.shared.js";
const TOKEN_EXPIRY_BUFFER_MS = 5 * 60 * 1000;
const GOOGLE_OAUTH_TOKEN_ERROR_BODY_LIMIT_BYTES = 8 * 1024;
async function requestTokenGrant(body: URLSearchParams): Promise<{
access_token?: string;
refresh_token?: string;
expires_in?: unknown;
}> {
const response = await fetchWithTimeout(TOKEN_URL, {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
Accept: "*/*",
"User-Agent": "google-api-nodejs-client/9.15.1",
},
body,
});
if (!response.ok) {
const errorText = await readResponseTextLimited(
response,
GOOGLE_OAUTH_TOKEN_ERROR_BODY_LIMIT_BYTES,
);
throw new Error(`Token exchange failed: ${errorText}`);
}
return (await response.json()) as {
access_token?: string;
refresh_token?: string;
expires_in?: unknown;
};
}
function resolveExpiredTokenTimestampMs(nowMs: number): number {
return asDateTimestampMs(nowMs - TOKEN_EXPIRY_BUFFER_MS) ?? nowMs;
}
function resolveTokenExpiresAt(value: unknown): number {
const nowMs = asDateTimestampMs(Date.now());
if (nowMs === undefined) {
return 0;
}
return (
resolveExpiresAtMsFromDurationSeconds(value, { nowMs, bufferMs: TOKEN_EXPIRY_BUFFER_MS }) ??
resolveExpiredTokenTimestampMs(nowMs)
);
}
async function buildGeminiCliCredentials(params: {
tokenResponse: {
access_token?: string;
refresh_token?: string;
expires_in?: unknown;
};
refreshTokenFallback?: string;
existing?: Pick<GeminiCliOAuthCredentials, "email" | "projectId">;
allowIdentityFallback?: boolean;
}): Promise<GeminiCliOAuthCredentials> {
const accessToken = params.tokenResponse.access_token;
if (!accessToken) {
throw new Error("No access token received. Please try again.");
}
let identity: { email?: string; projectId?: string } = params.existing ?? {};
try {
if (!identity.email || !identity.projectId) {
const discovered = await resolveGeminiCliIdentity(accessToken);
identity = {
email: identity.email ?? discovered.email,
projectId: identity.projectId ?? discovered.projectId,
};
}
} catch (error) {
if (!params.allowIdentityFallback || (!params.existing?.email && !params.existing?.projectId)) {
throw error;
}
// If identity discovery is temporarily unavailable during refresh, keep the
// already-stored identity binding instead of failing token renewal.
}
const expiresAt = resolveTokenExpiresAt(params.tokenResponse.expires_in);
return {
refresh: params.tokenResponse.refresh_token ?? params.refreshTokenFallback ?? "",
access: accessToken,
expires: expiresAt,
projectId: identity.projectId,
email: identity.email,
};
}
async function resolveGeminiCliIdentity(
accessToken: string,
): Promise<{ email?: string; projectId?: string }> {
return isGeminiCliPersonalOAuth()
? await resolveGooglePersonalOAuthIdentity(accessToken)
: await resolveGoogleOAuthIdentity(accessToken);
}
export async function exchangeCodeForTokens(
code: string,
verifier: string,
): Promise<GeminiCliOAuthCredentials> {
const { clientId, clientSecret } = resolveOAuthClientConfig();
const body = new URLSearchParams({
client_id: clientId,
code,
grant_type: "authorization_code",
redirect_uri: REDIRECT_URI,
code_verifier: verifier,
});
if (clientSecret) {
body.set("client_secret", clientSecret);
}
const refreshed = await buildGeminiCliCredentials({
tokenResponse: await requestTokenGrant(body),
});
if (!refreshed.refresh) {
throw new Error("No refresh token received. Please try again.");
}
return refreshed;
}
export async function refreshTokensForGeminiCli(credentials: {
refresh: string;
email?: string;
projectId?: string;
}): Promise<GeminiCliOAuthCredentials> {
const { clientId, clientSecret } = resolveOAuthClientConfig();
const body = new URLSearchParams({
client_id: clientId,
grant_type: "refresh_token",
refresh_token: credentials.refresh,
});
if (clientSecret) {
body.set("client_secret", clientSecret);
}
return await buildGeminiCliCredentials({
tokenResponse: await requestTokenGrant(body),
refreshTokenFallback: credentials.refresh,
existing: {
email: credentials.email,
projectId: credentials.projectId,
},
allowIdentityFallback: true,
});
}

106
extensions/google/oauth.ts Normal file
View File

@@ -0,0 +1,106 @@
// Google plugin module implements oauth behavior.
import type { OAuthCredential } from "openclaw/plugin-sdk/provider-auth";
import { clearCredentialsCache, extractGeminiCliCredentials } from "./oauth.credentials.js";
import {
buildAuthUrl,
generateOAuthState,
generatePkce,
parseCallbackInput,
shouldUseManualOAuthFlow,
waitForLocalCallback,
} from "./oauth.flow.js";
import type { GeminiCliOAuthContext, GeminiCliOAuthCredentials } from "./oauth.shared.js";
import { exchangeCodeForTokens, refreshTokensForGeminiCli } from "./oauth.token.js";
export { clearCredentialsCache, extractGeminiCliCredentials };
export type { GeminiCliOAuthContext, GeminiCliOAuthCredentials };
export async function loginGeminiCliOAuth(
ctx: GeminiCliOAuthContext,
): Promise<GeminiCliOAuthCredentials> {
const needsManual = shouldUseManualOAuthFlow(ctx.isRemote);
await ctx.note(
needsManual
? [
"You are running in a remote/VPS environment.",
"A URL will be shown for you to open in your LOCAL browser.",
"After signing in, copy the redirect URL and paste it back here.",
].join("\n")
: [
"Browser will open for Google authentication.",
"Sign in with your Google account for Gemini CLI access.",
"The callback will be captured automatically on localhost:8085.",
].join("\n"),
"Gemini CLI OAuth",
);
const { verifier, challenge } = generatePkce();
const state = generateOAuthState();
const authUrl = buildAuthUrl(challenge, state);
if (needsManual) {
return manualFlow(ctx, authUrl, state, verifier);
}
ctx.progress.update("Complete sign-in in browser...");
ctx.log(`\nOpen this URL in your browser:\n\n${authUrl}\n`);
try {
await ctx.openUrl(authUrl);
} catch {
// The URL is already visible; browser launch is best-effort.
}
try {
const { code } = await waitForLocalCallback({
expectedState: state,
timeoutMs: 5 * 60 * 1000,
onProgress: (msg) => ctx.progress.update(msg),
});
ctx.progress.update("Exchanging authorization code for tokens...");
return await exchangeCodeForTokens(code, verifier);
} catch (err) {
if (
err instanceof Error &&
(err.message.includes("EADDRINUSE") ||
err.message.includes("port") ||
err.message.includes("listen"))
) {
ctx.progress.update("Local callback server failed. Switching to manual mode...");
return manualFlow(ctx, authUrl, state, verifier, err);
}
throw err;
}
}
async function manualFlow(
ctx: GeminiCliOAuthContext,
authUrl: string,
state: string,
verifier: string,
cause?: Error,
): Promise<GeminiCliOAuthCredentials> {
ctx.progress.update("OAuth URL ready");
ctx.log(`\nOpen this URL in your LOCAL browser:\n\n${authUrl}\n`);
ctx.progress.update("Waiting for you to paste the callback URL...");
const callbackInput = await ctx.prompt("Paste the redirect URL here: ");
const parsed = parseCallbackInput(callbackInput);
if ("error" in parsed) {
throw new Error(parsed.error, cause ? { cause } : undefined);
}
if (parsed.state !== state) {
throw new Error("OAuth state mismatch - please try again", cause ? { cause } : undefined);
}
ctx.progress.update("Exchanging authorization code for tokens...");
return exchangeCodeForTokens(parsed.code, verifier);
}
export async function refreshGeminiCliOAuthToken(
credentials: Pick<GeminiCliOAuthCredentials, "refresh" | "email" | "projectId">,
): Promise<OAuthCredential> {
const refreshed = await refreshTokensForGeminiCli(credentials);
return {
type: "oauth",
provider: "google-gemini-cli",
...refreshed,
};
}

View File

@@ -0,0 +1,79 @@
// Google setup module handles plugin onboarding behavior.
import {
applyAgentDefaultModelPrimary,
type OpenClawConfig,
} from "openclaw/plugin-sdk/provider-onboard";
export const GOOGLE_GEMINI_DEFAULT_MODEL = "google/gemini-3.1-pro-preview";
const RETIRED_GOOGLE_GEMINI_MODEL_REFS = new Set([
"gemini-3-pro",
"gemini-3-pro-preview",
"google/gemini-3-pro",
"google/gemini-3-pro-preview",
]);
function isRetiredGeminiModelRef(value: unknown): boolean {
if (typeof value !== "string") {
return false;
}
const modelRef = value.trim();
if (RETIRED_GOOGLE_GEMINI_MODEL_REFS.has(modelRef)) {
return true;
}
return modelRef.endsWith("/gemini-3-pro") || modelRef.endsWith("/gemini-3-pro-preview");
}
function hasRetiredGeminiDefaultModelRefs(cfg: OpenClawConfig): boolean {
const defaults = cfg.agents?.defaults;
const model = defaults?.model as unknown;
if (model && typeof model === "object") {
const fallbacks = (model as { fallbacks?: unknown }).fallbacks;
if (
Array.isArray(fallbacks) &&
fallbacks.some((fallback) => isRetiredGeminiModelRef(fallback))
) {
return true;
}
}
const models = defaults?.models;
if (
models &&
typeof models === "object" &&
Object.keys(models).some((modelRef) => isRetiredGeminiModelRef(modelRef))
) {
return true;
}
const providerConfigs = cfg.models?.providers;
if (!providerConfigs) {
return false;
}
return Object.values(providerConfigs).some((providerConfig) =>
Array.isArray(providerConfig.models)
? providerConfig.models.some((providerModel) => isRetiredGeminiModelRef(providerModel.id))
: false,
);
}
export function applyGoogleGeminiModelDefault(cfg: OpenClawConfig): {
next: OpenClawConfig;
changed: boolean;
} {
const current = cfg.agents?.defaults?.model as unknown;
const currentPrimary =
typeof current === "string"
? current.trim() || undefined
: current &&
typeof current === "object" &&
typeof (current as { primary?: unknown }).primary === "string"
? ((current as { primary: string }).primary || "").trim() || undefined
: undefined;
if (currentPrimary === GOOGLE_GEMINI_DEFAULT_MODEL && !hasRetiredGeminiDefaultModelRefs(cfg)) {
return { next: cfg, changed: false };
}
return {
next: applyAgentDefaultModelPrimary(cfg, GOOGLE_GEMINI_DEFAULT_MODEL),
changed: true,
};
}

View File

@@ -0,0 +1,719 @@
{
"id": "google",
"icon": "https://cdn.simpleicons.org/google",
"activation": {
"onStartup": false
},
"enabledByDefault": true,
"providers": ["google", "google-gemini-cli", "google-vertex"],
"providerCatalogEntry": "./provider-discovery.ts",
"autoEnableWhenConfiguredProviders": ["google-gemini-cli"],
"modelIdNormalization": {
"providers": {
"google": {
"aliases": {
"gemini-3-pro": "gemini-3.1-pro-preview",
"gemini-3-pro-preview": "gemini-3.1-pro-preview",
"gemini-3-flash": "gemini-3-flash-preview",
"gemini-3.1-pro": "gemini-3.1-pro-preview",
"gemini-3.1-flash-lite-preview": "gemini-3.1-flash-lite",
"gemini-3.1-flash": "gemini-3-flash-preview",
"gemini-3.1-flash-preview": "gemini-3-flash-preview"
}
},
"google-gemini-cli": {
"aliases": {
"gemini-3-pro": "gemini-3.1-pro-preview",
"gemini-3-pro-preview": "gemini-3.1-pro-preview",
"gemini-3-flash": "gemini-3-flash-preview",
"gemini-3.1-pro": "gemini-3.1-pro-preview",
"gemini-3.1-flash-lite-preview": "gemini-3.1-flash-lite",
"gemini-3.1-flash": "gemini-3-flash-preview",
"gemini-3.1-flash-preview": "gemini-3-flash-preview"
}
},
"google-vertex": {
"aliases": {
"gemini-3-pro": "gemini-3.1-pro-preview",
"gemini-3-pro-preview": "gemini-3.1-pro-preview",
"gemini-3-flash": "gemini-3-flash-preview",
"gemini-3.1-pro": "gemini-3.1-pro-preview",
"gemini-3.1-flash-lite-preview": "gemini-3.1-flash-lite",
"gemini-3.1-flash": "gemini-3-flash-preview",
"gemini-3.1-flash-preview": "gemini-3-flash-preview"
}
}
}
},
"modelCatalog": {
"suppressions": [
{
"provider": "google",
"model": "gemini-1.5-flash",
"reason": "Google shut down Gemini 1.5 Flash on 2025-09-29. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-1.5-flash-8b",
"reason": "Google shut down Gemini 1.5 Flash-8B on 2025-09-29. Use google/gemini-2.5-flash-lite."
},
{
"provider": "google",
"model": "gemini-1.5-pro",
"reason": "Google shut down Gemini 1.5 Pro on 2025-09-29. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-2.0-flash-exp",
"reason": "Google shut down this Gemini 2.0 experimental model. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-2.0-flash-exp-image-generation",
"reason": "Google shut down this Gemini 2.0 image preview. Use google/gemini-2.5-flash-image."
},
{
"provider": "google",
"model": "gemini-2.0-flash-live-001",
"reason": "Google shut down this Gemini Live model on 2025-12-09. Use google/gemini-3.1-flash-live-preview."
},
{
"provider": "google",
"model": "gemini-2.0-flash-lite-preview",
"reason": "Google shut down this Gemini 2.0 Flash-Lite preview on 2025-12-09. Use google/gemini-2.5-flash-lite."
},
{
"provider": "google",
"model": "gemini-2.0-flash-lite-preview-02-05",
"reason": "Google shut down this Gemini 2.0 Flash-Lite preview on 2025-12-09. Use google/gemini-2.5-flash-lite."
},
{
"provider": "google",
"model": "gemini-2.0-flash-preview-image-generation",
"reason": "Google shut down this Gemini 2.0 image preview. Use google/gemini-2.5-flash-image."
},
{
"provider": "google",
"model": "gemini-2.0-flash-thinking-exp",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-2.0-flash-thinking-exp-01-21",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-2.0-flash-thinking-exp-1219",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-2.0-pro-exp",
"reason": "Google shut down this Gemini 2.0 Pro experiment. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-2.0-pro-exp-02-05",
"reason": "Google shut down this Gemini 2.0 Pro experiment. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-2.5-flash-exp-native-audio-thinking-dialog",
"reason": "Google shut down this Gemini native-audio preview. Use google/gemini-3.1-flash-live-preview."
},
{
"provider": "google",
"model": "gemini-2.5-flash-image-preview",
"reason": "Google shut down this Gemini image preview on 2026-01-15. Use google/gemini-2.5-flash-image."
},
{
"provider": "google",
"model": "gemini-2.5-flash-lite-preview-06-17",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2025-11-18. Use google/gemini-2.5-flash-lite."
},
{
"provider": "google",
"model": "gemini-2.5-flash-lite-preview-09-25",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2026-03-31. Use google/gemini-3.1-flash-lite."
},
{
"provider": "google",
"model": "gemini-2.5-flash-lite-preview-09-2025",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2026-03-31. Use google/gemini-3.1-flash-lite."
},
{
"provider": "google",
"model": "gemini-2.5-flash-preview-04-17",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2025-07-15. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-2.5-flash-preview-05-20",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2025-11-18. Use google/gemini-2.5-flash."
},
{
"provider": "google",
"model": "gemini-2.5-flash-preview-09-25",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2026-02-17. Use google/gemini-3-flash-preview."
},
{
"provider": "google",
"model": "gemini-2.5-flash-preview-09-2025",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2026-02-17. Use google/gemini-3-flash-preview."
},
{
"provider": "google",
"model": "gemini-2.5-flash-preview-native-audio-dialog",
"reason": "Google shut down this Gemini native-audio preview. Use google/gemini-3.1-flash-live-preview."
},
{
"provider": "google",
"model": "gemini-2.5-pro-exp-03-25",
"reason": "Google shut down this Gemini 2.5 Pro experiment. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-2.5-pro-preview-03-25",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-2.5-pro-preview-05-06",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-2.5-pro-preview-06-05",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-3-pro-preview",
"reason": "Google shut down Gemini 3 Pro Preview on 2026-03-09. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-3.1-pro-preview-customtools",
"reason": "This is not a public Google Gemini chat model ID. Use google/gemini-3.1-pro-preview."
},
{
"provider": "google",
"model": "gemini-live-2.5-flash",
"reason": "This is not a current public Gemini chat model ID. Use google/gemini-3.1-flash-live-preview for Live API."
},
{
"provider": "google",
"model": "gemini-live-2.5-flash-preview",
"reason": "Google shut down this Gemini Live model on 2025-12-09. Use google/gemini-3.1-flash-live-preview."
},
{
"provider": "google",
"model": "gemini-live-2.5-flash-preview-native-audio",
"reason": "Google shut down this Gemini Live preview. Use google/gemini-3.1-flash-live-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-1.5-flash",
"reason": "Google shut down Gemini 1.5 Flash on 2025-09-29. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-1.5-flash-8b",
"reason": "Google shut down Gemini 1.5 Flash-8B on 2025-09-29. Use google-gemini-cli/gemini-2.5-flash-lite."
},
{
"provider": "google-gemini-cli",
"model": "gemini-1.5-pro",
"reason": "Google shut down Gemini 1.5 Pro on 2025-09-29. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-exp",
"reason": "Google shut down this Gemini 2.0 experimental model. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-exp-image-generation",
"reason": "Google shut down this Gemini 2.0 image preview. Use google-gemini-cli/gemini-2.5-flash-image."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-live-001",
"reason": "Google shut down this Gemini Live model on 2025-12-09. Use google-gemini-cli/gemini-3.1-flash-live-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-lite-preview",
"reason": "Google shut down this Gemini 2.0 Flash-Lite preview on 2025-12-09. Use google-gemini-cli/gemini-2.5-flash-lite."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-lite-preview-02-05",
"reason": "Google shut down this Gemini 2.0 Flash-Lite preview on 2025-12-09. Use google-gemini-cli/gemini-2.5-flash-lite."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-preview-image-generation",
"reason": "Google shut down this Gemini 2.0 image preview. Use google-gemini-cli/gemini-2.5-flash-image."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-thinking-exp",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-thinking-exp-01-21",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-flash-thinking-exp-1219",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-pro-exp",
"reason": "Google shut down this Gemini 2.0 Pro experiment. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.0-pro-exp-02-05",
"reason": "Google shut down this Gemini 2.0 Pro experiment. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-exp-native-audio-thinking-dialog",
"reason": "Google shut down this Gemini native-audio preview. Use google-gemini-cli/gemini-3.1-flash-live-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-image-preview",
"reason": "Google shut down this Gemini image preview on 2026-01-15. Use google-gemini-cli/gemini-2.5-flash-image."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-lite-preview-06-17",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2025-11-18. Use google-gemini-cli/gemini-2.5-flash-lite."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-lite-preview-09-25",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2026-03-31. Use google-gemini-cli/gemini-3.1-flash-lite."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-lite-preview-09-2025",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2026-03-31. Use google-gemini-cli/gemini-3.1-flash-lite."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-preview-04-17",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2025-07-15. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-preview-05-20",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2025-11-18. Use google-gemini-cli/gemini-2.5-flash."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-preview-09-25",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2026-02-17. Use google-gemini-cli/gemini-3-flash-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-preview-09-2025",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2026-02-17. Use google-gemini-cli/gemini-3-flash-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-flash-preview-native-audio-dialog",
"reason": "Google shut down this Gemini native-audio preview. Use google-gemini-cli/gemini-3.1-flash-live-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-pro-exp-03-25",
"reason": "Google shut down this Gemini 2.5 Pro experiment. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-pro-preview-03-25",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-pro-preview-05-06",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-2.5-pro-preview-06-05",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-3-pro-preview",
"reason": "Google shut down Gemini 3 Pro Preview on 2026-03-09. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-3.1-pro-preview-customtools",
"reason": "This is not a public Google Gemini chat model ID. Use google-gemini-cli/gemini-3.1-pro-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-live-2.5-flash",
"reason": "This is not a current public Gemini chat model ID. Use google-gemini-cli/gemini-3.1-flash-live-preview for Live API."
},
{
"provider": "google-gemini-cli",
"model": "gemini-live-2.5-flash-preview",
"reason": "Google shut down this Gemini Live model on 2025-12-09. Use google-gemini-cli/gemini-3.1-flash-live-preview."
},
{
"provider": "google-gemini-cli",
"model": "gemini-live-2.5-flash-preview-native-audio",
"reason": "Google shut down this Gemini Live preview. Use google-gemini-cli/gemini-3.1-flash-live-preview."
},
{
"provider": "google-vertex",
"model": "gemini-1.5-flash",
"reason": "Google retired Gemini 1.5 Flash. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-1.5-flash-8b",
"reason": "Google retired Gemini 1.5 Flash-8B. Use google-vertex/gemini-2.5-flash-lite."
},
{
"provider": "google-vertex",
"model": "gemini-1.5-pro",
"reason": "Google retired Gemini 1.5 Pro. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-exp",
"reason": "Google shut down this Gemini 2.0 experimental model. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-exp-image-generation",
"reason": "Google shut down this Gemini 2.0 image preview. Use google-vertex/gemini-2.5-flash-image."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-live-001",
"reason": "Google shut down this Gemini Live model on 2025-12-09. Use google-vertex/gemini-3.1-flash-live-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-lite-preview",
"reason": "Google shut down this Gemini 2.0 Flash-Lite preview on 2025-12-09. Use google-vertex/gemini-2.5-flash-lite."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-lite-preview-02-05",
"reason": "Google shut down this Gemini 2.0 Flash-Lite preview on 2025-12-09. Use google-vertex/gemini-2.5-flash-lite."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-preview-image-generation",
"reason": "Google shut down this Gemini 2.0 image preview. Use google-vertex/gemini-2.5-flash-image."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-thinking-exp",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-thinking-exp-01-21",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-flash-thinking-exp-1219",
"reason": "Google shut down this Gemini 2.0 thinking experiment. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-pro-exp",
"reason": "Google shut down this Gemini 2.0 Pro experiment. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.0-pro-exp-02-05",
"reason": "Google shut down this Gemini 2.0 Pro experiment. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-exp-native-audio-thinking-dialog",
"reason": "Google shut down this Gemini native-audio preview. Use google-vertex/gemini-3.1-flash-live-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-image-preview",
"reason": "Google shut down this Gemini image preview on 2026-01-15. Use google-vertex/gemini-2.5-flash-image."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-lite-preview-06-17",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2025-11-18. Use google-vertex/gemini-2.5-flash-lite."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-lite-preview-09-25",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2026-03-31. Use google-vertex/gemini-3.1-flash-lite."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-lite-preview-09-2025",
"reason": "Google shut down this Gemini 2.5 Flash-Lite preview on 2026-03-31. Use google-vertex/gemini-3.1-flash-lite."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-preview-04-17",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2025-07-15. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-preview-05-20",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2025-11-18. Use google-vertex/gemini-2.5-flash."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-preview-09-25",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2026-02-17. Use google-vertex/gemini-3-flash-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-preview-09-2025",
"reason": "Google shut down this Gemini 2.5 Flash preview on 2026-02-17. Use google-vertex/gemini-3-flash-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-flash-preview-native-audio-dialog",
"reason": "Google shut down this Gemini native-audio preview. Use google-vertex/gemini-3.1-flash-live-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-pro-exp-03-25",
"reason": "Google shut down this Gemini 2.5 Pro experiment. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-pro-preview-03-25",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-pro-preview-05-06",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-2.5-pro-preview-06-05",
"reason": "Google shut down this Gemini 2.5 Pro preview on 2025-12-02. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-3-pro-preview",
"reason": "Google shut down Gemini 3 Pro Preview on 2026-03-09. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-3.1-pro-preview-customtools",
"reason": "This is not a public Google Gemini chat model ID. Use google-vertex/gemini-3.1-pro-preview."
},
{
"provider": "google-vertex",
"model": "gemini-live-2.5-flash",
"reason": "This is not a current public Gemini chat model ID. Use google-vertex/gemini-3.1-flash-live-preview for Live API."
},
{
"provider": "google-vertex",
"model": "gemini-live-2.5-flash-preview",
"reason": "Google shut down this Gemini Live model on 2025-12-09. Use google-vertex/gemini-3.1-flash-live-preview."
},
{
"provider": "google-vertex",
"model": "gemini-live-2.5-flash-preview-native-audio",
"reason": "Google shut down this Gemini Live preview. Use google-vertex/gemini-3.1-flash-live-preview."
}
]
},
"modelPricing": {
"providers": {
"google-gemini-cli": {
"openRouter": {
"provider": "google"
},
"liteLLM": {
"provider": "google"
}
}
}
},
"providerEndpoints": [
{
"endpointClass": "google-generative-ai",
"hosts": ["generativelanguage.googleapis.com"]
},
{
"endpointClass": "google-vertex",
"hosts": ["aiplatform.googleapis.com"],
"googleVertexRegion": "global"
},
{
"endpointClass": "google-vertex",
"hosts": ["aiplatform.eu.rep.googleapis.com"],
"googleVertexRegion": "eu"
},
{
"endpointClass": "google-vertex",
"hosts": ["aiplatform.us.rep.googleapis.com"],
"googleVertexRegion": "us"
},
{
"endpointClass": "google-vertex",
"hostSuffixes": ["-aiplatform.googleapis.com"],
"googleVertexRegionHostSuffix": "-aiplatform.googleapis.com"
}
],
"providerRequest": {
"providers": {
"google": {
"family": "google"
},
"google-gemini-cli": {
"family": "google"
},
"google-vertex": {
"family": "google"
}
}
},
"setup": {
"providers": [
{
"id": "google-vertex",
"authMethods": ["api-key"],
"envVars": ["GOOGLE_CLOUD_API_KEY"],
"authEvidence": [
{
"type": "local-file-with-env",
"fileEnvVar": "GOOGLE_APPLICATION_CREDENTIALS",
"fallbackPaths": [
"${HOME}/.config/gcloud/application_default_credentials.json",
"${APPDATA}/gcloud/application_default_credentials.json"
],
"requiresAnyEnv": ["GOOGLE_CLOUD_PROJECT", "GCLOUD_PROJECT"],
"requiresAllEnv": ["GOOGLE_CLOUD_LOCATION"],
"credentialMarker": "gcp-vertex-credentials",
"source": "gcloud adc"
}
]
},
{
"id": "google",
"envVars": ["GEMINI_API_KEY", "GOOGLE_API_KEY"]
}
]
},
"cliBackends": ["google-gemini-cli"],
"providerAuthChoices": [
{
"provider": "google",
"method": "api-key",
"choiceId": "gemini-api-key",
"choiceLabel": "Google Gemini API key",
"groupId": "google",
"groupLabel": "Google",
"groupHint": "Gemini API key + OAuth",
"onboardingFeatured": true,
"optionKey": "geminiApiKey",
"cliFlag": "--gemini-api-key",
"cliOption": "--gemini-api-key <key>",
"cliDescription": "Gemini API key"
},
{
"provider": "google-gemini-cli",
"method": "oauth",
"choiceId": "google-gemini-cli",
"choiceLabel": "Gemini CLI OAuth",
"choiceHint": "Google OAuth with project-aware token payload",
"groupId": "google",
"groupLabel": "Google",
"groupHint": "Gemini API key + OAuth",
"onboardingFeatured": true
}
],
"uiHints": {
"webSearch.apiKey": {
"label": "Gemini Search API Key",
"help": "Gemini API key for Google Search grounding (fallback: GEMINI_API_KEY env var).",
"sensitive": true,
"placeholder": "AIza..."
},
"webSearch.model": {
"label": "Gemini Search Model",
"help": "Gemini model override for web search grounding."
},
"webSearch.baseUrl": {
"label": "Gemini Search Base URL",
"help": "Optional Gemini API base URL for web search grounding proxies."
}
},
"contracts": {
"mediaUnderstandingProviders": ["google"],
"memoryEmbeddingProviders": ["gemini"],
"imageGenerationProviders": ["google"],
"musicGenerationProviders": ["google"],
"realtimeVoiceProviders": ["google"],
"speechProviders": ["google"],
"videoGenerationProviders": ["google"],
"webSearchProviders": ["gemini"]
},
"mediaUnderstandingProviderMetadata": {
"google": {
"capabilities": ["image", "audio", "video"],
"defaultModels": {
"image": "gemini-3-flash-preview",
"audio": "gemini-3-flash-preview",
"video": "gemini-3-flash-preview"
},
"autoPriority": {
"image": 30,
"audio": 40,
"video": 10
},
"nativeDocumentInputs": ["pdf"]
}
},
"configContracts": {
"compatibilityRuntimePaths": ["tools.web.search.apiKey"]
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {
"webSearch": {
"type": "object",
"additionalProperties": false,
"properties": {
"apiKey": {
"type": ["string", "object"]
},
"model": {
"type": "string"
},
"baseUrl": {
"type": "string"
}
}
}
}
}
}

View File

@@ -0,0 +1,19 @@
{
"name": "@openclaw/google-plugin",
"version": "2026.6.11",
"private": true,
"description": "OpenClaw Google plugin",
"type": "module",
"dependencies": {
"@google/genai": "2.10.0",
"google-auth-library": "10.9.0"
},
"devDependencies": {
"@openclaw/plugin-sdk": "workspace:*"
},
"openclaw": {
"extensions": [
"./index.ts"
]
}
}

View File

@@ -0,0 +1,29 @@
// Google tests cover provider catalog plugin behavior.
import { describe, expect, it } from "vitest";
import {
buildGoogleStaticCatalogProvider,
buildGoogleVertexStaticCatalogProvider,
} from "./provider-catalog.js";
describe("google provider catalog", () => {
it("registers current Gemini rows for the Google Vertex provider", () => {
const provider = buildGoogleVertexStaticCatalogProvider();
expect(provider.api).toBe("google-vertex");
expect(provider.baseUrl).toBe("https://{location}-aiplatform.googleapis.com");
expect(provider.models.map((model) => model.id)).toEqual(
expect.arrayContaining(["gemini-2.5-pro", "gemini-3.1-pro-preview", "gemini-3.1-flash-lite"]),
);
expect(provider.models.find((model) => model.id === "gemini-3.1-flash-lite")).toMatchObject({
contextWindow: 1_048_576,
maxTokens: 65_536,
reasoning: true,
});
});
it("keeps Google AI Studio and Vertex model ids aligned", () => {
expect(buildGoogleVertexStaticCatalogProvider().models.map((model) => model.id)).toEqual(
buildGoogleStaticCatalogProvider().models.map((model) => model.id),
);
});
});

View File

@@ -0,0 +1,90 @@
// Google provider module implements model/runtime integration.
import type {
ModelDefinitionConfig,
ModelProviderConfig,
} from "openclaw/plugin-sdk/provider-model-shared";
const GOOGLE_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta";
const GOOGLE_VERTEX_BASE_URL = "https://{location}-aiplatform.googleapis.com";
const GOOGLE_GEMINI_COST = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 } as const;
const GOOGLE_GEMINI_TEXT_MODELS: ModelDefinitionConfig[] = [
{
id: "gemini-2.5-pro",
name: "Gemini 2.5 Pro",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
{
id: "gemini-2.5-flash",
name: "Gemini 2.5 Flash",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
{
id: "gemini-2.5-flash-lite",
name: "Gemini 2.5 Flash-Lite",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
{
id: "gemini-3.5-flash",
name: "Gemini 3.5 Flash",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
{
id: "gemini-3.1-pro-preview",
name: "Gemini 3.1 Pro Preview",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
{
id: "gemini-3.1-flash-lite",
name: "Gemini 3.1 Flash Lite",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
{
id: "gemini-3-flash-preview",
name: "Gemini 3 Flash Preview",
reasoning: true,
input: ["text", "image"],
cost: GOOGLE_GEMINI_COST,
contextWindow: 1_048_576,
maxTokens: 65_536,
},
];
export function buildGoogleStaticCatalogProvider(): ModelProviderConfig {
return {
baseUrl: GOOGLE_GEMINI_BASE_URL,
api: "google-generative-ai",
models: GOOGLE_GEMINI_TEXT_MODELS,
};
}
export function buildGoogleVertexStaticCatalogProvider(): ModelProviderConfig {
return {
baseUrl: GOOGLE_VERTEX_BASE_URL,
api: "google-vertex",
models: GOOGLE_GEMINI_TEXT_MODELS,
};
}

View File

@@ -0,0 +1,78 @@
// Google API module exposes the plugin public contract.
import type { ProviderPlugin } from "openclaw/plugin-sdk/provider-model-shared";
const noopAuth = async () => ({ profiles: [] });
export function createGoogleProvider(): ProviderPlugin {
return {
id: "google",
label: "Google AI Studio",
docsPath: "/providers/models",
hookAliases: ["google-antigravity", "google-vertex"],
envVars: ["GEMINI_API_KEY", "GOOGLE_API_KEY"],
auth: [
{
id: "api-key",
kind: "api_key",
label: "Google Gemini API key",
hint: "AI Studio / Gemini API key",
run: noopAuth,
wizard: {
choiceId: "gemini-api-key",
choiceLabel: "Google Gemini API key",
groupId: "google",
groupLabel: "Google",
groupHint: "Gemini API key + OAuth",
},
},
],
};
}
export function createGoogleVertexProvider(): ProviderPlugin {
return {
id: "google-vertex",
label: "Google Vertex AI",
docsPath: "/providers/models",
envVars: [
"GOOGLE_CLOUD_API_KEY",
"GOOGLE_CLOUD_PROJECT",
"GCLOUD_PROJECT",
"GOOGLE_CLOUD_LOCATION",
"GOOGLE_APPLICATION_CREDENTIALS",
],
auth: [],
};
}
export function createGoogleGeminiCliProvider(): ProviderPlugin {
return {
id: "google-gemini-cli",
label: "Gemini CLI OAuth",
docsPath: "/providers/models",
aliases: ["gemini-cli"],
envVars: [
"OPENCLAW_GEMINI_OAUTH_CLIENT_ID",
"OPENCLAW_GEMINI_OAUTH_CLIENT_SECRET",
"GEMINI_CLI_OAUTH_CLIENT_ID",
"GEMINI_CLI_OAUTH_CLIENT_SECRET",
],
auth: [
{
id: "oauth",
kind: "oauth",
label: "Google OAuth",
hint: "PKCE + localhost callback",
run: noopAuth,
},
],
wizard: {
setup: {
choiceId: "google-gemini-cli",
choiceLabel: "Gemini CLI OAuth",
choiceHint: "Google OAuth with project-aware token payload",
methodId: "oauth",
},
},
};
}

View File

@@ -0,0 +1,27 @@
// Google provider module implements model/runtime integration.
import type { ProviderPlugin } from "openclaw/plugin-sdk/provider-model-shared";
import {
buildGoogleStaticCatalogProvider,
buildGoogleVertexStaticCatalogProvider,
} from "./provider-catalog.js";
import { resolveGoogleVertexConfigApiKey } from "./vertex-adc.js";
const googleProviderDiscovery: ProviderPlugin = {
id: "google",
label: "Google AI Studio",
docsPath: "/providers/models",
auth: [],
resolveConfigApiKey: ({ provider, env }) =>
provider === "google-vertex" ? resolveGoogleVertexConfigApiKey(env) : undefined,
staticCatalog: {
order: "simple",
run: async () => ({
providers: {
google: buildGoogleStaticCatalogProvider(),
"google-vertex": buildGoogleVertexStaticCatalogProvider(),
},
}),
},
};
export default googleProviderDiscovery;

View File

@@ -0,0 +1,19 @@
// Google provider module implements model/runtime integration.
import type {
ProviderDefaultThinkingPolicyContext,
ProviderThinkingProfile,
} from "openclaw/plugin-sdk/core";
import { buildProviderReplayFamilyHooks } from "openclaw/plugin-sdk/provider-model-shared";
import { buildProviderToolCompatFamilyHooks } from "openclaw/plugin-sdk/provider-tools";
import { resolveGoogleThinkingProfile } from "./provider-policy.js";
import { createGoogleThinkingStreamWrapper } from "./thinking-api.js";
export const GOOGLE_GEMINI_PROVIDER_HOOKS = {
...buildProviderReplayFamilyHooks({
family: "google-gemini",
}),
...buildProviderToolCompatFamilyHooks("gemini"),
resolveThinkingProfile: (context: ProviderDefaultThinkingPolicyContext) =>
resolveGoogleThinkingProfile(context) satisfies ProviderThinkingProfile | undefined,
wrapStreamFn: createGoogleThinkingStreamWrapper,
};

View File

@@ -0,0 +1,532 @@
// Google tests cover provider models plugin behavior.
import type { ProviderRuntimeModel } from "openclaw/plugin-sdk/plugin-entry";
import { describe, expect, it } from "vitest";
import { createProviderDynamicModelContext as createContext } from "../test-support/provider-model-test-helpers.js";
import { isModernGoogleModel, resolveGoogleGeminiForwardCompatModel } from "./provider-models.js";
function createTemplateModel(
provider: string,
id: string,
overrides: Partial<ProviderRuntimeModel> = {},
): ProviderRuntimeModel {
return {
id,
name: id,
provider,
api: provider === "google-gemini-cli" ? "google-gemini-cli" : "google-generative-ai",
baseUrl:
provider === "google-gemini-cli"
? "https://cloudcode-pa.googleapis.com"
: "https://generativelanguage.googleapis.com/v1beta",
reasoning: false,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200_000,
maxTokens: 64_000,
...overrides,
} as ProviderRuntimeModel;
}
function expectModelFields(
model: ProviderRuntimeModel | undefined,
fields: Partial<ProviderRuntimeModel>,
) {
if (!model) {
throw new Error("expected provider model");
}
for (const [key, value] of Object.entries(fields)) {
expect(model[key as keyof ProviderRuntimeModel]).toEqual(value);
}
}
describe("resolveGoogleGeminiForwardCompatModel", () => {
it("resolves stable gemini 2.5 flash-lite from direct google templates for Gemini CLI when available", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-gemini-cli",
ctx: createContext({
provider: "google-gemini-cli",
modelId: "gemini-2.5-flash-lite",
models: [createTemplateModel("google", "gemini-2.5-flash-lite")],
}),
});
expectModelFields(model, {
provider: "google-gemini-cli",
id: "gemini-2.5-flash-lite",
api: "google-generative-ai",
reasoning: false,
});
});
it("resolves stable gemini 2.5 flash-lite from Gemini CLI templates when direct google templates are unavailable", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-gemini-cli",
ctx: createContext({
provider: "google-gemini-cli",
modelId: "gemini-2.5-flash-lite",
models: [
createTemplateModel("google-gemini-cli", "gemini-3.1-flash-lite", {
contextWindow: 1_048_576,
api: "google-gemini-cli",
baseUrl: "https://cloudcode-pa.googleapis.com",
}),
],
}),
});
expectModelFields(model, {
provider: "google-gemini-cli",
id: "gemini-2.5-flash-lite",
api: "google-gemini-cli",
contextWindow: 1_048_576,
reasoning: false,
});
});
it("resolves gemini 3.1 pro for google aliases via an alternate template provider", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-vertex",
ctx: createContext({
provider: "google-vertex",
modelId: "gemini-3.1-pro-preview",
models: [createTemplateModel("google-gemini-cli", "gemini-3-pro-preview")],
}),
});
expectModelFields(model, {
provider: "google-vertex",
id: "gemini-3.1-pro-preview",
api: "google-gemini-cli",
reasoning: false,
});
});
it("canonicalizes retired Gemini 3 Pro preview requests before cloning templates", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemini-3-pro-preview",
models: [createTemplateModel("google", "gemini-3-pro-preview")],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemini-3.1-pro-preview",
api: "google-generative-ai",
reasoning: true,
});
});
it("canonicalizes provider-qualified retired Gemini 3 Pro preview requests", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "google/gemini-3-pro-preview",
models: [createTemplateModel("google", "gemini-3.1-pro-preview")],
}),
});
expectModelFields(model, {
provider: "google",
id: "google/gemini-3.1-pro-preview",
api: "google-generative-ai",
reasoning: true,
});
});
it("keeps Gemini CLI 3.1 clones sourced from CLI templates when both catalogs exist", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-gemini-cli",
ctx: createContext({
provider: "google-gemini-cli",
modelId: "gemini-3.1-pro-preview",
models: [
createTemplateModel("google-gemini-cli", "gemini-3-pro-preview", {
api: "google-gemini-cli",
baseUrl: "https://cloudcode-pa.googleapis.com",
contextWindow: 1_048_576,
}),
createTemplateModel("google", "gemini-3-pro-preview", {
api: "google-generative-ai",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
contextWindow: 200_000,
}),
],
}),
});
expectModelFields(model, {
provider: "google-gemini-cli",
id: "gemini-3.1-pro-preview",
api: "google-gemini-cli",
baseUrl: "https://cloudcode-pa.googleapis.com",
contextWindow: 1_048_576,
});
});
it("prefers current Gemini 3.1 Pro templates over retired Gemini 3 Pro templates", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-gemini-cli",
ctx: createContext({
provider: "google-gemini-cli",
modelId: "gemini-3.1-pro-preview",
models: [
createTemplateModel("google-gemini-cli", "gemini-3-pro-preview", {
contextWindow: 100_000,
}),
createTemplateModel("google-gemini-cli", "gemini-3.1-pro-preview", {
contextWindow: 1_048_576,
}),
],
}),
});
expectModelFields(model, {
provider: "google-gemini-cli",
id: "gemini-3.1-pro-preview",
contextWindow: 1_048_576,
});
});
it("preserves template reasoning metadata instead of forcing it on forward-compat clones", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemini-3.1-flash-preview",
models: [
createTemplateModel("google-gemini-cli", "gemini-3-flash-preview", {
reasoning: true,
}),
],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemini-3.1-flash-preview",
api: "google-gemini-cli",
reasoning: true,
});
});
it("resolves gemini 3.1 flash from direct google templates", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemini-3.1-flash-preview",
models: [
createTemplateModel("google", "gemini-3-flash-preview", {
reasoning: false,
}),
],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemini-3.1-flash-preview",
api: "google-generative-ai",
reasoning: false,
});
});
it("resolves canonical gemini 3 flash from older Google flash templates when the exact row is missing", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemini-3-flash-preview",
models: [
createTemplateModel("google", "gemini-2.5-flash", {
contextWindow: 1_048_576,
reasoning: true,
}),
],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemini-3-flash-preview",
api: "google-generative-ai",
input: ["text", "image"],
contextWindow: 1_048_576,
reasoning: true,
});
});
it("resolves canonical Gemini CLI 3 flash from Google flash templates when the CLI row is missing", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-gemini-cli",
ctx: createContext({
provider: "google-gemini-cli",
modelId: "gemini-3-flash-preview",
models: [
createTemplateModel("google", "gemini-2.5-flash", {
contextWindow: 1_048_576,
}),
],
}),
});
expectModelFields(model, {
provider: "google-gemini-cli",
id: "gemini-3-flash-preview",
api: "google-generative-ai",
input: ["text", "image"],
contextWindow: 1_048_576,
});
});
it("resolves Gemini latest aliases from current Google templates", () => {
const models = [
createTemplateModel("google", "gemini-3-pro-preview", { reasoning: true }),
createTemplateModel("google", "gemini-3-flash-preview", { reasoning: true }),
createTemplateModel("google", "gemini-3.1-flash-lite", { reasoning: true }),
];
expectModelFields(
resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({ provider: "google", modelId: "gemini-pro-latest", models }),
}),
{
provider: "google",
id: "gemini-pro-latest",
api: "google-generative-ai",
reasoning: true,
},
);
expectModelFields(
resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({ provider: "google", modelId: "gemini-flash-latest", models }),
}),
{
provider: "google",
id: "gemini-flash-latest",
api: "google-generative-ai",
reasoning: true,
},
);
expectModelFields(
resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({ provider: "google", modelId: "gemini-flash-lite-latest", models }),
}),
{
provider: "google",
id: "gemini-flash-lite-latest",
api: "google-generative-ai",
reasoning: true,
},
);
});
it("resolves Antigravity Gemini 3.1 pro customtools from the low template", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-antigravity",
ctx: createContext({
provider: "google-antigravity",
modelId: "gemini-3.1-pro-preview-customtools",
models: [
createTemplateModel("google-antigravity", "gemini-3-pro-low", {
api: "openai-completions",
baseUrl: "https://antigravity.example/v1",
contextWindow: 1_048_576,
reasoning: true,
}),
],
}),
});
expectModelFields(model, {
provider: "google-antigravity",
id: "gemini-3.1-pro-preview-customtools",
api: "openai-completions",
baseUrl: "https://antigravity.example/v1",
contextWindow: 1_048_576,
reasoning: true,
});
});
it("falls back to the Antigravity high template when the low template is unavailable", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-antigravity",
ctx: createContext({
provider: "google-antigravity",
modelId: "gemini-3.1-pro-preview",
models: [
createTemplateModel("google-antigravity", "gemini-3-pro-high", {
api: "openai-completions",
maxTokens: 65_536,
reasoning: true,
}),
],
}),
});
expectModelFields(model, {
provider: "google-antigravity",
id: "gemini-3.1-pro-preview",
api: "openai-completions",
maxTokens: 65_536,
reasoning: true,
});
});
it("resolves Antigravity Gemini 3.1 flash variants from the flash template", () => {
const models = [
createTemplateModel("google-antigravity", "gemini-3-flash", {
api: "openai-completions",
contextWindow: 1_048_576,
}),
];
expectModelFields(
resolveGoogleGeminiForwardCompatModel({
providerId: "google-antigravity",
ctx: createContext({
provider: "google-antigravity",
modelId: "gemini-3.1-flash-preview",
models,
}),
}),
{
provider: "google-antigravity",
id: "gemini-3.1-flash-preview",
api: "openai-completions",
contextWindow: 1_048_576,
},
);
expectModelFields(
resolveGoogleGeminiForwardCompatModel({
providerId: "google-antigravity",
ctx: createContext({
provider: "google-antigravity",
modelId: "gemini-3.1-flash-lite",
models,
}),
}),
{
provider: "google-antigravity",
id: "gemini-3.1-flash-lite",
api: "openai-completions",
contextWindow: 1_048_576,
},
);
});
it("returns undefined for Antigravity Gemini 3.1 models without a matching template", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-antigravity",
ctx: createContext({
provider: "google-antigravity",
modelId: "gemini-3.1-pro-preview-customtools",
models: [createTemplateModel("google-antigravity", "claude-opus-4-6-thinking")],
}),
});
expect(model).toBeUndefined();
});
it("prefers the flash-lite template before the broader flash prefix", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google-vertex",
ctx: createContext({
provider: "google-vertex",
modelId: "gemini-3.1-flash-lite",
models: [
createTemplateModel("google-gemini-cli", "gemini-3-flash-preview", {
contextWindow: 128_000,
}),
createTemplateModel("google-gemini-cli", "gemini-3.1-flash-lite", {
contextWindow: 1_048_576,
}),
],
}),
});
expectModelFields(model, {
provider: "google-vertex",
id: "gemini-3.1-flash-lite",
contextWindow: 1_048_576,
reasoning: false,
});
});
it("treats gemini 2.5 ids as modern google models", () => {
expect(isModernGoogleModel("gemini-2.5-pro")).toBe(true);
expect(isModernGoogleModel("gemini-2.5-flash-lite")).toBe(true);
expect(isModernGoogleModel("gemini-1.5-pro")).toBe(false);
});
it("treats Gemini latest aliases as modern google models", () => {
expect(isModernGoogleModel("gemini-pro-latest")).toBe(true);
expect(isModernGoogleModel("gemini-flash-latest")).toBe(true);
expect(isModernGoogleModel("gemini-flash-lite-latest")).toBe(true);
});
it("treats gemma models as modern google models", () => {
expect(isModernGoogleModel("gemma-4-26b-a4b-it")).toBe(true);
expect(isModernGoogleModel("gemma-3-4b-it")).toBe(true);
});
it("resolves Gemma 4 models with reasoning enabled regardless of template", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemma-4-26b-a4b-it",
models: [createTemplateModel("google", "gemini-3-flash-preview", { reasoning: false })],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemma-4-26b-a4b-it",
reasoning: true,
});
});
it("canonicalizes Gemma 4 26B shorthand before cloning templates", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemma-4-26b",
models: [createTemplateModel("google", "gemini-3-flash-preview", { reasoning: false })],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemma-4-26b-a4b-it",
api: "google-generative-ai",
reasoning: true,
});
});
it("preserves template reasoning for non-Gemma 4 gemma models", () => {
const model = resolveGoogleGeminiForwardCompatModel({
providerId: "google",
ctx: createContext({
provider: "google",
modelId: "gemma-3-4b-it",
models: [createTemplateModel("google", "gemini-3-flash-preview", { reasoning: false })],
}),
});
expectModelFields(model, {
provider: "google",
id: "gemma-3-4b-it",
reasoning: false,
});
});
});

View File

@@ -0,0 +1,244 @@
// Google provider module implements model/runtime integration.
import type {
ProviderResolveDynamicModelContext,
ProviderRuntimeModel,
} from "openclaw/plugin-sdk/plugin-entry";
import { cloneFirstTemplateModel } from "openclaw/plugin-sdk/provider-model-shared";
import { normalizeOptionalLowercaseString } from "openclaw/plugin-sdk/string-coerce-runtime";
import { normalizeGoogleModelId } from "./model-id.js";
const GOOGLE_GEMINI_CLI_PROVIDER_ID = "google-gemini-cli";
const GOOGLE_ANTIGRAVITY_PROVIDER_ID = "google-antigravity";
const GEMINI_2_5_PRO_PREFIX = "gemini-2.5-pro";
const GEMINI_2_5_FLASH_LITE_PREFIX = "gemini-2.5-flash-lite";
const GEMINI_2_5_FLASH_PREFIX = "gemini-2.5-flash";
const GEMINI_3_1_PRO_PREFIX = "gemini-3.1-pro";
const GEMINI_3_1_FLASH_LITE_PREFIX = "gemini-3.1-flash-lite";
const GEMINI_3_1_FLASH_PREFIX = "gemini-3.1-flash";
const GEMINI_3_FLASH_LITE_PREFIX = "gemini-3-flash-lite";
const GEMINI_3_FLASH_PREFIX = "gemini-3-flash";
const GEMINI_3_5_FLASH_PREFIX = "gemini-3.5-flash";
const GEMINI_PRO_LATEST_ID = "gemini-pro-latest";
const GEMINI_FLASH_LATEST_ID = "gemini-flash-latest";
const GEMINI_FLASH_LITE_LATEST_ID = "gemini-flash-lite-latest";
const GEMMA_PREFIX = "gemma-";
const GEMINI_2_5_PRO_TEMPLATE_IDS = ["gemini-2.5-pro"] as const;
const GEMINI_2_5_FLASH_LITE_TEMPLATE_IDS = ["gemini-2.5-flash-lite"] as const;
const GEMINI_2_5_FLASH_TEMPLATE_IDS = ["gemini-2.5-flash"] as const;
const GEMINI_3_1_PRO_TEMPLATE_IDS = ["gemini-3.1-pro-preview", "gemini-3-pro-preview"] as const;
const GEMINI_3_1_FLASH_LITE_TEMPLATE_IDS = ["gemini-3.1-flash-lite"] as const;
const GEMINI_3_1_FLASH_TEMPLATE_IDS = ["gemini-3-flash-preview", "gemini-2.5-flash"] as const;
const GEMINI_3_PRO_ANTIGRAVITY_TEMPLATE_IDS = ["gemini-3-pro-low", "gemini-3-pro-high"] as const;
const GEMINI_3_FLASH_ANTIGRAVITY_TEMPLATE_IDS = ["gemini-3-flash"] as const;
// Gemma uses the Gemini flash template as a forward-compat approximation
// until a dedicated Gemma template is registered in the catalog.
const GEMMA_TEMPLATE_IDS = GEMINI_3_1_FLASH_TEMPLATE_IDS;
const GOOGLE_PROVIDER_PREFIX = "google/";
function normalizeGeminiProRequestId(id: string): string {
if (id.startsWith(GOOGLE_PROVIDER_PREFIX)) {
const modelId = id.slice(GOOGLE_PROVIDER_PREFIX.length);
const normalizedModelId = normalizeGeminiProRequestId(modelId);
return normalizedModelId === modelId ? id : `${GOOGLE_PROVIDER_PREFIX}${normalizedModelId}`;
}
if (id === "gemini-3-pro" || id === "gemini-3-pro-preview" || id === "gemini-3.1-pro") {
return "gemini-3.1-pro-preview";
}
if (id === "gemma-4-26b") {
return normalizeGoogleModelId(id);
}
return id;
}
function googleFamilyModelId(id: string): string {
return id.startsWith(GOOGLE_PROVIDER_PREFIX) ? id.slice(GOOGLE_PROVIDER_PREFIX.length) : id;
}
type GoogleForwardCompatFamily = {
googleTemplateIds: readonly string[];
cliTemplateIds: readonly string[];
antigravityTemplateIds?: readonly string[];
preferExternalFirstForCli?: boolean;
};
type GoogleTemplateSource = {
templateProviderId: string;
templateIds: readonly string[];
};
function cloneGoogleTemplateModel(params: {
providerId: string;
modelId: string;
templateProviderId: string;
templateIds: readonly string[];
ctx: ProviderResolveDynamicModelContext;
patch?: Partial<ProviderRuntimeModel>;
}): ProviderRuntimeModel | undefined {
return cloneFirstTemplateModel({
providerId: params.templateProviderId,
modelId: params.modelId,
templateIds: params.templateIds,
ctx: params.ctx,
patch: {
...params.patch,
provider: params.providerId,
},
});
}
function isGoogleGeminiCliProvider(providerId: string): boolean {
return normalizeOptionalLowercaseString(providerId) === GOOGLE_GEMINI_CLI_PROVIDER_ID;
}
function isGoogleAntigravityProvider(providerId: string): boolean {
return normalizeOptionalLowercaseString(providerId) === GOOGLE_ANTIGRAVITY_PROVIDER_ID;
}
function templateIdsForProvider(
templateProviderId: string,
family: GoogleForwardCompatFamily,
): readonly string[] {
if (isGoogleGeminiCliProvider(templateProviderId)) {
return family.cliTemplateIds;
}
if (isGoogleAntigravityProvider(templateProviderId)) {
return family.antigravityTemplateIds ?? family.googleTemplateIds;
}
return family.googleTemplateIds;
}
function buildGoogleTemplateSources(params: {
providerId: string;
templateProviderId?: string;
family: GoogleForwardCompatFamily;
}): GoogleTemplateSource[] {
const defaultTemplateProviderId = params.templateProviderId?.trim()
? params.templateProviderId
: isGoogleGeminiCliProvider(params.providerId)
? "google"
: GOOGLE_GEMINI_CLI_PROVIDER_ID;
const preferredExternalFirst =
isGoogleGeminiCliProvider(params.providerId) &&
params.family.preferExternalFirstForCli === true;
const orderedTemplateProviderIds = preferredExternalFirst
? [defaultTemplateProviderId, params.providerId]
: [params.providerId, defaultTemplateProviderId];
const seen = new Set<string>();
const sources: GoogleTemplateSource[] = [];
for (const providerId of orderedTemplateProviderIds) {
const trimmed = providerId?.trim();
if (!trimmed || seen.has(trimmed)) {
continue;
}
seen.add(trimmed);
sources.push({
templateProviderId: trimmed,
templateIds: templateIdsForProvider(trimmed, params.family),
});
}
return sources;
}
export function resolveGoogleGeminiForwardCompatModel(params: {
providerId: string;
templateProviderId?: string;
ctx: ProviderResolveDynamicModelContext;
}): ProviderRuntimeModel | undefined {
const trimmed = normalizeGeminiProRequestId(params.ctx.modelId.trim());
const lower = normalizeOptionalLowercaseString(googleFamilyModelId(trimmed)) ?? "";
let family: GoogleForwardCompatFamily;
let patch: Partial<ProviderRuntimeModel> | undefined;
if (lower.startsWith(GEMINI_2_5_PRO_PREFIX)) {
family = {
googleTemplateIds: GEMINI_2_5_PRO_TEMPLATE_IDS,
cliTemplateIds: GEMINI_3_1_PRO_TEMPLATE_IDS,
preferExternalFirstForCli: true,
};
} else if (lower.startsWith(GEMINI_2_5_FLASH_LITE_PREFIX)) {
family = {
googleTemplateIds: GEMINI_2_5_FLASH_LITE_TEMPLATE_IDS,
cliTemplateIds: GEMINI_3_1_FLASH_LITE_TEMPLATE_IDS,
preferExternalFirstForCli: true,
};
} else if (lower.startsWith(GEMINI_2_5_FLASH_PREFIX)) {
family = {
googleTemplateIds: GEMINI_2_5_FLASH_TEMPLATE_IDS,
cliTemplateIds: GEMINI_3_1_FLASH_TEMPLATE_IDS,
preferExternalFirstForCli: true,
};
} else if (lower.startsWith(GEMINI_3_1_PRO_PREFIX) || lower === GEMINI_PRO_LATEST_ID) {
family = {
googleTemplateIds: GEMINI_3_1_PRO_TEMPLATE_IDS,
cliTemplateIds: GEMINI_3_1_PRO_TEMPLATE_IDS,
antigravityTemplateIds: GEMINI_3_PRO_ANTIGRAVITY_TEMPLATE_IDS,
};
if (params.providerId === "google" || params.providerId === GOOGLE_GEMINI_CLI_PROVIDER_ID) {
patch = { reasoning: true };
}
} else if (
lower.startsWith(GEMINI_3_1_FLASH_LITE_PREFIX) ||
lower.startsWith(GEMINI_3_FLASH_LITE_PREFIX) ||
lower === GEMINI_FLASH_LITE_LATEST_ID
) {
family = {
googleTemplateIds: GEMINI_3_1_FLASH_LITE_TEMPLATE_IDS,
cliTemplateIds: GEMINI_3_1_FLASH_LITE_TEMPLATE_IDS,
antigravityTemplateIds: GEMINI_3_FLASH_ANTIGRAVITY_TEMPLATE_IDS,
};
} else if (
lower.startsWith(GEMINI_3_1_FLASH_PREFIX) ||
lower.startsWith(GEMINI_3_5_FLASH_PREFIX) ||
lower.startsWith(GEMINI_3_FLASH_PREFIX) ||
lower === GEMINI_FLASH_LATEST_ID
) {
family = {
googleTemplateIds: GEMINI_3_1_FLASH_TEMPLATE_IDS,
cliTemplateIds: GEMINI_3_1_FLASH_TEMPLATE_IDS,
antigravityTemplateIds: GEMINI_3_FLASH_ANTIGRAVITY_TEMPLATE_IDS,
};
} else if (lower.startsWith(GEMMA_PREFIX)) {
family = {
googleTemplateIds: GEMMA_TEMPLATE_IDS,
cliTemplateIds: GEMMA_TEMPLATE_IDS,
};
if (lower.startsWith("gemma-4")) {
patch = { reasoning: true };
}
} else {
return undefined;
}
for (const source of buildGoogleTemplateSources({
providerId: params.providerId,
templateProviderId: params.templateProviderId,
family,
})) {
const model = cloneGoogleTemplateModel({
providerId: params.providerId,
modelId: trimmed,
templateProviderId: source.templateProviderId,
templateIds: source.templateIds,
ctx: params.ctx,
patch,
});
if (model) {
return model;
}
}
return undefined;
}
export function isModernGoogleModel(modelId: string): boolean {
const lower = normalizeOptionalLowercaseString(modelId) ?? "";
return (
lower.startsWith("gemini-2.5") ||
lower.startsWith("gemini-3") ||
lower === GEMINI_PRO_LATEST_ID ||
lower === GEMINI_FLASH_LATEST_ID ||
lower === GEMINI_FLASH_LITE_LATEST_ID ||
lower.startsWith(GEMMA_PREFIX)
);
}

View File

@@ -0,0 +1,202 @@
// Google tests cover provider policy api plugin behavior.
import { describe, expect, it } from "vitest";
import { normalizeConfig, resolveThinkingProfile } from "./provider-policy-api.js";
describe("google provider policy public artifact", () => {
it("normalizes Google provider config without loading the full provider plugin", () => {
expect(
normalizeConfig({
provider: "google",
providerConfig: {
baseUrl: "https://generativelanguage.googleapis.com",
api: "google-generative-ai",
apiKey: "GEMINI_API_KEY",
models: [
{
id: "gemini-3-pro",
name: "Gemini 3 Pro",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
},
],
},
}),
).toEqual({
api: "google-generative-ai",
apiKey: "GEMINI_API_KEY",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
models: [
{
id: "gemini-3.1-pro-preview",
name: "Gemini 3 Pro",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
},
],
});
});
it("preserves explicit OpenAI-compatible Google endpoints during normalization", () => {
expect(
normalizeConfig({
provider: "google",
providerConfig: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
api: "openai-completions",
models: [],
},
}),
).toEqual({
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
api: "openai-completions",
models: [],
});
});
it("normalizes retired Google model ids even for explicit OpenAI-compatible endpoints", () => {
expect(
normalizeConfig({
provider: "google",
providerConfig: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
api: "openai-completions",
models: [
{
id: "google/gemini-3-pro-preview",
name: "Gemini 3 Pro",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
},
],
},
}),
).toEqual({
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
api: "openai-completions",
models: [
{
id: "google/gemini-3.1-pro-preview",
name: "Gemini 3 Pro",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
},
],
});
});
it("normalizes retired Gemini CLI config model ids before emission", () => {
expect(
normalizeConfig({
provider: "google-gemini-cli",
providerConfig: {
baseUrl: "openclaw://google-gemini-cli",
models: [
{
id: "google/gemini-3-pro-preview",
name: "Gemini CLI 3 Pro",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
},
],
},
}),
).toEqual({
baseUrl: "openclaw://google-gemini-cli",
models: [
{
id: "google/gemini-3.1-pro-preview",
name: "Gemini CLI 3 Pro",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
},
],
});
});
it("preserves Gemini 3 thinking levels when catalog reasoning metadata is stale", () => {
expect(
resolveThinkingProfile({
provider: "google",
modelId: "gemini-3-flash-preview",
reasoning: false,
}),
).toEqual({
levels: [
{ id: "off" },
{ id: "minimal" },
{ id: "low" },
{ id: "medium" },
{ id: "adaptive" },
{ id: "high" },
],
preserveWhenCatalogReasoningFalse: true,
});
});
it("preserves provider-prefixed Gemini 3 thinking levels when catalog reasoning metadata is stale", () => {
expect(
resolveThinkingProfile({
provider: "google",
modelId: "google/gemini-3-flash-preview",
reasoning: false,
}),
).toMatchObject({
levels: expect.arrayContaining([{ id: "low" }, { id: "medium" }, { id: "adaptive" }]),
preserveWhenCatalogReasoningFalse: true,
});
});
it("preserves normalized Gemini 3 aliases when catalog reasoning metadata is stale", () => {
expect(
resolveThinkingProfile({
provider: "google",
modelId: "google/gemini-3-pro",
reasoning: false,
}),
).toEqual({
levels: [{ id: "off" }, { id: "low" }, { id: "adaptive" }, { id: "high" }],
preserveWhenCatalogReasoningFalse: true,
});
});
it("preserves Gemini 3 Pro thinking levels when catalog reasoning metadata is stale", () => {
expect(
resolveThinkingProfile({
provider: "google",
modelId: "gemini-3.1-pro-preview",
reasoning: false,
}),
).toEqual({
levels: [{ id: "off" }, { id: "low" }, { id: "adaptive" }, { id: "high" }],
preserveWhenCatalogReasoningFalse: true,
});
});
it("honors catalog reasoning=false for non-Gemini 3 Google models", () => {
expect(
resolveThinkingProfile({
provider: "google",
modelId: "gemma-4-26b-a4b-it",
reasoning: false,
}),
).toBeUndefined();
});
});

View File

@@ -0,0 +1,12 @@
// Google API module exposes the plugin public contract.
import type { ProviderDefaultThinkingPolicyContext } from "openclaw/plugin-sdk/core";
import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-types";
import { normalizeGoogleProviderConfig, resolveGoogleThinkingProfile } from "./provider-policy.js";
export function normalizeConfig(params: { provider: string; providerConfig: ModelProviderConfig }) {
return normalizeGoogleProviderConfig(params.provider, params.providerConfig);
}
export function resolveThinkingProfile(context: ProviderDefaultThinkingPolicyContext) {
return resolveGoogleThinkingProfile(context);
}

View File

@@ -0,0 +1,245 @@
// Google provider module implements model/runtime integration.
import type {
ProviderDefaultThinkingPolicyContext,
ProviderThinkingProfile,
} from "openclaw/plugin-sdk/core";
import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-types";
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
import { normalizeAntigravityModelId, normalizeGoogleModelId } from "./model-id.js";
import { isGoogleGemini3ProModel, isGoogleGemini3ThinkingLevelModel } from "./thinking-api.js";
type GoogleApiCarrier = {
api?: string | null;
};
type GoogleProviderConfigLike = GoogleApiCarrier & {
baseUrl?: string | null;
models?: ReadonlyArray<GoogleApiCarrier | null | undefined> | null;
};
export const DEFAULT_GOOGLE_API_BASE_URL = "https://generativelanguage.googleapis.com/v1beta";
const GOOGLE_MODEL_ID_PROVIDERS = new Set(["google", "google-gemini-cli", "google-vertex"]);
function trimTrailingSlashes(value: string): string {
return value.replace(/\/+$/, "");
}
function isCanonicalGoogleApiOriginShorthand(value: string): boolean {
return /^https:\/\/generativelanguage\.googleapis\.com\/?$/i.test(value);
}
function isGoogleGenerativeAiUrl(url: URL): boolean {
return (
url.protocol === "https:" && url.hostname.toLowerCase() === "generativelanguage.googleapis.com"
);
}
function stripUrlUserInfo(url: URL): void {
url.username = "";
url.password = "";
}
const GOOGLE_VERTEX_HOST = "aiplatform.googleapis.com";
const GOOGLE_VERTEX_REGION_HOST_SUFFIX = "-aiplatform.googleapis.com";
const GOOGLE_VERTEX_MULTI_REGION_HOSTS = new Set([
"aiplatform.eu.rep.googleapis.com",
"aiplatform.us.rep.googleapis.com",
]);
export function isGoogleVertexHostname(hostname: string): boolean {
const normalized = hostname.toLowerCase();
return (
normalized === GOOGLE_VERTEX_HOST ||
normalized.endsWith(GOOGLE_VERTEX_REGION_HOST_SUFFIX) ||
GOOGLE_VERTEX_MULTI_REGION_HOSTS.has(normalized)
);
}
export function isGoogleVertexBaseUrl(baseUrl?: string | null): boolean {
const raw = normalizeOptionalString(baseUrl);
if (!raw) {
return false;
}
try {
return isGoogleVertexHostname(new URL(raw).hostname);
} catch {
return false;
}
}
export function normalizeGoogleApiBaseUrl(baseUrl?: string): string {
const raw = trimTrailingSlashes(normalizeOptionalString(baseUrl) || DEFAULT_GOOGLE_API_BASE_URL);
try {
const url = new URL(raw);
url.hash = "";
url.search = "";
stripUrlUserInfo(url);
if (isGoogleGenerativeAiUrl(url)) {
const normalizedPath = trimTrailingSlashes(url.pathname || "");
url.pathname = normalizedPath || "/v1beta";
}
return trimTrailingSlashes(url.toString());
} catch {
if (isCanonicalGoogleApiOriginShorthand(raw)) {
return DEFAULT_GOOGLE_API_BASE_URL;
}
return raw;
}
}
export function isGoogleGenerativeAiApi(api?: string | null): boolean {
return api === "google-generative-ai";
}
export function normalizeGoogleGenerativeAiBaseUrl(baseUrl?: string): string | undefined {
if (!baseUrl) {
return baseUrl;
}
const normalized = normalizeGoogleApiBaseUrl(baseUrl);
try {
const url = new URL(normalized);
stripUrlUserInfo(url);
if (isGoogleGenerativeAiUrl(url)) {
url.pathname = trimTrailingSlashes(url.pathname || "").replace(/\/openai$/i, "") || "/v1beta";
return trimTrailingSlashes(url.toString());
}
} catch {
// `normalizeGoogleApiBaseUrl` already returned the best-effort input form.
}
return normalized;
}
export function resolveGoogleGenerativeAiTransport<TApi extends string | null | undefined>(params: {
provider?: string;
api: TApi;
baseUrl?: string;
}): { api: TApi | "google-generative-ai" | "google-vertex"; baseUrl?: string } {
const api =
params.api ??
(params.provider === "google-vertex" && isGoogleVertexBaseUrl(params.baseUrl)
? "google-vertex"
: undefined) ??
(params.provider === "google" && params.baseUrl ? "google-generative-ai" : params.api);
return {
api,
baseUrl: isGoogleGenerativeAiApi(api)
? normalizeGoogleGenerativeAiBaseUrl(params.baseUrl)
: params.baseUrl,
};
}
export function resolveGoogleGenerativeAiApiOrigin(baseUrl?: string): string {
return (
normalizeGoogleGenerativeAiBaseUrl(baseUrl) ?? normalizeGoogleApiBaseUrl(baseUrl)
).replace(/\/v1beta$/i, "");
}
export function shouldNormalizeGoogleGenerativeAiProviderConfig(
providerKey: string,
provider: GoogleProviderConfigLike,
): boolean {
if (providerKey === "google-vertex" && isGoogleVertexBaseUrl(provider.baseUrl)) {
return false;
}
if (isGoogleGenerativeAiApi(provider.api)) {
return true;
}
const hasGoogleGenerativeAiModelApi =
provider.models?.some((model) => isGoogleGenerativeAiApi(model?.api)) ?? false;
if (hasGoogleGenerativeAiModelApi) {
return true;
}
if (providerKey !== "google" && providerKey !== "google-vertex") {
return false;
}
const hasExplicitNonGoogleApi = normalizeOptionalString(provider.api) !== undefined;
return !hasExplicitNonGoogleApi;
}
export function shouldNormalizeGoogleProviderConfig(
providerKey: string,
provider: GoogleProviderConfigLike,
): boolean {
return (
providerKey === "google-antigravity" ||
shouldNormalizeGoogleGenerativeAiProviderConfig(providerKey, provider)
);
}
function normalizeProviderModels(
provider: ModelProviderConfig,
normalizeId: (id: string) => string,
): ModelProviderConfig {
const models = provider.models;
if (!Array.isArray(models) || models.length === 0) {
return provider;
}
let mutated = false;
const nextModels = models.map((model) => {
const nextId = normalizeId(model.id);
if (nextId === model.id) {
return model;
}
mutated = true;
return Object.assign({}, model, { id: nextId });
});
return mutated ? { ...provider, models: nextModels } : provider;
}
export function normalizeGoogleProviderConfig(
providerKey: string,
provider: ModelProviderConfig,
): ModelProviderConfig {
let nextProvider = provider;
const shouldNormalizeModelIds = GOOGLE_MODEL_ID_PROVIDERS.has(providerKey);
if (shouldNormalizeModelIds) {
const modelNormalized = normalizeProviderModels(nextProvider, normalizeGoogleModelId);
if (shouldNormalizeGoogleGenerativeAiProviderConfig(providerKey, modelNormalized)) {
const normalizedBaseUrl = normalizeGoogleGenerativeAiBaseUrl(modelNormalized.baseUrl);
nextProvider =
normalizedBaseUrl !== modelNormalized.baseUrl
? { ...modelNormalized, baseUrl: normalizedBaseUrl ?? modelNormalized.baseUrl }
: modelNormalized;
} else {
nextProvider = modelNormalized;
}
}
if (providerKey === "google-antigravity") {
nextProvider = normalizeProviderModels(nextProvider, normalizeAntigravityModelId);
}
return nextProvider;
}
export function resolveGoogleThinkingProfile({
modelId,
reasoning,
}: ProviderDefaultThinkingPolicyContext): ProviderThinkingProfile | undefined {
const normalizedModelId = normalizeGoogleModelId(modelId);
const isGemini3ThinkingModel = isGoogleGemini3ThinkingLevelModel(normalizedModelId);
if (reasoning === false && !isGemini3ThinkingModel) {
return undefined;
}
const levels: ProviderThinkingProfile["levels"] = isGoogleGemini3ProModel(normalizedModelId)
? [{ id: "off" }, { id: "low" }, { id: "adaptive" }, { id: "high" }]
: [
{ id: "off" },
{ id: "minimal" },
{ id: "low" },
{ id: "medium" },
{ id: "adaptive" },
{ id: "high" },
];
return {
levels,
...(isGemini3ThinkingModel ? { preserveWhenCatalogReasoningFalse: true } : {}),
};
}

View File

@@ -0,0 +1,68 @@
// Google tests cover provider registration plugin behavior.
import type { Model } from "openclaw/plugin-sdk/llm";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { buildGoogleProvider } from "./provider-registration.js";
const streamFns = vi.hoisted(() => ({
createGenerativeAi: vi.fn(() => vi.fn()),
createVertex: vi.fn(() => vi.fn()),
}));
vi.mock("./transport-stream.js", () => ({
createGoogleGenerativeAiTransportStreamFn: streamFns.createGenerativeAi,
createGoogleVertexTransportStreamFn: streamFns.createVertex,
}));
function model(overrides: Partial<Model> = {}): Model {
return {
id: "gemini-2.5-flash",
name: "Gemini 2.5 Flash",
provider: "google-vertex",
api: "google-generative-ai",
baseUrl: "https://aiplatform.googleapis.com",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_048_576,
maxTokens: 65_536,
...overrides,
} as Model;
}
describe("buildGoogleProvider createStreamFn", () => {
beforeEach(() => {
streamFns.createGenerativeAi.mockClear();
streamFns.createVertex.mockClear();
});
it("routes native Vertex hosts through the Vertex transport", () => {
const provider = buildGoogleProvider();
provider.createStreamFn?.({
provider: "google-vertex",
modelId: "gemini-2.5-flash",
model: model(),
} as never);
expect(streamFns.createVertex).toHaveBeenCalledTimes(1);
expect(streamFns.createGenerativeAi).not.toHaveBeenCalled();
});
it("preserves explicit OpenAI-compatible Vertex endpoint configs", () => {
const provider = buildGoogleProvider();
const result = provider.createStreamFn?.({
provider: "google-vertex",
modelId: "gemini-2.5-flash",
model: model({
api: "openai-completions",
baseUrl:
"https://aiplatform.googleapis.com/v1/projects/test/locations/us-central1/endpoints/openapi",
}),
} as never);
expect(result).toBeUndefined();
expect(streamFns.createVertex).not.toHaveBeenCalled();
expect(streamFns.createGenerativeAi).not.toHaveBeenCalled();
});
});

View File

@@ -0,0 +1,114 @@
// Google provider module implements model/runtime integration.
import type {
OpenClawPluginApi,
ProviderReasoningOutputModeContext,
} from "openclaw/plugin-sdk/plugin-entry";
import { createProviderApiKeyAuthMethod } from "openclaw/plugin-sdk/provider-auth-api-key";
import type { ProviderPlugin } from "openclaw/plugin-sdk/provider-model-shared";
import { normalizeGoogleModelId } from "./model-id.js";
import { GOOGLE_GEMINI_DEFAULT_MODEL, applyGoogleGeminiModelDefault } from "./onboard.js";
import {
buildGoogleStaticCatalogProvider,
buildGoogleVertexStaticCatalogProvider,
} from "./provider-catalog.js";
import { GOOGLE_GEMINI_PROVIDER_HOOKS } from "./provider-hooks.js";
import { isModernGoogleModel, resolveGoogleGeminiForwardCompatModel } from "./provider-models.js";
import {
isGoogleVertexBaseUrl,
normalizeGoogleProviderConfig,
resolveGoogleGenerativeAiTransport,
} from "./provider-policy.js";
import {
createGoogleGenerativeAiTransportStreamFn,
createGoogleVertexTransportStreamFn,
} from "./transport-stream.js";
import { resolveGoogleVertexConfigApiKey } from "./vertex-adc.js";
function resolveGoogleReasoningOutputMode(
ctx: ProviderReasoningOutputModeContext,
): "native" | "tagged" {
if (ctx.provider === "google" || ctx.provider === "google-vertex") {
const api = ctx.model?.api ?? ctx.modelApi;
if (!api || api === "google-generative-ai" || api === "google-vertex") {
return "native";
}
}
return "tagged";
}
export function buildGoogleProvider(): ProviderPlugin {
return {
id: "google",
label: "Google AI Studio",
docsPath: "/providers/models",
hookAliases: ["google-antigravity", "google-vertex"],
envVars: ["GEMINI_API_KEY", "GOOGLE_API_KEY"],
auth: [
createProviderApiKeyAuthMethod({
providerId: "google",
methodId: "api-key",
label: "Google Gemini API key",
hint: "AI Studio / Gemini API key",
optionKey: "geminiApiKey",
flagName: "--gemini-api-key",
envVar: "GEMINI_API_KEY",
promptMessage: "Enter Gemini API key",
defaultModel: GOOGLE_GEMINI_DEFAULT_MODEL,
expectedProviders: ["google"],
applyConfig: (cfg) => applyGoogleGeminiModelDefault(cfg).next,
wizard: {
choiceId: "gemini-api-key",
choiceLabel: "Google Gemini API key",
groupId: "google",
groupLabel: "Google",
groupHint: "Gemini API key + OAuth",
},
}),
],
normalizeTransport: ({ provider, api, baseUrl }) =>
resolveGoogleGenerativeAiTransport({ provider, api, baseUrl }),
normalizeConfig: ({ provider, providerConfig }) =>
normalizeGoogleProviderConfig(provider, providerConfig),
resolveConfigApiKey: ({ provider, env }) =>
provider === "google-vertex" ? resolveGoogleVertexConfigApiKey(env) : undefined,
staticCatalog: {
order: "simple",
run: async () => ({
providers: {
google: buildGoogleStaticCatalogProvider(),
"google-vertex": buildGoogleVertexStaticCatalogProvider(),
},
}),
},
normalizeModelId: ({ modelId }) => normalizeGoogleModelId(modelId),
resolveDynamicModel: (ctx) =>
resolveGoogleGeminiForwardCompatModel({
providerId: ctx.provider,
ctx,
}),
createStreamFn: ({ model }) => {
if (
model.api === "google-vertex" ||
(model.api === "google-generative-ai" &&
(model.provider === "google-vertex" || isGoogleVertexBaseUrl(model.baseUrl)))
) {
return createGoogleVertexTransportStreamFn();
}
if (model.api === "google-generative-ai") {
return createGoogleGenerativeAiTransportStreamFn();
}
return undefined;
},
...GOOGLE_GEMINI_PROVIDER_HOOKS,
// Gemini 2.5+ delivers reasoning via native thinkingParts (thinkingConfig.includeThoughts).
// Tagged mode simultaneously injects <think>/<final> which the model opens before a tool
// call, never closes, leaving the post-tool turn empty (payloads=0). The CLI backend keeps
// tagged mode because it emits JSON text, not native thought parts.
resolveReasoningOutputMode: resolveGoogleReasoningOutputMode,
isModernModelRef: ({ modelId }) => isModernGoogleModel(modelId),
};
}
export function registerGoogleProvider(api: OpenClawPluginApi) {
api.registerProvider(buildGoogleProvider());
}

View File

@@ -0,0 +1,4 @@
// Google tests cover provider runtime.contract plugin behavior.
import { describeGoogleProviderRuntimeContract } from "openclaw/plugin-sdk/provider-test-contracts";
describeGoogleProviderRuntimeContract(() => import("./index.js"));

View File

@@ -0,0 +1,940 @@
// Google tests cover realtime voice provider plugin behavior.
import { REALTIME_VOICE_AUDIO_FORMAT_PCM16_24KHZ } from "openclaw/plugin-sdk/realtime-voice";
import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { buildGoogleRealtimeVoiceProvider } from "./realtime-voice-provider.js";
type MockGoogleLiveSession = {
close: ReturnType<typeof vi.fn>;
sendClientContent: ReturnType<typeof vi.fn>;
sendRealtimeInput: ReturnType<typeof vi.fn>;
sendToolResponse: ReturnType<typeof vi.fn>;
};
type MockGoogleLiveConnectParams = {
model: string;
config: Record<string, unknown>;
callbacks: {
onopen: () => void;
onmessage: (message: Record<string, unknown>) => void;
onerror: (event: { error?: unknown; message?: string }) => void;
onclose: (event?: { code?: number; reason?: string; wasClean?: boolean }) => void;
};
};
const { connectMock, createTokenMock, session } = vi.hoisted(() => {
const sessionValue: MockGoogleLiveSession = {
close: vi.fn(),
sendClientContent: vi.fn(),
sendRealtimeInput: vi.fn(),
sendToolResponse: vi.fn(),
};
const connectMockLocal = vi.fn(async (_params: MockGoogleLiveConnectParams) => sessionValue);
const createTokenMockLocal = vi.fn(async (_params: unknown) => ({
name: "auth_tokens/browser-session",
}));
return {
connectMock: connectMockLocal,
createTokenMock: createTokenMockLocal,
session: sessionValue,
};
});
vi.mock("./google-genai-runtime.js", () => ({
createGoogleGenAI: vi.fn(() => ({
authTokens: {
create: createTokenMock,
},
live: {
connect: connectMock,
},
})),
}));
const ENV_KEYS = ["GEMINI_API_KEY", "GOOGLE_API_KEY"] as const;
let envSnapshot: Partial<Record<(typeof ENV_KEYS)[number], string>>;
function lastConnectParams(): MockGoogleLiveConnectParams {
const params = connectMock.mock.calls.at(-1)?.[0];
if (!params) {
throw new Error("expected google live connect call");
}
return params;
}
function sentAudio(index = 0): { data?: unknown; mimeType?: unknown } {
const audio = session.sendRealtimeInput.mock.calls[index]?.[0]?.audio;
if (!audio) {
throw new Error(`Expected sent audio at index ${index}`);
}
return audio as { data?: unknown; mimeType?: unknown };
}
function requireFirstMockArg(mock: ReturnType<typeof vi.fn>, label: string): unknown {
const [call] = mock.mock.calls;
if (!call) {
throw new Error(`expected ${label}`);
}
return call[0];
}
function requireFirstError(mock: ReturnType<typeof vi.fn>): { message?: string } {
const error = requireFirstMockArg(mock, "Google Live error");
if (!error || typeof error !== "object" || Array.isArray(error)) {
throw new Error("expected Google Live error");
}
return error as { message?: string };
}
function requireFirstAudio(mock: ReturnType<typeof vi.fn>): unknown {
return requireFirstMockArg(mock, "Google Live audio");
}
describe("buildGoogleRealtimeVoiceProvider", () => {
beforeEach(() => {
envSnapshot = Object.fromEntries(ENV_KEYS.map((key) => [key, process.env[key]]));
connectMock.mockClear();
createTokenMock.mockClear();
session.close.mockClear();
session.sendClientContent.mockClear();
session.sendRealtimeInput.mockClear();
session.sendToolResponse.mockClear();
delete process.env.GEMINI_API_KEY;
delete process.env.GOOGLE_API_KEY;
});
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
for (const key of ENV_KEYS) {
const value = envSnapshot[key];
if (value === undefined) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
});
afterAll(() => {
vi.doUnmock("./google-genai-runtime.js");
vi.resetModules();
});
it("declares realtime Talk capabilities for catalog selection", () => {
const provider = buildGoogleRealtimeVoiceProvider();
expect(provider.capabilities).toEqual({
transports: ["provider-websocket", "gateway-relay"],
inputAudioFormats: [
{ encoding: "g711_ulaw", sampleRateHz: 8000, channels: 1 },
{ encoding: "pcm16", sampleRateHz: 24000, channels: 1 },
],
outputAudioFormats: [
{ encoding: "g711_ulaw", sampleRateHz: 8000, channels: 1 },
{ encoding: "pcm16", sampleRateHz: 24000, channels: 1 },
],
supportsBrowserSession: true,
supportsBargeIn: true,
supportsToolCalls: true,
supportsVideoFrames: true,
supportsSessionResumption: true,
});
});
it("normalizes provider config and cfg model-provider key fallback", () => {
const provider = buildGoogleRealtimeVoiceProvider();
const resolved = provider.resolveConfig?.({
cfg: {
models: {
providers: {
google: {
apiKey: "cfg-key",
},
},
},
} as never,
rawConfig: {
providers: {
google: {
model: "gemini-live-2.5-flash-preview",
voice: "Puck",
temperature: 0.4,
silenceDurationMs: 700,
startSensitivity: "high",
activityHandling: "no_interruption",
turnCoverage: "turn_includes_only_activity",
automaticActivityDetectionDisabled: false,
},
},
},
});
expect(resolved).toEqual({
apiKey: "cfg-key",
model: "gemini-live-2.5-flash-preview",
voice: "Puck",
temperature: 0.4,
apiVersion: undefined,
prefixPaddingMs: undefined,
silenceDurationMs: 700,
startSensitivity: "high",
endSensitivity: undefined,
activityHandling: "no-interruption",
turnCoverage: "only-activity",
automaticActivityDetectionDisabled: false,
enableAffectiveDialog: undefined,
sessionResumption: undefined,
contextWindowCompression: undefined,
thinkingLevel: undefined,
thinkingBudget: undefined,
});
});
it("connects with Google Live setup config and tool declarations", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
model: "gemini-live-2.5-flash-preview",
voice: "Kore",
temperature: 0.3,
startSensitivity: "low",
endSensitivity: "low",
activityHandling: "no-interruption",
turnCoverage: "only-activity",
},
instructions: "Speak briefly.",
tools: [
{
type: "function",
name: "lookup",
description: "Look something up",
parameters: {
type: "object",
properties: {
query: { type: "string" },
},
required: ["query"],
},
},
{
type: "function",
name: "openclaw_agent_consult",
description: "Ask OpenClaw",
parameters: {
type: "object",
properties: {
question: { type: "string" },
},
required: ["question"],
},
},
],
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
expect(connectMock).toHaveBeenCalledTimes(1);
const params = lastConnectParams();
expect(params.model).toBe("gemini-live-2.5-flash-preview");
const config = params.config as {
contextWindowCompression?: unknown;
outputAudioTranscription?: unknown;
realtimeInputConfig?: {
activityHandling?: string;
automaticActivityDetection?: {
endOfSpeechSensitivity?: string;
startOfSpeechSensitivity?: string;
};
turnCoverage?: string;
};
responseModalities?: string[];
sessionResumption?: unknown;
speechConfig?: { voiceConfig?: { prebuiltVoiceConfig?: { voiceName?: string } } };
systemInstruction?: string;
temperature?: number;
tools?: Array<{
functionDeclarations?: Array<{
behavior?: string;
description?: string;
name?: string;
parametersJsonSchema?: unknown;
}>;
}>;
};
expect(config.responseModalities).toEqual(["AUDIO"]);
expect(config.temperature).toBe(0.3);
expect(config.systemInstruction).toBe("Speak briefly.");
expect(config.speechConfig?.voiceConfig?.prebuiltVoiceConfig?.voiceName).toBe("Kore");
expect(config.outputAudioTranscription).toEqual({});
expect(config.realtimeInputConfig?.activityHandling).toBe("NO_INTERRUPTION");
expect(config.realtimeInputConfig?.automaticActivityDetection?.startOfSpeechSensitivity).toBe(
"START_SENSITIVITY_LOW",
);
expect(config.realtimeInputConfig?.automaticActivityDetection?.endOfSpeechSensitivity).toBe(
"END_SENSITIVITY_LOW",
);
expect(config.realtimeInputConfig?.turnCoverage).toBe("TURN_INCLUDES_ONLY_ACTIVITY");
expect(config.sessionResumption).toEqual({});
expect(config.contextWindowCompression).toEqual({ slidingWindow: {} });
const declarations = config.tools?.[0]?.functionDeclarations ?? [];
expect(declarations[0]?.name).toBe("lookup");
expect(declarations[0]?.description).toBe("Look something up");
expect(declarations[0]?.parametersJsonSchema).toEqual({
type: "object",
properties: {
query: { type: "string" },
},
required: ["query"],
});
expect(declarations[1]?.name).toBe("openclaw_agent_consult");
expect(declarations[1]?.description).toBe("Ask OpenClaw");
expect(declarations[1]?.parametersJsonSchema).toEqual({
type: "object",
properties: {
question: { type: "string" },
},
required: ["question"],
});
expect(declarations[1]?.behavior).toBe("NON_BLOCKING");
});
it("omits zero temperature for native audio responses", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
temperature: 0,
},
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
expect(lastConnectParams().config).not.toHaveProperty("temperature");
});
it("drops malformed VAD timing values before connecting", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
prefixPaddingMs: -1,
silenceDurationMs: 250.5,
},
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
expect(lastConnectParams().config).not.toHaveProperty("realtimeInputConfig");
});
it("drops malformed thinking budgets before connecting", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
thinkingBudget: 24_576.5,
},
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
expect(lastConnectParams().config).not.toHaveProperty("thinkingConfig");
});
it("passes Google Live dynamic thinking budget through", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
thinkingBudget: -1,
},
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
expect(lastConnectParams().config.thinkingConfig).toEqual({ thinkingBudget: -1 });
});
it("creates constrained browser sessions for Google Live Talk", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const sessionLocal = await provider.createBrowserSession?.({
providerConfig: {
apiKey: "gemini-key",
model: "gemini-live-2.5-flash-preview",
voice: "Puck",
temperature: 0.4,
},
instructions: "Speak briefly.",
tools: [
{
type: "function",
name: "openclaw_agent_consult",
description: "Ask OpenClaw",
parameters: {
type: "object",
properties: {
question: { type: "string" },
},
required: ["question"],
},
},
],
});
expect(createTokenMock).toHaveBeenCalledTimes(1);
const tokenConfig = requireFirstMockArg(createTokenMock, "Google Live auth token config") as {
config?: {
liveConnectConstraints?: {
config?: {
responseModalities?: string[];
speechConfig?: { voiceConfig?: { prebuiltVoiceConfig?: { voiceName?: string } } };
systemInstruction?: string;
temperature?: number;
tools?: Array<{ functionDeclarations?: Array<{ behavior?: string; name?: string }> }>;
};
model?: string;
};
uses?: number;
};
};
const liveConstraints = tokenConfig.config?.liveConnectConstraints;
expect(tokenConfig.config?.uses).toBe(1);
expect(liveConstraints?.model).toBe("gemini-live-2.5-flash-preview");
expect(liveConstraints?.config?.responseModalities).toEqual(["AUDIO"]);
expect(liveConstraints?.config?.temperature).toBe(0.4);
expect(liveConstraints?.config?.systemInstruction).toBe("Speak briefly.");
expect(liveConstraints?.config?.speechConfig?.voiceConfig?.prebuiltVoiceConfig?.voiceName).toBe(
"Puck",
);
expect(liveConstraints?.config?.tools?.[0]?.functionDeclarations?.[0]?.name).toBe(
"openclaw_agent_consult",
);
expect(liveConstraints?.config?.tools?.[0]?.functionDeclarations?.[0]?.behavior).toBe(
"NON_BLOCKING",
);
expect(sessionLocal?.provider).toBe("google");
expect(sessionLocal?.transport).toBe("provider-websocket");
const websocketSession = sessionLocal as {
audio: {
inputEncoding: string;
inputSampleRateHz: number;
outputEncoding: string;
outputSampleRateHz: number;
};
clientSecret: string;
initialMessage: {
setup: { generationConfig: { responseModalities: string[] }; model: string };
};
protocol: string;
websocketUrl: string;
};
expect(websocketSession.protocol).toBe("google-live-bidi");
expect(websocketSession.clientSecret).toBe("auth_tokens/browser-session");
expect(websocketSession.websocketUrl).toBe(
"wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1alpha.GenerativeService.BidiGenerateContentConstrained",
);
expect(websocketSession.audio.inputEncoding).toBe("pcm16");
expect(websocketSession.audio.inputSampleRateHz).toBe(16000);
expect(websocketSession.audio.outputEncoding).toBe("pcm16");
expect(websocketSession.audio.outputSampleRateHz).toBe(24000);
expect(websocketSession.initialMessage.setup.model).toBe(
"models/gemini-live-2.5-flash-preview",
);
expect(websocketSession.initialMessage.setup.generationConfig.responseModalities).toEqual([
"AUDIO",
]);
});
it("rejects browser session expiry outside Date range", async () => {
vi.spyOn(Date, "now").mockReturnValue(8_640_000_000_000_001);
const provider = buildGoogleRealtimeVoiceProvider();
await expect(
provider.createBrowserSession?.({
providerConfig: {
apiKey: "gemini-key",
},
}),
).rejects.toThrow("Google realtime browser session expiry is outside the supported Date range");
expect(createTokenMock).not.toHaveBeenCalled();
});
it("rejects browser session creation while the process clock is invalid", async () => {
vi.spyOn(Date, "now").mockReturnValue(Number.NaN);
const provider = buildGoogleRealtimeVoiceProvider();
await expect(
provider.createBrowserSession?.({
providerConfig: {
apiKey: "gemini-key",
},
}),
).rejects.toThrow("Google realtime browser session expiry is outside the supported Date range");
expect(createTokenMock).not.toHaveBeenCalled();
});
it("can opt out of Google Live session resumption and context compression", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
contextWindowCompression: false,
sessionResumption: false,
},
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
expect(lastConnectParams().config).not.toHaveProperty("contextWindowCompression");
expect(lastConnectParams().config).not.toHaveProperty("sessionResumption");
});
it("captures Google Live resumption handles and reuses them on reconnect", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
lastConnectParams().callbacks.onmessage({
sessionResumptionUpdate: { resumable: true, newHandle: "resume-1" },
});
await bridge.connect();
expect(lastConnectParams().config.sessionResumption).toEqual({ handle: "resume-1" });
});
it("reconnects unexpected Google Live closes with the latest resumption handle", async () => {
vi.useFakeTimers();
try {
const provider = buildGoogleRealtimeVoiceProvider();
const onClose = vi.fn();
const onError = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onClose,
onError,
});
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: { sessionId: "session-1" },
sessionResumptionUpdate: { resumable: true, newHandle: "resume-1" },
});
lastConnectParams().callbacks.onclose({
code: 1011,
reason: "temporary upstream close",
wasClean: false,
});
expect(onClose).not.toHaveBeenCalled();
const error = requireFirstError(onError);
expect(error.message).toContain("reconnecting 1/3");
await vi.advanceTimersByTimeAsync(250);
expect(connectMock).toHaveBeenCalledTimes(2);
expect(lastConnectParams().config.sessionResumption).toEqual({ handle: "resume-1" });
} finally {
vi.useRealTimers();
}
});
it("waits for setup completion before draining audio and firing ready", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onReady = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onReady,
});
await bridge.connect();
lastConnectParams().callbacks.onopen();
bridge.sendAudio(Buffer.from([0xff, 0xff]));
expect(session.sendRealtimeInput).not.toHaveBeenCalled();
expect(onReady).not.toHaveBeenCalled();
lastConnectParams().callbacks.onmessage({ setupComplete: { sessionId: "session-1" } });
expect(onReady).toHaveBeenCalledTimes(1);
expect(session.sendRealtimeInput).toHaveBeenCalledTimes(1);
const audio = sentAudio();
expect(typeof audio.data).toBe("string");
expect(audio.mimeType).toBe("audio/pcm;rate=16000");
});
it("marks the Google audio stream complete after sustained telephony silence", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key", silenceDurationMs: 60 },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
lastConnectParams().callbacks.onopen();
lastConnectParams().callbacks.onmessage({ setupComplete: { sessionId: "session-1" } });
const silence20ms = Buffer.alloc(160, 0xff);
bridge.sendAudio(silence20ms);
bridge.sendAudio(silence20ms);
bridge.sendAudio(silence20ms);
expect(session.sendRealtimeInput).toHaveBeenCalledWith({ audioStreamEnd: true });
const callsAfterStreamEnd = session.sendRealtimeInput.mock.calls.length;
bridge.sendAudio(silence20ms);
expect(session.sendRealtimeInput).toHaveBeenCalledTimes(callsAfterStreamEnd);
session.sendRealtimeInput.mockClear();
bridge.sendAudio(Buffer.alloc(160, 0x7f));
bridge.sendAudio(silence20ms);
bridge.sendAudio(silence20ms);
bridge.sendAudio(silence20ms);
expect(session.sendRealtimeInput).toHaveBeenCalledWith({ audioStreamEnd: true });
});
it("fuses telephony mu-law conversion into the Gemini 16 kHz PCM input frame", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
lastConnectParams().callbacks.onopen();
lastConnectParams().callbacks.onmessage({ setupComplete: { sessionId: "session-1" } });
bridge.sendAudio(Buffer.from([0xff, 0x00]));
const audio = sentAudio();
expect(typeof audio.data).toBe("string");
expect(audio.mimeType).toBe("audio/pcm;rate=16000");
const sent = Buffer.from(audio.data as string, "base64");
expect(Array.from({ length: sent.length / 2 }, (_, i) => sent.readInt16LE(i * 2))).toEqual([
0, -16062, -32124, -32124,
]);
});
it("accepts PCM16 24 kHz audio without the telephony mu-law hop", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
audioFormat: REALTIME_VOICE_AUDIO_FORMAT_PCM16_24KHZ,
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
lastConnectParams().callbacks.onopen();
lastConnectParams().callbacks.onmessage({ setupComplete: { sessionId: "session-1" } });
bridge.sendAudio(Buffer.alloc(480));
const audio = sentAudio();
expect(typeof audio.data).toBe("string");
expect(audio.mimeType).toBe("audio/pcm;rate=16000");
const sent = Buffer.from(audio.data as string, "base64");
expect(sent).toHaveLength(320);
});
it("can disable automatic VAD for manual activity signaling experiments", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: {
apiKey: "gemini-key",
automaticActivityDetectionDisabled: true,
},
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
const config = lastConnectParams().config as {
realtimeInputConfig?: { automaticActivityDetection?: { disabled?: boolean } };
};
expect(config.realtimeInputConfig?.automaticActivityDetection?.disabled).toBe(true);
});
it("sends text prompts as ordered client turns", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
});
await bridge.connect();
lastConnectParams().callbacks.onopen();
lastConnectParams().callbacks.onmessage({ setupComplete: { sessionId: "session-1" } });
bridge.sendUserMessage?.(" Say hello. ");
expect(session.sendClientContent).toHaveBeenCalledWith({
turns: [{ role: "user", parts: [{ text: "Say hello." }] }],
turnComplete: true,
});
});
it("converts Google PCM output to mu-law audio", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onAudio = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio,
onClearAudio: vi.fn(),
});
const pcm24k = Buffer.alloc(480);
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: { sessionId: "session-1" },
serverContent: {
modelTurn: {
parts: [
{
inlineData: {
mimeType: "audio/L16;codec=pcm;rate=24000",
data: pcm24k.toString("base64"),
},
},
],
},
},
});
expect(onAudio).toHaveBeenCalledTimes(1);
const audio = requireFirstAudio(onAudio);
expect(audio).toBeInstanceOf(Buffer);
expect(audio).toHaveLength(80);
});
it("can keep Google PCM output as PCM16 24 kHz audio", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onAudio = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
audioFormat: REALTIME_VOICE_AUDIO_FORMAT_PCM16_24KHZ,
onAudio,
onClearAudio: vi.fn(),
});
const pcm24k = Buffer.alloc(480);
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: { sessionId: "session-1" },
serverContent: {
modelTurn: {
parts: [
{
inlineData: {
mimeType: "audio/L16;codec=pcm;rate=24000",
data: pcm24k.toString("base64"),
},
},
],
},
},
});
expect(onAudio).toHaveBeenCalledTimes(1);
expect(requireFirstAudio(onAudio)).toEqual(pcm24k);
});
it("does not forward Google thought text as assistant transcript", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onTranscript = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onTranscript,
});
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: {},
serverContent: {
modelTurn: {
parts: [{ text: "internal reasoning", thought: true }],
},
},
});
expect(onTranscript).not.toHaveBeenCalled();
});
it("forwards Live API tool calls and submits matching function responses", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onToolCall = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onToolCall,
});
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: { sessionId: "session-1" },
toolCall: {
functionCalls: [{ id: "call-1", name: "lookup", args: { query: "hi" } }],
},
});
expect(onToolCall).toHaveBeenCalledWith({
itemId: "call-1",
callId: "call-1",
name: "lookup",
args: { query: "hi" },
});
bridge.submitToolResult("call-1", { result: "ok" });
expect(session.sendToolResponse).toHaveBeenCalledWith({
functionResponses: [
{
id: "call-1",
name: "lookup",
response: { result: "ok" },
},
],
});
});
it("keeps Google Live consult calls open after continuing tool responses", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onToolCall: vi.fn(),
});
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: { sessionId: "session-1" },
toolCall: {
functionCalls: [
{ id: "consult-call", name: "openclaw_agent_consult", args: { prompt: "hi" } },
],
},
});
bridge.submitToolResult(
"consult-call",
{ status: "working", message: "Tell the participant you are checking." },
{ willContinue: true },
);
bridge.submitToolResult("consult-call", { text: "The meeting starts at 3." });
expect(session.sendToolResponse).toHaveBeenNthCalledWith(1, {
functionResponses: [
{
id: "consult-call",
name: "openclaw_agent_consult",
scheduling: "WHEN_IDLE",
willContinue: true,
response: { status: "working", message: "Tell the participant you are checking." },
},
],
});
expect(session.sendToolResponse).toHaveBeenNthCalledWith(2, {
functionResponses: [
{
id: "consult-call",
name: "openclaw_agent_consult",
scheduling: "WHEN_IDLE",
response: { text: "The meeting starts at 3." },
},
],
});
});
it("does not send malformed Live API tool responses without a matching call name", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onError = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onError,
});
await bridge.connect();
bridge.submitToolResult("missing-call", { result: "ok" });
expect(session.sendToolResponse).not.toHaveBeenCalled();
const error = requireFirstError(onError);
expect(error.message).toBe(
"Google Live function response is missing a matching function call for missing-call",
);
});
it("reports Google Live tool response send failures without losing the call name", async () => {
const provider = buildGoogleRealtimeVoiceProvider();
const onError = vi.fn();
const bridge = provider.createBridge({
providerConfig: { apiKey: "gemini-key" },
onAudio: vi.fn(),
onClearAudio: vi.fn(),
onError,
});
await bridge.connect();
lastConnectParams().callbacks.onmessage({
setupComplete: { sessionId: "session-1" },
toolCall: {
functionCalls: [{ id: "call-1", name: "lookup", args: { query: "hi" } }],
},
});
const sendError = new Error("SDK send failed");
session.sendToolResponse.mockImplementationOnce(() => {
throw sendError;
});
bridge.submitToolResult("call-1", ["retryable"]);
expect(onError).toHaveBeenCalledWith(sendError);
bridge.submitToolResult("call-1", { result: "ok" });
expect(session.sendToolResponse).toHaveBeenLastCalledWith({
functionResponses: [
{
id: "call-1",
name: "lookup",
response: { result: "ok" },
},
],
});
});
});

View File

@@ -0,0 +1,972 @@
// Google provider module implements model/runtime integration.
import { randomUUID } from "node:crypto";
import type {
ActivityHandling,
Behavior,
EndSensitivity,
FunctionDeclaration,
FunctionResponse,
FunctionResponseScheduling,
LiveConnectConfig,
LiveServerContent,
LiveServerMessage,
LiveServerToolCall,
Modality,
RealtimeInputConfig,
StartSensitivity,
ThinkingConfig,
TurnCoverage,
} from "@google/genai";
import {
resolveExpiresAtMsFromDurationMs,
timestampMsToIsoString,
} from "openclaw/plugin-sdk/number-runtime";
import type { OpenClawConfig } from "openclaw/plugin-sdk/provider-onboard";
import type {
RealtimeVoiceAudioFormat,
RealtimeVoiceBridge,
RealtimeVoiceBrowserSession,
RealtimeVoiceBrowserSessionCreateRequest,
RealtimeVoiceBridgeCreateRequest,
RealtimeVoiceProviderConfig,
RealtimeVoiceProviderPlugin,
RealtimeVoiceTool,
RealtimeVoiceToolResultOptions,
} from "openclaw/plugin-sdk/realtime-voice";
import {
convertPcmToMulaw8k,
mulawToPcm,
REALTIME_VOICE_AUDIO_FORMAT_G711_ULAW_8KHZ,
REALTIME_VOICE_AUDIO_FORMAT_PCM16_24KHZ,
REALTIME_VOICE_AGENT_CONSULT_TOOL_NAME,
resamplePcm,
} from "openclaw/plugin-sdk/realtime-voice";
import { normalizeResolvedSecretInputString } from "openclaw/plugin-sdk/secret-input";
import {
asBoolean,
asFiniteNumber,
normalizeOptionalString,
} from "openclaw/plugin-sdk/string-coerce-runtime";
import { createGoogleGenAI } from "./google-genai-runtime.js";
const GOOGLE_REALTIME_DEFAULT_MODEL = "gemini-2.5-flash-native-audio-preview-12-2025";
const GOOGLE_REALTIME_DEFAULT_VOICE = "Kore";
const GOOGLE_REALTIME_DEFAULT_API_VERSION = "v1beta";
const GOOGLE_REALTIME_INPUT_SAMPLE_RATE = 16_000;
const GOOGLE_REALTIME_BROWSER_API_VERSION = "v1alpha";
const GOOGLE_REALTIME_BROWSER_WEBSOCKET_URL =
"wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1alpha.GenerativeService.BidiGenerateContentConstrained";
const MAX_PENDING_AUDIO_CHUNKS = 320;
const DEFAULT_AUDIO_STREAM_END_SILENCE_MS = 500;
const GOOGLE_REALTIME_BROWSER_SESSION_TTL_MS = 30 * 60 * 1000;
const GOOGLE_REALTIME_BROWSER_NEW_SESSION_TTL_MS = 60 * 1000;
const GOOGLE_REALTIME_RECONNECT_MAX_ATTEMPTS = 3;
const GOOGLE_REALTIME_RECONNECT_BASE_DELAY_MS = 250;
const GOOGLE_REALTIME_RECONNECT_MAX_DELAY_MS = 2_000;
const MULAW_LINEAR_SAMPLES = new Int16Array(256);
for (let i = 0; i < MULAW_LINEAR_SAMPLES.length; i += 1) {
MULAW_LINEAR_SAMPLES[i] = decodeMulawSample(i);
}
type GoogleRealtimeSensitivity = "low" | "high";
type GoogleRealtimeThinkingLevel = "minimal" | "low" | "medium" | "high";
type GoogleRealtimeActivityHandling = "start-of-activity-interrupts" | "no-interruption";
type GoogleRealtimeTurnCoverage = "only-activity" | "all-input" | "audio-activity-and-all-video";
type GoogleRealtimeVoiceProviderConfig = {
apiKey?: string;
model?: string;
voice?: string;
temperature?: number;
apiVersion?: string;
prefixPaddingMs?: number;
silenceDurationMs?: number;
startSensitivity?: GoogleRealtimeSensitivity;
endSensitivity?: GoogleRealtimeSensitivity;
activityHandling?: GoogleRealtimeActivityHandling;
turnCoverage?: GoogleRealtimeTurnCoverage;
automaticActivityDetectionDisabled?: boolean;
enableAffectiveDialog?: boolean;
sessionResumption?: boolean;
contextWindowCompression?: boolean;
thinkingLevel?: GoogleRealtimeThinkingLevel;
thinkingBudget?: number;
};
type GoogleRealtimeLiveConfig = {
apiKey: string;
instructions?: string;
tools?: RealtimeVoiceTool[];
model?: string;
voice?: string;
temperature?: number;
apiVersion?: string;
prefixPaddingMs?: number;
silenceDurationMs?: number;
startSensitivity?: GoogleRealtimeSensitivity;
endSensitivity?: GoogleRealtimeSensitivity;
activityHandling?: GoogleRealtimeActivityHandling;
turnCoverage?: GoogleRealtimeTurnCoverage;
automaticActivityDetectionDisabled?: boolean;
enableAffectiveDialog?: boolean;
sessionResumption?: boolean;
contextWindowCompression?: boolean;
thinkingLevel?: GoogleRealtimeThinkingLevel;
thinkingBudget?: number;
};
type GoogleRealtimeVoiceBridgeConfig = RealtimeVoiceBridgeCreateRequest & GoogleRealtimeLiveConfig;
type GoogleLiveSession = {
sendClientContent: (params: {
turns?: Array<{ role: string; parts: Array<{ text: string }> }>;
turnComplete?: boolean;
}) => void;
sendRealtimeInput: (params: {
audio?: { data: string; mimeType: string };
audioStreamEnd?: boolean;
}) => void;
sendToolResponse: (params: { functionResponses: FunctionResponse[] | FunctionResponse }) => void;
close: () => void;
};
function trimToUndefined(value: unknown): string | undefined {
return normalizeOptionalString(value);
}
function asSensitivity(value: unknown): GoogleRealtimeSensitivity | undefined {
const normalized = normalizeOptionalString(value)?.toLowerCase();
return normalized === "low" || normalized === "high" ? normalized : undefined;
}
function asThinkingLevel(value: unknown): GoogleRealtimeThinkingLevel | undefined {
const normalized = normalizeOptionalString(value)?.toLowerCase();
return normalized === "minimal" ||
normalized === "low" ||
normalized === "medium" ||
normalized === "high"
? normalized
: undefined;
}
function asActivityHandling(value: unknown): GoogleRealtimeActivityHandling | undefined {
const normalized = normalizeOptionalString(value)?.toLowerCase().replaceAll("_", "-");
switch (normalized) {
case "start-of-activity-interrupts":
case "start-of-activity-interrupt":
case "interrupt":
case "interrupts":
return "start-of-activity-interrupts";
case "no-interruption":
case "no-interruptions":
case "none":
return "no-interruption";
default:
return undefined;
}
}
function asTurnCoverage(value: unknown): GoogleRealtimeTurnCoverage | undefined {
const normalized = normalizeOptionalString(value)?.toLowerCase().replaceAll("_", "-");
switch (normalized) {
case "only-activity":
case "turn-includes-only-activity":
return "only-activity";
case "all-input":
case "turn-includes-all-input":
return "all-input";
case "audio-activity-and-all-video":
case "turn-includes-audio-activity-and-all-video":
return "audio-activity-and-all-video";
default:
return undefined;
}
}
function asNonNegativeInteger(value: unknown): number | undefined {
const number = asFiniteNumber(value);
return number !== undefined && Number.isSafeInteger(number) && number >= 0 ? number : undefined;
}
function asGoogleRealtimeThinkingBudget(value: unknown): number | undefined {
const budget = asFiniteNumber(value);
return budget !== undefined &&
Number.isSafeInteger(budget) &&
(budget === -1 || (budget >= 0 && budget <= 24_576))
? budget
: undefined;
}
function resolveGoogleRealtimeProviderConfigRecord(
config: Record<string, unknown>,
): Record<string, unknown> | undefined {
const providers =
typeof config.providers === "object" &&
config.providers !== null &&
!Array.isArray(config.providers)
? (config.providers as Record<string, unknown>)
: undefined;
const nested = providers?.google;
return typeof nested === "object" && nested !== null && !Array.isArray(nested)
? (nested as Record<string, unknown>)
: typeof config.google === "object" && config.google !== null && !Array.isArray(config.google)
? (config.google as Record<string, unknown>)
: config;
}
function normalizeProviderConfig(
config: RealtimeVoiceProviderConfig,
cfg?: OpenClawConfig,
): GoogleRealtimeVoiceProviderConfig {
const raw = resolveGoogleRealtimeProviderConfigRecord(config);
return {
apiKey: normalizeResolvedSecretInputString({
value: raw?.apiKey ?? cfg?.models?.providers?.google?.apiKey,
path: "plugins.entries.voice-call.config.realtime.providers.google.apiKey",
}),
model: trimToUndefined(raw?.model),
voice: trimToUndefined(raw?.speakerVoice) ?? trimToUndefined(raw?.voice),
temperature: asFiniteNumber(raw?.temperature),
apiVersion: trimToUndefined(raw?.apiVersion),
prefixPaddingMs: asNonNegativeInteger(raw?.prefixPaddingMs),
silenceDurationMs: asNonNegativeInteger(raw?.silenceDurationMs),
startSensitivity: asSensitivity(raw?.startSensitivity),
endSensitivity: asSensitivity(raw?.endSensitivity),
activityHandling: asActivityHandling(raw?.activityHandling),
turnCoverage: asTurnCoverage(raw?.turnCoverage),
automaticActivityDetectionDisabled: asBoolean(raw?.automaticActivityDetectionDisabled),
enableAffectiveDialog: asBoolean(raw?.enableAffectiveDialog),
sessionResumption: asBoolean(raw?.sessionResumption),
contextWindowCompression: asBoolean(raw?.contextWindowCompression),
thinkingLevel: asThinkingLevel(raw?.thinkingLevel),
thinkingBudget: asGoogleRealtimeThinkingBudget(raw?.thinkingBudget),
};
}
function resolveEnvApiKey(): string | undefined {
return trimToUndefined(process.env.GEMINI_API_KEY) ?? trimToUndefined(process.env.GOOGLE_API_KEY);
}
function mapStartSensitivity(
value: GoogleRealtimeSensitivity | undefined,
): StartSensitivity | undefined {
switch (value) {
case "high":
return "START_SENSITIVITY_HIGH" as StartSensitivity;
case "low":
return "START_SENSITIVITY_LOW" as StartSensitivity;
default:
return undefined;
}
}
function mapEndSensitivity(
value: GoogleRealtimeSensitivity | undefined,
): EndSensitivity | undefined {
switch (value) {
case "high":
return "END_SENSITIVITY_HIGH" as EndSensitivity;
case "low":
return "END_SENSITIVITY_LOW" as EndSensitivity;
default:
return undefined;
}
}
function mapActivityHandling(
value: GoogleRealtimeActivityHandling | undefined,
): ActivityHandling | undefined {
switch (value) {
case "no-interruption":
return "NO_INTERRUPTION" as ActivityHandling;
case "start-of-activity-interrupts":
return "START_OF_ACTIVITY_INTERRUPTS" as ActivityHandling;
default:
return undefined;
}
}
function mapTurnCoverage(value: GoogleRealtimeTurnCoverage | undefined): TurnCoverage | undefined {
switch (value) {
case "only-activity":
return "TURN_INCLUDES_ONLY_ACTIVITY" as TurnCoverage;
case "all-input":
return "TURN_INCLUDES_ALL_INPUT" as TurnCoverage;
case "audio-activity-and-all-video":
return "TURN_INCLUDES_AUDIO_ACTIVITY_AND_ALL_VIDEO" as TurnCoverage;
default:
return undefined;
}
}
function buildThinkingConfig(config: GoogleRealtimeLiveConfig): ThinkingConfig | undefined {
if (config.thinkingLevel) {
return { thinkingLevel: config.thinkingLevel.toUpperCase() as ThinkingConfig["thinkingLevel"] };
}
if (typeof config.thinkingBudget === "number") {
return { thinkingBudget: config.thinkingBudget };
}
return undefined;
}
function buildRealtimeInputConfig(
config: GoogleRealtimeLiveConfig,
): RealtimeInputConfig | undefined {
const startSensitivity = mapStartSensitivity(config.startSensitivity);
const endSensitivity = mapEndSensitivity(config.endSensitivity);
const activityHandling = mapActivityHandling(config.activityHandling);
const turnCoverage = mapTurnCoverage(config.turnCoverage);
const automaticActivityDetection = {
...(typeof config.automaticActivityDetectionDisabled === "boolean"
? { disabled: config.automaticActivityDetectionDisabled }
: {}),
...(startSensitivity ? { startOfSpeechSensitivity: startSensitivity } : {}),
...(endSensitivity ? { endOfSpeechSensitivity: endSensitivity } : {}),
...(typeof config.prefixPaddingMs === "number"
? { prefixPaddingMs: config.prefixPaddingMs }
: {}),
...(typeof config.silenceDurationMs === "number"
? { silenceDurationMs: config.silenceDurationMs }
: {}),
};
const realtimeInputConfig = {
...(Object.keys(automaticActivityDetection).length > 0 ? { automaticActivityDetection } : {}),
...(activityHandling ? { activityHandling } : {}),
...(turnCoverage ? { turnCoverage } : {}),
};
return Object.keys(realtimeInputConfig).length > 0 ? realtimeInputConfig : undefined;
}
function buildFunctionDeclarations(tools: RealtimeVoiceTool[] | undefined): FunctionDeclaration[] {
return (tools ?? []).map((tool) => {
const declaration: FunctionDeclaration = {
name: tool.name,
description: tool.description,
parametersJsonSchema: tool.parameters,
};
if (tool.name === REALTIME_VOICE_AGENT_CONSULT_TOOL_NAME) {
declaration.behavior = "NON_BLOCKING" as Behavior;
}
return declaration;
});
}
function buildGoogleLiveConnectConfig(config: GoogleRealtimeLiveConfig): LiveConnectConfig {
const functionDeclarations = buildFunctionDeclarations(config.tools);
const realtimeInputConfig = buildRealtimeInputConfig(config);
const thinkingConfig = buildThinkingConfig(config);
return {
responseModalities: ["AUDIO" as Modality],
...(typeof config.temperature === "number" && config.temperature > 0
? { temperature: config.temperature }
: {}),
speechConfig: {
voiceConfig: {
prebuiltVoiceConfig: {
voiceName: config.voice ?? GOOGLE_REALTIME_DEFAULT_VOICE,
},
},
},
systemInstruction: config.instructions,
...(functionDeclarations.length > 0 ? { tools: [{ functionDeclarations }] } : {}),
...(realtimeInputConfig ? { realtimeInputConfig } : {}),
inputAudioTranscription: {},
outputAudioTranscription: {},
...(typeof config.enableAffectiveDialog === "boolean"
? { enableAffectiveDialog: config.enableAffectiveDialog }
: {}),
...(thinkingConfig ? { thinkingConfig } : {}),
};
}
function toGoogleModelResource(model: string): string {
return model.startsWith("models/") ? model : `models/${model}`;
}
function buildBrowserInitialSetup(model: string) {
return {
setup: {
model: toGoogleModelResource(model),
generationConfig: {
responseModalities: ["AUDIO" as Modality],
},
inputAudioTranscription: {},
outputAudioTranscription: {},
},
};
}
function parsePcmSampleRate(mimeType: string | undefined): number {
const match = mimeType?.match(/(?:^|[;,\s])rate=(\d+)/i);
const parsed = match ? Number.parseInt(match[1] ?? "", 10) : Number.NaN;
return Number.isFinite(parsed) && parsed > 0 ? parsed : 24_000;
}
function isMulawSilence(audio: Buffer): boolean {
return audio.length > 0 && audio.every((sample) => sample === 0xff);
}
function isPcm16Silence(audio: Buffer): boolean {
const samples = Math.floor(audio.length / 2);
if (samples === 0) {
return false;
}
for (let i = 0; i < samples; i += 1) {
if (audio.readInt16LE(i * 2) !== 0) {
return false;
}
}
return true;
}
function formatGoogleLiveCloseEvent(
event:
| {
code?: number;
reason?: string;
wasClean?: boolean;
}
| undefined,
): string {
if (!event) {
return "code=unknown reason=unknown";
}
const code = typeof event.code === "number" ? event.code : "unknown";
const reason = event.reason?.trim() || "none";
const clean = typeof event.wasClean === "boolean" ? ` clean=${event.wasClean}` : "";
return `code=${code} reason=${reason}${clean}`;
}
class GoogleRealtimeVoiceBridge implements RealtimeVoiceBridge {
readonly supportsToolResultContinuation = true;
private session: GoogleLiveSession | null = null;
private connected = false;
private sessionConfigured = false;
private intentionallyClosed = false;
private pendingAudio: Buffer[] = [];
private sessionReadyFired = false;
private consecutiveSilenceMs = 0;
private audioStreamEnded = false;
private pendingFunctionNames = new Map<string, string>();
private readonly audioFormat: RealtimeVoiceAudioFormat;
private resumptionHandle: string | undefined;
private reconnectAttempts = 0;
private reconnectTimer: ReturnType<typeof setTimeout> | undefined;
constructor(private readonly config: GoogleRealtimeVoiceBridgeConfig) {
this.audioFormat = config.audioFormat ?? REALTIME_VOICE_AUDIO_FORMAT_G711_ULAW_8KHZ;
}
async connect(): Promise<void> {
this.intentionallyClosed = false;
this.sessionConfigured = false;
this.sessionReadyFired = false;
this.consecutiveSilenceMs = 0;
this.audioStreamEnded = false;
this.pendingFunctionNames.clear();
const ai = createGoogleGenAI({
apiKey: this.config.apiKey,
httpOptions: {
apiVersion: this.config.apiVersion ?? GOOGLE_REALTIME_DEFAULT_API_VERSION,
},
});
this.session = (await ai.live.connect({
model: this.config.model ?? GOOGLE_REALTIME_DEFAULT_MODEL,
config: {
...buildGoogleLiveConnectConfig(this.config),
...(this.config.sessionResumption === false
? {}
: {
sessionResumption: this.resumptionHandle ? { handle: this.resumptionHandle } : {},
}),
...(this.config.contextWindowCompression === false
? {}
: { contextWindowCompression: { slidingWindow: {} } }),
},
callbacks: {
onopen: () => {
this.connected = true;
},
onmessage: (message) => {
this.handleMessage(message);
},
onerror: (event) => {
const error =
event.error instanceof Error
? event.error
: new Error(
typeof event.message === "string" ? event.message : "Google Live API error",
);
this.config.onError?.(error);
},
onclose: (event) => {
this.connected = false;
this.sessionConfigured = false;
this.pendingFunctionNames.clear();
this.session = null;
if (this.intentionallyClosed) {
this.config.onClose?.("completed");
return;
}
const closeDetails = formatGoogleLiveCloseEvent(event);
if (this.scheduleReconnect(closeDetails)) {
return;
}
this.config.onError?.(
new Error(`Google Live session closed after reconnect attempts: ${closeDetails}`),
);
this.config.onClose?.("error");
},
},
})) as GoogleLiveSession;
}
sendAudio(audio: Buffer): void {
if (!this.session || !this.connected || !this.sessionConfigured) {
if (this.pendingAudio.length < MAX_PENDING_AUDIO_CHUNKS) {
this.pendingAudio.push(audio);
}
return;
}
const silent = this.isSilence(audio);
if (silent && this.audioStreamEnded) {
return;
}
if (!silent) {
this.consecutiveSilenceMs = 0;
this.audioStreamEnded = false;
}
const pcm16k = this.toGoogleInputPcm16k(audio);
this.session.sendRealtimeInput({
audio: {
data: pcm16k.toString("base64"),
mimeType: `audio/pcm;rate=${GOOGLE_REALTIME_INPUT_SAMPLE_RATE}`,
},
});
if (!silent) {
return;
}
const silenceThresholdMs =
typeof this.config.silenceDurationMs === "number"
? Math.max(0, Math.floor(this.config.silenceDurationMs))
: DEFAULT_AUDIO_STREAM_END_SILENCE_MS;
const bytesPerSample = this.audioFormat.encoding === "pcm16" ? 2 : 1;
this.consecutiveSilenceMs += Math.round(
(audio.length / bytesPerSample / this.audioFormat.sampleRateHz) * 1000,
);
if (!this.audioStreamEnded && this.consecutiveSilenceMs >= silenceThresholdMs) {
this.session.sendRealtimeInput({ audioStreamEnd: true });
this.audioStreamEnded = true;
}
}
setMediaTimestamp(_ts: number): void {}
sendUserMessage(text: string): void {
const normalized = text.trim();
if (!normalized || !this.session || !this.connected || !this.sessionConfigured) {
return;
}
this.session.sendClientContent({
turns: [{ role: "user", parts: [{ text: normalized }] }],
turnComplete: true,
});
}
triggerGreeting(instructions?: string): void {
const greetingPrompt =
instructions?.trim() || "Start the call now. Greet the caller naturally and keep it brief.";
this.sendUserMessage(greetingPrompt);
}
submitToolResult(
callId: string,
result: unknown,
options?: RealtimeVoiceToolResultOptions,
): void {
if (!this.session) {
return;
}
const name = this.pendingFunctionNames.get(callId);
if (!name) {
this.config.onError?.(
new Error(
`Google Live function response is missing a matching function call for ${callId}`,
),
);
return;
}
try {
const isConsultTool = name === REALTIME_VOICE_AGENT_CONSULT_TOOL_NAME;
const functionResponse: FunctionResponse = {
id: callId,
name,
response:
result && typeof result === "object" && !Array.isArray(result)
? (result as Record<string, unknown>)
: { output: result },
};
if (isConsultTool) {
functionResponse.scheduling = "WHEN_IDLE" as FunctionResponseScheduling;
if (options?.willContinue === true) {
functionResponse.willContinue = true;
}
} else if (options?.willContinue === true) {
this.config.onError?.(
new Error(
`Google Live continuation is only supported for ${REALTIME_VOICE_AGENT_CONSULT_TOOL_NAME}`,
),
);
return;
}
this.session.sendToolResponse({
functionResponses: [functionResponse],
});
if (options?.willContinue !== true) {
this.pendingFunctionNames.delete(callId);
}
} catch (error) {
this.config.onError?.(
error instanceof Error ? error : new Error("Failed to send Google Live function response"),
);
}
}
acknowledgeMark(): void {}
close(): void {
this.intentionallyClosed = true;
this.connected = false;
this.sessionConfigured = false;
if (this.reconnectTimer) {
clearTimeout(this.reconnectTimer);
this.reconnectTimer = undefined;
}
this.pendingAudio = [];
this.consecutiveSilenceMs = 0;
this.audioStreamEnded = false;
this.pendingFunctionNames.clear();
const session = this.session;
this.session = null;
session?.close();
}
isConnected(): boolean {
return this.connected && this.sessionConfigured;
}
private isSilence(audio: Buffer): boolean {
return this.audioFormat.encoding === "pcm16" ? isPcm16Silence(audio) : isMulawSilence(audio);
}
private toInputPcm(audio: Buffer): Buffer {
return this.audioFormat.encoding === "pcm16" ? audio : mulawToPcm(audio);
}
private toGoogleInputPcm16k(audio: Buffer): Buffer {
if (
this.audioFormat.encoding === "g711_ulaw" &&
this.audioFormat.sampleRateHz === 8_000 &&
GOOGLE_REALTIME_INPUT_SAMPLE_RATE === 16_000
) {
return convertMulaw8kToPcm16k(audio);
}
return resamplePcm(
this.toInputPcm(audio),
this.audioFormat.sampleRateHz,
GOOGLE_REALTIME_INPUT_SAMPLE_RATE,
);
}
private toOutputAudio(pcm: Buffer, sampleRate: number): Buffer {
return this.audioFormat.encoding === "pcm16"
? resamplePcm(pcm, sampleRate, this.audioFormat.sampleRateHz)
: convertPcmToMulaw8k(pcm, sampleRate);
}
private handleMessage(message: LiveServerMessage): void {
this.captureSessionLifecycle(message);
if (message.setupComplete) {
this.handleSetupComplete();
}
if (message.serverContent) {
this.handleServerContent(message.serverContent);
}
if (message.toolCall) {
this.handleToolCall(message.toolCall);
}
}
private captureSessionLifecycle(message: LiveServerMessage): void {
const raw = message as unknown as {
goAway?: { timeLeft?: string };
sessionResumptionUpdate?: { newHandle?: string; resumable?: boolean };
};
const update = raw.sessionResumptionUpdate;
if (update?.resumable && update.newHandle) {
this.resumptionHandle = update.newHandle;
}
if (raw.goAway?.timeLeft) {
this.config.onError?.(new Error(`Google Live session goAway: ${raw.goAway.timeLeft}`));
}
}
private handleSetupComplete(): void {
this.sessionConfigured = true;
this.reconnectAttempts = 0;
for (const chunk of this.pendingAudio.splice(0)) {
this.sendAudio(chunk);
}
if (!this.sessionReadyFired) {
this.sessionReadyFired = true;
this.config.onReady?.();
}
}
private handleServerContent(content: LiveServerContent): void {
if (content.interrupted) {
this.config.onClearAudio();
}
if (content.inputTranscription?.text) {
this.config.onTranscript?.(
"user",
content.inputTranscription.text,
content.inputTranscription.finished ?? false,
);
}
if (content.outputTranscription?.text) {
this.config.onTranscript?.(
"assistant",
content.outputTranscription.text,
content.outputTranscription.finished ?? false,
);
}
for (const part of content.modelTurn?.parts ?? []) {
if (part.inlineData?.data) {
const pcm = Buffer.from(part.inlineData.data, "base64");
const sampleRate = parsePcmSampleRate(part.inlineData.mimeType);
const audio = this.toOutputAudio(pcm, sampleRate);
if (audio.length > 0) {
this.config.onAudio(audio);
this.config.onMark?.(`audio-${randomUUID()}`);
}
continue;
}
if (part.thought) {
continue;
}
if (!content.outputTranscription?.text && typeof part.text === "string" && part.text.trim()) {
this.config.onTranscript?.("assistant", part.text, content.turnComplete ?? false);
}
}
}
private handleToolCall(toolCall: LiveServerToolCall): void {
for (const call of toolCall.functionCalls ?? []) {
const name = call.name?.trim();
if (!name) {
continue;
}
const callId = call.id?.trim() || `google-live-${randomUUID()}`;
this.pendingFunctionNames.set(callId, name);
this.config.onToolCall?.({
itemId: callId,
callId,
name,
args: call.args ?? {},
});
}
}
private scheduleReconnect(closeDetails: string): boolean {
if (this.reconnectAttempts >= GOOGLE_REALTIME_RECONNECT_MAX_ATTEMPTS) {
return false;
}
const attempt = ++this.reconnectAttempts;
const delayMs = Math.min(
GOOGLE_REALTIME_RECONNECT_MAX_DELAY_MS,
GOOGLE_REALTIME_RECONNECT_BASE_DELAY_MS * 2 ** (attempt - 1),
);
this.config.onError?.(
new Error(
`Google Live session closed unexpectedly (${closeDetails}); reconnecting ${attempt}/${GOOGLE_REALTIME_RECONNECT_MAX_ATTEMPTS} in ${delayMs}ms`,
),
);
this.reconnectTimer = setTimeout(() => {
this.reconnectTimer = undefined;
if (this.intentionallyClosed) {
return;
}
this.connect().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
this.config.onError?.(error instanceof Error ? error : new Error(message));
if (!this.scheduleReconnect(`connect failed: ${message}`)) {
this.config.onClose?.("error");
}
});
}, delayMs);
return true;
}
}
function convertMulaw8kToPcm16k(muLaw: Buffer): Buffer {
if (muLaw.length === 0) {
return Buffer.alloc(0);
}
const pcm = Buffer.alloc(muLaw.length * 4);
for (let i = 0; i < muLaw.length; i += 1) {
const current = MULAW_LINEAR_SAMPLES[muLaw[i] ?? 0] ?? 0;
const next = MULAW_LINEAR_SAMPLES[muLaw[i + 1] ?? muLaw[i] ?? 0] ?? current;
pcm.writeInt16LE(current, i * 4);
pcm.writeInt16LE(Math.round((current + next) / 2), i * 4 + 2);
}
return pcm;
}
function decodeMulawSample(value: number): number {
const muLaw = ~value & 0xff;
const sign = muLaw & 0x80;
const exponent = (muLaw >> 4) & 0x07;
const mantissa = muLaw & 0x0f;
let sample = ((mantissa << 3) + 132) << exponent;
sample -= 132;
return sign ? -sample : sample;
}
async function createGoogleRealtimeBrowserSession(
req: RealtimeVoiceBrowserSessionCreateRequest,
): Promise<RealtimeVoiceBrowserSession> {
const config = normalizeProviderConfig(req.providerConfig);
const apiKey = config.apiKey || resolveEnvApiKey();
if (!apiKey) {
throw new Error("Google Gemini API key missing");
}
const model = req.model ?? config.model ?? GOOGLE_REALTIME_DEFAULT_MODEL;
const voice = req.voice ?? config.voice ?? GOOGLE_REALTIME_DEFAULT_VOICE;
const nowMs = Date.now();
const expiresAtMs = resolveExpiresAtMsFromDurationMs(GOOGLE_REALTIME_BROWSER_SESSION_TTL_MS, {
nowMs,
});
const newSessionExpiresAtMs = resolveExpiresAtMsFromDurationMs(
GOOGLE_REALTIME_BROWSER_NEW_SESSION_TTL_MS,
{ nowMs },
);
const expireTime = timestampMsToIsoString(expiresAtMs);
const newSessionExpireTime = timestampMsToIsoString(newSessionExpiresAtMs);
if (expiresAtMs === undefined || !expireTime || !newSessionExpireTime) {
throw new Error("Google realtime browser session expiry is outside the supported Date range");
}
const ai = createGoogleGenAI({
apiKey,
httpOptions: {
apiVersion: GOOGLE_REALTIME_BROWSER_API_VERSION,
},
});
const token = await ai.authTokens.create({
config: {
uses: 1,
expireTime,
newSessionExpireTime,
liveConnectConstraints: {
model,
config: buildGoogleLiveConnectConfig({
...config,
apiKey,
model,
voice,
instructions: req.instructions,
tools: req.tools,
}),
},
},
});
const clientSecret = token.name?.trim();
if (!clientSecret) {
throw new Error("Google Live browser session did not return an ephemeral token");
}
return {
provider: "google",
transport: "provider-websocket",
protocol: "google-live-bidi",
clientSecret,
websocketUrl: GOOGLE_REALTIME_BROWSER_WEBSOCKET_URL,
audio: {
inputEncoding: "pcm16",
inputSampleRateHz: GOOGLE_REALTIME_INPUT_SAMPLE_RATE,
outputEncoding: "pcm16",
outputSampleRateHz: 24_000,
},
initialMessage: buildBrowserInitialSetup(model),
model,
voice,
expiresAt: Math.floor(expiresAtMs / 1000),
};
}
export function buildGoogleRealtimeVoiceProvider(): RealtimeVoiceProviderPlugin {
return {
id: "google",
label: "Google Live Voice",
defaultModel: GOOGLE_REALTIME_DEFAULT_MODEL,
autoSelectOrder: 20,
capabilities: {
transports: ["provider-websocket", "gateway-relay"],
inputAudioFormats: [
REALTIME_VOICE_AUDIO_FORMAT_G711_ULAW_8KHZ,
REALTIME_VOICE_AUDIO_FORMAT_PCM16_24KHZ,
],
outputAudioFormats: [
REALTIME_VOICE_AUDIO_FORMAT_G711_ULAW_8KHZ,
REALTIME_VOICE_AUDIO_FORMAT_PCM16_24KHZ,
],
supportsBrowserSession: true,
supportsBargeIn: true,
supportsToolCalls: true,
supportsVideoFrames: true,
supportsSessionResumption: true,
},
resolveConfig: ({ cfg, rawConfig }) => normalizeProviderConfig(rawConfig, cfg),
isConfigured: ({ providerConfig }) =>
Boolean(normalizeProviderConfig(providerConfig).apiKey || resolveEnvApiKey()),
createBridge: (req) => {
const config = normalizeProviderConfig(req.providerConfig);
const apiKey = config.apiKey || resolveEnvApiKey();
if (!apiKey) {
throw new Error("Google Gemini API key missing");
}
return new GoogleRealtimeVoiceBridge({
...req,
apiKey,
model: config.model,
voice: config.voice,
temperature: config.temperature,
apiVersion: config.apiVersion,
prefixPaddingMs: config.prefixPaddingMs,
silenceDurationMs: config.silenceDurationMs,
startSensitivity: config.startSensitivity,
endSensitivity: config.endSensitivity,
activityHandling: config.activityHandling,
turnCoverage: config.turnCoverage,
automaticActivityDetectionDisabled: config.automaticActivityDetectionDisabled,
enableAffectiveDialog: config.enableAffectiveDialog,
sessionResumption: config.sessionResumption,
contextWindowCompression: config.contextWindowCompression,
thinkingLevel: config.thinkingLevel,
thinkingBudget: config.thinkingBudget,
});
},
createBrowserSession: createGoogleRealtimeBrowserSession,
};
}

View File

@@ -0,0 +1,20 @@
// Google API module exposes the plugin public contract.
export {
DEFAULT_GOOGLE_API_BASE_URL,
createGoogleThinkingPayloadWrapper,
createGoogleThinkingStreamWrapper,
isGoogleGemini3FlashModel,
isGoogleGemini3ProModel,
isGoogleGemini3ThinkingLevelModel,
isGoogleThinkingRequiredModel,
normalizeGoogleApiBaseUrl,
normalizeGoogleModelId,
parseGeminiAuth,
buildGoogleGenerativeAiParams,
createGoogleGenerativeAiTransportStreamFn,
resolveGoogleGemini3ThinkingLevel,
resolveGoogleGenerativeAiHttpRequestConfig,
sanitizeGoogleThinkingPayload,
stripInvalidGoogleThinkingBudget,
} from "./api.js";
export type { GoogleThinkingInputLevel, GoogleThinkingLevel } from "./api.js";

View File

@@ -0,0 +1,564 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import type { CliBackendPlugin } from "openclaw/plugin-sdk/cli-backend";
import type { ProviderPlugin } from "openclaw/plugin-sdk/provider-model-shared";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import { withTempDir } from "openclaw/plugin-sdk/test-env";
import { describe, expect, it, vi } from "vitest";
import { buildGoogleGeminiCliBackend } from "./cli-backend.js";
import setupEntry from "./setup-api.js";
type GeminiPrepareContext = Parameters<
NonNullable<ReturnType<typeof buildGoogleGeminiCliBackend>["prepareExecution"]>
>[0] & {
env?: Record<string, string>;
authCredential?: {
type: "api_key" | "oauth" | "token";
provider: string;
access?: string;
refresh?: string;
expires?: number;
idToken?: string;
projectId?: string;
key?: string;
email?: string;
};
};
type GeminiPreparedExecution = Awaited<
ReturnType<NonNullable<ReturnType<typeof buildGoogleGeminiCliBackend>["prepareExecution"]>>
>;
async function stageGeminiPreparedExecution(
prepared: GeminiPreparedExecution | null | undefined,
): Promise<void> {
await prepared?.beforeExecution?.();
}
function buildGeminiOAuthPrepareContext(workspaceDir: string): GeminiPrepareContext {
const agentDir = path.join(workspaceDir, "agent");
return {
workspaceDir,
agentDir,
provider: "google-gemini-cli",
modelId: "gemini-3.1-pro-preview",
authProfileId: "google-gemini-cli:user@example.test",
// Private bundled-runtime bridge, not public Plugin SDK surface.
authCredential: {
type: "oauth",
provider: "google-gemini-cli",
access: "access-token",
refresh: "refresh-token",
expires: 1_800_000_000_000,
idToken: "id-token",
projectId: "profile-project",
email: "user@example.test",
},
};
}
function buildGeminiApiKeyPrepareContext(workspaceDir: string): GeminiPrepareContext {
const agentDir = path.join(workspaceDir, "agent");
return {
workspaceDir,
agentDir,
provider: "google-gemini-cli",
modelId: "gemini-3.1-flash-lite",
authProfileId: "google:api-key",
// Private bundled-runtime bridge, not public Plugin SDK surface.
authCredential: {
type: "api_key",
provider: "google",
key: "gemini-api-key",
email: "user@example.test",
},
};
}
function restoreEnv(name: string, value: string | undefined): void {
if (value === undefined) {
delete process.env[name];
return;
}
process.env[name] = value;
}
describe("google setup entry", () => {
it("registers setup runtime providers declared by the manifest", () => {
const providerIds: string[] = [];
const cliBackendIds: string[] = [];
setupEntry.register({
registerProvider(provider: ProviderPlugin) {
providerIds.push(provider.id);
},
registerCliBackend(backend: CliBackendPlugin) {
cliBackendIds.push(backend.id);
},
} as never);
expect(providerIds).toEqual(["google-vertex"]);
expect(cliBackendIds).toEqual(["google-gemini-cli"]);
});
});
describe("google gemini cli backend config", () => {
it("keeps legacy json output overrides on the json parser", () => {
const backend = buildGoogleGeminiCliBackend();
const normalized = backend.normalizeConfig?.({
...backend.config,
args: ["--skip-trust", "--output-format", "json", "--prompt", "{prompt}"],
resumeArgs: [
"--skip-trust",
"--resume",
"{sessionId}",
"--output-format=json",
"--prompt",
"{prompt}",
],
});
expect(normalized?.output).toBe("json");
expect(normalized?.resumeOutput).toBe("json");
expect(normalized?.jsonlDialect).toBeUndefined();
});
it("keeps short stream-json output overrides on the jsonl parser", () => {
const backend = buildGoogleGeminiCliBackend();
const normalized = backend.normalizeConfig?.({
...backend.config,
args: ["--skip-trust", "-o", "stream-json", "--prompt", "{prompt}"],
resumeArgs: [
"--skip-trust",
"--resume",
"{sessionId}",
"-o=stream-json",
"--prompt",
"{prompt}",
],
});
expect(normalized?.output).toBe("jsonl");
expect(normalized?.resumeOutput).toBe("jsonl");
expect(normalized?.jsonlDialect).toBe("gemini-stream-json");
});
});
describe("google gemini cli backend auth bridge", () => {
it("materializes selected OpenClaw OAuth credentials into a persistent profile-scoped Gemini CLI home", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
let home: string | undefined;
const cleanups: Array<() => Promise<void>> = [];
try {
const context = buildGeminiOAuthPrepareContext(workspaceDir);
const inheritedSettingsPath = path.join(workspaceDir, "generated-mcp-settings.json");
await fs.writeFile(
inheritedSettingsPath,
`${JSON.stringify({
security: {
auth: {
selectedType: "vertex-ai",
enforcedType: "oauth-personal",
useExternal: true,
},
},
mcp: { allowed: ["openclaw"] },
mcpServers: { openclaw: { url: "http://127.0.0.1:23119/mcp" } },
})}\n`,
"utf8",
);
context.env = { GEMINI_CLI_SYSTEM_SETTINGS_PATH: inheritedSettingsPath };
const prepared = await backend.prepareExecution?.(context);
if (prepared?.cleanup) {
cleanups.push(prepared.cleanup);
}
await stageGeminiPreparedExecution(prepared);
home = prepared?.env?.GEMINI_CLI_HOME;
const systemSettingsPath = prepared?.env?.GEMINI_CLI_SYSTEM_SETTINGS_PATH;
expect(home).toBeTruthy();
expect(systemSettingsPath).toBeTruthy();
expect(systemSettingsPath).not.toBe(inheritedSettingsPath);
expect(path.dirname(systemSettingsPath ?? "")).not.toBe(home);
expect(
path.relative(resolvePreferredOpenClawTmpDir(), path.dirname(systemSettingsPath ?? "")),
).toMatch(/^openclaw-gemini-cli-/);
expect(prepared?.env?.GEMINI_FORCE_FILE_STORAGE).toBe("true");
expect(prepared?.env?.GOOGLE_CLOUD_PROJECT).toBe("profile-project");
expect(prepared?.env?.GOOGLE_CLOUD_PROJECT_ID).toBe("profile-project");
expect(prepared?.env?.GOOGLE_CLOUD_QUOTA_PROJECT).toBe("profile-project");
if (!context.agentDir) {
throw new Error("expected Gemini test context to include an agent directory");
}
expect(home).toContain(path.join(context.agentDir, "google-gemini-cli-home"));
expect(home).not.toContain("user@example.test");
const raw = await fs.readFile(path.join(home ?? "", ".gemini", "oauth_creds.json"), "utf8");
expect(JSON.parse(raw)).toEqual({
access_token: "access-token",
refresh_token: "refresh-token",
id_token: "id-token",
expiry_date: 1_800_000_000_000,
token_type: "Bearer",
});
const nestedSettingsRaw = await fs.readFile(
path.join(home ?? "", ".gemini", "settings.json"),
"utf8",
);
const rootSettingsRaw = await fs.readFile(path.join(home ?? "", "settings.json"), "utf8");
expect(JSON.parse(nestedSettingsRaw)).toEqual({
security: { auth: { selectedType: "oauth-personal" } },
});
expect(JSON.parse(rootSettingsRaw)).toEqual(JSON.parse(nestedSettingsRaw));
const systemSettingsRaw = await fs.readFile(systemSettingsPath ?? "", "utf8");
expect(JSON.parse(systemSettingsRaw)).toEqual({
security: {
auth: {
selectedType: "oauth-personal",
enforcedType: "oauth-personal",
useExternal: true,
},
},
mcp: { allowed: ["openclaw"] },
mcpServers: { openclaw: { url: "http://127.0.0.1:23119/mcp" } },
});
const sessionMarker = path.join(home ?? "", ".gemini", "session-state.json");
await fs.writeFile(sessionMarker, '{"keep":true}\n', "utf8");
const cachedCredentialsPath = path.join(home ?? "", ".gemini", "gemini-credentials.json");
await fs.writeFile(cachedCredentialsPath, "stale-cache", "utf8");
const preparedAgain = await backend.prepareExecution?.(context);
if (preparedAgain?.cleanup) {
cleanups.push(preparedAgain.cleanup);
}
await stageGeminiPreparedExecution(preparedAgain);
expect(preparedAgain?.env?.GEMINI_CLI_HOME).toBe(home);
await expect(fs.access(sessionMarker)).resolves.toBeUndefined();
await expect(fs.access(cachedCredentialsPath)).rejects.toThrow();
} finally {
for (const cleanup of cleanups.toReversed()) {
await cleanup();
}
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("stages Gemini CLI JSON through same-directory atomic renames", async () => {
await withTempDir("openclaw-test-workspace-", async (workspaceDir) => {
const backend = buildGoogleGeminiCliBackend();
const realRename = fs.rename.bind(fs);
const renameCalls: Array<{ from: string; to: string }> = [];
const renameSpy = vi
.spyOn(fs, "rename")
.mockImplementation(async (...args: Parameters<typeof fs.rename>) => {
renameCalls.push({ from: String(args[0]), to: String(args[1]) });
await realRename(...args);
});
let prepared: GeminiPreparedExecution | null | undefined;
try {
prepared = await backend.prepareExecution?.(buildGeminiOAuthPrepareContext(workspaceDir));
await stageGeminiPreparedExecution(prepared);
const home = prepared?.env?.GEMINI_CLI_HOME;
const systemSettingsPath = prepared?.env?.GEMINI_CLI_SYSTEM_SETTINGS_PATH;
if (!home || !systemSettingsPath) {
throw new Error("expected Gemini CLI staging paths");
}
const expectedTargets = [
path.join(home, ".gemini", "settings.json"),
path.join(home, "settings.json"),
systemSettingsPath,
path.join(home, ".gemini", "oauth_creds.json"),
];
expect(renameCalls.map((call) => call.to).toSorted()).toEqual(expectedTargets.toSorted());
for (const call of renameCalls) {
expect(path.dirname(call.from)).toBe(path.dirname(call.to));
expect(path.basename(call.from).startsWith(`.${path.basename(call.to)}.`)).toBe(true);
expect(path.basename(call.from).endsWith(".tmp")).toBe(true);
}
const oauthStat = await fs.stat(path.join(home, ".gemini", "oauth_creds.json"));
expect(oauthStat.mode & 0o777).toBe(0o600);
} finally {
renameSpy.mockRestore();
await prepared?.cleanup?.();
}
});
});
it("prepares selected canonical Google API-key credentials and removes stale OAuth state for that profile home", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
let home: string | undefined;
const cleanups: Array<() => Promise<void>> = [];
try {
const context = buildGeminiApiKeyPrepareContext(workspaceDir);
const firstPrepared = await backend.prepareExecution?.(context);
if (firstPrepared?.cleanup) {
cleanups.push(firstPrepared.cleanup);
}
await stageGeminiPreparedExecution(firstPrepared);
home = firstPrepared?.env?.GEMINI_CLI_HOME;
expect(home).toBeTruthy();
await fs.writeFile(path.join(home ?? "", ".gemini", "oauth_creds.json"), "{}\n", "utf8");
await fs.writeFile(
path.join(home ?? "", ".gemini", "gemini-credentials.json"),
"stale-cache",
"utf8",
);
const prepared = await backend.prepareExecution?.(context);
if (prepared?.cleanup) {
cleanups.push(prepared.cleanup);
}
await stageGeminiPreparedExecution(prepared);
home = prepared?.env?.GEMINI_CLI_HOME;
expect(home).toBeTruthy();
expect(prepared?.env?.GEMINI_API_KEY).toBe("gemini-api-key");
expect(prepared?.env?.GEMINI_FORCE_FILE_STORAGE).toBe("true");
expect(prepared?.clearEnv).toContain("GEMINI_API_KEY");
expect(prepared?.clearEnv).toContain("GOOGLE_GENAI_USE_GCA");
expect(prepared?.clearEnv).toContain("GOOGLE_GENAI_USE_VERTEXAI");
expect(prepared?.clearEnv).toContain("GOOGLE_GEMINI_BASE_URL");
const settingsRaw = await fs.readFile(
path.join(home ?? "", ".gemini", "settings.json"),
"utf8",
);
expect(JSON.parse(settingsRaw)).toEqual({
security: { auth: { selectedType: "gemini-api-key" } },
});
await expect(
fs.access(path.join(home ?? "", ".gemini", "oauth_creds.json")),
).rejects.toThrow();
await expect(
fs.access(path.join(home ?? "", ".gemini", "gemini-credentials.json")),
).rejects.toThrow();
} finally {
for (const cleanup of cleanups.toReversed()) {
await cleanup();
}
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("rejects inherited Gemini system settings that enforce a different auth type", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
try {
const inheritedSettingsPath = path.join(workspaceDir, "generated-mcp-settings.json");
await fs.writeFile(
inheritedSettingsPath,
`${JSON.stringify({
security: { auth: { enforcedType: "gemini-api-key" } },
})}\n`,
"utf8",
);
const context = buildGeminiOAuthPrepareContext(workspaceDir);
context.env = { GEMINI_CLI_SYSTEM_SETTINGS_PATH: inheritedSettingsPath };
await expect(backend.prepareExecution?.(context)).rejects.toThrow(/enforce gemini-api-key/);
} finally {
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("inherits process Gemini system settings when no generated settings path is present", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
const originalSystemSettingsPath = process.env.GEMINI_CLI_SYSTEM_SETTINGS_PATH;
let prepared:
| Awaited<ReturnType<NonNullable<typeof backend.prepareExecution>>>
| null
| undefined;
try {
const inheritedSettingsPath = path.join(workspaceDir, "ambient-system-settings.json");
await fs.writeFile(
inheritedSettingsPath,
`${JSON.stringify({
security: {
auth: {
selectedType: "oauth-code-assist",
enforcedType: "oauth-personal",
},
folderTrust: { enabled: true },
},
})}\n`,
"utf8",
);
process.env.GEMINI_CLI_SYSTEM_SETTINGS_PATH = inheritedSettingsPath;
prepared = await backend.prepareExecution?.(buildGeminiOAuthPrepareContext(workspaceDir));
await stageGeminiPreparedExecution(prepared);
const systemSettingsRaw = await fs.readFile(
prepared?.env?.GEMINI_CLI_SYSTEM_SETTINGS_PATH ?? "",
"utf8",
);
expect(JSON.parse(systemSettingsRaw)).toEqual({
security: {
auth: {
selectedType: "oauth-personal",
enforcedType: "oauth-personal",
},
folderTrust: { enabled: true },
},
});
} finally {
restoreEnv("GEMINI_CLI_SYSTEM_SETTINGS_PATH", originalSystemSettingsPath);
await prepared?.cleanup?.();
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("rejects Vercel AI Gateway profiles for the Gemini CLI backend", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
try {
await expect(
backend.prepareExecution?.({
workspaceDir,
agentDir: path.join(workspaceDir, "agent"),
provider: "google-gemini-cli",
modelId: "gemini-3.1-flash-lite",
authProfileId: "vercel-ai-gateway:default",
authCredential: {
type: "api_key",
provider: "vercel-ai-gateway",
key: "vercel-key",
},
} as never),
).rejects.toThrow(/vercel-ai-gateway auth profile/);
} finally {
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("rejects selected Gemini token profiles before the CLI can use ambient auth", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
try {
await expect(
backend.prepareExecution?.({
workspaceDir,
agentDir: path.join(workspaceDir, "agent"),
provider: "google-gemini-cli",
modelId: "gemini-3.1-flash-lite",
authProfileId: "google-gemini-cli:token",
authCredential: {
type: "token",
provider: "google-gemini-cli",
token: "bearer-token",
},
} as never),
).rejects.toThrow(/OAuth or API-key auth profiles/);
} finally {
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("rejects selected Gemini profiles with no material before the CLI can use ambient auth", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
try {
await expect(
backend.prepareExecution?.({
workspaceDir,
agentDir: path.join(workspaceDir, "agent"),
provider: "google-gemini-cli",
modelId: "gemini-3.1-flash-lite",
authProfileId: "google-gemini-cli:missing",
} as never),
).rejects.toThrow(/no credential material/);
} finally {
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("clears inherited Gemini auth credentials when staging selected OAuth credentials", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
const originalUseGca = process.env.GOOGLE_GENAI_USE_GCA;
const originalCloudAccessToken = process.env.GOOGLE_CLOUD_ACCESS_TOKEN;
const originalGoogleApplicationCredentials = process.env.GOOGLE_APPLICATION_CREDENTIALS;
const originalForceEncryptedFileStorage = process.env.GEMINI_FORCE_ENCRYPTED_FILE_STORAGE;
const originalGeminiApiKey = process.env.GEMINI_API_KEY;
const originalGoogleApiKey = process.env.GOOGLE_API_KEY;
const originalQuotaProject = process.env.GOOGLE_CLOUD_QUOTA_PROJECT;
let prepared:
| Awaited<ReturnType<NonNullable<typeof backend.prepareExecution>>>
| null
| undefined;
process.env.GOOGLE_GENAI_USE_GCA = "true";
process.env.GOOGLE_CLOUD_ACCESS_TOKEN = "ambient-cloud-token";
process.env.GOOGLE_APPLICATION_CREDENTIALS = "/tmp/ambient-google-adc.json";
process.env.GEMINI_FORCE_ENCRYPTED_FILE_STORAGE = "true";
process.env.GEMINI_API_KEY = "ambient-gemini-key";
process.env.GOOGLE_API_KEY = "ambient-google-key";
process.env.GOOGLE_CLOUD_QUOTA_PROJECT = "ambient-project";
try {
prepared = await backend.prepareExecution?.(buildGeminiOAuthPrepareContext(workspaceDir));
expect(prepared?.env?.GEMINI_CLI_HOME).toBeTruthy();
expect(prepared?.clearEnv).toEqual([
"GOOGLE_GENAI_USE_GCA",
"GOOGLE_CLOUD_ACCESS_TOKEN",
"GOOGLE_APPLICATION_CREDENTIALS",
"GEMINI_FORCE_ENCRYPTED_FILE_STORAGE",
"GEMINI_FORCE_FILE_STORAGE",
"GOOGLE_GENAI_USE_VERTEXAI",
"GOOGLE_API_KEY",
"GOOGLE_CLOUD_PROJECT",
"GOOGLE_CLOUD_PROJECT_ID",
"GOOGLE_CLOUD_QUOTA_PROJECT",
"GOOGLE_CLOUD_LOCATION",
"GOOGLE_GEMINI_BASE_URL",
"GEMINI_CLI_CUSTOM_HEADERS",
"GEMINI_API_KEY_AUTH_MECHANISM",
"GEMINI_API_KEY",
"GEMINI_CLI_SYSTEM_SETTINGS_PATH",
]);
} finally {
restoreEnv("GOOGLE_GENAI_USE_GCA", originalUseGca);
restoreEnv("GOOGLE_CLOUD_ACCESS_TOKEN", originalCloudAccessToken);
restoreEnv("GOOGLE_APPLICATION_CREDENTIALS", originalGoogleApplicationCredentials);
restoreEnv("GEMINI_FORCE_ENCRYPTED_FILE_STORAGE", originalForceEncryptedFileStorage);
restoreEnv("GEMINI_API_KEY", originalGeminiApiKey);
restoreEnv("GOOGLE_API_KEY", originalGoogleApiKey);
restoreEnv("GOOGLE_CLOUD_QUOTA_PROJECT", originalQuotaProject);
await prepared?.cleanup?.();
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("requires an agent directory for profile-owned Gemini CLI state", async () => {
const backend = buildGoogleGeminiCliBackend();
const workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-test-workspace-"));
try {
const { agentDir: _agentDir, ...context } = buildGeminiOAuthPrepareContext(workspaceDir);
await expect(backend.prepareExecution?.(context)).rejects.toThrow(/agent directory/);
} finally {
await fs.rm(workspaceDir, { recursive: true, force: true });
}
});
it("uses profile-only auth epochs for the private Gemini CLI bridge", () => {
const backend = buildGoogleGeminiCliBackend();
expect(backend.authEpochMode).toBe("profile-only");
expect(backend.prepareExecution).toBeTypeOf("function");
});
});

View File

@@ -0,0 +1,14 @@
// Google API module exposes the plugin public contract.
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
import { buildGoogleGeminiCliBackend } from "./cli-backend.js";
import { createGoogleVertexProvider } from "./provider-contract-api.js";
export default definePluginEntry({
id: "google",
name: "Google Setup",
description: "Lightweight Google setup hooks",
register(api) {
api.registerProvider(createGoogleVertexProvider());
api.registerCliBackend(buildGoogleGeminiCliBackend());
},
});

View File

@@ -0,0 +1,738 @@
// Google tests cover speech provider plugin behavior.
import {
getProviderHttpMocks,
installProviderHttpMockCleanup,
} from "openclaw/plugin-sdk/provider-http-test-mocks";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
const transcodeAudioBufferToOpusMock = vi.hoisted(() => vi.fn());
vi.mock("openclaw/plugin-sdk/media-runtime", () => ({
transcodeAudioBufferToOpus: transcodeAudioBufferToOpusMock,
}));
const {
assertOkOrThrowProviderErrorMock,
postJsonRequestMock,
resolveProviderHttpRequestConfigMock,
} = getProviderHttpMocks();
let buildGoogleSpeechProvider: typeof import("./speech-provider.js").buildGoogleSpeechProvider;
let testing: typeof import("./speech-provider.js").testing;
const GOOGLE_TTS_JSON_CAP_BYTES = 16 * 1024 * 1024;
beforeAll(async () => {
({ buildGoogleSpeechProvider, testing } = await import("./speech-provider.js"));
});
installProviderHttpMockCleanup();
function googleTtsResponse(pcm = Buffer.from([1, 0, 2, 0])) {
return {
ok: true,
json: async () => ({
candidates: [
{
content: {
parts: [
{
inlineData: {
mimeType: "audio/L16;codec=pcm;rate=24000",
data: pcm.toString("base64"),
},
},
],
},
},
],
}),
};
}
function installGoogleTtsRequestMock(pcm = Buffer.from([1, 0, 2, 0])) {
postJsonRequestMock.mockResolvedValue({
response: googleTtsResponse(pcm),
release: vi.fn(async () => {}),
});
return postJsonRequestMock;
}
function oversizedGoogleTtsJsonResponse(onCancel: () => void): Response {
const response = new Response(
new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(new Uint8Array(GOOGLE_TTS_JSON_CAP_BYTES + 1));
},
cancel() {
onCancel();
},
}),
{ headers: { "content-type": "application/json" }, status: 200 },
);
Object.defineProperty(response, "json", {
value: async () => {
throw new Error("unbounded json reader was used");
},
});
return response;
}
function expectRecordFields(value: unknown, expected: Record<string, unknown>) {
if (!value || typeof value !== "object") {
throw new Error("Expected record");
}
const actual = value as Record<string, unknown>;
for (const [key, expectedValue] of Object.entries(expected)) {
expect(actual[key]).toEqual(expectedValue);
}
return actual;
}
function requireFirstMockArg(mock: ReturnType<typeof vi.fn>, label: string): unknown {
const [call] = mock.mock.calls;
if (!call) {
throw new Error(`Expected ${label}`);
}
return call[0];
}
function requireFirstRecordArg(
mock: ReturnType<typeof vi.fn>,
label: string,
): Record<string, unknown> {
const value = requireFirstMockArg(mock, label);
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error(`Expected ${label}`);
}
return value as Record<string, unknown>;
}
describe("Google speech provider", () => {
afterEach(() => {
vi.unstubAllGlobals();
vi.unstubAllEnvs();
transcodeAudioBufferToOpusMock.mockReset();
});
afterAll(() => {
vi.doUnmock("openclaw/plugin-sdk/media-runtime");
vi.resetModules();
});
it("synthesizes Gemini PCM as WAV and preserves audio tags in the request text", async () => {
const requestMock = installGoogleTtsRequestMock();
const provider = buildGoogleSpeechProvider();
const result = await provider.synthesize({
text: "[whispers] The door is open.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
model: "google/gemini-3.1-flash-tts",
voiceName: "Puck",
},
target: "audio-file",
timeoutMs: 12_345,
});
const request = expectRecordFields(requireFirstRecordArg(requestMock, "Google TTS request"), {
url: "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-tts-preview:generateContent",
body: {
contents: [
{
role: "user",
parts: [{ text: "[whispers] The door is open." }],
},
],
generationConfig: {
responseModalities: ["AUDIO"],
speechConfig: {
voiceConfig: {
prebuiltVoiceConfig: {
voiceName: "Puck",
},
},
},
},
},
fetchFn: fetch,
pinDns: false,
timeoutMs: 12_345,
}) as { headers?: HeadersInit };
expect(new Headers(request.headers).get("x-goog-api-key")).toBe("google-test-key");
expect(result.outputFormat).toBe("wav");
expect(result.fileExtension).toBe(".wav");
expect(result.voiceCompatible).toBe(false);
expect(result.audioBuffer.subarray(0, 4).toString("ascii")).toBe("RIFF");
expect(result.audioBuffer.subarray(8, 12).toString("ascii")).toBe("WAVE");
expect(result.audioBuffer.readUInt32LE(24)).toBe(testing.GOOGLE_TTS_SAMPLE_RATE);
expect(result.audioBuffer.subarray(44)).toEqual(Buffer.from([1, 0, 2, 0]));
expect(transcodeAudioBufferToOpusMock).not.toHaveBeenCalled();
});
it("bounds oversized Gemini TTS success JSON responses and cancels the stream", async () => {
let cancelCount = 0;
const release = vi.fn(async () => {});
postJsonRequestMock
.mockResolvedValueOnce({
response: oversizedGoogleTtsJsonResponse(() => {
cancelCount += 1;
}),
release,
})
.mockResolvedValueOnce({
response: oversizedGoogleTtsJsonResponse(() => {
cancelCount += 1;
}),
release,
});
const provider = buildGoogleSpeechProvider();
await expect(
provider.synthesize({
text: "oversized tts response",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
},
target: "audio-file",
timeoutMs: 12_000,
}),
).rejects.toThrow("Google TTS response: JSON response exceeds 16777216 bytes");
expect(cancelCount).toBe(2);
expect(release).toHaveBeenCalledTimes(2);
});
it("transcodes Gemini PCM to Opus for voice-note targets", async () => {
installGoogleTtsRequestMock(Buffer.from([5, 0, 6, 0]));
transcodeAudioBufferToOpusMock.mockResolvedValueOnce(Buffer.from("google-opus"));
const provider = buildGoogleSpeechProvider();
const result = await provider.synthesize({
text: "Send this as a voice note.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
},
target: "voice-note",
timeoutMs: 12_000,
});
expect(result).toEqual({
audioBuffer: Buffer.from("google-opus"),
outputFormat: "opus",
fileExtension: ".opus",
voiceCompatible: true,
});
const transcodeArg = expectRecordFields(
requireFirstRecordArg(transcodeAudioBufferToOpusMock, "Google TTS transcode request"),
{
inputExtension: "wav",
tempPrefix: "tts-google-",
timeoutMs: 12_000,
},
);
expect(Buffer.isBuffer(transcodeArg.audioBuffer)).toBe(true);
const audioBuffer = transcodeArg.audioBuffer as Buffer;
expect(audioBuffer.subarray(0, 4).toString("ascii")).toBe("RIFF");
expect(audioBuffer.subarray(8, 12).toString("ascii")).toBe("WAVE");
});
it("advertises all documented Gemini TTS-capable models", () => {
const provider = buildGoogleSpeechProvider();
expect(provider.models).toEqual(testing.GOOGLE_TTS_MODELS);
});
it("renders deterministic audio-profile-v1 prompts without generating tags", async () => {
const provider = buildGoogleSpeechProvider();
const prepared = await provider.prepareSynthesis?.({
text: "[whispers] The door is open.",
cfg: {},
providerConfig: {
promptTemplate: "audio-profile-v1",
personaPrompt: "Keep a close-mic feel.",
},
persona: {
id: "alfred",
label: "Alfred",
prompt: {
profile: "A brilliant British butler.",
scene: "A quiet late-night study.",
sampleContext: "The speaker is answering a trusted operator.",
style: "Refined and lightly amused.",
accent: "British English.",
pacing: "Measured.",
constraints: ["Do not read configuration values aloud."],
},
},
target: "audio-file",
timeoutMs: 1_000,
});
expect(prepared?.text).toBe(
[
"Synthesize speech from the TRANSCRIPT section only. Use the other sections only",
"as performance direction. Do not read section titles, notes, labels, or",
"configuration aloud.",
"",
"# AUDIO PROFILE: Alfred",
"A brilliant British butler.",
"",
"## THE SCENE",
"A quiet late-night study.",
"",
"### DIRECTOR'S NOTES",
"Style: Refined and lightly amused.",
"Accent: British English.",
"Pacing: Measured.",
"Constraints:",
"- Do not read configuration values aloud.",
"Provider notes:",
"Keep a close-mic feel.",
"",
"### SAMPLE CONTEXT",
"The speaker is answering a trusted operator.",
"",
"### TRANSCRIPT",
"[whispers] The door is open.",
].join("\n"),
);
});
it("does not wrap an OpenClaw audio-profile-v1 prompt twice", async () => {
const provider = buildGoogleSpeechProvider();
const text = [
"Synthesize speech from the TRANSCRIPT section only. Use the other sections only",
"as performance direction. Do not read section titles, notes, labels, or",
"configuration aloud.",
"",
"# AUDIO PROFILE: Alfred",
"A brilliant British butler.",
"",
"### TRANSCRIPT",
"Hello.",
].join("\n");
const prepared = await provider.prepareSynthesis?.({
text,
cfg: {},
providerConfig: {
promptTemplate: "audio-profile-v1",
},
persona: {
id: "alfred",
label: "Alfred",
prompt: {
profile: "A brilliant British butler.",
},
},
target: "audio-file",
timeoutMs: 1_000,
});
expect(prepared).toBeUndefined();
});
it("retries once when Gemini returns no audio payload", async () => {
const pcm = Buffer.from([5, 0, 6, 0]);
const requestSequence = vi
.fn()
.mockResolvedValueOnce({
response: {
ok: true,
json: async () => ({ candidates: [{ content: { parts: [{ text: "not audio" }] } }] }),
},
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: googleTtsResponse(pcm),
release: vi.fn(async () => {}),
});
postJsonRequestMock.mockImplementation(requestSequence);
const provider = buildGoogleSpeechProvider();
const result = await provider.synthesize({
text: "Retry this.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
},
target: "audio-file",
timeoutMs: 5_000,
});
expect(requestSequence).toHaveBeenCalledTimes(2);
expect(result.audioBuffer.subarray(44)).toEqual(pcm);
});
it("retries once when Gemini TTS fetch aborts", async () => {
const pcm = Buffer.from([7, 0, 8, 0]);
const abortError = new Error("This operation was aborted");
abortError.name = "AbortError";
const requestSequence = vi
.fn()
.mockRejectedValueOnce(abortError)
.mockResolvedValueOnce({
response: googleTtsResponse(pcm),
release: vi.fn(async () => {}),
});
postJsonRequestMock.mockImplementation(requestSequence);
const provider = buildGoogleSpeechProvider();
const result = await provider.synthesize({
text: "Retry aborted fetch.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
},
target: "audio-file",
timeoutMs: 5_000,
});
expect(requestSequence).toHaveBeenCalledTimes(2);
expect(result.audioBuffer.subarray(44)).toEqual(pcm);
});
it("does not retry non-transient Gemini TTS request failures", async () => {
const requestSequence = vi.fn().mockRejectedValueOnce(new Error("invalid request"));
postJsonRequestMock.mockImplementation(requestSequence);
const provider = buildGoogleSpeechProvider();
await expect(
provider.synthesize({
text: "Do not retry this.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
},
target: "audio-file",
timeoutMs: 5_000,
}),
).rejects.toThrow("invalid request");
expect(requestSequence).toHaveBeenCalledTimes(1);
});
it("falls back to GEMINI_API_KEY and configured Google API base URL", async () => {
vi.stubEnv("GEMINI_API_KEY", "env-google-key");
const requestMock = installGoogleTtsRequestMock();
const provider = buildGoogleSpeechProvider();
expect(provider.isConfigured({ providerConfig: {}, timeoutMs: 1 })).toBe(true);
await provider.synthesize({
text: "Read this plainly.",
cfg: {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
models: [],
},
},
},
},
providerConfig: {},
target: "voice-note",
timeoutMs: 10_000,
});
const request = expectRecordFields(requireFirstRecordArg(requestMock, "Google TTS request"), {
url: "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-tts-preview:generateContent",
}) as { headers?: HeadersInit };
expect(new Headers(request.headers).get("x-goog-api-key")).toBe("env-google-key");
});
it("can reuse a configured Google model-provider API key without auth profiles", async () => {
const requestMock = installGoogleTtsRequestMock();
const provider = buildGoogleSpeechProvider();
const cfg = {
models: {
providers: {
google: {
apiKey: "model-provider-google-key",
baseUrl: "https://generativelanguage.googleapis.com",
models: [],
},
},
},
};
expect(provider.isConfigured({ cfg, providerConfig: {}, timeoutMs: 1 })).toBe(true);
await provider.synthesize({
text: "Use the configured model provider key.",
cfg,
providerConfig: {},
target: "audio-file",
timeoutMs: 10_000,
});
const request = requireFirstRecordArg(requestMock, "Google TTS request") as {
headers?: HeadersInit;
};
expect(new Headers(request.headers).get("x-goog-api-key")).toBe("model-provider-google-key");
});
it("returns Gemini PCM directly for telephony synthesis", async () => {
const pcm = Buffer.from([3, 0, 4, 0]);
installGoogleTtsRequestMock(pcm);
const provider = buildGoogleSpeechProvider();
const result = await provider.synthesizeTelephony?.({
text: "Phone call audio.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
model: "google/gemini-3.1-flash-tts",
voice: "Kore",
audioProfile: "Speak calmly.",
speakerName: "Default speaker",
},
providerOverrides: {
model: "google/gemini-3.1-pro-tts",
voiceName: "Puck",
audioProfile: "Speak brightly.",
speakerName: "Override speaker",
},
timeoutMs: 5_000,
});
const request = expectRecordFields(
requireFirstRecordArg(postJsonRequestMock, "Google telephony TTS request"),
{
url: "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-pro-tts:generateContent",
},
);
const body = request.body as {
contents?: unknown;
generationConfig?: { speechConfig?: unknown };
};
expect(body.contents).toEqual([
{
role: "user",
parts: [{ text: "Speak brightly.\n\nSpeaker name: Override speaker\n\nPhone call audio." }],
},
]);
expect(body.generationConfig?.speechConfig).toEqual({
voiceConfig: {
prebuiltVoiceConfig: {
voiceName: "Puck",
},
},
});
expect(result).toEqual({
audioBuffer: pcm,
outputFormat: "pcm",
sampleRate: 24_000,
});
});
it("prepends configured Gemini TTS profile text", async () => {
const requestMock = installGoogleTtsRequestMock();
const provider = buildGoogleSpeechProvider();
await provider.synthesize({
text: "Status update starts now.",
cfg: {},
providerConfig: {
apiKey: "google-test-key",
audioProfile: "Speak professionally with a calm executive tone.",
speakerName: "Alex",
},
target: "audio-file",
timeoutMs: 10_000,
});
const request = requireFirstRecordArg(requestMock, "Google TTS request") as {
body?: { contents?: Array<{ parts?: Array<{ text?: string }> }> };
};
expect(request.body?.contents?.[0]?.parts?.[0]?.text).toBe(
"Speak professionally with a calm executive tone.\n\n" +
"Speaker name: Alex\n\n" +
"Status update starts now.",
);
});
it("resolves provider config and directive overrides", () => {
const provider = buildGoogleSpeechProvider();
expect(
provider.resolveConfig?.({
cfg: {},
rawConfig: {
providers: {
google: {
apiKey: "configured-key",
model: "google/gemini-3.1-flash-tts-preview",
voice: "Leda",
audioProfile: "Speak warmly.",
speakerName: "Narrator",
},
},
},
timeoutMs: 1,
}),
).toEqual({
apiKey: "configured-key",
audioProfile: "Speak warmly.",
baseUrl: undefined,
model: "gemini-3.1-flash-tts-preview",
speakerName: "Narrator",
voiceName: "Leda",
});
expect(
provider.parseDirectiveToken?.({
key: "google_voice",
value: "Aoede",
policy: {
enabled: true,
allowText: true,
allowProvider: true,
allowVoice: true,
allowModelId: true,
allowVoiceSettings: true,
allowNormalization: true,
allowSeed: true,
},
}),
).toEqual({
handled: true,
overrides: {
voiceName: "Aoede",
},
});
expect(
provider.parseDirectiveToken?.({
key: "google_model",
value: "gemini-3.1-flash-tts-preview",
policy: {
enabled: true,
allowText: true,
allowProvider: true,
allowVoice: true,
allowModelId: true,
allowVoiceSettings: true,
allowNormalization: true,
allowSeed: true,
},
}),
).toEqual({
handled: true,
overrides: {
model: "gemini-3.1-flash-tts-preview",
},
});
});
it("lists Gemini prebuilt TTS voices", async () => {
const provider = buildGoogleSpeechProvider();
const voices = await provider.listVoices?.({ providerConfig: {} });
const voiceLabels = voices?.map((voice) => `${voice.id}:${voice.name}`);
expect(voiceLabels).toContain("Kore:Kore");
expect(voiceLabels).toContain("Puck:Puck");
});
it("formats Google TTS HTTP errors with provider details", async () => {
assertOkOrThrowProviderErrorMock.mockRejectedValue(
new Error(
"Google TTS failed (429): Quota exceeded [code=RESOURCE_EXHAUSTED] [request_id=google_req_123]",
),
);
postJsonRequestMock.mockResolvedValue({
response: new Response(
JSON.stringify({
error: {
message: "Quota exceeded",
status: "RESOURCE_EXHAUSTED",
},
}),
{
status: 429,
headers: { "x-request-id": "google_req_123" },
},
),
release: vi.fn(async () => {}),
});
const provider = buildGoogleSpeechProvider();
await expect(
provider.synthesize({
text: "Read this plainly.",
cfg: {},
providerConfig: { apiKey: "google-test-key" },
target: "audio-file",
timeoutMs: 10_000,
}),
).rejects.toThrow(
"Google TTS failed (429): Quota exceeded [code=RESOURCE_EXHAUSTED] [request_id=google_req_123]",
);
});
it("honors configured private-network opt-in for Google TTS", async () => {
installGoogleTtsRequestMock();
const provider = buildGoogleSpeechProvider();
await provider.synthesize({
text: "hello",
cfg: {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
request: { allowPrivateNetwork: true },
models: [],
},
},
},
},
providerConfig: { apiKey: "google-test-key" },
target: "audio-file",
timeoutMs: 12_345,
});
const requestConfig = expectRecordFields(
requireFirstRecordArg(resolveProviderHttpRequestConfigMock, "Google TTS HTTP config request"),
{
allowPrivateNetwork: true,
},
);
expectRecordFields(requestConfig.request, { allowPrivateNetwork: true });
});
it("honors configured private-network opt-in for Google telephony TTS", async () => {
installGoogleTtsRequestMock();
const provider = buildGoogleSpeechProvider();
await provider.synthesizeTelephony?.({
text: "hello",
cfg: {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
request: { allowPrivateNetwork: true },
models: [],
},
},
},
},
providerConfig: { apiKey: "google-test-key" },
timeoutMs: 12_345,
});
const requestConfig = expectRecordFields(
requireFirstRecordArg(resolveProviderHttpRequestConfigMock, "Google TTS HTTP config request"),
{
allowPrivateNetwork: true,
},
);
expectRecordFields(requestConfig.request, { allowPrivateNetwork: true });
});
});

View File

@@ -0,0 +1,677 @@
// Google provider module implements model/runtime integration.
import { transcodeAudioBufferToOpus } from "openclaw/plugin-sdk/media-runtime";
import {
assertOkOrThrowProviderError,
postJsonRequest,
readProviderJsonResponse,
sanitizeConfiguredModelProviderRequest,
} from "openclaw/plugin-sdk/provider-http";
import type { OpenClawConfig } from "openclaw/plugin-sdk/provider-onboard";
import { normalizeResolvedSecretInputString } from "openclaw/plugin-sdk/secret-input";
import type {
SpeechDirectiveTokenParseContext,
SpeechProviderConfig,
SpeechProviderOverrides,
SpeechProviderPlugin,
SpeechSynthesisRequest,
} from "openclaw/plugin-sdk/speech-core";
import { asObject, trimToUndefined } from "openclaw/plugin-sdk/speech-core";
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolveGoogleGenerativeAiHttpRequestConfig } from "./api.js";
const DEFAULT_GOOGLE_TTS_MODEL = "gemini-3.1-flash-tts-preview";
const DEFAULT_GOOGLE_TTS_VOICE = "Kore";
const GOOGLE_TTS_SAMPLE_RATE = 24_000;
const GOOGLE_TTS_CHANNELS = 1;
const GOOGLE_TTS_BITS_PER_SAMPLE = 16;
const GOOGLE_AUDIO_PROFILE_PROMPT_TEMPLATE = "audio-profile-v1";
const GOOGLE_TTS_MODELS = [
"gemini-3.1-flash-tts-preview",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
] as const;
const GOOGLE_TTS_VOICES = [
"Zephyr",
"Puck",
"Charon",
"Kore",
"Fenrir",
"Leda",
"Orus",
"Aoede",
"Callirrhoe",
"Autonoe",
"Enceladus",
"Iapetus",
"Umbriel",
"Algieba",
"Despina",
"Erinome",
"Algenib",
"Rasalgethi",
"Laomedeia",
"Achernar",
"Alnilam",
"Schedar",
"Gacrux",
"Pulcherrima",
"Achird",
"Zubenelgenubi",
"Vindemiatrix",
"Sadachbia",
"Sadaltager",
"Sulafat",
] as const;
type GoogleTtsProviderConfig = {
apiKey?: string;
baseUrl?: string;
model: string;
voiceName: string;
audioProfile?: string;
speakerName?: string;
promptTemplate?: typeof GOOGLE_AUDIO_PROFILE_PROMPT_TEMPLATE;
personaPrompt?: string;
};
type GoogleTtsProviderOverrides = {
model?: string;
voiceName?: string;
audioProfile?: string;
speakerName?: string;
};
type Maybe<T> = T | undefined;
type GoogleInlineDataPart = {
mimeType?: string;
mime_type?: string;
data?: string;
};
type GoogleGenerateSpeechResponse = {
candidates?: Array<{
content?: {
parts?: Array<{
text?: string;
inlineData?: GoogleInlineDataPart;
inline_data?: GoogleInlineDataPart;
}>;
};
}>;
};
class GoogleTtsRetryableError extends Error {
constructor(message: string) {
super(message);
this.name = "GoogleTtsRetryableError";
}
}
function isGoogleTtsRetryableError(err: unknown): boolean {
if (err instanceof GoogleTtsRetryableError) {
return true;
}
if (!(err instanceof Error)) {
return false;
}
if (err.name === "AbortError") {
return true;
}
const message = err.message.toLowerCase();
return (
message.includes("aborted") ||
message.includes("timeout") ||
message.includes("fetch failed") ||
message.includes("network")
);
}
function normalizeGoogleTtsModel(model: unknown): string {
const trimmed = normalizeOptionalString(model);
if (!trimmed) {
return DEFAULT_GOOGLE_TTS_MODEL;
}
const withoutProvider = trimmed.startsWith("google/") ? trimmed.slice("google/".length) : trimmed;
return withoutProvider === "gemini-3.1-flash-tts" ? DEFAULT_GOOGLE_TTS_MODEL : withoutProvider;
}
function normalizeGoogleTtsVoiceName(voiceName: unknown): string {
return normalizeOptionalString(voiceName) ?? DEFAULT_GOOGLE_TTS_VOICE;
}
function normalizeGooglePromptTemplate(
value: unknown,
): typeof GOOGLE_AUDIO_PROFILE_PROMPT_TEMPLATE | undefined {
const trimmed = normalizeOptionalString(value);
if (!trimmed) {
return undefined;
}
if (trimmed === GOOGLE_AUDIO_PROFILE_PROMPT_TEMPLATE) {
return trimmed;
}
throw new Error(`Invalid Google TTS promptTemplate: ${trimmed}`);
}
function resolveGoogleTtsEnvApiKey(): string | undefined {
return (
normalizeOptionalString(process.env.GEMINI_API_KEY) ??
normalizeOptionalString(process.env.GOOGLE_API_KEY)
);
}
function resolveGoogleTtsModelProviderApiKey(cfg?: OpenClawConfig): string | undefined {
return normalizeResolvedSecretInputString({
value: cfg?.models?.providers?.google?.apiKey,
path: "models.providers.google.apiKey",
});
}
function resolveGoogleTtsApiKey(params: {
cfg?: OpenClawConfig;
providerConfig: SpeechProviderConfig;
}): string | undefined {
return (
readGoogleTtsProviderConfig(params.providerConfig).apiKey ??
resolveGoogleTtsModelProviderApiKey(params.cfg) ??
resolveGoogleTtsEnvApiKey()
);
}
function resolveGoogleTtsBaseUrl(params: {
cfg?: OpenClawConfig;
providerConfig: GoogleTtsProviderConfig;
}): string | undefined {
return (
params.providerConfig.baseUrl ?? trimToUndefined(params.cfg?.models?.providers?.google?.baseUrl)
);
}
function resolveGoogleTtsConfigRecord(
rawConfig: Record<string, unknown>,
): Record<string, unknown> | undefined {
const providers = asObject(rawConfig.providers);
return asObject(providers?.google) ?? asObject(rawConfig.google);
}
function normalizeGoogleTtsProviderConfig(
rawConfig: Record<string, unknown>,
): GoogleTtsProviderConfig {
const raw = resolveGoogleTtsConfigRecord(rawConfig);
const promptTemplate = normalizeGooglePromptTemplate(raw?.promptTemplate);
const personaPrompt = trimToUndefined(raw?.personaPrompt);
return {
apiKey: normalizeResolvedSecretInputString({
value: raw?.apiKey,
path: "messages.tts.providers.google.apiKey",
}),
baseUrl: trimToUndefined(raw?.baseUrl),
model: normalizeGoogleTtsModel(raw?.model),
voiceName: normalizeGoogleTtsVoiceName(raw?.voiceName ?? raw?.voice),
audioProfile: trimToUndefined(raw?.audioProfile),
speakerName: trimToUndefined(raw?.speakerName),
...(promptTemplate ? { promptTemplate } : {}),
...(personaPrompt ? { personaPrompt } : {}),
};
}
function readGoogleTtsProviderConfig(config: SpeechProviderConfig): GoogleTtsProviderConfig {
const normalized = normalizeGoogleTtsProviderConfig({});
const promptTemplate =
normalizeGooglePromptTemplate(config.promptTemplate) ?? normalized.promptTemplate;
const personaPrompt = trimToUndefined(config.personaPrompt) ?? normalized.personaPrompt;
return {
apiKey: trimToUndefined(config.apiKey) ?? normalized.apiKey,
baseUrl: trimToUndefined(config.baseUrl) ?? normalized.baseUrl,
model: normalizeGoogleTtsModel(config.model ?? normalized.model),
voiceName: normalizeGoogleTtsVoiceName(
config.voiceName ?? config.voice ?? normalized.voiceName,
),
audioProfile: trimToUndefined(config.audioProfile) ?? normalized.audioProfile,
speakerName: trimToUndefined(config.speakerName) ?? normalized.speakerName,
...(promptTemplate ? { promptTemplate } : {}),
...(personaPrompt ? { personaPrompt } : {}),
};
}
function readGoogleTtsOverrides(
overrides: Maybe<SpeechProviderOverrides>,
): GoogleTtsProviderOverrides {
if (!overrides) {
return {};
}
return {
model: normalizeOptionalString(overrides.model),
voiceName: normalizeOptionalString(overrides.voiceName ?? overrides.voice),
audioProfile: normalizeOptionalString(overrides.audioProfile),
speakerName: normalizeOptionalString(overrides.speakerName),
};
}
function composeGoogleTtsText(params: {
text: string;
audioProfile?: string;
speakerName?: string;
}): string {
return [
trimToUndefined(params.audioProfile),
trimToUndefined(params.speakerName) ? `Speaker name: ${params.speakerName}` : undefined,
params.text,
]
.filter((part): part is string => part !== undefined)
.join("\n\n");
}
function parseDirectiveToken(ctx: SpeechDirectiveTokenParseContext): {
handled: boolean;
overrides?: SpeechProviderOverrides;
warnings?: string[];
} {
switch (ctx.key) {
case "voicename":
case "voice_name":
case "google_voice":
case "googlevoice":
if (!ctx.policy.allowVoice) {
return { handled: true };
}
return { handled: true, overrides: { voiceName: ctx.value } };
case "google_model":
case "googlemodel":
if (!ctx.policy.allowModelId) {
return { handled: true };
}
return { handled: true, overrides: { model: ctx.value } };
default:
return { handled: false };
}
}
function extractGoogleSpeechPcm(payload: GoogleGenerateSpeechResponse): Buffer {
for (const candidate of payload.candidates ?? []) {
for (const part of candidate.content?.parts ?? []) {
const inline = part.inlineData ?? part.inline_data;
const data = normalizeOptionalString(inline?.data);
if (!data) {
continue;
}
return Buffer.from(data, "base64");
}
}
throw new Error("Google TTS response missing audio data");
}
function normalizePromptSectionText(value: string | undefined): string | undefined {
const trimmed = trimToUndefined(value?.replace(/\r\n?/g, "\n"));
if (!trimmed) {
return undefined;
}
let sanitized = "";
for (const char of trimmed) {
const code = char.charCodeAt(0);
if (
(code >= 0 && code <= 8) ||
code === 11 ||
code === 12 ||
(code >= 14 && code <= 31) ||
code === 127
) {
continue;
}
sanitized += char;
}
return sanitized;
}
function normalizePromptList(values: readonly string[] | undefined): string[] {
return (values ?? [])
.map((value) => normalizePromptSectionText(value))
.filter((value): value is string => Boolean(value));
}
function isOpenClawGoogleAudioProfilePrompt(text: string): boolean {
return (
text.includes("# AUDIO PROFILE:") &&
text.includes("### TRANSCRIPT") &&
text.startsWith("Synthesize speech from the TRANSCRIPT section only.")
);
}
function renderGoogleAudioProfilePrompt(params: {
text: string;
persona?: {
id: string;
label?: string;
prompt?: {
profile?: string;
scene?: string;
sampleContext?: string;
style?: string;
accent?: string;
pacing?: string;
constraints?: string[];
};
};
personaPrompt?: string;
}): string {
const transcript = params.text.replace(/\r\n?/g, "\n").trim();
const prompt = params.persona?.prompt;
const profile = normalizePromptSectionText(prompt?.profile);
const scene = normalizePromptSectionText(prompt?.scene);
const sampleContext = normalizePromptSectionText(prompt?.sampleContext);
const style = normalizePromptSectionText(prompt?.style);
const accent = normalizePromptSectionText(prompt?.accent);
const pacing = normalizePromptSectionText(prompt?.pacing);
const constraints = normalizePromptList(prompt?.constraints);
const personaPrompt = normalizePromptSectionText(params.personaPrompt);
const label =
normalizePromptSectionText(params.persona?.label) ??
normalizePromptSectionText(params.persona?.id);
const sections = [
[
"Synthesize speech from the TRANSCRIPT section only. Use the other sections only",
"as performance direction. Do not read section titles, notes, labels, or",
"configuration aloud.",
].join("\n"),
];
if (label || profile) {
sections.push([`# AUDIO PROFILE: ${label ?? "voice"}`, profile].filter(Boolean).join("\n"));
}
if (scene) {
sections.push(["## THE SCENE", scene].join("\n"));
}
const directorNotes: string[] = [];
if (style) {
directorNotes.push(`Style: ${style}`);
}
if (accent) {
directorNotes.push(`Accent: ${accent}`);
}
if (pacing) {
directorNotes.push(`Pacing: ${pacing}`);
}
if (constraints.length > 0) {
directorNotes.push(["Constraints:", ...constraints.map((item) => `- ${item}`)].join("\n"));
}
if (personaPrompt) {
directorNotes.push(["Provider notes:", personaPrompt].join("\n"));
}
if (directorNotes.length > 0) {
sections.push(["### DIRECTOR'S NOTES", ...directorNotes].join("\n"));
}
if (sampleContext) {
sections.push(["### SAMPLE CONTEXT", sampleContext].join("\n"));
}
sections.push(["### TRANSCRIPT", transcript].join("\n"));
return sections.join("\n\n");
}
function wrapPcm16MonoToWav(pcm: Buffer, sampleRate = GOOGLE_TTS_SAMPLE_RATE): Buffer {
const byteRate = sampleRate * GOOGLE_TTS_CHANNELS * (GOOGLE_TTS_BITS_PER_SAMPLE / 8);
const blockAlign = GOOGLE_TTS_CHANNELS * (GOOGLE_TTS_BITS_PER_SAMPLE / 8);
const header = Buffer.alloc(44);
header.write("RIFF", 0, "ascii");
header.writeUInt32LE(36 + pcm.length, 4);
header.write("WAVE", 8, "ascii");
header.write("fmt ", 12, "ascii");
header.writeUInt32LE(16, 16);
header.writeUInt16LE(1, 20);
header.writeUInt16LE(GOOGLE_TTS_CHANNELS, 22);
header.writeUInt32LE(sampleRate, 24);
header.writeUInt32LE(byteRate, 28);
header.writeUInt16LE(blockAlign, 32);
header.writeUInt16LE(GOOGLE_TTS_BITS_PER_SAMPLE, 34);
header.write("data", 36, "ascii");
header.writeUInt32LE(pcm.length, 40);
return Buffer.concat([header, pcm]);
}
async function synthesizeGoogleTtsPcmOnce(params: {
text: string;
apiKey: string;
baseUrl?: string;
request?: ReturnType<typeof sanitizeConfiguredModelProviderRequest>;
model: string;
voiceName: string;
audioProfile?: string;
speakerName?: string;
timeoutMs: number;
}): Promise<Buffer> {
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
resolveGoogleGenerativeAiHttpRequestConfig({
apiKey: params.apiKey,
baseUrl: params.baseUrl,
request: params.request,
capability: "audio",
transport: "http",
});
const { response: res, release } = await postJsonRequest({
url: `${baseUrl}/models/${params.model}:generateContent`,
headers,
body: {
contents: [
{
role: "user",
parts: [
{
text: composeGoogleTtsText({
text: params.text,
audioProfile: params.audioProfile,
speakerName: params.speakerName,
}),
},
],
},
],
generationConfig: {
responseModalities: ["AUDIO"],
speechConfig: {
voiceConfig: {
prebuiltVoiceConfig: {
voiceName: params.voiceName,
},
},
},
},
},
timeoutMs: params.timeoutMs,
fetchFn: fetch,
pinDns: false,
allowPrivateNetwork,
dispatcherPolicy,
});
try {
if (!res.ok) {
try {
await assertOkOrThrowProviderError(res, "Google TTS failed");
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
if (res.status >= 500 && res.status < 600) {
throw new GoogleTtsRetryableError(message);
}
throw err;
}
}
try {
const payload = await readProviderJsonResponse<GoogleGenerateSpeechResponse>(
res,
"Google TTS response",
);
return extractGoogleSpeechPcm(payload);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
throw new GoogleTtsRetryableError(message);
}
} finally {
await release();
}
}
async function synthesizeGoogleTtsPcm(params: {
text: string;
apiKey: string;
baseUrl?: string;
request?: ReturnType<typeof sanitizeConfiguredModelProviderRequest>;
model: string;
voiceName: string;
audioProfile?: string;
speakerName?: string;
timeoutMs: number;
}): Promise<Buffer> {
let lastError: unknown;
for (let attempt = 0; attempt < 2; attempt += 1) {
try {
return await synthesizeGoogleTtsPcmOnce(params);
} catch (err) {
lastError = err;
if (!isGoogleTtsRetryableError(err) || attempt > 0) {
throw err;
}
}
}
throw lastError instanceof Error ? lastError : new Error(String(lastError));
}
type GoogleTtsSynthesisRequest = Pick<
SpeechSynthesisRequest,
"cfg" | "providerConfig" | "providerOverrides" | "text" | "timeoutMs"
>;
async function synthesizeConfiguredGoogleTts(req: GoogleTtsSynthesisRequest): Promise<Buffer> {
const config = readGoogleTtsProviderConfig(req.providerConfig);
const overrides = readGoogleTtsOverrides(req.providerOverrides);
const apiKey = resolveGoogleTtsApiKey({
cfg: req.cfg,
providerConfig: req.providerConfig,
});
if (!apiKey) {
throw new Error("Google API key missing");
}
return synthesizeGoogleTtsPcm({
text: req.text,
apiKey,
baseUrl: resolveGoogleTtsBaseUrl({ cfg: req.cfg, providerConfig: config }),
request: sanitizeConfiguredModelProviderRequest(req.cfg?.models?.providers?.google?.request),
model: normalizeGoogleTtsModel(overrides.model ?? config.model),
voiceName: normalizeGoogleTtsVoiceName(overrides.voiceName ?? config.voiceName),
audioProfile: overrides.audioProfile ?? config.audioProfile,
speakerName: overrides.speakerName ?? config.speakerName,
timeoutMs: req.timeoutMs,
});
}
export function buildGoogleSpeechProvider(): SpeechProviderPlugin {
return {
id: "google",
label: "Google",
autoSelectOrder: 50,
defaultModel: DEFAULT_GOOGLE_TTS_MODEL,
models: GOOGLE_TTS_MODELS,
voices: GOOGLE_TTS_VOICES,
resolveConfig: ({ rawConfig }) => normalizeGoogleTtsProviderConfig(rawConfig),
parseDirectiveToken,
resolveTalkConfig: ({ baseTtsConfig, talkProviderConfig }) => {
const base = normalizeGoogleTtsProviderConfig(baseTtsConfig);
return {
...base,
...(talkProviderConfig.apiKey === undefined
? {}
: {
apiKey: normalizeResolvedSecretInputString({
value: talkProviderConfig.apiKey,
path: "talk.providers.google.apiKey",
}),
}),
...(trimToUndefined(talkProviderConfig.baseUrl) == null
? {}
: { baseUrl: trimToUndefined(talkProviderConfig.baseUrl) }),
...(trimToUndefined(talkProviderConfig.modelId) == null
? {}
: { model: normalizeGoogleTtsModel(talkProviderConfig.modelId) }),
...(trimToUndefined(talkProviderConfig.voiceId) == null
? {}
: { voiceName: normalizeGoogleTtsVoiceName(talkProviderConfig.voiceId) }),
};
},
resolveTalkOverrides: ({ params }) => ({
...(trimToUndefined(params.voiceId) == null
? {}
: { voiceName: normalizeGoogleTtsVoiceName(params.voiceId) }),
...(trimToUndefined(params.modelId) == null
? {}
: { model: normalizeGoogleTtsModel(params.modelId) }),
}),
listVoices: async () => GOOGLE_TTS_VOICES.map((voice) => ({ id: voice, name: voice })),
isConfigured: ({ cfg, providerConfig }) =>
Boolean(resolveGoogleTtsApiKey({ cfg, providerConfig })),
prepareSynthesis: (ctx) => {
const config = readGoogleTtsProviderConfig(ctx.providerConfig);
const shouldWrap =
config.promptTemplate === GOOGLE_AUDIO_PROFILE_PROMPT_TEMPLATE ||
Boolean(config.personaPrompt);
if (!shouldWrap || isOpenClawGoogleAudioProfilePrompt(ctx.text)) {
return undefined;
}
return {
text: renderGoogleAudioProfilePrompt({
text: ctx.text,
persona: ctx.persona,
personaPrompt: config.personaPrompt,
}),
};
},
synthesize: async (req) => {
const pcm = await synthesizeConfiguredGoogleTts(req);
if (req.target === "voice-note") {
return {
audioBuffer: await transcodeAudioBufferToOpus({
audioBuffer: wrapPcm16MonoToWav(pcm),
inputExtension: "wav",
tempPrefix: "tts-google-",
timeoutMs: req.timeoutMs,
}),
outputFormat: "opus",
fileExtension: ".opus",
voiceCompatible: true,
};
}
return {
audioBuffer: wrapPcm16MonoToWav(pcm),
outputFormat: "wav",
fileExtension: ".wav",
voiceCompatible: false,
};
},
synthesizeTelephony: async (req) => {
const pcm = await synthesizeConfiguredGoogleTts(req);
return {
audioBuffer: pcm,
outputFormat: "pcm",
sampleRate: GOOGLE_TTS_SAMPLE_RATE,
};
},
};
}
export const testing = {
DEFAULT_GOOGLE_TTS_MODEL,
DEFAULT_GOOGLE_TTS_VOICE,
GOOGLE_AUDIO_PROFILE_PROMPT_TEMPLATE,
GOOGLE_TTS_MODELS,
GOOGLE_TTS_SAMPLE_RATE,
normalizeGoogleTtsModel,
renderGoogleAudioProfilePrompt,
wrapPcm16MonoToWav,
};
export { testing as __testing };

View File

@@ -0,0 +1,375 @@
// Google provider module implements model/runtime integration.
import {
createProviderHttpError,
formatProviderHttpErrorMessage,
readProviderJsonObjectResponse,
} from "openclaw/plugin-sdk/provider-http";
import {
buildSearchCacheKey,
buildUnsupportedSearchFilterResponse,
DEFAULT_SEARCH_COUNT,
MAX_SEARCH_COUNT,
parseWebSearchTimeFilters,
readCachedSearchPayload,
readConfiguredSecretString,
readPositiveIntegerParam,
readProviderEnvValue,
readStringParam,
resolveCitationRedirectUrl,
resolveSearchCacheTtlMs,
resolveSearchCount,
resolveSearchTimeoutSeconds,
type SearchConfigRecord,
withTrustedWebSearchEndpoint,
wrapWebContent,
writeCachedSearchPayload,
} from "openclaw/plugin-sdk/provider-web-search";
import { isRecord } from "openclaw/plugin-sdk/string-coerce-runtime";
import {
resolveGeminiConfig,
resolveGeminiBaseUrl,
resolveGeminiModel,
type GeminiConfig,
} from "./gemini-web-search-provider.shared.js";
type GeminiFreshness = "day" | "week" | "month" | "year";
type GeminiTimeRangeFilter = {
startTime: string;
endTime: string;
};
type GeminiGroundingResponse = {
candidates?: Array<{
content?: {
parts?: Array<{
text?: string;
}>;
};
groundingMetadata?: {
groundingChunks?: Array<{
web?: {
uri?: string;
title?: string;
};
}>;
};
}>;
error?: {
code?: number;
message?: string;
status?: string;
};
};
function throwMalformedGeminiResponse(): never {
throw new Error("Gemini API error: malformed JSON response");
}
const GEMINI_FRESHNESS_DAYS: Record<GeminiFreshness, number> = {
day: 1,
week: 7,
month: 30,
year: 365,
};
const GEMINI_DAY_FRESHNESS_HINT = "Prioritize web sources published in the last 24 hours.";
// Gemini's google_search.time_range_filter accepts second-precision RFC 3339
// only. Despite the underlying google.protobuf.Timestamp type accepting "0, 3,
// 6 or 9 fractional digits", the Search grounding endpoint rejects any
// non-zero fractional component with
// "[FIELD_INVALID] Granularity of nano is not supported".
// Strip the fractional-second component before serializing.
function toGeminiTimeRangeTimestamp(date: Date): string {
return date.toISOString().replace(/\.\d+Z$/, "Z");
}
function isoDateStart(value: string): string {
return `${value}T00:00:00Z`;
}
function isoDateExclusiveEnd(value: string): string {
const end = new Date(`${value}T00:00:00Z`);
end.setUTCDate(end.getUTCDate() + 1);
return toGeminiTimeRangeTimestamp(end);
}
function freshnessStartTime(freshness: GeminiFreshness, now: Date): string {
const start = new Date(now);
start.setUTCDate(start.getUTCDate() - GEMINI_FRESHNESS_DAYS[freshness]);
return toGeminiTimeRangeTimestamp(start);
}
function queryWithSoftFreshness(query: string, freshness?: "day"): string {
if (freshness !== "day") {
return query;
}
return `${query}\n\nSearch recency instruction: ${GEMINI_DAY_FRESHNESS_HINT} If no matching recent sources are available, state that limitation and use the most relevant available sources.`;
}
function resolveGeminiTimeRangeFilter(
args: Record<string, unknown>,
now = new Date(),
):
| { timeRangeFilter?: GeminiTimeRangeFilter; freshness?: "day" }
| {
error:
| "invalid_freshness"
| "invalid_date"
| "invalid_date_range"
| "conflicting_time_filters";
message: string;
docs: string;
} {
const rawFreshness = readStringParam(args, "freshness");
const rawDateAfter = readStringParam(args, "date_after");
const rawDateBefore = readStringParam(args, "date_before");
const parsedTimeFilters = parseWebSearchTimeFilters({
rawDateAfter,
rawDateBefore,
rawFreshness,
freshnessProvider: "perplexity",
invalidFreshnessMessage:
"freshness must be day, week, month, year, or the shortcuts pd, pw, pm, py.",
invalidDateAfterMessage: "date_after must be YYYY-MM-DD format.",
invalidDateBeforeMessage: "date_before must be YYYY-MM-DD format.",
invalidDateRangeMessage: "date_after must be before date_before.",
});
if ("error" in parsedTimeFilters) {
return parsedTimeFilters;
}
const { freshness, dateAfter, dateBefore } = parsedTimeFilters;
if (freshness) {
// Gemini rejects 24-hour google_search.timeRangeFilter windows, while
// wider freshness windows still preserve the hard grounding contract.
if (freshness === "day") {
return {
freshness,
};
}
return {
timeRangeFilter: {
startTime: freshnessStartTime(freshness, now),
endTime: toGeminiTimeRangeTimestamp(now),
},
};
}
if (!dateAfter && !dateBefore) {
return {};
}
return {
timeRangeFilter: {
startTime: dateAfter ? isoDateStart(dateAfter) : "1970-01-01T00:00:00Z",
endTime: dateBefore ? isoDateExclusiveEnd(dateBefore) : toGeminiTimeRangeTimestamp(now),
},
};
}
export function resolveGeminiRuntimeApiKey(gemini?: GeminiConfig): string | undefined {
return (
readConfiguredSecretString(gemini?.apiKey, "tools.web.search.gemini.apiKey") ??
readProviderEnvValue(["GEMINI_API_KEY"]) ??
readConfiguredSecretString(gemini?.providerApiKey, "models.providers.google.apiKey")
);
}
async function runGeminiSearch(params: {
query: string;
apiKey: string;
baseUrl: string;
model: string;
timeoutSeconds: number;
signal?: AbortSignal;
timeRangeFilter?: GeminiTimeRangeFilter;
}): Promise<{ content: string; citations: Array<{ url: string; title?: string }> }> {
const endpoint = `${params.baseUrl}/models/${params.model}:generateContent`;
const googleSearch =
params.timeRangeFilter === undefined ? {} : { timeRangeFilter: params.timeRangeFilter };
return withTrustedWebSearchEndpoint(
{
url: endpoint,
timeoutSeconds: params.timeoutSeconds,
signal: params.signal,
init: {
method: "POST",
headers: {
"Content-Type": "application/json",
"x-goog-api-key": params.apiKey,
},
body: JSON.stringify({
contents: [{ parts: [{ text: params.query }] }],
tools: [{ google_search: googleSearch }],
}),
},
},
async (res) => {
if (!res.ok) {
const error = await createProviderHttpError(res, "Gemini API error");
throw new Error(error.message.replace(/key=[^&\s]+/giu, "key=***"));
}
const data = (await readProviderJsonObjectResponse(
res,
"Gemini API error",
)) as GeminiGroundingResponse;
if (data.error) {
const rawMessage = data.error.message || data.error.status || "unknown";
throw new Error(
formatProviderHttpErrorMessage({
label: "Gemini API error",
status: data.error.code ?? 0,
detail: rawMessage.replace(/key=[^&\s]+/giu, "key=***"),
}),
);
}
if (!Array.isArray(data.candidates)) {
throwMalformedGeminiResponse();
}
const candidate = data.candidates[0];
if (!isRecord(candidate) || !isRecord(candidate.content)) {
throwMalformedGeminiResponse();
}
const parts = candidate.content.parts;
if (!Array.isArray(parts)) {
throwMalformedGeminiResponse();
}
const content = parts
.map((part) => (isRecord(part) && typeof part.text === "string" ? part.text : undefined))
.filter((text): text is string => Boolean(text))
.join("\n");
if (!content) {
throwMalformedGeminiResponse();
}
const groundingMetadata = candidate.groundingMetadata;
const groundingChunks =
groundingMetadata === undefined
? []
: isRecord(groundingMetadata)
? groundingMetadata.groundingChunks === undefined
? []
: Array.isArray(groundingMetadata.groundingChunks)
? groundingMetadata.groundingChunks
: undefined
: undefined;
if (!groundingChunks) {
throwMalformedGeminiResponse();
}
const rawCitations = groundingChunks.flatMap((chunk) => {
if (!isRecord(chunk) || !isRecord(chunk.web) || typeof chunk.web.uri !== "string") {
return [];
}
return [
{
url: chunk.web.uri,
title: typeof chunk.web.title === "string" ? chunk.web.title : undefined,
},
];
});
const citations: Array<{ url: string; title?: string }> = [];
for (let index = 0; index < rawCitations.length; index += 10) {
const batch = rawCitations.slice(index, index + 10);
const resolved = await Promise.all(
batch.map(async (citation) =>
Object.assign({}, citation, { url: await resolveCitationRedirectUrl(citation.url) }),
),
);
citations.push(...resolved);
}
return { content, citations };
},
);
}
export async function executeGeminiSearch(
args: Record<string, unknown>,
searchConfig?: SearchConfigRecord,
context?: { signal?: AbortSignal },
): Promise<Record<string, unknown>> {
const unsupportedResponse = buildUnsupportedSearchFilterResponse(
{
country: args.country,
language: args.language,
},
"gemini",
);
if (unsupportedResponse) {
return unsupportedResponse;
}
const timeRange = resolveGeminiTimeRangeFilter(args);
if ("error" in timeRange) {
return timeRange;
}
const geminiConfig = resolveGeminiConfig(searchConfig);
const apiKey = resolveGeminiRuntimeApiKey(geminiConfig);
if (!apiKey) {
return {
error: "missing_gemini_api_key",
message:
"web_search (gemini) needs an API key. Set GEMINI_API_KEY in the Gateway environment, configure plugins.entries.google.config.webSearch.apiKey, or reuse models.providers.google.apiKey. If you do not want to configure a search API key, use web_fetch for a specific URL or the browser tool for interactive pages.",
docs: "https://docs.openclaw.ai/tools/web",
};
}
const query = readStringParam(args, "query", { required: true });
const count =
readPositiveIntegerParam(args, "count", {
max: MAX_SEARCH_COUNT,
message: `count must be an integer from 1 to ${MAX_SEARCH_COUNT}.`,
}) ??
searchConfig?.maxResults ??
undefined;
const model = resolveGeminiModel(geminiConfig);
const baseUrl = resolveGeminiBaseUrl(geminiConfig);
const cacheKey = buildSearchCacheKey([
"gemini",
query,
resolveSearchCount(count, DEFAULT_SEARCH_COUNT),
baseUrl,
model,
timeRange.freshness,
timeRange.timeRangeFilter?.startTime,
timeRange.timeRangeFilter?.endTime,
]);
const cached = readCachedSearchPayload(cacheKey);
if (cached) {
return cached;
}
const start = Date.now();
const result = await runGeminiSearch({
query: queryWithSoftFreshness(query, timeRange.freshness),
apiKey,
baseUrl,
model,
timeoutSeconds: resolveSearchTimeoutSeconds(searchConfig),
signal: context?.signal,
timeRangeFilter: timeRange.timeRangeFilter,
});
const payload = {
query,
provider: "gemini",
model,
tookMs: Date.now() - start,
externalContent: {
untrusted: true,
source: "web_search",
provider: "gemini",
wrapped: true,
},
content: wrapWebContent(result.content),
citations: result.citations,
};
writeCachedSearchPayload(cacheKey, payload, resolveSearchCacheTtlMs(searchConfig));
return payload;
}

View File

@@ -0,0 +1,42 @@
// Google provider module implements model/runtime integration.
import {
isRecord,
normalizeOptionalString as trimToUndefined,
} from "openclaw/plugin-sdk/string-coerce-runtime";
import { normalizeGoogleApiBaseUrl } from "../provider-policy.js";
const DEFAULT_GEMINI_WEB_SEARCH_MODEL = "gemini-2.5-flash";
export type GeminiConfig = {
apiKey?: unknown;
baseUrl?: unknown;
model?: unknown;
providerApiKey?: unknown;
providerBaseUrl?: unknown;
};
export function resolveGeminiConfig(searchConfig?: Record<string, unknown>): GeminiConfig {
const gemini = searchConfig?.gemini;
return isRecord(gemini) ? gemini : {};
}
export function resolveGeminiApiKey(
gemini?: GeminiConfig,
env: Record<string, string | undefined> = process.env,
): string | undefined {
return (
trimToUndefined(gemini?.apiKey) ??
trimToUndefined(env.GEMINI_API_KEY) ??
trimToUndefined(gemini?.providerApiKey)
);
}
export function resolveGeminiModel(gemini?: GeminiConfig): string {
return trimToUndefined(gemini?.model) ?? DEFAULT_GEMINI_WEB_SEARCH_MODEL;
}
export function resolveGeminiBaseUrl(gemini?: GeminiConfig): string {
return normalizeGoogleApiBaseUrl(
trimToUndefined(gemini?.baseUrl) ?? trimToUndefined(gemini?.providerBaseUrl),
);
}

View File

@@ -0,0 +1,155 @@
// Google provider module implements model/runtime integration.
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-contracts";
import { createLazyRuntimeModule } from "openclaw/plugin-sdk/lazy-runtime";
import {
createWebSearchProviderContractFields,
mergeScopedSearchConfig,
resolveProviderWebSearchPluginConfig,
type WebSearchProviderPlugin,
type WebSearchProviderToolDefinition,
} from "openclaw/plugin-sdk/provider-web-search-config-contract";
import { isRecord } from "openclaw/plugin-sdk/string-coerce-runtime";
import {
resolveGeminiApiKey,
resolveGeminiBaseUrl,
resolveGeminiModel,
} from "./gemini-web-search-provider.shared.js";
const GEMINI_CREDENTIAL_PATH = "plugins.entries.google.config.webSearch.apiKey";
const GOOGLE_PROVIDER_CREDENTIAL_PATH = "models.providers.google.apiKey";
const loadGeminiWebSearchRuntime = createLazyRuntimeModule(
() => import("./gemini-web-search-provider.runtime.js"),
);
const GEMINI_TOOL_PARAMETERS = {
type: "object",
properties: {
query: { type: "string", description: "Search query string." },
count: {
type: "integer",
description: "Number of results to return (1-10).",
minimum: 1,
maximum: 10,
},
country: { type: "string", description: "Not supported by Gemini." },
language: { type: "string", description: "Not supported by Gemini." },
freshness: {
type: "string",
description:
"Filter Gemini search freshness: week, month, and year use hard Google Search time ranges; day prioritizes the last 24 hours as a recency hint.",
},
date_after: {
type: "string",
description: "Only ground with results published after this date (YYYY-MM-DD).",
},
date_before: {
type: "string",
description: "Only ground with results published before this date (YYYY-MM-DD).",
},
},
required: ["query"],
} satisfies Record<string, unknown>;
function createGeminiToolDefinition(
searchConfig?: Record<string, unknown>,
): WebSearchProviderToolDefinition {
return {
description:
"Search the web using Gemini with Google Search grounding. Returns AI-synthesized answers with citations from Google Search.",
parameters: GEMINI_TOOL_PARAMETERS,
execute: async (args, context) => {
const { executeGeminiSearch } = await loadGeminiWebSearchRuntime();
return await executeGeminiSearch(args, searchConfig, context);
},
};
}
function resolveGoogleModelProviderConfig(
config?: OpenClawConfig,
): Record<string, unknown> | undefined {
const provider = config?.models?.providers?.google;
return isRecord(provider) ? provider : undefined;
}
function getGoogleModelProviderCredentialFallback(
config?: OpenClawConfig,
): { path: string; value: unknown } | undefined {
const provider = resolveGoogleModelProviderConfig(config);
return provider && provider.apiKey !== undefined
? { path: GOOGLE_PROVIDER_CREDENTIAL_PATH, value: provider.apiKey }
: undefined;
}
function withGoogleModelProviderFallbacks(
searchConfig: Record<string, unknown> | undefined,
config?: OpenClawConfig,
): Record<string, unknown> | undefined {
const provider = resolveGoogleModelProviderConfig(config);
if (!provider || (provider.apiKey === undefined && provider.baseUrl === undefined)) {
return searchConfig;
}
const gemini = isRecord(searchConfig?.gemini) ? { ...searchConfig.gemini } : {};
const mergedSearchConfig: Record<string, unknown> = searchConfig
? Object.defineProperties({}, Object.getOwnPropertyDescriptors(searchConfig))
: {};
const geminiDescriptor = searchConfig
? Object.getOwnPropertyDescriptor(searchConfig, "gemini")
: undefined;
if (provider.apiKey !== undefined) {
gemini.providerApiKey = provider.apiKey;
}
if (provider.baseUrl !== undefined) {
gemini.providerBaseUrl = provider.baseUrl;
}
Object.defineProperty(mergedSearchConfig, "gemini", {
value: gemini,
enumerable: geminiDescriptor?.enumerable ?? false,
configurable: true,
writable: true,
});
return mergedSearchConfig;
}
export function createGeminiWebSearchProvider(): WebSearchProviderPlugin {
const contractFields = createWebSearchProviderContractFields({
credentialPath: GEMINI_CREDENTIAL_PATH,
searchCredential: { type: "scoped", scopeId: "gemini" },
configuredCredential: { pluginId: "google" },
});
return {
id: "gemini",
label: "Gemini (Google Search)",
hint: "Requires Google Gemini API key · Google Search grounding",
onboardingScopes: ["text-inference"],
credentialLabel: "Google Gemini API key",
envVars: ["GEMINI_API_KEY"],
placeholder: "AIza...",
signupUrl: "https://aistudio.google.com/apikey",
docsUrl: "https://docs.openclaw.ai/tools/web",
autoDetectOrder: 20,
credentialPath: GEMINI_CREDENTIAL_PATH,
...contractFields,
getConfiguredCredentialFallback: getGoogleModelProviderCredentialFallback,
createTool: (ctx) =>
createGeminiToolDefinition(
withGoogleModelProviderFallbacks(
mergeScopedSearchConfig(
ctx.searchConfig,
"gemini",
resolveProviderWebSearchPluginConfig(ctx.config, "google"),
),
ctx.config,
),
),
};
}
export const testing = {
resolveGeminiApiKey,
resolveGeminiBaseUrl,
resolveGeminiModel,
withGoogleModelProviderFallbacks,
} as const;
export { testing as __testing };

View File

@@ -0,0 +1,7 @@
// Google API module exposes the plugin public contract.
export { buildGoogleGeminiCliBackend } from "./cli-backend.js";
export { buildGoogleImageGenerationProvider } from "./image-generation-provider.js";
export { buildGoogleMusicGenerationProvider } from "./music-generation-provider.js";
export { buildGoogleSpeechProvider } from "./speech-provider.js";
export { googleMediaUnderstandingProvider } from "./media-understanding-provider.js";
export { buildGoogleVideoGenerationProvider } from "./video-generation-provider.js";

View File

@@ -0,0 +1,15 @@
// Google API module exposes the plugin public contract.
export {
createGoogleThinkingPayloadWrapper,
createGoogleThinkingStreamWrapper,
isGoogleGemini25ThinkingBudgetModel,
isGoogleGemini3FlashModel,
isGoogleGemini3ProModel,
isGoogleGemini3ThinkingLevelModel,
isGoogleThinkingRequiredModel,
resolveGoogleGemini3ThinkingLevel,
sanitizeGoogleThinkingPayload,
stripInvalidGoogleThinkingBudget,
type GoogleThinkingInputLevel,
type GoogleThinkingLevel,
} from "./thinking.js";

View File

@@ -0,0 +1,154 @@
// Google tests cover thinking plugin behavior.
import { describe, expect, it } from "vitest";
import {
resolveGoogleGemini3ThinkingLevel,
sanitizeGoogleThinkingPayload,
} from "./thinking-api.js";
describe("google thinking policy", () => {
it.each([
["off", "LOW"],
["minimal", "LOW"],
["low", "LOW"],
["medium", "HIGH"],
["adaptive", undefined],
["high", "HIGH"],
["xhigh", "HIGH"],
] as const)("maps Gemini 3 Pro thinking level %s to %s", (thinkingLevel, expected) => {
expect(
resolveGoogleGemini3ThinkingLevel({
modelId: "gemini-3.1-pro-preview",
thinkingLevel,
}),
).toBe(expected);
});
it.each([
[0, "LOW"],
[2048, "LOW"],
[2049, "HIGH"],
] as const)("maps Gemini 3 Pro budget %s to %s", (thinkingBudget, expected) => {
expect(
resolveGoogleGemini3ThinkingLevel({
modelId: "gemini-pro-latest",
thinkingBudget,
}),
).toBe(expected);
});
it.each([
["off", "MINIMAL"],
["minimal", "MINIMAL"],
["low", "LOW"],
["medium", "MEDIUM"],
["adaptive", undefined],
["high", "HIGH"],
["xhigh", "HIGH"],
] as const)("maps Gemini 3 Flash thinking level %s to %s", (thinkingLevel, expected) => {
expect(
resolveGoogleGemini3ThinkingLevel({
modelId: "gemini-flash-latest",
thinkingLevel,
}),
).toBe(expected);
});
it.each([
[-1, undefined],
[0, "MINIMAL"],
[2048, "LOW"],
[8192, "MEDIUM"],
[8193, "HIGH"],
] as const)("maps Gemini 3 Flash budget %s to %s", (thinkingBudget, expected) => {
expect(
resolveGoogleGemini3ThinkingLevel({
modelId: "gemini-3.1-flash-lite",
thinkingBudget,
}),
).toBe(expected);
});
it("removes thinkingBudget=0 for Gemini 2.5 Pro", () => {
const payload = {
config: {
thinkingConfig: { thinkingBudget: 0 },
},
};
sanitizeGoogleThinkingPayload({ payload, modelId: "google/gemini-2.5-pro-preview" });
expect(payload.config).not.toHaveProperty("thinkingConfig");
});
it("rewrites Gemini 3 thinking budgets to thinkingLevel", () => {
const payload = {
generationConfig: {
thinkingConfig: { thinkingBudget: 8193, includeThoughts: true },
},
};
sanitizeGoogleThinkingPayload({
payload,
modelId: "gemini-3-flash-preview",
thinkingLevel: "medium",
});
expect(payload.generationConfig.thinkingConfig).toEqual({
includeThoughts: true,
thinkingLevel: "MEDIUM",
});
});
it("keeps Gemini 3 adaptive thinking provider-dynamic instead of forcing a fixed level", () => {
const payload = {
generationConfig: {
thinkingConfig: { thinkingBudget: 8192, includeThoughts: true },
},
};
sanitizeGoogleThinkingPayload({
payload,
modelId: "gemini-3-flash-preview",
thinkingLevel: "adaptive",
});
expect(payload.generationConfig.thinkingConfig).toEqual({
includeThoughts: true,
});
});
it("maps Gemini 2.5 adaptive thinking to dynamic thinkingBudget", () => {
const payload = {
config: {
thinkingConfig: { thinkingBudget: 8192, includeThoughts: true },
},
};
sanitizeGoogleThinkingPayload({
payload,
modelId: "gemini-2.5-flash",
thinkingLevel: "adaptive",
});
expect(payload.config.thinkingConfig).toEqual({
includeThoughts: true,
thinkingBudget: -1,
});
});
it("maps Gemma 4 thinking mode without sending thinkingBudget", () => {
const payload = {
config: {
thinkingConfig: { thinkingBudget: 4096 },
},
};
sanitizeGoogleThinkingPayload({
payload,
modelId: "gemma-4-26b-a4b-it",
thinkingLevel: "high",
});
expect(payload.config.thinkingConfig).toEqual({ thinkingLevel: "HIGH" });
});
});

View File

@@ -0,0 +1,15 @@
// Google plugin module implements thinking behavior.
export {
createGoogleThinkingPayloadWrapper,
createGoogleThinkingStreamWrapper,
isGoogleGemini25ThinkingBudgetModel,
isGoogleGemini3FlashModel,
isGoogleGemini3ProModel,
isGoogleGemini3ThinkingLevelModel,
isGoogleThinkingRequiredModel,
resolveGoogleGemini3ThinkingLevel,
sanitizeGoogleThinkingPayload,
stripInvalidGoogleThinkingBudget,
type GoogleThinkingInputLevel,
type GoogleThinkingLevel,
} from "openclaw/plugin-sdk/provider-stream-shared";

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,16 @@
{
"extends": "../tsconfig.package-boundary.base.json",
"compilerOptions": {
"rootDir": "."
},
"include": ["./*.ts", "./src/**/*.ts"],
"exclude": [
"./**/*.test.ts",
"./dist/**",
"./node_modules/**",
"./src/test-support/**",
"./src/**/*test-helpers.ts",
"./src/**/*test-harness.ts",
"./src/**/*test-support.ts"
]
}

View File

@@ -0,0 +1,398 @@
// Google plugin module implements vertex adc behavior.
import { existsSync, readFileSync } from "node:fs";
import { readFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { gunzipSync } from "node:zlib";
import {
asDateTimestampMs,
resolveExpiresAtMsFromDurationMs,
resolveExpiresAtMsFromDurationSeconds,
} from "openclaw/plugin-sdk/number-runtime";
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
type GoogleAuthorizedUserCredentials = {
type: "authorized_user";
client_id?: string;
client_secret?: string;
refresh_token?: string;
};
type GoogleVertexAuthorizedUserToken = {
token: string;
expiresAtMs: number;
credentialsPath: string;
refreshToken: string;
};
type GoogleVertexAdcToken = {
token: string;
expiresAtMs: number;
};
type GoogleOauthTokenResponsePayload = {
access_token?: unknown;
expires_in?: unknown;
error?: unknown;
error_description?: unknown;
};
const GCP_VERTEX_CREDENTIALS_MARKER = "gcp-vertex-credentials";
const GOOGLE_OAUTH_TOKEN_URL = "https://oauth2.googleapis.com/token";
const GOOGLE_VERTEX_OAUTH_SCOPE = "https://www.googleapis.com/auth/cloud-platform";
// Hold tokens slightly less long than reported expiry (Google's recommendation
// is a 60s buffer) so we don't ship a request that's already revoked when it
// leaves the gateway.
const GOOGLE_VERTEX_TOKEN_EXPIRY_BUFFER_MS = 60_000;
const GOOGLE_VERTEX_DEFAULT_TOKEN_LIFETIME_SECONDS = 3600;
const GOOGLE_VERTEX_AUTHLIB_TOKEN_CACHE_MS = 5 * 60_000;
let cachedGoogleVertexAuthorizedUserToken: GoogleVertexAuthorizedUserToken | undefined;
let cachedGoogleAuthClient:
| {
promise: Promise<{
getAccessToken: () => Promise<string | null | undefined>;
}>;
}
| undefined;
let cachedGoogleVertexAdcToken: GoogleVertexAdcToken | undefined;
function isGoogleVertexTokenFresh(expiresAtMsRaw: number, nowRaw = Date.now()): boolean {
const expiresAtMs = asDateTimestampMs(expiresAtMsRaw);
const nowMs = asDateTimestampMs(nowRaw);
if (expiresAtMs === undefined || nowMs === undefined) {
return false;
}
const minFreshExpiresAtMs = resolveExpiresAtMsFromDurationMs(
GOOGLE_VERTEX_TOKEN_EXPIRY_BUFFER_MS,
{ nowMs },
);
return minFreshExpiresAtMs !== undefined && expiresAtMs > minFreshExpiresAtMs;
}
function resolveAuthorizedUserTokenExpiresAtMs(value: unknown, nowRaw: number): number | undefined {
const nowMs = asDateTimestampMs(nowRaw);
if (nowMs === undefined) {
return undefined;
}
const lifetimeSeconds =
typeof value === "number" && Number.isFinite(value)
? Math.max(1, value)
: GOOGLE_VERTEX_DEFAULT_TOKEN_LIFETIME_SECONDS;
return resolveExpiresAtMsFromDurationSeconds(lifetimeSeconds, { nowMs }) ?? nowMs;
}
function resolveGoogleAuthLibraryTokenExpiresAtMs(nowRaw = Date.now()): number | undefined {
const nowMs = asDateTimestampMs(nowRaw);
return nowMs === undefined
? undefined
: resolveExpiresAtMsFromDurationMs(GOOGLE_VERTEX_AUTHLIB_TOKEN_CACHE_MS, { nowMs });
}
export function resetGoogleVertexAuthorizedUserTokenCacheForTest(): void {
cachedGoogleVertexAuthorizedUserToken = undefined;
cachedGoogleAuthClient = undefined;
cachedGoogleVertexAdcToken = undefined;
}
export function isGoogleVertexCredentialsMarker(
apiKey: string | undefined,
): apiKey is undefined | typeof GCP_VERTEX_CREDENTIALS_MARKER {
return apiKey === undefined || apiKey === GCP_VERTEX_CREDENTIALS_MARKER;
}
function hasGoogleVertexProjectEnv(env: NodeJS.ProcessEnv): boolean {
return Boolean(
normalizeOptionalString(env.GOOGLE_CLOUD_PROJECT) ||
normalizeOptionalString(env.GCLOUD_PROJECT),
);
}
function hasGoogleVertexLocationEnv(env: NodeJS.ProcessEnv): boolean {
return Boolean(normalizeOptionalString(env.GOOGLE_CLOUD_LOCATION));
}
function resolveGoogleApplicationCredentialsPath(
env: NodeJS.ProcessEnv = process.env,
): string | undefined {
const explicit = normalizeOptionalString(env.GOOGLE_APPLICATION_CREDENTIALS);
if (explicit) {
return existsSync(explicit) ? explicit : undefined;
}
const homeDir = normalizeOptionalString(env.HOME) ?? os.homedir();
const homeFallback = path.join(
homeDir,
".config",
"gcloud",
"application_default_credentials.json",
);
if (existsSync(homeFallback)) {
return homeFallback;
}
const appDataDir = normalizeOptionalString(env.APPDATA);
if (!appDataDir) {
return undefined;
}
const appDataFallback = path.join(appDataDir, "gcloud", "application_default_credentials.json");
return existsSync(appDataFallback) ? appDataFallback : undefined;
}
async function readGoogleAuthorizedUserCredentials(
credentialsPath: string,
): Promise<GoogleAuthorizedUserCredentials | undefined> {
let parsed: unknown;
try {
parsed = JSON.parse(await readFile(credentialsPath, "utf8")) as unknown;
} catch {
return undefined;
}
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return undefined;
}
const record = parsed as Record<string, unknown>;
if (record.type !== "authorized_user") {
return undefined;
}
return {
type: "authorized_user",
client_id: normalizeOptionalString(record.client_id),
client_secret: normalizeOptionalString(record.client_secret),
refresh_token: normalizeOptionalString(record.refresh_token),
};
}
function readGoogleAdcCredentialsTypeSync(credentialsPath: string): string | undefined {
try {
const parsed = JSON.parse(readFileSync(credentialsPath, "utf8")) as unknown;
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return undefined;
}
const type = (parsed as { type?: unknown }).type;
return typeof type === "string" ? type : undefined;
} catch {
return undefined;
}
}
/**
* Returns true when a file/env Application Default Credentials source usable
* for Google Vertex AI is detectable synchronously. We still call the function
* `...AuthorizedUserAdcSync` for backwards compatibility with older tests; the
* predicate now also covers:
*
* 1. `authorized_user` credentials file (existing case - `gcloud auth
* application-default login` produces this).
* 2. `external_account` credentials file (Workload Identity Federation).
* 3. `service_account` credentials file (raw GSA key - rarely used in
* OpenClaw, included for completeness).
* Metadata-server ADC is intentionally not detected here: `google-auth-library`
* probes the default metadata hosts asynchronously at request time, and the
* provider wires the Vertex transport without this sync predicate.
*/
export function hasGoogleVertexAuthorizedUserAdcSync(
env: NodeJS.ProcessEnv = process.env,
): boolean {
const credentialsPath = resolveGoogleApplicationCredentialsPath(env);
if (credentialsPath) {
const type = readGoogleAdcCredentialsTypeSync(credentialsPath);
if (type === "authorized_user" || type === "external_account" || type === "service_account") {
return true;
}
}
return false;
}
export function resolveGoogleVertexConfigApiKey(
env: NodeJS.ProcessEnv = process.env,
): string | undefined {
return hasGoogleVertexProjectEnv(env) &&
hasGoogleVertexLocationEnv(env) &&
hasGoogleVertexAuthorizedUserAdcSync(env)
? GCP_VERTEX_CREDENTIALS_MARKER
: undefined;
}
async function refreshGoogleVertexAuthorizedUserAccessToken(params: {
credentialsPath: string;
credentials: GoogleAuthorizedUserCredentials;
fetchImpl?: typeof fetch;
}): Promise<string> {
const clientId = normalizeOptionalString(params.credentials.client_id);
const clientSecret = normalizeOptionalString(params.credentials.client_secret);
const refreshToken = normalizeOptionalString(params.credentials.refresh_token);
if (!clientId || !clientSecret || !refreshToken) {
throw new Error(
"Google Vertex authorized_user ADC is missing client_id, client_secret, or refresh_token.",
);
}
const cached = cachedGoogleVertexAuthorizedUserToken;
if (
cached?.credentialsPath === params.credentialsPath &&
cached.refreshToken === refreshToken &&
isGoogleVertexTokenFresh(cached.expiresAtMs)
) {
return cached.token;
}
const body = new URLSearchParams({
client_id: clientId,
client_secret: clientSecret,
refresh_token: refreshToken,
grant_type: "refresh_token",
});
const response = await (params.fetchImpl ?? fetch)(GOOGLE_OAUTH_TOKEN_URL, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body,
});
const payload = await readGoogleOauthTokenResponsePayload(response);
if (!response.ok) {
const description = normalizeOptionalString(payload?.error_description);
const code = normalizeOptionalString(payload?.error);
throw new Error(
`Google Vertex ADC token refresh failed: ${response.status}${code ? ` ${code}` : ""}${description ? ` (${description})` : ""}`,
);
}
if (!payload) {
throw new Error("Google Vertex ADC token refresh response could not be parsed as JSON.");
}
const token = normalizeOptionalString(payload?.access_token);
if (!token) {
throw new Error("Google Vertex ADC token refresh response did not include an access_token.");
}
const nowMs = Date.now();
const expiresAtMs = resolveAuthorizedUserTokenExpiresAtMs(payload?.expires_in, nowMs);
if (expiresAtMs !== undefined) {
cachedGoogleVertexAuthorizedUserToken = {
token,
expiresAtMs,
credentialsPath: params.credentialsPath,
refreshToken,
};
}
return token;
}
async function readGoogleOauthTokenResponsePayload(
response: Response,
): Promise<GoogleOauthTokenResponsePayload | undefined> {
const bytes = Buffer.from(await response.arrayBuffer());
const text = decodeGoogleOauthTokenResponseBody(bytes, response.headers.get("content-encoding"));
if (!text.trim()) {
return undefined;
}
try {
return JSON.parse(text) as GoogleOauthTokenResponsePayload;
} catch {
return undefined;
}
}
function decodeGoogleOauthTokenResponseBody(bytes: Buffer, contentEncoding: string | null): string {
if (shouldGunzipGoogleOauthTokenResponse(bytes, contentEncoding)) {
try {
return gunzipSync(bytes).toString("utf8");
} catch {
return bytes.toString("utf8");
}
}
return bytes.toString("utf8");
}
function shouldGunzipGoogleOauthTokenResponse(
bytes: Buffer,
contentEncoding: string | null,
): boolean {
if (bytes[0] === 0x1f && bytes[1] === 0x8b) {
return true;
}
return (contentEncoding ?? "")
.split(",")
.map((encoding) => encoding.trim().toLowerCase())
.includes("gzip");
}
async function resolveGoogleVertexAccessTokenViaGoogleAuth(): Promise<string> {
// Lazy-import + cache so we don't pay the google-auth-library load cost on
// gateway startup; only when we actually need a non-authorized_user token.
if (!cachedGoogleAuthClient) {
cachedGoogleAuthClient = {
promise: import("google-auth-library").then(({ GoogleAuth }) => {
// GoogleAuth handles every ADC variant we care about for GKE:
// - external_account (Workload Identity Federation: STS exchange)
// - service_account (raw GSA key: JWT-bearer)
// - GKE Workload Identity (metadata server when no credentials file)
// - Compute Engine / Cloud Run / GAE metadata server fallback
// It also caches tokens internally and refreshes before expiry.
return new GoogleAuth({
scopes: [GOOGLE_VERTEX_OAUTH_SCOPE],
});
}),
};
}
const auth = await cachedGoogleAuthClient.promise;
const cached = cachedGoogleVertexAdcToken;
if (cached && isGoogleVertexTokenFresh(cached.expiresAtMs)) {
return cached.token;
}
const token = await auth.getAccessToken();
const normalized = normalizeOptionalString(token);
if (!normalized) {
throw new Error(
"Google Vertex ADC fallback (google-auth-library) did not return an access token. " +
"Verify the GKE Workload Identity binding (KSA \u2192 GSA), `GOOGLE_APPLICATION_CREDENTIALS`, " +
"or other ADC source is reachable from this pod.",
);
}
// google-auth-library doesn't expose token expiry on the simple
// `getAccessToken()` return type, so we cache for a conservative 5 minutes.
// The library itself already refreshes well before its own internal expiry,
// so this cache is mainly to avoid hot-loop calls into the auth client.
const expiresAtMs = resolveGoogleAuthLibraryTokenExpiresAtMs();
if (expiresAtMs !== undefined) {
cachedGoogleVertexAdcToken = {
token: normalized,
expiresAtMs,
};
}
return normalized;
}
/**
* Resolve `Authorization: Bearer ...` headers for Google Vertex calls.
*
* We try the hand-rolled `authorized_user` refresh path first (preserves the
* existing fetchImpl test seam and the OpenClaw upstream behaviour); when the
* configured ADC source is anything other than `authorized_user` (the common
* production cases on GKE: Workload Identity, Workload Identity Federation,
* service-account JSON keys), we hand off to `google-auth-library` which
* understands all of those natively.
*
* Note: the function is still named `...AuthorizedUserHeaders` to avoid a
* symbol rename across the existing patch surface; the docstring above is
* the truth, the name is legacy.
*/
export async function resolveGoogleVertexAuthorizedUserHeaders(
fetchImpl?: typeof fetch,
): Promise<Record<string, string>> {
const credentialsPath = resolveGoogleApplicationCredentialsPath();
if (credentialsPath) {
const credentials = await readGoogleAuthorizedUserCredentials(credentialsPath);
if (credentials) {
const token = await refreshGoogleVertexAuthorizedUserAccessToken({
credentialsPath,
credentials,
fetchImpl,
});
return { Authorization: `Bearer ${token}` };
}
}
// No file-based authorized_user ADC. Fall back to google-auth-library which
// handles GKE Workload Identity (metadata server), Workload Identity
// Federation (external_account), and service-account keys.
const token = await resolveGoogleVertexAccessTokenViaGoogleAuth();
return { Authorization: `Bearer ${token}` };
}

View File

@@ -0,0 +1,68 @@
import type { Model } from "openclaw/plugin-sdk/llm";
import { describe, expect, it } from "vitest";
import { isGoogleVertexHostname } from "./provider-policy.js";
import { resolveGoogleVertexBaseOrigin } from "./transport-stream.js";
// Minimal Vertex model whose baseUrl carries the {location} template, so the
// base-origin resolver falls through to location-based host construction
// (the configured-baseUrl early return only fires for a literal host).
function buildModel(overrides: Partial<Model<"google-vertex">> = {}): Model<"google-vertex"> {
return {
id: "gemini-3.5-flash",
name: "Gemini 3.5 Flash",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 8192,
...overrides,
};
}
describe("Google Vertex multi-region host construction", () => {
const model = buildModel();
it("routes the eu multi-region to the dedicated .rep.googleapis.com host", () => {
expect(resolveGoogleVertexBaseOrigin(model, "eu")).toBe(
"https://aiplatform.eu.rep.googleapis.com",
);
});
it("routes the us multi-region to the dedicated .rep.googleapis.com host", () => {
expect(resolveGoogleVertexBaseOrigin(model, "us")).toBe(
"https://aiplatform.us.rep.googleapis.com",
);
});
it("keeps the unprefixed host for the global location", () => {
expect(resolveGoogleVertexBaseOrigin(model, "global")).toBe(
"https://aiplatform.googleapis.com",
);
});
it("keeps the regional prefix for normal regions", () => {
expect(resolveGoogleVertexBaseOrigin(model, "europe-west1")).toBe(
"https://europe-west1-aiplatform.googleapis.com",
);
});
});
describe("Google Vertex hostname recognition", () => {
it("recognizes the multi-region rep host as a Vertex host", () => {
expect(isGoogleVertexHostname("aiplatform.eu.rep.googleapis.com")).toBe(true);
expect(isGoogleVertexHostname("aiplatform.us.rep.googleapis.com")).toBe(true);
});
it("does not classify unrelated rep hosts as Vertex hosts", () => {
expect(isGoogleVertexHostname("discoveryengine.eu.rep.googleapis.com")).toBe(false);
expect(isGoogleVertexHostname("not-aiplatform.eu.rep.googleapis.com")).toBe(false);
});
it("still recognizes the unprefixed and regional Vertex hosts", () => {
expect(isGoogleVertexHostname("aiplatform.googleapis.com")).toBe(true);
expect(isGoogleVertexHostname("europe-west1-aiplatform.googleapis.com")).toBe(true);
});
});

View File

@@ -0,0 +1,761 @@
// Google tests cover video generation provider plugin behavior.
import { mockPinnedHostnameResolution } from "openclaw/plugin-sdk/test-env";
import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const { createGoogleGenAIMock, downloadMock, generateVideosMock, getVideosOperationMock } =
vi.hoisted(() => {
const generateVideosMockLocal = vi.fn();
const getVideosOperationMockLocal = vi.fn();
const downloadMockLocal = vi.fn();
const createGoogleGenAIMockLocal = vi.fn(() => {
return {
models: {
generateVideos: generateVideosMockLocal,
},
operations: {
getVideosOperation: getVideosOperationMockLocal,
},
files: {
download: downloadMockLocal,
},
};
});
return {
createGoogleGenAIMock: createGoogleGenAIMockLocal,
downloadMock: downloadMockLocal,
generateVideosMock: generateVideosMockLocal,
getVideosOperationMock: getVideosOperationMockLocal,
};
});
vi.mock("./google-genai-runtime.js", () => ({
createGoogleGenAI: createGoogleGenAIMock,
}));
import * as providerAuthRuntime from "openclaw/plugin-sdk/provider-auth-runtime";
import { expectExplicitVideoGenerationCapabilities } from "openclaw/plugin-sdk/provider-test-contracts";
import { buildGoogleVideoGenerationProvider } from "./video-generation-provider.js";
type MockWithCalls = {
mock: { calls: unknown[][] };
};
function firstObjectArg(mock: MockWithCalls): Record<string, unknown> {
const [call] = mock.mock.calls;
if (!call) {
throw new Error("expected first mock call to receive an object argument");
}
const value = call[0];
if (value === undefined || value === null || typeof value !== "object" || Array.isArray(value)) {
throw new Error("expected first mock call to receive an object argument");
}
return value as Record<string, unknown>;
}
function recordField(value: unknown, field: string): Record<string, unknown> {
if (value === undefined || value === null || typeof value !== "object" || Array.isArray(value)) {
throw new Error(`expected ${field} to be an object`);
}
return value as Record<string, unknown>;
}
function firstGoogleClientHttpOptions(): Record<string, unknown> {
return recordField(firstObjectArg(createGoogleGenAIMock).httpOptions, "httpOptions");
}
function requireFetchCall(
fetchMock: ReturnType<typeof vi.fn>,
index: number,
): [RequestInfo | URL, RequestInit | undefined] {
const call = fetchMock.mock.calls[index];
if (!call) {
throw new Error(`expected Google video fetch call ${index}`);
}
return call as [RequestInfo | URL, RequestInit | undefined];
}
function parseFetchJsonBody(fetchMock: ReturnType<typeof vi.fn>, index: number): unknown {
const [, init] = requireFetchCall(fetchMock, index);
const body = init?.body;
if (typeof body !== "string") {
throw new Error(`expected Google video fetch body ${index}`);
}
return JSON.parse(body) as unknown;
}
function fetchInputUrl(fetchMock: ReturnType<typeof vi.fn>, index: number): string {
const [input] = requireFetchCall(fetchMock, index);
if (typeof input === "string") {
return input;
}
if (input instanceof URL) {
return input.toString();
}
return input.url;
}
function oversizedJsonResponse(params: { chunkCount: number; chunkSize: number }): {
response: Response;
getReadCount: () => number;
wasCanceled: () => boolean;
} {
const chunk = new Uint8Array(params.chunkSize);
let readCount = 0;
let canceled = false;
return {
response: new Response(
new ReadableStream<Uint8Array>({
pull(controller) {
if (readCount >= params.chunkCount) {
controller.close();
return;
}
readCount += 1;
controller.enqueue(chunk);
},
cancel() {
canceled = true;
},
}),
{
status: 200,
headers: { "Content-Type": "application/json" },
},
),
getReadCount: () => readCount,
wasCanceled: () => canceled,
};
}
let ssrfMock: { mockRestore: () => void } | undefined;
describe("google video generation provider", () => {
beforeEach(() => {
ssrfMock = mockPinnedHostnameResolution();
});
afterEach(() => {
ssrfMock?.mockRestore();
ssrfMock = undefined;
vi.restoreAllMocks();
vi.unstubAllGlobals();
downloadMock.mockReset();
generateVideosMock.mockReset();
getVideosOperationMock.mockReset();
createGoogleGenAIMock.mockClear();
vi.useRealTimers();
});
afterAll(() => {
vi.doUnmock("./google-genai-runtime.js");
vi.resetModules();
});
it("declares explicit mode capabilities", () => {
const provider = buildGoogleVideoGenerationProvider();
expectExplicitVideoGenerationCapabilities(provider);
expect(provider.capabilities.generate?.supportsAudio).toBe(false);
expect(provider.capabilities.imageToVideo?.supportsAudio).toBe(false);
expect(provider.capabilities.videoToVideo?.supportsAudio).toBe(false);
});
it("submits generation and returns inline video bytes", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
name: "operations/123",
response: {
generatedVideos: [
{
video: {
videoBytes: Buffer.from("mp4-bytes").toString("base64"),
mimeType: "video/mp4",
},
},
],
},
});
const provider = buildGoogleVideoGenerationProvider();
const result = await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
aspectRatio: "16:9",
resolution: "720P",
durationSeconds: 3,
audio: true,
});
expect(generateVideosMock).toHaveBeenCalledTimes(1);
const request = firstObjectArg(generateVideosMock);
expect(request.model).toBe("veo-3.1-fast-generate-preview");
expect(request.prompt).toBe("A tiny robot watering a windowsill garden");
const config = recordField(request.config, "config");
expect(config.durationSeconds).toBe(4);
expect(config.aspectRatio).toBe("16:9");
expect(config.resolution).toBe("720p");
expect(config).not.toHaveProperty("generateAudio");
expect(config).not.toHaveProperty("numberOfVideos");
expect(result.videos).toHaveLength(1);
expect(result.videos[0]?.mimeType).toBe("video/mp4");
const clientOptions = firstObjectArg(createGoogleGenAIMock);
expect(clientOptions.apiKey).toBe("google-key");
const httpOptions = recordField(clientOptions.httpOptions, "httpOptions");
expect(httpOptions).not.toHaveProperty("baseUrl");
expect(httpOptions).not.toHaveProperty("apiVersion");
});
it("rejects inline video bytes that exceed the configured media cap", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{
video: {
videoBytes: Buffer.from("too-large").toString("base64"),
mimeType: "video/mp4",
},
},
],
},
});
const provider = buildGoogleVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: { agents: { defaults: { mediaMaxMb: 0.000001 } } },
durationSeconds: 3,
}),
).rejects.toThrow("Google generated video download exceeds 1 bytes");
});
it("strips /v1beta suffix from configured baseUrl before passing to GoogleGenAI SDK", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{ video: { videoBytes: Buffer.from("mp4").toString("base64"), mimeType: "video/mp4" } },
],
},
});
const provider = buildGoogleVideoGenerationProvider();
await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {
models: {
providers: {
google: { baseUrl: "https://generativelanguage.googleapis.com/v1beta", models: [] },
},
},
},
durationSeconds: 3,
});
expect(firstGoogleClientHttpOptions().baseUrl).toBe(
"https://generativelanguage.googleapis.com",
);
});
it("downloads MLDev direct video uri responses without routing through the Files API", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{
video: {
uri: "https://generativelanguage.googleapis.com/v1beta/files/generated-video:download?alt=media",
mimeType: "video/mp4",
},
},
],
},
});
const fetchMock = vi.fn(async () => {
return new Response("direct-mp4", {
status: 200,
statusText: "OK",
headers: { "content-type": "video/mp4" },
});
});
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleVideoGenerationProvider();
const result = await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
durationSeconds: 3,
});
expect(fetchMock).toHaveBeenCalledTimes(1);
const [[downloadUrl]] = fetchMock.mock.calls as unknown as [[string, RequestInit?]];
expect(downloadUrl).toBe(
"https://generativelanguage.googleapis.com/v1beta/files/generated-video:download?alt=media&key=google-key",
);
expect(downloadMock).not.toHaveBeenCalled();
expect(result.videos[0]?.buffer).toEqual(Buffer.from("direct-mp4"));
expect(result.videos[0]?.mimeType).toBe("video/mp4");
});
it("rejects direct video uri downloads that exceed the configured media cap", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{
video: {
uri: "https://generativelanguage.googleapis.com/v1beta/files/generated-video:download?alt=media",
mimeType: "video/mp4",
},
},
],
},
});
vi.stubGlobal(
"fetch",
vi.fn(
async () =>
new Response("too-large", {
status: 200,
statusText: "OK",
headers: { "content-type": "video/mp4" },
}),
),
);
const provider = buildGoogleVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: { agents: { defaults: { mediaMaxMb: 0.000001 } } },
durationSeconds: 3,
}),
).rejects.toThrow("Google generated video download exceeds 1 bytes");
});
it("downloads SDK file handles through the bounded REST media endpoint", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{
video: {
uri: "files/generated-video",
mimeType: "video/mp4",
},
},
],
},
});
const fetchMock = vi.fn(async () => {
return new Response("sdk-video", {
status: 200,
statusText: "OK",
headers: { "content-type": "video/mp4" },
});
});
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleVideoGenerationProvider();
const result = await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
durationSeconds: 3,
});
expect(fetchInputUrl(fetchMock, 0)).toBe(
"https://generativelanguage.googleapis.com/v1beta/files/generated-video:download?alt=media&key=google-key",
);
expect(downloadMock).not.toHaveBeenCalled();
expect(result.videos[0]?.buffer).toEqual(Buffer.from("sdk-video"));
expect(result.videos[0]?.fileName).toBe("video-1.mp4");
});
it("rejects SDK file-handle downloads that exceed the configured media cap", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{
video: {
uri: "files/generated-video",
mimeType: "video/mp4",
},
},
],
},
});
vi.stubGlobal(
"fetch",
vi.fn(
async () =>
new Response("too-large", {
status: 200,
statusText: "OK",
headers: { "content-type": "video/mp4" },
}),
),
);
const provider = buildGoogleVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: { agents: { defaults: { mediaMaxMb: 0.000001 } } },
durationSeconds: 3,
}),
).rejects.toThrow("Google generated video download exceeds 1 bytes");
expect(downloadMock).not.toHaveBeenCalled();
});
it("falls back to REST predictLongRunning when text-only SDK video generation returns 404", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockRejectedValue(Object.assign(new Error("sdk 404"), { status: 404 }));
const fetchMock = vi
.fn()
.mockResolvedValueOnce(
new Response(
JSON.stringify({
done: true,
name: "operations/rest-123",
response: {
generateVideoResponse: {
generatedSamples: [
{
video: {
uri: "https://generativelanguage.googleapis.com/v1beta/files/rest-video:download?alt=media",
mimeType: "video/mp4",
},
},
],
},
},
}),
),
)
.mockResolvedValueOnce(
new Response("rest-video", {
status: 200,
statusText: "OK",
headers: { "content-type": "video/mp4" },
}),
);
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleVideoGenerationProvider();
const result = await provider.generateVideo({
provider: "google",
model: "google/models/veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
durationSeconds: 3,
});
expect(fetchMock).toHaveBeenCalledTimes(2);
expect(fetchInputUrl(fetchMock, 0)).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-fast-generate-preview:predictLongRunning",
);
expect(parseFetchJsonBody(fetchMock, 0)).toEqual({
instances: [{ prompt: "A tiny robot watering a windowsill garden" }],
parameters: { durationSeconds: 4 },
});
expect(fetchInputUrl(fetchMock, 1)).toBe(
"https://generativelanguage.googleapis.com/v1beta/files/rest-video:download?alt=media&key=google-key",
);
expect(downloadMock).not.toHaveBeenCalled();
expect(result.videos[0]?.buffer).toEqual(Buffer.from("rest-video"));
});
it("bounds successful Google REST operation JSON bodies instead of buffering the whole response", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockRejectedValue(Object.assign(new Error("sdk 404"), { status: 404 }));
const streamed = oversizedJsonResponse({ chunkCount: 64, chunkSize: 1024 * 1024 });
const fetchMock = vi.fn(async () => streamed.response);
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
durationSeconds: 3,
}),
).rejects.toThrow("Google video operation response exceeds 16777216 bytes");
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(streamed.getReadCount()).toBeLessThan(64);
expect(streamed.wasCanceled()).toBe(true);
});
it("retries transient Google REST poll failures with empty bodies", async () => {
vi.useFakeTimers();
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockRejectedValue(Object.assign(new Error("sdk 404"), { status: 404 }));
const fetchMock = vi
.fn()
.mockResolvedValueOnce(
new Response(
JSON.stringify({
done: false,
name: "operations/rest-123",
}),
),
)
.mockResolvedValueOnce(new Response("", { status: 503, statusText: "Service Unavailable" }))
.mockResolvedValueOnce(
new Response(
JSON.stringify({
done: true,
name: "operations/rest-123",
response: {
generateVideoResponse: {
generatedSamples: [
{
video: {
uri: "https://generativelanguage.googleapis.com/v1beta/files/rest-video:download?alt=media",
mimeType: "video/mp4",
},
},
],
},
},
}),
),
)
.mockResolvedValueOnce(
new Response("rest-video", {
status: 200,
statusText: "OK",
headers: { "content-type": "video/mp4" },
}),
);
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleVideoGenerationProvider();
const resultPromise = provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
durationSeconds: 3,
});
await vi.advanceTimersByTimeAsync(10_250);
const result = await resultPromise;
expect(fetchMock).toHaveBeenCalledTimes(4);
expect(fetchInputUrl(fetchMock, 1)).toBe(
"https://generativelanguage.googleapis.com/v1beta/operations/rest-123",
);
expect(fetchInputUrl(fetchMock, 2)).toBe(
"https://generativelanguage.googleapis.com/v1beta/operations/rest-123",
);
expect(result.videos[0]?.buffer).toEqual(Buffer.from("rest-video"));
});
it("does not fall back to REST when SDK video generation with reference inputs returns 404", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockRejectedValue(Object.assign(new Error("sdk 404"), { status: 404 }));
const fetchMock = vi.fn();
vi.stubGlobal("fetch", fetchMock);
const provider = buildGoogleVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "Animate this sketch",
cfg: {},
inputImages: [{ buffer: Buffer.from("img"), mimeType: "image/png" }],
}),
).rejects.toThrow("sdk 404");
expect(fetchMock).not.toHaveBeenCalled();
});
it("does NOT strip /v1beta when it appears mid-path (end-anchor proof)", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{ video: { videoBytes: Buffer.from("mp4").toString("base64"), mimeType: "video/mp4" } },
],
},
});
const provider = buildGoogleVideoGenerationProvider();
await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "test",
cfg: {
models: {
providers: { google: { baseUrl: "https://proxy.example.com/v1beta/route", models: [] } },
},
},
durationSeconds: 3,
});
expect(firstGoogleClientHttpOptions().baseUrl).toBe("https://proxy.example.com/v1beta/route");
});
it("passes baseUrl unchanged when no /v1beta suffix is present", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{ video: { videoBytes: Buffer.from("mp4").toString("base64"), mimeType: "video/mp4" } },
],
},
});
const provider = buildGoogleVideoGenerationProvider();
await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "test",
cfg: {
models: {
providers: {
google: { baseUrl: "https://generativelanguage.googleapis.com", models: [] },
},
},
},
durationSeconds: 3,
});
expect(firstGoogleClientHttpOptions().baseUrl).toBe(
"https://generativelanguage.googleapis.com",
);
});
it("rejects mixed image and video inputs", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
const provider = buildGoogleVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "Animate",
cfg: {},
inputImages: [{ buffer: Buffer.from("img"), mimeType: "image/png" }],
inputVideos: [{ buffer: Buffer.from("vid"), mimeType: "video/mp4" }],
}),
).rejects.toThrow("Google video generation does not support image and video inputs together.");
});
it("rounds unsupported durations to the nearest Veo value", async () => {
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
apiKey: "google-key",
source: "env",
mode: "api-key",
});
generateVideosMock.mockResolvedValue({
done: true,
response: {
generatedVideos: [
{
video: {
videoBytes: Buffer.from("mp4-bytes").toString("base64"),
mimeType: "video/mp4",
},
},
],
},
});
const provider = buildGoogleVideoGenerationProvider();
await provider.generateVideo({
provider: "google",
model: "veo-3.1-fast-generate-preview",
prompt: "A tiny robot watering a windowsill garden",
cfg: {},
durationSeconds: 5,
});
const request = firstObjectArg(generateVideosMock);
const config = recordField(request.config, "config");
expect(config.durationSeconds).toBe(6);
});
});

View File

@@ -0,0 +1,625 @@
// Google provider module implements model/runtime integration.
import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runtime";
import {
createProviderOperationDeadline,
executeProviderOperationWithRetry,
resolveProviderOperationTimeoutMs,
waitProviderOperationPollInterval,
} from "openclaw/plugin-sdk/provider-http";
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
import type {
GeneratedVideoAsset,
VideoGenerationProvider,
VideoGenerationRequest,
} from "openclaw/plugin-sdk/video-generation";
import { parseGeminiAuth, resolveGoogleGenerativeAiApiOrigin } from "./api.js";
import {
createGoogleVideoGenerationProviderMetadata,
DEFAULT_GOOGLE_VIDEO_MODEL,
GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS,
GOOGLE_VIDEO_MAX_DURATION_SECONDS,
GOOGLE_VIDEO_MIN_DURATION_SECONDS,
} from "./generation-provider-metadata.js";
import { createGoogleGenAI, type GoogleGenAIClient } from "./google-genai-runtime.js";
const DEFAULT_TIMEOUT_MS = 180_000;
const POLL_INTERVAL_MS = 10_000;
const MAX_POLL_ATTEMPTS = 120;
const DEFAULT_GENERATED_VIDEO_MAX_BYTES = 16 * 1024 * 1024;
const GOOGLE_VIDEO_OPERATION_RESPONSE_MAX_BYTES = 16 * 1024 * 1024;
const GOOGLE_VIDEO_EMPTY_RESULT_MESSAGE =
"Google video generation response missing generated videos";
function resolveConfiguredGoogleVideoBaseUrl(req: VideoGenerationRequest): string | undefined {
const configured = normalizeOptionalString(req.cfg?.models?.providers?.google?.baseUrl);
return configured ? resolveGoogleGenerativeAiApiOrigin(configured) : undefined;
}
function resolveGeneratedVideoMaxBytes(req: VideoGenerationRequest): number {
const configured = req.cfg.agents?.defaults?.mediaMaxMb;
if (typeof configured === "number" && Number.isFinite(configured) && configured > 0) {
return Math.floor(configured * 1024 * 1024);
}
return DEFAULT_GENERATED_VIDEO_MAX_BYTES;
}
function assertGeneratedVideoBufferWithinLimit(buffer: Buffer, maxBytes: number): void {
if (buffer.length > maxBytes) {
throw new Error(`Google generated video download exceeds ${maxBytes} bytes`);
}
}
function resolveGoogleVideoRestBaseUrl(configuredBaseUrl?: string): string {
return `${configuredBaseUrl ?? "https://generativelanguage.googleapis.com"}/v1beta`;
}
function resolveGoogleVideoRestModelPath(model: string): string {
const trimmed = normalizeOptionalString(model) || DEFAULT_GOOGLE_VIDEO_MODEL;
if (trimmed.startsWith("google/models/")) {
return trimmed.slice("google/".length);
}
if (trimmed.startsWith("models/")) {
return trimmed;
}
if (trimmed.startsWith("google/")) {
return `models/${trimmed.slice("google/".length)}`;
}
return `models/${trimmed}`;
}
function parseVideoSize(size: string | undefined): { width: number; height: number } | undefined {
const trimmed = normalizeOptionalString(size);
if (!trimmed) {
return undefined;
}
const match = /^(\d+)x(\d+)$/u.exec(trimmed);
if (!match) {
return undefined;
}
const width = Number.parseInt(match[1] ?? "", 10);
const height = Number.parseInt(match[2] ?? "", 10);
if (!Number.isFinite(width) || !Number.isFinite(height)) {
return undefined;
}
return { width, height };
}
function resolveAspectRatio(params: {
aspectRatio?: string;
size?: string;
}): "16:9" | "9:16" | undefined {
const direct = normalizeOptionalString(params.aspectRatio);
if (direct === "16:9" || direct === "9:16") {
return direct;
}
const parsedSize = parseVideoSize(params.size);
if (!parsedSize) {
return undefined;
}
return parsedSize.width >= parsedSize.height ? "16:9" : "9:16";
}
function resolveResolution(params: {
resolution?: string;
size?: string;
}): "720p" | "1080p" | undefined {
if (params.resolution === "720P") {
return "720p";
}
if (params.resolution === "1080P") {
return "1080p";
}
const parsedSize = parseVideoSize(params.size);
if (!parsedSize) {
return undefined;
}
const maxEdge = Math.max(parsedSize.width, parsedSize.height);
return maxEdge >= 1920 ? "1080p" : maxEdge >= 1280 ? "720p" : undefined;
}
function resolveDurationSeconds(durationSeconds: number | undefined): number | undefined {
if (typeof durationSeconds !== "number" || !Number.isFinite(durationSeconds)) {
return undefined;
}
const rounded = Math.min(
GOOGLE_VIDEO_MAX_DURATION_SECONDS,
Math.max(GOOGLE_VIDEO_MIN_DURATION_SECONDS, Math.round(durationSeconds)),
);
return GOOGLE_VIDEO_ALLOWED_DURATION_SECONDS.reduce((best, current) => {
const currentDistance = Math.abs(current - rounded);
const bestDistance = Math.abs(best - rounded);
if (currentDistance < bestDistance) {
return current;
}
if (currentDistance === bestDistance && current > best) {
return current;
}
return best;
});
}
function resolveInputImage(req: VideoGenerationRequest) {
const input = req.inputImages?.[0];
if (!input?.buffer) {
return undefined;
}
return {
imageBytes: input.buffer.toString("base64"),
mimeType: normalizeOptionalString(input.mimeType) || "image/png",
};
}
function resolveInputVideo(req: VideoGenerationRequest) {
const input = req.inputVideos?.[0];
if (!input?.buffer) {
return undefined;
}
return {
videoBytes: input.buffer.toString("base64"),
mimeType: normalizeOptionalString(input.mimeType) || "video/mp4",
};
}
function resolveGoogleGeneratedVideoDownloadUrl(params: {
uri: string | undefined;
apiKey: string;
configuredBaseUrl?: string;
}): string | undefined {
const trimmed = normalizeOptionalString(params.uri);
if (!trimmed) {
return undefined;
}
let url: URL;
try {
url = new URL(trimmed);
} catch {
return undefined;
}
if (url.protocol !== "https:") {
return undefined;
}
const allowedOrigins = new Set(["https://generativelanguage.googleapis.com"]);
if (params.configuredBaseUrl) {
try {
const configuredOrigin = new URL(params.configuredBaseUrl).origin;
if (configuredOrigin.startsWith("https://")) {
allowedOrigins.add(configuredOrigin);
}
} catch {
// Ignore invalid configured origins; resolveConfiguredGoogleVideoBaseUrl already normalizes.
}
}
if (!allowedOrigins.has(url.origin)) {
return undefined;
}
if (!url.searchParams.has("key")) {
url.searchParams.set("key", params.apiKey);
}
return url.toString();
}
function resolveGoogleGeneratedVideoFileDownloadUrl(params: {
file: unknown;
apiKey: string;
configuredBaseUrl?: string;
}): string | undefined {
const resource = params.file as { name?: unknown; uri?: unknown } | undefined;
const name = normalizeOptionalString(resource?.name) ?? normalizeOptionalString(resource?.uri);
if (!name || !/^files\/[^/?#]+$/u.test(name)) {
return undefined;
}
const baseUrl = resolveGoogleVideoRestBaseUrl(params.configuredBaseUrl);
const url = new URL(`${baseUrl}/${name}:download`);
url.searchParams.set("alt", "media");
url.searchParams.set("key", params.apiKey);
return url.toString();
}
async function downloadGeneratedVideoFromUri(params: {
uri: string | undefined;
apiKey: string;
configuredBaseUrl?: string;
mimeType?: string;
index: number;
maxBytes: number;
timeoutMs: number;
}): Promise<GeneratedVideoAsset | undefined> {
const downloadUrl = resolveGoogleGeneratedVideoDownloadUrl({
uri: params.uri,
apiKey: params.apiKey,
configuredBaseUrl: params.configuredBaseUrl,
});
if (!downloadUrl) {
return undefined;
}
return await executeProviderOperationWithRetry({
provider: "google",
stage: "download",
operation: async () => {
const { response, release } = await fetchWithSsrFGuard({
url: downloadUrl,
timeoutMs: params.timeoutMs,
});
try {
if (!response.ok) {
throw new Error(
`Failed to download Google generated video: ${response.status} ${response.statusText}`,
);
}
const buffer = await readResponseWithLimit(response, params.maxBytes, {
chunkTimeoutMs: params.timeoutMs,
onOverflow: ({ maxBytes }) =>
new Error(`Google generated video download exceeds ${maxBytes} bytes`),
onIdleTimeout: ({ chunkTimeoutMs }) =>
new Error(`Google generated video download stalled after ${chunkTimeoutMs}ms`),
});
return {
buffer,
mimeType:
normalizeOptionalString(response.headers.get("content-type")) ||
normalizeOptionalString(params.mimeType) ||
"video/mp4",
fileName: `video-${params.index + 1}.mp4`,
};
} finally {
await release();
}
},
});
}
function extractGoogleApiErrorCode(error: unknown): number | undefined {
const status = (error as { status?: unknown } | undefined)?.status;
if (typeof status === "number") {
return status;
}
const message = error instanceof Error ? error.message : String(error);
try {
const parsed = JSON.parse(message) as { code?: unknown; error?: { code?: unknown } };
const code = typeof parsed.code === "number" ? parsed.code : parsed.error?.code;
return typeof code === "number" ? code : undefined;
} catch {
return /\b404\b/u.test(message) ? 404 : undefined;
}
}
function extractGeneratedVideos(operation: unknown): Array<{ video?: unknown }> {
const response = (operation as { response?: Record<string, unknown> }).response;
const generatedVideos = response?.generatedVideos;
if (Array.isArray(generatedVideos) && generatedVideos.length > 0) {
return generatedVideos as Array<{ video?: unknown }>;
}
const generatedSamples = (response?.generateVideoResponse as { generatedSamples?: unknown })
?.generatedSamples;
return Array.isArray(generatedSamples) ? (generatedSamples as Array<{ video?: unknown }>) : [];
}
async function requestGoogleVideoJson(params: {
url: string;
method: "GET" | "POST";
headers: Record<string, string>;
deadline: ReturnType<typeof createProviderOperationDeadline>;
stage: "create" | "poll";
body?: unknown;
}): Promise<unknown> {
function createHttpError(response: Response, detail: unknown): Error {
const parts = [`HTTP ${response.status}`];
const statusText = response.statusText.trim();
if (statusText) {
parts.push(statusText);
}
if (typeof detail === "string") {
const trimmed = detail.trim();
if (trimmed) {
parts.push(trimmed);
}
} else if (detail && typeof detail === "object") {
parts.push(JSON.stringify(detail));
}
const error = new Error(parts.join(": "));
Object.assign(error, { status: response.status, statusCode: response.status });
return error;
}
return await executeProviderOperationWithRetry({
provider: "google",
stage: params.stage,
operation: async () => {
const controller = new AbortController();
const timeout = setTimeout(
() => {
const error = new Error("request timed out");
error.name = "TimeoutError";
controller.abort(error);
},
resolveProviderOperationTimeoutMs({
deadline: params.deadline,
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
}),
);
try {
const { response, release } = await fetchWithSsrFGuard({
url: params.url,
init: {
method: params.method,
headers: params.headers,
...(params.body === undefined ? {} : { body: JSON.stringify(params.body) }),
},
signal: controller.signal,
});
try {
const buffer = await readResponseWithLimit(
response,
GOOGLE_VIDEO_OPERATION_RESPONSE_MAX_BYTES,
{
onOverflow: ({ maxBytes }) =>
new Error(`Google video operation response exceeds ${maxBytes} bytes`),
},
);
const text = new TextDecoder().decode(buffer);
if (!response.ok) {
let detail: unknown = text;
if (text) {
try {
detail = JSON.parse(text) as unknown;
} catch {
detail = text;
}
}
throw createHttpError(response, detail);
}
const payload = text ? (JSON.parse(text) as unknown) : {};
return payload;
} finally {
await release();
}
} finally {
clearTimeout(timeout);
}
},
});
}
async function generateGoogleVideoViaRest(params: {
baseUrl: string;
headers: Record<string, string>;
deadline: ReturnType<typeof createProviderOperationDeadline>;
model: string;
prompt: string;
durationSeconds?: number;
aspectRatio?: "16:9" | "9:16";
resolution?: "720p" | "1080p";
}): Promise<unknown> {
let operation = await requestGoogleVideoJson({
url: `${params.baseUrl}/${resolveGoogleVideoRestModelPath(params.model)}:predictLongRunning`,
method: "POST",
headers: params.headers,
deadline: params.deadline,
stage: "create",
body: {
instances: [{ prompt: params.prompt }],
parameters: {
...(typeof params.durationSeconds === "number"
? { durationSeconds: params.durationSeconds }
: {}),
...(params.aspectRatio ? { aspectRatio: params.aspectRatio } : {}),
...(params.resolution ? { resolution: params.resolution } : {}),
},
},
});
for (let attempt = 0; !((operation as { done?: boolean }).done ?? false); attempt += 1) {
if (attempt >= MAX_POLL_ATTEMPTS) {
throw new Error("Google video generation did not finish in time");
}
await waitProviderOperationPollInterval({
deadline: params.deadline,
pollIntervalMs: POLL_INTERVAL_MS,
});
const operationName = (operation as { name?: unknown }).name;
if (typeof operationName !== "string" || !operationName) {
throw new Error("Google video operation response missing name for polling");
}
operation = await requestGoogleVideoJson({
url: `${params.baseUrl}/${operationName}`,
method: "GET",
headers: params.headers,
deadline: params.deadline,
stage: "poll",
});
}
const error = (operation as { error?: unknown }).error;
if (error) {
throw new Error(JSON.stringify(error));
}
return operation;
}
export function buildGoogleVideoGenerationProvider(): VideoGenerationProvider {
return {
...createGoogleVideoGenerationProviderMetadata(),
async generateVideo(req) {
if ((req.inputImages?.length ?? 0) > 1) {
throw new Error("Google video generation supports at most one input image.");
}
if ((req.inputVideos?.length ?? 0) > 1) {
throw new Error("Google video generation supports at most one input video.");
}
if ((req.inputImages?.length ?? 0) > 0 && (req.inputVideos?.length ?? 0) > 0) {
throw new Error(
"Google video generation does not support image and video inputs together.",
);
}
const auth = await resolveApiKeyForProvider({
provider: "google",
cfg: req.cfg,
agentDir: req.agentDir,
store: req.authStore,
});
if (!auth.apiKey) {
throw new Error("Google API key missing");
}
const apiKey = auth.apiKey;
const configuredBaseUrl = resolveConfiguredGoogleVideoBaseUrl(req);
const restBaseUrl = resolveGoogleVideoRestBaseUrl(configuredBaseUrl);
const authHeaders = parseGeminiAuth(apiKey).headers;
const durationSeconds = resolveDurationSeconds(req.durationSeconds);
const model = normalizeOptionalString(req.model) || DEFAULT_GOOGLE_VIDEO_MODEL;
const aspectRatio = resolveAspectRatio({ aspectRatio: req.aspectRatio, size: req.size });
const resolution = resolveResolution({ resolution: req.resolution, size: req.size });
const hasReferenceInputs =
(req.inputImages?.length ?? 0) > 0 || (req.inputVideos?.length ?? 0) > 0;
const deadline = createProviderOperationDeadline({
timeoutMs: req.timeoutMs,
label: "Google video generation",
});
const client = createGoogleGenAI({
apiKey,
httpOptions: {
...(configuredBaseUrl ? { baseUrl: configuredBaseUrl } : {}),
timeout: resolveProviderOperationTimeoutMs({
deadline,
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
}),
},
});
let usedRestFallback = false;
let operation;
try {
operation = await client.models.generateVideos({
model,
prompt: req.prompt,
image: resolveInputImage(req),
video: resolveInputVideo(req),
config: {
...(typeof durationSeconds === "number" ? { durationSeconds } : {}),
...(aspectRatio ? { aspectRatio } : {}),
...(resolution ? { resolution } : {}),
},
});
} catch (error) {
if (hasReferenceInputs || extractGoogleApiErrorCode(error) !== 404) {
throw error;
}
usedRestFallback = true;
operation = await generateGoogleVideoViaRest({
baseUrl: restBaseUrl,
headers: authHeaders,
deadline,
model,
prompt: req.prompt,
durationSeconds,
aspectRatio,
resolution,
});
}
if (!usedRestFallback) {
let sdkOperation = operation as Awaited<
ReturnType<GoogleGenAIClient["models"]["generateVideos"]>
>;
for (let attempt = 0; !(sdkOperation.done ?? false); attempt += 1) {
if (attempt >= MAX_POLL_ATTEMPTS) {
throw new Error("Google video generation did not finish in time");
}
await waitProviderOperationPollInterval({ deadline, pollIntervalMs: POLL_INTERVAL_MS });
resolveProviderOperationTimeoutMs({ deadline, defaultTimeoutMs: DEFAULT_TIMEOUT_MS });
sdkOperation = await executeProviderOperationWithRetry({
provider: "google",
stage: "poll",
operation: () => client.operations.getVideosOperation({ operation: sdkOperation }),
});
}
operation = sdkOperation;
}
const finalOperation = operation as { error?: unknown; name?: string };
if (finalOperation.error) {
throw new Error(JSON.stringify(finalOperation.error));
}
let generatedVideos = extractGeneratedVideos(operation);
if (generatedVideos.length === 0 && !hasReferenceInputs && !usedRestFallback) {
operation = await generateGoogleVideoViaRest({
baseUrl: restBaseUrl,
headers: authHeaders,
deadline,
model,
prompt: req.prompt,
durationSeconds,
aspectRatio,
resolution,
});
generatedVideos = extractGeneratedVideos(operation);
}
if (generatedVideos.length === 0) {
throw new Error(GOOGLE_VIDEO_EMPTY_RESULT_MESSAGE);
}
const maxVideoBytes = resolveGeneratedVideoMaxBytes(req);
const videos = await Promise.all(
generatedVideos.map(async (entry, index) => {
const inline = entry.video as
| { videoBytes?: string; uri?: string; mimeType?: string }
| undefined;
if (inline?.videoBytes) {
const buffer = Buffer.from(inline.videoBytes, "base64");
assertGeneratedVideoBufferWithinLimit(buffer, maxVideoBytes);
return {
buffer,
mimeType: normalizeOptionalString(inline.mimeType) || "video/mp4",
fileName: `video-${index + 1}.mp4`,
};
}
const directDownload = await downloadGeneratedVideoFromUri({
uri: inline?.uri,
apiKey,
configuredBaseUrl,
mimeType: inline?.mimeType,
index,
maxBytes: maxVideoBytes,
timeoutMs: resolveProviderOperationTimeoutMs({
deadline,
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
}),
});
if (directDownload) {
return directDownload;
}
if (!inline) {
throw new Error("Google generated video missing file handle");
}
const fileDownload = await downloadGeneratedVideoFromUri({
uri: resolveGoogleGeneratedVideoFileDownloadUrl({
file: inline,
apiKey,
configuredBaseUrl,
}),
apiKey,
configuredBaseUrl,
mimeType: inline.mimeType,
index,
maxBytes: maxVideoBytes,
timeoutMs: resolveProviderOperationTimeoutMs({
deadline,
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
}),
});
if (!fileDownload) {
throw new Error("Google generated video missing bounded download URL");
}
return fileDownload;
}),
);
return {
videos,
model,
metadata: finalOperation.name
? {
operationName: finalOperation.name,
}
: undefined,
};
},
};
}

View File

@@ -0,0 +1,2 @@
// Google API module exposes the plugin public contract.
export { createGeminiWebSearchProvider } from "./src/gemini-web-search-provider.js";

View File

@@ -0,0 +1,709 @@
// Google tests cover web search provider plugin behavior.
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-contracts";
import { withEnv, withEnvAsync, withFetchPreconnect } from "openclaw/plugin-sdk/test-env";
import { afterEach, describe, expect, it, vi } from "vitest";
import { testing, createGeminiWebSearchProvider } from "./src/gemini-web-search-provider.js";
type TestModelProviderConfig = NonNullable<
NonNullable<OpenClawConfig["models"]>["providers"]
>[string];
function installGeminiFetch() {
const mockFetch = vi.fn((_input?: RequestInfo | URL, _init?: RequestInit) =>
Promise.resolve(
new Response(
JSON.stringify({
candidates: [
{
content: { parts: [{ text: "Grounded answer" }] },
groundingMetadata: {
groundingChunks: [{ web: { uri: "https://example.com", title: "Example" } }],
},
},
],
}),
),
),
);
vi.stubGlobal("fetch", withFetchPreconnect(mockFetch));
return mockFetch;
}
function createGoogleModelProviderConfig(
overrides: Partial<TestModelProviderConfig>,
): TestModelProviderConfig {
return {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/",
models: [],
...overrides,
};
}
function requireFirstGeminiFetchCall(
mockFetch: ReturnType<typeof installGeminiFetch>,
): [RequestInfo | URL | undefined, RequestInit | undefined] {
const [call] = mockFetch.mock.calls;
if (!call) {
throw new Error("expected Gemini web search fetch call");
}
return call as [RequestInfo | URL | undefined, RequestInit | undefined];
}
function getFetchHeaders(mockFetch: ReturnType<typeof installGeminiFetch>): Record<string, string> {
const [, init] = requireFirstGeminiFetchCall(mockFetch);
return (init?.headers as Record<string, string> | undefined) ?? {};
}
function getGeminiFetchUrl(mockFetch: ReturnType<typeof installGeminiFetch>): string | undefined {
const [input] = requireFirstGeminiFetchCall(mockFetch);
if (typeof input === "string") {
return input;
}
if (input instanceof URL) {
return input.toString();
}
return input?.url;
}
function parseGeminiFetchBody(mockFetch: ReturnType<typeof installGeminiFetch>): {
contents?: Array<{ parts?: Array<{ text?: string }> }>;
tools?: Array<{ google_search?: { timeRangeFilter?: unknown } }>;
} {
const [, init] = requireFirstGeminiFetchCall(mockFetch);
const body = init?.body;
if (typeof body !== "string") {
throw new Error("Expected Gemini fetch body string");
}
return JSON.parse(body) as {
contents?: Array<{ parts?: Array<{ text?: string }> }>;
tools?: Array<{ google_search?: { timeRangeFilter?: unknown } }>;
};
}
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
describe("google web search provider", () => {
it("points missing-key users to fetch/browser alternatives", async () => {
await withEnvAsync({ GEMINI_API_KEY: undefined }, async () => {
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({ config: {}, searchConfig: {} });
if (!tool) {
throw new Error("Expected tool definition");
}
await expect(tool.execute({ query: "OpenClaw docs" })).resolves.toEqual({
docs: "https://docs.openclaw.ai/tools/web",
error: "missing_gemini_api_key",
message:
"web_search (gemini) needs an API key. Set GEMINI_API_KEY in the Gateway environment, configure plugins.entries.google.config.webSearch.apiKey, or reuse models.providers.google.apiKey. If you do not want to configure a search API key, use web_fetch for a specific URL or the browser tool for interactive pages.",
});
});
});
it("falls back to GEMINI_API_KEY from the environment", () => {
withEnv({ GEMINI_API_KEY: "AIza-env-test" }, () => {
expect(testing.resolveGeminiApiKey()).toBe("AIza-env-test");
});
});
it("prefers configured api keys over env fallbacks", () => {
withEnv({ GEMINI_API_KEY: "AIza-env-test" }, () => {
expect(testing.resolveGeminiApiKey({ apiKey: "AIza-configured-test" })).toBe(
"AIza-configured-test",
);
});
});
it("uses provider api keys only after env fallbacks", () => {
withEnv({ GEMINI_API_KEY: "AIza-env-test" }, () => {
expect(testing.resolveGeminiApiKey({ providerApiKey: "AIza-provider-test" })).toBe(
"AIza-env-test",
);
});
});
it("stores configured credentials at the canonical plugin config path", () => {
const provider = createGeminiWebSearchProvider();
const config = {} as OpenClawConfig;
provider.setConfiguredCredentialValue?.(config, "AIza-plugin-test");
expect(provider.credentialPath).toBe("plugins.entries.google.config.webSearch.apiKey");
expect(provider.getConfiguredCredentialValue?.(config)).toBe("AIza-plugin-test");
});
it("keeps model-provider fallback config runtime-only when Gemini config was injected", () => {
const searchConfig = Object.defineProperty({ provider: "gemini" }, "gemini", {
value: { apiKey: "AIza-plugin-test" },
enumerable: false,
configurable: true,
writable: true,
});
const merged = testing.withGoogleModelProviderFallbacks(searchConfig, {
models: {
providers: {
google: createGoogleModelProviderConfig({
apiKey: "AIza-provider-test",
baseUrl: "https://generativelanguage.googleapis.com/proxy/v1beta/",
}),
},
},
});
expect(merged?.gemini).toEqual({
apiKey: "AIza-plugin-test",
providerApiKey: "AIza-provider-test",
providerBaseUrl: "https://generativelanguage.googleapis.com/proxy/v1beta/",
});
expect(Object.keys(merged ?? {})).toEqual(["provider"]);
expect(Object.getOwnPropertyDescriptor(merged, "gemini")?.enumerable).toBe(false);
});
it("defaults the Gemini web search model and trims explicit overrides", () => {
expect(testing.resolveGeminiModel()).toBe("gemini-2.5-flash");
expect(testing.resolveGeminiModel({ model: " gemini-2.5-pro " })).toBe("gemini-2.5-pro");
});
it("routes Gemini web search through plugin webSearch.baseUrl", async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
baseUrl: "https://generativelanguage.googleapis.com/proxy/v1beta/",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "OpenClaw docs" });
expect(getGeminiFetchUrl(mockFetch)).toBe(
"https://generativelanguage.googleapis.com/proxy/v1beta/models/gemini-2.5-flash:generateContent",
);
});
it("accepts Gemini success JSON with empty grounding metadata", async () => {
vi.stubGlobal(
"fetch",
withFetchPreconnect(
vi.fn(() =>
Promise.resolve(
new Response(
JSON.stringify({
candidates: [
{
content: { parts: [{ text: "Today's date is Sunday, June 7, 2026." }] },
groundingMetadata: {},
},
],
}),
),
),
),
),
);
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
const result = await tool?.execute({ query: "current date today" });
expect(result).toMatchObject({
citations: [],
model: "gemini-2.5-flash",
provider: "gemini",
});
expect(String(result?.content)).toContain("Today's date is Sunday, June 7, 2026.");
});
it("reports malformed Gemini API JSON with a stable provider error", async () => {
vi.stubGlobal(
"fetch",
withFetchPreconnect(vi.fn(() => Promise.resolve(new Response("{ nope")))),
);
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await expect(tool?.execute({ query: "OpenClaw docs" })).rejects.toThrow(
"Gemini API error: malformed JSON response",
);
});
it("rejects wrong-root Gemini success JSON with a stable provider error", async () => {
vi.stubGlobal(
"fetch",
withFetchPreconnect(vi.fn(() => Promise.resolve(new Response(JSON.stringify([]))))),
);
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await expect(tool?.execute({ query: "OpenClaw docs" })).rejects.toThrow(
"Gemini API error: malformed JSON response",
);
});
it("rejects Gemini success JSON without candidate text", async () => {
vi.stubGlobal(
"fetch",
withFetchPreconnect(
vi.fn(() =>
Promise.resolve(
new Response(JSON.stringify({ candidates: [{ content: { parts: [] } }] })),
),
),
),
);
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await expect(tool?.execute({ query: "OpenClaw docs" })).rejects.toThrow(
"Gemini API error: malformed JSON response",
);
});
it("passes provider execution abort signals into the Gemini fetch", async () => {
const mockFetch = installGeminiFetch();
const controller = new AbortController();
controller.abort();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "OpenClaw docs" }, { signal: controller.signal });
const [, init] = requireFirstGeminiFetchCall(mockFetch);
expect(init?.signal?.aborted).toBe(true);
});
it("reuses the Google model provider key when no web search key or env key is set", async () => {
await withEnvAsync({ GEMINI_API_KEY: undefined }, async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
models: {
providers: {
google: createGoogleModelProviderConfig({
apiKey: "AIza-provider-test",
}),
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "OpenClaw provider key fallback" });
expect(getFetchHeaders(mockFetch)["x-goog-api-key"]).toBe("AIza-provider-test");
});
});
it("keeps plugin web search keys ahead of env and provider keys", async () => {
await withEnvAsync({ GEMINI_API_KEY: "AIza-env-test" }, async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
models: {
providers: {
google: createGoogleModelProviderConfig({
apiKey: "AIza-provider-test",
}),
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "OpenClaw plugin key precedence" });
expect(getFetchHeaders(mockFetch)["x-goog-api-key"]).toBe("AIza-plugin-test");
});
});
it("routes Gemini web search through provider-level google.baseUrl as a fallback", async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
models: {
providers: {
google: createGoogleModelProviderConfig({
apiKey: "AIza-provider-test",
baseUrl: "https://generativelanguage.googleapis.com/provider/v1beta/",
}),
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "OpenClaw provider baseUrl fallback" });
expect(getGeminiFetchUrl(mockFetch)).toBe(
"https://generativelanguage.googleapis.com/provider/v1beta/models/gemini-2.5-flash:generateContent",
);
});
it("keeps plugin webSearch.baseUrl ahead of provider-level google.baseUrl", async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
baseUrl: "https://generativelanguage.googleapis.com/plugin/v1beta/",
},
},
},
},
},
models: {
providers: {
google: createGoogleModelProviderConfig({
baseUrl: "https://generativelanguage.googleapis.com/provider/v1beta/",
}),
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "OpenClaw plugin baseUrl precedence" });
expect(getGeminiFetchUrl(mockFetch)).toBe(
"https://generativelanguage.googleapis.com/plugin/v1beta/models/gemini-2.5-flash:generateContent",
);
});
it("uses a soft recency hint for Gemini day freshness shortcuts instead of a 24-hour range", async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "latest ai news timestamp precision", freshness: "pd" });
const body = parseGeminiFetchBody(mockFetch);
expect(body.tools?.[0]?.google_search?.timeRangeFilter).toBeUndefined();
expect(body.contents?.[0]?.parts?.[0]?.text).toContain(
"Prioritize web sources published in the last 24 hours.",
);
});
it("preserves hard Gemini time ranges for wider freshness values", async () => {
vi.useFakeTimers({ toFake: ["Date"] });
vi.setSystemTime(new Date("2026-04-15T12:00:00.123Z"));
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "latest ai news timestamp precision", freshness: "week" });
const body = parseGeminiFetchBody(mockFetch);
expect(body.contents?.[0]?.parts?.[0]?.text).toBe("latest ai news timestamp precision");
expect(body.tools?.[0]?.google_search?.timeRangeFilter).toEqual({
startTime: "2026-04-08T12:00:00Z",
endTime: "2026-04-15T12:00:00Z",
});
});
it("partitions Gemini cache entries for soft day freshness, hard week freshness, and no freshness", async () => {
vi.useFakeTimers({ toFake: ["Date"] });
vi.setSystemTime(new Date("2026-04-15T12:00:00.123Z"));
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "same query cache partition", freshness: "day" });
await tool?.execute({ query: "same query cache partition", freshness: "week" });
await tool?.execute({ query: "same query cache partition" });
const postCalls = mockFetch.mock.calls.filter(([, init]) => typeof init?.body === "string");
expect(postCalls).toHaveLength(3);
const parsePostedBody = (call: (typeof postCalls)[number] | undefined) => {
const body = call?.[1]?.body;
if (typeof body !== "string") {
throw new Error("Expected Gemini fetch body to be a string");
}
return JSON.parse(body) as {
contents?: Array<{ parts?: Array<{ text?: string }> }>;
tools?: Array<{ google_search?: { timeRangeFilter?: unknown } }>;
};
};
const firstBody = parsePostedBody(postCalls[0]);
const secondBody = parsePostedBody(postCalls[1]);
const thirdBody = parsePostedBody(postCalls[2]);
expect(firstBody.tools?.[0]?.google_search?.timeRangeFilter).toBeUndefined();
expect(firstBody.contents?.[0]?.parts?.[0]?.text).toContain(
"Prioritize web sources published in the last 24 hours.",
);
expect(secondBody.tools?.[0]?.google_search?.timeRangeFilter).toEqual({
startTime: "2026-04-08T12:00:00Z",
endTime: "2026-04-15T12:00:00Z",
});
expect(secondBody.contents?.[0]?.parts?.[0]?.text).toBe("same query cache partition");
expect(thirdBody.tools?.[0]?.google_search?.timeRangeFilter).toBeUndefined();
expect(thirdBody.contents?.[0]?.parts?.[0]?.text).toBe("same query cache partition");
});
it("strips sub-second precision from date-range timestamps so Gemini accepts them", async () => {
vi.useFakeTimers({ toFake: ["Date"] });
// "now" with non-zero milliseconds. Without stripping, toISOString() emits
// "2026-04-15T12:00:00.123Z", which Gemini's google_search.time_range_filter
// rejects with "Granularity of nano is not supported".
vi.setSystemTime(new Date("2026-04-15T12:00:00.123Z"));
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({ query: "latest ai news", date_after: "2026-04-01" });
const body = parseGeminiFetchBody(mockFetch);
const filter = body.tools?.[0]?.google_search?.timeRangeFilter as
| { startTime: string; endTime: string }
| undefined;
expect(filter?.startTime).not.toMatch(/\.\d+Z$/);
expect(filter?.endTime).not.toMatch(/\.\d+Z$/);
expect(filter).toEqual({
startTime: "2026-04-01T00:00:00Z",
endTime: "2026-04-15T12:00:00Z",
});
});
it("passes date ranges to Gemini Google Search grounding", async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await tool?.execute({
query: "OpenClaw release notes",
date_after: "2026-04-01",
date_before: "2026-04-30",
});
const body = parseGeminiFetchBody(mockFetch);
expect(body.tools?.[0]?.google_search?.timeRangeFilter).toEqual({
startTime: "2026-04-01T00:00:00Z",
endTime: "2026-05-01T00:00:00Z",
});
});
it("returns validation errors for invalid Gemini time filters before fetch", async () => {
const mockFetch = installGeminiFetch();
const provider = createGeminiWebSearchProvider();
const tool = provider.createTool({
config: {
plugins: {
entries: {
google: {
config: {
webSearch: {
apiKey: "AIza-plugin-test",
},
},
},
},
},
},
searchConfig: { provider: "gemini" },
});
await expect(
tool?.execute({
query: "OpenClaw release notes",
freshness: "week",
date_after: "2026-04-01",
}),
).resolves.toEqual({
docs: "https://docs.openclaw.ai/tools/web",
error: "conflicting_time_filters",
message:
"freshness and date_after/date_before cannot be used together. Use either freshness (day/week/month/year) or a date range (date_after/date_before), not both.",
});
expect(mockFetch).not.toHaveBeenCalled();
});
it("normalizes Gemini shorthand base URLs", () => {
expect(
testing.resolveGeminiBaseUrl({ baseUrl: "https://generativelanguage.googleapis.com" }),
).toBe("https://generativelanguage.googleapis.com/v1beta");
});
});

View File

@@ -0,0 +1,2 @@
// Google provider module implements model/runtime integration.
export { createGeminiWebSearchProvider } from "./src/gemini-web-search-provider.js";