Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled

Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
2026-07-05 09:36:54 +00:00
parent 3216769225
commit bedb527145
21108 changed files with 6010766 additions and 0 deletions

View File

@@ -0,0 +1,129 @@
// Comfy tests cover comfy plugin behavior.
import { resolveDefaultAgentDir } from "openclaw/plugin-sdk/agent-runtime";
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-contracts";
import { createTestPluginApi } from "openclaw/plugin-sdk/plugin-test-api";
import { getRuntimeConfig } from "openclaw/plugin-sdk/runtime-config-snapshot";
import { isLiveTestEnabled } from "openclaw/plugin-sdk/test-env";
import { beforeAll, describe, expect, it } from "vitest";
import plugin from "./index.js";
import { getComfyConfig, isComfyCapabilityConfigured } from "./workflow-runtime.js";
const LIVE =
isLiveTestEnabled(["COMFY_LIVE_TEST"]) && (process.env.COMFY_LIVE_TEST ?? "").trim() === "1";
const describeLive = LIVE ? describe : describe.skip;
type RegisteredMusicProvider = {
id: string;
generateMusic: Function;
isConfigured?: Function;
};
function withPluginsEnabled<T>(cfg: T): T {
if (!cfg || typeof cfg !== "object") {
return cfg;
}
const record = cfg as Record<string, unknown>;
return {
...record,
plugins: {
...(record.plugins && typeof record.plugins === "object" ? record.plugins : {}),
enabled: true,
},
} as T;
}
function requireProvider<T extends { id: string }>(providers: T[], id: string): T {
const provider = providers.find((entry) => entry.id === id);
if (!provider) {
throw new Error(`expected ${id} provider to be registered`);
}
return provider;
}
describeLive("comfy live", () => {
let cfg = {} as OpenClawConfig;
let agentDir = "";
const imageProviders: Array<{ id: string; generateImage: Function; isConfigured?: Function }> =
[];
const musicProviders: RegisteredMusicProvider[] = [];
const videoProviders: Array<{ id: string; generateVideo: Function; isConfigured?: Function }> =
[];
beforeAll(async () => {
cfg = withPluginsEnabled(getRuntimeConfig());
agentDir = resolveDefaultAgentDir(cfg as never);
plugin.register(
createTestPluginApi({
config: cfg as never,
registerImageGenerationProvider(provider) {
imageProviders.push(provider as never);
},
registerMusicGenerationProvider(provider) {
musicProviders.push(provider as never);
},
registerVideoGenerationProvider(provider) {
videoProviders.push(provider as never);
},
}),
);
});
it.skipIf(!isComfyCapabilityConfigured({ cfg: cfg as never, agentDir, capability: "image" }))(
"runs an image workflow",
async () => {
const provider = requireProvider(imageProviders, "comfy");
const result = await provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "A tiny orange lobster icon on a clean background.",
cfg: cfg as never,
agentDir,
});
expect(result.images.length).toBeGreaterThan(0);
expect(result.images[0]?.mimeType.startsWith("image/")).toBe(true);
expect(result.images[0]?.buffer.byteLength).toBeGreaterThan(128);
},
120_000,
);
it.skipIf(!isComfyCapabilityConfigured({ cfg: cfg as never, agentDir, capability: "video" }))(
"runs a video workflow",
async () => {
const provider = requireProvider(videoProviders, "comfy");
const result = await provider.generateVideo({
provider: "comfy",
model: "workflow",
prompt: "A tiny paper lobster gently waving, cinematic motion.",
cfg: cfg as never,
agentDir,
});
expect(result.videos.length).toBeGreaterThan(0);
expect(result.videos[0]?.mimeType.startsWith("video/")).toBe(true);
expect(result.videos[0]?.buffer.byteLength).toBeGreaterThan(512);
},
180_000,
);
it.skipIf(!isComfyCapabilityConfigured({ cfg: cfg as never, agentDir, capability: "music" }))(
"runs a music workflow",
async () => {
const provider = requireProvider(musicProviders, "comfy");
const result = await provider.generateMusic({
provider: "comfy",
model: "workflow",
prompt: "A gentle ambient synth loop with warm analog pads.",
cfg: cfg as never,
agentDir,
});
expect(result.tracks.length).toBeGreaterThan(0);
expect(result.tracks[0]?.mimeType.startsWith("audio/")).toBe(true);
expect(result.tracks[0]?.buffer.byteLength).toBeGreaterThan(512);
},
180_000,
);
it("documents the effective comfy config shape for live debugging", () => {
const comfyConfig = getComfyConfig(cfg as never);
expect(typeof comfyConfig).toBe("object");
});
});

View File

@@ -0,0 +1,566 @@
// Comfy tests cover image generation provider plugin behavior.
import { MAX_TIMER_TIMEOUT_MS } from "openclaw/plugin-sdk/number-runtime";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
setComfyFetchGuardForTesting,
buildComfyImageGenerationProvider,
} from "./image-generation-provider.js";
import {
buildComfyConfig,
buildLegacyComfyConfig,
mockComfyCloudJobResponses,
mockComfyProviderApiKey,
parseComfyJsonBody,
} from "./test-helpers.js";
const { fetchWithSsrFGuardMock } = vi.hoisted(() => ({
fetchWithSsrFGuardMock: vi.fn(),
}));
type FetchGuardRequest = {
url?: unknown;
auditContext?: unknown;
timeoutMs?: unknown;
init?: {
method?: unknown;
headers?: HeadersInit;
body?: BodyInit | null;
};
};
function fetchRequest(call: number): FetchGuardRequest {
const request = fetchWithSsrFGuardMock.mock.calls[call - 1]?.[0] as FetchGuardRequest | undefined;
if (!request) {
throw new Error(`expected Comfy fetch call ${call}`);
}
return request;
}
function parseJsonBody(call: number): Record<string, unknown> {
return parseComfyJsonBody(fetchWithSsrFGuardMock, call);
}
describe("comfy image-generation provider", () => {
beforeEach(() => {
vi.clearAllMocks();
});
afterEach(() => {
setComfyFetchGuardForTesting(null);
vi.unstubAllEnvs();
vi.restoreAllMocks();
});
it("treats local comfy workflows as configured without an API key", () => {
const provider = buildComfyImageGenerationProvider();
expect(
provider.isConfigured?.({
cfg: buildComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
},
promptNodeId: "6",
}),
}),
).toBe(true);
});
it("falls back to legacy models.providers comfy config when plugin config is absent", () => {
const provider = buildComfyImageGenerationProvider();
expect(
provider.isConfigured?.({
cfg: buildLegacyComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
},
promptNodeId: "6",
}),
}),
).toBe(true);
});
it("treats cloud comfy workflows as configured with a plugin config API key", () => {
const provider = buildComfyImageGenerationProvider();
expect(
provider.isConfigured?.({
cfg: buildComfyConfig({
mode: "cloud",
apiKey: "comfy-test-key",
image: {
workflow: {
"6": { inputs: { text: "" } },
},
promptNodeId: "6",
},
}),
}),
).toBe(true);
});
it("treats cloud comfy workflows as configured with a plugin config env SecretRef", () => {
vi.stubEnv("COMFY_TEST_API_KEY", "comfy-secret-ref-key");
const provider = buildComfyImageGenerationProvider();
expect(
provider.isConfigured?.({
cfg: buildComfyConfig({
mode: "cloud",
apiKey: { source: "env", provider: "default", id: "COMFY_TEST_API_KEY" },
image: {
workflow: {
"6": { inputs: { text: "" } },
},
promptNodeId: "6",
},
}),
}),
).toBe(true);
});
it("submits a local workflow, waits for history, and downloads images", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "local-prompt-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"local-prompt-1": {
outputs: {
"9": {
images: [{ filename: "generated.png", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("png-data"), {
status: 200,
headers: { "content-type": "image/png" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyImageGenerationProvider();
const result = await provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "draw a lobster",
cfg: buildComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
}),
});
const submitRequest = fetchRequest(1);
expect(submitRequest.url).toBe("http://127.0.0.1:8188/prompt");
expect(submitRequest.auditContext).toBe("comfy-image-generate");
expect(parseJsonBody(1)).toEqual({
prompt: {
"6": { inputs: { text: "draw a lobster" } },
"9": { inputs: {} },
},
});
const historyRequest = fetchRequest(2);
expect(historyRequest.url).toBe("http://127.0.0.1:8188/history/local-prompt-1");
expect(historyRequest.auditContext).toBe("comfy-history");
const downloadRequest = fetchRequest(3);
expect(downloadRequest.url).toBe(
"http://127.0.0.1:8188/view?filename=generated.png&subfolder=&type=output",
);
expect(downloadRequest.auditContext).toBe("comfy-image-download");
expect(result).toEqual({
images: [
{
buffer: Buffer.from("png-data"),
mimeType: "image/png",
fileName: "generated.png",
metadata: {
nodeId: "9",
promptId: "local-prompt-1",
},
},
],
model: "workflow",
metadata: {
promptId: "local-prompt-1",
outputNodeIds: ["9"],
},
});
});
it("caps oversized local workflow timeouts", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
const nowSpy = vi.spyOn(Date, "now");
nowSpy
.mockReturnValueOnce(0)
.mockReturnValueOnce(0)
.mockReturnValueOnce(MAX_TIMER_TIMEOUT_MS + 1);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "local-prompt-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ "local-prompt-1": { outputs: {} } }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
});
try {
const provider = buildComfyImageGenerationProvider();
await expect(
provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "draw a bounded timer",
cfg: buildComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
timeoutMs: Number.MAX_SAFE_INTEGER,
}),
}),
).rejects.toThrow("Comfy workflow did not finish within 2147000s");
expect(fetchRequest(1).timeoutMs).toBe(MAX_TIMER_TIMEOUT_MS);
expect(fetchRequest(2).timeoutMs).toBe(MAX_TIMER_TIMEOUT_MS);
} finally {
nowSpy.mockRestore();
}
});
it("rejects generated image downloads that exceed the configured media cap", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "local-prompt-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"local-prompt-1": {
outputs: {
"9": {
images: [{ filename: "generated.png", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("too-large"), {
status: 200,
headers: { "content-type": "image/png" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyImageGenerationProvider();
await expect(
provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "draw a lobster",
cfg: {
...buildComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
}),
agents: { defaults: { mediaMaxMb: 0.000001 } },
} as never,
}),
).rejects.toThrow("Comfy image output download exceeds 1 bytes");
});
it("reports malformed local workflow submit JSON as a provider error", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
const release = vi.fn(async () => {});
fetchWithSsrFGuardMock.mockResolvedValueOnce({
response: new Response("{ nope", {
status: 200,
headers: { "content-type": "application/json" },
}),
release,
});
const provider = buildComfyImageGenerationProvider();
await expect(
provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "draw a lobster",
cfg: buildComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
}),
}),
).rejects.toThrow("Comfy workflow submit failed: malformed JSON response");
expect(release).toHaveBeenCalledTimes(1);
});
it("uploads reference images for local edit workflows", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ name: "upload.png" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "local-edit-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"local-edit-1": {
outputs: {
"9": {
images: [{ filename: "edited.png", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("edited-data"), {
status: 200,
headers: { "content-type": "image/png" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyImageGenerationProvider();
await provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "turn this into a poster",
cfg: buildComfyConfig({
workflow: {
"6": { inputs: { text: "" } },
"7": { inputs: { image: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
inputImageNodeId: "7",
outputNodeId: "9",
}),
inputImages: [
{
buffer: Buffer.from("source"),
mimeType: "image/png",
fileName: "source.png",
},
],
});
const uploadRequest = fetchRequest(1);
expect(uploadRequest?.url).toBe("http://127.0.0.1:8188/upload/image");
expect(uploadRequest?.auditContext).toBe("comfy-image-upload");
expect(uploadRequest?.init?.method).toBe("POST");
const uploadForm = uploadRequest?.init?.body;
if (!(uploadForm instanceof FormData)) {
throw new Error("expected Comfy upload request body to be FormData");
}
expect(uploadForm.get("type")).toBe("input");
expect(uploadForm.get("overwrite")).toBe("true");
expect(parseJsonBody(2)).toEqual({
prompt: {
"6": { inputs: { text: "turn this into a poster" } },
"7": { inputs: { image: "upload.png" } },
"9": { inputs: {} },
},
});
});
it("uses cloud endpoints, auth headers, and partner-node extra_data", async () => {
mockComfyProviderApiKey();
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
mockComfyCloudJobResponses(fetchWithSsrFGuardMock, {
body: Buffer.from("cloud-data"),
contentType: "image/png",
filename: "cloud.png",
outputKind: "images",
promptId: "cloud-job-1",
redirectLocation: "https://cdn.example.com/cloud.png",
});
const provider = buildComfyImageGenerationProvider();
const result = await provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "cloud workflow prompt",
cfg: buildComfyConfig({
mode: "cloud",
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
}),
});
const submitRequest = fetchRequest(1);
expect(submitRequest?.url).toBe("https://cloud.comfy.org/api/prompt");
expect(submitRequest?.auditContext).toBe("comfy-image-generate");
const submitHeaders = new Headers(submitRequest?.init?.headers);
expect(submitHeaders.get("x-api-key")).toBe("comfy-test-key");
expect(parseJsonBody(1)).toEqual({
prompt: {
"6": { inputs: { text: "cloud workflow prompt" } },
"9": { inputs: {} },
},
extra_data: {
api_key_comfy_org: "comfy-test-key",
},
});
const statusRequest = fetchRequest(2);
expect(statusRequest.url).toBe("https://cloud.comfy.org/api/job/cloud-job-1/status");
expect(statusRequest.auditContext).toBe("comfy-status");
const historyRequest = fetchRequest(3);
expect(historyRequest.url).toBe("https://cloud.comfy.org/api/history_v2/cloud-job-1");
expect(historyRequest.auditContext).toBe("comfy-history");
const viewRequest = fetchRequest(4);
expect(viewRequest.url).toBe(
"https://cloud.comfy.org/api/view?filename=cloud.png&subfolder=&type=output",
);
expect(viewRequest.auditContext).toBe("comfy-image-download");
const cdnRequest = fetchRequest(5);
expect(cdnRequest.url).toBe("https://cdn.example.com/cloud.png");
expect(cdnRequest.auditContext).toBe("comfy-image-download");
expect(result.metadata).toEqual({
promptId: "cloud-job-1",
outputNodeIds: ["9"],
});
});
it("uses plugin config env SecretRef auth for cloud workflows", async () => {
vi.stubEnv("COMFY_TEST_API_KEY", "comfy-secret-ref-key");
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
mockComfyCloudJobResponses(fetchWithSsrFGuardMock, {
body: Buffer.from("cloud-data"),
contentType: "image/png",
filename: "cloud.png",
outputKind: "images",
promptId: "cloud-secret-ref-1",
redirectLocation: "https://cdn.example.com/cloud.png",
});
const provider = buildComfyImageGenerationProvider();
await provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "cloud workflow prompt",
cfg: buildComfyConfig({
mode: "cloud",
apiKey: { source: "env", provider: "default", id: "COMFY_TEST_API_KEY" },
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
}),
});
const submitRequest = fetchRequest(1);
const submitHeaders = new Headers(submitRequest?.init?.headers);
expect(submitHeaders.get("x-api-key")).toBe("comfy-secret-ref-key");
const requestBody = parseJsonBody(1);
const extraData = requestBody.extra_data as { api_key_comfy_org?: unknown } | undefined;
expect(extraData?.api_key_comfy_org).toBe("comfy-secret-ref-key");
});
it("uses provider auth fallback for cloud workflows without plugin config API keys", async () => {
vi.stubEnv("COMFY_API_KEY", "stale-env-key");
mockComfyProviderApiKey("profile-key");
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
mockComfyCloudJobResponses(fetchWithSsrFGuardMock, {
body: Buffer.from("cloud-data"),
contentType: "image/png",
filename: "cloud.png",
outputKind: "images",
promptId: "cloud-profile-1",
redirectLocation: "https://cdn.example.com/cloud.png",
});
const provider = buildComfyImageGenerationProvider();
await provider.generateImage({
provider: "comfy",
model: "workflow",
prompt: "cloud workflow prompt",
cfg: buildComfyConfig({
mode: "cloud",
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
}),
});
const submitRequest = fetchRequest(1);
const submitHeaders = new Headers(submitRequest?.init?.headers);
expect(submitHeaders.get("x-api-key")).toBe("profile-key");
const requestBody = parseJsonBody(1);
const extraData = requestBody.extra_data as { api_key_comfy_org?: unknown } | undefined;
expect(extraData?.api_key_comfy_org).toBe("profile-key");
});
});

View File

@@ -0,0 +1,80 @@
// Comfy provider module implements model/runtime integration.
import type {
GeneratedImageAsset,
ImageGenerationProvider,
} from "openclaw/plugin-sdk/image-generation";
import {
DEFAULT_COMFY_MODEL,
setComfyFetchGuardForTesting,
isComfyCapabilityConfigured,
runComfyWorkflow,
} from "./workflow-runtime.js";
export { setComfyFetchGuardForTesting };
export function buildComfyImageGenerationProvider(): ImageGenerationProvider {
return {
id: "comfy",
label: "ComfyUI",
defaultModel: DEFAULT_COMFY_MODEL,
models: [DEFAULT_COMFY_MODEL],
isConfigured: ({ cfg, agentDir }) =>
isComfyCapabilityConfigured({
cfg,
agentDir,
capability: "image",
}),
capabilities: {
generate: {
maxCount: 1,
supportsSize: false,
supportsAspectRatio: false,
supportsResolution: false,
},
edit: {
enabled: true,
maxCount: 1,
maxInputImages: 1,
supportsSize: false,
supportsAspectRatio: false,
supportsResolution: false,
},
},
async generateImage(req) {
if ((req.inputImages?.length ?? 0) > 1) {
throw new Error("Comfy image generation currently supports at most one reference image");
}
const result = await runComfyWorkflow({
cfg: req.cfg,
agentDir: req.agentDir,
authStore: req.authStore,
prompt: req.prompt,
model: req.model,
timeoutMs: req.timeoutMs,
capability: "image",
outputKinds: ["images"],
inputImage: req.inputImages?.[0],
});
const images: GeneratedImageAsset[] = result.assets.map((asset) => ({
buffer: asset.buffer,
mimeType: asset.mimeType,
fileName: asset.fileName,
metadata: {
nodeId: asset.nodeId,
promptId: result.promptId,
},
}));
return {
images,
model: result.model,
metadata: {
promptId: result.promptId,
outputNodeIds: result.outputNodeIds,
},
};
},
};
}

View File

@@ -0,0 +1,52 @@
// Comfy tests cover index plugin behavior.
import fs from "node:fs";
import {
registerSingleProviderPlugin,
resolveProviderPluginChoice,
} from "openclaw/plugin-sdk/plugin-test-runtime";
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
type ComfyManifest = {
providerAuthChoices?: Array<{ choiceId?: string; method?: string; provider?: string }>;
};
function readManifest(): ComfyManifest {
return JSON.parse(
fs.readFileSync(new URL("./openclaw.plugin.json", import.meta.url), "utf8"),
) as ComfyManifest;
}
describe("comfy provider plugin", () => {
it("registers cloud API-key auth metadata", async () => {
const provider = await registerSingleProviderPlugin(plugin);
expect(provider.id).toBe("comfy");
expect(provider.envVars).toEqual(["COMFY_API_KEY", "COMFY_CLOUD_API_KEY"]);
expect(provider.auth?.map((method) => method.id)).toEqual(["cloud-api-key"]);
const choice = resolveProviderPluginChoice({
providers: [provider],
choice: "comfy-cloud-api-key",
});
expect(choice?.provider.id).toBe("comfy");
expect(choice?.method.id).toBe("cloud-api-key");
expect(readManifest().providerAuthChoices).toEqual([
{
provider: "comfy",
method: "cloud-api-key",
choiceId: "comfy-cloud-api-key",
choiceLabel: "Comfy Cloud API key",
choiceHint: "Required for cloud workflows",
cliOption: "--comfy-api-key <key>",
cliFlag: "--comfy-api-key",
cliDescription: "Comfy Cloud API key",
optionKey: "comfyApiKey",
groupId: "comfy",
groupLabel: "ComfyUI",
groupHint: "Local or cloud workflows",
onboardingScopes: ["image-generation"],
},
]);
});
});

46
extensions/comfy/index.ts Normal file
View File

@@ -0,0 +1,46 @@
// Comfy plugin entrypoint registers its OpenClaw integration.
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
import { createProviderApiKeyAuthMethod } from "openclaw/plugin-sdk/provider-auth-api-key";
import { buildComfyImageGenerationProvider } from "./image-generation-provider.js";
import { buildComfyMusicGenerationProvider } from "./music-generation-provider.js";
import { buildComfyVideoGenerationProvider } from "./video-generation-provider.js";
const PROVIDER_ID = "comfy";
export default definePluginEntry({
id: PROVIDER_ID,
name: "ComfyUI Provider",
description: "Bundled ComfyUI workflow media generation provider",
register(api) {
api.registerProvider({
id: PROVIDER_ID,
label: "ComfyUI",
docsPath: "/providers/comfy",
envVars: ["COMFY_API_KEY", "COMFY_CLOUD_API_KEY"],
auth: [
createProviderApiKeyAuthMethod({
providerId: PROVIDER_ID,
methodId: "cloud-api-key",
label: "Comfy Cloud API key",
hint: "API key for Comfy Cloud workflow runs",
optionKey: "comfyApiKey",
flagName: "--comfy-api-key",
envVar: "COMFY_API_KEY",
promptMessage: "Enter Comfy Cloud API key",
wizard: {
choiceId: "comfy-cloud-api-key",
choiceLabel: "Comfy Cloud API key",
choiceHint: "Required for cloud workflows",
groupId: "comfy",
groupLabel: "ComfyUI",
groupHint: "Local or cloud workflows",
onboardingScopes: ["image-generation"],
},
}),
],
});
api.registerImageGenerationProvider(buildComfyImageGenerationProvider());
api.registerMusicGenerationProvider(buildComfyMusicGenerationProvider());
api.registerVideoGenerationProvider(buildComfyVideoGenerationProvider());
},
});

View File

@@ -0,0 +1,167 @@
// Comfy tests cover music generation provider plugin behavior.
import { expectExplicitMusicGenerationCapabilities } from "openclaw/plugin-sdk/provider-test-contracts";
import { afterEach, describe, expect, it, vi } from "vitest";
import { buildComfyMusicGenerationProvider } from "./music-generation-provider.js";
import { setComfyFetchGuardForTesting } from "./workflow-runtime.js";
const { fetchWithSsrFGuardMock } = vi.hoisted(() => ({
fetchWithSsrFGuardMock: vi.fn(),
}));
describe("comfy music-generation provider", () => {
afterEach(() => {
setComfyFetchGuardForTesting(null);
vi.clearAllMocks();
});
it("registers the workflow model", () => {
const provider = buildComfyMusicGenerationProvider();
expect(provider.defaultModel).toBe("workflow");
expect(provider.models).toEqual(["workflow"]);
expectExplicitMusicGenerationCapabilities(provider);
});
it("runs a music workflow and returns audio outputs", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "music-job-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"music-job-1": {
outputs: {
"9": {
audio: [{ filename: "song.mp3", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("music-bytes"), {
status: 200,
headers: { "content-type": "audio/mpeg" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyMusicGenerationProvider();
const result = await provider.generateMusic({
provider: "comfy",
model: "workflow",
prompt: "gentle ambient synth loop",
cfg: {
plugins: {
entries: {
comfy: {
config: {
music: {
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
},
},
},
},
},
} as never,
});
expect(result).toEqual({
model: "workflow",
tracks: [
{
buffer: Buffer.from("music-bytes"),
mimeType: "audio/mpeg",
fileName: "song.mp3",
},
],
metadata: {
promptId: "music-job-1",
outputNodeIds: ["9"],
inputImageCount: 0,
},
});
});
it("rejects generated music downloads that exceed the configured media cap", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "music-job-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"music-job-1": {
outputs: {
"9": {
audio: [{ filename: "song.mp3", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("too-large"), {
status: 200,
headers: { "content-type": "audio/mpeg" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyMusicGenerationProvider();
await expect(
provider.generateMusic({
provider: "comfy",
model: "workflow",
prompt: "gentle ambient synth loop",
cfg: {
plugins: {
entries: {
comfy: {
config: {
music: {
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
},
},
},
},
},
agents: { defaults: { mediaMaxMb: 0.000001 } },
} as never,
}),
).rejects.toThrow("Comfy music output download exceeds 1 bytes");
});
});

View File

@@ -0,0 +1,89 @@
// Comfy provider module implements model/runtime integration.
import type {
GeneratedMusicAsset,
MusicGenerationProvider,
MusicGenerationSourceImage,
} from "openclaw/plugin-sdk/music-generation";
import {
DEFAULT_COMFY_MODEL,
isComfyCapabilityConfigured,
runComfyWorkflow,
} from "./workflow-runtime.js";
const COMFY_MAX_INPUT_IMAGES = 1;
function toGeneratedTrack(asset: {
buffer: Buffer;
mimeType: string;
fileName: string;
}): GeneratedMusicAsset {
return {
buffer: asset.buffer,
mimeType: asset.mimeType,
fileName: asset.fileName,
};
}
function resolveInputImage(inputImage: MusicGenerationSourceImage | undefined) {
if (!inputImage) {
return undefined;
}
if (!inputImage.buffer) {
throw new Error("Comfy music generation requires loaded reference image bytes.");
}
return {
buffer: inputImage.buffer,
mimeType: inputImage.mimeType ?? "image/png",
fileName: inputImage.fileName,
};
}
export function buildComfyMusicGenerationProvider(): MusicGenerationProvider {
return {
id: "comfy",
label: "ComfyUI",
defaultModel: DEFAULT_COMFY_MODEL,
models: [DEFAULT_COMFY_MODEL],
isConfigured: ({ cfg, agentDir }) =>
isComfyCapabilityConfigured({
cfg,
agentDir,
capability: "music",
}),
capabilities: {
generate: {},
edit: {
enabled: true,
maxInputImages: COMFY_MAX_INPUT_IMAGES,
},
},
async generateMusic(req) {
if ((req.inputImages?.length ?? 0) > COMFY_MAX_INPUT_IMAGES) {
throw new Error(
`Comfy music generation supports at most ${COMFY_MAX_INPUT_IMAGES} reference image.`,
);
}
const result = await runComfyWorkflow({
cfg: req.cfg,
agentDir: req.agentDir,
authStore: req.authStore,
prompt: req.prompt,
model: req.model,
capability: "music",
outputKinds: ["audio"],
inputImage: resolveInputImage(req.inputImages?.[0]),
});
return {
tracks: result.assets.map(toGeneratedTrack),
model: result.model,
metadata: {
promptId: result.promptId,
outputNodeIds: result.outputNodeIds,
inputImageCount: req.inputImages?.length ?? 0,
},
};
},
};
}

View File

@@ -0,0 +1,273 @@
{
"id": "comfy",
"activation": {
"onStartup": false
},
"enabledByDefault": true,
"providers": ["comfy"],
"setup": {
"providers": [
{
"id": "comfy",
"envVars": ["COMFY_API_KEY", "COMFY_CLOUD_API_KEY"]
}
]
},
"providerAuthChoices": [
{
"provider": "comfy",
"method": "cloud-api-key",
"choiceId": "comfy-cloud-api-key",
"choiceLabel": "Comfy Cloud API key",
"choiceHint": "Required for cloud workflows",
"groupId": "comfy",
"groupLabel": "ComfyUI",
"groupHint": "Local or cloud workflows",
"optionKey": "comfyApiKey",
"cliFlag": "--comfy-api-key",
"cliOption": "--comfy-api-key <key>",
"cliDescription": "Comfy Cloud API key",
"onboardingScopes": ["image-generation"]
}
],
"contracts": {
"imageGenerationProviders": ["comfy"],
"musicGenerationProviders": ["comfy"],
"videoGenerationProviders": ["comfy"]
},
"imageGenerationProviderMetadata": {
"comfy": {
"configSignals": [
{
"rootPath": "plugins.entries.comfy.config",
"overlayPath": "image",
"mode": {
"path": "mode",
"default": "local",
"allowed": ["local"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId"]
},
{
"rootPath": "models.providers.comfy",
"overlayPath": "image",
"mode": {
"path": "mode",
"default": "local",
"allowed": ["local"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId"]
},
{
"rootPath": "plugins.entries.comfy.config",
"overlayPath": "image",
"mode": {
"path": "mode",
"allowed": ["cloud"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId", "apiKey"]
},
{
"rootPath": "models.providers.comfy",
"overlayPath": "image",
"mode": {
"path": "mode",
"allowed": ["cloud"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId", "apiKey"]
}
]
}
},
"musicGenerationProviderMetadata": {
"comfy": {
"configSignals": [
{
"rootPath": "plugins.entries.comfy.config",
"overlayPath": "music",
"mode": {
"path": "mode",
"default": "local",
"allowed": ["local"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId"]
},
{
"rootPath": "models.providers.comfy",
"overlayPath": "music",
"mode": {
"path": "mode",
"default": "local",
"allowed": ["local"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId"]
},
{
"rootPath": "plugins.entries.comfy.config",
"overlayPath": "music",
"mode": {
"path": "mode",
"allowed": ["cloud"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId", "apiKey"]
},
{
"rootPath": "models.providers.comfy",
"overlayPath": "music",
"mode": {
"path": "mode",
"allowed": ["cloud"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId", "apiKey"]
}
]
}
},
"videoGenerationProviderMetadata": {
"comfy": {
"configSignals": [
{
"rootPath": "plugins.entries.comfy.config",
"overlayPath": "video",
"mode": {
"path": "mode",
"default": "local",
"allowed": ["local"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId"]
},
{
"rootPath": "models.providers.comfy",
"overlayPath": "video",
"mode": {
"path": "mode",
"default": "local",
"allowed": ["local"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId"]
},
{
"rootPath": "plugins.entries.comfy.config",
"overlayPath": "video",
"mode": {
"path": "mode",
"allowed": ["cloud"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId", "apiKey"]
},
{
"rootPath": "models.providers.comfy",
"overlayPath": "video",
"mode": {
"path": "mode",
"allowed": ["cloud"]
},
"requiredAny": ["workflow", "workflowPath"],
"required": ["promptNodeId", "apiKey"]
}
]
}
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {
"mode": {
"type": "string",
"enum": ["local", "cloud"]
},
"baseUrl": {
"type": "string"
},
"apiKey": {
"type": ["string", "object"]
},
"allowPrivateNetwork": {
"type": "boolean"
},
"workflowPath": {
"type": "string"
},
"workflow": {
"type": "object"
},
"promptNodeId": {
"type": "string"
},
"promptInputName": {
"type": "string"
},
"inputImageNodeId": {
"type": "string"
},
"inputImageInputName": {
"type": "string"
},
"outputNodeId": {
"type": "string"
},
"pollIntervalMs": {
"type": "integer",
"minimum": 100
},
"timeoutMs": {
"type": "integer",
"minimum": 1000
},
"image": {
"type": "object",
"additionalProperties": false,
"properties": {
"workflowPath": { "type": "string" },
"workflow": { "type": "object" },
"promptNodeId": { "type": "string" },
"promptInputName": { "type": "string" },
"inputImageNodeId": { "type": "string" },
"inputImageInputName": { "type": "string" },
"outputNodeId": { "type": "string" },
"pollIntervalMs": { "type": "integer", "minimum": 100 },
"timeoutMs": { "type": "integer", "minimum": 1000 }
}
},
"video": {
"type": "object",
"additionalProperties": false,
"properties": {
"workflowPath": { "type": "string" },
"workflow": { "type": "object" },
"promptNodeId": { "type": "string" },
"promptInputName": { "type": "string" },
"inputImageNodeId": { "type": "string" },
"inputImageInputName": { "type": "string" },
"outputNodeId": { "type": "string" },
"pollIntervalMs": { "type": "integer", "minimum": 100 },
"timeoutMs": { "type": "integer", "minimum": 1000 }
}
},
"music": {
"type": "object",
"additionalProperties": false,
"properties": {
"workflowPath": { "type": "string" },
"workflow": { "type": "object" },
"promptNodeId": { "type": "string" },
"promptInputName": { "type": "string" },
"outputNodeId": { "type": "string" },
"pollIntervalMs": { "type": "integer", "minimum": 100 },
"timeoutMs": { "type": "integer", "minimum": 1000 }
}
}
}
}
}

View File

@@ -0,0 +1,15 @@
{
"name": "@openclaw/comfy-provider",
"version": "2026.6.11",
"private": true,
"description": "OpenClaw ComfyUI provider plugin",
"type": "module",
"devDependencies": {
"@openclaw/plugin-sdk": "workspace:*"
},
"openclaw": {
"extensions": [
"./index.ts"
]
}
}

View File

@@ -0,0 +1,114 @@
// Comfy helper module supports test helpers behavior.
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-contracts";
import * as providerAuth from "openclaw/plugin-sdk/provider-auth-runtime";
import { expect, vi } from "vitest";
type FetchGuardMock = ReturnType<typeof vi.fn>;
type FetchGuardRequest = {
init?: {
body?: unknown;
};
};
type ComfyCloudJobResponseOptions = {
body: BodyInit;
contentType: string;
filename: string;
outputKind: "gifs" | "images";
promptId: string;
redirectLocation: string;
};
export function buildComfyConfig(config: Record<string, unknown>): OpenClawConfig {
return {
plugins: {
entries: {
comfy: { config },
},
},
} as unknown as OpenClawConfig;
}
export function buildLegacyComfyConfig(config: Record<string, unknown>): OpenClawConfig {
return {
models: {
providers: {
comfy: config,
},
},
} as unknown as OpenClawConfig;
}
export function parseComfyJsonBody(
fetchWithSsrFGuardMock: FetchGuardMock,
call: number,
): Record<string, unknown> {
const request = fetchWithSsrFGuardMock.mock.calls[call - 1]?.[0] as FetchGuardRequest | undefined;
const body = request?.init?.body;
expect(body).toBeTruthy();
if (typeof body !== "string") {
throw new Error(`Missing Comfy request body for fetch call ${call}`);
}
return JSON.parse(body) as Record<string, unknown>;
}
export function mockComfyProviderApiKey(apiKey = "comfy-test-key") {
return vi.spyOn(providerAuth, "resolveApiKeyForProvider").mockResolvedValue({
apiKey,
source: "env",
mode: "api-key",
});
}
export function mockComfyCloudJobResponses(
fetchWithSsrFGuardMock: FetchGuardMock,
options: ComfyCloudJobResponseOptions,
) {
fetchWithSsrFGuardMock
.mockResolvedValueOnce(fetchGuardJson({ prompt_id: options.promptId }))
.mockResolvedValueOnce(fetchGuardJson({ status: "completed" }))
.mockResolvedValueOnce(
fetchGuardJson({
[options.promptId]: {
outputs: {
"9": {
[options.outputKind]: [{ filename: options.filename, subfolder: "", type: "output" }],
},
},
},
}),
)
.mockResolvedValueOnce(
fetchGuardResponse(
new Response(null, {
status: 302,
headers: { location: options.redirectLocation },
}),
),
)
.mockResolvedValueOnce(
fetchGuardResponse(
new Response(options.body, {
status: 200,
headers: { "content-type": options.contentType },
}),
),
);
}
function fetchGuardJson(body: unknown) {
return fetchGuardResponse(
new Response(JSON.stringify(body), {
status: 200,
headers: { "content-type": "application/json" },
}),
);
}
function fetchGuardResponse(response: Response) {
return {
response,
release: vi.fn(async () => {}),
};
}

View File

@@ -0,0 +1,16 @@
{
"extends": "../tsconfig.package-boundary.base.json",
"compilerOptions": {
"rootDir": "."
},
"include": ["./*.ts", "./src/**/*.ts"],
"exclude": [
"./**/*.test.ts",
"./dist/**",
"./node_modules/**",
"./src/test-support/**",
"./src/**/*test-helpers.ts",
"./src/**/*test-harness.ts",
"./src/**/*test-support.ts"
]
}

View File

@@ -0,0 +1,244 @@
// Comfy tests cover video generation provider plugin behavior.
import { expectExplicitVideoGenerationCapabilities } from "openclaw/plugin-sdk/provider-test-contracts";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
buildComfyConfig,
mockComfyCloudJobResponses,
mockComfyProviderApiKey,
parseComfyJsonBody,
} from "./test-helpers.js";
import {
setComfyFetchGuardForTesting,
buildComfyVideoGenerationProvider,
} from "./video-generation-provider.js";
const { fetchWithSsrFGuardMock } = vi.hoisted(() => ({
fetchWithSsrFGuardMock: vi.fn(),
}));
function parseJsonBody(call: number): Record<string, unknown> {
return parseComfyJsonBody(fetchWithSsrFGuardMock, call);
}
function fetchGuardParams(call: number): { url?: unknown; auditContext?: unknown } {
const params = fetchWithSsrFGuardMock.mock.calls[call]?.[0];
if (!params || typeof params !== "object") {
throw new Error(`expected Comfy fetch guard call ${call}`);
}
return params as { url?: unknown; auditContext?: unknown };
}
describe("comfy video-generation provider", () => {
beforeEach(() => {
vi.clearAllMocks();
});
afterEach(() => {
setComfyFetchGuardForTesting(null);
vi.restoreAllMocks();
});
it("declares explicit mode capabilities", () => {
expectExplicitVideoGenerationCapabilities(buildComfyVideoGenerationProvider());
});
it("treats local comfy video workflows as configured without an API key", () => {
const provider = buildComfyVideoGenerationProvider();
expect(
provider.isConfigured?.({
cfg: buildComfyConfig({
video: {
workflow: {
"6": { inputs: { text: "" } },
},
promptNodeId: "6",
},
}),
}),
).toBe(true);
});
it("submits a local workflow, waits for history, and downloads videos", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "local-video-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"local-video-1": {
outputs: {
"9": {
gifs: [{ filename: "generated.mp4", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("mp4-data"), {
status: 200,
headers: { "content-type": "video/mp4" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyVideoGenerationProvider();
const result = await provider.generateVideo({
provider: "comfy",
model: "workflow",
prompt: "animate a lobster",
cfg: buildComfyConfig({
video: {
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
},
}),
});
expect(fetchGuardParams(0).url).toBe("http://127.0.0.1:8188/prompt");
expect(fetchGuardParams(0).auditContext).toBe("comfy-video-generate");
expect(parseJsonBody(1)).toEqual({
prompt: {
"6": { inputs: { text: "animate a lobster" } },
"9": { inputs: {} },
},
});
expect(fetchGuardParams(1).url).toBe("http://127.0.0.1:8188/history/local-video-1");
expect(fetchGuardParams(1).auditContext).toBe("comfy-history");
expect(fetchGuardParams(2).url).toBe(
"http://127.0.0.1:8188/view?filename=generated.mp4&subfolder=&type=output",
);
expect(fetchGuardParams(2).auditContext).toBe("comfy-video-download");
expect(result).toEqual({
videos: [
{
buffer: Buffer.from("mp4-data"),
mimeType: "video/mp4",
fileName: "generated.mp4",
metadata: {
nodeId: "9",
promptId: "local-video-1",
},
},
],
model: "workflow",
metadata: {
promptId: "local-video-1",
outputNodeIds: ["9"],
},
});
});
it("rejects generated video downloads that exceed the configured media cap", async () => {
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
fetchWithSsrFGuardMock
.mockResolvedValueOnce({
response: new Response(JSON.stringify({ prompt_id: "local-video-1" }), {
status: 200,
headers: { "content-type": "application/json" },
}),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(
JSON.stringify({
"local-video-1": {
outputs: {
"9": {
gifs: [{ filename: "generated.mp4", subfolder: "", type: "output" }],
},
},
},
}),
{
status: 200,
headers: { "content-type": "application/json" },
},
),
release: vi.fn(async () => {}),
})
.mockResolvedValueOnce({
response: new Response(Buffer.from("too-large"), {
status: 200,
headers: { "content-type": "video/mp4" },
}),
release: vi.fn(async () => {}),
});
const provider = buildComfyVideoGenerationProvider();
await expect(
provider.generateVideo({
provider: "comfy",
model: "workflow",
prompt: "animate a lobster",
cfg: {
...buildComfyConfig({
video: {
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
},
}),
agents: { defaults: { mediaMaxMb: 0.000001 } },
} as never,
}),
).rejects.toThrow("Comfy video output download exceeds 1 bytes");
});
it("uses cloud endpoints for video workflows", async () => {
mockComfyProviderApiKey();
setComfyFetchGuardForTesting(fetchWithSsrFGuardMock);
mockComfyCloudJobResponses(fetchWithSsrFGuardMock, {
body: Buffer.from("cloud-video-data"),
contentType: "video/mp4",
filename: "cloud.mp4",
outputKind: "gifs",
promptId: "cloud-video-1",
redirectLocation: "https://cdn.example.com/cloud.mp4",
});
const provider = buildComfyVideoGenerationProvider();
const result = await provider.generateVideo({
provider: "comfy",
model: "workflow",
prompt: "cloud video workflow",
cfg: buildComfyConfig({
mode: "cloud",
video: {
workflow: {
"6": { inputs: { text: "" } },
"9": { inputs: {} },
},
promptNodeId: "6",
outputNodeId: "9",
},
}),
});
expect(fetchGuardParams(0).url).toBe("https://cloud.comfy.org/api/prompt");
expect(fetchGuardParams(0).auditContext).toBe("comfy-video-generate");
expect(result.metadata).toEqual({
promptId: "cloud-video-1",
outputNodeIds: ["9"],
});
});
});

View File

@@ -0,0 +1,105 @@
// Comfy provider module implements model/runtime integration.
import type {
GeneratedVideoAsset,
VideoGenerationProvider,
VideoGenerationSourceAsset,
} from "openclaw/plugin-sdk/video-generation";
import {
DEFAULT_COMFY_MODEL,
setComfyFetchGuardForTesting,
isComfyCapabilityConfigured,
runComfyWorkflow,
} from "./workflow-runtime.js";
export { setComfyFetchGuardForTesting };
function toComfyInputImage(inputImage?: VideoGenerationSourceAsset) {
if (!inputImage) {
return undefined;
}
if (!inputImage.buffer || !inputImage.mimeType) {
throw new Error("Comfy video generation requires a local reference image file");
}
return {
buffer: inputImage.buffer,
mimeType: inputImage.mimeType,
fileName: inputImage.fileName,
};
}
export function buildComfyVideoGenerationProvider(): VideoGenerationProvider {
return {
id: "comfy",
label: "ComfyUI",
defaultModel: DEFAULT_COMFY_MODEL,
models: [DEFAULT_COMFY_MODEL],
isConfigured: ({ cfg, agentDir }) =>
isComfyCapabilityConfigured({
cfg,
agentDir,
capability: "video",
}),
capabilities: {
generate: {
maxVideos: 1,
supportsSize: false,
supportsAspectRatio: false,
supportsResolution: false,
supportsAudio: false,
supportsWatermark: false,
},
imageToVideo: {
enabled: true,
maxVideos: 1,
maxInputImages: 1,
supportsSize: false,
supportsAspectRatio: false,
supportsResolution: false,
supportsAudio: false,
supportsWatermark: false,
},
videoToVideo: {
enabled: false,
},
},
async generateVideo(req) {
if ((req.inputImages?.length ?? 0) > 1) {
throw new Error("Comfy video generation currently supports at most one reference image");
}
if ((req.inputVideos?.length ?? 0) > 0) {
throw new Error("Comfy video generation does not support input videos");
}
const result = await runComfyWorkflow({
cfg: req.cfg,
agentDir: req.agentDir,
authStore: req.authStore,
prompt: req.prompt,
model: req.model,
timeoutMs: req.timeoutMs,
capability: "video",
outputKinds: ["gifs", "videos"],
inputImage: toComfyInputImage(req.inputImages?.[0]),
});
const videos: GeneratedVideoAsset[] = result.assets.map((asset) => ({
buffer: asset.buffer,
mimeType: asset.mimeType,
fileName: asset.fileName,
metadata: {
nodeId: asset.nodeId,
promptId: result.promptId,
},
}));
return {
videos,
model: result.model,
metadata: {
promptId: result.promptId,
outputNodeIds: result.outputNodeIds,
},
};
},
};
}

View File

@@ -0,0 +1,171 @@
// Comfy tests cover workflow-runtime bounded-read delegation.
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { readJsonResponseForTest, setComfyFetchGuardForTesting } from "./workflow-runtime.js";
describe("readJsonResponse bounded read (readProviderJsonResponse delegation)", () => {
const fetchMock = vi.fn();
beforeEach(() => {
vi.clearAllMocks();
});
afterEach(() => {
setComfyFetchGuardForTesting(null);
vi.restoreAllMocks();
});
it("cancels oversized JSON body via the 16 MiB provider cap", async () => {
const ONE_MIB = 1024 * 1024;
const TOTAL_CHUNKS = 32;
const chunk = new Uint8Array(ONE_MIB);
let bytesPulled = 0;
let canceled = false;
const oversizedJson = new Response(
new ReadableStream<Uint8Array>({
pull(controller) {
if (bytesPulled >= TOTAL_CHUNKS * ONE_MIB) {
controller.close();
return;
}
bytesPulled += chunk.length;
controller.enqueue(chunk);
},
cancel() {
canceled = true;
},
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
const release = vi.fn(async () => {});
fetchMock.mockResolvedValueOnce({ response: oversizedJson, release });
setComfyFetchGuardForTesting(fetchMock);
await expect(
readJsonResponseForTest({
url: "http://127.0.0.1:9999/test",
init: { method: "GET" },
timeoutMs: 10_000,
auditContext: "comfy-test",
errorPrefix: "Comfy test failed",
}),
).rejects.toThrow(/JSON response exceeds 16777216 bytes/);
expect(canceled).toBe(true);
expect(bytesPulled).toBeLessThan(TOTAL_CHUNKS * ONE_MIB);
expect(release).toHaveBeenCalledOnce();
});
it("rejects oversized body with correct error prefix", async () => {
const ONE_MIB = 1024 * 1024;
const chunk = new Uint8Array(ONE_MIB);
let bytesPulled = 0;
let canceled = false;
const oversizedJson = new Response(
new ReadableStream<Uint8Array>({
pull(controller) {
if (bytesPulled >= 32 * ONE_MIB) {
controller.close();
return;
}
bytesPulled += chunk.length;
controller.enqueue(chunk);
},
cancel() {
canceled = true;
},
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
const release = vi.fn(async () => {});
fetchMock.mockResolvedValueOnce({ response: oversizedJson, release });
setComfyFetchGuardForTesting(fetchMock);
await expect(
readJsonResponseForTest({
url: "http://127.0.0.1:9999/test",
init: { method: "GET" },
timeoutMs: 10_000,
auditContext: "comfy-test",
errorPrefix: "Comfy test failed",
}),
).rejects.toThrow(/^Comfy test failed: JSON response exceeds 16777216 bytes/);
expect(canceled).toBe(true);
expect(bytesPulled).toBeLessThan(32 * ONE_MIB);
});
it("parses small valid JSON body (negative control)", async () => {
const smallBody = { status: "ok" };
const release = vi.fn(async () => {});
fetchMock.mockResolvedValueOnce({
response: new Response(JSON.stringify(smallBody), {
status: 200,
headers: { "Content-Type": "application/json" },
}),
release,
});
setComfyFetchGuardForTesting(fetchMock);
const result = await readJsonResponseForTest<{ status: string }>({
url: "http://127.0.0.1:9999/test",
init: { method: "GET" },
timeoutMs: 10_000,
auditContext: "comfy-test",
errorPrefix: "Comfy test failed",
});
expect(result.status).toBe("ok");
expect(release).toHaveBeenCalledOnce();
});
it("parses valid JSON with expected comfy response shape (happy path)", async () => {
const comfyResponse = { prompt_id: "abc-123" };
const release = vi.fn(async () => {});
fetchMock.mockResolvedValueOnce({
response: new Response(JSON.stringify(comfyResponse), {
status: 200,
headers: { "Content-Type": "application/json" },
}),
release,
});
setComfyFetchGuardForTesting(fetchMock);
const result = await readJsonResponseForTest<{ prompt_id: string }>({
url: "http://127.0.0.1:9999/test",
init: { method: "GET" },
timeoutMs: 10_000,
auditContext: "comfy-test",
errorPrefix: "Comfy test failed",
});
expect(result.prompt_id).toBe("abc-123");
expect(release).toHaveBeenCalledOnce();
});
it("propagates HTTP error status before reading body", async () => {
const release = vi.fn(async () => {});
fetchMock.mockResolvedValueOnce({
response: new Response(null, { status: 500, statusText: "Internal Server Error" }),
release,
});
setComfyFetchGuardForTesting(fetchMock);
await expect(
readJsonResponseForTest({
url: "http://127.0.0.1:9999/test",
init: { method: "GET" },
timeoutMs: 10_000,
auditContext: "comfy-test",
errorPrefix: "Comfy test failed",
}),
).rejects.toThrow(/Comfy test failed/);
expect(release).toHaveBeenCalledOnce();
});
});

View File

@@ -0,0 +1,887 @@
// Comfy plugin module implements workflow runtime behavior.
import fs from "node:fs/promises";
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-contracts";
import { canResolveEnvSecretRefInReadOnlyPath } from "openclaw/plugin-sdk/extension-shared";
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
import { resolvePositiveTimerTimeoutMs } from "openclaw/plugin-sdk/number-runtime";
import {
isProviderApiKeyConfigured,
type AuthProfileStore,
} from "openclaw/plugin-sdk/provider-auth";
import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runtime";
import {
assertOkOrThrowHttpError,
normalizeBaseUrl,
readProviderJsonResponse,
resolveProviderHttpRequestConfig,
} from "openclaw/plugin-sdk/provider-http";
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
import {
normalizeSecretInputString,
resolveSecretInputString,
} from "openclaw/plugin-sdk/secret-input-runtime";
import {
buildHostnameAllowlistPolicyFromSuffixAllowlist,
fetchWithSsrFGuard,
isPrivateOrLoopbackHost,
mergeSsrFPolicies,
ssrfPolicyFromDangerouslyAllowPrivateNetwork,
type SsrFPolicy,
} from "openclaw/plugin-sdk/ssrf-runtime";
import {
asBoolean,
isRecord,
normalizeOptionalLowercaseString,
normalizeOptionalString,
uniqueStrings,
} from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolveUserPath } from "openclaw/plugin-sdk/text-utility-runtime";
const DEFAULT_COMFY_LOCAL_BASE_URL = "http://127.0.0.1:8188";
const DEFAULT_COMFY_CLOUD_BASE_URL = "https://cloud.comfy.org";
const DEFAULT_PROMPT_INPUT_NAME = "text";
const DEFAULT_INPUT_IMAGE_INPUT_NAME = "image";
const DEFAULT_POLL_INTERVAL_MS = 1_500;
const DEFAULT_TIMEOUT_MS = 5 * 60_000;
const DEFAULT_GENERATED_IMAGE_MAX_BYTES = 6 * 1024 * 1024;
const DEFAULT_GENERATED_MEDIA_MAX_BYTES = 16 * 1024 * 1024;
export const DEFAULT_COMFY_MODEL = "workflow";
type ComfyMode = "local" | "cloud";
type ComfyCapability = "image" | "music" | "video";
type ComfyOutputKind = "audio" | "gifs" | "images" | "videos";
type ComfyWorkflow = Record<string, unknown>;
type ComfyProviderConfig = Record<string, unknown>;
type ComfyFetchGuardParams = Parameters<typeof fetchWithSsrFGuard>[0];
type ComfyDispatcherPolicy = ComfyFetchGuardParams["dispatcherPolicy"];
type ComfyPromptResponse = {
prompt_id?: string;
};
type ComfyOutputFile = {
filename?: string;
name?: string;
subfolder?: string;
type?: string;
};
type ComfyHistoryOutputEntry = Partial<Record<ComfyOutputKind, ComfyOutputFile[]>>;
type ComfyHistoryEntry = {
outputs?: Record<string, ComfyHistoryOutputEntry>;
};
type ComfyUploadResponse = {
name?: string;
filename?: string;
};
type ComfyStatusResponse = {
status?: string;
message?: string;
error?: string;
};
type ComfyNetworkPolicy = {
apiPolicy?: SsrFPolicy;
};
type ComfyApiKeyResolution =
| {
status: "available";
apiKey: string;
source: string;
}
| {
status: "missing";
}
| {
status: "configured_unavailable";
};
type ComfySourceImage = {
buffer: Buffer;
mimeType: string;
fileName?: string;
};
type ComfyGeneratedAsset = {
buffer: Buffer;
mimeType: string;
fileName: string;
nodeId: string;
};
type ComfyWorkflowResult = {
assets: ComfyGeneratedAsset[];
model: string;
promptId: string;
outputNodeIds: string[];
};
let comfyFetchGuard = fetchWithSsrFGuard;
export function setComfyFetchGuardForTesting(impl: typeof fetchWithSsrFGuard | null): void {
comfyFetchGuard = impl ?? fetchWithSsrFGuard;
}
function resolveComfyGeneratedOutputMaxBytes(params: {
cfg: OpenClawConfig;
capability: ComfyCapability;
}): number {
const configured = params.cfg.agents?.defaults?.mediaMaxMb;
if (typeof configured === "number" && Number.isFinite(configured) && configured > 0) {
return Math.floor(configured * 1024 * 1024);
}
return params.capability === "image"
? DEFAULT_GENERATED_IMAGE_MAX_BYTES
: DEFAULT_GENERATED_MEDIA_MAX_BYTES;
}
function readConfigBoolean(config: ComfyProviderConfig, key: string): boolean | undefined {
return asBoolean(config[key]);
}
function readConfigInteger(config: ComfyProviderConfig, key: string): number | undefined {
const value = config[key];
return typeof value === "number" && Number.isInteger(value) && value > 0 ? value : undefined;
}
export function getComfyConfig(cfg?: OpenClawConfig): ComfyProviderConfig {
const pluginConfig = cfg?.plugins?.entries?.comfy?.config;
if (isRecord(pluginConfig)) {
return pluginConfig;
}
const legacyConfig = cfg?.models?.providers?.comfy;
return isRecord(legacyConfig) ? legacyConfig : {};
}
function stripNestedCapabilityConfig(config: ComfyProviderConfig): ComfyProviderConfig {
const next = { ...config };
delete next.image;
delete next.video;
delete next.music;
return next;
}
function getComfyCapabilityConfig(
config: ComfyProviderConfig,
capability: ComfyCapability,
): ComfyProviderConfig {
const shared = stripNestedCapabilityConfig(config);
const nested = config[capability];
if (!isRecord(nested)) {
return shared;
}
return { ...shared, ...nested };
}
function resolveComfyMode(config: ComfyProviderConfig): ComfyMode {
return normalizeOptionalString(config.mode) === "cloud" ? "cloud" : "local";
}
function resolveComfyApiKey(
config: ComfyProviderConfig,
cfg?: OpenClawConfig,
): ComfyApiKeyResolution {
const resolved = resolveSecretInputString({
value: config.apiKey,
path: "plugins.entries.comfy.config.apiKey",
defaults: cfg?.secrets?.defaults,
mode: "inspect",
});
if (resolved.status === "available") {
const apiKey = normalizeSecretInputString(resolved.value);
return apiKey
? {
status: "available",
apiKey,
source: "plugins.entries.comfy.config.apiKey",
}
: { status: "missing" };
}
if (resolved.status === "configured_unavailable") {
if (resolved.ref.source !== "env") {
return { status: "configured_unavailable" };
}
const envVarName = resolved.ref.id.trim();
if (
!canResolveEnvSecretRefInReadOnlyPath({
cfg,
provider: resolved.ref.provider,
id: envVarName,
})
) {
return { status: "configured_unavailable" };
}
const apiKey = normalizeSecretInputString(process.env[envVarName]);
return apiKey
? {
status: "available",
apiKey,
source: `plugins.entries.comfy.config.apiKey (${envVarName})`,
}
: { status: "configured_unavailable" };
}
return { status: "missing" };
}
function getRequiredConfigString(config: ComfyProviderConfig, key: string): string {
const value = normalizeOptionalString(config[key]);
if (!value) {
throw new Error(`plugins.entries.comfy.config.${key} is required`);
}
return value;
}
function resolveComfyWorkflowSource(config: ComfyProviderConfig): {
workflow?: ComfyWorkflow;
workflowPath?: string;
} {
const workflow = config.workflow;
if (isRecord(workflow)) {
return { workflow: structuredClone(workflow) };
}
const workflowPath = normalizeOptionalString(config.workflowPath);
return { workflowPath };
}
async function loadComfyWorkflow(config: ComfyProviderConfig): Promise<ComfyWorkflow> {
const source = resolveComfyWorkflowSource(config);
if (source.workflow) {
return source.workflow;
}
if (!source.workflowPath) {
throw new Error(
"plugins.entries.comfy.config.<capability>.workflow or workflowPath is required",
);
}
const resolvedPath = resolveUserPath(source.workflowPath);
const raw = await fs.readFile(resolvedPath, "utf8");
const parsed = JSON.parse(raw) as unknown;
if (!isRecord(parsed)) {
throw new Error(`Comfy workflow at ${resolvedPath} must be a JSON object`);
}
return parsed;
}
function setWorkflowInput(params: {
workflow: ComfyWorkflow;
nodeId: string;
inputName: string;
value: unknown;
}): void {
const node = params.workflow[params.nodeId];
if (!isRecord(node)) {
throw new Error(`Comfy workflow missing node "${params.nodeId}"`);
}
const inputs = node.inputs;
if (!isRecord(inputs)) {
throw new Error(`Comfy workflow node "${params.nodeId}" is missing an inputs object`);
}
inputs[params.inputName] = params.value;
}
function resolveComfyNetworkPolicy(params: {
baseUrl: string;
allowPrivateNetwork: boolean;
}): ComfyNetworkPolicy {
let parsed: URL;
try {
parsed = new URL(params.baseUrl);
} catch {
return {};
}
const hostname = normalizeOptionalLowercaseString(parsed.hostname) ?? "";
if (!hostname || !params.allowPrivateNetwork || !isPrivateOrLoopbackHost(hostname)) {
return {};
}
const hostnamePolicy = buildHostnameAllowlistPolicyFromSuffixAllowlist([hostname]);
const privateNetworkPolicy = ssrfPolicyFromDangerouslyAllowPrivateNetwork(true);
return {
apiPolicy: mergeSsrFPolicies(hostnamePolicy, privateNetworkPolicy),
};
}
async function readJsonResponse<T>(params: {
url: string;
init?: RequestInit;
timeoutMs?: number;
policy?: SsrFPolicy;
dispatcherPolicy?: ComfyDispatcherPolicy;
auditContext: string;
errorPrefix: string;
}): Promise<T> {
const { response, release } = await comfyFetchGuard({
url: params.url,
init: params.init,
timeoutMs: params.timeoutMs,
policy: params.policy,
dispatcherPolicy: params.dispatcherPolicy,
auditContext: params.auditContext,
});
try {
await assertOkOrThrowHttpError(response, params.errorPrefix);
return (await readProviderJsonResponse(response, params.errorPrefix)) as T;
} finally {
await release();
}
}
/** @internal Test-only export. */
export const readJsonResponseForTest = readJsonResponse;
function resolveFileExtension(params: { fileName?: string; mimeType?: string }): string {
const extension = extensionForMime(params.mimeType);
if (extension) {
return extension.slice(1);
}
const fileName = params.fileName?.trim();
if (!fileName) {
return "bin";
}
const dotIndex = fileName.lastIndexOf(".");
if (dotIndex < 0 || dotIndex === fileName.length - 1) {
return "bin";
}
return fileName.slice(dotIndex + 1);
}
function toBlobBytes(buffer: Buffer): ArrayBuffer {
const arrayBuffer = new ArrayBuffer(buffer.byteLength);
new Uint8Array(arrayBuffer).set(buffer);
return arrayBuffer;
}
async function uploadInputImage(params: {
baseUrl: string;
headers: Headers;
timeoutMs: number;
policy?: SsrFPolicy;
dispatcherPolicy?: ComfyDispatcherPolicy;
image: ComfySourceImage;
mode: ComfyMode;
capability: ComfyCapability;
}): Promise<string> {
const form = new FormData();
form.set(
"image",
new Blob([toBlobBytes(params.image.buffer)], { type: params.image.mimeType }),
normalizeOptionalString(params.image.fileName) ||
`input.${resolveFileExtension({ mimeType: params.image.mimeType })}`,
);
form.set("type", "input");
form.set("overwrite", "true");
const headers = new Headers(params.headers);
headers.delete("Content-Type");
const payload = await readJsonResponse<ComfyUploadResponse>({
url: `${params.baseUrl}${params.mode === "cloud" ? "/api/upload/image" : "/upload/image"}`,
init: {
method: "POST",
headers,
body: form,
},
timeoutMs: params.timeoutMs,
policy: params.policy,
dispatcherPolicy: params.dispatcherPolicy,
auditContext: `comfy-${params.capability}-upload`,
errorPrefix: "Comfy image upload failed",
});
const uploadedName =
normalizeOptionalString(payload.filename) || normalizeOptionalString(payload.name);
if (!uploadedName) {
throw new Error("Comfy image upload response missing filename");
}
return uploadedName;
}
function extractHistoryEntry(history: unknown, promptId: string): ComfyHistoryEntry | null {
if (!isRecord(history)) {
return null;
}
const directOutputs = history.outputs;
if (isRecord(directOutputs)) {
return history as ComfyHistoryEntry;
}
const nested = history[promptId];
if (isRecord(nested)) {
return nested as ComfyHistoryEntry;
}
return null;
}
async function waitForLocalHistory(params: {
baseUrl: string;
promptId: string;
headers: Headers;
timeoutMs: number;
pollIntervalMs: number;
policy?: SsrFPolicy;
dispatcherPolicy?: ComfyDispatcherPolicy;
}): Promise<ComfyHistoryEntry> {
const deadline = Date.now() + params.timeoutMs;
for (;;) {
const requestTimeoutMs = resolveComfyRemainingMs(deadline, params.timeoutMs);
const history = await readJsonResponse<unknown>({
url: `${params.baseUrl}/history/${params.promptId}`,
init: {
method: "GET",
headers: params.headers,
},
timeoutMs: requestTimeoutMs,
policy: params.policy,
dispatcherPolicy: params.dispatcherPolicy,
auditContext: "comfy-history",
errorPrefix: "Comfy history lookup failed",
});
const entry = extractHistoryEntry(history, params.promptId);
if (entry?.outputs && Object.keys(entry.outputs).length > 0) {
return entry;
}
const pollDelayMs = resolveComfyRemainingMs(deadline, params.timeoutMs, params.pollIntervalMs);
await new Promise((resolve) => {
setTimeout(resolve, pollDelayMs);
});
}
}
async function waitForCloudCompletion(params: {
baseUrl: string;
promptId: string;
headers: Headers;
timeoutMs: number;
pollIntervalMs: number;
policy?: SsrFPolicy;
dispatcherPolicy?: ComfyDispatcherPolicy;
}): Promise<void> {
const deadline = Date.now() + params.timeoutMs;
for (;;) {
const requestTimeoutMs = resolveComfyRemainingMs(deadline, params.timeoutMs);
const status = await readJsonResponse<ComfyStatusResponse>({
url: `${params.baseUrl}/api/job/${params.promptId}/status`,
init: {
method: "GET",
headers: params.headers,
},
timeoutMs: requestTimeoutMs,
policy: params.policy,
dispatcherPolicy: params.dispatcherPolicy,
auditContext: "comfy-status",
errorPrefix: "Comfy status lookup failed",
});
if (status.status === "completed") {
return;
}
if (status.status === "failed" || status.status === "cancelled") {
throw new Error(
`Comfy workflow ${status.status}: ${status.error ?? status.message ?? params.promptId}`,
);
}
const pollDelayMs = resolveComfyRemainingMs(deadline, params.timeoutMs, params.pollIntervalMs);
await new Promise((resolve) => {
setTimeout(resolve, pollDelayMs);
});
}
}
function resolveComfyRemainingMs(
deadline: number,
timeoutMs: number,
defaultTimeoutMs = timeoutMs,
) {
const defaultMs = resolvePositiveTimerTimeoutMs(defaultTimeoutMs, 1);
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) {
throw new Error(`Comfy workflow did not finish within ${Math.ceil(timeoutMs / 1000)}s`);
}
return Math.max(1, Math.min(defaultMs, remainingMs));
}
function collectOutputFiles(params: {
history: ComfyHistoryEntry;
outputNodeId?: string;
outputKinds: readonly ComfyOutputKind[];
}): Array<{ nodeId: string; file: ComfyOutputFile }> {
const outputs = params.history.outputs;
if (!outputs) {
return [];
}
const nodeIds = params.outputNodeId ? [params.outputNodeId] : Object.keys(outputs);
const files: Array<{ nodeId: string; file: ComfyOutputFile }> = [];
for (const nodeId of nodeIds) {
const entry = outputs[nodeId];
if (!entry) {
continue;
}
for (const kind of params.outputKinds) {
const bucket = entry[kind];
if (!Array.isArray(bucket)) {
continue;
}
for (const file of bucket) {
files.push({ nodeId, file });
}
}
}
return files;
}
async function downloadOutputFile(params: {
baseUrl: string;
headers: Headers;
timeoutMs: number;
policy?: SsrFPolicy;
dispatcherPolicy?: ComfyDispatcherPolicy;
file: ComfyOutputFile;
mode: ComfyMode;
capability: ComfyCapability;
maxBytes: number;
}): Promise<{ buffer: Buffer; mimeType: string }> {
const fileName =
normalizeOptionalString(params.file.filename) || normalizeOptionalString(params.file.name);
if (!fileName) {
throw new Error("Comfy output entry missing filename");
}
const query = new URLSearchParams({
filename: fileName,
subfolder: normalizeOptionalString(params.file.subfolder) ?? "",
type: normalizeOptionalString(params.file.type) ?? "output",
});
const viewPath = params.mode === "cloud" ? "/api/view" : "/view";
const auditContext = `comfy-${params.capability}-download`;
const firstResponse = await comfyFetchGuard({
url: `${params.baseUrl}${viewPath}?${query.toString()}`,
init: {
method: "GET",
headers: params.headers,
...(params.mode === "cloud" ? { redirect: "manual" } : {}),
},
timeoutMs: params.timeoutMs,
policy: params.policy,
dispatcherPolicy: params.dispatcherPolicy,
auditContext,
});
try {
if (
params.mode === "cloud" &&
[301, 302, 303, 307, 308].includes(firstResponse.response.status)
) {
const redirectUrl = normalizeOptionalString(firstResponse.response.headers.get("location"));
if (!redirectUrl) {
throw new Error("Comfy cloud output redirect missing location header");
}
const redirected = await comfyFetchGuard({
url: redirectUrl,
init: {
method: "GET",
},
timeoutMs: params.timeoutMs,
dispatcherPolicy: params.dispatcherPolicy,
auditContext,
});
try {
await assertOkOrThrowHttpError(redirected.response, "Comfy output download failed");
const mimeType =
normalizeOptionalString(redirected.response.headers.get("content-type")) ||
"application/octet-stream";
return {
buffer: await readResponseWithLimit(redirected.response, params.maxBytes, {
chunkTimeoutMs: params.timeoutMs,
onOverflow: ({ maxBytes }) =>
new Error(`Comfy ${params.capability} output download exceeds ${maxBytes} bytes`),
onIdleTimeout: ({ chunkTimeoutMs }) =>
new Error(
`Comfy ${params.capability} output download stalled after ${chunkTimeoutMs}ms`,
),
}),
mimeType,
};
} finally {
await redirected.release();
}
}
await assertOkOrThrowHttpError(firstResponse.response, "Comfy output download failed");
const mimeType =
normalizeOptionalString(firstResponse.response.headers.get("content-type")) ||
"application/octet-stream";
return {
buffer: await readResponseWithLimit(firstResponse.response, params.maxBytes, {
chunkTimeoutMs: params.timeoutMs,
onOverflow: ({ maxBytes }) =>
new Error(`Comfy ${params.capability} output download exceeds ${maxBytes} bytes`),
onIdleTimeout: ({ chunkTimeoutMs }) =>
new Error(`Comfy ${params.capability} output download stalled after ${chunkTimeoutMs}ms`),
}),
mimeType,
};
} finally {
await firstResponse.release();
}
}
export function isComfyCapabilityConfigured(params: {
cfg?: OpenClawConfig;
agentDir?: string;
capability: ComfyCapability;
}): boolean {
const config = getComfyConfig(params.cfg);
const capabilityConfig = getComfyCapabilityConfig(config, params.capability);
const hasWorkflow = Boolean(
resolveComfyWorkflowSource(capabilityConfig).workflow ||
normalizeOptionalString(capabilityConfig.workflowPath),
);
const hasPromptNode = Boolean(normalizeOptionalString(capabilityConfig.promptNodeId));
if (!hasWorkflow || !hasPromptNode) {
return false;
}
if (resolveComfyMode(capabilityConfig) === "local") {
return true;
}
const configuredApiKey = resolveComfyApiKey(capabilityConfig, params.cfg);
if (configuredApiKey.status === "available") {
return true;
}
if (configuredApiKey.status === "configured_unavailable") {
return false;
}
return isProviderApiKeyConfigured({
provider: "comfy",
agentDir: params.agentDir,
});
}
export async function runComfyWorkflow(params: {
cfg: OpenClawConfig;
agentDir?: string;
authStore?: AuthProfileStore;
prompt: string;
model?: string;
timeoutMs?: number;
capability: ComfyCapability;
outputKinds: readonly ComfyOutputKind[];
inputImage?: ComfySourceImage;
}): Promise<ComfyWorkflowResult> {
const config = getComfyConfig(params.cfg);
const capabilityConfig = getComfyCapabilityConfig(config, params.capability);
const mode = resolveComfyMode(capabilityConfig);
const workflow = await loadComfyWorkflow(capabilityConfig);
const promptNodeId = getRequiredConfigString(capabilityConfig, "promptNodeId");
const promptInputName =
normalizeOptionalString(capabilityConfig.promptInputName) ?? DEFAULT_PROMPT_INPUT_NAME;
const inputImageNodeId = normalizeOptionalString(capabilityConfig.inputImageNodeId);
const inputImageInputName =
normalizeOptionalString(capabilityConfig.inputImageInputName) ?? DEFAULT_INPUT_IMAGE_INPUT_NAME;
const outputNodeId = normalizeOptionalString(capabilityConfig.outputNodeId);
const pollIntervalMs = resolvePositiveTimerTimeoutMs(
readConfigInteger(capabilityConfig, "pollIntervalMs"),
DEFAULT_POLL_INTERVAL_MS,
);
const timeoutMs = resolvePositiveTimerTimeoutMs(
readConfigInteger(capabilityConfig, "timeoutMs") ?? params.timeoutMs,
DEFAULT_TIMEOUT_MS,
);
const providerModel = normalizeOptionalString(params.model) || DEFAULT_COMFY_MODEL;
setWorkflowInput({
workflow,
nodeId: promptNodeId,
inputName: promptInputName,
value: params.prompt,
});
const pluginApiKey = resolveComfyApiKey(capabilityConfig, params.cfg);
const resolvedAuth =
mode === "cloud"
? pluginApiKey.status === "available"
? {
apiKey: pluginApiKey.apiKey,
source: pluginApiKey.source,
mode: "api-key" as const,
}
: pluginApiKey.status === "configured_unavailable"
? null
: await resolveApiKeyForProvider({
provider: "comfy",
cfg: params.cfg,
agentDir: params.agentDir,
store: params.authStore,
})
: null;
if (mode === "cloud" && !resolvedAuth?.apiKey) {
throw new Error("Comfy Cloud API key missing");
}
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
resolveProviderHttpRequestConfig({
baseUrl: normalizeOptionalString(capabilityConfig.baseUrl),
defaultBaseUrl:
mode === "cloud" ? DEFAULT_COMFY_CLOUD_BASE_URL : DEFAULT_COMFY_LOCAL_BASE_URL,
allowPrivateNetwork:
mode === "local" || readConfigBoolean(capabilityConfig, "allowPrivateNetwork") === true,
defaultHeaders:
mode === "cloud"
? {
"X-API-Key": resolvedAuth?.apiKey ?? "",
"Content-Type": "application/json",
}
: {
"Content-Type": "application/json",
},
provider: "comfy",
capability: params.capability === "music" ? "audio" : params.capability,
transport: "http",
});
const normalizedBaseUrl =
normalizeBaseUrl(baseUrl) ||
(mode === "cloud" ? DEFAULT_COMFY_CLOUD_BASE_URL : DEFAULT_COMFY_LOCAL_BASE_URL);
const networkPolicy = resolveComfyNetworkPolicy({
baseUrl: normalizedBaseUrl,
allowPrivateNetwork,
});
if (params.inputImage) {
if (!inputImageNodeId) {
throw new Error(
"Comfy edit requests require plugins.entries.comfy.config.<capability>.inputImageNodeId to be configured",
);
}
const uploadedName = await uploadInputImage({
baseUrl: normalizedBaseUrl,
headers: new Headers(headers),
timeoutMs,
policy: networkPolicy.apiPolicy,
dispatcherPolicy,
image: params.inputImage,
mode,
capability: params.capability,
});
setWorkflowInput({
workflow,
nodeId: inputImageNodeId,
inputName: inputImageInputName,
value: uploadedName,
});
}
const submitPayload = {
prompt: workflow,
...(mode === "cloud" && resolvedAuth?.apiKey
? { extra_data: { api_key_comfy_org: resolvedAuth.apiKey } }
: {}),
};
const promptResponse = await readJsonResponse<ComfyPromptResponse>({
url: `${normalizedBaseUrl}${mode === "cloud" ? "/api/prompt" : "/prompt"}`,
init: {
method: "POST",
headers,
body: JSON.stringify(submitPayload),
},
timeoutMs,
policy: networkPolicy.apiPolicy,
dispatcherPolicy,
auditContext: `comfy-${params.capability}-generate`,
errorPrefix: "Comfy workflow submit failed",
});
const promptId = normalizeOptionalString(promptResponse.prompt_id);
if (!promptId) {
throw new Error("Comfy workflow submit response missing prompt_id");
}
const history =
mode === "cloud"
? await (async () => {
await waitForCloudCompletion({
baseUrl: normalizedBaseUrl,
promptId,
headers: new Headers(headers),
timeoutMs,
pollIntervalMs,
policy: networkPolicy.apiPolicy,
dispatcherPolicy,
});
return await readJsonResponse<unknown>({
url: `${normalizedBaseUrl}/api/history_v2/${promptId}`,
init: {
method: "GET",
headers: new Headers(headers),
},
timeoutMs,
policy: networkPolicy.apiPolicy,
dispatcherPolicy,
auditContext: "comfy-history",
errorPrefix: "Comfy history lookup failed",
});
})()
: await waitForLocalHistory({
baseUrl: normalizedBaseUrl,
promptId,
headers: new Headers(headers),
timeoutMs,
pollIntervalMs,
policy: networkPolicy.apiPolicy,
dispatcherPolicy,
});
const historyEntry = extractHistoryEntry(history, promptId);
if (!historyEntry) {
throw new Error(`Comfy history response missing outputs for prompt ${promptId}`);
}
const outputFiles = collectOutputFiles({
history: historyEntry,
outputNodeId,
outputKinds: params.outputKinds,
});
if (outputFiles.length === 0) {
throw new Error(`Comfy workflow ${promptId} completed without ${params.capability} outputs`);
}
const assets: ComfyGeneratedAsset[] = [];
const maxOutputBytes = resolveComfyGeneratedOutputMaxBytes({
cfg: params.cfg,
capability: params.capability,
});
let assetIndex = 0;
for (const output of outputFiles) {
const downloaded = await downloadOutputFile({
baseUrl: normalizedBaseUrl,
headers: new Headers(headers),
timeoutMs,
policy: networkPolicy.apiPolicy,
dispatcherPolicy,
file: output.file,
mode,
capability: params.capability,
maxBytes: maxOutputBytes,
});
assetIndex += 1;
const originalName =
normalizeOptionalString(output.file.filename) || normalizeOptionalString(output.file.name);
assets.push({
buffer: downloaded.buffer,
mimeType: downloaded.mimeType,
fileName:
originalName ||
`${params.capability}-${assetIndex}.${resolveFileExtension({ mimeType: downloaded.mimeType })}`,
nodeId: output.nodeId,
});
}
return {
assets,
model: providerModel,
promptId,
outputNodeIds: uniqueStrings(outputFiles.map((entry) => entry.nodeId)),
};
}