Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
8
extensions/together/api.ts
Normal file
8
extensions/together/api.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
// Together API module exposes the plugin public contract.
|
||||
export {
|
||||
buildTogetherModelDefinition,
|
||||
TOGETHER_BASE_URL,
|
||||
TOGETHER_MODEL_CATALOG,
|
||||
} from "./models.js";
|
||||
export { buildTogetherProvider } from "./provider-catalog.js";
|
||||
export { applyTogetherConfig, TOGETHER_DEFAULT_MODEL_REF } from "./onboard.js";
|
||||
43
extensions/together/index.ts
Normal file
43
extensions/together/index.ts
Normal file
@@ -0,0 +1,43 @@
|
||||
// Together plugin entrypoint registers its OpenClaw integration.
|
||||
import { defineSingleProviderPluginEntry } from "openclaw/plugin-sdk/provider-entry";
|
||||
import { applyTogetherConfig, TOGETHER_DEFAULT_MODEL_REF } from "./onboard.js";
|
||||
import { buildTogetherProvider } from "./provider-catalog.js";
|
||||
import { buildTogetherVideoGenerationProvider } from "./video-generation-provider.js";
|
||||
|
||||
const PROVIDER_ID = "together";
|
||||
|
||||
export default defineSingleProviderPluginEntry({
|
||||
id: PROVIDER_ID,
|
||||
name: "Together Provider",
|
||||
description: "Bundled Together provider plugin",
|
||||
provider: {
|
||||
label: "Together",
|
||||
docsPath: "/providers/together",
|
||||
auth: [
|
||||
{
|
||||
methodId: "api-key",
|
||||
label: "Together AI API key",
|
||||
hint: "API key",
|
||||
optionKey: "togetherApiKey",
|
||||
flagName: "--together-api-key",
|
||||
envVar: "TOGETHER_API_KEY",
|
||||
promptMessage: "Enter Together AI API key",
|
||||
defaultModel: TOGETHER_DEFAULT_MODEL_REF,
|
||||
applyConfig: (cfg) => applyTogetherConfig(cfg),
|
||||
wizard: {
|
||||
groupLabel: "Together AI",
|
||||
},
|
||||
},
|
||||
],
|
||||
catalog: {
|
||||
buildProvider: buildTogetherProvider,
|
||||
},
|
||||
classifyFailoverReason: ({ errorMessage }) =>
|
||||
/\bconcurrency limit\b.*\b(?:breached|reached)\b/i.test(errorMessage)
|
||||
? "rate_limit"
|
||||
: undefined,
|
||||
},
|
||||
register(api) {
|
||||
api.registerVideoGenerationProvider(buildTogetherVideoGenerationProvider());
|
||||
},
|
||||
});
|
||||
24
extensions/together/models.ts
Normal file
24
extensions/together/models.ts
Normal file
@@ -0,0 +1,24 @@
|
||||
// Together plugin module implements models behavior.
|
||||
import { buildManifestModelProviderConfig } from "openclaw/plugin-sdk/provider-catalog-shared";
|
||||
import type { ModelDefinitionConfig } from "openclaw/plugin-sdk/provider-model-shared";
|
||||
import manifest from "./openclaw.plugin.json" with { type: "json" };
|
||||
|
||||
const TOGETHER_MANIFEST_PROVIDER = buildManifestModelProviderConfig({
|
||||
providerId: "together",
|
||||
catalog: manifest.modelCatalog.providers.together,
|
||||
});
|
||||
|
||||
export const TOGETHER_BASE_URL = TOGETHER_MANIFEST_PROVIDER.baseUrl;
|
||||
|
||||
export const TOGETHER_MODEL_CATALOG: ModelDefinitionConfig[] = TOGETHER_MANIFEST_PROVIDER.models;
|
||||
|
||||
export function buildTogetherModelDefinition(
|
||||
model: (typeof TOGETHER_MODEL_CATALOG)[number],
|
||||
): ModelDefinitionConfig {
|
||||
return {
|
||||
...model,
|
||||
api: "openai-completions",
|
||||
input: [...model.input],
|
||||
cost: { ...model.cost },
|
||||
};
|
||||
}
|
||||
27
extensions/together/onboard.ts
Normal file
27
extensions/together/onboard.ts
Normal file
@@ -0,0 +1,27 @@
|
||||
// Together setup module handles plugin onboarding behavior.
|
||||
import {
|
||||
createModelCatalogPresetAppliers,
|
||||
type OpenClawConfig,
|
||||
} from "openclaw/plugin-sdk/provider-onboard";
|
||||
import {
|
||||
buildTogetherModelDefinition,
|
||||
TOGETHER_BASE_URL,
|
||||
TOGETHER_MODEL_CATALOG,
|
||||
} from "./models.js";
|
||||
|
||||
export const TOGETHER_DEFAULT_MODEL_REF = "together/meta-llama/Llama-3.3-70B-Instruct-Turbo";
|
||||
|
||||
const togetherPresetAppliers = createModelCatalogPresetAppliers({
|
||||
primaryModelRef: TOGETHER_DEFAULT_MODEL_REF,
|
||||
resolveParams: (_cfg: OpenClawConfig) => ({
|
||||
providerId: "together",
|
||||
api: "openai-completions",
|
||||
baseUrl: TOGETHER_BASE_URL,
|
||||
catalogModels: TOGETHER_MODEL_CATALOG.map(buildTogetherModelDefinition),
|
||||
aliases: [{ modelRef: TOGETHER_DEFAULT_MODEL_REF, alias: "Together AI" }],
|
||||
}),
|
||||
});
|
||||
|
||||
export function applyTogetherConfig(cfg: OpenClawConfig): OpenClawConfig {
|
||||
return togetherPresetAppliers.applyConfig(cfg);
|
||||
}
|
||||
127
extensions/together/openclaw.plugin.json
Normal file
127
extensions/together/openclaw.plugin.json
Normal file
@@ -0,0 +1,127 @@
|
||||
{
|
||||
"id": "together",
|
||||
"activation": {
|
||||
"onStartup": false
|
||||
},
|
||||
"enabledByDefault": true,
|
||||
"providers": ["together"],
|
||||
"providerRequest": {
|
||||
"providers": {
|
||||
"together": {
|
||||
"family": "together"
|
||||
}
|
||||
}
|
||||
},
|
||||
"setup": {
|
||||
"providers": [
|
||||
{
|
||||
"id": "together",
|
||||
"envVars": ["TOGETHER_API_KEY"]
|
||||
}
|
||||
]
|
||||
},
|
||||
"providerAuthChoices": [
|
||||
{
|
||||
"provider": "together",
|
||||
"method": "api-key",
|
||||
"choiceId": "together-api-key",
|
||||
"choiceLabel": "Together AI API key",
|
||||
"groupId": "together",
|
||||
"groupLabel": "Together AI",
|
||||
"groupHint": "API key",
|
||||
"optionKey": "togetherApiKey",
|
||||
"cliFlag": "--together-api-key",
|
||||
"cliOption": "--together-api-key <key>",
|
||||
"cliDescription": "Together AI API key"
|
||||
}
|
||||
],
|
||||
"contracts": {
|
||||
"videoGenerationProviders": ["together"]
|
||||
},
|
||||
"modelCatalog": {
|
||||
"providers": {
|
||||
"together": {
|
||||
"baseUrl": "https://api.together.xyz/v1",
|
||||
"api": "openai-completions",
|
||||
"models": [
|
||||
{
|
||||
"id": "moonshotai/Kimi-K2.6",
|
||||
"name": "Kimi K2.6 FP4",
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"],
|
||||
"contextWindow": 262144,
|
||||
"maxTokens": 32768,
|
||||
"cost": {
|
||||
"input": 1.2,
|
||||
"output": 4.5,
|
||||
"cacheRead": 0.2,
|
||||
"cacheWrite": 4.5
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
|
||||
"name": "Llama 3.3 70B Instruct Turbo",
|
||||
"input": ["text"],
|
||||
"contextWindow": 131072,
|
||||
"maxTokens": 8192,
|
||||
"cost": {
|
||||
"input": 0.88,
|
||||
"output": 0.88,
|
||||
"cacheRead": 0.88,
|
||||
"cacheWrite": 0.88
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "deepseek-ai/DeepSeek-V4-Pro",
|
||||
"name": "DeepSeek V4 Pro",
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 512000,
|
||||
"maxTokens": 8192,
|
||||
"cost": {
|
||||
"input": 2.1,
|
||||
"output": 4.4,
|
||||
"cacheRead": 0.2,
|
||||
"cacheWrite": 4.4
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "Qwen/Qwen2.5-7B-Instruct-Turbo",
|
||||
"name": "Qwen2.5 7B Instruct Turbo",
|
||||
"input": ["text"],
|
||||
"contextWindow": 32768,
|
||||
"maxTokens": 8192,
|
||||
"cost": {
|
||||
"input": 0.3,
|
||||
"output": 0.3,
|
||||
"cacheRead": 0.3,
|
||||
"cacheWrite": 0.3
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "zai-org/GLM-5.1",
|
||||
"name": "GLM 5.1 FP4",
|
||||
"reasoning": true,
|
||||
"input": ["text"],
|
||||
"contextWindow": 202752,
|
||||
"maxTokens": 8192,
|
||||
"cost": {
|
||||
"input": 1.4,
|
||||
"output": 4.4,
|
||||
"cacheRead": 1.4,
|
||||
"cacheWrite": 4.4
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"discovery": {
|
||||
"together": "static"
|
||||
}
|
||||
},
|
||||
"configSchema": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {}
|
||||
}
|
||||
}
|
||||
15
extensions/together/package.json
Normal file
15
extensions/together/package.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "@openclaw/together-provider",
|
||||
"version": "2026.6.11",
|
||||
"private": true,
|
||||
"description": "OpenClaw Together provider plugin",
|
||||
"type": "module",
|
||||
"devDependencies": {
|
||||
"@openclaw/plugin-sdk": "workspace:*"
|
||||
},
|
||||
"openclaw": {
|
||||
"extensions": [
|
||||
"./index.ts"
|
||||
]
|
||||
}
|
||||
}
|
||||
11
extensions/together/provider-catalog.ts
Normal file
11
extensions/together/provider-catalog.ts
Normal file
@@ -0,0 +1,11 @@
|
||||
// Together provider module implements model/runtime integration.
|
||||
import { buildManifestModelProviderConfig } from "openclaw/plugin-sdk/provider-catalog-shared";
|
||||
import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-shared";
|
||||
import manifest from "./openclaw.plugin.json" with { type: "json" };
|
||||
|
||||
export function buildTogetherProvider(): ModelProviderConfig {
|
||||
return buildManifestModelProviderConfig({
|
||||
providerId: "together",
|
||||
catalog: manifest.modelCatalog.providers.together,
|
||||
});
|
||||
}
|
||||
80
extensions/together/together.live.test.ts
Normal file
80
extensions/together/together.live.test.ts
Normal file
@@ -0,0 +1,80 @@
|
||||
// Together tests cover together plugin behavior.
|
||||
import { completeSimple, type Model } from "openclaw/plugin-sdk/llm";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { TOGETHER_BASE_URL, TOGETHER_MODEL_CATALOG } from "./models.js";
|
||||
|
||||
const TOGETHER_KEY = process.env.TOGETHER_API_KEY ?? "";
|
||||
const LIVE = ["LIVE", "OPENCLAW_LIVE_TEST", "TOGETHER_LIVE_TEST"].some((name) => {
|
||||
const value = process.env[name]?.trim().toLowerCase();
|
||||
return value === "1" || value === "true" || value === "yes" || value === "on";
|
||||
});
|
||||
const TOGETHER_LIVE_TIMEOUT_MS = 45_000;
|
||||
|
||||
const describeLive = LIVE && TOGETHER_KEY ? describe : describe.skip;
|
||||
|
||||
function normalizeOpenAiCompletionInput(
|
||||
input: (typeof TOGETHER_MODEL_CATALOG)[number]["input"],
|
||||
): Array<"text" | "image"> {
|
||||
const supported = input.filter((value): value is "text" | "image" => {
|
||||
return value === "text" || value === "image";
|
||||
});
|
||||
return supported.length > 0 ? supported : ["text"];
|
||||
}
|
||||
|
||||
function buildLiveModel(model: (typeof TOGETHER_MODEL_CATALOG)[number]) {
|
||||
return {
|
||||
...model,
|
||||
api: "openai-completions",
|
||||
provider: "together",
|
||||
baseUrl: TOGETHER_BASE_URL,
|
||||
input: normalizeOpenAiCompletionInput(model.input),
|
||||
cost: { ...model.cost },
|
||||
} satisfies Model<"openai-completions">;
|
||||
}
|
||||
|
||||
function extractAssistantText(
|
||||
content: Array<{
|
||||
type?: string;
|
||||
text?: string;
|
||||
}>,
|
||||
) {
|
||||
return content
|
||||
.filter((block) => block.type === "text")
|
||||
.map((block) => block.text?.trim() ?? "")
|
||||
.filter(Boolean)
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
describeLive("together live catalog", () => {
|
||||
for (const catalogModel of TOGETHER_MODEL_CATALOG) {
|
||||
it(
|
||||
`${catalogModel.id} returns assistant text`,
|
||||
async () => {
|
||||
const model = buildLiveModel(catalogModel);
|
||||
const context = {
|
||||
messages: [
|
||||
{
|
||||
role: "user" as const,
|
||||
content: "Reply with the word ok.",
|
||||
timestamp: Date.now(),
|
||||
},
|
||||
],
|
||||
};
|
||||
let response = await completeSimple(model, context, {
|
||||
apiKey: TOGETHER_KEY,
|
||||
maxTokens: 128,
|
||||
});
|
||||
let text = extractAssistantText(response.content);
|
||||
if (text.length === 0 && response.stopReason === "length") {
|
||||
response = await completeSimple(model, context, {
|
||||
apiKey: TOGETHER_KEY,
|
||||
maxTokens: 512,
|
||||
});
|
||||
text = extractAssistantText(response.content);
|
||||
}
|
||||
expect(text.length).toBeGreaterThan(0);
|
||||
},
|
||||
TOGETHER_LIVE_TIMEOUT_MS,
|
||||
);
|
||||
}
|
||||
});
|
||||
16
extensions/together/tsconfig.json
Normal file
16
extensions/together/tsconfig.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"extends": "../tsconfig.package-boundary.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "."
|
||||
},
|
||||
"include": ["./*.ts", "./src/**/*.ts"],
|
||||
"exclude": [
|
||||
"./**/*.test.ts",
|
||||
"./dist/**",
|
||||
"./node_modules/**",
|
||||
"./src/test-support/**",
|
||||
"./src/**/*test-helpers.ts",
|
||||
"./src/**/*test-harness.ts",
|
||||
"./src/**/*test-support.ts"
|
||||
]
|
||||
}
|
||||
345
extensions/together/video-generation-provider.test.ts
Normal file
345
extensions/together/video-generation-provider.test.ts
Normal file
@@ -0,0 +1,345 @@
|
||||
// Together tests cover video generation provider plugin behavior.
|
||||
import {
|
||||
getProviderHttpMocks,
|
||||
installProviderHttpMockCleanup,
|
||||
} from "openclaw/plugin-sdk/provider-http-test-mocks";
|
||||
import { expectExplicitVideoGenerationCapabilities } from "openclaw/plugin-sdk/provider-test-contracts";
|
||||
import { beforeAll, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const { postJsonRequestMock, fetchWithTimeoutMock } = getProviderHttpMocks();
|
||||
|
||||
let buildTogetherVideoGenerationProvider: typeof import("./video-generation-provider.js").buildTogetherVideoGenerationProvider;
|
||||
|
||||
beforeAll(async () => {
|
||||
({ buildTogetherVideoGenerationProvider } = await import("./video-generation-provider.js"));
|
||||
});
|
||||
|
||||
installProviderHttpMockCleanup();
|
||||
|
||||
function requireRecord(value: unknown, label: string): Record<string, unknown> {
|
||||
if (value === null || typeof value !== "object" || Array.isArray(value)) {
|
||||
throw new Error(`expected ${label} to be a record`);
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function requireFirstPostJsonRequest(label: string): Record<string, unknown> {
|
||||
const [call] = postJsonRequestMock.mock.calls;
|
||||
if (!call) {
|
||||
throw new Error(`expected ${label}`);
|
||||
}
|
||||
return requireRecord(call[0], label);
|
||||
}
|
||||
|
||||
function streamingResponse(params: {
|
||||
body: string;
|
||||
headers?: HeadersInit;
|
||||
onCancel: () => void;
|
||||
}): Response {
|
||||
const encoded = new TextEncoder().encode(params.body);
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(encoded);
|
||||
},
|
||||
cancel() {
|
||||
params.onCancel();
|
||||
},
|
||||
});
|
||||
return new Response(stream, { headers: params.headers });
|
||||
}
|
||||
|
||||
function streamedJsonResponse(payload: unknown): Response {
|
||||
return new Response(
|
||||
new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(new TextEncoder().encode(JSON.stringify(payload)));
|
||||
controller.close();
|
||||
},
|
||||
}),
|
||||
{ headers: { "content-type": "application/json" } },
|
||||
);
|
||||
}
|
||||
|
||||
// Drives an unbounded JSON body (>16 MiB, no Content-Length) so the bounded
|
||||
// reader has to cancel the stream instead of buffering it all. A hard ceiling
|
||||
// guards the test from hanging if the reader ever fails to cancel.
|
||||
function oversizedJsonResponse(): {
|
||||
response: Response;
|
||||
state: { canceled: boolean; enqueuedBytes: number };
|
||||
} {
|
||||
const state = { canceled: false, enqueuedBytes: 0 };
|
||||
const chunk = 1024 * 1024;
|
||||
const maxChunks = 64; // 64 MiB ceiling, 4x the 16 MiB cap.
|
||||
let emitted = 0;
|
||||
const response = new Response(
|
||||
new ReadableStream({
|
||||
pull(controller) {
|
||||
if (emitted >= maxChunks) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
emitted += 1;
|
||||
state.enqueuedBytes += chunk;
|
||||
controller.enqueue(new Uint8Array(chunk));
|
||||
},
|
||||
cancel() {
|
||||
state.canceled = true;
|
||||
},
|
||||
}),
|
||||
{ headers: { "content-type": "application/json" } },
|
||||
);
|
||||
return { response, state };
|
||||
}
|
||||
|
||||
describe("together video generation provider", () => {
|
||||
it("declares explicit mode capabilities", () => {
|
||||
expectExplicitVideoGenerationCapabilities(buildTogetherVideoGenerationProvider());
|
||||
});
|
||||
|
||||
it("creates a video, polls completion, and downloads the output", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: streamedJsonResponse({
|
||||
id: "video_123",
|
||||
status: "in_progress",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
id: "video_123",
|
||||
status: "completed",
|
||||
outputs: { video_url: "https://example.com/together.mp4" },
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
headers: new Headers({ "content-type": "video/webm" }),
|
||||
arrayBuffer: async () => Buffer.from("webm-bytes"),
|
||||
});
|
||||
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
const result = await provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-T2V-A14B",
|
||||
prompt: "A bicycle weaving through a rainy neon street",
|
||||
cfg: {},
|
||||
});
|
||||
|
||||
expect(postJsonRequestMock).toHaveBeenCalledOnce();
|
||||
const request = requireFirstPostJsonRequest("Together request");
|
||||
expect(request.url).toBe("https://api.together.xyz/v2/videos");
|
||||
const body = requireRecord(request.body, "Together request body");
|
||||
expect(body.model).toBe("Wan-AI/Wan2.2-T2V-A14B");
|
||||
expect(body.prompt).toBe("A bicycle weaving through a rainy neon street");
|
||||
expect(result.videos).toHaveLength(1);
|
||||
const [video] = result.videos;
|
||||
if (!video) {
|
||||
throw new Error("Expected generated Together video");
|
||||
}
|
||||
expect(video.fileName).toBe("video-1.webm");
|
||||
expect(result.metadata).toEqual({
|
||||
videoId: "video_123",
|
||||
status: "completed",
|
||||
videoUrl: "https://example.com/together.mp4",
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds an unbounded successful Together create JSON body and cancels the stream", async () => {
|
||||
const oversized = oversizedJsonResponse();
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: oversized.response,
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
await expect(
|
||||
provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-T2V-A14B",
|
||||
prompt: "oversized create body",
|
||||
cfg: {},
|
||||
}),
|
||||
).rejects.toThrow("Together video generation failed: JSON response exceeds 16777216 bytes");
|
||||
// The bounded reader cancelled the stream rather than buffering the whole
|
||||
// body, and stopped reading well before the 64 MiB ceiling.
|
||||
expect(oversized.state.canceled).toBe(true);
|
||||
expect(oversized.state.enqueuedBytes).toBeLessThan(64 * 1024 * 1024);
|
||||
expect(fetchWithTimeoutMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("bounds downloaded videos before materializing them", async () => {
|
||||
let canceled = false;
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: streamedJsonResponse({
|
||||
id: "video_oversized",
|
||||
status: "in_progress",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
id: "video_oversized",
|
||||
status: "completed",
|
||||
outputs: { video_url: "https://example.com/oversized.mp4" },
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce(
|
||||
streamingResponse({
|
||||
body: "x".repeat(32),
|
||||
headers: { "content-type": "video/mp4" },
|
||||
onCancel: () => {
|
||||
canceled = true;
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
await expect(
|
||||
provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-T2V-A14B",
|
||||
prompt: "oversized video",
|
||||
cfg: { agents: { defaults: { mediaMaxMb: 0.00001 } } },
|
||||
}),
|
||||
).rejects.toThrow("Together generated video download exceeds");
|
||||
expect(canceled).toBe(true);
|
||||
});
|
||||
|
||||
it("uses the video API endpoint when the shared Together text base URL is configured", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: streamedJsonResponse({
|
||||
id: "video_123",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
id: "video_123",
|
||||
status: "completed",
|
||||
outputs: { video_url: "https://example.com/together.mp4" },
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
headers: new Headers({ "content-type": "video/mp4" }),
|
||||
arrayBuffer: async () => Buffer.from("mp4-bytes"),
|
||||
});
|
||||
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
await provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-T2V-A14B",
|
||||
prompt: "A bicycle weaving through a rainy neon street",
|
||||
cfg: {
|
||||
models: {
|
||||
providers: {
|
||||
together: {
|
||||
baseUrl: "https://api.together.xyz/v1",
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const request = requireFirstPostJsonRequest("Together request");
|
||||
expect(request.url).toBe("https://api.together.xyz/v2/videos");
|
||||
});
|
||||
|
||||
it("drops out-of-range duration values before creating videos", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: streamedJsonResponse({
|
||||
id: "video_123",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
id: "video_123",
|
||||
status: "completed",
|
||||
outputs: { video_url: "https://example.com/together.mp4" },
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
headers: new Headers({ "content-type": "video/mp4" }),
|
||||
arrayBuffer: async () => Buffer.from("mp4-bytes"),
|
||||
});
|
||||
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
await provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-T2V-A14B",
|
||||
prompt: "A bicycle weaving through a rainy neon street",
|
||||
durationSeconds: 99,
|
||||
cfg: {},
|
||||
});
|
||||
|
||||
const request = requireFirstPostJsonRequest("Together request");
|
||||
const body = requireRecord(request.body, "Together request body");
|
||||
expect(body).not.toHaveProperty("seconds");
|
||||
});
|
||||
|
||||
it("rejects reference images for Together text-to-video models before calling the API", async () => {
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
|
||||
await expect(
|
||||
provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-T2V-A14B",
|
||||
prompt: "A bicycle weaving through a rainy neon street",
|
||||
cfg: {},
|
||||
inputImages: [
|
||||
{
|
||||
buffer: Buffer.from("png"),
|
||||
mimeType: "image/png",
|
||||
fileName: "reference.png",
|
||||
},
|
||||
],
|
||||
}),
|
||||
).rejects.toThrow(/does not support image reference inputs/u);
|
||||
expect(postJsonRequestMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("sends reference images for the Together image-to-video model", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: streamedJsonResponse({
|
||||
id: "video_123",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
id: "video_123",
|
||||
status: "completed",
|
||||
outputs: { video_url: "https://example.com/together.mp4" },
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
headers: new Headers({ "content-type": "video/mp4" }),
|
||||
arrayBuffer: async () => Buffer.from("mp4-bytes"),
|
||||
});
|
||||
|
||||
const provider = buildTogetherVideoGenerationProvider();
|
||||
await provider.generateVideo({
|
||||
provider: "together",
|
||||
model: "Wan-AI/Wan2.2-I2V-A14B",
|
||||
prompt: "Animate the reference art.",
|
||||
cfg: {},
|
||||
inputImages: [
|
||||
{
|
||||
buffer: Buffer.from("png"),
|
||||
mimeType: "image/png",
|
||||
fileName: "reference.png",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const request = requireFirstPostJsonRequest("Together request");
|
||||
const body = requireRecord(request.body, "Together request body");
|
||||
expect(body.model).toBe("Wan-AI/Wan2.2-I2V-A14B");
|
||||
expect(body.reference_images).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
332
extensions/together/video-generation-provider.ts
Normal file
332
extensions/together/video-generation-provider.ts
Normal file
@@ -0,0 +1,332 @@
|
||||
// Together provider module implements model/runtime integration.
|
||||
import { toImageDataUrl } from "openclaw/plugin-sdk/image-generation";
|
||||
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
|
||||
import { isProviderApiKeyConfigured } from "openclaw/plugin-sdk/provider-auth";
|
||||
import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runtime";
|
||||
import {
|
||||
assertOkOrThrowHttpError,
|
||||
createProviderOperationDeadline,
|
||||
createProviderOperationTimeoutResolver,
|
||||
fetchProviderDownloadResponse,
|
||||
pollProviderOperationJson,
|
||||
postJsonRequest,
|
||||
readProviderJsonResponse,
|
||||
resolveProviderOperationTimeoutMs,
|
||||
resolveProviderHttpRequestConfig,
|
||||
type ProviderOperationTimeoutMs,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
|
||||
import {
|
||||
asSafeIntegerInRange,
|
||||
normalizeOptionalString,
|
||||
} from "openclaw/plugin-sdk/string-coerce-runtime";
|
||||
import type {
|
||||
GeneratedVideoAsset,
|
||||
VideoGenerationProvider,
|
||||
VideoGenerationRequest,
|
||||
} from "openclaw/plugin-sdk/video-generation";
|
||||
import { TOGETHER_BASE_URL } from "./models.js";
|
||||
|
||||
const DEFAULT_TOGETHER_VIDEO_MODEL = "Wan-AI/Wan2.2-T2V-A14B";
|
||||
const TOGETHER_IMAGE_TO_VIDEO_MODELS = new Set(["Wan-AI/Wan2.2-I2V-A14B"]);
|
||||
const TOGETHER_VIDEO_BASE_URL = "https://api.together.xyz/v2";
|
||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
||||
const POLL_INTERVAL_MS = 5_000;
|
||||
const MAX_POLL_ATTEMPTS = 120;
|
||||
const TOGETHER_MIN_DURATION_SECONDS = 1;
|
||||
const TOGETHER_MAX_DURATION_SECONDS = 10;
|
||||
const DEFAULT_GENERATED_VIDEO_MAX_BYTES = 16 * 1024 * 1024;
|
||||
|
||||
type TogetherVideoResponse = {
|
||||
id?: string;
|
||||
model?: string;
|
||||
status?: "in_progress" | "completed" | "failed";
|
||||
error?: {
|
||||
code?: string;
|
||||
message?: string;
|
||||
} | null;
|
||||
outputs?:
|
||||
| {
|
||||
video_url?: string;
|
||||
url?: string;
|
||||
}
|
||||
| Array<{
|
||||
video_url?: string;
|
||||
url?: string;
|
||||
}>;
|
||||
};
|
||||
|
||||
// Reads the Together create-video response through the shared provider JSON
|
||||
// reader so a provider that streams an unbounded JSON body cannot force the
|
||||
// runtime to buffer the whole payload before parsing it on the success path.
|
||||
// The shared helper applies the established 16 MiB provider JSON cap and the
|
||||
// standard malformed-JSON wrapping instead of a provider-local reimplementation.
|
||||
async function readTogetherVideoJson(response: Response): Promise<TogetherVideoResponse> {
|
||||
return (await readProviderJsonResponse(
|
||||
response,
|
||||
"Together video generation failed",
|
||||
)) as TogetherVideoResponse;
|
||||
}
|
||||
|
||||
function resolveTogetherVideoBaseUrl(req: VideoGenerationRequest): string {
|
||||
const configuredBaseUrl = normalizeOptionalString(req.cfg?.models?.providers?.together?.baseUrl);
|
||||
if (
|
||||
!configuredBaseUrl ||
|
||||
stripTrailingSlash(configuredBaseUrl) === stripTrailingSlash(TOGETHER_BASE_URL)
|
||||
) {
|
||||
return TOGETHER_VIDEO_BASE_URL;
|
||||
}
|
||||
return configuredBaseUrl;
|
||||
}
|
||||
|
||||
function stripTrailingSlash(value: string): string {
|
||||
return value.replace(/\/+$/u, "");
|
||||
}
|
||||
|
||||
function extractTogetherVideoUrl(payload: TogetherVideoResponse): string | undefined {
|
||||
if (Array.isArray(payload.outputs)) {
|
||||
for (const entry of payload.outputs) {
|
||||
const url = normalizeOptionalString(entry.video_url) ?? normalizeOptionalString(entry.url);
|
||||
if (url) {
|
||||
return url;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
return (
|
||||
normalizeOptionalString(payload.outputs?.video_url) ??
|
||||
normalizeOptionalString(payload.outputs?.url)
|
||||
);
|
||||
}
|
||||
|
||||
function resolveTogetherDurationSeconds(value: unknown): string | undefined {
|
||||
if (typeof value !== "number" || !Number.isFinite(value)) {
|
||||
return undefined;
|
||||
}
|
||||
const duration = asSafeIntegerInRange(Math.round(value), {
|
||||
min: TOGETHER_MIN_DURATION_SECONDS,
|
||||
max: TOGETHER_MAX_DURATION_SECONDS,
|
||||
});
|
||||
return duration === undefined ? undefined : String(duration);
|
||||
}
|
||||
|
||||
function resolveGeneratedVideoMaxBytes(req: VideoGenerationRequest): number {
|
||||
const configured = req.cfg.agents?.defaults?.mediaMaxMb;
|
||||
if (typeof configured === "number" && Number.isFinite(configured) && configured > 0) {
|
||||
return Math.floor(configured * 1024 * 1024);
|
||||
}
|
||||
return DEFAULT_GENERATED_VIDEO_MAX_BYTES;
|
||||
}
|
||||
|
||||
async function pollTogetherVideo(params: {
|
||||
videoId: string;
|
||||
headers: Headers;
|
||||
timeoutMs?: number;
|
||||
baseUrl: string;
|
||||
fetchFn: typeof fetch;
|
||||
}): Promise<TogetherVideoResponse> {
|
||||
const deadline = createProviderOperationDeadline({
|
||||
timeoutMs: params.timeoutMs,
|
||||
label: `Together video generation task ${params.videoId}`,
|
||||
});
|
||||
return await pollProviderOperationJson<TogetherVideoResponse>({
|
||||
url: `${params.baseUrl}/videos/${params.videoId}`,
|
||||
headers: params.headers,
|
||||
deadline,
|
||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
||||
fetchFn: params.fetchFn,
|
||||
maxAttempts: MAX_POLL_ATTEMPTS,
|
||||
pollIntervalMs: POLL_INTERVAL_MS,
|
||||
requestFailedMessage: "Together video status request failed",
|
||||
timeoutMessage: `Together video generation task ${params.videoId} did not finish in time`,
|
||||
isComplete: (payload) => payload.status === "completed",
|
||||
getFailureMessage: (payload) =>
|
||||
payload.status === "failed"
|
||||
? (normalizeOptionalString(payload.error?.message) ?? "Together video generation failed")
|
||||
: undefined,
|
||||
});
|
||||
}
|
||||
|
||||
async function downloadTogetherVideo(params: {
|
||||
url: string;
|
||||
timeoutMs?: ProviderOperationTimeoutMs;
|
||||
fetchFn: typeof fetch;
|
||||
maxBytes: number;
|
||||
}): Promise<GeneratedVideoAsset> {
|
||||
const response = await fetchProviderDownloadResponse({
|
||||
url: params.url,
|
||||
init: { method: "GET" },
|
||||
timeoutMs: params.timeoutMs ?? DEFAULT_TIMEOUT_MS,
|
||||
fetchFn: params.fetchFn,
|
||||
provider: "together",
|
||||
requestFailedMessage: "Together generated video download failed",
|
||||
});
|
||||
const mimeType = normalizeOptionalString(response.headers.get("content-type")) ?? "video/mp4";
|
||||
const buffer = await readResponseWithLimit(response, params.maxBytes, {
|
||||
onOverflow: ({ maxBytes }) =>
|
||||
new Error(`Together generated video download exceeds ${maxBytes} bytes`),
|
||||
});
|
||||
return {
|
||||
buffer,
|
||||
mimeType,
|
||||
fileName: `video-1.${extensionForMime(mimeType)?.slice(1) ?? "mp4"}`,
|
||||
};
|
||||
}
|
||||
|
||||
export function buildTogetherVideoGenerationProvider(): VideoGenerationProvider {
|
||||
return {
|
||||
id: "together",
|
||||
label: "Together",
|
||||
defaultModel: DEFAULT_TOGETHER_VIDEO_MODEL,
|
||||
models: [
|
||||
DEFAULT_TOGETHER_VIDEO_MODEL,
|
||||
"Wan-AI/Wan2.2-I2V-A14B",
|
||||
"minimax/Hailuo-02",
|
||||
"Kwai/Kling-2.1-Master",
|
||||
],
|
||||
isConfigured: ({ agentDir }) =>
|
||||
isProviderApiKeyConfigured({
|
||||
provider: "together",
|
||||
agentDir,
|
||||
}),
|
||||
capabilities: {
|
||||
generate: {
|
||||
maxVideos: 1,
|
||||
maxDurationSeconds: TOGETHER_MAX_DURATION_SECONDS,
|
||||
supportsSize: true,
|
||||
},
|
||||
imageToVideo: {
|
||||
enabled: true,
|
||||
maxInputImagesByModel: {
|
||||
"Wan-AI/Wan2.2-I2V-A14B": 1,
|
||||
},
|
||||
maxDurationSeconds: TOGETHER_MAX_DURATION_SECONDS,
|
||||
supportsSize: true,
|
||||
},
|
||||
videoToVideo: {
|
||||
enabled: false,
|
||||
},
|
||||
},
|
||||
async generateVideo(req) {
|
||||
if ((req.inputVideos?.length ?? 0) > 0) {
|
||||
throw new Error("Together video generation does not support video reference inputs.");
|
||||
}
|
||||
const auth = await resolveApiKeyForProvider({
|
||||
provider: "together",
|
||||
cfg: req.cfg,
|
||||
agentDir: req.agentDir,
|
||||
store: req.authStore,
|
||||
});
|
||||
if (!auth.apiKey) {
|
||||
throw new Error("Together API key missing");
|
||||
}
|
||||
|
||||
const fetchFn = fetch;
|
||||
const deadline = createProviderOperationDeadline({
|
||||
timeoutMs: req.timeoutMs,
|
||||
label: "Together video generation",
|
||||
});
|
||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
|
||||
resolveProviderHttpRequestConfig({
|
||||
baseUrl: resolveTogetherVideoBaseUrl(req),
|
||||
defaultBaseUrl: TOGETHER_VIDEO_BASE_URL,
|
||||
allowPrivateNetwork: false,
|
||||
defaultHeaders: {
|
||||
Authorization: `Bearer ${auth.apiKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
provider: "together",
|
||||
capability: "video",
|
||||
transport: "http",
|
||||
});
|
||||
const body: Record<string, unknown> = {
|
||||
model: normalizeOptionalString(req.model) ?? DEFAULT_TOGETHER_VIDEO_MODEL,
|
||||
prompt: req.prompt,
|
||||
};
|
||||
const model = String(body.model);
|
||||
const duration = resolveTogetherDurationSeconds(req.durationSeconds);
|
||||
if (duration !== undefined) {
|
||||
body.seconds = duration;
|
||||
}
|
||||
const size = normalizeOptionalString(req.size);
|
||||
if (size) {
|
||||
const match = /^(\d+)x(\d+)$/u.exec(size);
|
||||
if (match) {
|
||||
body.width = Number.parseInt(match[1] ?? "", 10);
|
||||
body.height = Number.parseInt(match[2] ?? "", 10);
|
||||
}
|
||||
}
|
||||
if (req.inputImages?.[0]) {
|
||||
if (!TOGETHER_IMAGE_TO_VIDEO_MODELS.has(model)) {
|
||||
throw new Error(
|
||||
`Together video model ${model} does not support image reference inputs. Use Wan-AI/Wan2.2-I2V-A14B or omit input images.`,
|
||||
);
|
||||
}
|
||||
const input = req.inputImages[0];
|
||||
const value = normalizeOptionalString(input.url)
|
||||
? normalizeOptionalString(input.url)
|
||||
: input.buffer
|
||||
? toImageDataUrl({ ...input, buffer: input.buffer, defaultMimeType: "image/png" })
|
||||
: undefined;
|
||||
if (!value) {
|
||||
throw new Error("Together reference image is missing image data.");
|
||||
}
|
||||
body.reference_images = [value];
|
||||
}
|
||||
const { response, release } = await postJsonRequest({
|
||||
url: `${baseUrl}/videos`,
|
||||
headers,
|
||||
body,
|
||||
timeoutMs: resolveProviderOperationTimeoutMs({
|
||||
deadline,
|
||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
||||
}),
|
||||
fetchFn,
|
||||
allowPrivateNetwork,
|
||||
dispatcherPolicy,
|
||||
});
|
||||
try {
|
||||
await assertOkOrThrowHttpError(response, "Together video generation failed");
|
||||
const submitted = await readTogetherVideoJson(response);
|
||||
const videoId = normalizeOptionalString(submitted.id);
|
||||
if (!videoId) {
|
||||
throw new Error("Together video generation response missing id");
|
||||
}
|
||||
const completed = await pollTogetherVideo({
|
||||
videoId,
|
||||
headers,
|
||||
timeoutMs: resolveProviderOperationTimeoutMs({
|
||||
deadline,
|
||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
||||
}),
|
||||
baseUrl,
|
||||
fetchFn,
|
||||
});
|
||||
const videoUrl = extractTogetherVideoUrl(completed);
|
||||
if (!videoUrl) {
|
||||
throw new Error("Together video generation completed without an output URL");
|
||||
}
|
||||
const video = await downloadTogetherVideo({
|
||||
url: videoUrl,
|
||||
timeoutMs: createProviderOperationTimeoutResolver({
|
||||
deadline,
|
||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
||||
}),
|
||||
fetchFn,
|
||||
maxBytes: resolveGeneratedVideoMaxBytes(req),
|
||||
});
|
||||
return {
|
||||
videos: [video],
|
||||
model: completed.model ?? req.model ?? DEFAULT_TOGETHER_VIDEO_MODEL,
|
||||
metadata: {
|
||||
videoId,
|
||||
status: completed.status,
|
||||
videoUrl,
|
||||
},
|
||||
};
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user