Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled

Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
2026-07-05 09:36:54 +00:00
parent 3216769225
commit bedb527145
21108 changed files with 6010766 additions and 0 deletions

View File

@@ -0,0 +1,3 @@
# LM Studio Provider
Bundled provider plugin for LM Studio discovery, auto-load, and setup.

View File

@@ -0,0 +1,37 @@
// Lmstudio API module exposes the plugin public contract.
export {
buildLmstudioAuthHeaders,
buildLmstudioModelName,
configureLmstudioNonInteractive,
discoverLmstudioProvider,
LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
LMSTUDIO_DEFAULT_BASE_URL,
LMSTUDIO_DEFAULT_EMBEDDING_MODEL,
LMSTUDIO_DEFAULT_INFERENCE_BASE_URL,
LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
LMSTUDIO_DEFAULT_MODEL_ID,
LMSTUDIO_DOCKER_HOST_BASE_URL,
LMSTUDIO_DOCKER_HOST_INFERENCE_BASE_URL,
LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
LMSTUDIO_MODEL_PLACEHOLDER,
LMSTUDIO_PROVIDER_ID,
LMSTUDIO_PROVIDER_LABEL,
type LmstudioModelBase,
type LmstudioModelWire,
mapLmstudioWireEntry,
mapLmstudioWireModelsToConfig,
normalizeLmstudioConfiguredCatalogEntries,
normalizeLmstudioConfiguredCatalogEntry,
normalizeLmstudioProviderConfig,
prepareLmstudioDynamicModels,
promptAndConfigureLmstudioInteractive,
resolveLmstudioConfiguredApiKey,
resolveLmstudioInferenceBase,
resolveLmstudioProviderHeaders,
resolveLmstudioReasoningCapability,
resolveLmstudioReasoningCompat,
resolveLmstudioRequestContext,
resolveLmstudioRuntimeApiKey,
resolveLmstudioServerBase,
resolveLoadedContextWindow,
} from "./src/api.js";

View File

@@ -0,0 +1,208 @@
// Lmstudio tests cover index plugin behavior.
import type { OpenClawConfig } from "openclaw/plugin-sdk/plugin-entry";
import { capturePluginRegistration } from "openclaw/plugin-sdk/plugin-test-runtime";
import { CUSTOM_LOCAL_AUTH_MARKER } from "openclaw/plugin-sdk/provider-auth";
import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-shared";
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
import { LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER } from "./src/defaults.js";
function registerProvider() {
const captured = capturePluginRegistration(plugin);
const provider = captured.providers[0];
expect(provider?.id).toBe("lmstudio");
return provider;
}
function createRemoteProviderConfig(overrides?: Partial<ModelProviderConfig>): ModelProviderConfig {
return {
api: "openai-completions",
baseUrl: "http://lmstudio.internal:1234/v1",
models: [
{
id: "qwen/qwen3.5-9b",
name: "Qwen 3.5 9B",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 131072,
maxTokens: 8192,
},
],
...overrides,
};
}
describe("lmstudio plugin", () => {
it("canonicalizes base URLs during provider normalization", () => {
const provider = registerProvider();
const providerConfig = createRemoteProviderConfig({
baseUrl: "http://localhost:1234/api/v1/",
});
expect(
provider?.normalizeConfig?.({
provider: "lmstudio",
providerConfig,
}),
).toEqual({
...providerConfig,
baseUrl: "http://localhost:1234/v1",
request: { allowPrivateNetwork: true },
});
});
it("synthesizes placeholder auth for configured lmstudio models without API key auth", () => {
const provider = registerProvider();
expect(
provider?.resolveSyntheticAuth?.({
provider: "lmstudio",
config: {},
providerConfig: createRemoteProviderConfig({
headers: {
"X-Proxy-Auth": "proxy-token",
},
}),
}),
).toEqual({
apiKey: CUSTOM_LOCAL_AUTH_MARKER,
source: "models.providers.lmstudio (synthetic local key)",
mode: "api-key",
});
});
it("still synthesizes placeholder auth when explicit api-key auth has no key", () => {
const provider = registerProvider();
expect(
provider?.resolveSyntheticAuth?.({
provider: "lmstudio",
config: {},
providerConfig: createRemoteProviderConfig({
auth: "api-key",
}),
}),
).toEqual({
apiKey: CUSTOM_LOCAL_AUTH_MARKER,
source: "models.providers.lmstudio (synthetic local key)",
mode: "api-key",
});
});
it("does not synthesize placeholder auth when Authorization header is configured", () => {
const provider = registerProvider();
expect(
provider?.resolveSyntheticAuth?.({
provider: "lmstudio",
config: {},
providerConfig: createRemoteProviderConfig({
headers: {
Authorization: "Bearer proxy-token",
},
}),
}),
).toBeUndefined();
});
it("defers stored lmstudio-local profile auth so real credentials can win", () => {
const provider = registerProvider();
expect(
provider?.shouldDeferSyntheticProfileAuth?.({
provider: "lmstudio",
config: {},
providerConfig: createRemoteProviderConfig(),
resolvedApiKey: LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
}),
).toBe(true);
expect(
provider?.shouldDeferSyntheticProfileAuth?.({
provider: "lmstudio",
config: {},
providerConfig: createRemoteProviderConfig(),
resolvedApiKey: CUSTOM_LOCAL_AUTH_MARKER,
}),
).toBe(true);
expect(
provider?.shouldDeferSyntheticProfileAuth?.({
provider: "lmstudio",
config: {},
providerConfig: createRemoteProviderConfig(),
resolvedApiKey: "lmstudio-real-key",
}),
).toBe(false);
});
it("augments the catalog with configured lmstudio models", () => {
const provider = registerProvider();
const config = {
models: {
providers: {
lmstudio: {
models: [
{
id: "qwen3-8b-instruct",
name: "Qwen 3 8B Instruct",
contextWindow: 32768,
contextTokens: 8192,
reasoning: true,
input: ["text", "image"],
compat: {
supportsReasoningEffort: true,
supportedReasoningEfforts: ["off", "on"],
reasoningEffortMap: { off: "off", high: "on" },
},
},
{
id: "phi-4",
},
{
id: " ",
name: "ignored",
},
],
},
},
},
} as unknown as OpenClawConfig;
expect(
provider?.augmentModelCatalog?.({
config,
agentDir: "/tmp/openclaw",
env: {},
entries: [],
}),
).toEqual([
{
provider: "lmstudio",
id: "qwen3-8b-instruct",
name: "Qwen 3 8B Instruct",
compat: {
supportsUsageInStreaming: true,
supportsReasoningEffort: true,
supportedReasoningEfforts: ["none", "minimal", "low", "medium", "high", "xhigh"],
reasoningEffortMap: { off: "none", none: "none", adaptive: "xhigh", max: "xhigh" },
},
contextWindow: 32768,
contextTokens: 8192,
reasoning: true,
input: ["text", "image"],
},
{
provider: "lmstudio",
id: "phi-4",
name: "phi-4",
compat: { supportsUsageInStreaming: true },
contextWindow: undefined,
contextTokens: undefined,
reasoning: undefined,
input: undefined,
},
]);
});
});

View File

@@ -0,0 +1,138 @@
// Lmstudio plugin entrypoint registers its OpenClaw integration.
import {
definePluginEntry,
type OpenClawPluginApi,
type ProviderAuthContext,
type ProviderAuthMethodNonInteractiveContext,
type ProviderAuthResult,
type ProviderRuntimeModel,
} from "openclaw/plugin-sdk/plugin-entry";
import type { OpenClawConfig } from "openclaw/plugin-sdk/plugin-entry";
import { CUSTOM_LOCAL_AUTH_MARKER } from "openclaw/plugin-sdk/provider-auth";
import { lmstudioMemoryEmbeddingProviderAdapter } from "./memory-embedding-adapter.js";
import {
LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
LMSTUDIO_PROVIDER_LABEL,
} from "./src/defaults.js";
import {
normalizeLmstudioConfiguredCatalogEntries,
normalizeLmstudioProviderConfig,
} from "./src/models.js";
import { shouldUseLmstudioSyntheticAuth } from "./src/provider-auth.js";
import { wrapLmstudioInferencePreload } from "./src/stream.js";
const PROVIDER_ID = "lmstudio";
// Intentional: dynamic models are cached per LM Studio endpoint (`baseUrl`) only.
const cachedDynamicModels = new Map<string, ProviderRuntimeModel[]>();
function resolveLmstudioAugmentedCatalogEntries(config: OpenClawConfig | undefined) {
if (!config) {
return [];
}
return normalizeLmstudioConfiguredCatalogEntries(config.models?.providers?.lmstudio?.models).map(
(entry) => ({
provider: PROVIDER_ID,
id: entry.id,
name: entry.name ?? entry.id,
compat: { ...entry.compat, supportsUsageInStreaming: true },
contextWindow: entry.contextWindow,
contextTokens: entry.contextTokens,
reasoning: entry.reasoning,
input: entry.input,
}),
);
}
/** Lazily loads setup helpers so provider wiring stays lightweight at startup. */
async function loadProviderSetup() {
return await import("./api.js");
}
export default definePluginEntry({
id: PROVIDER_ID,
name: "LM Studio Provider",
description: "Bundled LM Studio provider plugin",
register(api: OpenClawPluginApi) {
api.registerMemoryEmbeddingProvider(lmstudioMemoryEmbeddingProviderAdapter);
api.registerProvider({
id: PROVIDER_ID,
label: "LM Studio",
docsPath: "/providers/lmstudio",
envVars: [LMSTUDIO_DEFAULT_API_KEY_ENV_VAR],
auth: [
{
id: "custom",
label: LMSTUDIO_PROVIDER_LABEL,
hint: "Local/self-hosted LM Studio server",
kind: "custom",
run: async (ctx: ProviderAuthContext): Promise<ProviderAuthResult> => {
const providerSetup = await loadProviderSetup();
return await providerSetup.promptAndConfigureLmstudioInteractive({
config: ctx.config,
agentDir: ctx.agentDir,
prompter: ctx.prompter,
secretInputMode: ctx.secretInputMode,
allowSecretRefPrompt: ctx.allowSecretRefPrompt,
});
},
runNonInteractive: async (ctx: ProviderAuthMethodNonInteractiveContext) => {
const providerSetup = await loadProviderSetup();
return await providerSetup.configureLmstudioNonInteractive(ctx);
},
},
],
catalog: {
// Run after early providers so local LM Studio detection does not dominate resolution.
order: "late",
run: async (ctx) => {
const providerSetup = await loadProviderSetup();
return await providerSetup.discoverLmstudioProvider(ctx);
},
},
resolveSyntheticAuth: ({ providerConfig }) => {
if (!shouldUseLmstudioSyntheticAuth(providerConfig)) {
return undefined;
}
return {
apiKey: CUSTOM_LOCAL_AUTH_MARKER,
source: "models.providers.lmstudio (synthetic local key)",
mode: "api-key" as const,
};
},
shouldDeferSyntheticProfileAuth: ({ resolvedApiKey }) =>
resolvedApiKey?.trim() === LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER ||
resolvedApiKey?.trim() === CUSTOM_LOCAL_AUTH_MARKER,
normalizeConfig: ({ providerConfig }) => normalizeLmstudioProviderConfig(providerConfig),
prepareDynamicModel: async (ctx) => {
const providerSetup = await loadProviderSetup();
cachedDynamicModels.set(
ctx.providerConfig?.baseUrl ?? "",
await providerSetup.prepareLmstudioDynamicModels(ctx),
);
},
resolveDynamicModel: (ctx) =>
cachedDynamicModels
.get(ctx.providerConfig?.baseUrl ?? "")
?.find((model) => model.id === ctx.modelId),
augmentModelCatalog: (ctx) => resolveLmstudioAugmentedCatalogEntries(ctx.config),
wrapStreamFn: wrapLmstudioInferencePreload,
wizard: {
setup: {
choiceId: PROVIDER_ID,
choiceLabel: "LM Studio",
choiceHint: "Local/self-hosted LM Studio server",
groupId: PROVIDER_ID,
groupLabel: "LM Studio",
groupHint: "Self-hosted open-weight models",
methodId: "custom",
},
modelPicker: {
label: "LM Studio (custom)",
hint: "Detect models from LM Studio /api/v1/models",
methodId: "custom",
},
},
});
},
});

View File

@@ -0,0 +1,37 @@
// Lmstudio plugin module implements memory embedding adapter behavior.
import {
sanitizeEmbeddingCacheHeaders,
type MemoryEmbeddingProviderAdapter,
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
import {
createLmstudioEmbeddingProvider,
DEFAULT_LMSTUDIO_EMBEDDING_MODEL,
} from "./src/embedding-provider.js";
export const lmstudioMemoryEmbeddingProviderAdapter: MemoryEmbeddingProviderAdapter = {
id: "lmstudio",
defaultModel: DEFAULT_LMSTUDIO_EMBEDDING_MODEL,
transport: "remote",
authProviderId: "lmstudio",
allowExplicitWhenConfiguredAuto: true,
create: async (options) => {
const { provider, client } = await createLmstudioEmbeddingProvider({
...options,
provider: "lmstudio",
fallback: "none",
});
return {
provider,
runtime: {
id: "lmstudio",
inlineBatchTimeoutMs: 10 * 60_000,
cacheKeyData: {
provider: "lmstudio",
baseUrl: client.baseUrl,
model: client.model,
headers: sanitizeEmbeddingCacheHeaders(client.headers, ["authorization"]),
},
},
};
},
};

View File

@@ -0,0 +1,60 @@
{
"id": "lmstudio",
"icon": "https://cdn.simpleicons.org/lmstudio",
"activation": {
"onStartup": false
},
"enabledByDefault": true,
"providers": ["lmstudio"],
"providerRequest": {
"providers": {
"lmstudio": {
"family": "lmstudio",
"openAICompletions": {
"supportsStreamingUsage": true
}
}
}
},
"modelPricing": {
"providers": {
"lmstudio": {
"external": false
}
}
},
"nonSecretAuthMarkers": ["lmstudio-local"],
"syntheticAuthRefs": ["lmstudio"],
"setup": {
"providers": [
{
"id": "lmstudio",
"envVars": ["LM_API_TOKEN"]
}
]
},
"providerAuthChoices": [
{
"provider": "lmstudio",
"method": "custom",
"choiceId": "lmstudio",
"choiceLabel": "LM Studio",
"choiceHint": "Local/self-hosted LM Studio server",
"optionKey": "lmstudioApiKey",
"cliFlag": "--lmstudio-api-key",
"cliOption": "--lmstudio-api-key <key>",
"cliDescription": "LM Studio API key",
"groupId": "lmstudio",
"groupLabel": "LM Studio",
"groupHint": "Self-hosted open-weight models"
}
],
"contracts": {
"memoryEmbeddingProviders": ["lmstudio"]
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {}
}
}

View File

@@ -0,0 +1,12 @@
{
"name": "@openclaw/lmstudio-provider",
"version": "2026.6.11",
"private": true,
"description": "OpenClaw LM Studio provider plugin",
"type": "module",
"openclaw": {
"extensions": [
"./index.ts"
]
}
}

View File

@@ -0,0 +1,36 @@
// Lmstudio API module exposes the plugin public contract.
export {
LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
LMSTUDIO_DEFAULT_BASE_URL,
LMSTUDIO_DEFAULT_EMBEDDING_MODEL,
LMSTUDIO_DEFAULT_INFERENCE_BASE_URL,
LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
LMSTUDIO_DEFAULT_MODEL_ID,
LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
LMSTUDIO_MODEL_PLACEHOLDER,
LMSTUDIO_PROVIDER_ID,
LMSTUDIO_PROVIDER_LABEL,
} from "./src/defaults.js";
export {
discoverLmstudioModels,
ensureLmstudioModelLoaded,
fetchLmstudioModels,
} from "./src/models.fetch.js";
export {
mapLmstudioWireEntry,
mapLmstudioWireModelsToConfig,
normalizeLmstudioProviderConfig,
resolveLoadedContextWindow,
resolveLmstudioInferenceBase,
resolveLmstudioReasoningCapability,
resolveLmstudioServerBase,
type LmstudioModelBase,
type LmstudioModelWire,
} from "./src/models.js";
export {
buildLmstudioAuthHeaders,
resolveLmstudioConfiguredApiKey,
resolveLmstudioProviderHeaders,
resolveLmstudioRequestContext,
resolveLmstudioRuntimeApiKey,
} from "./src/runtime.js";

View File

@@ -0,0 +1,43 @@
// Lmstudio API module exposes the plugin public contract.
export {
LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
LMSTUDIO_DEFAULT_BASE_URL,
LMSTUDIO_DEFAULT_EMBEDDING_MODEL,
LMSTUDIO_DEFAULT_INFERENCE_BASE_URL,
LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
LMSTUDIO_DEFAULT_MODEL_ID,
LMSTUDIO_DOCKER_HOST_BASE_URL,
LMSTUDIO_DOCKER_HOST_INFERENCE_BASE_URL,
LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
LMSTUDIO_MODEL_PLACEHOLDER,
LMSTUDIO_PROVIDER_ID,
LMSTUDIO_PROVIDER_LABEL,
} from "./defaults.js";
export {
buildLmstudioModelName,
type LmstudioModelBase,
type LmstudioModelWire,
mapLmstudioWireEntry,
mapLmstudioWireModelsToConfig,
normalizeLmstudioConfiguredCatalogEntries,
normalizeLmstudioConfiguredCatalogEntry,
normalizeLmstudioProviderConfig,
resolveLmstudioInferenceBase,
resolveLmstudioReasoningCapability,
resolveLmstudioReasoningCompat,
resolveLmstudioServerBase,
resolveLoadedContextWindow,
} from "./models.js";
export {
buildLmstudioAuthHeaders,
resolveLmstudioConfiguredApiKey,
resolveLmstudioProviderHeaders,
resolveLmstudioRequestContext,
resolveLmstudioRuntimeApiKey,
} from "./runtime.js";
export {
configureLmstudioNonInteractive,
discoverLmstudioProvider,
prepareLmstudioDynamicModels,
promptAndConfigureLmstudioInteractive,
} from "./setup.js";

View File

@@ -0,0 +1,14 @@
/** Shared LM Studio defaults used by setup, runtime discovery, and embeddings paths. */
export const LMSTUDIO_DEFAULT_BASE_URL = "http://localhost:1234";
export const LMSTUDIO_DEFAULT_INFERENCE_BASE_URL = `${LMSTUDIO_DEFAULT_BASE_URL}/v1`;
export const LMSTUDIO_DOCKER_HOST_BASE_URL = "http://host.docker.internal:1234";
export const LMSTUDIO_DOCKER_HOST_INFERENCE_BASE_URL = `${LMSTUDIO_DOCKER_HOST_BASE_URL}/v1`;
export const LMSTUDIO_DEFAULT_EMBEDDING_MODEL = "text-embedding-nomic-embed-text-v1.5";
export const LMSTUDIO_PROVIDER_LABEL = "LM Studio";
export const LMSTUDIO_DEFAULT_API_KEY_ENV_VAR = "LM_API_TOKEN";
export const LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER = "lmstudio-local";
export const LMSTUDIO_MODEL_PLACEHOLDER = "model-key-from-api-v1-models";
// Default context length sent when requesting LM Studio to load a model.
export const LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH = 64000;
export const LMSTUDIO_DEFAULT_MODEL_ID = "qwen/qwen3.5-9b";
export const LMSTUDIO_PROVIDER_ID = "lmstudio";

View File

@@ -0,0 +1,148 @@
// Lmstudio provider module implements model/runtime integration.
import { createSubsystemLogger } from "openclaw/plugin-sdk/logging-core";
import {
buildRemoteBaseUrlPolicy,
createRemoteEmbeddingProvider,
normalizeEmbeddingModelWithPrefixes,
type MemoryEmbeddingProvider,
type MemoryEmbeddingProviderCreateOptions,
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
import { resolveMemorySecretInputString } from "openclaw/plugin-sdk/memory-core-host-secret";
import { formatErrorMessage, type SsrFPolicy } from "openclaw/plugin-sdk/ssrf-runtime";
import { LMSTUDIO_DEFAULT_EMBEDDING_MODEL, LMSTUDIO_PROVIDER_ID } from "./defaults.js";
import { ensureLmstudioModelLoaded } from "./models.fetch.js";
import { resolveLmstudioInferenceBase } from "./models.js";
import {
buildLmstudioAuthHeaders,
resolveLmstudioProviderHeaders,
resolveLmstudioRuntimeApiKey,
} from "./runtime.js";
const log = createSubsystemLogger("memory/embeddings");
type LmstudioEmbeddingClient = {
baseUrl: string;
headers: Record<string, string>;
ssrfPolicy?: SsrFPolicy;
model: string;
};
export const DEFAULT_LMSTUDIO_EMBEDDING_MODEL = LMSTUDIO_DEFAULT_EMBEDDING_MODEL;
/** Normalizes LM Studio embedding model refs and accepts `lmstudio/` prefix. */
function normalizeLmstudioModel(model: string): string {
return normalizeEmbeddingModelWithPrefixes({
model,
defaultModel: DEFAULT_LMSTUDIO_EMBEDDING_MODEL,
prefixes: ["lmstudio/"],
});
}
function hasAuthorizationHeader(headers: Record<string, string> | undefined): boolean {
if (!headers) {
return false;
}
return Object.entries(headers).some(
([headerName, value]) =>
headerName.trim().toLowerCase() === "authorization" && value.trim().length > 0,
);
}
/** Resolves API key (real or synthetic placeholder) from runtime/provider auth config. */
async function resolveLmstudioApiKey(
options: MemoryEmbeddingProviderCreateOptions,
): Promise<string | undefined> {
try {
return await resolveLmstudioRuntimeApiKey({
config: options.config,
agentDir: options.agentDir,
});
} catch (error) {
// Embeddings can target local LM Studio instances that do not require auth.
if (/LM Studio API key is required/i.test(formatErrorMessage(error))) {
return undefined;
}
throw error;
}
}
/** Creates the LM Studio embedding provider client and preloads the target model before return. */
export async function createLmstudioEmbeddingProvider(
options: MemoryEmbeddingProviderCreateOptions,
): Promise<{ provider: MemoryEmbeddingProvider; client: LmstudioEmbeddingClient }> {
const providerConfig = options.config.models?.providers?.lmstudio;
const providerBaseUrl = providerConfig?.baseUrl?.trim();
const isFallbackActivation = options.fallback === "lmstudio" && options.provider !== "lmstudio";
const remoteBaseUrl = options.remote?.baseUrl?.trim();
const remoteApiKey = !isFallbackActivation
? resolveMemorySecretInputString({
value: options.remote?.apiKey,
path: "agents.*.memorySearch.remote.apiKey",
})
: undefined;
// memorySearch.remote is shared across primary + fallback providers.
// Ignore it during fallback activation to avoid inheriting another provider's
// endpoint/headers/credentials when LM Studio activates as a fallback.
const baseUrlSource = !isFallbackActivation ? remoteBaseUrl : undefined;
const configuredBaseUrl =
baseUrlSource && baseUrlSource.length > 0
? baseUrlSource
: providerBaseUrl && providerBaseUrl.length > 0
? providerBaseUrl
: undefined;
const baseUrl = resolveLmstudioInferenceBase(configuredBaseUrl);
const model = normalizeLmstudioModel(options.model);
const providerHeaders = await resolveLmstudioProviderHeaders({
config: options.config,
env: process.env,
headers: Object.assign(
{},
providerConfig?.headers,
!isFallbackActivation ? options.remote?.headers : {},
),
});
const apiKey = hasAuthorizationHeader(providerHeaders)
? undefined
: !isFallbackActivation
? remoteApiKey?.trim() || (await resolveLmstudioApiKey(options))
: await resolveLmstudioApiKey(options);
const headerOverrides = Object.assign({}, providerHeaders);
const headers =
buildLmstudioAuthHeaders({
apiKey,
json: true,
headers: headerOverrides,
}) ?? {};
const ssrfPolicy = buildRemoteBaseUrlPolicy(baseUrl);
const client: LmstudioEmbeddingClient = {
baseUrl,
model,
headers,
ssrfPolicy,
};
try {
await ensureLmstudioModelLoaded({
baseUrl,
apiKey,
headers: headerOverrides,
ssrfPolicy,
modelKey: model,
timeoutMs: 120_000,
});
} catch (error) {
log.warn("lmstudio embeddings warmup failed; continuing without preload", {
baseUrl,
model,
error: formatErrorMessage(error),
});
}
return {
provider: createRemoteEmbeddingProvider({
id: LMSTUDIO_PROVIDER_ID,
client,
errorPrefix: "lmstudio embeddings failed",
}),
client,
};
}

View File

@@ -0,0 +1,306 @@
// Lmstudio plugin module implements models.fetch behavior.
import { createSubsystemLogger } from "openclaw/plugin-sdk/logging-core";
import { resolveTimerTimeoutMs } from "openclaw/plugin-sdk/number-runtime";
import {
readProviderJsonArrayFieldResponse,
readProviderJsonResponse,
readResponseTextLimited,
} from "openclaw/plugin-sdk/provider-http";
import type { ModelDefinitionConfig } from "openclaw/plugin-sdk/provider-model-shared";
import { SELF_HOSTED_DEFAULT_COST } from "openclaw/plugin-sdk/provider-setup";
import { fetchWithSsrFGuard, type SsrFPolicy } from "openclaw/plugin-sdk/ssrf-runtime";
import { asPositiveSafeInteger } from "openclaw/plugin-sdk/string-coerce-runtime";
import { LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH } from "./defaults.js";
import {
buildLmstudioModelName,
mapLmstudioWireEntry,
resolveLmstudioCanonicalModelKey,
resolveLmstudioServerBase,
resolveLoadedContextWindow,
type LmstudioModelWire,
} from "./models.js";
import { buildLmstudioAuthHeaders } from "./runtime.js";
const log = createSubsystemLogger("extensions/lmstudio/models");
const LMSTUDIO_ERROR_BODY_LIMIT_BYTES = 8 * 1024;
type LmstudioLoadResponse = {
status?: string;
};
type LmstudioResolvedModelKeyError = {
resolvedModelKey: string;
};
type FetchLmstudioModelsResult = {
reachable: boolean;
status?: number;
models: LmstudioModelWire[];
error?: unknown;
};
type DiscoverLmstudioModelsParams = {
baseUrl: string;
apiKey: string;
headers?: Record<string, string>;
quiet: boolean;
/** Injectable fetch implementation; defaults to the global fetch. */
fetchImpl?: typeof fetch;
};
async function fetchLmstudioEndpoint(params: {
url: string;
init?: RequestInit;
timeoutMs: number;
fetchImpl?: typeof fetch;
ssrfPolicy?: SsrFPolicy;
auditContext: string;
}): Promise<{ response: Response; release: () => Promise<void> }> {
const timeoutMs = resolveTimerTimeoutMs(params.timeoutMs, 1);
if (params.ssrfPolicy) {
return await fetchWithSsrFGuard({
url: params.url,
init: params.init,
timeoutMs,
fetchImpl: params.fetchImpl,
policy: params.ssrfPolicy,
auditContext: params.auditContext,
});
}
const fetchFn = params.fetchImpl ?? fetch;
return {
response: await fetchFn(params.url, {
...params.init,
signal: AbortSignal.timeout(timeoutMs),
}),
release: async () => {},
};
}
function asLmstudioModelWire(value: unknown): LmstudioModelWire {
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new Error("LM Studio model list: malformed JSON response");
}
return value as LmstudioModelWire;
}
function withResolvedLmstudioModelKey(
error: unknown,
resolvedModelKey: string,
): Error & LmstudioResolvedModelKeyError {
if (error instanceof Error) {
return Object.assign(error, { resolvedModelKey });
}
return Object.assign(new Error(String(error)), {
cause: error,
resolvedModelKey,
});
}
/** Fetches /api/v1/models and reports transport reachability separately from HTTP status. */
export async function fetchLmstudioModels(params: {
baseUrl?: string;
apiKey?: string;
headers?: Record<string, string>;
ssrfPolicy?: SsrFPolicy;
timeoutMs?: number;
/** Injectable fetch implementation; defaults to the global fetch. */
fetchImpl?: typeof fetch;
}): Promise<FetchLmstudioModelsResult> {
const baseUrl = resolveLmstudioServerBase(params.baseUrl);
const timeoutMs = params.timeoutMs ?? 5000;
try {
const { response, release } = await fetchLmstudioEndpoint({
url: `${baseUrl}/api/v1/models`,
init: {
headers: buildLmstudioAuthHeaders({
apiKey: params.apiKey,
headers: params.headers,
}),
},
timeoutMs,
fetchImpl: params.fetchImpl,
ssrfPolicy: params.ssrfPolicy,
auditContext: "lmstudio-model-discovery",
});
try {
if (!response.ok) {
return {
reachable: true,
status: response.status,
models: [],
};
}
const models = await readProviderJsonArrayFieldResponse(
response,
"LM Studio model list",
"models",
);
return {
reachable: true,
status: response.status,
models: models.map(asLmstudioModelWire),
};
} finally {
await release();
}
} catch (error) {
return {
reachable: false,
models: [],
error,
};
}
}
/** Discovers LLM models from LM Studio and maps them to OpenClaw model definitions. */
export async function discoverLmstudioModels(
params: DiscoverLmstudioModelsParams,
): Promise<ModelDefinitionConfig[]> {
const fetched = await fetchLmstudioModels({
baseUrl: params.baseUrl,
apiKey: params.apiKey,
headers: params.headers,
fetchImpl: params.fetchImpl,
});
const quiet = params.quiet;
if (!fetched.reachable) {
if (!quiet) {
log.debug(`Failed to discover LM Studio models: ${String(fetched.error)}`);
}
return [];
}
if (fetched.status !== undefined && fetched.status >= 400) {
if (!quiet) {
log.debug(`Failed to discover LM Studio models: ${fetched.status}`);
}
return [];
}
const models = fetched.models;
if (models.length === 0) {
if (!quiet) {
log.debug("No LM Studio models found on local instance");
}
return [];
}
return models
.map((entry): ModelDefinitionConfig | null => {
const base = mapLmstudioWireEntry(entry);
if (!base) {
return null;
}
return {
id: base.id,
// Runtime display: include format/vision/tool-use/loaded tags in the name.
name: buildLmstudioModelName(base),
reasoning: base.reasoning,
input: base.input,
cost: SELF_HOSTED_DEFAULT_COST,
compat: { ...base.compat, supportsUsageInStreaming: true },
contextWindow: base.contextWindow,
contextTokens: base.contextTokens,
maxTokens: base.maxTokens,
};
})
.filter((entry): entry is ModelDefinitionConfig => entry !== null);
}
/** Ensures a model is loaded in LM Studio before first real inference/embedding call. */
export async function ensureLmstudioModelLoaded(params: {
baseUrl?: string;
apiKey?: string;
headers?: Record<string, string>;
ssrfPolicy?: SsrFPolicy;
modelKey: string;
requestedContextLength?: number;
timeoutMs?: number;
/** Injectable fetch implementation; defaults to the global fetch. */
fetchImpl?: typeof fetch;
}): Promise<string> {
const modelKey = params.modelKey.trim();
if (!modelKey) {
throw new Error("LM Studio model key is required");
}
const timeoutMs = params.timeoutMs ?? 30_000;
const baseUrl = resolveLmstudioServerBase(params.baseUrl);
const preflight = await fetchLmstudioModels({
baseUrl,
apiKey: params.apiKey,
headers: params.headers,
ssrfPolicy: params.ssrfPolicy,
timeoutMs,
fetchImpl: params.fetchImpl,
});
if (!preflight.reachable) {
throw new Error(`LM Studio model discovery failed: ${String(preflight.error)}`);
}
if (preflight.status !== undefined && preflight.status >= 400) {
throw new Error(`LM Studio model discovery failed (${preflight.status})`);
}
const canonicalModelKey = resolveLmstudioCanonicalModelKey({
modelKey,
models: preflight.models,
});
const matchingModel = preflight.models.find((entry) => entry.key?.trim() === canonicalModelKey);
const loadedContextWindow = matchingModel ? resolveLoadedContextWindow(matchingModel) : null;
const advertisedContextLimit = asPositiveSafeInteger(matchingModel?.max_context_length) ?? null;
const requestedContextLength = asPositiveSafeInteger(params.requestedContextLength) ?? null;
const contextLengthForLoad =
advertisedContextLimit === null
? (requestedContextLength ?? LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH)
: Math.min(
requestedContextLength ?? LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
advertisedContextLimit,
);
if (loadedContextWindow !== null && loadedContextWindow >= contextLengthForLoad) {
return canonicalModelKey;
}
try {
const { response, release } = await fetchLmstudioEndpoint({
url: `${baseUrl}/api/v1/models/load`,
init: {
method: "POST",
headers: buildLmstudioAuthHeaders({
apiKey: params.apiKey,
headers: params.headers,
json: true,
}),
body: JSON.stringify({
model: canonicalModelKey,
// Ask LM Studio to load with our default target, capped to the model's own limit.
context_length: contextLengthForLoad,
}),
},
timeoutMs,
fetchImpl: params.fetchImpl,
ssrfPolicy: params.ssrfPolicy,
auditContext: "lmstudio-model-load",
});
try {
if (!response.ok) {
const body = await readResponseTextLimited(response, LMSTUDIO_ERROR_BODY_LIMIT_BYTES);
throw new Error(
`LM Studio model load failed (${response.status})${body ? `: ${body}` : ""}`,
);
}
// Read the success body through the shared byte-capped reader so a misbehaving
// or compromised LM Studio server cannot stream an unbounded JSON payload into
// memory before we parse it. Malformed JSON is wrapped with our own label.
const payload = await readProviderJsonResponse<LmstudioLoadResponse>(
response,
"LM Studio model load",
);
if (typeof payload.status === "string" && payload.status.toLowerCase() !== "loaded") {
throw new Error(`LM Studio model load returned unexpected status: ${payload.status}`);
}
} finally {
await release();
}
} catch (error) {
throw withResolvedLmstudioModelKey(error, canonicalModelKey);
}
return canonicalModelKey;
}

View File

@@ -0,0 +1,771 @@
// Lmstudio tests cover models plugin behavior.
import { MAX_TIMER_TIMEOUT_MS } from "openclaw/plugin-sdk/number-runtime";
import {
SELF_HOSTED_DEFAULT_CONTEXT_WINDOW,
SELF_HOSTED_DEFAULT_MAX_TOKENS,
} from "openclaw/plugin-sdk/provider-setup";
import { afterAll, afterEach, describe, expect, it, vi } from "vitest";
import { LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH } from "./defaults.js";
import {
discoverLmstudioModels,
ensureLmstudioModelLoaded,
fetchLmstudioModels,
} from "./models.fetch.js";
import {
mapLmstudioWireEntry,
normalizeLmstudioConfiguredCatalogEntry,
normalizeLmstudioProviderConfig,
resolveLmstudioInferenceBase,
resolveLmstudioReasoningCompat,
resolveLmstudioReasoningCapability,
resolveLmstudioServerBase,
} from "./models.js";
const fetchWithSsrFGuardMock = vi.hoisted(() => vi.fn());
vi.mock("openclaw/plugin-sdk/ssrf-runtime", async (importOriginal) => {
const actual = await importOriginal<typeof import("openclaw/plugin-sdk/ssrf-runtime")>();
return {
...actual,
fetchWithSsrFGuard: (...args: unknown[]) => fetchWithSsrFGuardMock(...args),
};
});
function jsonResponse(payload: unknown, init?: ResponseInit): Response {
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
...init,
});
}
function malformedJsonResponse(): Response {
return new Response("{ nope", {
status: 200,
headers: { "content-type": "application/json" },
});
}
afterAll(() => {
vi.doUnmock("openclaw/plugin-sdk/ssrf-runtime");
vi.resetModules();
});
describe("lmstudio-models", () => {
const asFetch = (mock: unknown) => mock as typeof fetch;
const parseJsonRequestBody = (init: RequestInit | undefined): unknown => {
if (typeof init?.body !== "string") {
throw new Error("Expected request body to be a JSON string");
}
return JSON.parse(init.body) as unknown;
};
const cancelTrackedResponse = (
text: string,
init: ResponseInit,
): {
response: Response;
wasCanceled: () => boolean;
} => {
let canceled = false;
const stream = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(new TextEncoder().encode(text));
},
cancel() {
canceled = true;
},
});
return {
response: new Response(stream, init),
wasCanceled: () => canceled,
};
};
const createModelLoadFetchMock = (params?: {
key?: string;
variants?: unknown;
selectedVariant?: unknown;
loadedContextLength?: number;
maxContextLength?: number;
}) =>
vi.fn(async (url: string | URL, _init?: RequestInit) => {
const key = params?.key ?? "qwen3-8b-instruct";
if (String(url).endsWith("/api/v1/models")) {
return jsonResponse({
models: [
{
type: "llm",
key,
max_context_length: params?.maxContextLength,
variants: params?.variants,
selected_variant: params?.selectedVariant,
loaded_instances: params?.loadedContextLength
? [{ id: "inst-1", config: { context_length: params.loadedContextLength } }]
: [],
},
],
});
}
if (String(url).endsWith("/api/v1/models/load")) {
return jsonResponse({ status: "loaded" });
}
throw new Error(`Unexpected fetch URL: ${String(url)}`);
});
const findModelLoadCall = (fetchMock: ReturnType<typeof createModelLoadFetchMock>) =>
fetchMock.mock.calls.find((call) => String(call[0]).endsWith("/models/load"));
const expectLoadContextLength = (
fetchMock: ReturnType<typeof createModelLoadFetchMock>,
contextLength: number,
) => {
const loadCall = findModelLoadCall(fetchMock);
if (!loadCall) {
throw new Error("expected LM Studio model load request");
}
const loadInit = loadCall[1] as RequestInit;
const loadBody = parseJsonRequestBody(loadInit) as { context_length: number };
expect(loadBody.context_length).toBe(contextLength);
};
const expectLoadModelKey = (
fetchMock: ReturnType<typeof createModelLoadFetchMock>,
modelKey: string,
) => {
const loadCall = findModelLoadCall(fetchMock);
if (!loadCall) {
throw new Error("expected LM Studio model load request");
}
const loadInit = loadCall[1] as RequestInit;
const loadBody = parseJsonRequestBody(loadInit) as { model: string };
expect(loadBody.model).toBe(modelKey);
};
afterEach(() => {
fetchWithSsrFGuardMock.mockReset();
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
it("normalizes LM Studio base URLs", () => {
expect(resolveLmstudioServerBase()).toBe("http://localhost:1234");
expect(resolveLmstudioInferenceBase()).toBe("http://localhost:1234/v1");
expect(resolveLmstudioServerBase("http://localhost:1234/api/v1")).toBe("http://localhost:1234");
expect(resolveLmstudioInferenceBase("http://localhost:1234/api/v1")).toBe(
"http://localhost:1234/v1",
);
expect(resolveLmstudioServerBase("localhost:1234/api/v1")).toBe("http://localhost:1234");
expect(resolveLmstudioInferenceBase("localhost:1234/api/v1")).toBe("http://localhost:1234/v1");
});
it("marks configured LM Studio endpoints as trusted private-network model targets", () => {
expect(
normalizeLmstudioProviderConfig({
baseUrl: "http://192.168.1.10:1234",
models: [],
}),
).toEqual({
baseUrl: "http://192.168.1.10:1234/v1",
request: { allowPrivateNetwork: true },
models: [],
});
expect(
normalizeLmstudioProviderConfig({
baseUrl: "http://gpu-box.local:1234/v1",
request: {
allowPrivateNetwork: false,
headers: { "X-Proxy-Auth": "token" },
},
models: [],
}),
).toEqual({
baseUrl: "http://gpu-box.local:1234/v1",
request: {
allowPrivateNetwork: false,
headers: { "X-Proxy-Auth": "token" },
},
models: [],
});
});
it("drops malformed configured catalog token metadata", () => {
expect(
normalizeLmstudioConfiguredCatalogEntry({
id: "bad-window",
contextWindow: Number.POSITIVE_INFINITY,
contextTokens: 4096.5,
}),
).toMatchObject({
id: "bad-window",
contextWindow: undefined,
contextTokens: undefined,
});
expect(
normalizeLmstudioConfiguredCatalogEntry({
id: "bad-tokens",
contextWindow: -1,
contextTokens: 0,
}),
).toMatchObject({
id: "bad-tokens",
contextWindow: undefined,
contextTokens: undefined,
});
});
it("drops malformed discovered context metadata", () => {
const model = mapLmstudioWireEntry({
type: "llm",
key: "bad-context",
max_context_length: 32768.5,
loaded_instances: [{ id: "loaded", config: { context_length: Number.POSITIVE_INFINITY } }],
});
expect(model).toMatchObject({
id: "bad-context",
contextWindow: SELF_HOSTED_DEFAULT_CONTEXT_WINDOW,
contextTokens: LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
maxTokens: SELF_HOSTED_DEFAULT_MAX_TOKENS,
loaded: false,
});
});
it("resolves reasoning capability for supported and unsupported options", () => {
expect(resolveLmstudioReasoningCapability({ capabilities: undefined })).toBe(false);
expect(
resolveLmstudioReasoningCapability({
capabilities: {
reasoning: {
allowed_options: ["low", "medium", "high"],
default: "low",
},
},
}),
).toBe(true);
expect(
resolveLmstudioReasoningCapability({
capabilities: {
reasoning: {
allowed_options: ["off"],
default: "off",
},
},
}),
).toBe(false);
});
it("maps LM Studio binary reasoning options into OpenAI-compatible effort compat", () => {
expect(
resolveLmstudioReasoningCompat({
capabilities: {
reasoning: {
allowed_options: ["off", "on"],
default: "on",
},
},
}),
).toEqual({
supportsReasoningEffort: true,
supportedReasoningEfforts: ["none", "minimal", "low", "medium", "high", "xhigh"],
reasoningEffortMap: {
off: "none",
none: "none",
adaptive: "xhigh",
max: "xhigh",
},
});
expect(
resolveLmstudioReasoningCompat({
capabilities: {
reasoning: {
allowed_options: ["low", "medium", "high"],
default: "low",
},
},
}),
).toEqual({
supportsReasoningEffort: true,
supportedReasoningEfforts: ["low", "medium", "high"],
reasoningEffortMap: {
adaptive: "high",
max: "high",
},
});
expect(
resolveLmstudioReasoningCompat({
capabilities: {
reasoning: {
allowed_options: ["off"],
default: "off",
},
},
}),
).toBeUndefined();
});
it("discovers llm models and maps metadata", async () => {
const fetchMock = vi.fn(async (_url: string | URL, _init?: RequestInit) =>
jsonResponse({
models: [
{
type: "llm",
key: "qwen3-8b-instruct",
display_name: "Qwen3 8B",
max_context_length: 262144,
format: "mlx",
capabilities: {
vision: true,
trained_for_tool_use: true,
reasoning: {
allowed_options: ["off", "on"],
default: "on",
},
},
loaded_instances: [{ id: "inst-1", config: { context_length: 64000 } }],
},
{
type: "llm",
key: "deepseek-r1",
},
{
type: "embedding",
key: "text-embedding-nomic-embed-text-v1.5",
},
{
type: "llm",
key: " ",
},
],
}),
);
const models = await discoverLmstudioModels({
baseUrl: "http://localhost:1234/v1",
apiKey: "lm-token",
quiet: false,
fetchImpl: asFetch(fetchMock),
});
const modelsRequest = fetchMock.mock.calls.find(
([url]) => url === "http://localhost:1234/api/v1/models",
);
const modelsRequestOptions = modelsRequest?.[1] as
| { headers?: Record<string, string>; signal?: unknown }
| undefined;
expect(modelsRequestOptions?.headers).toEqual({
Authorization: "Bearer lm-token",
});
expect(modelsRequestOptions?.signal).toBeInstanceOf(AbortSignal);
expect(models).toHaveLength(2);
expect(models[0]).toEqual({
id: "qwen3-8b-instruct",
name: "Qwen3 8B (MLX, vision, tool-use, loaded)",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
compat: {
supportsUsageInStreaming: true,
supportsReasoningEffort: true,
supportedReasoningEfforts: ["none", "minimal", "low", "medium", "high", "xhigh"],
reasoningEffortMap: {
off: "none",
none: "none",
adaptive: "xhigh",
max: "xhigh",
},
},
contextWindow: 262144,
contextTokens: LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
maxTokens: SELF_HOSTED_DEFAULT_MAX_TOKENS,
});
expect(models[1]).toEqual({
id: "deepseek-r1",
name: "deepseek-r1",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
compat: { supportsUsageInStreaming: true },
contextWindow: SELF_HOSTED_DEFAULT_CONTEXT_WINDOW,
contextTokens: LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH,
maxTokens: SELF_HOSTED_DEFAULT_MAX_TOKENS,
});
});
it("reports malformed model list JSON with an owned error", async () => {
const fetchMock = vi.fn(async () => malformedJsonResponse());
const result = await fetchLmstudioModels({
baseUrl: "http://localhost:1234/v1",
fetchImpl: asFetch(fetchMock),
});
expect(result.reachable).toBe(false);
expect((result.error as Error).message).toBe("LM Studio model list: malformed JSON response");
});
it("reports wrong-shaped model list payloads with owned errors", async () => {
for (const payload of [[], { models: {} }, { models: [null] }]) {
const fetchMock = vi.fn(async () => jsonResponse(payload));
const result = await fetchLmstudioModels({
baseUrl: "http://localhost:1234/v1",
fetchImpl: asFetch(fetchMock),
});
expect(result.reachable).toBe(false);
expect((result.error as Error).message).toBe("LM Studio model list: malformed JSON response");
}
});
it("caps oversized direct fetch timeouts before discovering models", async () => {
const timeoutController = new AbortController();
const timeoutSpy = vi.spyOn(AbortSignal, "timeout").mockReturnValue(timeoutController.signal);
const fetchMock = vi.fn(async (_url: string | URL, _init?: RequestInit) =>
jsonResponse({ models: [] }),
);
const result = await fetchLmstudioModels({
baseUrl: "http://localhost:1234/v1",
timeoutMs: Number.MAX_SAFE_INTEGER,
fetchImpl: asFetch(fetchMock),
});
expect(result.reachable).toBe(true);
expect(timeoutSpy).toHaveBeenCalledWith(MAX_TIMER_TIMEOUT_MS);
expect(fetchMock.mock.calls[0]?.[1]?.signal).toBe(timeoutController.signal);
});
it("caps oversized guarded-fetch timeouts before discovering models", async () => {
fetchWithSsrFGuardMock.mockResolvedValue({
response: new Response(JSON.stringify({ models: [] }), { status: 200 }),
release: vi.fn(async () => undefined),
});
const result = await fetchLmstudioModels({
baseUrl: "http://localhost:1234/v1",
timeoutMs: Number.MAX_SAFE_INTEGER,
ssrfPolicy: {},
});
expect(result.reachable).toBe(true);
expect(fetchWithSsrFGuardMock.mock.calls[0]?.[0]).toMatchObject({
timeoutMs: MAX_TIMER_TIMEOUT_MS,
});
});
it("skips model load when already loaded", async () => {
const fetchMock = createModelLoadFetchMock({ loadedContextLength: 64000 });
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
}),
).resolves.toBe("qwen3-8b-instruct");
expect(fetchMock).toHaveBeenCalledTimes(1);
const calledUrls = fetchMock.mock.calls.map((call) => String(call[0]));
expect(calledUrls).not.toContain("http://localhost:1234/api/v1/models/load");
});
it("reloads model when requested context length exceeds the loaded window", async () => {
const fetchMock = createModelLoadFetchMock({
loadedContextLength: 4096,
maxContextLength: 32768,
});
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
requestedContextLength: 8192,
}),
).resolves.toBe("qwen3-8b-instruct");
expect(fetchMock).toHaveBeenCalledTimes(2);
expectLoadContextLength(fetchMock, 8192);
});
it("loads the canonical model key when the requested key is an advertised variant", async () => {
const canonicalKey = "gemma-4-e4b-it-ultra-uncensored-heretic";
const variantKey = `${canonicalKey}@q4_k_m`;
const fetchMock = createModelLoadFetchMock({
key: canonicalKey,
variants: [variantKey],
selectedVariant: variantKey,
});
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: variantKey,
}),
).resolves.toBe(canonicalKey);
expect(fetchMock).toHaveBeenCalledTimes(2);
expectLoadModelKey(fetchMock, canonicalKey);
});
it("keeps the canonical model key on load failures after variant discovery", async () => {
const canonicalKey = "gemma-4-e4b-it-ultra-uncensored-heretic";
const variantKey = `${canonicalKey}@q4_k_m`;
const fetchMock = vi.fn(async (url: string | URL) => {
if (String(url).endsWith("/api/v1/models")) {
return jsonResponse({
models: [
{
type: "llm",
key: canonicalKey,
variants: [variantKey],
selected_variant: variantKey,
loaded_instances: [],
},
],
});
}
if (String(url).endsWith("/api/v1/models/load")) {
return new Response("load failed", { status: 503 });
}
throw new Error(`Unexpected fetch URL: ${String(url)}`);
});
vi.stubGlobal("fetch", asFetch(fetchMock));
const error = await ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: variantKey,
}).catch((caught: unknown) => caught);
expect(error).toBeInstanceOf(Error);
expect(error).toMatchObject({ resolvedModelKey: canonicalKey });
});
it("preserves a suffixed key when LM Studio advertises it as the model key", async () => {
const suffixedKey = "local/special-model@q4_k_m";
const fetchMock = createModelLoadFetchMock({
key: suffixedKey,
variants: ["local/special-model@q8_0"],
selectedVariant: "local/special-model@q8_0",
});
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: suffixedKey,
}),
).resolves.toBe(suffixedKey);
expect(fetchMock).toHaveBeenCalledTimes(2);
expectLoadModelKey(fetchMock, suffixedKey);
});
it("reports malformed model load JSON with an owned error", async () => {
const fetchMock = vi.fn(async (url: string | URL) => {
if (String(url).endsWith("/api/v1/models")) {
return jsonResponse({
models: [{ type: "llm", key: "qwen3-8b-instruct", loaded_instances: [] }],
});
}
if (String(url).endsWith("/api/v1/models/load")) {
return malformedJsonResponse();
}
throw new Error(`Unexpected fetch URL: ${String(url)}`);
});
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
}),
).rejects.toThrow("LM Studio model load: malformed JSON response");
});
it("bounds oversized model load success bodies", async () => {
// A misbehaving server may stream an unbounded success JSON body; the load
// path must stop reading at the byte cap instead of buffering it all.
let canceled = false;
let bytesEmitted = 0;
const oversizedStream = new ReadableStream<Uint8Array>({
pull(controller) {
// Far exceeds the 16 MiB provider JSON cap if read to completion.
if (bytesEmitted >= 32 * 1024 * 1024) {
controller.close();
return;
}
bytesEmitted += 64 * 1024;
controller.enqueue(new Uint8Array(64 * 1024).fill(0x61));
},
cancel() {
canceled = true;
},
});
const fetchMock = vi.fn(async (url: string | URL) => {
if (String(url).endsWith("/api/v1/models")) {
return jsonResponse({
models: [{ type: "llm", key: "qwen3-8b-instruct", loaded_instances: [] }],
});
}
if (String(url).endsWith("/api/v1/models/load")) {
return new Response(oversizedStream, {
status: 200,
headers: { "content-type": "application/json" },
});
}
throw new Error(`Unexpected fetch URL: ${String(url)}`);
});
vi.stubGlobal("fetch", asFetch(fetchMock));
const error = await ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
}).catch((caught: unknown) => caught);
expect(error).toBeInstanceOf(Error);
expect((error as Error).message).toMatch(/JSON response exceeds \d+ bytes/);
expect(canceled).toBe(true);
expect(bytesEmitted).toBeLessThan(32 * 1024 * 1024);
});
it("bounds model load error bodies", async () => {
const body = `${"lmstudio load unavailable ".repeat(512)}tail`;
const tracked = cancelTrackedResponse(body, { status: 503 });
const textSpy = vi.spyOn(tracked.response, "text").mockRejectedValue(new Error("unbounded"));
const fetchMock = vi.fn(async (url: string | URL) => {
if (String(url).endsWith("/api/v1/models")) {
return jsonResponse({
models: [{ type: "llm", key: "qwen3-8b-instruct", loaded_instances: [] }],
});
}
if (String(url).endsWith("/api/v1/models/load")) {
return tracked.response;
}
throw new Error(`Unexpected fetch URL: ${String(url)}`);
});
vi.stubGlobal("fetch", asFetch(fetchMock));
const error = await ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
}).catch((caught: unknown) => caught);
expect(error).toBeInstanceOf(Error);
expect((error as Error).message).toMatch(
/LM Studio model load failed \(503\): lmstudio load unavailable/,
);
expect((error as Error).message).not.toContain("tail");
expect(tracked.wasCanceled()).toBe(true);
expect(textSpy).not.toHaveBeenCalled();
});
it("reloads model to the clamped default target when already loaded below the default window", async () => {
const fetchMock = createModelLoadFetchMock({
loadedContextLength: 4096,
maxContextLength: 32768,
});
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
}),
).resolves.toBe("qwen3-8b-instruct");
expect(fetchMock).toHaveBeenCalledTimes(2);
expectLoadContextLength(fetchMock, 32768);
});
it("loads model with clamped context length and merged headers", async () => {
const fetchMock = createModelLoadFetchMock({ maxContextLength: 32768 });
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
apiKey: "lm-token",
headers: {
"X-Proxy-Auth": "required",
Authorization: "Bearer override",
},
modelKey: " qwen3-8b-instruct ",
}),
).resolves.toBe("qwen3-8b-instruct");
expect(fetchMock).toHaveBeenCalledTimes(2);
const loadCall = findModelLoadCall(fetchMock);
if (!loadCall) {
throw new Error("expected LM Studio model load request");
}
const loadInit = loadCall[1] as RequestInit;
const { signal, ...stableLoadInit } = loadInit;
expect(signal).toBeInstanceOf(AbortSignal);
expect(stableLoadInit).toEqual({
method: "POST",
headers: {
"X-Proxy-Auth": "required",
Authorization: "Bearer lm-token",
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "qwen3-8b-instruct",
context_length: 32768,
}),
});
const loadBody = parseJsonRequestBody(loadInit) as { context_length: number };
expect(loadBody.context_length).not.toBe(LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH);
});
it("uses requested context length when provided for model load", async () => {
const fetchMock = createModelLoadFetchMock({ maxContextLength: 32768 });
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
requestedContextLength: 8192,
}),
).resolves.toBe("qwen3-8b-instruct");
expectLoadContextLength(fetchMock, 8192);
});
it("omits malformed context lengths before loading models", async () => {
const fetchMock = createModelLoadFetchMock({
loadedContextLength: 4096.5,
maxContextLength: 32768.5,
});
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
requestedContextLength: 8192.5,
}),
).resolves.toBe("qwen3-8b-instruct");
expectLoadContextLength(fetchMock, LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH);
});
it("throws when model discovery fails", async () => {
const fetchMock = vi.fn(async () => ({
ok: false,
status: 401,
}));
vi.stubGlobal("fetch", asFetch(fetchMock));
await expect(
ensureLmstudioModelLoaded({
baseUrl: "http://localhost:1234/v1",
modelKey: "qwen3-8b-instruct",
}),
).rejects.toThrow("LM Studio model discovery failed (401)");
expect(fetchMock).toHaveBeenCalledTimes(1);
});
});

View File

@@ -0,0 +1,570 @@
// Lmstudio plugin module implements models behavior.
import type {
ModelDefinitionConfig,
ModelProviderConfig,
} from "openclaw/plugin-sdk/provider-model-shared";
import {
SELF_HOSTED_DEFAULT_CONTEXT_WINDOW,
SELF_HOSTED_DEFAULT_COST,
SELF_HOSTED_DEFAULT_MAX_TOKENS,
} from "openclaw/plugin-sdk/provider-setup";
import { asPositiveSafeInteger, uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { LMSTUDIO_DEFAULT_BASE_URL, LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH } from "./defaults.js";
export type LmstudioModelWire = {
type?: "llm" | "embedding";
key?: string;
display_name?: string;
max_context_length?: number;
format?: "gguf" | "mlx" | null;
variants?: unknown;
selected_variant?: unknown;
capabilities?: {
vision?: boolean;
trained_for_tool_use?: boolean;
reasoning?: LmstudioReasoningCapabilityWire;
};
loaded_instances?: Array<{
id?: string;
config?: {
context_length?: number;
} | null;
} | null>;
};
type LmstudioReasoningCapabilityWire = {
allowed_options?: unknown;
default?: unknown;
};
type LmstudioConfiguredCatalogEntry = {
id: string;
name?: string;
contextWindow?: number;
contextTokens?: number;
reasoning?: boolean;
input?: ("text" | "image" | "document")[];
compat?: ModelDefinitionConfig["compat"];
};
const LMSTUDIO_OPENAI_COMPAT_ENABLED_REASONING_EFFORTS = [
"minimal",
"low",
"medium",
"high",
"xhigh",
] as const;
const LMSTUDIO_OPENAI_COMPAT_REASONING_EFFORTS = [
"none",
...LMSTUDIO_OPENAI_COMPAT_ENABLED_REASONING_EFFORTS,
] as const;
function normalizeReasoningOption(value: unknown): string | null {
if (typeof value !== "string") {
return null;
}
const normalized = value.trim().toLowerCase();
return normalized.length > 0 ? normalized : null;
}
function isReasoningEnabledOption(value: unknown): boolean {
const normalized = normalizeReasoningOption(value);
if (!normalized) {
return false;
}
return normalized !== "off";
}
function normalizeReasoningOptions(value: unknown): string[] {
if (!Array.isArray(value)) {
return [];
}
return uniqueStrings(value.flatMap((option) => normalizeReasoningOption(option) ?? []));
}
function isLmstudioBinaryReasoningOptions(allowedOptions: readonly string[]): boolean {
return (
allowedOptions.some((option) => option === "on") &&
allowedOptions.every((option) => option === "on" || option === "off")
);
}
function resolveLmstudioTransportReasoningEfforts(allowedOptions: readonly string[]): string[] {
if (isLmstudioBinaryReasoningOptions(allowedOptions)) {
return allowedOptions.includes("off")
? [...LMSTUDIO_OPENAI_COMPAT_REASONING_EFFORTS]
: [...LMSTUDIO_OPENAI_COMPAT_ENABLED_REASONING_EFFORTS];
}
return uniqueStrings(
allowedOptions
.map((option) => (option === "off" ? "none" : option))
.filter((option) => option !== "on"),
);
}
function resolveLmstudioEnabledTransportReasoningOption(
supportedReasoningEfforts: readonly string[],
): string | undefined {
return (
supportedReasoningEfforts.find((option) => option === "xhigh") ??
supportedReasoningEfforts.find((option) => option === "high") ??
supportedReasoningEfforts.find((option) => option !== "none")
);
}
function buildLmstudioReasoningEffortMap(
supportedReasoningEfforts: readonly string[],
): Record<string, string> | undefined {
const disabled = supportedReasoningEfforts.includes("none") ? "none" : undefined;
const max = resolveLmstudioEnabledTransportReasoningOption(supportedReasoningEfforts);
const map = {
...(disabled ? { off: disabled, none: disabled } : {}),
...(max ? { adaptive: max, max } : {}),
};
return Object.keys(map).length > 0 ? map : undefined;
}
function buildLmstudioReasoningCompat(
allowedOptions: readonly string[],
): ModelDefinitionConfig["compat"] | undefined {
const supportedReasoningEfforts = resolveLmstudioTransportReasoningEfforts(allowedOptions);
if (supportedReasoningEfforts.length === 0) {
return undefined;
}
if (!supportedReasoningEfforts.some((option) => option !== "none")) {
return undefined;
}
return {
supportsReasoningEffort: true,
supportedReasoningEfforts,
reasoningEffortMap: buildLmstudioReasoningEffortMap(supportedReasoningEfforts),
};
}
function normalizeLmstudioTransportReasoningCompat(
compat: NonNullable<ModelDefinitionConfig["compat"]>,
): NonNullable<ModelDefinitionConfig["compat"]> {
const supportedReasoningEfforts = compat.supportedReasoningEfforts;
const map = compat.reasoningEffortMap;
const hasBinarySupported =
Array.isArray(supportedReasoningEfforts) &&
supportedReasoningEfforts.some((option) => option === "on");
const hasBinaryMapValue =
map !== undefined && Object.values(map).some((value) => value === "on" || value === "off");
if (!hasBinarySupported && !hasBinaryMapValue) {
return compat;
}
const hasDisabled =
supportedReasoningEfforts?.includes("off") === true ||
supportedReasoningEfforts?.includes("none") === true ||
Object.values(map ?? {}).some((value) => value === "off" || value === "none");
const normalizedSupportedReasoningEfforts = hasDisabled
? [...LMSTUDIO_OPENAI_COMPAT_REASONING_EFFORTS]
: [...LMSTUDIO_OPENAI_COMPAT_ENABLED_REASONING_EFFORTS];
return {
...compat,
supportedReasoningEfforts: normalizedSupportedReasoningEfforts,
reasoningEffortMap: buildLmstudioReasoningEffortMap(normalizedSupportedReasoningEfforts),
};
}
export function resolveLmstudioReasoningCompat(
entry: Pick<LmstudioModelWire, "capabilities">,
): ModelDefinitionConfig["compat"] | undefined {
const reasoning = entry.capabilities?.reasoning;
if (reasoning === undefined || reasoning === null) {
return undefined;
}
const allowedOptions = normalizeReasoningOptions(reasoning.allowed_options);
if (allowedOptions.length === 0) {
return undefined;
}
return buildLmstudioReasoningCompat(allowedOptions);
}
/**
* Resolves LM Studio reasoning support from capabilities payloads.
* Defaults to false when the server omits reasoning metadata.
*/
export function resolveLmstudioReasoningCapability(
entry: Pick<LmstudioModelWire, "capabilities">,
): boolean {
const reasoning = entry.capabilities?.reasoning;
if (reasoning === undefined || reasoning === null) {
return false;
}
const allowedOptions = normalizeReasoningOptions(reasoning.allowed_options);
if (allowedOptions.length > 0) {
return allowedOptions.some((option) => isReasoningEnabledOption(option));
}
return isReasoningEnabledOption(reasoning.default);
}
/**
* Reads loaded LM Studio instances and returns the largest valid context window.
* Returns null when no usable loaded context is present.
*/
export function resolveLoadedContextWindow(
entry: Pick<LmstudioModelWire, "loaded_instances">,
): number | null {
const loadedInstances = Array.isArray(entry.loaded_instances) ? entry.loaded_instances : [];
let contextWindow: number | null = null;
for (const instance of loadedInstances) {
// Discovery payload is external JSON, so tolerate malformed entries.
const normalized = asPositiveSafeInteger(instance?.config?.context_length);
if (normalized === undefined) {
continue;
}
contextWindow = contextWindow === null ? normalized : Math.max(contextWindow, normalized);
}
return contextWindow;
}
function normalizeLmstudioVariantIds(value: unknown): string[] {
if (!Array.isArray(value)) {
return [];
}
return uniqueStrings(
value.flatMap((variant) =>
typeof variant === "string" && variant.trim().length > 0 ? variant.trim() : [],
),
);
}
/**
* Resolves LM Studio variant ids back to their loadable model key.
*
* LM Studio exposes quantized variants separately from the canonical `key`, but
* `/api/v1/models/load` expects the key. Exact key matches still win so unusual
* servers that expose a suffix as the real key are preserved.
*/
export function resolveLmstudioCanonicalModelKey(params: {
modelKey: string;
models: LmstudioModelWire[];
}): string {
const modelKey = params.modelKey.trim();
if (!modelKey) {
return modelKey;
}
const normalizedModelKey = modelKey.toLowerCase();
for (const entry of params.models) {
if (entry.key?.trim() === modelKey) {
return modelKey;
}
}
for (const entry of params.models) {
const key = entry.key?.trim();
if (!key) {
continue;
}
const selectedVariant =
typeof entry.selected_variant === "string" ? entry.selected_variant.trim() : "";
const variants = normalizeLmstudioVariantIds(entry.variants);
if (
selectedVariant.toLowerCase() === normalizedModelKey ||
variants.some((variant) => variant.toLowerCase() === normalizedModelKey)
) {
return key;
}
}
return modelKey;
}
/**
* Normalizes a server path by stripping trailing slash and inference suffixes.
*
* LM Studio users often copy their inference URL (e.g. "http://localhost:1234/v1") instead
* of the server root. This function strips a trailing "/v1" or "/api/v1" so the caller always
* receives a clean root base URL. The expected input is the server root without any API version
* path (e.g. "http://localhost:1234").
*/
function normalizeUrlPath(pathname: string): string {
const trimmed = pathname.replace(/\/+$/, "");
if (!trimmed) {
return "";
}
return trimmed.replace(/\/api\/v1$/i, "").replace(/\/v1$/i, "");
}
function hasExplicitHttpScheme(value: string): boolean {
return /^https?:\/\//i.test(value);
}
function isLikelyHostBaseUrl(value: string): boolean {
return (
/^(?:localhost|(?:\d{1,3}\.){3}\d{1,3}|[a-z0-9.-]+\.[a-z]{2,}|[^/\s?#]+:\d+)(?:[/?#].*)?$/i.test(
value,
) && !value.startsWith("/")
);
}
function normalizeConfiguredReasoningEffortMap(value: unknown): Record<string, string> | undefined {
if (!value || typeof value !== "object" || Array.isArray(value)) {
return undefined;
}
const normalized = Object.fromEntries(
Object.entries(value)
.map(([key, mapped]) => [key.trim(), typeof mapped === "string" ? mapped.trim() : ""])
.filter(([key, mapped]) => key.length > 0 && mapped.length > 0),
);
return Object.keys(normalized).length > 0 ? normalized : undefined;
}
function normalizeLmstudioConfiguredCompat(value: unknown): ModelDefinitionConfig["compat"] {
if (!value || typeof value !== "object" || Array.isArray(value)) {
return undefined;
}
const record = value as Record<string, unknown>;
const supportedReasoningEfforts = normalizeReasoningOptions(record.supportedReasoningEfforts);
const reasoningEffortMap = normalizeConfiguredReasoningEffortMap(record.reasoningEffortMap);
const compat: NonNullable<ModelDefinitionConfig["compat"]> = {};
if (typeof record.supportsUsageInStreaming === "boolean") {
compat.supportsUsageInStreaming = record.supportsUsageInStreaming;
}
if (typeof record.supportsReasoningEffort === "boolean") {
compat.supportsReasoningEffort = record.supportsReasoningEffort;
}
if (supportedReasoningEfforts.length > 0) {
compat.supportedReasoningEfforts = supportedReasoningEfforts;
}
if (reasoningEffortMap) {
compat.reasoningEffortMap = reasoningEffortMap;
}
return Object.keys(compat).length > 0
? normalizeLmstudioTransportReasoningCompat(compat)
: undefined;
}
function toFetchableLmstudioBaseUrl(value: string): string {
if (hasExplicitHttpScheme(value) || !isLikelyHostBaseUrl(value)) {
return value;
}
return `http://${value}`;
}
/** Resolves LM Studio server base URL (without /v1 or /api/v1). */
export function resolveLmstudioServerBase(configuredBaseUrl?: string): string {
// Use configured value when present; otherwise target local LM Studio default.
const configured = configuredBaseUrl?.trim();
const resolved = configured && configured.length > 0 ? configured : LMSTUDIO_DEFAULT_BASE_URL;
const fetchableBaseUrl = toFetchableLmstudioBaseUrl(resolved);
try {
const parsed = new URL(fetchableBaseUrl);
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
throw new TypeError(`Unsupported LM Studio protocol: ${parsed.protocol}`);
}
const pathname = normalizeUrlPath(parsed.pathname);
parsed.pathname = pathname.length > 0 ? pathname : "/";
parsed.search = "";
parsed.hash = "";
return parsed.toString().replace(/\/$/, "");
} catch {
const trimmed = resolved.replace(/\/+$/, "");
const normalized = normalizeUrlPath(trimmed);
return normalized.length > 0 ? normalized : LMSTUDIO_DEFAULT_BASE_URL;
}
}
/** Resolves LM Studio inference base URL and always appends /v1. */
export function resolveLmstudioInferenceBase(configuredBaseUrl?: string): string {
const serverBase = resolveLmstudioServerBase(configuredBaseUrl);
return `${serverBase}/v1`;
}
/** Canonicalizes persisted LM Studio provider config to the inference base URL form. */
export function normalizeLmstudioProviderConfig(
provider: ModelProviderConfig,
): ModelProviderConfig {
const configuredBaseUrl = typeof provider.baseUrl === "string" ? provider.baseUrl.trim() : "";
if (!configuredBaseUrl) {
return provider;
}
const normalizedBaseUrl = resolveLmstudioInferenceBase(configuredBaseUrl);
const request =
provider.request && typeof provider.request === "object" && !Array.isArray(provider.request)
? provider.request
: undefined;
const requestWithPrivateNetworkDefault =
typeof request?.allowPrivateNetwork === "boolean"
? request
: {
...request,
allowPrivateNetwork: true,
};
if (
normalizedBaseUrl === provider.baseUrl &&
requestWithPrivateNetworkDefault === provider.request
) {
return provider;
}
return {
...provider,
baseUrl: normalizedBaseUrl,
request: requestWithPrivateNetworkDefault,
};
}
export function normalizeLmstudioConfiguredCatalogEntry(
entry: unknown,
): LmstudioConfiguredCatalogEntry | null {
if (!entry || typeof entry !== "object") {
return null;
}
const record = entry as Record<string, unknown>;
if (typeof record.id !== "string" || record.id.trim().length === 0) {
return null;
}
const id = record.id.trim();
const name = typeof record.name === "string" && record.name.trim().length > 0 ? record.name : id;
const contextWindow = asPositiveSafeInteger(record.contextWindow);
const contextTokens = asPositiveSafeInteger(record.contextTokens);
const reasoning = typeof record.reasoning === "boolean" ? record.reasoning : undefined;
const input = Array.isArray(record.input)
? record.input.filter(
(item): item is "text" | "image" | "document" =>
item === "text" || item === "image" || item === "document",
)
: undefined;
const compat = normalizeLmstudioConfiguredCompat(record.compat);
return {
id,
name,
contextWindow,
contextTokens,
reasoning,
input: input && input.length > 0 ? input : undefined,
compat,
};
}
export function normalizeLmstudioConfiguredCatalogEntries(
models: unknown,
): LmstudioConfiguredCatalogEntry[] {
if (!Array.isArray(models)) {
return [];
}
return models
.map((entry) => normalizeLmstudioConfiguredCatalogEntry(entry))
.filter((entry): entry is LmstudioConfiguredCatalogEntry => entry !== null);
}
export function buildLmstudioModelName(model: {
displayName: string;
format: "gguf" | "mlx" | null;
vision: boolean;
trainedForToolUse: boolean;
loaded: boolean;
}): string {
const tags: string[] = [];
if (model.format === "mlx") {
tags.push("MLX");
} else if (model.format === "gguf") {
tags.push("GGUF");
}
if (model.vision) {
tags.push("vision");
}
if (model.trainedForToolUse) {
tags.push("tool-use");
}
if (model.loaded) {
tags.push("loaded");
}
if (tags.length === 0) {
return model.displayName;
}
return `${model.displayName} (${tags.join(", ")})`;
}
/**
* Base model fields extracted from a single LM Studio wire entry.
* Shared by the setup layer (persists simple names to config) and the runtime
* discovery path (which enriches the name with format/state tags).
*/
export type LmstudioModelBase = {
id: string;
displayName: string;
format: "gguf" | "mlx" | null;
vision: boolean;
trainedForToolUse: boolean;
loaded: boolean;
reasoning: boolean;
input: Array<"text" | "image">;
cost: ModelDefinitionConfig["cost"];
compat?: ModelDefinitionConfig["compat"];
contextWindow: number;
contextTokens: number;
maxTokens: number;
};
/**
* Maps a single LM Studio wire entry to its base model fields.
* Returns null for non-LLM entries or entries with no usable key.
*
* Shared by both the setup layer (persists simple names to config) and the
* runtime discovery path (which enriches the name with format/state tags via
* buildLmstudioModelName).
*/
export function mapLmstudioWireEntry(entry: LmstudioModelWire): LmstudioModelBase | null {
if (entry.type !== "llm") {
return null;
}
const id = entry.key?.trim() ?? "";
if (!id) {
return null;
}
const loadedContextWindow = resolveLoadedContextWindow(entry);
const advertisedContextWindow = asPositiveSafeInteger(entry.max_context_length) ?? null;
const contextWindow = advertisedContextWindow ?? SELF_HOSTED_DEFAULT_CONTEXT_WINDOW;
// Keep native/advertised context window metadata in catalog, but use a practical
// default target for model loading unless callers explicitly override it.
const contextTokens = Math.min(contextWindow, LMSTUDIO_DEFAULT_LOAD_CONTEXT_LENGTH);
const rawDisplayName = entry.display_name?.trim();
return {
id,
displayName: rawDisplayName && rawDisplayName.length > 0 ? rawDisplayName : id,
format: entry.format ?? null,
vision: entry.capabilities?.vision === true,
trainedForToolUse: entry.capabilities?.trained_for_tool_use === true,
// Use the same validity check as resolveLoadedContextWindow so malformed entries
// like [null, {}] don't produce a false positive "loaded" tag.
loaded: loadedContextWindow !== null,
reasoning: resolveLmstudioReasoningCapability(entry),
input: entry.capabilities?.vision ? ["text", "image"] : ["text"],
cost: SELF_HOSTED_DEFAULT_COST,
compat: resolveLmstudioReasoningCompat(entry),
contextWindow,
contextTokens,
maxTokens: Math.max(1, Math.min(contextWindow, SELF_HOSTED_DEFAULT_MAX_TOKENS)),
};
}
/**
* Maps LM Studio wire models to config entries using plain display names.
* Use this for config persistence where runtime format/state tags are not needed.
* For runtime discovery with enriched names, use discoverLmstudioModels from models.fetch.ts.
*/
export function mapLmstudioWireModelsToConfig(
models: LmstudioModelWire[],
): ModelDefinitionConfig[] {
return models
.map((entry): ModelDefinitionConfig | null => {
const base = mapLmstudioWireEntry(entry);
if (!base) {
return null;
}
return {
id: base.id,
name: base.displayName,
reasoning: base.reasoning,
input: base.input,
cost: base.cost,
...(base.compat ? { compat: base.compat } : {}),
contextWindow: base.contextWindow,
contextTokens: base.contextTokens,
maxTokens: base.maxTokens,
};
})
.filter((entry): entry is ModelDefinitionConfig => entry !== null);
}

View File

@@ -0,0 +1,60 @@
// Lmstudio provider module implements model/runtime integration.
import {
CUSTOM_LOCAL_AUTH_MARKER,
hasConfiguredSecretInput,
normalizeOptionalSecretInput,
} from "openclaw/plugin-sdk/provider-auth";
import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-shared";
import { LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER } from "./defaults.js";
export function hasLmstudioAuthorizationHeader(headers: unknown): boolean {
if (!headers || typeof headers !== "object" || Array.isArray(headers)) {
return false;
}
for (const [headerName, headerValue] of Object.entries(headers)) {
if (headerName.trim().toLowerCase() !== "authorization") {
continue;
}
if (hasConfiguredSecretInput(headerValue)) {
return true;
}
}
return false;
}
export function resolveLmstudioProviderAuthMode(
apiKey: ModelProviderConfig["apiKey"] | undefined,
): ModelProviderConfig["auth"] | undefined {
const normalized = normalizeOptionalSecretInput(apiKey);
if (normalized !== undefined) {
const trimmed = normalized.trim();
if (
!trimmed ||
trimmed === LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER ||
trimmed === CUSTOM_LOCAL_AUTH_MARKER
) {
return undefined;
}
return "api-key";
}
return hasConfiguredSecretInput(apiKey) ? "api-key" : undefined;
}
export function shouldUseLmstudioApiKeyPlaceholder(params: {
hasModels: boolean;
resolvedApiKey: ModelProviderConfig["apiKey"] | undefined;
hasAuthorizationHeader?: boolean;
}): boolean {
return params.hasModels && !params.resolvedApiKey && !params.hasAuthorizationHeader;
}
export function shouldUseLmstudioSyntheticAuth(
providerConfig: ModelProviderConfig | undefined,
): boolean {
const hasModels = Array.isArray(providerConfig?.models) && providerConfig.models.length > 0;
return (
hasModels &&
!resolveLmstudioProviderAuthMode(providerConfig?.apiKey) &&
!hasLmstudioAuthorizationHeader(providerConfig?.headers)
);
}

View File

@@ -0,0 +1,358 @@
// Lmstudio tests cover runtime plugin behavior.
import type { OpenClawConfig } from "openclaw/plugin-sdk/provider-auth";
import { CUSTOM_LOCAL_AUTH_MARKER } from "openclaw/plugin-sdk/provider-auth";
import { afterAll, beforeEach, describe, expect, it, vi } from "vitest";
import { LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER } from "./defaults.js";
import {
buildLmstudioAuthHeaders,
resolveLmstudioConfiguredApiKey,
resolveLmstudioProviderHeaders,
resolveLmstudioRuntimeApiKey,
} from "./runtime.js";
const resolveApiKeyForProviderMock = vi.hoisted(() => vi.fn());
vi.mock("openclaw/plugin-sdk/provider-auth-runtime", async (importOriginal) => {
const actual = await importOriginal<typeof import("openclaw/plugin-sdk/provider-auth-runtime")>();
return {
...actual,
resolveApiKeyForProvider: (...args: unknown[]) => resolveApiKeyForProviderMock(...args),
};
});
afterAll(() => {
vi.doUnmock("openclaw/plugin-sdk/provider-auth-runtime");
vi.resetModules();
});
function buildLmstudioConfig(overrides?: {
apiKey?: unknown;
headers?: unknown;
auth?: "api-key";
}): OpenClawConfig {
return {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234/v1",
api: "openai-completions",
...(overrides?.auth ? { auth: overrides.auth } : {}),
...(overrides?.apiKey !== undefined ? { apiKey: overrides.apiKey } : {}),
...(overrides?.headers !== undefined ? { headers: overrides.headers } : {}),
models: [],
},
},
},
} as OpenClawConfig;
}
describe("lmstudio-runtime", () => {
beforeEach(() => {
resolveApiKeyForProviderMock.mockReset();
});
it("throws when runtime auth resolves to blank and no configured key exists", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: " ",
source: "profile:lmstudio:default",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({ auth: "api-key" }),
}),
).rejects.toThrow(/LM Studio API key is required/i);
});
it("falls back to configured env marker key when profile resolution fails", async () => {
resolveApiKeyForProviderMock.mockRejectedValueOnce(
new Error('No API key found for provider "lmstudio". Auth store: /tmp/auth-profiles.json.'),
);
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({
auth: "api-key",
apiKey: "${LM_API_TOKEN}",
}),
env: {
LM_API_TOKEN: "template-lmstudio-key",
},
}),
).resolves.toBe("template-lmstudio-key");
});
it("accepts synthesized lmstudio-local for non-explicit auth mode", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
source: "models.providers.lmstudio (synthetic local key)",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig(),
}),
).resolves.toBe(LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER);
});
it("accepts synthesized lmstudio-local for explicit api-key mode", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
source: "models.providers.lmstudio (synthetic local key)",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({ auth: "api-key" }),
}),
).resolves.toBe(LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER);
});
it("accepts shared synthetic local marker for keyless runtime auth", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: CUSTOM_LOCAL_AUTH_MARKER,
source: "models.providers.lmstudio (synthetic local key)",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig(),
}),
).resolves.toBe(CUSTOM_LOCAL_AUTH_MARKER);
});
it("allows header-only runtime auth when Authorization is configured", async () => {
resolveApiKeyForProviderMock.mockRejectedValueOnce(
new Error('No API key found for provider "lmstudio". Auth store: /tmp/auth-profiles.json.'),
);
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({
headers: {
Authorization: "Bearer proxy-token",
},
}),
}),
).resolves.toBeUndefined();
});
it("allows header-only runtime auth when an api key env template is unset", async () => {
resolveApiKeyForProviderMock.mockRejectedValueOnce(
new Error('No API key found for provider "lmstudio". Auth store: /tmp/auth-profiles.json.'),
);
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({
apiKey: "${LMSTUDIO_API_KEY}",
headers: {
Authorization: "Bearer proxy-token",
},
}),
env: {},
}),
).resolves.toBeUndefined();
});
it("suppresses profile runtime auth when Authorization is configured", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: "stale-profile-key",
source: "profile:lmstudio:default",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({
headers: {
Authorization: "Bearer proxy-token",
},
}),
}),
).resolves.toBeUndefined();
});
it("suppresses env runtime auth when Authorization is configured", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: "stale-env-key",
source: "env:LM_API_TOKEN",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({
headers: {
Authorization: "Bearer proxy-token",
},
}),
}),
).resolves.toBeUndefined();
});
it("suppresses shell env runtime auth when Authorization is configured", async () => {
resolveApiKeyForProviderMock.mockResolvedValueOnce({
apiKey: "stale-shell-env-key",
source: "shell env: LM_API_TOKEN",
mode: "api-key",
});
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({
headers: {
Authorization: "Bearer proxy-token",
},
}),
}),
).resolves.toBeUndefined();
});
it("throws when explicit api-key mode cannot resolve any key", async () => {
resolveApiKeyForProviderMock.mockRejectedValue(
new Error('No API key found for provider "lmstudio". Auth store: /tmp/auth-profiles.json.'),
);
await expect(
resolveLmstudioRuntimeApiKey({
config: buildLmstudioConfig({ auth: "api-key" }),
}),
).rejects.toThrow(/LM Studio API key is required/i);
await expect(
resolveLmstudioConfiguredApiKey({
config: buildLmstudioConfig({ auth: "api-key" }),
}),
).resolves.toBeUndefined();
});
it("resolves SecretRef api key and headers", async () => {
const headerRef = {
"X-Proxy-Auth": {
source: "env" as const,
provider: "default" as const,
id: "LMSTUDIO_PROXY_TOKEN",
},
};
await expect(
resolveLmstudioConfiguredApiKey({
config: buildLmstudioConfig({
apiKey: {
source: "env",
provider: "default",
id: "LM_API_TOKEN",
},
}),
env: {
LM_API_TOKEN: "secretref-lmstudio-key",
},
}),
).resolves.toBe("secretref-lmstudio-key");
await expect(
resolveLmstudioProviderHeaders({
config: buildLmstudioConfig({ headers: headerRef }),
env: {
LMSTUDIO_PROXY_TOKEN: "proxy-token",
},
headers: headerRef,
}),
).resolves.toEqual({
"X-Proxy-Auth": "proxy-token",
});
});
it("resolves env-template api keys from config", async () => {
await expect(
resolveLmstudioConfiguredApiKey({
config: buildLmstudioConfig({
apiKey: "${LM_API_TOKEN}",
}),
env: {
LM_API_TOKEN: "template-lmstudio-key",
},
}),
).resolves.toBe("template-lmstudio-key");
});
it("resolves arbitrary env-template api keys from config", async () => {
await expect(
resolveLmstudioConfiguredApiKey({
config: buildLmstudioConfig({
apiKey: "${LMSTUDIO_API_KEY}",
}),
env: {
LMSTUDIO_API_KEY: "custom-template-lmstudio-key",
},
}),
).resolves.toBe("custom-template-lmstudio-key");
});
it("throws a path-specific error when an env-template api key cannot be resolved", async () => {
await expect(
resolveLmstudioConfiguredApiKey({
config: buildLmstudioConfig({
apiKey: "${LMSTUDIO_API_KEY}",
}),
env: {},
}),
).rejects.toThrow(/models\.providers\.lmstudio\.apiKey/i);
});
it("throws a path-specific error when a SecretRef header cannot be resolved", async () => {
const headerRef = {
"X-Proxy-Auth": {
source: "env" as const,
provider: "default" as const,
id: "LMSTUDIO_PROXY_TOKEN",
},
};
await expect(
resolveLmstudioProviderHeaders({
config: buildLmstudioConfig({ headers: headerRef }),
env: {},
headers: headerRef,
}),
).rejects.toThrow(/models\.providers\.lmstudio\.headers\.X-Proxy-Auth/i);
});
it("builds auth headers with key precedence and json support", () => {
expect(buildLmstudioAuthHeaders({})).toBeUndefined();
expect(buildLmstudioAuthHeaders({ apiKey: " sk-test " })).toEqual({
Authorization: "Bearer sk-test",
});
expect(buildLmstudioAuthHeaders({ apiKey: " " })).toBeUndefined();
expect(
buildLmstudioAuthHeaders({ apiKey: LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER }),
).toBeUndefined();
expect(
buildLmstudioAuthHeaders({
apiKey: LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
headers: {
Authorization: "Bearer proxy-token",
},
}),
).toEqual({
Authorization: "Bearer proxy-token",
});
expect(
buildLmstudioAuthHeaders({
apiKey: "sk-new",
json: true,
headers: {
authorization: "Bearer sk-old",
"X-Proxy": "proxy-token",
},
}),
).toEqual({
"Content-Type": "application/json",
"X-Proxy": "proxy-token",
Authorization: "Bearer sk-new",
});
});
});

View File

@@ -0,0 +1,277 @@
// Lmstudio plugin module implements runtime behavior.
import {
CUSTOM_LOCAL_AUTH_MARKER,
isKnownEnvApiKeyMarker,
isNonSecretApiKeyMarker,
normalizeApiKeyConfig,
normalizeOptionalSecretInput,
type OpenClawConfig,
} from "openclaw/plugin-sdk/provider-auth";
import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runtime";
import { resolveConfiguredSecretInputString } from "openclaw/plugin-sdk/secret-input-runtime";
import {
LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
LMSTUDIO_PROVIDER_ID,
} from "./defaults.js";
import { hasLmstudioAuthorizationHeader } from "./provider-auth.js";
type LmstudioAuthHeadersParams = {
apiKey?: string;
json?: boolean;
headers?: Record<string, string>;
};
export function buildLmstudioAuthHeaders(
params: LmstudioAuthHeadersParams,
): Record<string, string> | undefined {
const headers: Record<string, string> = { ...params.headers };
// Runtime auth resolution is strict, but guard known non-secret markers here.
const apiKey = params.apiKey?.trim();
const isSyntheticLocalKey = apiKey === LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER;
if (apiKey && !isSyntheticLocalKey && !isNonSecretApiKeyMarker(apiKey)) {
for (const headerName of Object.keys(headers)) {
if (headerName.toLowerCase() === "authorization") {
delete headers[headerName];
}
}
headers.Authorization = `Bearer ${apiKey}`;
}
if (params.json) {
headers["Content-Type"] = "application/json";
}
return Object.keys(headers).length > 0 ? headers : undefined;
}
function sanitizeStringHeaders(headers: unknown): Record<string, string> | undefined {
if (!headers || typeof headers !== "object" || Array.isArray(headers)) {
return undefined;
}
const next: Record<string, string> = {};
for (const [headerName, headerValue] of Object.entries(headers)) {
if (typeof headerValue !== "string") {
continue;
}
const normalized = headerValue.trim();
if (!normalized) {
continue;
}
next[headerName] = normalized;
}
return Object.keys(next).length > 0 ? next : undefined;
}
function shouldSuppressResolvedRuntimeApiKeyForHeaderAuth(
source: string | undefined,
hasAuthorizationHeader: boolean,
): boolean {
if (!hasAuthorizationHeader || !source) {
return false;
}
return /^profile:|^(?:shell )?env(?::|$)/.test(source);
}
export async function resolveLmstudioConfiguredApiKey(params: {
config?: OpenClawConfig;
env?: NodeJS.ProcessEnv;
path?: string;
allowUnresolved?: boolean;
}): Promise<string | undefined> {
const providerConfig = params.config?.models?.providers?.[LMSTUDIO_PROVIDER_ID];
const apiKeyInput = providerConfig?.apiKey;
if (apiKeyInput === undefined || apiKeyInput === null) {
return undefined;
}
const path = params.path ?? "models.providers.lmstudio.apiKey";
const env = params.env ?? process.env;
const directApiKey = normalizeOptionalSecretInput(apiKeyInput);
if (directApiKey !== undefined) {
const resolved = params.config
? await resolveConfiguredSecretInputString({
config: params.config,
env,
value: directApiKey,
path,
unresolvedReasonStyle: "detailed",
})
: { value: directApiKey };
if (resolved.unresolvedRefReason) {
if (params.allowUnresolved) {
return undefined;
}
throw new Error(`${path}: ${resolved.unresolvedRefReason}`);
}
const resolvedValue = normalizeOptionalSecretInput(resolved.value);
const trimmed = resolvedValue ? normalizeApiKeyConfig(resolvedValue).trim() : "";
if (!trimmed) {
return undefined;
}
if (isKnownEnvApiKeyMarker(trimmed)) {
const envValue = normalizeOptionalSecretInput(env[trimmed]);
return envValue;
}
return isNonSecretApiKeyMarker(trimmed) ? undefined : trimmed;
}
if (!params.config) {
return undefined;
}
const resolved = await resolveConfiguredSecretInputString({
config: params.config,
env,
value: apiKeyInput,
path,
unresolvedReasonStyle: "detailed",
});
if (resolved.unresolvedRefReason) {
if (params.allowUnresolved) {
return undefined;
}
throw new Error(`${path}: ${resolved.unresolvedRefReason}`);
}
const resolvedValue = normalizeOptionalSecretInput(resolved.value);
const trimmedResolvedValue = resolvedValue ? normalizeApiKeyConfig(resolvedValue).trim() : "";
if (!trimmedResolvedValue) {
return undefined;
}
if (isNonSecretApiKeyMarker(trimmedResolvedValue)) {
return undefined;
}
return trimmedResolvedValue;
}
export async function resolveLmstudioProviderHeaders(params: {
config?: OpenClawConfig;
env?: NodeJS.ProcessEnv;
headers?: unknown;
path?: string;
}): Promise<Record<string, string> | undefined> {
const headerInputs = params.headers;
if (!headerInputs || typeof headerInputs !== "object" || Array.isArray(headerInputs)) {
return undefined;
}
if (!params.config) {
return sanitizeStringHeaders(headerInputs);
}
const pathPrefix = params.path ?? "models.providers.lmstudio.headers";
const resolved: Record<string, string> = {};
for (const [headerName, headerValue] of Object.entries(headerInputs)) {
const resolvedHeader = await resolveConfiguredSecretInputString({
config: params.config,
env: params.env ?? process.env,
value: headerValue,
path: `${pathPrefix}.${headerName}`,
unresolvedReasonStyle: "detailed",
});
if (resolvedHeader.unresolvedRefReason) {
throw new Error(`${pathPrefix}.${headerName}: ${resolvedHeader.unresolvedRefReason}`);
}
const resolvedValue = resolvedHeader.value;
if (!resolvedValue) {
continue;
}
resolved[headerName] = resolvedValue;
}
return Object.keys(resolved).length > 0 ? resolved : undefined;
}
/**
* Resolves LM Studio API key and provider headers in parallel.
* Use this as the standard auth setup step before discovery or model load calls.
*/
export async function resolveLmstudioRequestContext(params: {
config?: OpenClawConfig;
agentDir?: string;
env?: NodeJS.ProcessEnv;
providerHeaders?: unknown;
}): Promise<{ apiKey: string | undefined; headers: Record<string, string> | undefined }> {
const providerHeaders =
params.providerHeaders ?? params.config?.models?.providers?.[LMSTUDIO_PROVIDER_ID]?.headers;
const [apiKey, headers] = await Promise.all([
resolveLmstudioRuntimeApiKey({
config: params.config,
agentDir: params.agentDir,
env: params.env,
headers: providerHeaders,
}),
resolveLmstudioProviderHeaders({
config: params.config,
env: params.env,
headers: providerHeaders,
}),
]);
return { apiKey, headers };
}
/**
* Resolves LM Studio runtime API key from config.
*/
export async function resolveLmstudioRuntimeApiKey(params: {
config?: OpenClawConfig;
agentDir?: string;
env?: NodeJS.ProcessEnv;
headers?: unknown;
}): Promise<string | undefined> {
const config = params.config;
if (!config) {
return undefined;
}
const providerHeaders =
params.headers ?? config.models?.providers?.[LMSTUDIO_PROVIDER_ID]?.headers;
const hasAuthorizationHeader = hasLmstudioAuthorizationHeader(providerHeaders);
let configuredApiKeyPromise: Promise<string | undefined> | undefined;
const getConfiguredApiKey = async () => {
configuredApiKeyPromise ??= resolveLmstudioConfiguredApiKey({
config,
env: params.env,
allowUnresolved: hasAuthorizationHeader,
});
return await configuredApiKeyPromise;
};
const resolveConfiguredApiKeyOrThrow = async () => {
const configuredApiKey = await getConfiguredApiKey();
if (configuredApiKey) {
return configuredApiKey;
}
if (hasAuthorizationHeader) {
return undefined;
}
const envMarker = `\${${LMSTUDIO_DEFAULT_API_KEY_ENV_VAR}}`;
throw new Error(
[
"LM Studio API key is required.",
`Set models.providers.lmstudio.apiKey (for example "${envMarker}")`,
'or run "openclaw models auth lmstudio".',
].join(" "),
);
};
let resolved: Awaited<ReturnType<typeof resolveApiKeyForProvider>>;
try {
resolved = await resolveApiKeyForProvider({
provider: LMSTUDIO_PROVIDER_ID,
cfg: config,
agentDir: params.agentDir,
});
} catch {
return await resolveConfiguredApiKeyOrThrow();
}
// Normalize empty/whitespace keys to undefined for callers.
const resolvedApiKey = resolved.apiKey?.trim();
if (!resolvedApiKey || resolvedApiKey.length === 0) {
return await resolveConfiguredApiKeyOrThrow();
}
if (shouldSuppressResolvedRuntimeApiKeyForHeaderAuth(resolved.source, hasAuthorizationHeader)) {
return await resolveConfiguredApiKeyOrThrow();
}
if (
isNonSecretApiKeyMarker(resolvedApiKey) &&
resolvedApiKey !== CUSTOM_LOCAL_AUTH_MARKER &&
resolvedApiKey !== LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER
) {
return await resolveConfiguredApiKeyOrThrow();
}
return resolvedApiKey;
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,881 @@
// Lmstudio setup module handles plugin onboarding behavior.
import { parseStrictPositiveInteger } from "openclaw/plugin-sdk/number-runtime";
import {
removeProviderAuthProfilesWithLock,
buildApiKeyCredential,
ensureApiKeyFromEnvOrPrompt,
hasConfiguredSecretInput,
normalizeOptionalSecretInput,
type OpenClawConfig,
type SecretInput,
type SecretInputMode,
} from "openclaw/plugin-sdk/provider-auth";
import type {
ModelDefinitionConfig,
ModelProviderConfig,
} from "openclaw/plugin-sdk/provider-model-shared";
import { withAgentModelAliases } from "openclaw/plugin-sdk/provider-onboard";
import {
applyProviderDefaultModel,
configureOpenAICompatibleSelfHostedProviderNonInteractive,
type ProviderAuthMethodNonInteractiveContext,
type ProviderAuthResult,
type ProviderCatalogContext,
type ProviderPrepareDynamicModelContext,
type ProviderRuntimeModel,
} from "openclaw/plugin-sdk/provider-setup";
import { WizardCancelledError, type WizardPrompter } from "openclaw/plugin-sdk/setup";
import { normalizeStringEntries } from "openclaw/plugin-sdk/string-coerce-runtime";
import {
LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
LMSTUDIO_DEFAULT_INFERENCE_BASE_URL,
LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER,
LMSTUDIO_MODEL_PLACEHOLDER,
LMSTUDIO_DEFAULT_BASE_URL,
LMSTUDIO_DOCKER_HOST_BASE_URL,
LMSTUDIO_DOCKER_HOST_INFERENCE_BASE_URL,
LMSTUDIO_PROVIDER_LABEL,
LMSTUDIO_DEFAULT_MODEL_ID,
LMSTUDIO_PROVIDER_ID as PROVIDER_ID,
} from "./defaults.js";
import { discoverLmstudioModels, fetchLmstudioModels } from "./models.fetch.js";
import {
mapLmstudioWireModelsToConfig,
type LmstudioModelWire,
resolveLmstudioInferenceBase,
} from "./models.js";
import {
hasLmstudioAuthorizationHeader,
resolveLmstudioProviderAuthMode,
shouldUseLmstudioApiKeyPlaceholder,
} from "./provider-auth.js";
import {
resolveLmstudioConfiguredApiKey,
resolveLmstudioProviderHeaders,
resolveLmstudioRequestContext,
} from "./runtime.js";
type ProviderPromptText = (params: {
message: string;
initialValue?: string;
placeholder?: string;
validate?: (value: string | undefined) => string | undefined;
}) => Promise<string | undefined>;
type ProviderPromptNote = (message: string, title?: string) => Promise<void> | void;
type LmstudioDiscoveryResult = Awaited<ReturnType<typeof fetchLmstudioModels>>;
type LmstudioSetupDiscovery = {
discovery: LmstudioDiscoveryResult;
models: ModelDefinitionConfig[];
defaultModel: string | undefined;
defaultModelId: string | undefined;
};
function isTruthyEnvValue(value: string | undefined): boolean {
return ["1", "true", "yes", "on"].includes(value?.trim().toLowerCase() ?? "");
}
function resolveLmstudioSetupDefaultBaseUrl(env: NodeJS.ProcessEnv = process.env): string {
return isTruthyEnvValue(env.OPENCLAW_DOCKER_SETUP)
? LMSTUDIO_DOCKER_HOST_BASE_URL
: LMSTUDIO_DEFAULT_BASE_URL;
}
function resolveLmstudioSetupDefaultInferenceBaseUrl(env: NodeJS.ProcessEnv = process.env): string {
return isTruthyEnvValue(env.OPENCLAW_DOCKER_SETUP)
? LMSTUDIO_DOCKER_HOST_INFERENCE_BASE_URL
: LMSTUDIO_DEFAULT_INFERENCE_BASE_URL;
}
function stripLmstudioStoredAuthConfig(cfg: OpenClawConfig): OpenClawConfig {
const { profiles: _profiles, order: _order, ...restAuth } = cfg.auth ?? {};
const nextProfiles = Object.fromEntries(
Object.entries(cfg.auth?.profiles ?? {}).filter(
([, profile]) => profile.provider !== PROVIDER_ID,
),
);
const nextOrder = Object.fromEntries(
Object.entries(cfg.auth?.order ?? {}).filter(([providerId]) => providerId !== PROVIDER_ID),
);
return {
...cfg,
auth:
Object.keys(restAuth).length > 0 ||
Object.keys(nextProfiles).length > 0 ||
Object.keys(nextOrder).length > 0
? {
...restAuth,
...(Object.keys(nextProfiles).length > 0 ? { profiles: nextProfiles } : {}),
...(Object.keys(nextOrder).length > 0 ? { order: nextOrder } : {}),
}
: undefined,
};
}
function resolvePositiveInteger(value: unknown): number | undefined {
if (typeof value === "number" && Number.isFinite(value)) {
const normalized = Math.floor(value);
return normalized > 0 ? normalized : undefined;
}
if (typeof value !== "string") {
return undefined;
}
const trimmed = value.trim();
if (!trimmed || !/^\d+$/.test(trimmed)) {
return undefined;
}
return parseStrictPositiveInteger(trimmed);
}
function buildLmstudioSetupProviderConfig(params: {
existingProvider: ModelProviderConfig | undefined;
sharedProvider?: ModelProviderConfig;
baseUrl: string;
apiKey?: ModelProviderConfig["apiKey"];
headers: ModelProviderConfig["headers"] | undefined;
models: ModelDefinitionConfig[];
}): ModelProviderConfig {
const existingWithoutAuth = params.existingProvider
? (({ auth: _auth, apiKey: _apiKey, ...rest }) => rest)(params.existingProvider)
: undefined;
const sharedWithoutAuth = params.sharedProvider
? (({ auth: _auth, apiKey: _apiKey, ...rest }) => rest)(params.sharedProvider)
: undefined;
const resolvedAuth = resolveLmstudioProviderAuthMode(params.apiKey);
return {
...existingWithoutAuth,
...sharedWithoutAuth,
baseUrl: params.baseUrl,
api: params.sharedProvider?.api ?? params.existingProvider?.api ?? "openai-completions",
...(resolvedAuth ? { auth: resolvedAuth } : {}),
...(params.apiKey !== undefined ? { apiKey: params.apiKey } : {}),
headers: params.headers,
models: params.models,
};
}
function resolveLmstudioModelAdvertisedContextLimit(entry: LmstudioModelWire): number | undefined {
const raw = entry.max_context_length;
if (raw === undefined || !Number.isFinite(raw) || raw <= 0) {
return undefined;
}
return Math.floor(raw);
}
function applyModelContextTokensOverride(
model: ModelDefinitionConfig,
contextTokens: number,
): ModelDefinitionConfig {
return {
...model,
contextTokens,
maxTokens: Math.min(model.maxTokens, contextTokens),
};
}
function applyRequestedContextWindowToAllModels(params: {
models: ModelDefinitionConfig[];
discoveryModels: LmstudioModelWire[];
requestedContextWindow?: number;
}): ModelDefinitionConfig[] {
const requestedContextWindow = params.requestedContextWindow;
if (!requestedContextWindow) {
return params.models;
}
const contextLimitByModelId = new Map(
params.discoveryModels
.map((entry) => {
const modelId = entry.key?.trim();
if (!modelId) {
return null;
}
return [modelId, resolveLmstudioModelAdvertisedContextLimit(entry)] as const;
})
.filter((entry): entry is readonly [string, number | undefined] => Boolean(entry)),
);
return params.models.map((model) =>
applyModelContextTokensOverride(
model,
Math.min(
requestedContextWindow,
contextLimitByModelId.get(model.id) ?? requestedContextWindow,
),
),
);
}
function resolveLmstudioDiscoveryFailure(params: {
baseUrl: string;
discovery: LmstudioDiscoveryResult;
}): { noteLines: [string, string]; reason: string } | null {
const { baseUrl, discovery } = params;
if (!discovery.reachable) {
return {
noteLines: [
`LM Studio could not be reached at ${baseUrl}.`,
"Start LM Studio (or run lms server start) and re-run setup.",
],
reason: "LM Studio not reachable",
};
}
if (discovery.status !== undefined && discovery.status >= 400) {
return {
noteLines: [
`LM Studio returned HTTP ${discovery.status} while listing models at ${baseUrl}.`,
"Check the base URL and API key, then re-run setup.",
],
reason: `LM Studio discovery failed (${discovery.status})`,
};
}
const hasUsableModel = discovery.models.some(
(model) => model.type === "llm" && Boolean(model.key?.trim()),
);
if (!hasUsableModel) {
return {
noteLines: [
`No LM Studio LLM models were found at ${baseUrl}.`,
"Load at least one model in LM Studio (or run lms load), then re-run setup.",
],
reason: "No LM Studio models found",
};
}
return null;
}
function resolvePersistedLmstudioApiKey(params: {
currentApiKey: ModelProviderConfig["apiKey"] | undefined;
explicitAuth: ModelProviderConfig["auth"] | undefined;
fallbackApiKey: ModelProviderConfig["apiKey"] | undefined;
preferFallbackApiKey?: boolean;
hasModels: boolean;
hasAuthorizationHeader?: boolean;
}): ModelProviderConfig["apiKey"] | undefined {
if (params.explicitAuth === "api-key") {
if (params.preferFallbackApiKey && params.fallbackApiKey !== undefined) {
return params.fallbackApiKey;
}
if (resolveLmstudioProviderAuthMode(params.currentApiKey)) {
return params.currentApiKey;
}
return params.fallbackApiKey;
}
return shouldUseLmstudioApiKeyPlaceholder({
hasModels: params.hasModels,
resolvedApiKey: params.currentApiKey,
hasAuthorizationHeader: params.hasAuthorizationHeader,
})
? LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER
: undefined;
}
/** Keeps explicit model entries first and appends unique discovered entries. */
function mergeDiscoveredModels(params: {
explicitModels?: ModelDefinitionConfig[];
discoveredModels?: ModelDefinitionConfig[];
}): ModelDefinitionConfig[] {
const explicitModels = Array.isArray(params.explicitModels) ? params.explicitModels : [];
const discoveredModels = Array.isArray(params.discoveredModels) ? params.discoveredModels : [];
if (explicitModels.length === 0) {
return discoveredModels;
}
if (discoveredModels.length === 0) {
return explicitModels;
}
const merged = [...explicitModels];
const seen = new Set(normalizeStringEntries(explicitModels.map((model) => model.id)));
for (const model of discoveredModels) {
const id = model.id.trim();
if (!id || seen.has(id)) {
continue;
}
seen.add(id);
merged.push(model);
}
return merged;
}
async function discoverLmstudioProviderCatalog(params: {
baseUrl?: string;
apiKey?: string;
headers?: Record<string, string>;
quiet: boolean;
}): Promise<ModelProviderConfig> {
const baseUrl = resolveLmstudioInferenceBase(params.baseUrl);
const models = await discoverLmstudioModels({
baseUrl,
apiKey: params.apiKey ?? "",
headers: params.headers,
quiet: params.quiet,
});
return {
baseUrl,
api: "openai-completions",
models,
};
}
function isLmstudioDiscoveryConfigResolutionError(error: unknown): boolean {
const message = error instanceof Error ? error.message : String(error);
return (
message.includes("models.providers.lmstudio.apiKey") ||
message.includes("models.providers.lmstudio.headers.")
);
}
/** Preserves existing allowlist metadata and appends discovered LM Studio model refs. */
function mergeDiscoveredLmstudioAllowlistEntries(params: {
existing?: NonNullable<NonNullable<OpenClawConfig["agents"]>["defaults"]>["models"];
discoveredModels: ModelDefinitionConfig[];
}) {
return withAgentModelAliases(
params.existing,
normalizeStringEntries(params.discoveredModels.map((model) => model.id)).map(
(id) => `${PROVIDER_ID}/${id}`,
),
);
}
function selectDefaultLmstudioModelId(
discoveredModels: ModelDefinitionConfig[],
): string | undefined {
const ids = normalizeStringEntries(discoveredModels.map((model) => model.id));
if (ids.length === 0) {
return undefined;
}
return ids.includes(LMSTUDIO_DEFAULT_MODEL_ID) ? LMSTUDIO_DEFAULT_MODEL_ID : ids[0];
}
async function discoverLmstudioSetupModels(params: {
baseUrl: string;
apiKey?: string;
headers?: Record<string, string>;
timeoutMs?: number;
}): Promise<
| { value: LmstudioSetupDiscovery }
| { failure: NonNullable<ReturnType<typeof resolveLmstudioDiscoveryFailure>> }
> {
const discovery = await fetchLmstudioModels({
baseUrl: params.baseUrl,
apiKey: params.apiKey,
...(params.headers ? { headers: params.headers } : {}),
timeoutMs: params.timeoutMs ?? 5000,
});
const failure = resolveLmstudioDiscoveryFailure({
baseUrl: params.baseUrl,
discovery,
});
if (failure) {
return { failure };
}
const models = mapLmstudioWireModelsToConfig(discovery.models);
const defaultModelId = selectDefaultLmstudioModelId(models);
return {
value: {
discovery,
models,
defaultModel: defaultModelId ? `${PROVIDER_ID}/${defaultModelId}` : undefined,
defaultModelId,
},
};
}
/** Interactive LM Studio setup with connectivity and model-availability checks. */
export async function promptAndConfigureLmstudioInteractive(params: {
config: OpenClawConfig;
agentDir?: string;
prompter?: WizardPrompter;
secretInputMode?: SecretInputMode;
allowSecretRefPrompt?: boolean;
promptText?: ProviderPromptText;
note?: ProviderPromptNote;
}): Promise<ProviderAuthResult> {
const promptText = params.prompter
? params.prompter.text.bind(params.prompter)
: params.promptText;
if (!promptText) {
throw new Error("LM Studio interactive setup requires a text prompter.");
}
const note = params.prompter ? params.prompter.note.bind(params.prompter) : params.note;
const defaultBaseUrl = resolveLmstudioSetupDefaultBaseUrl();
const baseUrlRaw = await promptText({
message: `${LMSTUDIO_PROVIDER_LABEL} base URL`,
initialValue: defaultBaseUrl,
placeholder: defaultBaseUrl,
validate: (value) => (value?.trim() ? undefined : "Required"),
});
const baseUrl = resolveLmstudioInferenceBase(baseUrlRaw ?? defaultBaseUrl);
let credentialInput: SecretInput | undefined;
let credentialMode: SecretInputMode | undefined;
const implicitRefMode = params.allowSecretRefPrompt === false && !params.secretInputMode;
const autoRefEnvKey = process.env[LMSTUDIO_DEFAULT_API_KEY_ENV_VAR]?.trim();
const apiKey =
params.prompter && implicitRefMode && autoRefEnvKey
? autoRefEnvKey
: params.prompter
? await ensureApiKeyFromEnvOrPrompt({
config: params.config,
provider: PROVIDER_ID,
envLabel: LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
promptMessage: `${LMSTUDIO_PROVIDER_LABEL} API key`,
normalize: (value) => value.trim(),
validate: () => undefined,
prompter: params.prompter,
secretInputMode:
params.allowSecretRefPrompt === false
? (params.secretInputMode ?? "plaintext")
: params.secretInputMode,
setCredential: async (apiKeyValue, mode) => {
credentialInput = apiKeyValue;
credentialMode = mode;
},
})
: (
(await promptText({
message: `${LMSTUDIO_PROVIDER_LABEL} API key`,
placeholder: "sk-... (leave blank if auth is disabled)",
validate: () => undefined,
})) ?? ""
).trim();
const normalizedApiKey = normalizeOptionalSecretInput(apiKey);
const credentialSource =
credentialInput ??
(implicitRefMode && autoRefEnvKey ? `\${${LMSTUDIO_DEFAULT_API_KEY_ENV_VAR}}` : apiKey);
const shouldStoreCredential = params.prompter
? credentialMode === "ref" || hasConfiguredSecretInput(credentialSource)
: normalizedApiKey !== undefined;
const credential = shouldStoreCredential
? params.prompter
? buildApiKeyCredential(
PROVIDER_ID,
credentialSource,
undefined,
credentialMode
? { secretInputMode: credentialMode }
: implicitRefMode && autoRefEnvKey
? { secretInputMode: "ref" }
: undefined,
)
: {
type: "api_key" as const,
provider: PROVIDER_ID,
key: normalizedApiKey ?? apiKey,
}
: undefined;
const existingProvider = params.config.models?.providers?.[PROVIDER_ID];
// Auth setup updates auth/profile/provider model fields but does not mutate
// user-provided header overrides. Runtime request assembly is the source of truth for auth.
const persistedHeaders = existingProvider?.headers;
const resolvedHeaders = await resolveLmstudioProviderHeaders({
config: params.config,
env: process.env,
headers: persistedHeaders,
});
const hasAuthorizationHeader = hasLmstudioAuthorizationHeader(resolvedHeaders);
const setupDiscoveryApiKey =
normalizedApiKey ??
(shouldUseLmstudioApiKeyPlaceholder({
hasModels: true,
resolvedApiKey: undefined,
hasAuthorizationHeader,
})
? LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER
: undefined);
const setupDiscovery = await discoverLmstudioSetupModels({
baseUrl,
apiKey: setupDiscoveryApiKey,
...(resolvedHeaders ? { headers: resolvedHeaders } : {}),
timeoutMs: 5000,
});
if ("failure" in setupDiscovery) {
await note?.(setupDiscovery.failure.noteLines.join("\n"), "LM Studio");
throw new WizardCancelledError(setupDiscovery.failure.reason);
}
let discoveredModels = setupDiscovery.value.models;
if (params.prompter) {
const requestedRaw = await params.prompter.text({
message: "Preferred context length to load LM Studio models with (optional)",
placeholder: "e.g. 32768 (leave blank to skip)",
validate: (value) =>
value?.trim()
? resolvePositiveInteger(value)
? undefined
: "Enter a positive integer token count"
: undefined,
});
const requestedContextWindow = resolvePositiveInteger(requestedRaw);
discoveredModels = applyRequestedContextWindowToAllModels({
models: discoveredModels,
discoveryModels: setupDiscovery.value.discovery.models,
requestedContextWindow,
});
}
const allowlistEntries = mergeDiscoveredLmstudioAllowlistEntries({
existing: params.config.agents?.defaults?.models,
discoveredModels,
});
const defaultModel = setupDiscovery.value.defaultModel;
const persistedApiKey =
resolvePersistedLmstudioApiKey({
currentApiKey: normalizedApiKey ? existingProvider?.apiKey : undefined,
explicitAuth: resolveLmstudioProviderAuthMode(normalizedApiKey),
fallbackApiKey: normalizedApiKey ? LMSTUDIO_DEFAULT_API_KEY_ENV_VAR : undefined,
preferFallbackApiKey: true,
hasModels: discoveredModels.length > 0,
hasAuthorizationHeader,
}) ?? (normalizedApiKey ? LMSTUDIO_DEFAULT_API_KEY_ENV_VAR : undefined);
if (!credential) {
await removeProviderAuthProfilesWithLock({
provider: PROVIDER_ID,
agentDir: params.agentDir,
});
}
return {
profiles: credential
? [
{
profileId: `${PROVIDER_ID}:default`,
credential,
},
]
: [],
configPatch: {
agents: {
defaults: {
models: allowlistEntries,
},
},
models: {
// Respect existing global mode; self-hosted provider setup should merge by default.
mode: params.config.models?.mode ?? "merge",
providers: {
[PROVIDER_ID]: buildLmstudioSetupProviderConfig({
existingProvider,
baseUrl,
apiKey: persistedApiKey,
headers: persistedHeaders,
models: discoveredModels,
}),
},
},
},
defaultModel,
};
}
/** Non-interactive setup path backed by the shared self-hosted helper. */
export async function configureLmstudioNonInteractive(
ctx: ProviderAuthMethodNonInteractiveContext,
): Promise<OpenClawConfig | null> {
const customBaseUrl = normalizeOptionalSecretInput(ctx.opts.customBaseUrl);
const baseUrl = resolveLmstudioInferenceBase(
customBaseUrl || resolveLmstudioSetupDefaultInferenceBaseUrl(),
);
const normalizedCtx = customBaseUrl
? {
...ctx,
opts: {
...ctx.opts,
customBaseUrl: baseUrl,
},
}
: ctx;
const configureShared = async (configureCtx: ProviderAuthMethodNonInteractiveContext) =>
await configureOpenAICompatibleSelfHostedProviderNonInteractive({
ctx: configureCtx,
providerId: PROVIDER_ID,
providerLabel: LMSTUDIO_PROVIDER_LABEL,
defaultBaseUrl: resolveLmstudioSetupDefaultInferenceBaseUrl(),
defaultApiKeyEnvVar: LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
modelPlaceholder: LMSTUDIO_MODEL_PLACEHOLDER,
});
const requestedModelId = normalizeOptionalSecretInput(normalizedCtx.opts.customModelId);
const resolved = await normalizedCtx.resolveApiKey({
provider: PROVIDER_ID,
flagValue:
normalizeOptionalSecretInput(normalizedCtx.opts.lmstudioApiKey) ??
normalizeOptionalSecretInput(normalizedCtx.opts.customApiKey),
flagName:
normalizeOptionalSecretInput(normalizedCtx.opts.lmstudioApiKey) !== undefined
? "--lmstudio-api-key"
: "--custom-api-key",
envVar: LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
envVarName: LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
required: false,
});
const existingProvider = normalizedCtx.config.models?.providers?.[PROVIDER_ID];
// Auth setup updates auth/profile/provider model fields but does not mutate
// user-provided header overrides. Runtime request assembly is the source of truth for auth.
const persistedHeaders = existingProvider?.headers;
const resolvedHeaders = await resolveLmstudioProviderHeaders({
config: normalizedCtx.config,
env: process.env,
headers: persistedHeaders,
});
const hasAuthorizationHeader = hasLmstudioAuthorizationHeader(resolvedHeaders);
const useHeaderOnlyAuth = hasAuthorizationHeader && (!resolved || resolved.source !== "flag");
const setupDiscoveryApiKey =
(useHeaderOnlyAuth ? undefined : resolved?.key) ??
(shouldUseLmstudioApiKeyPlaceholder({
hasModels: true,
resolvedApiKey: undefined,
hasAuthorizationHeader,
})
? LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER
: undefined);
if (!setupDiscoveryApiKey && !hasAuthorizationHeader) {
normalizedCtx.runtime.error(
`LM Studio API key is required. Set ${LMSTUDIO_DEFAULT_API_KEY_ENV_VAR} or pass --lmstudio-api-key.`,
);
normalizedCtx.runtime.exit(1);
return null;
}
const setupDiscovery = await discoverLmstudioSetupModels({
baseUrl,
apiKey: setupDiscoveryApiKey,
...(resolvedHeaders ? { headers: resolvedHeaders } : {}),
timeoutMs: 5000,
});
if ("failure" in setupDiscovery) {
normalizedCtx.runtime.error(setupDiscovery.failure.noteLines.join("\n"));
normalizedCtx.runtime.exit(1);
return null;
}
const discoveredModels = setupDiscovery.value.models;
const selectedModelId = requestedModelId ?? setupDiscovery.value.defaultModelId;
const selectedModel = selectedModelId
? discoveredModels.find((model) => model.id === selectedModelId)
: undefined;
if (!selectedModelId || !selectedModel) {
const availableModels = discoveredModels.map((model) => model.id).join(", ");
normalizedCtx.runtime.error(
requestedModelId
? [
`LM Studio model ${requestedModelId} was not found at ${baseUrl}.`,
`Available models: ${availableModels}`,
].join("\n")
: [
`LM Studio did not expose a usable default model at ${baseUrl}.`,
`Available models: ${availableModels || "(none)"}`,
].join("\n"),
);
normalizedCtx.runtime.exit(1);
return null;
}
if (useHeaderOnlyAuth) {
await removeProviderAuthProfilesWithLock({
provider: PROVIDER_ID,
agentDir: normalizedCtx.agentDir,
});
const configWithoutStoredLmstudioAuth = stripLmstudioStoredAuthConfig(normalizedCtx.config);
return applyProviderDefaultModel(
{
...configWithoutStoredLmstudioAuth,
models: {
...configWithoutStoredLmstudioAuth.models,
mode: configWithoutStoredLmstudioAuth.models?.mode ?? "merge",
providers: {
...configWithoutStoredLmstudioAuth.models?.providers,
[PROVIDER_ID]: buildLmstudioSetupProviderConfig({
existingProvider,
baseUrl,
headers: persistedHeaders,
models: discoveredModels,
}),
},
},
},
`${PROVIDER_ID}/${selectedModelId}`,
);
}
const resolvedOrSynthetic =
resolved ??
(setupDiscoveryApiKey
? {
key: setupDiscoveryApiKey,
source: "flag" as const,
}
: null);
if (!resolvedOrSynthetic) {
return null;
}
// Delegate to the shared helper even when modelId is set so that onboarding
// state and credential storage are handled consistently. The pre-resolved key
// is injected via resolveApiKey to skip a second prompt. The returned config
// is then post-patched below to add the discovered model list and base URL.
const configured = await configureShared({
...normalizedCtx,
opts: {
...normalizedCtx.opts,
customModelId: selectedModelId,
},
resolveApiKey: async () => resolvedOrSynthetic,
});
if (!configured) {
return null;
}
const sharedProvider = configured.models?.providers?.[PROVIDER_ID];
const resolvedSyntheticLocalKey = resolvedOrSynthetic.key === LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER;
const persistedApiKey = resolvePersistedLmstudioApiKey({
// If this run resolved to keyless local mode, avoid preserving stale env markers.
currentApiKey: resolvedSyntheticLocalKey ? undefined : existingProvider?.apiKey,
explicitAuth: resolveLmstudioProviderAuthMode(resolvedOrSynthetic.key),
fallbackApiKey: resolvedSyntheticLocalKey
? LMSTUDIO_LOCAL_API_KEY_PLACEHOLDER
: (configured.models?.providers?.[PROVIDER_ID]?.apiKey ?? LMSTUDIO_DEFAULT_API_KEY_ENV_VAR),
preferFallbackApiKey: true,
hasModels: discoveredModels.length > 0,
hasAuthorizationHeader: hasLmstudioAuthorizationHeader(resolvedHeaders),
});
return {
...configured,
models: {
...configured.models,
providers: {
...configured.models?.providers,
[PROVIDER_ID]: buildLmstudioSetupProviderConfig({
existingProvider,
sharedProvider,
baseUrl,
apiKey: persistedApiKey,
headers: persistedHeaders,
models: discoveredModels,
}),
},
},
};
}
/** Discovers provider settings, merging explicit config with live model discovery. */
export async function discoverLmstudioProvider(ctx: ProviderCatalogContext): Promise<{
provider: ModelProviderConfig;
} | null> {
const explicit = ctx.config.models?.providers?.[PROVIDER_ID];
const explicitAuth = explicit?.auth;
let explicitWithoutHeaders: Omit<ModelProviderConfig, "headers" | "auth" | "apiKey"> | undefined;
if (explicit) {
const { headers: _headers, auth: _auth, apiKey: _apiKey, ...rest } = explicit;
explicitWithoutHeaders = rest;
}
const hasExplicitModels = Array.isArray(explicit?.models) && explicit.models.length > 0;
const { apiKey, discoveryApiKey } = ctx.resolveProviderApiKey(PROVIDER_ID);
let resolvedHeaders: Record<string, string> | undefined;
try {
resolvedHeaders = await resolveLmstudioProviderHeaders({
config: ctx.config,
env: ctx.env,
headers: explicit?.headers,
});
} catch (error) {
if (isLmstudioDiscoveryConfigResolutionError(error)) {
return null;
}
throw error;
}
const hasAuthorizationHeader = hasLmstudioAuthorizationHeader(resolvedHeaders);
let configuredDiscoveryApiKey: string | undefined;
try {
configuredDiscoveryApiKey = await resolveLmstudioConfiguredApiKey({
config: ctx.config,
env: ctx.env,
allowUnresolved: hasAuthorizationHeader || Boolean(discoveryApiKey),
});
} catch (error) {
if (isLmstudioDiscoveryConfigResolutionError(error)) {
return null;
}
throw error;
}
const resolvedDiscoveryApiKey = hasAuthorizationHeader
? undefined
: (discoveryApiKey ?? configuredDiscoveryApiKey);
// CLI/runtime-resolved key takes precedence over static provider config key.
const resolvedApiKey = apiKey ?? explicit?.apiKey;
if (hasExplicitModels && explicitWithoutHeaders) {
const persistedApiKey = resolvePersistedLmstudioApiKey({
currentApiKey: resolvedApiKey,
explicitAuth,
fallbackApiKey: LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
hasModels: hasExplicitModels,
hasAuthorizationHeader,
});
const persistedAuth = resolveLmstudioProviderAuthMode(persistedApiKey);
return {
provider: {
...explicitWithoutHeaders,
...(resolvedHeaders ? { headers: resolvedHeaders } : {}),
baseUrl: resolveLmstudioInferenceBase(explicitWithoutHeaders.baseUrl),
// Keep explicit API unless absent, then fall back to provider default.
api: explicitWithoutHeaders.api ?? "openai-completions",
...(persistedApiKey ? { apiKey: persistedApiKey } : {}),
...(persistedAuth ? { auth: persistedAuth } : {}),
models: explicitWithoutHeaders.models,
},
};
}
const provider = await discoverLmstudioProviderCatalog({
baseUrl: explicit?.baseUrl,
// Prefer resolved discovery auth, then configured provider auth.
apiKey: resolvedDiscoveryApiKey,
headers: resolvedHeaders,
quiet: !apiKey && !explicit && !resolvedDiscoveryApiKey,
});
const models = mergeDiscoveredModels({
explicitModels: explicit?.models,
discoveredModels: provider.models,
});
if (models.length === 0 && !apiKey && !explicit?.apiKey) {
return null;
}
const persistedApiKey = resolvePersistedLmstudioApiKey({
currentApiKey: resolvedApiKey,
explicitAuth,
fallbackApiKey: LMSTUDIO_DEFAULT_API_KEY_ENV_VAR,
hasModels: models.length > 0,
hasAuthorizationHeader,
});
const persistedAuth = resolveLmstudioProviderAuthMode(persistedApiKey);
return {
provider: {
...provider,
...explicitWithoutHeaders,
...(resolvedHeaders ? { headers: resolvedHeaders } : {}),
baseUrl: resolveLmstudioInferenceBase(explicit?.baseUrl ?? provider.baseUrl),
...(persistedApiKey ? { apiKey: persistedApiKey } : {}),
...(persistedAuth ? { auth: persistedAuth } : {}),
models,
},
};
}
export async function prepareLmstudioDynamicModels(
ctx: ProviderPrepareDynamicModelContext,
): Promise<ProviderRuntimeModel[]> {
const baseUrl = resolveLmstudioInferenceBase(ctx.providerConfig?.baseUrl);
const { apiKey, headers } = await resolveLmstudioRequestContext({
config: ctx.config,
agentDir: ctx.agentDir,
env: process.env,
providerHeaders: ctx.providerConfig?.headers,
});
const discoveredModels = await discoverLmstudioModels({
baseUrl,
apiKey: apiKey ?? "",
headers,
quiet: true,
});
return discoveredModels.map((model) =>
Object.assign({}, model, {
provider: PROVIDER_ID,
api: ctx.providerConfig?.api ?? `openai-completions`,
baseUrl,
input: model.input.filter(
(entry): entry is "text" | "image" => entry === "text" || entry === "image",
),
}),
);
}

View File

@@ -0,0 +1,850 @@
// Lmstudio tests cover stream plugin behavior.
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
import { createAssistantMessageEventStream } from "openclaw/plugin-sdk/llm";
import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { resetLmstudioPreloadCooldownForTest, wrapLmstudioInferencePreload } from "./stream.js";
const ensureLmstudioModelLoadedMock = vi.hoisted(() => vi.fn());
const resolveLmstudioProviderHeadersMock = vi.hoisted(() =>
vi.fn(async (_params?: unknown) => undefined),
);
const resolveLmstudioRuntimeApiKeyMock = vi.hoisted(() =>
vi.fn(async (_params?: unknown) => undefined),
);
vi.mock("./models.fetch.js", async (importOriginal) => {
const actual = await importOriginal<typeof import("./models.fetch.js")>();
return {
...actual,
ensureLmstudioModelLoaded: (params: unknown) => ensureLmstudioModelLoadedMock(params),
};
});
vi.mock("./runtime.js", async (importOriginal) => {
const actual = await importOriginal<typeof import("./runtime.js")>();
return {
...actual,
resolveLmstudioProviderHeaders: (params: unknown) => resolveLmstudioProviderHeadersMock(params),
resolveLmstudioRuntimeApiKey: (params: unknown) => resolveLmstudioRuntimeApiKeyMock(params),
};
});
afterAll(() => {
vi.doUnmock("./models.fetch.js");
vi.doUnmock("./runtime.js");
vi.resetModules();
});
type StreamEvent = { type: string } & Record<string, unknown>;
function requireRecord(value: unknown, label: string): Record<string, unknown> {
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error(`expected ${label} to be a record`);
}
return value as Record<string, unknown>;
}
function expectRecordFields(record: Record<string, unknown>, fields: Record<string, unknown>) {
for (const [key, value] of Object.entries(fields)) {
expect(record[key]).toEqual(value);
}
}
function expectSingleDoneEvent(events: StreamEvent[]) {
expect(events).toHaveLength(1);
expect(events[0]?.type).toBe("done");
}
function requireMockCallArg(mock: { mock: { calls: unknown[][] } }, label: string) {
const call = mock.mock.calls[0];
if (!call) {
throw new Error(`expected ${label} call`);
}
return call;
}
function expectEnsureLoadedFields(fields: Record<string, unknown>) {
const [params] = requireMockCallArg(ensureLmstudioModelLoadedMock, "ensureLmstudioModelLoaded");
const record = requireRecord(params, "ensureLmstudioModelLoaded params");
for (const [key, value] of Object.entries(fields)) {
if (key === "ssrfPolicy") {
expectRecordFields(
requireRecord(record.ssrfPolicy, "ssrfPolicy"),
value as Record<string, unknown>,
);
} else {
expect(record[key]).toEqual(value);
}
}
}
function expectBaseStreamModelFields(baseStream: StreamFn, fields: Record<string, unknown>) {
const call = requireMockCallArg(
baseStream as unknown as { mock: { calls: unknown[][] } },
"base stream",
);
expectRecordFields(requireRecord(call[0], "base stream model"), fields);
if (call[1] === undefined) {
throw new Error("Expected base stream context");
}
expect(call[2]).toBeUndefined();
}
function expectBaseStreamCallModelFields(
baseStream: StreamFn,
callIndex: number,
fields: Record<string, unknown>,
) {
const call = (baseStream as unknown as { mock: { calls: unknown[][] } }).mock.calls[callIndex];
if (!call) {
throw new Error(`expected base stream call ${callIndex}`);
}
expectRecordFields(requireRecord(call[0], "base stream model"), fields);
}
async function collectEvents(stream: ReturnType<StreamFn>): Promise<StreamEvent[]> {
const resolved = stream instanceof Promise ? await stream : stream;
const events: StreamEvent[] = [];
for await (const event of resolved) {
events.push(event as StreamEvent);
}
return events;
}
function buildDoneStreamFn(): StreamFn {
return vi.fn((_model, _context, _options) => {
const stream = createAssistantMessageEventStream();
queueMicrotask(() => {
stream.push({ type: "done", reason: "stop", message: {} as never });
stream.end();
});
return stream;
});
}
function buildEventStreamFn(events: unknown[]): StreamFn {
return vi.fn((_model, _context, _options) => {
const stream = createAssistantMessageEventStream();
queueMicrotask(() => {
for (const event of events) {
stream.push(event as never);
}
stream.end();
});
return stream;
});
}
function createWrappedLmstudioStream(
baseStream: StreamFn,
params?: { baseUrl?: string; thinkingLevel?: string },
): StreamFn {
return wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: params?.baseUrl ?? "http://localhost:1234",
models: [],
},
},
},
},
streamFn: baseStream,
thinkingLevel: params?.thinkingLevel,
} as never);
}
function buildPayloadStreamFn(payload: Record<string, unknown>): StreamFn {
return vi.fn((model, _context, options) => {
const stream = createAssistantMessageEventStream();
queueMicrotask(() => {
options?.onPayload?.(payload, model);
stream.push({ type: "done", reason: "stop", message: {} as never });
stream.end();
});
return stream;
});
}
const BINARY_REASONING_COMPAT = {
supportedReasoningEfforts: ["none", "minimal", "low", "medium", "high", "xhigh"],
reasoningEffortMap: { off: "none", none: "none", adaptive: "xhigh", max: "xhigh" },
};
function runWrappedLmstudioStream(
wrapped: StreamFn,
model: Record<string, unknown>,
options?: Record<string, unknown>,
context?: Record<string, unknown>,
) {
return wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "lmstudio/qwen3-8b-instruct",
...model,
} as never,
{ messages: [], ...context } as never,
options as never,
);
}
describe("lmstudio stream wrapper", () => {
beforeEach(() => {
resetLmstudioPreloadCooldownForTest();
});
afterEach(() => {
vi.restoreAllMocks();
ensureLmstudioModelLoadedMock.mockReset();
resolveLmstudioProviderHeadersMock.mockReset();
resolveLmstudioRuntimeApiKeyMock.mockReset();
resolveLmstudioProviderHeadersMock.mockResolvedValue(undefined);
resolveLmstudioRuntimeApiKeyMock.mockResolvedValue(undefined);
resetLmstudioPreloadCooldownForTest();
});
it("preloads LM Studio model before inference using model context window", async () => {
const baseStream = buildDoneStreamFn();
const wrapped = createWrappedLmstudioStream(baseStream, {
baseUrl: "http://lmstudio.internal:1234/v1",
});
const stream = runWrappedLmstudioStream(
wrapped,
{ contextWindow: 131072 },
{ apiKey: "lmstudio-token" },
);
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
expectEnsureLoadedFields({
baseUrl: "http://lmstudio.internal:1234/v1",
modelKey: "qwen3-8b-instruct",
requestedContextLength: 131072,
apiKey: "lmstudio-token",
ssrfPolicy: { allowedHostnames: ["lmstudio.internal"] },
});
});
it("streams with the canonical model key returned by preload", async () => {
ensureLmstudioModelLoadedMock.mockResolvedValueOnce("gemma-4-e4b-it-ultra-uncensored-heretic");
const baseStream = buildDoneStreamFn();
const wrapped = createWrappedLmstudioStream(baseStream);
const variantKey = "gemma-4-e4b-it-ultra-uncensored-heretic@q4_k_m";
const stream = runWrappedLmstudioStream(wrapped, { id: `lmstudio/${variantKey}` });
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expectEnsureLoadedFields({
modelKey: variantKey,
baseUrl: "http://localhost:1234/v1",
});
expectBaseStreamModelFields(baseStream, {
provider: "lmstudio",
id: "gemma-4-e4b-it-ultra-uncensored-heretic",
});
});
it("prefers model contextTokens over contextWindow for preload requests", async () => {
const baseStream = buildDoneStreamFn();
const wrapped = createWrappedLmstudioStream(baseStream, {
baseUrl: "http://lmstudio.internal:1234/v1",
});
const stream = runWrappedLmstudioStream(
wrapped,
{ contextWindow: 131072, contextTokens: 64000 },
{ apiKey: "lmstudio-token" },
);
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
expectEnsureLoadedFields({
baseUrl: "http://lmstudio.internal:1234/v1",
modelKey: "qwen3-8b-instruct",
requestedContextLength: 64000,
apiKey: "lmstudio-token",
ssrfPolicy: { allowedHostnames: ["lmstudio.internal"] },
});
});
it("omits malformed preload context lengths", async () => {
const baseStream = buildDoneStreamFn();
const wrapped = createWrappedLmstudioStream(baseStream, {
baseUrl: "http://lmstudio.internal:1234/v1",
});
const stream = runWrappedLmstudioStream(
wrapped,
{
contextTokens: 64000.5,
contextWindow: Number.POSITIVE_INFINITY,
},
{ apiKey: "lmstudio-token" },
);
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
expectEnsureLoadedFields({
baseUrl: "http://lmstudio.internal:1234/v1",
modelKey: "qwen3-8b-instruct",
requestedContextLength: undefined,
apiKey: "lmstudio-token",
ssrfPolicy: { allowedHostnames: ["lmstudio.internal"] },
});
});
it("continues inference when preload fails", async () => {
ensureLmstudioModelLoadedMock.mockRejectedValueOnce(new Error("load failed"));
const baseStream = buildDoneStreamFn();
const wrapped = wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234",
models: [],
},
},
},
},
streamFn: baseStream,
} as never);
const stream = wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
} as never,
{ messages: [] } as never,
undefined as never,
);
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expect(baseStream).toHaveBeenCalledTimes(1);
});
it("streams with the canonical model key when preload fails after discovery", async () => {
ensureLmstudioModelLoadedMock.mockRejectedValueOnce(
Object.assign(new Error("load failed"), {
resolvedModelKey: "gemma-4-e4b-it-ultra-uncensored-heretic",
}),
);
const baseStream = buildDoneStreamFn();
const wrapped = createWrappedLmstudioStream(baseStream);
const stream = runWrappedLmstudioStream(wrapped, {
id: "lmstudio/gemma-4-e4b-it-ultra-uncensored-heretic@q4_k_m",
});
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expect(baseStream).toHaveBeenCalledTimes(1);
expectBaseStreamModelFields(baseStream, {
provider: "lmstudio",
id: "gemma-4-e4b-it-ultra-uncensored-heretic",
});
});
it("reuses the canonical model key while preload failure cooldown is active", async () => {
const canonicalKey = "gemma-4-e4b-it-ultra-uncensored-heretic";
const variantModel = {
id: `lmstudio/${canonicalKey}@q4_k_m`,
};
ensureLmstudioModelLoadedMock.mockRejectedValueOnce(
Object.assign(new Error("load failed"), {
resolvedModelKey: canonicalKey,
}),
);
const baseStream = buildDoneStreamFn();
const wrapped = createWrappedLmstudioStream(baseStream);
const firstEvents = await collectEvents(runWrappedLmstudioStream(wrapped, variantModel));
const secondEvents = await collectEvents(runWrappedLmstudioStream(wrapped, variantModel));
expectSingleDoneEvent(firstEvents);
expectSingleDoneEvent(secondEvents);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
expect(baseStream).toHaveBeenCalledTimes(2);
expectBaseStreamCallModelFields(baseStream, 0, {
provider: "lmstudio",
id: canonicalKey,
});
expectBaseStreamCallModelFields(baseStream, 1, {
provider: "lmstudio",
id: canonicalKey,
});
});
it("skips native model preload when provider params disable it", async () => {
const baseStream = buildDoneStreamFn();
const wrapped = wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234",
params: { preload: false },
models: [],
},
},
},
},
streamFn: baseStream,
} as never);
const events = await collectEvents(
wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
} as never,
{ messages: [] } as never,
undefined as never,
),
);
expectSingleDoneEvent(events);
expect(ensureLmstudioModelLoadedMock).not.toHaveBeenCalled();
expect(baseStream).toHaveBeenCalledTimes(1);
const [model] = requireMockCallArg(
baseStream as unknown as { mock: { calls: unknown[][] } },
"base stream",
);
expectRecordFields(requireRecord(requireRecord(model, "base stream model").compat, "compat"), {
supportsUsageInStreaming: true,
});
});
it("dedupes concurrent preload requests for the same model and context", async () => {
let resolvePreload: (() => void) | undefined;
ensureLmstudioModelLoadedMock.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
resolvePreload = resolve;
}),
);
const baseStream = buildDoneStreamFn();
const wrapped = wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234",
models: [],
},
},
},
},
streamFn: baseStream,
} as never);
const first = wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
contextWindow: 32768,
} as never,
{ messages: [] } as never,
undefined as never,
);
const second = wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
contextWindow: 32768,
} as never,
{ messages: [] } as never,
undefined as never,
);
const firstPromise = collectEvents(first);
const secondPromise = collectEvents(second);
await vi.waitFor(() => {
if (!resolvePreload) {
throw new Error("LM Studio preload resolver not initialized");
}
});
if (!resolvePreload) {
throw new Error("LM Studio preload resolver not initialized");
}
resolvePreload();
const [firstEvents, secondEvents] = await Promise.all([firstPromise, secondPromise]);
expectSingleDoneEvent(firstEvents);
expectSingleDoneEvent(secondEvents);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
});
it("skips preload on the second attempt while the failure backoff is active", async () => {
ensureLmstudioModelLoadedMock.mockRejectedValue(new Error("out of memory"));
const baseStream = buildDoneStreamFn();
const wrapped = wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234",
models: [],
},
},
},
},
streamFn: baseStream,
} as never);
const firstEvents = await collectEvents(
wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
} as never,
{ messages: [] } as never,
undefined as never,
),
);
expectSingleDoneEvent(firstEvents);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
const secondEvents = await collectEvents(
wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
} as never,
{ messages: [] } as never,
undefined as never,
),
);
expectSingleDoneEvent(secondEvents);
// The second call must NOT retry preload because cooldown is active, but
// the underlying stream must still run so the user gets a response.
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
expect(baseStream).toHaveBeenCalledTimes(2);
});
it("retries preload once the cooldown expires", async () => {
ensureLmstudioModelLoadedMock.mockRejectedValueOnce(new Error("out of memory"));
ensureLmstudioModelLoadedMock.mockResolvedValueOnce(undefined);
const baseStream = buildDoneStreamFn();
const wrapped = wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234",
models: [],
},
},
},
},
streamFn: baseStream,
} as never);
// Freeze Date.now at a known base so we can jump past the first backoff
// window (5s by default) between the two preload attempts.
const baseTime = 1_000_000;
const nowSpy = vi.spyOn(Date, "now");
nowSpy.mockReturnValue(baseTime);
await collectEvents(
wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
} as never,
{ messages: [] } as never,
undefined as never,
),
);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(1);
// Move the clock past the initial 5s cooldown window so the next call is
// allowed to retry preload.
nowSpy.mockReturnValue(baseTime + 6_000);
await collectEvents(
wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
} as never,
{ messages: [] } as never,
undefined as never,
),
);
expect(ensureLmstudioModelLoadedMock).toHaveBeenCalledTimes(2);
nowSpy.mockRestore();
});
it("forces supportsUsageInStreaming compat before calling the underlying stream", async () => {
const baseStream = buildDoneStreamFn();
const wrapped = wrapLmstudioInferencePreload({
provider: "lmstudio",
modelId: "qwen3-8b-instruct",
config: {
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234",
models: [],
},
},
},
},
streamFn: baseStream,
} as never);
const stream = wrapped(
{
provider: "lmstudio",
api: "openai-completions",
id: "qwen3-8b-instruct",
compat: { supportsDeveloperRole: false },
} as never,
{ messages: [] } as never,
undefined as never,
);
const events = await collectEvents(stream);
expectSingleDoneEvent(events);
expect(baseStream).toHaveBeenCalledTimes(1);
expectBaseStreamModelFields(baseStream, { provider: "lmstudio" });
const [model] = requireMockCallArg(
baseStream as unknown as { mock: { calls: unknown[][] } },
"base stream",
);
expectRecordFields(requireRecord(requireRecord(model, "base stream model").compat, "compat"), {
supportsDeveloperRole: false,
supportsUsageInStreaming: true,
});
});
it("promotes standalone bracketed local-model tool text to a structured tool call", async () => {
const rawToolText = [
"[mempalace_mempalace_search]",
'{"query":"codename","wing":"personal","room":"identities"}',
"[END_TOOL_REQUEST]",
].join("\n");
const baseStream = buildEventStreamFn([
{ type: "start", partial: { content: [] } },
{ type: "text_start", contentIndex: 0, partial: { content: [{ type: "text", text: "" }] } },
{ type: "text_delta", contentIndex: 0, delta: rawToolText },
{ type: "text_end", contentIndex: 0, content: rawToolText },
{
type: "done",
reason: "stop",
message: {
role: "assistant",
content: [{ type: "text", text: rawToolText }],
stopReason: "stop",
},
},
]);
const wrapped = createWrappedLmstudioStream(baseStream);
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, {}, undefined, {
tools: [
{
name: "mempalace_mempalace_search",
description: "Search MemPalace",
parameters: { type: "object", properties: {} },
},
],
}),
);
expect(events.map((event) => event.type)).toEqual([
"start",
"toolcall_start",
"toolcall_delta",
"done",
]);
const done = events.find((event) => event.type === "done") as {
message?: { content?: Array<Record<string, unknown>>; stopReason?: string };
reason?: string;
};
expect(done.reason).toBe("toolUse");
expect(done.message?.stopReason).toBe("toolUse");
const toolCall = requireRecord(done.message?.content?.[0], "tool call content");
expectRecordFields(toolCall, {
type: "toolCall",
name: "mempalace_mempalace_search",
arguments: { query: "codename", wing: "personal", room: "identities" },
});
expect(String(toolCall.id)).toMatch(/^call_[a-f0-9]{24}$/);
});
it("promotes standalone Harmony local-model tool text to a structured tool call", async () => {
const rawToolText =
'commentary to=read code {"path":"/path/to/file","line_start":1,"line_end":400}';
const baseStream = buildEventStreamFn([
{ type: "start", partial: { content: [] } },
{ type: "text_start", contentIndex: 0, partial: { content: [{ type: "text", text: "" }] } },
{ type: "text_delta", contentIndex: 0, delta: rawToolText },
{ type: "text_end", contentIndex: 0, content: rawToolText },
{
type: "done",
reason: "stop",
message: {
role: "assistant",
content: [{ type: "text", text: rawToolText }],
stopReason: "stop",
},
},
]);
const wrapped = createWrappedLmstudioStream(baseStream);
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, {}, undefined, {
tools: [{ name: "read", description: "Read", parameters: { type: "object" } }],
}),
);
expect(events.map((event) => event.type)).toEqual([
"start",
"toolcall_start",
"toolcall_delta",
"done",
]);
const done = events.find((event) => event.type === "done") as {
message?: { content?: Array<Record<string, unknown>>; stopReason?: string };
reason?: string;
};
expect(done.reason).toBe("toolUse");
expectRecordFields(requireRecord(done.message?.content?.[0], "tool call content"), {
type: "toolCall",
name: "read",
arguments: { path: "/path/to/file", line_start: 1, line_end: 400 },
});
});
it("passes through bracketed text when the tool is not registered", async () => {
const rawToolText = [
"[mempalace_mempalace_search]",
'{"query":"codename"}',
"[/mempalace_mempalace_search]",
].join("\n");
const baseStream = buildEventStreamFn([
{ type: "start", partial: { content: [] } },
{ type: "text_start", contentIndex: 0, partial: { content: [{ type: "text", text: "" }] } },
{ type: "text_delta", contentIndex: 0, delta: rawToolText },
{ type: "text_end", contentIndex: 0, content: rawToolText },
{
type: "done",
reason: "stop",
message: {
role: "assistant",
content: [{ type: "text", text: rawToolText }],
stopReason: "stop",
},
},
]);
const wrapped = createWrappedLmstudioStream(baseStream);
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, {}, undefined, {
tools: [{ name: "read", description: "Read", parameters: { type: "object" } }],
}),
);
expect(events.map((event) => event.type)).toEqual([
"start",
"text_start",
"text_delta",
"text_end",
"done",
]);
expectRecordFields(
requireRecord(
events.find((event) => event.type === "text_delta"),
"text delta",
),
{
delta: rawToolText,
},
);
});
it("rewrites reasoning_effort to the disabled effort when thinking is off", async () => {
const payload: Record<string, unknown> = {
model: "qwen3-8b-instruct",
reasoning_effort: "high",
};
const baseStream = buildPayloadStreamFn(payload);
const wrapped = createWrappedLmstudioStream(baseStream, { thinkingLevel: "off" });
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, { compat: BINARY_REASONING_COMPAT }),
);
expectSingleDoneEvent(events);
expect(payload.reasoning_effort).toBe("none");
});
it("drops reasoning_effort on thinking off when the model has no disabled effort", async () => {
const payload: Record<string, unknown> = {
model: "qwen3-8b-instruct",
reasoning_effort: "high",
};
const baseStream = buildPayloadStreamFn(payload);
const wrapped = createWrappedLmstudioStream(baseStream, { thinkingLevel: "off" });
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, {
compat: {
supportedReasoningEfforts: ["minimal", "low", "medium", "high", "xhigh"],
reasoningEffortMap: { adaptive: "xhigh", max: "xhigh" },
},
}),
);
expectSingleDoneEvent(events);
expect("reasoning_effort" in payload).toBe(false);
});
it("keeps reasoning_effort untouched for enabled thinking levels", async () => {
const payload: Record<string, unknown> = {
model: "qwen3-8b-instruct",
reasoning_effort: "high",
};
const baseStream = buildPayloadStreamFn(payload);
const wrapped = createWrappedLmstudioStream(baseStream, { thinkingLevel: "high" });
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, { compat: BINARY_REASONING_COMPAT }),
);
expectSingleDoneEvent(events);
expect(payload.reasoning_effort).toBe("high");
});
it("keeps reasoning_effort untouched without a thinking level", async () => {
const payload: Record<string, unknown> = {
model: "qwen3-8b-instruct",
reasoning_effort: "high",
};
const baseStream = buildPayloadStreamFn(payload);
const wrapped = createWrappedLmstudioStream(baseStream);
const events = await collectEvents(
runWrappedLmstudioStream(wrapped, { compat: BINARY_REASONING_COMPAT }),
);
expectSingleDoneEvent(events);
expect(payload.reasoning_effort).toBe("high");
});
});

View File

@@ -0,0 +1,326 @@
// Lmstudio plugin module implements stream behavior.
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
import { streamSimple } from "openclaw/plugin-sdk/llm";
import { createSubsystemLogger } from "openclaw/plugin-sdk/logging-core";
import type { ProviderWrapStreamFnContext } from "openclaw/plugin-sdk/plugin-entry";
import {
createOpenAICompatibleCompletionsThinkingOffWrapper,
createPlainTextToolCallCompatWrapper,
} from "openclaw/plugin-sdk/provider-stream-shared";
import { ssrfPolicyFromHttpBaseUrlAllowedHostname } from "openclaw/plugin-sdk/ssrf-runtime";
import { asPositiveSafeInteger } from "openclaw/plugin-sdk/string-coerce-runtime";
import { LMSTUDIO_PROVIDER_ID } from "./defaults.js";
import { ensureLmstudioModelLoaded } from "./models.fetch.js";
import { resolveLmstudioInferenceBase } from "./models.js";
import { resolveLmstudioProviderHeaders, resolveLmstudioRuntimeApiKey } from "./runtime.js";
const log = createSubsystemLogger("extensions/lmstudio/stream");
type StreamOptions = Parameters<StreamFn>[2];
type StreamModel = Parameters<StreamFn>[0];
const preloadInFlight = new Map<string, Promise<string | undefined>>();
/**
* Cooldown state for the LM Studio preload endpoint.
*
* Without this, every chat request would retry preload ~every 2s even when
* LM Studio has rejected the load (for example the memory guardrail will keep
* rejecting until the user adjusts the setting or frees RAM). That produced
* hundreds of `LM Studio inference preload failed` WARN lines per hour without
* actually helping the user. The cooldown applies an exponential backoff per
* preloadKey and, while the cooldown is active, the wrapper skips the preload
* step entirely and proceeds directly to streaming — the model is often
* already loaded from the user's LM Studio UI, so inference can succeed even
* when preload keeps being rejected.
*/
type PreloadCooldownEntry = {
untilMs: number;
consecutiveFailures: number;
resolvedModelKey?: string;
};
const preloadCooldown = new Map<string, PreloadCooldownEntry>();
const PRELOAD_BACKOFF_BASE_MS = 5_000;
const PRELOAD_BACKOFF_MAX_MS = 300_000;
function computePreloadBackoffMs(consecutiveFailures: number): number {
const exponent = Math.max(0, consecutiveFailures - 1);
const raw = PRELOAD_BACKOFF_BASE_MS * 2 ** exponent;
return Math.min(PRELOAD_BACKOFF_MAX_MS, raw);
}
function recordPreloadSuccess(preloadKey: string): void {
preloadCooldown.delete(preloadKey);
}
function recordPreloadFailure(
preloadKey: string,
now: number,
resolvedModelKey?: string,
): PreloadCooldownEntry {
const existing = preloadCooldown.get(preloadKey);
const consecutiveFailures = (existing?.consecutiveFailures ?? 0) + 1;
const persistedResolvedModelKey = resolvedModelKey ?? existing?.resolvedModelKey;
const entry: PreloadCooldownEntry = {
consecutiveFailures,
untilMs: now + computePreloadBackoffMs(consecutiveFailures),
...(persistedResolvedModelKey ? { resolvedModelKey: persistedResolvedModelKey } : {}),
};
preloadCooldown.set(preloadKey, entry);
return entry;
}
function isPreloadCoolingDown(preloadKey: string, now: number): PreloadCooldownEntry | undefined {
const entry = preloadCooldown.get(preloadKey);
if (!entry) {
return undefined;
}
if (entry.untilMs <= now) {
preloadCooldown.delete(preloadKey);
return undefined;
}
return entry;
}
/** Test-only hook for clearing preload cooldown state between cases. */
export function resetLmstudioPreloadCooldownForTest(): void {
preloadCooldown.clear();
preloadInFlight.clear();
}
function normalizeLmstudioModelKey(modelId: string): string {
const trimmed = modelId.trim();
if (trimmed.toLowerCase().startsWith("lmstudio/")) {
return trimmed.slice("lmstudio/".length).trim();
}
return trimmed;
}
function resolveRequestedContextLength(model: StreamModel): number | undefined {
const withContextTokens = model as StreamModel & { contextTokens?: unknown };
const contextTokens = asPositiveSafeInteger(withContextTokens.contextTokens);
if (contextTokens !== undefined) {
return contextTokens;
}
const contextWindow = asPositiveSafeInteger(model.contextWindow);
if (contextWindow !== undefined) {
return contextWindow;
}
return undefined;
}
function resolveModelHeaders(model: StreamModel): Record<string, string> | undefined {
if (!model.headers || typeof model.headers !== "object" || Array.isArray(model.headers)) {
return undefined;
}
return model.headers;
}
function toRecord(value: unknown): Record<string, unknown> | undefined {
return value && typeof value === "object" ? (value as Record<string, unknown>) : undefined;
}
function shouldPreloadLmstudioModels(value: unknown): boolean {
const providerConfig = toRecord(value);
const params = toRecord(providerConfig?.params);
return params?.preload !== false;
}
function withLmstudioUsageCompat(model: StreamModel): StreamModel {
return {
...model,
compat: {
...(model.compat && typeof model.compat === "object" ? model.compat : {}),
supportsUsageInStreaming: true,
},
};
}
function withLmstudioResolvedModelKey(
model: StreamModel,
resolvedModelKey: string | undefined,
): StreamModel {
if (!resolvedModelKey || model.id === resolvedModelKey) {
return model;
}
return {
...model,
id: resolvedModelKey,
};
}
function resolveLmstudioModelKeyFromError(error: unknown): string | undefined {
let current = error;
const seen = new Set<object>();
while (current && typeof current === "object" && !seen.has(current)) {
seen.add(current);
const record = current as { cause?: unknown; resolvedModelKey?: unknown };
const resolvedModelKey =
typeof record.resolvedModelKey === "string" ? record.resolvedModelKey.trim() : "";
if (resolvedModelKey) {
return resolvedModelKey;
}
current = record.cause;
}
return undefined;
}
function createPreloadKey(params: {
baseUrl: string;
modelKey: string;
requestedContextLength?: number;
}) {
return `${params.baseUrl}::${params.modelKey}::${params.requestedContextLength ?? "default"}`;
}
async function ensureLmstudioModelLoadedBestEffort(params: {
baseUrl: string;
modelKey: string;
requestedContextLength?: number;
options: StreamOptions;
ctx: ProviderWrapStreamFnContext;
modelHeaders?: Record<string, string>;
}): Promise<string> {
const providerConfig = params.ctx.config?.models?.providers?.[LMSTUDIO_PROVIDER_ID];
const providerHeaders = { ...providerConfig?.headers, ...params.modelHeaders };
const runtimeApiKey =
typeof params.options?.apiKey === "string" && params.options.apiKey.trim().length > 0
? params.options.apiKey.trim()
: undefined;
const headers = await resolveLmstudioProviderHeaders({
config: params.ctx.config,
headers: providerHeaders,
});
const configuredApiKey =
runtimeApiKey !== undefined
? undefined
: await resolveLmstudioRuntimeApiKey({
config: params.ctx.config,
agentDir: params.ctx.agentDir,
headers: providerHeaders,
});
return await ensureLmstudioModelLoaded({
baseUrl: params.baseUrl,
apiKey: runtimeApiKey ?? configuredApiKey,
headers,
ssrfPolicy: ssrfPolicyFromHttpBaseUrlAllowedHostname(params.baseUrl),
modelKey: params.modelKey,
requestedContextLength: params.requestedContextLength,
});
}
export function wrapLmstudioInferencePreload(ctx: ProviderWrapStreamFnContext): StreamFn {
const underlying = ctx.streamFn ?? streamSimple;
// LM Studio does not ride the shared OpenAI provider hook stack, so the
// thinking-level payload rewrite must be composed here: without it, thinking
// "off" leaves the transport's defaulted reasoning_effort (an enabled level)
// in requests to binary-thinking servers.
const streamWithThinkingLevel = createOpenAICompatibleCompletionsThinkingOffWrapper(
createPlainTextToolCallCompatWrapper(underlying),
ctx.thinkingLevel,
);
return (model, context, options) => {
if (model.provider !== LMSTUDIO_PROVIDER_ID) {
return underlying(model, context, options);
}
const modelKey = normalizeLmstudioModelKey(model.id);
if (!modelKey) {
return underlying(model, context, options);
}
const providerConfig = ctx.config?.models?.providers?.[LMSTUDIO_PROVIDER_ID];
if (!shouldPreloadLmstudioModels(providerConfig)) {
return streamWithThinkingLevel(withLmstudioUsageCompat(model), context, options);
}
const providerBaseUrl = providerConfig?.baseUrl;
const resolvedBaseUrl = resolveLmstudioInferenceBase(
typeof model.baseUrl === "string" ? model.baseUrl : providerBaseUrl,
);
const requestedContextLength = resolveRequestedContextLength(model);
const preloadKey = createPreloadKey({
baseUrl: resolvedBaseUrl,
modelKey,
requestedContextLength,
});
const cooldownEntry = isPreloadCoolingDown(preloadKey, Date.now());
const existing = preloadInFlight.get(preloadKey);
const preloadPromise: Promise<string | undefined> | undefined =
existing ??
(cooldownEntry
? undefined
: (() => {
const created = ensureLmstudioModelLoadedBestEffort({
baseUrl: resolvedBaseUrl,
modelKey,
requestedContextLength,
options,
ctx,
modelHeaders: resolveModelHeaders(model),
})
.then(
(resolvedModelKey) => {
recordPreloadSuccess(preloadKey);
return resolvedModelKey;
},
(error: unknown) => {
const resolvedModelKey = resolveLmstudioModelKeyFromError(error);
const entry = recordPreloadFailure(preloadKey, Date.now(), resolvedModelKey);
throw Object.assign(new Error("preload-failed"), {
cause: error,
consecutiveFailures: entry.consecutiveFailures,
cooldownMs: entry.untilMs - Date.now(),
resolvedModelKey,
});
},
)
.finally(() => {
preloadInFlight.delete(preloadKey);
});
preloadInFlight.set(preloadKey, created);
return created;
})());
return (async () => {
let resolvedModelKey: string | undefined;
if (preloadPromise) {
try {
resolvedModelKey = await preloadPromise;
} catch (error) {
const annotated = error as {
cause?: unknown;
consecutiveFailures?: number;
cooldownMs?: number;
};
resolvedModelKey = resolveLmstudioModelKeyFromError(error);
const cause = annotated.cause ?? error;
const failures = annotated.consecutiveFailures ?? 1;
const cooldownSec = Math.max(0, Math.round((annotated.cooldownMs ?? 0) / 1000));
log.warn(
`LM Studio inference preload failed for "${modelKey}" (${failures} consecutive failure${
failures === 1 ? "" : "s"
}, next preload attempt skipped for ~${cooldownSec}s); continuing without preload: ${String(cause)}`,
);
}
} else if (cooldownEntry) {
resolvedModelKey = cooldownEntry.resolvedModelKey;
log.debug(
`LM Studio inference preload for "${modelKey}" skipped while backoff active (${cooldownEntry.consecutiveFailures} prior failures)`,
);
}
// LM Studio uses OpenAI-compatible streaming usage payloads when requested via
// `stream_options.include_usage`. Force this compat flag at call time so usage
// reporting remains enabled even when catalog entries omitted compat metadata.
const streamModel = withLmstudioResolvedModelKey(model, resolvedModelKey);
const stream = streamWithThinkingLevel(
withLmstudioUsageCompat(streamModel),
context,
options,
);
const resolvedStream = stream instanceof Promise ? await stream : stream;
return resolvedStream;
})();
};
}