Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
@@ -0,0 +1,36 @@
|
||||
title: Hosted image generation providers live
|
||||
|
||||
scenario:
|
||||
id: hosted-image-generation-providers-live
|
||||
surface: long-tail-hosted-providers
|
||||
category: long-tail-hosted-providers.hosted-media-providers
|
||||
runtimeParityTier: live-only
|
||||
coverage:
|
||||
primary:
|
||||
- hosted-providers.image-generation-providers
|
||||
objective: Verify hosted image generation providers can generate real images through the live provider runtime sweep.
|
||||
successCriteria:
|
||||
- The QA producer runs the canonical live media image harness instead of duplicating provider assertions.
|
||||
- At least one auth-available hosted image provider runs through `provider.generateImage`.
|
||||
- Generated image output includes image MIME type and non-trivial bytes.
|
||||
- Missing live provider credentials produce blocked evidence rather than false passing primary coverage.
|
||||
docsRefs:
|
||||
- docs/help/testing.md
|
||||
- docs/tools/image-generation.md
|
||||
- docs/tools/media-overview.md
|
||||
codeRefs:
|
||||
- test/e2e/qa-lab/media/hosted-media-provider-live.ts
|
||||
- test/image-generation.runtime.live.test.ts
|
||||
- src/image-generation/live-test-helpers.ts
|
||||
execution:
|
||||
kind: script
|
||||
path: test/e2e/qa-lab/media/hosted-media-provider-live.ts
|
||||
summary: Runs the existing hosted image-generation live provider sweep and emits QA Lab evidence.
|
||||
allowBlockedEvidence: true
|
||||
timeoutMs: 7200000
|
||||
args:
|
||||
- --qa-evidence
|
||||
- --suite
|
||||
- image
|
||||
- --artifact-base
|
||||
- ${outputDir}
|
||||
@@ -0,0 +1,41 @@
|
||||
title: Hosted video generation providers live
|
||||
|
||||
scenario:
|
||||
id: hosted-video-generation-providers-live
|
||||
surface: long-tail-hosted-providers
|
||||
category: long-tail-hosted-providers.hosted-media-providers
|
||||
runtimeParityTier: live-only
|
||||
coverage:
|
||||
primary:
|
||||
- hosted-providers.video-generation-providers
|
||||
- media.reference-image-video-and-audio-inputs
|
||||
secondary:
|
||||
- media.video-generation-tool-invocation
|
||||
objective: Verify hosted video generation providers can generate real videos and exercise full reference-input modes where supported.
|
||||
successCriteria:
|
||||
- The QA producer runs the canonical live media video harness instead of duplicating provider assertions.
|
||||
- At least one auth-available hosted video provider runs through `provider.generateVideo`.
|
||||
- Full live video modes are enabled so image-to-video and video-to-video reference inputs are attempted where provider capabilities allow them.
|
||||
- Generated video output includes video MIME type with bytes or an HTTP(S) URL.
|
||||
- Missing live provider credentials produce blocked evidence rather than false passing primary coverage.
|
||||
docsRefs:
|
||||
- docs/help/testing.md
|
||||
- docs/tools/video-generation.md
|
||||
- docs/tools/media-overview.md
|
||||
codeRefs:
|
||||
- test/e2e/qa-lab/media/hosted-media-provider-live.ts
|
||||
- extensions/video-generation-providers.live.test.ts
|
||||
- src/video-generation/runtime.ts
|
||||
- src/agents/tools/video-generate-tool.ts
|
||||
execution:
|
||||
kind: script
|
||||
path: test/e2e/qa-lab/media/hosted-media-provider-live.ts
|
||||
summary: Runs the existing hosted video-generation live provider sweep with full reference-input modes and emits QA Lab evidence.
|
||||
allowBlockedEvidence: true
|
||||
timeoutMs: 14400000
|
||||
args:
|
||||
- --qa-evidence
|
||||
- --suite
|
||||
- video
|
||||
- --artifact-base
|
||||
- ${outputDir}
|
||||
100
qa/scenarios/media/image-generation-roundtrip.yaml
Normal file
100
qa/scenarios/media/image-generation-roundtrip.yaml
Normal file
@@ -0,0 +1,100 @@
|
||||
title: Image generation roundtrip
|
||||
|
||||
scenario:
|
||||
id: image-generation-roundtrip
|
||||
surface: image-generation
|
||||
coverage:
|
||||
primary:
|
||||
- media.image-generation
|
||||
- media.image-generation-delivery
|
||||
secondary:
|
||||
- channels.qa-channel
|
||||
objective: Verify a generated image is saved as media, reattached on the next turn, and described correctly through the vision path.
|
||||
successCriteria:
|
||||
- image_generate produces a saved MEDIA artifact.
|
||||
- The generated artifact is reattached on a follow-up turn.
|
||||
- The follow-up vision answer describes the generated scene rather than a generic attachment placeholder.
|
||||
docsRefs:
|
||||
- docs/tools/image-generation.md
|
||||
- docs/help/testing.md
|
||||
codeRefs:
|
||||
- src/agents/tools/image-generate-tool.ts
|
||||
- src/gateway/chat-attachments.ts
|
||||
- extensions/qa-lab/src/mock-openai-server.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify a generated image is saved as media, reattached on the next turn, and described correctly through the vision path.
|
||||
config:
|
||||
requiredChannelDriver: qa-channel
|
||||
generatePrompt: "Image generation check: generate a QA lighthouse image and summarize it in one short sentence."
|
||||
generatePromptSnippet: "Image generation check"
|
||||
inspectPrompt: "Roundtrip image inspection check: describe the generated lighthouse attachment in one short sentence."
|
||||
expectedNeedle: "lighthouse"
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: reattaches the generated media artifact on the follow-up turn
|
||||
actions:
|
||||
- call: ensureImageGenerationConfigured
|
||||
args:
|
||||
- ref: env
|
||||
- call: createSession
|
||||
args:
|
||||
- ref: env
|
||||
- Image roundtrip
|
||||
- agent:qa:image-roundtrip
|
||||
- call: reset
|
||||
- set: generatedStartedAtMs
|
||||
value:
|
||||
expr: Date.now()
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey: agent:qa:image-roundtrip
|
||||
message:
|
||||
expr: config.generatePrompt
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- call: resolveGeneratedImagePath
|
||||
saveAs: mediaPath
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
promptSnippet:
|
||||
expr: config.generatePromptSnippet
|
||||
startedAtMs:
|
||||
ref: generatedStartedAtMs
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- call: fs.readFile
|
||||
saveAs: imageBuffer
|
||||
args:
|
||||
- ref: mediaPath
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey: agent:qa:image-roundtrip
|
||||
message:
|
||||
expr: config.inspectPrompt
|
||||
attachments:
|
||||
- mimeType: image/png
|
||||
fileName:
|
||||
expr: path.basename(mediaPath)
|
||||
content:
|
||||
expr: imageBuffer.toString('base64')
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- call: waitForCondition
|
||||
saveAs: outbound
|
||||
args:
|
||||
- lambda:
|
||||
expr: "state.getSnapshot().messages.filter((candidate) => candidate.direction === 'outbound' && candidate.conversation.id === 'qa-operator' && normalizeLowercaseStringOrEmpty(candidate.text).includes(normalizeLowercaseStringOrEmpty(config.expectedNeedle))).at(-1)"
|
||||
- expr: liveTurnTimeoutMs(env, 45000)
|
||||
- assert:
|
||||
expr: "!env.mock || Boolean((await fetchJson(`${env.mock.baseUrl}/debug/requests`)).find((request) => request.plannedToolName === 'image_generate' && String(request.prompt ?? '').includes(config.generatePromptSnippet)))"
|
||||
message: expected image_generate call before roundtrip inspection
|
||||
- assert:
|
||||
expr: "!env.mock || (await fetchJson(`${env.mock.baseUrl}/debug/requests`)).some((request) => String(request.prompt ?? '').includes(config.inspectPrompt) && (request.imageInputCount ?? 0) >= 1)"
|
||||
message:
|
||||
expr: "`expected generated artifact to be reattached on follow-up turn; recentRequests=${JSON.stringify((await fetchJson(`${env.mock.baseUrl}/debug/requests`)).slice(-12).map((request) => ({ prompt: String(request.prompt ?? '').slice(0, 240), imageInputCount: request.imageInputCount, allInputText: String(request.allInputText ?? '').slice(0, 240) })))}`"
|
||||
detailsExpr: "`MEDIA:${mediaPath}\\n${outbound.text}`"
|
||||
92
qa/scenarios/media/image-understanding-attachment.yaml
Normal file
92
qa/scenarios/media/image-understanding-attachment.yaml
Normal file
@@ -0,0 +1,92 @@
|
||||
title: Image understanding from attachment
|
||||
|
||||
scenario:
|
||||
id: image-understanding-attachment
|
||||
surface: image-understanding
|
||||
coverage:
|
||||
primary:
|
||||
- media.image-understanding
|
||||
secondary:
|
||||
- channels.qa-channel
|
||||
objective: Verify an attached image reaches the agent model and the agent can describe what it sees.
|
||||
successCriteria:
|
||||
- Agent receives at least one image attachment.
|
||||
- Final answer describes the visible image content in one short sentence.
|
||||
- The description mentions the expected red and blue regions.
|
||||
docsRefs:
|
||||
- docs/help/testing.md
|
||||
- docs/tools/index.md
|
||||
codeRefs:
|
||||
- src/gateway/server-methods/agent.ts
|
||||
- extensions/qa-lab/src/suite.ts
|
||||
- extensions/qa-lab/src/mock-openai-server.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify an attached image reaches the agent model and the agent can describe what it sees.
|
||||
config:
|
||||
requiredChannelDriver: qa-channel
|
||||
prompt: "Image understanding check: describe the top and bottom colors in the attached image in one short sentence."
|
||||
requiredColorGroups:
|
||||
- [red, scarlet, crimson]
|
||||
- [blue, azure, teal, cyan, aqua]
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: describes an attached image in one short sentence
|
||||
actions:
|
||||
- call: reset
|
||||
- set: outboundStartIndex
|
||||
value:
|
||||
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').length"
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey: agent:qa:image-understanding
|
||||
message:
|
||||
expr: config.prompt
|
||||
attachments:
|
||||
- mimeType: image/png
|
||||
fileName: red-top-blue-bottom.png
|
||||
content:
|
||||
expr: imageUnderstandingValidPngBase64
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: outbound
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.conversation.id === 'qa-operator' && config.requiredColorGroups.every((group) => group.some((color) => normalizeLowercaseStringOrEmpty(candidate.text).includes(color)))"
|
||||
- expr: liveTurnTimeoutMs(env, 45000)
|
||||
- sinceIndex:
|
||||
ref: outboundStartIndex
|
||||
- set: missingColorGroup
|
||||
value:
|
||||
expr: "config.requiredColorGroups.find((group) => !group.some((candidate) => normalizeLowercaseStringOrEmpty(outbound.text).includes(candidate)))"
|
||||
- assert:
|
||||
expr: "!missingColorGroup"
|
||||
message:
|
||||
expr: "`missing expected colors in image description: ${outbound.text}`"
|
||||
# Image-processing assertion: verify the mock actually received an
|
||||
# image on the scenario-unique prompt. This is as strong as a
|
||||
# tool-call assertion for this scenario — unlike the
|
||||
# `source-docs-discovery-report` / `subagent-handoff` /
|
||||
# `config-restart-capability-flip` scenarios that rely on a real
|
||||
# tool call to satisfy the parity criterion, image understanding
|
||||
# is handled inside the provider's vision capability and does NOT
|
||||
# emit a tool call the mock can record as `plannedToolName`. The
|
||||
# `imageInputCount` field IS the tool-call evidence for vision
|
||||
# scenarios: it proves the attachment reached the provider, which
|
||||
# is the only thing an external harness can verify in mock mode.
|
||||
# Match on the scenario-unique prompt substring so the assertion
|
||||
# can't be accidentally satisfied by some other scenario's image
|
||||
# request that happens to share a debug log with this one.
|
||||
- set: imageRequest
|
||||
value:
|
||||
expr: "env.mock ? [...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))].find((request) => String(request.prompt ?? '').includes('Image understanding check')) : null"
|
||||
- assert:
|
||||
expr: "!env.mock || (imageRequest && (imageRequest.imageInputCount ?? 0) >= 1)"
|
||||
message:
|
||||
expr: "`expected at least one input image on the Image understanding check request, got imageInputCount=${String(imageRequest?.imageInputCount ?? 0)}`"
|
||||
detailsExpr: outbound.text
|
||||
88
qa/scenarios/media/native-image-generation.yaml
Normal file
88
qa/scenarios/media/native-image-generation.yaml
Normal file
@@ -0,0 +1,88 @@
|
||||
title: Native image generation
|
||||
|
||||
scenario:
|
||||
id: native-image-generation
|
||||
surface: image-generation
|
||||
coverage:
|
||||
primary:
|
||||
- media.image-generation
|
||||
secondary:
|
||||
- tools.native-image-generation
|
||||
objective: Verify image_generate appears when configured and returns a real saved media artifact.
|
||||
successCriteria:
|
||||
- image_generate appears in the effective tool inventory.
|
||||
- Agent triggers native image_generate.
|
||||
- Tool output returns a saved MEDIA path and the file exists.
|
||||
docsRefs:
|
||||
- docs/tools/image-generation.md
|
||||
- docs/providers/openai.md
|
||||
codeRefs:
|
||||
- src/agents/tools/image-generate-tool.ts
|
||||
- extensions/qa-lab/src/mock-openai-server.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify image_generate appears when configured and returns a real saved media artifact.
|
||||
config:
|
||||
requiredChannelDriver: qa-channel
|
||||
prompt: "Image generation check: generate a QA lighthouse image and summarize it in one short sentence."
|
||||
promptSnippet: "Image generation check"
|
||||
generatedNeedle: "QA lighthouse"
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: enables image_generate and saves a real media artifact
|
||||
actions:
|
||||
- call: ensureImageGenerationConfigured
|
||||
args:
|
||||
- ref: env
|
||||
- call: createSession
|
||||
saveAs: sessionKey
|
||||
args:
|
||||
- ref: env
|
||||
- Image generation
|
||||
- call: readEffectiveTools
|
||||
saveAs: tools
|
||||
args:
|
||||
- ref: env
|
||||
- ref: sessionKey
|
||||
- assert:
|
||||
expr: "tools.has('image_generate')"
|
||||
message: image_generate not present after imageGenerationModel patch
|
||||
- call: reset
|
||||
- set: generationStartedAt
|
||||
value:
|
||||
expr: Date.now()
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey: agent:qa:image-generate
|
||||
message:
|
||||
expr: config.prompt
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: outbound
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.conversation.id === 'qa-operator'"
|
||||
- expr: liveTurnTimeoutMs(env, 45000)
|
||||
- assert:
|
||||
expr: "!env.mock || ((await fetchJson(`${env.mock.baseUrl}/debug/requests`)).find((request) => String(request.allInputText ?? '').includes(config.promptSnippet))?.plannedToolName === 'image_generate')"
|
||||
message:
|
||||
expr: "`expected image_generate, got ${String((await fetchJson(`${env.mock.baseUrl}/debug/requests`)).find((request) => String(request.allInputText ?? '').includes(config.promptSnippet))?.plannedToolName ?? '')}`"
|
||||
- call: resolveGeneratedImagePath
|
||||
saveAs: generatedPath
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
promptSnippet:
|
||||
expr: config.promptSnippet
|
||||
startedAtMs:
|
||||
ref: generationStartedAt
|
||||
timeoutMs: 15000
|
||||
- assert:
|
||||
expr: "typeof generatedPath === 'string' && generatedPath.length > 0"
|
||||
message: image generation did not produce a saved media path
|
||||
detailsExpr: "`${outbound.text}\\nIMAGE_PATH:${generatedPath}`"
|
||||
29
qa/scenarios/media/webchat-auto-tts.yaml
Normal file
29
qa/scenarios/media/webchat-auto-tts.yaml
Normal file
@@ -0,0 +1,29 @@
|
||||
title: WebChat auto TTS delivery
|
||||
|
||||
scenario:
|
||||
id: webchat-auto-tts
|
||||
surface: media-understanding-and-media-generation
|
||||
category: media-understanding-and-media-generation.text-to-speech-delivery
|
||||
coverage:
|
||||
secondary:
|
||||
- media.tts
|
||||
- media.outbound-voice-audio-delivery
|
||||
objective: Verify WebChat auto-TTS synthesizes only the final reply tail and serves trusted local audio through scoped browser media tickets.
|
||||
successCriteria:
|
||||
- WebChat block delivery does not synthesize intermediate TTS audio.
|
||||
- WebChat final delivery synthesizes one local TTS audio file with spoken text metadata.
|
||||
- Trusted local TTS media becomes a WebChat audio attachment while untrusted local paths are rejected.
|
||||
- The real Gateway HTTP route mints a scoped ticket and serves the synthesized audio only with that ticket.
|
||||
docsRefs:
|
||||
- docs/tools/tts.md
|
||||
- docs/tools/media-overview.md
|
||||
- docs/concepts/qa-e2e-automation.md
|
||||
codeRefs:
|
||||
- packages/speech-core/src/tts.ts
|
||||
- src/gateway/server-methods/chat-webchat-media.ts
|
||||
- src/gateway/control-ui.ts
|
||||
- test/e2e/qa-lab/media/webchat-auto-tts.e2e.test.ts
|
||||
execution:
|
||||
kind: vitest
|
||||
path: test/e2e/qa-lab/media/webchat-auto-tts.e2e.test.ts
|
||||
summary: Vitest QA Lab coverage for mock WebChat TTS synthesis and real scoped media-ticket delivery.
|
||||
Reference in New Issue
Block a user