Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
27
qa/scenarios/ui/browser-talk-start-stop.yaml
Normal file
27
qa/scenarios/ui/browser-talk-start-stop.yaml
Normal file
@@ -0,0 +1,27 @@
|
||||
title: Browser realtime Talk start-stop evidence
|
||||
|
||||
scenario:
|
||||
id: browser-talk-start-stop
|
||||
surface: browser-control-ui-and-webchat
|
||||
category: browser-control-ui-and-webchat.browser-realtime-talk
|
||||
coverage:
|
||||
primary:
|
||||
- voice.browser-talk-start-stop-ui
|
||||
secondary:
|
||||
- ui.browser-talk-start-stop
|
||||
objective: Link deterministic browser realtime Talk transport start/stop coverage to UI maturity accounting.
|
||||
successCriteria:
|
||||
- Browser Talk starts the Google Live WebSocket transport and reaches listening state.
|
||||
- Stopped transports ignore late WebSocket events.
|
||||
- Active consult shutdown aborts the active run without reviving listening state.
|
||||
docsRefs:
|
||||
- docs/web/control-ui.md
|
||||
- docs/help/testing.md
|
||||
codeRefs:
|
||||
- ui/src/e2e/browser-talk-start-stop.e2e.test.ts
|
||||
- ui/src/test-helpers/control-ui-e2e.ts
|
||||
- ui/src/pages/chat/realtime-talk-google-live.test.ts
|
||||
execution:
|
||||
kind: playwright
|
||||
path: ui/src/e2e/browser-talk-start-stop.e2e.test.ts
|
||||
summary: Playwright coverage for browser realtime Talk start and stop through the Control UI.
|
||||
23
qa/scenarios/ui/control-ui-assistant-media-tickets.yaml
Normal file
23
qa/scenarios/ui/control-ui-assistant-media-tickets.yaml
Normal file
@@ -0,0 +1,23 @@
|
||||
title: Control UI assistant media ticket evidence
|
||||
|
||||
scenario:
|
||||
id: control-ui-assistant-media-tickets
|
||||
surface: browser-control-ui-and-webchat
|
||||
category: browser-control-ui-and-webchat.webchat-conversations
|
||||
coverage:
|
||||
primary:
|
||||
- ui.assistant-media-tickets
|
||||
objective: Link scoped assistant media ticket e2e coverage to Control UI maturity accounting.
|
||||
successCriteria:
|
||||
- Gateway metadata requests mint scoped assistant media tickets.
|
||||
- Assistant media fetches without a ticket are rejected.
|
||||
- Tickets cannot be replayed for a different source file.
|
||||
docsRefs:
|
||||
- docs/web/control-ui.md
|
||||
- docs/help/testing.md
|
||||
codeRefs:
|
||||
- src/gateway/control-ui-assistant-media.e2e.test.ts
|
||||
execution:
|
||||
kind: vitest
|
||||
path: src/gateway/control-ui-assistant-media.e2e.test.ts
|
||||
summary: Vitest e2e coverage for scoped Control UI assistant media tickets.
|
||||
19
qa/scenarios/ui/control-ui-chat-flow-playwright.yaml
Normal file
19
qa/scenarios/ui/control-ui-chat-flow-playwright.yaml
Normal file
@@ -0,0 +1,19 @@
|
||||
title: Control UI chat flow Playwright coverage
|
||||
|
||||
scenario:
|
||||
id: control-ui-chat-flow-playwright
|
||||
surface: control-ui
|
||||
coverage:
|
||||
primary:
|
||||
- ui.control
|
||||
objective: Link the Control UI chat-flow Playwright suite to the QA coverage inventory.
|
||||
successCriteria:
|
||||
- Playwright covers the hosted Control UI chat surface.
|
||||
docsRefs:
|
||||
- docs/web/control-ui.md
|
||||
codeRefs:
|
||||
- ui/src/e2e/chat-flow.e2e.test.ts
|
||||
execution:
|
||||
kind: playwright
|
||||
path: ui/src/e2e/chat-flow.e2e.test.ts
|
||||
summary: Playwright coverage for the Control UI chat flow.
|
||||
303
qa/scenarios/ui/control-ui-qa-channel-image-roundtrip.yaml
Normal file
303
qa/scenarios/ui/control-ui-qa-channel-image-roundtrip.yaml
Normal file
@@ -0,0 +1,303 @@
|
||||
title: Control UI plus qa-channel image roundtrip
|
||||
|
||||
scenario:
|
||||
id: control-ui-qa-channel-image-roundtrip
|
||||
surface: control-ui
|
||||
coverage:
|
||||
primary:
|
||||
- ui.control
|
||||
- ui.dashboard-auth-bootstrap
|
||||
secondary:
|
||||
- media.image-understanding
|
||||
- channels.qa-channel
|
||||
objective: Verify the embedded Control UI can observe a qa-channel-backed session while the fake channel injects text and image turns that the agent answers correctly.
|
||||
successCriteria:
|
||||
- Control UI opens directly on the target qa-channel session.
|
||||
- A text prompt delivered through qa-channel produces a correct outbound reply.
|
||||
- A later qa-channel image message produces a correct image-aware reply.
|
||||
- The Control UI transcript shows both transport-side prompts and both final answers.
|
||||
docsRefs:
|
||||
- docs/concepts/qa-e2e-automation.md
|
||||
- docs/channels/qa-channel.md
|
||||
codeRefs:
|
||||
- extensions/qa-lab/src/scenario-runtime-api.ts
|
||||
- extensions/qa-lab/src/suite.ts
|
||||
- extensions/qa-lab/src/web-runtime.ts
|
||||
- ui/src/ui/views/chat.ts
|
||||
gatewayRuntime:
|
||||
forwardHostHome: true
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Open the Control UI on a qa-channel session with the generic QA web driver, inject text and image turns through qa-channel, and verify the replies in both the transport log and the UI transcript.
|
||||
config:
|
||||
requiredChannelDriver: qa-channel
|
||||
conversationId: control-ui-e2e
|
||||
textPrompt: "Control UI bridge check. Marker exact marker: `ui bridge armed`"
|
||||
uiExpectedNeedle: ui bridge armed
|
||||
imagePrompt: "Image understanding check: describe the top and bottom colors in the attached image in one short sentence."
|
||||
imagePromptNeedle: image understanding check
|
||||
requiredColorGroups:
|
||||
- [red, scarlet, crimson]
|
||||
- [blue, azure, teal, cyan, aqua]
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: opens control ui on the qa-channel-backed session
|
||||
actions:
|
||||
- call: reset
|
||||
- call: waitForGatewayHealthy
|
||||
args:
|
||||
- ref: env
|
||||
- expr: liveTurnTimeoutMs(env, 60000)
|
||||
- call: waitForQaChannelReady
|
||||
args:
|
||||
- ref: env
|
||||
- expr: liveTurnTimeoutMs(env, 60000)
|
||||
- call: fetchJson
|
||||
saveAs: bootstrap
|
||||
args:
|
||||
- expr: "`${lab.baseUrl}/api/bootstrap`"
|
||||
- assert:
|
||||
expr: "Boolean(bootstrap.controlUiEmbeddedUrl)"
|
||||
message: qa-lab bootstrap did not expose controlUiEmbeddedUrl
|
||||
- set: uiSessionKey
|
||||
value:
|
||||
expr: "buildAgentSessionKey({ agentId: env.cfg.agents?.list?.find((agent) => agent.default)?.id ?? env.cfg.agents?.list?.[0]?.id ?? 'main', channel: 'qa-channel', accountId: 'default', peer: { kind: 'direct', id: config.conversationId }, dmScope: env.cfg.session?.dmScope, identityLinks: env.cfg.session?.identityLinks })"
|
||||
- set: controlUiChatUrl
|
||||
value:
|
||||
expr: "(() => { const url = new URL(`${env.gateway.baseUrl}/`); url.searchParams.set('session', uiSessionKey); url.hash = `token=${encodeURIComponent(env.gateway.token ?? '')}`; return url.toString(); })()"
|
||||
- call: webOpenPage
|
||||
saveAs: uiTab
|
||||
args:
|
||||
- url:
|
||||
ref: controlUiChatUrl
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 60000)
|
||||
- set: uiPageId
|
||||
value:
|
||||
expr: "uiTab.pageId"
|
||||
- call: webWait
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiPageId
|
||||
selector: openclaw-app
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForCondition
|
||||
saveAs: uiReadySnapshot
|
||||
args:
|
||||
- lambda:
|
||||
async: true
|
||||
expr: "await (async () => { const snapshot = await webSnapshot({ pageId: uiPageId, maxChars: 12000, timeoutMs: liveTurnTimeoutMs(env, 30000) }); const text = normalizeLowercaseStringOrEmpty(snapshot.text); return text.includes('ready to chat') ? snapshot : undefined; })()"
|
||||
- expr: liveTurnTimeoutMs(env, 45000)
|
||||
- 500
|
||||
catch:
|
||||
- call: webSnapshot
|
||||
saveAs: uiReadyFailureSnapshot
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiPageId
|
||||
maxChars: 12000
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 15000)
|
||||
- call: webEvaluate
|
||||
saveAs: uiReadyFailureState
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiPageId
|
||||
expression: "(() => { const app = document.querySelector('openclaw-app'); const resources = performance.getEntriesByType('resource').map((entry) => ({ name: entry.name, type: entry.initiatorType, duration: Math.round(entry.duration), transferSize: entry.transferSize, decodedBodySize: entry.decodedBodySize })); return { url: location.href, readyState: document.readyState, appDefined: Boolean(customElements.get('openclaw-app')), appState: app ? { sessionKey: app.sessionKey, settingsSessionKey: app.settings?.sessionKey, lastActiveSessionKey: app.settings?.lastActiveSessionKey, chatMessages: Array.isArray(app.chatMessages) ? app.chatMessages.length : null, chatLoading: app.chatLoading, lastError: app.lastError, connected: app.connected, tab: app.tab } : null, scripts: Array.from(document.scripts).map((script) => script.src || script.textContent?.slice(0, 80)), links: Array.from(document.querySelectorAll('link')).map((link) => link.href), resources, bodyHtml: document.body.innerHTML.slice(0, 400) }; })()"
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 15000)
|
||||
- throw:
|
||||
expr: "`control ui did not become ready. state=${JSON.stringify(uiReadyFailureState)} diagnostics=${JSON.stringify(uiReadyFailureSnapshot.diagnostics ?? [])} snapshot: ${uiReadyFailureSnapshot.text}`"
|
||||
- assert:
|
||||
expr: "Boolean(uiPageId)"
|
||||
message: control ui page was not available
|
||||
detailsExpr: "uiReadySnapshot.text"
|
||||
- name: text injected through qa-channel gets a correct transport reply
|
||||
actions:
|
||||
- set: firstInboundStartIndex
|
||||
value:
|
||||
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'inbound').length"
|
||||
- set: firstOutboundStartIndex
|
||||
value:
|
||||
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').length"
|
||||
- sendInbound:
|
||||
accountId: default
|
||||
conversation:
|
||||
id:
|
||||
expr: config.conversationId
|
||||
kind: direct
|
||||
senderId:
|
||||
expr: config.conversationId
|
||||
senderName: Control UI QA
|
||||
text:
|
||||
expr: config.textPrompt
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: uiOutbound
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.conversation.id === config.conversationId && normalizeLowercaseStringOrEmpty(candidate.text).includes(config.uiExpectedNeedle)"
|
||||
- expr: liveTurnTimeoutMs(env, 45000)
|
||||
- sinceIndex:
|
||||
ref: firstOutboundStartIndex
|
||||
- call: readRawQaSessionStore
|
||||
saveAs: rawSessionStore
|
||||
args:
|
||||
- ref: env
|
||||
- set: rawSessionStoreKeys
|
||||
value:
|
||||
expr: "Object.keys(rawSessionStore)"
|
||||
detailsExpr: "`${uiOutbound.text}\\nSTORE:${JSON.stringify(rawSessionStoreKeys)}`"
|
||||
- name: text injected through qa-channel renders in a fresh control ui load
|
||||
actions:
|
||||
- call: webOpenPage
|
||||
saveAs: uiAckTab
|
||||
args:
|
||||
- url:
|
||||
ref: controlUiChatUrl
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 60000)
|
||||
- set: uiAckPageId
|
||||
value:
|
||||
expr: "uiAckTab.pageId"
|
||||
- call: webWait
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiAckPageId
|
||||
selector: openclaw-app
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForCondition
|
||||
saveAs: uiAckSnapshot
|
||||
args:
|
||||
- lambda:
|
||||
async: true
|
||||
expr: "await (async () => { const snapshot = await webSnapshot({ pageId: uiAckPageId, maxChars: 12000, timeoutMs: liveTurnTimeoutMs(env, 30000) }); const text = normalizeLowercaseStringOrEmpty(snapshot.text); return text.includes(config.uiExpectedNeedle) && text.includes('control ui bridge check') ? snapshot : undefined; })()"
|
||||
- expr: liveTurnTimeoutMs(env, 45000)
|
||||
- 500
|
||||
catch:
|
||||
- call: webSnapshot
|
||||
saveAs: uiAckFailureSnapshot
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiAckPageId
|
||||
maxChars: 12000
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 15000)
|
||||
- call: webEvaluate
|
||||
saveAs: uiAckFailureState
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiAckPageId
|
||||
expression: "(() => { const app = document.querySelector('openclaw-app'); return app ? { sessionKey: app.sessionKey, settingsSessionKey: app.settings?.sessionKey, lastActiveSessionKey: app.settings?.lastActiveSessionKey, chatMessages: Array.isArray(app.chatMessages) ? app.chatMessages.length : null, chatLoading: app.chatLoading, lastError: app.lastError, connected: app.connected } : null; })()"
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 15000)
|
||||
- throw:
|
||||
expr: "`control ui text transcript missing after fresh load. state=${JSON.stringify(uiAckFailureState)} snapshot: ${uiAckFailureSnapshot.text}`"
|
||||
detailsExpr: "uiAckSnapshot.text"
|
||||
- name: image injected through qa-channel gets a correct transport reply
|
||||
actions:
|
||||
- set: secondOutboundStartIndex
|
||||
value:
|
||||
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').length"
|
||||
- sendInbound:
|
||||
accountId: default
|
||||
conversation:
|
||||
id:
|
||||
expr: config.conversationId
|
||||
kind: direct
|
||||
senderId:
|
||||
expr: config.conversationId
|
||||
senderName: Control UI QA
|
||||
text:
|
||||
expr: config.imagePrompt
|
||||
attachments:
|
||||
- kind: image
|
||||
mimeType: image/png
|
||||
fileName: red-top-blue-bottom.png
|
||||
altText: red on top blue on bottom
|
||||
contentBase64:
|
||||
expr: imageUnderstandingValidPngBase64
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: imageOutbound
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.conversation.id === config.conversationId && config.requiredColorGroups.every((group) => group.some((color) => normalizeLowercaseStringOrEmpty(candidate.text).includes(color)))"
|
||||
- expr: liveTurnTimeoutMs(env, 90000)
|
||||
- sinceIndex:
|
||||
ref: secondOutboundStartIndex
|
||||
catchAs: imageWaitError
|
||||
catch:
|
||||
- set: imageDebugRequests
|
||||
value:
|
||||
expr: "env.mock ? [...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))].slice(-16).map((request) => ({ plannedToolName: request.plannedToolName ?? null, prompt: String(request.prompt ?? '').slice(0, 260), allInputText: String(request.allInputText ?? '').slice(0, 260), imageInputCount: request.imageInputCount ?? null })) : []"
|
||||
- throw:
|
||||
expr: "`qa-channel image reply missing: ${imageWaitError?.message ?? imageWaitError}; outbound=${recentOutboundSummary(state, 8)} requests=${JSON.stringify(imageDebugRequests)}`"
|
||||
- set: missingColorGroup
|
||||
value:
|
||||
expr: "config.requiredColorGroups.find((group) => !group.some((color) => normalizeLowercaseStringOrEmpty(imageOutbound.text).includes(color)))"
|
||||
- assert:
|
||||
expr: "!missingColorGroup"
|
||||
message:
|
||||
expr: "`missing expected colors in image reply: ${imageOutbound.text}`"
|
||||
detailsExpr: "imageOutbound.text"
|
||||
- name: image injected through qa-channel renders in a fresh control ui load
|
||||
actions:
|
||||
- call: webOpenPage
|
||||
saveAs: uiImageTab
|
||||
args:
|
||||
- url:
|
||||
ref: controlUiChatUrl
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 60000)
|
||||
- set: uiImagePageId
|
||||
value:
|
||||
expr: "uiImageTab.pageId"
|
||||
- call: webWait
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiImagePageId
|
||||
selector: openclaw-app
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 45000)
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForCondition
|
||||
saveAs: uiImageSnapshot
|
||||
args:
|
||||
- lambda:
|
||||
async: true
|
||||
expr: "await (async () => { const snapshot = await webSnapshot({ pageId: uiImagePageId, maxChars: 12000, timeoutMs: liveTurnTimeoutMs(env, 30000) }); const text = normalizeLowercaseStringOrEmpty(snapshot.text); const hasPrompt = text.includes(config.imagePromptNeedle); const hasColors = config.requiredColorGroups.every((group) => group.some((color) => text.includes(color))); return hasPrompt && hasColors ? snapshot : undefined; })()"
|
||||
- expr: liveTurnTimeoutMs(env, 90000)
|
||||
- 500
|
||||
catch:
|
||||
- call: webSnapshot
|
||||
saveAs: uiImageFailureSnapshot
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiImagePageId
|
||||
maxChars: 12000
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 15000)
|
||||
- call: webEvaluate
|
||||
saveAs: uiImageFailureState
|
||||
args:
|
||||
- pageId:
|
||||
ref: uiImagePageId
|
||||
expression: "(() => { const app = document.querySelector('openclaw-app'); return app ? { sessionKey: app.sessionKey, settingsSessionKey: app.settings?.sessionKey, lastActiveSessionKey: app.settings?.lastActiveSessionKey, chatMessages: Array.isArray(app.chatMessages) ? app.chatMessages.length : null, chatLoading: app.chatLoading, lastError: app.lastError, connected: app.connected } : null; })()"
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 15000)
|
||||
- throw:
|
||||
expr: "`control ui image transcript missing after fresh load. state=${JSON.stringify(uiImageFailureState)} snapshot: ${uiImageFailureSnapshot.text}`"
|
||||
detailsExpr: "uiImageSnapshot.text"
|
||||
35
qa/scenarios/ui/ux-matrix-evidence-dashboard.yaml
Normal file
35
qa/scenarios/ui/ux-matrix-evidence-dashboard.yaml
Normal file
@@ -0,0 +1,35 @@
|
||||
title: UX Matrix evidence dashboard
|
||||
|
||||
scenario:
|
||||
id: ux-matrix-evidence-dashboard
|
||||
surface: control-ui
|
||||
category: browser-control-ui-and-webchat.browser-ui
|
||||
coverage:
|
||||
primary:
|
||||
- qa.artifact-safety
|
||||
secondary:
|
||||
- ui.control
|
||||
- gateway.control-ui-hosting
|
||||
- cli.entrypoint
|
||||
- cli.status-snapshots
|
||||
- tools.evidence
|
||||
- workspace.artifacts
|
||||
objective: Produce UX Matrix evidence artifacts through the QA Lab script producer contract.
|
||||
successCriteria:
|
||||
- The script producer writes manifest.json, matrix.json, release-ledger.json, scorecard.md, latest-run.json, and qa-evidence.json.
|
||||
- Evidence entries declare coverage IDs and artifact paths for generated logs, screenshot proof, and video-style proof.
|
||||
- Missing local browser or CLI prerequisites are represented as blocked evidence with logs instead of fake artifacts.
|
||||
docsRefs:
|
||||
- docs/concepts/qa-e2e-automation.md
|
||||
- docs/web/control-ui.md
|
||||
codeRefs:
|
||||
- scripts/qa/ux-matrix-evidence-producer.ts
|
||||
- extensions/qa-lab/src/test-file-scenario-runner.ts
|
||||
execution:
|
||||
kind: script
|
||||
path: scripts/qa/ux-matrix-evidence-producer.ts
|
||||
summary: Script-backed UX Matrix producer that emits QA Lab evidence artifacts.
|
||||
allowBlockedEvidence: true
|
||||
args:
|
||||
- --artifact-base
|
||||
- ${outputDir}
|
||||
Reference in New Issue
Block a user