Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
30
qa/scenarios/config/cli-channel-picker.yaml
Normal file
30
qa/scenarios/config/cli-channel-picker.yaml
Normal file
@@ -0,0 +1,30 @@
|
||||
title: CLI channel picker
|
||||
|
||||
scenario:
|
||||
id: cli-channel-picker
|
||||
surface: cli-install-update-onboard-doctor
|
||||
category: cli-install-update-onboard-doctor.plugin-and-channel-setup
|
||||
coverage:
|
||||
primary:
|
||||
- cli.channel-picker
|
||||
objective: Drive the real interactive channel picker in an isolated OpenClaw home and verify its persisted configuration.
|
||||
successCriteria:
|
||||
- The producer launches the compiled OpenClaw CLI through a real PTY with an isolated OPENCLAW_HOME.
|
||||
- The interaction selects Telegram, enters a test token, finishes the picker, and exits successfully.
|
||||
- The written config enables the Telegram plugin and channel, preserves the default group mention gate, and records configure wizard metadata.
|
||||
- Missing final output, a stalled picker, non-zero exit, or mismatched configuration produces failed evidence.
|
||||
docsRefs:
|
||||
- docs/channels/telegram.md
|
||||
- docs/help/testing.md
|
||||
codeRefs:
|
||||
- test/e2e/qa-lab/config/cli-channel-picker.ts
|
||||
- scripts/e2e/lib/run-with-pty.mjs
|
||||
- src/flows/channel-setup.ts
|
||||
execution:
|
||||
kind: script
|
||||
path: test/e2e/qa-lab/config/cli-channel-picker.ts
|
||||
summary: Spawns the real channel picker under an isolated home, drives it through a PTY, asserts the config write, and emits QA evidence.
|
||||
timeoutMs: 180000
|
||||
args:
|
||||
- --artifact-base
|
||||
- ${outputDir}
|
||||
118
qa/scenarios/config/config-apply-restart-wakeup.yaml
Normal file
118
qa/scenarios/config/config-apply-restart-wakeup.yaml
Normal file
@@ -0,0 +1,118 @@
|
||||
title: Config apply restart wake-up
|
||||
|
||||
scenario:
|
||||
id: config-apply-restart-wakeup
|
||||
surface: config
|
||||
coverage:
|
||||
primary:
|
||||
- config.restart-apply
|
||||
secondary:
|
||||
- runtime.gateway-restart
|
||||
objective: Verify a restart-required config.apply restarts cleanly and delivers the post-restart wake message back into the QA channel.
|
||||
successCriteria:
|
||||
- config.apply schedules a restart-required change.
|
||||
- Gateway becomes healthy again after restart.
|
||||
- Restart sentinel wake-up message arrives in the QA channel.
|
||||
docsRefs:
|
||||
- docs/gateway/configuration.md
|
||||
- docs/gateway/protocol.md
|
||||
codeRefs:
|
||||
- src/gateway/server-methods/config.ts
|
||||
- src/gateway/server-restart-sentinel.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify a restart-required config.apply restarts cleanly and delivers the post-restart wake message back into the QA channel.
|
||||
config:
|
||||
requiredChannelDriver: qa-channel
|
||||
channelId: qa-room
|
||||
announcePrompt: "Acknowledge restart wake-up setup in qa-room."
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: restarts cleanly and posts the restart sentinel back into qa-channel
|
||||
actions:
|
||||
- call: reset
|
||||
- set: sessionKey
|
||||
value:
|
||||
expr: "buildAgentSessionKey({ agentId: 'qa', channel: 'qa-channel', peer: { kind: 'channel', id: config.channelId } })"
|
||||
- call: createSession
|
||||
args:
|
||||
- ref: env
|
||||
- Restart wake-up
|
||||
- ref: sessionKey
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey:
|
||||
ref: sessionKey
|
||||
to:
|
||||
expr: "`channel:${config.channelId}`"
|
||||
message:
|
||||
expr: config.announcePrompt
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, 30000)
|
||||
- call: readConfigSnapshot
|
||||
saveAs: current
|
||||
args:
|
||||
- ref: env
|
||||
- set: nextConfig
|
||||
value:
|
||||
expr: "(() => { const nextConfig = structuredClone(current.config); const gatewayConfig = (nextConfig.gateway ??= {}); const controlUi = (gatewayConfig.controlUi ??= {}); const allowedOrigins = Array.isArray(controlUi.allowedOrigins) ? [...controlUi.allowedOrigins] : []; if (!allowedOrigins.includes('http://127.0.0.1:65535')) allowedOrigins.push('http://127.0.0.1:65535'); controlUi.allowedOrigins = allowedOrigins; return nextConfig; })()"
|
||||
- set: wakeMarker
|
||||
value:
|
||||
expr: "`QA-RESTART-${randomUUID().slice(0, 8)}`"
|
||||
- set: wakeStartIndex
|
||||
value:
|
||||
expr: "state.getSnapshot().messages.length"
|
||||
- call: applyConfig
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
nextConfig:
|
||||
ref: nextConfig
|
||||
sessionKey:
|
||||
ref: sessionKey
|
||||
deliveryContext:
|
||||
expr: "({ channel: 'qa-channel', to: `channel:${config.channelId}` })"
|
||||
note:
|
||||
ref: wakeMarker
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForGatewayHealthy
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
catchAs: healthyError
|
||||
catch:
|
||||
- throw:
|
||||
message:
|
||||
expr: "`gateway never returned healthy after config.apply: ${formatErrorMessage(healthyError)}`"
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForQaChannelReady
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
catchAs: readyError
|
||||
catch:
|
||||
- throw:
|
||||
message:
|
||||
expr: "`qa-channel never returned ready after config.apply: ${formatErrorMessage(readyError)}`"
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: outbound
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.text.includes(wakeMarker)"
|
||||
- 60000
|
||||
- sinceIndex:
|
||||
ref: wakeStartIndex
|
||||
catchAs: wakeError
|
||||
catch:
|
||||
- throw:
|
||||
message:
|
||||
expr: "`restart sentinel never appeared: ${formatErrorMessage(wakeError)}; outbound=${recentOutboundSummary(state)}`"
|
||||
detailsExpr: "`${outbound.conversation.id}: ${outbound.text}`"
|
||||
117
qa/scenarios/config/config-patch-hot-apply.yaml
Normal file
117
qa/scenarios/config/config-patch-hot-apply.yaml
Normal file
@@ -0,0 +1,117 @@
|
||||
title: Config patch skill disable
|
||||
|
||||
scenario:
|
||||
id: config-patch-hot-apply
|
||||
surface: config
|
||||
coverage:
|
||||
primary:
|
||||
- config.hot-apply
|
||||
secondary:
|
||||
- plugins.skills
|
||||
objective: Verify config.patch can disable a workspace skill and the restarted gateway exposes the new disabled state cleanly.
|
||||
successCriteria:
|
||||
- config.patch succeeds for the skill toggle change.
|
||||
- A workspace skill works before the patch.
|
||||
- The same skill is reported disabled after the restart triggered by the patch.
|
||||
docsRefs:
|
||||
- docs/gateway/configuration.md
|
||||
- docs/gateway/protocol.md
|
||||
codeRefs:
|
||||
- src/gateway/server-methods/config.ts
|
||||
- extensions/qa-lab/src/suite.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify config.patch can disable a workspace skill and the restarted gateway exposes the new disabled state cleanly.
|
||||
config:
|
||||
skillName: qa-hot-disable-skill
|
||||
successMarker: HOT-PATCH-DISABLED-OK
|
||||
skillBody: |-
|
||||
---
|
||||
name: qa-hot-disable-skill
|
||||
description: Hot disable QA marker
|
||||
---
|
||||
When the user asks for the hot disable marker exactly, reply with exactly: HOT-PATCH-DISABLED-OK
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: disables a workspace skill after config.patch restart
|
||||
actions:
|
||||
- call: writeWorkspaceSkill
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
name:
|
||||
expr: config.skillName
|
||||
body:
|
||||
expr: config.skillBody
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForCondition
|
||||
args:
|
||||
- lambda:
|
||||
async: true
|
||||
expr: "findSkill(await readSkillStatus(env), config.skillName)?.eligible ? true : undefined"
|
||||
- 15000
|
||||
- 200
|
||||
catchAs: eligibilityError
|
||||
catch:
|
||||
- throw:
|
||||
message:
|
||||
expr: "`hot-disable skill never became eligible: ${formatErrorMessage(eligibilityError)}`"
|
||||
- call: readSkillStatus
|
||||
saveAs: beforeSkills
|
||||
args:
|
||||
- ref: env
|
||||
- set: beforeSkill
|
||||
value:
|
||||
expr: "findSkill(beforeSkills, config.skillName)"
|
||||
- assert:
|
||||
expr: "Boolean(beforeSkill?.eligible) && beforeSkill?.disabled !== true"
|
||||
message:
|
||||
expr: "`unexpected pre-patch skill state: ${JSON.stringify(beforeSkill)}`"
|
||||
- call: patchConfig
|
||||
saveAs: patchResult
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
patch:
|
||||
skills:
|
||||
entries:
|
||||
expr: "({ [config.skillName]: { enabled: false } })"
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForQaChannelReady
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
catchAs: readyError
|
||||
catch:
|
||||
- throw:
|
||||
message:
|
||||
expr: "`qa-channel never returned ready after config.patch: ${formatErrorMessage(readyError)}`"
|
||||
- try:
|
||||
actions:
|
||||
- call: waitForCondition
|
||||
args:
|
||||
- lambda:
|
||||
async: true
|
||||
expr: "findSkill(await readSkillStatus(env), config.skillName)?.disabled ? true : undefined"
|
||||
- 15000
|
||||
- 200
|
||||
catchAs: disabledError
|
||||
catch:
|
||||
- throw:
|
||||
message:
|
||||
expr: "`hot-disable skill never flipped to disabled: ${formatErrorMessage(disabledError)}`"
|
||||
- call: readSkillStatus
|
||||
saveAs: afterSkills
|
||||
args:
|
||||
- ref: env
|
||||
- set: afterSkill
|
||||
value:
|
||||
expr: "findSkill(afterSkills, config.skillName)"
|
||||
- assert:
|
||||
expr: "Boolean(afterSkill?.disabled)"
|
||||
message:
|
||||
expr: "`unexpected post-patch skill state: ${JSON.stringify(afterSkill)}`"
|
||||
detailsExpr: " `restartDelayMs=${String(patchResult.restart?.delayMs ?? '')}\\nmarker=${config.successMarker}\\npre=${JSON.stringify(beforeSkill)}\\npost=${JSON.stringify(afterSkill)}` "
|
||||
236
qa/scenarios/config/config-restart-capability-flip.yaml
Normal file
236
qa/scenarios/config/config-restart-capability-flip.yaml
Normal file
@@ -0,0 +1,236 @@
|
||||
title: "Config restart capability flip"
|
||||
|
||||
scenario:
|
||||
id: config-restart-capability-flip
|
||||
surface: config
|
||||
coverage:
|
||||
primary:
|
||||
- config.restart-apply
|
||||
secondary:
|
||||
- plugins.capabilities
|
||||
objective: Verify a restart-triggering config change flips capability inventory and the same session successfully uses the newly restored tool after wake-up.
|
||||
successCriteria:
|
||||
- Capability is absent before the restart-triggering patch.
|
||||
- Restart sentinel wakes the same session back up after config patch.
|
||||
- The restored capability appears in tools.effective and works in the follow-up turn.
|
||||
docsRefs:
|
||||
- docs/gateway/configuration.md
|
||||
- docs/gateway/protocol.md
|
||||
- docs/tools/image-generation.md
|
||||
codeRefs:
|
||||
- src/gateway/server-methods/config.ts
|
||||
- src/gateway/server-restart-sentinel.ts
|
||||
- src/gateway/server-methods/tools-effective.ts
|
||||
- extensions/qa-lab/src/suite.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Verify a restart-triggering config change flips capability inventory and the same session successfully uses the newly restored tool after wake-up.
|
||||
config:
|
||||
imagePrompt: "Capability flip image check: generate a QA lighthouse image in this turn right now. Do not acknowledge first, do not promise future work, and do not stop before using image_generate. Final reply must include the MEDIA path."
|
||||
imagePromptSnippet: "Capability flip image check"
|
||||
deniedTool: image_generate
|
||||
imageTurnTimeoutMs: 120000
|
||||
mediaPathTimeoutMs: 30000
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: restores image_generate after restart and uses it in the same session
|
||||
actions:
|
||||
- call: ensureImageGenerationConfigured
|
||||
args:
|
||||
- ref: env
|
||||
- call: readConfigSnapshot
|
||||
saveAs: original
|
||||
args:
|
||||
- ref: env
|
||||
- set: originalTools
|
||||
value:
|
||||
expr: "original.config.tools && typeof original.config.tools === 'object' ? original.config.tools : null"
|
||||
- set: originalToolsDeny
|
||||
value:
|
||||
expr: "originalTools ? (Object.prototype.hasOwnProperty.call(originalTools, 'deny') ? structuredClone(originalTools.deny) : undefined) : undefined"
|
||||
- set: originalImageGenerationModelPrimary
|
||||
value:
|
||||
expr: "original.config.agents?.defaults?.imageGenerationModel?.primary ?? null"
|
||||
- set: denied
|
||||
value:
|
||||
expr: "Array.isArray(originalToolsDeny) ? originalToolsDeny.map((entry) => String(entry)) : []"
|
||||
- set: deniedWithImage
|
||||
value:
|
||||
expr: "denied.includes(config.deniedTool) ? denied : [...denied, config.deniedTool]"
|
||||
- set: sessionKey
|
||||
value: agent:qa:capability-flip
|
||||
- call: createSession
|
||||
args:
|
||||
- ref: env
|
||||
- Capability flip
|
||||
- ref: sessionKey
|
||||
- try:
|
||||
actions:
|
||||
- call: patchConfig
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
patch:
|
||||
tools:
|
||||
deny:
|
||||
ref: deniedWithImage
|
||||
- call: waitForGatewayHealthy
|
||||
args:
|
||||
- ref: env
|
||||
- call: waitForQaChannelReady
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
- call: readEffectiveTools
|
||||
saveAs: beforeTools
|
||||
args:
|
||||
- ref: env
|
||||
- ref: sessionKey
|
||||
- assert:
|
||||
expr: "!beforeTools.has(config.deniedTool)"
|
||||
message:
|
||||
expr: "`${config.deniedTool} still present before capability flip`"
|
||||
- set: wakeMarker
|
||||
value:
|
||||
expr: "`QA-CAPABILITY-${randomUUID().slice(0, 8)}`"
|
||||
- call: patchConfig
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
patch:
|
||||
tools:
|
||||
deny:
|
||||
expr: "originalToolsDeny === undefined ? null : originalToolsDeny"
|
||||
agents:
|
||||
defaults:
|
||||
imageGenerationModel:
|
||||
primary:
|
||||
ref: originalImageGenerationModelPrimary
|
||||
sessionKey:
|
||||
ref: sessionKey
|
||||
note:
|
||||
ref: wakeMarker
|
||||
replacePaths:
|
||||
- tools.deny
|
||||
- call: waitForGatewayHealthy
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
- call: waitForQaChannelReady
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
- call: waitForCondition
|
||||
saveAs: afterTools
|
||||
args:
|
||||
- lambda:
|
||||
async: true
|
||||
expr: "(() => readEffectiveTools(env, sessionKey).then((tools) => (tools.has('image_generate') ? tools : undefined)))()"
|
||||
- expr: liveTurnTimeoutMs(env, config.imageTurnTimeoutMs)
|
||||
- 500
|
||||
- set: imageStartedAtMs
|
||||
value:
|
||||
expr: "Date.now()"
|
||||
- set: mediaPath
|
||||
value: ""
|
||||
- set: imageReplyText
|
||||
value: ""
|
||||
- set: imageReplyStartIndex
|
||||
value:
|
||||
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').length"
|
||||
- try:
|
||||
actions:
|
||||
- call: runAgentPrompt
|
||||
args:
|
||||
- ref: env
|
||||
- sessionKey:
|
||||
ref: sessionKey
|
||||
message:
|
||||
expr: config.imagePrompt
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, config.imageTurnTimeoutMs)
|
||||
catchAs: imageRunError
|
||||
catch:
|
||||
- if:
|
||||
expr: "!env.mock || !/agent run aborted/i.test(formatErrorMessage(imageRunError))"
|
||||
then:
|
||||
- throw:
|
||||
message:
|
||||
expr: "formatErrorMessage(imageRunError)"
|
||||
- try:
|
||||
actions:
|
||||
- call: resolveGeneratedImagePath
|
||||
saveAs: mediaPath
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
promptSnippet:
|
||||
expr: config.imagePromptSnippet
|
||||
startedAtMs:
|
||||
ref: imageStartedAtMs
|
||||
timeoutMs:
|
||||
expr: liveTurnTimeoutMs(env, config.mediaPathTimeoutMs)
|
||||
catch:
|
||||
- set: mediaPath
|
||||
value: ""
|
||||
- if:
|
||||
expr: "!mediaPath"
|
||||
then:
|
||||
- call: waitForOutboundMessage
|
||||
saveAs: imageReply
|
||||
args:
|
||||
- ref: state
|
||||
- lambda:
|
||||
params: [candidate]
|
||||
expr: "candidate.conversation.id === 'qa-operator' && (String(candidate.text ?? '').includes('MEDIA:') || /media failed|image generation failed/i.test(String(candidate.text ?? '')))"
|
||||
- expr: liveTurnTimeoutMs(env, config.imageTurnTimeoutMs)
|
||||
- sinceIndex:
|
||||
ref: imageReplyStartIndex
|
||||
- set: imageReplyText
|
||||
value:
|
||||
expr: "String(imageReply.text ?? '')"
|
||||
else:
|
||||
- set: imageReplyText
|
||||
value:
|
||||
expr: "`MEDIA:${mediaPath}`"
|
||||
- set: imageReplyLower
|
||||
value:
|
||||
expr: "imageReplyText.toLowerCase()"
|
||||
- assert:
|
||||
expr: "Boolean(mediaPath) || (!env.mock && /media failed|image generation failed/.test(imageReplyLower))"
|
||||
message:
|
||||
expr: "`expected restored ${config.deniedTool} to either produce media or, in live mode only, surface a provider-side image failure; got ${imageReplyText}`"
|
||||
# Tool-call assertion (criterion 2 of the parity completion
|
||||
# gate in #64227): the restored `image_generate` capability
|
||||
# must have actually fired as a real tool call. Without this
|
||||
# assertion, a prose reply that just mentions a MEDIA path
|
||||
# could satisfy the scenario, so strengthen it by requiring
|
||||
# the mock to have recorded `plannedToolName: "image_generate"`
|
||||
# against a post-restart request. The `!env.mock || ...`
|
||||
# guard means this check only runs in mock mode (where
|
||||
# `/debug/requests` is available); live-frontier runs skip
|
||||
# it and still pass the rest of the scenario.
|
||||
- assert:
|
||||
expr: "!env.mock || [...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))].some((request) => String(request.allInputText ?? '').toLowerCase().includes('capability flip image check') && request.plannedToolName === 'image_generate')"
|
||||
message:
|
||||
expr: "`expected image_generate tool call during capability flip scenario, saw plannedToolNames=${JSON.stringify([...(await fetchJson(`${env.mock.baseUrl}/debug/requests`))].filter((request) => String(request.allInputText ?? '').toLowerCase().includes('capability flip image check')).map((request) => request.plannedToolName ?? null))}`"
|
||||
finally:
|
||||
- call: patchConfig
|
||||
args:
|
||||
- env:
|
||||
ref: env
|
||||
patch:
|
||||
tools:
|
||||
deny:
|
||||
expr: "originalToolsDeny === undefined ? null : originalToolsDeny"
|
||||
replacePaths:
|
||||
- tools.deny
|
||||
- call: waitForGatewayHealthy
|
||||
args:
|
||||
- ref: env
|
||||
- call: waitForQaChannelReady
|
||||
args:
|
||||
- ref: env
|
||||
- 60000
|
||||
detailsExpr: "`${wakeMarker}\\n${config.deniedTool}=${String(afterTools.has(config.deniedTool))}\\n${mediaPath ? `MEDIA:${mediaPath}` : imageReplyText}`"
|
||||
155
qa/scenarios/config/crestodian-ring-zero-setup.yaml
Normal file
155
qa/scenarios/config/crestodian-ring-zero-setup.yaml
Normal file
@@ -0,0 +1,155 @@
|
||||
title: Crestodian ring-zero setup
|
||||
|
||||
scenario:
|
||||
id: crestodian-ring-zero-setup
|
||||
surface: config
|
||||
coverage:
|
||||
primary:
|
||||
- config.crestodian-setup
|
||||
secondary:
|
||||
- channels.discord-config
|
||||
- agents.create
|
||||
objective: Verify Crestodian can bootstrap a fresh OpenClaw config, set the default model, create an agent, configure Discord through a SecretRef, validate config, and leave an audit trail.
|
||||
successCriteria:
|
||||
- Crestodian reports missing config in an empty state dir.
|
||||
- Crestodian setup writes a workspace and default model.
|
||||
- Crestodian creates a non-main agent with its own workspace and model.
|
||||
- Crestodian enables the Discord plugin before writing Discord channel config.
|
||||
- Crestodian configures Discord through an env SecretRef without persisting the raw token.
|
||||
- Config validation passes and audit entries exist for every applied write.
|
||||
docsRefs:
|
||||
- docs/cli/crestodian.md
|
||||
- docs/channels/discord.md
|
||||
- docs/help/testing.md
|
||||
codeRefs:
|
||||
- src/crestodian/operations.ts
|
||||
- scripts/e2e/crestodian-first-run-spec.json
|
||||
- test/e2e/qa-lab/runtime/crestodian-first-run-docker-client.ts
|
||||
- extensions/qa-lab/src/suite-runtime-agent-process.ts
|
||||
execution:
|
||||
kind: flow
|
||||
summary: Drive the public Crestodian CLI in an isolated fresh state dir and verify setup/model/agent/Discord/audit results.
|
||||
config:
|
||||
specPath: scripts/e2e/crestodian-first-run-spec.json
|
||||
|
||||
flow:
|
||||
steps:
|
||||
- name: bootstraps config through Crestodian CLI
|
||||
actions:
|
||||
- set: setupSpec
|
||||
value:
|
||||
expr: "JSON.parse(await fs.readFile(path.join(env.repoRoot, config.specPath), 'utf8'))"
|
||||
- set: stateDir
|
||||
value:
|
||||
expr: "path.join(env.gateway.tempRoot, setupSpec.stateDirName)"
|
||||
- set: configPath
|
||||
value:
|
||||
expr: "path.join(stateDir, 'openclaw.json')"
|
||||
- set: defaultWorkspace
|
||||
value:
|
||||
expr: "path.join(env.gateway.tempRoot, setupSpec.defaultWorkspaceName)"
|
||||
- set: agentWorkspace
|
||||
value:
|
||||
expr: "path.join(env.gateway.tempRoot, setupSpec.agentWorkspaceName)"
|
||||
- set: commandVars
|
||||
value:
|
||||
expr: "({ defaultWorkspace, agentWorkspace, agentId: setupSpec.agentId, model: setupSpec.model, discordEnv: setupSpec.discordEnv })"
|
||||
- set: renderCommand
|
||||
value:
|
||||
lambda:
|
||||
params:
|
||||
- template
|
||||
expr: "String(template).replace(/\\{([A-Za-z0-9_]+)\\}/g, (match, key) => String(commandVars[key] ?? match))"
|
||||
- set: crestodianEnv
|
||||
value:
|
||||
expr: "({ OPENCLAW_STATE_DIR: stateDir, OPENCLAW_CONFIG_PATH: configPath, OPENCLAW_BUNDLED_PLUGINS_DIR: path.join(env.repoRoot, 'dist', 'extensions'), [setupSpec.discordEnv]: setupSpec.discordToken })"
|
||||
- call: fs.rm
|
||||
args:
|
||||
- ref: stateDir
|
||||
- recursive: true
|
||||
force: true
|
||||
- call: fs.mkdir
|
||||
args:
|
||||
- ref: stateDir
|
||||
- recursive: true
|
||||
- call: runQaCli
|
||||
saveAs: overviewOutput
|
||||
args:
|
||||
- ref: env
|
||||
- - crestodian
|
||||
- -m
|
||||
- overview
|
||||
- timeoutMs: 60000
|
||||
env:
|
||||
ref: crestodianEnv
|
||||
- assert:
|
||||
expr: "String(overviewOutput).includes('Config: missing')"
|
||||
message:
|
||||
expr: "`fresh Crestodian overview did not report missing config: ${overviewOutput}`"
|
||||
- assert:
|
||||
expr: 'String(overviewOutput).includes(''Next: run "setup" to create a starter config'')'
|
||||
message:
|
||||
expr: "`fresh Crestodian overview did not recommend setup: ${overviewOutput}`"
|
||||
- forEach:
|
||||
items:
|
||||
ref: setupSpec.commands
|
||||
item: commandStep
|
||||
actions:
|
||||
- call: runQaCli
|
||||
saveAs: commandOutput
|
||||
args:
|
||||
- ref: env
|
||||
- expr: "['crestodian', ...(commandStep.approve ? ['--yes'] : []), '-m', renderCommand(commandStep.message)]"
|
||||
- timeoutMs: 60000
|
||||
env:
|
||||
ref: crestodianEnv
|
||||
- assert:
|
||||
expr: "String(commandOutput).includes(commandStep.expectOutput)"
|
||||
message:
|
||||
expr: "`Crestodian command ${commandStep.id} did not produce ${commandStep.expectOutput}: ${commandOutput}`"
|
||||
- set: writtenConfig
|
||||
value:
|
||||
expr: "JSON.parse(await fs.readFile(configPath, 'utf8'))"
|
||||
- set: agent
|
||||
value:
|
||||
expr: "writtenConfig.agents?.list?.find((candidate) => candidate.id === setupSpec.agentId)"
|
||||
- assert:
|
||||
expr: "writtenConfig.agents?.defaults?.workspace === defaultWorkspace"
|
||||
message:
|
||||
expr: "`default workspace mismatch: ${JSON.stringify(writtenConfig.agents?.defaults)}`"
|
||||
- assert:
|
||||
expr: "writtenConfig.agents?.defaults?.model?.primary === setupSpec.model"
|
||||
message:
|
||||
expr: "`default model mismatch: ${JSON.stringify(writtenConfig.agents?.defaults?.model)}`"
|
||||
- assert:
|
||||
expr: "agent?.workspace === agentWorkspace && agent?.model === setupSpec.model"
|
||||
message:
|
||||
expr: "`agent config mismatch: ${JSON.stringify(agent)}`"
|
||||
- assert:
|
||||
expr: "writtenConfig.plugins?.allow?.includes('discord') && writtenConfig.plugins?.entries?.discord?.enabled === true"
|
||||
message:
|
||||
expr: "`Discord plugin was not enabled: ${JSON.stringify(writtenConfig.plugins)}`"
|
||||
- assert:
|
||||
expr: "writtenConfig.channels?.discord?.enabled === true"
|
||||
message:
|
||||
expr: "`Discord was not enabled: ${JSON.stringify(writtenConfig.channels?.discord)}`"
|
||||
- assert:
|
||||
expr: "writtenConfig.channels?.discord?.token?.source === 'env' && writtenConfig.channels?.discord?.token?.id === setupSpec.discordEnv"
|
||||
message:
|
||||
expr: "`Discord token was not an env SecretRef: ${JSON.stringify(writtenConfig.channels?.discord?.token)}`"
|
||||
- assert:
|
||||
expr: "!JSON.stringify(writtenConfig.channels?.discord ?? {}).includes(setupSpec.discordToken)"
|
||||
message: Crestodian persisted the raw Discord token.
|
||||
- set: auditText
|
||||
value:
|
||||
expr: "await fs.readFile(path.join(stateDir, 'audit', 'crestodian.jsonl'), 'utf8')"
|
||||
- forEach:
|
||||
items:
|
||||
ref: setupSpec.auditOperations
|
||||
item: operation
|
||||
actions:
|
||||
- assert:
|
||||
expr: 'auditText.includes(`"operation":"${operation}"`)'
|
||||
message:
|
||||
expr: "`missing audit entry for ${operation}: ${auditText}`"
|
||||
detailsExpr: "`stateDir=${stateDir}\\nconfigPath=${configPath}\\nagent=${JSON.stringify(agent)}\\nDiscord SecretRef=${JSON.stringify(writtenConfig.channels?.discord?.token)}`"
|
||||
Reference in New Issue
Block a user