Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
151
qa/new-scenarios-2026-04.md
Normal file
151
qa/new-scenarios-2026-04.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# QA Scenario Expansion - Round 2
|
||||
|
||||
Ten repo-grounded candidate scenarios to add after the current seed suite.
|
||||
|
||||
## 1. On-demand memory tools in channel context
|
||||
|
||||
- Goal: verify the agent uses `memory_search` plus `memory_get` instead of bluffing when a channel message asks about prior notes.
|
||||
- Flow:
|
||||
- Seed `MEMORY.md` or `memory/*.md` with a fact not present in the current transcript.
|
||||
- Ask in a channel thread for that fact.
|
||||
- Verify tool usage and final answer accuracy.
|
||||
- Pass:
|
||||
- `memory_search` runs first.
|
||||
- `memory_get` narrows to the right lines.
|
||||
- Final answer cites the remembered fact correctly without cross-session leakage.
|
||||
- Docs: `docs/concepts/memory.md`, `docs/concepts/memory-search.md`
|
||||
- Code: `extensions/memory-core/src/tools.ts`, `extensions/memory-core/src/prompt-section.ts`
|
||||
|
||||
## 2. Memory failure fallback
|
||||
|
||||
- Goal: verify memory failure is graceful when embeddings/search are unavailable.
|
||||
- Flow:
|
||||
- Disable or break the embedding-backed memory path.
|
||||
- Ask for prior-note recall.
|
||||
- Verify the agent surfaces uncertainty and next action instead of hallucinating.
|
||||
- Pass:
|
||||
- Tool failure does not crash the run.
|
||||
- Agent says it checked and could not confirm.
|
||||
- Report includes the remediation hint.
|
||||
- Docs: `docs/concepts/memory.md`, `docs/help/faq.md`
|
||||
- Code: `extensions/memory-core/src/tools.shared.ts`, `extensions/memory-core/src/tools.citations.test.ts`
|
||||
|
||||
## 3. Model switch with tool continuity
|
||||
|
||||
- Goal: verify model switching preserves session context and tool availability, not just plain text continuity.
|
||||
- Flow:
|
||||
- Start on one model.
|
||||
- Switch to another configured model.
|
||||
- Ask for a tool-using follow-up such as file read or memory lookup.
|
||||
- Pass:
|
||||
- Switch is reflected in runtime state.
|
||||
- Tool call still succeeds after the switch.
|
||||
- Final answer keeps prior context.
|
||||
- Docs: `docs/help/testing.md`, `docs/concepts/model-failover.md`
|
||||
- Code: `extensions/qa-lab/src/suite.ts`, `docs/web/webchat.md`
|
||||
|
||||
## 4. MCP-backed recall via QMD/mcporter
|
||||
|
||||
- Goal: verify an MCP-backed tool path works end to end, not just core tools.
|
||||
- Flow:
|
||||
- Enable `memory.qmd.mcporter`.
|
||||
- Ask for recall that should route through the QMD MCP bridge.
|
||||
- Verify response and captured MCP execution path.
|
||||
- Pass:
|
||||
- MCP-backed search path is used.
|
||||
- Returned snippet matches the right note.
|
||||
- Failure mode is explicit if the daemon/tool is missing.
|
||||
- Docs: `docs/gateway/secrets.md`, `docs/concepts/memory-qmd.md`
|
||||
- Code: `extensions/memory-core/src/memory/qmd-manager.ts`, `extensions/memory-core/src/memory/qmd-manager.test.ts`
|
||||
|
||||
## 5. Skill visibility and invocation
|
||||
|
||||
- Goal: verify the agent sees a workspace/project skill and actually uses it.
|
||||
- Flow:
|
||||
- Add a simple workspace or `.agents` skill.
|
||||
- Confirm skill visibility through runtime inventory.
|
||||
- Ask for a task that should trigger the skill.
|
||||
- Pass:
|
||||
- Skill appears in `skills.status`.
|
||||
- Agent invocation reflects the installed skill instructions.
|
||||
- Per-agent allowlist behavior is respected.
|
||||
- Docs: `docs/tools/skills.md`, `docs/gateway/protocol.md`, `docs/gateway/configuration.md`
|
||||
- Code: `.agents/skills/openclaw-qa-testing/SKILL.md`, `docs/gateway/protocol.md`
|
||||
|
||||
## 6. Skill install and hot availability
|
||||
|
||||
- Goal: verify a newly installed skill becomes usable without a broken intermediate state.
|
||||
- Flow:
|
||||
- Install a ClawHub or gateway-managed skill.
|
||||
- Re-check skill inventory.
|
||||
- Ask the agent to perform the skill-backed task.
|
||||
- Pass:
|
||||
- Install succeeds.
|
||||
- `skills.status` or `skills.bins` reflects the new skill.
|
||||
- Agent can use the skill immediately or after the expected reload path.
|
||||
- Docs: `docs/tools/skills.md`, `docs/cli/skills.md`, `docs/gateway/protocol.md`
|
||||
- Code: `docs/gateway/protocol.md`, `docs/tools/skills.md`
|
||||
|
||||
## 7. Native image generation
|
||||
|
||||
- Goal: verify `image_generate` appears only when configured and returns a real attachment/artifact.
|
||||
- Flow:
|
||||
- Configure `agents.defaults.imageGenerationModel.primary`.
|
||||
- Ask for a simple generated image.
|
||||
- Verify generated media is returned in the reply path.
|
||||
- Pass:
|
||||
- `image_generate` is in the effective tool set.
|
||||
- Generation succeeds with the configured provider/model.
|
||||
- Output is attached and the agent summarizes what it created.
|
||||
- Docs: `docs/tools/image-generation.md`, `docs/providers/openai.md`
|
||||
- Code: `src/agents/openclaw-tools.image-generation.test.ts`, `src/image-generation/runtime.ts`
|
||||
|
||||
## 8. Config patch skill disable
|
||||
|
||||
- Goal: verify `config.patch` can disable a workspace skill and the restarted gateway exposes the disabled state cleanly.
|
||||
- Flow:
|
||||
- Add a workspace skill and verify it is eligible.
|
||||
- Use `config.patch` to disable that skill.
|
||||
- Wait for the gateway restart and read `skills.status` again.
|
||||
- Pass:
|
||||
- Patch succeeds.
|
||||
- Gateway restarts cleanly.
|
||||
- The skill flips from eligible to disabled.
|
||||
- Docs: `docs/gateway/configuration.md`, `docs/gateway/protocol.md`
|
||||
- Code: `docs/gateway/configuration.md`, `docs/web/control-ui.md`
|
||||
|
||||
## 9. Restart-required config apply with wake-up
|
||||
|
||||
- Goal: verify a restart-required config change restarts cleanly and wakes the session back up.
|
||||
- Flow:
|
||||
- Use `config.apply` or `update.run` on a restart-required surface.
|
||||
- Provide `sessionKey` so the operator gets the post-restart ping.
|
||||
- Resume the task after restart.
|
||||
- Pass:
|
||||
- Restart happens once.
|
||||
- Session wake-up ping arrives.
|
||||
- Agent continues in the same logical workflow after restart.
|
||||
- Docs: `docs/gateway/configuration.md`, `docs/web/control-ui.md`
|
||||
- Code: `docs/gateway/configuration.md`, `docs/gateway/protocol.md`
|
||||
|
||||
## 10. Runtime inventory drift check
|
||||
|
||||
- Goal: verify the reported tool and skill inventory matches what the agent can really use after config/plugin changes.
|
||||
- Flow:
|
||||
- Read `tools.effective` and `skills.status`.
|
||||
- Ask the agent to use one enabled thing and one disabled thing.
|
||||
- Compare actual behavior vs reported inventory.
|
||||
- Pass:
|
||||
- Enabled item is callable.
|
||||
- Disabled item is absent or blocked for the right reason.
|
||||
- Inventory and runtime behavior stay in sync.
|
||||
- Docs: `docs/gateway/protocol.md`, `docs/web/webchat.md`
|
||||
- Code: `docs/gateway/protocol.md`, `docs/web/control-ui.md`
|
||||
|
||||
## Best next additions to the executable suite
|
||||
|
||||
If we only promote three right away:
|
||||
|
||||
1. On-demand memory tools in channel context
|
||||
2. Native image generation
|
||||
3. Config patch skill disable
|
||||
Reference in New Issue
Block a user