Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
187 lines
7.6 KiB
Markdown
187 lines
7.6 KiB
Markdown
---
|
|
name: claw-score
|
|
description: Audit or refresh OpenClaw maturity scorecard docs from root taxonomy, maturity scores, and QA evidence artifacts without using maintainer discrawl data or committed inventory reports.
|
|
---
|
|
|
|
# claw-score
|
|
|
|
Use this skill when working on the OpenClaw maturity scorecard in this repo.
|
|
This is the openclaw-local version of the maintainer `claw-score` workflow:
|
|
it keeps the taxonomy and scorecard concepts, but excludes discrawl and the old
|
|
committed `inventory/` report tree.
|
|
|
|
## Authority
|
|
|
|
This skill owns the operational workflow for:
|
|
|
|
- `taxonomy.yaml`
|
|
- `qa/maturity-scores.yaml`
|
|
- `docs/concepts/qa-e2e-automation.md`
|
|
- `qa/scenarios/index.yaml`
|
|
|
|
Keep person-specific, maintainer-private, Discord archive, and discrawl facts
|
|
out of this repo. If a score needs private evidence, use the redacted
|
|
`qa-evidence.json` artifact shape generated by OpenClaw QA workflows.
|
|
|
|
## Source Model
|
|
|
|
- `taxonomy.yaml` is the hand-edited source of truth for surfaces, levels,
|
|
QA profiles, categories, feature coverage IDs, docs refs, LTS overrides, and
|
|
completeness-instruction paths.
|
|
- Feature `coverageIds` are ANDed proof targets, not aliases. A feature may
|
|
list multiple IDs when each ID proves part of one capability.
|
|
- Coverage IDs use dotted `namespace.behavior` form, with lowercase
|
|
alphanumeric/dash segments. Profile, surface, and category IDs may remain
|
|
dashed or dotted.
|
|
- Keep categories and feature names unique, product-shaped, and broader than raw
|
|
coverage IDs. Do not promote generic IDs into standalone feature names.
|
|
- Avoid duplicate coverage-ID bundles under different feature names in one
|
|
category.
|
|
- `qa/maturity-scores.yaml` is the committed aggregate source for Quality,
|
|
Completeness, and LTS review state.
|
|
- `extensions/qa-lab/src/scorecard-taxonomy.ts` exports
|
|
`qaMaturityScoresSchema` and `readValidatedQaMaturityScoreSources`; use those
|
|
QA Lab utilities to validate score output.
|
|
- Generated public docs are `docs/maturity/scorecard.md` and
|
|
`docs/maturity/taxonomy.md`; both come from `pnpm maturity:render`. Do not
|
|
hand-edit generated Markdown to change score results.
|
|
- `qa-evidence.json` artifacts provide per-run QA scorecard evidence. Release
|
|
profile artifacts are the source of truth for Coverage. They can enrich
|
|
generated artifact docs, but they are not committed as inventory.
|
|
|
|
## Commands
|
|
|
|
Run from the openclaw repo root.
|
|
|
|
Validate taxonomy YAML structure and the maturity score schema after source
|
|
edits:
|
|
|
|
```bash
|
|
node --import tsx --input-type=module <<'NODE'
|
|
import fs from "node:fs";
|
|
import YAML from "yaml";
|
|
import { readValidatedQaMaturityScoreSources } from "./extensions/qa-lab/src/scorecard-taxonomy.ts";
|
|
|
|
for (const file of ["taxonomy.yaml", "qa/scenarios/index.yaml"]) {
|
|
YAML.parse(fs.readFileSync(file, "utf8"));
|
|
}
|
|
readValidatedQaMaturityScoreSources();
|
|
NODE
|
|
```
|
|
|
|
Check docs when touching docs prose:
|
|
|
|
```bash
|
|
pnpm check:docs
|
|
```
|
|
|
|
Run focused QA/profile checks when changing coverage IDs or profile membership:
|
|
|
|
```bash
|
|
pnpm openclaw qa coverage --json
|
|
```
|
|
|
|
## Scoring Workflow
|
|
|
|
When asked to score or refresh a surface:
|
|
|
|
1. Read the surface in `taxonomy.yaml`.
|
|
2. Read the surface completeness rubric under
|
|
`.agents/skills/claw-score/references/completeness/`.
|
|
3. Gather public repo evidence from docs, source, tests, and QA scenario
|
|
metadata.
|
|
4. Prefer existing release profile `qa-evidence.json` artifacts for executed
|
|
proof.
|
|
5. Update `qa/maturity-scores.yaml` only for Quality, Completeness, and LTS
|
|
review state backed by public or redacted artifact evidence.
|
|
6. Run the schema validation command from this skill.
|
|
7. Run `pnpm check:docs` if docs prose changed, and focused QA coverage checks
|
|
if coverage IDs or profile membership changed.
|
|
|
|
For subjective score changes, make the smallest defensible edit and leave the
|
|
evidence path in the PR or task summary. Keep manual prose in current docs and
|
|
keep score data in `qa/maturity-scores.yaml`.
|
|
|
|
## Default Completeness Process
|
|
|
|
Completeness is scored against the intended operator-visible workflow for each
|
|
category, not against test breadth or implementation quality. The completeness
|
|
reference files under `references/completeness/` define the category scope and
|
|
any surface-specific variation from this default process.
|
|
|
|
By default, Completeness measures how fully OpenClaw exposes the intended
|
|
surface capability set to the user, operator, author, or maintainer persona for
|
|
that surface. Score whether each category delivers the full expected workflow,
|
|
including setup, normal use, status or inspection, recovery, and important
|
|
platform, provider, channel, security, or lifecycle variants where they apply.
|
|
|
|
Treat `Surface-Specific Scoring Questions` and `Surface-Specific Guidance` as
|
|
higher-priority instructions for that surface. The surface instructions may
|
|
flesh out, narrow, or intentionally conflict with the default ideas here; when
|
|
they do, follow the surface instructions and make the score rationale reflect
|
|
that surface-specific instruction. If a reference file does not include
|
|
surface-specific questions or guidance, apply this default process to the
|
|
surface's `Category Scope`.
|
|
|
|
For each category, ask:
|
|
|
|
- Can the intended user or operator complete the category workflow end to end?
|
|
- Are the taxonomy features present as supported capabilities rather than
|
|
isolated implementation fragments?
|
|
- Are the important lifecycle stages represented: setup, normal operation,
|
|
status/inspection, recovery, and upgrade or removal where relevant?
|
|
- Are the important environment, provider, platform, channel, or security
|
|
branches present for this surface?
|
|
- Do the known gaps leave major user-visible capability branches missing?
|
|
|
|
Default guidance:
|
|
|
|
- Favor higher Completeness when the category supports the full
|
|
operator-visible workflow described by taxonomy and category evidence.
|
|
- Lower Completeness when only the happy path exists, when important variants
|
|
are undocumented or unimplemented, or when recovery/status paths are missing.
|
|
- Do not lower Completeness because tests are thin; that is Coverage.
|
|
- Do not lower Completeness because implementation quality is fragile; that is
|
|
Quality.
|
|
|
|
Default Completeness bands:
|
|
|
|
- `Clawesome` (95-100): complete across expected workflows, variants, and
|
|
recovery branches, with only minor polish gaps.
|
|
- `Stable` (80-95): the expected workflow set is broadly present, with only
|
|
bounded missing branches.
|
|
- `Beta` (70-80): the main workflow exists, but meaningful branches or recovery
|
|
paths are still absent.
|
|
- `Alpha` (50-70): only a partial capability set is present; users can complete
|
|
some core tasks but not the full expected workflow.
|
|
- `Experimental` (0-50): the category exposes only fragments of the intended
|
|
capability.
|
|
|
|
## Score Semantics
|
|
|
|
- Coverage: deterministic release validation coverage derived from the release
|
|
profile `qa-evidence.json.scorecard` feature fulfillment data.
|
|
- Quality: reliability, maintainability, operator safety, and regression
|
|
confidence for the category.
|
|
- Completeness: how much of the intended operator-visible workflow exists for
|
|
the category. Use the default completeness process plus any surface-specific
|
|
variation before changing this score.
|
|
- LTS: derived from Quality, release-evidence Coverage, and
|
|
`human_lts_override`; do not hand-edit generated Markdown to change LTS
|
|
status.
|
|
|
|
Bands:
|
|
|
|
- `Clawesome`: 95-100
|
|
- `Stable`: 80-95
|
|
- `Beta`: 70-80
|
|
- `Alpha`: 50-70
|
|
- `Experimental`: 0-50
|
|
|
|
## Artifacts
|
|
|
|
Do not add the maintainer repo's `docs/kevinslin/maturity-scorecard/inventory/`
|
|
tree to openclaw. Evidence-enriched scorecard outputs belong in short-lived
|
|
artifacts, not committed generated docs, unless this repo adds an explicit
|
|
renderer/check workflow first.
|