Files
adolf/docs/platforms/mac/peekaboo.md
alvis bedb527145
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Vendor OpenClaw source as Adolf fork baseline
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
2026-07-05 09:36:54 +00:00

4.1 KiB

summary, read_when, title
summary read_when title
PeekabooBridge integration for macOS UI automation
Hosting PeekabooBridge in OpenClaw.app
Integrating Peekaboo via Swift Package Manager
Changing PeekabooBridge protocol/paths
Deciding between PeekabooBridge, Codex Computer Use, and cua-driver MCP
Peekaboo bridge

OpenClaw can host PeekabooBridge as a local, permission-aware UI automation broker (PeekabooBridgeHostCoordinator, backed by the steipete/Peekaboo Swift package). This lets the peekaboo CLI drive UI automation while reusing the macOS app's TCC permissions.

What this is (and is not)

  • Host: OpenClaw.app can act as a PeekabooBridge host.
  • Client: the peekaboo CLI (there is no separate openclaw ui ... surface).
  • UI: visual overlays stay in Peekaboo.app; OpenClaw is a thin broker host.

Relationship to other desktop-control paths

OpenClaw has three desktop-control paths that intentionally stay separate:

  • PeekabooBridge host: OpenClaw.app hosts the local PeekabooBridge socket. The peekaboo CLI is the client and uses OpenClaw.app's macOS permissions for screenshots, clicks, menus, dialogs, Dock actions, and window management.
  • Codex Computer Use: the bundled codex plugin checks and can install Codex's computer-use MCP plugin (extensions/codex/src/app-server/computer-use.ts), then lets Codex own native desktop-control tool calls during Codex-mode turns. OpenClaw does not proxy those actions through PeekabooBridge.
  • Direct cua-driver MCP: OpenClaw can register TryCua's upstream cua-driver mcp server as a normal MCP server, giving agents the CUA driver's own schemas and pid/window/element-index workflow without routing through the Codex marketplace or the PeekabooBridge socket.

Use Peekaboo for the broad macOS automation surface via OpenClaw.app's permission-aware bridge host. Use Codex Computer Use when a Codex-mode agent should rely on Codex's native plugin. Use direct cua-driver mcp to expose the CUA driver to any OpenClaw-managed runtime as a normal MCP server.

Enable the bridge

In the macOS app: Settings -> Enable Peekaboo Bridge.

When enabled, OpenClaw starts a local UNIX socket server at ~/Library/Application Support/OpenClaw/<socket-name>. If disabled, the host stops and peekaboo falls back to other available hosts. The coordinator also maintains legacy socket symlinks (clawdbot, clawdis, moltbot under Application Support) pointing at the current socket for older peekaboo installs.

Client discovery order

Peekaboo clients typically try hosts in this order:

  1. Peekaboo.app (full UX)
  2. Claude.app (if installed)
  3. OpenClaw.app (thin broker)

Use peekaboo bridge status --verbose to see which host is active and which socket path is in use. Override with:

export PEEKABOO_BRIDGE_SOCKET=/path/to/bridge.sock

Security and permissions

  • The bridge validates caller code signatures; an allowlist of TeamIDs is enforced (Peekaboo host TeamID plus the running app's own TeamID).
  • Prefer the signed bridge/app identity over a generic node runtime for Accessibility. Granting Accessibility to node lets any package launched by that Node executable inherit GUI automation access; see macOS permissions.
  • Requests time out after 10 seconds (requestTimeoutSec: 10).
  • If required permissions are missing, the bridge returns a clear error message rather than launching System Settings.

Snapshot behavior (automation)

Snapshots are stored in memory with a 10-minute validity window and a cap of 50 snapshots (InMemorySnapshotManager); artifacts are not deleted on cleanup. If you need longer retention, re-capture from the client.

Troubleshooting

  • If peekaboo reports "bridge client is not authorized", ensure the client is properly signed or run the host with PEEKABOO_ALLOW_UNSIGNED_SOCKET_CLIENTS=1 in debug mode only.
  • If no hosts are found, open one of the host apps (Peekaboo.app or OpenClaw.app) and confirm permissions are granted.