Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled

Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
2026-07-05 09:36:54 +00:00
parent 3216769225
commit bedb527145
21108 changed files with 6010766 additions and 0 deletions

19949
apps/.i18n/native-source.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/ar.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/de.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/es.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/fa.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/fr.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/hi.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/id.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/it.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/ja-JP.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/ko.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/nl.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/pl.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/pt-BR.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/ru.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/sv.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/th.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/tr.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/uk.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/vi.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/zh-CN.json Normal file

File diff suppressed because it is too large Load Diff

12472
apps/.i18n/native/zh-TW.json Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,16 @@
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
[*.{kt,kts}]
indent_style = space
indent_size = 2
max_line_length = off
ktlint_standard_filename = disabled
ktlint_standard_function-naming = disabled
ktlint_standard_max-line-length = disabled
ktlint_standard_property-naming = disabled

26
apps/android/AGENTS.md Normal file
View File

@@ -0,0 +1,26 @@
# Android Release Agent Policy
Root rules still apply. This file adds the Android release guardrails.
## Google Play Releases
- Agent-driven Google Play uploads must use only `pnpm android:release:upload`.
- If `pnpm android:release:upload` exits non-zero, stop immediately and report the failing step.
- After a failed `pnpm android:release:upload`, do not continue with `pnpm android:release:archive`, `pnpm android:release:metadata`, `fastlane android play_store`, `fastlane android metadata`, direct Gradle release artifacts plus Google Play upload commands, Google Play API mutation commands, or mobile release ref recording.
- Do not promote an Android release to production. Production promotion stays manual in Google Play Console unless the user explicitly asks to promote a specific already-prepared release after the failed state has been reported.
- `pnpm android:release:archive` is for local archive validation only. It is not a fallback release path after screenshot, metadata, signing, validation, or upload-lane failure.
## Licenses Screen
- Maintain the Settings-tab Licenses screen when Android app dependencies change.
- Bundled license files live in `apps/android/THIRD_PARTY_LICENSES/openclaw/licenses/`.
- License files must be UTF-8 `.txt` files. Do not add Markdown, HTML, RTF, JSON, XML, or generated notice bundles for this screen.
- The Licenses screen discovers bundled `.txt` files at runtime through `AndroidLicenseNotices`; do not hardcode individual license rows in Compose.
- License rows are ordered alphabetically in code by derived display title, case-insensitive, with filename as the tiebreaker. Do not use numeric filename prefixes for ordering.
- The display title is the license filename without the `.txt` extension.
- Filenames should be plain dependency names, for example `Manrope.txt`; the filename is the row title and must not be shown as a row subtitle.
- Do not add OpenClaw, OpenClaw Foundation, or other first-party/self-owned license entries. The screen is for third-party/open-source dependency acknowledgements.
- When adding, removing, or upgrading Android dependencies, audit whether `apps/android/THIRD_PARTY_LICENSES/openclaw/licenses/` needs updates. Exclude dependencies owned by OpenClaw Foundation from the published license list.
- Keep license detail bodies rendered as verbatim monospace text.
- Keep the Settings `Licenses` section at the bottom of Settings, after `Account`, with a single `Licenses` row and no row subtitle unless product direction changes.
- When changing license loading or presentation, update `apps/android/app/src/test/java/ai/openclaw/app/AndroidLicenseNoticesTest.kt`, then run focused Android validation.

19
apps/android/CHANGELOG.md Normal file
View File

@@ -0,0 +1,19 @@
# OpenClaw Android Changelog
## Unreleased
The OpenClaw mascot now comes alive across onboarding and the app headers with the same float, blink, antenna-wiggle, and claw-snap animation as openclaw.ai.
## 2026.6.11 - 2026-07-01
Improves Android gateway setup with localized onboarding, QR pairing fixes, and support for local mDNS gateway hosts.
Adds clearer recovery guidance for TLS fingerprint timeouts, mobile protocol mismatches, and gateway auth states.
Refreshes native Android localization coverage, including Swedish app naming and localized gateway trust flows.
## 2026.6.2 - 2026-06-02
OpenClaw is now available on Android.
Connect to your OpenClaw Gateway to chat with your assistant, use realtime Talk mode, review approvals, and bring Android device capabilities like camera, location, screen, and notifications into your private automation workflows.

1
apps/android/CLAUDE.md Symbolic link
View File

@@ -0,0 +1 @@
AGENTS.md

View File

@@ -0,0 +1,14 @@
{
"signingRepo": "git@github.com:openclaw/apps-signing.git",
"signingBranch": "main",
"assetPath": "android/openclaw",
"uploadKeystoreEncryptedFile": "upload-keystore.jks.enc",
"gradlePropertiesEncryptedFile": "gradle.properties.enc",
"materializedRoot": "apps/android/build/release-signing",
"gradlePropertyNames": [
"OPENCLAW_ANDROID_STORE_FILE",
"OPENCLAW_ANDROID_STORE_PASSWORD",
"OPENCLAW_ANDROID_KEY_ALIAS",
"OPENCLAW_ANDROID_KEY_PASSWORD"
]
}

View File

@@ -0,0 +1,6 @@
# Shared Android version defaults.
# Source of truth: apps/android/version.json
# Generated by scripts/android-sync-versioning.ts.
OPENCLAW_ANDROID_VERSION_NAME=2026.6.11
OPENCLAW_ANDROID_VERSION_CODE=2026061101

345
apps/android/README.md Normal file
View File

@@ -0,0 +1,345 @@
## OpenClaw Android App
OpenClaw Android is the officially released Google Play app. It connects to an OpenClaw Gateway as a companion node for chat, voice, approvals, screen, and device-aware automation.
### Current App Surface
- [x] New 4-step onboarding flow
- [x] Connect tab with `Setup Code` + `Manual` modes
- [x] Encrypted persistence for gateway setup/auth state
- [x] Chat UI restyled
- [x] Settings UI restyled and de-duplicated (gateway controls moved to Connect)
- [x] QR code scanning in onboarding
- [x] Performance improvements
- [x] Streaming support in chat UI
- [x] Request camera/location and other permissions in onboarding/settings flow
- [x] Push notifications for gateway/chat status updates
- [x] Security hardening (biometric lock, token handling, safer defaults)
- [x] Authenticated background presence beacons
- [x] Voice tab full functionality
- [x] Screen tab full functionality
## Open in Android Studio
- Open the folder `apps/android`.
## Build / Run
```bash
cd apps/android
./gradlew :app:assemblePlayDebug
./gradlew :app:installPlayDebug
./gradlew :app:testPlayDebugUnitTest
cd ../..
pnpm android:release:archive
```
Third-party debug flavor:
```bash
cd apps/android
./gradlew :app:assembleThirdPartyDebug
./gradlew :app:installThirdPartyDebug
./gradlew :app:testThirdPartyDebugUnitTest
```
Android release archives use the pinned version in `apps/android/version.json`. Update it with:
```bash
pnpm android:version
pnpm android:version:check
pnpm android:version:pin -- --from-gateway
pnpm android:version:pin -- --version 2026.6.5 --version-code 2026060501
```
Release-owner signing sync:
```bash
pnpm android:release:signing:plan
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:sync:pull
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:check
```
The signing sync pulls encrypted Android upload-key assets from the shared `apps-signing` repo and materializes decrypted files under `apps/android/build/release-signing/`.
Generate raw Google Play screenshots:
```bash
pnpm android:screenshots
```
To make screenshot capture own emulator startup, pass a named AVD:
```bash
ANDROID_SCREENSHOT_AVD=OpenClaw_QA_API35 pnpm android:screenshots
```
The screenshot script uses one connected ADB device when available. If none is
connected and `ANDROID_SCREENSHOT_AVD` is set, it boots that emulator
headlessly, waits for Android to finish booting, disables animations, captures
the screenshots, then shuts down the emulator it started.
`pnpm android:release:archive` builds signed release artifacts into `apps/android/build/release-artifacts/` and writes `.sha256` checksum files:
- Play build: `openclaw-<version>-play-release.aab`
- Third-party build: `openclaw-<version>-third-party-release.apk`
`pnpm android:bundle:release` is an alias for the same Fastlane archive lane.
`pnpm android:release:archive` is for local archive validation only. It is not a
fallback upload path after `pnpm android:release:upload` fails.
Agent-driven Google Play uploads must use `pnpm android:release:upload` as the
only release path. If that command fails, stop and fix the failing screenshot,
metadata, signing, validation, archive, or upload step before trying again. Do
not upload archived artifacts through direct Fastlane lanes, Gradle artifacts,
Google Play API commands, or Play Console mutation commands.
See `apps/android/VERSIONING.md` and `apps/android/fastlane/SETUP.md` for the release workflow.
Flavor-specific direct Gradle tasks:
```bash
cd apps/android
./gradlew :app:bundlePlayRelease
./gradlew :app:bundleThirdPartyRelease
```
## Kotlin Lint + Format
```bash
pnpm android:lint
pnpm android:format
```
Android framework/resource lint (separate pass):
```bash
pnpm android:lint:android
```
Direct Gradle tasks:
```bash
cd apps/android
./gradlew :app:ktlintCheck :benchmark:ktlintCheck
./gradlew :app:ktlintFormat :benchmark:ktlintFormat
./gradlew :app:lintPlayDebug :app:lintThirdPartyDebug
```
`gradlew` auto-detects the Android SDK at `~/Library/Android/sdk` (macOS default) if `ANDROID_SDK_ROOT` / `ANDROID_HOME` are unset.
## Macrobenchmark (Startup + Frame Timing)
```bash
cd apps/android
./gradlew :benchmark:connectedDebugAndroidTest
```
Reports are written under:
- `apps/android/benchmark/build/reports/androidTests/connected/`
## Perf CLI (low-noise)
Deterministic startup measurement + hotspot extraction with compact CLI output:
```bash
cd apps/android
./scripts/perf-startup-benchmark.sh
./scripts/perf-startup-hotspots.sh
```
Benchmark script behavior:
- Runs only `StartupMacrobenchmark#coldStartup` (10 iterations).
- Prints median/min/max/COV in one line.
- Writes timestamped snapshot JSON to `apps/android/benchmark/results/`.
- Auto-compares with previous local snapshot (or pass explicit baseline: `--baseline <old-benchmarkData.json>`).
Hotspot script behavior:
- Ensures debug app installed, captures startup `simpleperf` data for `.MainActivity`.
- Prints top DSOs, top symbols, and key app-path clues (Compose/MainActivity/WebView).
- Writes raw `perf.data` path for deeper follow-up if needed.
## Run on a Real Android Phone (USB)
1) On phone, enable **Developer options** + **USB debugging**.
2) Connect by USB and accept the debugging trust prompt on phone.
3) Verify ADB can see the device:
```bash
adb devices -l
```
4) Install + launch debug build:
```bash
pnpm android:install
pnpm android:run
```
If `adb devices -l` shows `unauthorized`, re-plug and accept the trust prompt again.
### USB-only gateway testing (no LAN dependency)
Use `adb reverse` so Android `localhost:18789` tunnels to your laptop `localhost:18789`.
Terminal A (gateway):
```bash
pnpm openclaw gateway --port 18789 --verbose
```
Terminal B (USB tunnel):
```bash
adb reverse tcp:18789 tcp:18789
```
Then in app **Connect → Manual**:
- Host: `127.0.0.1`
- Port: `18789`
- TLS: off
## Hot Reload / Fast Iteration
This app is native Kotlin + Jetpack Compose.
- For Compose UI edits: use Android Studio **Live Edit** on a debug build (works on physical devices; project `minSdk=31` already meets API requirement).
- For many non-structural code/resource changes: use Android Studio **Apply Changes**.
- For structural/native/manifest/Gradle changes: do full reinstall (`pnpm android:run`).
- Canvas web content already supports live reload when loaded from Gateway `__openclaw__/canvas/` (see `docs/platforms/android.md`).
## Connect / Pair
1) Start the gateway (on your main machine):
```bash
pnpm openclaw gateway --port 18789 --verbose
```
2) In the Android app:
- Open the **Connect** tab.
- Use **Setup Code** or **Manual** mode to connect.
3) Approve pairing (on the gateway machine):
```bash
openclaw devices list
openclaw devices approve <requestId>
```
More details: `docs/platforms/android.md`.
## Permissions
- Discovery:
- Android 13+ (`API 33+`): `NEARBY_WIFI_DEVICES`
- Android 12 and below: `ACCESS_FINE_LOCATION` (required for NSD scanning)
- Foreground service notification (Android 13+): `POST_NOTIFICATIONS`
- Camera:
- `CAMERA` for `camera.snap` and `camera.clip`
- `RECORD_AUDIO` for `camera.clip` when `includeAudio=true`
## Google Play Restricted Permissions
As of March 19, 2026, these manifest permissions are the main Google Play policy risk for this app:
- `READ_SMS`
- `SEND_SMS`
- `READ_CALL_LOG`
Why these matter:
- Google Play treats SMS and Call Log access as highly restricted. In most cases, Play only allows them for the default SMS app, default Phone app, default Assistant, or a narrow policy exception.
- Review usually involves a `Permissions Declaration Form`, policy justification, and demo video evidence in Play Console.
- The Play build removes these behind the `play` flavor.
- Photo library access is also removed from the Play build. Use third-party builds for `photos.latest`.
Current OpenClaw Android implication:
- APK / sideload build can keep SMS, Call Log, and recent-photo features.
- Google Play build excludes SMS send/search, Call Log search, and recent-photo access unless the product is intentionally positioned and approved under the relevant policy exception.
- The repo now ships this split as Android product flavors:
- `play`: removes `READ_SMS`, `SEND_SMS`, `READ_CALL_LOG`, `READ_MEDIA_IMAGES`, `READ_MEDIA_VISUAL_USER_SELECTED`, and `READ_EXTERNAL_STORAGE`; hides SMS, Call Log, and Photos surfaces in onboarding, settings, and advertised node capabilities.
- Installed-app listing is user controlled. `device.apps` is advertised only after the user enables **Settings > Phone Capabilities > Installed Apps**. The command defaults to launcher-visible apps and does not require `QUERY_ALL_PACKAGES`.
- `thirdParty`: keeps the full permission set and the existing SMS / Call Log / Photos functionality.
Policy links:
- [Google Play SMS and Call Log policy](https://support.google.com/googleplay/android-developer/answer/10208820?hl=en)
- [Google Play sensitive permissions policy hub](https://support.google.com/googleplay/android-developer/answer/16558241)
- [Android default handlers guide](https://developer.android.com/guide/topics/permissions/default-handlers)
Other Play-restricted surfaces to watch if added later:
- `ACCESS_BACKGROUND_LOCATION`
- `MANAGE_EXTERNAL_STORAGE`
- `QUERY_ALL_PACKAGES`
- `REQUEST_INSTALL_PACKAGES`
- `AccessibilityService`
Reference links:
- [Background location policy](https://support.google.com/googleplay/android-developer/answer/9799150)
- [AccessibilityService policy](https://support.google.com/googleplay/android-developer/answer/10964491?hl=en-GB)
- [Photo and Video Permissions policy](https://support.google.com/googleplay/android-developer/answer/14594990)
## Integration Capability Test (Preconditioned)
This suite assumes setup is already done manually. It does **not** install/run/pair automatically.
Pre-req checklist:
1) Gateway is running and reachable from the Android app.
2) Android app is connected to that gateway and `openclaw nodes status` shows it as paired + connected.
3) App stays unlocked and in foreground for the whole run.
4) Open the app **Screen** tab and keep it active during the run (canvas/A2UI commands require the canvas WebView attached there).
5) Grant runtime permissions for capabilities you expect to pass (camera/mic/location/notification listener/location, etc.).
6) No interactive system dialogs should be pending before test start.
7) Canvas host is enabled and reachable from the device for remote Canvas checks (do not run gateway with `OPENCLAW_SKIP_CANVAS_HOST=1`; startup logs should include `canvas host mounted at .../__openclaw__/`).
8) Local operator test client pairing is approved. If first run fails with `pairing required`, preview the latest pending request, approve the printed request ID, then rerun:
9) For A2UI checks, keep the app on **Screen** tab; the node uses its bundled app-owned A2UI page for message application.
```bash
openclaw devices list
openclaw devices approve --latest # preview only; copy the requestId from output
openclaw devices approve <requestId>
```
Run:
```bash
pnpm android:test:integration
```
Optional overrides:
- `OPENCLAW_ANDROID_GATEWAY_URL=ws://...` (default: from your local OpenClaw config)
- `OPENCLAW_ANDROID_GATEWAY_TOKEN=...`
- `OPENCLAW_ANDROID_GATEWAY_PASSWORD=...`
- `OPENCLAW_ANDROID_NODE_ID=...` or `OPENCLAW_ANDROID_NODE_NAME=...`
What it does:
- Reads `node.describe` command list from the selected Android node.
- Invokes advertised non-interactive commands.
- Skips `screen.record` in this suite (Android requires interactive per-invocation screen-capture consent).
- Asserts command contracts (success or expected deterministic error for safe-invalid calls like `sms.send` and `notifications.actions`).
Common failure quick-fixes:
- `pairing required` before tests start:
- list pending requests (`openclaw devices list`), then approve with the exact ID (`openclaw devices approve <requestId>`) and rerun.
- `A2UI host not reachable` / `A2UI_HOST_UNAVAILABLE`:
- keep the app foregrounded on the **Screen** tab and rerun. A2UI commands use the bundled app-owned A2UI page; the Gateway Canvas host is still needed for remote Canvas checks, but not for A2UI message application.
- `NODE_BACKGROUND_UNAVAILABLE: canvas unavailable`:
- app is not effectively ready for canvas commands; keep app foregrounded and **Screen** tab active.
## Contributions
Maintainer: @obviyus. For issues/questions/contributions, please open an issue or reach out on Discord.

View File

@@ -0,0 +1,19 @@
Bouncy Castle Provider
Artifact: org.bouncycastle:bcprov-jdk18on
License: Bouncy Castle Licence
Bouncy Castle License
Copyright (c) 2000 - 2026 The Legion of the Bouncy Castle Inc. (https://www.bouncycastle.org)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files
(the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -0,0 +1,32 @@
CommonMark Java
Artifacts:
- org.commonmark:commonmark
- org.commonmark:commonmark-ext-autolink
- org.commonmark:commonmark-ext-gfm-strikethrough
- org.commonmark:commonmark-ext-gfm-tables
- org.commonmark:commonmark-ext-task-list-items
License: BSD 2-Clause
Copyright (c) 2015, Atlassian Pty Ltd
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -0,0 +1,184 @@
Kotlin libraries
Artifacts:
- org.jetbrains.kotlin:kotlin-stdlib
- org.jetbrains.kotlinx:kotlinx-coroutines-android
- org.jetbrains.kotlinx:kotlinx-coroutines-core
- org.jetbrains.kotlinx:kotlinx-serialization-json
License: Apache License 2.0
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS

View File

@@ -0,0 +1,93 @@
Copyright 2018 The Manrope Project Authors (https://github.com/sharanda/manrope)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.

View File

@@ -0,0 +1,184 @@
OkHttp and Okio
Artifacts:
- com.squareup.okhttp3:okhttp
- com.squareup.okhttp3:okhttp-android
- com.squareup.okio:okio
- com.squareup.okio:okio-jvm
License: Apache License 2.0
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS

View File

@@ -0,0 +1,25 @@
SLF4J API
Artifact: org.slf4j:slf4j-api
License: MIT License
Copyright (c) 2004-2022 QOS.ch Sarl (Switzerland)
All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -0,0 +1,34 @@
dnsjava
Artifact: dnsjava:dnsjava
License: BSD 3-Clause
Copyright (c) 1998-2019, Brian Wellington
Copyright (c) 2005 VeriSign. All rights reserved.
Copyright (c) 2019-2023, dnsjava authors
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -0,0 +1,23 @@
nibor autolink
Artifact: org.nibor.autolink:autolink
License: MIT License
Copyright (c) 2015 Robin Stocker
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

110
apps/android/VERSIONING.md Normal file
View File

@@ -0,0 +1,110 @@
# OpenClaw Android Versioning
Android release builds use pinned app metadata instead of auto-bumping `build.gradle.kts`.
## Version model
- `apps/android/version.json` is the source of truth.
- `version` is the Play `versionName` and uses CalVer: `YYYY.M.D`.
- `versionCode` uses `YYYYMMDDNN`, where `NN` is a two-digit build number for that pinned app version.
- `apps/android/Config/Version.properties` is generated from `version.json` and read by Gradle.
- `apps/android/CHANGELOG.md` is the Android-only changelog and release-note source.
- `apps/android/fastlane/metadata/android/en-US/release_notes.txt` is generated from the changelog.
Examples:
- `version = 2026.6.2`
- `versionCode = 2026060201`
- another upload on the same release train: `versionCode = 2026060202`
## Commands
```bash
pnpm android:version
pnpm android:version:check
pnpm android:version:sync
pnpm android:version:pin -- --from-gateway
pnpm android:version:pin -- --version 2026.6.5 --version-code 2026060501
pnpm android:release:signing:plan
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:sync:pull
pnpm android:release:preflight
```
## Release-note resolution order
When generating `apps/android/fastlane/metadata/android/en-US/release_notes.txt`, the tooling reads the first available changelog section in this order:
1. exact pinned version, for example `## 2026.6.2`
2. `## Unreleased`
Recommended workflow:
- while iterating on a Google Play release train, keep pending notes under `## Unreleased`
- before the production release, move or copy the final notes under `## <pinned version>` and run sync again
## Release Workflow
1. Pin Android to the intended release version.
2. Run `pnpm android:version:sync`.
3. Update `apps/android/CHANGELOG.md`, then run `pnpm android:version:sync` again if needed.
4. Run `MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:sync:pull` to materialize encrypted Android signing assets from `apps-signing`.
5. Run `pnpm android:release:preflight` to validate Play auth, signing, synced versioning, and release notes.
6. Run `ANDROID_SCREENSHOT_AVD=<avd-name> pnpm android:screenshots` to refresh raw Google Play screenshots with a script-managed emulator, or run `pnpm android:screenshots` when exactly one ADB device is already connected.
7. Run `pnpm android:release:archive` to produce the signed Play AAB and third-party APK.
8. Run `pnpm android:release:upload` to upload metadata, screenshots, and the Play AAB to the configured Google Play track.
9. Complete production rollout manually in Google Play Console when needed.
If `pnpm android:release:upload` fails, stop at that failure. Do not continue by
uploading archived artifacts through `pnpm android:release:archive`,
`pnpm android:release:metadata`, direct Fastlane lanes, Gradle release artifacts,
Google Play API mutation commands, or Play Console mutation commands. Fix the
failing release-lane step, then rerun `pnpm android:release:upload`.
The third-party flavor is archived as a signed APK for non-Play distribution. It is not uploaded by the Play release lane.
## Release SHA tracking
Successful Play build uploads create a non-tag Git ref that records the source
commit for the uploaded store build:
```text
refs/openclaw/mobile-releases/android/<versionName>-<versionCode>
```
Example:
```text
refs/openclaw/mobile-releases/android/2026.6.10-2026061008
```
These refs are intentionally outside `refs/tags/*` and `refs/heads/*`. They do
not appear on GitHub release or tag pages, and they do not participate in the
core OpenClaw release machinery.
`pnpm android:release:upload` checks the ref before uploading the Play build and
records it only after `upload_to_play_store` succeeds. Existing refs are
immutable: the same ref at the same SHA is accepted, while the same ref at a
different SHA fails. `GOOGLE_PLAY_VALIDATE_ONLY=1` still checks the ref but does
not record it because no Play build is published.
Do not create this ref after a manual fallback upload. The ref is release-lane
evidence, not a repair mechanism for a failed `pnpm android:release:upload` run.
Useful direct commands:
```bash
pnpm mobile:release:preflight -- --platform android --version 2026.6.10 --version-code 2026061008
pnpm mobile:release:resolve -- --platform android --version 2026.6.10 --version-code 2026061008
```
## Signing model
`apps/android/Config/ReleaseSigning.json` pins the Android signing assets in the shared private `apps-signing` repo. The Android pipeline uses the same `MATCH_PASSWORD` release-owner secret as iOS, but the Android files are managed by `scripts/android-release-signing.mjs` instead of Fastlane `match`.
`sync:pull` decrypts the Play upload keystore and Gradle signing properties into `apps/android/build/release-signing/`. That directory is gitignored, and Fastlane exports the materialized values as Gradle project properties for the current release command.
If `MATCH_PASSWORD` is not set, the existing manual Gradle-property signing path still works: provide `OPENCLAW_ANDROID_STORE_FILE`, `OPENCLAW_ANDROID_STORE_PASSWORD`, `OPENCLAW_ANDROID_KEY_ALIAS`, and `OPENCLAW_ANDROID_KEY_PASSWORD` through your local Gradle user properties before running release tasks.
Agent-driven releases must not use those lower-level signing and upload surfaces
to bypass a failed `pnpm android:release:upload` attempt. Report the failing
step and wait for maintainer direction instead.

View File

@@ -0,0 +1,328 @@
import com.android.build.api.variant.impl.VariantOutputImpl
import java.util.Properties
val dnsjavaInetAddressResolverService = "META-INF/services/java.net.spi.InetAddressResolverProvider"
val openClawAndroidVersionFile = rootProject.file("Config/Version.properties")
val thirdPartyLicensesDir = rootProject.file("THIRD_PARTY_LICENSES")
val openClawAndroidVersionProperties =
Properties().apply {
if (!openClawAndroidVersionFile.isFile) {
error("Missing Android version properties. Run `pnpm android:version:sync`.")
}
openClawAndroidVersionFile.inputStream().use(::load)
}
fun requireOpenClawAndroidVersionProperty(name: String): String =
openClawAndroidVersionProperties.getProperty(name)?.trim()?.takeIf { it.isNotEmpty() }
?: error("Missing $name in Config/Version.properties. Run `pnpm android:version:sync`.")
val openClawAndroidVersionName = requireOpenClawAndroidVersionProperty("OPENCLAW_ANDROID_VERSION_NAME")
val openClawAndroidVersionCode =
requireOpenClawAndroidVersionProperty("OPENCLAW_ANDROID_VERSION_CODE").toIntOrNull()
?: error("OPENCLAW_ANDROID_VERSION_CODE must be an integer in Config/Version.properties.")
val androidStoreFile = providers.gradleProperty("OPENCLAW_ANDROID_STORE_FILE").orNull?.takeIf { it.isNotBlank() }
val androidStorePassword = providers.gradleProperty("OPENCLAW_ANDROID_STORE_PASSWORD").orNull?.takeIf { it.isNotBlank() }
val androidKeyAlias = providers.gradleProperty("OPENCLAW_ANDROID_KEY_ALIAS").orNull?.takeIf { it.isNotBlank() }
val androidKeyPassword = providers.gradleProperty("OPENCLAW_ANDROID_KEY_PASSWORD").orNull?.takeIf { it.isNotBlank() }
val resolvedAndroidStoreFile =
androidStoreFile?.let { storeFilePath ->
if (storeFilePath.startsWith("~/")) {
"${System.getProperty("user.home")}/${storeFilePath.removePrefix("~/")}"
} else {
storeFilePath
}
}
val hasAndroidReleaseSigning =
listOf(resolvedAndroidStoreFile, androidStorePassword, androidKeyAlias, androidKeyPassword).all { it != null }
val wantsAndroidReleaseBuild =
gradle.startParameter.taskNames.any { taskName ->
taskName.contains("Release", ignoreCase = true) ||
Regex("""(^|:)(bundle|assemble)$""").containsMatchIn(taskName)
}
if (wantsAndroidReleaseBuild && !hasAndroidReleaseSigning) {
error(
"Missing Android release signing properties. Set OPENCLAW_ANDROID_STORE_FILE, " +
"OPENCLAW_ANDROID_STORE_PASSWORD, OPENCLAW_ANDROID_KEY_ALIAS, and " +
"OPENCLAW_ANDROID_KEY_PASSWORD in ~/.gradle/gradle.properties.",
)
}
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.ktlint)
alias(libs.plugins.kotlin.compose)
alias(libs.plugins.kotlin.serialization)
}
android {
namespace = "ai.openclaw.app"
compileSdk = 36
// Release signing is local-only; keep the keystore path and passwords out of the repo.
signingConfigs {
if (hasAndroidReleaseSigning) {
create("release") {
storeFile = project.file(checkNotNull(resolvedAndroidStoreFile))
storePassword = checkNotNull(androidStorePassword)
keyAlias = checkNotNull(androidKeyAlias)
keyPassword = checkNotNull(androidKeyPassword)
}
}
}
sourceSets {
getByName("main") {
assets.directories.add("../../shared/OpenClawKit/Sources/OpenClawKit/Resources")
assets.directories.add(thirdPartyLicensesDir.path)
}
}
defaultConfig {
applicationId = "ai.openclaw.app"
minSdk = 31
targetSdk = 36
versionCode = openClawAndroidVersionCode
versionName = openClawAndroidVersionName
ndk {
// Support all major ABIs — native libs are tiny (~47 KB per ABI)
abiFilters += listOf("armeabi-v7a", "arm64-v8a", "x86", "x86_64")
}
}
flavorDimensions += "store"
productFlavors {
create("play") {
dimension = "store"
}
create("thirdParty") {
dimension = "store"
}
}
buildTypes {
release {
if (hasAndroidReleaseSigning) {
signingConfig = signingConfigs.getByName("release")
}
isMinifyEnabled = true
isShrinkResources = true
ndk {
debugSymbolLevel = "SYMBOL_TABLE"
}
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
}
debug {
isMinifyEnabled = false
}
}
buildFeatures {
compose = true
buildConfig = true
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
packaging {
resources {
excludes +=
setOf(
"/META-INF/{AL2.0,LGPL2.1}",
"/META-INF/*.version",
"/META-INF/LICENSE*.txt",
"DebugProbesKt.bin",
"kotlin-tooling-metadata.json",
"org/bouncycastle/pqc/crypto/picnic/lowmcL1.bin.properties",
"org/bouncycastle/pqc/crypto/picnic/lowmcL3.bin.properties",
"org/bouncycastle/pqc/crypto/picnic/lowmcL5.bin.properties",
"org/bouncycastle/x509/CertPathReviewerMessages*.properties",
)
}
}
lint {
lintConfig = file("lint.xml")
warningsAsErrors = true
}
testOptions {
unitTests.isIncludeAndroidResources = true
}
}
androidComponents {
onVariants { variant ->
variant.outputs
.filterIsInstance<VariantOutputImpl>()
.forEach { output ->
val versionName = output.versionName.orNull ?: "0"
val buildType = variant.buildType
val flavorName = variant.flavorName?.takeIf { it.isNotBlank() }
val outputFileName =
if (flavorName == null) {
"openclaw-$versionName-$buildType.apk"
} else {
"openclaw-$versionName-$flavorName-$buildType.apk"
}
output.outputFileName = outputFileName
}
}
}
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
allWarningsAsErrors.set(true)
}
}
ktlint {
android.set(true)
ignoreFailures.set(false)
filter {
exclude("**/build/**")
}
}
dependencies {
val composeBom = platform(libs.androidx.compose.bom)
implementation(composeBom)
androidTestImplementation(composeBom)
implementation(libs.androidx.core.ktx)
implementation(libs.androidx.lifecycle.runtime.ktx)
implementation(libs.androidx.activity.compose)
implementation(libs.androidx.webkit)
implementation(libs.androidx.compose.ui)
implementation(libs.androidx.compose.ui.tooling.preview)
implementation(libs.androidx.compose.material3)
// material-icons-extended pulled in full icon set (~20 MB DEX). Only ~18 icons used.
// R8 will tree-shake unused icons when minify is enabled on release builds.
implementation(libs.androidx.compose.material.icons.extended)
debugImplementation(libs.androidx.compose.ui.tooling)
// Material Components (XML theme + resources)
implementation(libs.material)
implementation(libs.kotlinx.coroutines.android)
implementation(libs.kotlinx.serialization.json)
implementation(libs.androidx.security.crypto)
implementation(libs.androidx.exifinterface)
implementation(libs.okhttp)
implementation(libs.bcprov)
implementation(libs.commonmark)
implementation(libs.commonmark.ext.autolink)
implementation(libs.commonmark.ext.gfm.strikethrough)
implementation(libs.commonmark.ext.gfm.tables)
implementation(libs.commonmark.ext.task.list.items)
// CameraX (for node.invoke camera.* parity)
implementation(libs.androidx.camera.core)
implementation(libs.androidx.camera.camera2)
implementation(libs.androidx.camera.lifecycle)
implementation(libs.androidx.camera.view)
implementation(libs.androidx.camera.video)
implementation(libs.barcode.scanning)
// Unicast DNS-SD (Wide-Area Bonjour) for tailnet discovery domains.
implementation(libs.dnsjava)
testImplementation(libs.junit)
testImplementation(libs.kotlinx.coroutines.test)
testImplementation(libs.kotest.runner.junit5)
testImplementation(libs.kotest.assertions.core)
testImplementation(libs.mockwebserver)
testImplementation(libs.robolectric)
testRuntimeOnly(libs.junit.vintage.engine)
}
tasks.withType<Test>().configureEach {
useJUnitPlatform()
}
val validateThirdPartyLicenseAssets =
tasks.register("validateThirdPartyLicenseAssets") {
inputs.dir(thirdPartyLicensesDir)
doLast {
if (!thirdPartyLicensesDir.isDirectory) {
error("Missing Android third-party license directory: ${thirdPartyLicensesDir.relativeTo(rootProject.projectDir)}")
}
val invalidFiles =
thirdPartyLicensesDir
.walkTopDown()
.filter { file -> file.isFile && file.extension.lowercase() != "txt" }
.map { file -> file.relativeTo(thirdPartyLicensesDir).path }
.toList()
if (invalidFiles.isNotEmpty()) {
error(
"Android third-party license assets must be .txt files:\n" +
invalidFiles.joinToString(separator = "\n") { path -> "- $path" },
)
}
}
}
tasks.matching { task -> task.name == "preBuild" }.configureEach {
dependsOn(validateThirdPartyLicenseAssets)
}
androidComponents {
onVariants(selector().withBuildType("release")) { variant ->
val variantName = variant.name
val variantNameCapitalized = variantName.replaceFirstChar(Char::titlecase)
val stripTaskName = "strip${variantNameCapitalized}DnsjavaServiceDescriptor"
val mergeTaskName = "merge${variantNameCapitalized}JavaResource"
val minifyTaskName = "minify${variantNameCapitalized}WithR8"
val mergedJar =
layout.buildDirectory.file(
"intermediates/merged_java_res/$variantName/$mergeTaskName/base.jar",
)
val stripTask =
tasks.register(stripTaskName) {
inputs.file(mergedJar)
outputs.file(mergedJar)
doLast {
val jarFile = mergedJar.get().asFile
if (!jarFile.exists()) {
return@doLast
}
val unpackDir = temporaryDir.resolve("merged-java-res")
delete(unpackDir)
copy {
from(zipTree(jarFile))
into(unpackDir)
exclude(dnsjavaInetAddressResolverService)
}
delete(jarFile)
ant.invokeMethod(
"zip",
mapOf(
"destfile" to jarFile.absolutePath,
"basedir" to unpackDir.absolutePath,
),
)
}
}
tasks.matching { it.name == mergeTaskName }.configureEach {
finalizedBy(stripTask)
}
tasks.matching { it.name == minifyTaskName }.configureEach {
dependsOn(stripTask)
}
}
}

13
apps/android/app/lint.xml Normal file
View File

@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<lint>
<issue id="AndroidGradlePluginVersion" severity="ignore" />
<issue id="GradleDependency" severity="ignore" />
<issue id="IconLauncherShape" severity="ignore" />
<issue id="NewerVersionAvailable" severity="ignore" />
<!-- OpenClaw uses date-based version codes (yyyyMMddNN), which are high but still below the Android max. -->
<issue id="HighAppVersionCode" severity="ignore" />
<!-- Target SDK follows the current release train; bump only after platform compatibility testing. -->
<issue id="OldTargetApi" severity="ignore" />
</lint>

8
apps/android/app/proguard-rules.pro vendored Normal file
View File

@@ -0,0 +1,8 @@
-dontwarn org.bouncycastle.**
-dontwarn okhttp3.**
-dontwarn okio.**
-dontwarn com.sun.jna.**
-dontwarn javax.naming.**
-dontwarn lombok.Generated
-dontwarn org.slf4j.impl.StaticLoggerBinder
-dontwarn sun.net.spi.nameservice.NameServiceDescriptor

View File

@@ -0,0 +1,22 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<permission
android:name="${applicationId}.permission.RUN_VOICE_E2E"
android:protectionLevel="signature" />
<uses-permission android:name="${applicationId}.permission.RUN_VOICE_E2E" />
<application>
<receiver
android:name=".VoiceE2eReceiver"
android:permission="${applicationId}.permission.RUN_VOICE_E2E"
android:exported="false">
<intent-filter>
<action android:name="ai.openclaw.app.debug.RUN_VOICE_E2E" />
</intent-filter>
</receiver>
<service
android:name=".VoiceE2eService"
android:exported="false" />
</application>
</manifest>

View File

@@ -0,0 +1,195 @@
package ai.openclaw.app
import android.app.Service
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.os.IBinder
import android.util.Base64
import android.util.Log
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withTimeout
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import java.io.File
private const val tag = "VoiceE2E"
private const val resultFileName = "voice_e2e_result.json"
class VoiceE2eReceiver : BroadcastReceiver() {
override fun onReceive(
context: Context,
intent: Intent,
) {
context.startService(
Intent(context, VoiceE2eService::class.java)
.putExtras(intent),
)
}
}
class VoiceE2eService : Service() {
private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
override fun onBind(intent: Intent?): IBinder? = null
override fun onStartCommand(
intent: Intent?,
flags: Int,
startId: Int,
): Int {
val command = intent ?: return START_NOT_STICKY
serviceScope.launch {
try {
runCommand(command)
} finally {
stopSelf(startId)
}
}
return START_NOT_STICKY
}
override fun onDestroy() {
serviceScope.cancel()
super.onDestroy()
}
private suspend fun runCommand(intent: Intent) {
try {
val app = applicationContext as NodeApp
val runtime = app.ensureRuntime()
val mode =
intent
.getDecodedStringExtra("mode")
?.trim()
.orEmpty()
.ifEmpty { "both" }
if (mode == "stop") {
runtime.cancelMicCapture()
runtime.setTalkModeEnabled(false)
writeResult("""{"ok":true,"mode":"stop"}""")
return
}
val connect = !intent.getBooleanExtra("noConnect", false)
val connectTimeoutMs = intent.getLongExtra("connectTimeoutMs", 20_000L)
if (connect) {
configureGateway(runtime = runtime, intent = intent)
}
if (connect || !runtime.isConnected.value) {
awaitGateway(runtime = runtime, timeoutMs = connectTimeoutMs)
}
startActivity(
Intent(actionOpenVoiceE2e)
.setClass(this, MainActivity::class.java)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP),
)
if (mode == "connect") {
val resultJson = """{"ok":true,"mode":"connect","connected":true}"""
writeResult(resultJson)
Log.i(tag, "PASS $resultJson")
return
}
val transcript =
intent
.getDecodedStringExtra("transcript")
?.trim()
.orEmpty()
.ifEmpty { "Reply exactly: Android voice e2e normal path ok." }
val realtimeReply =
intent
.getDecodedStringExtra("realtimeAssistant")
?.trim()
.orEmpty()
.ifEmpty { "Android realtime voice e2e relay path ok." }
val timeoutMs = intent.getLongExtra("timeoutMs", 60_000L)
val result =
runtime.runVoiceE2e(
mode = mode,
transcript = transcript,
realtimeAssistantText = realtimeReply,
timeoutMs = timeoutMs,
)
val resultJson = encodeResult(result)
writeResult(resultJson)
Log.i(tag, "PASS $resultJson")
} catch (err: Throwable) {
val resultJson =
buildJsonObject {
put("ok", JsonPrimitive(false))
put("error", JsonPrimitive(err.message ?: err::class.java.simpleName))
}.toString()
writeResult(resultJson)
Log.e(tag, "FAIL $resultJson", err)
}
}
private fun configureGateway(
runtime: NodeRuntime,
intent: Intent,
) {
val host =
intent
.getDecodedStringExtra("host")
?.trim()
.orEmpty()
.ifEmpty { "127.0.0.1" }
val port = intent.getIntExtra("port", 18789)
runtime.setManualEnabled(true)
runtime.setManualHost(host)
runtime.setManualPort(port)
runtime.setManualTls(intent.getBooleanExtra("tls", false))
runtime.setGatewayToken(intent.getDecodedStringExtra("token").orEmpty())
runtime.setGatewayBootstrapToken(intent.getDecodedStringExtra("bootstrapToken").orEmpty())
runtime.setGatewayPassword(intent.getDecodedStringExtra("password").orEmpty())
runtime.setOnboardingCompleted(true)
runtime.connectManual()
}
private suspend fun awaitGateway(
runtime: NodeRuntime,
timeoutMs: Long,
) {
withTimeout(timeoutMs) {
while (!runtime.isConnected.value) {
delay(100L)
}
}
}
private fun encodeResult(result: NodeRuntime.VoiceE2eResult): String =
buildJsonObject {
put("ok", JsonPrimitive(true))
put("normal", result.normal?.let(::encodeSlice) ?: JsonNull)
put("realtime", result.realtime?.let(::encodeSlice) ?: JsonNull)
}.toString()
private fun encodeSlice(slice: NodeRuntime.VoiceE2eSliceResult) =
buildJsonObject {
put("mode", JsonPrimitive(slice.mode))
put("status", JsonPrimitive(slice.status))
put("userText", slice.userText?.let(::JsonPrimitive) ?: JsonNull)
put("assistantText", slice.assistantText?.let(::JsonPrimitive) ?: JsonNull)
}
private fun writeResult(json: String) {
File(cacheDir, resultFileName).writeText(json)
}
}
private fun Intent.getDecodedStringExtra(name: String): String? {
val encoded = getStringExtra("${name}Base64")
if (!encoded.isNullOrBlank()) {
return String(Base64.decode(encoded, Base64.NO_WRAP), Charsets.UTF_8)
}
return getStringExtra(name)
}

View File

@@ -0,0 +1,91 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_CONNECTED_DEVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission
android:name="android.permission.NEARBY_WIFI_DEVICES"
android:usesPermissionFlags="neverForLocation" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
<uses-permission android:name="android.permission.READ_MEDIA_VISUAL_USER_SELECTED" />
<uses-permission
android:name="android.permission.READ_EXTERNAL_STORAGE"
android:maxSdkVersion="32" />
<uses-permission android:name="android.permission.READ_CONTACTS" />
<uses-permission android:name="android.permission.WRITE_CONTACTS" />
<uses-permission android:name="android.permission.READ_CALENDAR" />
<uses-permission android:name="android.permission.WRITE_CALENDAR" />
<uses-permission android:name="android.permission.ACTIVITY_RECOGNITION" />
<uses-feature
android:name="android.hardware.camera"
android:required="false" />
<uses-feature
android:name="android.hardware.telephony"
android:required="false" />
<queries>
<intent>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent>
</queries>
<application
android:name=".NodeApp"
android:allowBackup="false"
android:dataExtractionRules="@xml/data_extraction_rules"
android:fullBackupContent="@xml/backup_rules"
android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round"
android:label="@string/app_name"
android:supportsRtl="true"
android:networkSecurityConfig="@xml/network_security_config"
android:theme="@style/Theme.OpenClawNode">
<service
android:name=".NodeForegroundService"
android:exported="false"
android:foregroundServiceType="connectedDevice|microphone" />
<service
android:name=".node.DeviceNotificationListenerService"
android:label="@string/app_name"
android:permission="android.permission.BIND_NOTIFICATION_LISTENER_SERVICE"
android:exported="false">
<intent-filter>
<action android:name="android.service.notification.NotificationListenerService" />
</intent-filter>
</service>
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.fileprovider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/file_paths" />
</provider>
<activity
android:name=".MainActivity"
android:exported="true"
android:windowSoftInputMode="adjustResize"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|uiMode|density|keyboard|keyboardHidden|navigation">
<meta-data
android:name="android.app.shortcuts"
android:resource="@xml/shortcuts" />
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.ASSIST" />
<category android:name="android.intent.category.DEFAULT" />
</intent-filter>
</activity>
</application>
</manifest>

View File

@@ -0,0 +1,40 @@
package ai.openclaw.app
import android.content.res.AssetManager
internal const val ANDROID_LICENSE_ASSET_DIRECTORY = "openclaw/licenses"
internal data class AndroidLicenseNotice(
val title: String,
val fileName: String,
val text: String,
)
internal fun loadAndroidLicenseNotices(assetManager: AssetManager): List<AndroidLicenseNotice> {
val files =
assetManager
.list(ANDROID_LICENSE_ASSET_DIRECTORY)
.orEmpty()
.filter(::isAndroidLicenseFileName)
return files
.map { fileName ->
val rawText =
assetManager
.open("$ANDROID_LICENSE_ASSET_DIRECTORY/$fileName")
.bufferedReader(Charsets.UTF_8)
.use { reader -> reader.readText() }
AndroidLicenseNotice(title = androidLicenseTitleFromFileName(fileName), fileName = fileName, text = rawText)
}.sortedWith(
compareBy<AndroidLicenseNotice, String>(String.CASE_INSENSITIVE_ORDER) { notice -> notice.title }
.thenBy(String.CASE_INSENSITIVE_ORDER) { notice -> notice.fileName },
)
}
internal fun isAndroidLicenseFileName(fileName: String): Boolean = fileName.endsWith(".txt", ignoreCase = true)
internal fun androidLicenseTitleFromFileName(fileName: String): String =
fileName
.substringBeforeLast('.')
.trim()
.ifBlank { "License" }

View File

@@ -0,0 +1,28 @@
package ai.openclaw.app
import android.content.Intent
const val extraAndroidScreenshotMode = "openclaw.screenshotMode"
const val extraAndroidScreenshotScene = "openclaw.screenshotScene"
enum class AndroidScreenshotScene(
val rawValue: String,
) {
Connect("connect"),
Chat("chat"),
Voice("voice"),
Screen("screen"),
Settings("settings"),
;
companion object {
fun fromRawValue(raw: String?): AndroidScreenshotScene = entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Connect
}
}
fun parseAndroidScreenshotModeIntent(intent: Intent?): AndroidScreenshotScene? {
if (intent?.getBooleanExtra(extraAndroidScreenshotMode, false) != true) {
return null
}
return AndroidScreenshotScene.fromRawValue(intent.getStringExtra(extraAndroidScreenshotScene))
}

View File

@@ -0,0 +1,25 @@
package ai.openclaw.app
/** User-selectable app theme mode for Android appearance settings. */
enum class AppearanceThemeMode(
val rawValue: String,
val displayLabel: String,
) {
System(rawValue = "system", displayLabel = "System"),
Dark(rawValue = "dark", displayLabel = "Dark"),
Light(rawValue = "light", displayLabel = "Light"),
;
fun isDark(systemDark: Boolean): Boolean =
when (this) {
System -> systemDark
Dark -> true
Light -> false
}
companion object {
fun fromRawValue(value: String?): AppearanceThemeMode = entries.firstOrNull { it.rawValue == value?.trim()?.lowercase() } ?: Dark
fun fromDisplayLabel(label: String): AppearanceThemeMode = entries.firstOrNull { it.displayLabel.equals(label.trim(), ignoreCase = true) } ?: Dark
}
}

View File

@@ -0,0 +1,70 @@
package ai.openclaw.app
import android.content.Intent
/** Android Assistant entry point used by manifest-declared app actions. */
const val actionAskOpenClaw = "ai.openclaw.app.action.ASK_OPENCLAW"
/** Debug action that opens the Voice tab directly for Android E2E automation. */
const val actionOpenVoiceE2e = "ai.openclaw.app.debug.OPEN_VOICE_E2E"
/** Intent extra that carries an optional assistant prompt for app actions. */
const val extraAssistantPrompt = "prompt"
/**
* Top-level home destinations that external actions may request.
*/
enum class HomeDestination {
Connect,
Chat,
Voice,
Screen,
Settings,
}
/**
* Normalized launch request from Android Assistant or explicit app actions.
*/
data class AssistantLaunchRequest(
val source: String,
val prompt: String?,
val autoSend: Boolean,
)
/**
* Parses app-owned navigation actions that should open a specific home tab.
*/
fun parseHomeDestinationIntent(intent: Intent?): HomeDestination? {
val action = intent?.action ?: return null
return when {
// Debug-only shortcut keeps E2E navigation out of release builds.
BuildConfig.DEBUG && action == actionOpenVoiceE2e -> HomeDestination.Voice
else -> null
}
}
/**
* Parse external assistant entry points without starting any UI side effects.
*/
fun parseAssistantLaunchIntent(intent: Intent?): AssistantLaunchRequest? {
val action = intent?.action ?: return null
return when (action) {
Intent.ACTION_ASSIST ->
AssistantLaunchRequest(
source = "assist",
prompt = null,
autoSend = false,
)
actionAskOpenClaw -> {
val prompt = intent.getStringExtra(extraAssistantPrompt)?.trim()?.ifEmpty { null }
AssistantLaunchRequest(
source = "app_action",
prompt = prompt,
autoSend = false,
)
}
else -> null
}
}

View File

@@ -0,0 +1,16 @@
package ai.openclaw.app
/** Camera HUD state categories shown over the Android UI during capture. */
enum class CameraHudKind {
Photo,
Recording,
Success,
Error,
}
/** One-shot camera HUD message keyed by token so repeated text still replays. */
data class CameraHudState(
val token: Long,
val kind: CameraHudKind,
val message: String,
)

View File

@@ -0,0 +1,28 @@
package ai.openclaw.app
import android.content.Context
import android.os.Build
import android.provider.Settings
object DeviceNames {
/** Prefers the user-visible Android device name, then falls back to manufacturer/model text. */
fun bestDefaultNodeName(context: Context): String {
val deviceName =
runCatching {
Settings.Global.getString(context.contentResolver, "device_name")
}.getOrNull()
?.trim()
.orEmpty()
if (deviceName.isNotEmpty()) return deviceName
// Manufacturer/model are best-effort platform fields; keep the final
// fallback stable so stored default names do not become blank.
val model =
listOfNotNull(Build.MANUFACTURER?.takeIf { it.isNotBlank() }, Build.MODEL?.takeIf { it.isNotBlank() })
.joinToString(" ")
.trim()
return model.ifEmpty { "Android Node" }
}
}

View File

@@ -0,0 +1,163 @@
package ai.openclaw.app
import ai.openclaw.app.node.asObjectOrNull
import ai.openclaw.app.node.asStringOrNull
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
data class GatewayExecApprovalSummary(
val id: String,
val commandText: String,
val commandPreview: String?,
val allowedDecisions: List<String>,
val host: String?,
val nodeId: String?,
val agentId: String?,
val createdAtMs: Long?,
val expiresAtMs: Long?,
val resolvingDecision: String? = null,
val errorText: String? = null,
)
internal fun parseGatewayExecApprovalListPayload(
payloadJson: String,
json: Json,
): List<GatewayExecApprovalSummary> =
try {
(json.parseToJsonElement(payloadJson) as? JsonArray)
?.mapNotNull(::parseGatewayExecApprovalListEntry)
?.sortedBy { it.createdAtMs ?: Long.MAX_VALUE }
.orEmpty()
} catch (_: Throwable) {
emptyList()
}
internal fun parseGatewayExecApprovalListEntry(item: JsonElement): GatewayExecApprovalSummary? {
val obj = item.asObjectOrNull() ?: return null
val id = obj["id"].asStringOrNull()?.trim().orEmpty()
if (id.isEmpty()) return null
val request = obj["request"].asObjectOrNull()
val commandText = gatewayExecApprovalListCommandText(obj, request)
return GatewayExecApprovalSummary(
id = id,
commandText = commandText,
commandPreview = gatewayExecApprovalListCommandPreview(obj, request, commandText),
allowedDecisions = emptyList(),
host =
request
?.get("host")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
nodeId =
request
?.get("nodeId")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
agentId =
request
?.get("agentId")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
createdAtMs = obj.long("createdAtMs"),
expiresAtMs = obj.long("expiresAtMs"),
)
}
internal fun parseGatewayExecApprovalDetail(
obj: JsonObject,
createdAtMs: Long?,
): GatewayExecApprovalSummary? {
val id = obj["id"].asStringOrNull()?.trim().orEmpty()
if (id.isEmpty()) return null
return GatewayExecApprovalSummary(
id = id,
commandText =
obj["commandText"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: "Command request",
commandPreview =
obj["commandPreview"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
allowedDecisions = gatewayExecApprovalAllowedDecisions(obj),
host = obj["host"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() },
nodeId = obj["nodeId"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() },
agentId = obj["agentId"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() },
createdAtMs = createdAtMs,
expiresAtMs = obj.long("expiresAtMs"),
)
}
private fun gatewayExecApprovalListCommandText(
obj: JsonObject,
request: JsonObject?,
): String =
obj["commandText"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: request
?.get("command")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: "Command request"
private fun gatewayExecApprovalListCommandPreview(
obj: JsonObject,
request: JsonObject?,
commandText: String,
): String? {
val preview =
obj["commandPreview"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: request
?.get("commandPreview")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
return preview?.takeIf { it != commandText }
}
private fun gatewayExecApprovalAllowedDecisions(request: JsonObject?): List<String> {
val explicit = parseGatewayExecApprovalDecisions(request?.get("allowedDecisions") as? JsonArray)
if (explicit.isNotEmpty()) return explicit
val allowed =
if (request
?.get("ask")
.asStringOrNull()
?.trim()
?.lowercase() == "always"
) {
listOf("allow-once", "deny")
} else {
listOf("allow-once", "allow-always", "deny")
}
val unavailable = parseGatewayExecApprovalDecisions(request?.get("unavailableDecisions") as? JsonArray).toSet()
return allowed.filterNot { it == "allow-always" && it in unavailable }
}
private fun parseGatewayExecApprovalDecisions(items: JsonArray?): List<String> =
items
?.mapNotNull { item ->
when (item.asStringOrNull()?.trim()) {
"allow-once" -> "allow-once"
"allow-always" -> "allow-always"
"deny" -> "deny"
else -> null
}
}?.distinct()
.orEmpty()
private fun JsonObject?.long(key: String): Long? = (this?.get(key) as? JsonPrimitive)?.content?.trim()?.toLongOrNull()

View File

@@ -0,0 +1,216 @@
package ai.openclaw.app
import ai.openclaw.app.node.asObjectOrNull
import ai.openclaw.app.node.asStringOrNull
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.booleanOrNull
data class GatewayTalkSetupReadiness(
val realtimeTalk: GatewayTalkSetupState,
val dictation: GatewayTalkSetupState,
) {
companion object {
fun unverified(
issue: GatewayTalkSetupIssue = GatewayTalkSetupIssue.CatalogNotLoaded,
): GatewayTalkSetupReadiness =
GatewayTalkSetupReadiness(
realtimeTalk = GatewayTalkSetupState.Unverified(issue),
dictation = GatewayTalkSetupState.Unverified(issue),
)
}
}
sealed interface GatewayTalkSetupState {
data class Ready(
val provider: GatewayTalkProvider,
) : GatewayTalkSetupState
data class NeedsSetup(
val issue: GatewayTalkSetupIssue,
val provider: GatewayTalkProvider? = null,
) : GatewayTalkSetupState
/** Catalog failures must not disable a startup path that the Gateway still validates. */
data class Unverified(
val issue: GatewayTalkSetupIssue,
) : GatewayTalkSetupState
}
enum class GatewayTalkSetupTarget(
val title: String,
) {
REALTIME_TALK("Realtime Talk"),
DICTATION("Dictation"),
}
sealed interface GatewayTalkSetupIssue {
data object CatalogNotLoaded : GatewayTalkSetupIssue
data object CatalogLoadFailed : GatewayTalkSetupIssue
data class GroupMissing(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class NoProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class UnknownProvider(
val target: GatewayTalkSetupTarget,
val providerId: String,
) : GatewayTalkSetupIssue
data class MissingReadiness(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class ConfigureProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class MissingActiveProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class UnsupportedProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class ConfigureSelectedProvider(
val providerLabel: String,
) : GatewayTalkSetupIssue
}
data class GatewayTalkProvider(
val id: String,
val label: String,
)
val GatewayTalkSetupState.isReady: Boolean
get() = this is GatewayTalkSetupState.Ready
val GatewayTalkSetupState.requiresSetup: Boolean
get() = this is GatewayTalkSetupState.NeedsSetup
fun gatewayTalkSetupStatusText(state: GatewayTalkSetupState): String =
when (state) {
is GatewayTalkSetupState.Ready -> "Ready"
is GatewayTalkSetupState.NeedsSetup -> "Needs setup"
is GatewayTalkSetupState.Unverified -> "Unverified"
}
fun gatewayTalkSetupDescription(state: GatewayTalkSetupState): String =
when (state) {
is GatewayTalkSetupState.Ready -> "${state.provider.label} via Gateway relay"
is GatewayTalkSetupState.NeedsSetup -> gatewayTalkSetupIssueDescription(state.issue)
is GatewayTalkSetupState.Unverified -> gatewayTalkSetupIssueDescription(state.issue)
}
private fun gatewayTalkSetupIssueDescription(issue: GatewayTalkSetupIssue): String =
when (issue) {
GatewayTalkSetupIssue.CatalogNotLoaded -> "Gateway talk catalog not loaded"
GatewayTalkSetupIssue.CatalogLoadFailed -> "Could not load Gateway talk catalog"
is GatewayTalkSetupIssue.GroupMissing -> "Gateway did not return ${issue.target.title} setup"
is GatewayTalkSetupIssue.NoProvider -> "No ${issue.target.title} provider is configured on the Gateway"
is GatewayTalkSetupIssue.UnknownProvider -> "Gateway selected unknown provider ${issue.providerId}"
is GatewayTalkSetupIssue.MissingReadiness -> "Gateway did not return ${issue.target.title} readiness"
is GatewayTalkSetupIssue.ConfigureProvider -> "Configure a ${issue.target.title} provider on the Gateway"
is GatewayTalkSetupIssue.MissingActiveProvider ->
"Gateway did not identify the active ${issue.target.title} provider"
is GatewayTalkSetupIssue.UnsupportedProvider ->
"Choose a supported ${issue.target.title} provider on the Gateway"
is GatewayTalkSetupIssue.ConfigureSelectedProvider -> "Configure ${issue.providerLabel} on the Gateway"
}
internal fun parseGatewayTalkSetupReadiness(catalog: JsonObject?): GatewayTalkSetupReadiness {
if (catalog == null) return GatewayTalkSetupReadiness.unverified()
return GatewayTalkSetupReadiness(
realtimeTalk =
parseTalkCatalogGroup(catalog = catalog, key = "realtime", target = GatewayTalkSetupTarget.REALTIME_TALK),
dictation =
parseTalkCatalogGroup(catalog = catalog, key = "transcription", target = GatewayTalkSetupTarget.DICTATION),
)
}
private fun parseTalkCatalogGroup(
catalog: JsonObject,
key: String,
target: GatewayTalkSetupTarget,
): GatewayTalkSetupState {
val group =
catalog[key].asObjectOrNull()
?: return GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.GroupMissing(target))
val providers =
(group["providers"] as? JsonArray)
?.mapNotNull(::parseTalkCatalogProvider)
.orEmpty()
val ready = (group["ready"] as? JsonPrimitive)?.booleanOrNull
val activeProviderId = group["activeProvider"].asStringOrNull()?.trim()?.takeIf(String::isNotEmpty)
if (providers.isEmpty()) {
return when {
ready == false -> GatewayTalkSetupState.NeedsSetup(GatewayTalkSetupIssue.NoProvider(target))
activeProviderId != null ->
GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.UnknownProvider(target, activeProviderId))
else -> GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.MissingReadiness(target))
}
}
if (activeProviderId == null) {
if (ready == false) {
return GatewayTalkSetupState.NeedsSetup(GatewayTalkSetupIssue.ConfigureProvider(target))
}
// Older Gateways can omit the selected provider and report alias-backed rows as unconfigured
// even though session startup resolves them. Only an explicit readiness result is authoritative.
return GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.MissingActiveProvider(target))
}
val selected =
// Match Gateway registry precedence: canonical ids win before alias fallback.
providers.firstOrNull { it.matchesId(activeProviderId) }
?: providers.firstOrNull { it.matchesAlias(activeProviderId) }
?: return if (ready == false) {
GatewayTalkSetupState.NeedsSetup(GatewayTalkSetupIssue.UnsupportedProvider(target))
} else {
GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.UnknownProvider(target, activeProviderId))
}
val provider = GatewayTalkProvider(id = selected.id, label = selected.label)
return when (ready) {
true -> GatewayTalkSetupState.Ready(provider)
false ->
GatewayTalkSetupState.NeedsSetup(
issue = GatewayTalkSetupIssue.ConfigureSelectedProvider(selected.label),
provider = provider,
)
null -> GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.MissingReadiness(target))
}
}
private data class TalkCatalogProvider(
val id: String,
val label: String,
val configured: Boolean,
val aliases: List<String>,
) {
fun matchesId(candidate: String): Boolean = id.equals(candidate, ignoreCase = true)
fun matchesAlias(candidate: String): Boolean = aliases.any { it.equals(candidate, ignoreCase = true) }
}
private fun parseTalkCatalogProvider(item: JsonElement): TalkCatalogProvider? {
val value = item.asObjectOrNull() ?: return null
val id = value["id"].asStringOrNull()?.trim()?.takeIf(String::isNotEmpty) ?: return null
val label = value["label"].asStringOrNull()?.trim()?.takeIf(String::isNotEmpty) ?: id
val aliases =
(value["aliases"] as? JsonArray)
?.mapNotNull { it.asStringOrNull()?.trim()?.takeIf(String::isNotEmpty) }
.orEmpty()
return TalkCatalogProvider(
id = id,
label = label,
configured = (value["configured"] as? JsonPrimitive)?.booleanOrNull == true,
aliases = aliases,
)
}

View File

@@ -0,0 +1,22 @@
package ai.openclaw.app
/**
* Persisted location capture mode advertised to the gateway.
*/
enum class LocationMode(
val rawValue: String,
) {
Off("off"),
WhileUsing("whileUsing"),
;
companion object {
/** Parses persisted location mode text while migrating old always-on configs to while-using. */
fun fromRawValue(raw: String?): LocationMode {
val normalized = raw?.trim()?.lowercase()
// Older configs used "always"; Android node currently exposes while-using location only.
if (normalized == "always") return WhileUsing
return entries.firstOrNull { it.rawValue.lowercase() == normalized } ?: Off
}
}
}

View File

@@ -0,0 +1,209 @@
package ai.openclaw.app
import ai.openclaw.app.ui.AndroidScreenshotModeScreen
import ai.openclaw.app.ui.OpenClawTheme
import ai.openclaw.app.ui.RootScreen
import android.content.Intent
import android.os.Bundle
import android.view.WindowManager
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.viewModels
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.withFrameNanos
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.sp
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.lifecycleScope
import androidx.lifecycle.repeatOnLifecycle
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
/**
* Main Android activity that owns Compose UI attachment and runtime UI wiring.
*/
class MainActivity : ComponentActivity() {
private val viewModel: MainViewModel by viewModels()
private lateinit var permissionRequester: PermissionRequester
private var initializedViewModel: MainViewModel? = null
private var didAttachRuntimeUi = false
private var didStartNodeService = false
private var didStartViewModelCollectors = false
private var foreground = false
private var pendingIntent: Intent? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
pendingIntent = intent
WindowCompat.setDecorFitsSystemWindows(window, false)
permissionRequester = PermissionRequester(this)
if (BuildConfig.DEBUG) {
parseAndroidScreenshotModeIntent(intent)?.let { scene ->
enterScreenshotMode(scene)
return
}
}
setContent {
var activeViewModel by remember { mutableStateOf<MainViewModel?>(null) }
LaunchedEffect(Unit) {
withFrameNanos { }
withContext(Dispatchers.Default) {
(application as NodeApp).prefs
}
val readyViewModel = viewModel
activateViewModel(readyViewModel)
activeViewModel = readyViewModel
}
val currentViewModel = activeViewModel
if (currentViewModel == null) {
OpenClawTheme {
StartupSurface()
}
} else {
val appearanceThemeMode by currentViewModel.appearanceThemeMode.collectAsState()
OpenClawTheme(themeMode = appearanceThemeMode) {
RootScreen(viewModel = currentViewModel)
}
}
}
}
private fun enterScreenshotMode(scene: AndroidScreenshotScene) {
hideScreenshotModeStatusBar()
setContent {
AndroidScreenshotModeScreen(scene = scene)
}
}
private fun hideScreenshotModeStatusBar() {
WindowCompat
.getInsetsController(window, window.decorView)
.apply {
systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
hide(WindowInsetsCompat.Type.statusBars())
}
}
override fun onStart() {
super.onStart()
foreground = true
initializedViewModel?.setForeground(true)
}
override fun onStop() {
foreground = false
initializedViewModel?.setForeground(false)
super.onStop()
}
override fun onNewIntent(intent: android.content.Intent) {
super.onNewIntent(intent)
setIntent(intent)
pendingIntent = intent
initializedViewModel?.let { handleAssistantIntent(viewModel = it, intent = intent) }
}
/**
* Wires MainViewModel only after Activity first draw and background prefs warm-up.
*/
private fun activateViewModel(readyViewModel: MainViewModel) {
if (initializedViewModel != null) return
initializedViewModel = readyViewModel
readyViewModel.setForeground(foreground)
startViewModelCollectors(readyViewModel)
pendingIntent?.let { initialIntent ->
handleAssistantIntent(viewModel = readyViewModel, intent = initialIntent)
pendingIntent = null
}
}
/**
* Starts lifecycle collectors after ViewModel construction so they cannot force early startup.
*/
private fun startViewModelCollectors(readyViewModel: MainViewModel) {
if (didStartViewModelCollectors) return
didStartViewModelCollectors = true
lifecycleScope.launch {
repeatOnLifecycle(Lifecycle.State.STARTED) {
readyViewModel.preventSleep.collect { enabled ->
if (enabled) {
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
} else {
window.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
}
}
}
}
lifecycleScope.launch {
repeatOnLifecycle(Lifecycle.State.STARTED) {
readyViewModel.runtimeInitialized.collect { ready ->
if (!ready || didAttachRuntimeUi) return@collect
// Runtime UI helpers need an Activity owner, so attach once after NodeRuntime is ready.
readyViewModel.attachRuntimeUi(owner = this@MainActivity, permissionRequester = permissionRequester)
didAttachRuntimeUi = true
if (!didStartNodeService) {
NodeForegroundService.start(this@MainActivity)
didStartNodeService = true
}
}
}
}
}
/**
* Routes assistant/app-action intents into ViewModel state without recreating the activity.
*/
private fun handleAssistantIntent(
viewModel: MainViewModel,
intent: Intent?,
) {
parseHomeDestinationIntent(intent)?.let { destination ->
viewModel.requestHomeDestination(destination)
return
}
val request = parseAssistantLaunchIntent(intent) ?: return
viewModel.handleAssistantLaunch(request)
}
}
@Composable
private fun StartupSurface() {
Surface(
modifier = Modifier.fillMaxSize(),
color = Color.Black,
contentColor = Color.White,
) {
Box(
modifier = Modifier.fillMaxSize(),
contentAlignment = Alignment.Center,
) {
Text(
text = "OPENCLAW",
fontSize = 22.sp,
fontWeight = FontWeight.Medium,
)
}
}
}

View File

@@ -0,0 +1,633 @@
package ai.openclaw.app
import ai.openclaw.app.chat.ChatCommandEntry
import ai.openclaw.app.chat.ChatMessage
import ai.openclaw.app.chat.ChatPendingToolCall
import ai.openclaw.app.chat.ChatSessionEntry
import ai.openclaw.app.chat.OutgoingAttachment
import ai.openclaw.app.gateway.DeviceAuthStore
import ai.openclaw.app.gateway.DeviceIdentityStore
import ai.openclaw.app.gateway.GatewayEndpoint
import ai.openclaw.app.gateway.GatewayUpdateAvailableSummary
import ai.openclaw.app.node.CameraCaptureManager
import ai.openclaw.app.node.CanvasController
import ai.openclaw.app.node.SmsManager
import ai.openclaw.app.ui.GatewayConnectPlan
import ai.openclaw.app.ui.GatewaySavedAuthAction
import ai.openclaw.app.voice.VoiceConversationEntry
import android.app.Application
import androidx.lifecycle.AndroidViewModel
import androidx.lifecycle.LifecycleOwner
import androidx.lifecycle.viewModelScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.flatMapLatest
import kotlinx.coroutines.flow.flowOf
import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.launch
/**
* UI-facing bridge that exposes NodeRuntime and preference state as Compose-friendly StateFlows.
*/
@OptIn(ExperimentalCoroutinesApi::class)
class MainViewModel(
app: Application,
) : AndroidViewModel(app) {
private val nodeApp = app as NodeApp
private val prefs = nodeApp.prefs
private val runtimeRef = MutableStateFlow<NodeRuntime?>(null)
@Volatile private var foreground = false
@Volatile private var runtimeStartupQueued = false
private val _requestedHomeDestination = MutableStateFlow<HomeDestination?>(null)
val requestedHomeDestination: StateFlow<HomeDestination?> = _requestedHomeDestination
private val _startOnboardingAtGatewaySetup = MutableStateFlow(false)
val startOnboardingAtGatewaySetup: StateFlow<Boolean> = _startOnboardingAtGatewaySetup
private val _chatDraft = MutableStateFlow<String?>(null)
val chatDraft: StateFlow<String?> = _chatDraft
private val _pendingAssistantAutoSend = MutableStateFlow<String?>(null)
val pendingAssistantAutoSend: StateFlow<String?> = _pendingAssistantAutoSend
/**
* Lazily starts NodeRuntime and preserves the current foreground bit across startup.
*/
private fun ensureRuntime(): NodeRuntime {
runtimeRef.value?.let { return it }
val runtime = nodeApp.ensureRuntime()
runtime.setForeground(foreground)
runtimeRef.value = runtime
return runtime
}
/**
* Starts the node runtime off the main thread so fresh installs can render
* the shell before encrypted prefs, device identity, and gateway setup warm up.
*/
private fun queueRuntimeStartup() {
if (runtimeRef.value != null || runtimeStartupQueued) return
runtimeStartupQueued = true
viewModelScope.launch(Dispatchers.Default) {
runCatching { ensureRuntime() }
runtimeStartupQueued = false
}
}
/**
* Adapts a runtime StateFlow to a stable ViewModel StateFlow before runtime startup.
*/
private fun <T> runtimeState(
initial: T,
selector: (NodeRuntime) -> StateFlow<T>,
): StateFlow<T> =
runtimeRef
.flatMapLatest { runtime -> runtime?.let(selector) ?: flowOf(initial) }
.stateIn(viewModelScope, SharingStarted.Eagerly, initial)
val runtimeInitialized: StateFlow<Boolean> =
runtimeRef
.flatMapLatest { runtime -> flowOf(runtime != null) }
.stateIn(viewModelScope, SharingStarted.Eagerly, false)
val canvasCurrentUrl: StateFlow<String?> = runtimeState(initial = null) { it.canvas.currentUrl }
val canvasA2uiHydrated: StateFlow<Boolean> = runtimeState(initial = false) { it.canvasA2uiHydrated }
val canvasRehydratePending: StateFlow<Boolean> = runtimeState(initial = false) { it.canvasRehydratePending }
val canvasRehydrateErrorText: StateFlow<String?> = runtimeState(initial = null) { it.canvasRehydrateErrorText }
val gateways: StateFlow<List<GatewayEndpoint>> = runtimeState(initial = emptyList()) { it.gateways }
val discoveryStatusText: StateFlow<String> = runtimeState(initial = "Searching…") { it.discoveryStatusText }
val notificationForwardingEnabled: StateFlow<Boolean> = prefs.notificationForwardingEnabled
val notificationForwardingMode: StateFlow<NotificationPackageFilterMode> =
prefs.notificationForwardingMode
val notificationForwardingPackages: StateFlow<Set<String>> = prefs.notificationForwardingPackages
val notificationForwardingQuietHoursEnabled: StateFlow<Boolean> =
prefs.notificationForwardingQuietHoursEnabled
val notificationForwardingQuietStart: StateFlow<String> = prefs.notificationForwardingQuietStart
val notificationForwardingQuietEnd: StateFlow<String> = prefs.notificationForwardingQuietEnd
val notificationForwardingMaxEventsPerMinute: StateFlow<Int> =
prefs.notificationForwardingMaxEventsPerMinute
val notificationForwardingSessionKey: StateFlow<String?> = prefs.notificationForwardingSessionKey
val isConnected: StateFlow<Boolean> = runtimeState(initial = false) { it.isConnected }
val gatewayControlPage: StateFlow<NodeRuntime.GatewayControlPage?> =
runtimeState(initial = null) { it.gatewayControlPage }
val isNodeConnected: StateFlow<Boolean> = runtimeState(initial = false) { it.nodeConnected }
val nodeCapabilityApproval: StateFlow<GatewayNodeCapabilityApproval> =
runtimeState(initial = GatewayNodeCapabilityApproval.Loading) { it.nodeCapabilityApproval }
val statusText: StateFlow<String> = runtimeState(initial = "Offline") { it.statusText }
val gatewayConnectionProblem: StateFlow<GatewayConnectionProblem?> = runtimeState(initial = null) { it.gatewayConnectionProblem }
val gatewayConnectionDisplay: StateFlow<GatewayConnectionDisplay> =
runtimeState(initial = GatewayConnectionDisplay(false, "Offline", null)) { it.gatewayConnectionDisplay }
val serverName: StateFlow<String?> = runtimeState(initial = null) { it.serverName }
val remoteAddress: StateFlow<String?> = runtimeState(initial = null) { it.remoteAddress }
val gatewayVersion: StateFlow<String?> = runtimeState(initial = null) { it.gatewayVersion }
val gatewayUpdateAvailable: StateFlow<GatewayUpdateAvailableSummary?> = runtimeState(initial = null) { it.gatewayUpdateAvailable }
val modelCatalog: StateFlow<List<GatewayModelSummary>> = runtimeState(initial = emptyList()) { it.modelCatalog }
val modelAuthProviders: StateFlow<List<GatewayModelProviderSummary>> = runtimeState(initial = emptyList()) { it.modelAuthProviders }
val modelCatalogRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.modelCatalogRefreshing }
val modelCatalogErrorText: StateFlow<String?> = runtimeState(initial = null) { it.modelCatalogErrorText }
val talkSetupReadiness: StateFlow<GatewayTalkSetupReadiness> =
runtimeState(initial = GatewayTalkSetupReadiness.unverified()) { it.talkSetupReadiness }
val gatewayDefaultAgentId: StateFlow<String?> = runtimeState(initial = null) { it.gatewayDefaultAgentId }
val gatewayAgents: StateFlow<List<GatewayAgentSummary>> = runtimeState(initial = emptyList()) { it.gatewayAgents }
val cronStatus: StateFlow<GatewayCronStatus> = runtimeState(initial = GatewayCronStatus(enabled = false, jobs = 0, nextWakeAtMs = null)) { it.cronStatus }
val cronJobs: StateFlow<List<GatewayCronJobSummary>> = runtimeState(initial = emptyList()) { it.cronJobs }
val cronRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.cronRefreshing }
val cronErrorText: StateFlow<String?> = runtimeState(initial = null) { it.cronErrorText }
val usageSummary: StateFlow<GatewayUsageSummary> = runtimeState(initial = GatewayUsageSummary(updatedAtMs = null, providers = emptyList())) { it.usageSummary }
val usageRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.usageRefreshing }
val usageErrorText: StateFlow<String?> = runtimeState(initial = null) { it.usageErrorText }
val skillsSummary: StateFlow<GatewaySkillsSummary> = runtimeState(initial = GatewaySkillsSummary(skills = emptyList())) { it.skillsSummary }
val skillsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.skillsRefreshing }
val skillsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.skillsErrorText }
val nodesDevicesSummary: StateFlow<GatewayNodesDevicesSummary> =
runtimeState(initial = GatewayNodesDevicesSummary(nodes = emptyList(), pendingDevices = emptyList(), pairedDevices = emptyList())) { it.nodesDevicesSummary }
val nodesDevicesRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.nodesDevicesRefreshing }
val nodesDevicesErrorText: StateFlow<String?> = runtimeState(initial = null) { it.nodesDevicesErrorText }
val channelsSummary: StateFlow<GatewayChannelsSummary> =
runtimeState(initial = GatewayChannelsSummary(channels = emptyList())) { it.channelsSummary }
val channelsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.channelsRefreshing }
val channelsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.channelsErrorText }
val dreamingSummary: StateFlow<GatewayDreamingSummary> =
runtimeState(initial = GatewayDreamingSummary()) { it.dreamingSummary }
val dreamingRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.dreamingRefreshing }
val dreamingErrorText: StateFlow<String?> = runtimeState(initial = null) { it.dreamingErrorText }
val healthLogsSummary: StateFlow<GatewayHealthLogsSummary> =
runtimeState(initial = GatewayHealthLogsSummary()) { it.healthLogsSummary }
val healthLogsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.healthLogsRefreshing }
val healthLogsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.healthLogsErrorText }
val pendingGatewayTrust: StateFlow<NodeRuntime.GatewayTrustPrompt?> = runtimeState(initial = null) { it.pendingGatewayTrust }
val seamColorArgb: StateFlow<Long> = runtimeState(initial = 0xFF0EA5E9) { it.seamColorArgb }
val mainSessionKey: StateFlow<String> = runtimeState(initial = "main") { it.mainSessionKey }
val cameraHud: StateFlow<CameraHudState?> = runtimeState(initial = null) { it.cameraHud }
val cameraFlashToken: StateFlow<Long> = runtimeState(initial = 0L) { it.cameraFlashToken }
val instanceId: StateFlow<String> = prefs.instanceId
val displayName: StateFlow<String> = prefs.displayName
val cameraEnabled: StateFlow<Boolean> = prefs.cameraEnabled
val locationMode: StateFlow<LocationMode> = prefs.locationMode
val locationPreciseEnabled: StateFlow<Boolean> = prefs.locationPreciseEnabled
val preventSleep: StateFlow<Boolean> = prefs.preventSleep
val manualEnabled: StateFlow<Boolean> = prefs.manualEnabled
val manualHost: StateFlow<String> = prefs.manualHost
val manualPort: StateFlow<Int> = prefs.manualPort
val manualTls: StateFlow<Boolean> = prefs.manualTls
val gatewayToken: StateFlow<String> = prefs.gatewayToken
val gatewayBootstrapToken: StateFlow<String> = prefs.gatewayBootstrapToken
val onboardingCompleted: StateFlow<Boolean> = prefs.onboardingCompleted
val canvasDebugStatusEnabled: StateFlow<Boolean> = prefs.canvasDebugStatusEnabled
val installedAppsSharingEnabled: StateFlow<Boolean> = prefs.installedAppsSharingEnabled
val speakerEnabled: StateFlow<Boolean> = prefs.speakerEnabled
val appearanceThemeMode: StateFlow<AppearanceThemeMode> = prefs.appearanceThemeMode
val voiceCaptureMode: StateFlow<VoiceCaptureMode> = runtimeState(initial = VoiceCaptureMode.Off) { it.voiceCaptureMode }
val micEnabled: StateFlow<Boolean> = runtimeState(initial = false) { it.micEnabled }
val micCooldown: StateFlow<Boolean> = runtimeState(initial = false) { it.micCooldown }
val micStatusText: StateFlow<String> = runtimeState(initial = "Mic off") { it.micStatusText }
val micLiveTranscript: StateFlow<String?> = runtimeState(initial = null) { it.micLiveTranscript }
val micIsListening: StateFlow<Boolean> = runtimeState(initial = false) { it.micIsListening }
val micQueuedMessages: StateFlow<List<String>> = runtimeState(initial = emptyList()) { it.micQueuedMessages }
val micConversation: StateFlow<List<VoiceConversationEntry>> = runtimeState(initial = emptyList()) { it.micConversation }
val micInputLevel: StateFlow<Float> = runtimeState(initial = 0f) { it.micInputLevel }
val micIsSending: StateFlow<Boolean> = runtimeState(initial = false) { it.micIsSending }
val talkModeEnabled: StateFlow<Boolean> = runtimeState(initial = false) { it.talkModeEnabled }
val talkModeListening: StateFlow<Boolean> = runtimeState(initial = false) { it.talkModeListening }
val talkModeSpeaking: StateFlow<Boolean> = runtimeState(initial = false) { it.talkModeSpeaking }
val talkModeStatusText: StateFlow<String> = runtimeState(initial = "Off") { it.talkModeStatusText }
val talkModeConversation: StateFlow<List<VoiceConversationEntry>> =
runtimeState(initial = emptyList()) { it.talkModeConversation }
val chatSessionKey: StateFlow<String> = runtimeState(initial = "main") { it.chatSessionKey }
val chatSessionId: StateFlow<String?> = runtimeState(initial = null) { it.chatSessionId }
val chatMessages: StateFlow<List<ChatMessage>> = runtimeState(initial = emptyList()) { it.chatMessages }
val chatHistoryLoading: StateFlow<Boolean> = runtimeState(initial = false) { it.chatHistoryLoading }
val chatError: StateFlow<String?> = runtimeState(initial = null) { it.chatError }
val chatHealthOk: StateFlow<Boolean> = runtimeState(initial = false) { it.chatHealthOk }
val chatThinkingLevel: StateFlow<String> = runtimeState(initial = "off") { it.chatThinkingLevel }
val chatStreamingAssistantText: StateFlow<String?> = runtimeState(initial = null) { it.chatStreamingAssistantText }
val chatPendingToolCalls: StateFlow<List<ChatPendingToolCall>> = runtimeState(initial = emptyList()) { it.chatPendingToolCalls }
val chatSessions: StateFlow<List<ChatSessionEntry>> = runtimeState(initial = emptyList()) { it.chatSessions }
val pendingRunCount: StateFlow<Int> = runtimeState(initial = 0) { it.pendingRunCount }
val chatCommands: StateFlow<List<ChatCommandEntry>> = runtimeState(initial = emptyList<ChatCommandEntry>()) { it.chatCommands }
val execApprovals: StateFlow<List<GatewayExecApprovalSummary>> = runtimeState(initial = emptyList()) { it.execApprovals }
val execApprovalsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.execApprovalsRefreshing }
val execApprovalsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.execApprovalsErrorText }
val canvas: CanvasController
get() = ensureRuntime().canvas
val camera: CameraCaptureManager
get() = ensureRuntime().camera
val sms: SmsManager
get() = ensureRuntime().sms
/**
* Attaches Activity-owned permission and lifecycle seams after runtime initialization.
*/
fun attachRuntimeUi(
owner: LifecycleOwner,
permissionRequester: PermissionRequester,
) {
val runtime = runtimeRef.value ?: return
runtime.camera.attachLifecycleOwner(owner)
runtime.camera.attachPermissionRequester(permissionRequester)
runtime.sms.attachPermissionRequester(permissionRequester)
}
/**
* Starts runtime on foreground entry only after onboarding has completed.
*/
fun setForeground(value: Boolean) {
foreground = value
if (value && prefs.onboardingCompleted.value) {
queueRuntimeStartup()
}
runtimeRef.value?.setForeground(value)
}
fun setDisplayName(value: String) {
prefs.setDisplayName(value)
}
fun setCameraEnabled(value: Boolean) {
prefs.setCameraEnabled(value)
}
fun setLocationMode(mode: LocationMode) {
prefs.setLocationMode(mode)
}
fun setLocationPreciseEnabled(value: Boolean) {
prefs.setLocationPreciseEnabled(value)
}
fun setPreventSleep(value: Boolean) {
prefs.setPreventSleep(value)
}
fun setManualEnabled(value: Boolean) {
prefs.setManualEnabled(value)
}
fun setManualHost(value: String) {
prefs.setManualHost(value)
}
fun setManualPort(value: Int) {
prefs.setManualPort(value)
}
fun setManualTls(value: Boolean) {
prefs.setManualTls(value)
}
fun setGatewayBootstrapToken(value: String) {
prefs.setGatewayBootstrapToken(value)
}
fun setGatewayPassword(value: String) {
prefs.setGatewayPassword(value)
}
/** Clears setup credentials without starting the runtime just to discard first-run pairing auth. */
private fun resetGatewaySetupAuth() {
runtimeRef.value?.resetGatewaySetupAuth() ?: resetGatewaySetupAuthWithoutRuntime()
}
private fun resetGatewaySetupAuthWithoutRuntime() {
prefs.clearGatewaySetupAuth()
val deviceId = DeviceIdentityStore(nodeApp).loadOrCreate().deviceId
val deviceAuthStore = DeviceAuthStore(prefs)
deviceAuthStore.clearToken(deviceId, "node")
deviceAuthStore.clearToken(deviceId, "operator")
}
internal fun saveGatewayConfigAndConnect(plan: GatewayConnectPlan) {
// Gateway pairing touches encrypted prefs, identity files, and sockets; keep
// the whole sequence off the Compose thread so retries cannot trigger ANRs.
viewModelScope.launch(Dispatchers.Default) {
val config = plan.config
val replacesSavedAuth = plan.savedAuthAction != GatewaySavedAuthAction.PRESERVE
val hasExplicitAuth =
config.token.isNotEmpty() || config.bootstrapToken.isNotEmpty() || config.password.isNotEmpty()
if (replacesSavedAuth) {
resetGatewaySetupAuth()
}
prefs.setManualEnabled(true)
prefs.setManualHost(config.host)
prefs.setManualPort(config.port)
prefs.setManualTls(config.tls)
// A blank same-endpoint save means "keep access". Secrets remain runtime-owned,
// including password-only setups that Compose deliberately cannot read back.
if (replacesSavedAuth || hasExplicitAuth) {
prefs.setGatewayBootstrapToken(config.bootstrapToken)
prefs.setGatewayToken(config.token)
prefs.setGatewayPassword(config.password)
}
val runtime = ensureRuntime()
val endpoint = GatewayEndpoint.manual(host = config.host, port = config.port)
if (replacesSavedAuth || hasExplicitAuth) {
runtime.connect(
endpoint,
NodeRuntime.GatewayConnectAuth(
token = config.token.ifEmpty { null },
bootstrapToken = config.bootstrapToken.ifEmpty { null },
password = config.password.ifEmpty { null },
),
)
} else {
runtime.connect(endpoint)
}
}
}
/** Marks onboarding complete and starts the runtime before UI observes connected-state flows. */
fun setOnboardingCompleted(value: Boolean) {
if (value) {
ensureRuntime()
}
prefs.setOnboardingCompleted(value)
}
/** Re-enters gateway setup after disconnecting and clearing one-time setup credentials. */
fun pairNewGateway() {
viewModelScope.launch(Dispatchers.Default) {
runtimeRef.value?.disconnect()
resetGatewaySetupAuth()
prefs.setOnboardingCompleted(false)
_startOnboardingAtGatewaySetup.value = true
}
}
/** Acknowledges the one-shot request that opens onboarding at the gateway setup step. */
fun clearGatewaySetupStartRequest() {
_startOnboardingAtGatewaySetup.value = false
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
prefs.setCanvasDebugStatusEnabled(value)
}
fun setInstalledAppsSharingEnabled(value: Boolean) {
ensureRuntime().setInstalledAppsSharingEnabled(value)
}
fun setNotificationForwardingEnabled(value: Boolean) {
ensureRuntime().setNotificationForwardingEnabled(value)
}
fun setNotificationForwardingMode(mode: NotificationPackageFilterMode) {
ensureRuntime().setNotificationForwardingMode(mode)
}
fun setNotificationForwardingPackagesCsv(csv: String) {
val packages =
csv
.split(',')
.map { it.trim() }
.filter { it.isNotEmpty() }
ensureRuntime().setNotificationForwardingPackages(packages)
}
fun setNotificationForwardingQuietHours(
enabled: Boolean,
start: String,
end: String,
): Boolean = ensureRuntime().setNotificationForwardingQuietHours(enabled = enabled, start = start, end = end)
fun setNotificationForwardingMaxEventsPerMinute(value: Int) {
ensureRuntime().setNotificationForwardingMaxEventsPerMinute(value)
}
fun setNotificationForwardingSessionKey(value: String?) {
ensureRuntime().setNotificationForwardingSessionKey(value)
}
fun setVoiceScreenActive(active: Boolean) {
ensureRuntime().setVoiceScreenActive(active)
}
/** Routes assistant intents into chat, either as a draft or queued auto-send prompt. */
fun handleAssistantLaunch(request: AssistantLaunchRequest) {
_requestedHomeDestination.value = HomeDestination.Chat
if (request.autoSend) {
_pendingAssistantAutoSend.value = request.prompt
_chatDraft.value = null
return
}
_pendingAssistantAutoSend.value = null
_chatDraft.value = request.prompt
}
fun clearRequestedHomeDestination() {
_requestedHomeDestination.value = null
}
fun requestHomeDestination(destination: HomeDestination) {
_requestedHomeDestination.value = destination
}
fun clearChatDraft() {
_chatDraft.value = null
}
fun clearPendingAssistantAutoSend() {
_pendingAssistantAutoSend.value = null
}
fun setMicEnabled(enabled: Boolean) {
ensureRuntime().setMicEnabled(enabled)
}
fun cancelMicCapture() {
ensureRuntime().cancelMicCapture()
}
fun setTalkModeEnabled(enabled: Boolean) {
ensureRuntime().setTalkModeEnabled(enabled)
}
fun setSpeakerEnabled(enabled: Boolean) {
ensureRuntime().setSpeakerEnabled(enabled)
}
fun setAppearanceThemeMode(mode: AppearanceThemeMode) {
prefs.setAppearanceThemeMode(mode)
}
fun refreshGatewayConnection() {
viewModelScope.launch(Dispatchers.Default) {
ensureRuntime().refreshGatewayConnection()
}
}
fun startGatewayDiscovery() {
queueRuntimeStartup()
}
fun connect(endpoint: GatewayEndpoint) {
ensureRuntime().connect(endpoint)
}
fun connectInBackground(endpoint: GatewayEndpoint) {
viewModelScope.launch(Dispatchers.Default) {
ensureRuntime().connect(endpoint)
}
}
fun connect(
endpoint: GatewayEndpoint,
token: String?,
bootstrapToken: String?,
password: String?,
) {
ensureRuntime().connect(
endpoint,
NodeRuntime.GatewayConnectAuth(
token = token,
bootstrapToken = bootstrapToken,
password = password,
),
)
}
fun connectManual() {
ensureRuntime().connectManual()
}
fun disconnect() {
runtimeRef.value?.disconnect()
}
fun acceptGatewayTrustPrompt() {
runtimeRef.value?.acceptGatewayTrustPrompt()
}
fun declineGatewayTrustPrompt() {
runtimeRef.value?.declineGatewayTrustPrompt()
}
fun handleCanvasA2UIActionFromWebView(payloadJson: String) {
ensureRuntime().handleCanvasA2UIActionFromWebView(payloadJson)
}
fun isTrustedCanvasActionUrl(rawUrl: String?): Boolean = ensureRuntime().isTrustedCanvasActionUrl(rawUrl)
fun requestCanvasRehydrate(source: String = "screen_tab") {
ensureRuntime().requestCanvasRehydrate(source = source, force = true)
}
fun refreshHomeCanvasOverviewIfConnected() {
ensureRuntime().refreshHomeCanvasOverviewIfConnected()
}
fun refreshModelCatalog() {
ensureRuntime().refreshModelCatalog()
}
fun refreshTalkSetupReadiness() {
ensureRuntime().refreshTalkSetupReadiness()
}
fun refreshAgents() {
ensureRuntime().refreshAgents()
}
fun refreshCronJobs() {
ensureRuntime().refreshCronJobs()
}
fun refreshUsage() {
ensureRuntime().refreshUsage()
}
fun refreshSkills() {
ensureRuntime().refreshSkills()
}
fun refreshNodesDevices() {
ensureRuntime().refreshNodesDevices()
}
fun refreshExecApprovals() {
ensureRuntime().refreshExecApprovals()
}
fun resolveExecApproval(
id: String,
decision: String,
) {
ensureRuntime().resolveExecApproval(id = id, decision = decision)
}
fun refreshChannels() {
ensureRuntime().refreshChannels()
}
fun refreshDreaming() {
ensureRuntime().refreshDreaming()
}
fun refreshHealthLogs() {
ensureRuntime().refreshHealthLogs()
}
fun loadChat(sessionKey: String) {
ensureRuntime().loadChat(sessionKey)
}
fun refreshChat() {
ensureRuntime().refreshChat()
}
fun refreshChatSessions(limit: Int? = null) {
ensureRuntime().refreshChatSessions(limit = limit)
}
fun setChatThinkingLevel(level: String) {
ensureRuntime().setChatThinkingLevel(level)
}
fun switchChatSession(sessionKey: String) {
ensureRuntime().switchChatSession(sessionKey)
}
fun abortChat() {
ensureRuntime().abortChat()
}
fun startNewChat() {
ensureRuntime().startNewChat()
}
fun refreshChatCommands() {
ensureRuntime().refreshChatCommands()
}
fun sendChat(
message: String,
thinking: String,
attachments: List<OutgoingAttachment>,
) {
ensureRuntime().sendChat(message = message, thinking = thinking, attachments = attachments)
}
suspend fun sendChatAwaitAcceptance(
message: String,
thinking: String,
attachments: List<OutgoingAttachment>,
): Boolean =
ensureRuntime().sendChatAwaitAcceptance(
message = message,
thinking = thinking,
attachments = attachments,
)
}

View File

@@ -0,0 +1,48 @@
package ai.openclaw.app
import android.app.Application
import android.os.StrictMode
/**
* Android Application singleton that owns process-wide secure prefs and lazy NodeRuntime startup.
*/
class NodeApp : Application() {
val prefs: SecurePrefs by lazy { SecurePrefs(this) }
@Volatile private var runtimeInstance: NodeRuntime? = null
/**
* Returns the single NodeRuntime for this process, creating it on first use.
*/
fun ensureRuntime(): NodeRuntime {
runtimeInstance?.let { return it }
return synchronized(this) {
runtimeInstance ?: NodeRuntime(this, prefs).also { runtimeInstance = it }
}
}
/**
* Reads the runtime without forcing startup, used by lifecycle probes and services.
*/
fun peekRuntime(): NodeRuntime? = runtimeInstance
override fun onCreate() {
super.onCreate()
if (BuildConfig.DEBUG) {
StrictMode.setThreadPolicy(
StrictMode.ThreadPolicy
.Builder()
.detectAll()
.penaltyLog()
.build(),
)
StrictMode.setVmPolicy(
StrictMode.VmPolicy
.Builder()
.detectAll()
.penaltyLog()
.build(),
)
}
}
}

View File

@@ -0,0 +1,290 @@
package ai.openclaw.app
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import androidx.core.app.NotificationCompat
import androidx.core.app.ServiceCompat
import androidx.core.content.ContextCompat
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.launch
/** Foreground service that keeps the Android node connection and voice capture visible to the OS. */
class NodeForegroundService : Service() {
private val scope: CoroutineScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
private var notificationJob: Job? = null
private var voiceCaptureMode = VoiceCaptureMode.Off
override fun onCreate() {
super.onCreate()
ensureChannel()
val initial = buildNotification(title = "OpenClaw Node", text = "Starting…")
startForegroundWithTypes(notification = initial)
val runtime = (application as NodeApp).peekRuntime()
if (runtime == null) {
stopSelf()
return
}
// Keep the connection tuple atomic, then split connection and capture work so notification text
// can update without restarting runtime-owned connection work.
notificationJob =
scope.launch {
combine(
combine(
runtime.gatewayConnectionDisplay,
runtime.serverName,
runtime.voiceCaptureMode,
) { connection, server, mode ->
VoiceNotificationBase(
status = connection.statusText,
server = server,
connected = connection.isConnected,
mode = mode,
)
},
combine(
runtime.micEnabled,
runtime.micIsListening,
runtime.talkModeListening,
runtime.talkModeSpeaking,
) { micEnabled, micListening, talkListening, talkSpeaking ->
VoiceNotificationCapture(
micEnabled = micEnabled,
micListening = micListening,
talkListening = talkListening,
talkSpeaking = talkSpeaking,
)
},
) { base, capture ->
VoiceNotificationState(base = base, capture = capture)
}.collect { state ->
voiceCaptureMode = state.mode
val title =
when {
state.connected && state.mode == VoiceCaptureMode.TalkMode -> "OpenClaw Node · Talk"
state.connected -> "OpenClaw Node · Connected"
else -> "OpenClaw Node"
}
val text =
(state.server?.let { "${state.status} · $it" } ?: state.status) +
voiceNotificationSuffix(
mode = state.mode,
manualMicEnabled = state.capture.micEnabled,
manualMicListening = state.capture.micListening,
talkListening = state.capture.talkListening,
talkSpeaking = state.capture.talkSpeaking,
)
startForegroundWithTypes(
notification = buildNotification(title = title, text = text),
)
}
}
}
override fun onStartCommand(
intent: Intent?,
flags: Int,
startId: Int,
): Int {
when (intent?.action) {
ACTION_STOP -> {
(application as NodeApp).peekRuntime()?.disconnect()
stopSelf()
return START_NOT_STICKY
}
ACTION_SET_VOICE_CAPTURE_MODE -> {
voiceCaptureMode = intent.getStringExtra(EXTRA_VOICE_CAPTURE_MODE).toVoiceCaptureMode()
startForegroundWithTypes(
notification =
buildNotification(
title = "OpenClaw Node",
text = if (voiceCaptureMode == VoiceCaptureMode.TalkMode) "Talk mode active" else "Connected",
),
)
}
}
// Keep running; connection is managed by NodeRuntime (auto-reconnect + manual).
return START_STICKY
}
override fun onDestroy() {
notificationJob?.cancel()
scope.cancel()
super.onDestroy()
}
override fun onBind(intent: Intent?) = null
private fun ensureChannel() {
val mgr = getSystemService(NotificationManager::class.java)
val channel =
NotificationChannel(
CHANNEL_ID,
"Connection",
NotificationManager.IMPORTANCE_LOW,
).apply {
description = "OpenClaw node connection status"
setShowBadge(false)
}
mgr.createNotificationChannel(channel)
}
private fun buildNotification(
title: String,
text: String,
): Notification {
val launchIntent =
Intent(this, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP
}
val launchPending =
PendingIntent.getActivity(
this,
1,
launchIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
val stopIntent = Intent(this, NodeForegroundService::class.java).setAction(ACTION_STOP)
val stopPending =
PendingIntent.getService(
this,
2,
stopIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
return NotificationCompat
.Builder(this, CHANNEL_ID)
.setSmallIcon(R.mipmap.ic_launcher)
.setContentTitle(title)
.setContentText(text)
.setContentIntent(launchPending)
.setOngoing(true)
.setOnlyAlertOnce(true)
.setForegroundServiceBehavior(NotificationCompat.FOREGROUND_SERVICE_IMMEDIATE)
.addAction(0, "Disconnect", stopPending)
.build()
}
private fun startForegroundWithTypes(notification: Notification) {
val serviceTypes = foregroundServiceTypesForVoiceMode(voiceCaptureMode)
ServiceCompat.startForeground(this, NOTIFICATION_ID, notification, serviceTypes)
}
companion object {
private const val CHANNEL_ID = "connection"
private const val NOTIFICATION_ID = 1
private const val ACTION_STOP = "ai.openclaw.app.action.STOP"
private const val ACTION_SET_VOICE_CAPTURE_MODE = "ai.openclaw.app.action.SET_VOICE_CAPTURE_MODE"
private const val EXTRA_VOICE_CAPTURE_MODE = "ai.openclaw.app.extra.VOICE_CAPTURE_MODE"
fun start(context: Context) {
val intent = Intent(context, NodeForegroundService::class.java)
context.startForegroundService(intent)
}
fun stop(context: Context) {
val intent = Intent(context, NodeForegroundService::class.java).setAction(ACTION_STOP)
context.startService(intent)
}
fun setVoiceCaptureMode(
context: Context,
mode: VoiceCaptureMode,
) {
val intent =
Intent(context, NodeForegroundService::class.java)
.setAction(ACTION_SET_VOICE_CAPTURE_MODE)
.putExtra(EXTRA_VOICE_CAPTURE_MODE, mode.name)
if (mode == VoiceCaptureMode.TalkMode) {
// Microphone foreground service type must be declared before Talk capture starts.
ContextCompat.startForegroundService(context, intent)
} else {
context.startService(intent)
}
}
}
}
internal fun foregroundServiceTypesForVoiceMode(mode: VoiceCaptureMode): Int {
val base = ServiceInfo.FOREGROUND_SERVICE_TYPE_CONNECTED_DEVICE
return when (mode) {
VoiceCaptureMode.Off -> base
VoiceCaptureMode.ManualMic,
VoiceCaptureMode.TalkMode,
-> base or ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE
}
}
internal fun voiceNotificationSuffix(
mode: VoiceCaptureMode,
manualMicEnabled: Boolean,
manualMicListening: Boolean,
talkListening: Boolean,
talkSpeaking: Boolean,
): String =
when (mode) {
VoiceCaptureMode.TalkMode ->
when {
talkSpeaking -> " · Talk: Speaking"
talkListening -> " · Talk: Listening"
else -> " · Talk: On"
}
VoiceCaptureMode.ManualMic ->
if (manualMicEnabled) {
if (manualMicListening) " · Mic: Listening" else " · Mic: Pending"
} else {
""
}
VoiceCaptureMode.Off -> ""
}
private fun String?.toVoiceCaptureMode(): VoiceCaptureMode =
VoiceCaptureMode.entries.firstOrNull {
it.name == this
} ?: VoiceCaptureMode.Off
/** Connection fields that drive foreground notification title/body text. */
private data class VoiceNotificationBase(
val status: String,
val server: String?,
val connected: Boolean,
val mode: VoiceCaptureMode,
)
/** Voice capture fields that affect foreground-service type and suffix. */
private data class VoiceNotificationCapture(
val micEnabled: Boolean,
val micListening: Boolean,
val talkListening: Boolean,
val talkSpeaking: Boolean,
)
/** Aggregated notification state from runtime flows. */
private data class VoiceNotificationState(
val base: VoiceNotificationBase,
val capture: VoiceNotificationCapture,
) {
val status: String
get() = base.status
val server: String?
get() = base.server
val connected: Boolean
get() = base.connected
val mode: VoiceCaptureMode
get() = base.mode
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,122 @@
package ai.openclaw.app
import java.time.Instant
import java.time.ZoneId
/** Package-filter mode used before notification events are forwarded to the gateway. */
enum class NotificationPackageFilterMode(
val rawValue: String,
) {
Allowlist("allowlist"),
Blocklist("blocklist"),
;
companion object {
/** Parses persisted filter mode text, defaulting to blocklist for safer forwarding. */
fun fromRawValue(raw: String?): NotificationPackageFilterMode = entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Blocklist
}
}
/** Runtime policy used before forwarding notification events to a node session. */
internal data class NotificationForwardingPolicy(
val enabled: Boolean,
val mode: NotificationPackageFilterMode,
val packages: Set<String>,
val quietHoursEnabled: Boolean,
val quietStart: String,
val quietEnd: String,
val maxEventsPerMinute: Int,
val sessionKey: String?,
val selfPackageName: String = "",
)
/** Applies the operator-configured package allow/block list after trimming input. */
internal fun NotificationForwardingPolicy.allowsPackage(packageName: String): Boolean {
val normalized = packageName.trim()
if (normalized.isEmpty()) {
return false
}
val self = selfPackageName.trim()
if (self.isNotEmpty() && normalized == self) {
return false
}
return when (mode) {
NotificationPackageFilterMode.Allowlist -> packages.contains(normalized)
NotificationPackageFilterMode.Blocklist -> !packages.contains(normalized)
}
}
/** Returns true for both same-day and overnight quiet-hour windows. */
internal fun NotificationForwardingPolicy.isWithinQuietHours(
nowEpochMs: Long,
zoneId: ZoneId = ZoneId.systemDefault(),
): Boolean {
if (!quietHoursEnabled) {
return false
}
val startMinutes = parseLocalHourMinute(quietStart) ?: return false
val endMinutes = parseLocalHourMinute(quietEnd) ?: return false
if (startMinutes == endMinutes) {
return true
}
val now =
Instant
.ofEpochMilli(nowEpochMs)
.atZone(zoneId)
.toLocalTime()
val nowMinutes = now.hour * 60 + now.minute
return if (startMinutes < endMinutes) {
nowMinutes in startMinutes until endMinutes
} else {
nowMinutes >= startMinutes || nowMinutes < endMinutes
}
}
private val localHourMinuteRegex = Regex("""^([01]\d|2[0-3]):([0-5]\d)$""")
/** Normalizes persisted or user-entered local times to strict HH:mm form. */
internal fun normalizeLocalHourMinute(raw: String): String? {
val trimmed = raw.trim()
val match = localHourMinuteRegex.matchEntire(trimmed) ?: return null
return "${match.groupValues[1]}:${match.groupValues[2]}"
}
/** Converts strict local HH:mm text to minutes since midnight for window checks. */
internal fun parseLocalHourMinute(raw: String): Int? {
val normalized = normalizeLocalHourMinute(raw) ?: return null
val parts = normalized.split(':')
val hour = parts[0].toInt()
val minute = parts[1].toInt()
return hour * 60 + minute
}
/** Fixed-window limiter that bounds notification bursts per wall-clock minute. */
internal class NotificationBurstLimiter {
private val lock = Any()
private var windowStartMs: Long = -1L
private var eventsInWindow: Int = 0
/** Returns true when the current minute bucket still has forwarding capacity. */
fun allow(
nowEpochMs: Long,
maxEventsPerMinute: Int,
): Boolean {
if (maxEventsPerMinute <= 0) {
return false
}
// Align all callers to the same minute bucket so concurrent notifications
// share the quota even when they arrive with slightly different timestamps.
val currentWindow = nowEpochMs - (nowEpochMs % 60_000L)
synchronized(lock) {
if (currentWindow != windowStartMs) {
windowStartMs = currentWindow
eventsInWindow = 0
}
if (eventsInWindow >= maxEventsPerMinute) {
return false
}
eventsInWindow += 1
return true
}
}
}

View File

@@ -0,0 +1,285 @@
package ai.openclaw.app
import android.Manifest
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Handler
import android.os.Looper
import android.provider.Settings
import androidx.activity.ComponentActivity
import androidx.activity.result.ActivityResultLauncher
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AlertDialog
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.TimeoutCancellationException
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.util.concurrent.atomic.AtomicBoolean
import kotlin.coroutines.resume
/**
* Serializes Android runtime-permission prompts behind coroutine-friendly request calls.
*/
class PermissionRequester internal constructor(
private val activity: ComponentActivity,
launcherFactory: ((Map<String, Boolean>) -> Unit) -> ActivityResultLauncher<Array<String>>,
) {
private data class PendingPermissionRequest(
val deferred: CompletableDeferred<Map<String, Boolean>>,
var timedOut: Boolean = false,
)
private class PermissionRequestSlot(
val launcher: ActivityResultLauncher<Array<String>>,
var request: PendingPermissionRequest? = null,
)
constructor(activity: ComponentActivity) : this(
activity = activity,
launcherFactory = { callback ->
activity.registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions(), callback)
},
)
private val mutex = Mutex()
private val requestSlotsLock = Any()
private val mainHandler = Handler(Looper.getMainLooper())
// ActivityResult launchers cannot be registered after start; pre-register a small pool for nested UI flows.
private val launchers = List(4) { createPermissionRequestSlot(launcherFactory) }
/**
* Request missing Android runtime permissions and return the final grant state for every requested permission.
*/
suspend fun requestIfMissing(
permissions: List<String>,
timeoutMs: Long = 20_000,
): Map<String, Boolean> {
return mutex.withLock {
while (true) {
val missing =
permissions.filter { perm ->
ContextCompat.checkSelfPermission(activity, perm) != PackageManager.PERMISSION_GRANTED
}
if (missing.isEmpty()) {
return permissions.associateWith { true }
}
val needsRationale =
missing.any { ActivityCompat.shouldShowRequestPermissionRationale(activity, it) }
if (needsRationale) {
val proceed = showRationaleDialog(missing)
if (!proceed) {
return permissions.associateWith { perm ->
ContextCompat.checkSelfPermission(activity, perm) == PackageManager.PERMISSION_GRANTED
}
}
}
val deferred = CompletableDeferred<Map<String, Boolean>>()
val request = PendingPermissionRequest(deferred)
val slot = reservePermissionRequestSlot(request)
try {
withContext(Dispatchers.Main) {
slot.launcher.launch(missing.toTypedArray())
}
} catch (err: Throwable) {
clearPermissionRequestSlot(slot, request)
throw err
}
val result =
try {
withTimeout(timeoutMs) { deferred.await() }
} catch (err: TimeoutCancellationException) {
// Late ActivityResult callbacks are ignored by completePermissionRequest.
request.timedOut = true
throw err
}
val merged =
permissions.associateWith { perm ->
val nowGranted =
ContextCompat.checkSelfPermission(activity, perm) == PackageManager.PERMISSION_GRANTED
result[perm] == true || nowGranted
}
val denied =
merged.filterValues { !it }.keys.filter {
!ActivityCompat.shouldShowRequestPermissionRationale(activity, it)
}
if (denied.isNotEmpty()) {
showSettingsDialog(denied)
}
return merged
}
error("unreachable")
}
}
private fun createPermissionRequestSlot(
launcherFactory: ((Map<String, Boolean>) -> Unit) -> ActivityResultLauncher<Array<String>>,
): PermissionRequestSlot {
var slot: PermissionRequestSlot? = null
val launcher = launcherFactory { result -> completePermissionRequest(checkNotNull(slot), result) }
val created = PermissionRequestSlot(launcher)
slot = created
return created
}
private fun reservePermissionRequestSlot(request: PendingPermissionRequest): PermissionRequestSlot =
synchronized(requestSlotsLock) {
// The outer mutex serializes normal callers; this guard catches accidental concurrent launchers in tests.
val slot = launchers.firstOrNull { it.request == null } ?: error("permission request launcher busy")
slot.request = request
slot
}
private fun completePermissionRequest(
slot: PermissionRequestSlot,
result: Map<String, Boolean>,
) {
val request =
synchronized(requestSlotsLock) {
slot.request.also {
slot.request = null
}
} ?: return
// Timed-out requests have already resumed callers with failure; ignore any late platform callback.
if (request.timedOut) return
request.deferred.complete(result)
}
private fun clearPermissionRequestSlot(
slot: PermissionRequestSlot,
request: PendingPermissionRequest,
) {
synchronized(requestSlotsLock) {
if (slot.request === request) {
slot.request = null
}
}
}
private suspend fun showRationaleDialog(permissions: List<String>): Boolean =
withContext(Dispatchers.Main) {
if (activity.isFinishing || activity.isDestroyed) {
return@withContext false
}
suspendCancellableCoroutine { cont ->
val lifecycle = activity.lifecycle
var dialog: AlertDialog? = null
var observer: LifecycleEventObserver? = null
val finished = AtomicBoolean(false)
val removeObserver = {
observer?.let(lifecycle::removeObserver)
observer = null
}
fun finish(result: Boolean?) {
if (!finished.compareAndSet(false, true)) return
removeObserver()
dialog?.dismiss()
if (result != null) {
cont.resume(result)
}
}
val actualObserver =
LifecycleEventObserver { _, event ->
if (event != Lifecycle.Event.ON_DESTROY) return@LifecycleEventObserver
// Do not resume a destroyed Activity with a positive result.
finish(false)
}
observer = actualObserver
lifecycle.addObserver(actualObserver)
cont.invokeOnCancellation {
mainHandler.post {
finish(null)
}
}
dialog =
AlertDialog
.Builder(activity)
.setTitle("Permission required")
.setMessage(buildRationaleMessage(permissions))
.setPositiveButton("Continue") { _, _ -> finish(true) }
.setNegativeButton("Not now") { _, _ -> finish(false) }
.setOnCancelListener { finish(false) }
.show()
}
}
private suspend fun showSettingsDialog(permissions: List<String>) =
withContext(Dispatchers.Main) {
if (activity.isFinishing || activity.isDestroyed) return@withContext
val lifecycle = activity.lifecycle
var dialog: AlertDialog? = null
var observer: LifecycleEventObserver? = null
val removeObserver = {
observer?.let(lifecycle::removeObserver)
observer = null
}
val actualObserver =
LifecycleEventObserver { _, event ->
if (event != Lifecycle.Event.ON_DESTROY) return@LifecycleEventObserver
removeObserver()
dialog?.dismiss()
}
observer = actualObserver
lifecycle.addObserver(actualObserver)
dialog =
AlertDialog
.Builder(activity)
.setTitle("Enable permission in Settings")
.setMessage(buildSettingsMessage(permissions))
.setPositiveButton("Open Settings") { _, _ ->
if (activity.isFinishing || activity.isDestroyed) return@setPositiveButton
val intent =
Intent(
Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", activity.packageName, null),
)
activity.startActivity(intent)
}.setNegativeButton("Cancel", null)
.setOnDismissListener { removeObserver() }
.show()
}
private fun buildRationaleMessage(permissions: List<String>): String {
val labels = permissions.map { permissionLabel(it) }
return "OpenClaw needs ${labels.joinToString(", ")} permissions to continue."
}
private fun buildSettingsMessage(permissions: List<String>): String {
val labels = permissions.map { permissionLabel(it) }
return "Please enable ${labels.joinToString(", ")} in Android Settings to continue."
}
private fun permissionLabel(permission: String): String =
when (permission) {
Manifest.permission.CAMERA -> "Camera"
Manifest.permission.RECORD_AUDIO -> "Microphone"
Manifest.permission.SEND_SMS -> "Send SMS"
Manifest.permission.READ_SMS -> "Read SMS"
Manifest.permission.READ_CONTACTS -> "Read Contacts"
Manifest.permission.WRITE_CONTACTS -> "Write Contacts"
Manifest.permission.READ_CALENDAR -> "Read Calendar"
Manifest.permission.WRITE_CALENDAR -> "Write Calendar"
Manifest.permission.READ_CALL_LOG -> "Read Call Log"
Manifest.permission.ACTIVITY_RECOGNITION -> "Motion Activity"
Manifest.permission.READ_MEDIA_IMAGES -> "Photos"
Manifest.permission.READ_MEDIA_VISUAL_USER_SELECTED -> "Photos"
Manifest.permission.READ_EXTERNAL_STORAGE -> "Photos"
else -> permission
}
}

View File

@@ -0,0 +1,23 @@
package ai.openclaw.app
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import androidx.core.content.ContextCompat
internal fun photoReadPermissionsForRequest(): List<String> =
when {
Build.VERSION.SDK_INT >= 34 ->
listOf(
Manifest.permission.READ_MEDIA_IMAGES,
Manifest.permission.READ_MEDIA_VISUAL_USER_SELECTED,
)
Build.VERSION.SDK_INT >= 33 -> listOf(Manifest.permission.READ_MEDIA_IMAGES)
else -> listOf(Manifest.permission.READ_EXTERNAL_STORAGE)
}
internal fun hasPhotoReadPermission(context: Context): Boolean =
photoReadPermissionsForRequest().any { permission ->
ContextCompat.checkSelfPermission(context, permission) == PackageManager.PERMISSION_GRANTED
}

View File

@@ -0,0 +1,606 @@
@file:Suppress("DEPRECATION")
package ai.openclaw.app
import android.content.Context
import android.content.SharedPreferences
import androidx.core.content.edit
import androidx.security.crypto.EncryptedSharedPreferences
import androidx.security.crypto.MasterKey
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonPrimitive
import java.util.UUID
/**
* Reactive settings facade for Android node preferences and encrypted gateway credentials.
*/
class SecurePrefs(
context: Context,
private val securePrefsOverride: SharedPreferences? = null,
) {
companion object {
val defaultWakeWords: List<String> = listOf("openclaw", "claude")
private const val displayNameKey = "node.displayName"
private const val locationModeKey = "location.enabledMode"
private const val voiceWakeModeKey = "voiceWake.mode"
private const val plainPrefsName = "openclaw.node"
private const val securePrefsName = "openclaw.node.secure"
private const val notificationsForwardingEnabledKey = "notifications.forwarding.enabled"
private const val defaultNotificationForwardingEnabled = false
private const val notificationsForwardingModeKey = "notifications.forwarding.mode"
private const val notificationsForwardingPackagesKey = "notifications.forwarding.packages"
private const val notificationsForwardingQuietHoursEnabledKey =
"notifications.forwarding.quietHoursEnabled"
private const val notificationsForwardingQuietStartKey = "notifications.forwarding.quietStart"
private const val notificationsForwardingQuietEndKey = "notifications.forwarding.quietEnd"
private const val notificationsForwardingMaxEventsPerMinuteKey =
"notifications.forwarding.maxEventsPerMinute"
private const val notificationsForwardingSessionKeyKey = "notifications.forwarding.sessionKey"
private const val installedAppsSharingEnabledKey = "device.apps.sharing.enabled"
private const val cameraEnabledKey = "camera.enabled"
private const val voiceMicEnabledKey = "voice.micEnabled"
private const val appearanceThemeModeKey = "appearance.themeMode"
}
private val appContext = context.applicationContext
private val json = Json { ignoreUnknownKeys = true }
// Non-secret UI/runtime preferences stay readable for migration and backup behavior.
private val plainPrefs: SharedPreferences =
appContext.getSharedPreferences(plainPrefsName, Context.MODE_PRIVATE)
private val hadPlainPrefsBeforeInit = plainPrefs.all.isNotEmpty()
// Gateway credentials and arbitrary secret strings are isolated behind EncryptedSharedPreferences.
private val masterKey by lazy {
MasterKey
.Builder(appContext)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
.build()
}
private val securePrefs: SharedPreferences by lazy { securePrefsOverride ?: createSecurePrefs(appContext, securePrefsName) }
private val _instanceId = MutableStateFlow(loadOrCreateInstanceId())
val instanceId: StateFlow<String> = _instanceId
private val _displayName =
MutableStateFlow(loadOrMigrateDisplayName(context = context))
val displayName: StateFlow<String> = _displayName
private val _cameraEnabled = MutableStateFlow(loadCameraEnabled())
val cameraEnabled: StateFlow<Boolean> = _cameraEnabled
private val _locationMode = MutableStateFlow(loadLocationMode())
val locationMode: StateFlow<LocationMode> = _locationMode
private val _locationPreciseEnabled =
MutableStateFlow(plainPrefs.getBoolean("location.preciseEnabled", true))
val locationPreciseEnabled: StateFlow<Boolean> = _locationPreciseEnabled
private val _preventSleep = MutableStateFlow(plainPrefs.getBoolean("screen.preventSleep", true))
val preventSleep: StateFlow<Boolean> = _preventSleep
private val _manualEnabled =
MutableStateFlow(plainPrefs.getBoolean("gateway.manual.enabled", false))
val manualEnabled: StateFlow<Boolean> = _manualEnabled
private val _manualHost =
MutableStateFlow(plainPrefs.getString("gateway.manual.host", "") ?: "")
val manualHost: StateFlow<String> = _manualHost
private val _manualPort =
MutableStateFlow(plainPrefs.getInt("gateway.manual.port", 18789))
val manualPort: StateFlow<Int> = _manualPort
private val _manualTls =
MutableStateFlow(plainPrefs.getBoolean("gateway.manual.tls", true))
val manualTls: StateFlow<Boolean> = _manualTls
private val _gatewayToken = MutableStateFlow("")
val gatewayToken: StateFlow<String> = _gatewayToken
private val _gatewayBootstrapToken = MutableStateFlow("")
val gatewayBootstrapToken: StateFlow<String> = _gatewayBootstrapToken
private val _onboardingCompleted =
MutableStateFlow(plainPrefs.getBoolean("onboarding.completed", false))
val onboardingCompleted: StateFlow<Boolean> = _onboardingCompleted
private val _lastDiscoveredStableId =
MutableStateFlow(
plainPrefs.getString("gateway.lastDiscoveredStableID", "") ?: "",
)
val lastDiscoveredStableId: StateFlow<String> = _lastDiscoveredStableId
private val _canvasDebugStatusEnabled =
MutableStateFlow(plainPrefs.getBoolean("canvas.debugStatusEnabled", false))
val canvasDebugStatusEnabled: StateFlow<Boolean> = _canvasDebugStatusEnabled
private val _installedAppsSharingEnabled =
MutableStateFlow(plainPrefs.getBoolean(installedAppsSharingEnabledKey, false))
val installedAppsSharingEnabled: StateFlow<Boolean> = _installedAppsSharingEnabled
private val _notificationForwardingEnabled =
MutableStateFlow(plainPrefs.getBoolean(notificationsForwardingEnabledKey, defaultNotificationForwardingEnabled))
val notificationForwardingEnabled: StateFlow<Boolean> = _notificationForwardingEnabled
private val _notificationForwardingMode =
MutableStateFlow(
NotificationPackageFilterMode.fromRawValue(
plainPrefs.getString(notificationsForwardingModeKey, null),
),
)
val notificationForwardingMode: StateFlow<NotificationPackageFilterMode> = _notificationForwardingMode
private val _notificationForwardingPackages = MutableStateFlow(loadNotificationForwardingPackages())
val notificationForwardingPackages: StateFlow<Set<String>> = _notificationForwardingPackages
private val storedQuietStart =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty())
?: "22:00"
private val storedQuietEnd =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty())
?: "07:00"
private val storedQuietHoursEnabled =
plainPrefs.getBoolean(notificationsForwardingQuietHoursEnabledKey, false) &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty()) != null &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty()) != null
private val _notificationForwardingQuietHoursEnabled =
MutableStateFlow(storedQuietHoursEnabled)
val notificationForwardingQuietHoursEnabled: StateFlow<Boolean> = _notificationForwardingQuietHoursEnabled
private val _notificationForwardingQuietStart = MutableStateFlow(storedQuietStart)
val notificationForwardingQuietStart: StateFlow<String> = _notificationForwardingQuietStart
private val _notificationForwardingQuietEnd = MutableStateFlow(storedQuietEnd)
val notificationForwardingQuietEnd: StateFlow<String> = _notificationForwardingQuietEnd
private val _notificationForwardingMaxEventsPerMinute =
MutableStateFlow(plainPrefs.getInt(notificationsForwardingMaxEventsPerMinuteKey, 20).coerceAtLeast(1))
val notificationForwardingMaxEventsPerMinute: StateFlow<Int> = _notificationForwardingMaxEventsPerMinute
private val _notificationForwardingSessionKey =
MutableStateFlow(
plainPrefs
.getString(notificationsForwardingSessionKeyKey, "")
?.trim()
?.takeIf { it.isNotEmpty() },
)
val notificationForwardingSessionKey: StateFlow<String?> = _notificationForwardingSessionKey
private val _wakeWords = MutableStateFlow(loadWakeWords())
val wakeWords: StateFlow<List<String>> = _wakeWords
private val _voiceWakeMode = MutableStateFlow(loadVoiceWakeMode())
val voiceWakeMode: StateFlow<VoiceWakeMode> = _voiceWakeMode
private val _voiceMicEnabled = MutableStateFlow(plainPrefs.getBoolean(voiceMicEnabledKey, false))
val voiceMicEnabled: StateFlow<Boolean> = _voiceMicEnabled
private val _speakerEnabled = MutableStateFlow(plainPrefs.getBoolean("voice.speakerEnabled", true))
val speakerEnabled: StateFlow<Boolean> = _speakerEnabled
private val _appearanceThemeMode =
MutableStateFlow(AppearanceThemeMode.fromRawValue(plainPrefs.getString(appearanceThemeModeKey, null)))
val appearanceThemeMode: StateFlow<AppearanceThemeMode> = _appearanceThemeMode
fun setLastDiscoveredStableId(value: String) {
val trimmed = value.trim()
plainPrefs.edit { putString("gateway.lastDiscoveredStableID", trimmed) }
_lastDiscoveredStableId.value = trimmed
}
fun setDisplayName(value: String) {
val trimmed = value.trim()
plainPrefs.edit { putString(displayNameKey, trimmed) }
_displayName.value = trimmed
}
fun setCameraEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(cameraEnabledKey, value) }
_cameraEnabled.value = value
}
fun setLocationMode(mode: LocationMode) {
plainPrefs.edit { putString(locationModeKey, mode.rawValue) }
_locationMode.value = mode
}
fun setLocationPreciseEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("location.preciseEnabled", value) }
_locationPreciseEnabled.value = value
}
fun setPreventSleep(value: Boolean) {
plainPrefs.edit { putBoolean("screen.preventSleep", value) }
_preventSleep.value = value
}
fun setManualEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("gateway.manual.enabled", value) }
_manualEnabled.value = value
}
fun setManualHost(value: String) {
val trimmed = value.trim()
plainPrefs.edit { putString("gateway.manual.host", trimmed) }
_manualHost.value = trimmed
}
fun setManualPort(value: Int) {
plainPrefs.edit { putInt("gateway.manual.port", value) }
_manualPort.value = value
}
fun setManualTls(value: Boolean) {
plainPrefs.edit { putBoolean("gateway.manual.tls", value) }
_manualTls.value = value
}
fun setGatewayToken(value: String) {
val trimmed = value.trim()
securePrefs.edit { putString("gateway.manual.token", trimmed) }
_gatewayToken.value = trimmed
}
fun setGatewayPassword(value: String) {
saveGatewayPassword(value)
}
fun setGatewayBootstrapToken(value: String) {
saveGatewayBootstrapToken(value)
}
fun setOnboardingCompleted(value: Boolean) {
plainPrefs.edit { putBoolean("onboarding.completed", value) }
_onboardingCompleted.value = value
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("canvas.debugStatusEnabled", value) }
_canvasDebugStatusEnabled.value = value
}
fun setInstalledAppsSharingEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(installedAppsSharingEnabledKey, value) }
_installedAppsSharingEnabled.value = value
}
internal fun getNotificationForwardingPolicy(appPackageName: String): NotificationForwardingPolicy {
val modeRaw = plainPrefs.getString(notificationsForwardingModeKey, null)
val mode = NotificationPackageFilterMode.fromRawValue(modeRaw)
val configuredPackages = loadNotificationForwardingPackages()
val normalizedAppPackage = appPackageName.trim()
// Always block OpenClaw's own notifications in blocklist mode to prevent forwarding loops.
val defaultBlockedPackages =
if (normalizedAppPackage.isNotEmpty()) setOf(normalizedAppPackage) else emptySet()
val packages =
when (mode) {
NotificationPackageFilterMode.Allowlist -> configuredPackages
NotificationPackageFilterMode.Blocklist -> configuredPackages + defaultBlockedPackages
}
val maxEvents = plainPrefs.getInt(notificationsForwardingMaxEventsPerMinuteKey, 20)
val quietStart =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty())
?: "22:00"
val quietEnd =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty())
?: "07:00"
val sessionKey =
plainPrefs
.getString(notificationsForwardingSessionKeyKey, "")
?.trim()
?.takeIf { it.isNotEmpty() }
val quietHoursEnabled =
plainPrefs.getBoolean(notificationsForwardingQuietHoursEnabledKey, false) &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty()) != null &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty()) != null
return NotificationForwardingPolicy(
enabled = plainPrefs.getBoolean(notificationsForwardingEnabledKey, defaultNotificationForwardingEnabled),
mode = mode,
packages = packages,
quietHoursEnabled = quietHoursEnabled,
quietStart = quietStart,
quietEnd = quietEnd,
maxEventsPerMinute = maxEvents.coerceAtLeast(1),
sessionKey = sessionKey,
selfPackageName = normalizedAppPackage,
)
}
internal fun setNotificationForwardingEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(notificationsForwardingEnabledKey, value) }
_notificationForwardingEnabled.value = value
}
internal fun setNotificationForwardingMode(mode: NotificationPackageFilterMode) {
plainPrefs.edit { putString(notificationsForwardingModeKey, mode.rawValue) }
_notificationForwardingMode.value = mode
}
internal fun setNotificationForwardingPackages(packages: List<String>) {
val sanitized =
packages
.asSequence()
.map { it.trim() }
.filter { it.isNotEmpty() }
.toSet()
.toList()
.sorted()
// Persist deterministic JSON so settings diffs and state restoration are stable.
val encoded = JsonArray(sanitized.map { JsonPrimitive(it) }).toString()
plainPrefs.edit { putString(notificationsForwardingPackagesKey, encoded) }
_notificationForwardingPackages.value = sanitized.toSet()
}
internal fun setNotificationForwardingQuietHours(
enabled: Boolean,
start: String,
end: String,
): Boolean {
if (!enabled) {
plainPrefs.edit { putBoolean(notificationsForwardingQuietHoursEnabledKey, false) }
_notificationForwardingQuietHoursEnabled.value = false
return true
}
val normalizedStart = normalizeLocalHourMinute(start) ?: return false
val normalizedEnd = normalizeLocalHourMinute(end) ?: return false
plainPrefs.edit {
putBoolean(notificationsForwardingQuietHoursEnabledKey, enabled)
putString(notificationsForwardingQuietStartKey, normalizedStart)
putString(notificationsForwardingQuietEndKey, normalizedEnd)
}
_notificationForwardingQuietHoursEnabled.value = enabled
_notificationForwardingQuietStart.value = normalizedStart
_notificationForwardingQuietEnd.value = normalizedEnd
return true
}
internal fun setNotificationForwardingMaxEventsPerMinute(value: Int) {
val normalized = value.coerceAtLeast(1)
plainPrefs.edit {
putInt(notificationsForwardingMaxEventsPerMinuteKey, normalized)
}
_notificationForwardingMaxEventsPerMinute.value = normalized
}
internal fun setNotificationForwardingSessionKey(value: String?) {
val normalized = value?.trim()?.takeIf { it.isNotEmpty() }
plainPrefs.edit {
putString(notificationsForwardingSessionKeyKey, normalized.orEmpty())
}
_notificationForwardingSessionKey.value = normalized
}
/** Loads manual or instance-scoped gateway token material from encrypted preferences. */
fun loadGatewayToken(): String? {
val manual =
_gatewayToken.value.trim().ifEmpty {
val stored = securePrefs.getString("gateway.manual.token", null)?.trim().orEmpty()
if (stored.isNotEmpty()) _gatewayToken.value = stored
stored
}
if (manual.isNotEmpty()) return manual
// Per-instance tokens keep reused Android installs from sharing stale gateway auth.
val key = "gateway.token.${_instanceId.value}"
val stored = securePrefs.getString(key, null)?.trim()
return stored?.takeIf { it.isNotEmpty() }
}
/** Loads the bootstrap token used during gateway setup and device-token handoff. */
fun loadGatewayBootstrapToken(): String? {
val key = "gateway.bootstrapToken.${_instanceId.value}"
val stored =
_gatewayBootstrapToken.value.trim().ifEmpty {
val persisted = securePrefs.getString(key, null)?.trim().orEmpty()
if (persisted.isNotEmpty()) {
_gatewayBootstrapToken.value = persisted
}
persisted
}
return stored.takeIf { it.isNotEmpty() }
}
fun saveGatewayBootstrapToken(token: String) {
val key = "gateway.bootstrapToken.${_instanceId.value}"
val trimmed = token.trim()
securePrefs.edit { putString(key, trimmed) }
_gatewayBootstrapToken.value = trimmed
}
fun loadGatewayPassword(): String? {
val key = "gateway.password.${_instanceId.value}"
val stored = securePrefs.getString(key, null)?.trim()
return stored?.takeIf { it.isNotEmpty() }
}
fun saveGatewayPassword(password: String) {
val key = "gateway.password.${_instanceId.value}"
securePrefs.edit { putString(key, password.trim()) }
}
/** Clears manual/setup credentials without removing persisted role-specific device tokens. */
fun clearGatewaySetupAuth() {
val instanceId = _instanceId.value
securePrefs.edit {
// Clear both current manual credentials and instance-scoped setup credentials after pairing/reset.
remove("gateway.manual.token")
remove("gateway.token.$instanceId")
remove("gateway.bootstrapToken.$instanceId")
remove("gateway.password.$instanceId")
}
_gatewayToken.value = ""
_gatewayBootstrapToken.value = ""
}
/** Loads the pinned gateway TLS fingerprint for a discovered/manual stable endpoint id. */
fun loadGatewayTlsFingerprint(stableId: String): String? {
val key = "gateway.tls.$stableId"
return plainPrefs.getString(key, null)?.trim()?.takeIf { it.isNotEmpty() }
}
/** Persists the gateway TLS fingerprint captured through TOFU or explicit trust. */
fun saveGatewayTlsFingerprint(
stableId: String,
fingerprint: String,
) {
val key = "gateway.tls.$stableId"
plainPrefs.edit { putString(key, fingerprint.trim()) }
}
fun getString(key: String): String? = securePrefs.getString(key, null)
fun putString(
key: String,
value: String,
) {
securePrefs.edit { putString(key, value) }
}
fun remove(key: String) {
securePrefs.edit { remove(key) }
}
private fun createSecurePrefs(
context: Context,
name: String,
): SharedPreferences =
EncryptedSharedPreferences.create(
context,
name,
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM,
)
private fun loadOrCreateInstanceId(): String {
val existing = plainPrefs.getString("node.instanceId", null)?.trim()
if (!existing.isNullOrBlank()) return existing
// Instance id is not secret; it scopes local credentials and survives display-name changes.
val fresh = UUID.randomUUID().toString()
plainPrefs.edit { putString("node.instanceId", fresh) }
return fresh
}
private fun loadOrMigrateDisplayName(context: Context): String {
val existing = plainPrefs.getString(displayNameKey, null)?.trim().orEmpty()
if (existing.isNotEmpty() && existing != "Android Node") return existing
// Replace the historical generic name with a device-specific default once.
val candidate = DeviceNames.bestDefaultNodeName(context).trim()
val resolved = candidate.ifEmpty { "Android Node" }
plainPrefs.edit { putString(displayNameKey, resolved) }
return resolved
}
/** Persists sanitized voice wake triggers and updates the reactive settings flow. */
fun setWakeWords(words: List<String>) {
val sanitized = WakeWords.sanitize(words, defaultWakeWords)
val encoded =
JsonArray(sanitized.map { JsonPrimitive(it) }).toString()
plainPrefs.edit { putString("voiceWake.triggerWords", encoded) }
_wakeWords.value = sanitized
}
fun setVoiceWakeMode(mode: VoiceWakeMode) {
plainPrefs.edit { putString(voiceWakeModeKey, mode.rawValue) }
_voiceWakeMode.value = mode
}
fun setVoiceMicEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(voiceMicEnabledKey, value) }
_voiceMicEnabled.value = value
}
fun setSpeakerEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("voice.speakerEnabled", value) }
_speakerEnabled.value = value
}
fun setAppearanceThemeMode(mode: AppearanceThemeMode) {
plainPrefs.edit { putString(appearanceThemeModeKey, mode.rawValue) }
_appearanceThemeMode.value = mode
}
private fun loadNotificationForwardingPackages(): Set<String> {
val raw = plainPrefs.getString(notificationsForwardingPackagesKey, null)?.trim()
if (raw.isNullOrEmpty()) {
return emptySet()
}
return try {
val element = json.parseToJsonElement(raw)
val array = element as? JsonArray ?: return emptySet()
array
.mapNotNull { item ->
when (item) {
is JsonNull -> null
is JsonPrimitive -> item.content.trim().takeIf { it.isNotEmpty() }
else -> null
}
}.toSet()
} catch (_: Throwable) {
emptySet()
}
}
private fun loadVoiceWakeMode(): VoiceWakeMode {
val raw = plainPrefs.getString(voiceWakeModeKey, null)
val resolved = VoiceWakeMode.fromRawValue(raw)
// Default ON (foreground) when unset, but keep "always" opt-in through explicit settings.
if (raw.isNullOrBlank()) {
plainPrefs.edit { putString(voiceWakeModeKey, resolved.rawValue) }
}
return resolved
}
private fun loadLocationMode(): LocationMode {
val raw = plainPrefs.getString(locationModeKey, "off")
val resolved = LocationMode.fromRawValue(raw)
if (raw?.trim()?.lowercase() == "always") {
// Migrate old "always" configs to the current while-using contract.
plainPrefs.edit { putString(locationModeKey, resolved.rawValue) }
}
return resolved
}
private fun loadCameraEnabled(): Boolean {
if (plainPrefs.contains(cameraEnabledKey)) {
return plainPrefs.getBoolean(cameraEnabledKey, false)
}
val migratedValue = hadPlainPrefsBeforeInit
plainPrefs.edit { putBoolean(cameraEnabledKey, migratedValue) }
return migratedValue
}
private fun loadWakeWords(): List<String> {
val raw = plainPrefs.getString("voiceWake.triggerWords", null)?.trim()
if (raw.isNullOrEmpty()) return defaultWakeWords
return try {
val element = json.parseToJsonElement(raw)
val array = element as? JsonArray ?: return defaultWakeWords
val decoded =
array.mapNotNull { item ->
when (item) {
is JsonNull -> null
is JsonPrimitive -> item.content.trim().takeIf { it.isNotEmpty() }
else -> null
}
}
WakeWords.sanitize(decoded, defaultWakeWords)
} catch (_: Throwable) {
defaultWakeWords
}
}
}

View File

@@ -0,0 +1,27 @@
package ai.openclaw.app
/** Normalizes blank gateway session keys to the legacy main session alias. */
internal fun normalizeMainKey(raw: String?): String {
val trimmed = raw?.trim()
return if (!trimmed.isNullOrEmpty()) trimmed else "main"
}
/** Extracts the agent id from canonical agent-scoped main session keys. */
internal fun resolveAgentIdFromMainSessionKey(raw: String?): String? {
val trimmed = raw?.trim().orEmpty()
if (!trimmed.startsWith("agent:")) return null
return trimmed
.removePrefix("agent:")
.substringBefore(':')
.trim()
.ifEmpty { null }
}
/** Builds the node session key shape consumed by gateway chat and presence APIs. */
internal fun buildNodeMainSessionKey(
deviceId: String,
agentId: String?,
): String {
val resolvedAgentId = agentId?.trim().orEmpty().ifEmpty { "main" }
return "agent:$resolvedAgentId:node-${deviceId.take(12)}"
}

View File

@@ -0,0 +1,10 @@
package ai.openclaw.app
/**
* Persisted voice capture mode that controls foreground-service microphone requirements.
*/
enum class VoiceCaptureMode {
Off,
ManualMic,
TalkMode,
}

View File

@@ -0,0 +1,20 @@
package ai.openclaw.app
/**
* Persisted wake-word mode; raw values are stored in secure preferences.
*/
enum class VoiceWakeMode(
val rawValue: String,
) {
Off("off"),
Foreground("foreground"),
Always("always"),
;
companion object {
/**
* Invalid stored values fall back to foreground wake so hands-free behavior stays opt-in.
*/
fun fromRawValue(raw: String?): VoiceWakeMode = entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Foreground
}
}

View File

@@ -0,0 +1,35 @@
package ai.openclaw.app
/**
* Wake-word parsing limits and sanitizers shared by settings and voice runtime paths.
*/
object WakeWords {
const val maxWords: Int = 32
const val maxWordLength: Int = 64
/** Splits comma-separated user input into non-empty wake-word entries. */
fun parseCommaSeparated(input: String): List<String> = input.split(",").map { it.trim() }.filter { it.isNotEmpty() }
/** Returns null when edited text normalizes to the current wake-word list. */
fun parseIfChanged(
input: String,
current: List<String>,
): List<String>? {
val parsed = parseCommaSeparated(input)
return if (parsed == current) null else parsed
}
/** Applies persisted-list bounds and falls back to defaults when all entries are empty. */
fun sanitize(
words: List<String>,
defaults: List<String>,
): List<String> {
val cleaned =
words
.map { it.trim() }
.filter { it.isNotEmpty() }
.take(maxWords)
.map { it.take(maxWordLength) }
return cleaned.ifEmpty { defaults }
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,80 @@
package ai.openclaw.app.chat
/**
* Chat transcript item as delivered by gateway chat history and live chat events.
*/
data class ChatMessage(
val id: String,
val role: String,
val content: List<ChatMessageContent>,
val timestampMs: Long?,
val idempotencyKey: String? = null,
)
/**
* One content part in a chat message; binary parts carry base64 plus their MIME metadata.
*/
data class ChatMessageContent(
val type: String = "text",
val text: String? = null,
val mimeType: String? = null,
val fileName: String? = null,
val base64: String? = null,
)
/**
* Tool call placeholder shown while a gateway run is still streaming.
*/
data class ChatPendingToolCall(
val toolCallId: String,
val name: String,
val args: kotlinx.serialization.json.JsonObject? = null,
val startedAtMs: Long,
val isError: Boolean? = null,
)
/**
* Stable session selector row; [key] is the gateway session key used in chat requests.
*/
data class ChatSessionEntry(
val key: String,
val updatedAtMs: Long?,
val displayName: String? = null,
val totalTokens: Long? = null,
val totalTokensFresh: Boolean? = null,
val contextTokens: Long? = null,
val hasContextUsageMetadata: Boolean = totalTokens != null || totalTokensFresh != null || contextTokens != null,
)
/**
* Slash command metadata exposed by the gateway for text-surface chat clients.
*/
data class ChatCommandEntry(
val name: String,
val description: String,
val category: String? = null,
val textAliases: List<String> = emptyList(),
val acceptsArgs: Boolean = false,
)
/**
* Snapshot of one chat session, including optional thinking level selected on the gateway.
*/
data class ChatHistory(
val sessionKey: String,
val sessionId: String?,
val thinkingLevel: String?,
val messages: List<ChatMessage>,
val sessionInfo: ChatSessionEntry? = null,
)
/**
* User-selected attachment payload sent to the gateway as inline base64.
*/
data class OutgoingAttachment(
val type: String,
val mimeType: String,
val fileName: String,
val base64: String,
)

View File

@@ -0,0 +1,40 @@
package ai.openclaw.app.gateway
/**
* Decoder for Bonjour DNS-SD service names returned with decimal byte escapes.
*/
object BonjourEscapes {
/** Decodes Bonjour DNS-SD decimal escapes while preserving ordinary UTF-8. */
fun decode(input: String): String {
if (input.isEmpty()) return input
val bytes = mutableListOf<Byte>()
var i = 0
while (i < input.length) {
if (input[i] == '\\' && i + 3 < input.length) {
val d0 = input[i + 1]
val d1 = input[i + 2]
val d2 = input[i + 3]
if (d0.isDigit() && d1.isDigit() && d2.isDigit()) {
val value =
((d0.code - '0'.code) * 100) + ((d1.code - '0'.code) * 10) + (d2.code - '0'.code)
if (value in 0..255) {
// Bonjour escape bytes are decimal octets, not Unicode code points.
bytes.add(value.toByte())
i += 4
continue
}
}
}
val codePoint = Character.codePointAt(input, i)
val charBytes = String(Character.toChars(codePoint)).toByteArray(Charsets.UTF_8)
for (b in charBytes) {
bytes.add(b)
}
i += Character.charCount(codePoint)
}
return String(bytes.toByteArray(), Charsets.UTF_8)
}
}

View File

@@ -0,0 +1,46 @@
package ai.openclaw.app.gateway
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
internal data class ChatSendAck(
val runId: String?,
val status: String?,
) {
val normalizedStatus: String
get() = status?.trim()?.lowercase().orEmpty()
val isTerminalSuccess: Boolean
get() = normalizedStatus == "ok"
val isTerminalFailure: Boolean
get() = normalizedStatus == "timeout" || normalizedStatus == "error"
val isTerminal: Boolean
get() = isTerminalSuccess || isTerminalFailure
}
internal fun chatSendAckHistorySinceSeconds(
ack: ChatSendAck,
startedAtSeconds: Double,
): Double? = if (ack.isTerminalSuccess) null else startedAtSeconds
internal fun parseChatSendAck(
json: Json,
responseJson: String,
): ChatSendAck =
try {
val obj = json.parseToJsonElement(responseJson).asObjectOrNull()
ChatSendAck(
runId = obj?.get("runId").asStringOrNull(),
status = obj?.get("status").asStringOrNull(),
)
} catch (_: Throwable) {
ChatSendAck(runId = null, status = null)
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonElement?.asStringOrNull(): String? = (this as? JsonPrimitive)?.takeIf { it.isString }?.content

View File

@@ -0,0 +1,57 @@
package ai.openclaw.app.gateway
/**
* Canonical device-auth payload builder shared with gateway verification rules.
*/
internal object DeviceAuthPayload {
/** Builds the canonical v3 auth string signed by device registration flows. */
fun buildV3(
deviceId: String,
clientId: String,
clientMode: String,
role: String,
scopes: List<String>,
signedAtMs: Long,
token: String?,
nonce: String,
platform: String?,
deviceFamily: String?,
): String {
val scopeString = scopes.joinToString(",")
val authToken = token.orEmpty()
val platformNorm = normalizeMetadataField(platform)
val deviceFamilyNorm = normalizeMetadataField(deviceFamily)
return listOf(
"v3",
deviceId,
clientId,
clientMode,
role,
scopeString,
signedAtMs.toString(),
authToken,
nonce,
platformNorm,
deviceFamilyNorm,
).joinToString("|")
}
/** Normalizes signed metadata fields without locale-sensitive lowercasing. */
internal fun normalizeMetadataField(value: String?): String {
val trimmed = value?.trim().orEmpty()
if (trimmed.isEmpty()) {
return ""
}
// Keep cross-runtime normalization deterministic (TS/Swift/Kotlin):
// lowercase ASCII A-Z only for auth payload metadata fields.
val out = StringBuilder(trimmed.length)
for (ch in trimmed) {
if (ch in 'A'..'Z') {
out.append((ch.code + 32).toChar())
} else {
out.append(ch)
}
}
return out.toString()
}
}

View File

@@ -0,0 +1,142 @@
package ai.openclaw.app.gateway
import ai.openclaw.app.SecurePrefs
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
/** Stored gateway device-token material scoped by device id and role. */
data class DeviceAuthEntry(
val token: String,
val role: String,
val scopes: List<String>,
val updatedAtMs: Long,
)
@Serializable
private data class PersistedDeviceAuthMetadata(
val scopes: List<String> = emptyList(),
val updatedAtMs: Long = 0L,
)
/** Persistence interface used by gateway pairing/session code for role tokens. */
interface DeviceAuthTokenStore {
/** Loads the stored token plus metadata for one device/role pair. */
fun loadEntry(
deviceId: String,
role: String,
): DeviceAuthEntry?
/** Loads only the bearer token when callers do not need scope metadata. */
fun loadToken(
deviceId: String,
role: String,
): String? = loadEntry(deviceId, role)?.token
/** Persists a role token and deterministic scope metadata under normalized keys. */
fun saveToken(
deviceId: String,
role: String,
token: String,
scopes: List<String> = emptyList(),
)
/** Removes both token and metadata for the normalized device/role pair. */
fun clearToken(
deviceId: String,
role: String,
)
}
/** SecurePrefs-backed implementation of Android gateway device-token storage. */
class DeviceAuthStore(
private val prefs: SecurePrefs,
) : DeviceAuthTokenStore {
private val json = Json { ignoreUnknownKeys = true }
override fun loadEntry(
deviceId: String,
role: String,
): DeviceAuthEntry? {
val key = tokenKey(deviceId, role)
val token = prefs.getString(key)?.trim()?.takeIf { it.isNotEmpty() } ?: return null
val normalizedRole = normalizeRole(role)
val metadata =
prefs
.getString(metadataKey(deviceId, role))
?.let { raw ->
runCatching { json.decodeFromString<PersistedDeviceAuthMetadata>(raw) }.getOrNull()
}
return DeviceAuthEntry(
token = token,
role = normalizedRole,
scopes = metadata?.scopes ?: emptyList(),
updatedAtMs = metadata?.updatedAtMs ?: 0L,
)
}
override fun saveToken(
deviceId: String,
role: String,
token: String,
scopes: List<String>,
) {
val normalizedScopes = normalizeScopes(scopes)
val key = tokenKey(deviceId, role)
prefs.putString(key, token.trim())
prefs.putString(
metadataKey(deviceId, role),
json.encodeToString(
PersistedDeviceAuthMetadata(
scopes = normalizedScopes,
updatedAtMs = System.currentTimeMillis(),
),
),
)
}
override fun clearToken(
deviceId: String,
role: String,
) {
val key = tokenKey(deviceId, role)
prefs.remove(key)
prefs.remove(metadataKey(deviceId, role))
}
private fun tokenKey(
deviceId: String,
role: String,
): String {
val normalizedDevice = normalizeDeviceId(deviceId)
val normalizedRole = normalizeRole(role)
// Keep key normalization shared with metadata keys so token and metadata
// are added/removed as one logical auth entry.
return "gateway.deviceToken.$normalizedDevice.$normalizedRole"
}
private fun metadataKey(
deviceId: String,
role: String,
): String {
val normalizedDevice = normalizeDeviceId(deviceId)
val normalizedRole = normalizeRole(role)
return "gateway.deviceTokenMeta.$normalizedDevice.$normalizedRole"
}
/** Normalizes device ids before they become encrypted preference key segments. */
private fun normalizeDeviceId(deviceId: String): String = deviceId.trim().lowercase()
/** Normalizes role names so node/operator token slots are stable across callers. */
private fun normalizeRole(role: String): String = role.trim().lowercase()
/** Stores scopes in deterministic order for display and restart comparisons. */
private fun normalizeScopes(scopes: List<String>): List<String> =
scopes
.map { it.trim() }
.filter { it.isNotEmpty() }
// Persist deterministic scope lists because they are displayed and may be
// compared across process restarts.
.distinct()
.sorted()
}

View File

@@ -0,0 +1,211 @@
package ai.openclaw.app.gateway
import android.content.Context
import android.util.Base64
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import java.io.File
import java.security.MessageDigest
/** Persistent Ed25519 identity used to register this Android node with gateways. */
@Serializable
data class DeviceIdentity(
val deviceId: String,
val publicKeyRawBase64: String,
val privateKeyPkcs8Base64: String,
val createdAtMs: Long,
)
/** Owns device identity generation, persistence, and auth payload signatures. */
class DeviceIdentityStore(
context: Context,
) {
private val json = Json { ignoreUnknownKeys = true }
private val identityFile = File(context.filesDir, "openclaw/identity/device.json")
@Volatile private var cachedIdentity: DeviceIdentity? = null
/** Loads the persisted identity or creates one, repairing old device-id drift. */
@Synchronized
fun loadOrCreate(): DeviceIdentity {
cachedIdentity?.let { return it }
val existing = load()
if (existing != null) {
val derived = deriveDeviceId(existing.publicKeyRawBase64)
if (derived != null && derived != existing.deviceId) {
val updated = existing.copy(deviceId = derived)
save(updated)
cachedIdentity = updated
return updated
}
cachedIdentity = existing
return existing
}
val fresh = generate()
save(fresh)
cachedIdentity = fresh
return fresh
}
/** Signs gateway connect payload text with the persisted Ed25519 private key. */
fun signPayload(
payload: String,
identity: DeviceIdentity,
): String? =
try {
// Use BC lightweight API directly; R8 can break JCA provider registration.
val privateKeyBytes = Base64.decode(identity.privateKeyPkcs8Base64, Base64.DEFAULT)
val pkInfo =
org.bouncycastle.asn1.pkcs.PrivateKeyInfo
.getInstance(privateKeyBytes)
val parsed = pkInfo.parsePrivateKey()
val rawPrivate =
org.bouncycastle.asn1.DEROctetString
.getInstance(parsed)
.octets
val privateKey =
org.bouncycastle.crypto.params
.Ed25519PrivateKeyParameters(rawPrivate, 0)
val signer =
org.bouncycastle.crypto.signers
.Ed25519Signer()
signer.init(true, privateKey)
val payloadBytes = payload.toByteArray(Charsets.UTF_8)
signer.update(payloadBytes, 0, payloadBytes.size)
base64UrlEncode(signer.generateSignature())
} catch (e: Throwable) {
android.util.Log.e("DeviceAuth", "signPayload FAILED: ${e.javaClass.simpleName}: ${e.message}", e)
null
}
/** Verifies a signature against the persisted public key for debug diagnostics. */
fun verifySelfSignature(
payload: String,
signatureBase64Url: String,
identity: DeviceIdentity,
): Boolean =
try {
val rawPublicKey = Base64.decode(identity.publicKeyRawBase64, Base64.DEFAULT)
val pubKey =
org.bouncycastle.crypto.params
.Ed25519PublicKeyParameters(rawPublicKey, 0)
val sigBytes = base64UrlDecode(signatureBase64Url)
val verifier =
org.bouncycastle.crypto.signers
.Ed25519Signer()
verifier.init(false, pubKey)
val payloadBytes = payload.toByteArray(Charsets.UTF_8)
verifier.update(payloadBytes, 0, payloadBytes.size)
verifier.verifySignature(sigBytes)
} catch (e: Throwable) {
android.util.Log.e("DeviceAuth", "self-verify exception: ${e.message}", e)
false
}
/** Decodes gateway URL-safe base64 signatures, accepting unpadded input. */
private fun base64UrlDecode(input: String): ByteArray {
val normalized = input.replace('-', '+').replace('_', '/')
// Android Base64 expects padded input; gateway signatures are URL-safe
// unpadded strings.
val padded = normalized + "=".repeat((4 - normalized.length % 4) % 4)
return Base64.decode(padded, Base64.DEFAULT)
}
/** Returns the public key in the gateway's unpadded URL-safe base64 format. */
fun publicKeyBase64Url(identity: DeviceIdentity): String? =
try {
val raw = Base64.decode(identity.publicKeyRawBase64, Base64.DEFAULT)
base64UrlEncode(raw)
} catch (_: Throwable) {
null
}
private fun load(): DeviceIdentity? = readIdentity(identityFile)
private fun readIdentity(file: File): DeviceIdentity? {
return try {
if (!file.exists()) return null
val raw = file.readText(Charsets.UTF_8)
val decoded = json.decodeFromString(DeviceIdentity.serializer(), raw)
if (decoded.deviceId.isBlank() ||
decoded.publicKeyRawBase64.isBlank() ||
decoded.privateKeyPkcs8Base64.isBlank()
) {
null
} else {
decoded
}
} catch (_: Throwable) {
null
}
}
private fun save(identity: DeviceIdentity) {
try {
identityFile.parentFile?.mkdirs()
val encoded = json.encodeToString(DeviceIdentity.serializer(), identity)
identityFile.writeText(encoded, Charsets.UTF_8)
} catch (_: Throwable) {
// best-effort only
}
}
private fun generate(): DeviceIdentity {
// Use BC lightweight API directly to avoid JCA provider issues with R8.
val kpGen =
org.bouncycastle.crypto.generators
.Ed25519KeyPairGenerator()
kpGen.init(
org.bouncycastle.crypto.params
.Ed25519KeyGenerationParameters(java.security.SecureRandom()),
)
val kp = kpGen.generateKeyPair()
val pubKey = kp.public as org.bouncycastle.crypto.params.Ed25519PublicKeyParameters
val privKey = kp.private as org.bouncycastle.crypto.params.Ed25519PrivateKeyParameters
val rawPublic = pubKey.encoded // 32 bytes
val deviceId = sha256Hex(rawPublic)
// Store private key as PKCS8 so signPayload can parse the same persisted
// shape after app restarts and upgrades.
val privKeyInfo =
org.bouncycastle.crypto.util.PrivateKeyInfoFactory
.createPrivateKeyInfo(privKey)
val pkcs8Bytes = privKeyInfo.encoded
return DeviceIdentity(
deviceId = deviceId,
publicKeyRawBase64 = Base64.encodeToString(rawPublic, Base64.NO_WRAP),
privateKeyPkcs8Base64 = Base64.encodeToString(pkcs8Bytes, Base64.NO_WRAP),
createdAtMs = System.currentTimeMillis(),
)
}
/** Re-derives the stable device id from the raw Ed25519 public key bytes. */
private fun deriveDeviceId(publicKeyRawBase64: String): String? =
try {
val raw = Base64.decode(publicKeyRawBase64, Base64.DEFAULT)
sha256Hex(raw)
} catch (_: Throwable) {
null
}
private fun sha256Hex(data: ByteArray): String {
val digest = MessageDigest.getInstance("SHA-256").digest(data)
val out = CharArray(digest.size * 2)
var i = 0
for (byte in digest) {
val v = byte.toInt() and 0xff
out[i++] = HEX[v ushr 4]
out[i++] = HEX[v and 0x0f]
}
return String(out)
}
private fun base64UrlEncode(data: ByteArray): String =
Base64.encodeToString(
data,
Base64.URL_SAFE or Base64.NO_WRAP or Base64.NO_PADDING,
)
companion object {
private val HEX = "0123456789abcdef".toCharArray()
}
}

View File

@@ -0,0 +1,689 @@
package ai.openclaw.app.gateway
import android.content.Context
import android.net.ConnectivityManager
import android.net.DnsResolver
import android.net.Network
import android.net.NetworkCapabilities
import android.net.NetworkRequest
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import android.os.Build
import android.os.CancellationSignal
import android.util.Log
import androidx.annotation.RequiresApi
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.coroutines.suspendCancellableCoroutine
import org.xbill.DNS.AAAARecord
import org.xbill.DNS.ARecord
import org.xbill.DNS.DClass
import org.xbill.DNS.ExtendedResolver
import org.xbill.DNS.Message
import org.xbill.DNS.Name
import org.xbill.DNS.PTRRecord
import org.xbill.DNS.Rcode
import org.xbill.DNS.Record
import org.xbill.DNS.Resolver
import org.xbill.DNS.SRVRecord
import org.xbill.DNS.Section
import org.xbill.DNS.SimpleResolver
import org.xbill.DNS.TXTRecord
import org.xbill.DNS.TextParseException
import org.xbill.DNS.Type
import java.io.IOException
import java.net.InetAddress
import java.net.InetSocketAddress
import java.nio.ByteBuffer
import java.nio.charset.CodingErrorAction
import java.time.Duration
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.Executor
import java.util.concurrent.Executors
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
@Suppress("DEPRECATION")
private fun createDnsResolver(): DnsResolver = DnsResolver.getInstance()
/**
* Watches local DNS-SD and optional wide-area DNS-SD for reachable OpenClaw gateways.
*/
class GatewayDiscovery(
context: Context,
private val scope: CoroutineScope,
) {
private val nsd = context.getSystemService(NsdManager::class.java)
private val connectivity = context.getSystemService(ConnectivityManager::class.java)
private val dns = createDnsResolver()
private val serviceType = "_openclaw-gw._tcp."
private val wideAreaDomain = System.getenv("OPENCLAW_WIDE_AREA_DOMAIN")
private val logTag = "OpenClaw/GatewayDiscovery"
private val localById = ConcurrentHashMap<String, GatewayEndpoint>()
private val unicastById = ConcurrentHashMap<String, GatewayEndpoint>()
private val _gateways = MutableStateFlow<List<GatewayEndpoint>>(emptyList())
/** Current discovered gateway list, merged from local DNS-SD and optional wide-area DNS-SD. */
val gateways: StateFlow<List<GatewayEndpoint>> = _gateways.asStateFlow()
private val _statusText = MutableStateFlow("Searching…")
/** Short diagnostic text shown by connect UI while discovery is running. */
val statusText: StateFlow<String> = _statusText.asStateFlow()
private var unicastJob: Job? = null
private val dnsExecutor: Executor = Executors.newCachedThreadPool()
private val availableNetworks = ConcurrentHashMap.newKeySet<Network>()
private val serviceInfoCallbacks = ConcurrentHashMap<String, Any>()
@Volatile private var lastWideAreaRcode: Int? = null
@Volatile private var lastWideAreaCount: Int = 0
private val networkCallback =
object : ConnectivityManager.NetworkCallback() {
override fun onAvailable(network: Network) {
availableNetworks.add(network)
}
override fun onLost(network: Network) {
availableNetworks.remove(network)
}
}
private val discoveryListener =
object : NsdManager.DiscoveryListener {
override fun onStartDiscoveryFailed(
serviceType: String,
errorCode: Int,
) {}
override fun onStopDiscoveryFailed(
serviceType: String,
errorCode: Int,
) {}
override fun onDiscoveryStarted(serviceType: String) {}
override fun onDiscoveryStopped(serviceType: String) {}
override fun onServiceFound(serviceInfo: NsdServiceInfo) {
if (serviceInfo.serviceType != this@GatewayDiscovery.serviceType) return
resolve(serviceInfo)
}
override fun onServiceLost(serviceInfo: NsdServiceInfo) {
val serviceName = BonjourEscapes.decode(serviceInfo.serviceName)
val id = stableId(serviceName, "local.")
localById.remove(id)
unregisterServiceInfoCallback(id)
publish()
}
}
init {
startNetworkTracking()
startLocalDiscovery()
if (!wideAreaDomain.isNullOrBlank()) {
startUnicastDiscovery(wideAreaDomain)
}
}
private fun startNetworkTracking() {
val cm = connectivity ?: return
cm.activeNetwork?.let(availableNetworks::add)
try {
// Track all networks so wide-area DNS can prefer VPN/split-DNS answers
// even when Android's active network is not the VPN.
cm.registerNetworkCallback(NetworkRequest.Builder().build(), networkCallback)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun startLocalDiscovery() {
try {
nsd.discoverServices(serviceType, NsdManager.PROTOCOL_DNS_SD, discoveryListener)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun startUnicastDiscovery(domain: String) {
unicastJob =
scope.launch(Dispatchers.IO) {
while (true) {
try {
refreshUnicast(domain)
} catch (_: Throwable) {
// ignore (best-effort)
}
delay(5000)
}
}
}
private fun resolve(serviceInfo: NsdServiceInfo) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
// Android 14+ streams service updates; older releases require one-shot resolve calls.
resolveWithServiceInfoCallback(serviceInfo)
} else {
resolveLegacy(serviceInfo)
}
}
@RequiresApi(Build.VERSION_CODES.UPSIDE_DOWN_CAKE)
private fun resolveWithServiceInfoCallback(serviceInfo: NsdServiceInfo) {
val serviceName = BonjourEscapes.decode(serviceInfo.serviceName)
val id = stableId(serviceName, "local.")
if (serviceInfoCallbacks.containsKey(id)) return
val callback =
object : NsdManager.ServiceInfoCallback {
override fun onServiceInfoCallbackRegistrationFailed(errorCode: Int) {
serviceInfoCallbacks.remove(id, this)
}
override fun onServiceInfoCallbackUnregistered() {
serviceInfoCallbacks.remove(id, this)
}
override fun onServiceLost() {
localById.remove(id)
publish()
}
override fun onServiceUpdated(serviceInfo: NsdServiceInfo) {
upsertResolvedService(serviceInfo)
}
}
serviceInfoCallbacks[id] = callback
try {
nsd.registerServiceInfoCallback(serviceInfo, dnsExecutor, callback)
} catch (_: Throwable) {
serviceInfoCallbacks.remove(id, callback)
}
}
private fun unregisterServiceInfoCallback(id: String) {
val callback = serviceInfoCallbacks.remove(id) ?: return
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.UPSIDE_DOWN_CAKE) return
try {
nsd.unregisterServiceInfoCallback(callback as NsdManager.ServiceInfoCallback)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun resolveLegacy(serviceInfo: NsdServiceInfo) {
val listener =
object : NsdManager.ResolveListener {
override fun onResolveFailed(
serviceInfo: NsdServiceInfo,
errorCode: Int,
) {}
override fun onServiceResolved(resolved: NsdServiceInfo) {
upsertResolvedService(resolved)
}
}
try {
NsdManager::class.java
.getMethod("resolveService", NsdServiceInfo::class.java, NsdManager.ResolveListener::class.java)
.invoke(nsd, serviceInfo, listener)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun upsertResolvedService(resolved: NsdServiceInfo) {
val host = resolvedHostAddress(resolved) ?: return
val port = resolved.port
if (port <= 0) return
val rawServiceName = resolved.serviceName
val serviceName = BonjourEscapes.decode(rawServiceName)
val displayName = BonjourEscapes.decode(txt(resolved, "displayName") ?: serviceName)
val lanHost = txt(resolved, "lanHost")
val tailnetDns = txt(resolved, "tailnetDns")
val gatewayPort = txtInt(resolved, "gatewayPort")
val canvasPort = txtInt(resolved, "canvasPort")
val tlsEnabled = txtBool(resolved, "gatewayTls")
val tlsFingerprint = txt(resolved, "gatewayTlsSha256")
val id = stableId(serviceName, "local.")
// Local NSD gives the socket host/port; TXT ports are retained as gateway metadata only.
localById[id] =
GatewayEndpoint(
stableId = id,
name = displayName,
host = host,
port = port,
lanHost = lanHost,
tailnetDns = tailnetDns,
gatewayPort = gatewayPort,
canvasPort = canvasPort,
tlsEnabled = tlsEnabled,
tlsFingerprintSha256 = tlsFingerprint,
)
publish()
}
private fun resolvedHostAddress(resolved: NsdServiceInfo): String? {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
return resolved.hostAddresses.firstOrNull()?.hostAddress
}
return legacyHostAddress(resolved)
}
private fun legacyHostAddress(resolved: NsdServiceInfo): String? =
try {
val host = NsdServiceInfo::class.java.getMethod("getHost").invoke(resolved) as? InetAddress
host?.hostAddress
} catch (_: Throwable) {
null
}
private fun publish() {
_gateways.value =
// Merge local and wide-area results deterministically for stable UI selection.
(localById.values + unicastById.values).sortedBy { it.name.lowercase() }
_statusText.value = buildStatusText()
}
private fun buildStatusText(): String {
val localCount = localById.size
val wideRcode = lastWideAreaRcode
val wideCount = lastWideAreaCount
val wide =
when (wideRcode) {
null -> "Wide: ?"
Rcode.NOERROR -> "Wide: $wideCount"
Rcode.NXDOMAIN -> "Wide: NXDOMAIN"
else -> "Wide: ${Rcode.string(wideRcode)}"
}
return when {
localCount == 0 && wideRcode == null -> "Searching for gateways…"
localCount == 0 -> "$wide"
else -> "Local: $localCount$wide"
}
}
private fun stableId(
serviceName: String,
domain: String,
): String = "$serviceType|$domain|${normalizeName(serviceName)}"
private fun normalizeName(raw: String): String = raw.trim().split(Regex("\\s+")).joinToString(" ")
private fun txt(
info: NsdServiceInfo,
key: String,
): String? {
val bytes = info.attributes[key] ?: return null
return try {
String(bytes, Charsets.UTF_8).trim().ifEmpty { null }
} catch (_: Throwable) {
null
}
}
private fun txtInt(
info: NsdServiceInfo,
key: String,
): Int? = txt(info, key)?.toIntOrNull()
private fun txtBool(
info: NsdServiceInfo,
key: String,
): Boolean {
val raw = txt(info, key)?.trim()?.lowercase() ?: return false
return raw == "1" || raw == "true" || raw == "yes"
}
private suspend fun refreshUnicast(domain: String) {
val ptrName = "${serviceType}$domain"
val ptrMsg = lookupUnicastMessage(ptrName, Type.PTR) ?: return
val ptrRecords = records(ptrMsg, Section.ANSWER).mapNotNull { it as? PTRRecord }
val next = LinkedHashMap<String, GatewayEndpoint>()
for (ptr in ptrRecords) {
val instanceFqdn = ptr.target.toString()
val srv =
recordByName(ptrMsg, instanceFqdn, Type.SRV) as? SRVRecord
?: run {
val msg = lookupUnicastMessage(instanceFqdn, Type.SRV) ?: return@run null
recordByName(msg, instanceFqdn, Type.SRV) as? SRVRecord
}
?: continue
val port = srv.port
if (port <= 0) continue
val targetFqdn = srv.target.toString()
val host =
resolveHostFromMessage(ptrMsg, targetFqdn)
?: resolveHostFromMessage(lookupUnicastMessage(instanceFqdn, Type.SRV), targetFqdn)
?: resolveHostUnicast(targetFqdn)
?: continue
// Wide-area DNS-SD may put TXT in additional records; fall back to a direct TXT query.
val txtFromPtr =
recordsByName(ptrMsg, Section.ADDITIONAL)[keyName(instanceFqdn)]
.orEmpty()
.mapNotNull { it as? TXTRecord }
val txt =
if (txtFromPtr.isNotEmpty()) {
txtFromPtr
} else {
val msg = lookupUnicastMessage(instanceFqdn, Type.TXT)
records(msg, Section.ANSWER).mapNotNull { it as? TXTRecord }
}
val instanceName = BonjourEscapes.decode(decodeInstanceName(instanceFqdn, domain))
val displayName = BonjourEscapes.decode(txtValue(txt, "displayName") ?: instanceName)
val lanHost = txtValue(txt, "lanHost")
val tailnetDns = txtValue(txt, "tailnetDns")
val gatewayPort = txtIntValue(txt, "gatewayPort")
val canvasPort = txtIntValue(txt, "canvasPort")
val tlsEnabled = txtBoolValue(txt, "gatewayTls")
val tlsFingerprint = txtValue(txt, "gatewayTlsSha256")
val id = stableId(instanceName, domain)
next[id] =
GatewayEndpoint(
stableId = id,
name = displayName,
host = host,
port = port,
lanHost = lanHost,
tailnetDns = tailnetDns,
gatewayPort = gatewayPort,
canvasPort = canvasPort,
tlsEnabled = tlsEnabled,
tlsFingerprintSha256 = tlsFingerprint,
)
}
unicastById.clear()
unicastById.putAll(next)
lastWideAreaRcode = ptrMsg.header.rcode
lastWideAreaCount = next.size
publish()
if (next.isEmpty()) {
Log.d(
logTag,
"wide-area discovery: 0 results for $ptrName (rcode=${Rcode.string(ptrMsg.header.rcode)})",
)
}
}
private fun decodeInstanceName(
instanceFqdn: String,
domain: String,
): String {
val suffix = "${serviceType}$domain"
val withoutSuffix =
if (instanceFqdn.endsWith(suffix)) {
instanceFqdn.removeSuffix(suffix)
} else {
instanceFqdn.substringBefore(serviceType)
}
return normalizeName(stripTrailingDot(withoutSuffix))
}
private fun stripTrailingDot(raw: String): String = raw.removeSuffix(".")
private suspend fun lookupUnicastMessage(
name: String,
type: Int,
): Message? {
val query =
try {
Message.newQuery(
org.xbill.DNS.Record.newRecord(
Name.fromString(name),
type,
DClass.IN,
),
)
} catch (_: TextParseException) {
return null
}
val system = queryViaSystemDns(query)
if (records(system, Section.ANSWER).any { it.type == type }) return system
// Android's DnsResolver can miss split-DNS answers; retry with dnsjava against network DNS servers.
val direct = createDirectResolver() ?: return system
return try {
val msg = direct.send(query)
if (records(msg, Section.ANSWER).any { it.type == type }) msg else system
} catch (_: Throwable) {
system
}
}
private suspend fun queryViaSystemDns(query: Message): Message? {
val network = preferredDnsNetwork()
val bytes =
try {
rawQuery(network, query.toWire())
} catch (_: Throwable) {
return null
}
return try {
Message(bytes)
} catch (_: IOException) {
null
}
}
private fun records(
msg: Message?,
section: Int,
): List<Record> = msg?.getSection(section).orEmpty()
private fun keyName(raw: String): String = raw.trim().lowercase()
private fun recordsByName(
msg: Message,
section: Int,
): Map<String, List<Record>> {
val next = LinkedHashMap<String, MutableList<Record>>()
for (r in records(msg, section)) {
val name = r.name?.toString() ?: continue
next.getOrPut(keyName(name)) { mutableListOf() }.add(r)
}
return next
}
private fun recordByName(
msg: Message,
fqdn: String,
type: Int,
): Record? {
val key = keyName(fqdn)
val byNameAnswer = recordsByName(msg, Section.ANSWER)
val fromAnswer = byNameAnswer[key].orEmpty().firstOrNull { it.type == type }
if (fromAnswer != null) return fromAnswer
val byNameAdditional = recordsByName(msg, Section.ADDITIONAL)
return byNameAdditional[key].orEmpty().firstOrNull { it.type == type }
}
private fun resolveHostFromMessage(
msg: Message?,
hostname: String,
): String? {
val m = msg ?: return null
val key = keyName(hostname)
val additional = recordsByName(m, Section.ADDITIONAL)[key].orEmpty()
val a = additional.mapNotNull { it as? ARecord }.mapNotNull { it.address?.hostAddress }
val aaaa = additional.mapNotNull { it as? AAAARecord }.mapNotNull { it.address?.hostAddress }
return a.firstOrNull() ?: aaaa.firstOrNull()
}
private fun preferredDnsNetwork(): android.net.Network? {
val cm = connectivity ?: return null
// Prefer VPN (Tailscale) when present; otherwise use the active network.
trackedNetworks(cm)
.firstOrNull { n ->
val caps = cm.getNetworkCapabilities(n) ?: return@firstOrNull false
caps.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
}?.let { return it }
return cm.activeNetwork
}
private fun trackedNetworks(cm: ConnectivityManager): List<Network> =
buildList {
cm.activeNetwork?.let(::add)
addAll(availableNetworks)
}.distinct()
private fun createDirectResolver(): Resolver? {
val cm = connectivity ?: return null
val candidateNetworks =
buildList {
// Put VPN DNS first so Tailscale split-horizon names win over public DNS.
trackedNetworks(cm)
.firstOrNull { n ->
val caps = cm.getNetworkCapabilities(n) ?: return@firstOrNull false
caps.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
}?.let(::add)
cm.activeNetwork?.let(::add)
}.distinct()
val servers =
candidateNetworks
.asSequence()
.flatMap { n ->
cm.getLinkProperties(n)?.dnsServers?.asSequence() ?: emptySequence()
}.distinctBy { it.hostAddress ?: it.toString() }
.toList()
if (servers.isEmpty()) return null
return try {
val resolvers =
servers.mapNotNull { addr ->
try {
SimpleResolver().apply {
setAddress(InetSocketAddress(addr, 53))
setTimeout(Duration.ofSeconds(3))
}
} catch (_: Throwable) {
null
}
}
if (resolvers.isEmpty()) return null
ExtendedResolver(resolvers.toTypedArray()).apply { setTimeout(Duration.ofSeconds(3)) }
} catch (_: Throwable) {
null
}
}
private suspend fun rawQuery(
network: android.net.Network?,
wireQuery: ByteArray,
): ByteArray =
suspendCancellableCoroutine { cont ->
val signal = CancellationSignal()
cont.invokeOnCancellation { signal.cancel() }
dns.rawQuery(
network,
wireQuery,
DnsResolver.FLAG_EMPTY,
dnsExecutor,
signal,
object : DnsResolver.Callback<ByteArray> {
override fun onAnswer(
answer: ByteArray,
rcode: Int,
) {
cont.resume(answer)
}
override fun onError(error: DnsResolver.DnsException) {
cont.resumeWithException(error)
}
},
)
}
private fun txtValue(
records: List<TXTRecord>,
key: String,
): String? {
val prefix = "$key="
for (r in records) {
val strings: List<String> =
try {
r.strings
} catch (_: Throwable) {
emptyList()
}
for (s in strings) {
val trimmed = decodeDnsTxtString(s).trim()
if (trimmed.startsWith(prefix)) {
return trimmed.removePrefix(prefix).trim().ifEmpty { null }
}
}
}
return null
}
private fun txtIntValue(
records: List<TXTRecord>,
key: String,
): Int? = txtValue(records, key)?.toIntOrNull()
private fun txtBoolValue(
records: List<TXTRecord>,
key: String,
): Boolean {
val raw = txtValue(records, key)?.trim()?.lowercase() ?: return false
return raw == "1" || raw == "true" || raw == "yes"
}
private fun decodeDnsTxtString(raw: String): String {
// dnsjava treats TXT as opaque bytes and decodes as ISO-8859-1 to preserve bytes.
// Our TXT payload is UTF-8 (written by the gateway), so re-decode when possible.
val bytes = raw.toByteArray(Charsets.ISO_8859_1)
val decoder =
Charsets.UTF_8
.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT)
return try {
decoder.decode(ByteBuffer.wrap(bytes)).toString()
} catch (_: Throwable) {
raw
}
}
private suspend fun resolveHostUnicast(hostname: String): String? {
val a =
records(lookupUnicastMessage(hostname, Type.A), Section.ANSWER)
.mapNotNull { it as? ARecord }
.mapNotNull { it.address?.hostAddress }
val aaaa =
records(lookupUnicastMessage(hostname, Type.AAAA), Section.ANSWER)
.mapNotNull { it as? AAAARecord }
.mapNotNull { it.address?.hostAddress }
return a.firstOrNull() ?: aaaa.firstOrNull()
}
}

View File

@@ -0,0 +1,31 @@
package ai.openclaw.app.gateway
/** Resolved gateway address and optional metadata discovered from Bonjour/manual entry. */
data class GatewayEndpoint(
val stableId: String,
val name: String,
val host: String,
val port: Int,
val lanHost: String? = null,
val tailnetDns: String? = null,
val gatewayPort: Int? = null,
val canvasPort: Int? = null,
val tlsEnabled: Boolean = false,
val tlsFingerprintSha256: String? = null,
) {
companion object {
/** Builds a stable manual endpoint key that survives display-name changes. */
fun manual(
host: String,
port: Int,
): GatewayEndpoint =
GatewayEndpoint(
stableId = "manual|${host.lowercase()}|$port",
name = "$host:$port",
host = host,
port = port,
tlsEnabled = false,
tlsFingerprintSha256 = null,
)
}
}

View File

@@ -0,0 +1,148 @@
package ai.openclaw.app.gateway
import android.os.Build
import java.net.InetAddress
import java.util.Locale
/** Returns true only for loopback hosts safe to treat as local gateway origins. */
internal fun isLoopbackGatewayHost(
rawHost: String?,
allowEmulatorBridgeAlias: Boolean = isAndroidEmulatorRuntime(),
): Boolean {
var host =
rawHost
?.trim()
?.lowercase(Locale.US)
?.trim('[', ']')
.orEmpty()
if (host.endsWith(".")) {
host = host.dropLast(1)
}
val zoneIndex = host.indexOf('%')
// Scoped IPv6 literals are not stable origin identifiers; reject them for
// loopback trust instead of guessing which interface the zone names.
if (zoneIndex >= 0) return false
if (host.isEmpty()) return false
if (host == "localhost") return true
// Android emulator maps host loopback through this bridge alias.
if (allowEmulatorBridgeAlias && host == "10.0.2.2") return true
parseIpv4Address(host)?.let { ipv4 ->
return ipv4.first() == 127.toByte()
}
if (!host.contains(':') || !host.all(::isIpv6LiteralChar)) return false
val address = runCatching { InetAddress.getByName(host) }.getOrNull()?.address ?: return false
if (address.size == 4) {
return address[0] == 127.toByte()
}
if (address.size != 16) return false
// `::1` is 15 zero bytes followed by `0x01`.
val isIpv6Loopback = address.copyOfRange(0, 15).all { it == 0.toByte() } && address[15] == 1.toByte()
if (isIpv6Loopback) return true
val isMappedIpv4 =
address.copyOfRange(0, 10).all { it == 0.toByte() } &&
address[10] == 0xFF.toByte() &&
address[11] == 0xFF.toByte()
return isMappedIpv4 && address[12] == 127.toByte()
}
/** Allows cleartext only for loopback, `.local`, and private/link-local network ranges. */
internal fun isLocalCleartextGatewayHost(
rawHost: String?,
allowEmulatorBridgeAlias: Boolean = isAndroidEmulatorRuntime(),
): Boolean {
var host =
rawHost
?.trim()
?.lowercase(Locale.US)
?.trim('[', ']')
.orEmpty()
if (host.endsWith(".")) {
host = host.dropLast(1)
}
if (host.isEmpty()) return false
if (isLoopbackGatewayHost(host, allowEmulatorBridgeAlias = allowEmulatorBridgeAlias)) return true
if (isMdnsLocalHostname(host)) return true
val zoneIndex = host.indexOf('%')
if (zoneIndex >= 0) {
// Link-local cleartext policy is about the address range; strip the
// interface zone before InetAddress parsing rejects otherwise valid hosts.
host = host.substring(0, zoneIndex)
}
if (host.isEmpty()) return false
parseIpv4Address(host)?.let { ipv4 ->
val first = ipv4[0].toInt() and 0xff
val second = ipv4[1].toInt() and 0xff
return when {
first == 10 -> true
first == 172 && second in 16..31 -> true
first == 192 && second == 168 -> true
first == 169 && second == 254 -> true
else -> false
}
}
if (!host.contains(':') || !host.all(::isIpv6LiteralChar)) return false
val address = runCatching { InetAddress.getByName(host) }.getOrNull() ?: return false
return when {
address.isLinkLocalAddress -> true
address.isSiteLocalAddress -> true
else -> {
val bytes = address.address
bytes.size == 16 && (bytes[0].toInt() and 0xfe) == 0xfc
}
}
}
private fun isAndroidEmulatorRuntime(): Boolean {
val fingerprint = Build.FINGERPRINT?.lowercase(Locale.US).orEmpty()
val model = Build.MODEL?.lowercase(Locale.US).orEmpty()
val manufacturer = Build.MANUFACTURER?.lowercase(Locale.US).orEmpty()
val brand = Build.BRAND?.lowercase(Locale.US).orEmpty()
val device = Build.DEVICE?.lowercase(Locale.US).orEmpty()
val product = Build.PRODUCT?.lowercase(Locale.US).orEmpty()
return fingerprint.contains("generic") ||
fingerprint.contains("robolectric") ||
model.contains("emulator") ||
model.contains("sdk_gphone") ||
manufacturer.contains("genymotion") ||
(brand.contains("generic") && device.contains("generic")) ||
product.contains("sdk_gphone") ||
product.contains("emulator") ||
product.contains("simulator")
}
/** Parses strict dotted-quad IPv4, rejecting shorthand and out-of-range octets. */
private fun parseIpv4Address(host: String): ByteArray? {
val parts = host.split('.')
if (parts.size != 4) return null
val bytes = ByteArray(4)
for ((index, part) in parts.withIndex()) {
val value = part.toIntOrNull() ?: return null
if (value !in 0..255) return null
bytes[index] = value.toByte()
}
return bytes
}
private fun isMdnsLocalHostname(host: String): Boolean {
if (host.length > 253) return false
if (!host.endsWith(".local")) return false
val labels = host.split('.')
if (labels.size < 2 || labels.last() != "local") return false
return labels.dropLast(1).all(::isDnsHostnameLabel)
}
private fun isDnsHostnameLabel(label: String): Boolean {
if (label.isEmpty() || label.length > 63) return false
if (label.first() == '-' || label.last() == '-') return false
return label.all { it in 'a'..'z' || it in '0'..'9' || it == '-' }
}
/** Cheap prefilter before handing potential IPv6 literals to InetAddress. */
private fun isIpv6LiteralChar(char: Char): Boolean = char in '0'..'9' || char in 'a'..'f' || char == ':' || char == '.'

View File

@@ -0,0 +1,7 @@
package ai.openclaw.app.gateway
/** Gateway protocol version emitted by Android node clients. */
const val GATEWAY_PROTOCOL_VERSION = 4
/** Oldest gateway protocol version this Android client can speak safely. */
const val GATEWAY_MIN_PROTOCOL_VERSION = 4

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,258 @@
package ai.openclaw.app.gateway
import android.annotation.SuppressLint
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import java.io.EOFException
import java.net.ConnectException
import java.net.InetSocketAddress
import java.net.SocketException
import java.net.SocketTimeoutException
import java.net.UnknownHostException
import java.security.MessageDigest
import java.security.SecureRandom
import java.security.cert.CertificateException
import java.security.cert.X509Certificate
import java.util.Locale
import java.util.concurrent.atomic.AtomicReference
import javax.net.ssl.HostnameVerifier
import javax.net.ssl.HttpsURLConnection
import javax.net.ssl.SNIHostName
import javax.net.ssl.SSLContext
import javax.net.ssl.SSLException
import javax.net.ssl.SSLParameters
import javax.net.ssl.SSLSocket
import javax.net.ssl.SSLSocketFactory
import javax.net.ssl.TrustManagerFactory
import javax.net.ssl.X509TrustManager
/** TLS pinning inputs for a discovered or manually configured gateway endpoint. */
data class GatewayTlsParams(
val required: Boolean,
val expectedFingerprint: String?,
val allowTOFU: Boolean,
val stableId: String,
)
/** SSL primitives installed into OkHttp when a gateway needs TLS pinning/TOFU. */
data class GatewayTlsConfig(
val sslSocketFactory: SSLSocketFactory,
val trustManager: X509TrustManager,
val hostnameVerifier: HostnameVerifier,
)
/** Distinguishes non-TLS endpoints from unreachable endpoints during probing. */
enum class GatewayTlsProbeFailure {
TLS_UNAVAILABLE,
TLS_HANDSHAKE_TIMEOUT,
ENDPOINT_UNREACHABLE,
}
/** Result of probing a gateway TLS endpoint for first-use fingerprint capture. */
data class GatewayTlsProbeResult(
val fingerprintSha256: String? = null,
val failure: GatewayTlsProbeFailure? = null,
)
internal const val GATEWAY_TLS_PROBE_CONNECT_TIMEOUT_MS = 3_000
internal const val GATEWAY_TLS_PROBE_HANDSHAKE_TIMEOUT_MS = 10_000
/** Builds a TLS config that supports pinned fingerprints and trust-on-first-use. */
fun buildGatewayTlsConfig(
params: GatewayTlsParams?,
onStore: ((String) -> Unit)? = null,
): GatewayTlsConfig? {
if (params == null) return null
val expected =
params.expectedFingerprint
?.let(::normalizeGatewayTlsFingerprint)
?.takeIf { it.isNotBlank() }
val defaultTrust = defaultTrustManager()
@SuppressLint("CustomX509TrustManager")
val trustManager =
object : X509TrustManager {
override fun checkClientTrusted(
chain: Array<X509Certificate>,
authType: String,
) {
defaultTrust.checkClientTrusted(chain, authType)
}
override fun checkServerTrusted(
chain: Array<X509Certificate>,
authType: String,
) {
if (chain.isEmpty()) throw CertificateException("empty certificate chain")
val fingerprint = sha256Hex(chain[0].encoded)
if (expected != null) {
if (fingerprint != expected) {
throw CertificateException("gateway TLS fingerprint mismatch")
}
return
}
if (params.allowTOFU) {
// Store only after the TLS stack presents a concrete server cert; the
// caller persists the fingerprint against the endpoint's stable id,
// and later connects must come back through the pinned branch above.
onStore?.invoke(fingerprint)
return
}
defaultTrust.checkServerTrusted(chain, authType)
}
override fun getAcceptedIssuers(): Array<X509Certificate> = defaultTrust.acceptedIssuers
}
val context = SSLContext.getInstance("TLS")
context.init(null, arrayOf(trustManager), SecureRandom())
val verifier =
if (expected != null || params.allowTOFU) {
// When pinning, we intentionally ignore hostname mismatch (service discovery often yields IPs).
HostnameVerifier { _, _ -> true }
} else {
HttpsURLConnection.getDefaultHostnameVerifier()
}
return GatewayTlsConfig(
sslSocketFactory = context.socketFactory,
trustManager = trustManager,
hostnameVerifier = verifier,
)
}
/** Connects with a probe trust manager that captures the presented cert hash. */
suspend fun probeGatewayTlsFingerprint(
host: String,
port: Int,
): GatewayTlsProbeResult =
probeGatewayTlsFingerprint(
host = host,
port = port,
connectTimeoutMs = GATEWAY_TLS_PROBE_CONNECT_TIMEOUT_MS,
handshakeTimeoutMs = GATEWAY_TLS_PROBE_HANDSHAKE_TIMEOUT_MS,
)
internal suspend fun probeGatewayTlsFingerprint(
host: String,
port: Int,
connectTimeoutMs: Int,
handshakeTimeoutMs: Int,
): GatewayTlsProbeResult {
val trimmedHost = host.trim()
if (trimmedHost.isEmpty()) return GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE)
if (port !in 1..65535) return GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE)
if (connectTimeoutMs <= 0 || handshakeTimeoutMs <= 0) return GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE)
return withContext(Dispatchers.IO) {
val fingerprintRef = AtomicReference<String?>(null)
val probeTrustManager =
@SuppressLint("CustomX509TrustManager")
object : X509TrustManager {
override fun checkClientTrusted(
chain: Array<X509Certificate>,
authType: String,
): Unit = throw CertificateException("gateway TLS probe does not accept client certificates")
override fun checkServerTrusted(
chain: Array<X509Certificate>,
authType: String,
) {
if (chain.isEmpty()) throw CertificateException("empty certificate chain")
fingerprintRef.set(sha256Hex(chain[0].encoded))
// Abort validation after capture; the probe is not deciding trust.
throw CertificateException("gateway TLS probe captured fingerprint")
}
override fun getAcceptedIssuers(): Array<X509Certificate> = emptyArray()
}
val context = SSLContext.getInstance("TLS")
context.init(null, arrayOf(probeTrustManager), SecureRandom())
val socket = (context.socketFactory.createSocket() as SSLSocket)
var connected = false
try {
// TCP reachability and TLS handshake progress fail differently on mobile
// tailnets; keep the budgets separate so a reachable-but-slow secure
// endpoint does not collapse into generic gateway unreachable guidance.
socket.soTimeout = handshakeTimeoutMs
socket.connect(InetSocketAddress(trimmedHost, port), connectTimeoutMs)
connected = true
// Best-effort SNI for hostnames (avoid crashing on IP literals).
try {
if (trimmedHost.any { it.isLetter() }) {
val params = SSLParameters()
params.serverNames = listOf(SNIHostName(trimmedHost))
socket.sslParameters = params
}
} catch (_: Throwable) {
// SNI is only a probe hint. IP literals and odd Bonjour names should
// still be probed instead of failing before the TLS handshake.
}
socket.startHandshake()
val cert =
socket.session.peerCertificates.firstOrNull() as? X509Certificate
?: return@withContext GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.TLS_UNAVAILABLE)
GatewayTlsProbeResult(fingerprintSha256 = sha256Hex(cert.encoded))
} catch (err: Throwable) {
fingerprintRef.get()?.let { return@withContext GatewayTlsProbeResult(fingerprintSha256 = it) }
val failure =
when (err) {
is SSLException,
is EOFException,
-> GatewayTlsProbeFailure.TLS_UNAVAILABLE
is SocketTimeoutException ->
if (connected) {
GatewayTlsProbeFailure.TLS_HANDSHAKE_TIMEOUT
} else {
GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
}
is ConnectException,
is UnknownHostException,
-> GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
is SocketException ->
if (connected) {
GatewayTlsProbeFailure.TLS_UNAVAILABLE
} else {
GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
}
else -> GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
}
GatewayTlsProbeResult(failure = failure)
} finally {
try {
socket.close()
} catch (_: Throwable) {
// ignore
}
}
}
}
private fun defaultTrustManager(): X509TrustManager {
val factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
factory.init(null as java.security.KeyStore?)
val trust =
factory.trustManagers.firstOrNull { it is X509TrustManager } as? X509TrustManager
return trust ?: throw IllegalStateException("No default X509TrustManager found")
}
private fun sha256Hex(data: ByteArray): String {
val digest = MessageDigest.getInstance("SHA-256").digest(data)
val out = StringBuilder(digest.size * 2)
for (byte in digest) {
out.append(String.format(Locale.US, "%02x", byte))
}
return out.toString()
}
/** Normalizes user-visible fingerprint text to lowercase bare SHA-256 hex. */
fun normalizeGatewayTlsFingerprint(raw: String): String {
val stripped =
raw
.trim()
.replace(Regex("^sha-?256\\s*:?\\s*", RegexOption.IGNORE_CASE), "")
return stripped.lowercase(Locale.US).filter { it in '0'..'9' || it in 'a'..'f' }
}

View File

@@ -0,0 +1,47 @@
package ai.openclaw.app.gateway
private val invokeErrorCodePattern = Regex("^[A-Z][A-Z0-9_]*$")
data class ParsedInvokeError(
val code: String,
val message: String,
val hadExplicitCode: Boolean,
) {
/** Gateway-facing form expected by UI and retry copy. */
val prefixedMessage: String
get() = "$code: $message"
}
/**
* Parses gateway invoke errors encoded as CODE: message while preserving legacy
* plain-text errors as UNAVAILABLE.
*/
fun parseInvokeErrorMessage(raw: String): ParsedInvokeError {
val trimmed = raw.trim()
if (trimmed.isEmpty()) {
return ParsedInvokeError(code = "UNAVAILABLE", message = "error", hadExplicitCode = false)
}
val parts = trimmed.split(":", limit = 2)
if (parts.size == 2) {
val code = parts[0].trim()
val rest = parts[1].trim()
if (invokeErrorCodePattern.matches(code)) {
return ParsedInvokeError(
code = code,
message = rest.ifEmpty { trimmed },
hadExplicitCode = true,
)
}
}
return ParsedInvokeError(code = "UNAVAILABLE", message = trimmed, hadExplicitCode = false)
}
/** Extracts an invoke error from a throwable without exposing blank messages. */
fun parseInvokeErrorFromThrowable(
err: Throwable,
fallbackMessage: String = "error",
): ParsedInvokeError {
val raw = err.message?.trim().takeIf { !it.isNullOrEmpty() } ?: fallbackMessage
return parseInvokeErrorMessage(raw)
}

View File

@@ -0,0 +1,145 @@
package ai.openclaw.app.node
import kotlinx.coroutines.delay
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
/**
* Android bridge for applying gateway A2UI messages to the canvas WebView.
*/
class A2UIHandler(
private val canvas: CanvasController,
private val json: Json,
) {
fun isTrustedCanvasActionUrl(rawUrl: String?): Boolean = CanvasActionTrust.isTrustedCanvasActionUrl(rawUrl)
suspend fun ensureA2uiReady(): Boolean {
if (canvas.currentUrl()?.trim() == CanvasActionTrust.localA2uiAssetUrl && isA2uiReady()) {
return true
}
canvas.showLocalA2ui()
// The bundled A2UI host bootstraps asynchronously after navigation; poll briefly before failing the command.
repeat(50) {
if (isA2uiReady()) return true
delay(120)
}
return false
}
private suspend fun isA2uiReady(): Boolean =
try {
canvas.eval(a2uiReadyCheckJS) == "true"
} catch (_: Throwable) {
false
}
fun decodeA2uiMessages(
command: String,
paramsJson: String?,
): String {
val raw = paramsJson?.trim().orEmpty()
if (raw.isBlank()) throw IllegalArgumentException("INVALID_REQUEST: paramsJSON required")
val obj =
json.parseToJsonElement(raw) as? JsonObject
?: throw IllegalArgumentException("INVALID_REQUEST: expected object params")
val jsonlField = (obj["jsonl"] as? JsonPrimitive)?.content?.trim().orEmpty()
val hasMessagesArray = obj["messages"] is JsonArray
if (command == "canvas.a2ui.pushJSONL" || (!hasMessagesArray && jsonlField.isNotBlank())) {
val jsonl = jsonlField
if (jsonl.isBlank()) throw IllegalArgumentException("INVALID_REQUEST: jsonl required")
// JSONL keeps large A2UI streams model-friendly while still validating each message.
val messages =
jsonl
.lineSequence()
.map { it.trim() }
.filter { it.isNotBlank() }
.mapIndexed { idx, line ->
val el = json.parseToJsonElement(line)
val msg =
el as? JsonObject
?: throw IllegalArgumentException("A2UI JSONL line ${idx + 1}: expected a JSON object")
validateA2uiV0_8(msg, idx + 1)
msg
}.toList()
return JsonArray(messages).toString()
}
val arr = obj["messages"] as? JsonArray ?: throw IllegalArgumentException("INVALID_REQUEST: messages[] required")
val out =
arr.mapIndexed { idx, el ->
val msg =
el as? JsonObject
?: throw IllegalArgumentException("A2UI messages[$idx]: expected a JSON object")
validateA2uiV0_8(msg, idx + 1)
msg
}
return JsonArray(out).toString()
}
private fun validateA2uiV0_8(
msg: JsonObject,
lineNumber: Int,
) {
if (msg.containsKey("createSurface")) {
// Android scaffold currently implements A2UI v0.8, not the v0.9 createSurface shape.
throw IllegalArgumentException(
"A2UI JSONL line $lineNumber: looks like A2UI v0.9 (`createSurface`). Canvas supports v0.8 messages only.",
)
}
val allowed = setOf("beginRendering", "surfaceUpdate", "dataModelUpdate", "deleteSurface")
val matched = msg.keys.filter { allowed.contains(it) }
if (matched.size != 1) {
val found = msg.keys.sorted().joinToString(", ")
throw IllegalArgumentException(
"A2UI JSONL line $lineNumber: expected exactly one of ${allowed.sorted().joinToString(", ")}; found: $found",
)
}
}
companion object {
const val a2uiReadyCheckJS: String =
"""
(() => {
try {
const host = globalThis.openclawA2UI;
return !!host && typeof host.applyMessages === 'function';
} catch (_) {
return false;
}
})()
"""
const val a2uiResetJS: String =
"""
(() => {
try {
const host = globalThis.openclawA2UI;
if (!host) return { ok: false, error: "missing openclawA2UI" };
return host.reset();
} catch (e) {
return { ok: false, error: String(e?.message ?? e) };
}
})()
"""
fun a2uiApplyMessagesJS(messagesJson: String): String =
"""
(() => {
try {
const host = globalThis.openclawA2UI;
if (!host) return { ok: false, error: "missing openclawA2UI" };
const messages = $messagesJson;
return host.applyMessages(messages);
} catch (e) {
return { ok: false, error: String(e?.message ?? e) };
}
})()
""".trimIndent()
}
}

View File

@@ -0,0 +1,436 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.ContentResolver
import android.content.ContentUris
import android.content.ContentValues
import android.content.Context
import android.provider.CalendarContract
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.time.Instant
import java.time.temporal.ChronoUnit
import java.util.TimeZone
private const val DEFAULT_CALENDAR_LIMIT = 50
/**
* Parsed calendar.events request; times are epoch millis for CalendarContract queries.
*/
internal data class CalendarEventsRequest(
val startMs: Long,
val endMs: Long,
val limit: Int,
)
/**
* Parsed calendar.add request before resolving the target Android calendar.
*/
internal data class CalendarAddRequest(
val title: String,
val startMs: Long,
val endMs: Long,
val isAllDay: Boolean,
val location: String?,
val notes: String?,
val calendarId: Long?,
val calendarTitle: String?,
)
/**
* Normalized calendar event returned through gateway calendar commands.
*/
internal data class CalendarEventRecord(
val identifier: String,
val title: String,
val startISO: String,
val endISO: String,
val isAllDay: Boolean,
val location: String?,
val calendarTitle: String?,
)
/**
* Injectable CalendarProvider facade for command tests and Android runtime access.
*/
internal interface CalendarDataSource {
fun hasReadPermission(context: Context): Boolean
fun hasWritePermission(context: Context): Boolean
fun events(
context: Context,
request: CalendarEventsRequest,
): List<CalendarEventRecord>
fun add(
context: Context,
request: CalendarAddRequest,
): CalendarEventRecord
}
private object SystemCalendarDataSource : CalendarDataSource {
override fun hasReadPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CALENDAR) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun hasWritePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CALENDAR) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun events(
context: Context,
request: CalendarEventsRequest,
): List<CalendarEventRecord> {
val resolver = context.contentResolver
val builder = CalendarContract.Instances.CONTENT_URI.buildUpon()
// Instances expands recurring events inside the requested time window.
ContentUris.appendId(builder, request.startMs)
ContentUris.appendId(builder, request.endMs)
val projection =
arrayOf(
CalendarContract.Instances.EVENT_ID,
CalendarContract.Instances.TITLE,
CalendarContract.Instances.BEGIN,
CalendarContract.Instances.END,
CalendarContract.Instances.ALL_DAY,
CalendarContract.Instances.EVENT_LOCATION,
CalendarContract.Instances.CALENDAR_DISPLAY_NAME,
)
val sortOrder = "${CalendarContract.Instances.BEGIN} ASC LIMIT ${request.limit}"
resolver.query(builder.build(), projection, null, null, sortOrder).use { cursor ->
if (cursor == null) return emptyList()
val out = mutableListOf<CalendarEventRecord>()
while (cursor.moveToNext() && out.size < request.limit) {
val id = cursor.getLong(0)
val title =
cursor
.getString(1)
?.trim()
.orEmpty()
.ifEmpty { "(untitled)" }
val beginMs = cursor.getLong(2)
val endMs = cursor.getLong(3)
val isAllDay = cursor.getInt(4) == 1
val location = cursor.getString(5)?.trim()?.ifEmpty { null }
val calendarTitle = cursor.getString(6)?.trim()?.ifEmpty { null }
out +=
CalendarEventRecord(
identifier = id.toString(),
title = title,
startISO = Instant.ofEpochMilli(beginMs).toString(),
endISO = Instant.ofEpochMilli(endMs).toString(),
isAllDay = isAllDay,
location = location,
calendarTitle = calendarTitle,
)
}
return out
}
}
override fun add(
context: Context,
request: CalendarAddRequest,
): CalendarEventRecord {
val resolver = context.contentResolver
val resolvedCalendarId = resolveCalendarId(resolver, request.calendarId, request.calendarTitle)
val values =
ContentValues().apply {
put(CalendarContract.Events.CALENDAR_ID, resolvedCalendarId)
put(CalendarContract.Events.TITLE, request.title)
put(CalendarContract.Events.DTSTART, request.startMs)
put(CalendarContract.Events.DTEND, request.endMs)
put(CalendarContract.Events.ALL_DAY, if (request.isAllDay) 1 else 0)
put(CalendarContract.Events.EVENT_TIMEZONE, TimeZone.getDefault().id)
request.location?.let { put(CalendarContract.Events.EVENT_LOCATION, it) }
request.notes?.let { put(CalendarContract.Events.DESCRIPTION, it) }
}
val uri =
resolver.insert(CalendarContract.Events.CONTENT_URI, values)
?: throw IllegalStateException("calendar insert failed")
val eventId =
uri.lastPathSegment?.toLongOrNull()
?: throw IllegalStateException("calendar insert failed")
return loadEventById(resolver, eventId)
?: throw IllegalStateException("calendar insert failed")
}
private fun resolveCalendarId(
resolver: ContentResolver,
calendarId: Long?,
calendarTitle: String?,
): Long {
if (calendarId != null) {
// Explicit id wins over title/default selection and must already exist.
if (calendarExists(resolver, calendarId)) return calendarId
throw IllegalArgumentException("CALENDAR_NOT_FOUND: no calendar id $calendarId")
}
if (!calendarTitle.isNullOrEmpty()) {
// Title lookup is exact to avoid adding events to a similarly named calendar.
findCalendarByTitle(resolver, calendarTitle)?.let { return it }
throw IllegalArgumentException("CALENDAR_NOT_FOUND: no calendar named $calendarTitle")
}
findDefaultCalendarId(resolver)?.let { return it }
throw IllegalArgumentException("CALENDAR_NOT_FOUND: no default calendar")
}
private fun calendarExists(
resolver: ContentResolver,
id: Long,
): Boolean {
val projection = arrayOf(CalendarContract.Calendars._ID)
resolver
.query(
CalendarContract.Calendars.CONTENT_URI,
projection,
"${CalendarContract.Calendars._ID}=?",
arrayOf(id.toString()),
null,
).use { cursor ->
return cursor != null && cursor.moveToFirst()
}
}
private fun findCalendarByTitle(
resolver: ContentResolver,
title: String,
): Long? {
val projection = arrayOf(CalendarContract.Calendars._ID)
resolver
.query(
CalendarContract.Calendars.CONTENT_URI,
projection,
"${CalendarContract.Calendars.CALENDAR_DISPLAY_NAME}=?",
arrayOf(title),
"${CalendarContract.Calendars.IS_PRIMARY} DESC",
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return cursor.getLong(0)
}
}
private fun findDefaultCalendarId(resolver: ContentResolver): Long? {
val projection = arrayOf(CalendarContract.Calendars._ID)
resolver
.query(
CalendarContract.Calendars.CONTENT_URI,
projection,
"${CalendarContract.Calendars.VISIBLE}=1",
null,
// Prefer Android's primary visible calendar, then lowest id for deterministic fallback.
"${CalendarContract.Calendars.IS_PRIMARY} DESC, ${CalendarContract.Calendars._ID} ASC",
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return cursor.getLong(0)
}
}
private fun loadEventById(
resolver: ContentResolver,
eventId: Long,
): CalendarEventRecord? {
val projection =
arrayOf(
CalendarContract.Events._ID,
CalendarContract.Events.TITLE,
CalendarContract.Events.DTSTART,
CalendarContract.Events.DTEND,
CalendarContract.Events.ALL_DAY,
CalendarContract.Events.EVENT_LOCATION,
CalendarContract.Events.CALENDAR_DISPLAY_NAME,
)
resolver
.query(
CalendarContract.Events.CONTENT_URI,
projection,
"${CalendarContract.Events._ID}=?",
arrayOf(eventId.toString()),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return CalendarEventRecord(
identifier = cursor.getLong(0).toString(),
title =
cursor
.getString(1)
?.trim()
.orEmpty()
.ifEmpty { "(untitled)" },
startISO = Instant.ofEpochMilli(cursor.getLong(2)).toString(),
endISO = Instant.ofEpochMilli(cursor.getLong(3)).toString(),
isAllDay = cursor.getInt(4) == 1,
location = cursor.getString(5)?.trim()?.ifEmpty { null },
calendarTitle = cursor.getString(6)?.trim()?.ifEmpty { null },
)
}
}
}
class CalendarHandler private constructor(
private val appContext: Context,
private val dataSource: CalendarDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemCalendarDataSource)
fun handleCalendarEvents(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasReadPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_PERMISSION_REQUIRED",
message = "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission",
)
}
val request =
parseEventsRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val events = dataSource.events(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put(
"events",
buildJsonArray { events.forEach { add(eventJson(it)) } },
)
}.toString(),
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CALENDAR_UNAVAILABLE",
message = "CALENDAR_UNAVAILABLE: ${err.message ?: "calendar query failed"}",
)
}
}
fun handleCalendarAdd(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasWritePermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_PERMISSION_REQUIRED",
message = "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission",
)
}
val request =
parseAddRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
if (request.title.isEmpty()) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_INVALID",
message = "CALENDAR_INVALID: title required",
)
}
if (request.endMs <= request.startMs) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_INVALID",
message = "CALENDAR_INVALID: endISO must be after startISO",
)
}
return try {
val event = dataSource.add(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put("event", eventJson(event))
}.toString(),
)
} catch (err: IllegalArgumentException) {
val msg = err.message ?: "CALENDAR_INVALID: invalid request"
val code = if (msg.startsWith("CALENDAR_NOT_FOUND")) "CALENDAR_NOT_FOUND" else "CALENDAR_INVALID"
GatewaySession.InvokeResult.error(code = code, message = msg)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CALENDAR_UNAVAILABLE",
message = "CALENDAR_UNAVAILABLE: ${err.message ?: "calendar add failed"}",
)
}
}
private fun parseEventsRequest(paramsJson: String?): CalendarEventsRequest? {
if (paramsJson.isNullOrBlank()) {
val start = Instant.now()
val end = start.plus(7, ChronoUnit.DAYS)
// Default calendar read is a one-week window, not the full calendar store.
return CalendarEventsRequest(startMs = start.toEpochMilli(), endMs = end.toEpochMilli(), limit = DEFAULT_CALENDAR_LIMIT)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val start = parseISO((params["startISO"] as? JsonPrimitive)?.content)
val end = parseISO((params["endISO"] as? JsonPrimitive)?.content)
val resolvedStart = start ?: Instant.now()
val resolvedEnd = end ?: resolvedStart.plus(7, ChronoUnit.DAYS)
// Keep model-driven calendar reads bounded.
val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: DEFAULT_CALENDAR_LIMIT).coerceIn(1, 500)
return CalendarEventsRequest(
startMs = resolvedStart.toEpochMilli(),
endMs = resolvedEnd.toEpochMilli(),
limit = limit,
)
}
private fun parseAddRequest(paramsJson: String?): CalendarAddRequest? {
val params =
try {
paramsJson?.let { Json.parseToJsonElement(it).asObjectOrNull() }
} catch (_: Throwable) {
null
} ?: return null
val start =
parseISO((params["startISO"] as? JsonPrimitive)?.content)
?: return null
val end =
parseISO((params["endISO"] as? JsonPrimitive)?.content)
?: return null
return CalendarAddRequest(
title = (params["title"] as? JsonPrimitive)?.content?.trim().orEmpty(),
startMs = start.toEpochMilli(),
endMs = end.toEpochMilli(),
isAllDay = (params["isAllDay"] as? JsonPrimitive)?.content?.toBooleanStrictOrNull() ?: false,
location = (params["location"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
notes = (params["notes"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
calendarId = (params["calendarId"] as? JsonPrimitive)?.content?.toLongOrNull(),
calendarTitle = (params["calendarTitle"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
)
}
private fun parseISO(raw: String?): Instant? {
val value = raw?.trim().orEmpty()
if (value.isEmpty()) return null
// Gateway calendar payloads use UTC ISO-8601 instants for unambiguous Android storage.
return try {
Instant.parse(value)
} catch (_: Throwable) {
null
}
}
private fun eventJson(event: CalendarEventRecord): JsonObject =
buildJsonObject {
put("identifier", JsonPrimitive(event.identifier))
put("title", JsonPrimitive(event.title))
put("startISO", JsonPrimitive(event.startISO))
put("endISO", JsonPrimitive(event.endISO))
put("isAllDay", JsonPrimitive(event.isAllDay))
event.location?.let { put("location", JsonPrimitive(it)) }
event.calendarTitle?.let { put("calendarTitle", JsonPrimitive(it)) }
}
companion object {
internal fun forTesting(
appContext: Context,
dataSource: CalendarDataSource,
): CalendarHandler = CalendarHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,531 @@
package ai.openclaw.app.node
import ai.openclaw.app.PermissionRequester
import android.Manifest
import android.annotation.SuppressLint
import android.content.Context
import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.Matrix
import android.hardware.camera2.CameraCharacteristics
import android.util.Base64
import androidx.camera.camera2.interop.Camera2CameraInfo
import androidx.camera.core.CameraInfo
import androidx.camera.core.CameraSelector
import androidx.camera.core.ImageCapture
import androidx.camera.core.ImageCaptureException
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.camera.video.FallbackStrategy
import androidx.camera.video.FileOutputOptions
import androidx.camera.video.Quality
import androidx.camera.video.QualitySelector
import androidx.camera.video.Recorder
import androidx.camera.video.VideoCapture
import androidx.camera.video.VideoRecordEvent
import androidx.core.content.ContextCompat
import androidx.core.content.ContextCompat.checkSelfPermission
import androidx.core.graphics.scale
import androidx.exifinterface.media.ExifInterface
import androidx.lifecycle.LifecycleOwner
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import kotlinx.serialization.json.JsonObject
import java.io.ByteArrayOutputStream
import java.io.File
import java.util.concurrent.Executor
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
import kotlin.math.roundToInt
/**
* CameraX-backed capture service used by gateway camera commands.
*/
internal class CameraClipSession(
private val unbind: () -> Unit,
private val deleteTemporaryFile: (File) -> Unit,
) : AutoCloseable {
private var recording: AutoCloseable? = null
private var temporaryFile: File? = null
private var closed = false
fun ownRecording(recording: AutoCloseable) {
check(!closed) { "camera clip session is closed" }
this.recording = recording
}
fun ownFile(file: File): File {
check(!closed) { "camera clip session is closed" }
check(temporaryFile == null) { "camera clip session already owns a file" }
temporaryFile = file
return file
}
fun transferFile(): File {
check(!closed) { "camera clip session is closed" }
return checkNotNull(temporaryFile) { "camera clip session has no file" }
.also { temporaryFile = null }
}
override fun close() {
if (closed) return
closed = true
var failure: Throwable? = null
fun cleanup(action: () -> Unit) {
try {
action()
} catch (err: Throwable) {
failure?.addSuppressed(err) ?: run { failure = err }
}
}
// Keep teardown symmetric across bind, warmup, recording, finalize, and success exits.
cleanup { recording?.close() }
cleanup(unbind)
temporaryFile?.let { file -> cleanup { deleteTemporaryFile(file) } }
failure?.let { throw it }
}
}
class CameraCaptureManager(
private val context: Context,
) {
/** Base64 JSON response for camera.snap after resize and JPEG budget enforcement. */
data class Payload(
val payloadJson: String,
)
/** Temporary MP4 response for camera.clip before CameraHandler validates invoke size. */
data class FilePayload(
val file: File,
val durationMs: Long,
val hasAudio: Boolean,
)
/** Camera device metadata exposed through camera.list. */
data class CameraDeviceInfo(
val id: String,
val name: String,
val position: String,
val deviceType: String,
)
@Volatile private var lifecycleOwner: LifecycleOwner? = null
@Volatile private var permissionRequester: PermissionRequester? = null
/** Supplies the foreground Activity lifecycle required by CameraX use-case binding. */
fun attachLifecycleOwner(owner: LifecycleOwner) {
// CameraX binds use cases to an Activity lifecycle; background services cannot capture alone.
lifecycleOwner = owner
}
/** Supplies the Activity-owned permission launcher used by camera and microphone commands. */
fun attachPermissionRequester(requester: PermissionRequester) {
// Permission prompts must be launched by the Activity that owns the ActivityResult registry.
permissionRequester = requester
}
/** Lists CameraX devices with stable Camera2 ids where available. */
suspend fun listDevices(): List<CameraDeviceInfo> =
withContext(Dispatchers.Main) {
val provider = context.cameraProvider()
provider.availableCameraInfos
.mapNotNull { info -> cameraDeviceInfoOrNull(info) }
.sortedBy { it.id }
}
private suspend fun ensureCameraPermission() {
val granted = checkSelfPermission(context, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
if (granted) return
val requester =
permissionRequester
?: throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission")
val results = requester.requestIfMissing(listOf(Manifest.permission.CAMERA))
if (results[Manifest.permission.CAMERA] != true) {
throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission")
}
}
private suspend fun ensureMicPermission() {
val granted = checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED
if (granted) return
val requester =
permissionRequester
?: throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
val results = requester.requestIfMissing(listOf(Manifest.permission.RECORD_AUDIO))
if (results[Manifest.permission.RECORD_AUDIO] != true) {
throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
}
}
/** Captures one still image and returns a gateway-sized JPEG payload. */
suspend fun snap(paramsJson: String?): Payload =
withContext(Dispatchers.Main) {
ensureCameraPermission()
val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready")
val params = parseJsonParamsObject(paramsJson)
val facing = parseFacing(params) ?: "front"
val quality = (parseQuality(params) ?: 0.95).coerceIn(0.1, 1.0)
val maxWidth = parseMaxWidth(params) ?: 1600
val deviceId = parseDeviceId(params)
val provider = context.cameraProvider()
val capture = ImageCapture.Builder().build()
val selector = resolveCameraSelector(provider, facing, deviceId)
provider.unbindAll()
// Bind only the still capture use case; CameraX owns camera open/close through the lifecycle owner.
provider.bindToLifecycle(owner, selector, capture)
val (bytes, orientation) =
try {
capture.takeJpegWithExif(context.mainExecutor(), context.cacheDir)
} finally {
// The JPEG bytes are self-contained; release CameraX before decoding and recompressing them.
provider.unbind(capture)
}
val decoded =
BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
?: throw IllegalStateException("UNAVAILABLE: failed to decode captured image")
val rotated = rotateBitmapByExif(decoded, orientation)
val scaled =
if (maxWidth > 0 && rotated.width > maxWidth) {
val h =
(rotated.height.toDouble() * (maxWidth.toDouble() / rotated.width.toDouble()))
.toInt()
.coerceAtLeast(1)
val s = rotated.scale(maxWidth, h)
if (s !== rotated) rotated.recycle()
s
} else {
rotated
}
try {
val maxPayloadBytes = 5 * 1024 * 1024
// Base64 inflates payloads by ~4/3; cap encoded bytes so the payload stays under 5MB (API limit).
val maxEncodedBytes = (maxPayloadBytes / 4) * 3
val result =
JpegSizeLimiter.compressToLimit(
initialWidth = scaled.width,
initialHeight = scaled.height,
startQuality = (quality * 100.0).roundToInt().coerceIn(10, 100),
maxBytes = maxEncodedBytes,
encode = { width, height, q ->
val bitmap =
if (width == scaled.width && height == scaled.height) {
scaled
} else {
scaled.scale(width, height)
}
val out = ByteArrayOutputStream()
if (!bitmap.compress(Bitmap.CompressFormat.JPEG, q, out)) {
if (bitmap !== scaled) bitmap.recycle()
throw IllegalStateException("UNAVAILABLE: failed to encode JPEG")
}
if (bitmap !== scaled) {
bitmap.recycle()
}
out.toByteArray()
},
)
val base64 = Base64.encodeToString(result.bytes, Base64.NO_WRAP)
Payload(
"""{"format":"jpg","base64":"$base64","width":${result.width},"height":${result.height}}""",
)
} finally {
scaled.recycle()
}
}
/** Records a short MP4 clip into a temporary cache file for the caller to encode/delete. */
@SuppressLint("MissingPermission")
suspend fun clip(paramsJson: String?): FilePayload =
withContext(Dispatchers.Main) {
ensureCameraPermission()
val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready")
val params = parseJsonParamsObject(paramsJson)
val facing = parseFacing(params) ?: "front"
val durationMs = (parseDurationMs(params) ?: 3_000).coerceIn(200, 60_000)
val includeAudio = parseIncludeAudio(params) ?: true
val deviceId = parseDeviceId(params)
if (includeAudio) ensureMicPermission()
android.util.Log.w(
"CameraCaptureManager",
"clip: start facing=$facing duration=$durationMs audio=$includeAudio deviceId=${deviceId ?: "-"}",
)
val provider = context.cameraProvider()
android.util.Log.w("CameraCaptureManager", "clip: got camera provider")
// Use LOWEST quality for smallest files over WebSocket
val recorder =
Recorder
.Builder()
.setQualitySelector(
QualitySelector.from(Quality.LOWEST, FallbackStrategy.lowerQualityOrHigherThan(Quality.LOWEST)),
).build()
val videoCapture = VideoCapture.withOutput(recorder)
val selector = resolveCameraSelector(provider, facing, deviceId)
// CameraX requires a Preview use case for the camera to start producing frames;
// without it, the encoder may get no data (ERROR_NO_VALID_DATA).
val preview =
androidx.camera.core.Preview
.Builder()
.build()
// Allocate the dummy preview surface only after CameraX requests it; its result owns release.
preview.setSurfaceProvider { request ->
val surfaceTexture = android.graphics.SurfaceTexture(0)
surfaceTexture.setDefaultBufferSize(640, 480)
val surface = android.view.Surface(surfaceTexture)
request.provideSurface(surface, context.mainExecutor()) {
surface.release()
surfaceTexture.release()
}
}
provider.unbindAll()
CameraClipSession(
unbind = { provider.unbind(preview, videoCapture) },
deleteTemporaryFile = { file ->
check(!file.exists() || file.delete()) { "failed to delete temporary camera clip" }
},
).use { session ->
android.util.Log.w("CameraCaptureManager", "clip: binding preview + videoCapture to lifecycle")
val camera = provider.bindToLifecycle(owner, selector, preview, videoCapture)
android.util.Log.w("CameraCaptureManager", "clip: bound, cameraInfo=${camera.cameraInfo}")
// Give camera pipeline time to initialize before recording
android.util.Log.w("CameraCaptureManager", "clip: warming up camera 1.5s...")
kotlinx.coroutines.delay(1_500)
val clipFile = session.ownFile(File.createTempFile("openclaw-clip-", ".mp4", context.cacheDir))
val outputOptions = FileOutputOptions.Builder(clipFile).build()
val finalized = kotlinx.coroutines.CompletableDeferred<VideoRecordEvent.Finalize>()
android.util.Log.w("CameraCaptureManager", "clip: starting recording to ${clipFile.absolutePath}")
val recording =
videoCapture.output
.prepareRecording(context, outputOptions)
.apply {
if (includeAudio) withAudioEnabled()
}.start(context.mainExecutor()) { event ->
android.util.Log.w("CameraCaptureManager", "clip: event ${event.javaClass.simpleName}")
if (event is VideoRecordEvent.Status) {
android.util.Log.w("CameraCaptureManager", "clip: recording status update")
}
if (event is VideoRecordEvent.Finalize) {
android.util.Log.w(
"CameraCaptureManager",
"clip: finalize hasError=${event.hasError()} error=${event.error} cause=${event.cause}",
)
finalized.complete(event)
}
}
session.ownRecording(recording)
android.util.Log.w("CameraCaptureManager", "clip: recording started, delaying ${durationMs}ms")
kotlinx.coroutines.delay(durationMs.toLong())
android.util.Log.w("CameraCaptureManager", "clip: stopping recording")
recording.close()
val finalizeEvent =
try {
withTimeout(15_000) { finalized.await() }
} catch (err: kotlinx.coroutines.TimeoutCancellationException) {
android.util.Log.e("CameraCaptureManager", "clip: finalize timed out", err)
throw IllegalStateException("UNAVAILABLE: camera clip finalize timed out")
}
if (finalizeEvent.hasError()) {
android.util.Log.e(
"CameraCaptureManager",
"clip: FAILED error=${finalizeEvent.error}, cause=${finalizeEvent.cause}",
finalizeEvent.cause,
)
// Check file size for debugging
val fileSize = withContext(Dispatchers.IO) { if (clipFile.exists()) clipFile.length() else -1 }
android.util.Log.e("CameraCaptureManager", "clip: file exists=${clipFile.exists()} size=$fileSize")
throw IllegalStateException("UNAVAILABLE: camera clip failed (error=${finalizeEvent.error})")
}
val fileSize = withContext(Dispatchers.IO) { clipFile.length() }
android.util.Log.w("CameraCaptureManager", "clip: SUCCESS file size=$fileSize")
FilePayload(
file = session.transferFile(),
durationMs = durationMs.toLong(),
hasAudio = includeAudio,
)
}
}
private fun rotateBitmapByExif(
bitmap: Bitmap,
orientation: Int,
): Bitmap {
val matrix = Matrix()
// CameraX JPEG bytes keep sensor orientation in EXIF; normalize before resizing/encoding.
when (orientation) {
ExifInterface.ORIENTATION_ROTATE_90 -> matrix.postRotate(90f)
ExifInterface.ORIENTATION_ROTATE_180 -> matrix.postRotate(180f)
ExifInterface.ORIENTATION_ROTATE_270 -> matrix.postRotate(270f)
ExifInterface.ORIENTATION_FLIP_HORIZONTAL -> matrix.postScale(-1f, 1f)
ExifInterface.ORIENTATION_FLIP_VERTICAL -> matrix.postScale(1f, -1f)
ExifInterface.ORIENTATION_TRANSPOSE -> {
matrix.postRotate(90f)
matrix.postScale(-1f, 1f)
}
ExifInterface.ORIENTATION_TRANSVERSE -> {
matrix.postRotate(-90f)
matrix.postScale(-1f, 1f)
}
else -> return bitmap
}
val rotated = Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
if (rotated !== bitmap) {
bitmap.recycle()
}
return rotated
}
private fun parseFacing(params: JsonObject?): String? {
val value = parseJsonString(params, "facing")?.trim()?.lowercase() ?: return null
return when (value) {
"front", "back" -> value
else -> null
}
}
private fun parseQuality(params: JsonObject?): Double? = parseJsonDouble(params, "quality")
private fun parseMaxWidth(params: JsonObject?): Int? =
parseJsonInt(params, "maxWidth")
?.takeIf { it > 0 }
private fun parseDurationMs(params: JsonObject?): Int? = parseJsonInt(params, "durationMs")
private fun parseDeviceId(params: JsonObject?): String? =
parseJsonString(params, "deviceId")
?.trim()
?.takeIf { it.isNotEmpty() }
private fun parseIncludeAudio(params: JsonObject?): Boolean? = parseJsonBooleanFlag(params, "includeAudio")
private fun Context.mainExecutor(): Executor = ContextCompat.getMainExecutor(this)
private fun resolveCameraSelector(
provider: ProcessCameraProvider,
facing: String,
deviceId: String?,
): CameraSelector {
if (deviceId.isNullOrEmpty()) {
return if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA
}
val availableIds = provider.availableCameraInfos.mapNotNull { cameraIdOrNull(it) }.toSet()
if (!availableIds.contains(deviceId)) {
throw IllegalStateException("INVALID_REQUEST: unknown camera deviceId '$deviceId'")
}
return CameraSelector
.Builder()
// CameraX selectors are filters over CameraInfo; pin by Camera2 id for stable device selection.
.addCameraFilter { infos -> infos.filter { cameraIdOrNull(it) == deviceId } }
.build()
}
@SuppressLint("UnsafeOptInUsageError")
private fun cameraDeviceInfoOrNull(info: CameraInfo): CameraDeviceInfo? {
val cameraId = cameraIdOrNull(info) ?: return null
val lensFacing =
runCatching {
Camera2CameraInfo.from(info).getCameraCharacteristic(CameraCharacteristics.LENS_FACING)
}.getOrNull()
val position =
when (lensFacing) {
CameraCharacteristics.LENS_FACING_FRONT -> "front"
CameraCharacteristics.LENS_FACING_BACK -> "back"
CameraCharacteristics.LENS_FACING_EXTERNAL -> "external"
else -> "unspecified"
}
val deviceType =
if (lensFacing == CameraCharacteristics.LENS_FACING_EXTERNAL) "external" else "builtIn"
val name =
when (position) {
"front" -> "Front Camera"
"back" -> "Back Camera"
"external" -> "External Camera"
else -> "Camera $cameraId"
}
return CameraDeviceInfo(
id = cameraId,
name = name,
position = position,
deviceType = deviceType,
)
}
@SuppressLint("UnsafeOptInUsageError")
private fun cameraIdOrNull(info: CameraInfo): String? = runCatching { Camera2CameraInfo.from(info).cameraId }.getOrNull()
}
private suspend fun Context.cameraProvider(): ProcessCameraProvider =
suspendCancellableCoroutine { cont ->
val future = ProcessCameraProvider.getInstance(this)
future.addListener(
{
try {
cont.resume(future.get())
} catch (e: Exception) {
cont.resumeWithException(e)
}
},
ContextCompat.getMainExecutor(this),
)
}
/**
* Returns JPEG bytes plus EXIF orientation so callers can normalize the decoded bitmap.
*/
private suspend fun ImageCapture.takeJpegWithExif(
executor: Executor,
tempDir: File,
): Pair<ByteArray, Int> =
suspendCancellableCoroutine { cont ->
val file = File.createTempFile("openclaw-snap-", ".jpg", tempDir)
val options = ImageCapture.OutputFileOptions.Builder(file).build()
takePicture(
options,
executor,
object : ImageCapture.OnImageSavedCallback {
override fun onError(exception: ImageCaptureException) {
file.delete()
cont.resumeWithException(exception)
}
override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {
try {
val exif = ExifInterface(file.absolutePath)
val orientation =
exif.getAttributeInt(
ExifInterface.TAG_ORIENTATION,
ExifInterface.ORIENTATION_NORMAL,
)
val bytes = file.readBytes()
cont.resume(Pair(bytes, orientation))
} catch (e: Exception) {
cont.resumeWithException(e)
} finally {
file.delete()
}
}
},
)
}

View File

@@ -0,0 +1,189 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.CameraHudKind
import ai.openclaw.app.gateway.GatewaySession
import android.content.Context
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.put
internal const val CAMERA_CLIP_MAX_RAW_BYTES: Long = 18L * 1024L * 1024L
/**
* Raw MP4 size guard before base64 encoding the clip into a node.invoke response.
*/
internal fun isCameraClipWithinPayloadLimit(rawBytes: Long): Boolean = rawBytes in 0L..CAMERA_CLIP_MAX_RAW_BYTES
/**
* Gateway camera command adapter that adds HUD feedback and payload-size enforcement.
*/
class CameraHandler(
private val appContext: Context,
private val camera: CameraCaptureManager,
private val externalAudioCaptureActive: MutableStateFlow<Boolean>,
private val showCameraHud: (message: String, kind: CameraHudKind, autoHideMs: Long?) -> Unit,
private val triggerCameraFlash: () -> Unit,
private val invokeErrorFromThrowable: (err: Throwable) -> Pair<String, String>,
) {
/** Handles camera.list by exposing CameraX devices through gateway metadata. */
suspend fun handleList(_paramsJson: String?): GatewaySession.InvokeResult =
try {
val devices = camera.listDevices()
val payload =
buildJsonObject {
put(
"devices",
buildJsonArray {
devices.forEach { device ->
add(
buildJsonObject {
put("id", JsonPrimitive(device.id))
put("name", JsonPrimitive(device.name))
put("position", JsonPrimitive(device.position))
put("deviceType", JsonPrimitive(device.deviceType))
},
)
}
},
)
}.toString()
GatewaySession.InvokeResult.ok(payload)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
GatewaySession.InvokeResult.error(code = code, message = message)
}
/** Handles camera.snap with HUD progress, flash feedback, and normalized invoke errors. */
suspend fun handleSnap(paramsJson: String?): GatewaySession.InvokeResult {
val logFile = if (BuildConfig.DEBUG) java.io.File(appContext.cacheDir, "camera_debug.log") else null
fun camLog(msg: String) {
if (!BuildConfig.DEBUG) return
val ts = java.text.SimpleDateFormat("HH:mm:ss.SSS", java.util.Locale.US).format(java.util.Date())
logFile?.appendText("[$ts] $msg\n")
android.util.Log.w("openclaw", "camera.snap: $msg")
}
try {
logFile?.writeText("") // clear
camLog("starting, params=$paramsJson")
camLog("calling showCameraHud")
showCameraHud("Taking photo…", CameraHudKind.Photo, null)
camLog("calling triggerCameraFlash")
triggerCameraFlash()
val res =
try {
camLog("calling camera.snap()")
val r = camera.snap(paramsJson)
camLog("success, payload size=${r.payloadJson.length}")
r
} catch (err: Throwable) {
camLog("inner error: ${err::class.java.simpleName}: ${err.message}")
camLog("stack: ${err.stackTraceToString().take(2000)}")
val (code, message) = invokeErrorFromThrowable(err)
showCameraHud(message, CameraHudKind.Error, 2200)
return GatewaySession.InvokeResult.error(code = code, message = message)
}
camLog("returning result")
showCameraHud("Photo captured", CameraHudKind.Success, 1600)
return GatewaySession.InvokeResult.ok(res.payloadJson)
} catch (err: Throwable) {
camLog("outer error: ${err::class.java.simpleName}: ${err.message}")
camLog("stack: ${err.stackTraceToString().take(2000)}")
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = err.message ?: "camera snap failed")
}
}
/** Handles camera.clip and keeps external audio capture paused while camera audio is active. */
suspend fun handleClip(paramsJson: String?): GatewaySession.InvokeResult {
val clipLogFile = if (BuildConfig.DEBUG) java.io.File(appContext.cacheDir, "camera_debug.log") else null
fun clipLog(msg: String) {
if (!BuildConfig.DEBUG) return
val ts = java.text.SimpleDateFormat("HH:mm:ss.SSS", java.util.Locale.US).format(java.util.Date())
clipLogFile?.appendText("[CLIP $ts] $msg\n")
android.util.Log.w("openclaw", "camera.clip: $msg")
}
val includeAudio = parseIncludeAudio(paramsJson) ?: true
if (includeAudio) externalAudioCaptureActive.value = true
try {
clipLogFile?.writeText("") // clear
clipLog("starting, params=$paramsJson includeAudio=$includeAudio")
clipLog("calling showCameraHud")
showCameraHud("Recording…", CameraHudKind.Recording, null)
val filePayload =
try {
clipLog("calling camera.clip()")
val r = camera.clip(paramsJson)
clipLog("success, file size=${r.file.length()}")
r
} catch (err: Throwable) {
clipLog("inner error: ${err::class.java.simpleName}: ${err.message}")
clipLog("stack: ${err.stackTraceToString().take(2000)}")
val (code, message) = invokeErrorFromThrowable(err)
showCameraHud(message, CameraHudKind.Error, 2400)
return GatewaySession.InvokeResult.error(code = code, message = message)
}
val rawBytes = filePayload.file.length()
if (!isCameraClipWithinPayloadLimit(rawBytes)) {
clipLog("payload too large: bytes=$rawBytes max=$CAMERA_CLIP_MAX_RAW_BYTES")
// Delete oversized clips before returning so cache files do not accumulate after failed invokes.
withContext(Dispatchers.IO) { filePayload.file.delete() }
showCameraHud("Clip too large", CameraHudKind.Error, 2400)
return GatewaySession.InvokeResult.error(
code = "PAYLOAD_TOO_LARGE",
message =
"PAYLOAD_TOO_LARGE: camera clip is $rawBytes bytes; max is $CAMERA_CLIP_MAX_RAW_BYTES bytes. Reduce durationMs and retry.",
)
}
val bytes =
withContext(Dispatchers.IO) {
try {
filePayload.file.readBytes()
} finally {
filePayload.file.delete()
}
}
val base64 = android.util.Base64.encodeToString(bytes, android.util.Base64.NO_WRAP)
clipLog("returning base64 payload")
showCameraHud("Clip captured", CameraHudKind.Success, 1800)
return GatewaySession.InvokeResult.ok(
"""{"format":"mp4","base64":"$base64","durationMs":${filePayload.durationMs},"hasAudio":${filePayload.hasAudio}}""",
)
} catch (err: Throwable) {
clipLog("outer error: ${err::class.java.simpleName}: ${err.message}")
clipLog("stack: ${err.stackTraceToString().take(2000)}")
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = err.message ?: "camera clip failed")
} finally {
// Prevent talk/transcription capture from competing with camera audio after every exit path.
if (includeAudio) externalAudioCaptureActive.value = false
}
}
private fun parseIncludeAudio(paramsJson: String?): Boolean? {
if (paramsJson.isNullOrBlank()) return null
val root =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val value =
(root["includeAudio"] as? JsonPrimitive)
?.contentOrNull
?.trim()
?.lowercase()
return when (value) {
"true" -> true
"false" -> false
else -> null
}
}
}

View File

@@ -0,0 +1,21 @@
package ai.openclaw.app.node
/**
* Trust helper for WebView-originated canvas/A2UI actions.
*/
object CanvasActionTrust {
/** Local canvas scaffold is the only trusted file URL. */
const val scaffoldAssetUrl: String = "file:///android_asset/CanvasScaffold/scaffold.html"
/** Local bundled A2UI is the only action-capable A2UI host. */
const val localA2uiAssetUrl: String = "file:///android_asset/CanvasA2UI/index.html"
/** Accepts only app-owned bundled pages. Remote WebView content is render-only. */
fun isTrustedCanvasActionUrl(rawUrl: String?): Boolean {
val candidate = rawUrl?.trim().orEmpty()
if (candidate.isEmpty()) return false
if (candidate == scaffoldAssetUrl) return true
if (candidate == localA2uiAssetUrl) return true
return false
}
}

View File

@@ -0,0 +1,334 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import android.graphics.Bitmap
import android.graphics.Canvas
import android.os.Looper
import android.util.Base64
import android.util.Log
import android.webkit.WebView
import androidx.core.graphics.createBitmap
import androidx.core.graphics.scale
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import org.json.JSONObject
import java.io.ByteArrayOutputStream
import kotlin.coroutines.resume
/**
* Owns the Android WebView canvas surface used by canvas and A2UI commands.
*/
class CanvasController {
enum class SnapshotFormat(
val rawValue: String,
) {
Png("png"),
Jpeg("jpeg"),
}
@Volatile private var webView: WebView? = null
@Volatile private var url: String? = null
@Volatile private var debugStatusEnabled: Boolean = false
@Volatile private var debugStatusTitle: String? = null
@Volatile private var debugStatusSubtitle: String? = null
@Volatile private var homeCanvasStateJson: String? = null
private val _currentUrl = MutableStateFlow<String?>(null)
val currentUrl: StateFlow<String?> = _currentUrl.asStateFlow()
private val scaffoldAssetUrl = CanvasActionTrust.scaffoldAssetUrl
private val localA2uiAssetUrl = CanvasActionTrust.localA2uiAssetUrl
private fun clampJpegQuality(quality: Double?): Int {
val q = (quality ?: 0.82).coerceIn(0.1, 1.0)
return (q * 100.0).toInt().coerceIn(1, 100)
}
private fun Bitmap.scaleForMaxWidth(maxWidth: Int?): Bitmap {
if (maxWidth == null || maxWidth <= 0 || width <= maxWidth) {
return this
}
val scaledHeight = (height.toDouble() * (maxWidth.toDouble() / width.toDouble())).toInt().coerceAtLeast(1)
return scale(maxWidth, scaledHeight)
}
/** Attaches the active WebView and replays state that may have arrived before the view existed. */
fun attach(webView: WebView) {
this.webView = webView
// Replay persisted state because WebView attachment can happen after gateway events arrive.
reload()
applyDebugStatus()
applyHomeCanvasState()
}
/** Detaches only the currently attached WebView instance. */
fun detach(webView: WebView) {
if (this.webView === webView) {
this.webView = null
}
}
/** Navigates the canvas to a remote URL or back to the bundled scaffold for blank/root input. */
fun navigate(url: String) {
val trimmed = url.trim()
this.url = if (trimmed.isBlank() || trimmed == "/") null else trimmed
_currentUrl.value = this.url
reload()
}
/** Shows the app-owned A2UI renderer that is allowed to dispatch native actions. */
fun showLocalA2ui() {
this.url = localA2uiAssetUrl
_currentUrl.value = localA2uiAssetUrl
reload()
}
fun currentUrl(): String? = url
fun setDebugStatusEnabled(enabled: Boolean) {
debugStatusEnabled = enabled
applyDebugStatus()
}
fun setDebugStatus(
title: String?,
subtitle: String?,
) {
debugStatusTitle = title
debugStatusSubtitle = subtitle
applyDebugStatus()
}
fun onPageFinished() {
applyDebugStatus()
applyHomeCanvasState()
}
fun updateHomeCanvasState(json: String?) {
homeCanvasStateJson = json
applyHomeCanvasState()
}
private inline fun withWebViewOnMain(crossinline block: (WebView) -> Unit) {
val wv = webView ?: return
if (Looper.myLooper() == Looper.getMainLooper()) {
block(wv)
} else {
// WebView APIs must run on the main thread.
wv.post { block(wv) }
}
}
private fun reload() {
val currentUrl = url
withWebViewOnMain { wv ->
if (currentUrl == null) {
if (BuildConfig.DEBUG) {
Log.d("OpenClawCanvas", "load scaffold: $scaffoldAssetUrl")
}
wv.loadUrl(scaffoldAssetUrl)
} else {
if (BuildConfig.DEBUG) {
Log.d("OpenClawCanvas", "load url: $currentUrl")
}
wv.loadUrl(currentUrl)
}
}
}
private fun applyDebugStatus() {
val enabled = debugStatusEnabled
val title = debugStatusTitle
val subtitle = debugStatusSubtitle
withWebViewOnMain { wv ->
val titleJs = title?.let { JSONObject.quote(it) } ?: "null"
val subtitleJs = subtitle?.let { JSONObject.quote(it) } ?: "null"
val js =
"""
(() => {
try {
const api = globalThis.__openclaw;
if (!api) return;
if (typeof api.setDebugStatusEnabled === 'function') {
api.setDebugStatusEnabled(${if (enabled) "true" else "false"});
}
if (!${if (enabled) "true" else "false"}) return;
if (typeof api.setStatus === 'function') {
api.setStatus($titleJs, $subtitleJs);
}
} catch (_) {}
})();
""".trimIndent()
wv.evaluateJavascript(js, null)
}
}
private fun applyHomeCanvasState() {
val payload = homeCanvasStateJson ?: "null"
withWebViewOnMain { wv ->
val js =
"""
(() => {
try {
const api = globalThis.__openclaw;
if (!api || typeof api.renderHome !== 'function') return;
api.renderHome($payload);
} catch (_) {}
})();
""".trimIndent()
wv.evaluateJavascript(js, null)
}
}
/** Evaluates JavaScript against the attached WebView on the main thread. */
suspend fun eval(javaScript: String): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
suspendCancellableCoroutine { cont ->
wv.evaluateJavascript(javaScript) { result ->
cont.resume(result ?: "")
}
}
}
/** Captures the WebView as PNG/JPEG base64 with optional width and quality bounds. */
suspend fun snapshotBase64(
format: SnapshotFormat,
quality: Double?,
maxWidth: Int?,
): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
val bmp = wv.captureBitmap()
try {
val scaled = bmp.scaleForMaxWidth(maxWidth)
try {
val out = ByteArrayOutputStream()
val (compressFormat, compressQuality) =
when (format) {
SnapshotFormat.Png -> Bitmap.CompressFormat.PNG to 100
SnapshotFormat.Jpeg -> Bitmap.CompressFormat.JPEG to clampJpegQuality(quality)
}
scaled.compress(compressFormat, compressQuality, out)
Base64.encodeToString(out.toByteArray(), Base64.NO_WRAP)
} finally {
if (scaled !== bmp) scaled.recycle()
}
} finally {
bmp.recycle()
}
}
private suspend fun WebView.captureBitmap(): Bitmap =
suspendCancellableCoroutine { cont ->
val width = width.coerceAtLeast(1)
val height = height.coerceAtLeast(1)
val bitmap = createBitmap(width, height, Bitmap.Config.ARGB_8888)
// WebView isn't supported by PixelCopy.request(...) directly; draw() is the most reliable
// cross-version snapshot for this lightweight "canvas" use-case.
draw(Canvas(bitmap))
cont.resume(bitmap)
}
companion object {
/**
* Parsed canvas.snapshot options used by invoke dispatch.
*/
data class SnapshotParams(
val format: SnapshotFormat,
val quality: Double?,
val maxWidth: Int?,
)
/** Parses canvas.navigate params and returns blank when the payload is missing or invalid. */
fun parseNavigateUrl(paramsJson: String?): String {
val obj = parseParamsObject(paramsJson) ?: return ""
return obj.string("url").trim()
}
/** Parses non-blank JavaScript from canvas.eval params. */
fun parseEvalJs(paramsJson: String?): String? {
val obj = parseParamsObject(paramsJson) ?: return null
val js = obj.string("javaScript").trim()
return js.takeIf { it.isNotBlank() }
}
fun parseSnapshotMaxWidth(paramsJson: String?): Int? {
val obj = parseParamsObject(paramsJson) ?: return null
if (!obj.containsKey("maxWidth")) return null
val width = obj.int("maxWidth") ?: 0
return width.takeIf { it > 0 }
}
fun parseSnapshotFormat(paramsJson: String?): SnapshotFormat {
val obj = parseParamsObject(paramsJson) ?: return SnapshotFormat.Jpeg
val raw = obj.string("format").trim().lowercase()
return when (raw) {
"png" -> SnapshotFormat.Png
"jpeg", "jpg" -> SnapshotFormat.Jpeg
"" -> SnapshotFormat.Jpeg
else -> SnapshotFormat.Jpeg
}
}
fun parseSnapshotQuality(paramsJson: String?): Double? {
val obj = parseParamsObject(paramsJson) ?: return null
if (!obj.containsKey("quality")) return null
val q = obj.double("quality") ?: Double.NaN
if (!q.isFinite()) return null
// Keep JPEG quality inside encoder-safe bounds; PNG ignores it.
return q.coerceIn(0.1, 1.0)
}
/** Parses canvas.snapshot params using JPEG defaults and encoder-safe bounds. */
fun parseSnapshotParams(paramsJson: String?): SnapshotParams =
SnapshotParams(
format = parseSnapshotFormat(paramsJson),
quality = parseSnapshotQuality(paramsJson),
maxWidth = parseSnapshotMaxWidth(paramsJson),
)
private val json = Json { ignoreUnknownKeys = true }
private fun parseParamsObject(paramsJson: String?): JsonObject? {
val raw = paramsJson?.trim().orEmpty()
if (raw.isEmpty()) return null
return try {
json.parseToJsonElement(raw).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonObject.string(key: String): String {
val prim = this[key] as? JsonPrimitive ?: return ""
val raw = prim.content
return raw.takeIf { it != "null" }.orEmpty()
}
private fun JsonObject.int(key: String): Int? {
val prim = this[key] as? JsonPrimitive ?: return null
return prim.content.toIntOrNull()
}
private fun JsonObject.double(key: String): Double? {
val prim = this[key] as? JsonPrimitive ?: return null
return prim.content.toDoubleOrNull()
}
}
}

View File

@@ -0,0 +1,232 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.LocationMode
import ai.openclaw.app.SecurePrefs
import ai.openclaw.app.VoiceWakeMode
import ai.openclaw.app.gateway.GatewayClientInfo
import ai.openclaw.app.gateway.GatewayConnectOptions
import ai.openclaw.app.gateway.GatewayEndpoint
import ai.openclaw.app.gateway.GatewayTlsParams
import ai.openclaw.app.gateway.isLocalCleartextGatewayHost
import ai.openclaw.app.gateway.isLoopbackGatewayHost
import android.os.Build
/**
* Builds gateway connect metadata from current Android permissions, settings, and device identity.
*/
class ConnectionManager(
private val prefs: SecurePrefs,
private val cameraEnabled: () -> Boolean,
private val locationMode: () -> LocationMode,
private val voiceWakeMode: () -> VoiceWakeMode,
private val motionActivityAvailable: () -> Boolean,
private val motionPedometerAvailable: () -> Boolean,
private val sendSmsAvailable: () -> Boolean,
private val readSmsAvailable: () -> Boolean,
private val smsSearchPossible: () -> Boolean,
private val callLogAvailable: () -> Boolean,
private val photosAvailable: () -> Boolean,
private val hasRecordAudioPermission: () -> Boolean,
private val installedAppsSharingEnabled: () -> Boolean,
private val manualTls: () -> Boolean,
) {
companion object {
internal val legacyOperatorScopes: List<String> =
listOf(
"operator.approvals",
"operator.read",
"operator.write",
)
internal val nativeClientOperatorScopes: List<String> =
listOf(
"operator.approvals",
"operator.read",
"operator.talk.secrets",
"operator.write",
)
internal fun operatorScopesForStoredDeviceToken(storedScopes: List<String>): List<String> {
val normalized =
storedScopes
.map { it.trim() }
.filter { it.isNotEmpty() }
.distinct()
return normalized.ifEmpty { legacyOperatorScopes }
}
/**
* Decide whether a discovered/manual endpoint must use pinned TLS or can stay local cleartext.
*/
internal fun resolveTlsParamsForEndpoint(
endpoint: GatewayEndpoint,
storedFingerprint: String?,
manualTlsEnabled: Boolean,
): GatewayTlsParams? {
val stableId = endpoint.stableId
val stored = storedFingerprint?.trim().takeIf { !it.isNullOrEmpty() }
val isManual = stableId.startsWith("manual|")
val cleartextAllowedHost =
if (isManual) {
isLocalCleartextGatewayHost(endpoint.host)
} else {
isLoopbackGatewayHost(endpoint.host)
}
if (isManual) {
// Manual remote hosts default to TLS; only local manual hosts may honor the cleartext toggle.
if (!manualTlsEnabled && cleartextAllowedHost) return null
if (!stored.isNullOrBlank()) {
return GatewayTlsParams(
required = true,
expectedFingerprint = stored,
allowTOFU = false,
stableId = stableId,
)
}
return GatewayTlsParams(
required = true,
expectedFingerprint = null,
allowTOFU = false,
stableId = stableId,
)
}
// Prefer stored pins. Never let discovery-provided TXT override a stored fingerprint.
if (!stored.isNullOrBlank()) {
return GatewayTlsParams(
required = true,
expectedFingerprint = stored,
allowTOFU = false,
stableId = stableId,
)
}
val hinted = endpoint.tlsEnabled || !endpoint.tlsFingerprintSha256.isNullOrBlank()
if (hinted) {
// TXT is unauthenticated. Do not treat the advertised fingerprint as authoritative.
return GatewayTlsParams(
required = true,
expectedFingerprint = null,
allowTOFU = false,
stableId = stableId,
)
}
if (!cleartextAllowedHost) {
// Non-loopback discovered hosts require TLS even without TXT hints.
return GatewayTlsParams(
required = true,
expectedFingerprint = null,
allowTOFU = false,
stableId = stableId,
)
}
return null
}
}
private fun runtimeFlags(): NodeRuntimeFlags =
NodeRuntimeFlags(
cameraEnabled = cameraEnabled(),
locationEnabled = locationMode() != LocationMode.Off,
sendSmsAvailable = sendSmsAvailable(),
readSmsAvailable = readSmsAvailable(),
smsSearchPossible = smsSearchPossible(),
callLogAvailable = callLogAvailable(),
photosAvailable = photosAvailable(),
voiceWakeEnabled = voiceWakeMode() != VoiceWakeMode.Off && hasRecordAudioPermission(),
motionActivityAvailable = motionActivityAvailable(),
motionPedometerAvailable = motionPedometerAvailable(),
installedAppsSharingEnabled = installedAppsSharingEnabled(),
debugBuild = BuildConfig.DEBUG,
)
/** Builds the gateway-advertised node.invoke command list from current permission and feature state. */
fun buildInvokeCommands(): List<String> = InvokeCommandRegistry.advertisedCommands(runtimeFlags())
/** Builds the gateway-advertised capability list from current permission and feature state. */
fun buildCapabilities(): List<String> = InvokeCommandRegistry.advertisedCapabilities(runtimeFlags())
/**
* Debug Android builds advertise a dev version so gateway logs do not look like release clients.
*/
fun resolvedVersionName(): String {
val versionName = BuildConfig.VERSION_NAME.trim().ifEmpty { "dev" }
return if (BuildConfig.DEBUG && !versionName.contains("dev", ignoreCase = true)) {
"$versionName-dev"
} else {
versionName
}
}
/** Human-readable Android device model used in gateway client metadata. */
fun resolveModelIdentifier(): String? =
listOfNotNull(Build.MANUFACTURER, Build.MODEL)
.joinToString(" ")
.trim()
.ifEmpty { null }
/**
* User-Agent used for gateway telemetry and troubleshooting.
*/
fun buildUserAgent(): String {
val version = resolvedVersionName()
val release =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
val releaseLabel = if (release.isEmpty()) "unknown" else release
return "OpenClawAndroid/$version (Android $releaseLabel; SDK ${Build.VERSION.SDK_INT})"
}
/** Client identity block shared by node and operator gateway sessions. */
fun buildClientInfo(
clientId: String,
clientMode: String,
): GatewayClientInfo =
GatewayClientInfo(
id = clientId,
displayName = prefs.displayName.value,
version = resolvedVersionName(),
platform = "android",
mode = clientMode,
instanceId = prefs.instanceId.value,
deviceFamily = "Android",
modelIdentifier = resolveModelIdentifier(),
)
/** Connect options for the Android node session that exposes phone capabilities. */
fun buildNodeConnectOptions(): GatewayConnectOptions =
GatewayConnectOptions(
role = "node",
scopes = emptyList(),
caps = buildCapabilities(),
commands = buildInvokeCommands(),
permissions = emptyMap(),
client = buildClientInfo(clientId = "openclaw-android", clientMode = "node"),
userAgent = buildUserAgent(),
)
/** Connect options for the Android operator session that drives approvals and UI actions. */
fun buildOperatorConnectOptions(
scopes: List<String> = nativeClientOperatorScopes,
): GatewayConnectOptions =
GatewayConnectOptions(
role = "operator",
scopes = scopes,
caps = emptyList(),
commands = emptyList(),
permissions = emptyMap(),
client = buildClientInfo(clientId = "openclaw-android", clientMode = "ui"),
userAgent = buildUserAgent(),
)
/** Resolves persisted TLS pin policy for a concrete gateway endpoint. */
fun resolveTlsParams(endpoint: GatewayEndpoint): GatewayTlsParams? {
val stored = prefs.loadGatewayTlsFingerprint(endpoint.stableId)
return resolveTlsParamsForEndpoint(endpoint, storedFingerprint = stored, manualTlsEnabled = manualTls())
}
}

View File

@@ -0,0 +1,489 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.ContentProviderOperation
import android.content.ContentResolver
import android.content.Context
import android.provider.ContactsContract
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
private const val DEFAULT_CONTACTS_LIMIT = 25
/**
* Normalized Android contact row returned through the contacts commands.
*/
internal data class ContactRecord(
val identifier: String,
val displayName: String,
val givenName: String,
val familyName: String,
val organizationName: String,
val phoneNumbers: List<String>,
val emails: List<String>,
)
/**
* Parsed contacts.search request with bounded result count.
*/
internal data class ContactsSearchRequest(
val query: String?,
val limit: Int,
)
/**
* Parsed contacts.add request before ContentProviderOperation batching.
*/
internal data class ContactsAddRequest(
val givenName: String?,
val familyName: String?,
val organizationName: String?,
val displayName: String?,
val phoneNumbers: List<String>,
val emails: List<String>,
)
/**
* Injectable ContactsProvider facade for command tests and Android runtime access.
*/
internal interface ContactsDataSource {
fun hasReadPermission(context: Context): Boolean
fun hasWritePermission(context: Context): Boolean
fun search(
context: Context,
request: ContactsSearchRequest,
): List<ContactRecord>
fun add(
context: Context,
request: ContactsAddRequest,
): ContactRecord
}
private object SystemContactsDataSource : ContactsDataSource {
override fun hasReadPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CONTACTS) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun hasWritePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CONTACTS) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun search(
context: Context,
request: ContactsSearchRequest,
): List<ContactRecord> {
val resolver = context.contentResolver
val projection =
arrayOf(
ContactsContract.Contacts._ID,
ContactsContract.Contacts.DISPLAY_NAME_PRIMARY,
)
val selection: String?
val selectionArgs: Array<String>?
if (request.query.isNullOrBlank()) {
selection = null
selectionArgs = null
} else {
// Escape wildcard characters so user text remains a substring search, not a LIKE pattern.
selection = "${ContactsContract.Contacts.DISPLAY_NAME_PRIMARY} LIKE ? ESCAPE '\\'"
selectionArgs = arrayOf("%${escapeLikePattern(request.query)}%")
}
val sortOrder = "${ContactsContract.Contacts.DISPLAY_NAME_PRIMARY} COLLATE NOCASE ASC LIMIT ${request.limit}"
resolver
.query(
ContactsContract.Contacts.CONTENT_URI,
projection,
selection,
selectionArgs,
sortOrder,
).use { cursor ->
if (cursor == null) return emptyList()
val idIndex = cursor.getColumnIndexOrThrow(ContactsContract.Contacts._ID)
val displayNameIndex = cursor.getColumnIndexOrThrow(ContactsContract.Contacts.DISPLAY_NAME_PRIMARY)
val out = mutableListOf<ContactRecord>()
while (cursor.moveToNext() && out.size < request.limit) {
val contactId = cursor.getLong(idIndex)
val displayName = cursor.getString(displayNameIndex).orEmpty()
out += loadContactRecord(resolver, contactId, fallbackDisplayName = displayName)
}
return out
}
}
override fun add(
context: Context,
request: ContactsAddRequest,
): ContactRecord {
val resolver = context.contentResolver
val operations = ArrayList<ContentProviderOperation>()
operations +=
ContentProviderOperation
.newInsert(ContactsContract.RawContacts.CONTENT_URI)
.withValue(ContactsContract.RawContacts.ACCOUNT_TYPE, null)
.withValue(ContactsContract.RawContacts.ACCOUNT_NAME, null)
.build()
// Subsequent Data rows use back-reference 0 to attach to the RawContact inserted above.
if (!request.givenName.isNullOrEmpty() || !request.familyName.isNullOrEmpty() || !request.displayName.isNullOrEmpty()) {
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.StructuredName.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.StructuredName.GIVEN_NAME, request.givenName)
.withValue(ContactsContract.CommonDataKinds.StructuredName.FAMILY_NAME, request.familyName)
.withValue(ContactsContract.CommonDataKinds.StructuredName.DISPLAY_NAME, request.displayName)
.build()
}
if (!request.organizationName.isNullOrEmpty()) {
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Organization.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.Organization.COMPANY, request.organizationName)
.build()
}
request.phoneNumbers.forEach { number ->
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Phone.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.Phone.NUMBER, number)
.withValue(ContactsContract.CommonDataKinds.Phone.TYPE, ContactsContract.CommonDataKinds.Phone.TYPE_MOBILE)
.build()
}
request.emails.forEach { email ->
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Email.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.Email.ADDRESS, email)
.withValue(ContactsContract.CommonDataKinds.Email.TYPE, ContactsContract.CommonDataKinds.Email.TYPE_HOME)
.build()
}
val results = resolver.applyBatch(ContactsContract.AUTHORITY, operations)
val rawContactUri =
results.firstOrNull()?.uri
?: throw IllegalStateException("contact insert failed")
val rawContactId =
rawContactUri.lastPathSegment?.toLongOrNull()
?: throw IllegalStateException("contact insert failed")
val contactId =
// Android returns the RawContact id; resolve the aggregate Contact id used by search APIs.
resolveContactIdForRawContact(resolver, rawContactId)
?: throw IllegalStateException("contact insert failed")
return loadContactRecord(
resolver = resolver,
contactId = contactId,
fallbackDisplayName = request.displayName.orEmpty(),
)
}
private fun resolveContactIdForRawContact(
resolver: ContentResolver,
rawContactId: Long,
): Long? {
val projection = arrayOf(ContactsContract.RawContacts.CONTACT_ID)
resolver
.query(
ContactsContract.RawContacts.CONTENT_URI,
projection,
"${ContactsContract.RawContacts._ID}=?",
arrayOf(rawContactId.toString()),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
val index = cursor.getColumnIndexOrThrow(ContactsContract.RawContacts.CONTACT_ID)
return cursor.getLong(index)
}
}
private fun loadContactRecord(
resolver: ContentResolver,
contactId: Long,
fallbackDisplayName: String,
): ContactRecord {
val nameRow = loadNameRow(resolver, contactId)
val organization = loadOrganization(resolver, contactId)
val phones = loadPhones(resolver, contactId)
val emails = loadEmails(resolver, contactId)
val displayName =
when {
!nameRow.displayName.isNullOrEmpty() -> nameRow.displayName
!fallbackDisplayName.isNullOrEmpty() -> fallbackDisplayName
else -> listOfNotNull(nameRow.givenName, nameRow.familyName).joinToString(" ").trim()
}.ifEmpty { "(unnamed)" }
return ContactRecord(
identifier = contactId.toString(),
displayName = displayName,
givenName = nameRow.givenName.orEmpty(),
familyName = nameRow.familyName.orEmpty(),
organizationName = organization.orEmpty(),
phoneNumbers = phones,
emails = emails,
)
}
private data class NameRow(
val givenName: String?,
val familyName: String?,
val displayName: String?,
)
private fun loadNameRow(
resolver: ContentResolver,
contactId: Long,
): NameRow {
val projection =
arrayOf(
ContactsContract.CommonDataKinds.StructuredName.GIVEN_NAME,
ContactsContract.CommonDataKinds.StructuredName.FAMILY_NAME,
ContactsContract.CommonDataKinds.StructuredName.DISPLAY_NAME,
)
resolver
.query(
ContactsContract.Data.CONTENT_URI,
projection,
"${ContactsContract.Data.CONTACT_ID}=? AND ${ContactsContract.Data.MIMETYPE}=?",
arrayOf(
contactId.toString(),
ContactsContract.CommonDataKinds.StructuredName.CONTENT_ITEM_TYPE,
),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) {
return NameRow(givenName = null, familyName = null, displayName = null)
}
val given = cursor.getString(0)?.trim()?.ifEmpty { null }
val family = cursor.getString(1)?.trim()?.ifEmpty { null }
val display = cursor.getString(2)?.trim()?.ifEmpty { null }
return NameRow(givenName = given, familyName = family, displayName = display)
}
}
private fun loadOrganization(
resolver: ContentResolver,
contactId: Long,
): String? {
val projection = arrayOf(ContactsContract.CommonDataKinds.Organization.COMPANY)
resolver
.query(
ContactsContract.Data.CONTENT_URI,
projection,
"${ContactsContract.Data.CONTACT_ID}=? AND ${ContactsContract.Data.MIMETYPE}=?",
arrayOf(contactId.toString(), ContactsContract.CommonDataKinds.Organization.CONTENT_ITEM_TYPE),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return cursor.getString(0)?.trim()?.ifEmpty { null }
}
}
private fun escapeLikePattern(pattern: String): String = pattern.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
private fun loadPhones(
resolver: ContentResolver,
contactId: Long,
): List<String> =
queryContactValues(
resolver = resolver,
contentUri = ContactsContract.CommonDataKinds.Phone.CONTENT_URI,
valueColumn = ContactsContract.CommonDataKinds.Phone.NUMBER,
contactIdColumn = ContactsContract.CommonDataKinds.Phone.CONTACT_ID,
contactId = contactId,
)
private fun loadEmails(
resolver: ContentResolver,
contactId: Long,
): List<String> =
queryContactValues(
resolver = resolver,
contentUri = ContactsContract.CommonDataKinds.Email.CONTENT_URI,
valueColumn = ContactsContract.CommonDataKinds.Email.ADDRESS,
contactIdColumn = ContactsContract.CommonDataKinds.Email.CONTACT_ID,
contactId = contactId,
)
private fun queryContactValues(
resolver: ContentResolver,
contentUri: android.net.Uri,
valueColumn: String,
contactIdColumn: String,
contactId: Long,
): List<String> {
val projection = arrayOf(valueColumn)
resolver
.query(
contentUri,
projection,
"$contactIdColumn=?",
arrayOf(contactId.toString()),
null,
).use { cursor ->
if (cursor == null) return emptyList()
val out = LinkedHashSet<String>()
while (cursor.moveToNext()) {
val value = cursor.getString(0)?.trim().orEmpty()
if (value.isNotEmpty()) out += value
}
return out.toList()
}
}
}
/**
* Handles contacts.search and contacts.add gateway commands through Android ContactsProvider.
*/
class ContactsHandler private constructor(
private val appContext: Context,
private val dataSource: ContactsDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemContactsDataSource)
/** Searches contacts by optional display-name substring with bounded result count. */
fun handleContactsSearch(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasReadPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CONTACTS_PERMISSION_REQUIRED",
message = "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission",
)
}
val request =
parseSearchRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val contacts = dataSource.search(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put(
"contacts",
buildJsonArray {
contacts.forEach { add(contactJson(it)) }
},
)
}.toString(),
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CONTACTS_UNAVAILABLE",
message = "CONTACTS_UNAVAILABLE: ${err.message ?: "contacts query failed"}",
)
}
}
/** Adds a local contact after validating that at least one user-visible field is present. */
fun handleContactsAdd(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasWritePermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CONTACTS_PERMISSION_REQUIRED",
message = "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission",
)
}
val request =
parseAddRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
val hasName =
!(request.givenName.isNullOrEmpty() && request.familyName.isNullOrEmpty() && request.displayName.isNullOrEmpty())
val hasOrg = !request.organizationName.isNullOrEmpty()
val hasDetails = request.phoneNumbers.isNotEmpty() || request.emails.isNotEmpty()
if (!hasName && !hasOrg && !hasDetails) {
return GatewaySession.InvokeResult.error(
code = "CONTACTS_INVALID",
message = "CONTACTS_INVALID: include a name, organization, phone, or email",
)
}
return try {
val contact = dataSource.add(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put("contact", contactJson(contact))
}.toString(),
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CONTACTS_UNAVAILABLE",
message = "CONTACTS_UNAVAILABLE: ${err.message ?: "contact add failed"}",
)
}
}
private fun parseSearchRequest(paramsJson: String?): ContactsSearchRequest? {
if (paramsJson.isNullOrBlank()) {
return ContactsSearchRequest(query = null, limit = DEFAULT_CONTACTS_LIMIT)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val query = (params["query"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }
// Keep gateway-driven searches bounded even if the model asks for a large contact dump.
val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: DEFAULT_CONTACTS_LIMIT).coerceIn(1, 200)
return ContactsSearchRequest(query = query, limit = limit)
}
private fun parseAddRequest(paramsJson: String?): ContactsAddRequest? {
val params =
try {
paramsJson?.let { Json.parseToJsonElement(it).asObjectOrNull() }
} catch (_: Throwable) {
null
} ?: return null
return ContactsAddRequest(
givenName = (params["givenName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
familyName = (params["familyName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
organizationName = (params["organizationName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
displayName = (params["displayName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
phoneNumbers = stringArray(params["phoneNumbers"] as? JsonArray),
// Store emails case-normalized so repeated model calls do not create casing-only duplicates.
emails = stringArray(params["emails"] as? JsonArray).map { it.lowercase() },
)
}
private fun stringArray(array: JsonArray?): List<String> {
if (array == null) return emptyList()
return array.mapNotNull { element ->
(element as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }
}
}
private fun contactJson(contact: ContactRecord): JsonObject =
buildJsonObject {
put("identifier", JsonPrimitive(contact.identifier))
put("displayName", JsonPrimitive(contact.displayName))
put("givenName", JsonPrimitive(contact.givenName))
put("familyName", JsonPrimitive(contact.familyName))
put("organizationName", JsonPrimitive(contact.organizationName))
put("phoneNumbers", buildJsonArray { contact.phoneNumbers.forEach { add(JsonPrimitive(it)) } })
put("emails", buildJsonArray { contact.emails.forEach { add(JsonPrimitive(it)) } })
}
companion object {
/** Creates a handler with an injected contacts source for parser and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: ContactsDataSource,
): ContactsHandler = ContactsHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,155 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.gateway.DeviceIdentityStore
import ai.openclaw.app.gateway.GatewaySession
import android.content.Context
import kotlinx.serialization.json.JsonPrimitive
private const val LOGCAT_PATH = "/system/bin/logcat"
/**
* Debug-only node.invoke commands for Android cryptography and log diagnostics.
*/
class DebugHandler(
private val appContext: Context,
private val identityStore: DeviceIdentityStore,
) {
/**
* Runs an Ed25519 self-test and returns redacted diagnostics for debug builds.
*/
fun handleEd25519(): GatewaySession.InvokeResult {
if (!BuildConfig.DEBUG) {
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = "debug commands are disabled in release builds")
}
// Self-test Ed25519 signing without returning full private/public key material.
try {
val identity = identityStore.loadOrCreate()
val testPayload = "test|${identity.deviceId}|${System.currentTimeMillis()}"
val results = mutableListOf<String>()
results.add("deviceId: ${identity.deviceId}")
results.add("publicKeyRawBase64: ${identity.publicKeyRawBase64.take(20)}...")
results.add("privateKeyPkcs8Base64: ${identity.privateKeyPkcs8Base64.take(20)}...")
// Public-key URL encoding must match the gateway device-auth payload contract.
val pubKeyUrl = identityStore.publicKeyBase64Url(identity)
results.add("publicKeyBase64Url: ${pubKeyUrl ?: "NULL (FAILED)"}")
// Sign/verify through DeviceIdentityStore to catch provider and key-format failures together.
val signature = identityStore.signPayload(testPayload, identity)
results.add("signPayload: ${if (signature != null) "${signature.take(20)}... (OK)" else "NULL (FAILED)"}")
if (signature != null) {
val verifyOk = identityStore.verifySelfSignature(testPayload, signature, identity)
results.add("verifySelfSignature: $verifyOk")
}
// Check available providers
val providers = java.security.Security.getProviders()
val ed25519Providers =
providers.filter { p ->
p.services.any { s -> s.algorithm.contains("Ed25519", ignoreCase = true) }
}
results.add("Ed25519 providers: ${ed25519Providers.map { "${it.name} v${it.version}" }}")
results.add("Provider order: ${providers.take(5).map { it.name }}")
// Test KeyFactory directly
try {
val kf = java.security.KeyFactory.getInstance("Ed25519")
results.add("KeyFactory.Ed25519: ${kf.provider.name} (OK)")
} catch (e: Throwable) {
results.add("KeyFactory.Ed25519: FAILED - ${e.javaClass.simpleName}: ${e.message}")
}
// Test Signature directly
try {
val sig = java.security.Signature.getInstance("Ed25519")
results.add("Signature.Ed25519: ${sig.provider.name} (OK)")
} catch (e: Throwable) {
results.add("Signature.Ed25519: FAILED - ${e.javaClass.simpleName}: ${e.message}")
}
val diagnostics = results.joinToString("\n")
return GatewaySession.InvokeResult.ok("""{"diagnostics":${JsonPrimitive(diagnostics)}}""")
} catch (e: Throwable) {
return GatewaySession.InvokeResult.error(
code = "ED25519_TEST_FAILED",
message = "${e.javaClass.simpleName}: ${e.message}\n${e.stackTraceToString().take(500)}",
)
}
}
/**
* Returns a filtered logcat snapshot plus CameraX debug log for debug builds.
*/
fun handleLogs(): GatewaySession.InvokeResult {
if (!BuildConfig.DEBUG) {
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = "debug commands are disabled in release builds")
}
val pid = android.os.Process.myPid()
val rt = Runtime.getRuntime()
val info = "v6 pid=$pid thread=${Thread.currentThread().name} free=${rt.freeMemory() / 1024}K total=${rt.totalMemory() / 1024}K max=${rt.maxMemory() / 1024}K uptime=${android.os.SystemClock.elapsedRealtime() / 1000}s sdk=${android.os.Build.VERSION.SDK_INT} device=${android.os.Build.MODEL}\n"
// Capture only this process and redirect through a temp file to avoid blocking on pipe backpressure.
val logResult =
try {
val tmpFile = java.io.File(appContext.cacheDir, "debug_logs.txt")
if (tmpFile.exists()) tmpFile.delete()
val pb = ProcessBuilder(LOGCAT_PATH, "-d", "-t", "200", "--pid=$pid")
pb.redirectOutput(tmpFile)
pb.redirectErrorStream(true)
val proc = pb.start()
val finished = proc.waitFor(4, java.util.concurrent.TimeUnit.SECONDS)
if (!finished) proc.destroyForcibly()
val raw =
if (tmpFile.exists() && tmpFile.length() > 0) {
tmpFile.readText().take(128000)
} else {
"(no output, finished=$finished, exists=${tmpFile.exists()})"
}
tmpFile.delete()
val spamPatterns =
listOf(
"setRequestedFrameRate",
"I View :",
"BLASTBufferQueue",
"VRI[Pop-Up",
"InsetsController:",
"VRI[MainActivity",
"InsetsSource:",
"handleResized",
"ProfileInstaller",
"I VRI[",
"onStateChanged: host=",
"D StrictMode:",
"E StrictMode:",
"ImeFocusController",
"InputTransport",
"IncorrectContextUseViolation",
)
val sb = StringBuilder()
for (line in raw.lineSequence()) {
if (line.isBlank()) continue
if (spamPatterns.any { line.contains(it) }) continue
if (sb.length + line.length > 16000) {
// Keep debug.invoke responses small enough for the gateway WebSocket frame budget.
sb.append("\n(truncated)")
break
}
if (sb.isNotEmpty()) sb.append('\n')
sb.append(line)
}
sb.toString().ifEmpty { "(all ${raw.lines().size} lines filtered as spam)" }
} catch (e: Throwable) {
"(logcat error: ${e::class.java.simpleName}: ${e.message})"
}
// Camera capture writes a separate debug file because CameraX failures often happen off logcat's hot path.
val camLogFile = java.io.File(appContext.cacheDir, "camera_debug.log")
val camLog =
if (camLogFile.exists() && camLogFile.length() > 0) {
"\n--- camera_debug.log ---\n" + camLogFile.readText().take(4000)
} else {
""
}
return GatewaySession.InvokeResult.ok("""{"logs":${JsonPrimitive(info + logResult + camLog)}}""")
}
}

View File

@@ -0,0 +1,660 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.SensitiveFeatureConfig
import ai.openclaw.app.gateway.GatewaySession
import ai.openclaw.app.hasPhotoReadPermission
import android.Manifest
import android.annotation.SuppressLint
import android.app.ActivityManager
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.ApplicationInfo
import android.content.pm.PackageManager
import android.net.ConnectivityManager
import android.net.NetworkCapabilities
import android.os.BatteryManager
import android.os.Build
import android.os.Environment
import android.os.PowerManager
import android.os.StatFs
import android.os.SystemClock
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.util.Locale
private const val DEFAULT_DEVICE_APPS_LIMIT = 100
private const val MAX_DEVICE_APPS_LIMIT = 200
private const val DEVICE_APPS_SYSTEM_FLAGS =
ApplicationInfo.FLAG_SYSTEM or ApplicationInfo.FLAG_UPDATED_SYSTEM_APP
internal fun isSystemDeviceApp(appInfo: ApplicationInfo): Boolean = (appInfo.flags and DEVICE_APPS_SYSTEM_FLAGS) != 0
internal data class DeviceAppEntry(
val label: String,
val packageName: String,
val system: Boolean,
val enabled: Boolean,
val launchable: Boolean,
)
internal interface DeviceAppSource {
fun listApps(includeNonLaunchable: Boolean): List<DeviceAppEntry>
}
private class AndroidDeviceAppSource(
private val appContext: Context,
) : DeviceAppSource {
override fun listApps(includeNonLaunchable: Boolean): List<DeviceAppEntry> {
val packageManager = appContext.packageManager
val launcherIntent = Intent(Intent.ACTION_MAIN).apply { addCategory(Intent.CATEGORY_LAUNCHER) }
val launchablePackages =
packageManager
.queryIntentActivities(launcherIntent, PackageManager.MATCH_ALL)
.asSequence()
.mapNotNull {
it.activityInfo
?.packageName
?.trim()
?.takeIf(String::isNotEmpty)
}.toSet()
val appInfos =
if (includeNonLaunchable) {
visibleInstalledApplications(packageManager)
} else {
launchablePackages.mapNotNull { packageName ->
runCatching { packageManager.getApplicationInfo(packageName, 0) }.getOrNull()
}
}
return appInfos
.asSequence()
.mapNotNull { appInfo ->
appInfo.packageName
?.trim()
?.takeIf(String::isNotEmpty)
?.let { packageName ->
val label = packageManager.getApplicationLabel(appInfo).toString().trim()
DeviceAppEntry(
label = label.ifEmpty { packageName },
packageName = packageName,
system = isSystemDeviceApp(appInfo),
enabled = appInfo.enabled,
launchable = packageName in launchablePackages,
)
}
}.distinctBy { it.packageName }
.sortedWith(compareBy<DeviceAppEntry> { it.label.lowercase() }.thenBy { it.packageName })
.toList()
}
@SuppressLint("QueryPermissionsNeeded")
private fun visibleInstalledApplications(packageManager: PackageManager): List<ApplicationInfo> {
// Android package visibility intentionally bounds this result to packages the app can see.
// OpenClaw should not request QUERY_ALL_PACKAGES for this optional device-context surface.
return packageManager.getInstalledApplications(PackageManager.MATCH_ALL)
}
}
private data class DeviceAppsRequest(
val includeSystem: Boolean,
val includeDisabled: Boolean,
val includeNonLaunchable: Boolean,
val query: String?,
val limit: Int,
)
/**
* Gateway device command adapter for Android status, info, permission, and health snapshots.
*/
class DeviceHandler private constructor(
private val appContext: Context,
private val smsEnabled: Boolean = SensitiveFeatureConfig.smsEnabled,
private val callLogEnabled: Boolean = SensitiveFeatureConfig.callLogEnabled,
private val photosEnabled: Boolean = SensitiveFeatureConfig.photosEnabled,
private val appSource: DeviceAppSource = AndroidDeviceAppSource(appContext),
) {
constructor(
appContext: Context,
smsEnabled: Boolean = SensitiveFeatureConfig.smsEnabled,
callLogEnabled: Boolean = SensitiveFeatureConfig.callLogEnabled,
photosEnabled: Boolean = SensitiveFeatureConfig.photosEnabled,
) : this(
appContext = appContext,
smsEnabled = smsEnabled,
callLogEnabled = callLogEnabled,
photosEnabled = photosEnabled,
appSource = AndroidDeviceAppSource(appContext),
)
companion object {
internal fun forTesting(
appContext: Context,
appSource: DeviceAppSource,
smsEnabled: Boolean = SensitiveFeatureConfig.smsEnabled,
callLogEnabled: Boolean = SensitiveFeatureConfig.callLogEnabled,
photosEnabled: Boolean = SensitiveFeatureConfig.photosEnabled,
): DeviceHandler =
DeviceHandler(
appContext = appContext,
smsEnabled = smsEnabled,
callLogEnabled = callLogEnabled,
photosEnabled = photosEnabled,
appSource = appSource,
)
/**
* SMS is available only when the feature flag, telephony hardware, and at least one SMS permission align.
*/
internal fun hasAnySmsCapability(
smsEnabled: Boolean,
telephonyAvailable: Boolean,
smsSendGranted: Boolean,
smsReadGranted: Boolean,
): Boolean = smsEnabled && telephonyAvailable && (smsSendGranted || smsReadGranted)
/**
* Prompt only when Android can grant a missing SMS permission that this build can use.
*/
internal fun isSmsPromptable(
smsEnabled: Boolean,
telephonyAvailable: Boolean,
smsSendGranted: Boolean,
smsReadGranted: Boolean,
): Boolean = smsEnabled && telephonyAvailable && (!smsSendGranted || !smsReadGranted)
}
private data class BatterySnapshot(
val status: Int,
val plugged: Int,
val levelFraction: Double?,
val temperatureC: Double?,
)
/** Returns battery, storage, network, and uptime state for device.status. */
fun handleDeviceStatus(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(statusPayloadJson())
/** Returns stable Android hardware, OS, app, and locale metadata for device.info. */
fun handleDeviceInfo(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(infoPayloadJson())
/** Returns permission and promptability state for Android capabilities exposed to the gateway. */
fun handleDevicePermissions(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(permissionsPayloadJson())
/** Returns coarse device health for memory, power, thermal, battery, and security patch state. */
fun handleDeviceHealth(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(healthPayloadJson())
fun handleDeviceApps(paramsJson: String?): GatewaySession.InvokeResult {
val request = parseDeviceAppsRequest(paramsJson)
val matchingApps =
appSource
.listApps(includeNonLaunchable = request.includeNonLaunchable)
.asSequence()
.filter { request.includeSystem || !it.system }
.filter { request.includeDisabled || it.enabled }
.filter { app ->
val query = request.query ?: return@filter true
app.label.contains(query, ignoreCase = true) || app.packageName.contains(query, ignoreCase = true)
}.toList()
val limitedApps = matchingApps.take(request.limit)
return GatewaySession.InvokeResult.ok(
buildJsonObject {
put("count", JsonPrimitive(limitedApps.size))
put("totalMatched", JsonPrimitive(matchingApps.size))
put("truncated", JsonPrimitive(matchingApps.size > limitedApps.size))
put("visibility", JsonPrimitive(if (request.includeNonLaunchable) "android-visible" else "launcher"))
put("includeSystem", JsonPrimitive(request.includeSystem))
put("includeDisabled", JsonPrimitive(request.includeDisabled))
put(
"apps",
buildJsonArray {
for (app in limitedApps) {
add(
buildJsonObject {
put("label", JsonPrimitive(app.label))
put("packageName", JsonPrimitive(app.packageName))
put("system", JsonPrimitive(app.system))
put("enabled", JsonPrimitive(app.enabled))
put("launchable", JsonPrimitive(app.launchable))
},
)
}
},
)
}.toString(),
)
}
private fun statusPayloadJson(): String {
val battery = readBatterySnapshot()
val powerManager = appContext.getSystemService(PowerManager::class.java)
val storage = StatFs(Environment.getDataDirectory().absolutePath)
val totalBytes = storage.totalBytes
val freeBytes = storage.availableBytes
val usedBytes = (totalBytes - freeBytes).coerceAtLeast(0L)
val connectivity = appContext.getSystemService(ConnectivityManager::class.java)
val activeNetwork = connectivity?.activeNetwork
val caps = activeNetwork?.let { connectivity.getNetworkCapabilities(it) }
// elapsedRealtime is monotonic device uptime, not wall-clock time.
val uptimeSeconds = SystemClock.elapsedRealtime() / 1_000.0
return buildJsonObject {
put(
"battery",
buildJsonObject {
battery.levelFraction?.let { put("level", JsonPrimitive(it)) }
put("state", JsonPrimitive(mapBatteryState(battery.status)))
put("lowPowerModeEnabled", JsonPrimitive(powerManager?.isPowerSaveMode == true))
},
)
put(
"thermal",
buildJsonObject {
put("state", JsonPrimitive(mapThermalState(powerManager)))
},
)
put(
"storage",
buildJsonObject {
put("totalBytes", JsonPrimitive(totalBytes))
put("freeBytes", JsonPrimitive(freeBytes))
put("usedBytes", JsonPrimitive(usedBytes))
},
)
put(
"network",
buildJsonObject {
put("status", JsonPrimitive(mapNetworkStatus(caps)))
put(
"isExpensive",
JsonPrimitive(
caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_METERED)?.not() ?: false,
),
)
put(
"isConstrained",
JsonPrimitive(
caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_RESTRICTED)?.not() ?: false,
),
)
put("interfaces", networkInterfacesJson(caps))
},
)
put("uptimeSeconds", JsonPrimitive(uptimeSeconds))
}.toString()
}
private fun infoPayloadJson(): String {
val model = Build.MODEL?.trim().orEmpty()
val manufacturer = Build.MANUFACTURER?.trim().orEmpty()
val modelIdentifier = Build.DEVICE?.trim().orEmpty()
val systemVersion =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
val locale = Locale.getDefault().toLanguageTag().trim()
val appVersion = BuildConfig.VERSION_NAME.trim()
val appBuild = BuildConfig.VERSION_CODE.toString()
return buildJsonObject {
put("deviceName", JsonPrimitive(model.ifEmpty { "Android" }))
put(
"modelIdentifier",
JsonPrimitive(modelIdentifier.ifEmpty { listOf(manufacturer, model).filter { it.isNotEmpty() }.joinToString(" ") }),
)
put("systemName", JsonPrimitive("Android"))
put("systemVersion", JsonPrimitive(systemVersion.ifEmpty { Build.VERSION.SDK_INT.toString() }))
put("appVersion", JsonPrimitive(appVersion.ifEmpty { "dev" }))
put("appBuild", JsonPrimitive(appBuild.ifEmpty { "0" }))
put("locale", JsonPrimitive(locale.ifEmpty { Locale.getDefault().toString() }))
}.toString()
}
private fun permissionsPayloadJson(): String {
val canSendSms = appContext.packageManager.hasSystemFeature(PackageManager.FEATURE_TELEPHONY)
val smsSendGranted = hasPermission(Manifest.permission.SEND_SMS)
val smsReadGranted = hasPermission(Manifest.permission.READ_SMS)
val notificationAccess = DeviceNotificationListenerService.isAccessEnabled(appContext)
val photosGranted =
if (!photosEnabled) {
false
} else {
hasPhotoReadPermission(appContext)
}
val motionGranted = hasPermission(Manifest.permission.ACTIVITY_RECOGNITION)
val notificationsGranted =
if (Build.VERSION.SDK_INT >= 33) {
// POST_NOTIFICATIONS exists only on Android 13+.
hasPermission(Manifest.permission.POST_NOTIFICATIONS)
} else {
true
}
return buildJsonObject {
put(
"permissions",
buildJsonObject {
put(
"camera",
permissionStateJson(
granted = hasPermission(Manifest.permission.CAMERA),
promptableWhenDenied = true,
),
)
put(
"microphone",
permissionStateJson(
granted = hasPermission(Manifest.permission.RECORD_AUDIO),
promptableWhenDenied = true,
),
)
put(
"location",
permissionStateJson(
granted =
hasPermission(Manifest.permission.ACCESS_FINE_LOCATION) ||
hasPermission(Manifest.permission.ACCESS_COARSE_LOCATION),
promptableWhenDenied = true,
),
)
put(
"sms",
buildJsonObject {
put(
"status",
JsonPrimitive(
if (hasAnySmsCapability(
smsEnabled,
canSendSms,
smsSendGranted,
smsReadGranted,
)
) {
"granted"
} else {
"denied"
},
),
)
put("promptable", JsonPrimitive(isSmsPromptable(smsEnabled, canSendSms, smsSendGranted, smsReadGranted)))
put(
"capabilities",
buildJsonObject {
put(
"send",
permissionStateJson(
granted = smsEnabled && smsSendGranted && canSendSms,
promptableWhenDenied = smsEnabled && canSendSms,
),
)
put(
"read",
permissionStateJson(
granted = smsEnabled && smsReadGranted && canSendSms,
promptableWhenDenied = smsEnabled && canSendSms,
),
)
},
)
},
)
put(
"notificationListener",
permissionStateJson(
granted = notificationAccess,
promptableWhenDenied = true,
),
)
put(
"notifications",
permissionStateJson(
granted = notificationsGranted,
promptableWhenDenied = true,
),
)
put(
"photos",
permissionStateJson(
granted = photosGranted,
promptableWhenDenied = photosEnabled,
),
)
put(
"contacts",
permissionStateJson(
granted =
hasPermission(Manifest.permission.READ_CONTACTS) &&
hasPermission(Manifest.permission.WRITE_CONTACTS),
promptableWhenDenied = true,
),
)
put(
"calendar",
permissionStateJson(
granted =
hasPermission(Manifest.permission.READ_CALENDAR) &&
hasPermission(Manifest.permission.WRITE_CALENDAR),
promptableWhenDenied = true,
),
)
put(
"callLog",
permissionStateJson(
granted = callLogEnabled && hasPermission(Manifest.permission.READ_CALL_LOG),
promptableWhenDenied = callLogEnabled,
),
)
put(
"motion",
permissionStateJson(
granted = motionGranted,
promptableWhenDenied = true,
),
)
},
)
}.toString()
}
private fun healthPayloadJson(): String {
val battery = readBatterySnapshot()
val batteryManager = appContext.getSystemService(BatteryManager::class.java)
val currentNowUa = batteryManager?.getLongProperty(BatteryManager.BATTERY_PROPERTY_CURRENT_NOW)
val currentNowMa =
if (currentNowUa == null || currentNowUa == Long.MIN_VALUE) {
null
} else {
// BatteryManager reports microamps; expose milliamps in the gateway payload.
currentNowUa.toDouble() / 1_000.0
}
val powerManager = appContext.getSystemService(PowerManager::class.java)
val activityManager = appContext.getSystemService(ActivityManager::class.java)
val memoryInfo = ActivityManager.MemoryInfo()
activityManager?.getMemoryInfo(memoryInfo)
val totalRamBytes = memoryInfo.totalMem.coerceAtLeast(0L)
val availableRamBytes = memoryInfo.availMem.coerceAtLeast(0L)
val usedRamBytes = (totalRamBytes - availableRamBytes).coerceAtLeast(0L)
val lowMemory = memoryInfo.lowMemory
val memoryPressure = mapMemoryPressure(totalRamBytes, availableRamBytes, lowMemory)
return buildJsonObject {
put(
"memory",
buildJsonObject {
put("pressure", JsonPrimitive(memoryPressure))
put("totalRamBytes", JsonPrimitive(totalRamBytes))
put("availableRamBytes", JsonPrimitive(availableRamBytes))
put("usedRamBytes", JsonPrimitive(usedRamBytes))
put("thresholdBytes", JsonPrimitive(memoryInfo.threshold.coerceAtLeast(0L)))
put("lowMemory", JsonPrimitive(lowMemory))
},
)
put(
"battery",
buildJsonObject {
put("state", JsonPrimitive(mapBatteryState(battery.status)))
put("chargingType", JsonPrimitive(mapChargingType(battery.plugged)))
battery.temperatureC?.let { put("temperatureC", JsonPrimitive(it)) }
currentNowMa?.let { put("currentMa", JsonPrimitive(it)) }
},
)
put(
"power",
buildJsonObject {
put("dozeModeEnabled", JsonPrimitive(powerManager?.isDeviceIdleMode == true))
put("lowPowerModeEnabled", JsonPrimitive(powerManager?.isPowerSaveMode == true))
},
)
put(
"system",
buildJsonObject {
Build.VERSION.SECURITY_PATCH
?.trim()
?.takeIf { it.isNotEmpty() }
?.let { put("securityPatchLevel", JsonPrimitive(it)) }
},
)
}.toString()
}
private fun parseDeviceAppsRequest(paramsJson: String?): DeviceAppsRequest {
val params = parseJsonParamsObject(paramsJson)
val includeSystem = parseJsonBooleanFlag(params, "includeSystem") ?: false
val includeDisabled = parseJsonBooleanFlag(params, "includeDisabled") ?: false
val includeNonLaunchable = parseJsonBooleanFlag(params, "includeNonLaunchable") ?: false
val query = parseJsonString(params, "query")?.trim()?.takeIf { it.isNotEmpty() }
val limit =
(parseJsonInt(params, "limit") ?: DEFAULT_DEVICE_APPS_LIMIT)
.coerceIn(1, MAX_DEVICE_APPS_LIMIT)
return DeviceAppsRequest(
includeSystem = includeSystem,
includeDisabled = includeDisabled,
includeNonLaunchable = includeNonLaunchable,
query = query,
limit = limit,
)
}
private fun readBatterySnapshot(): BatterySnapshot {
// ACTION_BATTERY_CHANGED is sticky; registerReceiver(null, ...) reads the last system snapshot.
val intent = appContext.registerReceiver(null, IntentFilter(Intent.ACTION_BATTERY_CHANGED))
val status =
intent?.getIntExtra(BatteryManager.EXTRA_STATUS, BatteryManager.BATTERY_STATUS_UNKNOWN)
?: BatteryManager.BATTERY_STATUS_UNKNOWN
val plugged = intent?.getIntExtra(BatteryManager.EXTRA_PLUGGED, 0) ?: 0
val temperatureC =
intent
?.getIntExtra(BatteryManager.EXTRA_TEMPERATURE, Int.MIN_VALUE)
?.takeIf { it != Int.MIN_VALUE }
?.toDouble()
?.div(10.0)
return BatterySnapshot(
status = status,
plugged = plugged,
levelFraction = batteryLevelFraction(intent),
temperatureC = temperatureC,
)
}
private fun batteryLevelFraction(intent: Intent?): Double? {
val rawLevel = intent?.getIntExtra(BatteryManager.EXTRA_LEVEL, -1) ?: -1
val rawScale = intent?.getIntExtra(BatteryManager.EXTRA_SCALE, -1) ?: -1
if (rawLevel < 0 || rawScale <= 0) return null
return rawLevel.toDouble() / rawScale.toDouble()
}
private fun mapBatteryState(status: Int): String =
when (status) {
BatteryManager.BATTERY_STATUS_CHARGING -> "charging"
BatteryManager.BATTERY_STATUS_FULL -> "full"
BatteryManager.BATTERY_STATUS_DISCHARGING, BatteryManager.BATTERY_STATUS_NOT_CHARGING -> "unplugged"
else -> "unknown"
}
private fun mapChargingType(plugged: Int): String =
when (plugged) {
BatteryManager.BATTERY_PLUGGED_AC -> "ac"
BatteryManager.BATTERY_PLUGGED_USB -> "usb"
BatteryManager.BATTERY_PLUGGED_WIRELESS -> "wireless"
BatteryManager.BATTERY_PLUGGED_DOCK -> "dock"
else -> "none"
}
private fun mapThermalState(powerManager: PowerManager?): String {
val thermal = powerManager?.currentThermalStatus ?: return "nominal"
return when (thermal) {
PowerManager.THERMAL_STATUS_NONE, PowerManager.THERMAL_STATUS_LIGHT -> "nominal"
PowerManager.THERMAL_STATUS_MODERATE -> "fair"
PowerManager.THERMAL_STATUS_SEVERE -> "serious"
PowerManager.THERMAL_STATUS_CRITICAL,
PowerManager.THERMAL_STATUS_EMERGENCY,
PowerManager.THERMAL_STATUS_SHUTDOWN,
-> "critical"
else -> "nominal"
}
}
private fun mapNetworkStatus(caps: NetworkCapabilities?): String {
if (caps == null) return "unsatisfied"
return when {
caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_VALIDATED) -> "satisfied"
// Internet without validation mirrors iOS "requiresConnection" for captive or unproven networks.
caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET) -> "requiresConnection"
else -> "unsatisfied"
}
}
private fun permissionStateJson(
granted: Boolean,
promptableWhenDenied: Boolean,
) = buildJsonObject {
put("status", JsonPrimitive(if (granted) "granted" else "denied"))
put("promptable", JsonPrimitive(!granted && promptableWhenDenied))
}
private fun hasPermission(permission: String): Boolean =
(
ContextCompat.checkSelfPermission(appContext, permission) == PackageManager.PERMISSION_GRANTED
)
private fun mapMemoryPressure(
totalBytes: Long,
availableBytes: Long,
lowMemory: Boolean,
): String {
if (totalBytes <= 0L) return if (lowMemory) "critical" else "unknown"
if (lowMemory) return "critical"
val freeRatio = availableBytes.toDouble() / totalBytes.toDouble()
// Thresholds intentionally mirror coarse OS health labels instead of exact memory pressure.
return when {
freeRatio <= 0.05 -> "critical"
freeRatio <= 0.15 -> "high"
freeRatio <= 0.30 -> "moderate"
else -> "normal"
}
}
private fun networkInterfacesJson(caps: NetworkCapabilities?) =
buildJsonArray {
if (caps == null) return@buildJsonArray
var hasKnownTransport = false
if (caps.hasTransport(NetworkCapabilities.TRANSPORT_WIFI)) {
hasKnownTransport = true
add(JsonPrimitive("wifi"))
}
if (caps.hasTransport(NetworkCapabilities.TRANSPORT_CELLULAR)) {
hasKnownTransport = true
add(JsonPrimitive("cellular"))
}
if (caps.hasTransport(NetworkCapabilities.TRANSPORT_ETHERNET)) {
hasKnownTransport = true
add(JsonPrimitive("wired"))
}
if (!hasKnownTransport) add(JsonPrimitive("other"))
}
}

View File

@@ -0,0 +1,517 @@
package ai.openclaw.app.node
import ai.openclaw.app.NotificationBurstLimiter
import ai.openclaw.app.SecurePrefs
import ai.openclaw.app.allowsPackage
import ai.openclaw.app.isWithinQuietHours
import android.app.Notification
import android.app.NotificationManager
import android.app.RemoteInput
import android.content.ComponentName
import android.content.Context
import android.content.Intent
import android.service.notification.NotificationListenerService
import android.service.notification.StatusBarNotification
import androidx.core.content.edit
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
private const val MAX_NOTIFICATION_TEXT_CHARS = 512
private const val NOTIFICATIONS_CHANGED_EVENT = "notifications.changed"
/**
* Trims notification text and caps payload size before it enters gateway-visible state.
*/
internal fun sanitizeNotificationText(value: CharSequence?): String? {
val normalized = value?.toString()?.trim().orEmpty()
// Notification extras can include long previews; cap before sending over node events.
return normalized.take(MAX_NOTIFICATION_TEXT_CHARS).ifEmpty { null }
}
/**
* Stable notification snapshot entry exposed through the Android notifications command.
*/
data class DeviceNotificationEntry(
val key: String,
val packageName: String,
val title: String?,
val text: String?,
val subText: String?,
val category: String?,
val channelId: String?,
val postTimeMs: Long,
val isOngoing: Boolean,
val isClearable: Boolean,
)
internal fun DeviceNotificationEntry.toJsonObject(): JsonObject =
buildJsonObject {
put("key", JsonPrimitive(key))
put("packageName", JsonPrimitive(packageName))
put("postTimeMs", JsonPrimitive(postTimeMs))
put("isOngoing", JsonPrimitive(isOngoing))
put("isClearable", JsonPrimitive(isClearable))
title?.let { put("title", JsonPrimitive(it)) }
text?.let { put("text", JsonPrimitive(it)) }
subText?.let { put("subText", JsonPrimitive(it)) }
category?.let { put("category", JsonPrimitive(it)) }
channelId?.let { put("channelId", JsonPrimitive(it)) }
}
/**
* Listener state exposed to the gateway, including whether Android has connected the service.
*/
data class DeviceNotificationSnapshot(
val enabled: Boolean,
val connected: Boolean,
val notifications: List<DeviceNotificationEntry>,
)
/**
* Gateway-supported notification actions mapped to Android listener operations.
*/
enum class NotificationActionKind {
Open,
Dismiss,
Reply,
}
/**
* Gateway action request; [key] must match Android's StatusBarNotification key.
*/
data class NotificationActionRequest(
val key: String,
val kind: NotificationActionKind,
val replyText: String? = null,
)
/**
* Normalized notification action result returned through node.invoke.
*/
data class NotificationActionResult(
val ok: Boolean,
val code: String? = null,
val message: String? = null,
)
internal fun actionRequiresClearableNotification(kind: NotificationActionKind): Boolean = kind == NotificationActionKind.Dismiss
/**
* Process-local cache of active notifications mirrored from Android listener callbacks.
*/
private object DeviceNotificationStore {
private val lock = Any()
private var connected = false
private val byKey = LinkedHashMap<String, DeviceNotificationEntry>()
fun replace(entries: List<DeviceNotificationEntry>) {
synchronized(lock) {
byKey.clear()
for (entry in entries) {
byKey[entry.key] = entry
}
}
}
fun upsert(entry: DeviceNotificationEntry) {
synchronized(lock) {
byKey[entry.key] = entry
}
}
fun remove(key: String) {
synchronized(lock) {
byKey.remove(key)
}
}
fun setConnected(value: Boolean) {
synchronized(lock) {
connected = value
if (!value) {
// Android invalidates activeNotifications when the listener disconnects.
byKey.clear()
}
}
}
fun snapshot(enabled: Boolean): DeviceNotificationSnapshot {
val (isConnected, entries) =
synchronized(lock) {
connected to byKey.values.sortedByDescending { it.postTimeMs }
}
return DeviceNotificationSnapshot(
enabled = enabled,
connected = isConnected,
notifications = entries,
)
}
}
/**
* Android notification listener that mirrors notification state and executes gateway actions.
*/
class DeviceNotificationListenerService : NotificationListenerService() {
private val securePrefs by lazy { SecurePrefs(applicationContext) }
private val forwardingLimiter = NotificationBurstLimiter()
override fun onListenerConnected() {
super.onListenerConnected()
activeService = this
DeviceNotificationStore.setConnected(true)
refreshActiveNotifications()
}
override fun onListenerDisconnected() {
if (activeService === this) {
activeService = null
}
DeviceNotificationStore.setConnected(false)
super.onListenerDisconnected()
}
override fun onDestroy() {
if (activeService === this) {
activeService = null
}
super.onDestroy()
}
override fun onNotificationPosted(sbn: StatusBarNotification?) {
super.onNotificationPosted(sbn)
val entry = sbn?.toEntry() ?: return
DeviceNotificationStore.upsert(entry)
rememberRecentPackage(entry.packageName)
if (entry.packageName == packageName) {
return
}
val payload = notificationChangedPayload(entry) ?: return
emitNotificationsChanged(payload)
}
override fun onNotificationRemoved(sbn: StatusBarNotification?) {
super.onNotificationRemoved(sbn)
val removed = sbn ?: return
val key = removed.key.trim()
if (key.isEmpty()) {
return
}
DeviceNotificationStore.remove(key)
rememberRecentPackage(removed.packageName)
if (removed.packageName == packageName) {
return
}
val packageName = removed.packageName.trim()
val payload =
notificationChangedPayload(
entry = null,
change = "removed",
key = key,
packageName = packageName,
postTimeMs = removed.postTime,
isOngoing = removed.isOngoing,
isClearable = removed.isClearable,
) ?: return
emitNotificationsChanged(payload)
}
private fun notificationChangedPayload(entry: DeviceNotificationEntry): String? =
notificationChangedPayload(
entry = entry,
change = "posted",
key = entry.key,
packageName = entry.packageName,
postTimeMs = entry.postTimeMs,
isOngoing = entry.isOngoing,
isClearable = entry.isClearable,
)
private fun notificationChangedPayload(
entry: DeviceNotificationEntry?,
change: String,
key: String,
packageName: String,
postTimeMs: Long,
isOngoing: Boolean,
isClearable: Boolean,
): String? {
val normalizedPackage = packageName.trim()
if (normalizedPackage.isEmpty()) {
return null
}
val policy = securePrefs.getNotificationForwardingPolicy(appPackageName = this.packageName)
if (!policy.enabled) {
return null
}
if (!policy.allowsPackage(normalizedPackage)) {
return null
}
val nowEpochMs = System.currentTimeMillis()
if (policy.isWithinQuietHours(nowEpochMs = nowEpochMs)) {
return null
}
// Apply burst limits after package/quiet-hour filters so blocked notifications do not consume quota.
if (!forwardingLimiter.allow(nowEpochMs, policy.maxEventsPerMinute)) {
return null
}
return buildJsonObject {
put("change", JsonPrimitive(change))
put("key", JsonPrimitive(key))
put("packageName", JsonPrimitive(normalizedPackage))
put("postTimeMs", JsonPrimitive(postTimeMs))
put("isOngoing", JsonPrimitive(isOngoing))
put("isClearable", JsonPrimitive(isClearable))
policy.sessionKey?.let { put("sessionKey", JsonPrimitive(it)) }
entry?.title?.let { put("title", JsonPrimitive(it)) }
entry?.text?.let { put("text", JsonPrimitive(it)) }
entry?.subText?.let { put("subText", JsonPrimitive(it)) }
entry?.category?.let { put("category", JsonPrimitive(it)) }
entry?.channelId?.let { put("channelId", JsonPrimitive(it)) }
}.toString()
}
private fun refreshActiveNotifications() {
val entries =
runCatching {
activeNotifications
?.mapNotNull { it.toEntry() }
?: emptyList()
}.getOrElse { emptyList() }
DeviceNotificationStore.replace(entries)
}
private fun StatusBarNotification.toEntry(): DeviceNotificationEntry {
val extras = notification.extras
val keyValue = key.takeIf { it.isNotBlank() } ?: "$packageName:$id:$postTime"
val title = sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_TITLE))
val body =
sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_BIG_TEXT))
?: sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_TEXT))
val subText = sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_SUB_TEXT))
return DeviceNotificationEntry(
key = keyValue,
packageName = packageName,
title = title,
text = body,
subText = subText,
category = notification.category?.trim()?.ifEmpty { null },
channelId = notification.channelId?.trim()?.ifEmpty { null },
postTimeMs = postTime,
isOngoing = isOngoing,
isClearable = isClearable,
)
}
companion object {
private const val recentPackagesPref = "notifications.forwarding.recentPackages"
private const val legacyRecentPackagesPref = "notifications.recentPackages"
private const val recentPackagesLimit = 64
@Volatile private var activeService: DeviceNotificationListenerService? = null
@Volatile private var nodeEventSink: ((event: String, payloadJson: String?) -> Unit)? = null
private fun serviceComponent(context: Context): ComponentName = ComponentName(context, DeviceNotificationListenerService::class.java)
/** Installs the node event sink used to emit filtered notification change events. */
fun setNodeEventSink(sink: ((event: String, payloadJson: String?) -> Unit)?) {
nodeEventSink = sink
}
private fun recentPackagesPrefs(context: Context) = context.applicationContext.getSharedPreferences("openclaw.secure", Context.MODE_PRIVATE)
private fun migrateLegacyRecentPackagesIfNeeded(context: Context) {
val prefs = recentPackagesPrefs(context)
val hasNew = prefs.contains(recentPackagesPref)
val legacy = prefs.getString(legacyRecentPackagesPref, null)?.trim().orEmpty()
if (!hasNew && legacy.isNotEmpty()) {
// Keep recent package suggestions across the preference-key rename.
prefs.edit {
putString(recentPackagesPref, legacy)
remove(legacyRecentPackagesPref)
}
} else if (hasNew && prefs.contains(legacyRecentPackagesPref)) {
prefs.edit { remove(legacyRecentPackagesPref) }
}
}
/** Returns recent third-party packages seen by the listener for settings suggestions. */
fun recentPackages(context: Context): List<String> {
migrateLegacyRecentPackagesIfNeeded(context)
val prefs = recentPackagesPrefs(context)
val stored = prefs.getString(recentPackagesPref, null).orEmpty()
return stored
.split(',')
.map { it.trim() }
.filter { it.isNotEmpty() }
.distinct()
}
/** Checks whether Android has granted listener access to this service component. */
fun isAccessEnabled(context: Context): Boolean {
val manager = context.getSystemService(NotificationManager::class.java) ?: return false
return manager.isNotificationListenerAccessGranted(serviceComponent(context))
}
/** Reads the current mirrored notification snapshot without forcing service startup. */
fun snapshot(
context: Context,
enabled: Boolean = isAccessEnabled(context),
): DeviceNotificationSnapshot = DeviceNotificationStore.snapshot(enabled = enabled)
/** Asks Android to rebind the listener after settings grant access but callbacks have not arrived. */
fun requestServiceRebind(context: Context) {
runCatching {
NotificationListenerService.requestRebind(serviceComponent(context))
}
}
/** Executes an open, dismiss, or reply action through the active listener instance. */
fun executeAction(
context: Context,
request: NotificationActionRequest,
): NotificationActionResult {
if (!isAccessEnabled(context)) {
return NotificationActionResult(
ok = false,
code = "NOTIFICATIONS_DISABLED",
message = "NOTIFICATIONS_DISABLED: enable notification access in system Settings",
)
}
val service =
activeService
?: return NotificationActionResult(
ok = false,
code = "NOTIFICATIONS_UNAVAILABLE",
message = "NOTIFICATIONS_UNAVAILABLE: notification listener not connected",
)
return service.executeActionInternal(request)
}
private fun emitNotificationsChanged(payloadJson: String) {
runCatching {
nodeEventSink?.invoke(NOTIFICATIONS_CHANGED_EVENT, payloadJson)
}
}
private fun rememberRecentPackage(packageName: String?) {
val service = activeService ?: return
val normalized = packageName?.trim().orEmpty()
if (normalized.isEmpty() || normalized == service.packageName) return
migrateLegacyRecentPackagesIfNeeded(service.applicationContext)
val prefs = recentPackagesPrefs(service.applicationContext)
val existing =
prefs
.getString(recentPackagesPref, null)
.orEmpty()
.split(',')
.map { it.trim() }
.filter { it.isNotEmpty() && it != normalized }
.take(recentPackagesLimit - 1)
// Most recent package first keeps settings suggestions useful without storing notification content.
val updated = listOf(normalized) + existing
prefs.edit { putString(recentPackagesPref, updated.joinToString(",")) }
}
}
private fun executeActionInternal(request: NotificationActionRequest): NotificationActionResult {
val sbn =
activeNotifications
?.firstOrNull { it.key == request.key }
?: return NotificationActionResult(
ok = false,
code = "NOTIFICATION_NOT_FOUND",
message = "NOTIFICATION_NOT_FOUND: notification key not found",
)
if (actionRequiresClearableNotification(request.kind) && !sbn.isClearable) {
return NotificationActionResult(
ok = false,
code = "NOTIFICATION_NOT_CLEARABLE",
message = "NOTIFICATION_NOT_CLEARABLE: notification is ongoing or protected",
)
}
return when (request.kind) {
NotificationActionKind.Open -> {
val pendingIntent =
sbn.notification.contentIntent
?: return NotificationActionResult(
ok = false,
code = "ACTION_UNAVAILABLE",
message = "ACTION_UNAVAILABLE: notification has no open action",
)
runCatching {
pendingIntent.send()
}.fold(
onSuccess = { NotificationActionResult(ok = true) },
onFailure = { err ->
NotificationActionResult(
ok = false,
code = "ACTION_FAILED",
message = "ACTION_FAILED: ${err.message ?: "open failed"}",
)
},
)
}
NotificationActionKind.Dismiss -> {
runCatching {
cancelNotification(sbn.key)
DeviceNotificationStore.remove(sbn.key)
}.fold(
onSuccess = { NotificationActionResult(ok = true) },
onFailure = { err ->
NotificationActionResult(
ok = false,
code = "ACTION_FAILED",
message = "ACTION_FAILED: ${err.message ?: "dismiss failed"}",
)
},
)
}
NotificationActionKind.Reply -> {
val replyText = request.replyText?.trim().orEmpty()
if (replyText.isEmpty()) {
return NotificationActionResult(
ok = false,
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: replyText required for reply action",
)
}
val action =
sbn.notification.actions
?.firstOrNull { candidate ->
// Android reply actions are identified by RemoteInput, not by a stable action title.
candidate.actionIntent != null && !candidate.remoteInputs.isNullOrEmpty()
}
?: return NotificationActionResult(
ok = false,
code = "ACTION_UNAVAILABLE",
message = "ACTION_UNAVAILABLE: notification has no reply action",
)
val remoteInputs = action.remoteInputs ?: emptyArray()
val fillInIntent = Intent()
val replyBundle = android.os.Bundle()
for (remoteInput in remoteInputs) {
replyBundle.putCharSequence(remoteInput.resultKey, replyText)
}
RemoteInput.addResultsToIntent(remoteInputs, fillInIntent, replyBundle)
runCatching {
action.actionIntent.send(this, 0, fillInIntent)
}.fold(
onSuccess = { NotificationActionResult(ok = true) },
onFailure = { err ->
NotificationActionResult(
ok = false,
code = "ACTION_FAILED",
message = "ACTION_FAILED: ${err.message ?: "reply failed"}",
)
},
)
}
}
}
}

View File

@@ -0,0 +1,78 @@
package ai.openclaw.app.node
import ai.openclaw.app.SecurePrefs
import ai.openclaw.app.gateway.GatewaySession
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
/**
* Handles gateway-originated events that need to update local Android preferences.
*/
class GatewayEventHandler(
private val scope: CoroutineScope,
private val prefs: SecurePrefs,
private val json: Json,
private val operatorSession: GatewaySession,
private val isConnected: () -> Boolean,
) {
private var suppressWakeWordsSync = false
private var wakeWordsSyncJob: Job? = null
/** Applies gateway wake words locally without echoing the same change back to the gateway. */
fun applyWakeWordsFromGateway(words: List<String>) {
suppressWakeWordsSync = true
prefs.setWakeWords(words)
suppressWakeWordsSync = false
}
/** Debounces local wake-word edits before sending voicewake.set to the operator session. */
fun scheduleWakeWordsSyncIfNeeded() {
if (suppressWakeWordsSync) return
if (!isConnected()) return
val snapshot = prefs.wakeWords.value
wakeWordsSyncJob?.cancel()
wakeWordsSyncJob =
scope.launch {
delay(650)
val jsonList = snapshot.joinToString(separator = ",") { it.toJsonString() }
val params = """{"triggers":[$jsonList]}"""
try {
operatorSession.request("voicewake.set", params)
} catch (_: Throwable) {
// ignore
}
}
}
/** Loads gateway wake words on connect so Android settings show server truth. */
suspend fun refreshWakeWordsFromGateway() {
if (!isConnected()) return
try {
val res = operatorSession.request("voicewake.get", "{}")
val payload = json.parseToJsonElement(res).asObjectOrNull() ?: return
val array = payload["triggers"] as? JsonArray ?: return
val triggers = array.mapNotNull { it.asStringOrNull() }
applyWakeWordsFromGateway(triggers)
} catch (_: Throwable) {
// ignore
}
}
/** Applies voicewake.changed event payloads emitted by the gateway. */
fun handleVoiceWakeChangedEvent(payloadJson: String?) {
if (payloadJson.isNullOrBlank()) return
try {
val payload = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: return
val array = payload["triggers"] as? JsonArray ?: return
val triggers = array.mapNotNull { it.asStringOrNull() }
applyWakeWordsFromGateway(triggers)
} catch (_: Throwable) {
// ignore
}
}
}

View File

@@ -0,0 +1,295 @@
package ai.openclaw.app.node
import ai.openclaw.app.protocol.OpenClawCalendarCommand
import ai.openclaw.app.protocol.OpenClawCallLogCommand
import ai.openclaw.app.protocol.OpenClawCameraCommand
import ai.openclaw.app.protocol.OpenClawCanvasA2UICommand
import ai.openclaw.app.protocol.OpenClawCanvasCommand
import ai.openclaw.app.protocol.OpenClawCapability
import ai.openclaw.app.protocol.OpenClawContactsCommand
import ai.openclaw.app.protocol.OpenClawDeviceCommand
import ai.openclaw.app.protocol.OpenClawLocationCommand
import ai.openclaw.app.protocol.OpenClawMotionCommand
import ai.openclaw.app.protocol.OpenClawNotificationsCommand
import ai.openclaw.app.protocol.OpenClawPhotosCommand
import ai.openclaw.app.protocol.OpenClawSmsCommand
import ai.openclaw.app.protocol.OpenClawSystemCommand
import ai.openclaw.app.protocol.OpenClawTalkCommand
/** Runtime feature flags used to decide which node tools are advertised. */
data class NodeRuntimeFlags(
val cameraEnabled: Boolean,
val locationEnabled: Boolean,
val sendSmsAvailable: Boolean,
val readSmsAvailable: Boolean,
val smsSearchPossible: Boolean,
val callLogAvailable: Boolean,
val photosAvailable: Boolean,
val voiceWakeEnabled: Boolean,
val motionActivityAvailable: Boolean,
val motionPedometerAvailable: Boolean,
val installedAppsSharingEnabled: Boolean,
val debugBuild: Boolean,
)
/** Per-command availability gates checked before advertising invoke methods. */
enum class InvokeCommandAvailability {
Always,
CameraEnabled,
LocationEnabled,
SendSmsAvailable,
ReadSmsAvailable,
RequestableSmsSearchAvailable,
CallLogAvailable,
PhotosAvailable,
MotionActivityAvailable,
MotionPedometerAvailable,
InstalledAppsSharingEnabled,
DebugBuild,
}
/** Per-capability availability gates for the node capabilities manifest. */
enum class NodeCapabilityAvailability {
Always,
CameraEnabled,
LocationEnabled,
SmsAvailable,
CallLogAvailable,
PhotosAvailable,
VoiceWakeEnabled,
MotionAvailable,
}
/** Capability entry reported to the gateway when its availability gate passes. */
data class NodeCapabilitySpec(
val name: String,
val availability: NodeCapabilityAvailability = NodeCapabilityAvailability.Always,
)
/** Invoke method entry advertised to gateway plus foreground routing metadata. */
data class InvokeCommandSpec(
val name: String,
val requiresForeground: Boolean = false,
val availability: InvokeCommandAvailability = InvokeCommandAvailability.Always,
)
object InvokeCommandRegistry {
/** Capabilities mirror gateway protocol ids and are filtered by device state. */
val capabilityManifest: List<NodeCapabilitySpec> =
listOf(
NodeCapabilitySpec(name = OpenClawCapability.Canvas.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.Device.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.Notifications.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.System.rawValue),
NodeCapabilitySpec(
name = OpenClawCapability.Camera.rawValue,
availability = NodeCapabilityAvailability.CameraEnabled,
),
NodeCapabilitySpec(
name = OpenClawCapability.Sms.rawValue,
availability = NodeCapabilityAvailability.SmsAvailable,
),
NodeCapabilitySpec(
name = OpenClawCapability.VoiceWake.rawValue,
availability = NodeCapabilityAvailability.VoiceWakeEnabled,
),
NodeCapabilitySpec(name = OpenClawCapability.Talk.rawValue),
NodeCapabilitySpec(
name = OpenClawCapability.Location.rawValue,
availability = NodeCapabilityAvailability.LocationEnabled,
),
NodeCapabilitySpec(
name = OpenClawCapability.Photos.rawValue,
availability = NodeCapabilityAvailability.PhotosAvailable,
),
NodeCapabilitySpec(name = OpenClawCapability.Contacts.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.Calendar.rawValue),
NodeCapabilitySpec(
name = OpenClawCapability.Motion.rawValue,
availability = NodeCapabilityAvailability.MotionAvailable,
),
NodeCapabilitySpec(
name = OpenClawCapability.CallLog.rawValue,
availability = NodeCapabilityAvailability.CallLogAvailable,
),
)
/** Complete Android node command catalog before runtime availability filtering. */
val all: List<InvokeCommandSpec> =
listOf(
InvokeCommandSpec(
name = OpenClawCanvasCommand.Present.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Hide.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Navigate.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Eval.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Snapshot.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasA2UICommand.Push.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasA2UICommand.PushJSONL.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasA2UICommand.Reset.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawSystemCommand.Notify.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttStart.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttStop.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttCancel.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttOnce.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCameraCommand.List.rawValue,
requiresForeground = true,
availability = InvokeCommandAvailability.CameraEnabled,
),
InvokeCommandSpec(
name = OpenClawCameraCommand.Snap.rawValue,
requiresForeground = true,
availability = InvokeCommandAvailability.CameraEnabled,
),
InvokeCommandSpec(
name = OpenClawCameraCommand.Clip.rawValue,
requiresForeground = true,
availability = InvokeCommandAvailability.CameraEnabled,
),
InvokeCommandSpec(
name = OpenClawLocationCommand.Get.rawValue,
availability = InvokeCommandAvailability.LocationEnabled,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Status.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Info.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Permissions.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Health.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Apps.rawValue,
availability = InvokeCommandAvailability.InstalledAppsSharingEnabled,
),
InvokeCommandSpec(
name = OpenClawNotificationsCommand.List.rawValue,
),
InvokeCommandSpec(
name = OpenClawNotificationsCommand.Actions.rawValue,
),
InvokeCommandSpec(
name = OpenClawPhotosCommand.Latest.rawValue,
availability = InvokeCommandAvailability.PhotosAvailable,
),
InvokeCommandSpec(
name = OpenClawContactsCommand.Search.rawValue,
),
InvokeCommandSpec(
name = OpenClawContactsCommand.Add.rawValue,
),
InvokeCommandSpec(
name = OpenClawCalendarCommand.Events.rawValue,
),
InvokeCommandSpec(
name = OpenClawCalendarCommand.Add.rawValue,
),
InvokeCommandSpec(
name = OpenClawMotionCommand.Activity.rawValue,
availability = InvokeCommandAvailability.MotionActivityAvailable,
),
InvokeCommandSpec(
name = OpenClawMotionCommand.Pedometer.rawValue,
availability = InvokeCommandAvailability.MotionPedometerAvailable,
),
InvokeCommandSpec(
name = OpenClawSmsCommand.Send.rawValue,
availability = InvokeCommandAvailability.SendSmsAvailable,
),
InvokeCommandSpec(
name = OpenClawSmsCommand.Search.rawValue,
availability = InvokeCommandAvailability.RequestableSmsSearchAvailable,
),
InvokeCommandSpec(
name = OpenClawCallLogCommand.Search.rawValue,
availability = InvokeCommandAvailability.CallLogAvailable,
),
InvokeCommandSpec(
name = "debug.logs",
availability = InvokeCommandAvailability.DebugBuild,
),
InvokeCommandSpec(
name = "debug.ed25519",
availability = InvokeCommandAvailability.DebugBuild,
),
)
private val byNameInternal: Map<String, InvokeCommandSpec> = all.associateBy { it.name }
/** Finds the command metadata used by dispatch and advertised-method builders. */
fun find(command: String): InvokeCommandSpec? = byNameInternal[command]
/** Returns gateway capability ids the current Android device can actually serve. */
fun advertisedCapabilities(flags: NodeRuntimeFlags): List<String> =
capabilityManifest
.filter { spec ->
when (spec.availability) {
NodeCapabilityAvailability.Always -> true
NodeCapabilityAvailability.CameraEnabled -> flags.cameraEnabled
NodeCapabilityAvailability.LocationEnabled -> flags.locationEnabled
NodeCapabilityAvailability.SmsAvailable -> flags.sendSmsAvailable || flags.readSmsAvailable
NodeCapabilityAvailability.CallLogAvailable -> flags.callLogAvailable
NodeCapabilityAvailability.PhotosAvailable -> flags.photosAvailable
NodeCapabilityAvailability.VoiceWakeEnabled -> flags.voiceWakeEnabled
NodeCapabilityAvailability.MotionAvailable -> flags.motionActivityAvailable || flags.motionPedometerAvailable
}
}.map { it.name }
/** Returns gateway invoke method ids available under current permissions/build flags. */
fun advertisedCommands(flags: NodeRuntimeFlags): List<String> =
all
.filter { spec ->
when (spec.availability) {
InvokeCommandAvailability.Always -> true
InvokeCommandAvailability.CameraEnabled -> flags.cameraEnabled
InvokeCommandAvailability.LocationEnabled -> flags.locationEnabled
InvokeCommandAvailability.SendSmsAvailable -> flags.sendSmsAvailable
InvokeCommandAvailability.ReadSmsAvailable -> flags.readSmsAvailable
InvokeCommandAvailability.RequestableSmsSearchAvailable -> flags.smsSearchPossible
InvokeCommandAvailability.CallLogAvailable -> flags.callLogAvailable
InvokeCommandAvailability.PhotosAvailable -> flags.photosAvailable
InvokeCommandAvailability.MotionActivityAvailable -> flags.motionActivityAvailable
InvokeCommandAvailability.MotionPedometerAvailable -> flags.motionPedometerAvailable
InvokeCommandAvailability.InstalledAppsSharingEnabled -> flags.installedAppsSharingEnabled
InvokeCommandAvailability.DebugBuild -> flags.debugBuild
}
}.map { it.name }
}

View File

@@ -0,0 +1,375 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import ai.openclaw.app.protocol.OpenClawCalendarCommand
import ai.openclaw.app.protocol.OpenClawCallLogCommand
import ai.openclaw.app.protocol.OpenClawCameraCommand
import ai.openclaw.app.protocol.OpenClawCanvasA2UICommand
import ai.openclaw.app.protocol.OpenClawCanvasCommand
import ai.openclaw.app.protocol.OpenClawContactsCommand
import ai.openclaw.app.protocol.OpenClawDeviceCommand
import ai.openclaw.app.protocol.OpenClawLocationCommand
import ai.openclaw.app.protocol.OpenClawMotionCommand
import ai.openclaw.app.protocol.OpenClawNotificationsCommand
import ai.openclaw.app.protocol.OpenClawSmsCommand
import ai.openclaw.app.protocol.OpenClawSystemCommand
import ai.openclaw.app.protocol.OpenClawTalkCommand
/** Runtime state for SMS search, split so permission prompts are not reported as hard unavailability. */
internal enum class SmsSearchAvailabilityReason {
Available,
PermissionRequired,
Unavailable,
}
/**
* Distinguish permanent SMS search unavailability from permission-gated search.
*/
internal fun classifySmsSearchAvailability(
readSmsAvailable: Boolean,
smsFeatureEnabled: Boolean,
smsTelephonyAvailable: Boolean,
): SmsSearchAvailabilityReason {
if (readSmsAvailable) return SmsSearchAvailabilityReason.Available
if (!smsFeatureEnabled || !smsTelephonyAvailable) return SmsSearchAvailabilityReason.Unavailable
return SmsSearchAvailabilityReason.PermissionRequired
}
internal fun smsSearchAvailabilityError(
readSmsAvailable: Boolean,
smsFeatureEnabled: Boolean,
smsTelephonyAvailable: Boolean,
): GatewaySession.InvokeResult? =
when (
classifySmsSearchAvailability(
readSmsAvailable = readSmsAvailable,
smsFeatureEnabled = smsFeatureEnabled,
smsTelephonyAvailable = smsTelephonyAvailable,
)
) {
SmsSearchAvailabilityReason.Available,
SmsSearchAvailabilityReason.PermissionRequired,
-> null
SmsSearchAvailabilityReason.Unavailable ->
GatewaySession.InvokeResult.error(
code = "SMS_UNAVAILABLE",
message = "SMS_UNAVAILABLE: SMS not available on this device",
)
}
/**
* Gateway node.invoke command router for Android-owned capabilities.
*/
class InvokeDispatcher(
private val canvas: CanvasController,
private val cameraHandler: CameraHandler,
private val locationHandler: LocationHandler,
private val deviceHandler: DeviceHandler,
private val notificationsHandler: NotificationsHandler,
private val systemHandler: SystemHandler,
private val talkHandler: TalkHandler,
private val photosHandler: PhotosHandler,
private val contactsHandler: ContactsHandler,
private val calendarHandler: CalendarHandler,
private val motionHandler: MotionHandler,
private val smsHandler: SmsHandler,
private val a2uiHandler: A2UIHandler,
private val debugHandler: DebugHandler,
private val callLogHandler: CallLogHandler,
private val isForeground: () -> Boolean,
private val cameraEnabled: () -> Boolean,
private val locationEnabled: () -> Boolean,
private val sendSmsAvailable: () -> Boolean,
private val readSmsAvailable: () -> Boolean,
private val smsFeatureEnabled: () -> Boolean,
private val smsTelephonyAvailable: () -> Boolean,
private val callLogAvailable: () -> Boolean,
private val photosAvailable: () -> Boolean,
private val installedAppsSharingEnabled: () -> Boolean,
private val debugBuild: () -> Boolean,
private val onCanvasA2uiPush: () -> Unit,
private val onCanvasA2uiReset: () -> Unit,
private val motionActivityAvailable: () -> Boolean,
private val motionPedometerAvailable: () -> Boolean,
) {
/** Dispatches one gateway node.invoke command after foreground and availability gates pass. */
suspend fun handleInvoke(
command: String,
paramsJson: String?,
): GatewaySession.InvokeResult {
val spec =
InvokeCommandRegistry.find(command)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: unknown command",
)
if (spec.requiresForeground && !isForeground()) {
// Foreground-only commands need an active Activity surface before touching UI or capture APIs.
return GatewaySession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: command requires foreground",
)
}
availabilityError(spec.availability)?.let { return it }
// Command strings come from OpenClawProtocolConstants; the registry above owns advertised availability.
return when (command) {
// Canvas commands
OpenClawCanvasCommand.Present.rawValue -> {
val url = CanvasController.parseNavigateUrl(paramsJson)
canvas.navigate(url)
GatewaySession.InvokeResult.ok(null)
}
OpenClawCanvasCommand.Hide.rawValue -> GatewaySession.InvokeResult.ok(null)
OpenClawCanvasCommand.Navigate.rawValue -> {
val url = CanvasController.parseNavigateUrl(paramsJson)
canvas.navigate(url)
GatewaySession.InvokeResult.ok(null)
}
OpenClawCanvasCommand.Eval.rawValue -> {
val js =
CanvasController.parseEvalJs(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: javaScript required",
)
withCanvasAvailable {
val result = canvas.eval(js)
GatewaySession.InvokeResult.ok("""{"result":${result.toJsonString()}}""")
}
}
OpenClawCanvasCommand.Snapshot.rawValue -> {
val snapshotParams = CanvasController.parseSnapshotParams(paramsJson)
withCanvasAvailable {
val base64 =
canvas.snapshotBase64(
format = snapshotParams.format,
quality = snapshotParams.quality,
maxWidth = snapshotParams.maxWidth,
)
GatewaySession.InvokeResult.ok("""{"format":"${snapshotParams.format.rawValue}","base64":"$base64"}""")
}
}
// A2UI commands
OpenClawCanvasA2UICommand.Reset.rawValue ->
withReadyA2ui {
withCanvasAvailable {
val res = canvas.eval(A2UIHandler.a2uiResetJS)
onCanvasA2uiReset()
GatewaySession.InvokeResult.ok(res)
}
}
OpenClawCanvasA2UICommand.Push.rawValue, OpenClawCanvasA2UICommand.PushJSONL.rawValue -> {
val messages =
try {
a2uiHandler.decodeA2uiMessages(command, paramsJson)
} catch (err: Throwable) {
return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = err.message ?: "invalid A2UI payload",
)
}
withReadyA2ui {
withCanvasAvailable {
val js = A2UIHandler.a2uiApplyMessagesJS(messages)
val res = canvas.eval(js)
onCanvasA2uiPush()
GatewaySession.InvokeResult.ok(res)
}
}
}
// Camera commands
OpenClawCameraCommand.List.rawValue -> cameraHandler.handleList(paramsJson)
OpenClawCameraCommand.Snap.rawValue -> cameraHandler.handleSnap(paramsJson)
OpenClawCameraCommand.Clip.rawValue -> cameraHandler.handleClip(paramsJson)
// Location command
OpenClawLocationCommand.Get.rawValue -> locationHandler.handleLocationGet(paramsJson)
// Device commands
OpenClawDeviceCommand.Status.rawValue -> deviceHandler.handleDeviceStatus(paramsJson)
OpenClawDeviceCommand.Info.rawValue -> deviceHandler.handleDeviceInfo(paramsJson)
OpenClawDeviceCommand.Permissions.rawValue -> deviceHandler.handleDevicePermissions(paramsJson)
OpenClawDeviceCommand.Health.rawValue -> deviceHandler.handleDeviceHealth(paramsJson)
OpenClawDeviceCommand.Apps.rawValue -> deviceHandler.handleDeviceApps(paramsJson)
// Notifications command
OpenClawNotificationsCommand.List.rawValue -> notificationsHandler.handleNotificationsList(paramsJson)
OpenClawNotificationsCommand.Actions.rawValue -> notificationsHandler.handleNotificationsActions(paramsJson)
// System command
OpenClawSystemCommand.Notify.rawValue -> systemHandler.handleSystemNotify(paramsJson)
// Talk commands
OpenClawTalkCommand.PttStart.rawValue -> talkHandler.handlePttStart(paramsJson)
OpenClawTalkCommand.PttStop.rawValue -> talkHandler.handlePttStop(paramsJson)
OpenClawTalkCommand.PttCancel.rawValue -> talkHandler.handlePttCancel(paramsJson)
OpenClawTalkCommand.PttOnce.rawValue -> talkHandler.handlePttOnce(paramsJson)
// Photos command
ai.openclaw.app.protocol.OpenClawPhotosCommand.Latest.rawValue ->
photosHandler.handlePhotosLatest(
paramsJson,
)
// Contacts command
OpenClawContactsCommand.Search.rawValue -> contactsHandler.handleContactsSearch(paramsJson)
OpenClawContactsCommand.Add.rawValue -> contactsHandler.handleContactsAdd(paramsJson)
// Calendar command
OpenClawCalendarCommand.Events.rawValue -> calendarHandler.handleCalendarEvents(paramsJson)
OpenClawCalendarCommand.Add.rawValue -> calendarHandler.handleCalendarAdd(paramsJson)
// Motion command
OpenClawMotionCommand.Activity.rawValue -> motionHandler.handleMotionActivity(paramsJson)
OpenClawMotionCommand.Pedometer.rawValue -> motionHandler.handleMotionPedometer(paramsJson)
// SMS command
OpenClawSmsCommand.Send.rawValue -> smsHandler.handleSmsSend(paramsJson)
OpenClawSmsCommand.Search.rawValue -> smsHandler.handleSmsSearch(paramsJson)
// CallLog command
OpenClawCallLogCommand.Search.rawValue -> callLogHandler.handleCallLogSearch(paramsJson)
// Debug commands
"debug.ed25519" -> debugHandler.handleEd25519()
"debug.logs" -> debugHandler.handleLogs()
else -> GatewaySession.InvokeResult.error(code = "INVALID_REQUEST", message = "INVALID_REQUEST: unknown command")
}
}
private suspend fun withReadyA2ui(block: suspend () -> GatewaySession.InvokeResult): GatewaySession.InvokeResult {
if (!a2uiHandler.ensureA2uiReady()) {
return GatewaySession.InvokeResult.error(
code = "A2UI_HOST_UNAVAILABLE",
message = "A2UI_HOST_UNAVAILABLE: bundled A2UI host not reachable",
)
}
return block()
}
private suspend fun withCanvasAvailable(block: suspend () -> GatewaySession.InvokeResult): GatewaySession.InvokeResult =
try {
block()
} catch (_: Throwable) {
// WebView calls throw when the Activity is backgrounded between the foreground check and execution.
GatewaySession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable",
)
}
private fun availabilityError(availability: InvokeCommandAvailability): GatewaySession.InvokeResult? =
when (availability) {
InvokeCommandAvailability.Always -> null
InvokeCommandAvailability.CameraEnabled ->
if (cameraEnabled()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "CAMERA_DISABLED",
message = "CAMERA_DISABLED: enable Camera in Settings",
)
}
InvokeCommandAvailability.LocationEnabled ->
if (locationEnabled()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "LOCATION_DISABLED",
message = "LOCATION_DISABLED: enable Location in Settings",
)
}
InvokeCommandAvailability.MotionActivityAvailable ->
if (motionActivityAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "MOTION_UNAVAILABLE",
message = "MOTION_UNAVAILABLE: accelerometer not available",
)
}
InvokeCommandAvailability.MotionPedometerAvailable ->
if (motionPedometerAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "PEDOMETER_UNAVAILABLE",
message = "PEDOMETER_UNAVAILABLE: step counter not available",
)
}
InvokeCommandAvailability.SendSmsAvailable ->
if (sendSmsAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "SMS_UNAVAILABLE",
message = "SMS_UNAVAILABLE: SMS not available on this device",
)
}
InvokeCommandAvailability.ReadSmsAvailable,
InvokeCommandAvailability.RequestableSmsSearchAvailable,
->
// SMS search may still be advertised as promptable; runtime invoke fails only on permanent unavailability.
smsSearchAvailabilityError(
readSmsAvailable = readSmsAvailable(),
smsFeatureEnabled = smsFeatureEnabled(),
smsTelephonyAvailable = smsTelephonyAvailable(),
)
InvokeCommandAvailability.CallLogAvailable ->
if (callLogAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "CALL_LOG_UNAVAILABLE",
message = "CALL_LOG_UNAVAILABLE: call log not available on this build",
)
}
InvokeCommandAvailability.PhotosAvailable ->
if (photosAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "PHOTOS_UNAVAILABLE",
message = "PHOTOS_UNAVAILABLE: photos not available on this build",
)
}
InvokeCommandAvailability.InstalledAppsSharingEnabled ->
if (installedAppsSharingEnabled()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "INSTALLED_APPS_SHARING_DISABLED",
message = "INSTALLED_APPS_SHARING_DISABLED: enable Installed Apps in Settings",
)
}
InvokeCommandAvailability.DebugBuild ->
if (debugBuild()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: unknown command",
)
}
}
}
/**
* Talk-mode command adapter implemented by the voice subsystem.
*/
interface TalkHandler {
/** Starts a push-to-talk capture session and keeps it open until stop or cancel. */
suspend fun handlePttStart(paramsJson: String?): GatewaySession.InvokeResult
/** Finishes the active push-to-talk capture and submits recognized speech. */
suspend fun handlePttStop(paramsJson: String?): GatewaySession.InvokeResult
/** Aborts the active push-to-talk capture without submitting speech. */
suspend fun handlePttCancel(paramsJson: String?): GatewaySession.InvokeResult
/** Runs a bounded one-shot push-to-talk capture. */
suspend fun handlePttOnce(paramsJson: String?): GatewaySession.InvokeResult
}

View File

@@ -0,0 +1,70 @@
package ai.openclaw.app.node
import kotlin.math.max
import kotlin.math.min
import kotlin.math.roundToInt
/**
* Result of a JPEG compression attempt after quality and scale reductions.
*/
internal data class JpegSizeLimiterResult(
val bytes: ByteArray,
val width: Int,
val height: Int,
val quality: Int,
)
/**
* Utility that searches quality/scale combinations until a JPEG fits a byte budget.
*/
internal object JpegSizeLimiter {
/** Compresses with the caller-provided encoder, reducing quality before image dimensions. */
fun compressToLimit(
initialWidth: Int,
initialHeight: Int,
startQuality: Int,
maxBytes: Int,
minQuality: Int = 20,
minSize: Int = 256,
scaleStep: Double = 0.85,
maxScaleAttempts: Int = 6,
maxQualityAttempts: Int = 6,
encode: (width: Int, height: Int, quality: Int) -> ByteArray,
): JpegSizeLimiterResult {
require(initialWidth > 0 && initialHeight > 0) { "Invalid image size" }
require(maxBytes > 0) { "Invalid maxBytes" }
val clampedStartQuality = startQuality.coerceIn(minQuality, 100)
var width = initialWidth
var height = initialHeight
var best: JpegSizeLimiterResult? = null
repeat(maxScaleAttempts + 1) { scaleAttempt ->
var quality = clampedStartQuality
repeat(maxQualityAttempts) {
val bytes = encode(width, height, quality)
val attempt = JpegSizeLimiterResult(bytes = bytes, width = width, height = height, quality = quality)
best = attempt
if (bytes.size <= maxBytes) return best
if (quality <= minQuality) return@repeat
quality = max(minQuality, (quality * 0.75).roundToInt())
}
if (scaleAttempt == maxScaleAttempts) return@repeat
val minScale = (minSize.toDouble() / min(width, height).toDouble()).coerceAtMost(1.0)
val nextScale = max(scaleStep, minScale)
val nextWidth = max(minSize, (width * nextScale).roundToInt())
val nextHeight = max(minSize, (height * nextScale).roundToInt())
if (nextWidth == width && nextHeight == height) return@repeat
width = min(nextWidth, width)
height = min(nextHeight, height)
}
val failed = checkNotNull(best)
if (failed.bytes.size > maxBytes) {
throw IllegalStateException("CAMERA_TOO_LARGE: ${failed.bytes.size} bytes > $maxBytes bytes")
}
return failed
}
}

View File

@@ -0,0 +1,123 @@
package ai.openclaw.app.node
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.location.Location
import android.location.LocationManager
import android.os.CancellationSignal
import androidx.core.content.ContextCompat
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.time.Instant
import java.time.format.DateTimeFormatter
/**
* Android LocationManager-backed capture used by gateway location commands.
*/
class LocationCaptureManager(
private val context: Context,
) {
data class Payload(
val payloadJson: String,
)
suspend fun getLocation(
desiredProviders: List<String>,
maxAgeMs: Long?,
timeoutMs: Long,
isPrecise: Boolean,
): Payload =
withContext(Dispatchers.Main) {
val manager = context.getSystemService(Context.LOCATION_SERVICE) as LocationManager
if (!manager.isProviderEnabled(LocationManager.GPS_PROVIDER) &&
!manager.isProviderEnabled(LocationManager.NETWORK_PROVIDER)
) {
throw IllegalStateException("LOCATION_UNAVAILABLE: no location providers enabled")
}
// Prefer a recent cached fix before waking GPS/network providers.
val cached = bestLastKnown(manager, desiredProviders, maxAgeMs)
val location =
cached ?: requestCurrent(manager, desiredProviders, timeoutMs)
val timestamp = DateTimeFormatter.ISO_INSTANT.format(Instant.ofEpochMilli(location.time))
val source = location.provider
val altitudeMeters = if (location.hasAltitude()) location.altitude else null
val speedMps = if (location.hasSpeed()) location.speed.toDouble() else null
val headingDeg = if (location.hasBearing()) location.bearing.toDouble() else null
Payload(
buildString {
append("{\"lat\":")
append(location.latitude)
append(",\"lon\":")
append(location.longitude)
append(",\"accuracyMeters\":")
append(location.accuracy.toDouble())
if (altitudeMeters != null) append(",\"altitudeMeters\":").append(altitudeMeters)
if (speedMps != null) append(",\"speedMps\":").append(speedMps)
if (headingDeg != null) append(",\"headingDeg\":").append(headingDeg)
append(",\"timestamp\":\"").append(timestamp).append('"')
append(",\"isPrecise\":").append(isPrecise)
append(",\"source\":\"").append(source).append('"')
append('}')
},
)
}
private fun bestLastKnown(
manager: LocationManager,
providers: List<String>,
maxAgeMs: Long?,
): Location? {
val fineOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_FINE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
val coarseOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_COARSE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
if (!fineOk && !coarseOk) {
throw IllegalStateException("LOCATION_PERMISSION_REQUIRED: grant Location permission")
}
val now = System.currentTimeMillis()
val candidates =
providers.mapNotNull { provider -> manager.getLastKnownLocation(provider) }
val freshest = candidates.maxByOrNull { it.time } ?: return null
// maxAgeMs is a caller contract; stale cached fixes force a live provider request.
if (maxAgeMs != null && now - freshest.time > maxAgeMs) return null
return freshest
}
private suspend fun requestCurrent(
manager: LocationManager,
providers: List<String>,
timeoutMs: Long,
): Location {
val fineOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_FINE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
val coarseOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_COARSE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
if (!fineOk && !coarseOk) {
throw IllegalStateException("LOCATION_PERMISSION_REQUIRED: grant Location permission")
}
val resolved =
providers.firstOrNull { manager.isProviderEnabled(it) }
?: throw IllegalStateException("LOCATION_UNAVAILABLE: no providers available")
// getCurrentLocation can return null; the handler maps timeout/null fixes to gateway error shapes.
val location =
withTimeout(timeoutMs.coerceAtLeast(1)) {
suspendCancellableCoroutine<Location?> { cont ->
val signal = CancellationSignal()
cont.invokeOnCancellation { signal.cancel() }
manager.getCurrentLocation(resolved, signal, context.mainExecutor) { location ->
cont.resume(location) { _, _, _ -> }
}
}
}
return location ?: throw IllegalStateException("LOCATION_UNAVAILABLE: no fix")
}
}

View File

@@ -0,0 +1,170 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.location.LocationManager
import androidx.core.content.ContextCompat
import kotlinx.coroutines.TimeoutCancellationException
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
/**
* Injectable location facade for command tests and Android runtime access.
*/
internal interface LocationDataSource {
fun hasFinePermission(context: Context): Boolean
fun hasCoarsePermission(context: Context): Boolean
suspend fun fetchLocation(
desiredProviders: List<String>,
maxAgeMs: Long?,
timeoutMs: Long,
isPrecise: Boolean,
): LocationCaptureManager.Payload
}
private class DefaultLocationDataSource(
private val capture: LocationCaptureManager,
) : LocationDataSource {
override fun hasFinePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_FINE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
override fun hasCoarsePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_COARSE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
override suspend fun fetchLocation(
desiredProviders: List<String>,
maxAgeMs: Long?,
timeoutMs: Long,
isPrecise: Boolean,
): LocationCaptureManager.Payload =
capture.getLocation(
desiredProviders = desiredProviders,
maxAgeMs = maxAgeMs,
timeoutMs = timeoutMs,
isPrecise = isPrecise,
)
}
class LocationHandler private constructor(
private val appContext: Context,
private val dataSource: LocationDataSource,
private val json: Json,
private val isForeground: () -> Boolean,
private val locationPreciseEnabled: () -> Boolean,
) {
constructor(
appContext: Context,
location: LocationCaptureManager,
json: Json,
isForeground: () -> Boolean,
locationPreciseEnabled: () -> Boolean,
) : this(
appContext = appContext,
dataSource = DefaultLocationDataSource(location),
json = json,
isForeground = isForeground,
locationPreciseEnabled = locationPreciseEnabled,
)
/** Reports whether precise GPS-backed location can be requested from Android. */
fun hasFineLocationPermission(): Boolean = dataSource.hasFinePermission(appContext)
/** Reports whether network/coarse location can be requested from Android. */
fun hasCoarseLocationPermission(): Boolean = dataSource.hasCoarsePermission(appContext)
companion object {
/** Creates a handler with injected location state for permission and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: LocationDataSource,
json: Json = Json { ignoreUnknownKeys = true },
isForeground: () -> Boolean = { true },
locationPreciseEnabled: () -> Boolean = { true },
): LocationHandler =
LocationHandler(
appContext = appContext,
dataSource = dataSource,
json = json,
isForeground = isForeground,
locationPreciseEnabled = locationPreciseEnabled,
)
}
/** Handles location.get with foreground, permission, and user precision gates applied. */
suspend fun handleLocationGet(paramsJson: String?): GatewaySession.InvokeResult {
if (!isForeground()) {
// Android foreground restrictions and user expectation keep live location tied to the visible app.
return GatewaySession.InvokeResult.error(
code = "LOCATION_BACKGROUND_UNAVAILABLE",
message = "LOCATION_BACKGROUND_UNAVAILABLE: location requires OpenClaw to stay open",
)
}
if (!dataSource.hasFinePermission(appContext) && !dataSource.hasCoarsePermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "LOCATION_PERMISSION_REQUIRED",
message = "LOCATION_PERMISSION_REQUIRED: grant Location permission",
)
}
val (maxAgeMs, timeoutMs, desiredAccuracy) = parseLocationParams(paramsJson)
val preciseEnabled = locationPreciseEnabled()
// Gateway requests are advisory; Android permission and user settings decide
// whether precise capture is actually allowed for this invocation.
val accuracy =
when (desiredAccuracy) {
"precise" -> if (preciseEnabled && dataSource.hasFinePermission(appContext)) "precise" else "balanced"
"coarse" -> "coarse"
else -> if (preciseEnabled && dataSource.hasFinePermission(appContext)) "precise" else "balanced"
}
val providers =
when (accuracy) {
// Provider order is part of the accuracy policy: GPS first for precise, network first otherwise.
"precise" -> listOf(LocationManager.GPS_PROVIDER, LocationManager.NETWORK_PROVIDER)
"coarse" -> listOf(LocationManager.NETWORK_PROVIDER, LocationManager.GPS_PROVIDER)
else -> listOf(LocationManager.NETWORK_PROVIDER, LocationManager.GPS_PROVIDER)
}
try {
val payload =
dataSource.fetchLocation(
desiredProviders = providers,
maxAgeMs = maxAgeMs,
timeoutMs = timeoutMs,
isPrecise = accuracy == "precise",
)
return GatewaySession.InvokeResult.ok(payload.payloadJson)
} catch (err: TimeoutCancellationException) {
return GatewaySession.InvokeResult.error(
code = "LOCATION_TIMEOUT",
message = "LOCATION_TIMEOUT: no fix in time",
)
} catch (err: Throwable) {
val message = err.message ?: "LOCATION_UNAVAILABLE: no fix"
return GatewaySession.InvokeResult.error(code = "LOCATION_UNAVAILABLE", message = message)
}
}
private fun parseLocationParams(paramsJson: String?): Triple<Long?, Long, String?> {
if (paramsJson.isNullOrBlank()) {
return Triple(null, 10_000L, null)
}
val root =
try {
json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
}
val maxAgeMs = (root?.get("maxAgeMs") as? JsonPrimitive)?.content?.toLongOrNull()
val timeoutMs =
(root?.get("timeoutMs") as? JsonPrimitive)?.content?.toLongOrNull()?.coerceIn(1_000L, 60_000L)
?: 10_000L
// desiredAccuracy is advisory; invalid values fall through to the default policy.
val desiredAccuracy =
(root?.get("desiredAccuracy") as? JsonPrimitive)?.content?.trim()?.lowercase()
return Triple(maxAgeMs, timeoutMs, desiredAccuracy)
}
}

View File

@@ -0,0 +1,418 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.Context
import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.hardware.SensorManager
import android.os.SystemClock
import androidx.core.content.ContextCompat
import kotlinx.coroutines.InternalCoroutinesApi
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withTimeoutOrNull
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.time.Instant
import kotlin.math.abs
import kotlin.math.max
import kotlin.math.sqrt
private const val ACCELEROMETER_SAMPLE_TARGET = 20
private const val ACCELEROMETER_SAMPLE_TIMEOUT_MS = 6_000L
/** Gateway request for motion.activity after parsing and limit bounds. */
internal data class MotionActivityRequest(
val startISO: String?,
val endISO: String?,
val limit: Int,
)
/** Gateway request for motion.pedometer. */
internal data class MotionPedometerRequest(
val startISO: String?,
val endISO: String?,
)
/** Motion activity sample returned in gateway-compatible boolean flags. */
internal data class MotionActivityRecord(
val startISO: String,
val endISO: String,
val confidence: String,
val isWalking: Boolean,
val isRunning: Boolean,
val isCycling: Boolean,
val isAutomotive: Boolean,
val isStationary: Boolean,
val isUnknown: Boolean,
)
/** Pedometer sample returned from Android's cumulative step counter. */
internal data class PedometerRecord(
val startISO: String,
val endISO: String,
val steps: Int?,
val distanceMeters: Double?,
val floorsAscended: Int?,
val floorsDescended: Int?,
)
/** Motion data seam for Android sensors and tests. */
internal interface MotionDataSource {
fun isActivityAvailable(context: Context): Boolean
fun isPedometerAvailable(context: Context): Boolean
fun isAvailable(context: Context): Boolean = isActivityAvailable(context) || isPedometerAvailable(context)
fun hasPermission(context: Context): Boolean
suspend fun activity(
context: Context,
request: MotionActivityRequest,
): MotionActivityRecord
suspend fun pedometer(
context: Context,
request: MotionPedometerRequest,
): PedometerRecord
}
private object SystemMotionDataSource : MotionDataSource {
override fun isActivityAvailable(context: Context): Boolean {
val sensorManager = context.getSystemService(SensorManager::class.java)
return sensorManager?.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null
}
override fun isPedometerAvailable(context: Context): Boolean {
val sensorManager = context.getSystemService(SensorManager::class.java)
return sensorManager?.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) != null
}
override fun hasPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACTIVITY_RECOGNITION) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override suspend fun activity(
context: Context,
request: MotionActivityRequest,
): MotionActivityRecord {
if (!request.startISO.isNullOrBlank() || !request.endISO.isNullOrBlank()) {
// Android does not expose historical activity samples here; fail with a
// stable gateway code instead of pretending the range is empty.
throw IllegalArgumentException("MOTION_RANGE_UNAVAILABLE: historical activity range not supported on Android")
}
val sensorManager =
context.getSystemService(SensorManager::class.java)
?: throw IllegalStateException("MOTION_UNAVAILABLE: sensor manager unavailable")
val accelerometer =
sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)
?: throw IllegalStateException("MOTION_UNAVAILABLE: accelerometer not available")
val sample =
readAccelerometerSample(sensorManager, accelerometer)
?: throw IllegalStateException("MOTION_UNAVAILABLE: no accelerometer sample")
val end = Instant.now()
val start = end.minusSeconds(2)
val classification = classifyActivity(sample.averageDelta)
return MotionActivityRecord(
startISO = start.toString(),
endISO = end.toString(),
confidence = classifyConfidence(sample.samples, sample.averageDelta),
isWalking = classification == "walking",
isRunning = classification == "running",
isCycling = false,
isAutomotive = false,
isStationary = classification == "stationary",
isUnknown = classification == "unknown",
)
}
override suspend fun pedometer(
context: Context,
request: MotionPedometerRequest,
): PedometerRecord {
if (!request.startISO.isNullOrBlank() || !request.endISO.isNullOrBlank()) {
// TYPE_STEP_COUNTER is cumulative since boot, not a historical query API.
throw IllegalArgumentException("PEDOMETER_RANGE_UNAVAILABLE: historical pedometer range not supported on Android")
}
val sensorManager =
context.getSystemService(SensorManager::class.java)
?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: sensor manager unavailable")
val stepCounter =
sensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER)
?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: step counting not supported")
val steps =
readStepCounter(sensorManager, stepCounter)
?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: no step counter sample")
val bootMs = System.currentTimeMillis() - SystemClock.elapsedRealtime()
return PedometerRecord(
startISO = Instant.ofEpochMilli(max(0L, bootMs)).toString(),
endISO = Instant.now().toString(),
steps = steps,
distanceMeters = null,
floorsAscended = null,
floorsDescended = null,
)
}
private data class AccelerometerSample(
val samples: Int,
val averageDelta: Double,
)
@OptIn(InternalCoroutinesApi::class)
private suspend fun readStepCounter(
sensorManager: SensorManager,
sensor: Sensor,
): Int? {
val sample =
withTimeoutOrNull(1200L) {
suspendCancellableCoroutine<Float?> { cont ->
val listener =
object : SensorEventListener {
override fun onSensorChanged(event: SensorEvent?) {
val value = event?.values?.firstOrNull()
val token = cont.tryResume(value) ?: return
cont.completeResume(token)
sensorManager.unregisterListener(this)
}
override fun onAccuracyChanged(
sensor: Sensor?,
accuracy: Int,
) = Unit
}
val registered = sensorManager.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_NORMAL)
if (!registered) {
sensorManager.unregisterListener(listener)
cont.resume(null) { _, _, _ -> }
return@suspendCancellableCoroutine
}
cont.invokeOnCancellation { sensorManager.unregisterListener(listener) }
}
}
return sample?.toInt()?.takeIf { it >= 0 }
}
@OptIn(InternalCoroutinesApi::class)
private suspend fun readAccelerometerSample(
sensorManager: SensorManager,
sensor: Sensor,
): AccelerometerSample? {
val sample =
withTimeoutOrNull(ACCELEROMETER_SAMPLE_TIMEOUT_MS) {
suspendCancellableCoroutine<AccelerometerSample?> { cont ->
var count = 0
var sumDelta = 0.0
val listener =
object : SensorEventListener {
override fun onSensorChanged(event: SensorEvent?) {
val values = event?.values ?: return
if (values.size < 3) return
val magnitude =
sqrt(
values[0] * values[0] +
values[1] * values[1] +
values[2] * values[2],
).toDouble()
sumDelta += abs(magnitude - SensorManager.GRAVITY_EARTH.toDouble())
count += 1
if (count >= ACCELEROMETER_SAMPLE_TARGET) {
// Average gravity-adjusted magnitude across a short window so
// one noisy sensor event cannot decide the activity label.
val result =
AccelerometerSample(
samples = count,
averageDelta = sumDelta / count,
)
val token = cont.tryResume(result) ?: return
cont.completeResume(token)
sensorManager.unregisterListener(this)
}
}
override fun onAccuracyChanged(
sensor: Sensor?,
accuracy: Int,
) = Unit
}
val registered = sensorManager.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_NORMAL)
if (!registered) {
cont.resume(null) { _, _, _ -> }
return@suspendCancellableCoroutine
}
cont.invokeOnCancellation { sensorManager.unregisterListener(listener) }
}
}
return sample
}
private fun classifyActivity(averageDelta: Double): String =
when {
averageDelta <= 0.55 -> "stationary"
averageDelta <= 1.80 -> "walking"
else -> "running"
}
private fun classifyConfidence(
samples: Int,
averageDelta: Double,
): String {
if (samples < 6) return "low"
if (samples >= 14 && averageDelta > 0.4) return "high"
return "medium"
}
}
/** Handles Android motion-related node.invoke commands backed by live sensors. */
class MotionHandler private constructor(
private val appContext: Context,
private val dataSource: MotionDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemMotionDataSource)
/** Classifies a short accelerometer sample into the gateway activity shape. */
suspend fun handleMotionActivity(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "MOTION_PERMISSION_REQUIRED",
message = "MOTION_PERMISSION_REQUIRED: grant Motion permission",
)
}
val request =
parseActivityRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val activity = dataSource.activity(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put(
"activities",
buildJsonArray {
add(
buildJsonObject {
put("startISO", JsonPrimitive(activity.startISO))
put("endISO", JsonPrimitive(activity.endISO))
put("confidence", JsonPrimitive(activity.confidence))
put("isWalking", JsonPrimitive(activity.isWalking))
put("isRunning", JsonPrimitive(activity.isRunning))
put("isCycling", JsonPrimitive(activity.isCycling))
put("isAutomotive", JsonPrimitive(activity.isAutomotive))
put("isStationary", JsonPrimitive(activity.isStationary))
put("isUnknown", JsonPrimitive(activity.isUnknown))
},
)
},
)
}.toString(),
)
} catch (err: IllegalArgumentException) {
GatewaySession.InvokeResult.error(code = "MOTION_UNAVAILABLE", message = err.message ?: "MOTION_UNAVAILABLE")
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "MOTION_UNAVAILABLE",
message = "MOTION_UNAVAILABLE: ${err.message ?: "motion activity failed"}",
)
}
}
/** Returns the current boot-scoped Android step-counter reading. */
suspend fun handleMotionPedometer(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "MOTION_PERMISSION_REQUIRED",
message = "MOTION_PERMISSION_REQUIRED: grant Motion permission",
)
}
val request =
parsePedometerRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val payload = dataSource.pedometer(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put("startISO", JsonPrimitive(payload.startISO))
put("endISO", JsonPrimitive(payload.endISO))
payload.steps?.let { put("steps", JsonPrimitive(it)) }
payload.distanceMeters?.let { put("distanceMeters", JsonPrimitive(it)) }
payload.floorsAscended?.let { put("floorsAscended", JsonPrimitive(it)) }
payload.floorsDescended?.let { put("floorsDescended", JsonPrimitive(it)) }
}.toString(),
)
} catch (err: IllegalArgumentException) {
GatewaySession.InvokeResult.error(code = "MOTION_UNAVAILABLE", message = err.message ?: "MOTION_UNAVAILABLE")
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "MOTION_UNAVAILABLE",
message = "MOTION_UNAVAILABLE: ${err.message ?: "pedometer query failed"}",
)
}
}
fun isAvailable(): Boolean = dataSource.isAvailable(appContext)
/** Returns true when live accelerometer classification can be sampled. */
fun isActivityAvailable(): Boolean = dataSource.isActivityAvailable(appContext)
/** Returns true when Android exposes a cumulative step-counter sensor. */
fun isPedometerAvailable(): Boolean = dataSource.isPedometerAvailable(appContext)
private fun parseActivityRequest(paramsJson: String?): MotionActivityRequest? {
if (paramsJson.isNullOrBlank()) {
return MotionActivityRequest(startISO = null, endISO = null, limit = 200)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
// Keep the accepted gateway parameter even though Android can only return
// one live classification sample for now.
val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: 200).coerceIn(1, 1000)
return MotionActivityRequest(
startISO = (params["startISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
endISO = (params["endISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
limit = limit,
)
}
private fun parsePedometerRequest(paramsJson: String?): MotionPedometerRequest? {
if (paramsJson.isNullOrBlank()) {
return MotionPedometerRequest(startISO = null, endISO = null)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
return MotionPedometerRequest(
startISO = (params["startISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
endISO = (params["endISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
)
}
companion object {
/** Static capability probe used before a MotionHandler instance is needed. */
fun isMotionCapabilityAvailable(context: Context): Boolean = SystemMotionDataSource.isAvailable(context)
/** Creates a handler with an injected sensor source for parser and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: MotionDataSource,
): MotionHandler = MotionHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,114 @@
package ai.openclaw.app.node
import android.os.Build
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
internal object NodePresenceAliveBeacon {
/** Gateway event emitted by Android when background execution confirms liveness. */
const val EVENT_NAME: String = "node.presence.alive"
/** Avoids spamming presence when multiple background triggers fire together. */
const val MIN_SUCCESS_INTERVAL_MS: Long = 10 * 60 * 1000
private const val MAX_RESPONSE_JSON_CHARS: Int = 16 * 1024
/**
* Source of the liveness event, serialized as gateway-stable wire values.
*/
enum class Trigger(
val rawValue: String,
) {
Background("background"),
SilentPush("silent_push"),
BackgroundAppRefresh("bg_app_refresh"),
SignificantLocation("significant_location"),
Manual("manual"),
Connect("connect"),
}
/**
* Minimal gateway response fields used to decide whether a liveness event was accepted.
*/
data class ResponsePayload(
val ok: Boolean?,
val event: String?,
val handled: Boolean?,
val reason: String?,
)
private val json = Json { ignoreUnknownKeys = true }
/** Skips sends after a recent successful presence update. */
fun shouldSkipRecentSuccess(
nowMs: Long,
lastSuccessAtMs: Long?,
minIntervalMs: Long = MIN_SUCCESS_INTERVAL_MS,
): Boolean {
val last = lastSuccessAtMs ?: return false
if (last <= 0) return false
val elapsed = nowMs - last
return elapsed >= 0 && elapsed < minIntervalMs
}
/** Human-readable Android version label included in presence payloads. */
fun androidPlatformLabel(): String {
val release =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
.ifEmpty { "unknown" }
return "Android $release (SDK ${Build.VERSION.SDK_INT})"
}
/** Builds the compact JSON payload consumed by gateway node-presence handlers. */
fun makePayloadJson(
trigger: Trigger,
sentAtMs: Long,
displayName: String,
version: String,
platform: String,
deviceFamily: String?,
modelIdentifier: String?,
pushTransport: String? = null,
): String =
buildJsonObject {
put("trigger", JsonPrimitive(trigger.rawValue))
put("sentAtMs", JsonPrimitive(sentAtMs))
put("displayName", JsonPrimitive(displayName))
put("version", JsonPrimitive(version))
put("platform", JsonPrimitive(platform))
deviceFamily?.trim()?.takeIf { it.isNotEmpty() }?.let { put("deviceFamily", JsonPrimitive(it)) }
modelIdentifier?.trim()?.takeIf { it.isNotEmpty() }?.let { put("modelIdentifier", JsonPrimitive(it)) }
pushTransport?.trim()?.takeIf { it.isNotEmpty() }?.let { put("pushTransport", JsonPrimitive(it)) }
}.toString()
/** Parses the gateway response while rejecting empty, oversized, or malformed payloads. */
fun decodeResponse(payloadJson: String?): ResponsePayload? {
val raw = payloadJson?.trim()?.takeIf { it.isNotEmpty() } ?: return null
// Bound log/IPC responses before JSON parsing to avoid memory spikes from
// malformed gateway replies.
if (raw.length > MAX_RESPONSE_JSON_CHARS) return null
val obj =
try {
json.parseToJsonElement(raw).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
return ResponsePayload(
ok = parseJsonBooleanFlag(obj, "ok"),
event = parseJsonString(obj, "event"),
handled = parseJsonBooleanFlag(obj, "handled"),
reason = parseJsonString(obj, "reason"),
)
}
/** Sanitizes gateway response reasons before writing them into Android logs. */
fun sanitizeReasonForLog(raw: String?): String {
val value = raw?.trim()?.takeIf { it.isNotEmpty() } ?: "unsupported"
return value
.map { ch -> if (ch.isISOControl()) ' ' else ch }
.joinToString("")
.take(200)
}
}

View File

@@ -0,0 +1,111 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.parseInvokeErrorFromThrowable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
/** Default canvas seam color used when gateway/user params omit a hex color. */
const val DEFAULT_SEAM_COLOR_ARGB: Long = 0xFF4F7A9A
/** Small tuple used by Android node handlers that need four return values. */
data class Quad<A, B, C, D>(
val first: A,
val second: B,
val third: C,
val fourth: D,
)
/** Escapes a Kotlin string into a JSON string literal without building a JsonElement. */
fun String.toJsonString(): String {
val escaped =
this
.replace("\\", "\\\\")
.replace("\"", "\\\"")
.replace("\n", "\\n")
.replace("\r", "\\r")
return "\"$escaped\""
}
fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
/** Parses invoke params into a JSON object, returning null for absent/malformed input. */
fun parseJsonParamsObject(paramsJson: String?): JsonObject? {
if (paramsJson.isNullOrBlank()) return null
return try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
/** Reads a primitive field from invoke params without accepting arrays/objects. */
fun readJsonPrimitive(
params: JsonObject?,
key: String,
): JsonPrimitive? = params?.get(key) as? JsonPrimitive
/** Parses an optional integer invoke param. */
fun parseJsonInt(
params: JsonObject?,
key: String,
): Int? = readJsonPrimitive(params, key)?.contentOrNull?.toIntOrNull()
/** Parses an optional decimal invoke param. */
fun parseJsonDouble(
params: JsonObject?,
key: String,
): Double? = readJsonPrimitive(params, key)?.contentOrNull?.toDoubleOrNull()
/** Parses an optional string invoke param. */
fun parseJsonString(
params: JsonObject?,
key: String,
): String? = readJsonPrimitive(params, key)?.contentOrNull
/** Parses strict true/false flags from string-like JSON primitives. */
fun parseJsonBooleanFlag(
params: JsonObject?,
key: String,
): Boolean? {
val value = readJsonPrimitive(params, key)?.contentOrNull?.trim()?.lowercase() ?: return null
return when (value) {
"true" -> true
"false" -> false
else -> null
}
}
/** Converts JSON null to Kotlin null while preserving primitive text content. */
fun JsonElement?.asStringOrNull(): String? =
when (this) {
is JsonNull -> null
is JsonPrimitive -> content
else -> null
}
/** Parses #RRGGBB or RRGGBB into opaque ARGB. */
fun parseHexColorArgb(raw: String?): Long? {
val trimmed = raw?.trim().orEmpty()
if (trimmed.isEmpty()) return null
val hex = if (trimmed.startsWith("#")) trimmed.drop(1) else trimmed
if (hex.length != 6) return null
val rgb = hex.toLongOrNull(16) ?: return null
return 0xFF000000L or rgb
}
/** Converts gateway invocation throwables into protocol code/message pairs. */
fun invokeErrorFromThrowable(err: Throwable): Pair<String, String> {
val parsed = parseInvokeErrorFromThrowable(err, fallbackMessage = "UNAVAILABLE: error")
val message = if (parsed.hadExplicitCode) parsed.prefixedMessage else parsed.message
return parsed.code to message
}
/** Normalizes user/session keys while preserving main as the canonical session id. */
fun normalizeMainKey(raw: String?): String? {
val trimmed = raw?.trim().orEmpty()
return if (trimmed.isEmpty()) null else trimmed
}

Some files were not shown because too many files have changed in this diff Show More