Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled

Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
2026-07-05 09:36:54 +00:00
parent 3216769225
commit bedb527145
21108 changed files with 6010766 additions and 0 deletions

View File

@@ -0,0 +1,328 @@
import com.android.build.api.variant.impl.VariantOutputImpl
import java.util.Properties
val dnsjavaInetAddressResolverService = "META-INF/services/java.net.spi.InetAddressResolverProvider"
val openClawAndroidVersionFile = rootProject.file("Config/Version.properties")
val thirdPartyLicensesDir = rootProject.file("THIRD_PARTY_LICENSES")
val openClawAndroidVersionProperties =
Properties().apply {
if (!openClawAndroidVersionFile.isFile) {
error("Missing Android version properties. Run `pnpm android:version:sync`.")
}
openClawAndroidVersionFile.inputStream().use(::load)
}
fun requireOpenClawAndroidVersionProperty(name: String): String =
openClawAndroidVersionProperties.getProperty(name)?.trim()?.takeIf { it.isNotEmpty() }
?: error("Missing $name in Config/Version.properties. Run `pnpm android:version:sync`.")
val openClawAndroidVersionName = requireOpenClawAndroidVersionProperty("OPENCLAW_ANDROID_VERSION_NAME")
val openClawAndroidVersionCode =
requireOpenClawAndroidVersionProperty("OPENCLAW_ANDROID_VERSION_CODE").toIntOrNull()
?: error("OPENCLAW_ANDROID_VERSION_CODE must be an integer in Config/Version.properties.")
val androidStoreFile = providers.gradleProperty("OPENCLAW_ANDROID_STORE_FILE").orNull?.takeIf { it.isNotBlank() }
val androidStorePassword = providers.gradleProperty("OPENCLAW_ANDROID_STORE_PASSWORD").orNull?.takeIf { it.isNotBlank() }
val androidKeyAlias = providers.gradleProperty("OPENCLAW_ANDROID_KEY_ALIAS").orNull?.takeIf { it.isNotBlank() }
val androidKeyPassword = providers.gradleProperty("OPENCLAW_ANDROID_KEY_PASSWORD").orNull?.takeIf { it.isNotBlank() }
val resolvedAndroidStoreFile =
androidStoreFile?.let { storeFilePath ->
if (storeFilePath.startsWith("~/")) {
"${System.getProperty("user.home")}/${storeFilePath.removePrefix("~/")}"
} else {
storeFilePath
}
}
val hasAndroidReleaseSigning =
listOf(resolvedAndroidStoreFile, androidStorePassword, androidKeyAlias, androidKeyPassword).all { it != null }
val wantsAndroidReleaseBuild =
gradle.startParameter.taskNames.any { taskName ->
taskName.contains("Release", ignoreCase = true) ||
Regex("""(^|:)(bundle|assemble)$""").containsMatchIn(taskName)
}
if (wantsAndroidReleaseBuild && !hasAndroidReleaseSigning) {
error(
"Missing Android release signing properties. Set OPENCLAW_ANDROID_STORE_FILE, " +
"OPENCLAW_ANDROID_STORE_PASSWORD, OPENCLAW_ANDROID_KEY_ALIAS, and " +
"OPENCLAW_ANDROID_KEY_PASSWORD in ~/.gradle/gradle.properties.",
)
}
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.ktlint)
alias(libs.plugins.kotlin.compose)
alias(libs.plugins.kotlin.serialization)
}
android {
namespace = "ai.openclaw.app"
compileSdk = 36
// Release signing is local-only; keep the keystore path and passwords out of the repo.
signingConfigs {
if (hasAndroidReleaseSigning) {
create("release") {
storeFile = project.file(checkNotNull(resolvedAndroidStoreFile))
storePassword = checkNotNull(androidStorePassword)
keyAlias = checkNotNull(androidKeyAlias)
keyPassword = checkNotNull(androidKeyPassword)
}
}
}
sourceSets {
getByName("main") {
assets.directories.add("../../shared/OpenClawKit/Sources/OpenClawKit/Resources")
assets.directories.add(thirdPartyLicensesDir.path)
}
}
defaultConfig {
applicationId = "ai.openclaw.app"
minSdk = 31
targetSdk = 36
versionCode = openClawAndroidVersionCode
versionName = openClawAndroidVersionName
ndk {
// Support all major ABIs — native libs are tiny (~47 KB per ABI)
abiFilters += listOf("armeabi-v7a", "arm64-v8a", "x86", "x86_64")
}
}
flavorDimensions += "store"
productFlavors {
create("play") {
dimension = "store"
}
create("thirdParty") {
dimension = "store"
}
}
buildTypes {
release {
if (hasAndroidReleaseSigning) {
signingConfig = signingConfigs.getByName("release")
}
isMinifyEnabled = true
isShrinkResources = true
ndk {
debugSymbolLevel = "SYMBOL_TABLE"
}
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
}
debug {
isMinifyEnabled = false
}
}
buildFeatures {
compose = true
buildConfig = true
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
packaging {
resources {
excludes +=
setOf(
"/META-INF/{AL2.0,LGPL2.1}",
"/META-INF/*.version",
"/META-INF/LICENSE*.txt",
"DebugProbesKt.bin",
"kotlin-tooling-metadata.json",
"org/bouncycastle/pqc/crypto/picnic/lowmcL1.bin.properties",
"org/bouncycastle/pqc/crypto/picnic/lowmcL3.bin.properties",
"org/bouncycastle/pqc/crypto/picnic/lowmcL5.bin.properties",
"org/bouncycastle/x509/CertPathReviewerMessages*.properties",
)
}
}
lint {
lintConfig = file("lint.xml")
warningsAsErrors = true
}
testOptions {
unitTests.isIncludeAndroidResources = true
}
}
androidComponents {
onVariants { variant ->
variant.outputs
.filterIsInstance<VariantOutputImpl>()
.forEach { output ->
val versionName = output.versionName.orNull ?: "0"
val buildType = variant.buildType
val flavorName = variant.flavorName?.takeIf { it.isNotBlank() }
val outputFileName =
if (flavorName == null) {
"openclaw-$versionName-$buildType.apk"
} else {
"openclaw-$versionName-$flavorName-$buildType.apk"
}
output.outputFileName = outputFileName
}
}
}
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
allWarningsAsErrors.set(true)
}
}
ktlint {
android.set(true)
ignoreFailures.set(false)
filter {
exclude("**/build/**")
}
}
dependencies {
val composeBom = platform(libs.androidx.compose.bom)
implementation(composeBom)
androidTestImplementation(composeBom)
implementation(libs.androidx.core.ktx)
implementation(libs.androidx.lifecycle.runtime.ktx)
implementation(libs.androidx.activity.compose)
implementation(libs.androidx.webkit)
implementation(libs.androidx.compose.ui)
implementation(libs.androidx.compose.ui.tooling.preview)
implementation(libs.androidx.compose.material3)
// material-icons-extended pulled in full icon set (~20 MB DEX). Only ~18 icons used.
// R8 will tree-shake unused icons when minify is enabled on release builds.
implementation(libs.androidx.compose.material.icons.extended)
debugImplementation(libs.androidx.compose.ui.tooling)
// Material Components (XML theme + resources)
implementation(libs.material)
implementation(libs.kotlinx.coroutines.android)
implementation(libs.kotlinx.serialization.json)
implementation(libs.androidx.security.crypto)
implementation(libs.androidx.exifinterface)
implementation(libs.okhttp)
implementation(libs.bcprov)
implementation(libs.commonmark)
implementation(libs.commonmark.ext.autolink)
implementation(libs.commonmark.ext.gfm.strikethrough)
implementation(libs.commonmark.ext.gfm.tables)
implementation(libs.commonmark.ext.task.list.items)
// CameraX (for node.invoke camera.* parity)
implementation(libs.androidx.camera.core)
implementation(libs.androidx.camera.camera2)
implementation(libs.androidx.camera.lifecycle)
implementation(libs.androidx.camera.view)
implementation(libs.androidx.camera.video)
implementation(libs.barcode.scanning)
// Unicast DNS-SD (Wide-Area Bonjour) for tailnet discovery domains.
implementation(libs.dnsjava)
testImplementation(libs.junit)
testImplementation(libs.kotlinx.coroutines.test)
testImplementation(libs.kotest.runner.junit5)
testImplementation(libs.kotest.assertions.core)
testImplementation(libs.mockwebserver)
testImplementation(libs.robolectric)
testRuntimeOnly(libs.junit.vintage.engine)
}
tasks.withType<Test>().configureEach {
useJUnitPlatform()
}
val validateThirdPartyLicenseAssets =
tasks.register("validateThirdPartyLicenseAssets") {
inputs.dir(thirdPartyLicensesDir)
doLast {
if (!thirdPartyLicensesDir.isDirectory) {
error("Missing Android third-party license directory: ${thirdPartyLicensesDir.relativeTo(rootProject.projectDir)}")
}
val invalidFiles =
thirdPartyLicensesDir
.walkTopDown()
.filter { file -> file.isFile && file.extension.lowercase() != "txt" }
.map { file -> file.relativeTo(thirdPartyLicensesDir).path }
.toList()
if (invalidFiles.isNotEmpty()) {
error(
"Android third-party license assets must be .txt files:\n" +
invalidFiles.joinToString(separator = "\n") { path -> "- $path" },
)
}
}
}
tasks.matching { task -> task.name == "preBuild" }.configureEach {
dependsOn(validateThirdPartyLicenseAssets)
}
androidComponents {
onVariants(selector().withBuildType("release")) { variant ->
val variantName = variant.name
val variantNameCapitalized = variantName.replaceFirstChar(Char::titlecase)
val stripTaskName = "strip${variantNameCapitalized}DnsjavaServiceDescriptor"
val mergeTaskName = "merge${variantNameCapitalized}JavaResource"
val minifyTaskName = "minify${variantNameCapitalized}WithR8"
val mergedJar =
layout.buildDirectory.file(
"intermediates/merged_java_res/$variantName/$mergeTaskName/base.jar",
)
val stripTask =
tasks.register(stripTaskName) {
inputs.file(mergedJar)
outputs.file(mergedJar)
doLast {
val jarFile = mergedJar.get().asFile
if (!jarFile.exists()) {
return@doLast
}
val unpackDir = temporaryDir.resolve("merged-java-res")
delete(unpackDir)
copy {
from(zipTree(jarFile))
into(unpackDir)
exclude(dnsjavaInetAddressResolverService)
}
delete(jarFile)
ant.invokeMethod(
"zip",
mapOf(
"destfile" to jarFile.absolutePath,
"basedir" to unpackDir.absolutePath,
),
)
}
}
tasks.matching { it.name == mergeTaskName }.configureEach {
finalizedBy(stripTask)
}
tasks.matching { it.name == minifyTaskName }.configureEach {
dependsOn(stripTask)
}
}
}

13
apps/android/app/lint.xml Normal file
View File

@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<lint>
<issue id="AndroidGradlePluginVersion" severity="ignore" />
<issue id="GradleDependency" severity="ignore" />
<issue id="IconLauncherShape" severity="ignore" />
<issue id="NewerVersionAvailable" severity="ignore" />
<!-- OpenClaw uses date-based version codes (yyyyMMddNN), which are high but still below the Android max. -->
<issue id="HighAppVersionCode" severity="ignore" />
<!-- Target SDK follows the current release train; bump only after platform compatibility testing. -->
<issue id="OldTargetApi" severity="ignore" />
</lint>

8
apps/android/app/proguard-rules.pro vendored Normal file
View File

@@ -0,0 +1,8 @@
-dontwarn org.bouncycastle.**
-dontwarn okhttp3.**
-dontwarn okio.**
-dontwarn com.sun.jna.**
-dontwarn javax.naming.**
-dontwarn lombok.Generated
-dontwarn org.slf4j.impl.StaticLoggerBinder
-dontwarn sun.net.spi.nameservice.NameServiceDescriptor

View File

@@ -0,0 +1,22 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<permission
android:name="${applicationId}.permission.RUN_VOICE_E2E"
android:protectionLevel="signature" />
<uses-permission android:name="${applicationId}.permission.RUN_VOICE_E2E" />
<application>
<receiver
android:name=".VoiceE2eReceiver"
android:permission="${applicationId}.permission.RUN_VOICE_E2E"
android:exported="false">
<intent-filter>
<action android:name="ai.openclaw.app.debug.RUN_VOICE_E2E" />
</intent-filter>
</receiver>
<service
android:name=".VoiceE2eService"
android:exported="false" />
</application>
</manifest>

View File

@@ -0,0 +1,195 @@
package ai.openclaw.app
import android.app.Service
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.os.IBinder
import android.util.Base64
import android.util.Log
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withTimeout
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import java.io.File
private const val tag = "VoiceE2E"
private const val resultFileName = "voice_e2e_result.json"
class VoiceE2eReceiver : BroadcastReceiver() {
override fun onReceive(
context: Context,
intent: Intent,
) {
context.startService(
Intent(context, VoiceE2eService::class.java)
.putExtras(intent),
)
}
}
class VoiceE2eService : Service() {
private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
override fun onBind(intent: Intent?): IBinder? = null
override fun onStartCommand(
intent: Intent?,
flags: Int,
startId: Int,
): Int {
val command = intent ?: return START_NOT_STICKY
serviceScope.launch {
try {
runCommand(command)
} finally {
stopSelf(startId)
}
}
return START_NOT_STICKY
}
override fun onDestroy() {
serviceScope.cancel()
super.onDestroy()
}
private suspend fun runCommand(intent: Intent) {
try {
val app = applicationContext as NodeApp
val runtime = app.ensureRuntime()
val mode =
intent
.getDecodedStringExtra("mode")
?.trim()
.orEmpty()
.ifEmpty { "both" }
if (mode == "stop") {
runtime.cancelMicCapture()
runtime.setTalkModeEnabled(false)
writeResult("""{"ok":true,"mode":"stop"}""")
return
}
val connect = !intent.getBooleanExtra("noConnect", false)
val connectTimeoutMs = intent.getLongExtra("connectTimeoutMs", 20_000L)
if (connect) {
configureGateway(runtime = runtime, intent = intent)
}
if (connect || !runtime.isConnected.value) {
awaitGateway(runtime = runtime, timeoutMs = connectTimeoutMs)
}
startActivity(
Intent(actionOpenVoiceE2e)
.setClass(this, MainActivity::class.java)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP),
)
if (mode == "connect") {
val resultJson = """{"ok":true,"mode":"connect","connected":true}"""
writeResult(resultJson)
Log.i(tag, "PASS $resultJson")
return
}
val transcript =
intent
.getDecodedStringExtra("transcript")
?.trim()
.orEmpty()
.ifEmpty { "Reply exactly: Android voice e2e normal path ok." }
val realtimeReply =
intent
.getDecodedStringExtra("realtimeAssistant")
?.trim()
.orEmpty()
.ifEmpty { "Android realtime voice e2e relay path ok." }
val timeoutMs = intent.getLongExtra("timeoutMs", 60_000L)
val result =
runtime.runVoiceE2e(
mode = mode,
transcript = transcript,
realtimeAssistantText = realtimeReply,
timeoutMs = timeoutMs,
)
val resultJson = encodeResult(result)
writeResult(resultJson)
Log.i(tag, "PASS $resultJson")
} catch (err: Throwable) {
val resultJson =
buildJsonObject {
put("ok", JsonPrimitive(false))
put("error", JsonPrimitive(err.message ?: err::class.java.simpleName))
}.toString()
writeResult(resultJson)
Log.e(tag, "FAIL $resultJson", err)
}
}
private fun configureGateway(
runtime: NodeRuntime,
intent: Intent,
) {
val host =
intent
.getDecodedStringExtra("host")
?.trim()
.orEmpty()
.ifEmpty { "127.0.0.1" }
val port = intent.getIntExtra("port", 18789)
runtime.setManualEnabled(true)
runtime.setManualHost(host)
runtime.setManualPort(port)
runtime.setManualTls(intent.getBooleanExtra("tls", false))
runtime.setGatewayToken(intent.getDecodedStringExtra("token").orEmpty())
runtime.setGatewayBootstrapToken(intent.getDecodedStringExtra("bootstrapToken").orEmpty())
runtime.setGatewayPassword(intent.getDecodedStringExtra("password").orEmpty())
runtime.setOnboardingCompleted(true)
runtime.connectManual()
}
private suspend fun awaitGateway(
runtime: NodeRuntime,
timeoutMs: Long,
) {
withTimeout(timeoutMs) {
while (!runtime.isConnected.value) {
delay(100L)
}
}
}
private fun encodeResult(result: NodeRuntime.VoiceE2eResult): String =
buildJsonObject {
put("ok", JsonPrimitive(true))
put("normal", result.normal?.let(::encodeSlice) ?: JsonNull)
put("realtime", result.realtime?.let(::encodeSlice) ?: JsonNull)
}.toString()
private fun encodeSlice(slice: NodeRuntime.VoiceE2eSliceResult) =
buildJsonObject {
put("mode", JsonPrimitive(slice.mode))
put("status", JsonPrimitive(slice.status))
put("userText", slice.userText?.let(::JsonPrimitive) ?: JsonNull)
put("assistantText", slice.assistantText?.let(::JsonPrimitive) ?: JsonNull)
}
private fun writeResult(json: String) {
File(cacheDir, resultFileName).writeText(json)
}
}
private fun Intent.getDecodedStringExtra(name: String): String? {
val encoded = getStringExtra("${name}Base64")
if (!encoded.isNullOrBlank()) {
return String(Base64.decode(encoded, Base64.NO_WRAP), Charsets.UTF_8)
}
return getStringExtra(name)
}

View File

@@ -0,0 +1,91 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_CONNECTED_DEVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission
android:name="android.permission.NEARBY_WIFI_DEVICES"
android:usesPermissionFlags="neverForLocation" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
<uses-permission android:name="android.permission.READ_MEDIA_VISUAL_USER_SELECTED" />
<uses-permission
android:name="android.permission.READ_EXTERNAL_STORAGE"
android:maxSdkVersion="32" />
<uses-permission android:name="android.permission.READ_CONTACTS" />
<uses-permission android:name="android.permission.WRITE_CONTACTS" />
<uses-permission android:name="android.permission.READ_CALENDAR" />
<uses-permission android:name="android.permission.WRITE_CALENDAR" />
<uses-permission android:name="android.permission.ACTIVITY_RECOGNITION" />
<uses-feature
android:name="android.hardware.camera"
android:required="false" />
<uses-feature
android:name="android.hardware.telephony"
android:required="false" />
<queries>
<intent>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent>
</queries>
<application
android:name=".NodeApp"
android:allowBackup="false"
android:dataExtractionRules="@xml/data_extraction_rules"
android:fullBackupContent="@xml/backup_rules"
android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round"
android:label="@string/app_name"
android:supportsRtl="true"
android:networkSecurityConfig="@xml/network_security_config"
android:theme="@style/Theme.OpenClawNode">
<service
android:name=".NodeForegroundService"
android:exported="false"
android:foregroundServiceType="connectedDevice|microphone" />
<service
android:name=".node.DeviceNotificationListenerService"
android:label="@string/app_name"
android:permission="android.permission.BIND_NOTIFICATION_LISTENER_SERVICE"
android:exported="false">
<intent-filter>
<action android:name="android.service.notification.NotificationListenerService" />
</intent-filter>
</service>
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.fileprovider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/file_paths" />
</provider>
<activity
android:name=".MainActivity"
android:exported="true"
android:windowSoftInputMode="adjustResize"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|uiMode|density|keyboard|keyboardHidden|navigation">
<meta-data
android:name="android.app.shortcuts"
android:resource="@xml/shortcuts" />
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.ASSIST" />
<category android:name="android.intent.category.DEFAULT" />
</intent-filter>
</activity>
</application>
</manifest>

View File

@@ -0,0 +1,40 @@
package ai.openclaw.app
import android.content.res.AssetManager
internal const val ANDROID_LICENSE_ASSET_DIRECTORY = "openclaw/licenses"
internal data class AndroidLicenseNotice(
val title: String,
val fileName: String,
val text: String,
)
internal fun loadAndroidLicenseNotices(assetManager: AssetManager): List<AndroidLicenseNotice> {
val files =
assetManager
.list(ANDROID_LICENSE_ASSET_DIRECTORY)
.orEmpty()
.filter(::isAndroidLicenseFileName)
return files
.map { fileName ->
val rawText =
assetManager
.open("$ANDROID_LICENSE_ASSET_DIRECTORY/$fileName")
.bufferedReader(Charsets.UTF_8)
.use { reader -> reader.readText() }
AndroidLicenseNotice(title = androidLicenseTitleFromFileName(fileName), fileName = fileName, text = rawText)
}.sortedWith(
compareBy<AndroidLicenseNotice, String>(String.CASE_INSENSITIVE_ORDER) { notice -> notice.title }
.thenBy(String.CASE_INSENSITIVE_ORDER) { notice -> notice.fileName },
)
}
internal fun isAndroidLicenseFileName(fileName: String): Boolean = fileName.endsWith(".txt", ignoreCase = true)
internal fun androidLicenseTitleFromFileName(fileName: String): String =
fileName
.substringBeforeLast('.')
.trim()
.ifBlank { "License" }

View File

@@ -0,0 +1,28 @@
package ai.openclaw.app
import android.content.Intent
const val extraAndroidScreenshotMode = "openclaw.screenshotMode"
const val extraAndroidScreenshotScene = "openclaw.screenshotScene"
enum class AndroidScreenshotScene(
val rawValue: String,
) {
Connect("connect"),
Chat("chat"),
Voice("voice"),
Screen("screen"),
Settings("settings"),
;
companion object {
fun fromRawValue(raw: String?): AndroidScreenshotScene = entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Connect
}
}
fun parseAndroidScreenshotModeIntent(intent: Intent?): AndroidScreenshotScene? {
if (intent?.getBooleanExtra(extraAndroidScreenshotMode, false) != true) {
return null
}
return AndroidScreenshotScene.fromRawValue(intent.getStringExtra(extraAndroidScreenshotScene))
}

View File

@@ -0,0 +1,25 @@
package ai.openclaw.app
/** User-selectable app theme mode for Android appearance settings. */
enum class AppearanceThemeMode(
val rawValue: String,
val displayLabel: String,
) {
System(rawValue = "system", displayLabel = "System"),
Dark(rawValue = "dark", displayLabel = "Dark"),
Light(rawValue = "light", displayLabel = "Light"),
;
fun isDark(systemDark: Boolean): Boolean =
when (this) {
System -> systemDark
Dark -> true
Light -> false
}
companion object {
fun fromRawValue(value: String?): AppearanceThemeMode = entries.firstOrNull { it.rawValue == value?.trim()?.lowercase() } ?: Dark
fun fromDisplayLabel(label: String): AppearanceThemeMode = entries.firstOrNull { it.displayLabel.equals(label.trim(), ignoreCase = true) } ?: Dark
}
}

View File

@@ -0,0 +1,70 @@
package ai.openclaw.app
import android.content.Intent
/** Android Assistant entry point used by manifest-declared app actions. */
const val actionAskOpenClaw = "ai.openclaw.app.action.ASK_OPENCLAW"
/** Debug action that opens the Voice tab directly for Android E2E automation. */
const val actionOpenVoiceE2e = "ai.openclaw.app.debug.OPEN_VOICE_E2E"
/** Intent extra that carries an optional assistant prompt for app actions. */
const val extraAssistantPrompt = "prompt"
/**
* Top-level home destinations that external actions may request.
*/
enum class HomeDestination {
Connect,
Chat,
Voice,
Screen,
Settings,
}
/**
* Normalized launch request from Android Assistant or explicit app actions.
*/
data class AssistantLaunchRequest(
val source: String,
val prompt: String?,
val autoSend: Boolean,
)
/**
* Parses app-owned navigation actions that should open a specific home tab.
*/
fun parseHomeDestinationIntent(intent: Intent?): HomeDestination? {
val action = intent?.action ?: return null
return when {
// Debug-only shortcut keeps E2E navigation out of release builds.
BuildConfig.DEBUG && action == actionOpenVoiceE2e -> HomeDestination.Voice
else -> null
}
}
/**
* Parse external assistant entry points without starting any UI side effects.
*/
fun parseAssistantLaunchIntent(intent: Intent?): AssistantLaunchRequest? {
val action = intent?.action ?: return null
return when (action) {
Intent.ACTION_ASSIST ->
AssistantLaunchRequest(
source = "assist",
prompt = null,
autoSend = false,
)
actionAskOpenClaw -> {
val prompt = intent.getStringExtra(extraAssistantPrompt)?.trim()?.ifEmpty { null }
AssistantLaunchRequest(
source = "app_action",
prompt = prompt,
autoSend = false,
)
}
else -> null
}
}

View File

@@ -0,0 +1,16 @@
package ai.openclaw.app
/** Camera HUD state categories shown over the Android UI during capture. */
enum class CameraHudKind {
Photo,
Recording,
Success,
Error,
}
/** One-shot camera HUD message keyed by token so repeated text still replays. */
data class CameraHudState(
val token: Long,
val kind: CameraHudKind,
val message: String,
)

View File

@@ -0,0 +1,28 @@
package ai.openclaw.app
import android.content.Context
import android.os.Build
import android.provider.Settings
object DeviceNames {
/** Prefers the user-visible Android device name, then falls back to manufacturer/model text. */
fun bestDefaultNodeName(context: Context): String {
val deviceName =
runCatching {
Settings.Global.getString(context.contentResolver, "device_name")
}.getOrNull()
?.trim()
.orEmpty()
if (deviceName.isNotEmpty()) return deviceName
// Manufacturer/model are best-effort platform fields; keep the final
// fallback stable so stored default names do not become blank.
val model =
listOfNotNull(Build.MANUFACTURER?.takeIf { it.isNotBlank() }, Build.MODEL?.takeIf { it.isNotBlank() })
.joinToString(" ")
.trim()
return model.ifEmpty { "Android Node" }
}
}

View File

@@ -0,0 +1,163 @@
package ai.openclaw.app
import ai.openclaw.app.node.asObjectOrNull
import ai.openclaw.app.node.asStringOrNull
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
data class GatewayExecApprovalSummary(
val id: String,
val commandText: String,
val commandPreview: String?,
val allowedDecisions: List<String>,
val host: String?,
val nodeId: String?,
val agentId: String?,
val createdAtMs: Long?,
val expiresAtMs: Long?,
val resolvingDecision: String? = null,
val errorText: String? = null,
)
internal fun parseGatewayExecApprovalListPayload(
payloadJson: String,
json: Json,
): List<GatewayExecApprovalSummary> =
try {
(json.parseToJsonElement(payloadJson) as? JsonArray)
?.mapNotNull(::parseGatewayExecApprovalListEntry)
?.sortedBy { it.createdAtMs ?: Long.MAX_VALUE }
.orEmpty()
} catch (_: Throwable) {
emptyList()
}
internal fun parseGatewayExecApprovalListEntry(item: JsonElement): GatewayExecApprovalSummary? {
val obj = item.asObjectOrNull() ?: return null
val id = obj["id"].asStringOrNull()?.trim().orEmpty()
if (id.isEmpty()) return null
val request = obj["request"].asObjectOrNull()
val commandText = gatewayExecApprovalListCommandText(obj, request)
return GatewayExecApprovalSummary(
id = id,
commandText = commandText,
commandPreview = gatewayExecApprovalListCommandPreview(obj, request, commandText),
allowedDecisions = emptyList(),
host =
request
?.get("host")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
nodeId =
request
?.get("nodeId")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
agentId =
request
?.get("agentId")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
createdAtMs = obj.long("createdAtMs"),
expiresAtMs = obj.long("expiresAtMs"),
)
}
internal fun parseGatewayExecApprovalDetail(
obj: JsonObject,
createdAtMs: Long?,
): GatewayExecApprovalSummary? {
val id = obj["id"].asStringOrNull()?.trim().orEmpty()
if (id.isEmpty()) return null
return GatewayExecApprovalSummary(
id = id,
commandText =
obj["commandText"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: "Command request",
commandPreview =
obj["commandPreview"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() },
allowedDecisions = gatewayExecApprovalAllowedDecisions(obj),
host = obj["host"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() },
nodeId = obj["nodeId"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() },
agentId = obj["agentId"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() },
createdAtMs = createdAtMs,
expiresAtMs = obj.long("expiresAtMs"),
)
}
private fun gatewayExecApprovalListCommandText(
obj: JsonObject,
request: JsonObject?,
): String =
obj["commandText"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: request
?.get("command")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: "Command request"
private fun gatewayExecApprovalListCommandPreview(
obj: JsonObject,
request: JsonObject?,
commandText: String,
): String? {
val preview =
obj["commandPreview"]
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
?: request
?.get("commandPreview")
.asStringOrNull()
?.trim()
?.takeIf { it.isNotEmpty() }
return preview?.takeIf { it != commandText }
}
private fun gatewayExecApprovalAllowedDecisions(request: JsonObject?): List<String> {
val explicit = parseGatewayExecApprovalDecisions(request?.get("allowedDecisions") as? JsonArray)
if (explicit.isNotEmpty()) return explicit
val allowed =
if (request
?.get("ask")
.asStringOrNull()
?.trim()
?.lowercase() == "always"
) {
listOf("allow-once", "deny")
} else {
listOf("allow-once", "allow-always", "deny")
}
val unavailable = parseGatewayExecApprovalDecisions(request?.get("unavailableDecisions") as? JsonArray).toSet()
return allowed.filterNot { it == "allow-always" && it in unavailable }
}
private fun parseGatewayExecApprovalDecisions(items: JsonArray?): List<String> =
items
?.mapNotNull { item ->
when (item.asStringOrNull()?.trim()) {
"allow-once" -> "allow-once"
"allow-always" -> "allow-always"
"deny" -> "deny"
else -> null
}
}?.distinct()
.orEmpty()
private fun JsonObject?.long(key: String): Long? = (this?.get(key) as? JsonPrimitive)?.content?.trim()?.toLongOrNull()

View File

@@ -0,0 +1,216 @@
package ai.openclaw.app
import ai.openclaw.app.node.asObjectOrNull
import ai.openclaw.app.node.asStringOrNull
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.booleanOrNull
data class GatewayTalkSetupReadiness(
val realtimeTalk: GatewayTalkSetupState,
val dictation: GatewayTalkSetupState,
) {
companion object {
fun unverified(
issue: GatewayTalkSetupIssue = GatewayTalkSetupIssue.CatalogNotLoaded,
): GatewayTalkSetupReadiness =
GatewayTalkSetupReadiness(
realtimeTalk = GatewayTalkSetupState.Unverified(issue),
dictation = GatewayTalkSetupState.Unverified(issue),
)
}
}
sealed interface GatewayTalkSetupState {
data class Ready(
val provider: GatewayTalkProvider,
) : GatewayTalkSetupState
data class NeedsSetup(
val issue: GatewayTalkSetupIssue,
val provider: GatewayTalkProvider? = null,
) : GatewayTalkSetupState
/** Catalog failures must not disable a startup path that the Gateway still validates. */
data class Unverified(
val issue: GatewayTalkSetupIssue,
) : GatewayTalkSetupState
}
enum class GatewayTalkSetupTarget(
val title: String,
) {
REALTIME_TALK("Realtime Talk"),
DICTATION("Dictation"),
}
sealed interface GatewayTalkSetupIssue {
data object CatalogNotLoaded : GatewayTalkSetupIssue
data object CatalogLoadFailed : GatewayTalkSetupIssue
data class GroupMissing(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class NoProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class UnknownProvider(
val target: GatewayTalkSetupTarget,
val providerId: String,
) : GatewayTalkSetupIssue
data class MissingReadiness(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class ConfigureProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class MissingActiveProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class UnsupportedProvider(
val target: GatewayTalkSetupTarget,
) : GatewayTalkSetupIssue
data class ConfigureSelectedProvider(
val providerLabel: String,
) : GatewayTalkSetupIssue
}
data class GatewayTalkProvider(
val id: String,
val label: String,
)
val GatewayTalkSetupState.isReady: Boolean
get() = this is GatewayTalkSetupState.Ready
val GatewayTalkSetupState.requiresSetup: Boolean
get() = this is GatewayTalkSetupState.NeedsSetup
fun gatewayTalkSetupStatusText(state: GatewayTalkSetupState): String =
when (state) {
is GatewayTalkSetupState.Ready -> "Ready"
is GatewayTalkSetupState.NeedsSetup -> "Needs setup"
is GatewayTalkSetupState.Unverified -> "Unverified"
}
fun gatewayTalkSetupDescription(state: GatewayTalkSetupState): String =
when (state) {
is GatewayTalkSetupState.Ready -> "${state.provider.label} via Gateway relay"
is GatewayTalkSetupState.NeedsSetup -> gatewayTalkSetupIssueDescription(state.issue)
is GatewayTalkSetupState.Unverified -> gatewayTalkSetupIssueDescription(state.issue)
}
private fun gatewayTalkSetupIssueDescription(issue: GatewayTalkSetupIssue): String =
when (issue) {
GatewayTalkSetupIssue.CatalogNotLoaded -> "Gateway talk catalog not loaded"
GatewayTalkSetupIssue.CatalogLoadFailed -> "Could not load Gateway talk catalog"
is GatewayTalkSetupIssue.GroupMissing -> "Gateway did not return ${issue.target.title} setup"
is GatewayTalkSetupIssue.NoProvider -> "No ${issue.target.title} provider is configured on the Gateway"
is GatewayTalkSetupIssue.UnknownProvider -> "Gateway selected unknown provider ${issue.providerId}"
is GatewayTalkSetupIssue.MissingReadiness -> "Gateway did not return ${issue.target.title} readiness"
is GatewayTalkSetupIssue.ConfigureProvider -> "Configure a ${issue.target.title} provider on the Gateway"
is GatewayTalkSetupIssue.MissingActiveProvider ->
"Gateway did not identify the active ${issue.target.title} provider"
is GatewayTalkSetupIssue.UnsupportedProvider ->
"Choose a supported ${issue.target.title} provider on the Gateway"
is GatewayTalkSetupIssue.ConfigureSelectedProvider -> "Configure ${issue.providerLabel} on the Gateway"
}
internal fun parseGatewayTalkSetupReadiness(catalog: JsonObject?): GatewayTalkSetupReadiness {
if (catalog == null) return GatewayTalkSetupReadiness.unverified()
return GatewayTalkSetupReadiness(
realtimeTalk =
parseTalkCatalogGroup(catalog = catalog, key = "realtime", target = GatewayTalkSetupTarget.REALTIME_TALK),
dictation =
parseTalkCatalogGroup(catalog = catalog, key = "transcription", target = GatewayTalkSetupTarget.DICTATION),
)
}
private fun parseTalkCatalogGroup(
catalog: JsonObject,
key: String,
target: GatewayTalkSetupTarget,
): GatewayTalkSetupState {
val group =
catalog[key].asObjectOrNull()
?: return GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.GroupMissing(target))
val providers =
(group["providers"] as? JsonArray)
?.mapNotNull(::parseTalkCatalogProvider)
.orEmpty()
val ready = (group["ready"] as? JsonPrimitive)?.booleanOrNull
val activeProviderId = group["activeProvider"].asStringOrNull()?.trim()?.takeIf(String::isNotEmpty)
if (providers.isEmpty()) {
return when {
ready == false -> GatewayTalkSetupState.NeedsSetup(GatewayTalkSetupIssue.NoProvider(target))
activeProviderId != null ->
GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.UnknownProvider(target, activeProviderId))
else -> GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.MissingReadiness(target))
}
}
if (activeProviderId == null) {
if (ready == false) {
return GatewayTalkSetupState.NeedsSetup(GatewayTalkSetupIssue.ConfigureProvider(target))
}
// Older Gateways can omit the selected provider and report alias-backed rows as unconfigured
// even though session startup resolves them. Only an explicit readiness result is authoritative.
return GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.MissingActiveProvider(target))
}
val selected =
// Match Gateway registry precedence: canonical ids win before alias fallback.
providers.firstOrNull { it.matchesId(activeProviderId) }
?: providers.firstOrNull { it.matchesAlias(activeProviderId) }
?: return if (ready == false) {
GatewayTalkSetupState.NeedsSetup(GatewayTalkSetupIssue.UnsupportedProvider(target))
} else {
GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.UnknownProvider(target, activeProviderId))
}
val provider = GatewayTalkProvider(id = selected.id, label = selected.label)
return when (ready) {
true -> GatewayTalkSetupState.Ready(provider)
false ->
GatewayTalkSetupState.NeedsSetup(
issue = GatewayTalkSetupIssue.ConfigureSelectedProvider(selected.label),
provider = provider,
)
null -> GatewayTalkSetupState.Unverified(GatewayTalkSetupIssue.MissingReadiness(target))
}
}
private data class TalkCatalogProvider(
val id: String,
val label: String,
val configured: Boolean,
val aliases: List<String>,
) {
fun matchesId(candidate: String): Boolean = id.equals(candidate, ignoreCase = true)
fun matchesAlias(candidate: String): Boolean = aliases.any { it.equals(candidate, ignoreCase = true) }
}
private fun parseTalkCatalogProvider(item: JsonElement): TalkCatalogProvider? {
val value = item.asObjectOrNull() ?: return null
val id = value["id"].asStringOrNull()?.trim()?.takeIf(String::isNotEmpty) ?: return null
val label = value["label"].asStringOrNull()?.trim()?.takeIf(String::isNotEmpty) ?: id
val aliases =
(value["aliases"] as? JsonArray)
?.mapNotNull { it.asStringOrNull()?.trim()?.takeIf(String::isNotEmpty) }
.orEmpty()
return TalkCatalogProvider(
id = id,
label = label,
configured = (value["configured"] as? JsonPrimitive)?.booleanOrNull == true,
aliases = aliases,
)
}

View File

@@ -0,0 +1,22 @@
package ai.openclaw.app
/**
* Persisted location capture mode advertised to the gateway.
*/
enum class LocationMode(
val rawValue: String,
) {
Off("off"),
WhileUsing("whileUsing"),
;
companion object {
/** Parses persisted location mode text while migrating old always-on configs to while-using. */
fun fromRawValue(raw: String?): LocationMode {
val normalized = raw?.trim()?.lowercase()
// Older configs used "always"; Android node currently exposes while-using location only.
if (normalized == "always") return WhileUsing
return entries.firstOrNull { it.rawValue.lowercase() == normalized } ?: Off
}
}
}

View File

@@ -0,0 +1,209 @@
package ai.openclaw.app
import ai.openclaw.app.ui.AndroidScreenshotModeScreen
import ai.openclaw.app.ui.OpenClawTheme
import ai.openclaw.app.ui.RootScreen
import android.content.Intent
import android.os.Bundle
import android.view.WindowManager
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.viewModels
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.withFrameNanos
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.sp
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.lifecycleScope
import androidx.lifecycle.repeatOnLifecycle
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
/**
* Main Android activity that owns Compose UI attachment and runtime UI wiring.
*/
class MainActivity : ComponentActivity() {
private val viewModel: MainViewModel by viewModels()
private lateinit var permissionRequester: PermissionRequester
private var initializedViewModel: MainViewModel? = null
private var didAttachRuntimeUi = false
private var didStartNodeService = false
private var didStartViewModelCollectors = false
private var foreground = false
private var pendingIntent: Intent? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
pendingIntent = intent
WindowCompat.setDecorFitsSystemWindows(window, false)
permissionRequester = PermissionRequester(this)
if (BuildConfig.DEBUG) {
parseAndroidScreenshotModeIntent(intent)?.let { scene ->
enterScreenshotMode(scene)
return
}
}
setContent {
var activeViewModel by remember { mutableStateOf<MainViewModel?>(null) }
LaunchedEffect(Unit) {
withFrameNanos { }
withContext(Dispatchers.Default) {
(application as NodeApp).prefs
}
val readyViewModel = viewModel
activateViewModel(readyViewModel)
activeViewModel = readyViewModel
}
val currentViewModel = activeViewModel
if (currentViewModel == null) {
OpenClawTheme {
StartupSurface()
}
} else {
val appearanceThemeMode by currentViewModel.appearanceThemeMode.collectAsState()
OpenClawTheme(themeMode = appearanceThemeMode) {
RootScreen(viewModel = currentViewModel)
}
}
}
}
private fun enterScreenshotMode(scene: AndroidScreenshotScene) {
hideScreenshotModeStatusBar()
setContent {
AndroidScreenshotModeScreen(scene = scene)
}
}
private fun hideScreenshotModeStatusBar() {
WindowCompat
.getInsetsController(window, window.decorView)
.apply {
systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
hide(WindowInsetsCompat.Type.statusBars())
}
}
override fun onStart() {
super.onStart()
foreground = true
initializedViewModel?.setForeground(true)
}
override fun onStop() {
foreground = false
initializedViewModel?.setForeground(false)
super.onStop()
}
override fun onNewIntent(intent: android.content.Intent) {
super.onNewIntent(intent)
setIntent(intent)
pendingIntent = intent
initializedViewModel?.let { handleAssistantIntent(viewModel = it, intent = intent) }
}
/**
* Wires MainViewModel only after Activity first draw and background prefs warm-up.
*/
private fun activateViewModel(readyViewModel: MainViewModel) {
if (initializedViewModel != null) return
initializedViewModel = readyViewModel
readyViewModel.setForeground(foreground)
startViewModelCollectors(readyViewModel)
pendingIntent?.let { initialIntent ->
handleAssistantIntent(viewModel = readyViewModel, intent = initialIntent)
pendingIntent = null
}
}
/**
* Starts lifecycle collectors after ViewModel construction so they cannot force early startup.
*/
private fun startViewModelCollectors(readyViewModel: MainViewModel) {
if (didStartViewModelCollectors) return
didStartViewModelCollectors = true
lifecycleScope.launch {
repeatOnLifecycle(Lifecycle.State.STARTED) {
readyViewModel.preventSleep.collect { enabled ->
if (enabled) {
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
} else {
window.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
}
}
}
}
lifecycleScope.launch {
repeatOnLifecycle(Lifecycle.State.STARTED) {
readyViewModel.runtimeInitialized.collect { ready ->
if (!ready || didAttachRuntimeUi) return@collect
// Runtime UI helpers need an Activity owner, so attach once after NodeRuntime is ready.
readyViewModel.attachRuntimeUi(owner = this@MainActivity, permissionRequester = permissionRequester)
didAttachRuntimeUi = true
if (!didStartNodeService) {
NodeForegroundService.start(this@MainActivity)
didStartNodeService = true
}
}
}
}
}
/**
* Routes assistant/app-action intents into ViewModel state without recreating the activity.
*/
private fun handleAssistantIntent(
viewModel: MainViewModel,
intent: Intent?,
) {
parseHomeDestinationIntent(intent)?.let { destination ->
viewModel.requestHomeDestination(destination)
return
}
val request = parseAssistantLaunchIntent(intent) ?: return
viewModel.handleAssistantLaunch(request)
}
}
@Composable
private fun StartupSurface() {
Surface(
modifier = Modifier.fillMaxSize(),
color = Color.Black,
contentColor = Color.White,
) {
Box(
modifier = Modifier.fillMaxSize(),
contentAlignment = Alignment.Center,
) {
Text(
text = "OPENCLAW",
fontSize = 22.sp,
fontWeight = FontWeight.Medium,
)
}
}
}

View File

@@ -0,0 +1,633 @@
package ai.openclaw.app
import ai.openclaw.app.chat.ChatCommandEntry
import ai.openclaw.app.chat.ChatMessage
import ai.openclaw.app.chat.ChatPendingToolCall
import ai.openclaw.app.chat.ChatSessionEntry
import ai.openclaw.app.chat.OutgoingAttachment
import ai.openclaw.app.gateway.DeviceAuthStore
import ai.openclaw.app.gateway.DeviceIdentityStore
import ai.openclaw.app.gateway.GatewayEndpoint
import ai.openclaw.app.gateway.GatewayUpdateAvailableSummary
import ai.openclaw.app.node.CameraCaptureManager
import ai.openclaw.app.node.CanvasController
import ai.openclaw.app.node.SmsManager
import ai.openclaw.app.ui.GatewayConnectPlan
import ai.openclaw.app.ui.GatewaySavedAuthAction
import ai.openclaw.app.voice.VoiceConversationEntry
import android.app.Application
import androidx.lifecycle.AndroidViewModel
import androidx.lifecycle.LifecycleOwner
import androidx.lifecycle.viewModelScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.flatMapLatest
import kotlinx.coroutines.flow.flowOf
import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.launch
/**
* UI-facing bridge that exposes NodeRuntime and preference state as Compose-friendly StateFlows.
*/
@OptIn(ExperimentalCoroutinesApi::class)
class MainViewModel(
app: Application,
) : AndroidViewModel(app) {
private val nodeApp = app as NodeApp
private val prefs = nodeApp.prefs
private val runtimeRef = MutableStateFlow<NodeRuntime?>(null)
@Volatile private var foreground = false
@Volatile private var runtimeStartupQueued = false
private val _requestedHomeDestination = MutableStateFlow<HomeDestination?>(null)
val requestedHomeDestination: StateFlow<HomeDestination?> = _requestedHomeDestination
private val _startOnboardingAtGatewaySetup = MutableStateFlow(false)
val startOnboardingAtGatewaySetup: StateFlow<Boolean> = _startOnboardingAtGatewaySetup
private val _chatDraft = MutableStateFlow<String?>(null)
val chatDraft: StateFlow<String?> = _chatDraft
private val _pendingAssistantAutoSend = MutableStateFlow<String?>(null)
val pendingAssistantAutoSend: StateFlow<String?> = _pendingAssistantAutoSend
/**
* Lazily starts NodeRuntime and preserves the current foreground bit across startup.
*/
private fun ensureRuntime(): NodeRuntime {
runtimeRef.value?.let { return it }
val runtime = nodeApp.ensureRuntime()
runtime.setForeground(foreground)
runtimeRef.value = runtime
return runtime
}
/**
* Starts the node runtime off the main thread so fresh installs can render
* the shell before encrypted prefs, device identity, and gateway setup warm up.
*/
private fun queueRuntimeStartup() {
if (runtimeRef.value != null || runtimeStartupQueued) return
runtimeStartupQueued = true
viewModelScope.launch(Dispatchers.Default) {
runCatching { ensureRuntime() }
runtimeStartupQueued = false
}
}
/**
* Adapts a runtime StateFlow to a stable ViewModel StateFlow before runtime startup.
*/
private fun <T> runtimeState(
initial: T,
selector: (NodeRuntime) -> StateFlow<T>,
): StateFlow<T> =
runtimeRef
.flatMapLatest { runtime -> runtime?.let(selector) ?: flowOf(initial) }
.stateIn(viewModelScope, SharingStarted.Eagerly, initial)
val runtimeInitialized: StateFlow<Boolean> =
runtimeRef
.flatMapLatest { runtime -> flowOf(runtime != null) }
.stateIn(viewModelScope, SharingStarted.Eagerly, false)
val canvasCurrentUrl: StateFlow<String?> = runtimeState(initial = null) { it.canvas.currentUrl }
val canvasA2uiHydrated: StateFlow<Boolean> = runtimeState(initial = false) { it.canvasA2uiHydrated }
val canvasRehydratePending: StateFlow<Boolean> = runtimeState(initial = false) { it.canvasRehydratePending }
val canvasRehydrateErrorText: StateFlow<String?> = runtimeState(initial = null) { it.canvasRehydrateErrorText }
val gateways: StateFlow<List<GatewayEndpoint>> = runtimeState(initial = emptyList()) { it.gateways }
val discoveryStatusText: StateFlow<String> = runtimeState(initial = "Searching…") { it.discoveryStatusText }
val notificationForwardingEnabled: StateFlow<Boolean> = prefs.notificationForwardingEnabled
val notificationForwardingMode: StateFlow<NotificationPackageFilterMode> =
prefs.notificationForwardingMode
val notificationForwardingPackages: StateFlow<Set<String>> = prefs.notificationForwardingPackages
val notificationForwardingQuietHoursEnabled: StateFlow<Boolean> =
prefs.notificationForwardingQuietHoursEnabled
val notificationForwardingQuietStart: StateFlow<String> = prefs.notificationForwardingQuietStart
val notificationForwardingQuietEnd: StateFlow<String> = prefs.notificationForwardingQuietEnd
val notificationForwardingMaxEventsPerMinute: StateFlow<Int> =
prefs.notificationForwardingMaxEventsPerMinute
val notificationForwardingSessionKey: StateFlow<String?> = prefs.notificationForwardingSessionKey
val isConnected: StateFlow<Boolean> = runtimeState(initial = false) { it.isConnected }
val gatewayControlPage: StateFlow<NodeRuntime.GatewayControlPage?> =
runtimeState(initial = null) { it.gatewayControlPage }
val isNodeConnected: StateFlow<Boolean> = runtimeState(initial = false) { it.nodeConnected }
val nodeCapabilityApproval: StateFlow<GatewayNodeCapabilityApproval> =
runtimeState(initial = GatewayNodeCapabilityApproval.Loading) { it.nodeCapabilityApproval }
val statusText: StateFlow<String> = runtimeState(initial = "Offline") { it.statusText }
val gatewayConnectionProblem: StateFlow<GatewayConnectionProblem?> = runtimeState(initial = null) { it.gatewayConnectionProblem }
val gatewayConnectionDisplay: StateFlow<GatewayConnectionDisplay> =
runtimeState(initial = GatewayConnectionDisplay(false, "Offline", null)) { it.gatewayConnectionDisplay }
val serverName: StateFlow<String?> = runtimeState(initial = null) { it.serverName }
val remoteAddress: StateFlow<String?> = runtimeState(initial = null) { it.remoteAddress }
val gatewayVersion: StateFlow<String?> = runtimeState(initial = null) { it.gatewayVersion }
val gatewayUpdateAvailable: StateFlow<GatewayUpdateAvailableSummary?> = runtimeState(initial = null) { it.gatewayUpdateAvailable }
val modelCatalog: StateFlow<List<GatewayModelSummary>> = runtimeState(initial = emptyList()) { it.modelCatalog }
val modelAuthProviders: StateFlow<List<GatewayModelProviderSummary>> = runtimeState(initial = emptyList()) { it.modelAuthProviders }
val modelCatalogRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.modelCatalogRefreshing }
val modelCatalogErrorText: StateFlow<String?> = runtimeState(initial = null) { it.modelCatalogErrorText }
val talkSetupReadiness: StateFlow<GatewayTalkSetupReadiness> =
runtimeState(initial = GatewayTalkSetupReadiness.unverified()) { it.talkSetupReadiness }
val gatewayDefaultAgentId: StateFlow<String?> = runtimeState(initial = null) { it.gatewayDefaultAgentId }
val gatewayAgents: StateFlow<List<GatewayAgentSummary>> = runtimeState(initial = emptyList()) { it.gatewayAgents }
val cronStatus: StateFlow<GatewayCronStatus> = runtimeState(initial = GatewayCronStatus(enabled = false, jobs = 0, nextWakeAtMs = null)) { it.cronStatus }
val cronJobs: StateFlow<List<GatewayCronJobSummary>> = runtimeState(initial = emptyList()) { it.cronJobs }
val cronRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.cronRefreshing }
val cronErrorText: StateFlow<String?> = runtimeState(initial = null) { it.cronErrorText }
val usageSummary: StateFlow<GatewayUsageSummary> = runtimeState(initial = GatewayUsageSummary(updatedAtMs = null, providers = emptyList())) { it.usageSummary }
val usageRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.usageRefreshing }
val usageErrorText: StateFlow<String?> = runtimeState(initial = null) { it.usageErrorText }
val skillsSummary: StateFlow<GatewaySkillsSummary> = runtimeState(initial = GatewaySkillsSummary(skills = emptyList())) { it.skillsSummary }
val skillsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.skillsRefreshing }
val skillsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.skillsErrorText }
val nodesDevicesSummary: StateFlow<GatewayNodesDevicesSummary> =
runtimeState(initial = GatewayNodesDevicesSummary(nodes = emptyList(), pendingDevices = emptyList(), pairedDevices = emptyList())) { it.nodesDevicesSummary }
val nodesDevicesRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.nodesDevicesRefreshing }
val nodesDevicesErrorText: StateFlow<String?> = runtimeState(initial = null) { it.nodesDevicesErrorText }
val channelsSummary: StateFlow<GatewayChannelsSummary> =
runtimeState(initial = GatewayChannelsSummary(channels = emptyList())) { it.channelsSummary }
val channelsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.channelsRefreshing }
val channelsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.channelsErrorText }
val dreamingSummary: StateFlow<GatewayDreamingSummary> =
runtimeState(initial = GatewayDreamingSummary()) { it.dreamingSummary }
val dreamingRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.dreamingRefreshing }
val dreamingErrorText: StateFlow<String?> = runtimeState(initial = null) { it.dreamingErrorText }
val healthLogsSummary: StateFlow<GatewayHealthLogsSummary> =
runtimeState(initial = GatewayHealthLogsSummary()) { it.healthLogsSummary }
val healthLogsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.healthLogsRefreshing }
val healthLogsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.healthLogsErrorText }
val pendingGatewayTrust: StateFlow<NodeRuntime.GatewayTrustPrompt?> = runtimeState(initial = null) { it.pendingGatewayTrust }
val seamColorArgb: StateFlow<Long> = runtimeState(initial = 0xFF0EA5E9) { it.seamColorArgb }
val mainSessionKey: StateFlow<String> = runtimeState(initial = "main") { it.mainSessionKey }
val cameraHud: StateFlow<CameraHudState?> = runtimeState(initial = null) { it.cameraHud }
val cameraFlashToken: StateFlow<Long> = runtimeState(initial = 0L) { it.cameraFlashToken }
val instanceId: StateFlow<String> = prefs.instanceId
val displayName: StateFlow<String> = prefs.displayName
val cameraEnabled: StateFlow<Boolean> = prefs.cameraEnabled
val locationMode: StateFlow<LocationMode> = prefs.locationMode
val locationPreciseEnabled: StateFlow<Boolean> = prefs.locationPreciseEnabled
val preventSleep: StateFlow<Boolean> = prefs.preventSleep
val manualEnabled: StateFlow<Boolean> = prefs.manualEnabled
val manualHost: StateFlow<String> = prefs.manualHost
val manualPort: StateFlow<Int> = prefs.manualPort
val manualTls: StateFlow<Boolean> = prefs.manualTls
val gatewayToken: StateFlow<String> = prefs.gatewayToken
val gatewayBootstrapToken: StateFlow<String> = prefs.gatewayBootstrapToken
val onboardingCompleted: StateFlow<Boolean> = prefs.onboardingCompleted
val canvasDebugStatusEnabled: StateFlow<Boolean> = prefs.canvasDebugStatusEnabled
val installedAppsSharingEnabled: StateFlow<Boolean> = prefs.installedAppsSharingEnabled
val speakerEnabled: StateFlow<Boolean> = prefs.speakerEnabled
val appearanceThemeMode: StateFlow<AppearanceThemeMode> = prefs.appearanceThemeMode
val voiceCaptureMode: StateFlow<VoiceCaptureMode> = runtimeState(initial = VoiceCaptureMode.Off) { it.voiceCaptureMode }
val micEnabled: StateFlow<Boolean> = runtimeState(initial = false) { it.micEnabled }
val micCooldown: StateFlow<Boolean> = runtimeState(initial = false) { it.micCooldown }
val micStatusText: StateFlow<String> = runtimeState(initial = "Mic off") { it.micStatusText }
val micLiveTranscript: StateFlow<String?> = runtimeState(initial = null) { it.micLiveTranscript }
val micIsListening: StateFlow<Boolean> = runtimeState(initial = false) { it.micIsListening }
val micQueuedMessages: StateFlow<List<String>> = runtimeState(initial = emptyList()) { it.micQueuedMessages }
val micConversation: StateFlow<List<VoiceConversationEntry>> = runtimeState(initial = emptyList()) { it.micConversation }
val micInputLevel: StateFlow<Float> = runtimeState(initial = 0f) { it.micInputLevel }
val micIsSending: StateFlow<Boolean> = runtimeState(initial = false) { it.micIsSending }
val talkModeEnabled: StateFlow<Boolean> = runtimeState(initial = false) { it.talkModeEnabled }
val talkModeListening: StateFlow<Boolean> = runtimeState(initial = false) { it.talkModeListening }
val talkModeSpeaking: StateFlow<Boolean> = runtimeState(initial = false) { it.talkModeSpeaking }
val talkModeStatusText: StateFlow<String> = runtimeState(initial = "Off") { it.talkModeStatusText }
val talkModeConversation: StateFlow<List<VoiceConversationEntry>> =
runtimeState(initial = emptyList()) { it.talkModeConversation }
val chatSessionKey: StateFlow<String> = runtimeState(initial = "main") { it.chatSessionKey }
val chatSessionId: StateFlow<String?> = runtimeState(initial = null) { it.chatSessionId }
val chatMessages: StateFlow<List<ChatMessage>> = runtimeState(initial = emptyList()) { it.chatMessages }
val chatHistoryLoading: StateFlow<Boolean> = runtimeState(initial = false) { it.chatHistoryLoading }
val chatError: StateFlow<String?> = runtimeState(initial = null) { it.chatError }
val chatHealthOk: StateFlow<Boolean> = runtimeState(initial = false) { it.chatHealthOk }
val chatThinkingLevel: StateFlow<String> = runtimeState(initial = "off") { it.chatThinkingLevel }
val chatStreamingAssistantText: StateFlow<String?> = runtimeState(initial = null) { it.chatStreamingAssistantText }
val chatPendingToolCalls: StateFlow<List<ChatPendingToolCall>> = runtimeState(initial = emptyList()) { it.chatPendingToolCalls }
val chatSessions: StateFlow<List<ChatSessionEntry>> = runtimeState(initial = emptyList()) { it.chatSessions }
val pendingRunCount: StateFlow<Int> = runtimeState(initial = 0) { it.pendingRunCount }
val chatCommands: StateFlow<List<ChatCommandEntry>> = runtimeState(initial = emptyList<ChatCommandEntry>()) { it.chatCommands }
val execApprovals: StateFlow<List<GatewayExecApprovalSummary>> = runtimeState(initial = emptyList()) { it.execApprovals }
val execApprovalsRefreshing: StateFlow<Boolean> = runtimeState(initial = false) { it.execApprovalsRefreshing }
val execApprovalsErrorText: StateFlow<String?> = runtimeState(initial = null) { it.execApprovalsErrorText }
val canvas: CanvasController
get() = ensureRuntime().canvas
val camera: CameraCaptureManager
get() = ensureRuntime().camera
val sms: SmsManager
get() = ensureRuntime().sms
/**
* Attaches Activity-owned permission and lifecycle seams after runtime initialization.
*/
fun attachRuntimeUi(
owner: LifecycleOwner,
permissionRequester: PermissionRequester,
) {
val runtime = runtimeRef.value ?: return
runtime.camera.attachLifecycleOwner(owner)
runtime.camera.attachPermissionRequester(permissionRequester)
runtime.sms.attachPermissionRequester(permissionRequester)
}
/**
* Starts runtime on foreground entry only after onboarding has completed.
*/
fun setForeground(value: Boolean) {
foreground = value
if (value && prefs.onboardingCompleted.value) {
queueRuntimeStartup()
}
runtimeRef.value?.setForeground(value)
}
fun setDisplayName(value: String) {
prefs.setDisplayName(value)
}
fun setCameraEnabled(value: Boolean) {
prefs.setCameraEnabled(value)
}
fun setLocationMode(mode: LocationMode) {
prefs.setLocationMode(mode)
}
fun setLocationPreciseEnabled(value: Boolean) {
prefs.setLocationPreciseEnabled(value)
}
fun setPreventSleep(value: Boolean) {
prefs.setPreventSleep(value)
}
fun setManualEnabled(value: Boolean) {
prefs.setManualEnabled(value)
}
fun setManualHost(value: String) {
prefs.setManualHost(value)
}
fun setManualPort(value: Int) {
prefs.setManualPort(value)
}
fun setManualTls(value: Boolean) {
prefs.setManualTls(value)
}
fun setGatewayBootstrapToken(value: String) {
prefs.setGatewayBootstrapToken(value)
}
fun setGatewayPassword(value: String) {
prefs.setGatewayPassword(value)
}
/** Clears setup credentials without starting the runtime just to discard first-run pairing auth. */
private fun resetGatewaySetupAuth() {
runtimeRef.value?.resetGatewaySetupAuth() ?: resetGatewaySetupAuthWithoutRuntime()
}
private fun resetGatewaySetupAuthWithoutRuntime() {
prefs.clearGatewaySetupAuth()
val deviceId = DeviceIdentityStore(nodeApp).loadOrCreate().deviceId
val deviceAuthStore = DeviceAuthStore(prefs)
deviceAuthStore.clearToken(deviceId, "node")
deviceAuthStore.clearToken(deviceId, "operator")
}
internal fun saveGatewayConfigAndConnect(plan: GatewayConnectPlan) {
// Gateway pairing touches encrypted prefs, identity files, and sockets; keep
// the whole sequence off the Compose thread so retries cannot trigger ANRs.
viewModelScope.launch(Dispatchers.Default) {
val config = plan.config
val replacesSavedAuth = plan.savedAuthAction != GatewaySavedAuthAction.PRESERVE
val hasExplicitAuth =
config.token.isNotEmpty() || config.bootstrapToken.isNotEmpty() || config.password.isNotEmpty()
if (replacesSavedAuth) {
resetGatewaySetupAuth()
}
prefs.setManualEnabled(true)
prefs.setManualHost(config.host)
prefs.setManualPort(config.port)
prefs.setManualTls(config.tls)
// A blank same-endpoint save means "keep access". Secrets remain runtime-owned,
// including password-only setups that Compose deliberately cannot read back.
if (replacesSavedAuth || hasExplicitAuth) {
prefs.setGatewayBootstrapToken(config.bootstrapToken)
prefs.setGatewayToken(config.token)
prefs.setGatewayPassword(config.password)
}
val runtime = ensureRuntime()
val endpoint = GatewayEndpoint.manual(host = config.host, port = config.port)
if (replacesSavedAuth || hasExplicitAuth) {
runtime.connect(
endpoint,
NodeRuntime.GatewayConnectAuth(
token = config.token.ifEmpty { null },
bootstrapToken = config.bootstrapToken.ifEmpty { null },
password = config.password.ifEmpty { null },
),
)
} else {
runtime.connect(endpoint)
}
}
}
/** Marks onboarding complete and starts the runtime before UI observes connected-state flows. */
fun setOnboardingCompleted(value: Boolean) {
if (value) {
ensureRuntime()
}
prefs.setOnboardingCompleted(value)
}
/** Re-enters gateway setup after disconnecting and clearing one-time setup credentials. */
fun pairNewGateway() {
viewModelScope.launch(Dispatchers.Default) {
runtimeRef.value?.disconnect()
resetGatewaySetupAuth()
prefs.setOnboardingCompleted(false)
_startOnboardingAtGatewaySetup.value = true
}
}
/** Acknowledges the one-shot request that opens onboarding at the gateway setup step. */
fun clearGatewaySetupStartRequest() {
_startOnboardingAtGatewaySetup.value = false
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
prefs.setCanvasDebugStatusEnabled(value)
}
fun setInstalledAppsSharingEnabled(value: Boolean) {
ensureRuntime().setInstalledAppsSharingEnabled(value)
}
fun setNotificationForwardingEnabled(value: Boolean) {
ensureRuntime().setNotificationForwardingEnabled(value)
}
fun setNotificationForwardingMode(mode: NotificationPackageFilterMode) {
ensureRuntime().setNotificationForwardingMode(mode)
}
fun setNotificationForwardingPackagesCsv(csv: String) {
val packages =
csv
.split(',')
.map { it.trim() }
.filter { it.isNotEmpty() }
ensureRuntime().setNotificationForwardingPackages(packages)
}
fun setNotificationForwardingQuietHours(
enabled: Boolean,
start: String,
end: String,
): Boolean = ensureRuntime().setNotificationForwardingQuietHours(enabled = enabled, start = start, end = end)
fun setNotificationForwardingMaxEventsPerMinute(value: Int) {
ensureRuntime().setNotificationForwardingMaxEventsPerMinute(value)
}
fun setNotificationForwardingSessionKey(value: String?) {
ensureRuntime().setNotificationForwardingSessionKey(value)
}
fun setVoiceScreenActive(active: Boolean) {
ensureRuntime().setVoiceScreenActive(active)
}
/** Routes assistant intents into chat, either as a draft or queued auto-send prompt. */
fun handleAssistantLaunch(request: AssistantLaunchRequest) {
_requestedHomeDestination.value = HomeDestination.Chat
if (request.autoSend) {
_pendingAssistantAutoSend.value = request.prompt
_chatDraft.value = null
return
}
_pendingAssistantAutoSend.value = null
_chatDraft.value = request.prompt
}
fun clearRequestedHomeDestination() {
_requestedHomeDestination.value = null
}
fun requestHomeDestination(destination: HomeDestination) {
_requestedHomeDestination.value = destination
}
fun clearChatDraft() {
_chatDraft.value = null
}
fun clearPendingAssistantAutoSend() {
_pendingAssistantAutoSend.value = null
}
fun setMicEnabled(enabled: Boolean) {
ensureRuntime().setMicEnabled(enabled)
}
fun cancelMicCapture() {
ensureRuntime().cancelMicCapture()
}
fun setTalkModeEnabled(enabled: Boolean) {
ensureRuntime().setTalkModeEnabled(enabled)
}
fun setSpeakerEnabled(enabled: Boolean) {
ensureRuntime().setSpeakerEnabled(enabled)
}
fun setAppearanceThemeMode(mode: AppearanceThemeMode) {
prefs.setAppearanceThemeMode(mode)
}
fun refreshGatewayConnection() {
viewModelScope.launch(Dispatchers.Default) {
ensureRuntime().refreshGatewayConnection()
}
}
fun startGatewayDiscovery() {
queueRuntimeStartup()
}
fun connect(endpoint: GatewayEndpoint) {
ensureRuntime().connect(endpoint)
}
fun connectInBackground(endpoint: GatewayEndpoint) {
viewModelScope.launch(Dispatchers.Default) {
ensureRuntime().connect(endpoint)
}
}
fun connect(
endpoint: GatewayEndpoint,
token: String?,
bootstrapToken: String?,
password: String?,
) {
ensureRuntime().connect(
endpoint,
NodeRuntime.GatewayConnectAuth(
token = token,
bootstrapToken = bootstrapToken,
password = password,
),
)
}
fun connectManual() {
ensureRuntime().connectManual()
}
fun disconnect() {
runtimeRef.value?.disconnect()
}
fun acceptGatewayTrustPrompt() {
runtimeRef.value?.acceptGatewayTrustPrompt()
}
fun declineGatewayTrustPrompt() {
runtimeRef.value?.declineGatewayTrustPrompt()
}
fun handleCanvasA2UIActionFromWebView(payloadJson: String) {
ensureRuntime().handleCanvasA2UIActionFromWebView(payloadJson)
}
fun isTrustedCanvasActionUrl(rawUrl: String?): Boolean = ensureRuntime().isTrustedCanvasActionUrl(rawUrl)
fun requestCanvasRehydrate(source: String = "screen_tab") {
ensureRuntime().requestCanvasRehydrate(source = source, force = true)
}
fun refreshHomeCanvasOverviewIfConnected() {
ensureRuntime().refreshHomeCanvasOverviewIfConnected()
}
fun refreshModelCatalog() {
ensureRuntime().refreshModelCatalog()
}
fun refreshTalkSetupReadiness() {
ensureRuntime().refreshTalkSetupReadiness()
}
fun refreshAgents() {
ensureRuntime().refreshAgents()
}
fun refreshCronJobs() {
ensureRuntime().refreshCronJobs()
}
fun refreshUsage() {
ensureRuntime().refreshUsage()
}
fun refreshSkills() {
ensureRuntime().refreshSkills()
}
fun refreshNodesDevices() {
ensureRuntime().refreshNodesDevices()
}
fun refreshExecApprovals() {
ensureRuntime().refreshExecApprovals()
}
fun resolveExecApproval(
id: String,
decision: String,
) {
ensureRuntime().resolveExecApproval(id = id, decision = decision)
}
fun refreshChannels() {
ensureRuntime().refreshChannels()
}
fun refreshDreaming() {
ensureRuntime().refreshDreaming()
}
fun refreshHealthLogs() {
ensureRuntime().refreshHealthLogs()
}
fun loadChat(sessionKey: String) {
ensureRuntime().loadChat(sessionKey)
}
fun refreshChat() {
ensureRuntime().refreshChat()
}
fun refreshChatSessions(limit: Int? = null) {
ensureRuntime().refreshChatSessions(limit = limit)
}
fun setChatThinkingLevel(level: String) {
ensureRuntime().setChatThinkingLevel(level)
}
fun switchChatSession(sessionKey: String) {
ensureRuntime().switchChatSession(sessionKey)
}
fun abortChat() {
ensureRuntime().abortChat()
}
fun startNewChat() {
ensureRuntime().startNewChat()
}
fun refreshChatCommands() {
ensureRuntime().refreshChatCommands()
}
fun sendChat(
message: String,
thinking: String,
attachments: List<OutgoingAttachment>,
) {
ensureRuntime().sendChat(message = message, thinking = thinking, attachments = attachments)
}
suspend fun sendChatAwaitAcceptance(
message: String,
thinking: String,
attachments: List<OutgoingAttachment>,
): Boolean =
ensureRuntime().sendChatAwaitAcceptance(
message = message,
thinking = thinking,
attachments = attachments,
)
}

View File

@@ -0,0 +1,48 @@
package ai.openclaw.app
import android.app.Application
import android.os.StrictMode
/**
* Android Application singleton that owns process-wide secure prefs and lazy NodeRuntime startup.
*/
class NodeApp : Application() {
val prefs: SecurePrefs by lazy { SecurePrefs(this) }
@Volatile private var runtimeInstance: NodeRuntime? = null
/**
* Returns the single NodeRuntime for this process, creating it on first use.
*/
fun ensureRuntime(): NodeRuntime {
runtimeInstance?.let { return it }
return synchronized(this) {
runtimeInstance ?: NodeRuntime(this, prefs).also { runtimeInstance = it }
}
}
/**
* Reads the runtime without forcing startup, used by lifecycle probes and services.
*/
fun peekRuntime(): NodeRuntime? = runtimeInstance
override fun onCreate() {
super.onCreate()
if (BuildConfig.DEBUG) {
StrictMode.setThreadPolicy(
StrictMode.ThreadPolicy
.Builder()
.detectAll()
.penaltyLog()
.build(),
)
StrictMode.setVmPolicy(
StrictMode.VmPolicy
.Builder()
.detectAll()
.penaltyLog()
.build(),
)
}
}
}

View File

@@ -0,0 +1,290 @@
package ai.openclaw.app
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import androidx.core.app.NotificationCompat
import androidx.core.app.ServiceCompat
import androidx.core.content.ContextCompat
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.launch
/** Foreground service that keeps the Android node connection and voice capture visible to the OS. */
class NodeForegroundService : Service() {
private val scope: CoroutineScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
private var notificationJob: Job? = null
private var voiceCaptureMode = VoiceCaptureMode.Off
override fun onCreate() {
super.onCreate()
ensureChannel()
val initial = buildNotification(title = "OpenClaw Node", text = "Starting…")
startForegroundWithTypes(notification = initial)
val runtime = (application as NodeApp).peekRuntime()
if (runtime == null) {
stopSelf()
return
}
// Keep the connection tuple atomic, then split connection and capture work so notification text
// can update without restarting runtime-owned connection work.
notificationJob =
scope.launch {
combine(
combine(
runtime.gatewayConnectionDisplay,
runtime.serverName,
runtime.voiceCaptureMode,
) { connection, server, mode ->
VoiceNotificationBase(
status = connection.statusText,
server = server,
connected = connection.isConnected,
mode = mode,
)
},
combine(
runtime.micEnabled,
runtime.micIsListening,
runtime.talkModeListening,
runtime.talkModeSpeaking,
) { micEnabled, micListening, talkListening, talkSpeaking ->
VoiceNotificationCapture(
micEnabled = micEnabled,
micListening = micListening,
talkListening = talkListening,
talkSpeaking = talkSpeaking,
)
},
) { base, capture ->
VoiceNotificationState(base = base, capture = capture)
}.collect { state ->
voiceCaptureMode = state.mode
val title =
when {
state.connected && state.mode == VoiceCaptureMode.TalkMode -> "OpenClaw Node · Talk"
state.connected -> "OpenClaw Node · Connected"
else -> "OpenClaw Node"
}
val text =
(state.server?.let { "${state.status} · $it" } ?: state.status) +
voiceNotificationSuffix(
mode = state.mode,
manualMicEnabled = state.capture.micEnabled,
manualMicListening = state.capture.micListening,
talkListening = state.capture.talkListening,
talkSpeaking = state.capture.talkSpeaking,
)
startForegroundWithTypes(
notification = buildNotification(title = title, text = text),
)
}
}
}
override fun onStartCommand(
intent: Intent?,
flags: Int,
startId: Int,
): Int {
when (intent?.action) {
ACTION_STOP -> {
(application as NodeApp).peekRuntime()?.disconnect()
stopSelf()
return START_NOT_STICKY
}
ACTION_SET_VOICE_CAPTURE_MODE -> {
voiceCaptureMode = intent.getStringExtra(EXTRA_VOICE_CAPTURE_MODE).toVoiceCaptureMode()
startForegroundWithTypes(
notification =
buildNotification(
title = "OpenClaw Node",
text = if (voiceCaptureMode == VoiceCaptureMode.TalkMode) "Talk mode active" else "Connected",
),
)
}
}
// Keep running; connection is managed by NodeRuntime (auto-reconnect + manual).
return START_STICKY
}
override fun onDestroy() {
notificationJob?.cancel()
scope.cancel()
super.onDestroy()
}
override fun onBind(intent: Intent?) = null
private fun ensureChannel() {
val mgr = getSystemService(NotificationManager::class.java)
val channel =
NotificationChannel(
CHANNEL_ID,
"Connection",
NotificationManager.IMPORTANCE_LOW,
).apply {
description = "OpenClaw node connection status"
setShowBadge(false)
}
mgr.createNotificationChannel(channel)
}
private fun buildNotification(
title: String,
text: String,
): Notification {
val launchIntent =
Intent(this, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP
}
val launchPending =
PendingIntent.getActivity(
this,
1,
launchIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
val stopIntent = Intent(this, NodeForegroundService::class.java).setAction(ACTION_STOP)
val stopPending =
PendingIntent.getService(
this,
2,
stopIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
return NotificationCompat
.Builder(this, CHANNEL_ID)
.setSmallIcon(R.mipmap.ic_launcher)
.setContentTitle(title)
.setContentText(text)
.setContentIntent(launchPending)
.setOngoing(true)
.setOnlyAlertOnce(true)
.setForegroundServiceBehavior(NotificationCompat.FOREGROUND_SERVICE_IMMEDIATE)
.addAction(0, "Disconnect", stopPending)
.build()
}
private fun startForegroundWithTypes(notification: Notification) {
val serviceTypes = foregroundServiceTypesForVoiceMode(voiceCaptureMode)
ServiceCompat.startForeground(this, NOTIFICATION_ID, notification, serviceTypes)
}
companion object {
private const val CHANNEL_ID = "connection"
private const val NOTIFICATION_ID = 1
private const val ACTION_STOP = "ai.openclaw.app.action.STOP"
private const val ACTION_SET_VOICE_CAPTURE_MODE = "ai.openclaw.app.action.SET_VOICE_CAPTURE_MODE"
private const val EXTRA_VOICE_CAPTURE_MODE = "ai.openclaw.app.extra.VOICE_CAPTURE_MODE"
fun start(context: Context) {
val intent = Intent(context, NodeForegroundService::class.java)
context.startForegroundService(intent)
}
fun stop(context: Context) {
val intent = Intent(context, NodeForegroundService::class.java).setAction(ACTION_STOP)
context.startService(intent)
}
fun setVoiceCaptureMode(
context: Context,
mode: VoiceCaptureMode,
) {
val intent =
Intent(context, NodeForegroundService::class.java)
.setAction(ACTION_SET_VOICE_CAPTURE_MODE)
.putExtra(EXTRA_VOICE_CAPTURE_MODE, mode.name)
if (mode == VoiceCaptureMode.TalkMode) {
// Microphone foreground service type must be declared before Talk capture starts.
ContextCompat.startForegroundService(context, intent)
} else {
context.startService(intent)
}
}
}
}
internal fun foregroundServiceTypesForVoiceMode(mode: VoiceCaptureMode): Int {
val base = ServiceInfo.FOREGROUND_SERVICE_TYPE_CONNECTED_DEVICE
return when (mode) {
VoiceCaptureMode.Off -> base
VoiceCaptureMode.ManualMic,
VoiceCaptureMode.TalkMode,
-> base or ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE
}
}
internal fun voiceNotificationSuffix(
mode: VoiceCaptureMode,
manualMicEnabled: Boolean,
manualMicListening: Boolean,
talkListening: Boolean,
talkSpeaking: Boolean,
): String =
when (mode) {
VoiceCaptureMode.TalkMode ->
when {
talkSpeaking -> " · Talk: Speaking"
talkListening -> " · Talk: Listening"
else -> " · Talk: On"
}
VoiceCaptureMode.ManualMic ->
if (manualMicEnabled) {
if (manualMicListening) " · Mic: Listening" else " · Mic: Pending"
} else {
""
}
VoiceCaptureMode.Off -> ""
}
private fun String?.toVoiceCaptureMode(): VoiceCaptureMode =
VoiceCaptureMode.entries.firstOrNull {
it.name == this
} ?: VoiceCaptureMode.Off
/** Connection fields that drive foreground notification title/body text. */
private data class VoiceNotificationBase(
val status: String,
val server: String?,
val connected: Boolean,
val mode: VoiceCaptureMode,
)
/** Voice capture fields that affect foreground-service type and suffix. */
private data class VoiceNotificationCapture(
val micEnabled: Boolean,
val micListening: Boolean,
val talkListening: Boolean,
val talkSpeaking: Boolean,
)
/** Aggregated notification state from runtime flows. */
private data class VoiceNotificationState(
val base: VoiceNotificationBase,
val capture: VoiceNotificationCapture,
) {
val status: String
get() = base.status
val server: String?
get() = base.server
val connected: Boolean
get() = base.connected
val mode: VoiceCaptureMode
get() = base.mode
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,122 @@
package ai.openclaw.app
import java.time.Instant
import java.time.ZoneId
/** Package-filter mode used before notification events are forwarded to the gateway. */
enum class NotificationPackageFilterMode(
val rawValue: String,
) {
Allowlist("allowlist"),
Blocklist("blocklist"),
;
companion object {
/** Parses persisted filter mode text, defaulting to blocklist for safer forwarding. */
fun fromRawValue(raw: String?): NotificationPackageFilterMode = entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Blocklist
}
}
/** Runtime policy used before forwarding notification events to a node session. */
internal data class NotificationForwardingPolicy(
val enabled: Boolean,
val mode: NotificationPackageFilterMode,
val packages: Set<String>,
val quietHoursEnabled: Boolean,
val quietStart: String,
val quietEnd: String,
val maxEventsPerMinute: Int,
val sessionKey: String?,
val selfPackageName: String = "",
)
/** Applies the operator-configured package allow/block list after trimming input. */
internal fun NotificationForwardingPolicy.allowsPackage(packageName: String): Boolean {
val normalized = packageName.trim()
if (normalized.isEmpty()) {
return false
}
val self = selfPackageName.trim()
if (self.isNotEmpty() && normalized == self) {
return false
}
return when (mode) {
NotificationPackageFilterMode.Allowlist -> packages.contains(normalized)
NotificationPackageFilterMode.Blocklist -> !packages.contains(normalized)
}
}
/** Returns true for both same-day and overnight quiet-hour windows. */
internal fun NotificationForwardingPolicy.isWithinQuietHours(
nowEpochMs: Long,
zoneId: ZoneId = ZoneId.systemDefault(),
): Boolean {
if (!quietHoursEnabled) {
return false
}
val startMinutes = parseLocalHourMinute(quietStart) ?: return false
val endMinutes = parseLocalHourMinute(quietEnd) ?: return false
if (startMinutes == endMinutes) {
return true
}
val now =
Instant
.ofEpochMilli(nowEpochMs)
.atZone(zoneId)
.toLocalTime()
val nowMinutes = now.hour * 60 + now.minute
return if (startMinutes < endMinutes) {
nowMinutes in startMinutes until endMinutes
} else {
nowMinutes >= startMinutes || nowMinutes < endMinutes
}
}
private val localHourMinuteRegex = Regex("""^([01]\d|2[0-3]):([0-5]\d)$""")
/** Normalizes persisted or user-entered local times to strict HH:mm form. */
internal fun normalizeLocalHourMinute(raw: String): String? {
val trimmed = raw.trim()
val match = localHourMinuteRegex.matchEntire(trimmed) ?: return null
return "${match.groupValues[1]}:${match.groupValues[2]}"
}
/** Converts strict local HH:mm text to minutes since midnight for window checks. */
internal fun parseLocalHourMinute(raw: String): Int? {
val normalized = normalizeLocalHourMinute(raw) ?: return null
val parts = normalized.split(':')
val hour = parts[0].toInt()
val minute = parts[1].toInt()
return hour * 60 + minute
}
/** Fixed-window limiter that bounds notification bursts per wall-clock minute. */
internal class NotificationBurstLimiter {
private val lock = Any()
private var windowStartMs: Long = -1L
private var eventsInWindow: Int = 0
/** Returns true when the current minute bucket still has forwarding capacity. */
fun allow(
nowEpochMs: Long,
maxEventsPerMinute: Int,
): Boolean {
if (maxEventsPerMinute <= 0) {
return false
}
// Align all callers to the same minute bucket so concurrent notifications
// share the quota even when they arrive with slightly different timestamps.
val currentWindow = nowEpochMs - (nowEpochMs % 60_000L)
synchronized(lock) {
if (currentWindow != windowStartMs) {
windowStartMs = currentWindow
eventsInWindow = 0
}
if (eventsInWindow >= maxEventsPerMinute) {
return false
}
eventsInWindow += 1
return true
}
}
}

View File

@@ -0,0 +1,285 @@
package ai.openclaw.app
import android.Manifest
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Handler
import android.os.Looper
import android.provider.Settings
import androidx.activity.ComponentActivity
import androidx.activity.result.ActivityResultLauncher
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AlertDialog
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.TimeoutCancellationException
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.util.concurrent.atomic.AtomicBoolean
import kotlin.coroutines.resume
/**
* Serializes Android runtime-permission prompts behind coroutine-friendly request calls.
*/
class PermissionRequester internal constructor(
private val activity: ComponentActivity,
launcherFactory: ((Map<String, Boolean>) -> Unit) -> ActivityResultLauncher<Array<String>>,
) {
private data class PendingPermissionRequest(
val deferred: CompletableDeferred<Map<String, Boolean>>,
var timedOut: Boolean = false,
)
private class PermissionRequestSlot(
val launcher: ActivityResultLauncher<Array<String>>,
var request: PendingPermissionRequest? = null,
)
constructor(activity: ComponentActivity) : this(
activity = activity,
launcherFactory = { callback ->
activity.registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions(), callback)
},
)
private val mutex = Mutex()
private val requestSlotsLock = Any()
private val mainHandler = Handler(Looper.getMainLooper())
// ActivityResult launchers cannot be registered after start; pre-register a small pool for nested UI flows.
private val launchers = List(4) { createPermissionRequestSlot(launcherFactory) }
/**
* Request missing Android runtime permissions and return the final grant state for every requested permission.
*/
suspend fun requestIfMissing(
permissions: List<String>,
timeoutMs: Long = 20_000,
): Map<String, Boolean> {
return mutex.withLock {
while (true) {
val missing =
permissions.filter { perm ->
ContextCompat.checkSelfPermission(activity, perm) != PackageManager.PERMISSION_GRANTED
}
if (missing.isEmpty()) {
return permissions.associateWith { true }
}
val needsRationale =
missing.any { ActivityCompat.shouldShowRequestPermissionRationale(activity, it) }
if (needsRationale) {
val proceed = showRationaleDialog(missing)
if (!proceed) {
return permissions.associateWith { perm ->
ContextCompat.checkSelfPermission(activity, perm) == PackageManager.PERMISSION_GRANTED
}
}
}
val deferred = CompletableDeferred<Map<String, Boolean>>()
val request = PendingPermissionRequest(deferred)
val slot = reservePermissionRequestSlot(request)
try {
withContext(Dispatchers.Main) {
slot.launcher.launch(missing.toTypedArray())
}
} catch (err: Throwable) {
clearPermissionRequestSlot(slot, request)
throw err
}
val result =
try {
withTimeout(timeoutMs) { deferred.await() }
} catch (err: TimeoutCancellationException) {
// Late ActivityResult callbacks are ignored by completePermissionRequest.
request.timedOut = true
throw err
}
val merged =
permissions.associateWith { perm ->
val nowGranted =
ContextCompat.checkSelfPermission(activity, perm) == PackageManager.PERMISSION_GRANTED
result[perm] == true || nowGranted
}
val denied =
merged.filterValues { !it }.keys.filter {
!ActivityCompat.shouldShowRequestPermissionRationale(activity, it)
}
if (denied.isNotEmpty()) {
showSettingsDialog(denied)
}
return merged
}
error("unreachable")
}
}
private fun createPermissionRequestSlot(
launcherFactory: ((Map<String, Boolean>) -> Unit) -> ActivityResultLauncher<Array<String>>,
): PermissionRequestSlot {
var slot: PermissionRequestSlot? = null
val launcher = launcherFactory { result -> completePermissionRequest(checkNotNull(slot), result) }
val created = PermissionRequestSlot(launcher)
slot = created
return created
}
private fun reservePermissionRequestSlot(request: PendingPermissionRequest): PermissionRequestSlot =
synchronized(requestSlotsLock) {
// The outer mutex serializes normal callers; this guard catches accidental concurrent launchers in tests.
val slot = launchers.firstOrNull { it.request == null } ?: error("permission request launcher busy")
slot.request = request
slot
}
private fun completePermissionRequest(
slot: PermissionRequestSlot,
result: Map<String, Boolean>,
) {
val request =
synchronized(requestSlotsLock) {
slot.request.also {
slot.request = null
}
} ?: return
// Timed-out requests have already resumed callers with failure; ignore any late platform callback.
if (request.timedOut) return
request.deferred.complete(result)
}
private fun clearPermissionRequestSlot(
slot: PermissionRequestSlot,
request: PendingPermissionRequest,
) {
synchronized(requestSlotsLock) {
if (slot.request === request) {
slot.request = null
}
}
}
private suspend fun showRationaleDialog(permissions: List<String>): Boolean =
withContext(Dispatchers.Main) {
if (activity.isFinishing || activity.isDestroyed) {
return@withContext false
}
suspendCancellableCoroutine { cont ->
val lifecycle = activity.lifecycle
var dialog: AlertDialog? = null
var observer: LifecycleEventObserver? = null
val finished = AtomicBoolean(false)
val removeObserver = {
observer?.let(lifecycle::removeObserver)
observer = null
}
fun finish(result: Boolean?) {
if (!finished.compareAndSet(false, true)) return
removeObserver()
dialog?.dismiss()
if (result != null) {
cont.resume(result)
}
}
val actualObserver =
LifecycleEventObserver { _, event ->
if (event != Lifecycle.Event.ON_DESTROY) return@LifecycleEventObserver
// Do not resume a destroyed Activity with a positive result.
finish(false)
}
observer = actualObserver
lifecycle.addObserver(actualObserver)
cont.invokeOnCancellation {
mainHandler.post {
finish(null)
}
}
dialog =
AlertDialog
.Builder(activity)
.setTitle("Permission required")
.setMessage(buildRationaleMessage(permissions))
.setPositiveButton("Continue") { _, _ -> finish(true) }
.setNegativeButton("Not now") { _, _ -> finish(false) }
.setOnCancelListener { finish(false) }
.show()
}
}
private suspend fun showSettingsDialog(permissions: List<String>) =
withContext(Dispatchers.Main) {
if (activity.isFinishing || activity.isDestroyed) return@withContext
val lifecycle = activity.lifecycle
var dialog: AlertDialog? = null
var observer: LifecycleEventObserver? = null
val removeObserver = {
observer?.let(lifecycle::removeObserver)
observer = null
}
val actualObserver =
LifecycleEventObserver { _, event ->
if (event != Lifecycle.Event.ON_DESTROY) return@LifecycleEventObserver
removeObserver()
dialog?.dismiss()
}
observer = actualObserver
lifecycle.addObserver(actualObserver)
dialog =
AlertDialog
.Builder(activity)
.setTitle("Enable permission in Settings")
.setMessage(buildSettingsMessage(permissions))
.setPositiveButton("Open Settings") { _, _ ->
if (activity.isFinishing || activity.isDestroyed) return@setPositiveButton
val intent =
Intent(
Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", activity.packageName, null),
)
activity.startActivity(intent)
}.setNegativeButton("Cancel", null)
.setOnDismissListener { removeObserver() }
.show()
}
private fun buildRationaleMessage(permissions: List<String>): String {
val labels = permissions.map { permissionLabel(it) }
return "OpenClaw needs ${labels.joinToString(", ")} permissions to continue."
}
private fun buildSettingsMessage(permissions: List<String>): String {
val labels = permissions.map { permissionLabel(it) }
return "Please enable ${labels.joinToString(", ")} in Android Settings to continue."
}
private fun permissionLabel(permission: String): String =
when (permission) {
Manifest.permission.CAMERA -> "Camera"
Manifest.permission.RECORD_AUDIO -> "Microphone"
Manifest.permission.SEND_SMS -> "Send SMS"
Manifest.permission.READ_SMS -> "Read SMS"
Manifest.permission.READ_CONTACTS -> "Read Contacts"
Manifest.permission.WRITE_CONTACTS -> "Write Contacts"
Manifest.permission.READ_CALENDAR -> "Read Calendar"
Manifest.permission.WRITE_CALENDAR -> "Write Calendar"
Manifest.permission.READ_CALL_LOG -> "Read Call Log"
Manifest.permission.ACTIVITY_RECOGNITION -> "Motion Activity"
Manifest.permission.READ_MEDIA_IMAGES -> "Photos"
Manifest.permission.READ_MEDIA_VISUAL_USER_SELECTED -> "Photos"
Manifest.permission.READ_EXTERNAL_STORAGE -> "Photos"
else -> permission
}
}

View File

@@ -0,0 +1,23 @@
package ai.openclaw.app
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import androidx.core.content.ContextCompat
internal fun photoReadPermissionsForRequest(): List<String> =
when {
Build.VERSION.SDK_INT >= 34 ->
listOf(
Manifest.permission.READ_MEDIA_IMAGES,
Manifest.permission.READ_MEDIA_VISUAL_USER_SELECTED,
)
Build.VERSION.SDK_INT >= 33 -> listOf(Manifest.permission.READ_MEDIA_IMAGES)
else -> listOf(Manifest.permission.READ_EXTERNAL_STORAGE)
}
internal fun hasPhotoReadPermission(context: Context): Boolean =
photoReadPermissionsForRequest().any { permission ->
ContextCompat.checkSelfPermission(context, permission) == PackageManager.PERMISSION_GRANTED
}

View File

@@ -0,0 +1,606 @@
@file:Suppress("DEPRECATION")
package ai.openclaw.app
import android.content.Context
import android.content.SharedPreferences
import androidx.core.content.edit
import androidx.security.crypto.EncryptedSharedPreferences
import androidx.security.crypto.MasterKey
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonPrimitive
import java.util.UUID
/**
* Reactive settings facade for Android node preferences and encrypted gateway credentials.
*/
class SecurePrefs(
context: Context,
private val securePrefsOverride: SharedPreferences? = null,
) {
companion object {
val defaultWakeWords: List<String> = listOf("openclaw", "claude")
private const val displayNameKey = "node.displayName"
private const val locationModeKey = "location.enabledMode"
private const val voiceWakeModeKey = "voiceWake.mode"
private const val plainPrefsName = "openclaw.node"
private const val securePrefsName = "openclaw.node.secure"
private const val notificationsForwardingEnabledKey = "notifications.forwarding.enabled"
private const val defaultNotificationForwardingEnabled = false
private const val notificationsForwardingModeKey = "notifications.forwarding.mode"
private const val notificationsForwardingPackagesKey = "notifications.forwarding.packages"
private const val notificationsForwardingQuietHoursEnabledKey =
"notifications.forwarding.quietHoursEnabled"
private const val notificationsForwardingQuietStartKey = "notifications.forwarding.quietStart"
private const val notificationsForwardingQuietEndKey = "notifications.forwarding.quietEnd"
private const val notificationsForwardingMaxEventsPerMinuteKey =
"notifications.forwarding.maxEventsPerMinute"
private const val notificationsForwardingSessionKeyKey = "notifications.forwarding.sessionKey"
private const val installedAppsSharingEnabledKey = "device.apps.sharing.enabled"
private const val cameraEnabledKey = "camera.enabled"
private const val voiceMicEnabledKey = "voice.micEnabled"
private const val appearanceThemeModeKey = "appearance.themeMode"
}
private val appContext = context.applicationContext
private val json = Json { ignoreUnknownKeys = true }
// Non-secret UI/runtime preferences stay readable for migration and backup behavior.
private val plainPrefs: SharedPreferences =
appContext.getSharedPreferences(plainPrefsName, Context.MODE_PRIVATE)
private val hadPlainPrefsBeforeInit = plainPrefs.all.isNotEmpty()
// Gateway credentials and arbitrary secret strings are isolated behind EncryptedSharedPreferences.
private val masterKey by lazy {
MasterKey
.Builder(appContext)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
.build()
}
private val securePrefs: SharedPreferences by lazy { securePrefsOverride ?: createSecurePrefs(appContext, securePrefsName) }
private val _instanceId = MutableStateFlow(loadOrCreateInstanceId())
val instanceId: StateFlow<String> = _instanceId
private val _displayName =
MutableStateFlow(loadOrMigrateDisplayName(context = context))
val displayName: StateFlow<String> = _displayName
private val _cameraEnabled = MutableStateFlow(loadCameraEnabled())
val cameraEnabled: StateFlow<Boolean> = _cameraEnabled
private val _locationMode = MutableStateFlow(loadLocationMode())
val locationMode: StateFlow<LocationMode> = _locationMode
private val _locationPreciseEnabled =
MutableStateFlow(plainPrefs.getBoolean("location.preciseEnabled", true))
val locationPreciseEnabled: StateFlow<Boolean> = _locationPreciseEnabled
private val _preventSleep = MutableStateFlow(plainPrefs.getBoolean("screen.preventSleep", true))
val preventSleep: StateFlow<Boolean> = _preventSleep
private val _manualEnabled =
MutableStateFlow(plainPrefs.getBoolean("gateway.manual.enabled", false))
val manualEnabled: StateFlow<Boolean> = _manualEnabled
private val _manualHost =
MutableStateFlow(plainPrefs.getString("gateway.manual.host", "") ?: "")
val manualHost: StateFlow<String> = _manualHost
private val _manualPort =
MutableStateFlow(plainPrefs.getInt("gateway.manual.port", 18789))
val manualPort: StateFlow<Int> = _manualPort
private val _manualTls =
MutableStateFlow(plainPrefs.getBoolean("gateway.manual.tls", true))
val manualTls: StateFlow<Boolean> = _manualTls
private val _gatewayToken = MutableStateFlow("")
val gatewayToken: StateFlow<String> = _gatewayToken
private val _gatewayBootstrapToken = MutableStateFlow("")
val gatewayBootstrapToken: StateFlow<String> = _gatewayBootstrapToken
private val _onboardingCompleted =
MutableStateFlow(plainPrefs.getBoolean("onboarding.completed", false))
val onboardingCompleted: StateFlow<Boolean> = _onboardingCompleted
private val _lastDiscoveredStableId =
MutableStateFlow(
plainPrefs.getString("gateway.lastDiscoveredStableID", "") ?: "",
)
val lastDiscoveredStableId: StateFlow<String> = _lastDiscoveredStableId
private val _canvasDebugStatusEnabled =
MutableStateFlow(plainPrefs.getBoolean("canvas.debugStatusEnabled", false))
val canvasDebugStatusEnabled: StateFlow<Boolean> = _canvasDebugStatusEnabled
private val _installedAppsSharingEnabled =
MutableStateFlow(plainPrefs.getBoolean(installedAppsSharingEnabledKey, false))
val installedAppsSharingEnabled: StateFlow<Boolean> = _installedAppsSharingEnabled
private val _notificationForwardingEnabled =
MutableStateFlow(plainPrefs.getBoolean(notificationsForwardingEnabledKey, defaultNotificationForwardingEnabled))
val notificationForwardingEnabled: StateFlow<Boolean> = _notificationForwardingEnabled
private val _notificationForwardingMode =
MutableStateFlow(
NotificationPackageFilterMode.fromRawValue(
plainPrefs.getString(notificationsForwardingModeKey, null),
),
)
val notificationForwardingMode: StateFlow<NotificationPackageFilterMode> = _notificationForwardingMode
private val _notificationForwardingPackages = MutableStateFlow(loadNotificationForwardingPackages())
val notificationForwardingPackages: StateFlow<Set<String>> = _notificationForwardingPackages
private val storedQuietStart =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty())
?: "22:00"
private val storedQuietEnd =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty())
?: "07:00"
private val storedQuietHoursEnabled =
plainPrefs.getBoolean(notificationsForwardingQuietHoursEnabledKey, false) &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty()) != null &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty()) != null
private val _notificationForwardingQuietHoursEnabled =
MutableStateFlow(storedQuietHoursEnabled)
val notificationForwardingQuietHoursEnabled: StateFlow<Boolean> = _notificationForwardingQuietHoursEnabled
private val _notificationForwardingQuietStart = MutableStateFlow(storedQuietStart)
val notificationForwardingQuietStart: StateFlow<String> = _notificationForwardingQuietStart
private val _notificationForwardingQuietEnd = MutableStateFlow(storedQuietEnd)
val notificationForwardingQuietEnd: StateFlow<String> = _notificationForwardingQuietEnd
private val _notificationForwardingMaxEventsPerMinute =
MutableStateFlow(plainPrefs.getInt(notificationsForwardingMaxEventsPerMinuteKey, 20).coerceAtLeast(1))
val notificationForwardingMaxEventsPerMinute: StateFlow<Int> = _notificationForwardingMaxEventsPerMinute
private val _notificationForwardingSessionKey =
MutableStateFlow(
plainPrefs
.getString(notificationsForwardingSessionKeyKey, "")
?.trim()
?.takeIf { it.isNotEmpty() },
)
val notificationForwardingSessionKey: StateFlow<String?> = _notificationForwardingSessionKey
private val _wakeWords = MutableStateFlow(loadWakeWords())
val wakeWords: StateFlow<List<String>> = _wakeWords
private val _voiceWakeMode = MutableStateFlow(loadVoiceWakeMode())
val voiceWakeMode: StateFlow<VoiceWakeMode> = _voiceWakeMode
private val _voiceMicEnabled = MutableStateFlow(plainPrefs.getBoolean(voiceMicEnabledKey, false))
val voiceMicEnabled: StateFlow<Boolean> = _voiceMicEnabled
private val _speakerEnabled = MutableStateFlow(plainPrefs.getBoolean("voice.speakerEnabled", true))
val speakerEnabled: StateFlow<Boolean> = _speakerEnabled
private val _appearanceThemeMode =
MutableStateFlow(AppearanceThemeMode.fromRawValue(plainPrefs.getString(appearanceThemeModeKey, null)))
val appearanceThemeMode: StateFlow<AppearanceThemeMode> = _appearanceThemeMode
fun setLastDiscoveredStableId(value: String) {
val trimmed = value.trim()
plainPrefs.edit { putString("gateway.lastDiscoveredStableID", trimmed) }
_lastDiscoveredStableId.value = trimmed
}
fun setDisplayName(value: String) {
val trimmed = value.trim()
plainPrefs.edit { putString(displayNameKey, trimmed) }
_displayName.value = trimmed
}
fun setCameraEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(cameraEnabledKey, value) }
_cameraEnabled.value = value
}
fun setLocationMode(mode: LocationMode) {
plainPrefs.edit { putString(locationModeKey, mode.rawValue) }
_locationMode.value = mode
}
fun setLocationPreciseEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("location.preciseEnabled", value) }
_locationPreciseEnabled.value = value
}
fun setPreventSleep(value: Boolean) {
plainPrefs.edit { putBoolean("screen.preventSleep", value) }
_preventSleep.value = value
}
fun setManualEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("gateway.manual.enabled", value) }
_manualEnabled.value = value
}
fun setManualHost(value: String) {
val trimmed = value.trim()
plainPrefs.edit { putString("gateway.manual.host", trimmed) }
_manualHost.value = trimmed
}
fun setManualPort(value: Int) {
plainPrefs.edit { putInt("gateway.manual.port", value) }
_manualPort.value = value
}
fun setManualTls(value: Boolean) {
plainPrefs.edit { putBoolean("gateway.manual.tls", value) }
_manualTls.value = value
}
fun setGatewayToken(value: String) {
val trimmed = value.trim()
securePrefs.edit { putString("gateway.manual.token", trimmed) }
_gatewayToken.value = trimmed
}
fun setGatewayPassword(value: String) {
saveGatewayPassword(value)
}
fun setGatewayBootstrapToken(value: String) {
saveGatewayBootstrapToken(value)
}
fun setOnboardingCompleted(value: Boolean) {
plainPrefs.edit { putBoolean("onboarding.completed", value) }
_onboardingCompleted.value = value
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("canvas.debugStatusEnabled", value) }
_canvasDebugStatusEnabled.value = value
}
fun setInstalledAppsSharingEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(installedAppsSharingEnabledKey, value) }
_installedAppsSharingEnabled.value = value
}
internal fun getNotificationForwardingPolicy(appPackageName: String): NotificationForwardingPolicy {
val modeRaw = plainPrefs.getString(notificationsForwardingModeKey, null)
val mode = NotificationPackageFilterMode.fromRawValue(modeRaw)
val configuredPackages = loadNotificationForwardingPackages()
val normalizedAppPackage = appPackageName.trim()
// Always block OpenClaw's own notifications in blocklist mode to prevent forwarding loops.
val defaultBlockedPackages =
if (normalizedAppPackage.isNotEmpty()) setOf(normalizedAppPackage) else emptySet()
val packages =
when (mode) {
NotificationPackageFilterMode.Allowlist -> configuredPackages
NotificationPackageFilterMode.Blocklist -> configuredPackages + defaultBlockedPackages
}
val maxEvents = plainPrefs.getInt(notificationsForwardingMaxEventsPerMinuteKey, 20)
val quietStart =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty())
?: "22:00"
val quietEnd =
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty())
?: "07:00"
val sessionKey =
plainPrefs
.getString(notificationsForwardingSessionKeyKey, "")
?.trim()
?.takeIf { it.isNotEmpty() }
val quietHoursEnabled =
plainPrefs.getBoolean(notificationsForwardingQuietHoursEnabledKey, false) &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietStartKey, "22:00").orEmpty()) != null &&
normalizeLocalHourMinute(plainPrefs.getString(notificationsForwardingQuietEndKey, "07:00").orEmpty()) != null
return NotificationForwardingPolicy(
enabled = plainPrefs.getBoolean(notificationsForwardingEnabledKey, defaultNotificationForwardingEnabled),
mode = mode,
packages = packages,
quietHoursEnabled = quietHoursEnabled,
quietStart = quietStart,
quietEnd = quietEnd,
maxEventsPerMinute = maxEvents.coerceAtLeast(1),
sessionKey = sessionKey,
selfPackageName = normalizedAppPackage,
)
}
internal fun setNotificationForwardingEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(notificationsForwardingEnabledKey, value) }
_notificationForwardingEnabled.value = value
}
internal fun setNotificationForwardingMode(mode: NotificationPackageFilterMode) {
plainPrefs.edit { putString(notificationsForwardingModeKey, mode.rawValue) }
_notificationForwardingMode.value = mode
}
internal fun setNotificationForwardingPackages(packages: List<String>) {
val sanitized =
packages
.asSequence()
.map { it.trim() }
.filter { it.isNotEmpty() }
.toSet()
.toList()
.sorted()
// Persist deterministic JSON so settings diffs and state restoration are stable.
val encoded = JsonArray(sanitized.map { JsonPrimitive(it) }).toString()
plainPrefs.edit { putString(notificationsForwardingPackagesKey, encoded) }
_notificationForwardingPackages.value = sanitized.toSet()
}
internal fun setNotificationForwardingQuietHours(
enabled: Boolean,
start: String,
end: String,
): Boolean {
if (!enabled) {
plainPrefs.edit { putBoolean(notificationsForwardingQuietHoursEnabledKey, false) }
_notificationForwardingQuietHoursEnabled.value = false
return true
}
val normalizedStart = normalizeLocalHourMinute(start) ?: return false
val normalizedEnd = normalizeLocalHourMinute(end) ?: return false
plainPrefs.edit {
putBoolean(notificationsForwardingQuietHoursEnabledKey, enabled)
putString(notificationsForwardingQuietStartKey, normalizedStart)
putString(notificationsForwardingQuietEndKey, normalizedEnd)
}
_notificationForwardingQuietHoursEnabled.value = enabled
_notificationForwardingQuietStart.value = normalizedStart
_notificationForwardingQuietEnd.value = normalizedEnd
return true
}
internal fun setNotificationForwardingMaxEventsPerMinute(value: Int) {
val normalized = value.coerceAtLeast(1)
plainPrefs.edit {
putInt(notificationsForwardingMaxEventsPerMinuteKey, normalized)
}
_notificationForwardingMaxEventsPerMinute.value = normalized
}
internal fun setNotificationForwardingSessionKey(value: String?) {
val normalized = value?.trim()?.takeIf { it.isNotEmpty() }
plainPrefs.edit {
putString(notificationsForwardingSessionKeyKey, normalized.orEmpty())
}
_notificationForwardingSessionKey.value = normalized
}
/** Loads manual or instance-scoped gateway token material from encrypted preferences. */
fun loadGatewayToken(): String? {
val manual =
_gatewayToken.value.trim().ifEmpty {
val stored = securePrefs.getString("gateway.manual.token", null)?.trim().orEmpty()
if (stored.isNotEmpty()) _gatewayToken.value = stored
stored
}
if (manual.isNotEmpty()) return manual
// Per-instance tokens keep reused Android installs from sharing stale gateway auth.
val key = "gateway.token.${_instanceId.value}"
val stored = securePrefs.getString(key, null)?.trim()
return stored?.takeIf { it.isNotEmpty() }
}
/** Loads the bootstrap token used during gateway setup and device-token handoff. */
fun loadGatewayBootstrapToken(): String? {
val key = "gateway.bootstrapToken.${_instanceId.value}"
val stored =
_gatewayBootstrapToken.value.trim().ifEmpty {
val persisted = securePrefs.getString(key, null)?.trim().orEmpty()
if (persisted.isNotEmpty()) {
_gatewayBootstrapToken.value = persisted
}
persisted
}
return stored.takeIf { it.isNotEmpty() }
}
fun saveGatewayBootstrapToken(token: String) {
val key = "gateway.bootstrapToken.${_instanceId.value}"
val trimmed = token.trim()
securePrefs.edit { putString(key, trimmed) }
_gatewayBootstrapToken.value = trimmed
}
fun loadGatewayPassword(): String? {
val key = "gateway.password.${_instanceId.value}"
val stored = securePrefs.getString(key, null)?.trim()
return stored?.takeIf { it.isNotEmpty() }
}
fun saveGatewayPassword(password: String) {
val key = "gateway.password.${_instanceId.value}"
securePrefs.edit { putString(key, password.trim()) }
}
/** Clears manual/setup credentials without removing persisted role-specific device tokens. */
fun clearGatewaySetupAuth() {
val instanceId = _instanceId.value
securePrefs.edit {
// Clear both current manual credentials and instance-scoped setup credentials after pairing/reset.
remove("gateway.manual.token")
remove("gateway.token.$instanceId")
remove("gateway.bootstrapToken.$instanceId")
remove("gateway.password.$instanceId")
}
_gatewayToken.value = ""
_gatewayBootstrapToken.value = ""
}
/** Loads the pinned gateway TLS fingerprint for a discovered/manual stable endpoint id. */
fun loadGatewayTlsFingerprint(stableId: String): String? {
val key = "gateway.tls.$stableId"
return plainPrefs.getString(key, null)?.trim()?.takeIf { it.isNotEmpty() }
}
/** Persists the gateway TLS fingerprint captured through TOFU or explicit trust. */
fun saveGatewayTlsFingerprint(
stableId: String,
fingerprint: String,
) {
val key = "gateway.tls.$stableId"
plainPrefs.edit { putString(key, fingerprint.trim()) }
}
fun getString(key: String): String? = securePrefs.getString(key, null)
fun putString(
key: String,
value: String,
) {
securePrefs.edit { putString(key, value) }
}
fun remove(key: String) {
securePrefs.edit { remove(key) }
}
private fun createSecurePrefs(
context: Context,
name: String,
): SharedPreferences =
EncryptedSharedPreferences.create(
context,
name,
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM,
)
private fun loadOrCreateInstanceId(): String {
val existing = plainPrefs.getString("node.instanceId", null)?.trim()
if (!existing.isNullOrBlank()) return existing
// Instance id is not secret; it scopes local credentials and survives display-name changes.
val fresh = UUID.randomUUID().toString()
plainPrefs.edit { putString("node.instanceId", fresh) }
return fresh
}
private fun loadOrMigrateDisplayName(context: Context): String {
val existing = plainPrefs.getString(displayNameKey, null)?.trim().orEmpty()
if (existing.isNotEmpty() && existing != "Android Node") return existing
// Replace the historical generic name with a device-specific default once.
val candidate = DeviceNames.bestDefaultNodeName(context).trim()
val resolved = candidate.ifEmpty { "Android Node" }
plainPrefs.edit { putString(displayNameKey, resolved) }
return resolved
}
/** Persists sanitized voice wake triggers and updates the reactive settings flow. */
fun setWakeWords(words: List<String>) {
val sanitized = WakeWords.sanitize(words, defaultWakeWords)
val encoded =
JsonArray(sanitized.map { JsonPrimitive(it) }).toString()
plainPrefs.edit { putString("voiceWake.triggerWords", encoded) }
_wakeWords.value = sanitized
}
fun setVoiceWakeMode(mode: VoiceWakeMode) {
plainPrefs.edit { putString(voiceWakeModeKey, mode.rawValue) }
_voiceWakeMode.value = mode
}
fun setVoiceMicEnabled(value: Boolean) {
plainPrefs.edit { putBoolean(voiceMicEnabledKey, value) }
_voiceMicEnabled.value = value
}
fun setSpeakerEnabled(value: Boolean) {
plainPrefs.edit { putBoolean("voice.speakerEnabled", value) }
_speakerEnabled.value = value
}
fun setAppearanceThemeMode(mode: AppearanceThemeMode) {
plainPrefs.edit { putString(appearanceThemeModeKey, mode.rawValue) }
_appearanceThemeMode.value = mode
}
private fun loadNotificationForwardingPackages(): Set<String> {
val raw = plainPrefs.getString(notificationsForwardingPackagesKey, null)?.trim()
if (raw.isNullOrEmpty()) {
return emptySet()
}
return try {
val element = json.parseToJsonElement(raw)
val array = element as? JsonArray ?: return emptySet()
array
.mapNotNull { item ->
when (item) {
is JsonNull -> null
is JsonPrimitive -> item.content.trim().takeIf { it.isNotEmpty() }
else -> null
}
}.toSet()
} catch (_: Throwable) {
emptySet()
}
}
private fun loadVoiceWakeMode(): VoiceWakeMode {
val raw = plainPrefs.getString(voiceWakeModeKey, null)
val resolved = VoiceWakeMode.fromRawValue(raw)
// Default ON (foreground) when unset, but keep "always" opt-in through explicit settings.
if (raw.isNullOrBlank()) {
plainPrefs.edit { putString(voiceWakeModeKey, resolved.rawValue) }
}
return resolved
}
private fun loadLocationMode(): LocationMode {
val raw = plainPrefs.getString(locationModeKey, "off")
val resolved = LocationMode.fromRawValue(raw)
if (raw?.trim()?.lowercase() == "always") {
// Migrate old "always" configs to the current while-using contract.
plainPrefs.edit { putString(locationModeKey, resolved.rawValue) }
}
return resolved
}
private fun loadCameraEnabled(): Boolean {
if (plainPrefs.contains(cameraEnabledKey)) {
return plainPrefs.getBoolean(cameraEnabledKey, false)
}
val migratedValue = hadPlainPrefsBeforeInit
plainPrefs.edit { putBoolean(cameraEnabledKey, migratedValue) }
return migratedValue
}
private fun loadWakeWords(): List<String> {
val raw = plainPrefs.getString("voiceWake.triggerWords", null)?.trim()
if (raw.isNullOrEmpty()) return defaultWakeWords
return try {
val element = json.parseToJsonElement(raw)
val array = element as? JsonArray ?: return defaultWakeWords
val decoded =
array.mapNotNull { item ->
when (item) {
is JsonNull -> null
is JsonPrimitive -> item.content.trim().takeIf { it.isNotEmpty() }
else -> null
}
}
WakeWords.sanitize(decoded, defaultWakeWords)
} catch (_: Throwable) {
defaultWakeWords
}
}
}

View File

@@ -0,0 +1,27 @@
package ai.openclaw.app
/** Normalizes blank gateway session keys to the legacy main session alias. */
internal fun normalizeMainKey(raw: String?): String {
val trimmed = raw?.trim()
return if (!trimmed.isNullOrEmpty()) trimmed else "main"
}
/** Extracts the agent id from canonical agent-scoped main session keys. */
internal fun resolveAgentIdFromMainSessionKey(raw: String?): String? {
val trimmed = raw?.trim().orEmpty()
if (!trimmed.startsWith("agent:")) return null
return trimmed
.removePrefix("agent:")
.substringBefore(':')
.trim()
.ifEmpty { null }
}
/** Builds the node session key shape consumed by gateway chat and presence APIs. */
internal fun buildNodeMainSessionKey(
deviceId: String,
agentId: String?,
): String {
val resolvedAgentId = agentId?.trim().orEmpty().ifEmpty { "main" }
return "agent:$resolvedAgentId:node-${deviceId.take(12)}"
}

View File

@@ -0,0 +1,10 @@
package ai.openclaw.app
/**
* Persisted voice capture mode that controls foreground-service microphone requirements.
*/
enum class VoiceCaptureMode {
Off,
ManualMic,
TalkMode,
}

View File

@@ -0,0 +1,20 @@
package ai.openclaw.app
/**
* Persisted wake-word mode; raw values are stored in secure preferences.
*/
enum class VoiceWakeMode(
val rawValue: String,
) {
Off("off"),
Foreground("foreground"),
Always("always"),
;
companion object {
/**
* Invalid stored values fall back to foreground wake so hands-free behavior stays opt-in.
*/
fun fromRawValue(raw: String?): VoiceWakeMode = entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Foreground
}
}

View File

@@ -0,0 +1,35 @@
package ai.openclaw.app
/**
* Wake-word parsing limits and sanitizers shared by settings and voice runtime paths.
*/
object WakeWords {
const val maxWords: Int = 32
const val maxWordLength: Int = 64
/** Splits comma-separated user input into non-empty wake-word entries. */
fun parseCommaSeparated(input: String): List<String> = input.split(",").map { it.trim() }.filter { it.isNotEmpty() }
/** Returns null when edited text normalizes to the current wake-word list. */
fun parseIfChanged(
input: String,
current: List<String>,
): List<String>? {
val parsed = parseCommaSeparated(input)
return if (parsed == current) null else parsed
}
/** Applies persisted-list bounds and falls back to defaults when all entries are empty. */
fun sanitize(
words: List<String>,
defaults: List<String>,
): List<String> {
val cleaned =
words
.map { it.trim() }
.filter { it.isNotEmpty() }
.take(maxWords)
.map { it.take(maxWordLength) }
return cleaned.ifEmpty { defaults }
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,80 @@
package ai.openclaw.app.chat
/**
* Chat transcript item as delivered by gateway chat history and live chat events.
*/
data class ChatMessage(
val id: String,
val role: String,
val content: List<ChatMessageContent>,
val timestampMs: Long?,
val idempotencyKey: String? = null,
)
/**
* One content part in a chat message; binary parts carry base64 plus their MIME metadata.
*/
data class ChatMessageContent(
val type: String = "text",
val text: String? = null,
val mimeType: String? = null,
val fileName: String? = null,
val base64: String? = null,
)
/**
* Tool call placeholder shown while a gateway run is still streaming.
*/
data class ChatPendingToolCall(
val toolCallId: String,
val name: String,
val args: kotlinx.serialization.json.JsonObject? = null,
val startedAtMs: Long,
val isError: Boolean? = null,
)
/**
* Stable session selector row; [key] is the gateway session key used in chat requests.
*/
data class ChatSessionEntry(
val key: String,
val updatedAtMs: Long?,
val displayName: String? = null,
val totalTokens: Long? = null,
val totalTokensFresh: Boolean? = null,
val contextTokens: Long? = null,
val hasContextUsageMetadata: Boolean = totalTokens != null || totalTokensFresh != null || contextTokens != null,
)
/**
* Slash command metadata exposed by the gateway for text-surface chat clients.
*/
data class ChatCommandEntry(
val name: String,
val description: String,
val category: String? = null,
val textAliases: List<String> = emptyList(),
val acceptsArgs: Boolean = false,
)
/**
* Snapshot of one chat session, including optional thinking level selected on the gateway.
*/
data class ChatHistory(
val sessionKey: String,
val sessionId: String?,
val thinkingLevel: String?,
val messages: List<ChatMessage>,
val sessionInfo: ChatSessionEntry? = null,
)
/**
* User-selected attachment payload sent to the gateway as inline base64.
*/
data class OutgoingAttachment(
val type: String,
val mimeType: String,
val fileName: String,
val base64: String,
)

View File

@@ -0,0 +1,40 @@
package ai.openclaw.app.gateway
/**
* Decoder for Bonjour DNS-SD service names returned with decimal byte escapes.
*/
object BonjourEscapes {
/** Decodes Bonjour DNS-SD decimal escapes while preserving ordinary UTF-8. */
fun decode(input: String): String {
if (input.isEmpty()) return input
val bytes = mutableListOf<Byte>()
var i = 0
while (i < input.length) {
if (input[i] == '\\' && i + 3 < input.length) {
val d0 = input[i + 1]
val d1 = input[i + 2]
val d2 = input[i + 3]
if (d0.isDigit() && d1.isDigit() && d2.isDigit()) {
val value =
((d0.code - '0'.code) * 100) + ((d1.code - '0'.code) * 10) + (d2.code - '0'.code)
if (value in 0..255) {
// Bonjour escape bytes are decimal octets, not Unicode code points.
bytes.add(value.toByte())
i += 4
continue
}
}
}
val codePoint = Character.codePointAt(input, i)
val charBytes = String(Character.toChars(codePoint)).toByteArray(Charsets.UTF_8)
for (b in charBytes) {
bytes.add(b)
}
i += Character.charCount(codePoint)
}
return String(bytes.toByteArray(), Charsets.UTF_8)
}
}

View File

@@ -0,0 +1,46 @@
package ai.openclaw.app.gateway
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
internal data class ChatSendAck(
val runId: String?,
val status: String?,
) {
val normalizedStatus: String
get() = status?.trim()?.lowercase().orEmpty()
val isTerminalSuccess: Boolean
get() = normalizedStatus == "ok"
val isTerminalFailure: Boolean
get() = normalizedStatus == "timeout" || normalizedStatus == "error"
val isTerminal: Boolean
get() = isTerminalSuccess || isTerminalFailure
}
internal fun chatSendAckHistorySinceSeconds(
ack: ChatSendAck,
startedAtSeconds: Double,
): Double? = if (ack.isTerminalSuccess) null else startedAtSeconds
internal fun parseChatSendAck(
json: Json,
responseJson: String,
): ChatSendAck =
try {
val obj = json.parseToJsonElement(responseJson).asObjectOrNull()
ChatSendAck(
runId = obj?.get("runId").asStringOrNull(),
status = obj?.get("status").asStringOrNull(),
)
} catch (_: Throwable) {
ChatSendAck(runId = null, status = null)
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonElement?.asStringOrNull(): String? = (this as? JsonPrimitive)?.takeIf { it.isString }?.content

View File

@@ -0,0 +1,57 @@
package ai.openclaw.app.gateway
/**
* Canonical device-auth payload builder shared with gateway verification rules.
*/
internal object DeviceAuthPayload {
/** Builds the canonical v3 auth string signed by device registration flows. */
fun buildV3(
deviceId: String,
clientId: String,
clientMode: String,
role: String,
scopes: List<String>,
signedAtMs: Long,
token: String?,
nonce: String,
platform: String?,
deviceFamily: String?,
): String {
val scopeString = scopes.joinToString(",")
val authToken = token.orEmpty()
val platformNorm = normalizeMetadataField(platform)
val deviceFamilyNorm = normalizeMetadataField(deviceFamily)
return listOf(
"v3",
deviceId,
clientId,
clientMode,
role,
scopeString,
signedAtMs.toString(),
authToken,
nonce,
platformNorm,
deviceFamilyNorm,
).joinToString("|")
}
/** Normalizes signed metadata fields without locale-sensitive lowercasing. */
internal fun normalizeMetadataField(value: String?): String {
val trimmed = value?.trim().orEmpty()
if (trimmed.isEmpty()) {
return ""
}
// Keep cross-runtime normalization deterministic (TS/Swift/Kotlin):
// lowercase ASCII A-Z only for auth payload metadata fields.
val out = StringBuilder(trimmed.length)
for (ch in trimmed) {
if (ch in 'A'..'Z') {
out.append((ch.code + 32).toChar())
} else {
out.append(ch)
}
}
return out.toString()
}
}

View File

@@ -0,0 +1,142 @@
package ai.openclaw.app.gateway
import ai.openclaw.app.SecurePrefs
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
/** Stored gateway device-token material scoped by device id and role. */
data class DeviceAuthEntry(
val token: String,
val role: String,
val scopes: List<String>,
val updatedAtMs: Long,
)
@Serializable
private data class PersistedDeviceAuthMetadata(
val scopes: List<String> = emptyList(),
val updatedAtMs: Long = 0L,
)
/** Persistence interface used by gateway pairing/session code for role tokens. */
interface DeviceAuthTokenStore {
/** Loads the stored token plus metadata for one device/role pair. */
fun loadEntry(
deviceId: String,
role: String,
): DeviceAuthEntry?
/** Loads only the bearer token when callers do not need scope metadata. */
fun loadToken(
deviceId: String,
role: String,
): String? = loadEntry(deviceId, role)?.token
/** Persists a role token and deterministic scope metadata under normalized keys. */
fun saveToken(
deviceId: String,
role: String,
token: String,
scopes: List<String> = emptyList(),
)
/** Removes both token and metadata for the normalized device/role pair. */
fun clearToken(
deviceId: String,
role: String,
)
}
/** SecurePrefs-backed implementation of Android gateway device-token storage. */
class DeviceAuthStore(
private val prefs: SecurePrefs,
) : DeviceAuthTokenStore {
private val json = Json { ignoreUnknownKeys = true }
override fun loadEntry(
deviceId: String,
role: String,
): DeviceAuthEntry? {
val key = tokenKey(deviceId, role)
val token = prefs.getString(key)?.trim()?.takeIf { it.isNotEmpty() } ?: return null
val normalizedRole = normalizeRole(role)
val metadata =
prefs
.getString(metadataKey(deviceId, role))
?.let { raw ->
runCatching { json.decodeFromString<PersistedDeviceAuthMetadata>(raw) }.getOrNull()
}
return DeviceAuthEntry(
token = token,
role = normalizedRole,
scopes = metadata?.scopes ?: emptyList(),
updatedAtMs = metadata?.updatedAtMs ?: 0L,
)
}
override fun saveToken(
deviceId: String,
role: String,
token: String,
scopes: List<String>,
) {
val normalizedScopes = normalizeScopes(scopes)
val key = tokenKey(deviceId, role)
prefs.putString(key, token.trim())
prefs.putString(
metadataKey(deviceId, role),
json.encodeToString(
PersistedDeviceAuthMetadata(
scopes = normalizedScopes,
updatedAtMs = System.currentTimeMillis(),
),
),
)
}
override fun clearToken(
deviceId: String,
role: String,
) {
val key = tokenKey(deviceId, role)
prefs.remove(key)
prefs.remove(metadataKey(deviceId, role))
}
private fun tokenKey(
deviceId: String,
role: String,
): String {
val normalizedDevice = normalizeDeviceId(deviceId)
val normalizedRole = normalizeRole(role)
// Keep key normalization shared with metadata keys so token and metadata
// are added/removed as one logical auth entry.
return "gateway.deviceToken.$normalizedDevice.$normalizedRole"
}
private fun metadataKey(
deviceId: String,
role: String,
): String {
val normalizedDevice = normalizeDeviceId(deviceId)
val normalizedRole = normalizeRole(role)
return "gateway.deviceTokenMeta.$normalizedDevice.$normalizedRole"
}
/** Normalizes device ids before they become encrypted preference key segments. */
private fun normalizeDeviceId(deviceId: String): String = deviceId.trim().lowercase()
/** Normalizes role names so node/operator token slots are stable across callers. */
private fun normalizeRole(role: String): String = role.trim().lowercase()
/** Stores scopes in deterministic order for display and restart comparisons. */
private fun normalizeScopes(scopes: List<String>): List<String> =
scopes
.map { it.trim() }
.filter { it.isNotEmpty() }
// Persist deterministic scope lists because they are displayed and may be
// compared across process restarts.
.distinct()
.sorted()
}

View File

@@ -0,0 +1,211 @@
package ai.openclaw.app.gateway
import android.content.Context
import android.util.Base64
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import java.io.File
import java.security.MessageDigest
/** Persistent Ed25519 identity used to register this Android node with gateways. */
@Serializable
data class DeviceIdentity(
val deviceId: String,
val publicKeyRawBase64: String,
val privateKeyPkcs8Base64: String,
val createdAtMs: Long,
)
/** Owns device identity generation, persistence, and auth payload signatures. */
class DeviceIdentityStore(
context: Context,
) {
private val json = Json { ignoreUnknownKeys = true }
private val identityFile = File(context.filesDir, "openclaw/identity/device.json")
@Volatile private var cachedIdentity: DeviceIdentity? = null
/** Loads the persisted identity or creates one, repairing old device-id drift. */
@Synchronized
fun loadOrCreate(): DeviceIdentity {
cachedIdentity?.let { return it }
val existing = load()
if (existing != null) {
val derived = deriveDeviceId(existing.publicKeyRawBase64)
if (derived != null && derived != existing.deviceId) {
val updated = existing.copy(deviceId = derived)
save(updated)
cachedIdentity = updated
return updated
}
cachedIdentity = existing
return existing
}
val fresh = generate()
save(fresh)
cachedIdentity = fresh
return fresh
}
/** Signs gateway connect payload text with the persisted Ed25519 private key. */
fun signPayload(
payload: String,
identity: DeviceIdentity,
): String? =
try {
// Use BC lightweight API directly; R8 can break JCA provider registration.
val privateKeyBytes = Base64.decode(identity.privateKeyPkcs8Base64, Base64.DEFAULT)
val pkInfo =
org.bouncycastle.asn1.pkcs.PrivateKeyInfo
.getInstance(privateKeyBytes)
val parsed = pkInfo.parsePrivateKey()
val rawPrivate =
org.bouncycastle.asn1.DEROctetString
.getInstance(parsed)
.octets
val privateKey =
org.bouncycastle.crypto.params
.Ed25519PrivateKeyParameters(rawPrivate, 0)
val signer =
org.bouncycastle.crypto.signers
.Ed25519Signer()
signer.init(true, privateKey)
val payloadBytes = payload.toByteArray(Charsets.UTF_8)
signer.update(payloadBytes, 0, payloadBytes.size)
base64UrlEncode(signer.generateSignature())
} catch (e: Throwable) {
android.util.Log.e("DeviceAuth", "signPayload FAILED: ${e.javaClass.simpleName}: ${e.message}", e)
null
}
/** Verifies a signature against the persisted public key for debug diagnostics. */
fun verifySelfSignature(
payload: String,
signatureBase64Url: String,
identity: DeviceIdentity,
): Boolean =
try {
val rawPublicKey = Base64.decode(identity.publicKeyRawBase64, Base64.DEFAULT)
val pubKey =
org.bouncycastle.crypto.params
.Ed25519PublicKeyParameters(rawPublicKey, 0)
val sigBytes = base64UrlDecode(signatureBase64Url)
val verifier =
org.bouncycastle.crypto.signers
.Ed25519Signer()
verifier.init(false, pubKey)
val payloadBytes = payload.toByteArray(Charsets.UTF_8)
verifier.update(payloadBytes, 0, payloadBytes.size)
verifier.verifySignature(sigBytes)
} catch (e: Throwable) {
android.util.Log.e("DeviceAuth", "self-verify exception: ${e.message}", e)
false
}
/** Decodes gateway URL-safe base64 signatures, accepting unpadded input. */
private fun base64UrlDecode(input: String): ByteArray {
val normalized = input.replace('-', '+').replace('_', '/')
// Android Base64 expects padded input; gateway signatures are URL-safe
// unpadded strings.
val padded = normalized + "=".repeat((4 - normalized.length % 4) % 4)
return Base64.decode(padded, Base64.DEFAULT)
}
/** Returns the public key in the gateway's unpadded URL-safe base64 format. */
fun publicKeyBase64Url(identity: DeviceIdentity): String? =
try {
val raw = Base64.decode(identity.publicKeyRawBase64, Base64.DEFAULT)
base64UrlEncode(raw)
} catch (_: Throwable) {
null
}
private fun load(): DeviceIdentity? = readIdentity(identityFile)
private fun readIdentity(file: File): DeviceIdentity? {
return try {
if (!file.exists()) return null
val raw = file.readText(Charsets.UTF_8)
val decoded = json.decodeFromString(DeviceIdentity.serializer(), raw)
if (decoded.deviceId.isBlank() ||
decoded.publicKeyRawBase64.isBlank() ||
decoded.privateKeyPkcs8Base64.isBlank()
) {
null
} else {
decoded
}
} catch (_: Throwable) {
null
}
}
private fun save(identity: DeviceIdentity) {
try {
identityFile.parentFile?.mkdirs()
val encoded = json.encodeToString(DeviceIdentity.serializer(), identity)
identityFile.writeText(encoded, Charsets.UTF_8)
} catch (_: Throwable) {
// best-effort only
}
}
private fun generate(): DeviceIdentity {
// Use BC lightweight API directly to avoid JCA provider issues with R8.
val kpGen =
org.bouncycastle.crypto.generators
.Ed25519KeyPairGenerator()
kpGen.init(
org.bouncycastle.crypto.params
.Ed25519KeyGenerationParameters(java.security.SecureRandom()),
)
val kp = kpGen.generateKeyPair()
val pubKey = kp.public as org.bouncycastle.crypto.params.Ed25519PublicKeyParameters
val privKey = kp.private as org.bouncycastle.crypto.params.Ed25519PrivateKeyParameters
val rawPublic = pubKey.encoded // 32 bytes
val deviceId = sha256Hex(rawPublic)
// Store private key as PKCS8 so signPayload can parse the same persisted
// shape after app restarts and upgrades.
val privKeyInfo =
org.bouncycastle.crypto.util.PrivateKeyInfoFactory
.createPrivateKeyInfo(privKey)
val pkcs8Bytes = privKeyInfo.encoded
return DeviceIdentity(
deviceId = deviceId,
publicKeyRawBase64 = Base64.encodeToString(rawPublic, Base64.NO_WRAP),
privateKeyPkcs8Base64 = Base64.encodeToString(pkcs8Bytes, Base64.NO_WRAP),
createdAtMs = System.currentTimeMillis(),
)
}
/** Re-derives the stable device id from the raw Ed25519 public key bytes. */
private fun deriveDeviceId(publicKeyRawBase64: String): String? =
try {
val raw = Base64.decode(publicKeyRawBase64, Base64.DEFAULT)
sha256Hex(raw)
} catch (_: Throwable) {
null
}
private fun sha256Hex(data: ByteArray): String {
val digest = MessageDigest.getInstance("SHA-256").digest(data)
val out = CharArray(digest.size * 2)
var i = 0
for (byte in digest) {
val v = byte.toInt() and 0xff
out[i++] = HEX[v ushr 4]
out[i++] = HEX[v and 0x0f]
}
return String(out)
}
private fun base64UrlEncode(data: ByteArray): String =
Base64.encodeToString(
data,
Base64.URL_SAFE or Base64.NO_WRAP or Base64.NO_PADDING,
)
companion object {
private val HEX = "0123456789abcdef".toCharArray()
}
}

View File

@@ -0,0 +1,689 @@
package ai.openclaw.app.gateway
import android.content.Context
import android.net.ConnectivityManager
import android.net.DnsResolver
import android.net.Network
import android.net.NetworkCapabilities
import android.net.NetworkRequest
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import android.os.Build
import android.os.CancellationSignal
import android.util.Log
import androidx.annotation.RequiresApi
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.coroutines.suspendCancellableCoroutine
import org.xbill.DNS.AAAARecord
import org.xbill.DNS.ARecord
import org.xbill.DNS.DClass
import org.xbill.DNS.ExtendedResolver
import org.xbill.DNS.Message
import org.xbill.DNS.Name
import org.xbill.DNS.PTRRecord
import org.xbill.DNS.Rcode
import org.xbill.DNS.Record
import org.xbill.DNS.Resolver
import org.xbill.DNS.SRVRecord
import org.xbill.DNS.Section
import org.xbill.DNS.SimpleResolver
import org.xbill.DNS.TXTRecord
import org.xbill.DNS.TextParseException
import org.xbill.DNS.Type
import java.io.IOException
import java.net.InetAddress
import java.net.InetSocketAddress
import java.nio.ByteBuffer
import java.nio.charset.CodingErrorAction
import java.time.Duration
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.Executor
import java.util.concurrent.Executors
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
@Suppress("DEPRECATION")
private fun createDnsResolver(): DnsResolver = DnsResolver.getInstance()
/**
* Watches local DNS-SD and optional wide-area DNS-SD for reachable OpenClaw gateways.
*/
class GatewayDiscovery(
context: Context,
private val scope: CoroutineScope,
) {
private val nsd = context.getSystemService(NsdManager::class.java)
private val connectivity = context.getSystemService(ConnectivityManager::class.java)
private val dns = createDnsResolver()
private val serviceType = "_openclaw-gw._tcp."
private val wideAreaDomain = System.getenv("OPENCLAW_WIDE_AREA_DOMAIN")
private val logTag = "OpenClaw/GatewayDiscovery"
private val localById = ConcurrentHashMap<String, GatewayEndpoint>()
private val unicastById = ConcurrentHashMap<String, GatewayEndpoint>()
private val _gateways = MutableStateFlow<List<GatewayEndpoint>>(emptyList())
/** Current discovered gateway list, merged from local DNS-SD and optional wide-area DNS-SD. */
val gateways: StateFlow<List<GatewayEndpoint>> = _gateways.asStateFlow()
private val _statusText = MutableStateFlow("Searching…")
/** Short diagnostic text shown by connect UI while discovery is running. */
val statusText: StateFlow<String> = _statusText.asStateFlow()
private var unicastJob: Job? = null
private val dnsExecutor: Executor = Executors.newCachedThreadPool()
private val availableNetworks = ConcurrentHashMap.newKeySet<Network>()
private val serviceInfoCallbacks = ConcurrentHashMap<String, Any>()
@Volatile private var lastWideAreaRcode: Int? = null
@Volatile private var lastWideAreaCount: Int = 0
private val networkCallback =
object : ConnectivityManager.NetworkCallback() {
override fun onAvailable(network: Network) {
availableNetworks.add(network)
}
override fun onLost(network: Network) {
availableNetworks.remove(network)
}
}
private val discoveryListener =
object : NsdManager.DiscoveryListener {
override fun onStartDiscoveryFailed(
serviceType: String,
errorCode: Int,
) {}
override fun onStopDiscoveryFailed(
serviceType: String,
errorCode: Int,
) {}
override fun onDiscoveryStarted(serviceType: String) {}
override fun onDiscoveryStopped(serviceType: String) {}
override fun onServiceFound(serviceInfo: NsdServiceInfo) {
if (serviceInfo.serviceType != this@GatewayDiscovery.serviceType) return
resolve(serviceInfo)
}
override fun onServiceLost(serviceInfo: NsdServiceInfo) {
val serviceName = BonjourEscapes.decode(serviceInfo.serviceName)
val id = stableId(serviceName, "local.")
localById.remove(id)
unregisterServiceInfoCallback(id)
publish()
}
}
init {
startNetworkTracking()
startLocalDiscovery()
if (!wideAreaDomain.isNullOrBlank()) {
startUnicastDiscovery(wideAreaDomain)
}
}
private fun startNetworkTracking() {
val cm = connectivity ?: return
cm.activeNetwork?.let(availableNetworks::add)
try {
// Track all networks so wide-area DNS can prefer VPN/split-DNS answers
// even when Android's active network is not the VPN.
cm.registerNetworkCallback(NetworkRequest.Builder().build(), networkCallback)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun startLocalDiscovery() {
try {
nsd.discoverServices(serviceType, NsdManager.PROTOCOL_DNS_SD, discoveryListener)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun startUnicastDiscovery(domain: String) {
unicastJob =
scope.launch(Dispatchers.IO) {
while (true) {
try {
refreshUnicast(domain)
} catch (_: Throwable) {
// ignore (best-effort)
}
delay(5000)
}
}
}
private fun resolve(serviceInfo: NsdServiceInfo) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
// Android 14+ streams service updates; older releases require one-shot resolve calls.
resolveWithServiceInfoCallback(serviceInfo)
} else {
resolveLegacy(serviceInfo)
}
}
@RequiresApi(Build.VERSION_CODES.UPSIDE_DOWN_CAKE)
private fun resolveWithServiceInfoCallback(serviceInfo: NsdServiceInfo) {
val serviceName = BonjourEscapes.decode(serviceInfo.serviceName)
val id = stableId(serviceName, "local.")
if (serviceInfoCallbacks.containsKey(id)) return
val callback =
object : NsdManager.ServiceInfoCallback {
override fun onServiceInfoCallbackRegistrationFailed(errorCode: Int) {
serviceInfoCallbacks.remove(id, this)
}
override fun onServiceInfoCallbackUnregistered() {
serviceInfoCallbacks.remove(id, this)
}
override fun onServiceLost() {
localById.remove(id)
publish()
}
override fun onServiceUpdated(serviceInfo: NsdServiceInfo) {
upsertResolvedService(serviceInfo)
}
}
serviceInfoCallbacks[id] = callback
try {
nsd.registerServiceInfoCallback(serviceInfo, dnsExecutor, callback)
} catch (_: Throwable) {
serviceInfoCallbacks.remove(id, callback)
}
}
private fun unregisterServiceInfoCallback(id: String) {
val callback = serviceInfoCallbacks.remove(id) ?: return
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.UPSIDE_DOWN_CAKE) return
try {
nsd.unregisterServiceInfoCallback(callback as NsdManager.ServiceInfoCallback)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun resolveLegacy(serviceInfo: NsdServiceInfo) {
val listener =
object : NsdManager.ResolveListener {
override fun onResolveFailed(
serviceInfo: NsdServiceInfo,
errorCode: Int,
) {}
override fun onServiceResolved(resolved: NsdServiceInfo) {
upsertResolvedService(resolved)
}
}
try {
NsdManager::class.java
.getMethod("resolveService", NsdServiceInfo::class.java, NsdManager.ResolveListener::class.java)
.invoke(nsd, serviceInfo, listener)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun upsertResolvedService(resolved: NsdServiceInfo) {
val host = resolvedHostAddress(resolved) ?: return
val port = resolved.port
if (port <= 0) return
val rawServiceName = resolved.serviceName
val serviceName = BonjourEscapes.decode(rawServiceName)
val displayName = BonjourEscapes.decode(txt(resolved, "displayName") ?: serviceName)
val lanHost = txt(resolved, "lanHost")
val tailnetDns = txt(resolved, "tailnetDns")
val gatewayPort = txtInt(resolved, "gatewayPort")
val canvasPort = txtInt(resolved, "canvasPort")
val tlsEnabled = txtBool(resolved, "gatewayTls")
val tlsFingerprint = txt(resolved, "gatewayTlsSha256")
val id = stableId(serviceName, "local.")
// Local NSD gives the socket host/port; TXT ports are retained as gateway metadata only.
localById[id] =
GatewayEndpoint(
stableId = id,
name = displayName,
host = host,
port = port,
lanHost = lanHost,
tailnetDns = tailnetDns,
gatewayPort = gatewayPort,
canvasPort = canvasPort,
tlsEnabled = tlsEnabled,
tlsFingerprintSha256 = tlsFingerprint,
)
publish()
}
private fun resolvedHostAddress(resolved: NsdServiceInfo): String? {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
return resolved.hostAddresses.firstOrNull()?.hostAddress
}
return legacyHostAddress(resolved)
}
private fun legacyHostAddress(resolved: NsdServiceInfo): String? =
try {
val host = NsdServiceInfo::class.java.getMethod("getHost").invoke(resolved) as? InetAddress
host?.hostAddress
} catch (_: Throwable) {
null
}
private fun publish() {
_gateways.value =
// Merge local and wide-area results deterministically for stable UI selection.
(localById.values + unicastById.values).sortedBy { it.name.lowercase() }
_statusText.value = buildStatusText()
}
private fun buildStatusText(): String {
val localCount = localById.size
val wideRcode = lastWideAreaRcode
val wideCount = lastWideAreaCount
val wide =
when (wideRcode) {
null -> "Wide: ?"
Rcode.NOERROR -> "Wide: $wideCount"
Rcode.NXDOMAIN -> "Wide: NXDOMAIN"
else -> "Wide: ${Rcode.string(wideRcode)}"
}
return when {
localCount == 0 && wideRcode == null -> "Searching for gateways…"
localCount == 0 -> "$wide"
else -> "Local: $localCount$wide"
}
}
private fun stableId(
serviceName: String,
domain: String,
): String = "$serviceType|$domain|${normalizeName(serviceName)}"
private fun normalizeName(raw: String): String = raw.trim().split(Regex("\\s+")).joinToString(" ")
private fun txt(
info: NsdServiceInfo,
key: String,
): String? {
val bytes = info.attributes[key] ?: return null
return try {
String(bytes, Charsets.UTF_8).trim().ifEmpty { null }
} catch (_: Throwable) {
null
}
}
private fun txtInt(
info: NsdServiceInfo,
key: String,
): Int? = txt(info, key)?.toIntOrNull()
private fun txtBool(
info: NsdServiceInfo,
key: String,
): Boolean {
val raw = txt(info, key)?.trim()?.lowercase() ?: return false
return raw == "1" || raw == "true" || raw == "yes"
}
private suspend fun refreshUnicast(domain: String) {
val ptrName = "${serviceType}$domain"
val ptrMsg = lookupUnicastMessage(ptrName, Type.PTR) ?: return
val ptrRecords = records(ptrMsg, Section.ANSWER).mapNotNull { it as? PTRRecord }
val next = LinkedHashMap<String, GatewayEndpoint>()
for (ptr in ptrRecords) {
val instanceFqdn = ptr.target.toString()
val srv =
recordByName(ptrMsg, instanceFqdn, Type.SRV) as? SRVRecord
?: run {
val msg = lookupUnicastMessage(instanceFqdn, Type.SRV) ?: return@run null
recordByName(msg, instanceFqdn, Type.SRV) as? SRVRecord
}
?: continue
val port = srv.port
if (port <= 0) continue
val targetFqdn = srv.target.toString()
val host =
resolveHostFromMessage(ptrMsg, targetFqdn)
?: resolveHostFromMessage(lookupUnicastMessage(instanceFqdn, Type.SRV), targetFqdn)
?: resolveHostUnicast(targetFqdn)
?: continue
// Wide-area DNS-SD may put TXT in additional records; fall back to a direct TXT query.
val txtFromPtr =
recordsByName(ptrMsg, Section.ADDITIONAL)[keyName(instanceFqdn)]
.orEmpty()
.mapNotNull { it as? TXTRecord }
val txt =
if (txtFromPtr.isNotEmpty()) {
txtFromPtr
} else {
val msg = lookupUnicastMessage(instanceFqdn, Type.TXT)
records(msg, Section.ANSWER).mapNotNull { it as? TXTRecord }
}
val instanceName = BonjourEscapes.decode(decodeInstanceName(instanceFqdn, domain))
val displayName = BonjourEscapes.decode(txtValue(txt, "displayName") ?: instanceName)
val lanHost = txtValue(txt, "lanHost")
val tailnetDns = txtValue(txt, "tailnetDns")
val gatewayPort = txtIntValue(txt, "gatewayPort")
val canvasPort = txtIntValue(txt, "canvasPort")
val tlsEnabled = txtBoolValue(txt, "gatewayTls")
val tlsFingerprint = txtValue(txt, "gatewayTlsSha256")
val id = stableId(instanceName, domain)
next[id] =
GatewayEndpoint(
stableId = id,
name = displayName,
host = host,
port = port,
lanHost = lanHost,
tailnetDns = tailnetDns,
gatewayPort = gatewayPort,
canvasPort = canvasPort,
tlsEnabled = tlsEnabled,
tlsFingerprintSha256 = tlsFingerprint,
)
}
unicastById.clear()
unicastById.putAll(next)
lastWideAreaRcode = ptrMsg.header.rcode
lastWideAreaCount = next.size
publish()
if (next.isEmpty()) {
Log.d(
logTag,
"wide-area discovery: 0 results for $ptrName (rcode=${Rcode.string(ptrMsg.header.rcode)})",
)
}
}
private fun decodeInstanceName(
instanceFqdn: String,
domain: String,
): String {
val suffix = "${serviceType}$domain"
val withoutSuffix =
if (instanceFqdn.endsWith(suffix)) {
instanceFqdn.removeSuffix(suffix)
} else {
instanceFqdn.substringBefore(serviceType)
}
return normalizeName(stripTrailingDot(withoutSuffix))
}
private fun stripTrailingDot(raw: String): String = raw.removeSuffix(".")
private suspend fun lookupUnicastMessage(
name: String,
type: Int,
): Message? {
val query =
try {
Message.newQuery(
org.xbill.DNS.Record.newRecord(
Name.fromString(name),
type,
DClass.IN,
),
)
} catch (_: TextParseException) {
return null
}
val system = queryViaSystemDns(query)
if (records(system, Section.ANSWER).any { it.type == type }) return system
// Android's DnsResolver can miss split-DNS answers; retry with dnsjava against network DNS servers.
val direct = createDirectResolver() ?: return system
return try {
val msg = direct.send(query)
if (records(msg, Section.ANSWER).any { it.type == type }) msg else system
} catch (_: Throwable) {
system
}
}
private suspend fun queryViaSystemDns(query: Message): Message? {
val network = preferredDnsNetwork()
val bytes =
try {
rawQuery(network, query.toWire())
} catch (_: Throwable) {
return null
}
return try {
Message(bytes)
} catch (_: IOException) {
null
}
}
private fun records(
msg: Message?,
section: Int,
): List<Record> = msg?.getSection(section).orEmpty()
private fun keyName(raw: String): String = raw.trim().lowercase()
private fun recordsByName(
msg: Message,
section: Int,
): Map<String, List<Record>> {
val next = LinkedHashMap<String, MutableList<Record>>()
for (r in records(msg, section)) {
val name = r.name?.toString() ?: continue
next.getOrPut(keyName(name)) { mutableListOf() }.add(r)
}
return next
}
private fun recordByName(
msg: Message,
fqdn: String,
type: Int,
): Record? {
val key = keyName(fqdn)
val byNameAnswer = recordsByName(msg, Section.ANSWER)
val fromAnswer = byNameAnswer[key].orEmpty().firstOrNull { it.type == type }
if (fromAnswer != null) return fromAnswer
val byNameAdditional = recordsByName(msg, Section.ADDITIONAL)
return byNameAdditional[key].orEmpty().firstOrNull { it.type == type }
}
private fun resolveHostFromMessage(
msg: Message?,
hostname: String,
): String? {
val m = msg ?: return null
val key = keyName(hostname)
val additional = recordsByName(m, Section.ADDITIONAL)[key].orEmpty()
val a = additional.mapNotNull { it as? ARecord }.mapNotNull { it.address?.hostAddress }
val aaaa = additional.mapNotNull { it as? AAAARecord }.mapNotNull { it.address?.hostAddress }
return a.firstOrNull() ?: aaaa.firstOrNull()
}
private fun preferredDnsNetwork(): android.net.Network? {
val cm = connectivity ?: return null
// Prefer VPN (Tailscale) when present; otherwise use the active network.
trackedNetworks(cm)
.firstOrNull { n ->
val caps = cm.getNetworkCapabilities(n) ?: return@firstOrNull false
caps.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
}?.let { return it }
return cm.activeNetwork
}
private fun trackedNetworks(cm: ConnectivityManager): List<Network> =
buildList {
cm.activeNetwork?.let(::add)
addAll(availableNetworks)
}.distinct()
private fun createDirectResolver(): Resolver? {
val cm = connectivity ?: return null
val candidateNetworks =
buildList {
// Put VPN DNS first so Tailscale split-horizon names win over public DNS.
trackedNetworks(cm)
.firstOrNull { n ->
val caps = cm.getNetworkCapabilities(n) ?: return@firstOrNull false
caps.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
}?.let(::add)
cm.activeNetwork?.let(::add)
}.distinct()
val servers =
candidateNetworks
.asSequence()
.flatMap { n ->
cm.getLinkProperties(n)?.dnsServers?.asSequence() ?: emptySequence()
}.distinctBy { it.hostAddress ?: it.toString() }
.toList()
if (servers.isEmpty()) return null
return try {
val resolvers =
servers.mapNotNull { addr ->
try {
SimpleResolver().apply {
setAddress(InetSocketAddress(addr, 53))
setTimeout(Duration.ofSeconds(3))
}
} catch (_: Throwable) {
null
}
}
if (resolvers.isEmpty()) return null
ExtendedResolver(resolvers.toTypedArray()).apply { setTimeout(Duration.ofSeconds(3)) }
} catch (_: Throwable) {
null
}
}
private suspend fun rawQuery(
network: android.net.Network?,
wireQuery: ByteArray,
): ByteArray =
suspendCancellableCoroutine { cont ->
val signal = CancellationSignal()
cont.invokeOnCancellation { signal.cancel() }
dns.rawQuery(
network,
wireQuery,
DnsResolver.FLAG_EMPTY,
dnsExecutor,
signal,
object : DnsResolver.Callback<ByteArray> {
override fun onAnswer(
answer: ByteArray,
rcode: Int,
) {
cont.resume(answer)
}
override fun onError(error: DnsResolver.DnsException) {
cont.resumeWithException(error)
}
},
)
}
private fun txtValue(
records: List<TXTRecord>,
key: String,
): String? {
val prefix = "$key="
for (r in records) {
val strings: List<String> =
try {
r.strings
} catch (_: Throwable) {
emptyList()
}
for (s in strings) {
val trimmed = decodeDnsTxtString(s).trim()
if (trimmed.startsWith(prefix)) {
return trimmed.removePrefix(prefix).trim().ifEmpty { null }
}
}
}
return null
}
private fun txtIntValue(
records: List<TXTRecord>,
key: String,
): Int? = txtValue(records, key)?.toIntOrNull()
private fun txtBoolValue(
records: List<TXTRecord>,
key: String,
): Boolean {
val raw = txtValue(records, key)?.trim()?.lowercase() ?: return false
return raw == "1" || raw == "true" || raw == "yes"
}
private fun decodeDnsTxtString(raw: String): String {
// dnsjava treats TXT as opaque bytes and decodes as ISO-8859-1 to preserve bytes.
// Our TXT payload is UTF-8 (written by the gateway), so re-decode when possible.
val bytes = raw.toByteArray(Charsets.ISO_8859_1)
val decoder =
Charsets.UTF_8
.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT)
return try {
decoder.decode(ByteBuffer.wrap(bytes)).toString()
} catch (_: Throwable) {
raw
}
}
private suspend fun resolveHostUnicast(hostname: String): String? {
val a =
records(lookupUnicastMessage(hostname, Type.A), Section.ANSWER)
.mapNotNull { it as? ARecord }
.mapNotNull { it.address?.hostAddress }
val aaaa =
records(lookupUnicastMessage(hostname, Type.AAAA), Section.ANSWER)
.mapNotNull { it as? AAAARecord }
.mapNotNull { it.address?.hostAddress }
return a.firstOrNull() ?: aaaa.firstOrNull()
}
}

View File

@@ -0,0 +1,31 @@
package ai.openclaw.app.gateway
/** Resolved gateway address and optional metadata discovered from Bonjour/manual entry. */
data class GatewayEndpoint(
val stableId: String,
val name: String,
val host: String,
val port: Int,
val lanHost: String? = null,
val tailnetDns: String? = null,
val gatewayPort: Int? = null,
val canvasPort: Int? = null,
val tlsEnabled: Boolean = false,
val tlsFingerprintSha256: String? = null,
) {
companion object {
/** Builds a stable manual endpoint key that survives display-name changes. */
fun manual(
host: String,
port: Int,
): GatewayEndpoint =
GatewayEndpoint(
stableId = "manual|${host.lowercase()}|$port",
name = "$host:$port",
host = host,
port = port,
tlsEnabled = false,
tlsFingerprintSha256 = null,
)
}
}

View File

@@ -0,0 +1,148 @@
package ai.openclaw.app.gateway
import android.os.Build
import java.net.InetAddress
import java.util.Locale
/** Returns true only for loopback hosts safe to treat as local gateway origins. */
internal fun isLoopbackGatewayHost(
rawHost: String?,
allowEmulatorBridgeAlias: Boolean = isAndroidEmulatorRuntime(),
): Boolean {
var host =
rawHost
?.trim()
?.lowercase(Locale.US)
?.trim('[', ']')
.orEmpty()
if (host.endsWith(".")) {
host = host.dropLast(1)
}
val zoneIndex = host.indexOf('%')
// Scoped IPv6 literals are not stable origin identifiers; reject them for
// loopback trust instead of guessing which interface the zone names.
if (zoneIndex >= 0) return false
if (host.isEmpty()) return false
if (host == "localhost") return true
// Android emulator maps host loopback through this bridge alias.
if (allowEmulatorBridgeAlias && host == "10.0.2.2") return true
parseIpv4Address(host)?.let { ipv4 ->
return ipv4.first() == 127.toByte()
}
if (!host.contains(':') || !host.all(::isIpv6LiteralChar)) return false
val address = runCatching { InetAddress.getByName(host) }.getOrNull()?.address ?: return false
if (address.size == 4) {
return address[0] == 127.toByte()
}
if (address.size != 16) return false
// `::1` is 15 zero bytes followed by `0x01`.
val isIpv6Loopback = address.copyOfRange(0, 15).all { it == 0.toByte() } && address[15] == 1.toByte()
if (isIpv6Loopback) return true
val isMappedIpv4 =
address.copyOfRange(0, 10).all { it == 0.toByte() } &&
address[10] == 0xFF.toByte() &&
address[11] == 0xFF.toByte()
return isMappedIpv4 && address[12] == 127.toByte()
}
/** Allows cleartext only for loopback, `.local`, and private/link-local network ranges. */
internal fun isLocalCleartextGatewayHost(
rawHost: String?,
allowEmulatorBridgeAlias: Boolean = isAndroidEmulatorRuntime(),
): Boolean {
var host =
rawHost
?.trim()
?.lowercase(Locale.US)
?.trim('[', ']')
.orEmpty()
if (host.endsWith(".")) {
host = host.dropLast(1)
}
if (host.isEmpty()) return false
if (isLoopbackGatewayHost(host, allowEmulatorBridgeAlias = allowEmulatorBridgeAlias)) return true
if (isMdnsLocalHostname(host)) return true
val zoneIndex = host.indexOf('%')
if (zoneIndex >= 0) {
// Link-local cleartext policy is about the address range; strip the
// interface zone before InetAddress parsing rejects otherwise valid hosts.
host = host.substring(0, zoneIndex)
}
if (host.isEmpty()) return false
parseIpv4Address(host)?.let { ipv4 ->
val first = ipv4[0].toInt() and 0xff
val second = ipv4[1].toInt() and 0xff
return when {
first == 10 -> true
first == 172 && second in 16..31 -> true
first == 192 && second == 168 -> true
first == 169 && second == 254 -> true
else -> false
}
}
if (!host.contains(':') || !host.all(::isIpv6LiteralChar)) return false
val address = runCatching { InetAddress.getByName(host) }.getOrNull() ?: return false
return when {
address.isLinkLocalAddress -> true
address.isSiteLocalAddress -> true
else -> {
val bytes = address.address
bytes.size == 16 && (bytes[0].toInt() and 0xfe) == 0xfc
}
}
}
private fun isAndroidEmulatorRuntime(): Boolean {
val fingerprint = Build.FINGERPRINT?.lowercase(Locale.US).orEmpty()
val model = Build.MODEL?.lowercase(Locale.US).orEmpty()
val manufacturer = Build.MANUFACTURER?.lowercase(Locale.US).orEmpty()
val brand = Build.BRAND?.lowercase(Locale.US).orEmpty()
val device = Build.DEVICE?.lowercase(Locale.US).orEmpty()
val product = Build.PRODUCT?.lowercase(Locale.US).orEmpty()
return fingerprint.contains("generic") ||
fingerprint.contains("robolectric") ||
model.contains("emulator") ||
model.contains("sdk_gphone") ||
manufacturer.contains("genymotion") ||
(brand.contains("generic") && device.contains("generic")) ||
product.contains("sdk_gphone") ||
product.contains("emulator") ||
product.contains("simulator")
}
/** Parses strict dotted-quad IPv4, rejecting shorthand and out-of-range octets. */
private fun parseIpv4Address(host: String): ByteArray? {
val parts = host.split('.')
if (parts.size != 4) return null
val bytes = ByteArray(4)
for ((index, part) in parts.withIndex()) {
val value = part.toIntOrNull() ?: return null
if (value !in 0..255) return null
bytes[index] = value.toByte()
}
return bytes
}
private fun isMdnsLocalHostname(host: String): Boolean {
if (host.length > 253) return false
if (!host.endsWith(".local")) return false
val labels = host.split('.')
if (labels.size < 2 || labels.last() != "local") return false
return labels.dropLast(1).all(::isDnsHostnameLabel)
}
private fun isDnsHostnameLabel(label: String): Boolean {
if (label.isEmpty() || label.length > 63) return false
if (label.first() == '-' || label.last() == '-') return false
return label.all { it in 'a'..'z' || it in '0'..'9' || it == '-' }
}
/** Cheap prefilter before handing potential IPv6 literals to InetAddress. */
private fun isIpv6LiteralChar(char: Char): Boolean = char in '0'..'9' || char in 'a'..'f' || char == ':' || char == '.'

View File

@@ -0,0 +1,7 @@
package ai.openclaw.app.gateway
/** Gateway protocol version emitted by Android node clients. */
const val GATEWAY_PROTOCOL_VERSION = 4
/** Oldest gateway protocol version this Android client can speak safely. */
const val GATEWAY_MIN_PROTOCOL_VERSION = 4

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,258 @@
package ai.openclaw.app.gateway
import android.annotation.SuppressLint
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import java.io.EOFException
import java.net.ConnectException
import java.net.InetSocketAddress
import java.net.SocketException
import java.net.SocketTimeoutException
import java.net.UnknownHostException
import java.security.MessageDigest
import java.security.SecureRandom
import java.security.cert.CertificateException
import java.security.cert.X509Certificate
import java.util.Locale
import java.util.concurrent.atomic.AtomicReference
import javax.net.ssl.HostnameVerifier
import javax.net.ssl.HttpsURLConnection
import javax.net.ssl.SNIHostName
import javax.net.ssl.SSLContext
import javax.net.ssl.SSLException
import javax.net.ssl.SSLParameters
import javax.net.ssl.SSLSocket
import javax.net.ssl.SSLSocketFactory
import javax.net.ssl.TrustManagerFactory
import javax.net.ssl.X509TrustManager
/** TLS pinning inputs for a discovered or manually configured gateway endpoint. */
data class GatewayTlsParams(
val required: Boolean,
val expectedFingerprint: String?,
val allowTOFU: Boolean,
val stableId: String,
)
/** SSL primitives installed into OkHttp when a gateway needs TLS pinning/TOFU. */
data class GatewayTlsConfig(
val sslSocketFactory: SSLSocketFactory,
val trustManager: X509TrustManager,
val hostnameVerifier: HostnameVerifier,
)
/** Distinguishes non-TLS endpoints from unreachable endpoints during probing. */
enum class GatewayTlsProbeFailure {
TLS_UNAVAILABLE,
TLS_HANDSHAKE_TIMEOUT,
ENDPOINT_UNREACHABLE,
}
/** Result of probing a gateway TLS endpoint for first-use fingerprint capture. */
data class GatewayTlsProbeResult(
val fingerprintSha256: String? = null,
val failure: GatewayTlsProbeFailure? = null,
)
internal const val GATEWAY_TLS_PROBE_CONNECT_TIMEOUT_MS = 3_000
internal const val GATEWAY_TLS_PROBE_HANDSHAKE_TIMEOUT_MS = 10_000
/** Builds a TLS config that supports pinned fingerprints and trust-on-first-use. */
fun buildGatewayTlsConfig(
params: GatewayTlsParams?,
onStore: ((String) -> Unit)? = null,
): GatewayTlsConfig? {
if (params == null) return null
val expected =
params.expectedFingerprint
?.let(::normalizeGatewayTlsFingerprint)
?.takeIf { it.isNotBlank() }
val defaultTrust = defaultTrustManager()
@SuppressLint("CustomX509TrustManager")
val trustManager =
object : X509TrustManager {
override fun checkClientTrusted(
chain: Array<X509Certificate>,
authType: String,
) {
defaultTrust.checkClientTrusted(chain, authType)
}
override fun checkServerTrusted(
chain: Array<X509Certificate>,
authType: String,
) {
if (chain.isEmpty()) throw CertificateException("empty certificate chain")
val fingerprint = sha256Hex(chain[0].encoded)
if (expected != null) {
if (fingerprint != expected) {
throw CertificateException("gateway TLS fingerprint mismatch")
}
return
}
if (params.allowTOFU) {
// Store only after the TLS stack presents a concrete server cert; the
// caller persists the fingerprint against the endpoint's stable id,
// and later connects must come back through the pinned branch above.
onStore?.invoke(fingerprint)
return
}
defaultTrust.checkServerTrusted(chain, authType)
}
override fun getAcceptedIssuers(): Array<X509Certificate> = defaultTrust.acceptedIssuers
}
val context = SSLContext.getInstance("TLS")
context.init(null, arrayOf(trustManager), SecureRandom())
val verifier =
if (expected != null || params.allowTOFU) {
// When pinning, we intentionally ignore hostname mismatch (service discovery often yields IPs).
HostnameVerifier { _, _ -> true }
} else {
HttpsURLConnection.getDefaultHostnameVerifier()
}
return GatewayTlsConfig(
sslSocketFactory = context.socketFactory,
trustManager = trustManager,
hostnameVerifier = verifier,
)
}
/** Connects with a probe trust manager that captures the presented cert hash. */
suspend fun probeGatewayTlsFingerprint(
host: String,
port: Int,
): GatewayTlsProbeResult =
probeGatewayTlsFingerprint(
host = host,
port = port,
connectTimeoutMs = GATEWAY_TLS_PROBE_CONNECT_TIMEOUT_MS,
handshakeTimeoutMs = GATEWAY_TLS_PROBE_HANDSHAKE_TIMEOUT_MS,
)
internal suspend fun probeGatewayTlsFingerprint(
host: String,
port: Int,
connectTimeoutMs: Int,
handshakeTimeoutMs: Int,
): GatewayTlsProbeResult {
val trimmedHost = host.trim()
if (trimmedHost.isEmpty()) return GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE)
if (port !in 1..65535) return GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE)
if (connectTimeoutMs <= 0 || handshakeTimeoutMs <= 0) return GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE)
return withContext(Dispatchers.IO) {
val fingerprintRef = AtomicReference<String?>(null)
val probeTrustManager =
@SuppressLint("CustomX509TrustManager")
object : X509TrustManager {
override fun checkClientTrusted(
chain: Array<X509Certificate>,
authType: String,
): Unit = throw CertificateException("gateway TLS probe does not accept client certificates")
override fun checkServerTrusted(
chain: Array<X509Certificate>,
authType: String,
) {
if (chain.isEmpty()) throw CertificateException("empty certificate chain")
fingerprintRef.set(sha256Hex(chain[0].encoded))
// Abort validation after capture; the probe is not deciding trust.
throw CertificateException("gateway TLS probe captured fingerprint")
}
override fun getAcceptedIssuers(): Array<X509Certificate> = emptyArray()
}
val context = SSLContext.getInstance("TLS")
context.init(null, arrayOf(probeTrustManager), SecureRandom())
val socket = (context.socketFactory.createSocket() as SSLSocket)
var connected = false
try {
// TCP reachability and TLS handshake progress fail differently on mobile
// tailnets; keep the budgets separate so a reachable-but-slow secure
// endpoint does not collapse into generic gateway unreachable guidance.
socket.soTimeout = handshakeTimeoutMs
socket.connect(InetSocketAddress(trimmedHost, port), connectTimeoutMs)
connected = true
// Best-effort SNI for hostnames (avoid crashing on IP literals).
try {
if (trimmedHost.any { it.isLetter() }) {
val params = SSLParameters()
params.serverNames = listOf(SNIHostName(trimmedHost))
socket.sslParameters = params
}
} catch (_: Throwable) {
// SNI is only a probe hint. IP literals and odd Bonjour names should
// still be probed instead of failing before the TLS handshake.
}
socket.startHandshake()
val cert =
socket.session.peerCertificates.firstOrNull() as? X509Certificate
?: return@withContext GatewayTlsProbeResult(failure = GatewayTlsProbeFailure.TLS_UNAVAILABLE)
GatewayTlsProbeResult(fingerprintSha256 = sha256Hex(cert.encoded))
} catch (err: Throwable) {
fingerprintRef.get()?.let { return@withContext GatewayTlsProbeResult(fingerprintSha256 = it) }
val failure =
when (err) {
is SSLException,
is EOFException,
-> GatewayTlsProbeFailure.TLS_UNAVAILABLE
is SocketTimeoutException ->
if (connected) {
GatewayTlsProbeFailure.TLS_HANDSHAKE_TIMEOUT
} else {
GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
}
is ConnectException,
is UnknownHostException,
-> GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
is SocketException ->
if (connected) {
GatewayTlsProbeFailure.TLS_UNAVAILABLE
} else {
GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
}
else -> GatewayTlsProbeFailure.ENDPOINT_UNREACHABLE
}
GatewayTlsProbeResult(failure = failure)
} finally {
try {
socket.close()
} catch (_: Throwable) {
// ignore
}
}
}
}
private fun defaultTrustManager(): X509TrustManager {
val factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
factory.init(null as java.security.KeyStore?)
val trust =
factory.trustManagers.firstOrNull { it is X509TrustManager } as? X509TrustManager
return trust ?: throw IllegalStateException("No default X509TrustManager found")
}
private fun sha256Hex(data: ByteArray): String {
val digest = MessageDigest.getInstance("SHA-256").digest(data)
val out = StringBuilder(digest.size * 2)
for (byte in digest) {
out.append(String.format(Locale.US, "%02x", byte))
}
return out.toString()
}
/** Normalizes user-visible fingerprint text to lowercase bare SHA-256 hex. */
fun normalizeGatewayTlsFingerprint(raw: String): String {
val stripped =
raw
.trim()
.replace(Regex("^sha-?256\\s*:?\\s*", RegexOption.IGNORE_CASE), "")
return stripped.lowercase(Locale.US).filter { it in '0'..'9' || it in 'a'..'f' }
}

View File

@@ -0,0 +1,47 @@
package ai.openclaw.app.gateway
private val invokeErrorCodePattern = Regex("^[A-Z][A-Z0-9_]*$")
data class ParsedInvokeError(
val code: String,
val message: String,
val hadExplicitCode: Boolean,
) {
/** Gateway-facing form expected by UI and retry copy. */
val prefixedMessage: String
get() = "$code: $message"
}
/**
* Parses gateway invoke errors encoded as CODE: message while preserving legacy
* plain-text errors as UNAVAILABLE.
*/
fun parseInvokeErrorMessage(raw: String): ParsedInvokeError {
val trimmed = raw.trim()
if (trimmed.isEmpty()) {
return ParsedInvokeError(code = "UNAVAILABLE", message = "error", hadExplicitCode = false)
}
val parts = trimmed.split(":", limit = 2)
if (parts.size == 2) {
val code = parts[0].trim()
val rest = parts[1].trim()
if (invokeErrorCodePattern.matches(code)) {
return ParsedInvokeError(
code = code,
message = rest.ifEmpty { trimmed },
hadExplicitCode = true,
)
}
}
return ParsedInvokeError(code = "UNAVAILABLE", message = trimmed, hadExplicitCode = false)
}
/** Extracts an invoke error from a throwable without exposing blank messages. */
fun parseInvokeErrorFromThrowable(
err: Throwable,
fallbackMessage: String = "error",
): ParsedInvokeError {
val raw = err.message?.trim().takeIf { !it.isNullOrEmpty() } ?: fallbackMessage
return parseInvokeErrorMessage(raw)
}

View File

@@ -0,0 +1,145 @@
package ai.openclaw.app.node
import kotlinx.coroutines.delay
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
/**
* Android bridge for applying gateway A2UI messages to the canvas WebView.
*/
class A2UIHandler(
private val canvas: CanvasController,
private val json: Json,
) {
fun isTrustedCanvasActionUrl(rawUrl: String?): Boolean = CanvasActionTrust.isTrustedCanvasActionUrl(rawUrl)
suspend fun ensureA2uiReady(): Boolean {
if (canvas.currentUrl()?.trim() == CanvasActionTrust.localA2uiAssetUrl && isA2uiReady()) {
return true
}
canvas.showLocalA2ui()
// The bundled A2UI host bootstraps asynchronously after navigation; poll briefly before failing the command.
repeat(50) {
if (isA2uiReady()) return true
delay(120)
}
return false
}
private suspend fun isA2uiReady(): Boolean =
try {
canvas.eval(a2uiReadyCheckJS) == "true"
} catch (_: Throwable) {
false
}
fun decodeA2uiMessages(
command: String,
paramsJson: String?,
): String {
val raw = paramsJson?.trim().orEmpty()
if (raw.isBlank()) throw IllegalArgumentException("INVALID_REQUEST: paramsJSON required")
val obj =
json.parseToJsonElement(raw) as? JsonObject
?: throw IllegalArgumentException("INVALID_REQUEST: expected object params")
val jsonlField = (obj["jsonl"] as? JsonPrimitive)?.content?.trim().orEmpty()
val hasMessagesArray = obj["messages"] is JsonArray
if (command == "canvas.a2ui.pushJSONL" || (!hasMessagesArray && jsonlField.isNotBlank())) {
val jsonl = jsonlField
if (jsonl.isBlank()) throw IllegalArgumentException("INVALID_REQUEST: jsonl required")
// JSONL keeps large A2UI streams model-friendly while still validating each message.
val messages =
jsonl
.lineSequence()
.map { it.trim() }
.filter { it.isNotBlank() }
.mapIndexed { idx, line ->
val el = json.parseToJsonElement(line)
val msg =
el as? JsonObject
?: throw IllegalArgumentException("A2UI JSONL line ${idx + 1}: expected a JSON object")
validateA2uiV0_8(msg, idx + 1)
msg
}.toList()
return JsonArray(messages).toString()
}
val arr = obj["messages"] as? JsonArray ?: throw IllegalArgumentException("INVALID_REQUEST: messages[] required")
val out =
arr.mapIndexed { idx, el ->
val msg =
el as? JsonObject
?: throw IllegalArgumentException("A2UI messages[$idx]: expected a JSON object")
validateA2uiV0_8(msg, idx + 1)
msg
}
return JsonArray(out).toString()
}
private fun validateA2uiV0_8(
msg: JsonObject,
lineNumber: Int,
) {
if (msg.containsKey("createSurface")) {
// Android scaffold currently implements A2UI v0.8, not the v0.9 createSurface shape.
throw IllegalArgumentException(
"A2UI JSONL line $lineNumber: looks like A2UI v0.9 (`createSurface`). Canvas supports v0.8 messages only.",
)
}
val allowed = setOf("beginRendering", "surfaceUpdate", "dataModelUpdate", "deleteSurface")
val matched = msg.keys.filter { allowed.contains(it) }
if (matched.size != 1) {
val found = msg.keys.sorted().joinToString(", ")
throw IllegalArgumentException(
"A2UI JSONL line $lineNumber: expected exactly one of ${allowed.sorted().joinToString(", ")}; found: $found",
)
}
}
companion object {
const val a2uiReadyCheckJS: String =
"""
(() => {
try {
const host = globalThis.openclawA2UI;
return !!host && typeof host.applyMessages === 'function';
} catch (_) {
return false;
}
})()
"""
const val a2uiResetJS: String =
"""
(() => {
try {
const host = globalThis.openclawA2UI;
if (!host) return { ok: false, error: "missing openclawA2UI" };
return host.reset();
} catch (e) {
return { ok: false, error: String(e?.message ?? e) };
}
})()
"""
fun a2uiApplyMessagesJS(messagesJson: String): String =
"""
(() => {
try {
const host = globalThis.openclawA2UI;
if (!host) return { ok: false, error: "missing openclawA2UI" };
const messages = $messagesJson;
return host.applyMessages(messages);
} catch (e) {
return { ok: false, error: String(e?.message ?? e) };
}
})()
""".trimIndent()
}
}

View File

@@ -0,0 +1,436 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.ContentResolver
import android.content.ContentUris
import android.content.ContentValues
import android.content.Context
import android.provider.CalendarContract
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.time.Instant
import java.time.temporal.ChronoUnit
import java.util.TimeZone
private const val DEFAULT_CALENDAR_LIMIT = 50
/**
* Parsed calendar.events request; times are epoch millis for CalendarContract queries.
*/
internal data class CalendarEventsRequest(
val startMs: Long,
val endMs: Long,
val limit: Int,
)
/**
* Parsed calendar.add request before resolving the target Android calendar.
*/
internal data class CalendarAddRequest(
val title: String,
val startMs: Long,
val endMs: Long,
val isAllDay: Boolean,
val location: String?,
val notes: String?,
val calendarId: Long?,
val calendarTitle: String?,
)
/**
* Normalized calendar event returned through gateway calendar commands.
*/
internal data class CalendarEventRecord(
val identifier: String,
val title: String,
val startISO: String,
val endISO: String,
val isAllDay: Boolean,
val location: String?,
val calendarTitle: String?,
)
/**
* Injectable CalendarProvider facade for command tests and Android runtime access.
*/
internal interface CalendarDataSource {
fun hasReadPermission(context: Context): Boolean
fun hasWritePermission(context: Context): Boolean
fun events(
context: Context,
request: CalendarEventsRequest,
): List<CalendarEventRecord>
fun add(
context: Context,
request: CalendarAddRequest,
): CalendarEventRecord
}
private object SystemCalendarDataSource : CalendarDataSource {
override fun hasReadPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CALENDAR) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun hasWritePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CALENDAR) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun events(
context: Context,
request: CalendarEventsRequest,
): List<CalendarEventRecord> {
val resolver = context.contentResolver
val builder = CalendarContract.Instances.CONTENT_URI.buildUpon()
// Instances expands recurring events inside the requested time window.
ContentUris.appendId(builder, request.startMs)
ContentUris.appendId(builder, request.endMs)
val projection =
arrayOf(
CalendarContract.Instances.EVENT_ID,
CalendarContract.Instances.TITLE,
CalendarContract.Instances.BEGIN,
CalendarContract.Instances.END,
CalendarContract.Instances.ALL_DAY,
CalendarContract.Instances.EVENT_LOCATION,
CalendarContract.Instances.CALENDAR_DISPLAY_NAME,
)
val sortOrder = "${CalendarContract.Instances.BEGIN} ASC LIMIT ${request.limit}"
resolver.query(builder.build(), projection, null, null, sortOrder).use { cursor ->
if (cursor == null) return emptyList()
val out = mutableListOf<CalendarEventRecord>()
while (cursor.moveToNext() && out.size < request.limit) {
val id = cursor.getLong(0)
val title =
cursor
.getString(1)
?.trim()
.orEmpty()
.ifEmpty { "(untitled)" }
val beginMs = cursor.getLong(2)
val endMs = cursor.getLong(3)
val isAllDay = cursor.getInt(4) == 1
val location = cursor.getString(5)?.trim()?.ifEmpty { null }
val calendarTitle = cursor.getString(6)?.trim()?.ifEmpty { null }
out +=
CalendarEventRecord(
identifier = id.toString(),
title = title,
startISO = Instant.ofEpochMilli(beginMs).toString(),
endISO = Instant.ofEpochMilli(endMs).toString(),
isAllDay = isAllDay,
location = location,
calendarTitle = calendarTitle,
)
}
return out
}
}
override fun add(
context: Context,
request: CalendarAddRequest,
): CalendarEventRecord {
val resolver = context.contentResolver
val resolvedCalendarId = resolveCalendarId(resolver, request.calendarId, request.calendarTitle)
val values =
ContentValues().apply {
put(CalendarContract.Events.CALENDAR_ID, resolvedCalendarId)
put(CalendarContract.Events.TITLE, request.title)
put(CalendarContract.Events.DTSTART, request.startMs)
put(CalendarContract.Events.DTEND, request.endMs)
put(CalendarContract.Events.ALL_DAY, if (request.isAllDay) 1 else 0)
put(CalendarContract.Events.EVENT_TIMEZONE, TimeZone.getDefault().id)
request.location?.let { put(CalendarContract.Events.EVENT_LOCATION, it) }
request.notes?.let { put(CalendarContract.Events.DESCRIPTION, it) }
}
val uri =
resolver.insert(CalendarContract.Events.CONTENT_URI, values)
?: throw IllegalStateException("calendar insert failed")
val eventId =
uri.lastPathSegment?.toLongOrNull()
?: throw IllegalStateException("calendar insert failed")
return loadEventById(resolver, eventId)
?: throw IllegalStateException("calendar insert failed")
}
private fun resolveCalendarId(
resolver: ContentResolver,
calendarId: Long?,
calendarTitle: String?,
): Long {
if (calendarId != null) {
// Explicit id wins over title/default selection and must already exist.
if (calendarExists(resolver, calendarId)) return calendarId
throw IllegalArgumentException("CALENDAR_NOT_FOUND: no calendar id $calendarId")
}
if (!calendarTitle.isNullOrEmpty()) {
// Title lookup is exact to avoid adding events to a similarly named calendar.
findCalendarByTitle(resolver, calendarTitle)?.let { return it }
throw IllegalArgumentException("CALENDAR_NOT_FOUND: no calendar named $calendarTitle")
}
findDefaultCalendarId(resolver)?.let { return it }
throw IllegalArgumentException("CALENDAR_NOT_FOUND: no default calendar")
}
private fun calendarExists(
resolver: ContentResolver,
id: Long,
): Boolean {
val projection = arrayOf(CalendarContract.Calendars._ID)
resolver
.query(
CalendarContract.Calendars.CONTENT_URI,
projection,
"${CalendarContract.Calendars._ID}=?",
arrayOf(id.toString()),
null,
).use { cursor ->
return cursor != null && cursor.moveToFirst()
}
}
private fun findCalendarByTitle(
resolver: ContentResolver,
title: String,
): Long? {
val projection = arrayOf(CalendarContract.Calendars._ID)
resolver
.query(
CalendarContract.Calendars.CONTENT_URI,
projection,
"${CalendarContract.Calendars.CALENDAR_DISPLAY_NAME}=?",
arrayOf(title),
"${CalendarContract.Calendars.IS_PRIMARY} DESC",
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return cursor.getLong(0)
}
}
private fun findDefaultCalendarId(resolver: ContentResolver): Long? {
val projection = arrayOf(CalendarContract.Calendars._ID)
resolver
.query(
CalendarContract.Calendars.CONTENT_URI,
projection,
"${CalendarContract.Calendars.VISIBLE}=1",
null,
// Prefer Android's primary visible calendar, then lowest id for deterministic fallback.
"${CalendarContract.Calendars.IS_PRIMARY} DESC, ${CalendarContract.Calendars._ID} ASC",
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return cursor.getLong(0)
}
}
private fun loadEventById(
resolver: ContentResolver,
eventId: Long,
): CalendarEventRecord? {
val projection =
arrayOf(
CalendarContract.Events._ID,
CalendarContract.Events.TITLE,
CalendarContract.Events.DTSTART,
CalendarContract.Events.DTEND,
CalendarContract.Events.ALL_DAY,
CalendarContract.Events.EVENT_LOCATION,
CalendarContract.Events.CALENDAR_DISPLAY_NAME,
)
resolver
.query(
CalendarContract.Events.CONTENT_URI,
projection,
"${CalendarContract.Events._ID}=?",
arrayOf(eventId.toString()),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return CalendarEventRecord(
identifier = cursor.getLong(0).toString(),
title =
cursor
.getString(1)
?.trim()
.orEmpty()
.ifEmpty { "(untitled)" },
startISO = Instant.ofEpochMilli(cursor.getLong(2)).toString(),
endISO = Instant.ofEpochMilli(cursor.getLong(3)).toString(),
isAllDay = cursor.getInt(4) == 1,
location = cursor.getString(5)?.trim()?.ifEmpty { null },
calendarTitle = cursor.getString(6)?.trim()?.ifEmpty { null },
)
}
}
}
class CalendarHandler private constructor(
private val appContext: Context,
private val dataSource: CalendarDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemCalendarDataSource)
fun handleCalendarEvents(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasReadPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_PERMISSION_REQUIRED",
message = "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission",
)
}
val request =
parseEventsRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val events = dataSource.events(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put(
"events",
buildJsonArray { events.forEach { add(eventJson(it)) } },
)
}.toString(),
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CALENDAR_UNAVAILABLE",
message = "CALENDAR_UNAVAILABLE: ${err.message ?: "calendar query failed"}",
)
}
}
fun handleCalendarAdd(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasWritePermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_PERMISSION_REQUIRED",
message = "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission",
)
}
val request =
parseAddRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
if (request.title.isEmpty()) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_INVALID",
message = "CALENDAR_INVALID: title required",
)
}
if (request.endMs <= request.startMs) {
return GatewaySession.InvokeResult.error(
code = "CALENDAR_INVALID",
message = "CALENDAR_INVALID: endISO must be after startISO",
)
}
return try {
val event = dataSource.add(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put("event", eventJson(event))
}.toString(),
)
} catch (err: IllegalArgumentException) {
val msg = err.message ?: "CALENDAR_INVALID: invalid request"
val code = if (msg.startsWith("CALENDAR_NOT_FOUND")) "CALENDAR_NOT_FOUND" else "CALENDAR_INVALID"
GatewaySession.InvokeResult.error(code = code, message = msg)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CALENDAR_UNAVAILABLE",
message = "CALENDAR_UNAVAILABLE: ${err.message ?: "calendar add failed"}",
)
}
}
private fun parseEventsRequest(paramsJson: String?): CalendarEventsRequest? {
if (paramsJson.isNullOrBlank()) {
val start = Instant.now()
val end = start.plus(7, ChronoUnit.DAYS)
// Default calendar read is a one-week window, not the full calendar store.
return CalendarEventsRequest(startMs = start.toEpochMilli(), endMs = end.toEpochMilli(), limit = DEFAULT_CALENDAR_LIMIT)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val start = parseISO((params["startISO"] as? JsonPrimitive)?.content)
val end = parseISO((params["endISO"] as? JsonPrimitive)?.content)
val resolvedStart = start ?: Instant.now()
val resolvedEnd = end ?: resolvedStart.plus(7, ChronoUnit.DAYS)
// Keep model-driven calendar reads bounded.
val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: DEFAULT_CALENDAR_LIMIT).coerceIn(1, 500)
return CalendarEventsRequest(
startMs = resolvedStart.toEpochMilli(),
endMs = resolvedEnd.toEpochMilli(),
limit = limit,
)
}
private fun parseAddRequest(paramsJson: String?): CalendarAddRequest? {
val params =
try {
paramsJson?.let { Json.parseToJsonElement(it).asObjectOrNull() }
} catch (_: Throwable) {
null
} ?: return null
val start =
parseISO((params["startISO"] as? JsonPrimitive)?.content)
?: return null
val end =
parseISO((params["endISO"] as? JsonPrimitive)?.content)
?: return null
return CalendarAddRequest(
title = (params["title"] as? JsonPrimitive)?.content?.trim().orEmpty(),
startMs = start.toEpochMilli(),
endMs = end.toEpochMilli(),
isAllDay = (params["isAllDay"] as? JsonPrimitive)?.content?.toBooleanStrictOrNull() ?: false,
location = (params["location"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
notes = (params["notes"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
calendarId = (params["calendarId"] as? JsonPrimitive)?.content?.toLongOrNull(),
calendarTitle = (params["calendarTitle"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
)
}
private fun parseISO(raw: String?): Instant? {
val value = raw?.trim().orEmpty()
if (value.isEmpty()) return null
// Gateway calendar payloads use UTC ISO-8601 instants for unambiguous Android storage.
return try {
Instant.parse(value)
} catch (_: Throwable) {
null
}
}
private fun eventJson(event: CalendarEventRecord): JsonObject =
buildJsonObject {
put("identifier", JsonPrimitive(event.identifier))
put("title", JsonPrimitive(event.title))
put("startISO", JsonPrimitive(event.startISO))
put("endISO", JsonPrimitive(event.endISO))
put("isAllDay", JsonPrimitive(event.isAllDay))
event.location?.let { put("location", JsonPrimitive(it)) }
event.calendarTitle?.let { put("calendarTitle", JsonPrimitive(it)) }
}
companion object {
internal fun forTesting(
appContext: Context,
dataSource: CalendarDataSource,
): CalendarHandler = CalendarHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,531 @@
package ai.openclaw.app.node
import ai.openclaw.app.PermissionRequester
import android.Manifest
import android.annotation.SuppressLint
import android.content.Context
import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.Matrix
import android.hardware.camera2.CameraCharacteristics
import android.util.Base64
import androidx.camera.camera2.interop.Camera2CameraInfo
import androidx.camera.core.CameraInfo
import androidx.camera.core.CameraSelector
import androidx.camera.core.ImageCapture
import androidx.camera.core.ImageCaptureException
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.camera.video.FallbackStrategy
import androidx.camera.video.FileOutputOptions
import androidx.camera.video.Quality
import androidx.camera.video.QualitySelector
import androidx.camera.video.Recorder
import androidx.camera.video.VideoCapture
import androidx.camera.video.VideoRecordEvent
import androidx.core.content.ContextCompat
import androidx.core.content.ContextCompat.checkSelfPermission
import androidx.core.graphics.scale
import androidx.exifinterface.media.ExifInterface
import androidx.lifecycle.LifecycleOwner
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import kotlinx.serialization.json.JsonObject
import java.io.ByteArrayOutputStream
import java.io.File
import java.util.concurrent.Executor
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
import kotlin.math.roundToInt
/**
* CameraX-backed capture service used by gateway camera commands.
*/
internal class CameraClipSession(
private val unbind: () -> Unit,
private val deleteTemporaryFile: (File) -> Unit,
) : AutoCloseable {
private var recording: AutoCloseable? = null
private var temporaryFile: File? = null
private var closed = false
fun ownRecording(recording: AutoCloseable) {
check(!closed) { "camera clip session is closed" }
this.recording = recording
}
fun ownFile(file: File): File {
check(!closed) { "camera clip session is closed" }
check(temporaryFile == null) { "camera clip session already owns a file" }
temporaryFile = file
return file
}
fun transferFile(): File {
check(!closed) { "camera clip session is closed" }
return checkNotNull(temporaryFile) { "camera clip session has no file" }
.also { temporaryFile = null }
}
override fun close() {
if (closed) return
closed = true
var failure: Throwable? = null
fun cleanup(action: () -> Unit) {
try {
action()
} catch (err: Throwable) {
failure?.addSuppressed(err) ?: run { failure = err }
}
}
// Keep teardown symmetric across bind, warmup, recording, finalize, and success exits.
cleanup { recording?.close() }
cleanup(unbind)
temporaryFile?.let { file -> cleanup { deleteTemporaryFile(file) } }
failure?.let { throw it }
}
}
class CameraCaptureManager(
private val context: Context,
) {
/** Base64 JSON response for camera.snap after resize and JPEG budget enforcement. */
data class Payload(
val payloadJson: String,
)
/** Temporary MP4 response for camera.clip before CameraHandler validates invoke size. */
data class FilePayload(
val file: File,
val durationMs: Long,
val hasAudio: Boolean,
)
/** Camera device metadata exposed through camera.list. */
data class CameraDeviceInfo(
val id: String,
val name: String,
val position: String,
val deviceType: String,
)
@Volatile private var lifecycleOwner: LifecycleOwner? = null
@Volatile private var permissionRequester: PermissionRequester? = null
/** Supplies the foreground Activity lifecycle required by CameraX use-case binding. */
fun attachLifecycleOwner(owner: LifecycleOwner) {
// CameraX binds use cases to an Activity lifecycle; background services cannot capture alone.
lifecycleOwner = owner
}
/** Supplies the Activity-owned permission launcher used by camera and microphone commands. */
fun attachPermissionRequester(requester: PermissionRequester) {
// Permission prompts must be launched by the Activity that owns the ActivityResult registry.
permissionRequester = requester
}
/** Lists CameraX devices with stable Camera2 ids where available. */
suspend fun listDevices(): List<CameraDeviceInfo> =
withContext(Dispatchers.Main) {
val provider = context.cameraProvider()
provider.availableCameraInfos
.mapNotNull { info -> cameraDeviceInfoOrNull(info) }
.sortedBy { it.id }
}
private suspend fun ensureCameraPermission() {
val granted = checkSelfPermission(context, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
if (granted) return
val requester =
permissionRequester
?: throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission")
val results = requester.requestIfMissing(listOf(Manifest.permission.CAMERA))
if (results[Manifest.permission.CAMERA] != true) {
throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission")
}
}
private suspend fun ensureMicPermission() {
val granted = checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED
if (granted) return
val requester =
permissionRequester
?: throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
val results = requester.requestIfMissing(listOf(Manifest.permission.RECORD_AUDIO))
if (results[Manifest.permission.RECORD_AUDIO] != true) {
throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
}
}
/** Captures one still image and returns a gateway-sized JPEG payload. */
suspend fun snap(paramsJson: String?): Payload =
withContext(Dispatchers.Main) {
ensureCameraPermission()
val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready")
val params = parseJsonParamsObject(paramsJson)
val facing = parseFacing(params) ?: "front"
val quality = (parseQuality(params) ?: 0.95).coerceIn(0.1, 1.0)
val maxWidth = parseMaxWidth(params) ?: 1600
val deviceId = parseDeviceId(params)
val provider = context.cameraProvider()
val capture = ImageCapture.Builder().build()
val selector = resolveCameraSelector(provider, facing, deviceId)
provider.unbindAll()
// Bind only the still capture use case; CameraX owns camera open/close through the lifecycle owner.
provider.bindToLifecycle(owner, selector, capture)
val (bytes, orientation) =
try {
capture.takeJpegWithExif(context.mainExecutor(), context.cacheDir)
} finally {
// The JPEG bytes are self-contained; release CameraX before decoding and recompressing them.
provider.unbind(capture)
}
val decoded =
BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
?: throw IllegalStateException("UNAVAILABLE: failed to decode captured image")
val rotated = rotateBitmapByExif(decoded, orientation)
val scaled =
if (maxWidth > 0 && rotated.width > maxWidth) {
val h =
(rotated.height.toDouble() * (maxWidth.toDouble() / rotated.width.toDouble()))
.toInt()
.coerceAtLeast(1)
val s = rotated.scale(maxWidth, h)
if (s !== rotated) rotated.recycle()
s
} else {
rotated
}
try {
val maxPayloadBytes = 5 * 1024 * 1024
// Base64 inflates payloads by ~4/3; cap encoded bytes so the payload stays under 5MB (API limit).
val maxEncodedBytes = (maxPayloadBytes / 4) * 3
val result =
JpegSizeLimiter.compressToLimit(
initialWidth = scaled.width,
initialHeight = scaled.height,
startQuality = (quality * 100.0).roundToInt().coerceIn(10, 100),
maxBytes = maxEncodedBytes,
encode = { width, height, q ->
val bitmap =
if (width == scaled.width && height == scaled.height) {
scaled
} else {
scaled.scale(width, height)
}
val out = ByteArrayOutputStream()
if (!bitmap.compress(Bitmap.CompressFormat.JPEG, q, out)) {
if (bitmap !== scaled) bitmap.recycle()
throw IllegalStateException("UNAVAILABLE: failed to encode JPEG")
}
if (bitmap !== scaled) {
bitmap.recycle()
}
out.toByteArray()
},
)
val base64 = Base64.encodeToString(result.bytes, Base64.NO_WRAP)
Payload(
"""{"format":"jpg","base64":"$base64","width":${result.width},"height":${result.height}}""",
)
} finally {
scaled.recycle()
}
}
/** Records a short MP4 clip into a temporary cache file for the caller to encode/delete. */
@SuppressLint("MissingPermission")
suspend fun clip(paramsJson: String?): FilePayload =
withContext(Dispatchers.Main) {
ensureCameraPermission()
val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready")
val params = parseJsonParamsObject(paramsJson)
val facing = parseFacing(params) ?: "front"
val durationMs = (parseDurationMs(params) ?: 3_000).coerceIn(200, 60_000)
val includeAudio = parseIncludeAudio(params) ?: true
val deviceId = parseDeviceId(params)
if (includeAudio) ensureMicPermission()
android.util.Log.w(
"CameraCaptureManager",
"clip: start facing=$facing duration=$durationMs audio=$includeAudio deviceId=${deviceId ?: "-"}",
)
val provider = context.cameraProvider()
android.util.Log.w("CameraCaptureManager", "clip: got camera provider")
// Use LOWEST quality for smallest files over WebSocket
val recorder =
Recorder
.Builder()
.setQualitySelector(
QualitySelector.from(Quality.LOWEST, FallbackStrategy.lowerQualityOrHigherThan(Quality.LOWEST)),
).build()
val videoCapture = VideoCapture.withOutput(recorder)
val selector = resolveCameraSelector(provider, facing, deviceId)
// CameraX requires a Preview use case for the camera to start producing frames;
// without it, the encoder may get no data (ERROR_NO_VALID_DATA).
val preview =
androidx.camera.core.Preview
.Builder()
.build()
// Allocate the dummy preview surface only after CameraX requests it; its result owns release.
preview.setSurfaceProvider { request ->
val surfaceTexture = android.graphics.SurfaceTexture(0)
surfaceTexture.setDefaultBufferSize(640, 480)
val surface = android.view.Surface(surfaceTexture)
request.provideSurface(surface, context.mainExecutor()) {
surface.release()
surfaceTexture.release()
}
}
provider.unbindAll()
CameraClipSession(
unbind = { provider.unbind(preview, videoCapture) },
deleteTemporaryFile = { file ->
check(!file.exists() || file.delete()) { "failed to delete temporary camera clip" }
},
).use { session ->
android.util.Log.w("CameraCaptureManager", "clip: binding preview + videoCapture to lifecycle")
val camera = provider.bindToLifecycle(owner, selector, preview, videoCapture)
android.util.Log.w("CameraCaptureManager", "clip: bound, cameraInfo=${camera.cameraInfo}")
// Give camera pipeline time to initialize before recording
android.util.Log.w("CameraCaptureManager", "clip: warming up camera 1.5s...")
kotlinx.coroutines.delay(1_500)
val clipFile = session.ownFile(File.createTempFile("openclaw-clip-", ".mp4", context.cacheDir))
val outputOptions = FileOutputOptions.Builder(clipFile).build()
val finalized = kotlinx.coroutines.CompletableDeferred<VideoRecordEvent.Finalize>()
android.util.Log.w("CameraCaptureManager", "clip: starting recording to ${clipFile.absolutePath}")
val recording =
videoCapture.output
.prepareRecording(context, outputOptions)
.apply {
if (includeAudio) withAudioEnabled()
}.start(context.mainExecutor()) { event ->
android.util.Log.w("CameraCaptureManager", "clip: event ${event.javaClass.simpleName}")
if (event is VideoRecordEvent.Status) {
android.util.Log.w("CameraCaptureManager", "clip: recording status update")
}
if (event is VideoRecordEvent.Finalize) {
android.util.Log.w(
"CameraCaptureManager",
"clip: finalize hasError=${event.hasError()} error=${event.error} cause=${event.cause}",
)
finalized.complete(event)
}
}
session.ownRecording(recording)
android.util.Log.w("CameraCaptureManager", "clip: recording started, delaying ${durationMs}ms")
kotlinx.coroutines.delay(durationMs.toLong())
android.util.Log.w("CameraCaptureManager", "clip: stopping recording")
recording.close()
val finalizeEvent =
try {
withTimeout(15_000) { finalized.await() }
} catch (err: kotlinx.coroutines.TimeoutCancellationException) {
android.util.Log.e("CameraCaptureManager", "clip: finalize timed out", err)
throw IllegalStateException("UNAVAILABLE: camera clip finalize timed out")
}
if (finalizeEvent.hasError()) {
android.util.Log.e(
"CameraCaptureManager",
"clip: FAILED error=${finalizeEvent.error}, cause=${finalizeEvent.cause}",
finalizeEvent.cause,
)
// Check file size for debugging
val fileSize = withContext(Dispatchers.IO) { if (clipFile.exists()) clipFile.length() else -1 }
android.util.Log.e("CameraCaptureManager", "clip: file exists=${clipFile.exists()} size=$fileSize")
throw IllegalStateException("UNAVAILABLE: camera clip failed (error=${finalizeEvent.error})")
}
val fileSize = withContext(Dispatchers.IO) { clipFile.length() }
android.util.Log.w("CameraCaptureManager", "clip: SUCCESS file size=$fileSize")
FilePayload(
file = session.transferFile(),
durationMs = durationMs.toLong(),
hasAudio = includeAudio,
)
}
}
private fun rotateBitmapByExif(
bitmap: Bitmap,
orientation: Int,
): Bitmap {
val matrix = Matrix()
// CameraX JPEG bytes keep sensor orientation in EXIF; normalize before resizing/encoding.
when (orientation) {
ExifInterface.ORIENTATION_ROTATE_90 -> matrix.postRotate(90f)
ExifInterface.ORIENTATION_ROTATE_180 -> matrix.postRotate(180f)
ExifInterface.ORIENTATION_ROTATE_270 -> matrix.postRotate(270f)
ExifInterface.ORIENTATION_FLIP_HORIZONTAL -> matrix.postScale(-1f, 1f)
ExifInterface.ORIENTATION_FLIP_VERTICAL -> matrix.postScale(1f, -1f)
ExifInterface.ORIENTATION_TRANSPOSE -> {
matrix.postRotate(90f)
matrix.postScale(-1f, 1f)
}
ExifInterface.ORIENTATION_TRANSVERSE -> {
matrix.postRotate(-90f)
matrix.postScale(-1f, 1f)
}
else -> return bitmap
}
val rotated = Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
if (rotated !== bitmap) {
bitmap.recycle()
}
return rotated
}
private fun parseFacing(params: JsonObject?): String? {
val value = parseJsonString(params, "facing")?.trim()?.lowercase() ?: return null
return when (value) {
"front", "back" -> value
else -> null
}
}
private fun parseQuality(params: JsonObject?): Double? = parseJsonDouble(params, "quality")
private fun parseMaxWidth(params: JsonObject?): Int? =
parseJsonInt(params, "maxWidth")
?.takeIf { it > 0 }
private fun parseDurationMs(params: JsonObject?): Int? = parseJsonInt(params, "durationMs")
private fun parseDeviceId(params: JsonObject?): String? =
parseJsonString(params, "deviceId")
?.trim()
?.takeIf { it.isNotEmpty() }
private fun parseIncludeAudio(params: JsonObject?): Boolean? = parseJsonBooleanFlag(params, "includeAudio")
private fun Context.mainExecutor(): Executor = ContextCompat.getMainExecutor(this)
private fun resolveCameraSelector(
provider: ProcessCameraProvider,
facing: String,
deviceId: String?,
): CameraSelector {
if (deviceId.isNullOrEmpty()) {
return if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA
}
val availableIds = provider.availableCameraInfos.mapNotNull { cameraIdOrNull(it) }.toSet()
if (!availableIds.contains(deviceId)) {
throw IllegalStateException("INVALID_REQUEST: unknown camera deviceId '$deviceId'")
}
return CameraSelector
.Builder()
// CameraX selectors are filters over CameraInfo; pin by Camera2 id for stable device selection.
.addCameraFilter { infos -> infos.filter { cameraIdOrNull(it) == deviceId } }
.build()
}
@SuppressLint("UnsafeOptInUsageError")
private fun cameraDeviceInfoOrNull(info: CameraInfo): CameraDeviceInfo? {
val cameraId = cameraIdOrNull(info) ?: return null
val lensFacing =
runCatching {
Camera2CameraInfo.from(info).getCameraCharacteristic(CameraCharacteristics.LENS_FACING)
}.getOrNull()
val position =
when (lensFacing) {
CameraCharacteristics.LENS_FACING_FRONT -> "front"
CameraCharacteristics.LENS_FACING_BACK -> "back"
CameraCharacteristics.LENS_FACING_EXTERNAL -> "external"
else -> "unspecified"
}
val deviceType =
if (lensFacing == CameraCharacteristics.LENS_FACING_EXTERNAL) "external" else "builtIn"
val name =
when (position) {
"front" -> "Front Camera"
"back" -> "Back Camera"
"external" -> "External Camera"
else -> "Camera $cameraId"
}
return CameraDeviceInfo(
id = cameraId,
name = name,
position = position,
deviceType = deviceType,
)
}
@SuppressLint("UnsafeOptInUsageError")
private fun cameraIdOrNull(info: CameraInfo): String? = runCatching { Camera2CameraInfo.from(info).cameraId }.getOrNull()
}
private suspend fun Context.cameraProvider(): ProcessCameraProvider =
suspendCancellableCoroutine { cont ->
val future = ProcessCameraProvider.getInstance(this)
future.addListener(
{
try {
cont.resume(future.get())
} catch (e: Exception) {
cont.resumeWithException(e)
}
},
ContextCompat.getMainExecutor(this),
)
}
/**
* Returns JPEG bytes plus EXIF orientation so callers can normalize the decoded bitmap.
*/
private suspend fun ImageCapture.takeJpegWithExif(
executor: Executor,
tempDir: File,
): Pair<ByteArray, Int> =
suspendCancellableCoroutine { cont ->
val file = File.createTempFile("openclaw-snap-", ".jpg", tempDir)
val options = ImageCapture.OutputFileOptions.Builder(file).build()
takePicture(
options,
executor,
object : ImageCapture.OnImageSavedCallback {
override fun onError(exception: ImageCaptureException) {
file.delete()
cont.resumeWithException(exception)
}
override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {
try {
val exif = ExifInterface(file.absolutePath)
val orientation =
exif.getAttributeInt(
ExifInterface.TAG_ORIENTATION,
ExifInterface.ORIENTATION_NORMAL,
)
val bytes = file.readBytes()
cont.resume(Pair(bytes, orientation))
} catch (e: Exception) {
cont.resumeWithException(e)
} finally {
file.delete()
}
}
},
)
}

View File

@@ -0,0 +1,189 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.CameraHudKind
import ai.openclaw.app.gateway.GatewaySession
import android.content.Context
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.put
internal const val CAMERA_CLIP_MAX_RAW_BYTES: Long = 18L * 1024L * 1024L
/**
* Raw MP4 size guard before base64 encoding the clip into a node.invoke response.
*/
internal fun isCameraClipWithinPayloadLimit(rawBytes: Long): Boolean = rawBytes in 0L..CAMERA_CLIP_MAX_RAW_BYTES
/**
* Gateway camera command adapter that adds HUD feedback and payload-size enforcement.
*/
class CameraHandler(
private val appContext: Context,
private val camera: CameraCaptureManager,
private val externalAudioCaptureActive: MutableStateFlow<Boolean>,
private val showCameraHud: (message: String, kind: CameraHudKind, autoHideMs: Long?) -> Unit,
private val triggerCameraFlash: () -> Unit,
private val invokeErrorFromThrowable: (err: Throwable) -> Pair<String, String>,
) {
/** Handles camera.list by exposing CameraX devices through gateway metadata. */
suspend fun handleList(_paramsJson: String?): GatewaySession.InvokeResult =
try {
val devices = camera.listDevices()
val payload =
buildJsonObject {
put(
"devices",
buildJsonArray {
devices.forEach { device ->
add(
buildJsonObject {
put("id", JsonPrimitive(device.id))
put("name", JsonPrimitive(device.name))
put("position", JsonPrimitive(device.position))
put("deviceType", JsonPrimitive(device.deviceType))
},
)
}
},
)
}.toString()
GatewaySession.InvokeResult.ok(payload)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
GatewaySession.InvokeResult.error(code = code, message = message)
}
/** Handles camera.snap with HUD progress, flash feedback, and normalized invoke errors. */
suspend fun handleSnap(paramsJson: String?): GatewaySession.InvokeResult {
val logFile = if (BuildConfig.DEBUG) java.io.File(appContext.cacheDir, "camera_debug.log") else null
fun camLog(msg: String) {
if (!BuildConfig.DEBUG) return
val ts = java.text.SimpleDateFormat("HH:mm:ss.SSS", java.util.Locale.US).format(java.util.Date())
logFile?.appendText("[$ts] $msg\n")
android.util.Log.w("openclaw", "camera.snap: $msg")
}
try {
logFile?.writeText("") // clear
camLog("starting, params=$paramsJson")
camLog("calling showCameraHud")
showCameraHud("Taking photo…", CameraHudKind.Photo, null)
camLog("calling triggerCameraFlash")
triggerCameraFlash()
val res =
try {
camLog("calling camera.snap()")
val r = camera.snap(paramsJson)
camLog("success, payload size=${r.payloadJson.length}")
r
} catch (err: Throwable) {
camLog("inner error: ${err::class.java.simpleName}: ${err.message}")
camLog("stack: ${err.stackTraceToString().take(2000)}")
val (code, message) = invokeErrorFromThrowable(err)
showCameraHud(message, CameraHudKind.Error, 2200)
return GatewaySession.InvokeResult.error(code = code, message = message)
}
camLog("returning result")
showCameraHud("Photo captured", CameraHudKind.Success, 1600)
return GatewaySession.InvokeResult.ok(res.payloadJson)
} catch (err: Throwable) {
camLog("outer error: ${err::class.java.simpleName}: ${err.message}")
camLog("stack: ${err.stackTraceToString().take(2000)}")
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = err.message ?: "camera snap failed")
}
}
/** Handles camera.clip and keeps external audio capture paused while camera audio is active. */
suspend fun handleClip(paramsJson: String?): GatewaySession.InvokeResult {
val clipLogFile = if (BuildConfig.DEBUG) java.io.File(appContext.cacheDir, "camera_debug.log") else null
fun clipLog(msg: String) {
if (!BuildConfig.DEBUG) return
val ts = java.text.SimpleDateFormat("HH:mm:ss.SSS", java.util.Locale.US).format(java.util.Date())
clipLogFile?.appendText("[CLIP $ts] $msg\n")
android.util.Log.w("openclaw", "camera.clip: $msg")
}
val includeAudio = parseIncludeAudio(paramsJson) ?: true
if (includeAudio) externalAudioCaptureActive.value = true
try {
clipLogFile?.writeText("") // clear
clipLog("starting, params=$paramsJson includeAudio=$includeAudio")
clipLog("calling showCameraHud")
showCameraHud("Recording…", CameraHudKind.Recording, null)
val filePayload =
try {
clipLog("calling camera.clip()")
val r = camera.clip(paramsJson)
clipLog("success, file size=${r.file.length()}")
r
} catch (err: Throwable) {
clipLog("inner error: ${err::class.java.simpleName}: ${err.message}")
clipLog("stack: ${err.stackTraceToString().take(2000)}")
val (code, message) = invokeErrorFromThrowable(err)
showCameraHud(message, CameraHudKind.Error, 2400)
return GatewaySession.InvokeResult.error(code = code, message = message)
}
val rawBytes = filePayload.file.length()
if (!isCameraClipWithinPayloadLimit(rawBytes)) {
clipLog("payload too large: bytes=$rawBytes max=$CAMERA_CLIP_MAX_RAW_BYTES")
// Delete oversized clips before returning so cache files do not accumulate after failed invokes.
withContext(Dispatchers.IO) { filePayload.file.delete() }
showCameraHud("Clip too large", CameraHudKind.Error, 2400)
return GatewaySession.InvokeResult.error(
code = "PAYLOAD_TOO_LARGE",
message =
"PAYLOAD_TOO_LARGE: camera clip is $rawBytes bytes; max is $CAMERA_CLIP_MAX_RAW_BYTES bytes. Reduce durationMs and retry.",
)
}
val bytes =
withContext(Dispatchers.IO) {
try {
filePayload.file.readBytes()
} finally {
filePayload.file.delete()
}
}
val base64 = android.util.Base64.encodeToString(bytes, android.util.Base64.NO_WRAP)
clipLog("returning base64 payload")
showCameraHud("Clip captured", CameraHudKind.Success, 1800)
return GatewaySession.InvokeResult.ok(
"""{"format":"mp4","base64":"$base64","durationMs":${filePayload.durationMs},"hasAudio":${filePayload.hasAudio}}""",
)
} catch (err: Throwable) {
clipLog("outer error: ${err::class.java.simpleName}: ${err.message}")
clipLog("stack: ${err.stackTraceToString().take(2000)}")
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = err.message ?: "camera clip failed")
} finally {
// Prevent talk/transcription capture from competing with camera audio after every exit path.
if (includeAudio) externalAudioCaptureActive.value = false
}
}
private fun parseIncludeAudio(paramsJson: String?): Boolean? {
if (paramsJson.isNullOrBlank()) return null
val root =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val value =
(root["includeAudio"] as? JsonPrimitive)
?.contentOrNull
?.trim()
?.lowercase()
return when (value) {
"true" -> true
"false" -> false
else -> null
}
}
}

View File

@@ -0,0 +1,21 @@
package ai.openclaw.app.node
/**
* Trust helper for WebView-originated canvas/A2UI actions.
*/
object CanvasActionTrust {
/** Local canvas scaffold is the only trusted file URL. */
const val scaffoldAssetUrl: String = "file:///android_asset/CanvasScaffold/scaffold.html"
/** Local bundled A2UI is the only action-capable A2UI host. */
const val localA2uiAssetUrl: String = "file:///android_asset/CanvasA2UI/index.html"
/** Accepts only app-owned bundled pages. Remote WebView content is render-only. */
fun isTrustedCanvasActionUrl(rawUrl: String?): Boolean {
val candidate = rawUrl?.trim().orEmpty()
if (candidate.isEmpty()) return false
if (candidate == scaffoldAssetUrl) return true
if (candidate == localA2uiAssetUrl) return true
return false
}
}

View File

@@ -0,0 +1,334 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import android.graphics.Bitmap
import android.graphics.Canvas
import android.os.Looper
import android.util.Base64
import android.util.Log
import android.webkit.WebView
import androidx.core.graphics.createBitmap
import androidx.core.graphics.scale
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import org.json.JSONObject
import java.io.ByteArrayOutputStream
import kotlin.coroutines.resume
/**
* Owns the Android WebView canvas surface used by canvas and A2UI commands.
*/
class CanvasController {
enum class SnapshotFormat(
val rawValue: String,
) {
Png("png"),
Jpeg("jpeg"),
}
@Volatile private var webView: WebView? = null
@Volatile private var url: String? = null
@Volatile private var debugStatusEnabled: Boolean = false
@Volatile private var debugStatusTitle: String? = null
@Volatile private var debugStatusSubtitle: String? = null
@Volatile private var homeCanvasStateJson: String? = null
private val _currentUrl = MutableStateFlow<String?>(null)
val currentUrl: StateFlow<String?> = _currentUrl.asStateFlow()
private val scaffoldAssetUrl = CanvasActionTrust.scaffoldAssetUrl
private val localA2uiAssetUrl = CanvasActionTrust.localA2uiAssetUrl
private fun clampJpegQuality(quality: Double?): Int {
val q = (quality ?: 0.82).coerceIn(0.1, 1.0)
return (q * 100.0).toInt().coerceIn(1, 100)
}
private fun Bitmap.scaleForMaxWidth(maxWidth: Int?): Bitmap {
if (maxWidth == null || maxWidth <= 0 || width <= maxWidth) {
return this
}
val scaledHeight = (height.toDouble() * (maxWidth.toDouble() / width.toDouble())).toInt().coerceAtLeast(1)
return scale(maxWidth, scaledHeight)
}
/** Attaches the active WebView and replays state that may have arrived before the view existed. */
fun attach(webView: WebView) {
this.webView = webView
// Replay persisted state because WebView attachment can happen after gateway events arrive.
reload()
applyDebugStatus()
applyHomeCanvasState()
}
/** Detaches only the currently attached WebView instance. */
fun detach(webView: WebView) {
if (this.webView === webView) {
this.webView = null
}
}
/** Navigates the canvas to a remote URL or back to the bundled scaffold for blank/root input. */
fun navigate(url: String) {
val trimmed = url.trim()
this.url = if (trimmed.isBlank() || trimmed == "/") null else trimmed
_currentUrl.value = this.url
reload()
}
/** Shows the app-owned A2UI renderer that is allowed to dispatch native actions. */
fun showLocalA2ui() {
this.url = localA2uiAssetUrl
_currentUrl.value = localA2uiAssetUrl
reload()
}
fun currentUrl(): String? = url
fun setDebugStatusEnabled(enabled: Boolean) {
debugStatusEnabled = enabled
applyDebugStatus()
}
fun setDebugStatus(
title: String?,
subtitle: String?,
) {
debugStatusTitle = title
debugStatusSubtitle = subtitle
applyDebugStatus()
}
fun onPageFinished() {
applyDebugStatus()
applyHomeCanvasState()
}
fun updateHomeCanvasState(json: String?) {
homeCanvasStateJson = json
applyHomeCanvasState()
}
private inline fun withWebViewOnMain(crossinline block: (WebView) -> Unit) {
val wv = webView ?: return
if (Looper.myLooper() == Looper.getMainLooper()) {
block(wv)
} else {
// WebView APIs must run on the main thread.
wv.post { block(wv) }
}
}
private fun reload() {
val currentUrl = url
withWebViewOnMain { wv ->
if (currentUrl == null) {
if (BuildConfig.DEBUG) {
Log.d("OpenClawCanvas", "load scaffold: $scaffoldAssetUrl")
}
wv.loadUrl(scaffoldAssetUrl)
} else {
if (BuildConfig.DEBUG) {
Log.d("OpenClawCanvas", "load url: $currentUrl")
}
wv.loadUrl(currentUrl)
}
}
}
private fun applyDebugStatus() {
val enabled = debugStatusEnabled
val title = debugStatusTitle
val subtitle = debugStatusSubtitle
withWebViewOnMain { wv ->
val titleJs = title?.let { JSONObject.quote(it) } ?: "null"
val subtitleJs = subtitle?.let { JSONObject.quote(it) } ?: "null"
val js =
"""
(() => {
try {
const api = globalThis.__openclaw;
if (!api) return;
if (typeof api.setDebugStatusEnabled === 'function') {
api.setDebugStatusEnabled(${if (enabled) "true" else "false"});
}
if (!${if (enabled) "true" else "false"}) return;
if (typeof api.setStatus === 'function') {
api.setStatus($titleJs, $subtitleJs);
}
} catch (_) {}
})();
""".trimIndent()
wv.evaluateJavascript(js, null)
}
}
private fun applyHomeCanvasState() {
val payload = homeCanvasStateJson ?: "null"
withWebViewOnMain { wv ->
val js =
"""
(() => {
try {
const api = globalThis.__openclaw;
if (!api || typeof api.renderHome !== 'function') return;
api.renderHome($payload);
} catch (_) {}
})();
""".trimIndent()
wv.evaluateJavascript(js, null)
}
}
/** Evaluates JavaScript against the attached WebView on the main thread. */
suspend fun eval(javaScript: String): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
suspendCancellableCoroutine { cont ->
wv.evaluateJavascript(javaScript) { result ->
cont.resume(result ?: "")
}
}
}
/** Captures the WebView as PNG/JPEG base64 with optional width and quality bounds. */
suspend fun snapshotBase64(
format: SnapshotFormat,
quality: Double?,
maxWidth: Int?,
): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
val bmp = wv.captureBitmap()
try {
val scaled = bmp.scaleForMaxWidth(maxWidth)
try {
val out = ByteArrayOutputStream()
val (compressFormat, compressQuality) =
when (format) {
SnapshotFormat.Png -> Bitmap.CompressFormat.PNG to 100
SnapshotFormat.Jpeg -> Bitmap.CompressFormat.JPEG to clampJpegQuality(quality)
}
scaled.compress(compressFormat, compressQuality, out)
Base64.encodeToString(out.toByteArray(), Base64.NO_WRAP)
} finally {
if (scaled !== bmp) scaled.recycle()
}
} finally {
bmp.recycle()
}
}
private suspend fun WebView.captureBitmap(): Bitmap =
suspendCancellableCoroutine { cont ->
val width = width.coerceAtLeast(1)
val height = height.coerceAtLeast(1)
val bitmap = createBitmap(width, height, Bitmap.Config.ARGB_8888)
// WebView isn't supported by PixelCopy.request(...) directly; draw() is the most reliable
// cross-version snapshot for this lightweight "canvas" use-case.
draw(Canvas(bitmap))
cont.resume(bitmap)
}
companion object {
/**
* Parsed canvas.snapshot options used by invoke dispatch.
*/
data class SnapshotParams(
val format: SnapshotFormat,
val quality: Double?,
val maxWidth: Int?,
)
/** Parses canvas.navigate params and returns blank when the payload is missing or invalid. */
fun parseNavigateUrl(paramsJson: String?): String {
val obj = parseParamsObject(paramsJson) ?: return ""
return obj.string("url").trim()
}
/** Parses non-blank JavaScript from canvas.eval params. */
fun parseEvalJs(paramsJson: String?): String? {
val obj = parseParamsObject(paramsJson) ?: return null
val js = obj.string("javaScript").trim()
return js.takeIf { it.isNotBlank() }
}
fun parseSnapshotMaxWidth(paramsJson: String?): Int? {
val obj = parseParamsObject(paramsJson) ?: return null
if (!obj.containsKey("maxWidth")) return null
val width = obj.int("maxWidth") ?: 0
return width.takeIf { it > 0 }
}
fun parseSnapshotFormat(paramsJson: String?): SnapshotFormat {
val obj = parseParamsObject(paramsJson) ?: return SnapshotFormat.Jpeg
val raw = obj.string("format").trim().lowercase()
return when (raw) {
"png" -> SnapshotFormat.Png
"jpeg", "jpg" -> SnapshotFormat.Jpeg
"" -> SnapshotFormat.Jpeg
else -> SnapshotFormat.Jpeg
}
}
fun parseSnapshotQuality(paramsJson: String?): Double? {
val obj = parseParamsObject(paramsJson) ?: return null
if (!obj.containsKey("quality")) return null
val q = obj.double("quality") ?: Double.NaN
if (!q.isFinite()) return null
// Keep JPEG quality inside encoder-safe bounds; PNG ignores it.
return q.coerceIn(0.1, 1.0)
}
/** Parses canvas.snapshot params using JPEG defaults and encoder-safe bounds. */
fun parseSnapshotParams(paramsJson: String?): SnapshotParams =
SnapshotParams(
format = parseSnapshotFormat(paramsJson),
quality = parseSnapshotQuality(paramsJson),
maxWidth = parseSnapshotMaxWidth(paramsJson),
)
private val json = Json { ignoreUnknownKeys = true }
private fun parseParamsObject(paramsJson: String?): JsonObject? {
val raw = paramsJson?.trim().orEmpty()
if (raw.isEmpty()) return null
return try {
json.parseToJsonElement(raw).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonObject.string(key: String): String {
val prim = this[key] as? JsonPrimitive ?: return ""
val raw = prim.content
return raw.takeIf { it != "null" }.orEmpty()
}
private fun JsonObject.int(key: String): Int? {
val prim = this[key] as? JsonPrimitive ?: return null
return prim.content.toIntOrNull()
}
private fun JsonObject.double(key: String): Double? {
val prim = this[key] as? JsonPrimitive ?: return null
return prim.content.toDoubleOrNull()
}
}
}

View File

@@ -0,0 +1,232 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.LocationMode
import ai.openclaw.app.SecurePrefs
import ai.openclaw.app.VoiceWakeMode
import ai.openclaw.app.gateway.GatewayClientInfo
import ai.openclaw.app.gateway.GatewayConnectOptions
import ai.openclaw.app.gateway.GatewayEndpoint
import ai.openclaw.app.gateway.GatewayTlsParams
import ai.openclaw.app.gateway.isLocalCleartextGatewayHost
import ai.openclaw.app.gateway.isLoopbackGatewayHost
import android.os.Build
/**
* Builds gateway connect metadata from current Android permissions, settings, and device identity.
*/
class ConnectionManager(
private val prefs: SecurePrefs,
private val cameraEnabled: () -> Boolean,
private val locationMode: () -> LocationMode,
private val voiceWakeMode: () -> VoiceWakeMode,
private val motionActivityAvailable: () -> Boolean,
private val motionPedometerAvailable: () -> Boolean,
private val sendSmsAvailable: () -> Boolean,
private val readSmsAvailable: () -> Boolean,
private val smsSearchPossible: () -> Boolean,
private val callLogAvailable: () -> Boolean,
private val photosAvailable: () -> Boolean,
private val hasRecordAudioPermission: () -> Boolean,
private val installedAppsSharingEnabled: () -> Boolean,
private val manualTls: () -> Boolean,
) {
companion object {
internal val legacyOperatorScopes: List<String> =
listOf(
"operator.approvals",
"operator.read",
"operator.write",
)
internal val nativeClientOperatorScopes: List<String> =
listOf(
"operator.approvals",
"operator.read",
"operator.talk.secrets",
"operator.write",
)
internal fun operatorScopesForStoredDeviceToken(storedScopes: List<String>): List<String> {
val normalized =
storedScopes
.map { it.trim() }
.filter { it.isNotEmpty() }
.distinct()
return normalized.ifEmpty { legacyOperatorScopes }
}
/**
* Decide whether a discovered/manual endpoint must use pinned TLS or can stay local cleartext.
*/
internal fun resolveTlsParamsForEndpoint(
endpoint: GatewayEndpoint,
storedFingerprint: String?,
manualTlsEnabled: Boolean,
): GatewayTlsParams? {
val stableId = endpoint.stableId
val stored = storedFingerprint?.trim().takeIf { !it.isNullOrEmpty() }
val isManual = stableId.startsWith("manual|")
val cleartextAllowedHost =
if (isManual) {
isLocalCleartextGatewayHost(endpoint.host)
} else {
isLoopbackGatewayHost(endpoint.host)
}
if (isManual) {
// Manual remote hosts default to TLS; only local manual hosts may honor the cleartext toggle.
if (!manualTlsEnabled && cleartextAllowedHost) return null
if (!stored.isNullOrBlank()) {
return GatewayTlsParams(
required = true,
expectedFingerprint = stored,
allowTOFU = false,
stableId = stableId,
)
}
return GatewayTlsParams(
required = true,
expectedFingerprint = null,
allowTOFU = false,
stableId = stableId,
)
}
// Prefer stored pins. Never let discovery-provided TXT override a stored fingerprint.
if (!stored.isNullOrBlank()) {
return GatewayTlsParams(
required = true,
expectedFingerprint = stored,
allowTOFU = false,
stableId = stableId,
)
}
val hinted = endpoint.tlsEnabled || !endpoint.tlsFingerprintSha256.isNullOrBlank()
if (hinted) {
// TXT is unauthenticated. Do not treat the advertised fingerprint as authoritative.
return GatewayTlsParams(
required = true,
expectedFingerprint = null,
allowTOFU = false,
stableId = stableId,
)
}
if (!cleartextAllowedHost) {
// Non-loopback discovered hosts require TLS even without TXT hints.
return GatewayTlsParams(
required = true,
expectedFingerprint = null,
allowTOFU = false,
stableId = stableId,
)
}
return null
}
}
private fun runtimeFlags(): NodeRuntimeFlags =
NodeRuntimeFlags(
cameraEnabled = cameraEnabled(),
locationEnabled = locationMode() != LocationMode.Off,
sendSmsAvailable = sendSmsAvailable(),
readSmsAvailable = readSmsAvailable(),
smsSearchPossible = smsSearchPossible(),
callLogAvailable = callLogAvailable(),
photosAvailable = photosAvailable(),
voiceWakeEnabled = voiceWakeMode() != VoiceWakeMode.Off && hasRecordAudioPermission(),
motionActivityAvailable = motionActivityAvailable(),
motionPedometerAvailable = motionPedometerAvailable(),
installedAppsSharingEnabled = installedAppsSharingEnabled(),
debugBuild = BuildConfig.DEBUG,
)
/** Builds the gateway-advertised node.invoke command list from current permission and feature state. */
fun buildInvokeCommands(): List<String> = InvokeCommandRegistry.advertisedCommands(runtimeFlags())
/** Builds the gateway-advertised capability list from current permission and feature state. */
fun buildCapabilities(): List<String> = InvokeCommandRegistry.advertisedCapabilities(runtimeFlags())
/**
* Debug Android builds advertise a dev version so gateway logs do not look like release clients.
*/
fun resolvedVersionName(): String {
val versionName = BuildConfig.VERSION_NAME.trim().ifEmpty { "dev" }
return if (BuildConfig.DEBUG && !versionName.contains("dev", ignoreCase = true)) {
"$versionName-dev"
} else {
versionName
}
}
/** Human-readable Android device model used in gateway client metadata. */
fun resolveModelIdentifier(): String? =
listOfNotNull(Build.MANUFACTURER, Build.MODEL)
.joinToString(" ")
.trim()
.ifEmpty { null }
/**
* User-Agent used for gateway telemetry and troubleshooting.
*/
fun buildUserAgent(): String {
val version = resolvedVersionName()
val release =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
val releaseLabel = if (release.isEmpty()) "unknown" else release
return "OpenClawAndroid/$version (Android $releaseLabel; SDK ${Build.VERSION.SDK_INT})"
}
/** Client identity block shared by node and operator gateway sessions. */
fun buildClientInfo(
clientId: String,
clientMode: String,
): GatewayClientInfo =
GatewayClientInfo(
id = clientId,
displayName = prefs.displayName.value,
version = resolvedVersionName(),
platform = "android",
mode = clientMode,
instanceId = prefs.instanceId.value,
deviceFamily = "Android",
modelIdentifier = resolveModelIdentifier(),
)
/** Connect options for the Android node session that exposes phone capabilities. */
fun buildNodeConnectOptions(): GatewayConnectOptions =
GatewayConnectOptions(
role = "node",
scopes = emptyList(),
caps = buildCapabilities(),
commands = buildInvokeCommands(),
permissions = emptyMap(),
client = buildClientInfo(clientId = "openclaw-android", clientMode = "node"),
userAgent = buildUserAgent(),
)
/** Connect options for the Android operator session that drives approvals and UI actions. */
fun buildOperatorConnectOptions(
scopes: List<String> = nativeClientOperatorScopes,
): GatewayConnectOptions =
GatewayConnectOptions(
role = "operator",
scopes = scopes,
caps = emptyList(),
commands = emptyList(),
permissions = emptyMap(),
client = buildClientInfo(clientId = "openclaw-android", clientMode = "ui"),
userAgent = buildUserAgent(),
)
/** Resolves persisted TLS pin policy for a concrete gateway endpoint. */
fun resolveTlsParams(endpoint: GatewayEndpoint): GatewayTlsParams? {
val stored = prefs.loadGatewayTlsFingerprint(endpoint.stableId)
return resolveTlsParamsForEndpoint(endpoint, storedFingerprint = stored, manualTlsEnabled = manualTls())
}
}

View File

@@ -0,0 +1,489 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.ContentProviderOperation
import android.content.ContentResolver
import android.content.Context
import android.provider.ContactsContract
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
private const val DEFAULT_CONTACTS_LIMIT = 25
/**
* Normalized Android contact row returned through the contacts commands.
*/
internal data class ContactRecord(
val identifier: String,
val displayName: String,
val givenName: String,
val familyName: String,
val organizationName: String,
val phoneNumbers: List<String>,
val emails: List<String>,
)
/**
* Parsed contacts.search request with bounded result count.
*/
internal data class ContactsSearchRequest(
val query: String?,
val limit: Int,
)
/**
* Parsed contacts.add request before ContentProviderOperation batching.
*/
internal data class ContactsAddRequest(
val givenName: String?,
val familyName: String?,
val organizationName: String?,
val displayName: String?,
val phoneNumbers: List<String>,
val emails: List<String>,
)
/**
* Injectable ContactsProvider facade for command tests and Android runtime access.
*/
internal interface ContactsDataSource {
fun hasReadPermission(context: Context): Boolean
fun hasWritePermission(context: Context): Boolean
fun search(
context: Context,
request: ContactsSearchRequest,
): List<ContactRecord>
fun add(
context: Context,
request: ContactsAddRequest,
): ContactRecord
}
private object SystemContactsDataSource : ContactsDataSource {
override fun hasReadPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CONTACTS) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun hasWritePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CONTACTS) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override fun search(
context: Context,
request: ContactsSearchRequest,
): List<ContactRecord> {
val resolver = context.contentResolver
val projection =
arrayOf(
ContactsContract.Contacts._ID,
ContactsContract.Contacts.DISPLAY_NAME_PRIMARY,
)
val selection: String?
val selectionArgs: Array<String>?
if (request.query.isNullOrBlank()) {
selection = null
selectionArgs = null
} else {
// Escape wildcard characters so user text remains a substring search, not a LIKE pattern.
selection = "${ContactsContract.Contacts.DISPLAY_NAME_PRIMARY} LIKE ? ESCAPE '\\'"
selectionArgs = arrayOf("%${escapeLikePattern(request.query)}%")
}
val sortOrder = "${ContactsContract.Contacts.DISPLAY_NAME_PRIMARY} COLLATE NOCASE ASC LIMIT ${request.limit}"
resolver
.query(
ContactsContract.Contacts.CONTENT_URI,
projection,
selection,
selectionArgs,
sortOrder,
).use { cursor ->
if (cursor == null) return emptyList()
val idIndex = cursor.getColumnIndexOrThrow(ContactsContract.Contacts._ID)
val displayNameIndex = cursor.getColumnIndexOrThrow(ContactsContract.Contacts.DISPLAY_NAME_PRIMARY)
val out = mutableListOf<ContactRecord>()
while (cursor.moveToNext() && out.size < request.limit) {
val contactId = cursor.getLong(idIndex)
val displayName = cursor.getString(displayNameIndex).orEmpty()
out += loadContactRecord(resolver, contactId, fallbackDisplayName = displayName)
}
return out
}
}
override fun add(
context: Context,
request: ContactsAddRequest,
): ContactRecord {
val resolver = context.contentResolver
val operations = ArrayList<ContentProviderOperation>()
operations +=
ContentProviderOperation
.newInsert(ContactsContract.RawContacts.CONTENT_URI)
.withValue(ContactsContract.RawContacts.ACCOUNT_TYPE, null)
.withValue(ContactsContract.RawContacts.ACCOUNT_NAME, null)
.build()
// Subsequent Data rows use back-reference 0 to attach to the RawContact inserted above.
if (!request.givenName.isNullOrEmpty() || !request.familyName.isNullOrEmpty() || !request.displayName.isNullOrEmpty()) {
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.StructuredName.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.StructuredName.GIVEN_NAME, request.givenName)
.withValue(ContactsContract.CommonDataKinds.StructuredName.FAMILY_NAME, request.familyName)
.withValue(ContactsContract.CommonDataKinds.StructuredName.DISPLAY_NAME, request.displayName)
.build()
}
if (!request.organizationName.isNullOrEmpty()) {
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Organization.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.Organization.COMPANY, request.organizationName)
.build()
}
request.phoneNumbers.forEach { number ->
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Phone.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.Phone.NUMBER, number)
.withValue(ContactsContract.CommonDataKinds.Phone.TYPE, ContactsContract.CommonDataKinds.Phone.TYPE_MOBILE)
.build()
}
request.emails.forEach { email ->
operations +=
ContentProviderOperation
.newInsert(ContactsContract.Data.CONTENT_URI)
.withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0)
.withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Email.CONTENT_ITEM_TYPE)
.withValue(ContactsContract.CommonDataKinds.Email.ADDRESS, email)
.withValue(ContactsContract.CommonDataKinds.Email.TYPE, ContactsContract.CommonDataKinds.Email.TYPE_HOME)
.build()
}
val results = resolver.applyBatch(ContactsContract.AUTHORITY, operations)
val rawContactUri =
results.firstOrNull()?.uri
?: throw IllegalStateException("contact insert failed")
val rawContactId =
rawContactUri.lastPathSegment?.toLongOrNull()
?: throw IllegalStateException("contact insert failed")
val contactId =
// Android returns the RawContact id; resolve the aggregate Contact id used by search APIs.
resolveContactIdForRawContact(resolver, rawContactId)
?: throw IllegalStateException("contact insert failed")
return loadContactRecord(
resolver = resolver,
contactId = contactId,
fallbackDisplayName = request.displayName.orEmpty(),
)
}
private fun resolveContactIdForRawContact(
resolver: ContentResolver,
rawContactId: Long,
): Long? {
val projection = arrayOf(ContactsContract.RawContacts.CONTACT_ID)
resolver
.query(
ContactsContract.RawContacts.CONTENT_URI,
projection,
"${ContactsContract.RawContacts._ID}=?",
arrayOf(rawContactId.toString()),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
val index = cursor.getColumnIndexOrThrow(ContactsContract.RawContacts.CONTACT_ID)
return cursor.getLong(index)
}
}
private fun loadContactRecord(
resolver: ContentResolver,
contactId: Long,
fallbackDisplayName: String,
): ContactRecord {
val nameRow = loadNameRow(resolver, contactId)
val organization = loadOrganization(resolver, contactId)
val phones = loadPhones(resolver, contactId)
val emails = loadEmails(resolver, contactId)
val displayName =
when {
!nameRow.displayName.isNullOrEmpty() -> nameRow.displayName
!fallbackDisplayName.isNullOrEmpty() -> fallbackDisplayName
else -> listOfNotNull(nameRow.givenName, nameRow.familyName).joinToString(" ").trim()
}.ifEmpty { "(unnamed)" }
return ContactRecord(
identifier = contactId.toString(),
displayName = displayName,
givenName = nameRow.givenName.orEmpty(),
familyName = nameRow.familyName.orEmpty(),
organizationName = organization.orEmpty(),
phoneNumbers = phones,
emails = emails,
)
}
private data class NameRow(
val givenName: String?,
val familyName: String?,
val displayName: String?,
)
private fun loadNameRow(
resolver: ContentResolver,
contactId: Long,
): NameRow {
val projection =
arrayOf(
ContactsContract.CommonDataKinds.StructuredName.GIVEN_NAME,
ContactsContract.CommonDataKinds.StructuredName.FAMILY_NAME,
ContactsContract.CommonDataKinds.StructuredName.DISPLAY_NAME,
)
resolver
.query(
ContactsContract.Data.CONTENT_URI,
projection,
"${ContactsContract.Data.CONTACT_ID}=? AND ${ContactsContract.Data.MIMETYPE}=?",
arrayOf(
contactId.toString(),
ContactsContract.CommonDataKinds.StructuredName.CONTENT_ITEM_TYPE,
),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) {
return NameRow(givenName = null, familyName = null, displayName = null)
}
val given = cursor.getString(0)?.trim()?.ifEmpty { null }
val family = cursor.getString(1)?.trim()?.ifEmpty { null }
val display = cursor.getString(2)?.trim()?.ifEmpty { null }
return NameRow(givenName = given, familyName = family, displayName = display)
}
}
private fun loadOrganization(
resolver: ContentResolver,
contactId: Long,
): String? {
val projection = arrayOf(ContactsContract.CommonDataKinds.Organization.COMPANY)
resolver
.query(
ContactsContract.Data.CONTENT_URI,
projection,
"${ContactsContract.Data.CONTACT_ID}=? AND ${ContactsContract.Data.MIMETYPE}=?",
arrayOf(contactId.toString(), ContactsContract.CommonDataKinds.Organization.CONTENT_ITEM_TYPE),
null,
).use { cursor ->
if (cursor == null || !cursor.moveToFirst()) return null
return cursor.getString(0)?.trim()?.ifEmpty { null }
}
}
private fun escapeLikePattern(pattern: String): String = pattern.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
private fun loadPhones(
resolver: ContentResolver,
contactId: Long,
): List<String> =
queryContactValues(
resolver = resolver,
contentUri = ContactsContract.CommonDataKinds.Phone.CONTENT_URI,
valueColumn = ContactsContract.CommonDataKinds.Phone.NUMBER,
contactIdColumn = ContactsContract.CommonDataKinds.Phone.CONTACT_ID,
contactId = contactId,
)
private fun loadEmails(
resolver: ContentResolver,
contactId: Long,
): List<String> =
queryContactValues(
resolver = resolver,
contentUri = ContactsContract.CommonDataKinds.Email.CONTENT_URI,
valueColumn = ContactsContract.CommonDataKinds.Email.ADDRESS,
contactIdColumn = ContactsContract.CommonDataKinds.Email.CONTACT_ID,
contactId = contactId,
)
private fun queryContactValues(
resolver: ContentResolver,
contentUri: android.net.Uri,
valueColumn: String,
contactIdColumn: String,
contactId: Long,
): List<String> {
val projection = arrayOf(valueColumn)
resolver
.query(
contentUri,
projection,
"$contactIdColumn=?",
arrayOf(contactId.toString()),
null,
).use { cursor ->
if (cursor == null) return emptyList()
val out = LinkedHashSet<String>()
while (cursor.moveToNext()) {
val value = cursor.getString(0)?.trim().orEmpty()
if (value.isNotEmpty()) out += value
}
return out.toList()
}
}
}
/**
* Handles contacts.search and contacts.add gateway commands through Android ContactsProvider.
*/
class ContactsHandler private constructor(
private val appContext: Context,
private val dataSource: ContactsDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemContactsDataSource)
/** Searches contacts by optional display-name substring with bounded result count. */
fun handleContactsSearch(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasReadPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CONTACTS_PERMISSION_REQUIRED",
message = "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission",
)
}
val request =
parseSearchRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val contacts = dataSource.search(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put(
"contacts",
buildJsonArray {
contacts.forEach { add(contactJson(it)) }
},
)
}.toString(),
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CONTACTS_UNAVAILABLE",
message = "CONTACTS_UNAVAILABLE: ${err.message ?: "contacts query failed"}",
)
}
}
/** Adds a local contact after validating that at least one user-visible field is present. */
fun handleContactsAdd(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasWritePermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "CONTACTS_PERMISSION_REQUIRED",
message = "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission",
)
}
val request =
parseAddRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
val hasName =
!(request.givenName.isNullOrEmpty() && request.familyName.isNullOrEmpty() && request.displayName.isNullOrEmpty())
val hasOrg = !request.organizationName.isNullOrEmpty()
val hasDetails = request.phoneNumbers.isNotEmpty() || request.emails.isNotEmpty()
if (!hasName && !hasOrg && !hasDetails) {
return GatewaySession.InvokeResult.error(
code = "CONTACTS_INVALID",
message = "CONTACTS_INVALID: include a name, organization, phone, or email",
)
}
return try {
val contact = dataSource.add(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put("contact", contactJson(contact))
}.toString(),
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "CONTACTS_UNAVAILABLE",
message = "CONTACTS_UNAVAILABLE: ${err.message ?: "contact add failed"}",
)
}
}
private fun parseSearchRequest(paramsJson: String?): ContactsSearchRequest? {
if (paramsJson.isNullOrBlank()) {
return ContactsSearchRequest(query = null, limit = DEFAULT_CONTACTS_LIMIT)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val query = (params["query"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }
// Keep gateway-driven searches bounded even if the model asks for a large contact dump.
val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: DEFAULT_CONTACTS_LIMIT).coerceIn(1, 200)
return ContactsSearchRequest(query = query, limit = limit)
}
private fun parseAddRequest(paramsJson: String?): ContactsAddRequest? {
val params =
try {
paramsJson?.let { Json.parseToJsonElement(it).asObjectOrNull() }
} catch (_: Throwable) {
null
} ?: return null
return ContactsAddRequest(
givenName = (params["givenName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
familyName = (params["familyName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
organizationName = (params["organizationName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
displayName = (params["displayName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
phoneNumbers = stringArray(params["phoneNumbers"] as? JsonArray),
// Store emails case-normalized so repeated model calls do not create casing-only duplicates.
emails = stringArray(params["emails"] as? JsonArray).map { it.lowercase() },
)
}
private fun stringArray(array: JsonArray?): List<String> {
if (array == null) return emptyList()
return array.mapNotNull { element ->
(element as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }
}
}
private fun contactJson(contact: ContactRecord): JsonObject =
buildJsonObject {
put("identifier", JsonPrimitive(contact.identifier))
put("displayName", JsonPrimitive(contact.displayName))
put("givenName", JsonPrimitive(contact.givenName))
put("familyName", JsonPrimitive(contact.familyName))
put("organizationName", JsonPrimitive(contact.organizationName))
put("phoneNumbers", buildJsonArray { contact.phoneNumbers.forEach { add(JsonPrimitive(it)) } })
put("emails", buildJsonArray { contact.emails.forEach { add(JsonPrimitive(it)) } })
}
companion object {
/** Creates a handler with an injected contacts source for parser and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: ContactsDataSource,
): ContactsHandler = ContactsHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,155 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.gateway.DeviceIdentityStore
import ai.openclaw.app.gateway.GatewaySession
import android.content.Context
import kotlinx.serialization.json.JsonPrimitive
private const val LOGCAT_PATH = "/system/bin/logcat"
/**
* Debug-only node.invoke commands for Android cryptography and log diagnostics.
*/
class DebugHandler(
private val appContext: Context,
private val identityStore: DeviceIdentityStore,
) {
/**
* Runs an Ed25519 self-test and returns redacted diagnostics for debug builds.
*/
fun handleEd25519(): GatewaySession.InvokeResult {
if (!BuildConfig.DEBUG) {
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = "debug commands are disabled in release builds")
}
// Self-test Ed25519 signing without returning full private/public key material.
try {
val identity = identityStore.loadOrCreate()
val testPayload = "test|${identity.deviceId}|${System.currentTimeMillis()}"
val results = mutableListOf<String>()
results.add("deviceId: ${identity.deviceId}")
results.add("publicKeyRawBase64: ${identity.publicKeyRawBase64.take(20)}...")
results.add("privateKeyPkcs8Base64: ${identity.privateKeyPkcs8Base64.take(20)}...")
// Public-key URL encoding must match the gateway device-auth payload contract.
val pubKeyUrl = identityStore.publicKeyBase64Url(identity)
results.add("publicKeyBase64Url: ${pubKeyUrl ?: "NULL (FAILED)"}")
// Sign/verify through DeviceIdentityStore to catch provider and key-format failures together.
val signature = identityStore.signPayload(testPayload, identity)
results.add("signPayload: ${if (signature != null) "${signature.take(20)}... (OK)" else "NULL (FAILED)"}")
if (signature != null) {
val verifyOk = identityStore.verifySelfSignature(testPayload, signature, identity)
results.add("verifySelfSignature: $verifyOk")
}
// Check available providers
val providers = java.security.Security.getProviders()
val ed25519Providers =
providers.filter { p ->
p.services.any { s -> s.algorithm.contains("Ed25519", ignoreCase = true) }
}
results.add("Ed25519 providers: ${ed25519Providers.map { "${it.name} v${it.version}" }}")
results.add("Provider order: ${providers.take(5).map { it.name }}")
// Test KeyFactory directly
try {
val kf = java.security.KeyFactory.getInstance("Ed25519")
results.add("KeyFactory.Ed25519: ${kf.provider.name} (OK)")
} catch (e: Throwable) {
results.add("KeyFactory.Ed25519: FAILED - ${e.javaClass.simpleName}: ${e.message}")
}
// Test Signature directly
try {
val sig = java.security.Signature.getInstance("Ed25519")
results.add("Signature.Ed25519: ${sig.provider.name} (OK)")
} catch (e: Throwable) {
results.add("Signature.Ed25519: FAILED - ${e.javaClass.simpleName}: ${e.message}")
}
val diagnostics = results.joinToString("\n")
return GatewaySession.InvokeResult.ok("""{"diagnostics":${JsonPrimitive(diagnostics)}}""")
} catch (e: Throwable) {
return GatewaySession.InvokeResult.error(
code = "ED25519_TEST_FAILED",
message = "${e.javaClass.simpleName}: ${e.message}\n${e.stackTraceToString().take(500)}",
)
}
}
/**
* Returns a filtered logcat snapshot plus CameraX debug log for debug builds.
*/
fun handleLogs(): GatewaySession.InvokeResult {
if (!BuildConfig.DEBUG) {
return GatewaySession.InvokeResult.error(code = "UNAVAILABLE", message = "debug commands are disabled in release builds")
}
val pid = android.os.Process.myPid()
val rt = Runtime.getRuntime()
val info = "v6 pid=$pid thread=${Thread.currentThread().name} free=${rt.freeMemory() / 1024}K total=${rt.totalMemory() / 1024}K max=${rt.maxMemory() / 1024}K uptime=${android.os.SystemClock.elapsedRealtime() / 1000}s sdk=${android.os.Build.VERSION.SDK_INT} device=${android.os.Build.MODEL}\n"
// Capture only this process and redirect through a temp file to avoid blocking on pipe backpressure.
val logResult =
try {
val tmpFile = java.io.File(appContext.cacheDir, "debug_logs.txt")
if (tmpFile.exists()) tmpFile.delete()
val pb = ProcessBuilder(LOGCAT_PATH, "-d", "-t", "200", "--pid=$pid")
pb.redirectOutput(tmpFile)
pb.redirectErrorStream(true)
val proc = pb.start()
val finished = proc.waitFor(4, java.util.concurrent.TimeUnit.SECONDS)
if (!finished) proc.destroyForcibly()
val raw =
if (tmpFile.exists() && tmpFile.length() > 0) {
tmpFile.readText().take(128000)
} else {
"(no output, finished=$finished, exists=${tmpFile.exists()})"
}
tmpFile.delete()
val spamPatterns =
listOf(
"setRequestedFrameRate",
"I View :",
"BLASTBufferQueue",
"VRI[Pop-Up",
"InsetsController:",
"VRI[MainActivity",
"InsetsSource:",
"handleResized",
"ProfileInstaller",
"I VRI[",
"onStateChanged: host=",
"D StrictMode:",
"E StrictMode:",
"ImeFocusController",
"InputTransport",
"IncorrectContextUseViolation",
)
val sb = StringBuilder()
for (line in raw.lineSequence()) {
if (line.isBlank()) continue
if (spamPatterns.any { line.contains(it) }) continue
if (sb.length + line.length > 16000) {
// Keep debug.invoke responses small enough for the gateway WebSocket frame budget.
sb.append("\n(truncated)")
break
}
if (sb.isNotEmpty()) sb.append('\n')
sb.append(line)
}
sb.toString().ifEmpty { "(all ${raw.lines().size} lines filtered as spam)" }
} catch (e: Throwable) {
"(logcat error: ${e::class.java.simpleName}: ${e.message})"
}
// Camera capture writes a separate debug file because CameraX failures often happen off logcat's hot path.
val camLogFile = java.io.File(appContext.cacheDir, "camera_debug.log")
val camLog =
if (camLogFile.exists() && camLogFile.length() > 0) {
"\n--- camera_debug.log ---\n" + camLogFile.readText().take(4000)
} else {
""
}
return GatewaySession.InvokeResult.ok("""{"logs":${JsonPrimitive(info + logResult + camLog)}}""")
}
}

View File

@@ -0,0 +1,660 @@
package ai.openclaw.app.node
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.SensitiveFeatureConfig
import ai.openclaw.app.gateway.GatewaySession
import ai.openclaw.app.hasPhotoReadPermission
import android.Manifest
import android.annotation.SuppressLint
import android.app.ActivityManager
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.ApplicationInfo
import android.content.pm.PackageManager
import android.net.ConnectivityManager
import android.net.NetworkCapabilities
import android.os.BatteryManager
import android.os.Build
import android.os.Environment
import android.os.PowerManager
import android.os.StatFs
import android.os.SystemClock
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.util.Locale
private const val DEFAULT_DEVICE_APPS_LIMIT = 100
private const val MAX_DEVICE_APPS_LIMIT = 200
private const val DEVICE_APPS_SYSTEM_FLAGS =
ApplicationInfo.FLAG_SYSTEM or ApplicationInfo.FLAG_UPDATED_SYSTEM_APP
internal fun isSystemDeviceApp(appInfo: ApplicationInfo): Boolean = (appInfo.flags and DEVICE_APPS_SYSTEM_FLAGS) != 0
internal data class DeviceAppEntry(
val label: String,
val packageName: String,
val system: Boolean,
val enabled: Boolean,
val launchable: Boolean,
)
internal interface DeviceAppSource {
fun listApps(includeNonLaunchable: Boolean): List<DeviceAppEntry>
}
private class AndroidDeviceAppSource(
private val appContext: Context,
) : DeviceAppSource {
override fun listApps(includeNonLaunchable: Boolean): List<DeviceAppEntry> {
val packageManager = appContext.packageManager
val launcherIntent = Intent(Intent.ACTION_MAIN).apply { addCategory(Intent.CATEGORY_LAUNCHER) }
val launchablePackages =
packageManager
.queryIntentActivities(launcherIntent, PackageManager.MATCH_ALL)
.asSequence()
.mapNotNull {
it.activityInfo
?.packageName
?.trim()
?.takeIf(String::isNotEmpty)
}.toSet()
val appInfos =
if (includeNonLaunchable) {
visibleInstalledApplications(packageManager)
} else {
launchablePackages.mapNotNull { packageName ->
runCatching { packageManager.getApplicationInfo(packageName, 0) }.getOrNull()
}
}
return appInfos
.asSequence()
.mapNotNull { appInfo ->
appInfo.packageName
?.trim()
?.takeIf(String::isNotEmpty)
?.let { packageName ->
val label = packageManager.getApplicationLabel(appInfo).toString().trim()
DeviceAppEntry(
label = label.ifEmpty { packageName },
packageName = packageName,
system = isSystemDeviceApp(appInfo),
enabled = appInfo.enabled,
launchable = packageName in launchablePackages,
)
}
}.distinctBy { it.packageName }
.sortedWith(compareBy<DeviceAppEntry> { it.label.lowercase() }.thenBy { it.packageName })
.toList()
}
@SuppressLint("QueryPermissionsNeeded")
private fun visibleInstalledApplications(packageManager: PackageManager): List<ApplicationInfo> {
// Android package visibility intentionally bounds this result to packages the app can see.
// OpenClaw should not request QUERY_ALL_PACKAGES for this optional device-context surface.
return packageManager.getInstalledApplications(PackageManager.MATCH_ALL)
}
}
private data class DeviceAppsRequest(
val includeSystem: Boolean,
val includeDisabled: Boolean,
val includeNonLaunchable: Boolean,
val query: String?,
val limit: Int,
)
/**
* Gateway device command adapter for Android status, info, permission, and health snapshots.
*/
class DeviceHandler private constructor(
private val appContext: Context,
private val smsEnabled: Boolean = SensitiveFeatureConfig.smsEnabled,
private val callLogEnabled: Boolean = SensitiveFeatureConfig.callLogEnabled,
private val photosEnabled: Boolean = SensitiveFeatureConfig.photosEnabled,
private val appSource: DeviceAppSource = AndroidDeviceAppSource(appContext),
) {
constructor(
appContext: Context,
smsEnabled: Boolean = SensitiveFeatureConfig.smsEnabled,
callLogEnabled: Boolean = SensitiveFeatureConfig.callLogEnabled,
photosEnabled: Boolean = SensitiveFeatureConfig.photosEnabled,
) : this(
appContext = appContext,
smsEnabled = smsEnabled,
callLogEnabled = callLogEnabled,
photosEnabled = photosEnabled,
appSource = AndroidDeviceAppSource(appContext),
)
companion object {
internal fun forTesting(
appContext: Context,
appSource: DeviceAppSource,
smsEnabled: Boolean = SensitiveFeatureConfig.smsEnabled,
callLogEnabled: Boolean = SensitiveFeatureConfig.callLogEnabled,
photosEnabled: Boolean = SensitiveFeatureConfig.photosEnabled,
): DeviceHandler =
DeviceHandler(
appContext = appContext,
smsEnabled = smsEnabled,
callLogEnabled = callLogEnabled,
photosEnabled = photosEnabled,
appSource = appSource,
)
/**
* SMS is available only when the feature flag, telephony hardware, and at least one SMS permission align.
*/
internal fun hasAnySmsCapability(
smsEnabled: Boolean,
telephonyAvailable: Boolean,
smsSendGranted: Boolean,
smsReadGranted: Boolean,
): Boolean = smsEnabled && telephonyAvailable && (smsSendGranted || smsReadGranted)
/**
* Prompt only when Android can grant a missing SMS permission that this build can use.
*/
internal fun isSmsPromptable(
smsEnabled: Boolean,
telephonyAvailable: Boolean,
smsSendGranted: Boolean,
smsReadGranted: Boolean,
): Boolean = smsEnabled && telephonyAvailable && (!smsSendGranted || !smsReadGranted)
}
private data class BatterySnapshot(
val status: Int,
val plugged: Int,
val levelFraction: Double?,
val temperatureC: Double?,
)
/** Returns battery, storage, network, and uptime state for device.status. */
fun handleDeviceStatus(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(statusPayloadJson())
/** Returns stable Android hardware, OS, app, and locale metadata for device.info. */
fun handleDeviceInfo(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(infoPayloadJson())
/** Returns permission and promptability state for Android capabilities exposed to the gateway. */
fun handleDevicePermissions(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(permissionsPayloadJson())
/** Returns coarse device health for memory, power, thermal, battery, and security patch state. */
fun handleDeviceHealth(_paramsJson: String?): GatewaySession.InvokeResult = GatewaySession.InvokeResult.ok(healthPayloadJson())
fun handleDeviceApps(paramsJson: String?): GatewaySession.InvokeResult {
val request = parseDeviceAppsRequest(paramsJson)
val matchingApps =
appSource
.listApps(includeNonLaunchable = request.includeNonLaunchable)
.asSequence()
.filter { request.includeSystem || !it.system }
.filter { request.includeDisabled || it.enabled }
.filter { app ->
val query = request.query ?: return@filter true
app.label.contains(query, ignoreCase = true) || app.packageName.contains(query, ignoreCase = true)
}.toList()
val limitedApps = matchingApps.take(request.limit)
return GatewaySession.InvokeResult.ok(
buildJsonObject {
put("count", JsonPrimitive(limitedApps.size))
put("totalMatched", JsonPrimitive(matchingApps.size))
put("truncated", JsonPrimitive(matchingApps.size > limitedApps.size))
put("visibility", JsonPrimitive(if (request.includeNonLaunchable) "android-visible" else "launcher"))
put("includeSystem", JsonPrimitive(request.includeSystem))
put("includeDisabled", JsonPrimitive(request.includeDisabled))
put(
"apps",
buildJsonArray {
for (app in limitedApps) {
add(
buildJsonObject {
put("label", JsonPrimitive(app.label))
put("packageName", JsonPrimitive(app.packageName))
put("system", JsonPrimitive(app.system))
put("enabled", JsonPrimitive(app.enabled))
put("launchable", JsonPrimitive(app.launchable))
},
)
}
},
)
}.toString(),
)
}
private fun statusPayloadJson(): String {
val battery = readBatterySnapshot()
val powerManager = appContext.getSystemService(PowerManager::class.java)
val storage = StatFs(Environment.getDataDirectory().absolutePath)
val totalBytes = storage.totalBytes
val freeBytes = storage.availableBytes
val usedBytes = (totalBytes - freeBytes).coerceAtLeast(0L)
val connectivity = appContext.getSystemService(ConnectivityManager::class.java)
val activeNetwork = connectivity?.activeNetwork
val caps = activeNetwork?.let { connectivity.getNetworkCapabilities(it) }
// elapsedRealtime is monotonic device uptime, not wall-clock time.
val uptimeSeconds = SystemClock.elapsedRealtime() / 1_000.0
return buildJsonObject {
put(
"battery",
buildJsonObject {
battery.levelFraction?.let { put("level", JsonPrimitive(it)) }
put("state", JsonPrimitive(mapBatteryState(battery.status)))
put("lowPowerModeEnabled", JsonPrimitive(powerManager?.isPowerSaveMode == true))
},
)
put(
"thermal",
buildJsonObject {
put("state", JsonPrimitive(mapThermalState(powerManager)))
},
)
put(
"storage",
buildJsonObject {
put("totalBytes", JsonPrimitive(totalBytes))
put("freeBytes", JsonPrimitive(freeBytes))
put("usedBytes", JsonPrimitive(usedBytes))
},
)
put(
"network",
buildJsonObject {
put("status", JsonPrimitive(mapNetworkStatus(caps)))
put(
"isExpensive",
JsonPrimitive(
caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_METERED)?.not() ?: false,
),
)
put(
"isConstrained",
JsonPrimitive(
caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_RESTRICTED)?.not() ?: false,
),
)
put("interfaces", networkInterfacesJson(caps))
},
)
put("uptimeSeconds", JsonPrimitive(uptimeSeconds))
}.toString()
}
private fun infoPayloadJson(): String {
val model = Build.MODEL?.trim().orEmpty()
val manufacturer = Build.MANUFACTURER?.trim().orEmpty()
val modelIdentifier = Build.DEVICE?.trim().orEmpty()
val systemVersion =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
val locale = Locale.getDefault().toLanguageTag().trim()
val appVersion = BuildConfig.VERSION_NAME.trim()
val appBuild = BuildConfig.VERSION_CODE.toString()
return buildJsonObject {
put("deviceName", JsonPrimitive(model.ifEmpty { "Android" }))
put(
"modelIdentifier",
JsonPrimitive(modelIdentifier.ifEmpty { listOf(manufacturer, model).filter { it.isNotEmpty() }.joinToString(" ") }),
)
put("systemName", JsonPrimitive("Android"))
put("systemVersion", JsonPrimitive(systemVersion.ifEmpty { Build.VERSION.SDK_INT.toString() }))
put("appVersion", JsonPrimitive(appVersion.ifEmpty { "dev" }))
put("appBuild", JsonPrimitive(appBuild.ifEmpty { "0" }))
put("locale", JsonPrimitive(locale.ifEmpty { Locale.getDefault().toString() }))
}.toString()
}
private fun permissionsPayloadJson(): String {
val canSendSms = appContext.packageManager.hasSystemFeature(PackageManager.FEATURE_TELEPHONY)
val smsSendGranted = hasPermission(Manifest.permission.SEND_SMS)
val smsReadGranted = hasPermission(Manifest.permission.READ_SMS)
val notificationAccess = DeviceNotificationListenerService.isAccessEnabled(appContext)
val photosGranted =
if (!photosEnabled) {
false
} else {
hasPhotoReadPermission(appContext)
}
val motionGranted = hasPermission(Manifest.permission.ACTIVITY_RECOGNITION)
val notificationsGranted =
if (Build.VERSION.SDK_INT >= 33) {
// POST_NOTIFICATIONS exists only on Android 13+.
hasPermission(Manifest.permission.POST_NOTIFICATIONS)
} else {
true
}
return buildJsonObject {
put(
"permissions",
buildJsonObject {
put(
"camera",
permissionStateJson(
granted = hasPermission(Manifest.permission.CAMERA),
promptableWhenDenied = true,
),
)
put(
"microphone",
permissionStateJson(
granted = hasPermission(Manifest.permission.RECORD_AUDIO),
promptableWhenDenied = true,
),
)
put(
"location",
permissionStateJson(
granted =
hasPermission(Manifest.permission.ACCESS_FINE_LOCATION) ||
hasPermission(Manifest.permission.ACCESS_COARSE_LOCATION),
promptableWhenDenied = true,
),
)
put(
"sms",
buildJsonObject {
put(
"status",
JsonPrimitive(
if (hasAnySmsCapability(
smsEnabled,
canSendSms,
smsSendGranted,
smsReadGranted,
)
) {
"granted"
} else {
"denied"
},
),
)
put("promptable", JsonPrimitive(isSmsPromptable(smsEnabled, canSendSms, smsSendGranted, smsReadGranted)))
put(
"capabilities",
buildJsonObject {
put(
"send",
permissionStateJson(
granted = smsEnabled && smsSendGranted && canSendSms,
promptableWhenDenied = smsEnabled && canSendSms,
),
)
put(
"read",
permissionStateJson(
granted = smsEnabled && smsReadGranted && canSendSms,
promptableWhenDenied = smsEnabled && canSendSms,
),
)
},
)
},
)
put(
"notificationListener",
permissionStateJson(
granted = notificationAccess,
promptableWhenDenied = true,
),
)
put(
"notifications",
permissionStateJson(
granted = notificationsGranted,
promptableWhenDenied = true,
),
)
put(
"photos",
permissionStateJson(
granted = photosGranted,
promptableWhenDenied = photosEnabled,
),
)
put(
"contacts",
permissionStateJson(
granted =
hasPermission(Manifest.permission.READ_CONTACTS) &&
hasPermission(Manifest.permission.WRITE_CONTACTS),
promptableWhenDenied = true,
),
)
put(
"calendar",
permissionStateJson(
granted =
hasPermission(Manifest.permission.READ_CALENDAR) &&
hasPermission(Manifest.permission.WRITE_CALENDAR),
promptableWhenDenied = true,
),
)
put(
"callLog",
permissionStateJson(
granted = callLogEnabled && hasPermission(Manifest.permission.READ_CALL_LOG),
promptableWhenDenied = callLogEnabled,
),
)
put(
"motion",
permissionStateJson(
granted = motionGranted,
promptableWhenDenied = true,
),
)
},
)
}.toString()
}
private fun healthPayloadJson(): String {
val battery = readBatterySnapshot()
val batteryManager = appContext.getSystemService(BatteryManager::class.java)
val currentNowUa = batteryManager?.getLongProperty(BatteryManager.BATTERY_PROPERTY_CURRENT_NOW)
val currentNowMa =
if (currentNowUa == null || currentNowUa == Long.MIN_VALUE) {
null
} else {
// BatteryManager reports microamps; expose milliamps in the gateway payload.
currentNowUa.toDouble() / 1_000.0
}
val powerManager = appContext.getSystemService(PowerManager::class.java)
val activityManager = appContext.getSystemService(ActivityManager::class.java)
val memoryInfo = ActivityManager.MemoryInfo()
activityManager?.getMemoryInfo(memoryInfo)
val totalRamBytes = memoryInfo.totalMem.coerceAtLeast(0L)
val availableRamBytes = memoryInfo.availMem.coerceAtLeast(0L)
val usedRamBytes = (totalRamBytes - availableRamBytes).coerceAtLeast(0L)
val lowMemory = memoryInfo.lowMemory
val memoryPressure = mapMemoryPressure(totalRamBytes, availableRamBytes, lowMemory)
return buildJsonObject {
put(
"memory",
buildJsonObject {
put("pressure", JsonPrimitive(memoryPressure))
put("totalRamBytes", JsonPrimitive(totalRamBytes))
put("availableRamBytes", JsonPrimitive(availableRamBytes))
put("usedRamBytes", JsonPrimitive(usedRamBytes))
put("thresholdBytes", JsonPrimitive(memoryInfo.threshold.coerceAtLeast(0L)))
put("lowMemory", JsonPrimitive(lowMemory))
},
)
put(
"battery",
buildJsonObject {
put("state", JsonPrimitive(mapBatteryState(battery.status)))
put("chargingType", JsonPrimitive(mapChargingType(battery.plugged)))
battery.temperatureC?.let { put("temperatureC", JsonPrimitive(it)) }
currentNowMa?.let { put("currentMa", JsonPrimitive(it)) }
},
)
put(
"power",
buildJsonObject {
put("dozeModeEnabled", JsonPrimitive(powerManager?.isDeviceIdleMode == true))
put("lowPowerModeEnabled", JsonPrimitive(powerManager?.isPowerSaveMode == true))
},
)
put(
"system",
buildJsonObject {
Build.VERSION.SECURITY_PATCH
?.trim()
?.takeIf { it.isNotEmpty() }
?.let { put("securityPatchLevel", JsonPrimitive(it)) }
},
)
}.toString()
}
private fun parseDeviceAppsRequest(paramsJson: String?): DeviceAppsRequest {
val params = parseJsonParamsObject(paramsJson)
val includeSystem = parseJsonBooleanFlag(params, "includeSystem") ?: false
val includeDisabled = parseJsonBooleanFlag(params, "includeDisabled") ?: false
val includeNonLaunchable = parseJsonBooleanFlag(params, "includeNonLaunchable") ?: false
val query = parseJsonString(params, "query")?.trim()?.takeIf { it.isNotEmpty() }
val limit =
(parseJsonInt(params, "limit") ?: DEFAULT_DEVICE_APPS_LIMIT)
.coerceIn(1, MAX_DEVICE_APPS_LIMIT)
return DeviceAppsRequest(
includeSystem = includeSystem,
includeDisabled = includeDisabled,
includeNonLaunchable = includeNonLaunchable,
query = query,
limit = limit,
)
}
private fun readBatterySnapshot(): BatterySnapshot {
// ACTION_BATTERY_CHANGED is sticky; registerReceiver(null, ...) reads the last system snapshot.
val intent = appContext.registerReceiver(null, IntentFilter(Intent.ACTION_BATTERY_CHANGED))
val status =
intent?.getIntExtra(BatteryManager.EXTRA_STATUS, BatteryManager.BATTERY_STATUS_UNKNOWN)
?: BatteryManager.BATTERY_STATUS_UNKNOWN
val plugged = intent?.getIntExtra(BatteryManager.EXTRA_PLUGGED, 0) ?: 0
val temperatureC =
intent
?.getIntExtra(BatteryManager.EXTRA_TEMPERATURE, Int.MIN_VALUE)
?.takeIf { it != Int.MIN_VALUE }
?.toDouble()
?.div(10.0)
return BatterySnapshot(
status = status,
plugged = plugged,
levelFraction = batteryLevelFraction(intent),
temperatureC = temperatureC,
)
}
private fun batteryLevelFraction(intent: Intent?): Double? {
val rawLevel = intent?.getIntExtra(BatteryManager.EXTRA_LEVEL, -1) ?: -1
val rawScale = intent?.getIntExtra(BatteryManager.EXTRA_SCALE, -1) ?: -1
if (rawLevel < 0 || rawScale <= 0) return null
return rawLevel.toDouble() / rawScale.toDouble()
}
private fun mapBatteryState(status: Int): String =
when (status) {
BatteryManager.BATTERY_STATUS_CHARGING -> "charging"
BatteryManager.BATTERY_STATUS_FULL -> "full"
BatteryManager.BATTERY_STATUS_DISCHARGING, BatteryManager.BATTERY_STATUS_NOT_CHARGING -> "unplugged"
else -> "unknown"
}
private fun mapChargingType(plugged: Int): String =
when (plugged) {
BatteryManager.BATTERY_PLUGGED_AC -> "ac"
BatteryManager.BATTERY_PLUGGED_USB -> "usb"
BatteryManager.BATTERY_PLUGGED_WIRELESS -> "wireless"
BatteryManager.BATTERY_PLUGGED_DOCK -> "dock"
else -> "none"
}
private fun mapThermalState(powerManager: PowerManager?): String {
val thermal = powerManager?.currentThermalStatus ?: return "nominal"
return when (thermal) {
PowerManager.THERMAL_STATUS_NONE, PowerManager.THERMAL_STATUS_LIGHT -> "nominal"
PowerManager.THERMAL_STATUS_MODERATE -> "fair"
PowerManager.THERMAL_STATUS_SEVERE -> "serious"
PowerManager.THERMAL_STATUS_CRITICAL,
PowerManager.THERMAL_STATUS_EMERGENCY,
PowerManager.THERMAL_STATUS_SHUTDOWN,
-> "critical"
else -> "nominal"
}
}
private fun mapNetworkStatus(caps: NetworkCapabilities?): String {
if (caps == null) return "unsatisfied"
return when {
caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_VALIDATED) -> "satisfied"
// Internet without validation mirrors iOS "requiresConnection" for captive or unproven networks.
caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET) -> "requiresConnection"
else -> "unsatisfied"
}
}
private fun permissionStateJson(
granted: Boolean,
promptableWhenDenied: Boolean,
) = buildJsonObject {
put("status", JsonPrimitive(if (granted) "granted" else "denied"))
put("promptable", JsonPrimitive(!granted && promptableWhenDenied))
}
private fun hasPermission(permission: String): Boolean =
(
ContextCompat.checkSelfPermission(appContext, permission) == PackageManager.PERMISSION_GRANTED
)
private fun mapMemoryPressure(
totalBytes: Long,
availableBytes: Long,
lowMemory: Boolean,
): String {
if (totalBytes <= 0L) return if (lowMemory) "critical" else "unknown"
if (lowMemory) return "critical"
val freeRatio = availableBytes.toDouble() / totalBytes.toDouble()
// Thresholds intentionally mirror coarse OS health labels instead of exact memory pressure.
return when {
freeRatio <= 0.05 -> "critical"
freeRatio <= 0.15 -> "high"
freeRatio <= 0.30 -> "moderate"
else -> "normal"
}
}
private fun networkInterfacesJson(caps: NetworkCapabilities?) =
buildJsonArray {
if (caps == null) return@buildJsonArray
var hasKnownTransport = false
if (caps.hasTransport(NetworkCapabilities.TRANSPORT_WIFI)) {
hasKnownTransport = true
add(JsonPrimitive("wifi"))
}
if (caps.hasTransport(NetworkCapabilities.TRANSPORT_CELLULAR)) {
hasKnownTransport = true
add(JsonPrimitive("cellular"))
}
if (caps.hasTransport(NetworkCapabilities.TRANSPORT_ETHERNET)) {
hasKnownTransport = true
add(JsonPrimitive("wired"))
}
if (!hasKnownTransport) add(JsonPrimitive("other"))
}
}

View File

@@ -0,0 +1,517 @@
package ai.openclaw.app.node
import ai.openclaw.app.NotificationBurstLimiter
import ai.openclaw.app.SecurePrefs
import ai.openclaw.app.allowsPackage
import ai.openclaw.app.isWithinQuietHours
import android.app.Notification
import android.app.NotificationManager
import android.app.RemoteInput
import android.content.ComponentName
import android.content.Context
import android.content.Intent
import android.service.notification.NotificationListenerService
import android.service.notification.StatusBarNotification
import androidx.core.content.edit
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
private const val MAX_NOTIFICATION_TEXT_CHARS = 512
private const val NOTIFICATIONS_CHANGED_EVENT = "notifications.changed"
/**
* Trims notification text and caps payload size before it enters gateway-visible state.
*/
internal fun sanitizeNotificationText(value: CharSequence?): String? {
val normalized = value?.toString()?.trim().orEmpty()
// Notification extras can include long previews; cap before sending over node events.
return normalized.take(MAX_NOTIFICATION_TEXT_CHARS).ifEmpty { null }
}
/**
* Stable notification snapshot entry exposed through the Android notifications command.
*/
data class DeviceNotificationEntry(
val key: String,
val packageName: String,
val title: String?,
val text: String?,
val subText: String?,
val category: String?,
val channelId: String?,
val postTimeMs: Long,
val isOngoing: Boolean,
val isClearable: Boolean,
)
internal fun DeviceNotificationEntry.toJsonObject(): JsonObject =
buildJsonObject {
put("key", JsonPrimitive(key))
put("packageName", JsonPrimitive(packageName))
put("postTimeMs", JsonPrimitive(postTimeMs))
put("isOngoing", JsonPrimitive(isOngoing))
put("isClearable", JsonPrimitive(isClearable))
title?.let { put("title", JsonPrimitive(it)) }
text?.let { put("text", JsonPrimitive(it)) }
subText?.let { put("subText", JsonPrimitive(it)) }
category?.let { put("category", JsonPrimitive(it)) }
channelId?.let { put("channelId", JsonPrimitive(it)) }
}
/**
* Listener state exposed to the gateway, including whether Android has connected the service.
*/
data class DeviceNotificationSnapshot(
val enabled: Boolean,
val connected: Boolean,
val notifications: List<DeviceNotificationEntry>,
)
/**
* Gateway-supported notification actions mapped to Android listener operations.
*/
enum class NotificationActionKind {
Open,
Dismiss,
Reply,
}
/**
* Gateway action request; [key] must match Android's StatusBarNotification key.
*/
data class NotificationActionRequest(
val key: String,
val kind: NotificationActionKind,
val replyText: String? = null,
)
/**
* Normalized notification action result returned through node.invoke.
*/
data class NotificationActionResult(
val ok: Boolean,
val code: String? = null,
val message: String? = null,
)
internal fun actionRequiresClearableNotification(kind: NotificationActionKind): Boolean = kind == NotificationActionKind.Dismiss
/**
* Process-local cache of active notifications mirrored from Android listener callbacks.
*/
private object DeviceNotificationStore {
private val lock = Any()
private var connected = false
private val byKey = LinkedHashMap<String, DeviceNotificationEntry>()
fun replace(entries: List<DeviceNotificationEntry>) {
synchronized(lock) {
byKey.clear()
for (entry in entries) {
byKey[entry.key] = entry
}
}
}
fun upsert(entry: DeviceNotificationEntry) {
synchronized(lock) {
byKey[entry.key] = entry
}
}
fun remove(key: String) {
synchronized(lock) {
byKey.remove(key)
}
}
fun setConnected(value: Boolean) {
synchronized(lock) {
connected = value
if (!value) {
// Android invalidates activeNotifications when the listener disconnects.
byKey.clear()
}
}
}
fun snapshot(enabled: Boolean): DeviceNotificationSnapshot {
val (isConnected, entries) =
synchronized(lock) {
connected to byKey.values.sortedByDescending { it.postTimeMs }
}
return DeviceNotificationSnapshot(
enabled = enabled,
connected = isConnected,
notifications = entries,
)
}
}
/**
* Android notification listener that mirrors notification state and executes gateway actions.
*/
class DeviceNotificationListenerService : NotificationListenerService() {
private val securePrefs by lazy { SecurePrefs(applicationContext) }
private val forwardingLimiter = NotificationBurstLimiter()
override fun onListenerConnected() {
super.onListenerConnected()
activeService = this
DeviceNotificationStore.setConnected(true)
refreshActiveNotifications()
}
override fun onListenerDisconnected() {
if (activeService === this) {
activeService = null
}
DeviceNotificationStore.setConnected(false)
super.onListenerDisconnected()
}
override fun onDestroy() {
if (activeService === this) {
activeService = null
}
super.onDestroy()
}
override fun onNotificationPosted(sbn: StatusBarNotification?) {
super.onNotificationPosted(sbn)
val entry = sbn?.toEntry() ?: return
DeviceNotificationStore.upsert(entry)
rememberRecentPackage(entry.packageName)
if (entry.packageName == packageName) {
return
}
val payload = notificationChangedPayload(entry) ?: return
emitNotificationsChanged(payload)
}
override fun onNotificationRemoved(sbn: StatusBarNotification?) {
super.onNotificationRemoved(sbn)
val removed = sbn ?: return
val key = removed.key.trim()
if (key.isEmpty()) {
return
}
DeviceNotificationStore.remove(key)
rememberRecentPackage(removed.packageName)
if (removed.packageName == packageName) {
return
}
val packageName = removed.packageName.trim()
val payload =
notificationChangedPayload(
entry = null,
change = "removed",
key = key,
packageName = packageName,
postTimeMs = removed.postTime,
isOngoing = removed.isOngoing,
isClearable = removed.isClearable,
) ?: return
emitNotificationsChanged(payload)
}
private fun notificationChangedPayload(entry: DeviceNotificationEntry): String? =
notificationChangedPayload(
entry = entry,
change = "posted",
key = entry.key,
packageName = entry.packageName,
postTimeMs = entry.postTimeMs,
isOngoing = entry.isOngoing,
isClearable = entry.isClearable,
)
private fun notificationChangedPayload(
entry: DeviceNotificationEntry?,
change: String,
key: String,
packageName: String,
postTimeMs: Long,
isOngoing: Boolean,
isClearable: Boolean,
): String? {
val normalizedPackage = packageName.trim()
if (normalizedPackage.isEmpty()) {
return null
}
val policy = securePrefs.getNotificationForwardingPolicy(appPackageName = this.packageName)
if (!policy.enabled) {
return null
}
if (!policy.allowsPackage(normalizedPackage)) {
return null
}
val nowEpochMs = System.currentTimeMillis()
if (policy.isWithinQuietHours(nowEpochMs = nowEpochMs)) {
return null
}
// Apply burst limits after package/quiet-hour filters so blocked notifications do not consume quota.
if (!forwardingLimiter.allow(nowEpochMs, policy.maxEventsPerMinute)) {
return null
}
return buildJsonObject {
put("change", JsonPrimitive(change))
put("key", JsonPrimitive(key))
put("packageName", JsonPrimitive(normalizedPackage))
put("postTimeMs", JsonPrimitive(postTimeMs))
put("isOngoing", JsonPrimitive(isOngoing))
put("isClearable", JsonPrimitive(isClearable))
policy.sessionKey?.let { put("sessionKey", JsonPrimitive(it)) }
entry?.title?.let { put("title", JsonPrimitive(it)) }
entry?.text?.let { put("text", JsonPrimitive(it)) }
entry?.subText?.let { put("subText", JsonPrimitive(it)) }
entry?.category?.let { put("category", JsonPrimitive(it)) }
entry?.channelId?.let { put("channelId", JsonPrimitive(it)) }
}.toString()
}
private fun refreshActiveNotifications() {
val entries =
runCatching {
activeNotifications
?.mapNotNull { it.toEntry() }
?: emptyList()
}.getOrElse { emptyList() }
DeviceNotificationStore.replace(entries)
}
private fun StatusBarNotification.toEntry(): DeviceNotificationEntry {
val extras = notification.extras
val keyValue = key.takeIf { it.isNotBlank() } ?: "$packageName:$id:$postTime"
val title = sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_TITLE))
val body =
sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_BIG_TEXT))
?: sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_TEXT))
val subText = sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_SUB_TEXT))
return DeviceNotificationEntry(
key = keyValue,
packageName = packageName,
title = title,
text = body,
subText = subText,
category = notification.category?.trim()?.ifEmpty { null },
channelId = notification.channelId?.trim()?.ifEmpty { null },
postTimeMs = postTime,
isOngoing = isOngoing,
isClearable = isClearable,
)
}
companion object {
private const val recentPackagesPref = "notifications.forwarding.recentPackages"
private const val legacyRecentPackagesPref = "notifications.recentPackages"
private const val recentPackagesLimit = 64
@Volatile private var activeService: DeviceNotificationListenerService? = null
@Volatile private var nodeEventSink: ((event: String, payloadJson: String?) -> Unit)? = null
private fun serviceComponent(context: Context): ComponentName = ComponentName(context, DeviceNotificationListenerService::class.java)
/** Installs the node event sink used to emit filtered notification change events. */
fun setNodeEventSink(sink: ((event: String, payloadJson: String?) -> Unit)?) {
nodeEventSink = sink
}
private fun recentPackagesPrefs(context: Context) = context.applicationContext.getSharedPreferences("openclaw.secure", Context.MODE_PRIVATE)
private fun migrateLegacyRecentPackagesIfNeeded(context: Context) {
val prefs = recentPackagesPrefs(context)
val hasNew = prefs.contains(recentPackagesPref)
val legacy = prefs.getString(legacyRecentPackagesPref, null)?.trim().orEmpty()
if (!hasNew && legacy.isNotEmpty()) {
// Keep recent package suggestions across the preference-key rename.
prefs.edit {
putString(recentPackagesPref, legacy)
remove(legacyRecentPackagesPref)
}
} else if (hasNew && prefs.contains(legacyRecentPackagesPref)) {
prefs.edit { remove(legacyRecentPackagesPref) }
}
}
/** Returns recent third-party packages seen by the listener for settings suggestions. */
fun recentPackages(context: Context): List<String> {
migrateLegacyRecentPackagesIfNeeded(context)
val prefs = recentPackagesPrefs(context)
val stored = prefs.getString(recentPackagesPref, null).orEmpty()
return stored
.split(',')
.map { it.trim() }
.filter { it.isNotEmpty() }
.distinct()
}
/** Checks whether Android has granted listener access to this service component. */
fun isAccessEnabled(context: Context): Boolean {
val manager = context.getSystemService(NotificationManager::class.java) ?: return false
return manager.isNotificationListenerAccessGranted(serviceComponent(context))
}
/** Reads the current mirrored notification snapshot without forcing service startup. */
fun snapshot(
context: Context,
enabled: Boolean = isAccessEnabled(context),
): DeviceNotificationSnapshot = DeviceNotificationStore.snapshot(enabled = enabled)
/** Asks Android to rebind the listener after settings grant access but callbacks have not arrived. */
fun requestServiceRebind(context: Context) {
runCatching {
NotificationListenerService.requestRebind(serviceComponent(context))
}
}
/** Executes an open, dismiss, or reply action through the active listener instance. */
fun executeAction(
context: Context,
request: NotificationActionRequest,
): NotificationActionResult {
if (!isAccessEnabled(context)) {
return NotificationActionResult(
ok = false,
code = "NOTIFICATIONS_DISABLED",
message = "NOTIFICATIONS_DISABLED: enable notification access in system Settings",
)
}
val service =
activeService
?: return NotificationActionResult(
ok = false,
code = "NOTIFICATIONS_UNAVAILABLE",
message = "NOTIFICATIONS_UNAVAILABLE: notification listener not connected",
)
return service.executeActionInternal(request)
}
private fun emitNotificationsChanged(payloadJson: String) {
runCatching {
nodeEventSink?.invoke(NOTIFICATIONS_CHANGED_EVENT, payloadJson)
}
}
private fun rememberRecentPackage(packageName: String?) {
val service = activeService ?: return
val normalized = packageName?.trim().orEmpty()
if (normalized.isEmpty() || normalized == service.packageName) return
migrateLegacyRecentPackagesIfNeeded(service.applicationContext)
val prefs = recentPackagesPrefs(service.applicationContext)
val existing =
prefs
.getString(recentPackagesPref, null)
.orEmpty()
.split(',')
.map { it.trim() }
.filter { it.isNotEmpty() && it != normalized }
.take(recentPackagesLimit - 1)
// Most recent package first keeps settings suggestions useful without storing notification content.
val updated = listOf(normalized) + existing
prefs.edit { putString(recentPackagesPref, updated.joinToString(",")) }
}
}
private fun executeActionInternal(request: NotificationActionRequest): NotificationActionResult {
val sbn =
activeNotifications
?.firstOrNull { it.key == request.key }
?: return NotificationActionResult(
ok = false,
code = "NOTIFICATION_NOT_FOUND",
message = "NOTIFICATION_NOT_FOUND: notification key not found",
)
if (actionRequiresClearableNotification(request.kind) && !sbn.isClearable) {
return NotificationActionResult(
ok = false,
code = "NOTIFICATION_NOT_CLEARABLE",
message = "NOTIFICATION_NOT_CLEARABLE: notification is ongoing or protected",
)
}
return when (request.kind) {
NotificationActionKind.Open -> {
val pendingIntent =
sbn.notification.contentIntent
?: return NotificationActionResult(
ok = false,
code = "ACTION_UNAVAILABLE",
message = "ACTION_UNAVAILABLE: notification has no open action",
)
runCatching {
pendingIntent.send()
}.fold(
onSuccess = { NotificationActionResult(ok = true) },
onFailure = { err ->
NotificationActionResult(
ok = false,
code = "ACTION_FAILED",
message = "ACTION_FAILED: ${err.message ?: "open failed"}",
)
},
)
}
NotificationActionKind.Dismiss -> {
runCatching {
cancelNotification(sbn.key)
DeviceNotificationStore.remove(sbn.key)
}.fold(
onSuccess = { NotificationActionResult(ok = true) },
onFailure = { err ->
NotificationActionResult(
ok = false,
code = "ACTION_FAILED",
message = "ACTION_FAILED: ${err.message ?: "dismiss failed"}",
)
},
)
}
NotificationActionKind.Reply -> {
val replyText = request.replyText?.trim().orEmpty()
if (replyText.isEmpty()) {
return NotificationActionResult(
ok = false,
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: replyText required for reply action",
)
}
val action =
sbn.notification.actions
?.firstOrNull { candidate ->
// Android reply actions are identified by RemoteInput, not by a stable action title.
candidate.actionIntent != null && !candidate.remoteInputs.isNullOrEmpty()
}
?: return NotificationActionResult(
ok = false,
code = "ACTION_UNAVAILABLE",
message = "ACTION_UNAVAILABLE: notification has no reply action",
)
val remoteInputs = action.remoteInputs ?: emptyArray()
val fillInIntent = Intent()
val replyBundle = android.os.Bundle()
for (remoteInput in remoteInputs) {
replyBundle.putCharSequence(remoteInput.resultKey, replyText)
}
RemoteInput.addResultsToIntent(remoteInputs, fillInIntent, replyBundle)
runCatching {
action.actionIntent.send(this, 0, fillInIntent)
}.fold(
onSuccess = { NotificationActionResult(ok = true) },
onFailure = { err ->
NotificationActionResult(
ok = false,
code = "ACTION_FAILED",
message = "ACTION_FAILED: ${err.message ?: "reply failed"}",
)
},
)
}
}
}
}

View File

@@ -0,0 +1,78 @@
package ai.openclaw.app.node
import ai.openclaw.app.SecurePrefs
import ai.openclaw.app.gateway.GatewaySession
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
/**
* Handles gateway-originated events that need to update local Android preferences.
*/
class GatewayEventHandler(
private val scope: CoroutineScope,
private val prefs: SecurePrefs,
private val json: Json,
private val operatorSession: GatewaySession,
private val isConnected: () -> Boolean,
) {
private var suppressWakeWordsSync = false
private var wakeWordsSyncJob: Job? = null
/** Applies gateway wake words locally without echoing the same change back to the gateway. */
fun applyWakeWordsFromGateway(words: List<String>) {
suppressWakeWordsSync = true
prefs.setWakeWords(words)
suppressWakeWordsSync = false
}
/** Debounces local wake-word edits before sending voicewake.set to the operator session. */
fun scheduleWakeWordsSyncIfNeeded() {
if (suppressWakeWordsSync) return
if (!isConnected()) return
val snapshot = prefs.wakeWords.value
wakeWordsSyncJob?.cancel()
wakeWordsSyncJob =
scope.launch {
delay(650)
val jsonList = snapshot.joinToString(separator = ",") { it.toJsonString() }
val params = """{"triggers":[$jsonList]}"""
try {
operatorSession.request("voicewake.set", params)
} catch (_: Throwable) {
// ignore
}
}
}
/** Loads gateway wake words on connect so Android settings show server truth. */
suspend fun refreshWakeWordsFromGateway() {
if (!isConnected()) return
try {
val res = operatorSession.request("voicewake.get", "{}")
val payload = json.parseToJsonElement(res).asObjectOrNull() ?: return
val array = payload["triggers"] as? JsonArray ?: return
val triggers = array.mapNotNull { it.asStringOrNull() }
applyWakeWordsFromGateway(triggers)
} catch (_: Throwable) {
// ignore
}
}
/** Applies voicewake.changed event payloads emitted by the gateway. */
fun handleVoiceWakeChangedEvent(payloadJson: String?) {
if (payloadJson.isNullOrBlank()) return
try {
val payload = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: return
val array = payload["triggers"] as? JsonArray ?: return
val triggers = array.mapNotNull { it.asStringOrNull() }
applyWakeWordsFromGateway(triggers)
} catch (_: Throwable) {
// ignore
}
}
}

View File

@@ -0,0 +1,295 @@
package ai.openclaw.app.node
import ai.openclaw.app.protocol.OpenClawCalendarCommand
import ai.openclaw.app.protocol.OpenClawCallLogCommand
import ai.openclaw.app.protocol.OpenClawCameraCommand
import ai.openclaw.app.protocol.OpenClawCanvasA2UICommand
import ai.openclaw.app.protocol.OpenClawCanvasCommand
import ai.openclaw.app.protocol.OpenClawCapability
import ai.openclaw.app.protocol.OpenClawContactsCommand
import ai.openclaw.app.protocol.OpenClawDeviceCommand
import ai.openclaw.app.protocol.OpenClawLocationCommand
import ai.openclaw.app.protocol.OpenClawMotionCommand
import ai.openclaw.app.protocol.OpenClawNotificationsCommand
import ai.openclaw.app.protocol.OpenClawPhotosCommand
import ai.openclaw.app.protocol.OpenClawSmsCommand
import ai.openclaw.app.protocol.OpenClawSystemCommand
import ai.openclaw.app.protocol.OpenClawTalkCommand
/** Runtime feature flags used to decide which node tools are advertised. */
data class NodeRuntimeFlags(
val cameraEnabled: Boolean,
val locationEnabled: Boolean,
val sendSmsAvailable: Boolean,
val readSmsAvailable: Boolean,
val smsSearchPossible: Boolean,
val callLogAvailable: Boolean,
val photosAvailable: Boolean,
val voiceWakeEnabled: Boolean,
val motionActivityAvailable: Boolean,
val motionPedometerAvailable: Boolean,
val installedAppsSharingEnabled: Boolean,
val debugBuild: Boolean,
)
/** Per-command availability gates checked before advertising invoke methods. */
enum class InvokeCommandAvailability {
Always,
CameraEnabled,
LocationEnabled,
SendSmsAvailable,
ReadSmsAvailable,
RequestableSmsSearchAvailable,
CallLogAvailable,
PhotosAvailable,
MotionActivityAvailable,
MotionPedometerAvailable,
InstalledAppsSharingEnabled,
DebugBuild,
}
/** Per-capability availability gates for the node capabilities manifest. */
enum class NodeCapabilityAvailability {
Always,
CameraEnabled,
LocationEnabled,
SmsAvailable,
CallLogAvailable,
PhotosAvailable,
VoiceWakeEnabled,
MotionAvailable,
}
/** Capability entry reported to the gateway when its availability gate passes. */
data class NodeCapabilitySpec(
val name: String,
val availability: NodeCapabilityAvailability = NodeCapabilityAvailability.Always,
)
/** Invoke method entry advertised to gateway plus foreground routing metadata. */
data class InvokeCommandSpec(
val name: String,
val requiresForeground: Boolean = false,
val availability: InvokeCommandAvailability = InvokeCommandAvailability.Always,
)
object InvokeCommandRegistry {
/** Capabilities mirror gateway protocol ids and are filtered by device state. */
val capabilityManifest: List<NodeCapabilitySpec> =
listOf(
NodeCapabilitySpec(name = OpenClawCapability.Canvas.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.Device.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.Notifications.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.System.rawValue),
NodeCapabilitySpec(
name = OpenClawCapability.Camera.rawValue,
availability = NodeCapabilityAvailability.CameraEnabled,
),
NodeCapabilitySpec(
name = OpenClawCapability.Sms.rawValue,
availability = NodeCapabilityAvailability.SmsAvailable,
),
NodeCapabilitySpec(
name = OpenClawCapability.VoiceWake.rawValue,
availability = NodeCapabilityAvailability.VoiceWakeEnabled,
),
NodeCapabilitySpec(name = OpenClawCapability.Talk.rawValue),
NodeCapabilitySpec(
name = OpenClawCapability.Location.rawValue,
availability = NodeCapabilityAvailability.LocationEnabled,
),
NodeCapabilitySpec(
name = OpenClawCapability.Photos.rawValue,
availability = NodeCapabilityAvailability.PhotosAvailable,
),
NodeCapabilitySpec(name = OpenClawCapability.Contacts.rawValue),
NodeCapabilitySpec(name = OpenClawCapability.Calendar.rawValue),
NodeCapabilitySpec(
name = OpenClawCapability.Motion.rawValue,
availability = NodeCapabilityAvailability.MotionAvailable,
),
NodeCapabilitySpec(
name = OpenClawCapability.CallLog.rawValue,
availability = NodeCapabilityAvailability.CallLogAvailable,
),
)
/** Complete Android node command catalog before runtime availability filtering. */
val all: List<InvokeCommandSpec> =
listOf(
InvokeCommandSpec(
name = OpenClawCanvasCommand.Present.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Hide.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Navigate.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Eval.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasCommand.Snapshot.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasA2UICommand.Push.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasA2UICommand.PushJSONL.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCanvasA2UICommand.Reset.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawSystemCommand.Notify.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttStart.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttStop.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttCancel.rawValue,
),
InvokeCommandSpec(
name = OpenClawTalkCommand.PttOnce.rawValue,
requiresForeground = true,
),
InvokeCommandSpec(
name = OpenClawCameraCommand.List.rawValue,
requiresForeground = true,
availability = InvokeCommandAvailability.CameraEnabled,
),
InvokeCommandSpec(
name = OpenClawCameraCommand.Snap.rawValue,
requiresForeground = true,
availability = InvokeCommandAvailability.CameraEnabled,
),
InvokeCommandSpec(
name = OpenClawCameraCommand.Clip.rawValue,
requiresForeground = true,
availability = InvokeCommandAvailability.CameraEnabled,
),
InvokeCommandSpec(
name = OpenClawLocationCommand.Get.rawValue,
availability = InvokeCommandAvailability.LocationEnabled,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Status.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Info.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Permissions.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Health.rawValue,
),
InvokeCommandSpec(
name = OpenClawDeviceCommand.Apps.rawValue,
availability = InvokeCommandAvailability.InstalledAppsSharingEnabled,
),
InvokeCommandSpec(
name = OpenClawNotificationsCommand.List.rawValue,
),
InvokeCommandSpec(
name = OpenClawNotificationsCommand.Actions.rawValue,
),
InvokeCommandSpec(
name = OpenClawPhotosCommand.Latest.rawValue,
availability = InvokeCommandAvailability.PhotosAvailable,
),
InvokeCommandSpec(
name = OpenClawContactsCommand.Search.rawValue,
),
InvokeCommandSpec(
name = OpenClawContactsCommand.Add.rawValue,
),
InvokeCommandSpec(
name = OpenClawCalendarCommand.Events.rawValue,
),
InvokeCommandSpec(
name = OpenClawCalendarCommand.Add.rawValue,
),
InvokeCommandSpec(
name = OpenClawMotionCommand.Activity.rawValue,
availability = InvokeCommandAvailability.MotionActivityAvailable,
),
InvokeCommandSpec(
name = OpenClawMotionCommand.Pedometer.rawValue,
availability = InvokeCommandAvailability.MotionPedometerAvailable,
),
InvokeCommandSpec(
name = OpenClawSmsCommand.Send.rawValue,
availability = InvokeCommandAvailability.SendSmsAvailable,
),
InvokeCommandSpec(
name = OpenClawSmsCommand.Search.rawValue,
availability = InvokeCommandAvailability.RequestableSmsSearchAvailable,
),
InvokeCommandSpec(
name = OpenClawCallLogCommand.Search.rawValue,
availability = InvokeCommandAvailability.CallLogAvailable,
),
InvokeCommandSpec(
name = "debug.logs",
availability = InvokeCommandAvailability.DebugBuild,
),
InvokeCommandSpec(
name = "debug.ed25519",
availability = InvokeCommandAvailability.DebugBuild,
),
)
private val byNameInternal: Map<String, InvokeCommandSpec> = all.associateBy { it.name }
/** Finds the command metadata used by dispatch and advertised-method builders. */
fun find(command: String): InvokeCommandSpec? = byNameInternal[command]
/** Returns gateway capability ids the current Android device can actually serve. */
fun advertisedCapabilities(flags: NodeRuntimeFlags): List<String> =
capabilityManifest
.filter { spec ->
when (spec.availability) {
NodeCapabilityAvailability.Always -> true
NodeCapabilityAvailability.CameraEnabled -> flags.cameraEnabled
NodeCapabilityAvailability.LocationEnabled -> flags.locationEnabled
NodeCapabilityAvailability.SmsAvailable -> flags.sendSmsAvailable || flags.readSmsAvailable
NodeCapabilityAvailability.CallLogAvailable -> flags.callLogAvailable
NodeCapabilityAvailability.PhotosAvailable -> flags.photosAvailable
NodeCapabilityAvailability.VoiceWakeEnabled -> flags.voiceWakeEnabled
NodeCapabilityAvailability.MotionAvailable -> flags.motionActivityAvailable || flags.motionPedometerAvailable
}
}.map { it.name }
/** Returns gateway invoke method ids available under current permissions/build flags. */
fun advertisedCommands(flags: NodeRuntimeFlags): List<String> =
all
.filter { spec ->
when (spec.availability) {
InvokeCommandAvailability.Always -> true
InvokeCommandAvailability.CameraEnabled -> flags.cameraEnabled
InvokeCommandAvailability.LocationEnabled -> flags.locationEnabled
InvokeCommandAvailability.SendSmsAvailable -> flags.sendSmsAvailable
InvokeCommandAvailability.ReadSmsAvailable -> flags.readSmsAvailable
InvokeCommandAvailability.RequestableSmsSearchAvailable -> flags.smsSearchPossible
InvokeCommandAvailability.CallLogAvailable -> flags.callLogAvailable
InvokeCommandAvailability.PhotosAvailable -> flags.photosAvailable
InvokeCommandAvailability.MotionActivityAvailable -> flags.motionActivityAvailable
InvokeCommandAvailability.MotionPedometerAvailable -> flags.motionPedometerAvailable
InvokeCommandAvailability.InstalledAppsSharingEnabled -> flags.installedAppsSharingEnabled
InvokeCommandAvailability.DebugBuild -> flags.debugBuild
}
}.map { it.name }
}

View File

@@ -0,0 +1,375 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import ai.openclaw.app.protocol.OpenClawCalendarCommand
import ai.openclaw.app.protocol.OpenClawCallLogCommand
import ai.openclaw.app.protocol.OpenClawCameraCommand
import ai.openclaw.app.protocol.OpenClawCanvasA2UICommand
import ai.openclaw.app.protocol.OpenClawCanvasCommand
import ai.openclaw.app.protocol.OpenClawContactsCommand
import ai.openclaw.app.protocol.OpenClawDeviceCommand
import ai.openclaw.app.protocol.OpenClawLocationCommand
import ai.openclaw.app.protocol.OpenClawMotionCommand
import ai.openclaw.app.protocol.OpenClawNotificationsCommand
import ai.openclaw.app.protocol.OpenClawSmsCommand
import ai.openclaw.app.protocol.OpenClawSystemCommand
import ai.openclaw.app.protocol.OpenClawTalkCommand
/** Runtime state for SMS search, split so permission prompts are not reported as hard unavailability. */
internal enum class SmsSearchAvailabilityReason {
Available,
PermissionRequired,
Unavailable,
}
/**
* Distinguish permanent SMS search unavailability from permission-gated search.
*/
internal fun classifySmsSearchAvailability(
readSmsAvailable: Boolean,
smsFeatureEnabled: Boolean,
smsTelephonyAvailable: Boolean,
): SmsSearchAvailabilityReason {
if (readSmsAvailable) return SmsSearchAvailabilityReason.Available
if (!smsFeatureEnabled || !smsTelephonyAvailable) return SmsSearchAvailabilityReason.Unavailable
return SmsSearchAvailabilityReason.PermissionRequired
}
internal fun smsSearchAvailabilityError(
readSmsAvailable: Boolean,
smsFeatureEnabled: Boolean,
smsTelephonyAvailable: Boolean,
): GatewaySession.InvokeResult? =
when (
classifySmsSearchAvailability(
readSmsAvailable = readSmsAvailable,
smsFeatureEnabled = smsFeatureEnabled,
smsTelephonyAvailable = smsTelephonyAvailable,
)
) {
SmsSearchAvailabilityReason.Available,
SmsSearchAvailabilityReason.PermissionRequired,
-> null
SmsSearchAvailabilityReason.Unavailable ->
GatewaySession.InvokeResult.error(
code = "SMS_UNAVAILABLE",
message = "SMS_UNAVAILABLE: SMS not available on this device",
)
}
/**
* Gateway node.invoke command router for Android-owned capabilities.
*/
class InvokeDispatcher(
private val canvas: CanvasController,
private val cameraHandler: CameraHandler,
private val locationHandler: LocationHandler,
private val deviceHandler: DeviceHandler,
private val notificationsHandler: NotificationsHandler,
private val systemHandler: SystemHandler,
private val talkHandler: TalkHandler,
private val photosHandler: PhotosHandler,
private val contactsHandler: ContactsHandler,
private val calendarHandler: CalendarHandler,
private val motionHandler: MotionHandler,
private val smsHandler: SmsHandler,
private val a2uiHandler: A2UIHandler,
private val debugHandler: DebugHandler,
private val callLogHandler: CallLogHandler,
private val isForeground: () -> Boolean,
private val cameraEnabled: () -> Boolean,
private val locationEnabled: () -> Boolean,
private val sendSmsAvailable: () -> Boolean,
private val readSmsAvailable: () -> Boolean,
private val smsFeatureEnabled: () -> Boolean,
private val smsTelephonyAvailable: () -> Boolean,
private val callLogAvailable: () -> Boolean,
private val photosAvailable: () -> Boolean,
private val installedAppsSharingEnabled: () -> Boolean,
private val debugBuild: () -> Boolean,
private val onCanvasA2uiPush: () -> Unit,
private val onCanvasA2uiReset: () -> Unit,
private val motionActivityAvailable: () -> Boolean,
private val motionPedometerAvailable: () -> Boolean,
) {
/** Dispatches one gateway node.invoke command after foreground and availability gates pass. */
suspend fun handleInvoke(
command: String,
paramsJson: String?,
): GatewaySession.InvokeResult {
val spec =
InvokeCommandRegistry.find(command)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: unknown command",
)
if (spec.requiresForeground && !isForeground()) {
// Foreground-only commands need an active Activity surface before touching UI or capture APIs.
return GatewaySession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: command requires foreground",
)
}
availabilityError(spec.availability)?.let { return it }
// Command strings come from OpenClawProtocolConstants; the registry above owns advertised availability.
return when (command) {
// Canvas commands
OpenClawCanvasCommand.Present.rawValue -> {
val url = CanvasController.parseNavigateUrl(paramsJson)
canvas.navigate(url)
GatewaySession.InvokeResult.ok(null)
}
OpenClawCanvasCommand.Hide.rawValue -> GatewaySession.InvokeResult.ok(null)
OpenClawCanvasCommand.Navigate.rawValue -> {
val url = CanvasController.parseNavigateUrl(paramsJson)
canvas.navigate(url)
GatewaySession.InvokeResult.ok(null)
}
OpenClawCanvasCommand.Eval.rawValue -> {
val js =
CanvasController.parseEvalJs(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: javaScript required",
)
withCanvasAvailable {
val result = canvas.eval(js)
GatewaySession.InvokeResult.ok("""{"result":${result.toJsonString()}}""")
}
}
OpenClawCanvasCommand.Snapshot.rawValue -> {
val snapshotParams = CanvasController.parseSnapshotParams(paramsJson)
withCanvasAvailable {
val base64 =
canvas.snapshotBase64(
format = snapshotParams.format,
quality = snapshotParams.quality,
maxWidth = snapshotParams.maxWidth,
)
GatewaySession.InvokeResult.ok("""{"format":"${snapshotParams.format.rawValue}","base64":"$base64"}""")
}
}
// A2UI commands
OpenClawCanvasA2UICommand.Reset.rawValue ->
withReadyA2ui {
withCanvasAvailable {
val res = canvas.eval(A2UIHandler.a2uiResetJS)
onCanvasA2uiReset()
GatewaySession.InvokeResult.ok(res)
}
}
OpenClawCanvasA2UICommand.Push.rawValue, OpenClawCanvasA2UICommand.PushJSONL.rawValue -> {
val messages =
try {
a2uiHandler.decodeA2uiMessages(command, paramsJson)
} catch (err: Throwable) {
return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = err.message ?: "invalid A2UI payload",
)
}
withReadyA2ui {
withCanvasAvailable {
val js = A2UIHandler.a2uiApplyMessagesJS(messages)
val res = canvas.eval(js)
onCanvasA2uiPush()
GatewaySession.InvokeResult.ok(res)
}
}
}
// Camera commands
OpenClawCameraCommand.List.rawValue -> cameraHandler.handleList(paramsJson)
OpenClawCameraCommand.Snap.rawValue -> cameraHandler.handleSnap(paramsJson)
OpenClawCameraCommand.Clip.rawValue -> cameraHandler.handleClip(paramsJson)
// Location command
OpenClawLocationCommand.Get.rawValue -> locationHandler.handleLocationGet(paramsJson)
// Device commands
OpenClawDeviceCommand.Status.rawValue -> deviceHandler.handleDeviceStatus(paramsJson)
OpenClawDeviceCommand.Info.rawValue -> deviceHandler.handleDeviceInfo(paramsJson)
OpenClawDeviceCommand.Permissions.rawValue -> deviceHandler.handleDevicePermissions(paramsJson)
OpenClawDeviceCommand.Health.rawValue -> deviceHandler.handleDeviceHealth(paramsJson)
OpenClawDeviceCommand.Apps.rawValue -> deviceHandler.handleDeviceApps(paramsJson)
// Notifications command
OpenClawNotificationsCommand.List.rawValue -> notificationsHandler.handleNotificationsList(paramsJson)
OpenClawNotificationsCommand.Actions.rawValue -> notificationsHandler.handleNotificationsActions(paramsJson)
// System command
OpenClawSystemCommand.Notify.rawValue -> systemHandler.handleSystemNotify(paramsJson)
// Talk commands
OpenClawTalkCommand.PttStart.rawValue -> talkHandler.handlePttStart(paramsJson)
OpenClawTalkCommand.PttStop.rawValue -> talkHandler.handlePttStop(paramsJson)
OpenClawTalkCommand.PttCancel.rawValue -> talkHandler.handlePttCancel(paramsJson)
OpenClawTalkCommand.PttOnce.rawValue -> talkHandler.handlePttOnce(paramsJson)
// Photos command
ai.openclaw.app.protocol.OpenClawPhotosCommand.Latest.rawValue ->
photosHandler.handlePhotosLatest(
paramsJson,
)
// Contacts command
OpenClawContactsCommand.Search.rawValue -> contactsHandler.handleContactsSearch(paramsJson)
OpenClawContactsCommand.Add.rawValue -> contactsHandler.handleContactsAdd(paramsJson)
// Calendar command
OpenClawCalendarCommand.Events.rawValue -> calendarHandler.handleCalendarEvents(paramsJson)
OpenClawCalendarCommand.Add.rawValue -> calendarHandler.handleCalendarAdd(paramsJson)
// Motion command
OpenClawMotionCommand.Activity.rawValue -> motionHandler.handleMotionActivity(paramsJson)
OpenClawMotionCommand.Pedometer.rawValue -> motionHandler.handleMotionPedometer(paramsJson)
// SMS command
OpenClawSmsCommand.Send.rawValue -> smsHandler.handleSmsSend(paramsJson)
OpenClawSmsCommand.Search.rawValue -> smsHandler.handleSmsSearch(paramsJson)
// CallLog command
OpenClawCallLogCommand.Search.rawValue -> callLogHandler.handleCallLogSearch(paramsJson)
// Debug commands
"debug.ed25519" -> debugHandler.handleEd25519()
"debug.logs" -> debugHandler.handleLogs()
else -> GatewaySession.InvokeResult.error(code = "INVALID_REQUEST", message = "INVALID_REQUEST: unknown command")
}
}
private suspend fun withReadyA2ui(block: suspend () -> GatewaySession.InvokeResult): GatewaySession.InvokeResult {
if (!a2uiHandler.ensureA2uiReady()) {
return GatewaySession.InvokeResult.error(
code = "A2UI_HOST_UNAVAILABLE",
message = "A2UI_HOST_UNAVAILABLE: bundled A2UI host not reachable",
)
}
return block()
}
private suspend fun withCanvasAvailable(block: suspend () -> GatewaySession.InvokeResult): GatewaySession.InvokeResult =
try {
block()
} catch (_: Throwable) {
// WebView calls throw when the Activity is backgrounded between the foreground check and execution.
GatewaySession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable",
)
}
private fun availabilityError(availability: InvokeCommandAvailability): GatewaySession.InvokeResult? =
when (availability) {
InvokeCommandAvailability.Always -> null
InvokeCommandAvailability.CameraEnabled ->
if (cameraEnabled()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "CAMERA_DISABLED",
message = "CAMERA_DISABLED: enable Camera in Settings",
)
}
InvokeCommandAvailability.LocationEnabled ->
if (locationEnabled()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "LOCATION_DISABLED",
message = "LOCATION_DISABLED: enable Location in Settings",
)
}
InvokeCommandAvailability.MotionActivityAvailable ->
if (motionActivityAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "MOTION_UNAVAILABLE",
message = "MOTION_UNAVAILABLE: accelerometer not available",
)
}
InvokeCommandAvailability.MotionPedometerAvailable ->
if (motionPedometerAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "PEDOMETER_UNAVAILABLE",
message = "PEDOMETER_UNAVAILABLE: step counter not available",
)
}
InvokeCommandAvailability.SendSmsAvailable ->
if (sendSmsAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "SMS_UNAVAILABLE",
message = "SMS_UNAVAILABLE: SMS not available on this device",
)
}
InvokeCommandAvailability.ReadSmsAvailable,
InvokeCommandAvailability.RequestableSmsSearchAvailable,
->
// SMS search may still be advertised as promptable; runtime invoke fails only on permanent unavailability.
smsSearchAvailabilityError(
readSmsAvailable = readSmsAvailable(),
smsFeatureEnabled = smsFeatureEnabled(),
smsTelephonyAvailable = smsTelephonyAvailable(),
)
InvokeCommandAvailability.CallLogAvailable ->
if (callLogAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "CALL_LOG_UNAVAILABLE",
message = "CALL_LOG_UNAVAILABLE: call log not available on this build",
)
}
InvokeCommandAvailability.PhotosAvailable ->
if (photosAvailable()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "PHOTOS_UNAVAILABLE",
message = "PHOTOS_UNAVAILABLE: photos not available on this build",
)
}
InvokeCommandAvailability.InstalledAppsSharingEnabled ->
if (installedAppsSharingEnabled()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "INSTALLED_APPS_SHARING_DISABLED",
message = "INSTALLED_APPS_SHARING_DISABLED: enable Installed Apps in Settings",
)
}
InvokeCommandAvailability.DebugBuild ->
if (debugBuild()) {
null
} else {
GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: unknown command",
)
}
}
}
/**
* Talk-mode command adapter implemented by the voice subsystem.
*/
interface TalkHandler {
/** Starts a push-to-talk capture session and keeps it open until stop or cancel. */
suspend fun handlePttStart(paramsJson: String?): GatewaySession.InvokeResult
/** Finishes the active push-to-talk capture and submits recognized speech. */
suspend fun handlePttStop(paramsJson: String?): GatewaySession.InvokeResult
/** Aborts the active push-to-talk capture without submitting speech. */
suspend fun handlePttCancel(paramsJson: String?): GatewaySession.InvokeResult
/** Runs a bounded one-shot push-to-talk capture. */
suspend fun handlePttOnce(paramsJson: String?): GatewaySession.InvokeResult
}

View File

@@ -0,0 +1,70 @@
package ai.openclaw.app.node
import kotlin.math.max
import kotlin.math.min
import kotlin.math.roundToInt
/**
* Result of a JPEG compression attempt after quality and scale reductions.
*/
internal data class JpegSizeLimiterResult(
val bytes: ByteArray,
val width: Int,
val height: Int,
val quality: Int,
)
/**
* Utility that searches quality/scale combinations until a JPEG fits a byte budget.
*/
internal object JpegSizeLimiter {
/** Compresses with the caller-provided encoder, reducing quality before image dimensions. */
fun compressToLimit(
initialWidth: Int,
initialHeight: Int,
startQuality: Int,
maxBytes: Int,
minQuality: Int = 20,
minSize: Int = 256,
scaleStep: Double = 0.85,
maxScaleAttempts: Int = 6,
maxQualityAttempts: Int = 6,
encode: (width: Int, height: Int, quality: Int) -> ByteArray,
): JpegSizeLimiterResult {
require(initialWidth > 0 && initialHeight > 0) { "Invalid image size" }
require(maxBytes > 0) { "Invalid maxBytes" }
val clampedStartQuality = startQuality.coerceIn(minQuality, 100)
var width = initialWidth
var height = initialHeight
var best: JpegSizeLimiterResult? = null
repeat(maxScaleAttempts + 1) { scaleAttempt ->
var quality = clampedStartQuality
repeat(maxQualityAttempts) {
val bytes = encode(width, height, quality)
val attempt = JpegSizeLimiterResult(bytes = bytes, width = width, height = height, quality = quality)
best = attempt
if (bytes.size <= maxBytes) return best
if (quality <= minQuality) return@repeat
quality = max(minQuality, (quality * 0.75).roundToInt())
}
if (scaleAttempt == maxScaleAttempts) return@repeat
val minScale = (minSize.toDouble() / min(width, height).toDouble()).coerceAtMost(1.0)
val nextScale = max(scaleStep, minScale)
val nextWidth = max(minSize, (width * nextScale).roundToInt())
val nextHeight = max(minSize, (height * nextScale).roundToInt())
if (nextWidth == width && nextHeight == height) return@repeat
width = min(nextWidth, width)
height = min(nextHeight, height)
}
val failed = checkNotNull(best)
if (failed.bytes.size > maxBytes) {
throw IllegalStateException("CAMERA_TOO_LARGE: ${failed.bytes.size} bytes > $maxBytes bytes")
}
return failed
}
}

View File

@@ -0,0 +1,123 @@
package ai.openclaw.app.node
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.location.Location
import android.location.LocationManager
import android.os.CancellationSignal
import androidx.core.content.ContextCompat
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.time.Instant
import java.time.format.DateTimeFormatter
/**
* Android LocationManager-backed capture used by gateway location commands.
*/
class LocationCaptureManager(
private val context: Context,
) {
data class Payload(
val payloadJson: String,
)
suspend fun getLocation(
desiredProviders: List<String>,
maxAgeMs: Long?,
timeoutMs: Long,
isPrecise: Boolean,
): Payload =
withContext(Dispatchers.Main) {
val manager = context.getSystemService(Context.LOCATION_SERVICE) as LocationManager
if (!manager.isProviderEnabled(LocationManager.GPS_PROVIDER) &&
!manager.isProviderEnabled(LocationManager.NETWORK_PROVIDER)
) {
throw IllegalStateException("LOCATION_UNAVAILABLE: no location providers enabled")
}
// Prefer a recent cached fix before waking GPS/network providers.
val cached = bestLastKnown(manager, desiredProviders, maxAgeMs)
val location =
cached ?: requestCurrent(manager, desiredProviders, timeoutMs)
val timestamp = DateTimeFormatter.ISO_INSTANT.format(Instant.ofEpochMilli(location.time))
val source = location.provider
val altitudeMeters = if (location.hasAltitude()) location.altitude else null
val speedMps = if (location.hasSpeed()) location.speed.toDouble() else null
val headingDeg = if (location.hasBearing()) location.bearing.toDouble() else null
Payload(
buildString {
append("{\"lat\":")
append(location.latitude)
append(",\"lon\":")
append(location.longitude)
append(",\"accuracyMeters\":")
append(location.accuracy.toDouble())
if (altitudeMeters != null) append(",\"altitudeMeters\":").append(altitudeMeters)
if (speedMps != null) append(",\"speedMps\":").append(speedMps)
if (headingDeg != null) append(",\"headingDeg\":").append(headingDeg)
append(",\"timestamp\":\"").append(timestamp).append('"')
append(",\"isPrecise\":").append(isPrecise)
append(",\"source\":\"").append(source).append('"')
append('}')
},
)
}
private fun bestLastKnown(
manager: LocationManager,
providers: List<String>,
maxAgeMs: Long?,
): Location? {
val fineOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_FINE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
val coarseOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_COARSE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
if (!fineOk && !coarseOk) {
throw IllegalStateException("LOCATION_PERMISSION_REQUIRED: grant Location permission")
}
val now = System.currentTimeMillis()
val candidates =
providers.mapNotNull { provider -> manager.getLastKnownLocation(provider) }
val freshest = candidates.maxByOrNull { it.time } ?: return null
// maxAgeMs is a caller contract; stale cached fixes force a live provider request.
if (maxAgeMs != null && now - freshest.time > maxAgeMs) return null
return freshest
}
private suspend fun requestCurrent(
manager: LocationManager,
providers: List<String>,
timeoutMs: Long,
): Location {
val fineOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_FINE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
val coarseOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_COARSE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
if (!fineOk && !coarseOk) {
throw IllegalStateException("LOCATION_PERMISSION_REQUIRED: grant Location permission")
}
val resolved =
providers.firstOrNull { manager.isProviderEnabled(it) }
?: throw IllegalStateException("LOCATION_UNAVAILABLE: no providers available")
// getCurrentLocation can return null; the handler maps timeout/null fixes to gateway error shapes.
val location =
withTimeout(timeoutMs.coerceAtLeast(1)) {
suspendCancellableCoroutine<Location?> { cont ->
val signal = CancellationSignal()
cont.invokeOnCancellation { signal.cancel() }
manager.getCurrentLocation(resolved, signal, context.mainExecutor) { location ->
cont.resume(location) { _, _, _ -> }
}
}
}
return location ?: throw IllegalStateException("LOCATION_UNAVAILABLE: no fix")
}
}

View File

@@ -0,0 +1,170 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.location.LocationManager
import androidx.core.content.ContextCompat
import kotlinx.coroutines.TimeoutCancellationException
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
/**
* Injectable location facade for command tests and Android runtime access.
*/
internal interface LocationDataSource {
fun hasFinePermission(context: Context): Boolean
fun hasCoarsePermission(context: Context): Boolean
suspend fun fetchLocation(
desiredProviders: List<String>,
maxAgeMs: Long?,
timeoutMs: Long,
isPrecise: Boolean,
): LocationCaptureManager.Payload
}
private class DefaultLocationDataSource(
private val capture: LocationCaptureManager,
) : LocationDataSource {
override fun hasFinePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_FINE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
override fun hasCoarsePermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACCESS_COARSE_LOCATION) ==
PackageManager.PERMISSION_GRANTED
override suspend fun fetchLocation(
desiredProviders: List<String>,
maxAgeMs: Long?,
timeoutMs: Long,
isPrecise: Boolean,
): LocationCaptureManager.Payload =
capture.getLocation(
desiredProviders = desiredProviders,
maxAgeMs = maxAgeMs,
timeoutMs = timeoutMs,
isPrecise = isPrecise,
)
}
class LocationHandler private constructor(
private val appContext: Context,
private val dataSource: LocationDataSource,
private val json: Json,
private val isForeground: () -> Boolean,
private val locationPreciseEnabled: () -> Boolean,
) {
constructor(
appContext: Context,
location: LocationCaptureManager,
json: Json,
isForeground: () -> Boolean,
locationPreciseEnabled: () -> Boolean,
) : this(
appContext = appContext,
dataSource = DefaultLocationDataSource(location),
json = json,
isForeground = isForeground,
locationPreciseEnabled = locationPreciseEnabled,
)
/** Reports whether precise GPS-backed location can be requested from Android. */
fun hasFineLocationPermission(): Boolean = dataSource.hasFinePermission(appContext)
/** Reports whether network/coarse location can be requested from Android. */
fun hasCoarseLocationPermission(): Boolean = dataSource.hasCoarsePermission(appContext)
companion object {
/** Creates a handler with injected location state for permission and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: LocationDataSource,
json: Json = Json { ignoreUnknownKeys = true },
isForeground: () -> Boolean = { true },
locationPreciseEnabled: () -> Boolean = { true },
): LocationHandler =
LocationHandler(
appContext = appContext,
dataSource = dataSource,
json = json,
isForeground = isForeground,
locationPreciseEnabled = locationPreciseEnabled,
)
}
/** Handles location.get with foreground, permission, and user precision gates applied. */
suspend fun handleLocationGet(paramsJson: String?): GatewaySession.InvokeResult {
if (!isForeground()) {
// Android foreground restrictions and user expectation keep live location tied to the visible app.
return GatewaySession.InvokeResult.error(
code = "LOCATION_BACKGROUND_UNAVAILABLE",
message = "LOCATION_BACKGROUND_UNAVAILABLE: location requires OpenClaw to stay open",
)
}
if (!dataSource.hasFinePermission(appContext) && !dataSource.hasCoarsePermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "LOCATION_PERMISSION_REQUIRED",
message = "LOCATION_PERMISSION_REQUIRED: grant Location permission",
)
}
val (maxAgeMs, timeoutMs, desiredAccuracy) = parseLocationParams(paramsJson)
val preciseEnabled = locationPreciseEnabled()
// Gateway requests are advisory; Android permission and user settings decide
// whether precise capture is actually allowed for this invocation.
val accuracy =
when (desiredAccuracy) {
"precise" -> if (preciseEnabled && dataSource.hasFinePermission(appContext)) "precise" else "balanced"
"coarse" -> "coarse"
else -> if (preciseEnabled && dataSource.hasFinePermission(appContext)) "precise" else "balanced"
}
val providers =
when (accuracy) {
// Provider order is part of the accuracy policy: GPS first for precise, network first otherwise.
"precise" -> listOf(LocationManager.GPS_PROVIDER, LocationManager.NETWORK_PROVIDER)
"coarse" -> listOf(LocationManager.NETWORK_PROVIDER, LocationManager.GPS_PROVIDER)
else -> listOf(LocationManager.NETWORK_PROVIDER, LocationManager.GPS_PROVIDER)
}
try {
val payload =
dataSource.fetchLocation(
desiredProviders = providers,
maxAgeMs = maxAgeMs,
timeoutMs = timeoutMs,
isPrecise = accuracy == "precise",
)
return GatewaySession.InvokeResult.ok(payload.payloadJson)
} catch (err: TimeoutCancellationException) {
return GatewaySession.InvokeResult.error(
code = "LOCATION_TIMEOUT",
message = "LOCATION_TIMEOUT: no fix in time",
)
} catch (err: Throwable) {
val message = err.message ?: "LOCATION_UNAVAILABLE: no fix"
return GatewaySession.InvokeResult.error(code = "LOCATION_UNAVAILABLE", message = message)
}
}
private fun parseLocationParams(paramsJson: String?): Triple<Long?, Long, String?> {
if (paramsJson.isNullOrBlank()) {
return Triple(null, 10_000L, null)
}
val root =
try {
json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
}
val maxAgeMs = (root?.get("maxAgeMs") as? JsonPrimitive)?.content?.toLongOrNull()
val timeoutMs =
(root?.get("timeoutMs") as? JsonPrimitive)?.content?.toLongOrNull()?.coerceIn(1_000L, 60_000L)
?: 10_000L
// desiredAccuracy is advisory; invalid values fall through to the default policy.
val desiredAccuracy =
(root?.get("desiredAccuracy") as? JsonPrimitive)?.content?.trim()?.lowercase()
return Triple(maxAgeMs, timeoutMs, desiredAccuracy)
}
}

View File

@@ -0,0 +1,418 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.content.Context
import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.hardware.SensorManager
import android.os.SystemClock
import androidx.core.content.ContextCompat
import kotlinx.coroutines.InternalCoroutinesApi
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withTimeoutOrNull
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.time.Instant
import kotlin.math.abs
import kotlin.math.max
import kotlin.math.sqrt
private const val ACCELEROMETER_SAMPLE_TARGET = 20
private const val ACCELEROMETER_SAMPLE_TIMEOUT_MS = 6_000L
/** Gateway request for motion.activity after parsing and limit bounds. */
internal data class MotionActivityRequest(
val startISO: String?,
val endISO: String?,
val limit: Int,
)
/** Gateway request for motion.pedometer. */
internal data class MotionPedometerRequest(
val startISO: String?,
val endISO: String?,
)
/** Motion activity sample returned in gateway-compatible boolean flags. */
internal data class MotionActivityRecord(
val startISO: String,
val endISO: String,
val confidence: String,
val isWalking: Boolean,
val isRunning: Boolean,
val isCycling: Boolean,
val isAutomotive: Boolean,
val isStationary: Boolean,
val isUnknown: Boolean,
)
/** Pedometer sample returned from Android's cumulative step counter. */
internal data class PedometerRecord(
val startISO: String,
val endISO: String,
val steps: Int?,
val distanceMeters: Double?,
val floorsAscended: Int?,
val floorsDescended: Int?,
)
/** Motion data seam for Android sensors and tests. */
internal interface MotionDataSource {
fun isActivityAvailable(context: Context): Boolean
fun isPedometerAvailable(context: Context): Boolean
fun isAvailable(context: Context): Boolean = isActivityAvailable(context) || isPedometerAvailable(context)
fun hasPermission(context: Context): Boolean
suspend fun activity(
context: Context,
request: MotionActivityRequest,
): MotionActivityRecord
suspend fun pedometer(
context: Context,
request: MotionPedometerRequest,
): PedometerRecord
}
private object SystemMotionDataSource : MotionDataSource {
override fun isActivityAvailable(context: Context): Boolean {
val sensorManager = context.getSystemService(SensorManager::class.java)
return sensorManager?.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null
}
override fun isPedometerAvailable(context: Context): Boolean {
val sensorManager = context.getSystemService(SensorManager::class.java)
return sensorManager?.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) != null
}
override fun hasPermission(context: Context): Boolean =
ContextCompat.checkSelfPermission(context, Manifest.permission.ACTIVITY_RECOGNITION) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
override suspend fun activity(
context: Context,
request: MotionActivityRequest,
): MotionActivityRecord {
if (!request.startISO.isNullOrBlank() || !request.endISO.isNullOrBlank()) {
// Android does not expose historical activity samples here; fail with a
// stable gateway code instead of pretending the range is empty.
throw IllegalArgumentException("MOTION_RANGE_UNAVAILABLE: historical activity range not supported on Android")
}
val sensorManager =
context.getSystemService(SensorManager::class.java)
?: throw IllegalStateException("MOTION_UNAVAILABLE: sensor manager unavailable")
val accelerometer =
sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)
?: throw IllegalStateException("MOTION_UNAVAILABLE: accelerometer not available")
val sample =
readAccelerometerSample(sensorManager, accelerometer)
?: throw IllegalStateException("MOTION_UNAVAILABLE: no accelerometer sample")
val end = Instant.now()
val start = end.minusSeconds(2)
val classification = classifyActivity(sample.averageDelta)
return MotionActivityRecord(
startISO = start.toString(),
endISO = end.toString(),
confidence = classifyConfidence(sample.samples, sample.averageDelta),
isWalking = classification == "walking",
isRunning = classification == "running",
isCycling = false,
isAutomotive = false,
isStationary = classification == "stationary",
isUnknown = classification == "unknown",
)
}
override suspend fun pedometer(
context: Context,
request: MotionPedometerRequest,
): PedometerRecord {
if (!request.startISO.isNullOrBlank() || !request.endISO.isNullOrBlank()) {
// TYPE_STEP_COUNTER is cumulative since boot, not a historical query API.
throw IllegalArgumentException("PEDOMETER_RANGE_UNAVAILABLE: historical pedometer range not supported on Android")
}
val sensorManager =
context.getSystemService(SensorManager::class.java)
?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: sensor manager unavailable")
val stepCounter =
sensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER)
?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: step counting not supported")
val steps =
readStepCounter(sensorManager, stepCounter)
?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: no step counter sample")
val bootMs = System.currentTimeMillis() - SystemClock.elapsedRealtime()
return PedometerRecord(
startISO = Instant.ofEpochMilli(max(0L, bootMs)).toString(),
endISO = Instant.now().toString(),
steps = steps,
distanceMeters = null,
floorsAscended = null,
floorsDescended = null,
)
}
private data class AccelerometerSample(
val samples: Int,
val averageDelta: Double,
)
@OptIn(InternalCoroutinesApi::class)
private suspend fun readStepCounter(
sensorManager: SensorManager,
sensor: Sensor,
): Int? {
val sample =
withTimeoutOrNull(1200L) {
suspendCancellableCoroutine<Float?> { cont ->
val listener =
object : SensorEventListener {
override fun onSensorChanged(event: SensorEvent?) {
val value = event?.values?.firstOrNull()
val token = cont.tryResume(value) ?: return
cont.completeResume(token)
sensorManager.unregisterListener(this)
}
override fun onAccuracyChanged(
sensor: Sensor?,
accuracy: Int,
) = Unit
}
val registered = sensorManager.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_NORMAL)
if (!registered) {
sensorManager.unregisterListener(listener)
cont.resume(null) { _, _, _ -> }
return@suspendCancellableCoroutine
}
cont.invokeOnCancellation { sensorManager.unregisterListener(listener) }
}
}
return sample?.toInt()?.takeIf { it >= 0 }
}
@OptIn(InternalCoroutinesApi::class)
private suspend fun readAccelerometerSample(
sensorManager: SensorManager,
sensor: Sensor,
): AccelerometerSample? {
val sample =
withTimeoutOrNull(ACCELEROMETER_SAMPLE_TIMEOUT_MS) {
suspendCancellableCoroutine<AccelerometerSample?> { cont ->
var count = 0
var sumDelta = 0.0
val listener =
object : SensorEventListener {
override fun onSensorChanged(event: SensorEvent?) {
val values = event?.values ?: return
if (values.size < 3) return
val magnitude =
sqrt(
values[0] * values[0] +
values[1] * values[1] +
values[2] * values[2],
).toDouble()
sumDelta += abs(magnitude - SensorManager.GRAVITY_EARTH.toDouble())
count += 1
if (count >= ACCELEROMETER_SAMPLE_TARGET) {
// Average gravity-adjusted magnitude across a short window so
// one noisy sensor event cannot decide the activity label.
val result =
AccelerometerSample(
samples = count,
averageDelta = sumDelta / count,
)
val token = cont.tryResume(result) ?: return
cont.completeResume(token)
sensorManager.unregisterListener(this)
}
}
override fun onAccuracyChanged(
sensor: Sensor?,
accuracy: Int,
) = Unit
}
val registered = sensorManager.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_NORMAL)
if (!registered) {
cont.resume(null) { _, _, _ -> }
return@suspendCancellableCoroutine
}
cont.invokeOnCancellation { sensorManager.unregisterListener(listener) }
}
}
return sample
}
private fun classifyActivity(averageDelta: Double): String =
when {
averageDelta <= 0.55 -> "stationary"
averageDelta <= 1.80 -> "walking"
else -> "running"
}
private fun classifyConfidence(
samples: Int,
averageDelta: Double,
): String {
if (samples < 6) return "low"
if (samples >= 14 && averageDelta > 0.4) return "high"
return "medium"
}
}
/** Handles Android motion-related node.invoke commands backed by live sensors. */
class MotionHandler private constructor(
private val appContext: Context,
private val dataSource: MotionDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemMotionDataSource)
/** Classifies a short accelerometer sample into the gateway activity shape. */
suspend fun handleMotionActivity(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "MOTION_PERMISSION_REQUIRED",
message = "MOTION_PERMISSION_REQUIRED: grant Motion permission",
)
}
val request =
parseActivityRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val activity = dataSource.activity(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put(
"activities",
buildJsonArray {
add(
buildJsonObject {
put("startISO", JsonPrimitive(activity.startISO))
put("endISO", JsonPrimitive(activity.endISO))
put("confidence", JsonPrimitive(activity.confidence))
put("isWalking", JsonPrimitive(activity.isWalking))
put("isRunning", JsonPrimitive(activity.isRunning))
put("isCycling", JsonPrimitive(activity.isCycling))
put("isAutomotive", JsonPrimitive(activity.isAutomotive))
put("isStationary", JsonPrimitive(activity.isStationary))
put("isUnknown", JsonPrimitive(activity.isUnknown))
},
)
},
)
}.toString(),
)
} catch (err: IllegalArgumentException) {
GatewaySession.InvokeResult.error(code = "MOTION_UNAVAILABLE", message = err.message ?: "MOTION_UNAVAILABLE")
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "MOTION_UNAVAILABLE",
message = "MOTION_UNAVAILABLE: ${err.message ?: "motion activity failed"}",
)
}
}
/** Returns the current boot-scoped Android step-counter reading. */
suspend fun handleMotionPedometer(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "MOTION_PERMISSION_REQUIRED",
message = "MOTION_PERMISSION_REQUIRED: grant Motion permission",
)
}
val request =
parsePedometerRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val payload = dataSource.pedometer(appContext, request)
GatewaySession.InvokeResult.ok(
buildJsonObject {
put("startISO", JsonPrimitive(payload.startISO))
put("endISO", JsonPrimitive(payload.endISO))
payload.steps?.let { put("steps", JsonPrimitive(it)) }
payload.distanceMeters?.let { put("distanceMeters", JsonPrimitive(it)) }
payload.floorsAscended?.let { put("floorsAscended", JsonPrimitive(it)) }
payload.floorsDescended?.let { put("floorsDescended", JsonPrimitive(it)) }
}.toString(),
)
} catch (err: IllegalArgumentException) {
GatewaySession.InvokeResult.error(code = "MOTION_UNAVAILABLE", message = err.message ?: "MOTION_UNAVAILABLE")
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "MOTION_UNAVAILABLE",
message = "MOTION_UNAVAILABLE: ${err.message ?: "pedometer query failed"}",
)
}
}
fun isAvailable(): Boolean = dataSource.isAvailable(appContext)
/** Returns true when live accelerometer classification can be sampled. */
fun isActivityAvailable(): Boolean = dataSource.isActivityAvailable(appContext)
/** Returns true when Android exposes a cumulative step-counter sensor. */
fun isPedometerAvailable(): Boolean = dataSource.isPedometerAvailable(appContext)
private fun parseActivityRequest(paramsJson: String?): MotionActivityRequest? {
if (paramsJson.isNullOrBlank()) {
return MotionActivityRequest(startISO = null, endISO = null, limit = 200)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
// Keep the accepted gateway parameter even though Android can only return
// one live classification sample for now.
val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: 200).coerceIn(1, 1000)
return MotionActivityRequest(
startISO = (params["startISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
endISO = (params["endISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
limit = limit,
)
}
private fun parsePedometerRequest(paramsJson: String?): MotionPedometerRequest? {
if (paramsJson.isNullOrBlank()) {
return MotionPedometerRequest(startISO = null, endISO = null)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
return MotionPedometerRequest(
startISO = (params["startISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
endISO = (params["endISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null },
)
}
companion object {
/** Static capability probe used before a MotionHandler instance is needed. */
fun isMotionCapabilityAvailable(context: Context): Boolean = SystemMotionDataSource.isAvailable(context)
/** Creates a handler with an injected sensor source for parser and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: MotionDataSource,
): MotionHandler = MotionHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,114 @@
package ai.openclaw.app.node
import android.os.Build
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
internal object NodePresenceAliveBeacon {
/** Gateway event emitted by Android when background execution confirms liveness. */
const val EVENT_NAME: String = "node.presence.alive"
/** Avoids spamming presence when multiple background triggers fire together. */
const val MIN_SUCCESS_INTERVAL_MS: Long = 10 * 60 * 1000
private const val MAX_RESPONSE_JSON_CHARS: Int = 16 * 1024
/**
* Source of the liveness event, serialized as gateway-stable wire values.
*/
enum class Trigger(
val rawValue: String,
) {
Background("background"),
SilentPush("silent_push"),
BackgroundAppRefresh("bg_app_refresh"),
SignificantLocation("significant_location"),
Manual("manual"),
Connect("connect"),
}
/**
* Minimal gateway response fields used to decide whether a liveness event was accepted.
*/
data class ResponsePayload(
val ok: Boolean?,
val event: String?,
val handled: Boolean?,
val reason: String?,
)
private val json = Json { ignoreUnknownKeys = true }
/** Skips sends after a recent successful presence update. */
fun shouldSkipRecentSuccess(
nowMs: Long,
lastSuccessAtMs: Long?,
minIntervalMs: Long = MIN_SUCCESS_INTERVAL_MS,
): Boolean {
val last = lastSuccessAtMs ?: return false
if (last <= 0) return false
val elapsed = nowMs - last
return elapsed >= 0 && elapsed < minIntervalMs
}
/** Human-readable Android version label included in presence payloads. */
fun androidPlatformLabel(): String {
val release =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
.ifEmpty { "unknown" }
return "Android $release (SDK ${Build.VERSION.SDK_INT})"
}
/** Builds the compact JSON payload consumed by gateway node-presence handlers. */
fun makePayloadJson(
trigger: Trigger,
sentAtMs: Long,
displayName: String,
version: String,
platform: String,
deviceFamily: String?,
modelIdentifier: String?,
pushTransport: String? = null,
): String =
buildJsonObject {
put("trigger", JsonPrimitive(trigger.rawValue))
put("sentAtMs", JsonPrimitive(sentAtMs))
put("displayName", JsonPrimitive(displayName))
put("version", JsonPrimitive(version))
put("platform", JsonPrimitive(platform))
deviceFamily?.trim()?.takeIf { it.isNotEmpty() }?.let { put("deviceFamily", JsonPrimitive(it)) }
modelIdentifier?.trim()?.takeIf { it.isNotEmpty() }?.let { put("modelIdentifier", JsonPrimitive(it)) }
pushTransport?.trim()?.takeIf { it.isNotEmpty() }?.let { put("pushTransport", JsonPrimitive(it)) }
}.toString()
/** Parses the gateway response while rejecting empty, oversized, or malformed payloads. */
fun decodeResponse(payloadJson: String?): ResponsePayload? {
val raw = payloadJson?.trim()?.takeIf { it.isNotEmpty() } ?: return null
// Bound log/IPC responses before JSON parsing to avoid memory spikes from
// malformed gateway replies.
if (raw.length > MAX_RESPONSE_JSON_CHARS) return null
val obj =
try {
json.parseToJsonElement(raw).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
return ResponsePayload(
ok = parseJsonBooleanFlag(obj, "ok"),
event = parseJsonString(obj, "event"),
handled = parseJsonBooleanFlag(obj, "handled"),
reason = parseJsonString(obj, "reason"),
)
}
/** Sanitizes gateway response reasons before writing them into Android logs. */
fun sanitizeReasonForLog(raw: String?): String {
val value = raw?.trim()?.takeIf { it.isNotEmpty() } ?: "unsupported"
return value
.map { ch -> if (ch.isISOControl()) ' ' else ch }
.joinToString("")
.take(200)
}
}

View File

@@ -0,0 +1,111 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.parseInvokeErrorFromThrowable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
/** Default canvas seam color used when gateway/user params omit a hex color. */
const val DEFAULT_SEAM_COLOR_ARGB: Long = 0xFF4F7A9A
/** Small tuple used by Android node handlers that need four return values. */
data class Quad<A, B, C, D>(
val first: A,
val second: B,
val third: C,
val fourth: D,
)
/** Escapes a Kotlin string into a JSON string literal without building a JsonElement. */
fun String.toJsonString(): String {
val escaped =
this
.replace("\\", "\\\\")
.replace("\"", "\\\"")
.replace("\n", "\\n")
.replace("\r", "\\r")
return "\"$escaped\""
}
fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
/** Parses invoke params into a JSON object, returning null for absent/malformed input. */
fun parseJsonParamsObject(paramsJson: String?): JsonObject? {
if (paramsJson.isNullOrBlank()) return null
return try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
/** Reads a primitive field from invoke params without accepting arrays/objects. */
fun readJsonPrimitive(
params: JsonObject?,
key: String,
): JsonPrimitive? = params?.get(key) as? JsonPrimitive
/** Parses an optional integer invoke param. */
fun parseJsonInt(
params: JsonObject?,
key: String,
): Int? = readJsonPrimitive(params, key)?.contentOrNull?.toIntOrNull()
/** Parses an optional decimal invoke param. */
fun parseJsonDouble(
params: JsonObject?,
key: String,
): Double? = readJsonPrimitive(params, key)?.contentOrNull?.toDoubleOrNull()
/** Parses an optional string invoke param. */
fun parseJsonString(
params: JsonObject?,
key: String,
): String? = readJsonPrimitive(params, key)?.contentOrNull
/** Parses strict true/false flags from string-like JSON primitives. */
fun parseJsonBooleanFlag(
params: JsonObject?,
key: String,
): Boolean? {
val value = readJsonPrimitive(params, key)?.contentOrNull?.trim()?.lowercase() ?: return null
return when (value) {
"true" -> true
"false" -> false
else -> null
}
}
/** Converts JSON null to Kotlin null while preserving primitive text content. */
fun JsonElement?.asStringOrNull(): String? =
when (this) {
is JsonNull -> null
is JsonPrimitive -> content
else -> null
}
/** Parses #RRGGBB or RRGGBB into opaque ARGB. */
fun parseHexColorArgb(raw: String?): Long? {
val trimmed = raw?.trim().orEmpty()
if (trimmed.isEmpty()) return null
val hex = if (trimmed.startsWith("#")) trimmed.drop(1) else trimmed
if (hex.length != 6) return null
val rgb = hex.toLongOrNull(16) ?: return null
return 0xFF000000L or rgb
}
/** Converts gateway invocation throwables into protocol code/message pairs. */
fun invokeErrorFromThrowable(err: Throwable): Pair<String, String> {
val parsed = parseInvokeErrorFromThrowable(err, fallbackMessage = "UNAVAILABLE: error")
val message = if (parsed.hadExplicitCode) parsed.prefixedMessage else parsed.message
return parsed.code to message
}
/** Normalizes user/session keys while preserving main as the canonical session id. */
fun normalizeMainKey(raw: String?): String? {
val trimmed = raw?.trim().orEmpty()
return if (trimmed.isEmpty()) null else trimmed
}

View File

@@ -0,0 +1,180 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.content.Context
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.put
/**
* Injectable notification listener facade so command parsing can be tested without Android service state.
*/
internal interface NotificationsStateProvider {
fun readSnapshot(context: Context): DeviceNotificationSnapshot
fun requestServiceRebind(context: Context)
fun executeAction(
context: Context,
request: NotificationActionRequest,
): NotificationActionResult
}
private object SystemNotificationsStateProvider : NotificationsStateProvider {
/** Reads listener state through Android APIs and returns a disabled snapshot when access is missing. */
override fun readSnapshot(context: Context): DeviceNotificationSnapshot {
val enabled = DeviceNotificationListenerService.isAccessEnabled(context)
if (!enabled) {
return DeviceNotificationSnapshot(
enabled = false,
connected = false,
notifications = emptyList(),
)
}
return DeviceNotificationListenerService.snapshot(context, enabled = true)
}
/** Requests a platform listener rebind after access has been granted. */
override fun requestServiceRebind(context: Context) {
DeviceNotificationListenerService.requestServiceRebind(context)
}
/** Delegates actions to the active listener service instance. */
override fun executeAction(
context: Context,
request: NotificationActionRequest,
): NotificationActionResult = DeviceNotificationListenerService.executeAction(context, request)
}
/** Handles notification listing and actions via the Android listener service. */
class NotificationsHandler private constructor(
private val appContext: Context,
private val stateProvider: NotificationsStateProvider,
) {
constructor(appContext: Context) : this(appContext = appContext, stateProvider = SystemNotificationsStateProvider)
/** Lists the current listener snapshot after nudging Android to reconnect if needed. */
suspend fun handleNotificationsList(_paramsJson: String?): GatewaySession.InvokeResult {
val snapshot = readSnapshotWithRebind()
return GatewaySession.InvokeResult.ok(snapshotPayloadJson(snapshot))
}
/** Executes an action against a notification key from the current listener snapshot. */
suspend fun handleNotificationsActions(paramsJson: String?): GatewaySession.InvokeResult {
readSnapshotWithRebind()
val params =
parseParamsObject(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
val key =
readString(params, "key")
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: key required",
)
val actionRaw =
readString(params, "action")?.lowercase()
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: action required (open|dismiss|reply)",
)
// Keep accepted action names aligned with the cross-platform notification
// command contract rather than Android-specific PendingIntent labels.
val action =
when (actionRaw) {
"open" -> NotificationActionKind.Open
"dismiss" -> NotificationActionKind.Dismiss
"reply" -> NotificationActionKind.Reply
else ->
return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: action must be open|dismiss|reply",
)
}
val replyText = readString(params, "replyText")
if (action == NotificationActionKind.Reply && replyText.isNullOrBlank()) {
return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: replyText required for reply action",
)
}
val result =
stateProvider.executeAction(
appContext,
NotificationActionRequest(
key = key,
kind = action,
replyText = replyText,
),
)
if (!result.ok) {
return GatewaySession.InvokeResult.error(
code = result.code ?: "UNAVAILABLE",
message = result.message ?: "notification action failed",
)
}
val payload =
buildJsonObject {
put("ok", JsonPrimitive(true))
put("key", JsonPrimitive(key))
put("action", JsonPrimitive(actionRaw))
}.toString()
return GatewaySession.InvokeResult.ok(payload)
}
private fun readSnapshotWithRebind(): DeviceNotificationSnapshot {
val snapshot = stateProvider.readSnapshot(appContext)
if (snapshot.enabled && !snapshot.connected) {
// Access can be granted while Android has not rebound the listener yet.
stateProvider.requestServiceRebind(appContext)
}
return snapshot
}
private fun snapshotPayloadJson(snapshot: DeviceNotificationSnapshot): String =
buildJsonObject {
put("enabled", JsonPrimitive(snapshot.enabled))
put("connected", JsonPrimitive(snapshot.connected))
put("count", JsonPrimitive(snapshot.notifications.size))
put(
"notifications",
JsonArray(
snapshot.notifications.map { entry -> entry.toJsonObject() },
),
)
}.toString()
private fun parseParamsObject(paramsJson: String?): JsonObject? {
if (paramsJson.isNullOrBlank()) return null
return try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
private fun readString(
params: JsonObject,
key: String,
): String? =
(params[key] as? JsonPrimitive)
?.contentOrNull
?.trim()
?.takeIf { it.isNotEmpty() }
companion object {
internal fun forTesting(
appContext: Context,
stateProvider: NotificationsStateProvider,
): NotificationsHandler = NotificationsHandler(appContext = appContext, stateProvider = stateProvider)
}
}

View File

@@ -0,0 +1,317 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import ai.openclaw.app.hasPhotoReadPermission
import android.content.ContentResolver
import android.content.ContentUris
import android.content.Context
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.net.Uri
import android.os.Bundle
import android.provider.MediaStore
import androidx.core.graphics.scale
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonArray
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.io.ByteArrayOutputStream
import java.time.Instant
import kotlin.math.max
import kotlin.math.roundToInt
private const val DEFAULT_PHOTOS_LIMIT = 1
private const val DEFAULT_PHOTOS_MAX_WIDTH = 1600
private const val DEFAULT_PHOTOS_QUALITY = 0.85
private const val MAX_TOTAL_BASE64_CHARS = 340 * 1024
private const val MAX_PER_PHOTO_BASE64_CHARS = 300 * 1024
/** Request shape for photos.latest after defaults and bounds are applied. */
internal data class PhotosLatestRequest(
val limit: Int,
val maxWidth: Int,
val quality: Double,
)
/** Encoded photo payload returned to the gateway. */
internal data class EncodedPhotoPayload(
val format: String,
val base64: String,
val width: Int,
val height: Int,
val createdAt: String?,
)
/** Photo access seam for Android MediaStore and tests. */
internal interface PhotosDataSource {
fun hasPermission(context: Context): Boolean
fun latest(
context: Context,
request: PhotosLatestRequest,
): List<EncodedPhotoPayload>
}
private object SystemPhotosDataSource : PhotosDataSource {
override fun hasPermission(context: Context): Boolean = hasPhotoReadPermission(context)
override fun latest(
context: Context,
request: PhotosLatestRequest,
): List<EncodedPhotoPayload> {
val resolver = context.contentResolver
val rows = queryLatestRows(resolver, request.limit)
if (rows.isEmpty()) return emptyList()
var remainingBudget = MAX_TOTAL_BASE64_CHARS
val out = mutableListOf<EncodedPhotoPayload>()
for (row in rows) {
if (remainingBudget <= 0) break
val bitmap = decodeScaledBitmap(resolver, row.uri, request.maxWidth) ?: continue
try {
// Enforce both per-photo and total payload budgets before returning
// base64 data through the gateway invoke response.
val encoded = encodeJpegUnderBudget(bitmap, request.quality, MAX_PER_PHOTO_BASE64_CHARS)
if (encoded == null) continue
if (encoded.base64.length > remainingBudget) break
remainingBudget -= encoded.base64.length
out +=
EncodedPhotoPayload(
format = "jpeg",
base64 = encoded.base64,
width = encoded.width,
height = encoded.height,
createdAt = row.createdAtMs?.let { Instant.ofEpochMilli(it).toString() },
)
} finally {
bitmap.recycle()
}
}
return out
}
private data class PhotoRow(
val uri: Uri,
val createdAtMs: Long?,
)
private data class EncodedJpeg(
val base64: String,
val width: Int,
val height: Int,
)
private fun queryLatestRows(
resolver: ContentResolver,
limit: Int,
): List<PhotoRow> {
val projection =
arrayOf(
MediaStore.Images.Media._ID,
MediaStore.Images.Media.DATE_TAKEN,
MediaStore.Images.Media.DATE_ADDED,
)
val sortOrder =
"${MediaStore.Images.Media.DATE_TAKEN} DESC, ${MediaStore.Images.Media.DATE_ADDED} DESC"
val args =
Bundle().apply {
putString(ContentResolver.QUERY_ARG_SQL_SORT_ORDER, sortOrder)
putInt(ContentResolver.QUERY_ARG_LIMIT, limit)
}
resolver
.query(
MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
projection,
args,
null,
).use { cursor ->
if (cursor == null) return emptyList()
val idIndex = cursor.getColumnIndexOrThrow(MediaStore.Images.Media._ID)
val takenIndex = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATE_TAKEN)
val addedIndex = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATE_ADDED)
val rows = mutableListOf<PhotoRow>()
while (cursor.moveToNext()) {
val id = cursor.getLong(idIndex)
val takenMs = cursor.getLong(takenIndex).takeIf { it > 0L }
val addedMs = cursor.getLong(addedIndex).takeIf { it > 0L }?.times(1000L)
rows +=
PhotoRow(
uri = ContentUris.withAppendedId(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, id),
createdAtMs = takenMs ?: addedMs,
)
}
return rows
}
}
private fun decodeScaledBitmap(
resolver: ContentResolver,
uri: Uri,
maxWidth: Int,
): Bitmap? {
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
resolver.openInputStream(uri).use { input ->
if (input == null) return null
BitmapFactory.decodeStream(input, null, bounds)
}
if (bounds.outWidth <= 0 || bounds.outHeight <= 0) return null
val inSampleSize = computeInSampleSize(bounds.outWidth, maxWidth)
val decodeOptions = BitmapFactory.Options().apply { this.inSampleSize = inSampleSize }
val decoded =
resolver.openInputStream(uri).use { input ->
if (input == null) return null
BitmapFactory.decodeStream(input, null, decodeOptions)
} ?: return null
if (decoded.width <= maxWidth) return decoded
// Decode sampling is power-of-two only; finish with exact scaling when the
// sampled bitmap is still wider than the requested max width.
val targetHeight = max(1, ((decoded.height.toDouble() * maxWidth) / decoded.width).roundToInt())
return try {
decoded.scale(maxWidth, targetHeight, true)
} finally {
decoded.recycle()
}
}
private fun computeInSampleSize(
width: Int,
maxWidth: Int,
): Int {
var sample = 1
var candidate = width
while (candidate > maxWidth && sample < 64) {
sample *= 2
candidate = width / sample
}
return sample
}
private fun encodeJpegUnderBudget(
bitmap: Bitmap,
quality: Double,
maxBase64Chars: Int,
): EncodedJpeg? {
var working = bitmap
try {
var jpegQuality = (quality.coerceIn(0.1, 1.0) * 100.0).roundToInt().coerceIn(10, 100)
repeat(10) {
val out = ByteArrayOutputStream()
val ok = working.compress(Bitmap.CompressFormat.JPEG, jpegQuality, out)
if (!ok) return null
val bytes = out.toByteArray()
val base64 = android.util.Base64.encodeToString(bytes, android.util.Base64.NO_WRAP)
if (base64.length <= maxBase64Chars) {
return EncodedJpeg(
base64 = base64,
width = working.width,
height = working.height,
)
}
if (jpegQuality > 35) {
// Try quality reduction before resizing so small images keep detail.
jpegQuality = max(25, jpegQuality - 15)
return@repeat
}
val nextWidth = max(240, (working.width * 0.75f).roundToInt())
if (nextWidth >= working.width) return null
val nextHeight = max(1, ((working.height.toDouble() * nextWidth) / working.width).roundToInt())
val previous = working
working = working.scale(nextWidth, nextHeight, true)
if (previous !== bitmap) previous.recycle()
}
return null
} finally {
if (working !== bitmap) working.recycle()
}
}
}
/** Handles photos.latest by querying MediaStore and returning bounded JPEG payloads. */
class PhotosHandler private constructor(
private val appContext: Context,
private val dataSource: PhotosDataSource,
) {
constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemPhotosDataSource)
/** Returns the newest accessible photos as gateway-sized base64 JPEGs. */
fun handlePhotosLatest(paramsJson: String?): GatewaySession.InvokeResult {
if (!dataSource.hasPermission(appContext)) {
return GatewaySession.InvokeResult.error(
code = "PHOTOS_PERMISSION_REQUIRED",
message = "PHOTOS_PERMISSION_REQUIRED: grant Photos permission",
)
}
val request =
parseRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object",
)
return try {
val photos = dataSource.latest(appContext, request)
val payload =
buildJsonObject {
put(
"photos",
buildJsonArray {
photos.forEach { photo ->
add(
buildJsonObject {
put("format", JsonPrimitive(photo.format))
put("base64", JsonPrimitive(photo.base64))
put("width", JsonPrimitive(photo.width))
put("height", JsonPrimitive(photo.height))
photo.createdAt?.let { put("createdAt", JsonPrimitive(it)) }
},
)
}
},
)
}.toString()
GatewaySession.InvokeResult.ok(payload)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "PHOTOS_UNAVAILABLE",
message = "PHOTOS_UNAVAILABLE: ${err.message ?: "photo fetch failed"}",
)
}
}
private fun parseRequest(paramsJson: String?): PhotosLatestRequest? {
if (paramsJson.isNullOrBlank()) {
return PhotosLatestRequest(
limit = DEFAULT_PHOTOS_LIMIT,
maxWidth = DEFAULT_PHOTOS_MAX_WIDTH,
quality = DEFAULT_PHOTOS_QUALITY,
)
}
val params =
try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
} ?: return null
val limitRaw = (params["limit"] as? JsonPrimitive)?.content?.toIntOrNull()
val maxWidthRaw = (params["maxWidth"] as? JsonPrimitive)?.content?.toIntOrNull()
val qualityRaw = (params["quality"] as? JsonPrimitive)?.content?.toDoubleOrNull()
// Clamp model-supplied values to protect memory and response-size limits.
val limit = (limitRaw ?: DEFAULT_PHOTOS_LIMIT).coerceIn(1, 20)
val maxWidth = (maxWidthRaw ?: DEFAULT_PHOTOS_MAX_WIDTH).coerceIn(240, 4096)
val quality = (qualityRaw ?: DEFAULT_PHOTOS_QUALITY).coerceIn(0.1, 1.0)
return PhotosLatestRequest(limit = limit, maxWidth = maxWidth, quality = quality)
}
companion object {
/** Creates a handler with an injected photo source for parser and payload tests. */
internal fun forTesting(
appContext: Context,
dataSource: PhotosDataSource,
): PhotosHandler = PhotosHandler(appContext = appContext, dataSource = dataSource)
}
}

View File

@@ -0,0 +1,183 @@
package ai.openclaw.app.node
import ai.openclaw.app.gateway.GatewaySession
import android.Manifest
import android.app.NotificationChannel
import android.app.NotificationManager
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import androidx.core.app.NotificationCompat
import androidx.core.app.NotificationManagerCompat
import androidx.core.content.ContextCompat
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
private const val NOTIFICATION_CHANNEL_BASE_ID = "openclaw.system.notify"
/** Parsed payload for system.notify invocations. */
internal data class SystemNotifyRequest(
val title: String,
val body: String,
val sound: String?,
val priority: String?,
)
/** Notification posting seam used by production Android and unit tests. */
internal interface SystemNotificationPoster {
fun isAuthorized(): Boolean
fun post(request: SystemNotifyRequest)
}
private class AndroidSystemNotificationPoster(
private val appContext: Context,
) : SystemNotificationPoster {
/** Checks both Android 13 runtime permission and app-level notification enablement. */
override fun isAuthorized(): Boolean {
if (Build.VERSION.SDK_INT >= 33) {
val granted =
ContextCompat.checkSelfPermission(appContext, Manifest.permission.POST_NOTIFICATIONS) ==
PackageManager.PERMISSION_GRANTED
if (!granted) return false
}
return NotificationManagerCompat.from(appContext).areNotificationsEnabled()
}
/** Posts through a priority-specific channel so Android's immutable channel importance is respected. */
override fun post(request: SystemNotifyRequest) {
val channelId = ensureChannel(request.priority)
val silent = isSilentSound(request.sound)
val notification =
NotificationCompat
.Builder(appContext, channelId)
.setSmallIcon(android.R.drawable.ic_dialog_info)
.setContentTitle(request.title)
.setContentText(request.body)
.setPriority(compatPriority(request.priority))
.setAutoCancel(true)
.setOnlyAlertOnce(true)
.setSilent(silent)
.build()
if (
Build.VERSION.SDK_INT >= 33 &&
ContextCompat.checkSelfPermission(appContext, Manifest.permission.POST_NOTIFICATIONS) !=
PackageManager.PERMISSION_GRANTED
) {
throw SecurityException("notifications permission missing")
}
NotificationManagerCompat.from(appContext).notify((System.currentTimeMillis() and 0x7FFFFFFF).toInt(), notification)
}
private fun ensureChannel(priority: String?): String {
val normalizedPriority = priority.orEmpty().trim().lowercase()
// Android channel importance is immutable after creation, so priority maps
// to stable channel ids instead of mutating one shared channel.
val (suffix, importance, name) =
when (normalizedPriority) {
"passive" -> Triple("passive", NotificationManager.IMPORTANCE_LOW, "OpenClaw Passive")
"timesensitive" -> Triple("timesensitive", NotificationManager.IMPORTANCE_HIGH, "OpenClaw Time Sensitive")
else -> Triple("active", NotificationManager.IMPORTANCE_DEFAULT, "OpenClaw Active")
}
val channelId = "$NOTIFICATION_CHANNEL_BASE_ID.$suffix"
val manager = appContext.getSystemService(NotificationManager::class.java)
val existing = manager.getNotificationChannel(channelId)
if (existing == null) {
manager.createNotificationChannel(NotificationChannel(channelId, name, importance))
}
return channelId
}
private fun compatPriority(priority: String?): Int =
when (priority.orEmpty().trim().lowercase()) {
"passive" -> NotificationCompat.PRIORITY_LOW
"timesensitive" -> NotificationCompat.PRIORITY_HIGH
else -> NotificationCompat.PRIORITY_DEFAULT
}
private fun isSilentSound(sound: String?): Boolean {
val normalized = sound?.trim()?.lowercase() ?: return false
return normalized in setOf("none", "silent", "off", "false", "0")
}
}
/** Handles system-level node.invoke commands implemented by Android services. */
class SystemHandler private constructor(
private val poster: SystemNotificationPoster,
) {
constructor(appContext: Context) : this(poster = AndroidSystemNotificationPoster(appContext))
/** Posts an Android notification from the gateway system.notify command. */
fun handleSystemNotify(paramsJson: String?): GatewaySession.InvokeResult {
val params =
parseNotifyRequest(paramsJson)
?: return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: expected JSON object with title/body",
)
if (params.title.isEmpty() && params.body.isEmpty()) {
return GatewaySession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: empty notification",
)
}
if (!poster.isAuthorized()) {
return GatewaySession.InvokeResult.error(
code = "NOT_AUTHORIZED",
message = "NOT_AUTHORIZED: notifications",
)
}
return try {
poster.post(params)
GatewaySession.InvokeResult.ok(null)
} catch (_: SecurityException) {
GatewaySession.InvokeResult.error(
code = "NOT_AUTHORIZED",
message = "NOT_AUTHORIZED: notifications",
)
} catch (err: Throwable) {
GatewaySession.InvokeResult.error(
code = "UNAVAILABLE",
message = "NOTIFICATION_FAILED: ${err.message ?: "notification post failed"}",
)
}
}
private fun parseNotifyRequest(paramsJson: String?): SystemNotifyRequest? {
val params = parseParamsObject(paramsJson) ?: return null
// title/body are required by the gateway contract; optional fields only
// influence Android channel/silence behavior.
val rawTitle =
(params["title"] as? JsonPrimitive)
?.contentOrNull
?: return null
val rawBody =
(params["body"] as? JsonPrimitive)
?.contentOrNull
?: return null
val sound = (params["sound"] as? JsonPrimitive)?.contentOrNull
val priority = (params["priority"] as? JsonPrimitive)?.contentOrNull
return SystemNotifyRequest(
title = rawTitle.trim(),
body = rawBody.trim(),
sound = sound?.trim()?.ifEmpty { null },
priority = priority?.trim()?.ifEmpty { null },
)
}
private fun parseParamsObject(paramsJson: String?): JsonObject? {
if (paramsJson.isNullOrBlank()) return null
return try {
Json.parseToJsonElement(paramsJson).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
companion object {
/** Creates a handler with a fake poster for parser and authorization tests. */
internal fun forTesting(poster: SystemNotificationPoster): SystemHandler = SystemHandler(poster)
}
}

View File

@@ -0,0 +1,76 @@
package ai.openclaw.app.protocol
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
object OpenClawCanvasA2UIAction {
/** Reads the agent-facing action name from either the modern name field or legacy action field. */
fun extractActionName(userAction: JsonObject): String? {
val name =
(userAction["name"] as? JsonPrimitive)
?.content
?.trim()
.orEmpty()
if (name.isNotEmpty()) return name
val action =
(userAction["action"] as? JsonPrimitive)
?.content
?.trim()
.orEmpty()
return action.ifEmpty { null }
}
/** Normalizes prompt tag values so the compact CANVAS_A2UI envelope stays parser-friendly. */
fun sanitizeTagValue(value: String): String {
val trimmed = value.trim().ifEmpty { "-" }
val normalized = trimmed.replace(" ", "_")
val out = StringBuilder(normalized.length)
for (c in normalized) {
val ok =
c.isLetterOrDigit() ||
c == '_' ||
c == '-' ||
c == '.' ||
c == ':'
out.append(if (ok) c else '_')
}
return out.toString()
}
/** Formats the compact text envelope sent to the agent when a canvas UI action fires. */
fun formatAgentMessage(
actionName: String,
sessionKey: String,
surfaceId: String,
sourceComponentId: String,
host: String,
instanceId: String,
contextJson: String?,
): String {
val ctxSuffix = contextJson?.takeIf { it.isNotBlank() }?.let { " ctx=$it" }.orEmpty()
return listOf(
"CANVAS_A2UI",
"action=${sanitizeTagValue(actionName)}",
"session=${sanitizeTagValue(sessionKey)}",
"surface=${sanitizeTagValue(surfaceId)}",
"component=${sanitizeTagValue(sourceComponentId)}",
"host=${sanitizeTagValue(host)}",
"instance=${sanitizeTagValue(instanceId)}$ctxSuffix",
"default=update_canvas",
).joinToString(separator = " ")
}
/** Builds JS that reports an agent action result back to the canvas runtime. */
fun jsDispatchA2UIActionStatus(
actionId: String,
ok: Boolean,
error: String?,
): String {
val err = jsonStringLiteral(error ?: "")
val okLiteral = if (ok) "true" else "false"
val idLiteral = jsonStringLiteral(actionId)
return "window.dispatchEvent(new CustomEvent('openclaw:a2ui-action-status', { detail: { id: $idLiteral, ok: $okLiteral, error: $err } }));"
}
private fun jsonStringLiteral(raw: String): String = JsonPrimitive(raw).toString().replace("\u2028", "\\u2028").replace("\u2029", "\\u2029")
}

View File

@@ -0,0 +1,209 @@
package ai.openclaw.app.protocol
/** Capability ids advertised by the Android node to the OpenClaw gateway. */
enum class OpenClawCapability(
val rawValue: String,
) {
Canvas("canvas"),
Camera("camera"),
Sms("sms"),
VoiceWake("voiceWake"),
Talk("talk"),
Location("location"),
Device("device"),
Notifications("notifications"),
System("system"),
Photos("photos"),
Contacts("contacts"),
Calendar("calendar"),
Motion("motion"),
CallLog("callLog"),
}
/** Canvas command ids mirrored from the gateway tool namespace. */
enum class OpenClawCanvasCommand(
val rawValue: String,
) {
Present("canvas.present"),
Hide("canvas.hide"),
Navigate("canvas.navigate"),
Eval("canvas.eval"),
Snapshot("canvas.snapshot"),
;
companion object {
const val NamespacePrefix: String = "canvas."
}
}
/** Streaming canvas commands sent from agents back into the Android UI. */
enum class OpenClawCanvasA2UICommand(
val rawValue: String,
) {
Push("canvas.a2ui.push"),
PushJSONL("canvas.a2ui.pushJSONL"),
Reset("canvas.a2ui.reset"),
;
companion object {
const val NamespacePrefix: String = "canvas.a2ui."
}
}
/** Camera command ids accepted by the Android node. */
enum class OpenClawCameraCommand(
val rawValue: String,
) {
List("camera.list"),
Snap("camera.snap"),
Clip("camera.clip"),
;
companion object {
const val NamespacePrefix: String = "camera."
}
}
/** SMS command ids accepted by the Android node. */
enum class OpenClawSmsCommand(
val rawValue: String,
) {
Send("sms.send"),
Search("sms.search"),
;
companion object {
const val NamespacePrefix: String = "sms."
}
}
/** Push-to-talk command ids accepted by the Android node. */
enum class OpenClawTalkCommand(
val rawValue: String,
) {
PttStart("talk.ptt.start"),
PttStop("talk.ptt.stop"),
PttCancel("talk.ptt.cancel"),
PttOnce("talk.ptt.once"),
;
companion object {
const val NamespacePrefix: String = "talk."
}
}
/** Location command ids accepted by the Android node. */
enum class OpenClawLocationCommand(
val rawValue: String,
) {
Get("location.get"),
;
companion object {
const val NamespacePrefix: String = "location."
}
}
/** Device status and metadata command ids accepted by the Android node. */
enum class OpenClawDeviceCommand(
val rawValue: String,
) {
Status("device.status"),
Info("device.info"),
Permissions("device.permissions"),
Health("device.health"),
Apps("device.apps"),
;
companion object {
const val NamespacePrefix: String = "device."
}
}
/** Notification command ids accepted by the Android node. */
enum class OpenClawNotificationsCommand(
val rawValue: String,
) {
List("notifications.list"),
Actions("notifications.actions"),
;
companion object {
const val NamespacePrefix: String = "notifications."
}
}
/** System command ids accepted by the Android node. */
enum class OpenClawSystemCommand(
val rawValue: String,
) {
Notify("system.notify"),
;
companion object {
const val NamespacePrefix: String = "system."
}
}
/** Photos command ids accepted by the Android node. */
enum class OpenClawPhotosCommand(
val rawValue: String,
) {
Latest("photos.latest"),
;
companion object {
const val NamespacePrefix: String = "photos."
}
}
/** Contacts command ids accepted by the Android node. */
enum class OpenClawContactsCommand(
val rawValue: String,
) {
Search("contacts.search"),
Add("contacts.add"),
;
companion object {
const val NamespacePrefix: String = "contacts."
}
}
/** Calendar command ids accepted by the Android node. */
enum class OpenClawCalendarCommand(
val rawValue: String,
) {
Events("calendar.events"),
Add("calendar.add"),
;
companion object {
const val NamespacePrefix: String = "calendar."
}
}
/** Motion sensor command ids accepted by the Android node. */
enum class OpenClawMotionCommand(
val rawValue: String,
) {
Activity("motion.activity"),
Pedometer("motion.pedometer"),
;
companion object {
const val NamespacePrefix: String = "motion."
}
}
/** Call-log command ids accepted by the Android node. */
enum class OpenClawCallLogCommand(
val rawValue: String,
) {
Search("callLog.search"),
;
companion object {
const val NamespacePrefix: String = "callLog."
}
}

View File

@@ -0,0 +1,250 @@
package ai.openclaw.app.tools
import android.content.Context
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
@Serializable
private data class ToolDisplayActionSpec(
val label: String? = null,
val detailKeys: List<String>? = null,
)
@Serializable
private data class ToolDisplaySpec(
val emoji: String? = null,
val title: String? = null,
val label: String? = null,
val detailKeys: List<String>? = null,
val actions: Map<String, ToolDisplayActionSpec>? = null,
)
@Serializable
private data class ToolDisplayConfig(
val version: Int? = null,
val fallback: ToolDisplaySpec? = null,
val tools: Map<String, ToolDisplaySpec>? = null,
)
/** Compact UI summary for a running or pending tool call. */
data class ToolDisplaySummary(
val name: String,
val emoji: String,
val title: String,
val label: String,
val verb: String?,
val detail: String?,
) {
/** Optional second-line detail assembled from the action verb and best argument preview. */
val detailLine: String?
get() {
val parts = mutableListOf<String>()
if (!verb.isNullOrBlank()) parts.add(verb)
if (!detail.isNullOrBlank()) parts.add(detail)
return if (parts.isEmpty()) null else parts.joinToString(" · ")
}
/** Single-line fallback for compact tool rows that do not render detail separately. */
val summaryLine: String
get() = if (detailLine != null) "$emoji $label: $detailLine" else "$emoji $label"
}
/** Resolves tool-call names and args into user-facing Android display text. */
object ToolDisplayRegistry {
private const val CONFIG_ASSET = "tool-display.json"
private val json = Json { ignoreUnknownKeys = true }
@Volatile private var cachedConfig: ToolDisplayConfig? = null
/** Resolves a raw tool call into stable, bounded UI text for pending-tool surfaces. */
fun resolve(
context: Context,
name: String?,
args: JsonObject?,
meta: String? = null,
): ToolDisplaySummary {
val trimmedName = name?.trim().orEmpty().ifEmpty { "tool" }
val key = trimmedName.lowercase()
val config = loadConfig(context)
val spec = config.tools?.get(key)
val fallback = config.fallback
val emoji = spec?.emoji ?: fallback?.emoji ?: "🧩"
val title = spec?.title ?: titleFromName(trimmedName)
val label = spec?.label ?: trimmedName
val actionRaw = args?.get("action")?.asStringOrNull()?.trim()
val action = actionRaw?.takeIf { it.isNotEmpty() }
val actionSpec = action?.let { spec?.actions?.get(it) }
val verb = normalizeVerb(actionSpec?.label ?: action)
var detail: String? = null
if (key == "read") {
detail = readDetail(args)
} else if (key == "write" || key == "edit" || key == "attach") {
detail = pathDetail(args)
}
// Action-specific detail keys win over tool defaults so commands like
// read/write can surface the most useful argument for that action.
val detailKeys = actionSpec?.detailKeys ?: spec?.detailKeys ?: fallback?.detailKeys ?: emptyList()
if (detail == null) {
detail = firstValue(args, detailKeys)
}
if (detail == null) {
detail = meta
}
if (detail != null) {
detail = shortenHomeInString(detail)
}
return ToolDisplaySummary(
name = trimmedName,
emoji = emoji,
title = title,
label = label,
verb = verb,
detail = detail,
)
}
private fun loadConfig(context: Context): ToolDisplayConfig {
val existing = cachedConfig
if (existing != null) return existing
return try {
val jsonString =
context.assets
.open(CONFIG_ASSET)
.bufferedReader()
.use { it.readText() }
val decoded = json.decodeFromString(ToolDisplayConfig.serializer(), jsonString)
cachedConfig = decoded
decoded
} catch (_: Throwable) {
// The chat UI should still render pending tools if the asset is absent or
// malformed in debug builds.
val fallback = ToolDisplayConfig()
cachedConfig = fallback
fallback
}
}
private fun titleFromName(name: String): String {
val cleaned = name.replace("_", " ").trim()
if (cleaned.isEmpty()) return "Tool"
return cleaned
.split(Regex("\\s+"))
.joinToString(" ") { part ->
val upper = part.uppercase()
if (part.length <= 2 && part == upper) {
part
} else {
upper.firstOrNull()?.toString().orEmpty() + part.lowercase().drop(1)
}
}
}
private fun normalizeVerb(value: String?): String? {
val trimmed = value?.trim().orEmpty()
if (trimmed.isEmpty()) return null
return trimmed.replace("_", " ")
}
private fun readDetail(args: JsonObject?): String? {
val path = args?.get("path")?.asStringOrNull() ?: return null
val offset = args["offset"].asNumberOrNull()
val limit = args["limit"].asNumberOrNull()
return if (offset != null && limit != null) {
val end = offset + limit
"$path:${offset.toInt()}-${end.toInt()}"
} else {
path
}
}
private fun pathDetail(args: JsonObject?): String? = args?.get("path")?.asStringOrNull()
private fun firstValue(
args: JsonObject?,
keys: List<String>,
): String? {
for (key in keys) {
val value = valueForPath(args, key)
val rendered = renderValue(value)
if (!rendered.isNullOrBlank()) return rendered
}
return null
}
private fun valueForPath(
args: JsonObject?,
path: String,
): JsonElement? {
var current: JsonElement? = args
for (segment in path.split(".")) {
if (segment.isBlank()) return null
val obj = current as? JsonObject ?: return null
current = obj[segment]
}
return current
}
private fun renderValue(value: JsonElement?): String? {
if (value == null) return null
if (value is JsonPrimitive) {
if (value.isString) {
val trimmed = value.contentOrNull?.trim().orEmpty()
if (trimmed.isEmpty()) return null
val firstLine =
trimmed
.lineSequence()
.firstOrNull()
?.trim()
.orEmpty()
if (firstLine.isEmpty()) return null
return if (firstLine.length > 160) "${firstLine.take(157)}" else firstLine
}
val raw = value.contentOrNull?.trim().orEmpty()
raw.toBooleanStrictOrNull()?.let { return it.toString() }
raw.toLongOrNull()?.let { return it.toString() }
raw.toDoubleOrNull()?.let { return it.toString() }
}
if (value is JsonArray) {
val items = value.mapNotNull { renderValue(it) }
if (items.isEmpty()) return null
val preview = items.take(3).joinToString(", ")
return if (items.size > 3) "$preview" else preview
}
return null
}
private fun shortenHomeInString(value: String): String {
val home =
System.getProperty("user.home")?.takeIf { it.isNotBlank() }
?: System.getenv("HOME")?.takeIf { it.isNotBlank() }
if (home.isNullOrEmpty()) return value
return value
.replace(home, "~")
.replace(Regex("/Users/[^/]+"), "~")
.replace(Regex("/home/[^/]+"), "~")
}
private fun JsonElement?.asStringOrNull(): String? {
val primitive = this as? JsonPrimitive ?: return null
return if (primitive.isString) primitive.contentOrNull else primitive.toString()
}
private fun JsonElement?.asNumberOrNull(): Double? {
val primitive = this as? JsonPrimitive ?: return null
val raw = primitive.contentOrNull ?: return null
return raw.toDoubleOrNull()
}
}

View File

@@ -0,0 +1,390 @@
package ai.openclaw.app.ui
import ai.openclaw.app.AndroidScreenshotScene
import ai.openclaw.app.ui.design.ClawDesignTheme
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ScreenShare
import androidx.compose.material.icons.filled.ChatBubble
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material.icons.filled.WifiTethering
import androidx.compose.material3.Icon
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
@Composable
fun AndroidScreenshotModeScreen(scene: AndroidScreenshotScene) {
ClawDesignTheme(dark = true) {
Column(
modifier =
Modifier
.fillMaxSize()
.background(ClawTheme.colors.canvas)
.padding(horizontal = 20.dp, vertical = 26.dp),
verticalArrangement = Arrangement.SpaceBetween,
) {
ScreenshotHeader(scene)
ScreenshotSceneBody(scene = scene, modifier = Modifier.weight(1f))
ScreenshotTabBar(activeScene = scene)
}
}
}
@Composable
private fun ScreenshotHeader(scene: AndroidScreenshotScene) {
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically,
) {
Column {
Text(text = "OpenClaw", style = ClawTheme.type.title, color = ClawTheme.colors.text)
Text(
text = sceneTitle(scene),
style = ClawTheme.type.caption,
color = ClawTheme.colors.textMuted,
)
}
StatusPill(label = "Connected", color = ClawTheme.colors.success)
}
}
@Composable
private fun ScreenshotSceneBody(
scene: AndroidScreenshotScene,
modifier: Modifier = Modifier,
) {
Column(
modifier = modifier.fillMaxWidth().padding(vertical = 20.dp),
verticalArrangement = Arrangement.spacedBy(14.dp),
) {
when (scene) {
AndroidScreenshotScene.Connect -> ConnectScene()
AndroidScreenshotScene.Chat -> ChatScene()
AndroidScreenshotScene.Voice -> VoiceScene()
AndroidScreenshotScene.Screen -> ScreenScene()
AndroidScreenshotScene.Settings -> SettingsScene()
}
}
}
@Composable
private fun ConnectScene() {
FeaturePanel(icon = Icons.Default.WifiTethering, title = "Gateway paired", subtitle = "Mac Studio - Tailnet") {
MetricRow(label = "Node", value = "Android Pixel 9")
MetricRow(label = "Transport", value = "Secure WebSocket")
MetricRow(label = "Capabilities", value = "Chat, Talk, Camera, Screen")
}
CompactList(
title = "Ready",
rows =
listOf(
"Push wakes active",
"Approvals synced",
"Device tools available",
),
)
}
@Composable
private fun ChatScene() {
ChatBubble(label = "You", text = "Hi Molty, are you there?")
ChatBubble(
label = "Molty",
text = "Always. Lurking in the shadows, exfoliating.",
raised = true,
)
}
@Composable
private fun VoiceScene() {
Box(modifier = Modifier.fillMaxWidth().padding(vertical = 20.dp), contentAlignment = Alignment.Center) {
Surface(
modifier = Modifier.size(196.dp),
shape = CircleShape,
color = ClawTheme.colors.surfaceRaised,
border = BorderStroke(1.dp, ClawTheme.colors.borderStrong),
) {
Box(contentAlignment = Alignment.Center) {
Icon(
imageVector = Icons.Default.Mic,
contentDescription = null,
tint = ClawTheme.colors.primary,
modifier = Modifier.size(72.dp),
)
}
}
}
FeaturePanel(icon = Icons.Default.Mic, title = "Talk mode", subtitle = "Listening on device") {
MetricRow(label = "Wake phrase", value = "OpenClaw")
MetricRow(label = "Latency", value = "Realtime")
}
}
@Composable
private fun ScreenScene() {
FeaturePanel(icon = Icons.AutoMirrored.Filled.ScreenShare, title = "Screen tools", subtitle = "Shared with your gateway") {
MetricRow(label = "Canvas", value = "Available")
MetricRow(label = "Camera", value = "Permission granted")
MetricRow(label = "Location", value = "On request")
}
Surface(
modifier = Modifier.fillMaxWidth().height(168.dp),
shape = RoundedCornerShape(8.dp),
color = ClawTheme.colors.surfaceRaised,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Column(modifier = Modifier.padding(16.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) {
Text(text = "Live context", style = ClawTheme.type.section, color = ClawTheme.colors.text)
ContextBar(label = "Camera", fraction = 0.74f)
ContextBar(label = "Screen", fraction = 0.58f)
ContextBar(label = "Location", fraction = 0.38f)
}
}
}
@Composable
private fun SettingsScene() {
CompactList(
title = "Security",
rows = listOf("Biometric lock enabled", "Gateway token encrypted", "Tool approvals required"),
)
CompactList(
title = "Notifications",
rows = listOf("Gateway status", "Approval requests", "Background presence"),
)
}
@Composable
private fun FeaturePanel(
icon: ImageVector,
title: String,
subtitle: String,
content: @Composable () -> Unit,
) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(8.dp),
color = ClawTheme.colors.surface,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Column(modifier = Modifier.padding(16.dp), verticalArrangement = Arrangement.spacedBy(14.dp)) {
Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(12.dp)) {
IconBox(icon = icon)
Column {
Text(text = title, style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = subtitle, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
}
}
content()
}
}
}
@Composable
private fun CompactList(
title: String,
rows: List<String>,
) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(8.dp),
color = ClawTheme.colors.surfaceRaised,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Column(modifier = Modifier.padding(16.dp), verticalArrangement = Arrangement.spacedBy(12.dp)) {
Text(text = title, style = ClawTheme.type.section, color = ClawTheme.colors.text)
rows.forEach { row ->
Row(verticalAlignment = Alignment.CenterVertically) {
Box(modifier = Modifier.size(7.dp).clip(CircleShape).background(ClawTheme.colors.success))
Spacer(modifier = Modifier.width(10.dp))
Text(text = row, style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
}
}
}
@Composable
private fun ChatBubble(
label: String,
text: String,
raised: Boolean = false,
) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(8.dp),
color = if (raised) ClawTheme.colors.surfaceRaised else ClawTheme.colors.surface,
border = BorderStroke(1.dp, if (raised) ClawTheme.colors.borderStrong else ClawTheme.colors.border),
) {
Column(modifier = Modifier.padding(16.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(text = label, style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle)
Text(text = text, style = ClawTheme.type.body, color = ClawTheme.colors.text)
}
}
}
@Composable
private fun MetricRow(
label: String,
value: String,
) {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.SpaceBetween) {
Text(text = label, style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
Text(
text = value,
style = ClawTheme.type.label,
color = ClawTheme.colors.text,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
}
}
@Composable
private fun ContextBar(
label: String,
fraction: Float,
) {
Column(verticalArrangement = Arrangement.spacedBy(5.dp)) {
Text(text = label, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
Box(
modifier =
Modifier
.fillMaxWidth()
.height(7.dp)
.clip(RoundedCornerShape(4.dp))
.background(ClawTheme.colors.surfacePressed),
) {
Box(
modifier =
Modifier
.fillMaxWidth(fraction)
.height(7.dp)
.background(ClawTheme.colors.primary),
)
}
}
}
@Composable
private fun ScreenshotTabBar(activeScene: AndroidScreenshotScene) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(8.dp),
color = ClawTheme.colors.surface,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Row(
modifier = Modifier.padding(horizontal = 10.dp, vertical = 8.dp),
horizontalArrangement = Arrangement.SpaceBetween,
) {
TabIcon(icon = Icons.Default.CheckCircle, active = activeScene == AndroidScreenshotScene.Connect)
TabIcon(icon = Icons.Default.ChatBubble, active = activeScene == AndroidScreenshotScene.Chat)
TabIcon(icon = Icons.Default.Mic, active = activeScene == AndroidScreenshotScene.Voice)
TabIcon(icon = Icons.AutoMirrored.Filled.ScreenShare, active = activeScene == AndroidScreenshotScene.Screen)
TabIcon(icon = Icons.Default.Settings, active = activeScene == AndroidScreenshotScene.Settings)
}
}
}
@Composable
private fun TabIcon(
icon: ImageVector,
active: Boolean,
) {
Box(
modifier =
Modifier
.size(42.dp)
.clip(RoundedCornerShape(6.dp))
.background(if (active) ClawTheme.colors.surfacePressed else Color.Transparent),
contentAlignment = Alignment.Center,
) {
Icon(
imageVector = icon,
contentDescription = null,
tint = if (active) ClawTheme.colors.text else ClawTheme.colors.textSubtle,
modifier = Modifier.size(20.dp),
)
}
}
@Composable
private fun IconBox(icon: ImageVector) {
Box(
modifier =
Modifier
.size(42.dp)
.clip(RoundedCornerShape(8.dp))
.background(ClawTheme.colors.surfacePressed),
contentAlignment = Alignment.Center,
) {
Icon(
imageVector = icon,
contentDescription = null,
tint = ClawTheme.colors.primary,
modifier = Modifier.size(22.dp),
)
}
}
@Composable
private fun StatusPill(
label: String,
color: Color,
) {
Surface(
shape = RoundedCornerShape(8.dp),
color = ClawTheme.colors.surfaceRaised,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Row(
modifier = Modifier.padding(horizontal = 10.dp, vertical = 7.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Box(modifier = Modifier.size(7.dp).clip(CircleShape).background(color))
Spacer(modifier = Modifier.width(7.dp))
Text(
text = label,
style = ClawTheme.type.caption.copy(fontWeight = FontWeight.SemiBold),
color = color,
)
}
}
}
private fun sceneTitle(scene: AndroidScreenshotScene): String =
when (scene) {
AndroidScreenshotScene.Connect -> "Connect"
AndroidScreenshotScene.Chat -> "Chat"
AndroidScreenshotScene.Voice -> "Talk"
AndroidScreenshotScene.Screen -> "Device tools"
AndroidScreenshotScene.Settings -> "Settings"
}

View File

@@ -0,0 +1,47 @@
package ai.openclaw.app.ui
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableFloatStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Color
import kotlinx.coroutines.delay
/** Full-screen white flash keyed by camera capture tokens. */
@Composable
fun CameraFlashOverlay(
token: Long,
modifier: Modifier = Modifier,
) {
Box(modifier = modifier.fillMaxSize()) {
CameraFlash(token = token)
}
}
@Composable
private fun CameraFlash(token: Long) {
var alpha by remember { mutableFloatStateOf(0f) }
LaunchedEffect(token) {
if (token == 0L) return@LaunchedEffect
// Token changes replay the animation even when consecutive captures use
// the same HUD message.
alpha = 0.85f
delay(110)
alpha = 0f
}
Box(
modifier =
Modifier
.fillMaxSize()
.alpha(alpha)
.background(Color.White),
)
}

View File

@@ -0,0 +1,217 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import android.annotation.SuppressLint
import android.net.Uri
import android.util.Log
import android.view.View
import android.webkit.ConsoleMessage
import android.webkit.WebChromeClient
import android.webkit.WebResourceError
import android.webkit.WebResourceRequest
import android.webkit.WebResourceResponse
import android.webkit.WebSettings
import android.webkit.WebView
import android.webkit.WebViewClient
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.viewinterop.AndroidView
import androidx.webkit.JavaScriptReplyProxy
import androidx.webkit.WebMessageCompat
import androidx.webkit.WebSettingsCompat
import androidx.webkit.WebViewCompat
import androidx.webkit.WebViewFeature
import java.util.concurrent.atomic.AtomicReference
/** Hosts the gateway canvas WebView and attaches it to the runtime canvas controller. */
@SuppressLint("SetJavaScriptEnabled")
@Suppress("DEPRECATION")
@Composable
fun CanvasScreen(
viewModel: MainViewModel,
visible: Boolean,
modifier: Modifier = Modifier,
) {
val context = LocalContext.current
val isDebuggable = (context.applicationInfo.flags and android.content.pm.ApplicationInfo.FLAG_DEBUGGABLE) != 0
val webViewRef = remember { arrayOfNulls<WebView>(1) }
val currentPageUrlRef = remember { AtomicReference<String?>(null) }
DisposableEffect(viewModel) {
onDispose {
val webView = webViewRef[0] ?: return@onDispose
viewModel.canvas.detach(webView)
if (WebViewFeature.isFeatureSupported(WebViewFeature.WEB_MESSAGE_LISTENER)) {
WebViewCompat.removeWebMessageListener(webView, CanvasA2UIActionBridge.interfaceName)
}
webView.stopLoading()
webView.destroy()
webViewRef[0] = null
}
}
AndroidView(
modifier = modifier,
factory = {
val webView = WebView(context)
val webSettings = webView.settings
webSettings.setAllowContentAccess(false)
webSettings.setAllowFileAccess(false)
webSettings.setAllowFileAccessFromFileURLs(false)
webSettings.setAllowUniversalAccessFromFileURLs(false)
webSettings.setSafeBrowsingEnabled(true)
webSettings.javaScriptEnabled = true
webSettings.domStorageEnabled = true
webSettings.mixedContentMode = WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE
webSettings.useWideViewPort = false
webSettings.loadWithOverviewMode = false
webSettings.builtInZoomControls = false
webSettings.displayZoomControls = false
webSettings.setSupportZoom(false)
webView.visibility = if (visible) View.VISIBLE else View.INVISIBLE
// targetSdk 33+ ignores Force Dark APIs, so only opt out through the supported
// algorithmic darkening flag when this WebView implementation exposes it.
if (WebViewFeature.isFeatureSupported(WebViewFeature.ALGORITHMIC_DARKENING)) {
WebSettingsCompat.setAlgorithmicDarkeningAllowed(webSettings, false)
}
if (isDebuggable) {
Log.d("OpenClawWebView", "userAgent: ${webSettings.userAgentString}")
}
webView.isScrollContainer = true
webView.overScrollMode = View.OVER_SCROLL_IF_CONTENT_SCROLLS
webView.isVerticalScrollBarEnabled = true
webView.isHorizontalScrollBarEnabled = true
webView.webViewClient =
object : WebViewClient() {
override fun onPageStarted(
view: WebView,
url: String?,
favicon: android.graphics.Bitmap?,
) {
currentPageUrlRef.set(url)
}
override fun onReceivedError(
view: WebView,
request: WebResourceRequest,
error: WebResourceError,
) {
if (!isDebuggable || !request.isForMainFrame) return
Log.e("OpenClawWebView", "onReceivedError: ${error.errorCode} ${error.description} ${request.url}")
}
override fun onReceivedHttpError(
view: WebView,
request: WebResourceRequest,
errorResponse: WebResourceResponse,
) {
if (!isDebuggable || !request.isForMainFrame) return
Log.e(
"OpenClawWebView",
"onReceivedHttpError: ${errorResponse.statusCode} ${errorResponse.reasonPhrase} ${request.url}",
)
}
override fun onPageFinished(
view: WebView,
url: String?,
) {
currentPageUrlRef.set(url)
if (isDebuggable) {
Log.d("OpenClawWebView", "onPageFinished: $url")
}
viewModel.canvas.onPageFinished()
}
override fun onRenderProcessGone(
view: WebView,
detail: android.webkit.RenderProcessGoneDetail,
): Boolean {
if (isDebuggable) {
Log.e(
"OpenClawWebView",
"onRenderProcessGone didCrash=${detail.didCrash()} priorityAtExit=${detail.rendererPriorityAtExit()}",
)
}
return true
}
}
webView.webChromeClient =
object : WebChromeClient() {
override fun onConsoleMessage(consoleMessage: ConsoleMessage?): Boolean {
if (!isDebuggable) return false
val msg = consoleMessage ?: return false
Log.d(
"OpenClawWebView",
"console ${msg.messageLevel()} @ ${msg.sourceId()}:${msg.lineNumber()} ${msg.message()}",
)
return false
}
}
// The listener accepts any WebView origin at registration time; native
// dispatch still requires the live URL to be an app-owned bundled page.
val bridge =
CanvasA2UIActionBridge(
isTrustedPage = { viewModel.isTrustedCanvasActionUrl(currentPageUrlRef.get()) },
) { payload ->
viewModel.handleCanvasA2UIActionFromWebView(payload)
}
if (WebViewFeature.isFeatureSupported(WebViewFeature.WEB_MESSAGE_LISTENER)) {
WebViewCompat.addWebMessageListener(
webView,
CanvasA2UIActionBridge.interfaceName,
CanvasA2UIActionBridge.allowedOriginRules,
bridge,
)
} else if (isDebuggable) {
Log.w("OpenClawWebView", "WebMessageListener unsupported; canvas actions disabled")
}
viewModel.canvas.attach(webView)
webViewRef[0] = webView
webView
},
update = { webView ->
webView.visibility = if (visible) View.VISIBLE else View.INVISIBLE
if (visible) {
webView.resumeTimers()
webView.onResume()
} else {
webView.onPause()
webView.pauseTimers()
}
},
)
}
/** Filters WebView postMessage payloads before they enter the A2UI action handler. */
internal class CanvasA2UIActionBridge(
private val isTrustedPage: () -> Boolean,
private val onMessage: (String) -> Unit,
) : WebViewCompat.WebMessageListener {
override fun onPostMessage(
view: WebView,
message: WebMessageCompat,
sourceOrigin: Uri,
isMainFrame: Boolean,
replyProxy: JavaScriptReplyProxy,
) {
if (!isMainFrame) return
postMessage(message.data)
}
fun postMessage(payload: String?) {
val msg = payload?.trim().orEmpty()
if (msg.isEmpty()) return
if (!isTrustedPage()) return
onMessage(msg)
}
companion object {
const val interfaceName: String = "openclawCanvasA2UIAction"
val allowedOriginRules: Set<String> = setOf("*")
}
}

View File

@@ -0,0 +1,142 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawPrimaryButton
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ScreenShare
import androidx.compose.material3.Icon
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
/** Settings detail surface for live canvas status, refresh, and embedded preview. */
@Composable
internal fun CanvasSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val isConnected by viewModel.isConnected.collectAsState()
val currentUrl by viewModel.canvasCurrentUrl.collectAsState()
val hydrated by viewModel.canvasA2uiHydrated.collectAsState()
val rehydratePending by viewModel.canvasRehydratePending.collectAsState()
val rehydrateErrorText by viewModel.canvasRehydrateErrorText.collectAsState()
val hasLivePage = currentUrl?.isNotBlank() == true
val showCanvasSurface = isConnected
val canvasLabel = if (hasLivePage) "Live page" else "Home canvas"
LaunchedEffect(isConnected) {
if (isConnected) {
// Refresh once when the gateway comes online so the settings preview is
// populated before the user manually asks for a rehydrate.
viewModel.refreshHomeCanvasOverviewIfConnected()
}
}
SettingsDetailFrame(
title = "Canvas",
subtitle = "Current screen output and interactive app surface.",
icon = Icons.AutoMirrored.Filled.ScreenShare,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Connection", if (isConnected) "Online" else "Offline"),
SettingsMetric("Surface", canvasLabel),
SettingsMetric("Bridge", if (hasLivePage && hydrated) "Ready" else "Standby"),
),
)
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ClawPrimaryButton(
text = if (rehydratePending) "Refreshing" else "Refresh Screen",
onClick = { viewModel.requestCanvasRehydrate(source = "settings_canvas") },
enabled = isConnected && !rehydratePending,
modifier = Modifier.weight(1f),
)
ClawSecondaryButton(
text = "Reconnect",
onClick = viewModel::refreshGatewayConnection,
modifier = Modifier.weight(1f),
)
}
rehydrateErrorText?.let {
ClawPanel {
Text(text = it, style = ClawTheme.type.body, color = ClawTheme.colors.warning)
}
}
ClawPanel(contentPadding = PaddingValues(horizontal = 8.dp, vertical = 8.dp)) {
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(text = canvasLabel, style = ClawTheme.type.section, color = ClawTheme.colors.text, maxLines = 1, overflow = TextOverflow.Ellipsis)
Surface(
modifier = Modifier.fillMaxWidth().height(520.dp).clip(RoundedCornerShape(ClawTheme.radii.panel)),
shape = RoundedCornerShape(ClawTheme.radii.panel),
color = ClawTheme.colors.canvas,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Box {
if (showCanvasSurface) {
CanvasScreen(viewModel = viewModel, visible = true, modifier = Modifier.fillMaxWidth().height(520.dp))
} else {
CanvasStandbyPanel(isConnected = isConnected)
}
}
}
}
}
}
}
@Composable
private fun CanvasStandbyPanel(isConnected: Boolean) {
Column(
modifier = Modifier.fillMaxWidth().height(520.dp).padding(horizontal = 24.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center,
) {
Surface(
modifier = Modifier.size(54.dp),
shape = RoundedCornerShape(ClawTheme.radii.panel),
color = ClawTheme.colors.surfacePressed,
border = BorderStroke(1.dp, ClawTheme.colors.borderStrong),
contentColor = ClawTheme.colors.text,
) {
Box(contentAlignment = Alignment.Center) {
Icon(imageVector = Icons.AutoMirrored.Filled.ScreenShare, contentDescription = null, modifier = Modifier.size(26.dp))
}
}
Text(
text = if (isConnected) "Screen surface ready" else "Connect the gateway",
style = ClawTheme.type.title,
color = ClawTheme.colors.text,
modifier = Modifier.padding(top = 18.dp),
)
Text(
text = if (isConnected) "Canvas output appears here when OpenClaw opens an app surface." else "Canvas output needs an active gateway connection.",
style = ClawTheme.type.body,
color = ClawTheme.colors.textMuted,
modifier = Modifier.padding(top = 6.dp),
)
}
}

View File

@@ -0,0 +1,163 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayChannelSummary
import ai.openclaw.app.GatewayChannelsSummary
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawDetailRow
import ai.openclaw.app.ui.design.ClawListPanel
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawStatus
import ai.openclaw.app.ui.design.ClawStatusPill
import ai.openclaw.app.ui.design.ClawTextBadge
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Notifications
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
/** Settings screen for gateway channel readiness and account status. */
@Composable
internal fun ChannelsSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val summary by viewModel.channelsSummary.collectAsState()
val refreshing by viewModel.channelsRefreshing.collectAsState()
val errorText by viewModel.channelsErrorText.collectAsState()
val isConnected by viewModel.isConnected.collectAsState()
val channels = summary.channels
LaunchedEffect(isConnected) {
if (isConnected) {
viewModel.refreshChannels()
}
}
SettingsDetailFrame(
title = "Channels",
subtitle = "Messaging surfaces connected to this gateway.",
icon = Icons.Default.Notifications,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Channels", channels.size.toString()),
SettingsMetric("Connected", channels.count { it.connected }.toString()),
SettingsMetric("Configured", channels.count { it.configured }.toString()),
SettingsMetric("Issues", channels.count { it.error != null }.toString()),
),
)
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ClawSecondaryButton(
text = if (refreshing) "Refreshing" else "Refresh",
onClick = viewModel::refreshChannels,
enabled = isConnected && !refreshing,
modifier = Modifier.weight(1f),
)
}
errorText?.let { error ->
ClawPanel {
Text(text = error, style = ClawTheme.type.body, color = ClawTheme.colors.warning)
}
}
if (summary.partial || summary.warnings.isNotEmpty()) {
// Partial channel scans still include useful rows; surface the warning
// without hiding successful channel status.
ClawPanel {
Text(text = channelsWarningText(summary), style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
when {
!isConnected ->
ClawPanel {
Text(text = "Connect the gateway to load channels.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
channels.isEmpty() ->
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(3.dp)) {
Text(text = "No channels found.", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = "Telegram, WhatsApp, email, and other channels appear here after setup.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
else -> ChannelsPanel(channels = channels)
}
}
}
@Composable
private fun ChannelsPanel(channels: List<GatewayChannelSummary>) {
ClawListPanel(items = channels) { channel ->
ChannelRow(channel = channel)
}
}
@Composable
private fun ChannelRow(channel: GatewayChannelSummary) {
ClawDetailRow(
title = channel.label,
subtitle = channelSubtitle(channel),
leading = { ClawTextBadge(text = channelBadge(channel.label)) },
trailing = { ClawStatusPill(text = channelStatusText(channel), status = channelStatus(channel)) },
)
}
private fun channelSubtitle(channel: GatewayChannelSummary): String {
val accounts =
when (channel.accountCount) {
0 -> null
1 -> "1 account"
else -> "${channel.accountCount} accounts"
}
val lifecycle =
when {
channel.connected -> "Connected"
channel.running -> "Running"
channel.linked -> "Linked"
channel.configured -> "Configured"
channel.enabled -> "Enabled"
else -> "Off"
}
return listOfNotNull(accounts, lifecycle, channel.error).joinToString(" · ")
}
private fun channelStatusText(channel: GatewayChannelSummary): String =
when {
channel.error != null -> "Issue"
channel.connected -> "Connected"
channel.running -> "Running"
channel.linked || channel.configured -> "Ready"
channel.enabled -> "Setup"
else -> "Off"
}
private fun channelStatus(channel: GatewayChannelSummary): ClawStatus =
when {
channel.error != null -> ClawStatus.Danger
channel.connected || channel.running -> ClawStatus.Success
channel.linked || channel.configured -> ClawStatus.Neutral
channel.enabled -> ClawStatus.Warning
else -> ClawStatus.Neutral
}
private fun channelBadge(label: String): String =
label
.split(' ', '-', '_')
.filter { it.isNotBlank() }
.take(2)
.mapNotNull { it.firstOrNull()?.uppercaseChar()?.toString() }
.joinToString("")
.ifBlank { "C" }
/** Chooses the first gateway warning or a generic partial-scan message. */
private fun channelsWarningText(summary: GatewayChannelsSummary): String = summary.warnings.firstOrNull()?.takeIf { it.isNotBlank() } ?: "Some channel status checks did not complete."

View File

@@ -0,0 +1,11 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.chat.ChatSheetContent
import androidx.compose.runtime.Composable
/** Keeps the public shell entry point stable while chat internals live under ui.chat. */
@Composable
fun ChatSheet(viewModel: MainViewModel) {
ChatSheetContent(viewModel = viewModel)
}

View File

@@ -0,0 +1,315 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayModelProviderSummary
import ai.openclaw.app.GatewayModelSummary
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawEmptyState
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawPlainIconButton
import ai.openclaw.app.ui.design.ClawScaffold
import ai.openclaw.app.ui.design.ClawSeparatedColumn
import ai.openclaw.app.ui.design.ClawTextField
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.outlined.AccessTime
import androidx.compose.material.icons.outlined.ChatBubbleOutline
import androidx.compose.material.icons.outlined.Inventory2
import androidx.compose.material.icons.outlined.MicNone
import androidx.compose.material.icons.outlined.Settings
import androidx.compose.material3.Icon
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
/** Full-screen command palette for navigation and recent-session search. */
@Composable
internal fun CommandPalette(
viewModel: MainViewModel,
onDismiss: () -> Unit,
onOpenChat: () -> Unit,
onOpenVoice: () -> Unit,
onOpenSessions: () -> Unit,
onOpenProviders: () -> Unit,
onOpenSettings: () -> Unit,
onOpenSession: (String) -> Unit,
) {
val isConnected by viewModel.isConnected.collectAsState()
val sessions by viewModel.chatSessions.collectAsState()
val models by viewModel.modelCatalog.collectAsState()
val providers by viewModel.modelAuthProviders.collectAsState()
val pendingRunCount by viewModel.pendingRunCount.collectAsState()
var query by rememberSaveable { mutableStateOf("") }
val normalizedQuery = query.trim().lowercase()
val quickActions =
listOf(
CommandItem("Open Chat", "Start or continue a conversation", Icons.Outlined.ChatBubbleOutline, onOpenChat),
CommandItem("Start Voice", "Talk or dictate with OpenClaw", Icons.Outlined.MicNone, onOpenVoice),
CommandItem("Browse Sessions", "Find previous conversations", Icons.Outlined.AccessTime, onOpenSessions),
CommandItem("Providers & Models", providerCommandSubtitle(isConnected, providers, models), Icons.Outlined.Inventory2, onOpenProviders),
CommandItem("Settings", "Gateway, voice, notifications, privacy", Icons.Outlined.Settings, onOpenSettings),
)
val actionRows = quickActions.filter { it.matches(normalizedQuery) }
val sessionRows =
sessions
.filter { session ->
val title = commandSessionTitle(session.displayName)
normalizedQuery.isEmpty() || title.lowercase().contains(normalizedQuery)
}.take(5)
Surface(modifier = Modifier.fillMaxSize(), color = ClawTheme.colors.canvas, contentColor = ClawTheme.colors.text) {
ClawScaffold(contentPadding = PaddingValues(start = 20.dp, top = 14.dp, end = 20.dp, bottom = 20.dp)) {
LazyColumn(verticalArrangement = Arrangement.spacedBy(10.dp)) {
item {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(9.dp),
) {
ClawPlainIconButton(
icon = Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = "Close search",
onClick = onDismiss,
)
Text(text = "Search", style = ClawTheme.type.title, color = ClawTheme.colors.text, modifier = Modifier.weight(1f), textAlign = TextAlign.Center)
CommandAvatar(text = "OC")
}
}
item {
ClawTextField(value = query, onValueChange = { query = it }, placeholder = "Search OpenClaw")
}
item {
CommandSectionLabel(title = "Quick actions")
}
if (actionRows.isEmpty()) {
item {
ClawEmptyState(title = "No actions found", body = "Try Chat, Voice, Sessions, Providers, or Settings.")
}
} else {
item {
CommandActionList(rows = actionRows)
}
}
item {
CommandSectionLabel(title = "Sessions")
}
if (sessionRows.isEmpty()) {
item {
ClawPanel {
Text(
text = if (isConnected) "No matching sessions yet." else "Connect the Gateway to search sessions.",
style = ClawTheme.type.body,
color = ClawTheme.colors.textMuted,
)
}
}
} else {
item {
CommandSessionList(
rows =
sessionRows.map { session ->
CommandSessionRow(
key = session.key,
title = commandSessionTitle(session.displayName),
subtitle = if (pendingRunCount > 0) "Assistant working" else "OpenClaw session",
metadata = session.updatedAtMs?.let(::commandRelativeTime) ?: "now",
)
},
onOpen = onOpenSession,
)
}
}
}
}
}
}
private data class CommandItem(
val title: String,
val subtitle: String,
val icon: ImageVector,
val onClick: () -> Unit,
) {
/** Matches palette queries against both action title and explanatory subtitle. */
fun matches(query: String): Boolean = query.isEmpty() || title.lowercase().contains(query) || subtitle.lowercase().contains(query)
}
private data class CommandSessionRow(
val key: String,
val title: String,
val subtitle: String,
val metadata: String,
)
@Composable
private fun CommandActionList(rows: List<CommandItem>) {
ClawPanel(contentPadding = PaddingValues(horizontal = 8.dp, vertical = 0.dp)) {
ClawSeparatedColumn(items = rows) { row ->
CommandActionRow(row = row)
}
}
}
@Composable
private fun CommandActionRow(row: CommandItem) {
Surface(color = Color.Transparent, contentColor = ClawTheme.colors.text) {
Row(
modifier =
Modifier
.fillMaxWidth()
.heightIn(min = 52.dp)
.clip(RoundedCornerShape(ClawTheme.radii.row))
.clickable(onClick = row.onClick)
.padding(horizontal = 2.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(9.dp),
) {
Icon(imageVector = row.icon, contentDescription = null, modifier = Modifier.size(19.dp), tint = ClawTheme.colors.text)
Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(1.dp)) {
Text(text = row.title, style = ClawTheme.type.body, color = ClawTheme.colors.text, maxLines = 1, overflow = TextOverflow.Ellipsis)
Text(text = row.subtitle, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = "Open ${row.title}",
modifier = Modifier.size(17.dp),
tint = ClawTheme.colors.textMuted,
)
}
}
}
@Composable
private fun CommandSessionList(
rows: List<CommandSessionRow>,
onOpen: (String) -> Unit,
) {
ClawPanel(contentPadding = PaddingValues(horizontal = 8.dp, vertical = 0.dp)) {
ClawSeparatedColumn(items = rows) { row ->
CommandSessionListRow(row = row, onClick = { onOpen(row.key) })
}
}
}
@Composable
private fun CommandSessionListRow(
row: CommandSessionRow,
onClick: () -> Unit,
) {
Surface(color = ClawTheme.colors.canvas, contentColor = ClawTheme.colors.text) {
Row(
modifier =
Modifier
.fillMaxWidth()
.heightIn(min = 58.dp)
.clip(RoundedCornerShape(ClawTheme.radii.row))
.clickable(onClick = onClick)
.padding(horizontal = 2.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Surface(
modifier = Modifier.size(30.dp),
shape = CircleShape,
color = ClawTheme.colors.canvas,
border = BorderStroke(1.dp, ClawTheme.colors.borderStrong),
) {
Box(contentAlignment = Alignment.Center) {
Icon(imageVector = Icons.Outlined.ChatBubbleOutline, contentDescription = null, modifier = Modifier.size(15.dp), tint = ClawTheme.colors.text)
}
}
Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(1.dp)) {
Text(text = row.title, style = ClawTheme.type.body, color = ClawTheme.colors.text, maxLines = 1)
Text(text = row.subtitle, style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle, maxLines = 1)
}
Text(text = row.metadata, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = "Open session",
modifier = Modifier.size(17.dp),
tint = ClawTheme.colors.textMuted,
)
}
}
}
@Composable
private fun CommandAvatar(text: String) {
Surface(
modifier = Modifier.size(34.dp),
shape = CircleShape,
color = ClawTheme.colors.surfaceRaised,
contentColor = ClawTheme.colors.text,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Box(contentAlignment = Alignment.Center) {
Text(text = text.take(2).uppercase(), style = ClawTheme.type.label)
}
}
}
@Composable
private fun CommandSectionLabel(title: String) {
Row(modifier = Modifier.fillMaxWidth()) {
Text(text = title.uppercase(), style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
}
}
internal fun providerCommandSubtitle(
isConnected: Boolean,
providers: List<GatewayModelProviderSummary>,
models: List<GatewayModelSummary>,
): String {
if (!isConnected) return "Connect Gateway to view providers"
val readyProviderCount = providerRows(providers = providers, models = models).count { it.ready }
if (readyProviderCount > 0) return "$readyProviderCount providers ready"
return "No ready providers"
}
/** Falls back to the canonical main-session label when gateway display names are blank. */
private fun commandSessionTitle(displayName: String?): String = displayName?.takeIf { it.isNotBlank() } ?: "Main session"
/** Formats command-palette session timestamps for compact rows. */
private fun commandRelativeTime(updatedAtMs: Long): String {
val deltaMs = (System.currentTimeMillis() - updatedAtMs).coerceAtLeast(0L)
val minutes = deltaMs / 60_000L
if (minutes < 1) return "now"
if (minutes < 60) return "${minutes}m"
val hours = minutes / 60
if (hours < 24) return "${hours}h"
return "${hours / 24}d"
}

View File

@@ -0,0 +1,675 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayConnectionProblem
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.R
import ai.openclaw.app.ui.mobileCardSurface
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Cloud
import androidx.compose.material.icons.filled.ContentCopy
import androidx.compose.material.icons.filled.ExpandLess
import androidx.compose.material.icons.filled.ExpandMore
import androidx.compose.material.icons.filled.Link
import androidx.compose.material.icons.filled.PowerSettingsNew
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.OutlinedTextFieldDefaults
import androidx.compose.material3.Surface
import androidx.compose.material3.Switch
import androidx.compose.material3.SwitchDefaults
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
private enum class ConnectInputMode {
SetupCode,
Manual,
}
/** Gateway connection screen for setup-code and manual endpoint pairing. */
@Composable
fun ConnectTabScreen(viewModel: MainViewModel) {
val context = LocalContext.current
val gatewayConnectionDisplay by viewModel.gatewayConnectionDisplay.collectAsState()
val statusText = gatewayConnectionDisplay.statusText
val gatewayConnectionProblem = gatewayConnectionDisplay.problem
val isConnected = gatewayConnectionDisplay.isConnected
val remoteAddress by viewModel.remoteAddress.collectAsState()
val manualHost by viewModel.manualHost.collectAsState()
val manualPort by viewModel.manualPort.collectAsState()
val manualTls by viewModel.manualTls.collectAsState()
val manualEnabled by viewModel.manualEnabled.collectAsState()
val gatewayToken by viewModel.gatewayToken.collectAsState()
val pendingTrust by viewModel.pendingGatewayTrust.collectAsState()
var advancedOpen by rememberSaveable { mutableStateOf(false) }
var inputMode by
remember(manualEnabled, manualHost, gatewayToken) {
mutableStateOf(
if (manualEnabled || manualHost.isNotBlank() || gatewayToken.trim().isNotEmpty()) {
ConnectInputMode.Manual
} else {
ConnectInputMode.SetupCode
},
)
}
var setupCode by rememberSaveable { mutableStateOf("") }
var manualHostInput by rememberSaveable { mutableStateOf(manualHost.ifBlank { "10.0.2.2" }) }
var manualPortInput by rememberSaveable { mutableStateOf(manualPort.toString()) }
var manualTlsInput by rememberSaveable { mutableStateOf(manualTls) }
var tokenInput by remember { mutableStateOf("") }
var passwordInput by rememberSaveable { mutableStateOf("") }
var validationText by rememberSaveable { mutableStateOf<String?>(null) }
if (pendingTrust != null) {
val prompt = pendingTrust!!
AlertDialog(
onDismissRequest = { viewModel.declineGatewayTrustPrompt() },
containerColor = mobileCardSurface,
title = { Text(stringResource(R.string.trust_this_gateway), style = mobileHeadline, color = mobileText) },
text = {
val message =
if (prompt.previousFingerprintSha256.isNullOrBlank()) {
stringResource(R.string.gateway_trust_first_seen, prompt.fingerprintSha256)
} else {
stringResource(
R.string.gateway_trust_changed,
prompt.previousFingerprintSha256,
prompt.fingerprintSha256,
)
}
Text(
message,
style = mobileCallout,
color = mobileText,
)
},
confirmButton = {
TextButton(
onClick = { viewModel.acceptGatewayTrustPrompt() },
colors = ButtonDefaults.textButtonColors(contentColor = mobileAccent),
) {
Text(stringResource(R.string.trust_and_continue))
}
},
dismissButton = {
TextButton(
onClick = { viewModel.declineGatewayTrustPrompt() },
colors = ButtonDefaults.textButtonColors(contentColor = mobileTextSecondary),
) {
Text(stringResource(R.string.cancel))
}
},
)
}
val setupResolvedEndpoint = remember(setupCode) { decodeGatewaySetupCode(setupCode)?.url?.let { parseGatewayEndpoint(it)?.displayUrl } }
val manualResolvedEndpoint =
remember(manualHostInput, manualPortInput, manualTlsInput) {
composeGatewayManualUrl(manualHostInput, manualPortInput, manualTlsInput)?.let { parseGatewayEndpoint(it)?.displayUrl }
}
val activeEndpoint =
remember(isConnected, remoteAddress, setupResolvedEndpoint, manualResolvedEndpoint, inputMode) {
when {
isConnected && !remoteAddress.isNullOrBlank() -> remoteAddress!!
inputMode == ConnectInputMode.SetupCode -> setupResolvedEndpoint ?: "Not set"
else -> manualResolvedEndpoint ?: "Not set"
}
}
val showDiagnostics = !isConnected && (gatewayConnectionProblem != null || gatewayStatusHasDiagnostics(statusText))
val pairingRequired = !isConnected && (gatewayConnectionProblem?.isPairingRequired == true || gatewayStatusLooksLikePairing(statusText))
val pairingInstruction = gatewayPairingInstruction(gatewayConnectionProblem)
val statusLabel = gatewayStatusForDisplay(gatewayConnectionProblem?.message ?: statusText)
Column(
modifier = Modifier.verticalScroll(rememberScrollState()).padding(horizontal = 20.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(14.dp),
) {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(stringResource(R.string.gateway_connection), style = mobileTitle1, color = mobileText)
Text(
if (isConnected) {
stringResource(R.string.connected_gateway_ready)
} else {
stringResource(R.string.connect_gateway_get_started)
},
style = mobileCallout,
color = mobileTextSecondary,
)
}
// Status cards in a unified card group
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
color = mobileCardSurface,
border = BorderStroke(1.dp, mobileBorder),
) {
Column {
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
Surface(
shape = RoundedCornerShape(10.dp),
color = mobileAccentSoft,
) {
Icon(
imageVector = Icons.Default.Link,
contentDescription = null,
modifier = Modifier.padding(8.dp).size(18.dp),
tint = mobileAccent,
)
}
Column(verticalArrangement = Arrangement.spacedBy(2.dp)) {
Text(stringResource(R.string.endpoint), style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary)
Text(activeEndpoint, style = mobileBody.copy(fontFamily = FontFamily.Monospace), color = mobileText)
}
}
HorizontalDivider(color = mobileBorder)
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
Surface(
shape = RoundedCornerShape(10.dp),
color = if (isConnected) mobileSuccessSoft else mobileSurface,
) {
Icon(
imageVector = Icons.Default.Cloud,
contentDescription = null,
modifier = Modifier.padding(8.dp).size(18.dp),
tint = if (isConnected) mobileSuccess else mobileTextTertiary,
)
}
Column(verticalArrangement = Arrangement.spacedBy(2.dp)) {
Text(stringResource(R.string.status), style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary)
Text(statusText, style = mobileBody, color = if (isConnected) mobileSuccess else mobileText)
}
}
}
}
if (isConnected) {
// Outlined secondary button when connected — don't scream "danger"
Button(
onClick = {
viewModel.disconnect()
validationText = null
},
modifier = Modifier.fillMaxWidth().height(48.dp),
shape = RoundedCornerShape(14.dp),
colors =
ButtonDefaults.buttonColors(
containerColor = mobileCardSurface,
contentColor = mobileDanger,
),
border = BorderStroke(1.dp, mobileDanger.copy(alpha = 0.4f)),
) {
Icon(Icons.Default.PowerSettingsNew, contentDescription = null, modifier = Modifier.size(18.dp))
Spacer(modifier = Modifier.width(8.dp))
Text(stringResource(R.string.disconnect), style = mobileHeadline.copy(fontWeight = FontWeight.SemiBold))
}
} else {
Button(
onClick = {
if (statusText.contains("operator offline", ignoreCase = true)) {
validationText = null
viewModel.refreshGatewayConnection()
return@Button
}
val plan =
resolveGatewayConnectPlan(
useSetupCode = inputMode == ConnectInputMode.SetupCode,
setupCode = setupCode,
savedManualHost = manualHost,
savedManualPort = manualPort.toString(),
savedManualTls = manualTls,
manualHostInput = manualHostInput,
manualPortInput = manualPortInput,
manualTlsInput = manualTlsInput,
bootstrapTokenInput = "",
tokenInput = tokenInput,
passwordInput = passwordInput,
)
if (plan == null) {
validationText =
if (inputMode == ConnectInputMode.SetupCode) {
val parsedSetup = decodeGatewaySetupCode(setupCode)
if (parsedSetup == null) {
"Paste a valid setup code to connect."
} else {
val parsedGateway = parseGatewayEndpointResult(parsedSetup.url)
gatewayEndpointValidationMessage(
parsedGateway.error ?: GatewayEndpointValidationError.INVALID_URL,
GatewayEndpointInputSource.SETUP_CODE,
)
}
} else {
val manualUrl = composeGatewayManualUrl(manualHostInput, manualPortInput, manualTlsInput)
val parsedGateway = manualUrl?.let(::parseGatewayEndpointResult)
gatewayEndpointValidationMessage(
parsedGateway?.error ?: GatewayEndpointValidationError.INVALID_URL,
GatewayEndpointInputSource.MANUAL,
)
}
return@Button
}
validationText = null
viewModel.saveGatewayConfigAndConnect(plan)
tokenInput = ""
},
modifier = Modifier.fillMaxWidth().height(52.dp),
shape = RoundedCornerShape(14.dp),
colors =
ButtonDefaults.buttonColors(
containerColor = mobileAccent,
contentColor = Color.White,
),
) {
Text(stringResource(R.string.connect_gateway), style = mobileHeadline.copy(fontWeight = FontWeight.Bold))
}
}
if (showDiagnostics) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
color = mobileWarningSoft,
border = BorderStroke(1.dp, mobileWarning.copy(alpha = 0.25f)),
) {
Column(
modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 14.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
Text(if (pairingRequired) "Pairing required" else "Last gateway error", style = mobileHeadline, color = mobileWarning)
Text(statusLabel, style = mobileBody.copy(fontFamily = FontFamily.Monospace), color = mobileText)
if (pairingRequired) {
Text(
pairingInstruction,
style = mobileCallout,
color = mobileTextSecondary,
)
CommandBlock("openclaw devices list")
CommandBlock("openclaw devices approve <requestId>")
}
Text("OpenClaw Android ${openClawAndroidVersionLabel()}", style = mobileCaption1, color = mobileTextSecondary)
Button(
onClick = {
copyGatewayDiagnosticsReport(
context = context,
screen = "connect tab",
gatewayAddress = activeEndpoint,
statusText = statusLabel,
)
},
modifier = Modifier.fillMaxWidth().height(46.dp),
shape = RoundedCornerShape(12.dp),
colors =
ButtonDefaults.buttonColors(
containerColor = mobileCardSurface,
contentColor = mobileWarning,
),
border = BorderStroke(1.dp, mobileWarning.copy(alpha = 0.3f)),
) {
Icon(Icons.Default.ContentCopy, contentDescription = null, modifier = Modifier.size(18.dp))
Spacer(modifier = Modifier.width(8.dp))
Text(stringResource(R.string.copy_report_for_claw), style = mobileCallout.copy(fontWeight = FontWeight.Bold))
}
}
}
}
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
color = mobileSurface,
border = BorderStroke(1.dp, mobileBorder),
onClick = { advancedOpen = !advancedOpen },
) {
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Column(verticalArrangement = Arrangement.spacedBy(2.dp)) {
Text(stringResource(R.string.advanced_controls), style = mobileHeadline, color = mobileText)
Text("Setup code, endpoint, TLS, token, password, onboarding.", style = mobileCaption1, color = mobileTextSecondary)
}
Icon(
imageVector = if (advancedOpen) Icons.Default.ExpandLess else Icons.Default.ExpandMore,
contentDescription = if (advancedOpen) "Collapse advanced controls" else "Expand advanced controls",
tint = mobileTextSecondary,
)
}
}
AnimatedVisibility(visible = advancedOpen) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
color = mobileCardSurface,
border = BorderStroke(1.dp, mobileBorder),
) {
Column(
modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 14.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Text(stringResource(R.string.connection_method), style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary)
Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) {
MethodChip(
label = stringResource(R.string.setup_code),
active = inputMode == ConnectInputMode.SetupCode,
onClick = { inputMode = ConnectInputMode.SetupCode },
)
MethodChip(
label = stringResource(R.string.manual),
active = inputMode == ConnectInputMode.Manual,
onClick = { inputMode = ConnectInputMode.Manual },
)
}
Text("Run these on the gateway host:", style = mobileCallout, color = mobileTextSecondary)
CommandBlock("openclaw qr --setup-code-only")
CommandBlock("openclaw qr --json")
Text(
"For Tailscale or public hosts, use wss:// or Tailscale Serve. Private LAN ws:// remains supported.",
style = mobileCaption1,
color = mobileTextSecondary,
)
if (inputMode == ConnectInputMode.SetupCode) {
Text(stringResource(R.string.setup_code), style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary)
OutlinedTextField(
value = setupCode,
onValueChange = {
setupCode = it
validationText = null
},
placeholder = { Text(stringResource(R.string.paste_setup_code), style = mobileBody, color = mobileTextTertiary) },
modifier = Modifier.fillMaxWidth(),
minLines = 3,
maxLines = 5,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii),
textStyle = mobileBody.copy(fontFamily = FontFamily.Monospace, color = mobileText),
shape = RoundedCornerShape(14.dp),
colors = outlinedColors(),
)
if (!setupResolvedEndpoint.isNullOrBlank()) {
EndpointPreview(endpoint = setupResolvedEndpoint)
}
} else {
Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) {
QuickFillChip(
label = "Android Emulator",
onClick = {
manualHostInput = "10.0.2.2"
manualPortInput = "18789"
manualTlsInput = false
validationText = null
},
)
QuickFillChip(
label = "Localhost",
onClick = {
manualHostInput = "127.0.0.1"
manualPortInput = "18789"
manualTlsInput = false
validationText = null
},
)
}
Text(stringResource(R.string.host), style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary)
OutlinedTextField(
value = manualHostInput,
onValueChange = {
manualHostInput = it
validationText = null
},
placeholder = { Text("10.0.2.2", style = mobileBody, color = mobileTextTertiary) },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Uri),
textStyle = mobileBody.copy(color = mobileText),
shape = RoundedCornerShape(14.dp),
colors = outlinedColors(),
)
Text(
if (manualTlsInput) "Port (optional, defaults to 443)" else "Port",
style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold),
color = mobileTextSecondary,
)
OutlinedTextField(
value = manualPortInput,
onValueChange = {
manualPortInput = it
validationText = null
},
placeholder = { Text(if (manualTlsInput) "443" else "18789", style = mobileBody, color = mobileTextTertiary) },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
textStyle = mobileBody.copy(fontFamily = FontFamily.Monospace, color = mobileText),
shape = RoundedCornerShape(14.dp),
colors = outlinedColors(),
)
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Column(verticalArrangement = Arrangement.spacedBy(2.dp)) {
Text(stringResource(R.string.use_tls), style = mobileHeadline, color = mobileText)
Text(
"Turn this on for Tailscale or public hosts. Private LAN ws:// remains supported.",
style = mobileCallout,
color = mobileTextSecondary,
)
}
Switch(
checked = manualTlsInput,
onCheckedChange = {
manualTlsInput = it
validationText = null
},
colors =
SwitchDefaults.colors(
checkedTrackColor = mobileAccent,
uncheckedTrackColor = mobileBorderStrong,
checkedThumbColor = Color.White,
uncheckedThumbColor = Color.White,
),
)
}
Text(stringResource(R.string.token_optional), style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary)
OutlinedTextField(
value = tokenInput,
onValueChange = { tokenInput = it },
placeholder = { Text("Leave blank to keep saved token", style = mobileBody, color = mobileTextTertiary) },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii),
textStyle = mobileBody.copy(color = mobileText),
shape = RoundedCornerShape(14.dp),
colors = outlinedColors(),
)
Text(
"Password (optional)",
style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold),
color = mobileTextSecondary,
)
OutlinedTextField(
value = passwordInput,
onValueChange = { passwordInput = it },
placeholder = { Text(stringResource(R.string.password), style = mobileBody, color = mobileTextTertiary) },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii),
textStyle = mobileBody.copy(color = mobileText),
shape = RoundedCornerShape(14.dp),
colors = outlinedColors(),
)
if (!manualResolvedEndpoint.isNullOrBlank()) {
EndpointPreview(endpoint = manualResolvedEndpoint)
}
}
HorizontalDivider(color = mobileBorder)
TextButton(onClick = { viewModel.setOnboardingCompleted(false) }) {
Text(stringResource(R.string.run_onboarding_again), style = mobileCallout.copy(fontWeight = FontWeight.SemiBold), color = mobileAccent)
}
}
}
}
if (!validationText.isNullOrBlank()) {
Text(validationText!!, style = mobileCaption1, color = mobileWarning)
}
}
}
private fun gatewayPairingInstruction(problem: GatewayConnectionProblem?): String =
if (problem?.canAutoRetry == true) {
"Approve this phone on the gateway. OpenClaw will reconnect automatically."
} else {
"Approve this phone on the gateway, then retry the connection."
}
@Composable
private fun MethodChip(
label: String,
active: Boolean,
onClick: () -> Unit,
) {
Button(
onClick = onClick,
modifier = Modifier.height(40.dp),
shape = RoundedCornerShape(12.dp),
contentPadding = PaddingValues(horizontal = 12.dp, vertical = 8.dp),
colors =
ButtonDefaults.buttonColors(
containerColor = if (active) mobileAccent else mobileSurface,
contentColor = if (active) Color.White else mobileText,
),
border = BorderStroke(1.dp, if (active) mobileAccentBorderStrong else mobileBorderStrong),
) {
Text(label, style = mobileCaption1.copy(fontWeight = FontWeight.Bold))
}
}
@Composable
private fun QuickFillChip(
label: String,
onClick: () -> Unit,
) {
Button(
onClick = onClick,
shape = RoundedCornerShape(999.dp),
contentPadding = PaddingValues(horizontal = 12.dp, vertical = 6.dp),
colors =
ButtonDefaults.buttonColors(
containerColor = mobileAccentSoft,
contentColor = mobileAccent,
),
elevation = null,
) {
Text(label, style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold))
}
}
@Composable
private fun CommandBlock(command: String) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(12.dp),
color = mobileCodeBg,
border = BorderStroke(1.dp, mobileCodeBorder),
) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Box(modifier = Modifier.width(3.dp).height(42.dp).background(mobileCodeAccent))
Text(
text = command,
modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
style = mobileCallout.copy(fontFamily = FontFamily.Monospace),
color = mobileCodeText,
)
}
}
}
@Composable
private fun EndpointPreview(endpoint: String) {
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
HorizontalDivider(color = mobileBorder)
Text(
stringResource(R.string.resolved_endpoint),
style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold),
color = mobileTextSecondary,
)
Text(endpoint, style = mobileCallout.copy(fontFamily = FontFamily.Monospace), color = mobileText)
HorizontalDivider(color = mobileBorder)
}
}
@Composable
private fun outlinedColors() =
OutlinedTextFieldDefaults.colors(
focusedContainerColor = mobileSurface,
unfocusedContainerColor = mobileSurface,
focusedBorderColor = mobileAccent,
unfocusedBorderColor = mobileBorder,
focusedTextColor = mobileText,
unfocusedTextColor = mobileText,
cursorColor = mobileAccent,
)

View File

@@ -0,0 +1,184 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayDreamDiaryEntry
import ai.openclaw.app.GatewayDreamingSummary
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawStatusRow
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Storage
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
/** Settings screen for gateway dreaming state and recent dream diary entries. */
@Composable
internal fun DreamingSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val summary by viewModel.dreamingSummary.collectAsState()
val refreshing by viewModel.dreamingRefreshing.collectAsState()
val errorText by viewModel.dreamingErrorText.collectAsState()
val isConnected by viewModel.isConnected.collectAsState()
LaunchedEffect(isConnected) {
if (isConnected) {
viewModel.refreshDreaming()
}
}
SettingsDetailFrame(
title = "Dreaming",
subtitle = "Memory consolidation and dream diary.",
icon = Icons.Default.Storage,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Status", if (summary.enabled) "On" else "Off"),
SettingsMetric("Waiting", summary.shortTermCount.toString()),
SettingsMetric("Signals", summary.totalSignalCount.toString()),
SettingsMetric("Next Cycle", formatDreamingNextRun(summary.nextRunAtMs)),
),
)
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ClawSecondaryButton(
text = if (refreshing) "Refreshing" else "Refresh",
onClick = viewModel::refreshDreaming,
enabled = isConnected && !refreshing,
modifier = Modifier.weight(1f),
)
}
errorText?.let { error ->
ClawPanel {
Text(text = error, style = ClawTheme.type.body, color = ClawTheme.colors.warning)
}
}
when {
!isConnected ->
ClawPanel {
Text(text = "Connect the gateway to load dreaming.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
else -> DreamingPanel(summary = summary)
}
}
}
@Composable
private fun DreamingPanel(summary: GatewayDreamingSummary) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
ClawPanel(contentPadding = PaddingValues(horizontal = 0.dp, vertical = 0.dp)) {
Column {
ClawStatusRow(
title = "Memory Store",
value = if (summary.storeHealthy) "Healthy" else "Needs attention",
healthy = summary.storeHealthy,
)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(
title = "Signal Index",
value = if (summary.phaseSignalHealthy) "Healthy" else "Needs attention",
healthy = summary.phaseSignalHealthy,
)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(
title = "Promoted",
value = "${summary.promotedToday} today · ${summary.promotedTotal} total",
healthy = true,
)
}
}
DreamDiaryPanel(summary = summary)
}
}
@Composable
private fun DreamDiaryPanel(summary: GatewayDreamingSummary) {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = "DIARY", style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
if (!summary.diaryFound) {
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(3.dp)) {
Text(text = "No dream diary yet.", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = "Entries appear after a dreaming cycle writes a narrative summary.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
return
}
if (summary.diaryEntries.isEmpty()) {
ClawPanel {
Text(text = "The diary is waiting for its first entry.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
return
}
ClawPanel(contentPadding = PaddingValues(horizontal = 0.dp, vertical = 0.dp)) {
Column {
summary.diaryEntries.forEachIndexed { index, entry ->
DreamDiaryRow(entry = entry)
if (index != summary.diaryEntries.lastIndex) {
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
}
}
}
@Composable
private fun DreamDiaryRow(entry: GatewayDreamDiaryEntry) {
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 10.dp, vertical = 7.dp),
verticalAlignment = Alignment.Top,
horizontalArrangement = Arrangement.spacedBy(9.dp),
) {
Surface(
modifier = Modifier.size(30.dp),
shape = CircleShape,
color = ClawTheme.colors.surfacePressed,
border = BorderStroke(1.dp, ClawTheme.colors.border),
) {
Box(contentAlignment = Alignment.Center) {
Text(text = "D", style = ClawTheme.type.label, color = ClawTheme.colors.text)
}
}
Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(1.dp)) {
Text(text = entry.date, style = ClawTheme.type.body, color = ClawTheme.colors.text, maxLines = 1, overflow = TextOverflow.Ellipsis)
Text(text = entry.text, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted, maxLines = 2, overflow = TextOverflow.Ellipsis)
}
}
}
/** Formats the next dreaming cycle as a compact relative label. */
private fun formatDreamingNextRun(nextRunAtMs: Long?): String {
val next = nextRunAtMs ?: return "Not scheduled"
val deltaMinutes = ((next - System.currentTimeMillis()) / 60_000L).coerceAtLeast(0L)
val hours = deltaMinutes / 60L
return when {
hours >= 24L -> "In ${hours / 24L}d"
hours >= 1L -> "In ${hours}h"
deltaMinutes >= 1L -> "In ${deltaMinutes}m"
else -> "Soon"
}
}

View File

@@ -0,0 +1,370 @@
package ai.openclaw.app.ui
import ai.openclaw.app.gateway.isLocalCleartextGatewayHost
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.jsonObject
import java.net.URI
import java.util.Base64
import java.util.Locale
/** Parsed endpoint fields after URL validation and cleartext-safety checks. */
internal data class GatewayEndpointConfig(
val host: String,
val port: Int,
val tls: Boolean,
val displayUrl: String,
)
/** Decoded setup-code payload; only one credential family is expected to be populated. */
internal data class GatewaySetupCode(
val url: String,
val bootstrapToken: String?,
val token: String?,
val password: String?,
)
/** Final gateway connection fields selected from setup-code or manual UI input. */
internal data class GatewayConnectConfig(
val host: String,
val port: Int,
val tls: Boolean,
val bootstrapToken: String,
val token: String,
val password: String,
)
/** How a connection attempt may update credentials already owned by the runtime. */
internal enum class GatewaySavedAuthAction {
PRESERVE,
REPLACE_ENDPOINT,
REPLACE_SETUP,
}
/** Endpoint plus the credential ownership decision applied by MainViewModel. */
internal data class GatewayConnectPlan(
val config: GatewayConnectConfig,
val savedAuthAction: GatewaySavedAuthAction,
)
/** Validation reason used by setup, QR, and manual endpoint copy. */
internal enum class GatewayEndpointValidationError {
INVALID_URL,
INSECURE_REMOTE_URL,
IPV6_ZONE_ID_UNSUPPORTED,
}
/** User input source used to choose endpoint-validation wording. */
internal enum class GatewayEndpointInputSource {
SETUP_CODE,
MANUAL,
QR_SCAN,
}
/** Endpoint parse result that preserves the reason when no usable config exists. */
internal data class GatewayEndpointParseResult(
val config: GatewayEndpointConfig? = null,
val error: GatewayEndpointValidationError? = null,
)
/** QR scan result that separates a usable setup code from validation copy. */
internal data class GatewayScannedSetupCodeResult(
val setupCode: String? = null,
val error: GatewayEndpointValidationError? = null,
)
private val gatewaySetupJson = Json { ignoreUnknownKeys = true }
private const val remoteGatewaySecurityRule =
"Public gateways require wss:// or Tailscale Serve. ws:// is allowed for localhost, .local hosts, the Android emulator, and private LAN IPs."
private const val remoteGatewaySecurityFix =
"Use a private LAN IP for local setup, or enable Tailscale Serve / expose a wss:// gateway URL for remote access."
/** Resolves setup-code or manual UI fields without reading stored credentials. */
internal fun resolveGatewayConnectConfig(
useSetupCode: Boolean,
setupCode: String,
manualHostInput: String,
manualPortInput: String,
manualTlsInput: Boolean,
bootstrapTokenInput: String,
tokenInput: String,
passwordInput: String,
): GatewayConnectConfig? {
if (useSetupCode) {
val setup = resolveSetupCodeCandidate(setupCode)?.let(::decodeGatewaySetupCode) ?: return null
val parsed = parseGatewayEndpointResult(setup.url).config ?: return null
val setupBootstrapToken =
setup.bootstrapToken
?.trim()
.orEmpty()
.ifEmpty { bootstrapTokenInput.trim() }
// Bootstrap setup codes intentionally suppress stale shared credentials;
// the bootstrap token owns the first authenticated pairing exchange.
val sharedToken =
when {
!setup.token.isNullOrBlank() -> setup.token.trim()
setupBootstrapToken.isNotEmpty() -> ""
else -> tokenInput.trim()
}
val sharedPassword =
when {
!setup.password.isNullOrBlank() -> setup.password.trim()
setupBootstrapToken.isNotEmpty() || sharedToken.isNotEmpty() -> ""
else -> passwordInput.trim()
}
return GatewayConnectConfig(
host = parsed.host,
port = parsed.port,
tls = parsed.tls,
bootstrapToken = setupBootstrapToken,
token = sharedToken,
password = sharedPassword,
)
}
val manualUrl = composeGatewayManualUrl(manualHostInput, manualPortInput, manualTlsInput) ?: return null
val parsed = parseGatewayEndpointResult(manualUrl).config ?: return null
val token = tokenInput.trim()
val bootstrapToken = bootstrapTokenInput.trim().takeIf { token.isEmpty() }.orEmpty()
val password = passwordInput.trim().takeIf { token.isEmpty() && bootstrapToken.isEmpty() }.orEmpty()
return GatewayConnectConfig(
host = parsed.host,
port = parsed.port,
tls = parsed.tls,
bootstrapToken = bootstrapToken,
token = token,
password = password,
)
}
/**
* Produces one closed endpoint/auth plan. Blank auth fields preserve secrets
* only for the saved endpoint; neither Compose nor this resolver reads them.
*/
internal fun resolveGatewayConnectPlan(
useSetupCode: Boolean,
setupCode: String,
savedManualHost: String,
savedManualPort: String,
savedManualTls: Boolean,
manualHostInput: String,
manualPortInput: String,
manualTlsInput: Boolean,
tokenInput: String,
bootstrapTokenInput: String,
passwordInput: String,
): GatewayConnectPlan? {
val config =
resolveGatewayConnectConfig(
useSetupCode = useSetupCode,
setupCode = setupCode,
manualHostInput = manualHostInput,
manualPortInput = manualPortInput,
manualTlsInput = manualTlsInput,
tokenInput = tokenInput,
bootstrapTokenInput = bootstrapTokenInput,
passwordInput = passwordInput,
) ?: return null
if (useSetupCode) {
return GatewayConnectPlan(config, GatewaySavedAuthAction.REPLACE_SETUP)
}
if (config.bootstrapToken.isNotEmpty()) {
// Bootstrap auth requests a fresh pairing exchange. Retained role tokens
// would otherwise win before the bootstrap credential is attempted.
return GatewayConnectPlan(config, GatewaySavedAuthAction.REPLACE_SETUP)
}
val savedManualEndpoint =
composeGatewayManualUrl(savedManualHost, savedManualPort, savedManualTls)
?.let { parseGatewayEndpointResult(it).config }
val action =
if (savedManualEndpoint?.sameEndpoint(config) == true) {
GatewaySavedAuthAction.PRESERVE
} else {
GatewaySavedAuthAction.REPLACE_ENDPOINT
}
return GatewayConnectPlan(config, action)
}
private fun GatewayEndpointConfig.sameEndpoint(config: GatewayConnectConfig): Boolean = host.equals(config.host, ignoreCase = true) && port == config.port && tls == config.tls
/** Parses an endpoint string and returns only the valid connection config. */
internal fun parseGatewayEndpoint(rawInput: String): GatewayEndpointConfig? = parseGatewayEndpointResult(rawInput).config
/** Parses and validates gateway endpoint input with user-facing error reasons. */
internal fun parseGatewayEndpointResult(rawInput: String): GatewayEndpointParseResult {
val raw = rawInput.trim()
if (raw.isEmpty()) return GatewayEndpointParseResult(error = GatewayEndpointValidationError.INVALID_URL)
val normalized = if (raw.contains("://")) raw else "https://$raw"
val uri =
runCatching { URI(normalized) }.getOrNull()
?: return GatewayEndpointParseResult(error = GatewayEndpointValidationError.INVALID_URL)
val host = uri.host?.trim()?.trim('[', ']').orEmpty()
if (host.isEmpty()) return GatewayEndpointParseResult(error = GatewayEndpointValidationError.INVALID_URL)
// OkHttp rejects scoped IPv6 hosts after URI decoding, so fail before saving an endpoint that can never dial.
if (host.contains(':') && host.contains('%')) {
return GatewayEndpointParseResult(error = GatewayEndpointValidationError.IPV6_ZONE_ID_UNSUPPORTED)
}
val scheme =
uri.scheme
?.trim()
?.lowercase(Locale.US)
.orEmpty()
if (scheme !in setOf("ws", "wss", "http", "https")) {
return GatewayEndpointParseResult(error = GatewayEndpointValidationError.INVALID_URL)
}
val tls = scheme == "wss" || scheme == "https"
if (!tls && !isLocalCleartextGatewayHost(host)) {
return GatewayEndpointParseResult(error = GatewayEndpointValidationError.INSECURE_REMOTE_URL)
}
val defaultPort = if (tls) 443 else 18789
val displayPort = if (tls) 443 else 80
val port = gatewayPort(uri.port, defaultPort) ?: return GatewayEndpointParseResult(error = GatewayEndpointValidationError.INVALID_URL)
val displayHost = if (host.contains(":")) "[$host]" else host
val displayUrl =
if (port == displayPort && defaultPort == displayPort) {
"${if (tls) "https" else "http"}://$displayHost"
} else {
"${if (tls) "https" else "http"}://$displayHost:$port"
}
return GatewayEndpointParseResult(
config = GatewayEndpointConfig(host = host, port = port, tls = tls, displayUrl = displayUrl),
)
}
/** Decodes base64url setup-code payloads produced by gateway onboarding. */
internal fun decodeGatewaySetupCode(rawInput: String): GatewaySetupCode? {
val trimmed = rawInput.trim()
if (trimmed.isEmpty()) return null
val padded =
trimmed
.replace('-', '+')
.replace('_', '/')
.let { normalized ->
val remainder = normalized.length % 4
if (remainder == 0) normalized else normalized + "=".repeat(4 - remainder)
}
return try {
val decoded = String(Base64.getDecoder().decode(padded), Charsets.UTF_8)
val obj = parseJsonObject(decoded) ?: return null
val url = jsonField(obj, "url").orEmpty()
if (url.isEmpty()) return null
val bootstrapToken = jsonField(obj, "bootstrapToken")
val token = jsonField(obj, "token")
val password = jsonField(obj, "password")
GatewaySetupCode(url = url, bootstrapToken = bootstrapToken, token = token, password = password)
} catch (_: IllegalArgumentException) {
null
}
}
internal fun manualTokenLooksLikeSetupCode(rawInput: String): Boolean = resolveSetupCodeCandidate(rawInput)?.let(::decodeGatewaySetupCode) != null
/** Resolves QR scanner text to setup-code or validation error for UI copy. */
internal fun resolveScannedSetupCodeResult(rawInput: String): GatewayScannedSetupCodeResult {
val setupCode =
resolveSetupCodeCandidate(rawInput)
?: return GatewayScannedSetupCodeResult(error = GatewayEndpointValidationError.INVALID_URL)
val decoded =
decodeGatewaySetupCode(setupCode)
?: return GatewayScannedSetupCodeResult(error = GatewayEndpointValidationError.INVALID_URL)
val parsed = parseGatewayEndpointResult(decoded.url)
if (parsed.config == null) {
return GatewayScannedSetupCodeResult(error = parsed.error)
}
return GatewayScannedSetupCodeResult(setupCode = setupCode)
}
/** Converts endpoint validation errors into setup-source-specific UI copy. */
internal fun gatewayEndpointValidationMessage(
error: GatewayEndpointValidationError,
source: GatewayEndpointInputSource,
): String =
when (error) {
GatewayEndpointValidationError.INSECURE_REMOTE_URL ->
when (source) {
GatewayEndpointInputSource.SETUP_CODE ->
"Setup code points to an insecure remote gateway. $remoteGatewaySecurityRule $remoteGatewaySecurityFix"
GatewayEndpointInputSource.QR_SCAN ->
"QR code points to an insecure remote gateway. $remoteGatewaySecurityRule $remoteGatewaySecurityFix"
GatewayEndpointInputSource.MANUAL ->
"$remoteGatewaySecurityRule $remoteGatewaySecurityFix"
}
GatewayEndpointValidationError.IPV6_ZONE_ID_UNSUPPORTED ->
when (source) {
GatewayEndpointInputSource.SETUP_CODE ->
"Setup code uses an IPv6 zone ID. Use an unscoped IPv6 address or a LAN hostname."
GatewayEndpointInputSource.QR_SCAN ->
"QR code uses an IPv6 zone ID. Use an unscoped IPv6 address or a LAN hostname."
GatewayEndpointInputSource.MANUAL ->
"IPv6 zone IDs are not supported. Use an unscoped IPv6 address or a LAN hostname."
}
GatewayEndpointValidationError.INVALID_URL ->
when (source) {
GatewayEndpointInputSource.SETUP_CODE -> "Setup code has invalid gateway URL."
GatewayEndpointInputSource.QR_SCAN -> "QR code did not contain a valid setup code."
GatewayEndpointInputSource.MANUAL -> "Enter a valid manual endpoint to connect."
}
}
private fun gatewayPort(
port: Int,
defaultPort: Int,
): Int? =
when {
port == -1 -> defaultPort
port in 1..65535 -> port
else -> null
}
/** Builds a URL from manual host/port/tls fields for shared endpoint parsing. */
internal fun composeGatewayManualUrl(
hostInput: String,
portInput: String,
tls: Boolean,
): String? {
val host = hostInput.trim()
if (host.isEmpty()) return null
// A pasted endpoint is already a complete authority; its scheme and port
// must not be silently replaced by stale values from the separate controls.
if (host.contains("://")) {
val parsed = parseGatewayEndpointResult(host)
return host.takeUnless { parsed.error == GatewayEndpointValidationError.INVALID_URL }
}
val bareHost = host.trimEnd('/')
if (bareHost.isEmpty() || bareHost.contains('/')) return null
val portTrimmed = portInput.trim()
val port =
if (portTrimmed.isEmpty()) {
if (tls) 443 else return null
} else {
portTrimmed.toIntOrNull() ?: return null
}
if (port !in 1..65535) return null
val scheme = if (tls) "https" else "http"
return "$scheme://${ai.openclaw.app.gateway.formatGatewayAuthority(bareHost, port)}"
}
private fun parseJsonObject(input: String): JsonObject? = runCatching { gatewaySetupJson.parseToJsonElement(input).jsonObject }.getOrNull()
private fun resolveSetupCodeCandidate(rawInput: String): String? {
val trimmed = rawInput.trim()
if (trimmed.isEmpty()) return null
val qrSetupCode = parseJsonObject(trimmed)?.let { jsonField(it, "setupCode") }
return qrSetupCode ?: trimmed
}
private fun jsonField(
obj: JsonObject,
key: String,
): String? {
val value = (obj[key] as? JsonPrimitive)?.contentOrNull?.trim().orEmpty()
return value.ifEmpty { null }
}

View File

@@ -0,0 +1,178 @@
package ai.openclaw.app.ui
import ai.openclaw.app.BuildConfig
import ai.openclaw.app.GatewayConnectionProblem
import ai.openclaw.app.GatewayNodeApprovalState
import ai.openclaw.app.GatewayNodeCapabilityApproval
import ai.openclaw.app.gateway.normalizeGatewayApprovalRequestId
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Build
import android.widget.Toast
/** App version label shared by diagnostics and gateway-facing Android metadata. */
internal fun openClawAndroidVersionLabel(): String {
val versionName = BuildConfig.VERSION_NAME.trim().ifEmpty { "dev" }
return if (BuildConfig.DEBUG && !versionName.contains("dev", ignoreCase = true)) {
"$versionName-dev"
} else {
versionName
}
}
/** Normalizes blank gateway status text for display and diagnostics copy. */
internal fun gatewayStatusForDisplay(statusText: String): String = statusText.trim().ifEmpty { "Offline" }
/** Returns true when the status has enough signal to show diagnostics affordances. */
internal fun gatewayStatusHasDiagnostics(statusText: String): Boolean {
val lower = gatewayStatusForDisplay(statusText).lowercase()
return lower != "offline" && !lower.contains("connecting")
}
/** Resolves the best non-secret endpoint label available to diagnostics surfaces. */
internal fun gatewayDiagnosticsEndpoint(
remoteAddress: String?,
manualHost: String,
manualPort: Int,
manualTls: Boolean,
): String {
remoteAddress?.trim()?.takeIf { it.isNotEmpty() }?.let { return it }
return composeGatewayManualUrl(manualHost, manualPort.toString(), manualTls)?.let { parseGatewayEndpoint(it)?.displayUrl } ?: "Not set"
}
/** Detects pairing/approval status text so UI can offer pairing-specific actions. */
internal fun gatewayStatusLooksLikePairing(statusText: String): Boolean {
val lower = gatewayStatusForDisplay(statusText).lowercase()
return lower.contains("pair") || lower.contains("approve")
}
/** Maps structured gateway auth failures to the compact labels used by status surfaces. */
internal fun gatewayAuthRecoveryLabel(problem: GatewayConnectionProblem?): String? {
val kind =
when (problem?.code) {
"AUTH_BOOTSTRAP_TOKEN_INVALID" -> GatewayAuthRecoveryLabelKind.SETUP_CODE_EXPIRED
"AUTH_TOKEN_MISSING" -> GatewayAuthRecoveryLabelKind.TOKEN_NEEDED
"AUTH_TOKEN_NOT_CONFIGURED" -> GatewayAuthRecoveryLabelKind.TOKEN_NOT_CONFIGURED
"AUTH_PASSWORD_MISSING" -> GatewayAuthRecoveryLabelKind.PASSWORD_NEEDED
"AUTH_PASSWORD_MISMATCH" -> GatewayAuthRecoveryLabelKind.PASSWORD_INVALID
"AUTH_PASSWORD_NOT_CONFIGURED" -> GatewayAuthRecoveryLabelKind.PASSWORD_NOT_CONFIGURED
"AUTH_SCOPE_MISMATCH" -> GatewayAuthRecoveryLabelKind.ACCESS_NEEDS_REVIEW
"AUTH_TOKEN_MISMATCH",
"AUTH_DEVICE_TOKEN_MISMATCH",
-> GatewayAuthRecoveryLabelKind.SAVED_AUTH_INVALID
"CONTROL_UI_DEVICE_IDENTITY_REQUIRED",
"DEVICE_IDENTITY_REQUIRED",
-> GatewayAuthRecoveryLabelKind.DEVICE_IDENTITY_REQUIRED
else -> return null
}
return gatewayAuthRecoveryLabel(kind)
}
private enum class GatewayAuthRecoveryLabelKind {
SETUP_CODE_EXPIRED,
TOKEN_NEEDED,
TOKEN_NOT_CONFIGURED,
PASSWORD_NEEDED,
PASSWORD_INVALID,
PASSWORD_NOT_CONFIGURED,
ACCESS_NEEDS_REVIEW,
SAVED_AUTH_INVALID,
DEVICE_IDENTITY_REQUIRED,
}
private fun gatewayAuthRecoveryLabel(kind: GatewayAuthRecoveryLabelKind): String =
when (kind) {
GatewayAuthRecoveryLabelKind.SETUP_CODE_EXPIRED -> "Setup code expired"
GatewayAuthRecoveryLabelKind.TOKEN_NEEDED -> "Gateway token needed"
GatewayAuthRecoveryLabelKind.TOKEN_NOT_CONFIGURED -> "Gateway token not configured"
GatewayAuthRecoveryLabelKind.PASSWORD_NEEDED -> "Gateway password needed"
GatewayAuthRecoveryLabelKind.PASSWORD_INVALID -> "Gateway password invalid"
GatewayAuthRecoveryLabelKind.PASSWORD_NOT_CONFIGURED -> "Gateway password not configured"
GatewayAuthRecoveryLabelKind.ACCESS_NEEDS_REVIEW -> "Gateway access needs review"
GatewayAuthRecoveryLabelKind.SAVED_AUTH_INVALID -> "Saved auth invalid"
GatewayAuthRecoveryLabelKind.DEVICE_IDENTITY_REQUIRED -> "Device identity required"
}
/** Returns the exact host command for one node's approval state when available. */
internal fun gatewayNodeApprovalCommand(
state: GatewayNodeApprovalState,
requestId: String?,
): String? =
when (state) {
GatewayNodeApprovalState.PendingApproval,
GatewayNodeApprovalState.PendingReapproval,
-> normalizeGatewayApprovalRequestId(requestId)?.let { "openclaw nodes approve $it" } ?: "openclaw nodes status"
GatewayNodeApprovalState.Unapproved -> "openclaw nodes status"
GatewayNodeApprovalState.Loading,
GatewayNodeApprovalState.Unsupported,
GatewayNodeApprovalState.Approved,
-> null
}
internal fun gatewayNodeApprovalCommand(approval: GatewayNodeCapabilityApproval): String? =
when (approval) {
is GatewayNodeCapabilityApproval.PendingApproval ->
gatewayNodeApprovalCommand(GatewayNodeApprovalState.PendingApproval, approval.requestId)
is GatewayNodeCapabilityApproval.PendingReapproval ->
gatewayNodeApprovalCommand(GatewayNodeApprovalState.PendingReapproval, approval.requestId)
GatewayNodeCapabilityApproval.Unapproved ->
gatewayNodeApprovalCommand(GatewayNodeApprovalState.Unapproved, requestId = null)
GatewayNodeCapabilityApproval.Loading,
GatewayNodeCapabilityApproval.Unsupported,
GatewayNodeCapabilityApproval.Approved,
-> null
}
/** Builds the copyable support prompt with device, endpoint, and exact status context. */
internal fun buildGatewayDiagnosticsReport(
screen: String,
gatewayAddress: String,
statusText: String,
): String {
val device =
listOfNotNull(Build.MANUFACTURER, Build.MODEL)
.joinToString(" ")
.trim()
.ifEmpty { "Android" }
val androidVersion =
Build.VERSION.RELEASE
?.trim()
.orEmpty()
.ifEmpty { Build.VERSION.SDK_INT.toString() }
val endpoint = gatewayAddress.trim().ifEmpty { "unknown" }
val status = gatewayStatusForDisplay(statusText)
return """
Help diagnose this OpenClaw Android gateway connection failure.
Please:
- pick one route only: same machine, same LAN, Tailscale, or public URL
- classify this as pairing/auth, TLS trust, wrong advertised route, wrong address/port, or gateway down
- remember: public routes require wss:// or Tailscale Serve; ws:// is allowed for localhost, .local hosts, the Android emulator, and private LAN IPs
- quote the exact app status/error below
- tell me whether `openclaw devices list` should show a pending pairing request
- if more signal is needed, ask for `openclaw qr --json`, `openclaw devices list`, and `openclaw nodes status`
- give the next exact command or tap
Debug info:
- screen: $screen
- app version: ${openClawAndroidVersionLabel()}
- device: $device
- android: $androidVersion (SDK ${Build.VERSION.SDK_INT})
- gateway address: $endpoint
- status/error: $status
""".trimIndent()
}
/** Copies the diagnostics report to Android clipboard and shows a short confirmation toast. */
internal fun copyGatewayDiagnosticsReport(
context: Context,
screen: String,
gatewayAddress: String,
statusText: String,
) {
val clipboard = context.getSystemService(ClipboardManager::class.java) ?: return
val report = buildGatewayDiagnosticsReport(screen = screen, gatewayAddress = gatewayAddress, statusText = statusText)
clipboard.setPrimaryClip(ClipData.newPlainText("OpenClaw gateway diagnostics", report))
Toast.makeText(context, "Copied gateway diagnostics", Toast.LENGTH_SHORT).show()
}

View File

@@ -0,0 +1,275 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayHealthLogsSummary
import ai.openclaw.app.GatewayLogEntry
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawStatus
import ai.openclaw.app.ui.design.ClawStatusPill
import ai.openclaw.app.ui.design.ClawStatusRow
import ai.openclaw.app.ui.design.ClawTheme
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
/** Settings health screen for gateway/node status and recent gateway logs. */
@Composable
internal fun HealthLogsSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val gatewayConnectionDisplay by viewModel.gatewayConnectionDisplay.collectAsState()
val isConnected = gatewayConnectionDisplay.isConnected
val isNodeConnected by viewModel.isNodeConnected.collectAsState()
val chatHealthOk by viewModel.chatHealthOk.collectAsState()
val modelCount by viewModel.modelCatalog.collectAsState()
val pendingRunCount by viewModel.pendingRunCount.collectAsState()
val talkStatus by viewModel.talkModeStatusText.collectAsState()
val logsSummary by viewModel.healthLogsSummary.collectAsState()
val logsRefreshing by viewModel.healthLogsRefreshing.collectAsState()
val logsErrorText by viewModel.healthLogsErrorText.collectAsState()
var selectedLogEntry by remember { mutableStateOf<GatewayLogEntry?>(null) }
LaunchedEffect(isConnected) {
if (isConnected) {
// Load logs when the gateway becomes available; manual refresh covers
// later updates so this screen does not poll.
viewModel.refreshHealthLogs()
}
}
selectedLogEntry?.let { entry ->
GatewayLogDetailSettingsScreen(entry = entry, onBack = { selectedLogEntry = null })
return
}
SettingsDetailFrame(
title = "Health",
subtitle = "Gateway status, phone node readiness, and recent log stream.",
icon = Icons.Default.Settings,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Gateway", if (isConnected) "Online" else "Offline"),
SettingsMetric("Node", if (isNodeConnected) "Online" else "Waiting"),
SettingsMetric("Models", modelCount.size.toString()),
SettingsMetric("Logs", logsSummary.entries.size.toString()),
),
)
HealthStatusPanel(
gateway = gatewayConnectionDisplay.statusText,
node = if (isNodeConnected) "Online" else "Waiting",
chat = if (chatHealthOk) "Ready" else "Needs connection",
models = "${modelCount.size} available",
voice = talkStatus,
runs = if (pendingRunCount > 0) "$pendingRunCount active" else "Idle",
isConnected = isConnected,
isNodeConnected = isNodeConnected,
chatHealthOk = chatHealthOk,
modelsReady = modelCount.isNotEmpty(),
voiceReady = talkStatus.lowercase() != "off",
)
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ClawSecondaryButton(
text = if (logsRefreshing) "Refreshing" else "Refresh Logs",
onClick = viewModel::refreshHealthLogs,
enabled = isConnected && !logsRefreshing,
modifier = Modifier.weight(1f),
)
}
logsErrorText?.let { error ->
ClawPanel {
Text(text = error, style = ClawTheme.type.body, color = ClawTheme.colors.warning)
}
}
GatewayLogsPanel(isConnected = isConnected, summary = logsSummary, onLogClick = { selectedLogEntry = it })
}
}
@Composable
private fun GatewayLogDetailSettingsScreen(
entry: GatewayLogEntry,
onBack: () -> Unit,
) {
BackHandler(onBack = onBack)
SettingsDetailFrame(
title = "Log Entry",
subtitle = "Readable gateway log detail.",
icon = Icons.Default.Settings,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Time", compactLogTime(entry.time)),
SettingsMetric("Level", entry.level?.uppercase() ?: "LOG"),
SettingsMetric("Subsystem", entry.subsystem ?: "Unknown"),
),
)
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = "Message", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = entry.message, style = ClawTheme.type.body, color = ClawTheme.colors.text)
}
}
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = "Raw", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(
text = entry.raw.take(4_000),
style = ClawTheme.type.caption,
color = ClawTheme.colors.textMuted,
)
}
}
}
}
@Composable
private fun HealthStatusPanel(
gateway: String,
node: String,
chat: String,
models: String,
voice: String,
runs: String,
isConnected: Boolean,
isNodeConnected: Boolean,
chatHealthOk: Boolean,
modelsReady: Boolean,
voiceReady: Boolean,
) {
ClawPanel(contentPadding = PaddingValues(horizontal = 0.dp, vertical = 0.dp)) {
Column {
ClawStatusRow(title = "Gateway", value = gateway, healthy = isConnected)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(title = "Phone Node", value = node, healthy = isNodeConnected)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(title = "Chat", value = chat, healthy = chatHealthOk)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(title = "Models", value = models, healthy = modelsReady)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(title = "Voice", value = voice, healthy = voiceReady)
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
ClawStatusRow(title = "Runs", value = runs, healthy = true)
}
}
}
@Composable
private fun GatewayLogsPanel(
isConnected: Boolean,
summary: GatewayHealthLogsSummary,
onLogClick: (GatewayLogEntry) -> Unit,
) {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.SpaceBetween, verticalAlignment = Alignment.CenterVertically) {
Text(text = "RECENT LOGS", style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
summary.fileName?.let { fileName ->
Text(text = fileName, style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
}
when {
!isConnected ->
ClawPanel {
Text(text = "Connect the gateway to load recent logs.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
summary.entries.isEmpty() ->
ClawPanel {
Text(text = "No recent log entries.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
else ->
ClawPanel(contentPadding = PaddingValues(horizontal = 0.dp, vertical = 0.dp)) {
val entries = summary.entries.takeLast(12)
Column {
entries.forEachIndexed { index, entry ->
GatewayLogRow(entry = entry, onClick = { onLogClick(entry) })
if (index != entries.lastIndex) {
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
}
}
if (summary.truncated) {
Text(text = "Showing the latest log chunk.", style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle)
}
}
}
@Composable
private fun GatewayLogRow(
entry: GatewayLogEntry,
onClick: () -> Unit,
) {
Row(
modifier =
Modifier
.fillMaxWidth()
.clickable(onClickLabel = "Open log entry", onClick = onClick)
.padding(horizontal = 10.dp, vertical = 7.dp),
verticalAlignment = Alignment.Top,
horizontalArrangement = Arrangement.spacedBy(9.dp),
) {
Text(text = compactLogTime(entry.time), style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle, modifier = Modifier.weight(0.72f), maxLines = 1)
Column(modifier = Modifier.weight(2.7f), verticalArrangement = Arrangement.spacedBy(1.dp)) {
Text(text = entry.message, style = ClawTheme.type.caption, color = ClawTheme.colors.text, maxLines = 2, overflow = TextOverflow.Ellipsis)
entry.subsystem?.let { subsystem ->
Text(text = subsystem, style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
}
ClawStatusPill(text = entry.level?.uppercase() ?: "LOG", status = logLevelStatus(entry.level))
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = ClawTheme.colors.textSubtle,
)
}
}
private fun compactLogTime(value: String?): String {
val raw = value?.trim().orEmpty()
if (raw.isEmpty()) return "--:--"
// Gateway log timestamps may be ISO strings or already-compact fragments;
// keep only the HH:mm portion when present.
val time =
raw
.substringAfter('T', raw)
.substringBefore('.')
.substringBefore('+')
.substringBefore('Z')
return time.takeIf { it.length >= 5 }?.take(5) ?: raw.take(5)
}
private fun logLevelStatus(level: String?): ClawStatus =
when (level?.lowercase()) {
"error", "fatal" -> ClawStatus.Danger
"warn" -> ClawStatus.Warning
"info" -> ClawStatus.Success
else -> ClawStatus.Neutral
}

View File

@@ -0,0 +1,223 @@
package ai.openclaw.app.ui
import ai.openclaw.app.R
import androidx.compose.runtime.Composable
import androidx.compose.runtime.staticCompositionLocalOf
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.TextStyle
import androidx.compose.ui.text.font.Font
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.sp
internal data class MobileColors(
val surface: Color,
val surfaceStrong: Color,
val cardSurface: Color,
val border: Color,
val borderStrong: Color,
val text: Color,
val textSecondary: Color,
val textTertiary: Color,
val accent: Color,
val accentSoft: Color,
val accentBorderStrong: Color,
val success: Color,
val successSoft: Color,
val warning: Color,
val warningSoft: Color,
val danger: Color,
val dangerSoft: Color,
val codeBg: Color,
val codeText: Color,
val codeBorder: Color,
val codeAccent: Color,
val chipBorderConnected: Color,
val chipBorderConnecting: Color,
val chipBorderWarning: Color,
val chipBorderError: Color,
)
internal fun lightMobileColors() =
MobileColors(
surface = Color(0xFFFAFBFC),
surfaceStrong = Color(0xFFEFF3F8),
cardSurface = Color(0xFFFFFFFF),
border = Color(0xFFDDE3EC),
borderStrong = Color(0xFFC7D0DC),
text = Color(0xFF16181D),
textSecondary = Color(0xFF505B6A),
textTertiary = Color(0xFF8E98A7),
accent = Color(0xFF1B5ACB),
accentSoft = Color(0xFFEAF2FF),
accentBorderStrong = Color(0xFF174CA9),
success = Color(0xFF287F52),
successSoft = Color(0xFFEAF7F0),
warning = Color(0xFFAF7418),
warningSoft = Color(0xFFFFF4DF),
danger = Color(0xFFC94343),
dangerSoft = Color(0xFFFFECEC),
codeBg = Color(0xFFEFF3F8),
codeText = Color(0xFF172033),
codeBorder = Color(0xFFD7DDE7),
codeAccent = Color(0xFF287F52),
chipBorderConnected = Color(0xFFCFEBD8),
chipBorderConnecting = Color(0xFFD5E2FA),
chipBorderWarning = Color(0xFFEED8B8),
chipBorderError = Color(0xFFF3C8C8),
)
internal fun darkMobileColors() =
MobileColors(
surface = Color(0xFF1A1C20),
surfaceStrong = Color(0xFF24262B),
cardSurface = Color(0xFF1E2024),
border = Color(0xFF2E3038),
borderStrong = Color(0xFF3A3D46),
text = Color(0xFFE4E5EA),
textSecondary = Color(0xFFA0A6B4),
textTertiary = Color(0xFF6B7280),
accent = Color(0xFF6EA8FF),
accentSoft = Color(0xFF1A2A44),
accentBorderStrong = Color(0xFF5B93E8),
success = Color(0xFF5FBB85),
successSoft = Color(0xFF152E22),
warning = Color(0xFFE8A844),
warningSoft = Color(0xFF2E2212),
danger = Color(0xFFE87070),
dangerSoft = Color(0xFF2E1616),
codeBg = Color(0xFF111317),
codeText = Color(0xFFE8EAEE),
codeBorder = Color(0xFF2B2E35),
codeAccent = Color(0xFF3FC97A),
chipBorderConnected = Color(0xFF1E4A30),
chipBorderConnecting = Color(0xFF1E3358),
chipBorderWarning = Color(0xFF3E3018),
chipBorderError = Color(0xFF3E1E1E),
)
// Defaulting to light tokens keeps previews/tests usable when a screen forgets to
// provide the app theme; production roots override this composition local.
internal val LocalMobileColors = staticCompositionLocalOf { lightMobileColors() }
internal object MobileColorsAccessor {
val current: MobileColors
@Composable get() = LocalMobileColors.current
}
// Keep these accessors while screens migrate to `MobileColorsAccessor.current`.
// Each getter must stay composable so callers always read the active theme.
internal val mobileSurface: Color @Composable get() = LocalMobileColors.current.surface
internal val mobileSurfaceStrong: Color @Composable get() = LocalMobileColors.current.surfaceStrong
internal val mobileCardSurface: Color @Composable get() = LocalMobileColors.current.cardSurface
internal val mobileBorder: Color @Composable get() = LocalMobileColors.current.border
internal val mobileBorderStrong: Color @Composable get() = LocalMobileColors.current.borderStrong
internal val mobileText: Color @Composable get() = LocalMobileColors.current.text
internal val mobileTextSecondary: Color @Composable get() = LocalMobileColors.current.textSecondary
internal val mobileTextTertiary: Color @Composable get() = LocalMobileColors.current.textTertiary
internal val mobileAccent: Color @Composable get() = LocalMobileColors.current.accent
internal val mobileAccentSoft: Color @Composable get() = LocalMobileColors.current.accentSoft
internal val mobileAccentBorderStrong: Color @Composable get() = LocalMobileColors.current.accentBorderStrong
internal val mobileSuccess: Color @Composable get() = LocalMobileColors.current.success
internal val mobileSuccessSoft: Color @Composable get() = LocalMobileColors.current.successSoft
internal val mobileWarning: Color @Composable get() = LocalMobileColors.current.warning
internal val mobileWarningSoft: Color @Composable get() = LocalMobileColors.current.warningSoft
internal val mobileDanger: Color @Composable get() = LocalMobileColors.current.danger
internal val mobileDangerSoft: Color @Composable get() = LocalMobileColors.current.dangerSoft
internal val mobileCodeBg: Color @Composable get() = LocalMobileColors.current.codeBg
internal val mobileCodeText: Color @Composable get() = LocalMobileColors.current.codeText
internal val mobileCodeBorder: Color @Composable get() = LocalMobileColors.current.codeBorder
internal val mobileCodeAccent: Color @Composable get() = LocalMobileColors.current.codeAccent
// Build the page backdrop from semantic surfaces so light/dark palettes keep
// their contrast relationship without duplicating raw color stops.
internal val mobileBackgroundGradient: Brush
@Composable get() {
val colors = LocalMobileColors.current
return Brush.verticalGradient(
listOf(
colors.surface,
colors.surfaceStrong,
colors.surfaceStrong,
),
)
}
internal val mobileFontFamily =
FontFamily(
Font(resId = R.font.manrope_400_regular, weight = FontWeight.Normal),
Font(resId = R.font.manrope_500_medium, weight = FontWeight.Medium),
Font(resId = R.font.manrope_600_semibold, weight = FontWeight.SemiBold),
Font(resId = R.font.manrope_700_bold, weight = FontWeight.Bold),
)
internal val mobileDisplay =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.Bold,
fontSize = 34.sp,
lineHeight = 40.sp,
letterSpacing = (-0.8).sp,
)
internal val mobileTitle1 =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.SemiBold,
fontSize = 24.sp,
lineHeight = 30.sp,
letterSpacing = (-0.5).sp,
)
internal val mobileTitle2 =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.SemiBold,
fontSize = 20.sp,
lineHeight = 26.sp,
letterSpacing = (-0.3).sp,
)
internal val mobileHeadline =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.SemiBold,
fontSize = 16.sp,
lineHeight = 22.sp,
letterSpacing = (-0.1).sp,
)
internal val mobileBody =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.Medium,
fontSize = 15.sp,
lineHeight = 22.sp,
)
internal val mobileCallout =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.Medium,
fontSize = 14.sp,
lineHeight = 20.sp,
)
internal val mobileCaption1 =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.Medium,
fontSize = 12.sp,
lineHeight = 16.sp,
letterSpacing = 0.2.sp,
)
internal val mobileCaption2 =
TextStyle(
fontFamily = mobileFontFamily,
fontWeight = FontWeight.Medium,
fontSize = 11.sp,
lineHeight = 14.sp,
letterSpacing = 0.4.sp,
)

View File

@@ -0,0 +1,333 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayDeviceTokenSummary
import ai.openclaw.app.GatewayNodeApprovalState
import ai.openclaw.app.GatewayNodeSummary
import ai.openclaw.app.GatewayNodesDevicesSummary
import ai.openclaw.app.GatewayPairedDeviceSummary
import ai.openclaw.app.GatewayPendingDeviceSummary
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawDetailRow
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawStatus
import ai.openclaw.app.ui.design.ClawStatusPill
import ai.openclaw.app.ui.design.ClawTextBadge
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Cloud
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
/** Settings screen for gateway nodes, paired devices, and pending pairing requests. */
@Composable
internal fun NodesDevicesSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val summary by viewModel.nodesDevicesSummary.collectAsState()
val refreshing by viewModel.nodesDevicesRefreshing.collectAsState()
val errorText by viewModel.nodesDevicesErrorText.collectAsState()
val isConnected by viewModel.isConnected.collectAsState()
LaunchedEffect(isConnected) {
if (isConnected) {
// Refresh once on connection; user-triggered refresh handles later changes
// so device admin state is not polled from Compose.
viewModel.refreshNodesDevices()
}
}
SettingsDetailFrame(
title = "Nodes & Devices",
subtitle = "Live nodes, paired phones, and pending device requests.",
icon = Icons.Default.Cloud,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Nodes", summary.nodes.size.toString()),
SettingsMetric("Online", summary.nodes.count { it.connected }.toString()),
SettingsMetric("Devices", if (summary.devicePairingAvailable) summary.pairedDevices.size.toString() else "Admin"),
SettingsMetric("Pending", summary.pendingDevices.size.toString()),
),
)
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ClawSecondaryButton(
text = if (refreshing) "Refreshing" else "Refresh",
onClick = viewModel::refreshNodesDevices,
enabled = isConnected && !refreshing,
modifier = Modifier.weight(1f),
)
}
errorText?.let {
ClawPanel {
Text(text = it, style = ClawTheme.type.body, color = ClawTheme.colors.warning)
}
}
when {
!isConnected ->
ClawPanel {
Text(text = "Connect the gateway to load nodes and paired devices.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
summary.isEmpty() ->
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(3.dp)) {
Text(text = "No nodes or paired devices.", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = "Linked phones and node hosts will appear here after pairing.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
else -> NodesDevicesPanel(summary = summary)
}
}
}
@Composable
private fun NodesDevicesPanel(summary: GatewayNodesDevicesSummary) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
if (!summary.devicePairingAvailable) {
ClawPanel {
Text(text = devicePairingAdminUnavailableText(), style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
val approvalCommands = summary.nodes.mapNotNull(::nodeApprovalCommandRow)
if (approvalCommands.isNotEmpty()) {
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(text = "Node approval required", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = "Run on the Gateway host:", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
approvalCommands.forEach { (label, command) ->
Text(text = label, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
SelectionContainer {
Text(
text = command,
style = ClawTheme.type.body.copy(fontFamily = FontFamily.Monospace),
color = ClawTheme.colors.text,
)
}
}
}
}
}
if (summary.pendingDevices.isNotEmpty()) {
NodesSection(title = "Pending Requests") {
summary.pendingDevices.forEachIndexed { index, device ->
PendingDeviceRow(device = device)
if (index != summary.pendingDevices.lastIndex) {
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
}
if (summary.nodes.isNotEmpty()) {
NodesSection(title = "Nodes") {
summary.nodes.forEachIndexed { index, node ->
NodeRow(node = node)
if (index != summary.nodes.lastIndex) {
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
}
if (summary.pairedDevices.isNotEmpty()) {
NodesSection(title = "Paired Devices") {
summary.pairedDevices.forEachIndexed { index, device ->
PairedDeviceRow(device = device)
if (index != summary.pairedDevices.lastIndex) {
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
}
}
}
private fun nodeApprovalCommandRow(node: GatewayNodeSummary): Pair<String, String>? {
val command = gatewayNodeApprovalCommand(node.approvalState, node.pendingRequestId) ?: return null
return (node.displayName ?: node.id) to command
}
@Composable
private fun NodesSection(
title: String,
content: @Composable () -> Unit,
) {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = title.uppercase(), style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted)
ClawPanel(contentPadding = PaddingValues(horizontal = 0.dp, vertical = 0.dp)) {
Column {
content()
}
}
}
}
@Composable
private fun NodeRow(node: GatewayNodeSummary) {
DeviceListRow(
badge = nodeBadge(node.displayName ?: node.id),
title = node.displayName ?: node.id,
subtitle = nodeSubtitle(node),
statusText = nodeStatusText(node),
status = nodeStatus(node),
)
}
@Composable
private fun PendingDeviceRow(device: GatewayPendingDeviceSummary) {
DeviceListRow(
badge = nodeBadge(device.displayName ?: device.deviceId),
title = device.displayName ?: "New device",
subtitle = pendingDeviceSubtitle(device),
statusText = if (device.repair) "Repair" else "Review",
status = ClawStatus.Warning,
)
}
@Composable
private fun PairedDeviceRow(device: GatewayPairedDeviceSummary) {
DeviceListRow(
badge = nodeBadge(device.displayName ?: device.deviceId),
title = device.displayName ?: "Paired device",
subtitle = pairedDeviceSubtitle(device),
statusText = pairedDeviceStatusText(device.tokens),
status = pairedDeviceStatus(device.tokens),
)
}
@Composable
private fun DeviceListRow(
badge: String,
title: String,
subtitle: String,
statusText: String,
status: ClawStatus,
) {
ClawDetailRow(
title = title,
subtitle = subtitle,
leading = { ClawTextBadge(text = badge) },
trailing = { ClawStatusPill(text = statusText, status = status) },
)
}
/** True when the gateway returned no node or device rows to render. */
private fun GatewayNodesDevicesSummary.isEmpty(): Boolean = nodes.isEmpty() && pendingDevices.isEmpty() && pairedDevices.isEmpty()
private fun nodeSubtitle(node: GatewayNodeSummary): String {
val kind = node.deviceFamily ?: "Node host"
val version = node.version?.let { "OpenClaw $it" }
val status = if (node.paired) "Paired" else "Unpaired"
val approval = nodeApprovalSubtitle(node.approvalState)
val commands =
node.commands
.take(2)
.joinToString(", ")
.takeIf { it.isNotBlank() }
return listOfNotNull(kind, version, status, approval, commands).joinToString(" · ")
}
private fun nodeStatusText(node: GatewayNodeSummary): String =
when (node.approvalState) {
GatewayNodeApprovalState.PendingApproval -> "Needs approval"
GatewayNodeApprovalState.PendingReapproval -> "Needs reapproval"
GatewayNodeApprovalState.Unapproved -> "Unapproved"
else -> if (node.connected) "Online" else "Offline"
}
private fun nodeStatus(node: GatewayNodeSummary): ClawStatus =
when (node.approvalState) {
GatewayNodeApprovalState.Approved -> if (node.connected) ClawStatus.Success else ClawStatus.Warning
GatewayNodeApprovalState.PendingApproval,
GatewayNodeApprovalState.PendingReapproval,
GatewayNodeApprovalState.Unapproved,
-> ClawStatus.Warning
GatewayNodeApprovalState.Loading,
GatewayNodeApprovalState.Unsupported,
-> if (node.connected) ClawStatus.Neutral else ClawStatus.Warning
}
private fun nodeApprovalSubtitle(approvalState: GatewayNodeApprovalState): String? =
when (approvalState) {
GatewayNodeApprovalState.Approved -> "Approved"
GatewayNodeApprovalState.PendingApproval -> "Capability approval pending"
GatewayNodeApprovalState.PendingReapproval -> "Capability reapproval pending"
GatewayNodeApprovalState.Unapproved -> "Capability unapproved"
GatewayNodeApprovalState.Loading,
GatewayNodeApprovalState.Unsupported,
-> null
}
internal fun devicePairingAdminUnavailableText(): String =
"This gateway sign-in can list connected nodes, but it cannot approve new phone pairing. " +
"Pair new phones from a gateway admin session. Node capability approval is separate and still uses nodes approve <request id>."
private fun pendingDeviceSubtitle(device: GatewayPendingDeviceSummary): String {
val roles = formatDeviceList(device.roles, "role")
val scopes = formatDeviceList(device.scopes, "scope")
val requested = device.requestedAtMs?.let { "requested ${relativeDeviceTime(it)}" }
return listOfNotNull(roles, scopes, requested, device.remoteIp).joinToString(" · ")
}
private fun pairedDeviceSubtitle(device: GatewayPairedDeviceSummary): String {
val roles = formatDeviceList(device.roles, "role")
val scopes = formatDeviceList(device.scopes, "scope")
val tokens = "${device.tokens.count { !it.revoked }}/${device.tokens.size} active tokens"
return listOfNotNull(roles, scopes, tokens, device.remoteIp).joinToString(" · ")
}
private fun pairedDeviceStatusText(tokens: List<GatewayDeviceTokenSummary>): String =
when {
tokens.isEmpty() -> "Paired"
tokens.any { !it.revoked } -> "Active"
else -> "Needs Token"
}
private fun pairedDeviceStatus(tokens: List<GatewayDeviceTokenSummary>): ClawStatus =
when {
tokens.isEmpty() -> ClawStatus.Neutral
tokens.any { !it.revoked } -> ClawStatus.Success
else -> ClawStatus.Warning
}
private fun formatDeviceList(
values: List<String>,
fallback: String,
): String? =
when (values.size) {
0 -> null
1 -> values.first()
else -> "${values.size} ${fallback}s"
}
private fun nodeBadge(value: String): String =
value
.split(' ', '-', '_')
.filter { it.isNotBlank() }
.take(2)
.mapNotNull { it.firstOrNull()?.uppercaseChar()?.toString() }
.joinToString("")
.ifBlank { "N" }
private fun relativeDeviceTime(timeMs: Long): String {
val minutes = ((System.currentTimeMillis() - timeMs).coerceAtLeast(0L)) / 60_000L
if (minutes < 1) return "now"
if (minutes < 60) return "${minutes}m ago"
val hours = minutes / 60L
if (hours < 24) return "${hours}h ago"
return "${hours / 24L}d ago"
}

View File

@@ -0,0 +1,82 @@
package ai.openclaw.app.ui
import ai.openclaw.app.node.DeviceNotificationListenerService
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
/** App entry shown in the notification-forwarding package picker. */
data class InstalledApp(
val label: String,
val packageName: String,
val isSystemApp: Boolean,
)
/** Reads launcher, recent-notification, and configured packages for the picker. */
internal fun queryInstalledApps(
context: Context,
configuredPackages: Set<String>,
): List<InstalledApp> {
val packageManager = context.packageManager
val launcherIntent = Intent(Intent.ACTION_MAIN).apply { addCategory(Intent.CATEGORY_LAUNCHER) }
val launcherPackages =
packageManager
.queryIntentActivities(launcherIntent, PackageManager.MATCH_ALL)
.asSequence()
.mapNotNull {
it.activityInfo
?.packageName
?.trim()
?.takeIf(String::isNotEmpty)
}.toMutableSet()
val recentNotificationPackages =
DeviceNotificationListenerService
.recentPackages(context)
.asSequence()
.map { it.trim() }
.filter { it.isNotEmpty() }
.toList()
val candidatePackages =
resolveNotificationCandidatePackages(
launcherPackages = launcherPackages,
recentPackages = recentNotificationPackages,
configuredPackages = configuredPackages,
appPackageName = context.packageName,
)
return candidatePackages
.asSequence()
.mapNotNull { packageName ->
runCatching {
val appInfo = packageManager.getApplicationInfo(packageName, 0)
val label = packageManager.getApplicationLabel(appInfo).toString().trim()
InstalledApp(
label = if (label.isEmpty()) packageName else label,
packageName = packageName,
isSystemApp = (appInfo.flags and android.content.pm.ApplicationInfo.FLAG_SYSTEM) != 0,
)
}.getOrNull()
}.sortedWith(compareBy<InstalledApp> { it.label.lowercase() }.thenBy { it.packageName })
.toList()
}
/** Merges package sources while excluding OpenClaw from its own forwarding filter. */
internal fun resolveNotificationCandidatePackages(
launcherPackages: Set<String>,
recentPackages: List<String>,
configuredPackages: Set<String>,
appPackageName: String,
): Set<String> {
val blockedPackage = appPackageName.trim()
return sequenceOf(
configuredPackages.asSequence(),
launcherPackages.asSequence(),
recentPackages.asSequence(),
).flatten()
.map { it.trim() }
.filter { it.isNotEmpty() && it != blockedPackage }
.toSet()
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,52 @@
package ai.openclaw.app.ui
import ai.openclaw.app.AppearanceThemeMode
import android.app.Activity
import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.dynamicDarkColorScheme
import androidx.compose.material3.dynamicLightColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.runtime.SideEffect
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.LocalView
import androidx.core.view.WindowCompat
/**
* App theme wrapper that installs dynamic Material colors and legacy mobile color tokens.
*/
@Composable
fun OpenClawTheme(
themeMode: AppearanceThemeMode = AppearanceThemeMode.Dark,
content: @Composable () -> Unit,
) {
val context = LocalContext.current
val isDark = themeMode.isDark(systemDark = isSystemInDarkTheme())
val colorScheme = if (isDark) dynamicDarkColorScheme(context) else dynamicLightColorScheme(context)
val mobileColors = if (isDark) darkMobileColors() else lightMobileColors()
OpenClawSystemBarAppearance(lightAppearance = !isDark)
CompositionLocalProvider(
LocalMobileColors provides mobileColors,
) {
MaterialTheme(colorScheme = colorScheme, content = content)
}
}
@Composable
internal fun OpenClawSystemBarAppearance(lightAppearance: Boolean) {
val view = LocalView.current
if (!view.isInEditMode) {
SideEffect {
val window = (view.context as? Activity)?.window ?: return@SideEffect
WindowCompat
.getInsetsController(window, window.decorView)
.isAppearanceLightStatusBars = lightAppearance
WindowCompat
.getInsetsController(window, window.decorView)
.isAppearanceLightNavigationBars = lightAppearance
}
}
}

View File

@@ -0,0 +1,364 @@
package ai.openclaw.app.ui
import ai.openclaw.app.HomeDestination
import ai.openclaw.app.MainViewModel
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.WindowInsets
import androidx.compose.foundation.layout.WindowInsetsSides
import androidx.compose.foundation.layout.consumeWindowInsets
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.ime
import androidx.compose.foundation.layout.navigationBars
import androidx.compose.foundation.layout.only
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.safeDrawing
import androidx.compose.foundation.layout.windowInsetsPadding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ScreenShare
import androidx.compose.material.icons.filled.ChatBubble
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.RecordVoiceOver
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material3.Icon
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import androidx.compose.ui.zIndex
private enum class HomeTab(
val label: String,
val icon: ImageVector,
) {
Connect(label = "Connect", icon = Icons.Default.CheckCircle),
Chat(label = "Chat", icon = Icons.Default.ChatBubble),
Voice(label = "Voice", icon = Icons.Default.RecordVoiceOver),
Screen(label = "Screen", icon = Icons.AutoMirrored.Filled.ScreenShare),
Settings(label = "Settings", icon = Icons.Default.Settings),
}
private enum class StatusVisual {
Connected,
Connecting,
Warning,
Error,
Offline,
}
/** Legacy tab scaffold used by the mobile post-onboarding experience. */
@Composable
fun PostOnboardingTabs(
viewModel: MainViewModel,
modifier: Modifier = Modifier,
) {
var activeTab by rememberSaveable { mutableStateOf(HomeTab.Connect) }
var chatTabStarted by rememberSaveable { mutableStateOf(false) }
var screenTabStarted by rememberSaveable { mutableStateOf(false) }
val requestedHomeDestination by viewModel.requestedHomeDestination.collectAsState()
LaunchedEffect(requestedHomeDestination) {
val destination = requestedHomeDestination ?: return@LaunchedEffect
activeTab =
when (destination) {
HomeDestination.Connect -> HomeTab.Connect
HomeDestination.Chat -> HomeTab.Chat
HomeDestination.Voice -> HomeTab.Voice
HomeDestination.Screen -> HomeTab.Screen
HomeDestination.Settings -> HomeTab.Settings
}
viewModel.clearRequestedHomeDestination()
}
// Stop TTS when user navigates away from voice tab, and lazily keep the Chat/Screen tabs
// alive after the first visit so repeated tab switches do not rebuild their UI trees.
LaunchedEffect(activeTab) {
viewModel.setVoiceScreenActive(activeTab == HomeTab.Voice)
if (activeTab == HomeTab.Chat) {
chatTabStarted = true
}
if (activeTab == HomeTab.Screen) {
screenTabStarted = true
}
}
val gatewayConnectionDisplay by viewModel.gatewayConnectionDisplay.collectAsState()
val statusVisual =
remember(gatewayConnectionDisplay) {
val lower = gatewayConnectionDisplay.statusText.lowercase()
when {
gatewayConnectionDisplay.isConnected -> StatusVisual.Connected
lower.contains("connecting") || lower.contains("reconnecting") -> StatusVisual.Connecting
lower.contains("pairing") || lower.contains("approval") || lower.contains("auth") -> StatusVisual.Warning
lower.contains("error") || lower.contains("failed") -> StatusVisual.Error
else -> StatusVisual.Offline
}
}
val density = LocalDensity.current
val imeVisible = WindowInsets.ime.getBottom(density) > 0
val hideBottomTabBar = activeTab == HomeTab.Chat && imeVisible
Scaffold(
modifier = modifier,
containerColor = Color.Transparent,
contentWindowInsets = WindowInsets(0, 0, 0, 0),
topBar = {
TopStatusBar(
statusText = gatewayConnectionDisplay.statusText,
statusVisual = statusVisual,
)
},
bottomBar = {
if (!hideBottomTabBar) {
BottomTabBar(
activeTab = activeTab,
onSelect = { activeTab = it },
)
}
},
) { innerPadding ->
Box(
modifier =
Modifier
.fillMaxSize()
.padding(innerPadding)
.consumeWindowInsets(innerPadding)
.background(mobileBackgroundGradient),
) {
if (chatTabStarted) {
// Keep chat mounted after first use so session state and scroll position
// survive tab switches.
Box(
modifier =
Modifier
.matchParentSize()
.alpha(if (activeTab == HomeTab.Chat) 1f else 0f)
.zIndex(if (activeTab == HomeTab.Chat) 1f else 0f),
) {
ChatSheet(viewModel = viewModel)
}
}
if (screenTabStarted) {
// Canvas can be expensive to initialize; keep it mounted once visited
// and hide it by alpha/z-order instead of destroying the view tree.
ScreenTabScreen(
viewModel = viewModel,
visible = activeTab == HomeTab.Screen,
modifier =
Modifier
.matchParentSize()
.alpha(if (activeTab == HomeTab.Screen) 1f else 0f)
.zIndex(if (activeTab == HomeTab.Screen) 1f else 0f),
)
}
when (activeTab) {
HomeTab.Connect -> ConnectTabScreen(viewModel = viewModel)
HomeTab.Chat -> if (!chatTabStarted) ChatSheet(viewModel = viewModel)
HomeTab.Voice -> VoiceTabScreen(viewModel = viewModel)
HomeTab.Screen -> Unit
HomeTab.Settings -> SettingsSheet(viewModel = viewModel)
}
}
}
}
/** Screen tab wrapper that refreshes canvas data once per gateway connection. */
@Composable
private fun ScreenTabScreen(
viewModel: MainViewModel,
visible: Boolean,
modifier: Modifier = Modifier,
) {
val isConnected by viewModel.isConnected.collectAsState()
var refreshedForCurrentConnection by rememberSaveable(isConnected) { mutableStateOf(false) }
LaunchedEffect(isConnected, visible, refreshedForCurrentConnection) {
if (visible && isConnected && !refreshedForCurrentConnection) {
viewModel.refreshHomeCanvasOverviewIfConnected()
refreshedForCurrentConnection = true
}
}
Box(modifier = modifier.fillMaxSize()) {
CanvasScreen(viewModel = viewModel, visible = visible, modifier = Modifier.fillMaxSize())
}
}
/** Top status chip derived from gateway connection text. */
@Composable
private fun TopStatusBar(
statusText: String,
statusVisual: StatusVisual,
) {
val safeInsets = WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal)
val (chipBg, chipDot, chipText, chipBorder) =
when (statusVisual) {
StatusVisual.Connected ->
listOf(
mobileSuccessSoft,
mobileSuccess,
mobileSuccess,
LocalMobileColors.current.chipBorderConnected,
)
StatusVisual.Connecting ->
listOf(
mobileAccentSoft,
mobileAccent,
mobileAccent,
LocalMobileColors.current.chipBorderConnecting,
)
StatusVisual.Warning ->
listOf(
mobileWarningSoft,
mobileWarning,
mobileWarning,
LocalMobileColors.current.chipBorderWarning,
)
StatusVisual.Error ->
listOf(
mobileDangerSoft,
mobileDanger,
mobileDanger,
LocalMobileColors.current.chipBorderError,
)
StatusVisual.Offline ->
listOf(
mobileSurface,
mobileTextTertiary,
mobileTextSecondary,
mobileBorder,
)
}
Surface(
modifier = Modifier.fillMaxWidth().windowInsetsPadding(safeInsets),
color = Color.Transparent,
shadowElevation = 0.dp,
) {
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 18.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Text(
text = "OpenClaw",
style = mobileTitle2,
color = mobileText,
)
Surface(
shape = RoundedCornerShape(999.dp),
color = chipBg,
border = androidx.compose.foundation.BorderStroke(1.dp, chipBorder),
) {
Row(
modifier = Modifier.padding(horizontal = 10.dp, vertical = 5.dp),
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Surface(
modifier = Modifier.padding(top = 1.dp),
color = chipDot,
shape = RoundedCornerShape(999.dp),
) {
Box(modifier = Modifier.padding(4.dp))
}
Text(
text = statusText.trim().ifEmpty { "Offline" },
style = mobileCaption1,
color = chipText,
maxLines = 1,
)
}
}
}
}
}
/** Bottom navigation for the legacy tab scaffold. */
@Composable
private fun BottomTabBar(
activeTab: HomeTab,
onSelect: (HomeTab) -> Unit,
) {
val safeInsets = WindowInsets.navigationBars.only(WindowInsetsSides.Bottom + WindowInsetsSides.Horizontal)
Box(
modifier =
Modifier
.fillMaxWidth(),
) {
Surface(
modifier = Modifier.fillMaxWidth(),
color = mobileCardSurface.copy(alpha = 0.97f),
shape = RoundedCornerShape(topStart = 24.dp, topEnd = 24.dp),
border = BorderStroke(1.dp, mobileBorder),
shadowElevation = 6.dp,
) {
Row(
modifier =
Modifier
.fillMaxWidth()
.windowInsetsPadding(safeInsets)
.padding(horizontal = 10.dp, vertical = 10.dp),
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalAlignment = Alignment.CenterVertically,
) {
HomeTab.entries.forEach { tab ->
val active = tab == activeTab
Surface(
onClick = { onSelect(tab) },
modifier = Modifier.weight(1f).heightIn(min = 58.dp),
shape = RoundedCornerShape(16.dp),
color = if (active) mobileAccentSoft else Color.Transparent,
border = if (active) BorderStroke(1.dp, LocalMobileColors.current.chipBorderConnecting) else null,
shadowElevation = 0.dp,
) {
Column(
modifier = Modifier.fillMaxWidth().padding(horizontal = 6.dp, vertical = 7.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(2.dp),
) {
Icon(
imageVector = tab.icon,
contentDescription = tab.label,
tint = if (active) mobileAccent else mobileTextTertiary,
)
Text(
text = tab.label,
color = if (active) mobileAccent else mobileTextSecondary,
style = mobileCaption2.copy(fontWeight = if (active) FontWeight.Bold else FontWeight.Medium),
)
}
}
}
}
}
}
}

View File

@@ -0,0 +1,334 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewayModelProviderSummary
import ai.openclaw.app.GatewayModelSummary
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.providerDisplayName
import ai.openclaw.app.ui.design.ClawEmptyState
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawScaffold
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.WindowInsets
import androidx.compose.foundation.layout.WindowInsetsSides
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.only
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.safeDrawing
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
/** Android provider readiness screen backed by the configured gateway model view. */
@Composable
internal fun ProvidersModelsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val isConnected by viewModel.isConnected.collectAsState()
val models by viewModel.modelCatalog.collectAsState()
val providers by viewModel.modelAuthProviders.collectAsState()
val refreshing by viewModel.modelCatalogRefreshing.collectAsState()
val errorText by viewModel.modelCatalogErrorText.collectAsState()
val providerRows = providerRows(providers = providers, models = models)
LaunchedEffect(isConnected) {
if (isConnected) {
viewModel.refreshModelCatalog()
}
}
ClawScaffold(
contentPadding = PaddingValues(start = 20.dp, top = 13.dp, end = 20.dp, bottom = 6.dp),
contentWindowInsets = WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal),
) {
Box(modifier = Modifier.fillMaxSize()) {
LazyColumn(
modifier = Modifier.fillMaxSize(),
verticalArrangement = Arrangement.spacedBy(7.dp),
contentPadding = PaddingValues(bottom = 4.dp),
) {
item {
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
ProviderHeaderIconButton(icon = Icons.AutoMirrored.Filled.ArrowBack, contentDescription = "Back", onClick = onBack)
}
Column(verticalArrangement = Arrangement.spacedBy(3.dp)) {
Text(text = "Providers & Models", style = ClawTheme.type.display.copy(fontSize = 14.8.sp, lineHeight = 18.sp), color = ClawTheme.colors.text, maxLines = 1)
Text(
text = "Review provider readiness\nand configured models.",
style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp),
color = ClawTheme.colors.textMuted,
)
}
}
}
item {
ProviderOverviewPanel(
isConnected = isConnected,
providerRows = providerRows,
modelCount = models.size,
onRefresh = viewModel::refreshModelCatalog,
refreshing = refreshing,
)
}
item {
ProviderSectionLabel(title = "Connected providers")
}
item {
if (!isConnected && providerRows.isEmpty()) {
ClawEmptyState(title = "Gateway offline", body = "Connect your Gateway to load provider readiness.")
} else {
ProviderList(rows = providerRows, refreshing = refreshing)
}
}
errorText?.let { message ->
item {
ClawPanel {
Text(text = message, style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
}
}
}
}
}
internal data class ProviderRow(
val id: String,
val name: String,
val status: String,
val ready: Boolean,
val modelCount: Int,
)
/** Combines gateway auth-provider readiness with configured model providers. */
internal fun providerRows(
providers: List<GatewayModelProviderSummary>,
models: List<GatewayModelSummary>,
): List<ProviderRow> {
val modelCounts = models.groupingBy { it.provider }.eachCount()
val authRows =
providers
.map { provider ->
val ready = modelProviderReady(provider.status)
ProviderRow(
id = provider.id,
name = provider.displayName,
status = if (ready) "Ready" else "Needs attention",
ready = ready,
modelCount = modelCounts[provider.id] ?: 0,
)
}
val authProviderIds = authRows.mapTo(mutableSetOf()) { it.id.trim().lowercase() }
val configuredModelRows =
modelCounts.keys
.filter { provider -> provider.trim().lowercase() !in authProviderIds }
.map { provider ->
ProviderRow(
id = provider,
name = providerDisplayName(provider),
status = "Ready",
ready = true,
modelCount = modelCounts[provider] ?: 0,
)
}
return (authRows + configuredModelRows).sortedWith(compareBy(::providerPriority, { it.name.lowercase() }))
}
/** Normalizes gateway provider status strings into a ready/not-ready boolean. */
internal fun modelProviderReady(status: String): Boolean {
val normalized = status.trim().lowercase()
return normalized == "ok" ||
normalized == "ready" ||
normalized == "healthy" ||
normalized == "configured" ||
normalized == "static"
}
private fun providerPriority(row: ProviderRow): Int = providerPriority(row.id)
private fun providerPriority(provider: String): Int =
when (provider.trim().lowercase()) {
"openai" -> 0
"anthropic" -> 1
"google" -> 2
"openrouter" -> 3
"ollama", "ollama-local" -> 4
"codex" -> 5
else -> 100
}
@Composable
private fun ProviderList(
rows: List<ProviderRow>,
refreshing: Boolean,
) {
ClawPanel(contentPadding = PaddingValues(horizontal = 0.dp, vertical = 0.dp)) {
Column {
if (rows.isEmpty()) {
ProviderListRow(
ProviderRow(
id = "loading",
name = "Provider catalog",
status = if (refreshing) "Loading" else "No providers",
ready = false,
modelCount = 0,
),
)
} else {
val visibleRows = rows.take(5)
visibleRows.forEachIndexed { index, row ->
ProviderListRow(row)
if (index != visibleRows.lastIndex) {
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
}
}
}
@Composable
private fun ProviderOverviewPanel(
isConnected: Boolean,
providerRows: List<ProviderRow>,
modelCount: Int,
refreshing: Boolean,
onRefresh: () -> Unit,
) {
val readyCount = providerRows.count { it.ready }
val needsSetupCount = providerRows.count { !it.ready }
ClawPanel(contentPadding = PaddingValues(horizontal = 12.dp, vertical = 12.dp)) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ProviderMetricTile(label = "Ready", value = readyCount.toString(), modifier = Modifier.weight(1f))
ProviderMetricTile(label = "Models", value = modelCount.toString(), modifier = Modifier.weight(1f))
ProviderMetricTile(label = "Needs", value = needsSetupCount.toString(), modifier = Modifier.weight(1f))
}
Text(
text = if (isConnected) "Refresh to recheck provider readiness from your Gateway." else "Connect your Gateway to view provider readiness.",
style = ClawTheme.type.body,
color = ClawTheme.colors.textMuted,
)
ClawSecondaryButton(text = if (refreshing) "Refreshing" else "Refresh", onClick = onRefresh, enabled = isConnected && !refreshing, modifier = Modifier.fillMaxWidth())
}
}
}
@Composable
private fun ProviderMetricTile(
label: String,
value: String,
modifier: Modifier = Modifier,
) {
Surface(
modifier = modifier,
shape = RoundedCornerShape(ClawTheme.radii.panel),
color = ClawTheme.colors.surface,
border = BorderStroke(1.dp, ClawTheme.colors.border),
contentColor = ClawTheme.colors.text,
) {
Column(modifier = Modifier.padding(horizontal = 9.dp, vertical = 8.dp), verticalArrangement = Arrangement.spacedBy(2.dp)) {
Text(text = value, style = ClawTheme.type.title, color = ClawTheme.colors.text, maxLines = 1)
Text(text = label, style = ClawTheme.type.caption, color = ClawTheme.colors.textMuted, maxLines = 1)
}
}
}
@Composable
private fun ProviderListRow(row: ProviderRow) {
Row(modifier = Modifier.fillMaxWidth().heightIn(min = 58.dp).padding(horizontal = 10.dp, vertical = 6.dp), verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(10.dp)) {
ProviderBadge(text = row.name)
Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(1.dp)) {
Text(text = row.name, style = ClawTheme.type.body, color = ClawTheme.colors.text, maxLines = 1)
Text(text = if (row.modelCount > 0) "${row.modelCount} models" else "No configured models", style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp), color = ClawTheme.colors.textMuted, maxLines = 1)
}
Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(5.dp)) {
Box(modifier = Modifier.size(4.5.dp).clip(CircleShape).background(if (row.ready) ClawTheme.colors.success else ClawTheme.colors.warning))
Text(text = row.status, style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp), color = ClawTheme.colors.textMuted, maxLines = 1)
}
}
}
@Composable
private fun ProviderBadge(text: String) {
Surface(modifier = Modifier.size(30.dp), shape = RoundedCornerShape(ClawTheme.radii.row), color = ClawTheme.colors.surfacePressed, border = BorderStroke(1.dp, ClawTheme.colors.border)) {
Box(contentAlignment = Alignment.Center) {
Text(text = providerInitials(text), style = ClawTheme.type.label, color = ClawTheme.colors.text, textAlign = TextAlign.Center)
}
}
}
private fun providerInitials(value: String): String =
value
.split(' ', '-', '_')
.filter { it.isNotBlank() }
.take(2)
.mapNotNull { it.firstOrNull()?.uppercaseChar()?.toString() }
.joinToString("")
.ifBlank { "AI" }
@Composable
private fun ProviderSectionLabel(title: String) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.SpaceBetween) {
Text(text = title.uppercase(), style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp), color = ClawTheme.colors.textMuted)
}
}
@Composable
private fun ProviderHeaderIconButton(
icon: ImageVector,
contentDescription: String,
outlined: Boolean = false,
onClick: () -> Unit,
) {
Surface(
onClick = onClick,
modifier = Modifier.size(ClawTheme.spacing.touchTarget),
shape = CircleShape,
color = Color.Transparent,
contentColor = ClawTheme.colors.text,
border = if (outlined) BorderStroke(1.dp, ClawTheme.colors.borderStrong) else null,
) {
Box(contentAlignment = Alignment.Center) {
Icon(imageVector = icon, contentDescription = contentDescription, modifier = Modifier.size(if (outlined) 17.dp else 20.dp))
}
}
}

View File

@@ -0,0 +1,21 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
/** Chooses onboarding or the authenticated app shell from persisted app state. */
@Composable
fun RootScreen(viewModel: MainViewModel) {
val onboardingCompleted by viewModel.onboardingCompleted.collectAsState()
if (!onboardingCompleted) {
OnboardingFlow(viewModel = viewModel, modifier = Modifier.fillMaxSize())
return
}
ShellScreen(viewModel = viewModel, modifier = Modifier.fillMaxSize())
}

View File

@@ -0,0 +1,340 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawEmptyState
import ai.openclaw.app.ui.design.ClawPlainIconButton
import ai.openclaw.app.ui.design.ClawPrimaryButton
import ai.openclaw.app.ui.design.ClawScaffold
import ai.openclaw.app.ui.design.ClawTheme
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.WindowInsets
import androidx.compose.foundation.layout.WindowInsetsSides
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.only
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.safeDrawing
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.KeyboardArrowDown
import androidx.compose.material.icons.filled.Search
import androidx.compose.material.icons.filled.StarBorder
import androidx.compose.material.icons.filled.Storage
import androidx.compose.material.icons.filled.SwapVert
import androidx.compose.material.icons.outlined.AccessTime
import androidx.compose.material.icons.outlined.ChatBubbleOutline
import androidx.compose.material.icons.outlined.MicNone
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
/** Session browser for recent and current chat sessions. */
@Composable
internal fun SessionsScreen(
viewModel: MainViewModel,
onOpenCommand: () -> Unit,
onOpenChat: () -> Unit,
) {
val sessions by viewModel.chatSessions.collectAsState()
val chatSessionKey by viewModel.chatSessionKey.collectAsState()
val isConnected by viewModel.isConnected.collectAsState()
var filter by rememberSaveable { mutableStateOf(SessionFilter.Recent) }
var compactLayout by rememberSaveable { mutableStateOf(false) }
var recentFirst by rememberSaveable { mutableStateOf(true) }
val visibleSessions =
sessions
.let { rows ->
when (filter) {
SessionFilter.Recent -> rows
SessionFilter.Current -> rows.filter { it.key == chatSessionKey }
}
}.let { rows ->
if (recentFirst) {
rows.sortedByDescending { it.updatedAtMs ?: 0L }
} else {
rows.sortedBy { it.updatedAtMs ?: 0L }
}
}
LaunchedEffect(isConnected) {
if (isConnected) {
// Sessions are cheap to refresh on entry; subsequent sorting/filtering is
// local to avoid re-querying while the user explores the list.
viewModel.refreshChatSessions(limit = 200)
}
}
ClawScaffold(
contentPadding = PaddingValues(start = 16.dp, top = 10.dp, end = 16.dp, bottom = 4.dp),
contentWindowInsets = WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal),
) {
LazyColumn(
modifier = Modifier.fillMaxSize(),
verticalArrangement = Arrangement.spacedBy(9.dp),
contentPadding = PaddingValues(bottom = 4.dp),
) {
item {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(text = "Sessions", style = ClawTheme.type.display.copy(fontSize = 24.sp, lineHeight = 28.sp), color = ClawTheme.colors.text, modifier = Modifier.weight(1f))
ClawPlainIconButton(icon = Icons.Default.Search, contentDescription = "Search sessions", onClick = onOpenCommand)
ClawPlainIconButton(icon = Icons.Default.SwapVert, contentDescription = "Reverse session sort", onClick = { recentFirst = !recentFirst })
}
}
item {
Row(horizontalArrangement = Arrangement.spacedBy(5.dp)) {
FilterPill(text = "Recent", icon = Icons.Outlined.AccessTime, active = filter == SessionFilter.Recent, onClick = { filter = SessionFilter.Recent })
FilterPill(text = "Current", icon = Icons.Outlined.MicNone, active = filter == SessionFilter.Current, showDot = sessions.any { it.key == chatSessionKey }, onClick = { filter = SessionFilter.Current })
}
}
item {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.SpaceBetween, verticalAlignment = Alignment.CenterVertically) {
Row(
modifier =
Modifier
.clip(RoundedCornerShape(ClawTheme.radii.row))
.clickable { recentFirst = !recentFirst }
.padding(horizontal = 2.dp, vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
Text(text = "Sort: ${if (recentFirst) "Newest" else "Oldest"}", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
Icon(imageVector = Icons.Default.KeyboardArrowDown, contentDescription = null, modifier = Modifier.size(11.dp), tint = ClawTheme.colors.textMuted)
}
SessionOutlineIconButton(icon = Icons.Default.Storage, contentDescription = "Toggle session layout", onClick = { compactLayout = !compactLayout })
}
}
item {
Text(text = if (compactLayout) "Layout: Compact" else "Layout: Detailed", style = ClawTheme.type.caption, color = ClawTheme.colors.textSubtle)
}
if (visibleSessions.isEmpty()) {
item {
Box(
modifier = Modifier.fillParentMaxHeight(0.56f).fillMaxWidth(),
contentAlignment = Alignment.Center,
) {
ClawEmptyState(
title = emptySessionTitle(filter),
body = emptySessionBody(filter),
action = { ClawPrimaryButton(text = "Start Chat", onClick = onOpenChat) },
)
}
}
} else {
items(visibleSessions, key = { it.key }) { session ->
val active = session.key == chatSessionKey
SessionRow(
title = displaySessionTitle(session.displayName),
subtitle = if (active) "Current session" else "OpenClaw session",
metadata = session.updatedAtMs?.let(::relativeSessionTime) ?: "now",
active = active,
compact = compactLayout,
onClick = {
viewModel.switchChatSession(session.key)
onOpenChat()
},
)
}
}
}
}
}
@Composable
private fun FilterPill(
text: String,
icon: ImageVector? = null,
active: Boolean = false,
showDot: Boolean = false,
dropdown: Boolean = false,
onClick: (() -> Unit)? = null,
) {
Surface(
onClick = onClick ?: {},
enabled = onClick != null,
shape = RoundedCornerShape(7.dp),
color = if (active) ClawTheme.colors.surfaceRaised else Color.Transparent,
contentColor = ClawTheme.colors.text,
border = BorderStroke(1.dp, if (active) ClawTheme.colors.borderStrong else ClawTheme.colors.border),
) {
Row(
modifier = Modifier.padding(horizontal = 6.dp, vertical = 3.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
icon?.let { Icon(imageVector = it, contentDescription = null, modifier = Modifier.size(12.dp), tint = ClawTheme.colors.text) }
Text(text = text, style = ClawTheme.type.label, color = ClawTheme.colors.text, maxLines = 1)
if (showDot) {
Box(modifier = Modifier.size(4.dp).clip(CircleShape).background(ClawTheme.colors.success))
}
if (dropdown) {
Icon(imageVector = Icons.Default.KeyboardArrowDown, contentDescription = null, modifier = Modifier.size(11.dp), tint = ClawTheme.colors.textMuted)
}
}
}
}
@Composable
private fun SessionRow(
title: String,
subtitle: String,
metadata: String,
active: Boolean,
compact: Boolean,
onClick: () -> Unit,
) {
Surface(onClick = onClick, color = Color.Transparent, contentColor = ClawTheme.colors.text) {
Column {
Row(
modifier = Modifier.fillMaxWidth().heightIn(min = 58.dp).padding(vertical = 5.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(7.dp),
) {
Surface(
modifier = Modifier.size(30.dp),
shape = CircleShape,
color = Color.Transparent,
border = BorderStroke(1.dp, ClawTheme.colors.borderStrong),
) {
Box(contentAlignment = Alignment.Center) {
Icon(
imageVector = if (active) Icons.Default.StarBorder else Icons.Outlined.ChatBubbleOutline,
contentDescription = null,
modifier = Modifier.size(15.dp),
tint = ClawTheme.colors.text,
)
}
}
Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(1.5.dp)) {
Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(4.dp)) {
Text(
text = title,
style = ClawTheme.type.body,
color = ClawTheme.colors.text,
modifier = Modifier.weight(1f),
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
if (active) {
Box(modifier = Modifier.size(3.5.dp).clip(CircleShape).background(ClawTheme.colors.success))
}
}
if (!compact) {
Text(text = subtitle, style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp), color = ClawTheme.colors.textMuted, maxLines = 1)
Row(horizontalArrangement = Arrangement.spacedBy(4.dp)) {
SessionMiniTag(text = "Workspace")
SessionMiniTag(text = if (active) "Current" else "OpenClaw")
}
}
}
Column(horizontalAlignment = Alignment.End, verticalArrangement = Arrangement.spacedBy(5.dp)) {
Icon(imageVector = Icons.Outlined.ChatBubbleOutline, contentDescription = null, modifier = Modifier.size(13.dp), tint = ClawTheme.colors.textMuted)
Text(text = metadata, style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp), color = ClawTheme.colors.textMuted, maxLines = 1)
}
}
HorizontalDivider(color = ClawTheme.colors.border, thickness = 1.dp)
}
}
}
@Composable
private fun SessionOutlineIconButton(
icon: ImageVector,
contentDescription: String,
onClick: () -> Unit,
) {
Surface(
onClick = onClick,
modifier = Modifier.size(ClawTheme.spacing.touchTarget),
shape = RoundedCornerShape(7.dp),
color = Color.Transparent,
contentColor = ClawTheme.colors.text,
border = BorderStroke(1.dp, ClawTheme.colors.borderStrong),
) {
Box(contentAlignment = Alignment.Center) {
Icon(imageVector = icon, contentDescription = contentDescription, modifier = Modifier.size(14.dp))
}
}
}
@Composable
private fun SessionMiniTag(text: String) {
Surface(
shape = RoundedCornerShape(5.dp),
color = Color.Transparent,
border = BorderStroke(1.dp, ClawTheme.colors.border),
contentColor = ClawTheme.colors.textMuted,
) {
Text(text = text, modifier = Modifier.padding(horizontal = 4.dp, vertical = 0.5.dp), style = ClawTheme.type.caption.copy(fontSize = 12.5.sp, lineHeight = 16.sp), maxLines = 1)
}
}
private enum class SessionFilter {
Recent,
Current,
}
/** Empty-state title selected by the active session browser filter. */
private fun emptySessionTitle(filter: SessionFilter): String =
when (filter) {
SessionFilter.Recent -> "No sessions yet"
SessionFilter.Current -> "No current session"
}
/** Empty-state body selected by the active session browser filter. */
private fun emptySessionBody(filter: SessionFilter): String =
when (filter) {
SessionFilter.Recent -> "Start a new conversation and it will show up here."
SessionFilter.Current -> "Open Chat to start or resume the current session."
}
/** Formats session timestamps for compact mobile metadata. */
private fun relativeSessionTime(updatedAtMs: Long): String {
val deltaMs = (System.currentTimeMillis() - updatedAtMs).coerceAtLeast(0L)
val minutes = deltaMs / 60_000L
if (minutes < 1) return "now"
if (minutes < 60) return "${minutes}m"
val hours = minutes / 60
if (hours < 24) return "${hours}h"
return "${hours / 24}d"
}
/** Falls back to the canonical main-session label when gateway display names are blank. */
private fun displaySessionTitle(displayName: String?): String = displayName?.takeIf { it.isNotBlank() } ?: "Main session"

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,90 @@
package ai.openclaw.app.ui
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.saveable.listSaver
import androidx.compose.runtime.setValue
/**
* Shell navigation state: the visible tab, the open settings route, and where Back
* returns after a cross-tab detail open. All tab/route transitions go through this
* class so Back semantics stay consistent across every entry point.
*/
internal class ShellNavigation(
activeTab: Tab = Tab.Overview,
settingsRoute: SettingsRoute = SettingsRoute.Home,
returnTab: Tab? = null,
settingsRouteFromHome: Boolean = false,
) {
var activeTab by mutableStateOf(activeTab)
private set
var settingsRoute by mutableStateOf(settingsRoute)
private set
// Single-slot origin: Back from a cross-tab detail (settings route, Sessions,
// Providers) returns to the tab that opened it; deeper history intentionally
// collapses to Overview so the shell never accumulates a navigation stack.
private var returnTab by mutableStateOf(returnTab)
// Distinguishes a detail reached from the Settings Home list (Back unwinds to
// Home) from one opened cross-tab (Back leaves the Settings tab entirely).
private var settingsRouteFromHome by mutableStateOf(settingsRouteFromHome)
/** Tab-bar-style switch: Back from the selected tab returns to Overview. */
fun selectTab(tab: Tab) {
if (tab == Tab.Settings) settingsRoute = SettingsRoute.Home
settingsRouteFromHome = false
returnTab = null
activeTab = tab
}
/** Opens a settings route from another tab, remembering the origin for Back. */
fun openSettingsRoute(route: SettingsRoute) {
settingsRoute = route
settingsRouteFromHome = false
openDetailTab(Tab.Settings)
}
/** Opens a settings route from the Settings Home list; Back returns to Home. */
fun openSettingsRouteFromHome(route: SettingsRoute) {
settingsRoute = route
settingsRouteFromHome = true
}
/** Opens a detail tab (Sessions, Providers) from another tab, remembering the origin for Back. */
fun openDetailTab(tab: Tab) {
if (activeTab != tab) returnTab = activeTab
activeTab = tab
}
/** Unwinds one Back step: settings detail to Home or origin, otherwise tab to origin or Overview. */
fun back() {
if (activeTab == Tab.Settings && settingsRoute != SettingsRoute.Home) {
settingsRoute = SettingsRoute.Home
if (settingsRouteFromHome) {
settingsRouteFromHome = false
return
}
}
activeTab = returnTab ?: Tab.Overview
returnTab = null
}
companion object {
/** Persists shell navigation across process death for rememberSaveable. */
val Saver =
listSaver<ShellNavigation, String>(
save = { nav ->
listOf(nav.activeTab.name, nav.settingsRoute.name, nav.returnTab?.name.orEmpty(), nav.settingsRouteFromHome.toString())
},
restore = { saved ->
ShellNavigation(
activeTab = Tab.valueOf(saved[0]),
settingsRoute = SettingsRoute.valueOf(saved[1]),
returnTab = saved[2].takeIf { it.isNotEmpty() }?.let(Tab::valueOf),
settingsRouteFromHome = saved[3].toBoolean(),
)
},
)
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,276 @@
package ai.openclaw.app.ui
import ai.openclaw.app.GatewaySkillSummary
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.ui.design.ClawDetailRow
import ai.openclaw.app.ui.design.ClawListPanel
import ai.openclaw.app.ui.design.ClawPanel
import ai.openclaw.app.ui.design.ClawSecondaryButton
import ai.openclaw.app.ui.design.ClawStatus
import ai.openclaw.app.ui.design.ClawStatusPill
import ai.openclaw.app.ui.design.ClawTextBadge
import ai.openclaw.app.ui.design.ClawTheme
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material3.Icon
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
/** Settings screen for gateway skills and their readiness state. */
@Composable
internal fun SkillsSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val skillsSummary by viewModel.skillsSummary.collectAsState()
val skillsRefreshing by viewModel.skillsRefreshing.collectAsState()
val skillsErrorText by viewModel.skillsErrorText.collectAsState()
val isConnected by viewModel.isConnected.collectAsState()
val skills = skillsSummary.skills
val readyCount = skills.count { skillReady(it) }
val needsSetupCount = skills.count { skillNeedsSetup(it) }
var selectedSkillKey by remember { mutableStateOf<String?>(null) }
LaunchedEffect(isConnected) {
if (isConnected) {
viewModel.refreshSkills()
}
}
selectedSkillKey?.let { skillKey ->
val selectedSkill = skills.firstOrNull { it.skillKey == skillKey }
SkillDetailSettingsScreen(
skill = selectedSkill,
skillKey = skillKey,
isConnected = isConnected,
onBack = { selectedSkillKey = null },
)
return
}
SettingsDetailFrame(
title = "Skills",
subtitle = "Installed capabilities available to OpenClaw.",
icon = Icons.Default.Settings,
onBack = onBack,
) {
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Installed", skills.size.toString()),
SettingsMetric("Ready", readyCount.toString()),
SettingsMetric("Needs Setup", needsSetupCount.toString()),
),
)
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
ClawSecondaryButton(
text = if (skillsRefreshing) "Refreshing" else "Refresh",
onClick = viewModel::refreshSkills,
enabled = isConnected && !skillsRefreshing,
modifier = Modifier.weight(1f),
)
}
skillsErrorText?.let { errorText ->
ClawPanel {
Text(text = errorText, style = ClawTheme.type.body, color = ClawTheme.colors.warning)
}
}
when {
!isConnected ->
ClawPanel {
Text(text = "Connect the gateway to load skills.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
skills.isEmpty() ->
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(3.dp)) {
Text(text = "No skills installed.", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = "Skills installed on the gateway will appear here.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
else -> SkillsPanel(skills = skills, onSkillClick = { selectedSkillKey = it.skillKey })
}
}
}
@Composable
private fun SkillDetailSettingsScreen(
skill: GatewaySkillSummary?,
skillKey: String,
isConnected: Boolean,
onBack: () -> Unit,
) {
BackHandler(onBack = onBack)
SettingsDetailFrame(
title = skill?.name ?: skillKey,
subtitle = "Inspect installed skill capability and setup state.",
icon = Icons.Default.Settings,
onBack = onBack,
) {
skill?.let { summary ->
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Status", skillStatusText(summary)),
SettingsMetric("Source", skillSourceLabel(summary)),
SettingsMetric("Missing", summary.missingCount.toString()),
),
)
SkillSetupPanel(summary)
}
SkillDetailPanel(skill = skill, isConnected = isConnected)
}
}
@Composable
private fun SkillSetupPanel(skill: GatewaySkillSummary) {
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = "Setup", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = skillConfigurationText(skill), style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
}
@Composable
private fun SkillDetailPanel(
skill: GatewaySkillSummary?,
isConnected: Boolean,
) {
if (!isConnected) {
ClawPanel {
Text(text = "Connect the gateway to load skill details.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
return
}
if (skill == null) {
ClawPanel {
Text(text = "Skill detail is not available in the current skills status.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
return
}
SettingsMetricPanel(
rows =
listOf(
SettingsMetric("Skill Key", skill.skillKey),
SettingsMetric("Display", skill.name),
SettingsMetric("Source", skillSourceLabel(skill)),
SettingsMetric("Install Options", skill.installCount.toString()),
),
)
skill.description?.let { description ->
ClawPanel {
Column(verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = "Description", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = description, style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
}
}
@Composable
private fun SkillsPanel(
skills: List<GatewaySkillSummary>,
onSkillClick: (GatewaySkillSummary) -> Unit,
) {
ClawListPanel(items = skills) { skill ->
SkillListRow(skill = skill, onClick = { onSkillClick(skill) })
}
}
@Composable
private fun SkillListRow(
skill: GatewaySkillSummary,
onClick: () -> Unit,
) {
ClawDetailRow(
title = skill.name,
subtitle = skillSubtitle(skill),
modifier = Modifier.clickable(onClickLabel = "Open skill detail", onClick = onClick),
leading = { ClawTextBadge(text = skillBadge(skill)) },
trailing = {
Row(horizontalArrangement = Arrangement.spacedBy(6.dp)) {
ClawStatusPill(text = skillStatusText(skill), status = skillStatus(skill))
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = ClawTheme.colors.textSubtle,
)
}
},
)
}
private fun skillReady(skill: GatewaySkillSummary): Boolean = !skill.disabled && skill.eligible && skill.missingCount == 0
private fun skillNeedsSetup(skill: GatewaySkillSummary): Boolean = !skill.disabled && (skill.blockedByAllowlist || !skill.eligible || skill.missingCount > 0)
private fun skillStatusText(skill: GatewaySkillSummary): String =
when {
skill.disabled -> "Off"
skillNeedsSetup(skill) -> "Setup"
else -> "Ready"
}
private fun skillStatus(skill: GatewaySkillSummary): ClawStatus =
when {
skill.disabled -> ClawStatus.Neutral
skillNeedsSetup(skill) -> ClawStatus.Warning
else -> ClawStatus.Success
}
private fun skillSubtitle(skill: GatewaySkillSummary): String {
val issue =
when {
skill.disabled -> "Disabled"
skill.blockedByAllowlist -> "Blocked"
skill.missingCount > 0 -> "${skill.missingCount} missing"
!skill.eligible -> "Needs setup"
else -> null
}
return listOfNotNull(skill.description, skillSourceLabel(skill), issue).joinToString(" · ")
}
private fun skillConfigurationText(skill: GatewaySkillSummary): String =
when {
skill.disabled -> "This skill is disabled on the gateway. Android shows detail only; enable or configure it from desktop or CLI."
skill.blockedByAllowlist -> "This skill is blocked by the gateway allowlist. Android can inspect it, but allowlist changes stay on desktop or CLI."
skill.missingCount > 0 -> "This skill needs ${skill.missingCount} setup item(s). Android shows what is installed; setup/config changes stay on desktop or CLI."
!skill.eligible -> "This skill is installed but not currently eligible to run. Use desktop or CLI for configuration changes."
else -> "Ready on this gateway. Android detail is read-only; install, update, and configuration changes stay on desktop or CLI."
}
private fun skillSourceLabel(skill: GatewaySkillSummary): String =
when (skill.source) {
"openclaw-bundled" -> if (skill.bundled) "Built-in" else "Bundled"
"openclaw-managed" -> "Installed"
"openclaw-workspace" -> "Workspace"
"openclaw-extra" -> "Extra"
else -> "Skill"
}
private fun skillBadge(skill: GatewaySkillSummary): String {
skill.emoji?.let { return it }
return skill.name
.split(' ', '-', '_')
.filter { it.isNotBlank() }
.take(2)
.mapNotNull { it.firstOrNull()?.uppercaseChar()?.toString() }
.joinToString("")
.ifBlank { "S" }
}

View File

@@ -0,0 +1,137 @@
package ai.openclaw.app.ui
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
/**
* Full-screen talk-mode presence indicator with pulsing rings and status fallback text.
*/
@Composable
fun TalkOrbOverlay(
seamColor: Color,
statusText: String,
isListening: Boolean,
isSpeaking: Boolean,
modifier: Modifier = Modifier,
) {
val transition = rememberInfiniteTransition(label = "talk-orb")
val t by
transition.animateFloat(
initialValue = 0f,
targetValue = 1f,
animationSpec =
infiniteRepeatable(
animation = tween(durationMillis = 1500, easing = LinearEasing),
repeatMode = RepeatMode.Restart,
),
label = "pulse",
)
val trimmed = statusText.trim()
val showStatus = trimmed.isNotEmpty() && trimmed != "Off"
val phase =
when {
isSpeaking -> "Speaking"
isListening -> "Listening"
else -> "Thinking"
}
Column(
modifier = modifier.padding(24.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Box(contentAlignment = Alignment.Center) {
Canvas(modifier = Modifier.size(360.dp)) {
val center = this.center
val baseRadius = size.minDimension * 0.30f
val ring1 = 1.05f + (t * 0.25f)
val ring2 = 1.20f + (t * 0.55f)
val ringAlpha1 = (1f - t) * 0.34f
val ringAlpha2 = (1f - t) * 0.22f
drawCircle(
color = seamColor.copy(alpha = ringAlpha1),
radius = baseRadius * ring1,
center = center,
style = Stroke(width = 3.dp.toPx()),
)
drawCircle(
color = seamColor.copy(alpha = ringAlpha2),
radius = baseRadius * ring2,
center = center,
style = Stroke(width = 3.dp.toPx()),
)
drawCircle(
brush =
Brush.radialGradient(
colors =
listOf(
seamColor.copy(alpha = 0.92f),
seamColor.copy(alpha = 0.40f),
Color.Black.copy(alpha = 0.56f),
),
center = center,
radius = baseRadius * 1.35f,
),
radius = baseRadius,
center = center,
)
drawCircle(
color = seamColor.copy(alpha = 0.34f),
radius = baseRadius,
center = center,
style = Stroke(width = 1.dp.toPx()),
)
}
}
if (showStatus) {
Surface(
color = Color.Black.copy(alpha = 0.40f),
shape = CircleShape,
) {
Text(
text = trimmed,
modifier = Modifier.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White.copy(alpha = 0.92f),
style = MaterialTheme.typography.labelLarge,
fontWeight = FontWeight.SemiBold,
)
}
} else {
Text(
text = phase,
color = Color.White.copy(alpha = 0.80f),
style = MaterialTheme.typography.labelLarge,
fontWeight = FontWeight.SemiBold,
)
}
}
}

View File

@@ -0,0 +1,173 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.NodeRuntime
import ai.openclaw.app.ui.design.ClawPlainIconButton
import ai.openclaw.app.ui.design.ClawScaffold
import ai.openclaw.app.ui.design.ClawTheme
import android.annotation.SuppressLint
import android.view.View
import android.webkit.WebSettings
import android.webkit.WebView
import android.webkit.WebViewClient
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.imePadding
import androidx.compose.foundation.layout.padding
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material.icons.outlined.Terminal
import androidx.compose.material3.Icon
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.key
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.webkit.WebSettingsCompat
import androidx.webkit.WebViewCompat
import androidx.webkit.WebViewFeature
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
/**
* Full-height terminal surface: embeds the gateway-served terminal-only
* Control UI document (`/?view=terminal`, the same ghostty-web surface the
* desktop Control UI uses) for the currently connected gateway.
*/
@Composable
internal fun TerminalSettingsScreen(
viewModel: MainViewModel,
onBack: () -> Unit,
) {
val isConnected by viewModel.isConnected.collectAsState()
val controlPage by viewModel.gatewayControlPage.collectAsState()
ClawScaffold(
contentPadding = PaddingValues(start = ClawTheme.spacing.lg, top = 14.dp, end = ClawTheme.spacing.lg, bottom = 6.dp),
) {
Column(modifier = Modifier.fillMaxSize().imePadding(), verticalArrangement = Arrangement.spacedBy(10.dp)) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(9.dp)) {
ClawPlainIconButton(
icon = Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = "Back",
onClick = onBack,
)
Text(text = "Terminal", style = ClawTheme.type.title, color = ClawTheme.colors.text, modifier = Modifier.weight(1f), maxLines = 1, overflow = TextOverflow.Ellipsis)
Icon(imageVector = Icons.Outlined.Terminal, contentDescription = null, tint = ClawTheme.colors.textMuted)
}
Box(modifier = Modifier.fillMaxWidth().weight(1f)) {
val page = controlPage
if (isConnected && page != null) {
// Recreate the WebView only when the gateway page or credentials
// change; recompositions must not restart live shell sessions.
key(page) {
TerminalWebView(page = page, modifier = Modifier.fillMaxSize())
}
} else {
Column(modifier = Modifier.fillMaxWidth().padding(top = 48.dp), horizontalAlignment = Alignment.CenterHorizontally, verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(text = "Terminal needs a connected gateway", style = ClawTheme.type.section, color = ClawTheme.colors.text)
Text(text = "Connect to your gateway to open a shell in the agent workspace.", style = ClawTheme.type.body, color = ClawTheme.colors.textMuted)
}
}
}
}
}
}
/** Minimal WebView host for the terminal page; no script bridges needed. */
@SuppressLint("SetJavaScriptEnabled")
// Deprecated file-URL settings are still force-disabled defensively, like the canvas host.
@Suppress("DEPRECATION")
@Composable
private fun TerminalWebView(
page: NodeRuntime.GatewayControlPage,
modifier: Modifier = Modifier,
) {
val context = LocalContext.current
val webViewRef = remember { arrayOfNulls<WebView>(1) }
DisposableEffect(Unit) {
onDispose {
val webView = webViewRef[0] ?: return@onDispose
webView.stopLoading()
webView.destroy()
webViewRef[0] = null
}
}
AndroidView(
modifier = modifier,
factory = {
val webView = WebView(context)
val webSettings = webView.settings
webSettings.setAllowContentAccess(false)
webSettings.setAllowFileAccess(false)
webSettings.setAllowFileAccessFromFileURLs(false)
webSettings.setAllowUniversalAccessFromFileURLs(false)
webSettings.setSafeBrowsingEnabled(true)
webSettings.javaScriptEnabled = true
webSettings.domStorageEnabled = true
webSettings.mixedContentMode = WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE
webSettings.builtInZoomControls = false
webSettings.displayZoomControls = false
webSettings.setSupportZoom(false)
// targetSdk 33+ ignores Force Dark APIs; the terminal page owns its own
// dark palette, so opt out of algorithmic darkening like the canvas host.
if (WebViewFeature.isFeatureSupported(WebViewFeature.ALGORITHMIC_DARKENING)) {
WebSettingsCompat.setAlgorithmicDarkeningAllowed(webSettings, false)
}
webView.overScrollMode = View.OVER_SCROLL_NEVER
webView.webViewClient = WebViewClient()
installTerminalAuthScript(webView, page)
webView.loadUrl("${page.baseUrl}/?view=terminal")
webViewRef[0] = webView
webView
},
)
}
/**
* Hands the gateway credentials to the Control UI via its
* `__OPENCLAW_NATIVE_CONTROL_AUTH__` startup contract (the same mechanism the
* macOS Dashboard and iOS Terminal hub use), origin-locked by the platform's
* allowed-origin rules, so the token never appears in the page URL. Without
* document-start script support the page simply shows its own login gate.
*/
private fun installTerminalAuthScript(
webView: WebView,
page: NodeRuntime.GatewayControlPage,
) {
if (page.token == null && page.password == null) return
if (!WebViewFeature.isFeatureSupported(WebViewFeature.DOCUMENT_START_SCRIPT)) return
val gatewayUrl = page.baseUrl.replaceFirst("http", "ws")
val payload =
buildJsonObject {
put("gatewayUrl", gatewayUrl)
page.token?.let { put("token", it) }
page.password?.let { put("password", it) }
}
val script =
"""
(() => {
try {
Object.defineProperty(window, "__OPENCLAW_NATIVE_CONTROL_AUTH__", {
value: $payload,
configurable: true,
});
} catch (e) {}
})();
""".trimIndent()
WebViewCompat.addDocumentStartJavaScript(webView, script, setOf(page.baseUrl))
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,528 @@
package ai.openclaw.app.ui
import ai.openclaw.app.MainViewModel
import ai.openclaw.app.VoiceCaptureMode
import ai.openclaw.app.voice.VoiceConversationEntry
import ai.openclaw.app.voice.VoiceConversationRole
import android.Manifest
import android.app.Activity
import android.content.Context
import android.content.ContextWrapper
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.provider.Settings
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.WindowInsets
import androidx.compose.foundation.layout.WindowInsetsSides
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.imePadding
import androidx.compose.foundation.layout.only
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.safeDrawing
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.windowInsetsPadding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.lazy.rememberLazyListState
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.VolumeOff
import androidx.compose.material.icons.automirrored.filled.VolumeUp
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.MicOff
import androidx.compose.material.icons.filled.RecordVoiceOver
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.IconButtonDefaults
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.compose.LocalLifecycleOwner
import kotlin.math.max
/**
* Voice tab that switches between push-to-send mic capture and continuous Talk Mode.
*/
@Composable
fun VoiceTabScreen(viewModel: MainViewModel) {
val context = LocalContext.current
val lifecycleOwner = LocalLifecycleOwner.current
val activity = remember(context) { context.findActivity() }
val listState = rememberLazyListState()
val gatewayStatus by viewModel.statusText.collectAsState()
val voiceCaptureMode by viewModel.voiceCaptureMode.collectAsState()
val micEnabled by viewModel.micEnabled.collectAsState()
val micCooldown by viewModel.micCooldown.collectAsState()
val speakerEnabled by viewModel.speakerEnabled.collectAsState()
val micStatusText by viewModel.micStatusText.collectAsState()
val micLiveTranscript by viewModel.micLiveTranscript.collectAsState()
val micQueuedMessages by viewModel.micQueuedMessages.collectAsState()
val micConversation by viewModel.micConversation.collectAsState()
val micInputLevel by viewModel.micInputLevel.collectAsState()
val micIsSending by viewModel.micIsSending.collectAsState()
val talkModeEnabled by viewModel.talkModeEnabled.collectAsState()
val talkModeListening by viewModel.talkModeListening.collectAsState()
val talkModeSpeaking by viewModel.talkModeSpeaking.collectAsState()
val talkModeConversation by viewModel.talkModeConversation.collectAsState()
val activeConversation = if (voiceCaptureMode == VoiceCaptureMode.TalkMode) talkModeConversation else micConversation
val hasStreamingAssistant = activeConversation.any { it.role == VoiceConversationRole.Assistant && it.isStreaming }
val showThinkingBubble = micIsSending && !hasStreamingAssistant
var hasMicPermission by remember { mutableStateOf(context.hasRecordAudioPermission()) }
var pendingVoicePermissionAction by remember { mutableStateOf<PendingVoicePermissionAction?>(null) }
DisposableEffect(lifecycleOwner, context) {
val observer =
LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME) {
hasMicPermission = context.hasRecordAudioPermission()
}
}
lifecycleOwner.lifecycle.addObserver(observer)
onDispose {
lifecycleOwner.lifecycle.removeObserver(observer)
// Manual mic is tab-scoped; Talk Mode is user-enabled and can continue elsewhere.
viewModel.setVoiceScreenActive(false)
}
}
val requestMicPermission =
rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
hasMicPermission = granted
if (granted) {
when (pendingVoicePermissionAction) {
PendingVoicePermissionAction.ManualMic -> viewModel.setMicEnabled(true)
PendingVoicePermissionAction.TalkMode -> viewModel.setTalkModeEnabled(true)
null -> Unit
}
}
pendingVoicePermissionAction = null
}
LaunchedEffect(voiceCaptureMode, activeConversation.size, showThinkingBubble) {
val total = activeConversation.size + if (showThinkingBubble) 1 else 0
if (total > 0) {
listState.animateScrollToItem(total - 1)
}
}
Column(
modifier =
Modifier
.fillMaxSize()
.background(mobileBackgroundGradient)
.imePadding()
.windowInsetsPadding(WindowInsets.safeDrawing.only(WindowInsetsSides.Bottom))
.padding(horizontal = 20.dp, vertical = 14.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
LazyColumn(
state = listState,
modifier = Modifier.fillMaxWidth().weight(1f),
contentPadding = PaddingValues(vertical = 4.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
if (activeConversation.isEmpty() && !showThinkingBubble) {
item {
Box(
modifier = Modifier.fillParentMaxHeight().fillMaxWidth(),
contentAlignment = Alignment.Center,
) {
Column(
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
Icon(
imageVector = Icons.Default.Mic,
contentDescription = null,
modifier = Modifier.size(48.dp),
tint = mobileTextTertiary,
)
Text(
"Tap mic or Talk",
style = mobileHeadline,
color = mobileTextSecondary,
)
Text(
"Mic sends turns; Talk keeps the conversation open.",
style = mobileCallout,
color = mobileTextTertiary,
)
}
}
}
}
items(items = activeConversation, key = { it.id }) { entry ->
VoiceTurnBubble(entry = entry)
}
if (showThinkingBubble) {
item {
VoiceThinkingBubble()
}
}
}
Column(
modifier = Modifier.fillMaxWidth(),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
if (!micLiveTranscript.isNullOrBlank()) {
Surface(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
color = mobileAccentSoft,
border = BorderStroke(1.dp, mobileAccent.copy(alpha = 0.2f)),
) {
Text(
micLiveTranscript!!.trim(),
modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
style = mobileCallout,
color = mobileText,
)
}
}
// Mic button with input-reactive ring + speaker toggle
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.Center,
verticalAlignment = Alignment.CenterVertically,
) {
// Speaker toggle
Column(horizontalAlignment = Alignment.CenterHorizontally, verticalArrangement = Arrangement.spacedBy(4.dp)) {
IconButton(
onClick = { viewModel.setSpeakerEnabled(!speakerEnabled) },
modifier = Modifier.size(48.dp),
colors =
IconButtonDefaults.iconButtonColors(
containerColor = if (speakerEnabled) mobileSurface else mobileDangerSoft,
),
) {
Icon(
imageVector = if (speakerEnabled) Icons.AutoMirrored.Filled.VolumeUp else Icons.AutoMirrored.Filled.VolumeOff,
contentDescription = if (speakerEnabled) "Mute speaker" else "Unmute speaker",
modifier = Modifier.size(22.dp),
tint = if (speakerEnabled) mobileTextSecondary else mobileDanger,
)
}
Text(
if (speakerEnabled) "Speaker" else "Muted",
style = mobileCaption2,
color = if (speakerEnabled) mobileTextTertiary else mobileDanger,
)
}
// Ring size = 68dp base + up to 22dp driven by mic input level.
// The outer Box is fixed at 90dp (max ring size) so the ring never shifts the button.
Box(
modifier = Modifier.padding(horizontal = 16.dp).size(90.dp),
contentAlignment = Alignment.Center,
) {
if (micEnabled) {
val ringLevel = micInputLevel.coerceIn(0f, 1f)
val ringSize = 68.dp + (22.dp * max(ringLevel, 0.05f))
Box(
modifier =
Modifier
.size(ringSize)
.background(mobileAccent.copy(alpha = 0.12f + 0.14f * ringLevel), CircleShape),
)
}
Button(
onClick = {
if (micCooldown) return@Button
if (micEnabled) {
viewModel.setMicEnabled(false)
return@Button
}
if (hasMicPermission) {
viewModel.setMicEnabled(true)
} else {
pendingVoicePermissionAction = PendingVoicePermissionAction.ManualMic
requestMicPermission.launch(Manifest.permission.RECORD_AUDIO)
}
},
enabled = !micCooldown,
shape = CircleShape,
contentPadding = PaddingValues(0.dp),
modifier = Modifier.size(60.dp),
colors =
ButtonDefaults.buttonColors(
containerColor =
if (micCooldown) {
mobileTextSecondary
} else if (micEnabled) {
mobileDanger
} else {
mobileAccent
},
contentColor = Color.White,
disabledContainerColor = mobileTextSecondary,
disabledContentColor = Color.White.copy(alpha = 0.5f),
),
) {
Icon(
imageVector = if (micEnabled) Icons.Default.MicOff else Icons.Default.Mic,
contentDescription = if (micEnabled) "Turn microphone off" else "Turn microphone on",
modifier = Modifier.size(24.dp),
)
}
}
Column(horizontalAlignment = Alignment.CenterHorizontally, verticalArrangement = Arrangement.spacedBy(4.dp)) {
IconButton(
onClick = {
if (talkModeEnabled) {
viewModel.setTalkModeEnabled(false)
return@IconButton
}
if (hasMicPermission) {
viewModel.setTalkModeEnabled(true)
} else {
pendingVoicePermissionAction = PendingVoicePermissionAction.TalkMode
requestMicPermission.launch(Manifest.permission.RECORD_AUDIO)
}
},
modifier = Modifier.size(48.dp),
colors =
IconButtonDefaults.iconButtonColors(
containerColor = if (talkModeEnabled) mobileSuccessSoft else mobileSurface,
),
) {
Icon(
imageVector = Icons.Default.RecordVoiceOver,
contentDescription = if (talkModeEnabled) "Turn Talk Mode off" else "Turn Talk Mode on",
modifier = Modifier.size(22.dp),
tint = if (talkModeEnabled) mobileSuccess else mobileTextSecondary,
)
}
Spacer(modifier = Modifier.height(4.dp))
Text(
if (talkModeEnabled) "Talk on" else "Talk",
style = mobileCaption2,
color = if (talkModeEnabled) mobileSuccess else mobileTextTertiary,
)
}
}
// Status + labels
val queueCount = micQueuedMessages.size
val stateText =
when {
voiceCaptureMode == VoiceCaptureMode.TalkMode && talkModeSpeaking -> "Talk speaking"
voiceCaptureMode == VoiceCaptureMode.TalkMode && talkModeListening -> "Talk listening"
voiceCaptureMode == VoiceCaptureMode.TalkMode -> "Talk on"
micEnabled || micIsSending || micCooldown -> micStatusText
queueCount > 0 -> "$queueCount queued"
else -> "Mic off"
}
val stateColor =
when {
voiceCaptureMode == VoiceCaptureMode.TalkMode -> mobileSuccess
micEnabled -> mobileSuccess
micIsSending -> mobileAccent
else -> mobileTextSecondary
}
Surface(
shape = RoundedCornerShape(999.dp),
color = if (micEnabled || talkModeEnabled) mobileSuccessSoft else mobileSurface,
border = BorderStroke(1.dp, if (micEnabled || talkModeEnabled) mobileSuccess.copy(alpha = 0.3f) else mobileBorder),
) {
Text(
"$gatewayStatus · $stateText",
style = mobileCallout.copy(fontWeight = FontWeight.SemiBold),
color = stateColor,
modifier = Modifier.padding(horizontal = 14.dp, vertical = 6.dp),
)
}
if (!hasMicPermission) {
val showRationale =
if (activity == null) {
false
} else {
ActivityCompat.shouldShowRequestPermissionRationale(activity, Manifest.permission.RECORD_AUDIO)
}
Text(
if (showRationale) {
"Microphone permission is required for voice mode."
} else {
"Microphone blocked. Open app settings to enable it."
},
style = mobileCaption1,
color = mobileWarning,
textAlign = TextAlign.Center,
)
Button(
onClick = { openAppSettings(context) },
shape = RoundedCornerShape(12.dp),
colors = ButtonDefaults.buttonColors(containerColor = mobileSurfaceStrong, contentColor = mobileText),
) {
Text("Open settings", style = mobileCallout.copy(fontWeight = FontWeight.SemiBold))
}
}
}
}
}
/**
* Permission continuation captured before Android's runtime permission dialog suspends the action.
*/
private enum class PendingVoicePermissionAction {
ManualMic,
TalkMode,
}
/**
* Renders one transcript turn, preserving side and color by speaker role.
*/
@Composable
private fun VoiceTurnBubble(entry: VoiceConversationEntry) {
val isUser = entry.role == VoiceConversationRole.User
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = if (isUser) Arrangement.End else Arrangement.Start,
) {
Surface(
modifier = Modifier.fillMaxWidth(0.90f),
shape = RoundedCornerShape(12.dp),
color = if (isUser) mobileAccentSoft else mobileCardSurface,
border = BorderStroke(1.dp, if (isUser) mobileAccent else mobileBorderStrong),
) {
Column(
modifier = Modifier.fillMaxWidth().padding(horizontal = 11.dp, vertical = 8.dp),
verticalArrangement = Arrangement.spacedBy(3.dp),
) {
Text(
if (isUser) "You" else "OpenClaw",
style = mobileCaption2.copy(fontWeight = FontWeight.SemiBold, letterSpacing = 0.6.sp),
color = if (isUser) mobileAccent else mobileTextSecondary,
)
Text(
if (entry.isStreaming && entry.text.isBlank()) "Listening response…" else entry.text,
style = mobileCallout,
color = mobileText,
)
}
}
}
}
/**
* Placeholder assistant turn shown while a manual mic request is queued but not streaming yet.
*/
@Composable
private fun VoiceThinkingBubble() {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) {
Surface(
modifier = Modifier.fillMaxWidth(0.68f),
shape = RoundedCornerShape(12.dp),
color = mobileCardSurface,
border = BorderStroke(1.dp, mobileBorderStrong),
) {
Row(
modifier = Modifier.padding(horizontal = 11.dp, vertical = 8.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
ThinkingDots(color = mobileTextSecondary)
Text("OpenClaw is thinking…", style = mobileCallout, color = mobileTextSecondary)
}
}
}
}
/**
* Static dot cluster used by VoiceThinkingBubble to avoid starting another animation loop.
*/
@Composable
private fun ThinkingDots(color: Color) {
Row(horizontalArrangement = Arrangement.spacedBy(5.dp), verticalAlignment = Alignment.CenterVertically) {
ThinkingDot(alpha = 0.38f, color = color)
ThinkingDot(alpha = 0.62f, color = color)
ThinkingDot(alpha = 0.90f, color = color)
}
}
@Composable
private fun ThinkingDot(
alpha: Float,
color: Color,
) {
Surface(
modifier = Modifier.size(6.dp).alpha(alpha),
shape = CircleShape,
color = color,
) {}
}
/**
* Checks RECORD_AUDIO using ContextCompat so wrapped activity contexts behave the same.
*/
private fun Context.hasRecordAudioPermission(): Boolean =
(
ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
)
/**
* Walks ContextWrappers until an Activity is found for permission rationale checks.
*/
private fun Context.findActivity(): Activity? =
when (this) {
is Activity -> this
is ContextWrapper -> baseContext.findActivity()
else -> null
}
/**
* Opens this app's settings page after Android reports the mic permission as blocked.
*/
private fun openAppSettings(context: Context) {
val intent =
Intent(
Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", context.packageName, null),
)
context.startActivity(intent)
}

View File

@@ -0,0 +1,41 @@
package ai.openclaw.app.ui.chat
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.graphics.ImageBitmap
import androidx.compose.ui.graphics.asImageBitmap
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
/** Compose state for async base64 image decoding. */
internal data class Base64ImageState(
val image: ImageBitmap?,
val failed: Boolean,
)
/** Decodes a base64 image off the UI thread and reports failure state. */
@Composable
internal fun rememberBase64ImageState(base64: String): Base64ImageState {
var image by remember(base64) { mutableStateOf<ImageBitmap?>(null) }
var failed by remember(base64) { mutableStateOf(false) }
LaunchedEffect(base64) {
failed = false
image =
withContext(Dispatchers.Default) {
try {
val bitmap = decodeBase64Bitmap(base64) ?: return@withContext null
bitmap.asImageBitmap()
} catch (_: Throwable) {
null
}
}
if (image == null) failed = true
}
return Base64ImageState(image = image, failed = failed)
}

View File

@@ -0,0 +1,61 @@
package ai.openclaw.app.ui.chat
import ai.openclaw.app.chat.ChatCommandEntry
import java.util.Locale
internal fun slashCommandQuery(input: String): String? {
val trimmed = input.trimStart()
if (!trimmed.startsWith("/")) return null
val token = trimmed.drop(1).takeWhile { !it.isWhitespace() }
return token.lowercase(Locale.US)
}
internal fun shouldShowSlashCommandMenu(input: String): Boolean = slashCommandQuery(input) != null
internal fun matchingSlashCommands(
input: String,
commands: List<ChatCommandEntry>,
limit: Int = 6,
): List<ChatCommandEntry> {
val query = slashCommandQuery(input) ?: return emptyList()
val uniqueCommands = commands.map { command -> command.withMatchedSlashAliasFirst(query) }.distinctBy { slashCommandText(it) }
val matches =
if (query.isEmpty()) {
uniqueCommands
} else {
uniqueCommands.filter { command ->
slashCommandPrefixes(command).any { prefix -> prefix.startsWith(query) }
}
}
return matches.take(limit)
}
internal fun slashCommandText(command: ChatCommandEntry): String {
command.textAliases
.firstOrNull { alias -> alias.startsWith("/") && alias.length > 1 }
?.let { return it }
val name = command.name.trim().removePrefix("/").takeIf { it.isNotEmpty() } ?: "help"
return "/$name"
}
internal fun slashCommandCompletion(command: ChatCommandEntry): String {
val text = slashCommandText(command)
return if (command.acceptsArgs) "$text " else text
}
private fun slashCommandPrefixes(command: ChatCommandEntry): List<String> =
buildList {
add(normalizedSlashCommandName(command.name))
command.textAliases.forEach { alias ->
add(normalizedSlashCommandName(alias))
}
}.filter { it.isNotEmpty() }
private fun ChatCommandEntry.withMatchedSlashAliasFirst(query: String): ChatCommandEntry {
if (query.isEmpty()) return this
val match = textAliases.firstOrNull { alias -> normalizedSlashCommandName(alias).startsWith(query) } ?: return this
if (textAliases.firstOrNull() == match) return this
return copy(textAliases = listOf(match) + textAliases.filterNot { it == match })
}
private fun normalizedSlashCommandName(value: String): String = value.trim().removePrefix("/").lowercase(Locale.US)

Some files were not shown because too many files have changed in this diff Show More