feat(#195): chat attachment support for files and images (#418)

* feat(#195): add chat attachment support for files and images

- Add Attachment data model with computed properties (isImage, formattedSize, fileExtension)
- Extend ChatMessage with optional attachments list
- Add Coil 2.7.0 for async image loading in Compose
- Extend ChatViewModel with pending attachment state, addAttachment/removeAttachment/clearAttachments
- Build attachment-prefixed message text (data URIs for images, file refs for others)
- Add paperclip button and file picker launcher (ActivityResultContracts.GetContent) to ChatScreen
- Add animated attachment preview chips with thumbnails before sending
- Render attachments inline in UserBubble (AsyncImage for images, file cards for others)
- Add string resources for attachment UI features

* fix(#195): rewrite attachment upload protocol to match Hermes backend RPC methods

BREAKS: removes old buildAttachmentText() that embedded base64 data URIs in prompt text.

Changes:
- Replace inline data-URI encoding with proper backend RPC flow:
  - image.attach_bytes for images (fire-and-forget, auto-queues to session)
  - file.attach for non-image files (awaited, returns @file: ref text)
- Add CompletableDeferred to PendingRequest so sendMessage can await RPC responses
- Add readContentUriBytes() to read file content from content:// URIs
- Add sendRpcAndAwait() suspend function for sequential RPC calls
- Add RpcCallException for awaited RPC error handling
- Suppress error UI for awaited RPCs (caller handles them directly)
- Keep ChatScreen/ChatBubble UI layer unchanged (file picker, previews, inline images)

* feat(#195): add camera capture support via FileProvider + TakePicture contract

- Creates temporary photo files in cacheDir via FileProvider
- Launches system camera with ActivityResultContracts.TakePicture
- Adds camera icon button (📷) next to the paperclip in ChatInputBar
- Captured photos added as image/jpeg attachments via same addAttachment flow
- FileProvider configured with cache-root path in res/xml/file_paths.xml
- Error handling for launch failures and capture failures with snackbar

* refactor(#195): group camera/image/file into single attach dropdown

- Replaces separate 📷 and 📎 buttons with a single 📎 button
- DropdownMenu opens with three options:
  - 📷 Camera → TakePicture via FileProvider temp URI
  - 🖼️ Image → GetContent("image/*")
  - 📄 File → GetContent("*/*")
- Removes onAttachClick, adds onImageTap/onFileTap callbacks
- Input bar stays clean with one attachment affordance

* fix(#195): add missing imports for CI compilation

ChatScreen.kt:
- android.util.Log
- androidx.compose.animation.core.* (infiniteRepeatable, tween, LinearEasing, RepeatMode, rememberInfiniteTransition)
- androidx.compose.foundation.layout.widthIn
- androidx.compose.material.icons.filled.AttachFile
- androidx.compose.material.icons.filled.InsertDriveFile
- androidx.compose.ui.layout.ContentScale
- coil.compose.AsyncImage

ChatBubble.kt:
- androidx.compose.material.icons.filled.InsertDriveFile

* fix(#195): resolve android-lint LocalContextGetResourceValueCall error

Replace context.getString(R.string.chat_camera_error) with a
composable-level cameraErrorMsg captured via stringResource()
to avoid querying resources via LocalContext.current in a callback.

* fix(#195): migrate InsertDriveFile to AutoMirrored, fix chip double-remove

- Replace deprecated Icons.Filled.InsertDriveFile with
  Icons.AutoMirrored.Filled.InsertDriveFile in ChatBubble.kt
  and ChatScreen.kt
- Remove onClick from AttachmentChip Surface so only the
  dedicated close button triggers removal, not accidental taps
  on the chip body

---------

Co-authored-by: Hermes Agent <hermes@agent.local>
This commit is contained in:
M57andHermes Agent authored and GitHub committed 2026-06-29 16:33:12 +03:00
1 parent c64b025a54
commit 4849f4c1c6
12 files changed
+971 -30

No files matched your search

+3
View File
@@ -143,6 +143,9 @@ dependencies {
implementation(libs.retrofit.converter.gson)
implementation(libs.gson)
// Image loading
implementation(libs.coil.compose)
// Encrypted storage
implementation(libs.androidx.security.crypto)
implementation("androidx.startup:startup-runtime:1.1.1")
+10
View File
@@ -37,6 +37,16 @@
<receiver
android:name=".notification.NotificationReplyReceiver"
android:exported="false" />
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.fileprovider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/file_paths" />
</provider>
</application>
</manifest>
@@ -0,0 +1,31 @@
package com.m57.hermescontrol.data.model
/**
* Represents a file attached to a chat message.
*
* @param uri Content URI or file path to the attachment
* @param name Display name of the file
* @param mimeType MIME type (e.g., "image/jpeg", "application/pdf")
* @param size File size in bytes
* @param localPath Optional local file system path for direct access
*/
data class Attachment(
val uri: String,
val name: String,
val mimeType: String,
val size: Long,
) {
val isImage: Boolean
get() = mimeType.startsWith("image/")
val formattedSize: String
get() =
when {
size < 1024 -> "$size B"
size < 1024 * 1024 -> "${size / 1024} KB"
else -> "%.1f MB".format(size.toDouble() / (1024 * 1024))
}
val fileExtension: String
get() = name.substringAfterLast('.', "").lowercase()
}
@@ -22,4 +22,12 @@ object WsMethods {
const val COMMANDS_CATALOG = "commands.catalog"
const val COMMAND_DISPATCH = "command.dispatch"
const val COMMAND_RESOLVE = "command.resolve"
// ── Attachments ───────────────────────────────────────────────────────
/** Upload image bytes (base64) from a remote client. */
const val IMAGE_ATTACH_BYTES = "image.attach_bytes"
/** Stage a non-image file (data URL) for agent access. */
const val FILE_ATTACH = "file.attach"
}
@@ -31,6 +31,7 @@ import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.InsertDriveFile
import androidx.compose.material.icons.filled.Build
import androidx.compose.material.icons.filled.Check
import androidx.compose.material.icons.filled.CheckCircle
@@ -60,6 +61,7 @@ import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.luminance
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.platform.ClipEntry
import androidx.compose.ui.platform.LocalClipboard
import androidx.compose.ui.platform.LocalConfiguration
@@ -79,7 +81,9 @@ import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.text.withStyle
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import coil.compose.AsyncImage
import com.m57.hermescontrol.R
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.theme.AssistantBubble
import com.m57.hermescontrol.theme.AssistantBubbleLight
import com.m57.hermescontrol.theme.StatusGreen
@@ -238,6 +242,17 @@ private fun UserBubble(
style = MaterialTheme.typography.bodyMedium,
)
}
// Render inline attachments
if (!message.attachments.isNullOrEmpty()) {
Spacer(modifier = Modifier.height(6.dp))
message.attachments.forEach { attachment ->
InlineAttachment(
attachment = attachment,
textColor = userBubbleTextColor,
)
Spacer(modifier = Modifier.height(4.dp))
}
}
if (!message.isStreaming) {
Text(
text = formatTimestamp(message.timestamp, DateFormat.is24HourFormat(LocalContext.current)),
@@ -2070,3 +2085,60 @@ private fun buildHighlightedString(
}
}
}
/**
* Renders an attachment inline inside a chat bubble.
* Images are displayed as thumbnails; other files show a compact card.
*/
@Composable
private fun InlineAttachment(
attachment: Attachment,
textColor: Color,
) {
if (attachment.isImage) {
// Image attachment — show as a rounded thumbnail
AsyncImage(
model = attachment.uri,
contentDescription = attachment.name,
modifier =
Modifier
.fillMaxWidth()
.clip(RoundedCornerShape(12.dp)),
contentScale = ContentScale.FillWidth,
)
} else {
// Non-image file — show a card with file icon and name
Surface(
shape = RoundedCornerShape(8.dp),
color = textColor.copy(alpha = 0.1f),
border = BorderStroke(1.dp, textColor.copy(alpha = 0.2f)),
) {
Row(
modifier = Modifier.padding(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
imageVector = Icons.AutoMirrored.Filled.InsertDriveFile,
contentDescription = null,
modifier = Modifier.size(24.dp),
tint = textColor.copy(alpha = 0.8f),
)
Spacer(modifier = Modifier.width(8.dp))
Column {
Text(
text = attachment.name,
color = textColor,
style = MaterialTheme.typography.bodySmall,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
Text(
text = attachment.formattedSize,
color = textColor.copy(alpha = 0.6f),
style = MaterialTheme.typography.labelSmall,
)
}
}
}
}
}
@@ -1,5 +1,6 @@
package com.m57.hermescontrol.ui.chat
import com.m57.hermescontrol.data.model.Attachment
import java.util.UUID
/**
@@ -22,6 +23,8 @@ data class ChatMessage(
val toolName: String? = null,
val toolStatus: ToolStatus? = null,
val approvalInfo: ApprovalInfo? = null,
/** Files attached to this message — shown inline in the bubble. */
val attachments: List<Attachment>? = null,
)
enum class MessageRole {
@@ -4,27 +4,16 @@ import android.Manifest
import android.app.Activity
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Build
import android.provider.OpenableColumns
import android.speech.RecognizerIntent
import android.speech.SpeechRecognizer
import android.util.Log
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.animation.AnimatedContent
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.animateContentSize
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.scaleIn
import androidx.compose.animation.scaleOut
import androidx.compose.animation.slideInVertically
import androidx.compose.animation.slideOutVertically
import androidx.compose.animation.togetherWith
import androidx.compose.animation.*
import androidx.compose.animation.core.*
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.ExperimentalFoundationApi
import androidx.compose.foundation.background
@@ -46,6 +35,7 @@ import androidx.compose.foundation.layout.imePadding
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.LazyListState
import androidx.compose.foundation.lazy.items
@@ -56,8 +46,10 @@ import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.BasicTextField
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.InsertDriveFile
import androidx.compose.material.icons.automirrored.filled.Send
import androidx.compose.material.icons.filled.Add
import androidx.compose.material.icons.filled.AttachFile
import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.KeyboardArrowDown
import androidx.compose.material.icons.filled.KeyboardArrowUp
@@ -69,6 +61,7 @@ import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.DropdownMenu
import androidx.compose.material3.DropdownMenuItem
import androidx.compose.material3.FilledTonalButton
import androidx.compose.material3.FloatingActionButton
@@ -101,6 +94,7 @@ import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.testTag
import androidx.compose.ui.res.stringResource
@@ -111,10 +105,13 @@ import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.core.content.ContextCompat
import androidx.core.content.FileProvider
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.viewmodel.compose.viewModel
import coil.compose.AsyncImage
import com.m57.hermescontrol.NavigationController
import com.m57.hermescontrol.R
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.ws.CommandCatalog
import com.m57.hermescontrol.data.ws.ConnectionStatus
import com.m57.hermescontrol.notification.NotificationHelper
@@ -124,6 +121,10 @@ import com.m57.hermescontrol.ui.common.HermesScaffold
import com.m57.hermescontrol.ui.common.NavIcon
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import java.io.File
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
@OptIn(ExperimentalFoundationApi::class)
@Composable
@@ -208,6 +209,57 @@ fun ChatScreen(
}
}
// File picker launcher for attachments (issue #195)
val filePickerLauncher =
rememberLauncherForActivityResult(
ActivityResultContracts.GetContent(),
) { uri: Uri? ->
if (uri != null) {
val cursor = context.contentResolver.query(uri, null, null, null, null)
cursor?.use { c ->
if (c.moveToFirst()) {
val nameIdx = c.getColumnIndex(OpenableColumns.DISPLAY_NAME)
val sizeIdx = c.getColumnIndex(OpenableColumns.SIZE)
val name = if (nameIdx >= 0) c.getString(nameIdx) else uri.lastPathSegment ?: "file"
val size = if (sizeIdx >= 0) c.getLong(sizeIdx) else 0L
val mimeType = context.contentResolver.getType(uri) ?: "application/octet-stream"
viewModel.addAttachment(uri.toString(), name, mimeType, size)
}
}
}
}
// Camera photo launcher (issue #195)
var pendingCameraUri by remember { mutableStateOf<Uri?>(null) }
val cameraErrorMsg = stringResource(R.string.chat_camera_error)
val cameraLauncher =
rememberLauncherForActivityResult(
ActivityResultContracts.TakePicture(),
) { success ->
val uri = pendingCameraUri
pendingCameraUri = null
if (success && uri != null) {
try {
val fileName =
"photo_${
SimpleDateFormat("yyyyMMdd_HHmmss", Locale.US).format(
Date(),
)
}.jpg"
val inputStream = context.contentResolver.openInputStream(uri)
val size = inputStream?.use { it.available().toLong() } ?: 0L
viewModel.addAttachment(uri.toString(), fileName, "image/jpeg", size)
} catch (e: Exception) {
Log.e("ChatScreen", "Camera capture failed", e)
scrollScope.launch {
snackbarHostState.showSnackbar(
cameraErrorMsg,
)
}
}
}
}
// Lifecycle effects, permissions, session switching, auto-scroll, errors
ChatLifecycleEffects(
sessionId = sessionId,
@@ -401,6 +453,28 @@ fun ChatScreen(
isAgentTyping = state.isAgentTyping,
isConnected = state.isConnected,
commandCatalog = state.commandCatalog,
pendingAttachments = state.pendingAttachments,
onCameraTap = {
try {
val timeStamp =
SimpleDateFormat("yyyyMMdd_HHmmss", Locale.US).format(Date())
val photoFile =
File.createTempFile("camera_${timeStamp}_", ".jpg", context.cacheDir)
val uri =
FileProvider.getUriForFile(
context,
"${context.packageName}.fileprovider",
photoFile,
)
pendingCameraUri = uri
cameraLauncher.launch(uri)
} catch (e: Exception) {
Log.e("ChatScreen", "Camera launch failed", e)
}
},
onImageTap = { filePickerLauncher.launch("image/*") },
onFileTap = { filePickerLauncher.launch("*/*") },
onRemoveAttachment = viewModel::removeAttachment,
)
}
}
@@ -474,10 +548,20 @@ private fun ChatInputBar(
isAgentTyping: Boolean,
isConnected: Boolean,
commandCatalog: CommandCatalog,
pendingAttachments: List<Attachment> = emptyList(),
onCameraTap: () -> Unit = {},
onImageTap: () -> Unit = {},
onFileTap: () -> Unit = {},
onRemoveAttachment: (Int) -> Unit = {},
) {
// Allow sending slash commands even while agent is typing
val isSlashCommand = inputText.startsWith("/")
val canSend = inputText.isNotBlank() && isConnected && (!isAgentTyping || isSlashCommand)
val canSend =
(inputText.isNotBlank() || pendingAttachments.isNotEmpty()) &&
isConnected && (!isAgentTyping || isSlashCommand)
// Attachment menu state
var showAttachmentMenu by remember { mutableStateOf(false) }
AnimatedVisibility(
visible = true,
@@ -559,6 +643,28 @@ private fun ChatInputBar(
}
}
// Attachment preview chips
AnimatedVisibility(
visible = pendingAttachments.isNotEmpty(),
enter = fadeIn() + expandVertically(),
exit = fadeOut() + shrinkVertically(),
) {
Row(
modifier =
Modifier
.fillMaxWidth()
.padding(horizontal = 12.dp, vertical = 4.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
pendingAttachments.forEachIndexed { index, attachment ->
AttachmentChip(
attachment = attachment,
onRemove = { onRemoveAttachment(index) },
)
}
}
}
Row(
modifier =
Modifier
@@ -566,6 +672,66 @@ private fun ChatInputBar(
.padding(horizontal = 8.dp, vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Box {
// Single attach button that opens a dropdown menu
IconButton(
onClick = { showAttachmentMenu = true },
enabled = isConnected,
modifier = Modifier.size(36.dp),
) {
Icon(
imageVector = Icons.Default.AttachFile,
contentDescription = stringResource(R.string.chat_attach_desc),
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
DropdownMenu(
expanded = showAttachmentMenu,
onDismissRequest = { showAttachmentMenu = false },
) {
DropdownMenuItem(
text = { Text("Camera") },
onClick = {
showAttachmentMenu = false
onCameraTap()
},
leadingIcon = {
Text(
text = "📷",
fontSize = 18.sp,
)
},
)
DropdownMenuItem(
text = { Text("Image") },
onClick = {
showAttachmentMenu = false
onImageTap()
},
leadingIcon = {
Text(
text = "🖼️",
fontSize = 18.sp,
)
},
)
DropdownMenuItem(
text = { Text("File") },
onClick = {
showAttachmentMenu = false
onFileTap()
},
leadingIcon = {
Text(
text = "📄",
fontSize = 18.sp,
)
},
)
}
}
OutlinedTextField(
value = inputText,
onValueChange = onInputChange,
@@ -573,7 +739,7 @@ private fun ChatInputBar(
Modifier
.weight(1f)
.heightIn(min = 36.dp, max = 120.dp)
.padding(horizontal = 12.dp, vertical = 8.dp)
.padding(horizontal = 4.dp, vertical = 8.dp)
.testTag("chat_input"),
placeholder = {
Text(
@@ -599,14 +765,14 @@ private fun ChatInputBar(
),
)
Spacer(modifier = Modifier.width(6.dp))
Spacer(modifier = Modifier.width(4.dp))
// Mic / Stop / Send button — swaps based on input state
AnimatedContent(
targetState =
when {
isListening -> "listening"
inputText.isBlank() -> "mic"
inputText.isBlank() && pendingAttachments.isEmpty() -> "mic"
else -> "send"
},
transitionSpec = {
@@ -681,6 +847,66 @@ private fun ChatInputBar(
}
}
/**
* Compact chip showing a pending attachment with a remove button.
*/
@Composable
private fun AttachmentChip(
attachment: Attachment,
onRemove: () -> Unit,
) {
Surface(
shape = RoundedCornerShape(16.dp),
color = MaterialTheme.colorScheme.surfaceVariant,
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outline.copy(alpha = 0.3f)),
) {
Row(
modifier = Modifier.padding(start = 10.dp, end = 6.dp, top = 4.dp, bottom = 4.dp),
verticalAlignment = Alignment.CenterVertically,
) {
if (attachment.isImage) {
AsyncImage(
model = attachment.uri,
contentDescription = attachment.name,
modifier =
Modifier
.size(20.dp)
.clip(RoundedCornerShape(4.dp)),
contentScale = ContentScale.Crop,
)
Spacer(modifier = Modifier.width(4.dp))
} else {
Icon(
imageVector = Icons.AutoMirrored.Filled.InsertDriveFile,
contentDescription = null,
modifier = Modifier.size(16.dp),
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(modifier = Modifier.width(4.dp))
}
Text(
text = attachment.name,
style = MaterialTheme.typography.labelSmall,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
modifier = Modifier.widthIn(max = 120.dp),
)
Spacer(modifier = Modifier.width(4.dp))
IconButton(
onClick = onRemove,
modifier = Modifier.size(18.dp),
) {
Icon(
imageVector = Icons.Default.Close,
contentDescription = stringResource(R.string.chat_attach_remove_desc),
modifier = Modifier.size(14.dp),
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
}
@OptIn(ExperimentalFoundationApi::class)
@Composable
private fun ClarifyDialog(
@@ -1,11 +1,14 @@
package com.m57.hermescontrol.ui.chat
import android.app.Application
import android.net.Uri
import android.util.Base64
import android.util.Log
import androidx.lifecycle.AndroidViewModel
import androidx.lifecycle.viewModelScope
import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.local.HermesDatabase
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.remote.ApiClient
import com.m57.hermescontrol.data.remote.NetworkResult
import com.m57.hermescontrol.data.remote.OkHttpProvider
@@ -15,6 +18,7 @@ import com.m57.hermescontrol.data.ws.ConnectionStatus
import com.m57.hermescontrol.data.ws.HermesWsClient
import com.m57.hermescontrol.data.ws.WsEvent
import com.m57.hermescontrol.data.ws.WsMethods
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
@@ -27,6 +31,7 @@ import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.util.concurrent.ConcurrentHashMap
private const val TAG = "ChatViewModel"
@@ -37,6 +42,9 @@ private const val TAG = "ChatViewModel"
*/
private const val PENDING_REQUEST_TIMEOUT_MS = 30_000L
/** Thrown when an awaited RPC returns an error response. */
private class RpcCallException(message: String) : Exception(message)
data class ChatUiState(
val messages: List<ChatMessage> = emptyList(),
val currentSessionId: String? = null,
@@ -62,6 +70,8 @@ data class ChatUiState(
val typingEffectDelayMs: Int = 30,
// Commands catalog
val commandCatalog: CommandCatalog = CommandCatalog(),
// Attachment state
val pendingAttachments: List<Attachment> = emptyList(),
) {
/** Convenience — derived from [connectionStatus]. */
val isConnected: Boolean get() = connectionStatus == ConnectionStatus.CONNECTED
@@ -371,6 +381,9 @@ class ChatViewModel(
val pending = pendingRequests.remove(id) ?: return
val method = pending.method
// Complete the deferred so sendRpcAndAwait() callers can resume
pending.deferred?.complete(result)
when (method) {
WsMethods.SESSION_CREATE -> {
val resultMap = result as? Map<String, Any?> ?: return
@@ -511,18 +524,36 @@ class ChatViewModel(
is Map<*, *> -> error["message"] as? String ?: error.toString()
else -> error.toString()
}
_uiState.update {
it.copy(
isLoading = false,
errorMessage = "Error${if (method != null) " ($method)" else ""}: $errorMsg",
)
// Fail the deferred so sendRpcAndAwait() callers see the error
pending?.deferred?.completeExceptionally(RpcCallException(errorMsg))
// Only surface error in UI for non-awaited RPCs
if (pending?.deferred == null) {
_uiState.update {
it.copy(
isLoading = false,
errorMessage = "Error${if (method != null) " ($method)" else ""}: $errorMsg",
)
}
}
}
// ── Send message ─────────────────────────────────────────────────────
/**
* Send a user prompt, uploading any pending attachments to the backend
* first via their dedicated RPC methods.
*
* Flow:
* 1. Snapshot pending attachments (then clear them from UI)
* 2. Add user message to UI + DB immediately (optimistic UX)
* 3. For each image → fire-and-forget `image.attach_bytes` (queues into session)
* 4. For each file → await `file.attach`, collect @file: refs
* 5. Send `prompt.submit` with text + @file: refs — images auto-picked up by backend
*/
fun sendMessage(text: String) {
if (text.isBlank()) return
if (text.isBlank() && _uiState.value.pendingAttachments.isEmpty()) return
val sessionId = _uiState.value.currentSessionId ?: return
val trimmed = text.trim()
@@ -531,13 +562,18 @@ class ChatViewModel(
return
}
// Snapshot + clear attachments so the input bar empties immediately
val attachments = _uiState.value.pendingAttachments.toList()
clearAttachments()
val userMessage =
ChatMessage(
role = MessageRole.USER,
content = text,
attachments = if (attachments.isNotEmpty()) attachments else null,
)
// Update UI — no DB writes inside update{}
// Update UI immediately
_uiState.update { state ->
state.copy(
messages = state.messages + userMessage,
@@ -545,20 +581,149 @@ class ChatViewModel(
)
}
// Persist to Room — OUTSIDE update{}
// Persist to Room
viewModelScope.launch(Dispatchers.IO) {
repo.persistMessage(userMessage, sessionId)
}
// Upload attachments then submit prompt
viewModelScope.launch(Dispatchers.IO) {
val fileRefs = mutableListOf<String>()
for (attachment in attachments) {
val bytes = readContentUriBytes(attachment.uri)
if (bytes == null) {
Log.w(TAG, "Skipping unreadable attachment: ${attachment.name}")
continue
}
val b64 = Base64.encodeToString(bytes, Base64.NO_WRAP)
try {
if (attachment.isImage) {
// Fire-and-forget — backend queues into session["attached_images"],
// auto-picked by the subsequent prompt.submit
wsClient.send(
method = WsMethods.IMAGE_ATTACH_BYTES,
params =
mapOf(
"content_base64" to "data:${attachment.mimeType};base64,$b64",
"filename" to attachment.name,
"ext" to attachment.fileExtension,
),
)
} else {
// Await the @file: ref text so we can embed it in the prompt
sendRpcAndAwait(
method = WsMethods.FILE_ATTACH,
params =
mapOf(
"data_url" to "data:${attachment.mimeType};base64,$b64",
"name" to attachment.name,
),
)?.let { result ->
@Suppress("UNCHECKED_CAST")
val refText =
(result as? Map<String, Any?>)?.get("ref_text") as? String
if (!refText.isNullOrBlank()) fileRefs.add(refText)
}
}
} catch (e: Exception) {
Log.e(TAG, "Failed to upload attachment ${attachment.name}", e)
_uiState.update {
it.copy(errorMessage = "⚠️ Upload failed: ${attachment.name}")
}
}
}
// Build prompt text — prepend @file: refs for non-image files
val fullText =
if (fileRefs.isEmpty()) {
text
} else {
fileRefs.joinToString("\n") +
if (text.isNotBlank()) "\n\n$text" else ""
}
wsClient.sendMessage(
sessionId,
text,
fullText,
onSent = { id -> trackRequest(id, WsMethods.PROMPT_SUBMIT) },
)
}
}
/** Read bytes from a `content://` or `file://` URI via ContentResolver. */
private fun readContentUriBytes(uriString: String): ByteArray? {
return try {
val context = getApplication<Application>()
val uri = Uri.parse(uriString)
context.contentResolver.openInputStream(uri)?.use { stream -> stream.readBytes() }
} catch (e: Exception) {
Log.e(TAG, "Failed to read attachment bytes: ${e.message}", e)
null
}
}
/**
* Send a JSON-RPC call and suspend until the response arrives.
* Throws [RpcCallException] on error, [kotlinx.coroutines.TimeoutCancellationException] on timeout.
*/
private suspend fun sendRpcAndAwait(
method: String,
params: Map<String, Any>,
timeoutMs: Long = 30_000,
): Any? {
val deferred = CompletableDeferred<Any?>()
wsClient.send(
method = method,
params = params,
onSent = { id -> trackRequest(id, method, deferred) },
)
return withTimeout(timeoutMs) { deferred.await() }
}
// ── Attachment management ─────────────────────────────────────────────
/**
* Add a picked file as a pending attachment.
* [uri] should be a content:// URI string; the ViewModel will read
* the content and encode it for sending.
*/
fun addAttachment(
uri: String,
name: String,
mimeType: String,
size: Long,
) {
_uiState.update { state ->
val attachment =
Attachment(
uri = uri,
name = name,
mimeType = mimeType,
size = size,
)
state.copy(
pendingAttachments = state.pendingAttachments + attachment,
)
}
}
fun removeAttachment(index: Int) {
_uiState.update { state ->
state.copy(
pendingAttachments =
state.pendingAttachments.toMutableList().apply {
if (index in indices) removeAt(index)
},
)
}
}
fun clearAttachments() {
_uiState.update { it.copy(pendingAttachments = emptyList()) }
}
private fun handleSlashCommand(command: String) {
val userMsg = ChatMessage(role = MessageRole.USER, content = command)
val sessionId = _uiState.value.currentSessionId
@@ -1065,13 +1230,15 @@ class ChatViewModel(
private data class PendingRequest(
val method: String,
val createdAt: Long = System.currentTimeMillis(),
val deferred: CompletableDeferred<Any?>? = null,
)
private fun trackRequest(
id: String,
method: String,
deferred: CompletableDeferred<Any?>? = null,
) {
pendingRequests[id] = PendingRequest(method)
pendingRequests[id] = PendingRequest(method, deferred = deferred)
}
/**
+13
View File
@@ -100,6 +100,19 @@
<string name="content_desc_copy">Copy</string>
<string name="content_desc_edit">Edit</string>
<!-- Attachment / File Picker -->
<string name="chat_attach_desc">Attach file</string>
<string name="chat_attach_image_desc">Attach image</string>
<string name="chat_attach_remove_desc">Remove attachment</string>
<string name="chat_attachment_unknown">Unknown file</string>
<string name="chat_attachment_image">Image</string>
<string name="chat_attachment_file">%1$s</string>
<string name="chat_attachment_send_error">Failed to process attachment</string>
<!-- Camera -->
<string name="chat_camera_desc">Camera</string>
<string name="chat_camera_error">Failed to capture photo</string>
<!-- Settings Screen -->
<string name="settings_tab_connection">Connection</string>
<string name="settings_tab_appearance">Appearance</string>
+4
View File
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<paths>
<cache-path name="camera" path="." />
</paths>
+4
View File
@@ -24,6 +24,7 @@ retrofit = "3.0.0"
gson = "2.14.0"
securityCrypto = "1.0.0"
mockk = "1.14.11"
coil = "2.7.0"
room = "2.7.1"
sqlcipher = "4.16.0"
@@ -75,6 +76,9 @@ androidx-room-compiler = { module = "androidx.room:room-compiler", version.ref =
androidx-room-ktx = { module = "androidx.room:room-ktx", version.ref = "room" }
sqlcipher = { module = "net.zetetic:sqlcipher-android", version.ref = "sqlcipher" }
# Image loading
coil-compose = { module = "io.coil-kt:coil-compose", version.ref = "coil" }
[plugins]
android-application = { id = "com.android.application", version.ref = "androidGradlePlugin" }
+400
View File
@@ -0,0 +1,400 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>ChatInputBar — Layout (v2)</title>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
"Helvetica Neue", Arial, sans-serif;
-webkit-font-smoothing: antialiased;
background: #111;
color: #eee;
padding: 24px;
display: flex;
flex-direction: column;
gap: 36px;
}
.label {
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.06em;
color: #888;
margin-bottom: 6px;
}
.bar {
display: flex;
align-items: center;
gap: 4px;
width: 100%;
max-width: 500px;
background: rgba(30,30,35,0.92);
border: 1px solid rgba(255,255,255,0.08);
border-radius: 24px;
padding: 4px 8px;
backdrop-filter: blur(8px);
position: relative;
}
.icon-btn {
display: flex;
align-items: center;
justify-content: center;
width: 36px;
height: 36px;
border: none;
border-radius: 50%;
background: transparent;
cursor: pointer;
font-size: 18px;
flex-shrink: 0;
color: #999;
transition: background 0.15s;
}
.icon-btn:hover { background: rgba(255,255,255,0.08); }
.icon-btn:disabled { opacity: 0.25; cursor: default; }
.send-btn {
width: 40px;
height: 40px;
border-radius: 50%;
border: none;
display: flex;
align-items: center;
justify-content: center;
flex-shrink: 0;
cursor: pointer;
transition: all 0.15s;
background: var(--btn-bg, rgba(255,255,255,0.08));
color: var(--btn-color, #999);
}
.send-btn.mic { --btn-bg: rgba(255,255,255,0.08); --btn-color: #ccc; }
.send-btn.send { --btn-bg: #4f8cff; --btn-color: #fff; }
.send-btn.stop { --btn-bg: #e53935; --btn-color: #fff; }
.input-field {
flex: 1;
border: 1px solid rgba(255,255,255,0.12);
border-radius: 20px;
background: transparent;
padding: 8px 14px;
font-size: 14px;
color: #eee;
outline: none;
min-height: 36px;
max-height: 120px;
transition: border-color 0.15s;
font-family: inherit;
line-height: 1.5;
}
.input-field:focus { border-color: #4f8cff; }
.input-field::placeholder { color: #666; }
.input-field:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.chips-row {
display: flex;
gap: 8px;
padding: 4px 12px;
max-width: 500px;
overflow-x: auto;
}
.chip {
display: flex;
align-items: center;
gap: 4px;
background: rgba(255,255,255,0.06);
border: 1px solid rgba(255,255,255,0.10);
border-radius: 16px;
padding: 4px 6px 4px 10px;
cursor: pointer;
transition: background 0.12s;
}
.chip:hover { background: rgba(255,255,255,0.10); }
.chip .thumb {
width: 20px; height: 20px;
border-radius: 4px;
background: #333;
overflow: hidden;
flex-shrink: 0;
display: flex;
align-items: center;
justify-content: center;
font-size: 10px;
color: #888;
}
.chip .name {
font-size: 12px;
max-width: 100px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: #ccc;
}
.chip .close {
width: 18px; height: 18px;
border-radius: 50%;
border: none;
background: transparent;
cursor: pointer;
display: flex;
align-items: center;
justify-content: center;
font-size: 10px;
color: #888;
flex-shrink: 0;
}
.chip .close:hover { background: rgba(255,255,255,0.12); }
/* Dropdown menu */
.menu-wrapper {
position: relative;
}
.menu {
position: absolute;
bottom: 36px;
left: 0;
min-width: 180px;
background: rgba(42,42,50,0.98);
border: 1px solid rgba(255,255,255,0.08);
border-radius: 12px;
padding: 4px 0;
box-shadow: 0 8px 24px rgba(0,0,0,0.5);
z-index: 10;
overflow: hidden;
}
.menu-item {
display: flex;
align-items: center;
gap: 10px;
padding: 10px 16px;
cursor: pointer;
border: none;
background: transparent;
color: #ddd;
font-size: 14px;
width: 100%;
text-align: left;
transition: background 0.1s;
font-family: inherit;
}
.menu-item:hover { background: rgba(255,255,255,0.06); }
.menu-item .icon { font-size: 18px; width: 24px; text-align: center; flex-shrink: 0; }
.variant { margin-bottom: 8px; }
.note {
font-size: 12px;
color: #777;
margin-top: 4px;
line-height: 1.5;
}
.states {
display: flex;
flex-direction: column;
gap: 24px;
}
/* Simple arrow for "tap here" */
.tap-indicator {
display: inline-flex;
align-items: center;
gap: 6px;
background: rgba(79,140,255,0.12);
color: #7aadff;
padding: 2px 10px;
border-radius: 8px;
font-size: 11px;
margin-left: 4px;
}
</style>
</head>
<body>
<h2 style="color:#ddd; font-weight:500; font-size:18px; margin-bottom:4px;">📱 ChatInputBar (v2)</h2>
<p style="color:#777; font-size:13px; margin-bottom:20px;">
Single 📎 attach button → dropdown menu with Camera / Image / File
</p>
<div class="states">
<!-- 1. Empty + menu open -->
<div class="variant">
<div class="label">1. Empty — mic visible, dropdown open</div>
<div class="bar">
<div class="menu-wrapper">
<button class="icon-btn" title="Attach" style="background:rgba(255,255,255,0.08);">📎</button>
<div class="menu">
<button class="menu-item"><span class="icon">📷</span> Camera</button>
<button class="menu-item"><span class="icon">🖼️</span> Image</button>
<button class="menu-item"><span class="icon">📄</span> File</button>
</div>
</div>
<input class="input-field" placeholder="Type a message…" disabled />
<div style="width:4px;flex-shrink:0;"></div>
<button class="send-btn mic" title="Voice input">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z"/><path d="M19 10v2a7 7 0 0 1-14 0v-2"/><line x1="12" y1="19" x2="12" y2="23"/><line x1="8" y1="23" x2="16" y2="23"/></svg>
</button>
</div>
<div class="note">📎 button opens a Material 3 DropdownMenu with three options. Selecting one fires the launcher and closes the menu.</div>
</div>
<!-- 2. Empty, menu closed -->
<div class="variant">
<div class="label">2. Empty — normal state</div>
<div class="bar">
<button class="icon-btn" title="Attach">📎</button>
<input class="input-field" placeholder="Type a message…" disabled />
<div style="width:4px;flex-shrink:0;"></div>
<button class="send-btn mic" title="Voice input">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z"/><path d="M19 10v2a7 7 0 0 1-14 0v-2"/><line x1="12" y1="19" x2="12" y2="23"/><line x1="8" y1="23" x2="16" y2="23"/></svg>
</button>
</div>
<div class="note">Default look — single 📎, no clutter. Mic visible when input is blank &amp; no attachments.</div>
</div>
<!-- 3. Typing -->
<div class="variant">
<div class="label">3. Typing — send shown</div>
<div class="bar">
<button class="icon-btn" title="Attach">📎</button>
<input class="input-field" placeholder="Type a message…" value="Deploy the new config, but keep the old one as backup" style="color:#eee;" />
<div style="width:4px;flex-shrink:0;"></div>
<button class="send-btn send" title="Send">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="22" y1="2" x2="11" y2="13"/><polygon points="22 2 15 22 11 13 2 9 22 2"/></svg>
</button>
</div>
<div class="note">Mic → Send (blue) when text or attachments present.</div>
</div>
<!-- 4. Attachments -->
<div class="variant">
<div class="label">4. With attachments — preview chips</div>
<div class="chips-row">
<div class="chip">
<span class="thumb">📷</span>
<span class="name">photo_20260629.jpg</span>
<button class="close">✕</button>
</div>
<div class="chip">
<span class="thumb">📄</span>
<span class="name">config.yaml</span>
<button class="close">✕</button>
</div>
<div class="chip">
<span class="thumb">📷</span>
<span class="name">screenshot.png</span>
<button class="close">✕</button>
</div>
</div>
<div class="bar" style="margin-top:0;">
<button class="icon-btn" title="Attach">📎</button>
<input class="input-field" placeholder="Type a message…" value="Here's what I'm looking at" style="color:#eee;" />
<div style="width:4px;flex-shrink:0;"></div>
<button class="send-btn send" title="Send">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="22" y1="2" x2="11" y2="13"/><polygon points="22 2 15 22 11 13 2 9 22 2"/></svg>
</button>
</div>
<div class="note">Image chips → AsyncImage thumbnail. File chips → file icon. Close ✕ removes.</div>
</div>
<!-- 5. Listening -->
<div class="variant">
<div class="label">5. Listening — stop shown</div>
<div class="bar">
<button class="icon-btn" title="Attach" disabled>📎</button>
<input class="input-field" placeholder="Type a message…" disabled />
<div style="width:4px;flex-shrink:0;"></div>
<button class="send-btn stop" title="Stop listening">
<svg width="18" height="18" viewBox="0 0 24 24" fill="currentColor"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
</button>
</div>
<div class="note">While listening, input + attach disabled. Red Stop replaces Mic.</div>
</div>
<!-- 6. Disconnected -->
<div class="variant">
<div class="label">6. Disconnected — everything disabled</div>
<div class="bar" style="opacity:0.5;">
<button class="icon-btn" disabled title="Attach">📎</button>
<input class="input-field" placeholder="Not connected" disabled />
<div style="width:4px;flex-shrink:0;"></div>
<button class="send-btn mic" disabled title="Voice input">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z"/><path d="M19 10v2a7 7 0 0 1-14 0v-2"/><line x1="12" y1="19" x2="12" y2="23"/><line x1="8" y1="23" x2="16" y2="23"/></svg>
</button>
</div>
<div class="note">Everything dimmed, placeholder "Not connected".</div>
</div>
</div>
<!-- Full structural diagram -->
<hr style="border-color:rgba(255,255,255,0.06); margin: 28px 0 16px;" />
<h3 style="color:#ddd; font-weight:500; font-size:14px; margin-bottom:12px;">📐 Structural hierarchy</h3>
<pre style="font-size:12px; color:#aaa; line-height:1.6; background:rgba(255,255,255,0.03); padding:16px; border-radius:12px; max-width:600px;">
Card (roundedCornerShape(24dp), surface alpha 0.85)
└ Column
├ [conditional] Slash command suggestions
│ └ LazyColumn (maxHeight 200dp)
├ [conditional] Attachment preview chips
│ └ Row
│ └ AttachmentChip × N
│ └ Surface (roundedCorner 16dp)
│ ├ [if image] AsyncImage(20dp)
│ ├ [else] Icon(InsertDriveFile, 16dp)
│ └ name + Close IconButton(18dp)
└ Row (main input area)
├ Box (positioned for dropdown)
│ ├ IconButton 📎 AttachFile (36dp)
│ └ DropdownMenu (expanded = showAttachmentMenu)
│ ├ DropdownMenuItem: 📷 Camera → onCameraTap()
│ ├ DropdownMenuItem: 🖼️ Image → onImageTap() → launch("image/*")
│ └ DropdownMenuItem: 📄 File → onFileTap() → launch("*/*")
├ OutlinedTextField (weight=1f, maxLines=4, rounded 20dp)
├ Spacer(4dp)
└ AnimatedContent (mic ↔ send ↔ listening)
├ mic: FilledTonalButton(Mic icon, 40dp, circle)
├ send: FilledTonalButton(Send icon, 40dp, circle, blue)
└ listening: FilledTonalButton(Stop icon, 40dp, circle, red)
</pre>
<hr style="border-color:rgba(255,255,255,0.06); margin: 28px 0 16px;" />
<h3 style="color:#ddd; font-weight:500; font-size:14px; margin-bottom:12px;">🎯 What each dropdown option does</h3>
<table style="font-size:13px; color:#ccc; border-collapse: collapse; max-width:600px;">
<tr style="border-bottom:1px solid rgba(255,255,255,0.06);">
<th style="padding:8px 12px; text-align:left; color:#888;">Option</th>
<th style="padding:8px 12px; text-align:left; color:#888;">Launcher</th>
<th style="padding:8px 12px; text-align:left; color:#888;">MIME</th>
</tr>
<tr style="border-bottom:1px solid rgba(255,255,255,0.06);">
<td style="padding:8px 12px;">📷 Camera</td>
<td style="padding:8px 12px;"><code>TakePicture</code> → temp FileProvider URI</td>
<td style="padding:8px 12px;">image/jpeg</td>
</tr>
<tr style="border-bottom:1px solid rgba(255,255,255,0.06);">
<td style="padding:8px 12px;">🖼️ Image</td>
<td style="padding:8px 12px;"><code>GetContent("image/*")</code></td>
<td style="padding:8px 12px;">image/* (gallery picker)</td>
</tr>
<tr>
<td style="padding:8px 12px;">📄 File</td>
<td style="padding:8px 12px;"><code>GetContent("*/*")</code></td>
<td style="padding:8px 12px;">*/* (system file picker)</td>
</tr>
</table>
</body>
</html>