feat(#195): chat attachment support for files and images (#418)

* feat(#195): add chat attachment support for files and images

- Add Attachment data model with computed properties (isImage, formattedSize, fileExtension)
- Extend ChatMessage with optional attachments list
- Add Coil 2.7.0 for async image loading in Compose
- Extend ChatViewModel with pending attachment state, addAttachment/removeAttachment/clearAttachments
- Build attachment-prefixed message text (data URIs for images, file refs for others)
- Add paperclip button and file picker launcher (ActivityResultContracts.GetContent) to ChatScreen
- Add animated attachment preview chips with thumbnails before sending
- Render attachments inline in UserBubble (AsyncImage for images, file cards for others)
- Add string resources for attachment UI features

* fix(#195): rewrite attachment upload protocol to match Hermes backend RPC methods

BREAKS: removes old buildAttachmentText() that embedded base64 data URIs in prompt text.

Changes:
- Replace inline data-URI encoding with proper backend RPC flow:
  - image.attach_bytes for images (fire-and-forget, auto-queues to session)
  - file.attach for non-image files (awaited, returns @file: ref text)
- Add CompletableDeferred to PendingRequest so sendMessage can await RPC responses
- Add readContentUriBytes() to read file content from content:// URIs
- Add sendRpcAndAwait() suspend function for sequential RPC calls
- Add RpcCallException for awaited RPC error handling
- Suppress error UI for awaited RPCs (caller handles them directly)
- Keep ChatScreen/ChatBubble UI layer unchanged (file picker, previews, inline images)

* feat(#195): add camera capture support via FileProvider + TakePicture contract

- Creates temporary photo files in cacheDir via FileProvider
- Launches system camera with ActivityResultContracts.TakePicture
- Adds camera icon button (📷) next to the paperclip in ChatInputBar
- Captured photos added as image/jpeg attachments via same addAttachment flow
- FileProvider configured with cache-root path in res/xml/file_paths.xml
- Error handling for launch failures and capture failures with snackbar

* refactor(#195): group camera/image/file into single attach dropdown

- Replaces separate 📷 and 📎 buttons with a single 📎 button
- DropdownMenu opens with three options:
  - 📷 Camera → TakePicture via FileProvider temp URI
  - 🖼️ Image → GetContent("image/*")
  - 📄 File → GetContent("*/*")
- Removes onAttachClick, adds onImageTap/onFileTap callbacks
- Input bar stays clean with one attachment affordance

* fix(#195): add missing imports for CI compilation

ChatScreen.kt:
- android.util.Log
- androidx.compose.animation.core.* (infiniteRepeatable, tween, LinearEasing, RepeatMode, rememberInfiniteTransition)
- androidx.compose.foundation.layout.widthIn
- androidx.compose.material.icons.filled.AttachFile
- androidx.compose.material.icons.filled.InsertDriveFile
- androidx.compose.ui.layout.ContentScale
- coil.compose.AsyncImage

ChatBubble.kt:
- androidx.compose.material.icons.filled.InsertDriveFile

* fix(#195): resolve android-lint LocalContextGetResourceValueCall error

Replace context.getString(R.string.chat_camera_error) with a
composable-level cameraErrorMsg captured via stringResource()
to avoid querying resources via LocalContext.current in a callback.

* fix(#195): migrate InsertDriveFile to AutoMirrored, fix chip double-remove

- Replace deprecated Icons.Filled.InsertDriveFile with
  Icons.AutoMirrored.Filled.InsertDriveFile in ChatBubble.kt
  and ChatScreen.kt
- Remove onClick from AttachmentChip Surface so only the
  dedicated close button triggers removal, not accidental taps
  on the chip body

---------

Co-authored-by: Hermes Agent <hermes@agent.local>
This commit is contained in:
M57andHermes Agent authored and GitHub committed 2026-06-29 16:33:12 +03:00
1 parent c64b025a54
commit 4849f4c1c6
12 files changed
+971 -30

No files matched your search

@@ -0,0 +1,31 @@
package com.m57.hermescontrol.data.model
/**
* Represents a file attached to a chat message.
*
* @param uri Content URI or file path to the attachment
* @param name Display name of the file
* @param mimeType MIME type (e.g., "image/jpeg", "application/pdf")
* @param size File size in bytes
* @param localPath Optional local file system path for direct access
*/
data class Attachment(
val uri: String,
val name: String,
val mimeType: String,
val size: Long,
) {
val isImage: Boolean
get() = mimeType.startsWith("image/")
val formattedSize: String
get() =
when {
size < 1024 -> "$size B"
size < 1024 * 1024 -> "${size / 1024} KB"
else -> "%.1f MB".format(size.toDouble() / (1024 * 1024))
}
val fileExtension: String
get() = name.substringAfterLast('.', "").lowercase()
}
@@ -22,4 +22,12 @@ object WsMethods {
const val COMMANDS_CATALOG = "commands.catalog"
const val COMMAND_DISPATCH = "command.dispatch"
const val COMMAND_RESOLVE = "command.resolve"
// ── Attachments ───────────────────────────────────────────────────────
/** Upload image bytes (base64) from a remote client. */
const val IMAGE_ATTACH_BYTES = "image.attach_bytes"
/** Stage a non-image file (data URL) for agent access. */
const val FILE_ATTACH = "file.attach"
}
@@ -31,6 +31,7 @@ import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.InsertDriveFile
import androidx.compose.material.icons.filled.Build
import androidx.compose.material.icons.filled.Check
import androidx.compose.material.icons.filled.CheckCircle
@@ -60,6 +61,7 @@ import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.luminance
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.platform.ClipEntry
import androidx.compose.ui.platform.LocalClipboard
import androidx.compose.ui.platform.LocalConfiguration
@@ -79,7 +81,9 @@ import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.text.withStyle
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import coil.compose.AsyncImage
import com.m57.hermescontrol.R
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.theme.AssistantBubble
import com.m57.hermescontrol.theme.AssistantBubbleLight
import com.m57.hermescontrol.theme.StatusGreen
@@ -238,6 +242,17 @@ private fun UserBubble(
style = MaterialTheme.typography.bodyMedium,
)
}
// Render inline attachments
if (!message.attachments.isNullOrEmpty()) {
Spacer(modifier = Modifier.height(6.dp))
message.attachments.forEach { attachment ->
InlineAttachment(
attachment = attachment,
textColor = userBubbleTextColor,
)
Spacer(modifier = Modifier.height(4.dp))
}
}
if (!message.isStreaming) {
Text(
text = formatTimestamp(message.timestamp, DateFormat.is24HourFormat(LocalContext.current)),
@@ -2070,3 +2085,60 @@ private fun buildHighlightedString(
}
}
}
/**
* Renders an attachment inline inside a chat bubble.
* Images are displayed as thumbnails; other files show a compact card.
*/
@Composable
private fun InlineAttachment(
attachment: Attachment,
textColor: Color,
) {
if (attachment.isImage) {
// Image attachment — show as a rounded thumbnail
AsyncImage(
model = attachment.uri,
contentDescription = attachment.name,
modifier =
Modifier
.fillMaxWidth()
.clip(RoundedCornerShape(12.dp)),
contentScale = ContentScale.FillWidth,
)
} else {
// Non-image file — show a card with file icon and name
Surface(
shape = RoundedCornerShape(8.dp),
color = textColor.copy(alpha = 0.1f),
border = BorderStroke(1.dp, textColor.copy(alpha = 0.2f)),
) {
Row(
modifier = Modifier.padding(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
imageVector = Icons.AutoMirrored.Filled.InsertDriveFile,
contentDescription = null,
modifier = Modifier.size(24.dp),
tint = textColor.copy(alpha = 0.8f),
)
Spacer(modifier = Modifier.width(8.dp))
Column {
Text(
text = attachment.name,
color = textColor,
style = MaterialTheme.typography.bodySmall,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
Text(
text = attachment.formattedSize,
color = textColor.copy(alpha = 0.6f),
style = MaterialTheme.typography.labelSmall,
)
}
}
}
}
}
@@ -1,5 +1,6 @@
package com.m57.hermescontrol.ui.chat
import com.m57.hermescontrol.data.model.Attachment
import java.util.UUID
/**
@@ -22,6 +23,8 @@ data class ChatMessage(
val toolName: String? = null,
val toolStatus: ToolStatus? = null,
val approvalInfo: ApprovalInfo? = null,
/** Files attached to this message — shown inline in the bubble. */
val attachments: List<Attachment>? = null,
)
enum class MessageRole {
@@ -4,27 +4,16 @@ import android.Manifest
import android.app.Activity
import android.content.Intent
import android.content.pm.PackageManager
import android.net.Uri
import android.os.Build
import android.provider.OpenableColumns
import android.speech.RecognizerIntent
import android.speech.SpeechRecognizer
import android.util.Log
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.animation.AnimatedContent
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.animateContentSize
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.scaleIn
import androidx.compose.animation.scaleOut
import androidx.compose.animation.slideInVertically
import androidx.compose.animation.slideOutVertically
import androidx.compose.animation.togetherWith
import androidx.compose.animation.*
import androidx.compose.animation.core.*
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.ExperimentalFoundationApi
import androidx.compose.foundation.background
@@ -46,6 +35,7 @@ import androidx.compose.foundation.layout.imePadding
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.LazyListState
import androidx.compose.foundation.lazy.items
@@ -56,8 +46,10 @@ import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.BasicTextField
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.InsertDriveFile
import androidx.compose.material.icons.automirrored.filled.Send
import androidx.compose.material.icons.filled.Add
import androidx.compose.material.icons.filled.AttachFile
import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.KeyboardArrowDown
import androidx.compose.material.icons.filled.KeyboardArrowUp
@@ -69,6 +61,7 @@ import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.DropdownMenu
import androidx.compose.material3.DropdownMenuItem
import androidx.compose.material3.FilledTonalButton
import androidx.compose.material3.FloatingActionButton
@@ -101,6 +94,7 @@ import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.testTag
import androidx.compose.ui.res.stringResource
@@ -111,10 +105,13 @@ import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.core.content.ContextCompat
import androidx.core.content.FileProvider
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.viewmodel.compose.viewModel
import coil.compose.AsyncImage
import com.m57.hermescontrol.NavigationController
import com.m57.hermescontrol.R
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.ws.CommandCatalog
import com.m57.hermescontrol.data.ws.ConnectionStatus
import com.m57.hermescontrol.notification.NotificationHelper
@@ -124,6 +121,10 @@ import com.m57.hermescontrol.ui.common.HermesScaffold
import com.m57.hermescontrol.ui.common.NavIcon
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import java.io.File
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
@OptIn(ExperimentalFoundationApi::class)
@Composable
@@ -208,6 +209,57 @@ fun ChatScreen(
}
}
// File picker launcher for attachments (issue #195)
val filePickerLauncher =
rememberLauncherForActivityResult(
ActivityResultContracts.GetContent(),
) { uri: Uri? ->
if (uri != null) {
val cursor = context.contentResolver.query(uri, null, null, null, null)
cursor?.use { c ->
if (c.moveToFirst()) {
val nameIdx = c.getColumnIndex(OpenableColumns.DISPLAY_NAME)
val sizeIdx = c.getColumnIndex(OpenableColumns.SIZE)
val name = if (nameIdx >= 0) c.getString(nameIdx) else uri.lastPathSegment ?: "file"
val size = if (sizeIdx >= 0) c.getLong(sizeIdx) else 0L
val mimeType = context.contentResolver.getType(uri) ?: "application/octet-stream"
viewModel.addAttachment(uri.toString(), name, mimeType, size)
}
}
}
}
// Camera photo launcher (issue #195)
var pendingCameraUri by remember { mutableStateOf<Uri?>(null) }
val cameraErrorMsg = stringResource(R.string.chat_camera_error)
val cameraLauncher =
rememberLauncherForActivityResult(
ActivityResultContracts.TakePicture(),
) { success ->
val uri = pendingCameraUri
pendingCameraUri = null
if (success && uri != null) {
try {
val fileName =
"photo_${
SimpleDateFormat("yyyyMMdd_HHmmss", Locale.US).format(
Date(),
)
}.jpg"
val inputStream = context.contentResolver.openInputStream(uri)
val size = inputStream?.use { it.available().toLong() } ?: 0L
viewModel.addAttachment(uri.toString(), fileName, "image/jpeg", size)
} catch (e: Exception) {
Log.e("ChatScreen", "Camera capture failed", e)
scrollScope.launch {
snackbarHostState.showSnackbar(
cameraErrorMsg,
)
}
}
}
}
// Lifecycle effects, permissions, session switching, auto-scroll, errors
ChatLifecycleEffects(
sessionId = sessionId,
@@ -401,6 +453,28 @@ fun ChatScreen(
isAgentTyping = state.isAgentTyping,
isConnected = state.isConnected,
commandCatalog = state.commandCatalog,
pendingAttachments = state.pendingAttachments,
onCameraTap = {
try {
val timeStamp =
SimpleDateFormat("yyyyMMdd_HHmmss", Locale.US).format(Date())
val photoFile =
File.createTempFile("camera_${timeStamp}_", ".jpg", context.cacheDir)
val uri =
FileProvider.getUriForFile(
context,
"${context.packageName}.fileprovider",
photoFile,
)
pendingCameraUri = uri
cameraLauncher.launch(uri)
} catch (e: Exception) {
Log.e("ChatScreen", "Camera launch failed", e)
}
},
onImageTap = { filePickerLauncher.launch("image/*") },
onFileTap = { filePickerLauncher.launch("*/*") },
onRemoveAttachment = viewModel::removeAttachment,
)
}
}
@@ -474,10 +548,20 @@ private fun ChatInputBar(
isAgentTyping: Boolean,
isConnected: Boolean,
commandCatalog: CommandCatalog,
pendingAttachments: List<Attachment> = emptyList(),
onCameraTap: () -> Unit = {},
onImageTap: () -> Unit = {},
onFileTap: () -> Unit = {},
onRemoveAttachment: (Int) -> Unit = {},
) {
// Allow sending slash commands even while agent is typing
val isSlashCommand = inputText.startsWith("/")
val canSend = inputText.isNotBlank() && isConnected && (!isAgentTyping || isSlashCommand)
val canSend =
(inputText.isNotBlank() || pendingAttachments.isNotEmpty()) &&
isConnected && (!isAgentTyping || isSlashCommand)
// Attachment menu state
var showAttachmentMenu by remember { mutableStateOf(false) }
AnimatedVisibility(
visible = true,
@@ -559,6 +643,28 @@ private fun ChatInputBar(
}
}
// Attachment preview chips
AnimatedVisibility(
visible = pendingAttachments.isNotEmpty(),
enter = fadeIn() + expandVertically(),
exit = fadeOut() + shrinkVertically(),
) {
Row(
modifier =
Modifier
.fillMaxWidth()
.padding(horizontal = 12.dp, vertical = 4.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
pendingAttachments.forEachIndexed { index, attachment ->
AttachmentChip(
attachment = attachment,
onRemove = { onRemoveAttachment(index) },
)
}
}
}
Row(
modifier =
Modifier
@@ -566,6 +672,66 @@ private fun ChatInputBar(
.padding(horizontal = 8.dp, vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Box {
// Single attach button that opens a dropdown menu
IconButton(
onClick = { showAttachmentMenu = true },
enabled = isConnected,
modifier = Modifier.size(36.dp),
) {
Icon(
imageVector = Icons.Default.AttachFile,
contentDescription = stringResource(R.string.chat_attach_desc),
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
DropdownMenu(
expanded = showAttachmentMenu,
onDismissRequest = { showAttachmentMenu = false },
) {
DropdownMenuItem(
text = { Text("Camera") },
onClick = {
showAttachmentMenu = false
onCameraTap()
},
leadingIcon = {
Text(
text = "📷",
fontSize = 18.sp,
)
},
)
DropdownMenuItem(
text = { Text("Image") },
onClick = {
showAttachmentMenu = false
onImageTap()
},
leadingIcon = {
Text(
text = "🖼️",
fontSize = 18.sp,
)
},
)
DropdownMenuItem(
text = { Text("File") },
onClick = {
showAttachmentMenu = false
onFileTap()
},
leadingIcon = {
Text(
text = "📄",
fontSize = 18.sp,
)
},
)
}
}
OutlinedTextField(
value = inputText,
onValueChange = onInputChange,
@@ -573,7 +739,7 @@ private fun ChatInputBar(
Modifier
.weight(1f)
.heightIn(min = 36.dp, max = 120.dp)
.padding(horizontal = 12.dp, vertical = 8.dp)
.padding(horizontal = 4.dp, vertical = 8.dp)
.testTag("chat_input"),
placeholder = {
Text(
@@ -599,14 +765,14 @@ private fun ChatInputBar(
),
)
Spacer(modifier = Modifier.width(6.dp))
Spacer(modifier = Modifier.width(4.dp))
// Mic / Stop / Send button — swaps based on input state
AnimatedContent(
targetState =
when {
isListening -> "listening"
inputText.isBlank() -> "mic"
inputText.isBlank() && pendingAttachments.isEmpty() -> "mic"
else -> "send"
},
transitionSpec = {
@@ -681,6 +847,66 @@ private fun ChatInputBar(
}
}
/**
* Compact chip showing a pending attachment with a remove button.
*/
@Composable
private fun AttachmentChip(
attachment: Attachment,
onRemove: () -> Unit,
) {
Surface(
shape = RoundedCornerShape(16.dp),
color = MaterialTheme.colorScheme.surfaceVariant,
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outline.copy(alpha = 0.3f)),
) {
Row(
modifier = Modifier.padding(start = 10.dp, end = 6.dp, top = 4.dp, bottom = 4.dp),
verticalAlignment = Alignment.CenterVertically,
) {
if (attachment.isImage) {
AsyncImage(
model = attachment.uri,
contentDescription = attachment.name,
modifier =
Modifier
.size(20.dp)
.clip(RoundedCornerShape(4.dp)),
contentScale = ContentScale.Crop,
)
Spacer(modifier = Modifier.width(4.dp))
} else {
Icon(
imageVector = Icons.AutoMirrored.Filled.InsertDriveFile,
contentDescription = null,
modifier = Modifier.size(16.dp),
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(modifier = Modifier.width(4.dp))
}
Text(
text = attachment.name,
style = MaterialTheme.typography.labelSmall,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
modifier = Modifier.widthIn(max = 120.dp),
)
Spacer(modifier = Modifier.width(4.dp))
IconButton(
onClick = onRemove,
modifier = Modifier.size(18.dp),
) {
Icon(
imageVector = Icons.Default.Close,
contentDescription = stringResource(R.string.chat_attach_remove_desc),
modifier = Modifier.size(14.dp),
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
}
@OptIn(ExperimentalFoundationApi::class)
@Composable
private fun ClarifyDialog(
@@ -1,11 +1,14 @@
package com.m57.hermescontrol.ui.chat
import android.app.Application
import android.net.Uri
import android.util.Base64
import android.util.Log
import androidx.lifecycle.AndroidViewModel
import androidx.lifecycle.viewModelScope
import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.local.HermesDatabase
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.remote.ApiClient
import com.m57.hermescontrol.data.remote.NetworkResult
import com.m57.hermescontrol.data.remote.OkHttpProvider
@@ -15,6 +18,7 @@ import com.m57.hermescontrol.data.ws.ConnectionStatus
import com.m57.hermescontrol.data.ws.HermesWsClient
import com.m57.hermescontrol.data.ws.WsEvent
import com.m57.hermescontrol.data.ws.WsMethods
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
@@ -27,6 +31,7 @@ import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.util.concurrent.ConcurrentHashMap
private const val TAG = "ChatViewModel"
@@ -37,6 +42,9 @@ private const val TAG = "ChatViewModel"
*/
private const val PENDING_REQUEST_TIMEOUT_MS = 30_000L
/** Thrown when an awaited RPC returns an error response. */
private class RpcCallException(message: String) : Exception(message)
data class ChatUiState(
val messages: List<ChatMessage> = emptyList(),
val currentSessionId: String? = null,
@@ -62,6 +70,8 @@ data class ChatUiState(
val typingEffectDelayMs: Int = 30,
// Commands catalog
val commandCatalog: CommandCatalog = CommandCatalog(),
// Attachment state
val pendingAttachments: List<Attachment> = emptyList(),
) {
/** Convenience — derived from [connectionStatus]. */
val isConnected: Boolean get() = connectionStatus == ConnectionStatus.CONNECTED
@@ -371,6 +381,9 @@ class ChatViewModel(
val pending = pendingRequests.remove(id) ?: return
val method = pending.method
// Complete the deferred so sendRpcAndAwait() callers can resume
pending.deferred?.complete(result)
when (method) {
WsMethods.SESSION_CREATE -> {
val resultMap = result as? Map<String, Any?> ?: return
@@ -511,18 +524,36 @@ class ChatViewModel(
is Map<*, *> -> error["message"] as? String ?: error.toString()
else -> error.toString()
}
_uiState.update {
it.copy(
isLoading = false,
errorMessage = "Error${if (method != null) " ($method)" else ""}: $errorMsg",
)
// Fail the deferred so sendRpcAndAwait() callers see the error
pending?.deferred?.completeExceptionally(RpcCallException(errorMsg))
// Only surface error in UI for non-awaited RPCs
if (pending?.deferred == null) {
_uiState.update {
it.copy(
isLoading = false,
errorMessage = "Error${if (method != null) " ($method)" else ""}: $errorMsg",
)
}
}
}
// ── Send message ─────────────────────────────────────────────────────
/**
* Send a user prompt, uploading any pending attachments to the backend
* first via their dedicated RPC methods.
*
* Flow:
* 1. Snapshot pending attachments (then clear them from UI)
* 2. Add user message to UI + DB immediately (optimistic UX)
* 3. For each image → fire-and-forget `image.attach_bytes` (queues into session)
* 4. For each file → await `file.attach`, collect @file: refs
* 5. Send `prompt.submit` with text + @file: refs — images auto-picked up by backend
*/
fun sendMessage(text: String) {
if (text.isBlank()) return
if (text.isBlank() && _uiState.value.pendingAttachments.isEmpty()) return
val sessionId = _uiState.value.currentSessionId ?: return
val trimmed = text.trim()
@@ -531,13 +562,18 @@ class ChatViewModel(
return
}
// Snapshot + clear attachments so the input bar empties immediately
val attachments = _uiState.value.pendingAttachments.toList()
clearAttachments()
val userMessage =
ChatMessage(
role = MessageRole.USER,
content = text,
attachments = if (attachments.isNotEmpty()) attachments else null,
)
// Update UI — no DB writes inside update{}
// Update UI immediately
_uiState.update { state ->
state.copy(
messages = state.messages + userMessage,
@@ -545,20 +581,149 @@ class ChatViewModel(
)
}
// Persist to Room — OUTSIDE update{}
// Persist to Room
viewModelScope.launch(Dispatchers.IO) {
repo.persistMessage(userMessage, sessionId)
}
// Upload attachments then submit prompt
viewModelScope.launch(Dispatchers.IO) {
val fileRefs = mutableListOf<String>()
for (attachment in attachments) {
val bytes = readContentUriBytes(attachment.uri)
if (bytes == null) {
Log.w(TAG, "Skipping unreadable attachment: ${attachment.name}")
continue
}
val b64 = Base64.encodeToString(bytes, Base64.NO_WRAP)
try {
if (attachment.isImage) {
// Fire-and-forget — backend queues into session["attached_images"],
// auto-picked by the subsequent prompt.submit
wsClient.send(
method = WsMethods.IMAGE_ATTACH_BYTES,
params =
mapOf(
"content_base64" to "data:${attachment.mimeType};base64,$b64",
"filename" to attachment.name,
"ext" to attachment.fileExtension,
),
)
} else {
// Await the @file: ref text so we can embed it in the prompt
sendRpcAndAwait(
method = WsMethods.FILE_ATTACH,
params =
mapOf(
"data_url" to "data:${attachment.mimeType};base64,$b64",
"name" to attachment.name,
),
)?.let { result ->
@Suppress("UNCHECKED_CAST")
val refText =
(result as? Map<String, Any?>)?.get("ref_text") as? String
if (!refText.isNullOrBlank()) fileRefs.add(refText)
}
}
} catch (e: Exception) {
Log.e(TAG, "Failed to upload attachment ${attachment.name}", e)
_uiState.update {
it.copy(errorMessage = "⚠️ Upload failed: ${attachment.name}")
}
}
}
// Build prompt text — prepend @file: refs for non-image files
val fullText =
if (fileRefs.isEmpty()) {
text
} else {
fileRefs.joinToString("\n") +
if (text.isNotBlank()) "\n\n$text" else ""
}
wsClient.sendMessage(
sessionId,
text,
fullText,
onSent = { id -> trackRequest(id, WsMethods.PROMPT_SUBMIT) },
)
}
}
/** Read bytes from a `content://` or `file://` URI via ContentResolver. */
private fun readContentUriBytes(uriString: String): ByteArray? {
return try {
val context = getApplication<Application>()
val uri = Uri.parse(uriString)
context.contentResolver.openInputStream(uri)?.use { stream -> stream.readBytes() }
} catch (e: Exception) {
Log.e(TAG, "Failed to read attachment bytes: ${e.message}", e)
null
}
}
/**
* Send a JSON-RPC call and suspend until the response arrives.
* Throws [RpcCallException] on error, [kotlinx.coroutines.TimeoutCancellationException] on timeout.
*/
private suspend fun sendRpcAndAwait(
method: String,
params: Map<String, Any>,
timeoutMs: Long = 30_000,
): Any? {
val deferred = CompletableDeferred<Any?>()
wsClient.send(
method = method,
params = params,
onSent = { id -> trackRequest(id, method, deferred) },
)
return withTimeout(timeoutMs) { deferred.await() }
}
// ── Attachment management ─────────────────────────────────────────────
/**
* Add a picked file as a pending attachment.
* [uri] should be a content:// URI string; the ViewModel will read
* the content and encode it for sending.
*/
fun addAttachment(
uri: String,
name: String,
mimeType: String,
size: Long,
) {
_uiState.update { state ->
val attachment =
Attachment(
uri = uri,
name = name,
mimeType = mimeType,
size = size,
)
state.copy(
pendingAttachments = state.pendingAttachments + attachment,
)
}
}
fun removeAttachment(index: Int) {
_uiState.update { state ->
state.copy(
pendingAttachments =
state.pendingAttachments.toMutableList().apply {
if (index in indices) removeAt(index)
},
)
}
}
fun clearAttachments() {
_uiState.update { it.copy(pendingAttachments = emptyList()) }
}
private fun handleSlashCommand(command: String) {
val userMsg = ChatMessage(role = MessageRole.USER, content = command)
val sessionId = _uiState.value.currentSessionId
@@ -1065,13 +1230,15 @@ class ChatViewModel(
private data class PendingRequest(
val method: String,
val createdAt: Long = System.currentTimeMillis(),
val deferred: CompletableDeferred<Any?>? = null,
)
private fun trackRequest(
id: String,
method: String,
deferred: CompletableDeferred<Any?>? = null,
) {
pendingRequests[id] = PendingRequest(method)
pendingRequests[id] = PendingRequest(method, deferred = deferred)
}
/**