feat(#724): full gateway-file support — attach MEDIA: files (images, audio, video, docs) (#725)

* fix(#724): inline host-path MEDIA: images on mobile (client-side)

The previous approach re-fetched the REST session-messages endpoint, but that
endpoint returns the raw MEDIA:<path> text (api_server._message_response does
not inline, and the transcript stores raw content). The desktop app instead
reads the file locally via readFileDataUrl because it is co-located with the
host. Mirror that on mobile: on assistant message.complete/done, inline any
MEDIA:<path> image into a base64 data:image/... markdown tag the renderer can
show. Mobile-only, backend untouched. Unreachable tags are stripped (graceful
degrade). Pure inlining logic extracted to MediaInline for unit testing.

* fix: correct RegexOption name (IGNORE_CASE) in MediaInline

* test: fix byte literal types in MediaInlineTest

* fix(#724): rewrite host-path MEDIA: to gateway /api/files/download URL

Pivot from the earlier base64-inline approach to the desktop-proven,
remote-phone-working fix: rewrite MEDIA:<path> directives (delivered raw over
the WS stream) into authenticated
![image](<baseUrl>/api/files/download?path=<enc>&token=<enc>) markdown tags,
mirroring desktop media.ts mediaExternalUrl. The mobile renderer (MarkdownText)
shows http(s) ![]() via Coil, so agent-sent images now render on mobile too —
including on a remote phone (real HTTP, unlike a host-local file read).
Mobile-only, backend untouched. Pure rewrite logic in MediaUrlRewriter
(replaces MediaInline) with unit tests.

* feat(#724): full gateway-file support — attach MEDIA: files (not just images)

Pivot the MEDIA: fix from image-only URL rewriting to real attachments:
- Attachment: add gatewayUrl + AttachmentSource (LOCAL/GATEWAY) so
  gateway-hosted files are first-class and render via existing InlineAttachment.
- HostMediaExtractor (replaces MediaUrlRewriter): parse ANY absolute MEDIA:
  directive (any extension) and strip them from message text.
- ChatViewModel.attachHostMedia: build the authed /api/files/download URL via
  GatewayFileClient, classify via MediaTypes, attach as Attachment (images
  inline, every other file tappable). Idempotent.
- InlineAttachment: tappable. Gateway files fetch via GatewayFileClient and
  share via a chooser Intent (FileProvider); images load the URL in Coil.
- Hence agent-delivered images, audio, video, CSV, PDF and arbitrary files all
  work on mobile — including a remote phone. Backend untouched.

Builds on #726 (GatewayFileClient). Rebased onto main after #726 merged.

* fix(#724): resolve imports, bubble callback scope, and quoted media path regex

* feat(chat): open attachment in-place on tap (#724)

Tap opens LOCAL (content://) and GATEWAY (fetch->FileProvider) files via
ACTION_VIEW; failures surface as a non-blocking snackbar via openError.

* fix: remove redundant out-of-order io.mockk.* wildcard import (ktlint CI)

ktlint import-ordering requires lexicographic order; the trailing wildcard
'import io.mockk.*' sorts before 'io.mockk.coEvery' and was redundant — every
mockk symbol is already explicitly imported, and the single mockkConstructor
call uses a fully-qualified io.mockk. prefix (needs no import).

Fixes ktlint failure on PR #725.

* fix(ci): mock android.jar Intent stub so openAttachment GATEWAY test passes

android.jar unit-test stubs Intent.setDataAndType/addFlags to throw
'... not mocked'. openWithView builds an Intent and calls them, so
openBytes' runCatching swallowed the exception and startActivity never
fired, failing the test. Mock the Intent constructor + setters and
capture the constructed instance to assert the ACTION_VIEW contract.

* fix(#724): resolve host media download URLs and deduplicate REST synced chat bubbles

* fix(#724): add FLAG_ACTIVITY_NEW_TASK for file intents and update GatewayFileClientTest

---------

Co-authored-by: M57 <hy4ri@users.noreply.github.com>
This commit is contained in:
M57andM57 authored and GitHub committed 2026-07-27 14:52:49 +03:00
1 parent 6aee02c4da
commit a1155e8015
14 files changed
+622 -25

No files matched your search

@@ -1,11 +1,16 @@
package com.m57.hermescontrol package com.m57.hermescontrol
import android.app.Application import android.app.Application
import coil.ImageLoader
import coil.ImageLoaderFactory
import com.m57.hermescontrol.data.local.AuthManager import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.remote.NetworkMonitor import com.m57.hermescontrol.data.remote.NetworkMonitor
import com.m57.hermescontrol.data.remote.OkHttpProvider
import com.m57.hermescontrol.ui.analytics.AnalyticsPreloader import com.m57.hermescontrol.ui.analytics.AnalyticsPreloader
class HermesControlApp : Application() { class HermesControlApp :
Application(),
ImageLoaderFactory {
override fun onCreate() { override fun onCreate() {
super.onCreate() super.onCreate()
AuthManager.init(this) AuthManager.init(this)
@@ -18,4 +23,11 @@ class HermesControlApp : Application() {
// cold backend). Fire-and-forget; never blocks UI startup. // cold backend). Fire-and-forget; never blocks UI startup.
AnalyticsPreloader.preload(this) AnalyticsPreloader.preload(this)
} }
override fun newImageLoader(): ImageLoader =
ImageLoader
.Builder(this)
.okHttpClient(OkHttpProvider.base)
.crossfade(true)
.build()
} }
@@ -1,28 +1,40 @@
package com.m57.hermescontrol.data.model package com.m57.hermescontrol.data.model
import kotlinx.serialization.Serializable import kotlinx.serialization.Serializable
/** /**
* Represents a file attached to a chat message. * Represents a file attached to a chat message.
* *
* @param uri Content URI or file path to the attachment * @param uri Content URI or file path to the attachment. For gateway-sourced
* attachments this is the authenticated `/api/files/download?...` URL, which
* Coil can load directly for images.
* @param name Display name of the file * @param name Display name of the file
* @param mimeType MIME type (e.g., "image/jpeg", "application/pdf") * @param mimeType MIME type (e.g., "image/jpeg", "application/pdf")
* @param size File size in bytes * @param size File size in bytes (0 when unknown, e.g. gateway-hosted files)
* @param localPath Optional local file system path for direct access * @param gatewayUrl When non-null, the authenticated gateway download URL for a
* file that lives on the *gateway host* (agent-delivered `MEDIA:` directive).
* Null for locally-picked attachments.
* @param source Where the attachment originated.
*/ */
@Serializable @Serializable
data class Attachment( data class Attachment(
val uri: String, val uri: String,
val name: String, val name: String,
val mimeType: String, val mimeType: String,
val size: Long, val size: Long = 0,
val gatewayUrl: String? = null,
val source: AttachmentSource = AttachmentSource.LOCAL,
) { ) {
val isImage: Boolean val isImage: Boolean
get() = mimeType.startsWith("image/") get() = mimeType.startsWith("image/")
val isGateway: Boolean
get() = source == AttachmentSource.GATEWAY
val formattedSize: String val formattedSize: String
get() = get() =
when { when {
size <= 0 -> "—"
size < 1024 -> "$size B" size < 1024 -> "$size B"
size < 1024 * 1024 -> "${size / 1024} KB" size < 1024 * 1024 -> "${size / 1024} KB"
else -> "%.1f MB".format(size.toDouble() / (1024 * 1024)) else -> "%.1f MB".format(size.toDouble() / (1024 * 1024))
@@ -31,3 +43,13 @@ data class Attachment(
val fileExtension: String val fileExtension: String
get() = name.substringAfterLast('.', "").lowercase() get() = name.substringAfterLast('.', "").lowercase()
} }
/** Origin of an [Attachment]. */
@Serializable
enum class AttachmentSource {
/** Picked by the user on this device (content URI / local path). */
LOCAL,
/** Delivered by the agent via a gateway-hosted `MEDIA:` directive. */
GATEWAY,
}
@@ -41,11 +41,15 @@ object GatewayFileClient {
path: String, path: String,
): String? { ): String? {
val trimmedBase = baseUrl.trimEnd('/') val trimmedBase = baseUrl.trimEnd('/')
if (trimmedBase.isBlank() || token.isBlank()) return null if (trimmedBase.isBlank()) return null
val norm = normalizePath(path) ?: return null val norm = normalizePath(path) ?: return null
val encPath = URLEncoder.encode(norm, StandardCharsets.UTF_8.name()).replace("+", "%20") val encPath = URLEncoder.encode(norm, StandardCharsets.UTF_8.name()).replace("+", "%20")
return if (token.isNotBlank()) {
val encToken = URLEncoder.encode(token, StandardCharsets.UTF_8.name()).replace("+", "%20") val encToken = URLEncoder.encode(token, StandardCharsets.UTF_8.name()).replace("+", "%20")
return "$trimmedBase$DOWNLOAD_PATH?path=$encPath&token=$encToken" "$trimmedBase$DOWNLOAD_PATH?path=$encPath&token=$encToken"
} else {
"$trimmedBase$DOWNLOAD_PATH?path=$encPath"
}
} }
/** /**
@@ -114,6 +114,7 @@ fun ChatBubble(
searchQuery: String = "", searchQuery: String = "",
isCurrentMatch: Boolean = false, isCurrentMatch: Boolean = false,
onRespondApproval: (String) -> Unit = {}, onRespondApproval: (String) -> Unit = {},
onOpenAttachment: (Attachment) -> Unit = {},
modifier: Modifier = Modifier, modifier: Modifier = Modifier,
) { ) {
val screenWidth = LocalConfiguration.current.screenWidthDp.dp val screenWidth = LocalConfiguration.current.screenWidthDp.dp
@@ -129,17 +130,25 @@ fun ChatBubble(
) { ) {
when (message.role) { when (message.role) {
MessageRole.USER -> { MessageRole.USER -> {
UserBubble(message, maxBubbleWidth, searchQuery, isCurrentMatch, modifier) UserBubble(
message = message,
maxWidth = maxBubbleWidth,
searchQuery = searchQuery,
isCurrentMatch = isCurrentMatch,
onOpenAttachment = onOpenAttachment,
modifier = modifier,
)
} }
MessageRole.ASSISTANT -> { MessageRole.ASSISTANT -> {
AssistantBubble( AssistantBubble(
message, message = message,
maxBubbleWidth, maxWidth = maxBubbleWidth,
isDarkTheme, isDarkTheme = isDarkTheme,
searchQuery, searchQuery = searchQuery,
isCurrentMatch, isCurrentMatch = isCurrentMatch,
modifier, onOpenAttachment = onOpenAttachment,
modifier = modifier,
) )
} }
@@ -164,6 +173,7 @@ private fun UserBubble(
maxWidth: androidx.compose.ui.unit.Dp, maxWidth: androidx.compose.ui.unit.Dp,
searchQuery: String = "", searchQuery: String = "",
isCurrentMatch: Boolean = false, isCurrentMatch: Boolean = false,
onOpenAttachment: (Attachment) -> Unit = {},
modifier: Modifier = Modifier, modifier: Modifier = Modifier,
) { ) {
val clipboard = LocalClipboard.current val clipboard = LocalClipboard.current
@@ -263,6 +273,7 @@ private fun UserBubble(
InlineAttachment( InlineAttachment(
attachment = attachment, attachment = attachment,
textColor = userBubbleTextColor, textColor = userBubbleTextColor,
onOpen = { onOpenAttachment(it) },
) )
Spacer(modifier = Modifier.height(4.dp)) Spacer(modifier = Modifier.height(4.dp))
} }
@@ -325,6 +336,7 @@ private fun AssistantBubble(
isDarkTheme: Boolean, isDarkTheme: Boolean,
searchQuery: String = "", searchQuery: String = "",
isCurrentMatch: Boolean = false, isCurrentMatch: Boolean = false,
onOpenAttachment: (Attachment) -> Unit = {},
modifier: Modifier = Modifier, modifier: Modifier = Modifier,
) { ) {
val bubbleColor = if (isDarkTheme) AssistantBubble else AssistantBubbleLight val bubbleColor = if (isDarkTheme) AssistantBubble else AssistantBubbleLight
@@ -414,6 +426,7 @@ private fun AssistantBubble(
InlineAttachment( InlineAttachment(
attachment = attachment, attachment = attachment,
textColor = textColor, textColor = textColor,
onOpen = { onOpenAttachment(it) },
) )
Spacer(modifier = Modifier.height(4.dp)) Spacer(modifier = Modifier.height(4.dp))
} }
@@ -2293,24 +2306,30 @@ private fun buildHighlightedString(
private fun InlineAttachment( private fun InlineAttachment(
attachment: Attachment, attachment: Attachment,
textColor: Color, textColor: Color,
onOpen: (Attachment) -> Unit = {},
) { ) {
val clickable = Modifier.clickable { onOpen(attachment) }
if (attachment.isImage) { if (attachment.isImage) {
// Image attachment — show as a rounded thumbnail // Image attachment — show as a rounded thumbnail (Coil loads the
// gateway download URL directly when gateway-sourced).
AsyncImage( AsyncImage(
model = attachment.uri, model = attachment.uri,
contentDescription = attachment.name, contentDescription = attachment.name,
modifier = modifier =
Modifier Modifier
.fillMaxWidth() .fillMaxWidth()
.clip(RoundedCornerShape(12.dp)), .clip(RoundedCornerShape(12.dp))
.then(clickable),
contentScale = ContentScale.FillWidth, contentScale = ContentScale.FillWidth,
) )
} else { } else {
// Non-image file — show a card with file icon and name // Non-image file — show a card with file icon and name. Tapping fetches
// the bytes (gateway-sourced) or opens the local URI.
Surface( Surface(
shape = RoundedCornerShape(8.dp), shape = RoundedCornerShape(8.dp),
color = textColor.copy(alpha = 0.1f), color = textColor.copy(alpha = 0.1f),
border = BorderStroke(1.dp, textColor.copy(alpha = 0.2f)), border = BorderStroke(1.dp, textColor.copy(alpha = 0.2f)),
modifier = clickable,
) { ) {
Row( Row(
modifier = Modifier.padding(8.dp), modifier = Modifier.padding(8.dp),
@@ -334,6 +334,7 @@ fun ChatScreen(
messages = state.messages, messages = state.messages,
errorMessage = state.errorMessage, errorMessage = state.errorMessage,
backgroundCompleteMessage = state.backgroundCompleteMessage, backgroundCompleteMessage = state.backgroundCompleteMessage,
openError = state.openError,
isSearchActive = state.isSearchActive, isSearchActive = state.isSearchActive,
currentSearchMatchIndex = state.currentSearchMatchIndex, currentSearchMatchIndex = state.currentSearchMatchIndex,
searchMatchIndices = state.searchMatchIndices, searchMatchIndices = state.searchMatchIndices,
@@ -10,10 +10,14 @@ import androidx.lifecycle.viewModelScope
import com.m57.hermescontrol.data.local.AuthManager import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.local.HermesDatabase import com.m57.hermescontrol.data.local.HermesDatabase
import com.m57.hermescontrol.data.model.Attachment import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.model.AttachmentSource
import com.m57.hermescontrol.data.model.ModelProvider import com.m57.hermescontrol.data.model.ModelProvider
import com.m57.hermescontrol.data.model.PinnedModel import com.m57.hermescontrol.data.model.PinnedModel
import com.m57.hermescontrol.data.model.SessionMessage import com.m57.hermescontrol.data.model.SessionMessage
import com.m57.hermescontrol.data.remote.ApiClient import com.m57.hermescontrol.data.remote.ApiClient
import com.m57.hermescontrol.data.remote.GatewayFile
import com.m57.hermescontrol.data.remote.GatewayFileClient
import com.m57.hermescontrol.data.remote.GatewayFileResult
import com.m57.hermescontrol.data.remote.NetworkResult import com.m57.hermescontrol.data.remote.NetworkResult
import com.m57.hermescontrol.data.remote.OkHttpProvider import com.m57.hermescontrol.data.remote.OkHttpProvider
import com.m57.hermescontrol.data.remote.safeApiCall import com.m57.hermescontrol.data.remote.safeApiCall
@@ -66,6 +70,8 @@ data class ChatUiState(
val errorMessage: String? = null, val errorMessage: String? = null,
// Background job completion toast (issue #527) — non-blocking snackbar // Background job completion toast (issue #527) — non-blocking snackbar
val backgroundCompleteMessage: String? = null, val backgroundCompleteMessage: String? = null,
// Attachment open failure — surfaced as a non-blocking snackbar (issue #724)
val openError: String? = null,
val clarifyRequest: ClarifyUi? = null, val clarifyRequest: ClarifyUi? = null,
// Sudo / secret prompts — surfaced as dialogs (issue #524) // Sudo / secret prompts — surfaced as dialogs (issue #524)
val sudoPrompt: SudoPromptUi? = null, val sudoPrompt: SudoPromptUi? = null,
@@ -415,6 +421,16 @@ class ChatViewModel(
// than waiting up to 5s for the next session-sync poll. // than waiting up to 5s for the next session-sync poll.
viewModelScope.launch { fetchContextUsage() } viewModelScope.launch { fetchContextUsage() }
} }
is ReducerEffect.AttachHostMedia -> {
// Issue #724: turn host-path MEDIA: directives into real
// attachments (images inline, every other file tappable)
// via the gateway /api/files/download endpoint. Works on a
// remote phone too.
viewModelScope.launch(Dispatchers.IO) {
attachHostMedia(effect.sessionId, effect.messageId)
}
}
} }
} }
@@ -1052,7 +1068,8 @@ class ChatViewModel(
// #576). For those we fall back to slash.exec, which runs the // #576). For those we fall back to slash.exec, which runs the
// full COMMAND_REGISTRY through the worker. // full COMMAND_REGISTRY through the worker.
val result = val result =
wsClient.request( wsClient
.request(
WsMethods.COMMAND_DISPATCH, WsMethods.COMMAND_DISPATCH,
mapOf("name" to name, "arg" to arg, "session_id" to sessionId), mapOf("name" to name, "arg" to arg, "session_id" to sessionId),
).await() ).await()
@@ -1068,7 +1085,8 @@ class ChatViewModel(
// which routes the full CLI command set through the worker. // which routes the full CLI command set through the worker.
try { try {
val result = val result =
wsClient.request( wsClient
.request(
WsMethods.SLASH_EXEC, WsMethods.SLASH_EXEC,
mapOf( mapOf(
"command" to "/$name${if (arg.isNotEmpty()) " $arg" else ""}", "command" to "/$name${if (arg.isNotEmpty()) " $arg" else ""}",
@@ -1698,6 +1716,9 @@ class ChatViewModel(
.filter { it.reasoningText.isNotBlank() } .filter { it.reasoningText.isNotBlank() }
.associateBy { it.content } .associateBy { it.content }
val baseUrl = AuthManager.getBaseUrl()
val token = AuthManager.getToken().orEmpty()
return messages.mapIndexed { index, msg -> return messages.mapIndexed { index, msg ->
val role = val role =
when (msg.role?.lowercase()) { when (msg.role?.lowercase()) {
@@ -1714,25 +1735,249 @@ class ChatViewModel(
?.toLong() ?.toLong()
?: System.currentTimeMillis() ?: System.currentTimeMillis()
val content = msg.contentText val rawContent = msg.contentText
val reasoning = val reasoning =
if (msg.reasoningText.isNotBlank()) { if (msg.reasoningText.isNotBlank()) {
msg.reasoningText msg.reasoningText
} else { } else {
existingReasoningMap[content]?.reasoningText.orEmpty() existingReasoningMap[rawContent]?.reasoningText.orEmpty()
}
var finalContent = rawContent
var attachments: List<Attachment>? = null
if (role == MessageRole.ASSISTANT && rawContent.contains("MEDIA:")) {
val items = HostMediaExtractor.extract(rawContent)
if (items.isNotEmpty()) {
finalContent = HostMediaExtractor.strip(rawContent)
attachments =
items
.mapNotNull { item ->
val url =
GatewayFileClient.buildDownloadUrl(
baseUrl,
token,
item.path,
) ?: return@mapNotNull null
Attachment(
uri = url,
name = mediaNameFromPath(item.path),
mimeType = mediaMimeForPath(item.path),
size = 0,
gatewayUrl = url,
source = AttachmentSource.GATEWAY,
)
}.takeIf { it.isNotEmpty() }
}
} }
ChatMessage( ChatMessage(
id = "rest-$sessionId-$globalIndex", id = "rest-$sessionId-$globalIndex",
role = role, role = role,
content = content, content = finalContent,
reasoningText = reasoning, reasoningText = reasoning,
attachments = attachments,
timestamp = timestamp, timestamp = timestamp,
isStreaming = false, isStreaming = false,
) )
} }
} }
// ── Issue #724: attach host-path MEDIA: files as real attachments ────
//
// The gateway's WebSocket stream delivers the raw `MEDIA:<path>` directive
// the desktop app turns into an authenticated `/api/files/download?...`
// URL. We parse every directive, build the download URL via
// [GatewayFileClient], classify it (image / audio / video / file) using
// [mediaKindForPath], and attach it to the message. Images render inline
// (Coil loads the URL); every other type becomes a tappable, fetchable
// attachment. The directive text is stripped from the message body. Works
// on a remote phone (real HTTP). Mobile-only; backend untouched. Pure
// parsing lives in [HostMediaExtractor].
/**
* ViewModel-side handler for [ReducerEffect.AttachHostMedia]: find the local
* message by id, convert any `MEDIA:<path>` directives into [Attachment]s
* (via the gateway download URL) and strip them from the text. Role,
* reasoning, timestamp and existing attachments are preserved; new gateway
* attachments are appended. Idempotent — skips if gateway attachments for
* the same paths already exist.
*/
private fun attachHostMedia(
sessionId: String,
messageId: String,
) {
val current = _uiState.value.messages.find { it.id == messageId } ?: return
val content = current.content
val items = HostMediaExtractor.extract(content)
if (items.isEmpty()) return
val baseUrl = AuthManager.getBaseUrl()
val token = AuthManager.getToken()
if (baseUrl.isBlank() || token.isNullOrBlank()) return
val existingUrls =
current.attachments
.orEmpty()
.mapNotNull { it.gatewayUrl }
.toSet()
val newAttachments =
items.mapNotNull { item ->
val url = GatewayFileClient.buildDownloadUrl(baseUrl, token, item.path) ?: return@mapNotNull null
if (url in existingUrls) return@mapNotNull null
Attachment(
uri = url,
name = mediaNameFromPath(item.path),
mimeType = mediaMimeForPath(item.path),
size = 0,
gatewayUrl = url,
source = AttachmentSource.GATEWAY,
)
}
if (newAttachments.isEmpty()) return
val stripped = HostMediaExtractor.strip(content)
_uiState.update { state ->
state.copy(
messages =
state.messages.map { msg ->
if (msg.id == messageId) {
msg.copy(
content = stripped,
attachments =
(msg.attachments.orEmpty() + newAttachments)
.distinctBy { it.gatewayUrl ?: it.uri },
)
} else {
msg
}
},
)
}
}
/**
* Open an attachment when its chip/thumbnail is tapped.
*
* - LOCAL (user-picked) files: open the original `content://` URI
* directly via [android.content.Intent.ACTION_VIEW] — the resolver
* already grants read access for the picked document. If that fails
* (e.g. the permission lapsed), we copy to cache and retry via
* FileProvider so the tap is never a silent no-op.
* - GATEWAY (agent `MEDIA:`) files: fetch the bytes via
* [GatewayFileClient], write them to a cache file, and open with
* [android.content.Intent.ACTION_VIEW] through FileProvider — so a
* remote phone can view agent-delivered files in-place.
*
* Failures surface through [ChatUiState.openError] (non-blocking
* snackbar); the tap is never swallowed.
*/
fun openAttachment(attachment: Attachment) {
val ctx = getApplication<Application>().applicationContext
if (attachment.source == AttachmentSource.LOCAL) {
// Best-effort direct open of the picked content URI.
runCatching { openWithView(ctx, android.net.Uri.parse(attachment.uri), attachment.mimeType) }
.onSuccess { return }
.onFailure { /* fall through to cache-copy below */ }
}
// GATEWAY, or LOCAL direct-open failed → fetch/copy then open.
val path =
attachment.gatewayUrl?.let { url ->
runCatching {
java.net
.URL(url)
.query
.split('&')
.firstOrNull { it.startsWith("path=") }
?.removePrefix("path=")
?.let { java.net.URLDecoder.decode(it, "UTF-8") }
}.getOrNull()
} ?: attachment.name
viewModelScope.launch(Dispatchers.IO) {
when (val result = GatewayFileClient.fetch(path)) {
is GatewayFileResult.Success -> {
openBytes(ctx, result.file)
}
is GatewayFileResult.NotFound -> {
showOpenError("File not found on gateway: ${attachment.name}")
}
is GatewayFileResult.Forbidden -> {
showOpenError("Access denied: ${attachment.name}")
}
is GatewayFileResult.TooLarge -> {
showOpenError("File too large to open: ${attachment.name}")
}
is GatewayFileResult.Unauthorized -> {
showOpenError("Session expired — reconnect to open: ${attachment.name}")
}
is GatewayFileResult.Failure -> {
showOpenError("Could not open ${attachment.name}: ${result.throwable.message}")
}
}
}
}
/** Open bytes written to a cache file via FileProvider + ACTION_VIEW. */
private fun openBytes(
ctx: android.content.Context,
file: GatewayFile,
) {
runCatching {
val dir = java.io.File(ctx.cacheDir, "gateway_files").also { it.mkdirs() }
val safeName = file.name.replace(Regex("[/\\\\]"), "_").ifBlank { "file" }
val out = java.io.File(dir, safeName)
out.writeBytes(file.bytes)
val uri =
androidx.core.content.FileProvider.getUriForFile(
ctx,
"${ctx.packageName}.fileprovider",
out,
)
openWithView(ctx, uri, file.mimeType)
}.onFailure { showOpenError("Could not open ${file.name}: ${it.message}") }
}
/** Fire an ACTION_VIEW intent; throws if no activity can handle the type. */
private fun openWithView(
ctx: android.content.Context,
uri: android.net.Uri,
mimeType: String,
) {
val viewIntent =
android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
setDataAndType(uri, mimeType.ifBlank { "*/*" })
addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
}
try {
ctx.startActivity(viewIntent)
} catch (e: Throwable) {
val fallbackIntent =
android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
setDataAndType(uri, "*/*")
addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
}
val chooser =
android.content.Intent.createChooser(fallbackIntent, "Open file").apply {
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
}
ctx.startActivity(chooser)
}
}
private fun showOpenError(message: String) {
_uiState.update { it.copy(openError = message) }
}
fun clearOpenError() {
_uiState.update { it.copy(openError = null) }
}
private fun serverMessageIndex( private fun serverMessageIndex(
id: String, id: String,
sessionId: String, sessionId: String,
@@ -255,6 +255,13 @@ object ChatWsEventReducer {
// Refresh the context meter immediately on completion instead of // Refresh the context meter immediately on completion instead of
// waiting for the next 5s session-sync poll. // waiting for the next 5s session-sync poll.
effects.add(ReducerEffect.RefreshContextUsage) effects.add(ReducerEffect.RefreshContextUsage)
// Mobile-only media fix (issue #724): the WS stream delivers the raw
// `MEDIA:<host-path>` directive. Turn it into real attachments (images
// inline, every other file tappable) via the gateway /api/files/download
// endpoint — works on a remote phone too.
if (msg.role == MessageRole.ASSISTANT && sid != null) {
effects.add(ReducerEffect.AttachHostMedia(sid, msg.id))
}
return ReducerResult( return ReducerResult(
state = state =
state.copy( state.copy(
@@ -288,6 +295,12 @@ object ChatWsEventReducer {
if (sid != null) { if (sid != null) {
effects.add(ReducerEffect.PersistMessage(msg, sid)) effects.add(ReducerEffect.PersistMessage(msg, sid))
} }
// Mobile-only media fix (issue #724): attach any `MEDIA:<path>` file
// carried in the finalized assistant message as a real attachment. See
// the ReducerEffect.AttachHostMedia KDoc for the rationale.
if (msg.role == MessageRole.ASSISTANT && sid != null) {
effects.add(ReducerEffect.AttachHostMedia(sid, msg.id))
}
return ReducerResult( return ReducerResult(
state = state =
state.copy( state.copy(
@@ -669,4 +682,22 @@ sealed class ReducerEffect {
data object RefreshSessions : ReducerEffect() data object RefreshSessions : ReducerEffect()
data object RefreshContextUsage : ReducerEffect() data object RefreshContextUsage : ReducerEffect()
/**
* Ask the ViewModel to turn any `MEDIA:<host-path>` directives in the
* just-finalized assistant message into real [Attachment]s (images inline,
* every other file type tappable) and strip the directives from the text.
*
* Why: the gateway's WebSocket stream delivers the raw `MEDIA:<path>`
* directive (the same one the desktop app resolves via `mediaExternalUrl`
* to the authenticated `/api/files/download` endpoint, which streams the
* host file over HTTP). Attaching them lets the renderer show images inline
* and offer every other file as a fetchable attachment — even on a remote
* phone (real HTTP, unlike a host-local file read). Mirrors desktop's
* `mediaExternalUrl`; mobile-only, backend untouched. See issue #724.
*/
data class AttachHostMedia(
val sessionId: String,
val messageId: String,
) : ReducerEffect()
} }
@@ -0,0 +1,77 @@
package com.m57.hermescontrol.ui.chat
/**
* Issue #724: parse agent-delivered `MEDIA:<path>` directives out of message
* text so they can be attached (and rendered) as real files.
*
* The gateway's WebSocket stream delivers the raw `MEDIA:<path>` directive the
* desktop app resolves via `mediaExternalUrl` — an authenticated
* `/api/files/download?path=<enc>&token=<enc>` URL that streams the host file's
* bytes over HTTP. Mobile turns each directive into an [Attachment] (see
* [com.m57.hermescontrol.ui.chat.ChatViewModel.attachHostMedia]) so the
* renderer can show images inline and offer every other file type as a
* tappable, fetchable attachment — including on a *remote* phone.
*
* Matches ANY absolute (or `~/`) host path regardless of extension, so images,
* audio, video, CSV, PDF and arbitrary files are all supported. The pure parse
* logic lives here so it can be unit-tested without an Android context.
*/
internal object HostMediaExtractor {
/** A single `MEDIA:<path>` directive found in message text. */
data class Item(
val match: String, // the full matched directive (for stripping)
val path: String, // normalized absolute path
)
/** Matches `MEDIA:<path>` (any extension), supporting quotes/backticks around the path. */
private val MEDIA_TAG_RE =
Regex(
"""[`"']?MEDIA:\s*(?:"([^"]+)"|'([^']+)'|`([^`]+)`|((?:~|/|[A-Za-z]:)[^\s`"')]*))[`"']?""",
RegexOption.IGNORE_CASE,
)
/** Extract every `MEDIA:<path>` directive from [text] as a normalized item. */
fun extract(text: String): List<Item> {
if (!text.contains("MEDIA:")) return emptyList()
return MEDIA_TAG_RE
.findAll(text)
.mapNotNull { m ->
val raw = m.groupValues.drop(1).firstOrNull { it.isNotBlank() } ?: return@mapNotNull null
val path = normalizePath(raw) ?: return@mapNotNull null
Item(match = m.value, path = path)
}.toList()
}
/** Remove all `MEDIA:<path>` directives from [text] (leaves other content). */
fun strip(text: String): String {
if (!text.contains("MEDIA:")) return text
return MEDIA_TAG_RE
.replace(text) { "" }
.replace(Regex("""[ \t]+"""), " ")
.replace(Regex("""\n{3,}"""), "\n\n")
.trim()
}
/** Expand ~/ and verify the path is absolute. Null if not an absolute path. */
internal fun normalizePath(raw: String): String? {
val trimmed =
raw
.trim()
.removeSurrounding("`")
.removeSurrounding("\"")
.removeSurrounding("'")
val expanded =
if (trimmed.startsWith("~")) {
val home = System.getenv("HOME") ?: return null
home + trimmed.removePrefix("~")
} else {
trimmed
}
if (!expanded.startsWith("/") &&
!Regex("^[A-Za-z]:[/\\\\]").containsMatchIn(expanded)
) {
return null
}
return expanded
}
}
@@ -43,6 +43,8 @@ import androidx.compose.ui.text.style.TextDecoration
import androidx.compose.ui.text.withStyle import androidx.compose.ui.text.withStyle
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp import androidx.compose.ui.unit.sp
import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.remote.GatewayFileClient
import com.m57.hermescontrol.theme.LocalHermesStatusColors import com.m57.hermescontrol.theme.LocalHermesStatusColors
import com.m57.hermescontrol.theme.SearchHighlightColors import com.m57.hermescontrol.theme.SearchHighlightColors
import com.m57.hermescontrol.theme.searchHighlightColors import com.m57.hermescontrol.theme.searchHighlightColors
@@ -268,7 +270,7 @@ fun MarkdownText(
// AsyncImage is a Compose runtime composable — load it lazily // AsyncImage is a Compose runtime composable — load it lazily
// behind remember so the parser stays pure and testable (the // behind remember so the parser stays pure and testable (the
// parseBlocks/parseInline unit tests never touch Compose). // parseBlocks/parseInline unit tests never touch Compose).
val model: Any = block.uri val model: Any = remember(block.uri) { resolveImageUrl(block.uri) }
androidx.compose.foundation.layout.Box( androidx.compose.foundation.layout.Box(
modifier = modifier =
Modifier Modifier
@@ -1000,3 +1002,26 @@ internal enum class TableAlign {
CENTER, CENTER,
RIGHT, RIGHT,
} }
private fun resolveImageUrl(uri: String): String {
val trimmed = uri.trim()
if (trimmed.isBlank()) return uri
if (trimmed.startsWith("http://") || trimmed.startsWith("https://") || trimmed.startsWith("data:image/")) {
return trimmed
}
val baseUrl = runCatching { AuthManager.getBaseUrl() }.getOrDefault("")
val token = runCatching { AuthManager.getToken() }.getOrNull().orEmpty()
val downloadUrl = GatewayFileClient.buildDownloadUrl(baseUrl, token, trimmed)
if (downloadUrl != null) return downloadUrl
if (baseUrl.isNotBlank() && (trimmed.startsWith("/api/") || trimmed.startsWith("api/"))) {
val cleanPath = if (trimmed.startsWith("/")) trimmed else "/$trimmed"
val sep = if (cleanPath.contains("?")) "&" else "?"
return if (token.isNotBlank() && !cleanPath.contains("token=")) {
"${baseUrl.trimEnd('/')}$cleanPath${sep}token=$token"
} else {
"${baseUrl.trimEnd('/')}$cleanPath"
}
}
return uri
}
@@ -33,6 +33,7 @@ fun ChatLifecycleEffects(
messages: List<ChatMessage>, messages: List<ChatMessage>,
errorMessage: String?, errorMessage: String?,
backgroundCompleteMessage: String?, backgroundCompleteMessage: String?,
openError: String?,
isSearchActive: Boolean, isSearchActive: Boolean,
currentSearchMatchIndex: Int, currentSearchMatchIndex: Int,
searchMatchIndices: List<Int>, searchMatchIndices: List<Int>,
@@ -127,6 +128,14 @@ fun ChatLifecycleEffects(
} }
} }
// Show attachment-open failures as a non-blocking snackbar (issue #724)
LaunchedEffect(openError) {
openError?.let { message ->
snackbarHostState.showSnackbar(message)
viewModel.clearOpenError()
}
}
// Sudo / secret prompt dialogs (issue #524) // Sudo / secret prompt dialogs (issue #524)
sudoPrompt?.let { prompt -> sudoPrompt?.let { prompt ->
SudoPromptDialog( SudoPromptDialog(
@@ -108,6 +108,7 @@ fun ChatMessageList(
searchQuery = if (isSearchActive) searchQuery else "", searchQuery = if (isSearchActive) searchQuery else "",
isCurrentMatch = isCurrentMatch, isCurrentMatch = isCurrentMatch,
onRespondApproval = viewModel::respondToApproval, onRespondApproval = viewModel::respondToApproval,
onOpenAttachment = viewModel::openAttachment,
) )
} }
} }
@@ -127,6 +128,7 @@ fun ChatMessageList(
isDarkTheme = isDark, isDarkTheme = isDark,
searchQuery = "", searchQuery = "",
isCurrentMatch = false, isCurrentMatch = false,
onOpenAttachment = viewModel::openAttachment,
) )
} }
} }
@@ -25,9 +25,15 @@ class GatewayFileClientTest {
} }
@Test @Test
fun `buildDownloadUrl rejects blank base or token`() { fun `buildDownloadUrl rejects blank base`() {
assertNull(GatewayFileClient.buildDownloadUrl("", tok, "/tmp/x.png")) assertNull(GatewayFileClient.buildDownloadUrl("", tok, "/tmp/x.png"))
assertNull(GatewayFileClient.buildDownloadUrl(base, "", "/tmp/x.png")) }
@Test
fun `buildDownloadUrl permits blank token for cookie auth`() {
val url = GatewayFileClient.buildDownloadUrl(base, "", "/tmp/x.png")!!
assertEquals("$base/api/files/download?path=%2Ftmp%2Fx.png", url)
assertFalse(url.contains("token="))
} }
@Test @Test
@@ -2,18 +2,27 @@ package com.m57.hermescontrol.ui.chat
import android.app.Application import android.app.Application
import android.content.ContentResolver import android.content.ContentResolver
import android.content.Intent
import android.net.Uri import android.net.Uri
import android.util.Log import android.util.Log
import androidx.core.content.FileProvider
import com.m57.hermescontrol.data.local.AuthManager import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.local.HermesDatabase import com.m57.hermescontrol.data.local.HermesDatabase
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.model.AttachmentSource
import com.m57.hermescontrol.data.remote.ApiClient import com.m57.hermescontrol.data.remote.ApiClient
import com.m57.hermescontrol.data.remote.GatewayFile
import com.m57.hermescontrol.data.remote.GatewayFileClient
import com.m57.hermescontrol.data.remote.GatewayFileResult
import com.m57.hermescontrol.data.ws.ConnectionStatus import com.m57.hermescontrol.data.ws.ConnectionStatus
import com.m57.hermescontrol.data.ws.HermesWsClient import com.m57.hermescontrol.data.ws.HermesWsClient
import com.m57.hermescontrol.data.ws.JsonRpcError import com.m57.hermescontrol.data.ws.JsonRpcError
import com.m57.hermescontrol.data.ws.WsEvent import com.m57.hermescontrol.data.ws.WsEvent
import com.m57.hermescontrol.data.ws.WsMethods import com.m57.hermescontrol.data.ws.WsMethods
import com.m57.hermescontrol.ui.chat.fakes.FakeChatPersistenceRepository import com.m57.hermescontrol.ui.chat.fakes.FakeChatPersistenceRepository
import io.mockk.*
import io.mockk.coEvery import io.mockk.coEvery
import io.mockk.coVerify
import io.mockk.every import io.mockk.every
import io.mockk.mockk import io.mockk.mockk
import io.mockk.mockkObject import io.mockk.mockkObject
@@ -1946,4 +1955,83 @@ class ChatViewModelTest {
assertEquals(1, messages.size) assertEquals(1, messages.size)
assertEquals("{\"status\":\"ok\"}", messages[0].content) assertEquals("{\"status\":\"ok\"}", messages[0].content)
} }
// ── Attachment open (issue #724) ─────────────────────────────────────
@Test
fun `openAttachment GATEWAY success fires ACTION_VIEW with FileProvider uri`() =
runTest {
val cacheDir =
java.io.File(System.getProperty("java.io.tmpdir"), "hermes_open_test_${System.nanoTime()}")
cacheDir.mkdirs()
every { app.cacheDir } returns cacheDir
mockkObject(GatewayFileClient)
val bytes = "hello".toByteArray()
coEvery {
GatewayFileClient.fetch(any())
} returns GatewayFileResult.Success(GatewayFile("note.txt", "text/plain", bytes))
val intentSlot = slot<Intent>()
// android.jar stubs Intent ctor/setters to throw "not mocked" in unit tests;
// mock the Intent constructor so openWithView can build + deliver it,
// and capture the constructed instance to assert on its setters.
mockkConstructor(Intent::class)
every { anyConstructed<Intent>().setDataAndType(any(), any()) } answers { self as Intent }
every { anyConstructed<Intent>().addFlags(any()) } answers { self as Intent }
mockkStatic(FileProvider::class)
every {
FileProvider.getUriForFile(any(), any(), any())
} returns mockk(relaxed = true)
every { app.getApplicationContext() } returns app
every { app.applicationContext } returns app
every { app.startActivity(capture(intentSlot)) } returns Unit
val vm = createViewModel()
val attachment =
Attachment(
uri = "unused",
name = "note.txt",
mimeType = "text/plain",
gatewayUrl = "https://gw/api/files/download?path=%2Ftmp%2Fnote.txt&token=t",
source = AttachmentSource.GATEWAY,
)
vm.openAttachment(attachment)
advanceUntilIdle()
// fetch was invoked (proves we entered the IO launch)
coVerify { GatewayFileClient.fetch(any()) }
// ACTION_VIEW intent delivered, with the right type + grant flag.
verify { app.startActivity(any()) }
verify { intentSlot.captured.setDataAndType(any(), eq("text/plain")) }
verify { intentSlot.captured.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION) }
assertNull(vm.uiState.value.openError)
cacheDir.deleteRecursively()
}
@Test
fun `openAttachment GATEWAY not-found surfaces openError`() =
runTest {
mockkObject(GatewayFileClient)
coEvery {
GatewayFileClient.fetch(any())
} returns GatewayFileResult.NotFound
val vm = createViewModel()
val attachment =
Attachment(
uri = "unused",
name = "missing.pdf",
mimeType = "application/pdf",
gatewayUrl = "https://gw/api/files/download?path=%2Ftmp%2Fmissing.pdf&token=t",
source = AttachmentSource.GATEWAY,
)
vm.openAttachment(attachment)
advanceUntilIdle()
assertNotNull(vm.uiState.value.openError)
assertTrue(vm.uiState.value.openError!!.contains("missing.pdf"))
}
} }
@@ -0,0 +1,56 @@
package com.m57.hermescontrol.ui.chat
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Test
class HostMediaExtractorTest {
@Test
fun `extracts single image path`() {
val items = HostMediaExtractor.extract("Here: MEDIA:/tmp/foo.png")
assertEquals(1, items.size)
assertEquals("/tmp/foo.png", items[0].path)
assertEquals("MEDIA:/tmp/foo.png", items[0].match)
}
@Test
fun `extracts multiple directives of any type`() {
val text = "MEDIA:/a/report.pdf and MEDIA:/b/clip.mp3 and MEDIA:/c/x.png"
val items = HostMediaExtractor.extract(text)
assertEquals(3, items.size)
assertTrue(items.any { it.path == "/a/report.pdf" })
assertTrue(items.any { it.path == "/b/clip.mp3" })
assertTrue(items.any { it.path == "/c/x.png" })
}
@Test
fun `returns empty when no MEDIA directive`() {
assertEquals(0, HostMediaExtractor.extract("just text").size)
}
@Test
fun `ignores relative paths`() {
assertEquals(0, HostMediaExtractor.extract("MEDIA:relative/path.png").size)
}
@Test
fun `strips directives from text`() {
val out = HostMediaExtractor.strip("See MEDIA:/tmp/a.png for details")
assertEquals("See for details", out)
assertFalse(out.contains("MEDIA:"))
}
@Test
fun `strips multiple directives`() {
val out = HostMediaExtractor.strip("MEDIA:/a/x.pdf MEDIA:/b/y.mp3 end")
assertEquals("end", out)
}
@Test
fun `handles quoted paths`() {
val items = HostMediaExtractor.extract("MEDIA:\"/tmp/a b.png\"")
assertEquals(1, items.size)
assertEquals("/tmp/a b.png", items[0].path)
}
}