feat(#724): full gateway-file support — attach MEDIA: files (images, audio, video, docs) (#725)

* fix(#724): inline host-path MEDIA: images on mobile (client-side)

The previous approach re-fetched the REST session-messages endpoint, but that
endpoint returns the raw MEDIA:<path> text (api_server._message_response does
not inline, and the transcript stores raw content). The desktop app instead
reads the file locally via readFileDataUrl because it is co-located with the
host. Mirror that on mobile: on assistant message.complete/done, inline any
MEDIA:<path> image into a base64 data:image/... markdown tag the renderer can
show. Mobile-only, backend untouched. Unreachable tags are stripped (graceful
degrade). Pure inlining logic extracted to MediaInline for unit testing.

* fix: correct RegexOption name (IGNORE_CASE) in MediaInline

* test: fix byte literal types in MediaInlineTest

* fix(#724): rewrite host-path MEDIA: to gateway /api/files/download URL

Pivot from the earlier base64-inline approach to the desktop-proven,
remote-phone-working fix: rewrite MEDIA:<path> directives (delivered raw over
the WS stream) into authenticated
![image](<baseUrl>/api/files/download?path=<enc>&token=<enc>) markdown tags,
mirroring desktop media.ts mediaExternalUrl. The mobile renderer (MarkdownText)
shows http(s) ![]() via Coil, so agent-sent images now render on mobile too —
including on a remote phone (real HTTP, unlike a host-local file read).
Mobile-only, backend untouched. Pure rewrite logic in MediaUrlRewriter
(replaces MediaInline) with unit tests.

* feat(#724): full gateway-file support — attach MEDIA: files (not just images)

Pivot the MEDIA: fix from image-only URL rewriting to real attachments:
- Attachment: add gatewayUrl + AttachmentSource (LOCAL/GATEWAY) so
  gateway-hosted files are first-class and render via existing InlineAttachment.
- HostMediaExtractor (replaces MediaUrlRewriter): parse ANY absolute MEDIA:
  directive (any extension) and strip them from message text.
- ChatViewModel.attachHostMedia: build the authed /api/files/download URL via
  GatewayFileClient, classify via MediaTypes, attach as Attachment (images
  inline, every other file tappable). Idempotent.
- InlineAttachment: tappable. Gateway files fetch via GatewayFileClient and
  share via a chooser Intent (FileProvider); images load the URL in Coil.
- Hence agent-delivered images, audio, video, CSV, PDF and arbitrary files all
  work on mobile — including a remote phone. Backend untouched.

Builds on #726 (GatewayFileClient). Rebased onto main after #726 merged.

* fix(#724): resolve imports, bubble callback scope, and quoted media path regex

* feat(chat): open attachment in-place on tap (#724)

Tap opens LOCAL (content://) and GATEWAY (fetch->FileProvider) files via
ACTION_VIEW; failures surface as a non-blocking snackbar via openError.

* fix: remove redundant out-of-order io.mockk.* wildcard import (ktlint CI)

ktlint import-ordering requires lexicographic order; the trailing wildcard
'import io.mockk.*' sorts before 'io.mockk.coEvery' and was redundant — every
mockk symbol is already explicitly imported, and the single mockkConstructor
call uses a fully-qualified io.mockk. prefix (needs no import).

Fixes ktlint failure on PR #725.

* fix(ci): mock android.jar Intent stub so openAttachment GATEWAY test passes

android.jar unit-test stubs Intent.setDataAndType/addFlags to throw
'... not mocked'. openWithView builds an Intent and calls them, so
openBytes' runCatching swallowed the exception and startActivity never
fired, failing the test. Mock the Intent constructor + setters and
capture the constructed instance to assert the ACTION_VIEW contract.

* fix(#724): resolve host media download URLs and deduplicate REST synced chat bubbles

* fix(#724): add FLAG_ACTIVITY_NEW_TASK for file intents and update GatewayFileClientTest

---------

Co-authored-by: M57 <hy4ri@users.noreply.github.com>
This commit is contained in:
M57andM57 authored and GitHub committed 2026-07-27 14:52:49 +03:00
1 parent 6aee02c4da
commit a1155e8015
14 files changed
+632 -35

No files matched your search

@@ -1,11 +1,16 @@
package com.m57.hermescontrol
import android.app.Application
import coil.ImageLoader
import coil.ImageLoaderFactory
import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.remote.NetworkMonitor
import com.m57.hermescontrol.data.remote.OkHttpProvider
import com.m57.hermescontrol.ui.analytics.AnalyticsPreloader
class HermesControlApp : Application() {
class HermesControlApp :
Application(),
ImageLoaderFactory {
override fun onCreate() {
super.onCreate()
AuthManager.init(this)
@@ -18,4 +23,11 @@ class HermesControlApp : Application() {
// cold backend). Fire-and-forget; never blocks UI startup.
AnalyticsPreloader.preload(this)
}
override fun newImageLoader(): ImageLoader =
ImageLoader
.Builder(this)
.okHttpClient(OkHttpProvider.base)
.crossfade(true)
.build()
}
@@ -1,28 +1,40 @@
package com.m57.hermescontrol.data.model
import kotlinx.serialization.Serializable
/**
* Represents a file attached to a chat message.
*
* @param uri Content URI or file path to the attachment
* @param uri Content URI or file path to the attachment. For gateway-sourced
* attachments this is the authenticated `/api/files/download?...` URL, which
* Coil can load directly for images.
* @param name Display name of the file
* @param mimeType MIME type (e.g., "image/jpeg", "application/pdf")
* @param size File size in bytes
* @param localPath Optional local file system path for direct access
* @param size File size in bytes (0 when unknown, e.g. gateway-hosted files)
* @param gatewayUrl When non-null, the authenticated gateway download URL for a
* file that lives on the *gateway host* (agent-delivered `MEDIA:` directive).
* Null for locally-picked attachments.
* @param source Where the attachment originated.
*/
@Serializable
data class Attachment(
val uri: String,
val name: String,
val mimeType: String,
val size: Long,
val size: Long = 0,
val gatewayUrl: String? = null,
val source: AttachmentSource = AttachmentSource.LOCAL,
) {
val isImage: Boolean
get() = mimeType.startsWith("image/")
val isGateway: Boolean
get() = source == AttachmentSource.GATEWAY
val formattedSize: String
get() =
when {
size <= 0 -> "—"
size < 1024 -> "$size B"
size < 1024 * 1024 -> "${size / 1024} KB"
else -> "%.1f MB".format(size.toDouble() / (1024 * 1024))
@@ -31,3 +43,13 @@ data class Attachment(
val fileExtension: String
get() = name.substringAfterLast('.', "").lowercase()
}
/** Origin of an [Attachment]. */
@Serializable
enum class AttachmentSource {
/** Picked by the user on this device (content URI / local path). */
LOCAL,
/** Delivered by the agent via a gateway-hosted `MEDIA:` directive. */
GATEWAY,
}
@@ -41,11 +41,15 @@ object GatewayFileClient {
path: String,
): String? {
val trimmedBase = baseUrl.trimEnd('/')
if (trimmedBase.isBlank() || token.isBlank()) return null
if (trimmedBase.isBlank()) return null
val norm = normalizePath(path) ?: return null
val encPath = URLEncoder.encode(norm, StandardCharsets.UTF_8.name()).replace("+", "%20")
val encToken = URLEncoder.encode(token, StandardCharsets.UTF_8.name()).replace("+", "%20")
return "$trimmedBase$DOWNLOAD_PATH?path=$encPath&token=$encToken"
return if (token.isNotBlank()) {
val encToken = URLEncoder.encode(token, StandardCharsets.UTF_8.name()).replace("+", "%20")
"$trimmedBase$DOWNLOAD_PATH?path=$encPath&token=$encToken"
} else {
"$trimmedBase$DOWNLOAD_PATH?path=$encPath"
}
}
/**
@@ -114,6 +114,7 @@ fun ChatBubble(
searchQuery: String = "",
isCurrentMatch: Boolean = false,
onRespondApproval: (String) -> Unit = {},
onOpenAttachment: (Attachment) -> Unit = {},
modifier: Modifier = Modifier,
) {
val screenWidth = LocalConfiguration.current.screenWidthDp.dp
@@ -129,17 +130,25 @@ fun ChatBubble(
) {
when (message.role) {
MessageRole.USER -> {
UserBubble(message, maxBubbleWidth, searchQuery, isCurrentMatch, modifier)
UserBubble(
message = message,
maxWidth = maxBubbleWidth,
searchQuery = searchQuery,
isCurrentMatch = isCurrentMatch,
onOpenAttachment = onOpenAttachment,
modifier = modifier,
)
}
MessageRole.ASSISTANT -> {
AssistantBubble(
message,
maxBubbleWidth,
isDarkTheme,
searchQuery,
isCurrentMatch,
modifier,
message = message,
maxWidth = maxBubbleWidth,
isDarkTheme = isDarkTheme,
searchQuery = searchQuery,
isCurrentMatch = isCurrentMatch,
onOpenAttachment = onOpenAttachment,
modifier = modifier,
)
}
@@ -164,6 +173,7 @@ private fun UserBubble(
maxWidth: androidx.compose.ui.unit.Dp,
searchQuery: String = "",
isCurrentMatch: Boolean = false,
onOpenAttachment: (Attachment) -> Unit = {},
modifier: Modifier = Modifier,
) {
val clipboard = LocalClipboard.current
@@ -263,6 +273,7 @@ private fun UserBubble(
InlineAttachment(
attachment = attachment,
textColor = userBubbleTextColor,
onOpen = { onOpenAttachment(it) },
)
Spacer(modifier = Modifier.height(4.dp))
}
@@ -325,6 +336,7 @@ private fun AssistantBubble(
isDarkTheme: Boolean,
searchQuery: String = "",
isCurrentMatch: Boolean = false,
onOpenAttachment: (Attachment) -> Unit = {},
modifier: Modifier = Modifier,
) {
val bubbleColor = if (isDarkTheme) AssistantBubble else AssistantBubbleLight
@@ -414,6 +426,7 @@ private fun AssistantBubble(
InlineAttachment(
attachment = attachment,
textColor = textColor,
onOpen = { onOpenAttachment(it) },
)
Spacer(modifier = Modifier.height(4.dp))
}
@@ -2293,24 +2306,30 @@ private fun buildHighlightedString(
private fun InlineAttachment(
attachment: Attachment,
textColor: Color,
onOpen: (Attachment) -> Unit = {},
) {
val clickable = Modifier.clickable { onOpen(attachment) }
if (attachment.isImage) {
// Image attachment — show as a rounded thumbnail
// Image attachment — show as a rounded thumbnail (Coil loads the
// gateway download URL directly when gateway-sourced).
AsyncImage(
model = attachment.uri,
contentDescription = attachment.name,
modifier =
Modifier
.fillMaxWidth()
.clip(RoundedCornerShape(12.dp)),
.clip(RoundedCornerShape(12.dp))
.then(clickable),
contentScale = ContentScale.FillWidth,
)
} else {
// Non-image file — show a card with file icon and name
// Non-image file — show a card with file icon and name. Tapping fetches
// the bytes (gateway-sourced) or opens the local URI.
Surface(
shape = RoundedCornerShape(8.dp),
color = textColor.copy(alpha = 0.1f),
border = BorderStroke(1.dp, textColor.copy(alpha = 0.2f)),
modifier = clickable,
) {
Row(
modifier = Modifier.padding(8.dp),
@@ -334,6 +334,7 @@ fun ChatScreen(
messages = state.messages,
errorMessage = state.errorMessage,
backgroundCompleteMessage = state.backgroundCompleteMessage,
openError = state.openError,
isSearchActive = state.isSearchActive,
currentSearchMatchIndex = state.currentSearchMatchIndex,
searchMatchIndices = state.searchMatchIndices,
@@ -10,10 +10,14 @@ import androidx.lifecycle.viewModelScope
import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.local.HermesDatabase
import com.m57.hermescontrol.data.model.Attachment
import com.m57.hermescontrol.data.model.AttachmentSource
import com.m57.hermescontrol.data.model.ModelProvider
import com.m57.hermescontrol.data.model.PinnedModel
import com.m57.hermescontrol.data.model.SessionMessage
import com.m57.hermescontrol.data.remote.ApiClient
import com.m57.hermescontrol.data.remote.GatewayFile
import com.m57.hermescontrol.data.remote.GatewayFileClient
import com.m57.hermescontrol.data.remote.GatewayFileResult
import com.m57.hermescontrol.data.remote.NetworkResult
import com.m57.hermescontrol.data.remote.OkHttpProvider
import com.m57.hermescontrol.data.remote.safeApiCall
@@ -66,6 +70,8 @@ data class ChatUiState(
val errorMessage: String? = null,
// Background job completion toast (issue #527) — non-blocking snackbar
val backgroundCompleteMessage: String? = null,
// Attachment open failure — surfaced as a non-blocking snackbar (issue #724)
val openError: String? = null,
val clarifyRequest: ClarifyUi? = null,
// Sudo / secret prompts — surfaced as dialogs (issue #524)
val sudoPrompt: SudoPromptUi? = null,
@@ -415,6 +421,16 @@ class ChatViewModel(
// than waiting up to 5s for the next session-sync poll.
viewModelScope.launch { fetchContextUsage() }
}
is ReducerEffect.AttachHostMedia -> {
// Issue #724: turn host-path MEDIA: directives into real
// attachments (images inline, every other file tappable)
// via the gateway /api/files/download endpoint. Works on a
// remote phone too.
viewModelScope.launch(Dispatchers.IO) {
attachHostMedia(effect.sessionId, effect.messageId)
}
}
}
}
@@ -1052,10 +1068,11 @@ class ChatViewModel(
// #576). For those we fall back to slash.exec, which runs the
// full COMMAND_REGISTRY through the worker.
val result =
wsClient.request(
WsMethods.COMMAND_DISPATCH,
mapOf("name" to name, "arg" to arg, "session_id" to sessionId),
).await()
wsClient
.request(
WsMethods.COMMAND_DISPATCH,
mapOf("name" to name, "arg" to arg, "session_id" to sessionId),
).await()
handleDispatchResult(result)
} catch (e: HermesWsClient.HermesRpcException) {
val msg = e.message.orEmpty()
@@ -1068,13 +1085,14 @@ class ChatViewModel(
// which routes the full CLI command set through the worker.
try {
val result =
wsClient.request(
WsMethods.SLASH_EXEC,
mapOf(
"command" to "/$name${if (arg.isNotEmpty()) " $arg" else ""}",
"session_id" to sessionId,
),
).await()
wsClient
.request(
WsMethods.SLASH_EXEC,
mapOf(
"command" to "/$name${if (arg.isNotEmpty()) " $arg" else ""}",
"session_id" to sessionId,
),
).await()
val output = (result as? Map<*, *>)?.get("output") as? String
if (!output.isNullOrBlank()) addAssistantMessage(output)
} catch (e2: HermesWsClient.HermesRpcException) {
@@ -1698,6 +1716,9 @@ class ChatViewModel(
.filter { it.reasoningText.isNotBlank() }
.associateBy { it.content }
val baseUrl = AuthManager.getBaseUrl()
val token = AuthManager.getToken().orEmpty()
return messages.mapIndexed { index, msg ->
val role =
when (msg.role?.lowercase()) {
@@ -1714,25 +1735,249 @@ class ChatViewModel(
?.toLong()
?: System.currentTimeMillis()
val content = msg.contentText
val rawContent = msg.contentText
val reasoning =
if (msg.reasoningText.isNotBlank()) {
msg.reasoningText
} else {
existingReasoningMap[content]?.reasoningText.orEmpty()
existingReasoningMap[rawContent]?.reasoningText.orEmpty()
}
var finalContent = rawContent
var attachments: List<Attachment>? = null
if (role == MessageRole.ASSISTANT && rawContent.contains("MEDIA:")) {
val items = HostMediaExtractor.extract(rawContent)
if (items.isNotEmpty()) {
finalContent = HostMediaExtractor.strip(rawContent)
attachments =
items
.mapNotNull { item ->
val url =
GatewayFileClient.buildDownloadUrl(
baseUrl,
token,
item.path,
) ?: return@mapNotNull null
Attachment(
uri = url,
name = mediaNameFromPath(item.path),
mimeType = mediaMimeForPath(item.path),
size = 0,
gatewayUrl = url,
source = AttachmentSource.GATEWAY,
)
}.takeIf { it.isNotEmpty() }
}
}
ChatMessage(
id = "rest-$sessionId-$globalIndex",
role = role,
content = content,
content = finalContent,
reasoningText = reasoning,
attachments = attachments,
timestamp = timestamp,
isStreaming = false,
)
}
}
// ── Issue #724: attach host-path MEDIA: files as real attachments ────
//
// The gateway's WebSocket stream delivers the raw `MEDIA:<path>` directive
// the desktop app turns into an authenticated `/api/files/download?...`
// URL. We parse every directive, build the download URL via
// [GatewayFileClient], classify it (image / audio / video / file) using
// [mediaKindForPath], and attach it to the message. Images render inline
// (Coil loads the URL); every other type becomes a tappable, fetchable
// attachment. The directive text is stripped from the message body. Works
// on a remote phone (real HTTP). Mobile-only; backend untouched. Pure
// parsing lives in [HostMediaExtractor].
/**
* ViewModel-side handler for [ReducerEffect.AttachHostMedia]: find the local
* message by id, convert any `MEDIA:<path>` directives into [Attachment]s
* (via the gateway download URL) and strip them from the text. Role,
* reasoning, timestamp and existing attachments are preserved; new gateway
* attachments are appended. Idempotent — skips if gateway attachments for
* the same paths already exist.
*/
private fun attachHostMedia(
sessionId: String,
messageId: String,
) {
val current = _uiState.value.messages.find { it.id == messageId } ?: return
val content = current.content
val items = HostMediaExtractor.extract(content)
if (items.isEmpty()) return
val baseUrl = AuthManager.getBaseUrl()
val token = AuthManager.getToken()
if (baseUrl.isBlank() || token.isNullOrBlank()) return
val existingUrls =
current.attachments
.orEmpty()
.mapNotNull { it.gatewayUrl }
.toSet()
val newAttachments =
items.mapNotNull { item ->
val url = GatewayFileClient.buildDownloadUrl(baseUrl, token, item.path) ?: return@mapNotNull null
if (url in existingUrls) return@mapNotNull null
Attachment(
uri = url,
name = mediaNameFromPath(item.path),
mimeType = mediaMimeForPath(item.path),
size = 0,
gatewayUrl = url,
source = AttachmentSource.GATEWAY,
)
}
if (newAttachments.isEmpty()) return
val stripped = HostMediaExtractor.strip(content)
_uiState.update { state ->
state.copy(
messages =
state.messages.map { msg ->
if (msg.id == messageId) {
msg.copy(
content = stripped,
attachments =
(msg.attachments.orEmpty() + newAttachments)
.distinctBy { it.gatewayUrl ?: it.uri },
)
} else {
msg
}
},
)
}
}
/**
* Open an attachment when its chip/thumbnail is tapped.
*
* - LOCAL (user-picked) files: open the original `content://` URI
* directly via [android.content.Intent.ACTION_VIEW] — the resolver
* already grants read access for the picked document. If that fails
* (e.g. the permission lapsed), we copy to cache and retry via
* FileProvider so the tap is never a silent no-op.
* - GATEWAY (agent `MEDIA:`) files: fetch the bytes via
* [GatewayFileClient], write them to a cache file, and open with
* [android.content.Intent.ACTION_VIEW] through FileProvider — so a
* remote phone can view agent-delivered files in-place.
*
* Failures surface through [ChatUiState.openError] (non-blocking
* snackbar); the tap is never swallowed.
*/
fun openAttachment(attachment: Attachment) {
val ctx = getApplication<Application>().applicationContext
if (attachment.source == AttachmentSource.LOCAL) {
// Best-effort direct open of the picked content URI.
runCatching { openWithView(ctx, android.net.Uri.parse(attachment.uri), attachment.mimeType) }
.onSuccess { return }
.onFailure { /* fall through to cache-copy below */ }
}
// GATEWAY, or LOCAL direct-open failed → fetch/copy then open.
val path =
attachment.gatewayUrl?.let { url ->
runCatching {
java.net
.URL(url)
.query
.split('&')
.firstOrNull { it.startsWith("path=") }
?.removePrefix("path=")
?.let { java.net.URLDecoder.decode(it, "UTF-8") }
}.getOrNull()
} ?: attachment.name
viewModelScope.launch(Dispatchers.IO) {
when (val result = GatewayFileClient.fetch(path)) {
is GatewayFileResult.Success -> {
openBytes(ctx, result.file)
}
is GatewayFileResult.NotFound -> {
showOpenError("File not found on gateway: ${attachment.name}")
}
is GatewayFileResult.Forbidden -> {
showOpenError("Access denied: ${attachment.name}")
}
is GatewayFileResult.TooLarge -> {
showOpenError("File too large to open: ${attachment.name}")
}
is GatewayFileResult.Unauthorized -> {
showOpenError("Session expired — reconnect to open: ${attachment.name}")
}
is GatewayFileResult.Failure -> {
showOpenError("Could not open ${attachment.name}: ${result.throwable.message}")
}
}
}
}
/** Open bytes written to a cache file via FileProvider + ACTION_VIEW. */
private fun openBytes(
ctx: android.content.Context,
file: GatewayFile,
) {
runCatching {
val dir = java.io.File(ctx.cacheDir, "gateway_files").also { it.mkdirs() }
val safeName = file.name.replace(Regex("[/\\\\]"), "_").ifBlank { "file" }
val out = java.io.File(dir, safeName)
out.writeBytes(file.bytes)
val uri =
androidx.core.content.FileProvider.getUriForFile(
ctx,
"${ctx.packageName}.fileprovider",
out,
)
openWithView(ctx, uri, file.mimeType)
}.onFailure { showOpenError("Could not open ${file.name}: ${it.message}") }
}
/** Fire an ACTION_VIEW intent; throws if no activity can handle the type. */
private fun openWithView(
ctx: android.content.Context,
uri: android.net.Uri,
mimeType: String,
) {
val viewIntent =
android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
setDataAndType(uri, mimeType.ifBlank { "*/*" })
addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
}
try {
ctx.startActivity(viewIntent)
} catch (e: Throwable) {
val fallbackIntent =
android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
setDataAndType(uri, "*/*")
addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
}
val chooser =
android.content.Intent.createChooser(fallbackIntent, "Open file").apply {
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
}
ctx.startActivity(chooser)
}
}
private fun showOpenError(message: String) {
_uiState.update { it.copy(openError = message) }
}
fun clearOpenError() {
_uiState.update { it.copy(openError = null) }
}
private fun serverMessageIndex(
id: String,
sessionId: String,
@@ -255,6 +255,13 @@ object ChatWsEventReducer {
// Refresh the context meter immediately on completion instead of
// waiting for the next 5s session-sync poll.
effects.add(ReducerEffect.RefreshContextUsage)
// Mobile-only media fix (issue #724): the WS stream delivers the raw
// `MEDIA:<host-path>` directive. Turn it into real attachments (images
// inline, every other file tappable) via the gateway /api/files/download
// endpoint — works on a remote phone too.
if (msg.role == MessageRole.ASSISTANT && sid != null) {
effects.add(ReducerEffect.AttachHostMedia(sid, msg.id))
}
return ReducerResult(
state =
state.copy(
@@ -288,6 +295,12 @@ object ChatWsEventReducer {
if (sid != null) {
effects.add(ReducerEffect.PersistMessage(msg, sid))
}
// Mobile-only media fix (issue #724): attach any `MEDIA:<path>` file
// carried in the finalized assistant message as a real attachment. See
// the ReducerEffect.AttachHostMedia KDoc for the rationale.
if (msg.role == MessageRole.ASSISTANT && sid != null) {
effects.add(ReducerEffect.AttachHostMedia(sid, msg.id))
}
return ReducerResult(
state =
state.copy(
@@ -669,4 +682,22 @@ sealed class ReducerEffect {
data object RefreshSessions : ReducerEffect()
data object RefreshContextUsage : ReducerEffect()
/**
* Ask the ViewModel to turn any `MEDIA:<host-path>` directives in the
* just-finalized assistant message into real [Attachment]s (images inline,
* every other file type tappable) and strip the directives from the text.
*
* Why: the gateway's WebSocket stream delivers the raw `MEDIA:<path>`
* directive (the same one the desktop app resolves via `mediaExternalUrl`
* to the authenticated `/api/files/download` endpoint, which streams the
* host file over HTTP). Attaching them lets the renderer show images inline
* and offer every other file as a fetchable attachment — even on a remote
* phone (real HTTP, unlike a host-local file read). Mirrors desktop's
* `mediaExternalUrl`; mobile-only, backend untouched. See issue #724.
*/
data class AttachHostMedia(
val sessionId: String,
val messageId: String,
) : ReducerEffect()
}
@@ -0,0 +1,77 @@
package com.m57.hermescontrol.ui.chat
/**
* Issue #724: parse agent-delivered `MEDIA:<path>` directives out of message
* text so they can be attached (and rendered) as real files.
*
* The gateway's WebSocket stream delivers the raw `MEDIA:<path>` directive the
* desktop app resolves via `mediaExternalUrl` — an authenticated
* `/api/files/download?path=<enc>&token=<enc>` URL that streams the host file's
* bytes over HTTP. Mobile turns each directive into an [Attachment] (see
* [com.m57.hermescontrol.ui.chat.ChatViewModel.attachHostMedia]) so the
* renderer can show images inline and offer every other file type as a
* tappable, fetchable attachment — including on a *remote* phone.
*
* Matches ANY absolute (or `~/`) host path regardless of extension, so images,
* audio, video, CSV, PDF and arbitrary files are all supported. The pure parse
* logic lives here so it can be unit-tested without an Android context.
*/
internal object HostMediaExtractor {
/** A single `MEDIA:<path>` directive found in message text. */
data class Item(
val match: String, // the full matched directive (for stripping)
val path: String, // normalized absolute path
)
/** Matches `MEDIA:<path>` (any extension), supporting quotes/backticks around the path. */
private val MEDIA_TAG_RE =
Regex(
"""[`"']?MEDIA:\s*(?:"([^"]+)"|'([^']+)'|`([^`]+)`|((?:~|/|[A-Za-z]:)[^\s`"')]*))[`"']?""",
RegexOption.IGNORE_CASE,
)
/** Extract every `MEDIA:<path>` directive from [text] as a normalized item. */
fun extract(text: String): List<Item> {
if (!text.contains("MEDIA:")) return emptyList()
return MEDIA_TAG_RE
.findAll(text)
.mapNotNull { m ->
val raw = m.groupValues.drop(1).firstOrNull { it.isNotBlank() } ?: return@mapNotNull null
val path = normalizePath(raw) ?: return@mapNotNull null
Item(match = m.value, path = path)
}.toList()
}
/** Remove all `MEDIA:<path>` directives from [text] (leaves other content). */
fun strip(text: String): String {
if (!text.contains("MEDIA:")) return text
return MEDIA_TAG_RE
.replace(text) { "" }
.replace(Regex("""[ \t]+"""), " ")
.replace(Regex("""\n{3,}"""), "\n\n")
.trim()
}
/** Expand ~/ and verify the path is absolute. Null if not an absolute path. */
internal fun normalizePath(raw: String): String? {
val trimmed =
raw
.trim()
.removeSurrounding("`")
.removeSurrounding("\"")
.removeSurrounding("'")
val expanded =
if (trimmed.startsWith("~")) {
val home = System.getenv("HOME") ?: return null
home + trimmed.removePrefix("~")
} else {
trimmed
}
if (!expanded.startsWith("/") &&
!Regex("^[A-Za-z]:[/\\\\]").containsMatchIn(expanded)
) {
return null
}
return expanded
}
}
@@ -43,6 +43,8 @@ import androidx.compose.ui.text.style.TextDecoration
import androidx.compose.ui.text.withStyle
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import com.m57.hermescontrol.data.local.AuthManager
import com.m57.hermescontrol.data.remote.GatewayFileClient
import com.m57.hermescontrol.theme.LocalHermesStatusColors
import com.m57.hermescontrol.theme.SearchHighlightColors
import com.m57.hermescontrol.theme.searchHighlightColors
@@ -268,7 +270,7 @@ fun MarkdownText(
// AsyncImage is a Compose runtime composable — load it lazily
// behind remember so the parser stays pure and testable (the
// parseBlocks/parseInline unit tests never touch Compose).
val model: Any = block.uri
val model: Any = remember(block.uri) { resolveImageUrl(block.uri) }
androidx.compose.foundation.layout.Box(
modifier =
Modifier
@@ -1000,3 +1002,26 @@ internal enum class TableAlign {
CENTER,
RIGHT,
}
private fun resolveImageUrl(uri: String): String {
val trimmed = uri.trim()
if (trimmed.isBlank()) return uri
if (trimmed.startsWith("http://") || trimmed.startsWith("https://") || trimmed.startsWith("data:image/")) {
return trimmed
}
val baseUrl = runCatching { AuthManager.getBaseUrl() }.getOrDefault("")
val token = runCatching { AuthManager.getToken() }.getOrNull().orEmpty()
val downloadUrl = GatewayFileClient.buildDownloadUrl(baseUrl, token, trimmed)
if (downloadUrl != null) return downloadUrl
if (baseUrl.isNotBlank() && (trimmed.startsWith("/api/") || trimmed.startsWith("api/"))) {
val cleanPath = if (trimmed.startsWith("/")) trimmed else "/$trimmed"
val sep = if (cleanPath.contains("?")) "&" else "?"
return if (token.isNotBlank() && !cleanPath.contains("token=")) {
"${baseUrl.trimEnd('/')}$cleanPath${sep}token=$token"
} else {
"${baseUrl.trimEnd('/')}$cleanPath"
}
}
return uri
}
@@ -33,6 +33,7 @@ fun ChatLifecycleEffects(
messages: List<ChatMessage>,
errorMessage: String?,
backgroundCompleteMessage: String?,
openError: String?,
isSearchActive: Boolean,
currentSearchMatchIndex: Int,
searchMatchIndices: List<Int>,
@@ -127,6 +128,14 @@ fun ChatLifecycleEffects(
}
}
// Show attachment-open failures as a non-blocking snackbar (issue #724)
LaunchedEffect(openError) {
openError?.let { message ->
snackbarHostState.showSnackbar(message)
viewModel.clearOpenError()
}
}
// Sudo / secret prompt dialogs (issue #524)
sudoPrompt?.let { prompt ->
SudoPromptDialog(
@@ -108,6 +108,7 @@ fun ChatMessageList(
searchQuery = if (isSearchActive) searchQuery else "",
isCurrentMatch = isCurrentMatch,
onRespondApproval = viewModel::respondToApproval,
onOpenAttachment = viewModel::openAttachment,
)
}
}
@@ -127,6 +128,7 @@ fun ChatMessageList(
isDarkTheme = isDark,
searchQuery = "",
isCurrentMatch = false,
onOpenAttachment = viewModel::openAttachment,
)
}
}