* fix(#724): inline host-path MEDIA: images on mobile (client-side) The previous approach re-fetched the REST session-messages endpoint, but that endpoint returns the raw MEDIA:<path> text (api_server._message_response does not inline, and the transcript stores raw content). The desktop app instead reads the file locally via readFileDataUrl because it is co-located with the host. Mirror that on mobile: on assistant message.complete/done, inline any MEDIA:<path> image into a base64 data:image/... markdown tag the renderer can show. Mobile-only, backend untouched. Unreachable tags are stripped (graceful degrade). Pure inlining logic extracted to MediaInline for unit testing. * fix: correct RegexOption name (IGNORE_CASE) in MediaInline * test: fix byte literal types in MediaInlineTest * fix(#724): rewrite host-path MEDIA: to gateway /api/files/download URL Pivot from the earlier base64-inline approach to the desktop-proven, remote-phone-working fix: rewrite MEDIA:<path> directives (delivered raw over the WS stream) into authenticated  markdown tags, mirroring desktop media.ts mediaExternalUrl. The mobile renderer (MarkdownText) shows http(s) ![]() via Coil, so agent-sent images now render on mobile too — including on a remote phone (real HTTP, unlike a host-local file read). Mobile-only, backend untouched. Pure rewrite logic in MediaUrlRewriter (replaces MediaInline) with unit tests. * feat(#724): full gateway-file support — attach MEDIA: files (not just images) Pivot the MEDIA: fix from image-only URL rewriting to real attachments: - Attachment: add gatewayUrl + AttachmentSource (LOCAL/GATEWAY) so gateway-hosted files are first-class and render via existing InlineAttachment. - HostMediaExtractor (replaces MediaUrlRewriter): parse ANY absolute MEDIA: directive (any extension) and strip them from message text. - ChatViewModel.attachHostMedia: build the authed /api/files/download URL via GatewayFileClient, classify via MediaTypes, attach as Attachment (images inline, every other file tappable). Idempotent. - InlineAttachment: tappable. Gateway files fetch via GatewayFileClient and share via a chooser Intent (FileProvider); images load the URL in Coil. - Hence agent-delivered images, audio, video, CSV, PDF and arbitrary files all work on mobile — including a remote phone. Backend untouched. Builds on #726 (GatewayFileClient). Rebased onto main after #726 merged. * fix(#724): resolve imports, bubble callback scope, and quoted media path regex * feat(chat): open attachment in-place on tap (#724) Tap opens LOCAL (content://) and GATEWAY (fetch->FileProvider) files via ACTION_VIEW; failures surface as a non-blocking snackbar via openError. * fix: remove redundant out-of-order io.mockk.* wildcard import (ktlint CI) ktlint import-ordering requires lexicographic order; the trailing wildcard 'import io.mockk.*' sorts before 'io.mockk.coEvery' and was redundant — every mockk symbol is already explicitly imported, and the single mockkConstructor call uses a fully-qualified io.mockk. prefix (needs no import). Fixes ktlint failure on PR #725. * fix(ci): mock android.jar Intent stub so openAttachment GATEWAY test passes android.jar unit-test stubs Intent.setDataAndType/addFlags to throw '... not mocked'. openWithView builds an Intent and calls them, so openBytes' runCatching swallowed the exception and startActivity never fired, failing the test. Mock the Intent constructor + setters and capture the constructed instance to assert the ACTION_VIEW contract. * fix(#724): resolve host media download URLs and deduplicate REST synced chat bubbles * fix(#724): add FLAG_ACTIVITY_NEW_TASK for file intents and update GatewayFileClientTest --------- Co-authored-by: M57 <hy4ri@users.noreply.github.com>
This commit is contained in:
1 parent
6aee02c4da
commit
a1155e8015
14 files changed
+632
-35
No files matched your search
@@ -1,11 +1,16 @@
|
||||
package com.m57.hermescontrol
|
||||
|
||||
import android.app.Application
|
||||
import coil.ImageLoader
|
||||
import coil.ImageLoaderFactory
|
||||
import com.m57.hermescontrol.data.local.AuthManager
|
||||
import com.m57.hermescontrol.data.remote.NetworkMonitor
|
||||
import com.m57.hermescontrol.data.remote.OkHttpProvider
|
||||
import com.m57.hermescontrol.ui.analytics.AnalyticsPreloader
|
||||
|
||||
class HermesControlApp : Application() {
|
||||
class HermesControlApp :
|
||||
Application(),
|
||||
ImageLoaderFactory {
|
||||
override fun onCreate() {
|
||||
super.onCreate()
|
||||
AuthManager.init(this)
|
||||
@@ -18,4 +23,11 @@ class HermesControlApp : Application() {
|
||||
// cold backend). Fire-and-forget; never blocks UI startup.
|
||||
AnalyticsPreloader.preload(this)
|
||||
}
|
||||
|
||||
override fun newImageLoader(): ImageLoader =
|
||||
ImageLoader
|
||||
.Builder(this)
|
||||
.okHttpClient(OkHttpProvider.base)
|
||||
.crossfade(true)
|
||||
.build()
|
||||
}
|
||||
@@ -1,28 +1,40 @@
|
||||
package com.m57.hermescontrol.data.model
|
||||
|
||||
import kotlinx.serialization.Serializable
|
||||
|
||||
/**
|
||||
* Represents a file attached to a chat message.
|
||||
*
|
||||
* @param uri Content URI or file path to the attachment
|
||||
* @param uri Content URI or file path to the attachment. For gateway-sourced
|
||||
* attachments this is the authenticated `/api/files/download?...` URL, which
|
||||
* Coil can load directly for images.
|
||||
* @param name Display name of the file
|
||||
* @param mimeType MIME type (e.g., "image/jpeg", "application/pdf")
|
||||
* @param size File size in bytes
|
||||
* @param localPath Optional local file system path for direct access
|
||||
* @param size File size in bytes (0 when unknown, e.g. gateway-hosted files)
|
||||
* @param gatewayUrl When non-null, the authenticated gateway download URL for a
|
||||
* file that lives on the *gateway host* (agent-delivered `MEDIA:` directive).
|
||||
* Null for locally-picked attachments.
|
||||
* @param source Where the attachment originated.
|
||||
*/
|
||||
@Serializable
|
||||
data class Attachment(
|
||||
val uri: String,
|
||||
val name: String,
|
||||
val mimeType: String,
|
||||
val size: Long,
|
||||
val size: Long = 0,
|
||||
val gatewayUrl: String? = null,
|
||||
val source: AttachmentSource = AttachmentSource.LOCAL,
|
||||
) {
|
||||
val isImage: Boolean
|
||||
get() = mimeType.startsWith("image/")
|
||||
|
||||
val isGateway: Boolean
|
||||
get() = source == AttachmentSource.GATEWAY
|
||||
|
||||
val formattedSize: String
|
||||
get() =
|
||||
when {
|
||||
size <= 0 -> "—"
|
||||
size < 1024 -> "$size B"
|
||||
size < 1024 * 1024 -> "${size / 1024} KB"
|
||||
else -> "%.1f MB".format(size.toDouble() / (1024 * 1024))
|
||||
@@ -31,3 +43,13 @@ data class Attachment(
|
||||
val fileExtension: String
|
||||
get() = name.substringAfterLast('.', "").lowercase()
|
||||
}
|
||||
|
||||
/** Origin of an [Attachment]. */
|
||||
@Serializable
|
||||
enum class AttachmentSource {
|
||||
/** Picked by the user on this device (content URI / local path). */
|
||||
LOCAL,
|
||||
|
||||
/** Delivered by the agent via a gateway-hosted `MEDIA:` directive. */
|
||||
GATEWAY,
|
||||
}
|
||||
@@ -41,11 +41,15 @@ object GatewayFileClient {
|
||||
path: String,
|
||||
): String? {
|
||||
val trimmedBase = baseUrl.trimEnd('/')
|
||||
if (trimmedBase.isBlank() || token.isBlank()) return null
|
||||
if (trimmedBase.isBlank()) return null
|
||||
val norm = normalizePath(path) ?: return null
|
||||
val encPath = URLEncoder.encode(norm, StandardCharsets.UTF_8.name()).replace("+", "%20")
|
||||
val encToken = URLEncoder.encode(token, StandardCharsets.UTF_8.name()).replace("+", "%20")
|
||||
return "$trimmedBase$DOWNLOAD_PATH?path=$encPath&token=$encToken"
|
||||
return if (token.isNotBlank()) {
|
||||
val encToken = URLEncoder.encode(token, StandardCharsets.UTF_8.name()).replace("+", "%20")
|
||||
"$trimmedBase$DOWNLOAD_PATH?path=$encPath&token=$encToken"
|
||||
} else {
|
||||
"$trimmedBase$DOWNLOAD_PATH?path=$encPath"
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -114,6 +114,7 @@ fun ChatBubble(
|
||||
searchQuery: String = "",
|
||||
isCurrentMatch: Boolean = false,
|
||||
onRespondApproval: (String) -> Unit = {},
|
||||
onOpenAttachment: (Attachment) -> Unit = {},
|
||||
modifier: Modifier = Modifier,
|
||||
) {
|
||||
val screenWidth = LocalConfiguration.current.screenWidthDp.dp
|
||||
@@ -129,17 +130,25 @@ fun ChatBubble(
|
||||
) {
|
||||
when (message.role) {
|
||||
MessageRole.USER -> {
|
||||
UserBubble(message, maxBubbleWidth, searchQuery, isCurrentMatch, modifier)
|
||||
UserBubble(
|
||||
message = message,
|
||||
maxWidth = maxBubbleWidth,
|
||||
searchQuery = searchQuery,
|
||||
isCurrentMatch = isCurrentMatch,
|
||||
onOpenAttachment = onOpenAttachment,
|
||||
modifier = modifier,
|
||||
)
|
||||
}
|
||||
|
||||
MessageRole.ASSISTANT -> {
|
||||
AssistantBubble(
|
||||
message,
|
||||
maxBubbleWidth,
|
||||
isDarkTheme,
|
||||
searchQuery,
|
||||
isCurrentMatch,
|
||||
modifier,
|
||||
message = message,
|
||||
maxWidth = maxBubbleWidth,
|
||||
isDarkTheme = isDarkTheme,
|
||||
searchQuery = searchQuery,
|
||||
isCurrentMatch = isCurrentMatch,
|
||||
onOpenAttachment = onOpenAttachment,
|
||||
modifier = modifier,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -164,6 +173,7 @@ private fun UserBubble(
|
||||
maxWidth: androidx.compose.ui.unit.Dp,
|
||||
searchQuery: String = "",
|
||||
isCurrentMatch: Boolean = false,
|
||||
onOpenAttachment: (Attachment) -> Unit = {},
|
||||
modifier: Modifier = Modifier,
|
||||
) {
|
||||
val clipboard = LocalClipboard.current
|
||||
@@ -263,6 +273,7 @@ private fun UserBubble(
|
||||
InlineAttachment(
|
||||
attachment = attachment,
|
||||
textColor = userBubbleTextColor,
|
||||
onOpen = { onOpenAttachment(it) },
|
||||
)
|
||||
Spacer(modifier = Modifier.height(4.dp))
|
||||
}
|
||||
@@ -325,6 +336,7 @@ private fun AssistantBubble(
|
||||
isDarkTheme: Boolean,
|
||||
searchQuery: String = "",
|
||||
isCurrentMatch: Boolean = false,
|
||||
onOpenAttachment: (Attachment) -> Unit = {},
|
||||
modifier: Modifier = Modifier,
|
||||
) {
|
||||
val bubbleColor = if (isDarkTheme) AssistantBubble else AssistantBubbleLight
|
||||
@@ -414,6 +426,7 @@ private fun AssistantBubble(
|
||||
InlineAttachment(
|
||||
attachment = attachment,
|
||||
textColor = textColor,
|
||||
onOpen = { onOpenAttachment(it) },
|
||||
)
|
||||
Spacer(modifier = Modifier.height(4.dp))
|
||||
}
|
||||
@@ -2293,24 +2306,30 @@ private fun buildHighlightedString(
|
||||
private fun InlineAttachment(
|
||||
attachment: Attachment,
|
||||
textColor: Color,
|
||||
onOpen: (Attachment) -> Unit = {},
|
||||
) {
|
||||
val clickable = Modifier.clickable { onOpen(attachment) }
|
||||
if (attachment.isImage) {
|
||||
// Image attachment — show as a rounded thumbnail
|
||||
// Image attachment — show as a rounded thumbnail (Coil loads the
|
||||
// gateway download URL directly when gateway-sourced).
|
||||
AsyncImage(
|
||||
model = attachment.uri,
|
||||
contentDescription = attachment.name,
|
||||
modifier =
|
||||
Modifier
|
||||
.fillMaxWidth()
|
||||
.clip(RoundedCornerShape(12.dp)),
|
||||
.clip(RoundedCornerShape(12.dp))
|
||||
.then(clickable),
|
||||
contentScale = ContentScale.FillWidth,
|
||||
)
|
||||
} else {
|
||||
// Non-image file — show a card with file icon and name
|
||||
// Non-image file — show a card with file icon and name. Tapping fetches
|
||||
// the bytes (gateway-sourced) or opens the local URI.
|
||||
Surface(
|
||||
shape = RoundedCornerShape(8.dp),
|
||||
color = textColor.copy(alpha = 0.1f),
|
||||
border = BorderStroke(1.dp, textColor.copy(alpha = 0.2f)),
|
||||
modifier = clickable,
|
||||
) {
|
||||
Row(
|
||||
modifier = Modifier.padding(8.dp),
|
||||
|
||||
@@ -334,6 +334,7 @@ fun ChatScreen(
|
||||
messages = state.messages,
|
||||
errorMessage = state.errorMessage,
|
||||
backgroundCompleteMessage = state.backgroundCompleteMessage,
|
||||
openError = state.openError,
|
||||
isSearchActive = state.isSearchActive,
|
||||
currentSearchMatchIndex = state.currentSearchMatchIndex,
|
||||
searchMatchIndices = state.searchMatchIndices,
|
||||
|
||||
@@ -10,10 +10,14 @@ import androidx.lifecycle.viewModelScope
|
||||
import com.m57.hermescontrol.data.local.AuthManager
|
||||
import com.m57.hermescontrol.data.local.HermesDatabase
|
||||
import com.m57.hermescontrol.data.model.Attachment
|
||||
import com.m57.hermescontrol.data.model.AttachmentSource
|
||||
import com.m57.hermescontrol.data.model.ModelProvider
|
||||
import com.m57.hermescontrol.data.model.PinnedModel
|
||||
import com.m57.hermescontrol.data.model.SessionMessage
|
||||
import com.m57.hermescontrol.data.remote.ApiClient
|
||||
import com.m57.hermescontrol.data.remote.GatewayFile
|
||||
import com.m57.hermescontrol.data.remote.GatewayFileClient
|
||||
import com.m57.hermescontrol.data.remote.GatewayFileResult
|
||||
import com.m57.hermescontrol.data.remote.NetworkResult
|
||||
import com.m57.hermescontrol.data.remote.OkHttpProvider
|
||||
import com.m57.hermescontrol.data.remote.safeApiCall
|
||||
@@ -66,6 +70,8 @@ data class ChatUiState(
|
||||
val errorMessage: String? = null,
|
||||
// Background job completion toast (issue #527) — non-blocking snackbar
|
||||
val backgroundCompleteMessage: String? = null,
|
||||
// Attachment open failure — surfaced as a non-blocking snackbar (issue #724)
|
||||
val openError: String? = null,
|
||||
val clarifyRequest: ClarifyUi? = null,
|
||||
// Sudo / secret prompts — surfaced as dialogs (issue #524)
|
||||
val sudoPrompt: SudoPromptUi? = null,
|
||||
@@ -415,6 +421,16 @@ class ChatViewModel(
|
||||
// than waiting up to 5s for the next session-sync poll.
|
||||
viewModelScope.launch { fetchContextUsage() }
|
||||
}
|
||||
|
||||
is ReducerEffect.AttachHostMedia -> {
|
||||
// Issue #724: turn host-path MEDIA: directives into real
|
||||
// attachments (images inline, every other file tappable)
|
||||
// via the gateway /api/files/download endpoint. Works on a
|
||||
// remote phone too.
|
||||
viewModelScope.launch(Dispatchers.IO) {
|
||||
attachHostMedia(effect.sessionId, effect.messageId)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1052,10 +1068,11 @@ class ChatViewModel(
|
||||
// #576). For those we fall back to slash.exec, which runs the
|
||||
// full COMMAND_REGISTRY through the worker.
|
||||
val result =
|
||||
wsClient.request(
|
||||
WsMethods.COMMAND_DISPATCH,
|
||||
mapOf("name" to name, "arg" to arg, "session_id" to sessionId),
|
||||
).await()
|
||||
wsClient
|
||||
.request(
|
||||
WsMethods.COMMAND_DISPATCH,
|
||||
mapOf("name" to name, "arg" to arg, "session_id" to sessionId),
|
||||
).await()
|
||||
handleDispatchResult(result)
|
||||
} catch (e: HermesWsClient.HermesRpcException) {
|
||||
val msg = e.message.orEmpty()
|
||||
@@ -1068,13 +1085,14 @@ class ChatViewModel(
|
||||
// which routes the full CLI command set through the worker.
|
||||
try {
|
||||
val result =
|
||||
wsClient.request(
|
||||
WsMethods.SLASH_EXEC,
|
||||
mapOf(
|
||||
"command" to "/$name${if (arg.isNotEmpty()) " $arg" else ""}",
|
||||
"session_id" to sessionId,
|
||||
),
|
||||
).await()
|
||||
wsClient
|
||||
.request(
|
||||
WsMethods.SLASH_EXEC,
|
||||
mapOf(
|
||||
"command" to "/$name${if (arg.isNotEmpty()) " $arg" else ""}",
|
||||
"session_id" to sessionId,
|
||||
),
|
||||
).await()
|
||||
val output = (result as? Map<*, *>)?.get("output") as? String
|
||||
if (!output.isNullOrBlank()) addAssistantMessage(output)
|
||||
} catch (e2: HermesWsClient.HermesRpcException) {
|
||||
@@ -1698,6 +1716,9 @@ class ChatViewModel(
|
||||
.filter { it.reasoningText.isNotBlank() }
|
||||
.associateBy { it.content }
|
||||
|
||||
val baseUrl = AuthManager.getBaseUrl()
|
||||
val token = AuthManager.getToken().orEmpty()
|
||||
|
||||
return messages.mapIndexed { index, msg ->
|
||||
val role =
|
||||
when (msg.role?.lowercase()) {
|
||||
@@ -1714,25 +1735,249 @@ class ChatViewModel(
|
||||
?.toLong()
|
||||
?: System.currentTimeMillis()
|
||||
|
||||
val content = msg.contentText
|
||||
val rawContent = msg.contentText
|
||||
val reasoning =
|
||||
if (msg.reasoningText.isNotBlank()) {
|
||||
msg.reasoningText
|
||||
} else {
|
||||
existingReasoningMap[content]?.reasoningText.orEmpty()
|
||||
existingReasoningMap[rawContent]?.reasoningText.orEmpty()
|
||||
}
|
||||
|
||||
var finalContent = rawContent
|
||||
var attachments: List<Attachment>? = null
|
||||
if (role == MessageRole.ASSISTANT && rawContent.contains("MEDIA:")) {
|
||||
val items = HostMediaExtractor.extract(rawContent)
|
||||
if (items.isNotEmpty()) {
|
||||
finalContent = HostMediaExtractor.strip(rawContent)
|
||||
attachments =
|
||||
items
|
||||
.mapNotNull { item ->
|
||||
val url =
|
||||
GatewayFileClient.buildDownloadUrl(
|
||||
baseUrl,
|
||||
token,
|
||||
item.path,
|
||||
) ?: return@mapNotNull null
|
||||
Attachment(
|
||||
uri = url,
|
||||
name = mediaNameFromPath(item.path),
|
||||
mimeType = mediaMimeForPath(item.path),
|
||||
size = 0,
|
||||
gatewayUrl = url,
|
||||
source = AttachmentSource.GATEWAY,
|
||||
)
|
||||
}.takeIf { it.isNotEmpty() }
|
||||
}
|
||||
}
|
||||
|
||||
ChatMessage(
|
||||
id = "rest-$sessionId-$globalIndex",
|
||||
role = role,
|
||||
content = content,
|
||||
content = finalContent,
|
||||
reasoningText = reasoning,
|
||||
attachments = attachments,
|
||||
timestamp = timestamp,
|
||||
isStreaming = false,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Issue #724: attach host-path MEDIA: files as real attachments ────
|
||||
//
|
||||
// The gateway's WebSocket stream delivers the raw `MEDIA:<path>` directive
|
||||
// the desktop app turns into an authenticated `/api/files/download?...`
|
||||
// URL. We parse every directive, build the download URL via
|
||||
// [GatewayFileClient], classify it (image / audio / video / file) using
|
||||
// [mediaKindForPath], and attach it to the message. Images render inline
|
||||
// (Coil loads the URL); every other type becomes a tappable, fetchable
|
||||
// attachment. The directive text is stripped from the message body. Works
|
||||
// on a remote phone (real HTTP). Mobile-only; backend untouched. Pure
|
||||
// parsing lives in [HostMediaExtractor].
|
||||
|
||||
/**
|
||||
* ViewModel-side handler for [ReducerEffect.AttachHostMedia]: find the local
|
||||
* message by id, convert any `MEDIA:<path>` directives into [Attachment]s
|
||||
* (via the gateway download URL) and strip them from the text. Role,
|
||||
* reasoning, timestamp and existing attachments are preserved; new gateway
|
||||
* attachments are appended. Idempotent — skips if gateway attachments for
|
||||
* the same paths already exist.
|
||||
*/
|
||||
private fun attachHostMedia(
|
||||
sessionId: String,
|
||||
messageId: String,
|
||||
) {
|
||||
val current = _uiState.value.messages.find { it.id == messageId } ?: return
|
||||
val content = current.content
|
||||
val items = HostMediaExtractor.extract(content)
|
||||
if (items.isEmpty()) return
|
||||
|
||||
val baseUrl = AuthManager.getBaseUrl()
|
||||
val token = AuthManager.getToken()
|
||||
if (baseUrl.isBlank() || token.isNullOrBlank()) return
|
||||
|
||||
val existingUrls =
|
||||
current.attachments
|
||||
.orEmpty()
|
||||
.mapNotNull { it.gatewayUrl }
|
||||
.toSet()
|
||||
val newAttachments =
|
||||
items.mapNotNull { item ->
|
||||
val url = GatewayFileClient.buildDownloadUrl(baseUrl, token, item.path) ?: return@mapNotNull null
|
||||
if (url in existingUrls) return@mapNotNull null
|
||||
Attachment(
|
||||
uri = url,
|
||||
name = mediaNameFromPath(item.path),
|
||||
mimeType = mediaMimeForPath(item.path),
|
||||
size = 0,
|
||||
gatewayUrl = url,
|
||||
source = AttachmentSource.GATEWAY,
|
||||
)
|
||||
}
|
||||
if (newAttachments.isEmpty()) return
|
||||
|
||||
val stripped = HostMediaExtractor.strip(content)
|
||||
_uiState.update { state ->
|
||||
state.copy(
|
||||
messages =
|
||||
state.messages.map { msg ->
|
||||
if (msg.id == messageId) {
|
||||
msg.copy(
|
||||
content = stripped,
|
||||
attachments =
|
||||
(msg.attachments.orEmpty() + newAttachments)
|
||||
.distinctBy { it.gatewayUrl ?: it.uri },
|
||||
)
|
||||
} else {
|
||||
msg
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Open an attachment when its chip/thumbnail is tapped.
|
||||
*
|
||||
* - LOCAL (user-picked) files: open the original `content://` URI
|
||||
* directly via [android.content.Intent.ACTION_VIEW] — the resolver
|
||||
* already grants read access for the picked document. If that fails
|
||||
* (e.g. the permission lapsed), we copy to cache and retry via
|
||||
* FileProvider so the tap is never a silent no-op.
|
||||
* - GATEWAY (agent `MEDIA:`) files: fetch the bytes via
|
||||
* [GatewayFileClient], write them to a cache file, and open with
|
||||
* [android.content.Intent.ACTION_VIEW] through FileProvider — so a
|
||||
* remote phone can view agent-delivered files in-place.
|
||||
*
|
||||
* Failures surface through [ChatUiState.openError] (non-blocking
|
||||
* snackbar); the tap is never swallowed.
|
||||
*/
|
||||
fun openAttachment(attachment: Attachment) {
|
||||
val ctx = getApplication<Application>().applicationContext
|
||||
if (attachment.source == AttachmentSource.LOCAL) {
|
||||
// Best-effort direct open of the picked content URI.
|
||||
runCatching { openWithView(ctx, android.net.Uri.parse(attachment.uri), attachment.mimeType) }
|
||||
.onSuccess { return }
|
||||
.onFailure { /* fall through to cache-copy below */ }
|
||||
}
|
||||
// GATEWAY, or LOCAL direct-open failed → fetch/copy then open.
|
||||
val path =
|
||||
attachment.gatewayUrl?.let { url ->
|
||||
runCatching {
|
||||
java.net
|
||||
.URL(url)
|
||||
.query
|
||||
.split('&')
|
||||
.firstOrNull { it.startsWith("path=") }
|
||||
?.removePrefix("path=")
|
||||
?.let { java.net.URLDecoder.decode(it, "UTF-8") }
|
||||
}.getOrNull()
|
||||
} ?: attachment.name
|
||||
viewModelScope.launch(Dispatchers.IO) {
|
||||
when (val result = GatewayFileClient.fetch(path)) {
|
||||
is GatewayFileResult.Success -> {
|
||||
openBytes(ctx, result.file)
|
||||
}
|
||||
|
||||
is GatewayFileResult.NotFound -> {
|
||||
showOpenError("File not found on gateway: ${attachment.name}")
|
||||
}
|
||||
|
||||
is GatewayFileResult.Forbidden -> {
|
||||
showOpenError("Access denied: ${attachment.name}")
|
||||
}
|
||||
|
||||
is GatewayFileResult.TooLarge -> {
|
||||
showOpenError("File too large to open: ${attachment.name}")
|
||||
}
|
||||
|
||||
is GatewayFileResult.Unauthorized -> {
|
||||
showOpenError("Session expired — reconnect to open: ${attachment.name}")
|
||||
}
|
||||
|
||||
is GatewayFileResult.Failure -> {
|
||||
showOpenError("Could not open ${attachment.name}: ${result.throwable.message}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Open bytes written to a cache file via FileProvider + ACTION_VIEW. */
|
||||
private fun openBytes(
|
||||
ctx: android.content.Context,
|
||||
file: GatewayFile,
|
||||
) {
|
||||
runCatching {
|
||||
val dir = java.io.File(ctx.cacheDir, "gateway_files").also { it.mkdirs() }
|
||||
val safeName = file.name.replace(Regex("[/\\\\]"), "_").ifBlank { "file" }
|
||||
val out = java.io.File(dir, safeName)
|
||||
out.writeBytes(file.bytes)
|
||||
val uri =
|
||||
androidx.core.content.FileProvider.getUriForFile(
|
||||
ctx,
|
||||
"${ctx.packageName}.fileprovider",
|
||||
out,
|
||||
)
|
||||
openWithView(ctx, uri, file.mimeType)
|
||||
}.onFailure { showOpenError("Could not open ${file.name}: ${it.message}") }
|
||||
}
|
||||
|
||||
/** Fire an ACTION_VIEW intent; throws if no activity can handle the type. */
|
||||
private fun openWithView(
|
||||
ctx: android.content.Context,
|
||||
uri: android.net.Uri,
|
||||
mimeType: String,
|
||||
) {
|
||||
val viewIntent =
|
||||
android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
|
||||
setDataAndType(uri, mimeType.ifBlank { "*/*" })
|
||||
addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
|
||||
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
|
||||
}
|
||||
try {
|
||||
ctx.startActivity(viewIntent)
|
||||
} catch (e: Throwable) {
|
||||
val fallbackIntent =
|
||||
android.content.Intent(android.content.Intent.ACTION_VIEW).apply {
|
||||
setDataAndType(uri, "*/*")
|
||||
addFlags(android.content.Intent.FLAG_GRANT_READ_URI_PERMISSION)
|
||||
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
|
||||
}
|
||||
val chooser =
|
||||
android.content.Intent.createChooser(fallbackIntent, "Open file").apply {
|
||||
addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
|
||||
}
|
||||
ctx.startActivity(chooser)
|
||||
}
|
||||
}
|
||||
|
||||
private fun showOpenError(message: String) {
|
||||
_uiState.update { it.copy(openError = message) }
|
||||
}
|
||||
|
||||
fun clearOpenError() {
|
||||
_uiState.update { it.copy(openError = null) }
|
||||
}
|
||||
|
||||
private fun serverMessageIndex(
|
||||
id: String,
|
||||
sessionId: String,
|
||||
|
||||
@@ -255,6 +255,13 @@ object ChatWsEventReducer {
|
||||
// Refresh the context meter immediately on completion instead of
|
||||
// waiting for the next 5s session-sync poll.
|
||||
effects.add(ReducerEffect.RefreshContextUsage)
|
||||
// Mobile-only media fix (issue #724): the WS stream delivers the raw
|
||||
// `MEDIA:<host-path>` directive. Turn it into real attachments (images
|
||||
// inline, every other file tappable) via the gateway /api/files/download
|
||||
// endpoint — works on a remote phone too.
|
||||
if (msg.role == MessageRole.ASSISTANT && sid != null) {
|
||||
effects.add(ReducerEffect.AttachHostMedia(sid, msg.id))
|
||||
}
|
||||
return ReducerResult(
|
||||
state =
|
||||
state.copy(
|
||||
@@ -288,6 +295,12 @@ object ChatWsEventReducer {
|
||||
if (sid != null) {
|
||||
effects.add(ReducerEffect.PersistMessage(msg, sid))
|
||||
}
|
||||
// Mobile-only media fix (issue #724): attach any `MEDIA:<path>` file
|
||||
// carried in the finalized assistant message as a real attachment. See
|
||||
// the ReducerEffect.AttachHostMedia KDoc for the rationale.
|
||||
if (msg.role == MessageRole.ASSISTANT && sid != null) {
|
||||
effects.add(ReducerEffect.AttachHostMedia(sid, msg.id))
|
||||
}
|
||||
return ReducerResult(
|
||||
state =
|
||||
state.copy(
|
||||
@@ -669,4 +682,22 @@ sealed class ReducerEffect {
|
||||
data object RefreshSessions : ReducerEffect()
|
||||
|
||||
data object RefreshContextUsage : ReducerEffect()
|
||||
|
||||
/**
|
||||
* Ask the ViewModel to turn any `MEDIA:<host-path>` directives in the
|
||||
* just-finalized assistant message into real [Attachment]s (images inline,
|
||||
* every other file type tappable) and strip the directives from the text.
|
||||
*
|
||||
* Why: the gateway's WebSocket stream delivers the raw `MEDIA:<path>`
|
||||
* directive (the same one the desktop app resolves via `mediaExternalUrl`
|
||||
* to the authenticated `/api/files/download` endpoint, which streams the
|
||||
* host file over HTTP). Attaching them lets the renderer show images inline
|
||||
* and offer every other file as a fetchable attachment — even on a remote
|
||||
* phone (real HTTP, unlike a host-local file read). Mirrors desktop's
|
||||
* `mediaExternalUrl`; mobile-only, backend untouched. See issue #724.
|
||||
*/
|
||||
data class AttachHostMedia(
|
||||
val sessionId: String,
|
||||
val messageId: String,
|
||||
) : ReducerEffect()
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package com.m57.hermescontrol.ui.chat
|
||||
|
||||
/**
|
||||
* Issue #724: parse agent-delivered `MEDIA:<path>` directives out of message
|
||||
* text so they can be attached (and rendered) as real files.
|
||||
*
|
||||
* The gateway's WebSocket stream delivers the raw `MEDIA:<path>` directive the
|
||||
* desktop app resolves via `mediaExternalUrl` — an authenticated
|
||||
* `/api/files/download?path=<enc>&token=<enc>` URL that streams the host file's
|
||||
* bytes over HTTP. Mobile turns each directive into an [Attachment] (see
|
||||
* [com.m57.hermescontrol.ui.chat.ChatViewModel.attachHostMedia]) so the
|
||||
* renderer can show images inline and offer every other file type as a
|
||||
* tappable, fetchable attachment — including on a *remote* phone.
|
||||
*
|
||||
* Matches ANY absolute (or `~/`) host path regardless of extension, so images,
|
||||
* audio, video, CSV, PDF and arbitrary files are all supported. The pure parse
|
||||
* logic lives here so it can be unit-tested without an Android context.
|
||||
*/
|
||||
internal object HostMediaExtractor {
|
||||
/** A single `MEDIA:<path>` directive found in message text. */
|
||||
data class Item(
|
||||
val match: String, // the full matched directive (for stripping)
|
||||
val path: String, // normalized absolute path
|
||||
)
|
||||
|
||||
/** Matches `MEDIA:<path>` (any extension), supporting quotes/backticks around the path. */
|
||||
private val MEDIA_TAG_RE =
|
||||
Regex(
|
||||
"""[`"']?MEDIA:\s*(?:"([^"]+)"|'([^']+)'|`([^`]+)`|((?:~|/|[A-Za-z]:)[^\s`"')]*))[`"']?""",
|
||||
RegexOption.IGNORE_CASE,
|
||||
)
|
||||
|
||||
/** Extract every `MEDIA:<path>` directive from [text] as a normalized item. */
|
||||
fun extract(text: String): List<Item> {
|
||||
if (!text.contains("MEDIA:")) return emptyList()
|
||||
return MEDIA_TAG_RE
|
||||
.findAll(text)
|
||||
.mapNotNull { m ->
|
||||
val raw = m.groupValues.drop(1).firstOrNull { it.isNotBlank() } ?: return@mapNotNull null
|
||||
val path = normalizePath(raw) ?: return@mapNotNull null
|
||||
Item(match = m.value, path = path)
|
||||
}.toList()
|
||||
}
|
||||
|
||||
/** Remove all `MEDIA:<path>` directives from [text] (leaves other content). */
|
||||
fun strip(text: String): String {
|
||||
if (!text.contains("MEDIA:")) return text
|
||||
return MEDIA_TAG_RE
|
||||
.replace(text) { "" }
|
||||
.replace(Regex("""[ \t]+"""), " ")
|
||||
.replace(Regex("""\n{3,}"""), "\n\n")
|
||||
.trim()
|
||||
}
|
||||
|
||||
/** Expand ~/ and verify the path is absolute. Null if not an absolute path. */
|
||||
internal fun normalizePath(raw: String): String? {
|
||||
val trimmed =
|
||||
raw
|
||||
.trim()
|
||||
.removeSurrounding("`")
|
||||
.removeSurrounding("\"")
|
||||
.removeSurrounding("'")
|
||||
val expanded =
|
||||
if (trimmed.startsWith("~")) {
|
||||
val home = System.getenv("HOME") ?: return null
|
||||
home + trimmed.removePrefix("~")
|
||||
} else {
|
||||
trimmed
|
||||
}
|
||||
if (!expanded.startsWith("/") &&
|
||||
!Regex("^[A-Za-z]:[/\\\\]").containsMatchIn(expanded)
|
||||
) {
|
||||
return null
|
||||
}
|
||||
return expanded
|
||||
}
|
||||
}
|
||||
@@ -43,6 +43,8 @@ import androidx.compose.ui.text.style.TextDecoration
|
||||
import androidx.compose.ui.text.withStyle
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.unit.sp
|
||||
import com.m57.hermescontrol.data.local.AuthManager
|
||||
import com.m57.hermescontrol.data.remote.GatewayFileClient
|
||||
import com.m57.hermescontrol.theme.LocalHermesStatusColors
|
||||
import com.m57.hermescontrol.theme.SearchHighlightColors
|
||||
import com.m57.hermescontrol.theme.searchHighlightColors
|
||||
@@ -268,7 +270,7 @@ fun MarkdownText(
|
||||
// AsyncImage is a Compose runtime composable — load it lazily
|
||||
// behind remember so the parser stays pure and testable (the
|
||||
// parseBlocks/parseInline unit tests never touch Compose).
|
||||
val model: Any = block.uri
|
||||
val model: Any = remember(block.uri) { resolveImageUrl(block.uri) }
|
||||
androidx.compose.foundation.layout.Box(
|
||||
modifier =
|
||||
Modifier
|
||||
@@ -1000,3 +1002,26 @@ internal enum class TableAlign {
|
||||
CENTER,
|
||||
RIGHT,
|
||||
}
|
||||
|
||||
private fun resolveImageUrl(uri: String): String {
|
||||
val trimmed = uri.trim()
|
||||
if (trimmed.isBlank()) return uri
|
||||
if (trimmed.startsWith("http://") || trimmed.startsWith("https://") || trimmed.startsWith("data:image/")) {
|
||||
return trimmed
|
||||
}
|
||||
val baseUrl = runCatching { AuthManager.getBaseUrl() }.getOrDefault("")
|
||||
val token = runCatching { AuthManager.getToken() }.getOrNull().orEmpty()
|
||||
val downloadUrl = GatewayFileClient.buildDownloadUrl(baseUrl, token, trimmed)
|
||||
if (downloadUrl != null) return downloadUrl
|
||||
|
||||
if (baseUrl.isNotBlank() && (trimmed.startsWith("/api/") || trimmed.startsWith("api/"))) {
|
||||
val cleanPath = if (trimmed.startsWith("/")) trimmed else "/$trimmed"
|
||||
val sep = if (cleanPath.contains("?")) "&" else "?"
|
||||
return if (token.isNotBlank() && !cleanPath.contains("token=")) {
|
||||
"${baseUrl.trimEnd('/')}$cleanPath${sep}token=$token"
|
||||
} else {
|
||||
"${baseUrl.trimEnd('/')}$cleanPath"
|
||||
}
|
||||
}
|
||||
return uri
|
||||
}
|
||||
@@ -33,6 +33,7 @@ fun ChatLifecycleEffects(
|
||||
messages: List<ChatMessage>,
|
||||
errorMessage: String?,
|
||||
backgroundCompleteMessage: String?,
|
||||
openError: String?,
|
||||
isSearchActive: Boolean,
|
||||
currentSearchMatchIndex: Int,
|
||||
searchMatchIndices: List<Int>,
|
||||
@@ -127,6 +128,14 @@ fun ChatLifecycleEffects(
|
||||
}
|
||||
}
|
||||
|
||||
// Show attachment-open failures as a non-blocking snackbar (issue #724)
|
||||
LaunchedEffect(openError) {
|
||||
openError?.let { message ->
|
||||
snackbarHostState.showSnackbar(message)
|
||||
viewModel.clearOpenError()
|
||||
}
|
||||
}
|
||||
|
||||
// Sudo / secret prompt dialogs (issue #524)
|
||||
sudoPrompt?.let { prompt ->
|
||||
SudoPromptDialog(
|
||||
|
||||
@@ -108,6 +108,7 @@ fun ChatMessageList(
|
||||
searchQuery = if (isSearchActive) searchQuery else "",
|
||||
isCurrentMatch = isCurrentMatch,
|
||||
onRespondApproval = viewModel::respondToApproval,
|
||||
onOpenAttachment = viewModel::openAttachment,
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -127,6 +128,7 @@ fun ChatMessageList(
|
||||
isDarkTheme = isDark,
|
||||
searchQuery = "",
|
||||
isCurrentMatch = false,
|
||||
onOpenAttachment = viewModel::openAttachment,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user