fix(chat): context meter reflects compression via session.context_breakdown RPC (#759)
* fix(chat): context meter reflects compression via session.context_breakdown RPC
The chat context meter read GET /api/sessions/{id} input_tokens — a
cumulative lifetime counter that compression never resets, so the meter
only ever grew and sat red after the first compaction.
Mirror the desktop: read live occupancy from the session.context_breakdown
WS RPC (context_used = compressor last_prompt_tokens, falling back to an
estimate of the live system prompt + tools + compacted history). The
compressor zeroes last_prompt_tokens when it runs, so the meter now drops
right after auto-compaction or /compress, exactly like desktop/web.
REST getSessionDetail is kept solely to feed the detail sheet's cumulative
token accounting. RPC failures keep the last known values; the meter never
blanks. Fixes #756.
* feat(chat): compressed xN badge on the context widget
The context chip now shows how many times the current session has been
context-compressed, sourced from the session.usage WS RPC (compressions =
context_compressor.compression_count) — the same usage snapshot the desktop
status bar reads. The badge explains a meter drop after auto-compaction or
/compress: the widget no longer just shows the number, it shows the story.
- UsageSnapshotResponse model + parseUsageSnapshot (Number-safe like the
breakdown parser — WS parser decodes JSON numbers as Double)
- compressionCount added to ChatUiState, reset on session switch/create/branch
- session.usage fires alongside session.context_breakdown in the 5s poll;
failures keep the last known badge value (best-effort, never blank)
- 3 parser tests; full local suite 615 tests green
---------
Co-authored-by: M57 <hy4ri@users.noreply.github.com>
This commit is contained in:
1 parent
ed48e10ca6
commit
e014d2a0e7
9 files changed
+320
-32
No files matched your search
@@ -0,0 +1,49 @@
|
||||
package com.m57.hermescontrol.data.model
|
||||
|
||||
/**
|
||||
* Live context-window occupancy from the `session.context_breakdown` WS RPC —
|
||||
* the same RPC the Hermes desktop app's status-bar meter uses.
|
||||
*
|
||||
* Unlike the cumulative `input_tokens` REST counter, [contextUsed] is the
|
||||
* live agent's actual prompt occupancy: the context compressor's
|
||||
* `last_prompt_tokens` (the last prompt actually sent to the model), falling
|
||||
* back to an estimate of the live system prompt + tools + (compacted)
|
||||
* history. It therefore DROPS after context compression, which the cumulative
|
||||
* counter never does (issue #756).
|
||||
*
|
||||
* All counts are nullable — the backend omits/zeroes them when there is no
|
||||
* live agent or the engine doesn't track per-window occupancy.
|
||||
*/
|
||||
data class ContextBreakdownResponse(
|
||||
/** Actual current prompt size in tokens, or null when unknown. */
|
||||
val contextUsed: Long?,
|
||||
/** The active model's full context window in tokens, or null. */
|
||||
val contextMax: Long?,
|
||||
/** Clamped 0-100 fill percentage, or null. */
|
||||
val contextPercent: Int?,
|
||||
/** Rough estimate of system prompt + tools + history, or null. */
|
||||
val estimatedTotal: Long?,
|
||||
val model: String?,
|
||||
)
|
||||
|
||||
/**
|
||||
* Parse the decoded JSON-RPC result of `session.context_breakdown`.
|
||||
*
|
||||
* The WS event parser decodes JSON numbers as [Double] (`JsonRpcModels.toAny`),
|
||||
* so every count is read through [Number]. Returns null when the payload is
|
||||
* not a map (error/malformed response) — callers treat that as unknown and
|
||||
* keep the last known meter values.
|
||||
*/
|
||||
fun parseContextBreakdown(result: Any?): ContextBreakdownResponse? {
|
||||
val map = result as? Map<*, *> ?: return null
|
||||
|
||||
fun num(key: String): Long? = (map[key] as? Number)?.toLong()
|
||||
|
||||
return ContextBreakdownResponse(
|
||||
contextUsed = num("context_used"),
|
||||
contextMax = num("context_max"),
|
||||
contextPercent = (map["context_percent"] as? Number)?.toInt(),
|
||||
estimatedTotal = num("estimated_total"),
|
||||
model = map["model"] as? String,
|
||||
)
|
||||
}
|
||||
@@ -4,7 +4,7 @@ import kotlinx.serialization.Serializable
|
||||
|
||||
/**
|
||||
* Response from `GET /api/sessions/{id}` (gated — rides the app's existing
|
||||
* cookie auth). Only the fields the chat screen needs for the context meter
|
||||
* cookie auth). Only the fields the chat screen's context detail sheet needs
|
||||
* are modelled; the backend serializes many more and [kotlinx.serialization]
|
||||
* ignores the rest.
|
||||
*
|
||||
@@ -13,15 +13,19 @@ import kotlinx.serialization.Serializable
|
||||
* `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `reasoning_tokens`,
|
||||
* `message_count`, etc. — but it does NOT have a `last_prompt_tokens` column
|
||||
* (that field lives only on the in-memory `SessionEntry` in the gateway
|
||||
* process, never persisted to the REST response). So the *used* context window
|
||||
* is sourced from `input_tokens` (cumulative prompt tokens for the session),
|
||||
* paired with [ModelInfoResponse.effective_context_length] as the denominator.
|
||||
* process, never persisted to the REST response).
|
||||
*
|
||||
* NOTE (issue #756): this endpoint is NO LONGER the chat meter's numerator
|
||||
* source — `input_tokens` is a cumulative lifetime counter that never drops
|
||||
* after context compression. The meter reads live occupancy from the
|
||||
* `session.context_breakdown` WS RPC ([ContextBreakdownResponse]); this
|
||||
* response feeds only the detail sheet's cumulative token accounting.
|
||||
*/
|
||||
@Serializable
|
||||
data class SessionDetailResponse(
|
||||
val session_id: String? = null,
|
||||
val session_key: String? = null,
|
||||
/** Cumulative prompt tokens for the session — the *used* context window. */
|
||||
/** Cumulative prompt tokens for the session (lifetime accounting). */
|
||||
val input_tokens: Long? = null,
|
||||
val output_tokens: Long? = null,
|
||||
val cache_read_tokens: Long? = null,
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
package com.m57.hermescontrol.data.model
|
||||
|
||||
/**
|
||||
* Minimal view of the `session.usage` WS RPC result — only the fields the
|
||||
* context widget consumes today. The backend sends many more keys
|
||||
* (`input`/`output`/`total`/`calls`/`active_subagents`/`credits_lines` …);
|
||||
* parsing is key-by-key, so unmodelled keys are simply ignored.
|
||||
*
|
||||
* Verified backend shape (`tui_gateway/server.py` `_get_usage`): `compressions`
|
||||
* = `context_compressor.compression_count` (0 when never compressed, absent on
|
||||
* sessions with no live agent).
|
||||
*/
|
||||
data class UsageSnapshotResponse(
|
||||
/** How many times this session has been context-compressed. */
|
||||
val compressions: Int?,
|
||||
)
|
||||
|
||||
/**
|
||||
* Parse the decoded JSON-RPC result of `session.usage`.
|
||||
*
|
||||
* The WS event parser decodes JSON numbers as [Double] (`JsonRpcModels.toAny`),
|
||||
* so every count is read through [Number]. Returns null when the payload is
|
||||
* not a map (error/malformed response) — callers keep the last known value.
|
||||
*/
|
||||
fun parseUsageSnapshot(result: Any?): UsageSnapshotResponse? {
|
||||
val map = result as? Map<*, *> ?: return null
|
||||
return UsageSnapshotResponse(
|
||||
compressions = (map["compressions"] as? Number)?.toInt(),
|
||||
)
|
||||
}
|
||||
Reference in new issue
Block a user