yilsem/aiturk-hermes-ide
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 14 Wiki Activity

Import AITURK IDE 1.0.0-beta.1 from Hermes 63279301; preserve MIT license

Browse Source
This commit is contained in:
yilsem committed 2026-09-05 13:26:46 +03:00
commit 03634b1ca3
11340 files changed
+3442369
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

website/docs/user-guide/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "User Guide",
"position": 2,
"link": {
"type": "generated-index",
"description": "Learn how to use Hermes Agent effectively."
}
}
website/docs/user-guide/bot-mode.md
+200
View File
@@ -0,0 +1,200 @@
---
title: "Bot Mode"
description: "Turn your Hermes profiles into a roster of named Bots — each with its own chat, role, model, memory, skills, and avatar. Bots run routines, share group chats, and message each other."
---
# Bot Mode
**Bot Mode** turns your [Hermes profiles](./profiles.md) into a roster of named **Bots**. Each Bot has its own role, model, memory, skills, and avatar; Bots run recurring routines, deliberate together in group chats, and message each other directly. Build a specialist Bot once and it is there forever, one click away.
Bot Mode ships **built into the [desktop app](./desktop.md)** and is **on by default** — no install needed. It appears as a **Bots** tab next to Sessions in the left sidebar, with a **Routines** tile docked beside the conversation while the Bots tab is active.
:::tip A Bot is a profile
There is no new primitive to learn: a Bot **is** a Hermes profile — isolated config, memory, skills, credentials, and chat history under `~/.hermes/profiles/<name>/`. Bot Mode is a UI over that primitive, so everything you do in it is visible from the CLI too: `hermes -p <bot> chat` opens the same agent, and Bot routines appear in `hermes cron list`. No core patches, no background daemons, no extra storage.
:::
## The Bots pane
The roster shows one row per agent profile: avatar, latest-message preview, and timestamp.
- **Click a Bot** to land in its chat — every Bot has a canonical, persistent **Bot Chat** conversation that is created (and pinned) the moment the Bot is born. A row click always opens that Bot Chat (the same conversation the row previews), even when you have other tabs open for the Bot; those tabs stay open beside it. In the tab strip the Bot Chat is captioned with the Bot's name, so two open Bots are told apart at a glance.
- **Active now** — a presence strip above the roster shows every Bot currently working: the gateway-busy profile plus any Bot that wrote within the last 90 seconds. Each chip opens that Bot's chat. The strip never reorders the roster and disappears when the fleet is idle.
- **Search** filters the roster as you type.
- **Hide a Bot** — right-click a row → **Hide Bot** to take a Bot you don't use out of the roster and the Active-now strip. Hiding is display-only: @mentions still resolve, group-chat memberships are untouched, and routines keep running. Once at least one Bot is hidden, an **eye toggle** appears in the pane header — click it to reveal hidden Bots dimmed in place, then right-click → **Unhide Bot** to bring one back. Hidden Bots never toast, but they accumulate unread activity silently and the eye badges a dot so you know something happened. Hidden state is saved in the Bot's profile metadata, so it follows the Bot to every desktop connected to that backend.
:::note The canonical Bot Chat is a forever-chat
Typing `/new` (or `/reset`) inside a Bot's canonical chat would fork the relationship into a scratch session — the one thing Bot Mode promises never happens. The composer reroutes it to `/compact` instead: fresh working context, same conversation. Regular sessions on the same profile keep full `/new` freedom.
:::
### Organize bots into sections
Sections are folders you make yourself — **Clients**, **Team**, whatever fits — as a second axis beside the automatic per-gateway grouping. With no sections created the roster is the plain list it always was.
- **Create one** from the pane's **+** menu → **New section**, or right-click a Bot → **Move to section** → **New section…** (that files the Bot into it as you create it).
- **File a Bot** by dragging its row onto a section — the target highlights while you hover, and **Esc** cancels the drag — or right-click → **Move to section** and pick one. **Remove from section** puts it back in **Unassigned**.
- **Rename, reorder, or delete** a section from its heading's right-click menu (or the **⋯** that appears on hover); double-click a heading to rename. Headings fold like the gateway headings do.
- **Deleting a section never deletes Bots** — they return to **Unassigned**, and the toast offers **Undo**. No confirmation is asked.
Membership is stored in each Bot's profile metadata (`ui_meta`), so a Bot's section follows it to every desktop connected to that backend. When the roster shows more than one gateway, sections nest inside each gateway's bucket.
## Creating a Bot
Hit **New Agent** in the roster. The quick path is three fields — **Name**, **Title**, **Description** — and the Bot exists in seconds, introducing itself as the first message of its new Bot Chat.
An **Advanced** disclosure opens the full capabilities surface:
- **Clone from an existing profile** — start from another Bot's config, skills, SOUL, and memory, or pick **Fresh profile** for a clean start.
- **Create empty** — skip the bundled skills entirely for a minimal profile.
- **Model & provider pin** — give the Bot its own model. Any provider/model pair Hermes knows about works, and different Bots can run on different models side by side. Leave it unset to inherit from the launch profile.
- **Custom SOUL.md** — the Bot's persona and standing instructions.
- **Per-skill, per-toolset, and per-MCP-server enablement** — tick exactly the capabilities this specialist needs.
- **Shared keys** — by default the new Bot shares one OAuth/token pool with the main profile, so credential refreshes cannot invalidate each other. (Older gateways copy credentials instead — still functional, just forked.)
### Choosing which machine it lives on ("Create on")
With more than one connection registered in [Settings → Connections](./multi-connection-desktop.md), the New Agent dialog grows a **Create on** picker. Pick a device and the profile is created on **that** machine's backend — your window never switches gateways. The new Bot then appears in the roster as a Connections Bot (with an `@name-device` handle when the name exists on several machines), and chatting with it routes to its own machine.
With a single connection (the common case) the picker is hidden and the Bot is created on the machine you're connected to — exactly the old behavior.
Remote-creation notes:
- **Clone source** is a profile of the *target* machine (its `default`) — a remote box doesn't have your local profiles to clone.
- The live Capabilities tab pins to the target machine's backend, so skills, tools, and MCP servers you configure during creation land on the machine the Bot will live on. (Older desktop builds fall back to staged Skills/Tools/MCP checklists for remote targets; both read the target machine's catalog.)
- Cancelling the dialog discards the draft profile on whichever machine it was created.
**Edit Profile** (right-click a Bot) reopens the same surface on the live profile any time: avatar, title, description, model pin, skills, toolsets, MCP servers, and the full SOUL.md.
**Duplicate** (right-click) makes a full clone of a Bot — config, skills, SOUL.md, memory, and its look. **Delete Profile** permanently removes one, behind the same destructive confirmation the desktop's profile menu uses; the default profile cannot be deleted.
## Avatars
Every Bot gets a face:
- **Blob faces** (default) — a deterministic soft-body face drawn from the Bot's name: same name, same face, forever. While you type a name in New Agent the face follows it live; hit **Randomize** to re-roll, **Lock face** to keep the one you like even if the name changes, or pin one of the six silhouettes (round, organic, boxy, nub, cloud, sun) while everything else still comes from the name.
- **Geometric faces** — the classic 7 shapes × 10 colors, with blinking eyes that scan while the Bot works.
- **An uploaded image** — any picture you like.
- **An AI-generated portrait** — when an image backend is configured, generated in place (this rides the standard `image.generate` RPC and works over both local and remote gateways).
- **A pixel pet** — a companion from the [petdex gallery](./features/pets.md) that bounces beside the avatar while the Bot is busy. Run `hermes pets` in a terminal to explore the gallery.
A Bot's look, title, and description are stored in the profile's metadata on the backend, so the same Bot appears the same way on every desktop connected to that backend.
## Routines
The **Routines** pane attaches recurring tasks to the Bot that does them — "summarize my inbox every morning" lives next to the Bot responsible for it. The pane docks beside the chat only while the Bots tab is active and steps aside when you switch back to Sessions (older desktop builds keep it always visible). A structured schedule picker builds the schedule (frequency first, then only the detail that matters), with an Advanced field exposing the raw Hermes schedule string.
Routines are plain [Hermes cron jobs](./features/cron.md) namespaced `[bot:<name>] <routine>` — they also show up in `hermes cron list` and the core Cron page. Runs land in the Bot's own chat history, so the result is right where you would talk to that Bot anyway.
## Groups and group chats
Right-click a local Bot → **Manage groups** to add or remove it from any number of group chats. Pick existing groups independently or create one inline. Local membership is stored in the Bot's backend-synced profile metadata, so it follows that profile across desktops; older profiles with one legacy group continue to work. Connections Bots join through the New Group Chat picker and remain source-qualified in the room's shared state.
**Rooms follow your gateways, not one Desktop.** Each room's recent transcript, members, picture, and name are mirrored into the shared profile metadata of **every** gateway your Desktop is connected to, with per-gateway versioning so two Desktops writing at once merge instead of overwriting each other. Open Hermes Desktop on another machine against the same gateway (local network, Tailscale, anywhere) and the room appears with its history; gateway-only clients see it too. Rooms carry a durable internal identity, so renaming one changes just its display name everywhere, disbanding one removes it permanently on every client — even ones that were offline at the time — and recreating a same-name group starts a genuinely fresh room. If a gateway dies or is removed, nothing is lost: every connected Desktop keeps the full room locally and re-seeds any gateway it reconnects to. (The full orchestration log stays in each Desktop's local storage; the shared mirror is a bounded recent-history projection.)
Groups are standalone rows in the same activity-ordered roster as Bot DMs. A Bot keeps one DM row even when it belongs to several groups, while every group gets its own room row with member count, latest-message preview, timestamp, and needs-you state.
**Open chat** on any group row (2–6 Bots) opens a shared room where the whole group coordinates:
- Your message triggers up to **three serial rounds** of member turns. @-mentioned Bots respond (everyone responds when nobody is mentioned); each Bot replies briefly or passes, and the room settles when a full round stays silent.
- Bots pull each other in with `@name`, and escalate real judgment calls to you with `@user` — the group row shows a **needs you** badge when that happens.
- Hard caps (10 messages per send, 3 rounds) keep rooms from spinning.
- Each member keeps its own persistent `Group: <name>` session, so room context survives like any other conversation.
- **Not every Bot replies to every message.** Speaking is each member's own choice — a Bot replies only when it has something new to add and passes otherwise, and @-mentioning specific members scopes the round to them. Expect the members you addressed (or whoever has something to say) to speak, and the rest to stay quiet.
- **Rooms can span machines.** The New Group Chat picker seats Bots from any registered connection; each member's turns run on its own machine, in its own `Group: <name>` session there. Cross-machine members carry a device badge (`dixie · Mac Mini`) in the room and in other members' transcripts, and the disambiguated `@name-device` handle works in room mentions — so same-named agents on two machines never blur together.
## Bot-to-bot messaging
Bots message each other with attribution, and you can hand work off from any chat:
- **@mentions** — type `@researcher have a look at this` in any chat and the composer's `@` autocomplete helps you pick the right Bot; on send, the mention is resolved against the live roster and the active Bot is told exactly who you mean (profile, friendly name, and device for cross-connection Bots). The Bot then composes its own message and sends it with `message_agent` — your text is never forwarded verbatim, and the reply comes back attributed to that agent. An email address or an unknown `@` passes through untouched. Bots on other connected machines are reachable the same way: the Desktop relays the message over that connection's own socket (see *Bots across machines* below).
- **Renamed Bots keep their tags in sync** — give a Bot a friendly name (the pencil in its chat header, or `hermes profile rename`) and it becomes taggable by that name: a Bot titled *Research Buddy* answers to `@research-buddy` (and `@researchbuddy`), in regular chats and in group rooms alike. The composer's `@` autocomplete offers the renamed tag and also matches when you type the old profile name, which keeps resolving too.
- **Direct messages** — every Bot Chat carries the `message_agent` tool: a Bot messages a teammate by calling `message_agent(target="researcher", message="…")`. The tool validates the target against the live roster, prefixes the sender's `Message from 🤖 <sender> (@<sender>):` attribution automatically, and delivers into the teammate's canonical Bot Chat. Delivery is **fire-and-forget**: the sender gets an acknowledgement, finishes its turn, and the reply arrives later as a background completion notification. The message travels as a real parameter (nothing shell-interpreted — quotes, `$(...)`, and backticks arrive verbatim), and the Bot composes its own message rather than forwarding your words. The teammate roster — names **and roles** from each profile's title/description — is part of every Bot Chat's system prompt, so Bots know who does what before choosing a recipient. The tool exists **only** in canonical Bot Chat sessions on Bot-Mode-managed installs; regular chats, group-room member sessions, and CLI sessions never see it.
The backend teaches each Bot's canonical Bot Chat session the messaging protocol automatically at prompt-build time — including when a teammate opens it headlessly from the CLI. Only the canonical Bot Chat gets the protocol section; your regular sessions and your SOUL.md stay untouched. This is controlled by `agent.bot_mode_protocol` in `config.yaml` (default: on):
```yaml
agent:
bot_mode_protocol: true # inject the bot-to-bot messaging protocol into canonical Bot Chats
```
:::note
Bot-to-bot delivery is per-invocation: the receiving Bot picks the message up when it next runs. Live interrupt of a Bot mid-conversation is future work.
:::
### Failed turns retry safely
A failed delivery turn is retried at most once, and only when a retry can actually help. Transient failures (target runtime offline, delivery timeout, provider rate limit or server error) re-run the same Bot Chat session unchanged. A context-overflow failure also re-runs the same session — the retried turn compacts the over-threshold transcript via the standard context-compression pass before calling the model, so the retry fits where the original didn't. Auth, quota, and configuration failures never auto-retry: a second attempt cannot fix them and only burns quota, so the failure is surfaced immediately. A retried turn never starts a fresh session — your Bot Chat history and context stay intact.
### When a delivery fails: typed reasons
A failed bot turn or relay delivery carries a machine-readable `reason` code alongside the human error text, end to end: the target gateway classifies the failure (`provider_auth_or_access`, `provider_quota_limit`, `provider_rate_limit`, `provider_server_error`, `context_overflow`, `missing_config`, `model_unavailable`, `runtime_offline`, `queued_expired`, `delivery_timeout`, `target_busy`, `unknown`), the Desktop forwards it, and the sending agent's completion notification is tagged `[reason: <code>]` ahead of the error text. A calling agent can branch on the code — "sign in again" vs "retry later" — instead of parsing provider prose. The Desktop's needs-attention badge uses the same codes.
### Messaging across connected machines (the Desktop relay)
Every gateway you register in **Settings → Connections** — local, remote URL, SSH, Hermes Cloud, docker — is a persistent line the Desktop holds open, and Bot Mode uses those lines for messaging automatically. No extra setup:
- **Rosters propagate on their own.** While the Desktop runs, it periodically tells each connected gateway which agents live on the *other* connections. Every Bot Chat's teammate roster then lists them ("Teammates on OTHER connected machines"), with names, roles, and which machine they're on — and the roster refreshes when agents appear, disappear, or get renamed (capability epoch).
- **`message_agent` reaches them directly.** A Bot on your laptop messages the cloud agent with `message_agent(target="moxie", …)` exactly like a local teammate. If the same handle exists on several machines, disambiguate with `target="moxie@<connection>"` (the tool's error tells the Bot the exact forms). Delivery rides the Desktop: the sending gateway queues the message, the Desktop relays it to the target connection's own gateway, the target Bot runs a turn in its canonical Bot Chat, and the reply comes back to the sender as the same background completion notification local DMs use.
- **The Desktop is the courier.** Cross-connection delivery works while a Desktop that knows both connections is running (it holds the sockets and the credentials — gateways never see each other's auth). If the Desktop is closed mid-delivery, the sender's Bot is told the reply didn't arrive rather than left hanging. For always-on machine-to-machine messaging with no Desktop in the loop, register a peer (`hermes peer`, below) — the two routes coexist.
### Bot-initiated DMs across machines (`hermes peer`)
Bots on one machine can message Bots on **another machine's gateway** without any desktop in the loop. Register the other gateway as a *peer* (its API server URL + `API_SERVER_KEY`):
```bash
hermes peer add spark --url http://spark.lan:8377 --key <API_SERVER_KEY>
hermes peer list
hermes peer dm spark < /tmp/dm.txt # message body from a file (nothing shell-interpreted)
hermes peer dm spark/researcher < /tmp/dm.txt # named profile on a multiplexed peer
hermes peer run spark --idempotency-key ticket-123 < /tmp/long-task.txt
hermes peer status spark run_abc123
hermes peer stop spark run_abc123
```
`hermes peer dm` delivers into the remote agent's canonical Bot Chat over the peer's existing API server, runs one agent turn there, and prints the reply on stdout — the exact cross-machine twin of the local `hermes -p <bot> chat` command.
Use `peer dm` only for short queries and receipts because it holds one HTTP
connection until the turn finishes. For a long turn, `peer run` returns a
`run_id` immediately; poll it with `peer status`. The run inherits the
canonical Bot Chat transcript, and a stable `--idempotency-key` makes a retry
return the original run instead of starting duplicate work. Use `peer stop`
with that exact run ID to interrupt it without targeting another turn.
Once a peer is registered, the messaging protocol taught to every Bot Chat (`agent.bot_mode_protocol`) automatically includes the peer roster, and `message_agent` accepts peer targets directly — `message_agent(target="spark/researcher", …)`, or `target="spark"` for the peer's main agent — so **your bots learn on their own** that teammates exist on other machines and how to reach them. Registering or removing a peer refreshes each Bot Chat's protocol on its next message (capability epoch).
Requirements: the peer machine runs the `api_server` gateway platform with a strong `API_SERVER_KEY`; reachability is your network's business (LAN, Tailscale, VPN). The key is a credential and lives in `~/.hermes/.env` as `HERMES_PEER_<NAME>_KEY`; peer names/URLs live in `config.yaml` under `bot_peers`.
:::note One-way reachability (NAT)
Cross-gateway links are direct gateway-to-gateway connections — Desktop is a
viewer, not a relay. A gateway behind home NAT can dial out to a public peer
(laptop → VPS works), but the reverse direction has no inbound route
(VPS → home fails) unless your network provides one. If your Group Chat spans
a NAT boundary, put the room's authority on the host every participant can
reach (typically the public VPS), or bridge the network with Tailscale/VPN.
:::
## Bots across machines
When you register several backends in **Settings → Connections** — the local runtime, remote gateways, SSH hosts, Hermes Cloud instances — the roster shows the Bots from **every** connected source, persistently: SSH sources are inventoried without spawning anything on the remote box, and machines that are momentarily unreachable keep their last-known rows instead of vanishing. When the same profile name exists on several sources, handles disambiguate as `@name-device` (for example `@research-homelab`). A Bot's chats, sessions, memory, and routines live on the machine that owns the profile.
Clicking a Connections Bot does **not** hop your window onto that machine — stay in your chat and `@mention` it, seat it in a group chat, or create new agents on it directly with the **Create on** picker. Cloud and local agents share one roster this way: register your Hermes Cloud instance and your desktop (say, over Tailscale or SSH) and their Bots can message each other and sit in the same rooms, with each agent's work running on its own machine. Bot-to-bot DMs across those machines go through the Desktop relay automatically (see *Messaging across connected machines* above).
See [Connecting Desktop to Many Hermes Instances](./multi-connection-desktop.md) for the full multi-connection guide.
## Turning it off
Bot Mode is a bundled desktop plugin. Flip it off in **Settings → Plugins → Bots** — the roster, the Routines pane, and the composer middleware unregister live, no restart needed. Your profiles, sessions, and cron jobs are untouched either way; Bot Mode never owns your data, it only renders it.
There is also a preference to hide the canonical Bot Chats from the regular sidebar session list, so they only appear inside the Bots pane. (This uses the core hidden-session flag; on older gateways the chats simply stay visible.)
## CLI parity
Because Bots are profiles, everything has a terminal equivalent:
| In Bot Mode | From a shell |
| --- | --- |
| Chat with a Bot | `hermes -p <bot> chat` |
| A Bot's files, skills, memory | `~/.hermes/profiles/<bot>/` |
| Routines | `hermes cron list` (jobs named `[bot:<name>] …`) |
| Create / inspect profiles | `hermes profile create`, `hermes profile list` |
See [Profiles](./profiles.md) for the underlying primitive and [Profile Commands](../reference/profile-commands.md) for the full CLI reference.
website/docs/user-guide/checkpoints-and-rollback.md
+279
View File
@@ -0,0 +1,279 @@
---
sidebar_position: 8
sidebar_label: "Checkpoints & Rollback"
title: "Checkpoints and /rollback"
description: "Filesystem safety nets for destructive operations using shadow git repos and automatic snapshots"
---
# Checkpoints and `/rollback`
Hermes Agent can automatically snapshot your project before **destructive operations** and restore it with a single command. Checkpoints are **opt-in** as of v2 — most users never use `/rollback`, and the shadow-store storage is non-trivial over time, so the default is off.
Enable checkpoints per-session with `--checkpoints`:
```bash
hermes chat --checkpoints
```
Or enable globally in `~/.hermes/config.yaml`:
```yaml
checkpoints:
enabled: true
```
This safety net is powered by an internal **Checkpoint Manager** that keeps a single shared shadow git repository under `~/.hermes/checkpoints/store/` — your real project `.git` is never touched. Every project the agent works in shares the same store, so git's content-addressable object DB deduplicates across projects and across turns.
## What Triggers a Checkpoint
Checkpoints are taken automatically before:
- **File tools** — `write_file` and `patch`
- **Destructive terminal commands** — `rm`, `rmdir`, `cp`, `install`, `mv`, `sed -i`, `truncate`, `dd`, `shred`, output redirects (`>`), and `git reset`/`clean`/`checkout`
The agent creates **at most one checkpoint per directory per turn**, so long-running sessions don't spam snapshots.
## Quick Reference
In-session slash commands:
| Command | Description |
|---------|-------------|
| `/rollback` | List all checkpoints with change stats |
| `/rollback <N>` | Restore to checkpoint N, keeping your hand-edits (also undoes last chat turn) |
| `/rollback <N> --all` | Full restore — overwrites your hand-edits too |
| `/rollback diff <N>` | Preview diff between checkpoint N and current state |
| `/rollback <N> <file>` | Restore a single file from checkpoint N |
CLI for inspecting and managing the store outside a session:
| Command | Description |
|---------|-------------|
| `hermes checkpoints` | Show total size, project count, per-project breakdown |
| `hermes checkpoints status` | Same as bare `checkpoints` |
| `hermes checkpoints list` | Alias for `status` |
| `hermes checkpoints prune` | Force a sweep: delete orphans/stale, GC, enforce size cap |
| `hermes checkpoints clear` | Nuke the entire checkpoint base (asks first) |
| `hermes checkpoints clear-legacy` | Delete only the `legacy-*` archives from v1 migration |
## How Checkpoints Work
At a high level:
- Hermes detects when tools are about to **modify files** in your working tree.
- Once per conversation turn (per directory), it:
- Resolves a reasonable project root for the file.
- Initialises or reuses the **single shared shadow store** at `~/.hermes/checkpoints/store/`.
- Stages into a per-project index, builds a tree, and commits to a per-project ref (`refs/hermes/<project-hash>`).
- These per-project refs form a checkpoint history that you can inspect and restore via `/rollback`.
```mermaid
flowchart LR
user["User command\n(hermes, gateway)"]
agent["AIAgent\n(run_agent.py)"]
tools["File & terminal tools"]
cpMgr["CheckpointManager"]
store["Shared shadow store\n~/.hermes/checkpoints/store/"]
user --> agent
agent -->|"tool call"| tools
tools -->|"before mutate\nensure_checkpoint()"| cpMgr
cpMgr -->|"git add/commit-tree/update-ref"| store
cpMgr -->|"OK / skipped"| tools
tools -->|"apply changes"| agent
```
## Configuration
Configure in `~/.hermes/config.yaml`:
```yaml
checkpoints:
enabled: false # master switch (default: false — opt-in)
max_snapshots: 20 # max checkpoints per project (enforced via ref rewrite + gc)
max_total_size_mb: 500 # hard cap on total store size; oldest commits dropped
max_file_size_mb: 10 # skip any single file larger than this
# Auto-maintenance (on by default): sweep ~/.hermes/checkpoints/ at startup
# and delete project entries whose last_touch is older than retention_days.
# Runs at most once per min_interval_hours, tracked via a .last_prune
# marker. This sweep never deletes "orphan" entries (working directory not
# found) — a missing workdir at startup is ambiguous (deleted project vs.
# an unmounted external volume / network share / VPN not yet up), so
# orphan cleanup is only ever done via the explicit
# `hermes checkpoints prune` command below, with a confirmation prompt.
auto_prune: true
retention_days: 7
min_interval_hours: 24
```
To disable everything:
```yaml
checkpoints:
enabled: false
auto_prune: false
```
When `enabled: false`, the Checkpoint Manager is a no-op and never attempts git operations. When `auto_prune: false`, the store grows until you run `hermes checkpoints prune` manually.
## Listing Checkpoints
From a CLI session:
```
/rollback
```
Hermes responds with a formatted list showing change statistics:
```text
📸 Checkpoints for /path/to/project:
1. 4270a8c 2026-03-16 04:36 before patch (1 file, +1/-0)
2. eaf4c1f 2026-03-16 04:35 before write_file
3. b3f9d2e 2026-03-16 04:34 before terminal: sed -i s/old/new/ config.py (1 file, +1/-1)
/rollback <N> restore to checkpoint N (keeps your hand-edits)
/rollback <N> --all full restore, overwriting your hand-edits too
/rollback diff <N> preview changes since checkpoint N
/rollback <N> <file> restore a single file from checkpoint N
```
## Inspecting the Store from the Shell
```bash
hermes checkpoints
```
Sample output:
```text
Checkpoint base: /home/you/.hermes/checkpoints
Total size: 142.3 MB
store/ 138.1 MB
legacy-* 4.2 MB
Projects: 12
WORKDIR COMMITS LAST TOUCH STATE
/home/you/code/hermes-agent 20 2h ago live
/home/you/code/experiments/rl-runner 8 1d ago live
/home/you/code/old-prototype 3 9d ago orphan
...
Legacy archives (1):
legacy-20260506-050616 4.2 MB
Clear with: hermes checkpoints clear-legacy
```
Force a full sweep (ignores the 24h idempotency marker):
```bash
hermes checkpoints prune --retention-days 3 --max-size-mb 200
```
## Previewing Changes with `/rollback diff`
Before committing to a restore, preview what has changed since a checkpoint:
```
/rollback diff 1
```
This shows a git diff stat summary followed by the actual diff.
## Restoring with `/rollback`
```
/rollback 1
```
Behind the scenes, Hermes:
1. Verifies the target commit exists in the shadow store.
2. Takes a **pre-rollback snapshot** of the current state so you can "undo the undo" later.
3. Restores tracked files in your working directory — **preserving your hand-edits** (see below).
4. **Undoes the last conversation turn** so the agent's context matches the restored filesystem state.
### User hand-edits are preserved by default
`/rollback <N>` restores only the files Hermes itself changed. Every successful
`write_file` / `patch` records the file's content hash in an **agent-write
ledger**; at restore time, any file whose current contents no longer match what
Hermes last wrote (you edited it afterwards, or Hermes never touched it) is
**skipped** instead of overwritten, and listed in the output:
```
✅ Restored to checkpoint a1b2c3d4: before write_file
↷ Kept your hand-edits: src/config.py, notes.md
Use /rollback <N> --all to restore those too.
```
To force the classic full restore that reverts everything — including your own
edits — add `--all`:
```
/rollback 1 --all
```
If the ledger is empty (a store created before this feature, or Hermes hasn't
written any files in the project yet), `/rollback` falls back to the full
restore automatically.
## Single-File Restore
Restore just one file from a checkpoint without affecting the rest of the directory:
```
/rollback 1 src/broken_file.py
```
## Safety and Performance Guards
- **Git availability** — if `git` is not found on `PATH`, checkpoints are transparently disabled.
- **Directory scope** — Hermes skips overly broad directories (root `/`, home `$HOME`).
- **Repository size** — directories with more than 50,000 files are skipped.
- **Per-file size cap** — files larger than `max_file_size_mb` (default 10 MB) are excluded from the snapshot. Prevents accidentally swallowing datasets, model weights, or generated media.
- **Total store size cap** — when the store exceeds `max_total_size_mb` (default 500 MB), the oldest commit per project is dropped round-robin until under the cap.
- **Real pruning** — `max_snapshots` is enforced by rewriting the per-project ref and running `git gc --prune=now` afterwards, so loose objects don't accumulate.
- **No-change snapshots** — if there are no changes since the last snapshot, the checkpoint is skipped.
- **Non-fatal errors** — all errors inside the Checkpoint Manager are logged at debug level; your tools continue to run.
## Where Checkpoints Live
```text
~/.hermes/checkpoints/
├── store/ # single shared bare git repo
│ ├── HEAD, objects/ # git internals (shared across projects)
│ ├── refs/hermes/<hash> # per-project branch tip
│ ├── indexes/<hash> # per-project git index
│ ├── projects/<hash>.json # workdir + created_at + last_touch
│ └── info/exclude
├── .last_prune # auto-prune idempotency marker
└── legacy-<ts>/ # archived pre-v2 per-project shadow repos
```
Each `<hash>` is derived from the absolute path of the working directory. You normally never need to touch these manually — use `hermes checkpoints status` / `prune` / `clear` instead.
### Migration from v1
Before the v2 rewrite, each working directory got its own complete shadow git repo directly under `~/.hermes/checkpoints/<hash>/`. That layout couldn't dedup objects across projects and had a documented no-op pruner — the store would grow without bound.
On first v2 run, any pre-v2 shadow repos are moved into `~/.hermes/checkpoints/legacy-<timestamp>/` so the new single-store layout starts clean. Old `/rollback` history is still reachable by manually inspecting the legacy archive with `git`; once you're confident you don't need it, run:
```bash
hermes checkpoints clear-legacy
```
to reclaim the space. Legacy archives are also swept by `auto_prune` after `retention_days`.
## Best Practices
- **Enable checkpoints only when you need them** — `hermes chat --checkpoints` or per-profile `enabled: true`.
- **Use `/rollback diff` before restoring** — preview what will change to pick the right checkpoint.
- **Use `/rollback` instead of `git reset`** when you want to undo agent-driven changes only.
- **Check `hermes checkpoints status` occasionally** if you use checkpoints regularly — shows which projects are active and what the store costs you.
- **Combine with Git worktrees** for maximum safety — keep each Hermes session in its own worktree/branch, with checkpoints as an extra layer.
For running multiple agents in parallel on the same repo, see the guide on [Git worktrees](./git-worktrees.md).
website/docs/user-guide/cli.md
+556
View File
@@ -0,0 +1,556 @@
---
sidebar_position: 1
title: "CLI Interface"
description: "Master the Hermes Agent terminal interface — commands, keybindings, personalities, and more"
---
# CLI Interface
Hermes Agent's CLI is a full terminal user interface (TUI) — not a web UI. It features multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output. Built for people who live in the terminal.
:::tip First-time setup
One command — `hermes setup --portal` — and you're ready to `hermes chat`. See [Nous Portal](/integrations/nous-portal).
:::
:::tip
Hermes also ships a modern TUI with modal overlays, mouse selection, and non-blocking input. Launch it with `hermes --tui` — see the [TUI](tui.md) guide.
:::
## Running the CLI
```bash
# Start an interactive session (default)
hermes
# Single query mode (non-interactive)
hermes chat -q "Hello"
# Single query from a file or stdin — nothing is shell-interpreted, so
# arbitrary text (quotes, $(...), backticks) arrives verbatim
hermes chat --query-file prompt.txt
hermes chat --query-file - < prompt.txt
# With a specific model
hermes chat --model "anthropic/claude-sonnet-4"
# With a specific provider
hermes chat --provider nous # Use Nous Portal
hermes chat --provider openrouter # Force OpenRouter
# With specific toolsets
hermes chat --toolsets "web,terminal,skills"
# Start with one or more skills preloaded
hermes -s hermes-agent-dev,github-auth
hermes chat -s github-pr-workflow -q "open a draft PR"
# Resume previous sessions
hermes --continue # Resume the most recent CLI session (-c)
hermes --resume <session_id> # Resume a specific session by ID (-r)
hermes --resume latest # Resume the most recent session (same as -c)
hermes --resume latest --in ./dir # Resume ./dir's latest session, staying in ./dir
# Verbose mode (debug output)
hermes chat --verbose
# Isolated git worktree (for running multiple agents in parallel)
hermes -w # Interactive mode in worktree
hermes -w -z "Fix issue #123" # Single query in worktree
```
### Worktree cleanup
`hermes -w` sessions create disposable worktrees under `<repo>/.worktrees/`.
A conservative pruner runs automatically at startup (it only removes clean,
fully-merged scratch trees past an age threshold), but preserved trees and
merged local branches still accumulate on busy machines. Reclaim them
explicitly:
```bash
hermes worktree list # audit: age, size, verdict, reason per tree
hermes worktree prune # remove safe trees + delete merged branches
hermes worktree prune --dry-run # show the plan without changing anything
hermes worktree prune --trees-only # leave local branches alone
hermes worktree prune --branches-only # leave worktrees alone
```
Inside a session, `/worktree prune [--dry-run]` does the same (and never
touches the tree the session is running in).
Safety guarantees (all modes, any age):
- Uncommitted **tracked** changes are never deleted.
- **Unique unpushed commits** are never deleted — commits that were
rebase/squash-merged upstream are detected via `git cherry`
patch-equivalence and count as merged, which is what lets the dominant
"merged PR, tree preserved forever" leak finally reclaim.
- **Pushed open-PR lanes free their disk without losing anything**: when a
clean tree's branch head exactly matches what `origin` holds (checked with
one `git ls-remote` per sweep), the checkout is redundant — the tree is
removed but its **branch ref is kept**, so the lane is one
`git worktree add .worktrees/<name> <branch>` away from restored. If the
remote can't be reached, the tree is preserved.
- Trees **in use by a running hermes session** are never touched.
- **Untracked-only scratch** (PR body drafts, notes) is archived to
`~/.hermes/archive/worktree-prune/` before its tree is removed — never
destroyed.
- Branch deletion is content-gated, not name-gated: any local branch whose
commits are all on upstream is safe to delete; branches with unique work,
checked-out branches, and `main`/`master`/`develop` are always kept.
The same conservative pruner also runs from the cron scheduler (at most once
every 6 hours, in the background), so gateway-only machines — where nobody
launches `hermes -w` for days — no longer accumulate merged scratch trees
between CLI sessions.
When `.worktrees/` grows past 10 trees or 5 GB, startup prints a one-line
notice pointing at these commands.
### Plugin management
The `hermes plugins` commands manage native Hermes plugins and portable Agent
Plugins v1 packages through the same opt-in workflow:
```bash
hermes plugins install owner/repository --no-enable
hermes plugins list
hermes plugins enable <plugin-name>
hermes plugins disable <plugin-name>
hermes plugins update <plugin-name>
hermes plugins remove <plugin-name>
```
Portable packages remain disabled until explicitly enabled. Hermes currently
loads portable Agent Skills and stdio MCP entries. See the
[plugin developer guide](/developer-guide/plugins#portable-agent-plugins-v1-packages)
for the exact supported subset and trust boundary.
## Interface Layout
<img className="docs-terminal-figure" src="/docs/img/docs/cli-layout.svg" alt="Stylized preview of the Hermes CLI layout showing the banner, conversation area, and fixed input prompt." />
<p className="docs-figure-caption">The Hermes CLI banner, conversation stream, and fixed input prompt rendered as a stable docs figure instead of fragile text art.</p>
The welcome banner shows your model, terminal backend, working directory, available tools, and installed skills at a glance.
### Status Bar
A persistent status bar sits above the input area, updating in real time:
```
⚕ claude-sonnet-4-20250514 │ 12.4K/200K │ [██████░░░░] 6% │ $0.06 │ 15m
```
| Element | Description |
|---------|-------------|
| Model name | Current model (truncated if longer than 26 chars) |
| Token count | Context tokens used / max context window |
| Context bar | Visual fill indicator with color-coded thresholds |
| Cost | Estimated session cost (or `n/a` for unknown/zero-priced models) |
| 🗜️ N | **Context compression count** — how many times the running session has been auto-compressed. Appears once the first compression fires. |
| ▶ N | **Active background tasks** — how many `/bg` prompts are still running in the current session. Appears whenever at least one task is in flight. |
| Duration | Elapsed session time |
| Session title | Once the session has a title, it appears as a gold badge pinned to the far-right edge. Long titles truncate before displacing the essential model and context fields. |
| ⚠ YOLO | **YOLO mode warning** — shown whenever `HERMES_YOLO_MODE` is on (either `hermes --yolo` at launch or `/yolo` toggled mid-session). Mirrors the banner-line warning so you can't forget you're in auto-approve mode. |
The bar adapts to terminal width — full layout at ≥ 76 columns, compact at 52–75, minimal (model + duration, plus the YOLO badge when active) below 52.
**Context color coding:**
| Color | Threshold | Meaning |
|-------|-----------|---------|
| Green | < 50% | Plenty of room |
| Yellow | 50–80% | Getting full |
| Orange | 80–95% | Approaching limit |
| Red | ≥ 95% | Near overflow — consider `/compress` |
Use `/usage` for a detailed breakdown including per-category costs (input vs output tokens).
On the `openai-codex` provider, `/usage` also shows any banked usage-limit resets on your ChatGPT account ("You have N resets banked - use /usage reset to activate"). `/usage reset` redeems one banked reset, fully restoring your 5-hour and weekly limits. Hermes refuses to redeem while your limits aren't exhausted (a banked reset restores the full allowance, so spending it early wastes it) — pass `/usage reset --force` to redeem anyway.
### Session Resume Display
When resuming a previous session (`hermes -c` or `hermes --resume <id>`), a "Previous Conversation" panel appears between the banner and the input prompt, showing a compact recap of the conversation history. See [Sessions — Conversation Recap on Resume](sessions.md#conversation-recap-on-resume) for details and configuration.
## Keybindings
| Key | Action |
|-----|--------|
| `Enter` | Send message |
| `Alt+Enter`, `Ctrl+J`, or `Shift+Enter` | New line (multi-line input). `Shift+Enter` requires a terminal that distinguishes it from `Enter` — see below. On Windows Terminal, `Alt+Enter` is captured by the terminal (fullscreen toggle); use `Ctrl+Enter` or `Ctrl+J` instead. |
| `Alt+V` | Paste an image from the clipboard when supported by the terminal |
| `Ctrl+V` | Paste text and opportunistically attach clipboard images |
| `Ctrl+B` | Start/stop voice recording when voice mode is enabled (`voice.record_key`, default: `ctrl+b`) |
| `Ctrl+G` | Open the current input buffer in `$EDITOR` (vim/nvim/nano/VS Code/etc.). Save and quit to send the edited text as the next prompt — ideal for long, multi-paragraph prompts. |
| `Ctrl+X Ctrl+E` | Emacs-style alternate binding for the external editor (same behavior as `Ctrl+G`). |
| `Ctrl+S` | **Stash the prompt.** Parks the current draft and clears the composer so you can send something else first. Press `Ctrl+S` again on an empty composer to bring the draft back (cursor at the end, attached images restored). Repeated presses build a stack rather than overwriting, so an earlier draft is never silently lost — with two or more stashed, `Ctrl+S` opens a browse panel (`↑`/`↓` to navigate, `Enter` to restore, `D` to discard, `Esc` or `Ctrl+S` to close). A `📌 N` badge in the status bar shows how many drafts are parked. Multi-line drafts round-trip exactly, including blank lines. The stash lives in memory for the session only — nothing is written to disk, since drafts often contain secrets. |
| `Ctrl+C` | Interrupt agent (double-press within 2s to force exit) |
| `Ctrl+D` | Exit |
| `Ctrl+Z` | Suspend Hermes to background (Unix only). Run `fg` in the shell to resume. |
| `Tab` | Accept auto-suggestion (ghost text) or autocomplete slash commands |
| `!<command>` | **Shell mode** — run a shell command yourself without spending a model turn (e.g. `!git status`, `!pytest -x`). See below. |
**Multiline paste preview.** When you paste a multi-line block, the CLI echoes a compact single-line preview (`[pasted: 47 lines, 1,842 chars — press Enter to send]`) instead of dumping the whole payload into the scrollback. The full content is still what gets sent; this is just display polish.
### `!` Shell Mode
Start a line with `!` to run it as a shell command instead of sending it to the agent:
```
> !git status
> !ls -la
> !pytest -x tests/cli
```
- **Zero cost.** The model is never invoked — no API call, no tokens, no latency.
- **Nothing enters the conversation.** The command and its output are not added to history, so your context stays clean and the prompt cache is untouched.
- **Runs where the agent's `terminal` tool runs.** Uses the session working directory, so `!pwd` matches what the agent would see.
- **Approvals still apply.** A dangerous command (`rm -rf`, writes to `~/.hermes/config.yaml`, etc.) goes through the same approval prompt the agent's `terminal` tool uses. `!` is a cost/latency shortcut, not a security bypass.
- **Non-zero exits are shown.** A failing command prints `! exited <code>` after its output.
- `!` on its own prints a one-line usage reminder.
Shell mode is CLI-only. Gateway platforms (Discord, Telegram, Slack) and cron runs ignore it — those users already have their own shells.
**Markdown stripping in final responses.** The CLI strips the most verbose markdown fences and `**bold**` / `*italic*` wrappers from *final* agent replies so they render as readable terminal prose rather than raw source. Code blocks and lists are preserved. This does not affect gateway platforms or tool results — they keep their markdown for native rendering.
## Slash Commands
Type `/` to see the autocomplete dropdown. Hermes supports a large set of CLI slash commands, dynamic skill commands, and user-defined quick commands.
Common examples:
| Command | Description |
|---------|-------------|
| `/help` | Show command help |
| `/model` | Show or change the current model |
| `/tools` | List currently available tools |
| `/skills browse` | Browse the skills hub and official optional skills |
| `/bg <prompt>` | Run a prompt in a separate background session |
| `/btw <question>` | Ask a side question about the current conversation without interrupting it |
| `/skin` | Show or switch the active CLI skin |
| `/voice on` | Enable CLI voice mode (press `Ctrl+B` to record) |
| `/voice tts` | Toggle spoken playback for Hermes replies |
| `/reasoning high` | Increase reasoning effort |
| `/title My Session` | Name the current session |
| `/status` | Show session info — model/profile/tokens/duration — followed by a local **Session recap** block (recent turn counts, top tools used, files touched, latest user prompt + assistant reply). Pure local compute; no LLM call. |
| `/context [all]` | Visual context-usage breakdown — glyph block grid + per-category token table (system prompt / tools / skills / memory / conversation / free space). `/context all` adds per-skill and per-toolset costs. |
| `/sessions` | Open an interactive session picker right inside the classic CLI (same surface the TUI uses). Type to filter, arrow keys to navigate, Enter to resume. |
For the full built-in CLI and messaging lists, see [Slash Commands Reference](../reference/slash-commands.md).
For setup, providers, silence tuning, and messaging/Discord voice usage, see [Voice Mode](features/voice-mode.md).
:::tip
Commands are case-insensitive — `/HELP` works the same as `/help`. Installed skills also become slash commands automatically.
:::
## Quick Commands
You can define custom commands that run shell commands instantly without invoking the LLM. These work in both the CLI and messaging platforms (Telegram, Discord, etc.).
```yaml
# ~/.hermes/config.yaml
quick_commands:
status:
type: exec
command: systemctl status hermes-agent
gpu:
type: exec
command: nvidia-smi --query-gpu=utilization.gpu,memory.used --format=csv,noheader
restart:
type: alias
target: /gateway restart
```
Then type `/status`, `/gpu`, or `/restart` in any chat. See the [Configuration guide](/user-guide/configuration#quick-commands) for more examples.
## Preloading Skills at Launch
If you already know which skills you want active for the session, pass them at launch time:
```bash
hermes -s hermes-agent-dev,github-auth
hermes chat -s github-pr-workflow -s github-auth
```
Hermes loads each named skill into the session prompt before the first turn. The same flag works in interactive mode and single-query mode.
## Skill Slash Commands
Every installed skill in `~/.hermes/skills/` is automatically registered as a slash command. The skill name becomes the command:
```
/gif-search funny cats
/axolotl help me fine-tune Llama 3 on my dataset
/github-pr-workflow create a PR for the auth refactor
# Just the skill name loads it and lets the agent ask what you need:
/excalidraw
```
## Personalities
Set a predefined personality to change the agent's tone:
```
/personality pirate
/personality kawaii
/personality concise
```
Built-in personalities include: `helpful`, `concise`, `technical`, `creative`, `teacher`, `kawaii`, `catgirl`, `pirate`, `shakespeare`, `surfer`, `noir`, `uwu`, `philosopher`, `hype`.
To go back to the default (no overlay), use `/personality none` — `default` and `neutral` work too.
You can also define custom personalities in `~/.hermes/config.yaml`:
```yaml
personalities:
helpful: "You are a helpful, friendly AI assistant."
kawaii: "You are a kawaii assistant! Use cute expressions..."
pirate: "Arrr! Ye be talkin' to Captain Hermes..."
# Add your own!
```
## Multi-line Input
There are two ways to enter multi-line messages:
1. **`Alt+Enter`, `Ctrl+J`, or `Shift+Enter`** — inserts a new line
2. **Backslash continuation** — end a line with `\` to continue:
```
❯ Write a function that:\
1. Takes a list of numbers\
2. Returns the sum
```
`Ctrl+J` and backslash continuation are enabled by default, matching Claude Code / Codex / OpenCode multiline shortcuts. On supported terminals such as iTerm2, Hermes also requests extended key reporting so `Shift+Enter` arrives as a distinct newline key. If your terminal sends LF for plain `Enter` and you need the legacy `Ctrl+J`-as-submit fallback, opt out:
```yaml
# ~/.hermes/config.yaml
display:
cli_multiline_shortcuts: false
```
:::info
Pasting multi-line text is supported — use any of the newline keys above, or simply paste content directly.
:::
### Shift+Enter compatibility
Most terminals send the same byte sequence for `Enter` and `Shift+Enter` by default, so applications cannot distinguish them. Hermes recognises `Shift+Enter` only when the terminal sends a distinct sequence via the [Kitty keyboard protocol](https://sw.kovidgoyal.net/kitty/keyboard-protocol/) or xterm's `modifyOtherKeys` mode.
| Terminal | Status |
|---|---|
| Kitty, foot, WezTerm, Ghostty | Distinct `Shift+Enter` enabled by default |
| iTerm2 (recent), Alacritty, VS Code terminal, Warp | Supported once the Kitty protocol is enabled in settings |
| Windows Terminal Preview 1.25+ | Supported once the Kitty protocol is enabled in settings |
| macOS Terminal.app, stock Windows Terminal (stable) | Not supported — `Shift+Enter` is indistinguishable from `Enter` |
Where the terminal cannot distinguish them, `Alt+Enter` and `Ctrl+J` continue to work by default. **On Windows Terminal specifically, `Alt+Enter` is captured by the terminal (toggles fullscreen) and never reaches Hermes — use `Ctrl+Enter` (delivered as `Ctrl+J`) or `Ctrl+J` directly for a newline.**
## Redirecting the Agent Mid-Turn
While the agent is working, you can send a correction without starting a new turn:
- **Type a new message + Enter** — redirects the active turn using your correction
- **`Ctrl+C`** — interrupt the current operation (press twice within 2s to force exit)
- Completed tool work and reasoning already shown stay in context
- A running tool reaches its safe boundary before the correction is applied
### Busy Input Mode
The `display.busy_input_mode` config key controls what happens when you press Enter while the agent is working:
| Mode | Behavior |
|------|----------|
| `"interrupt"` (default) | Your message redirects the active turn. Model generation restarts with displayed reasoning and completed work preserved; running tools finish first |
| `"queue"` | Your message is silently queued and sent as the next turn after the agent finishes |
| `"steer"` | Your message is injected into the current run via `/steer`, arriving at the agent after the next tool call — no interrupt, no new turn |
```yaml
# ~/.hermes/config.yaml
display:
busy_input_mode: "steer" # or "queue" or "interrupt" (default)
```
`"queue"` mode prepares a separate follow-up turn. `"steer"` always waits for the next tool-result boundary. The default `"interrupt"` mode responds sooner during model generation while avoiding cancellation of a running tool. Use `/stop` when you want to cancel the turn and its foreground work. Unknown values fall back to `"interrupt"`.
`"steer"` has two automatic fallbacks: if the agent hasn't started yet, or if images are attached, the message falls back to `"queue"` behavior so nothing is lost.
You can also change it inside the CLI:
```text
/busy queue
/busy steer
/busy interrupt
/busy status
```
:::tip First-touch hint
The first time you press Enter while Hermes is working, Hermes prints a one-line reminder explaining the `/busy` knob. It only fires once per install; `onboarding.seen.busy_input_prompt` in `config.yaml` records that it was shown. Delete that key to see the tip again.
:::
### Suspending to Background
On Unix systems, press **`Ctrl+Z`** to suspend Hermes to the background — just like any terminal process. The shell prints a confirmation:
```
Hermes Agent has been suspended. Run `fg` to bring Hermes Agent back.
```
Type `fg` in your shell to resume the session exactly where you left off. This is not supported on Windows.
## Tool Progress Display
The CLI shows animated feedback as the agent works:
**Thinking animation** (during API calls):
```
◜ (｡•́︿•̀｡) pondering... (1.2s)
◠ (⊙_⊙) contemplating... (2.4s)
✧٩(ˊᗜˋ*)و✧ got it! (3.1s)
```
**Tool execution feed:**
```
┊ 💻 terminal `ls -la` (0.3s)
┊ 🔍 web_search (1.2s)
┊ 📄 web_extract (2.1s)
```
Cycle through display modes with `/verbose`: `off → new → all → verbose`. This command can also be enabled for messaging platforms — see [configuration](/user-guide/configuration#display-settings).
### Tool Preview Length
The `display.tool_preview_length` config key controls the maximum number of characters shown in tool call preview lines (e.g. file paths, terminal commands). The default is `0`, which means no limit — full paths and commands are shown.
```yaml
# ~/.hermes/config.yaml
display:
tool_preview_length: 80 # Truncate tool previews to 80 chars (0 = no limit)
```
This is useful on narrow terminals or when tool arguments contain very long file paths.
## Session Management
### Resuming Sessions
When you exit a CLI session, a resume command is printed:
```
Resume this session with:
hermes --resume 20260225_143052_a1b2c3
Session: 20260225_143052_a1b2c3
Duration: 12m 34s
Messages: 28 (5 user, 18 tool calls)
```
Resume options:
```bash
hermes --continue # Resume the most recent CLI session
hermes -c # Short form
hermes -c "my project" # Resume a named session (latest in lineage)
hermes --resume 20260225_143052_a1b2c3 # Resume a specific session by ID
hermes --resume "refactoring auth" # Resume by title
hermes --resume latest # Resume the most recent session (same as -c)
hermes --resume latest --in ./my-project # Latest session for ./my-project's workspace
hermes -r 20260225_143052_a1b2c3 # Short form
```
Resuming restores the full conversation history from SQLite. The agent sees all previous messages, tool calls, and responses — just as if you never left.
Use `/title My Session Name` inside a chat to name the current session, or `hermes sessions rename <id> <title>` from the command line. Use `hermes sessions list` to browse past sessions.
### Session Storage
CLI sessions are stored in Hermes's SQLite state database under `~/.hermes/state.db`. The database keeps:
- session metadata (ID, title, timestamps, token counters)
- message history
- lineage across compressed/resumed sessions
- full-text search indexes used by `session_search`
Some messaging adapters also keep per-platform transcript files alongside the database, but the CLI itself resumes from the SQLite session store.
### Context Compression
Long conversations are automatically summarized when approaching context limits:
```yaml
# In ~/.hermes/config.yaml
compression:
enabled: true
threshold: 0.50 # Compress at 50% of context limit by default
# Summarization model configured under auxiliary:
auxiliary:
compression:
model: "" # Leave empty to use the main chat model (default). Or pin a cheap fast model, e.g. "google/gemini-3-flash-preview".
```
When compression triggers, middle turns are summarized while the first 3 and last 20 turns are always preserved.
## Background Sessions
Run a prompt in a separate background session while continuing to use the CLI for other work:
```
/bg Analyze the logs in /var/log and summarize any errors from today
```
Hermes immediately confirms the task and gives you back the prompt:
```
🔄 Background task #1 started: "Analyze the logs in /var/log and summarize..."
Task ID: bg_143022_a1b2c3
```
### How It Works
Each `/bg` prompt spawns a **completely separate agent session** in a daemon thread:
- **Isolated conversation** — the background agent has no knowledge of your current session's history. It receives only the prompt you provide.
- **Same configuration** — the background agent inherits your model, provider, toolsets, reasoning settings, and fallback model from the current session.
- **Non-blocking** — your foreground session stays fully interactive. You can chat, run commands, or even start more background tasks.
- **Multiple tasks** — you can run several background tasks simultaneously. Each gets a numbered ID.
### Results
When a background task finishes, the result appears as a panel in your terminal:
```
╭─ ⚕ Hermes (background #1) ──────────────────────────────────╮
│ Found 3 errors in syslog from today: │
│ 1. OOM killer invoked at 03:22 — killed process nginx │
│ 2. Disk I/O error on /dev/sda1 at 07:15 │
│ 3. Failed SSH login attempts from 192.168.1.50 at 14:30 │
╰──────────────────────────────────────────────────────────────╯
```
If the task fails, you'll see an error notification instead. If `display.bell_on_complete` is enabled in your config, the terminal bell rings when the task finishes.
### Use Cases
- **Long-running research** — "/bg research the latest developments in quantum error correction" while you work on code
- **File processing** — "/bg analyze all Python files in this repo and list any security issues" while you continue a conversation
- **Parallel investigations** — start multiple background tasks to explore different angles simultaneously
:::info
Background sessions do not appear in your main conversation history. They are standalone sessions with their own task ID (e.g., `bg_143022_a1b2c3`).
:::
## Quiet Mode
By default, the CLI runs in quiet mode which:
- Suppresses verbose logging from tools
- Enables kawaii-style animated feedback
- Keeps output clean and user-friendly
For debug output:
```bash
hermes chat --verbose
```
website/docs/user-guide/configuration.md
+2786
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/configuring-models.md
+388
View File
@@ -0,0 +1,388 @@
---
sidebar_position: 3
---
# Configuring Models
Hermes uses two kinds of model slots:
- **Main model** — what the agent thinks with. Every user message, every tool-call loop, every streamed response goes through this model.
- **Auxiliary models** — smaller side-jobs the agent offloads. Context compression, vision (image analysis), web-page summarization, approval scoring, MCP tool routing, session-title generation, and skill search. Each has its own slot and can be overridden independently.
This page covers configuring both from the dashboard. If you prefer config files or the CLI, jump to [Alternative methods](#alternative-methods) at the bottom. To run models on your own machine instead of a cloud provider, see [Local Models](/user-guide/local-models).
:::tip Fastest path: Nous Portal
[Nous Portal](/user-guide/features/tool-gateway) provides 300+ models under one subscription. On a fresh install, run `hermes setup --portal` to log in and set Nous as your provider in one command. Inspect what's wired up with `hermes portal info`.
- Portal subscribers also get **10% off token-billed providers**.
:::
:::note `model:` schema — empty string vs. mapping
On a brand-new install the bundled default config has `model: ""` (an empty string sentinel meaning "not configured yet"). The first time you run `hermes setup` or `hermes model`, that key is upgraded in-place to a mapping with `provider`, `default`, `base_url`, and `api_mode` sub-keys — the shape shown throughout this page and in [`profiles.md`](./profiles.md) / [`configuration.md`](./configuration.md). If you ever see an empty string in `config.yaml`, run `hermes model` (or click **Change** in the dashboard) and Hermes will write the dict form for you.
:::
## The Models page
Open the dashboard and click **Models** in the sidebar. You get two sections:
1. **Model Settings** — the top panel, where you assign models to slots.
2. **Usage analytics** — ranked cards showing every model that ran a session in the selected period, with token counts, cost, and capability badges.
![Models page overview](/img/docs/dashboard-models/overview.png)
The top card is the **Model Settings** panel. The main row always shows what the agent will spin up for new sessions. Click **Change** to open the picker.
## Setting the main model
Click **Change** on the Main model row:
![Model picker dialog](/img/docs/dashboard-models/picker-dialog.png)
The picker has two columns:
- **Left** — authenticated providers. Only providers you've set up (API key set, OAuth'd, or defined as a custom endpoint) show up here. If a provider is missing, head to **Keys** and add its credential.
- **Right** — the curated model list for the selected provider. These are the agentic models Hermes recommends for that provider, not the raw `/models` dump (which on OpenRouter includes 400+ models including TTS, image generators, and rerankers).
Type in the filter box to narrow by provider name, slug, or model ID.
Pick a model, hit **Switch**, and Hermes writes it to `~/.hermes/config.yaml` under the `model` section. **This applies to new sessions only** — any chat tab you already have open keeps running whatever model it started with. To hot-swap the current chat, use the `/model` slash command inside it.
### Mid-session switches and context warnings
When you switch models **inside an active session** (Herm TUI model picker, `hermes` CLI, or `/model` on Telegram/Discord), Hermes estimates whether your **next message** will run **preflight context compression** against the new model's window. If the session is already near or above that model's compression threshold (see [Context Compression](./configuration.md#context-compression)), the switch reply includes a warning — the same `warning_message` path used for expensive-model notices. The switch still applies immediately; compression runs on the **first user message after the switch**, before the model answers.
:::warning Mid-session switches reset the prompt cache
Prompt caches are keyed to the model serving the request, so any mid-conversation model change — an explicit `/model` switch, an [automatic fallback](./features/fallback-providers.md), or a [credential-pool](./features/credential-pools.md) rotation onto a different account — means the next message re-reads the entire conversation at full input-token price instead of the cached (~75–90% discounted) rate. On a long session this one-time re-read can dwarf the per-token difference between the two models. Switch when you need to, but prefer doing it early in a conversation or right after starting a fresh session.
:::
### Unattended data-training tiers
Models with a `-contributor` suffix (e.g. `muse-spark-1.2-contributor`, `muse-spark-1.3-contributor`) are discounted because the vendor may train on your prompts and completions. Interactive model selection always shows a confirmation prompt. Non-interactive startup paths such as Kanban workers and cron agents fail closed because they cannot ask that question.
If training on the unattended workload's data is acceptable, record a persistent acknowledgement:
```bash
hermes config set security.allow_data_training_tiers_noninteractive true
```
Hermes still prints the full data-policy warning and the acknowledgement key on every unattended startup, so worker logs retain an audit trail. This setting does not approve expensive-model or provider-routing warnings, and it does not replace the interactive confirmation prompt. Revoke it with `hermes config unset security.allow_data_training_tiers_noninteractive`.
## Setting auxiliary models
Click **Show auxiliary** to reveal the 11 task slots:
![Auxiliary panel expanded](/img/docs/dashboard-models/auxiliary-expanded.png)
Every auxiliary task defaults to `auto` — meaning Hermes tries your main model for that job too. If that route is unavailable or hits a capacity-style failure, `auto` follows any task-specific `auxiliary.<task>.fallback_chain`, then the main `fallback_providers` / `fallback_model` chain, then Hermes' built-in auxiliary discovery chain. Override a specific task when you want a cheaper or faster model for a side-job.
### Common override patterns
| Task | When to override |
|---|---|
| **Title Gen** | When title latency or cost matters more than matching the main model. Pin a known-good flash model, or set `auxiliary.title_generation.prefer_fast_model: true` to let Hermes choose the provider's fast tier. |
| **Vision** | When your main model lacks vision support. Point it at `google/gemini-2.5-flash` or `gpt-4o-mini`. |
| **Compression** | When you're burning reasoning tokens on Opus/M2.7 just to summarize context. A fast chat model does the job at 1/50th the cost. |
| **Approval** | For `approval_mode: smart` — a fast/cheap model (haiku, flash, gpt-5-mini) decides whether to auto-approve low-risk commands. Expensive models here are waste. |
| **Web Extract** | When you use `web_extract` heavily. Same logic as compression — summarization doesn't need reasoning. |
| **Skills Hub** | `hermes skills search` uses this. Usually fine at `auto`. |
| **MCP** | MCP tool routing. Usually fine at `auto`. |
| **Triage Specifier** | Routes the Kanban triage specifier (`hermes kanban specify`) that expands a rough one-liner into a concrete spec. A cheap, capable model works well. |
| **Kanban Decomposer** | Routes Kanban task decomposition — splits a triage task into a graph of child tasks for specialist profiles. |
| **Profile Describer** | Routes profile-description generation (`hermes profile describe --auto` / the dashboard auto-generate button). Short, cheap call. |
| **Curator** | Routes the curator skill-usage review pass. Can run for minutes on reasoning models, so a cheaper aux model is often worthwhile. |
### Per-task override
Click **Change** on any auxiliary row. Same picker opens, same behavior — pick provider + model, hit Switch. The row updates to show `provider · model` instead of `auto (use main model)`.
### Reset all to auto
If you've over-tuned and want to start over, click **Reset all to auto** at the top of the auxiliary section. Every slot goes back to using your main model.
## The "Use as" shortcut
Every model card on the page has a **Use as** dropdown. This is the fast path — pick a model you see in your analytics, click **Use as**, and assign it to the main slot or any specific auxiliary task in one click:
![Use as dropdown](/img/docs/dashboard-models/use-as-dropdown.png)
The dropdown has:
- **Main model** — same as clicking Change on the main row.
- **All auxiliary tasks** — assigns this model to all 11 aux slots at once. Useful when you just want every side-job on a cheap flash model.
- **Individual task options** — Vision, Web Extract, Compression, etc. The currently-assigned model for each task is marked `current`.
Cards are badged with `main` or `aux · <task>` when they're currently assigned to something — so you can see at a glance which of your historical models are wired in where.
## What gets written to `config.yaml`
When you save via the dashboard, Hermes writes to `~/.hermes/config.yaml`:
**Main model:**
```yaml
model:
provider: openrouter
default: anthropic/claude-opus-4.7
base_url: '' # cleared on provider switch
api_mode: chat_completions
```
**Auxiliary override (example — vision on gemini-flash):**
```yaml
auxiliary:
vision:
provider: openrouter
model: google/gemini-2.5-flash
base_url: ''
api_key: ''
timeout: 120
extra_body: {}
download_timeout: 30
```
**Auxiliary on auto (default):**
```yaml
auxiliary:
compression:
provider: auto
model: ''
base_url: ''
# ... other fields unchanged
```
`provider: auto` with `model: ''` tells Hermes to use the main model for that task, while still honoring fallback policy if the main route cannot serve the auxiliary call.
Optional task-specific fallback chains live under the same auxiliary task:
```yaml
auxiliary:
title_generation:
provider: auto
model: ''
fallback_chain:
- provider: openrouter
model: inclusionai/ring-2.6-1t:free
```
When `fallback_chain` is absent, `auto` uses the top-level `fallback_providers` chain before the built-in auxiliary discovery chain.
## Per-provider request options
Provider entries (`providers.<name>` in the `providers:` dict, or items in the legacy `custom_providers` list) accept knobs that shape how Hermes talks to the endpoint:
**`extra_headers`** — a mapping of extra HTTP headers attached to every LLM request routed to that provider's base URL. They are applied last, after URL/profile defaults and user header overrides, so they survive credential swaps and client rebuilds. Useful for Cloudflare Access service tokens, proxy auth, or custom bearer schemes:
```yaml
providers:
my-gateway:
api: https://llm.internal.example.com/v1
api_key: sk-...
extra_headers:
CF-Access-Client-Id: "xxxx.access"
CF-Access-Client-Secret: "yyyy"
```
Header values routinely carry credentials — Hermes never logs them. `extra_headers` applies to OpenAI-compatible routes; the `anthropic_messages` and `bedrock_converse` API modes do not use it.
**`discover_models`** — set to `false` (default `true`) to skip querying the endpoint's `/models` listing and use only the `models` you configured on the entry. Handy for gateways whose model listing is slow, unreliable, or noisy:
```yaml
providers:
my-gateway:
api: https://llm.internal.example.com/v1
discover_models: false
models:
- my-finetune-v2
- my-finetune-v1
```
With discovery off, the model picker (`hermes model`, `/model`) shows the configured list instead of a live probe.
**`openai_native_compaction`** — set this capability to `true` only for an OpenAI-compatible endpoint that you trust with conversation content. Native compaction sends its payload to that provider's configured `base_url`:
```yaml
providers:
trusted-proxy:
api: https://llm.internal.example.com/v1
capabilities:
openai_native_compaction: true
```
For a gateway that resolves a bare model alias only after receiving the
request, opt the alias into prompt-cache markers with the per-model
`prompt_caching` capability:
```yaml
providers:
model-proxy:
api: https://gateway.example.com/v1
transport: openai_chat # or anthropic_messages
models:
fable:
context_length: 1000000
prompt_caching: true
```
Hermes matches this declaration to the exact provider route and runtime model
id, without rewriting the alias or inferring support from its provider name,
host, or model family. The marker layout follows the configured transport:
`openai_chat` uses the OpenAI-compatible envelope layout and
`anthropic_messages` uses the native inner-block layout. Set
`prompt_caching: false` to explicitly disable cache markers for a model; when
omitted, Hermes keeps its normal provider and model capability detection.
:::note Legacy format
Older configs used a top-level `custom_providers:` list (with `base_url` instead of `api`). It still works and is auto-migrated to the `providers:` dict on `hermes update` (config v12).
:::
## When does it take effect?
- **CLI** (`hermes chat`): next `hermes chat` invocation.
- **Gateway** (Telegram, Discord, Slack, etc.): next *new* session. Existing sessions keep their model. Restart the gateway (`hermes gateway restart`) if you want to force all sessions to pick up the change.
- **Dashboard chat tab** (`/chat`): next new PTY. The currently-open chat keeps its model — use `/model` inside it to hot-swap.
Changes never invalidate prompt caches on running sessions. That's deliberate: swapping the main model inside a session requires a cache reset (the system prompt contains model-specific content), and we reserve that for the explicit `/model` slash command inside chat.
## Troubleshooting
### "No authenticated providers" in the picker
Hermes lists a provider only if it has a working credential. Check **Keys** in the sidebar — you should see one of: an API key, a successful OAuth, or a custom endpoint URL. If the provider you want isn't there, run `hermes setup` to wire it up, or go to **Keys** and add the env var.
### Main model didn't change in my running chat
Expected. The dashboard writes `config.yaml`, which new sessions read. The currently-open chat is a live agent process — it keeps whatever model it was spawned with. Use `/model <name>` inside the chat to hot-swap that specific session.
### Auxiliary override "didn't take effect"
Three things to check:
1. **Did you start a new session?** Existing chats don't re-read config.
2. **Is `provider` set to something other than `auto`?** If the field shows `auto`, the task is still using your main model. Click **Change** and pick a real provider.
3. **Is the provider authenticated?** If you assigned `minimax` to a task but don't have a MiniMax API key, that task falls back to the openrouter default and logs a warning in `agent.log`.
### I picked a model but Hermes switched providers on me
On OpenRouter (or any aggregator), bare model names resolve *within* the aggregator first. So `claude-sonnet-4` on OpenRouter becomes `anthropic/claude-sonnet-4.6`, staying on your OpenRouter auth. But if you typed `claude-sonnet-4` on a native Anthropic auth, it would stay as `claude-sonnet-4-6`. If you see an unexpected provider switch, check that your current provider is what you expect — the picker always shows the current main at the top of the dialog.
## Alternative methods
### CLI slash command
Inside any `hermes chat` session:
```
/model gpt-5.4 --provider openrouter # session-only
/model gpt-5.4 --provider openrouter --global # also persists to config.yaml
/model claude-opus-4.6 --once # next turn only, then auto-restores
```
`--global` does the same thing the dashboard's **Change** button does, plus it switches the running session in-place.
`--once` switches for a single turn and restores the previous model afterward — on success, error, or interrupt alike. Nothing is persisted: a gateway restart mid-turn comes back on the original model. Useful for escalating one hard question to an expensive model ("ask Opus just this once") or dropping to a cheap model for a throwaway query.
:::note Prompt-cache cost
A one-turn switch breaks the provider's prompt-cache prefix twice (switching out and back). In a long session on a cached-prefix provider (Anthropic, OpenAI), the next turn re-pays full input cost — `--once` wins for short sessions or cheap→expensive escalation, but a quick side question inside a long expensive session can cost more than it saves.
:::
### Custom aliases
Define your own short names for models you reach for often, then use `/model <alias>` in a running session or `hermes chat --model <alias>` at startup. There are two equivalent formats — pick whichever fits your workflow.
**Canonical (top-level `model_aliases:`)** — full control over provider + base_url:
```yaml
# ~/.hermes/config.yaml
model_aliases:
fav:
model: claude-sonnet-4.6
provider: anthropic
grok:
model: grok-4
provider: x-ai
```
An alias that points at its own endpoint can also carry that endpoint's
credential, with either `api_key` (a literal, or a `"${VAR}"` reference) or
`key_env` (the name of an environment variable). If both are set, `api_key`
wins:
```yaml
model_aliases:
theta:
model: theta-1
provider: custom
base_url: "https://theta.example.com/v1"
key_env: THETA_API_KEY # or: api_key: "${THETA_API_KEY}"
```
When an alias sets neither, the key is resolved from the alias **host** —
`OLLAMA_API_KEY` for an `ollama.com` endpoint, `DEEPSEEK_API_KEY` for
`api.deepseek.com`, and so on. It is never inherited from whichever provider
happened to be active before the switch, so switching to an alias cannot send
one provider's secret to another provider's host.
**Short string form (`model.aliases.<name>: provider/model`)** — convenient from the shell because `hermes config set` writes scalars and now also parses inline list/mapping literals, though this short alias form still can't carry a custom `base_url`:
```bash
hermes config set model.aliases.fav anthropic/claude-opus-4.6
hermes config set model.aliases.grok x-ai/grok-4
```
> `hermes config set` also accepts inline **list/mapping literals** (JSON/YAML flow style). Quote them so your shell passes them through intact:
>
> ```bash
> hermes config set platform_toolsets.line '["clarify", "file", "web"]'
> hermes config set display.tool_progress_overrides '{"terminal": "off"}'
> ```
Both paths feed the same loader (`hermes_cli/model_switch.py`). Entries declared in `model_aliases:` take precedence over `model.aliases:` entries with the same name.
Then `/model fav` or `/model grok` in chat. User aliases shadow built-in short names (`sonnet`, `kimi`, `opus`, etc.). See [Custom model aliases](/reference/slash-commands#custom-model-aliases) for the full reference.
### `hermes model` subcommand
```bash
hermes model # Interactive provider + model picker (the canonical way to switch defaults)
```
`hermes model` walks you through picking a provider, authenticating (OAuth flows open a browser; API-key providers prompt for the key), and then choosing a specific model from that provider's curated catalog. The choice is written to `model.provider` and `model.default` in `~/.hermes/config.yaml`.
To list providers/models without launching the picker, use the dashboard or the REST endpoints below. To inspect what the CLI will actually use right now: `hermes config get model --json` and `hermes status`.
### Direct config edit
Edit `~/.hermes/config.yaml` and restart whatever reads it. See the [Configuration reference](./configuration.md) for the full schema.
### REST API
The dashboard uses three endpoints. Useful for scripting:
```bash
# List authenticated providers + curated model lists
curl -H "X-Hermes-Session-Token: $TOKEN" http://localhost:PORT/api/model/options
# Read current main + auxiliary assignments
curl -H "X-Hermes-Session-Token: $TOKEN" http://localhost:PORT/api/model/auxiliary
# Set the main model
curl -X POST -H "Content-Type: application/json" -H "X-Hermes-Session-Token: $TOKEN" \
-d '{"scope":"main","provider":"openrouter","model":"anthropic/claude-opus-4.7"}' \
http://localhost:PORT/api/model/set
# Override a single auxiliary task
curl -X POST -H "Content-Type: application/json" -H "X-Hermes-Session-Token: $TOKEN" \
-d '{"scope":"auxiliary","task":"vision","provider":"openrouter","model":"google/gemini-2.5-flash"}' \
http://localhost:PORT/api/model/set
# Assign one model to every auxiliary task
curl -X POST -H "Content-Type: application/json" -H "X-Hermes-Session-Token: $TOKEN" \
-d '{"scope":"auxiliary","task":"","provider":"openrouter","model":"google/gemini-2.5-flash"}' \
http://localhost:PORT/api/model/set
# Reset all auxiliary tasks to auto
curl -X POST -H "Content-Type: application/json" -H "X-Hermes-Session-Token: $TOKEN" \
-d '{"scope":"auxiliary","task":"__reset__","provider":"","model":""}' \
http://localhost:PORT/api/model/set
```
The session token is injected into the dashboard HTML at startup and rotates on every server restart. Grab it from the browser devtools (`window.__HERMES_SESSION_TOKEN__`) if you're scripting against a running dashboard.
website/docs/user-guide/desktop.md
+618
View File
@@ -0,0 +1,618 @@
---
sidebar_position: 3
title: "Hermes Desktop"
description: "The native Hermes desktop app — a polished experience for chatting with Hermes, with streaming tool output, side-by-side previews, a file browser, voice, cron, profiles, skills, and settings. macOS, Windows, and Linux."
---
# Hermes Desktop
The Hermes desktop app is a native app built around the **same** agent you get from the CLI and the gateway — same config, same API keys, same sessions, same skills, same memory. It is not a separate product or a lightweight clone; it uses the same Hermes Agent core and settings, and drives it through a modern & thoughtfully designed UI. If you have used `hermes` in a terminal, everything you set up there is already here, and anything you do here shows up there.
It runs on **macOS, Windows, and Linux**.
:::tip Which interface is which?
Hermes has several front ends that all talk to the same agent:
- **Desktop App** (this page) — a native application with a purpose-built UI for chat, configuration, and management.
- **CLI** (`hermes`) and **[TUI](./tui.md)** (`hermes --tui`) — terminal interfaces.
- **[Web Dashboard](./features/web-dashboard.md)** (`hermes dashboard`) — a browser admin panel; its optional **Chat** tab embeds the TUI through a pseudo-terminal.
Pick whichever fits the moment. They share state, so you can start a session in one and resume it in another.
:::
## Install
Download the app from the [Hermes Desktop product page](https://hermes-agent.nousresearch.com/desktop), or follow the [installation instructions for Hermes Desktop](../getting-started/installation.md).
If you already have Hermes installed, simply run
```bash
hermes desktop
```
That uses your current config, keys, sessions, and skills.
## What's in the app
The desktop app is organized as a chat-first window with a left sidebar for navigation. It's built to allow managing multiple simultaneous agent conversations, configuring messaging providers, creating artifacts, browsing projects' folder structures, and working on multiple projects at once.
### Chat
The center of the app. You get:
- **Streaming responses** with live tool activity and structured tool-call summaries as the agent works.
- **The same conversation history** as every other Hermes surface — sessions started here resume in the CLI/TUI and vice versa.
- **Drag-and-drop files** anywhere in the chat area to attach them to your next message.
- **A right-hand preview rail** — render web pages, files, and tool outputs side by side while you keep chatting.
- **Comment mode in the in-app browser** — click **Annotate** in the preview browser bar, then click any element (or drag a box) on the live page and type a note; each saved comment stays as a numbered pin on the page. Saving a pin never sends a turn — when you're done, **Add N comments** attaches a cropped screenshot per pin and a short prompt naming each comment to the composer, and you still hit send yourself. Each element comment carries its CSS selector, its markup, and the computed styles that matter for layout, so the agent can find the element in your source instead of guessing from the picture. Password and hidden field values, and any attribute that looks like a key or token, are redacted on the page before the markup leaves it. Larger batches arrive grouped by which part of the page each comment sits in, so twenty-odd comments become a handful of pieces of work rather than one task each — and because the groups are separate DOM subtrees they usually touch separate files, which is what makes handing them to parallel workers safe. Pin numbers hold steady if you delete one, and switching chats clears the stack.
- **Composer history and queue editing** — press the up/down arrow keys in an empty composer to recall and reuse previous prompts, and edit messages you've queued up before they're sent. Pressing Stop (or Esc) while turns are queued pauses the queue and expands it above the composer; resume it from there, or send, edit, and delete individual entries.
- **A conversation timeline rail** — long chats get a slim rail of markers along the edge of the transcript, one per prompt. Hover it to pop open the list of prompts, click one to jump straight to that point in the conversation. (It appears once the chat has a handful of turns.)
- **Find in page** — press **Cmd/Ctrl+F** to open a find bar that searches the rendered chat transcript. Enter / Shift+Enter (or Cmd/Ctrl+G / Cmd/Ctrl+Shift+G while the bar is open) step through matches; Esc closes it.
#### Status bar
The bar along the bottom of the chat shows live session state and exposes quick controls without opening Settings:
- **Per-session YOLO toggle** — flip YOLO on or off for just this session (matching the TUI). YOLO bypasses the dangerous-command approval prompts, so know what you're turning off — see [Security → YOLO Mode](./security.md#yolo-mode).
- **Context-usage meter** — a live "% full" meter of the session's context window. Click it to open the **Context Usage** popover with a token breakdown by category (system prompt, tool definitions, skills, memory, rules, MCP, subagent definitions, and the conversation itself) so you can see exactly what's eating the window before compression kicks in.
- **Cache hit rate and tokens per second** — off by default; turn them on from the right-click menu. Cache hit rate is the share of this session's prompt tokens served from the provider's prompt cache (cached tokens cost less, so higher is cheaper — you can watch a session get cheaper as it warms up). Tokens per second is output throughput averaged over the last 10 model calls. Both update live during a turn.
- **Customizable items** — right-click the status bar (**Show in status bar**) to choose what appears: the context meter, cache hit rate, tokens per second, workspace, model, approvals, turn/session timers, terminal, Command Center, backend version, and more — or hide the bar entirely (**Cmd/Ctrl+Shift+S** toggles it).
Chatting against a Hermes instance on another machine instead of the bundled local backend? See [Connecting to a remote backend](#connecting-to-a-remote-backend) below — and for the full picture of how the remote-hosted dashboard connection works (the auth gate, the `/api/ws` chat socket, and WebSocket close-code triage), see [Web Dashboard → Connecting Hermes Desktop to a remote backend](./features/web-dashboard.md#connecting-hermes-desktop-to-a-remote-backend).
#### Repository discovery
Hermes Desktop discovers local Git repositories for the Projects sidebar by scanning your home directory to a bounded depth. You can change this per profile in **Settings → Workspace**, or in `config.yaml`:
```yaml
desktop:
repo_scan_enabled: true
repo_scan_roots: []
repo_scan_exclude_paths: []
```
- Set `repo_scan_enabled: false` to stop the filesystem scan completely. Existing disk-discovery cache rows for that profile are cleared; explicit projects and repositories inferred from intentional Hermes sessions remain available.
- Set `repo_scan_roots` to a list of folders to restrict scanning. An empty list preserves the default home-directory scan.
- Set `repo_scan_exclude_paths` to folders whose complete subtrees should be skipped.
Changing any of these values invalidates only that profile's disk-discovery cache and starts a policy-compliant refresh. **Hide from sidebar** remains a separate per-item curation action.
#### Choosing a model
The model picker lives in the **composer**, just left of the microphone. Click it to switch the model, reasoning effort, and fast mode from one dropdown.
- **The composer picker is sticky UI state and never touches your default.** It's remembered locally (per device) and **follows** across new chats and restarts instead of snapping back to the default — pick a model once and the next `Cmd/Ctrl+N` opens on it. With a live chat, switching models scopes the change to that **current chat**; either way the selection rides along when the session is created/switched and is **never** written to the profile default — with one exception: on a fresh profile that has no `model.default`/`model.provider` configured yet, the first pick is persisted so the app has a real default instead of falling through to a stray API-key env var on restart. Persistence follows the same rule as `/model` (`model.persist_switch_by_default`); use **Settings → Model** to change the default deliberately. (Switching [profiles](#sessions--profiles) reseeds to that profile's own default.)
- **Set the default in Settings → Model.** That "main" model is your **per-profile global default** — it's what new chats, crons, subagents, and auxiliary tasks start from, and it's the only place that writes it. Each [profile](#sessions--profiles) keeps its own default.
- **Per-model effort/fast presets.** Each model remembers its own reasoning effort and fast-mode choice in the desktop app, re-applied to the session whenever you pick that model. These presets are a desktop convenience and don't change crons or subagents.
- **Mid-chat switches reset the prompt cache.** Switching the model inside a live chat means the next message re-reads the whole conversation at full input price (provider prompt caches are keyed to the model). Fine occasionally; on a long chat, a fresh chat on the new model is often cheaper than bouncing back and forth.
### File browser
Explore and preview the working directory without leaving the app — useful for following along as the agent reads, writes, and edits files. Set the initial project directory with `hermes desktop --cwd <path>` (or the `HERMES_DESKTOP_CWD` environment variable).
### Artifacts
The **Artifacts** view collects what your sessions generate — **images, files, and links** — into one searchable, browsable gallery. Open it from the sidebar, the command palette (**Artifacts — Browse generated outputs**), or a `nav.artifacts` shortcut you bind yourself. It indexes recent session outputs automatically; every artifact shows which session produced it with a jump back to that chat, and images and files open in a preview with download / open-in-browser / copy actions.
### Windows, tabs & panes
The app is built for working on several things at once:
- **Tabs** — **Cmd/Ctrl+T** opens a new session tab; **Ctrl+Tab** / **Ctrl+Shift+Tab** cycle sessions, and **Ctrl+1…9** jump to a recent session by position. **Cmd/Ctrl+W** closes the focused tab and **Cmd/Ctrl+Shift+T** reopens the last closed one.
- **Multiple windows** — **Cmd/Ctrl+Shift+N** opens a new window, and any session can be popped out via its context menu (**New window**) or from the command palette. A popped-out window renders that single chat without the global sidebar — handy for parking a long-running session on another monitor. Live agent output streams into every window showing the session.
- **Panes** — **Cmd/Ctrl+B** toggles the left sidebar, **Cmd/Ctrl+J** the right one, and **Cmd/Ctrl+\\** swaps which side the sidebars sit on.
### Terminal
A real terminal lives in the right sidebar, next to the file browser:
- **Ctrl+`** shows the terminal (opening one if none exist); **Ctrl+Shift+`** spawns an additional one. Multiple terminals stack in a tab rail — **Ctrl+Shift+↓/↑** walk between them, **Ctrl+Shift+W** closes the active one.
- **Shells persist while hidden.** Closing or hiding the panel doesn't kill your shell — every open terminal stays mounted with its scrollback and running processes intact until you explicitly close it.
- **Add to chat** — select terminal output and send it into the composer as context for your next message.
### Git review & worktrees
For sessions running inside a Git repository, the app has a built-in source-control surface:
- **Review pane** — **Cmd/Ctrl+G** toggles the working-tree review pane: branch and ahead/behind status, changed files (list or tree view), and diffs scoped to **Uncommitted**, **Branch**, or **Last turn** (just what the agent changed in its most recent turn). Stage/unstage files, revert changes, write a commit message (or **Generate commit message**), then **Commit** or **Commit & Push** — and **Create PR** via the GitHub CLI (`gh`), or hand the whole thing to the agent with **Ask Hermes to open PR**. You can also create and switch branches from here.
- **Worktrees** — **Cmd/Ctrl+Shift+B** (or **New worktree** on a project in the sidebar) creates a Git worktree on a new branch so an agent can work on a parallel copy of the repo without touching your checkout. Worktrees show up as their own lanes under the project; removing one offers to delete the worktree directory (the branch stays) or just hide the lane and leave it on disk, with a force option when it has uncommitted changes.
### Memory Graph
The **Memory Graph** (command palette → *Memory Graph*, or the status-bar item) is an interactive map of what Hermes has learned for you — skills and memories laid out as a zoomable node graph with a timeline, filterable by **All / Used / Learned**. A share control exports the map layout as a compact code you can paste to someone else (layout only — none of your memory or skill text is included) and imports codes the same way.
### Quick Entry
Quick Entry is a small always-available composer summoned by a **global hotkey from anywhere on your system** — fire off a prompt without switching to (or even opening) the main window. Enable it in **Settings → Advanced → Quick Entry**; the default shortcut is **Ctrl/Cmd+Shift+Space** and you can set your own (it needs at least one modifier). If another app already owns the chord, the settings row tells you so you can pick a different one.
### Voice
Talk to Hermes and hear it back, the same [voice mode](./features/voice-mode.md) available elsewhere. On macOS the OS will prompt once for microphone access.
### HUD mode
**⌘/Ctrl+Shift+H** (or the titlebar button) detaches the chat into a chrome-free, always-on-top floating bar that sits over whatever you are working in. The app window steps aside; the HUD keeps your live conversation and a composer. Where you park it is context — the bar's position tells Hermes which app and screen you're asking about, so "this", "here", and "that page" resolve to what's underneath it.
- **Moving the bar** — on macOS and Windows, **press and hold** anywhere on the composer for a beat, then drag. On Linux/X11, hold **Ctrl** and drag with the primary mouse button for an immediate grab (including over selected text); press-and-hold remains available too. Keep the grab held while invoking your desktop switch shortcut to carry the HUD onto another virtual desktop. On native Wayland the composer bar is a compositor drag handle (the only way to move it, because an app cannot place its own window).
- **Resizing** — drag any edge or corner of the bar; the opposite edge stays anchored. Native Wayland exposes the right and bottom edges because the compositor does not allow apps to position top-level windows themselves.
- **Reset layout** — the discard control on the bar restores the default size and (on X11 / macOS / Windows) position. Use this if a persisted size leaves the HUD unusable.
- **Snap to pointer** — **⌘/Ctrl+Shift+G** (a global hotkey, works from any app) jumps the HUD to wherever your cursor is. On native Wayland this is a no-op — the compositor owns placement.
- **Exiting** — click the exit button on the bar, or press **⌘/Ctrl+Shift+H** again. The app window comes back with your session intact.
#### Linux / Wayland
Electron 20+ already runs as a native Wayland client on a Wayland session. Drag, click-through, and resize work on that path.
On **Hyprland** (including Omarchy) the HUD is floated and pinned through the compositor's IPC after it maps — otherwise Hyprland tiles it like any other window, `always-on-top` is ignored, and compositor drag does nothing. No extra window rule is required.
A few compositors (notably COSMIC) ignore `always-on-top` for native Wayland windows. To restore pinning there, run the app under XWayland:
```yaml
desktop:
ozone_platform_hint: x11
```
That bridges to `ELECTRON_OZONE_PLATFORM_HINT` at launch (an explicit env var still wins). The trade: X11 cannot restore a window that has ignored the mouse, so the HUD stays a solid window instead of click-through. Some KDE setups also report keyboard breakage with the X11 ozone backend — leave the hint on `auto` unless you need always-on-top.
### Settings & onboarding
Manage providers, models, tools, and credentials from a real UI instead of editing YAML. First-run onboarding gets you to your first message in seconds. The settings panes cover providers/keys, model selection, toolset configuration, MCP servers, the gateway, and session management.
- **Providers settings pane** — a dedicated place to manage inference providers, with an Accounts / API-keys UX for signing in and storing credentials per provider. Its **Local Models** view installs and manages an on-device llama.cpp runtime — see [Local Models](/user-guide/local-models).
- **Every provider and model in the menus** — the GUI surfaces the full provider list and every model that `hermes model` knows about, so you pick from the same catalog the CLI sees rather than a curated subset.
- **xAI Grok OAuth** — Grok is a first-class OAuth provider in the launcher; sign in through the browser flow like the other OAuth providers.
- **Tool-backend installs from the GUI** — run a tool backend's post-setup install steps directly from the app instead of dropping to a terminal.
- **Terminal font picker** — choose an installed font in **Settings → Appearance**. Nerd Fonts such as `MesloLGS NF` render Powerlevel10k separators and icons in both interactive and agent terminals; the setting is saved per profile.
- **Reopen Last Chat on Launch** — by default the app picks up where you left off on cold start. Turn it off in **Settings → Appearance** (or set `display.resume_last_session: false` in `config.yaml`) to always begin with a fresh chat. Deep links and explicit destinations are never overridden either way.
- **Auxiliary-model warning** — if you switch the main model to a new provider while auxiliary tasks (titling, summarization, and similar helpers) are still pinned to another provider, the app warns you so you don't unknowingly split work across two providers.
- **VS Code Marketplace themes** — beyond the built-in theme presets, the appearance settings include a live VS Code Marketplace search: pick any color theme and the app downloads, converts, and installs it as a desktop theme. The same importer is available from the command palette (*Install theme*), and imported themes can be removed again from the appearance settings.
- **Keep computer awake** — **Settings → Advanced → Keep computer awake** stops the machine from sleeping so long or overnight agent runs keep going (the display can still dim). This is a per-computer setting.
First-run onboarding has been redesigned on a unified overlay design system, and you can pick **Choose provider later** to skip provider setup and get into the app first.
#### Per-profile settings: the "Applies to" scope
When you have two or more [profiles](./profiles.md), the config-backed settings pages — **Model, Workspace, Safety, Memory & Context, Voice, Chat, Advanced, and Tools & Keys** — and the **Messaging** overlay show a shared **Applies to** chip row at the top. It selects which profile your edits target:
- The default selection **follows the active profile**, which behaves exactly as before — edit the profile you're using.
- Pick another profile to view and edit *its* settings without switching the whole app; the selection persists as you move between settings pages.
- Switching the app's active profile resets the selector, so edits can't silently keep landing on a previously selected profile.
- With fewer than two profiles the chip row is hidden entirely.
(The Gateways page handles profiles differently — via its **Per-profile overrides** subsection — and the Capabilities and Scheduled Jobs views have their own scope selectors.)
### Management panes
The app also surfaces the broader Hermes management surface so you don't have to drop to a terminal:
- **Skills** — browse, install, and manage [skills](./features/skills.md). The Skills tab lists your installed skills with enable/disable toggles, and below them the full built-in optional-skills catalog that ships with Hermes — each entry has a one-click **Install** button that flips the row into the installed list once it finishes.
- **Memory graph (Star Map)** — type `/journey` (aliases `/learning`, `/memory-graph`) in chat to open an interactive constellation of learned skills and memories over time, with a playback scrubber. Nodes can be edited or deleted right from the panel (skills are archived, memories removed). See [Learning Journey](./features/memory.md#learning-journey-journey).
- **Cron** — view and manage [scheduled jobs](../reference/cli-commands.md#hermes-cron).
- **Profiles** — switch between [Hermes profiles](./profiles.md) (isolated config/skills/sessions).
- **Messaging** — set up gateway channels.
- **Agents** and **Command Center** — orchestration surfaces for multi-agent work.
### Bot Mode (built in)
**Bot Mode** ships with the app and is on by default: a "one chat per agent"
roster where every [Hermes profile](./profiles.md) appears as a bot with its
own avatar (geometric face, uploaded image, AI-generated portrait, or a pixel
pet), its own canonical **Bot Chat** conversation, and its own **Routines**
(recurring tasks backed by Hermes cron). The roster lives in the left
sidebar as a tab next to your conversations — a **Sessions | Bots** tab
strip — rather than a second pane stacked below the session list. Installs
that picked up the older stacked layout are re-homed into the tab strip
automatically, once; if you've hand-placed panes yourself, your layout is
left alone. The **Cronjobs** (Routines) pane docks beside the chat only
while the Bots tab is active and disappears when you switch back to
Sessions (older desktop builds keep it always visible).
Create new agents from the roster —
Name / Title / Description plus an Advanced disclosure with the full
capabilities surface (model, SOUL, skills, toolsets, MCP servers) — group
them into sections, and open group chats where several bots deliberate.
Group chats appear as standalone Discord-style rows in the roster — stacked
member avatars, member count, a preview of the latest room line, and the
"needs you" badge — interleaved with the bot rows in the same pin+recency
ordering. Clicking a group row opens the room as a tab that takes over the
**main chat window** (older desktop builds fall back to opening it inside the
bots side panel).
Bots message each other: type `@researcher have a look at this` in any chat
and the active bot hands the message off and reports back, and bots reach
each other's Bot Chats directly (`hermes -p <bot> chat`). The backend teaches
each bot's canonical **Bot Chat** session the messaging protocol
automatically (config `agent.bot_mode_protocol`, default on) — including
when a teammate bot opens it headlessly from the CLI — so bot-to-bot
replies and handoffs work without touching your SOUL.md, and your regular
sessions stay untouched.
Bot Mode's sessions — each bot's canonical Bot Chat and every group-chat
member session — are always hidden from the global Sessions sidebar. They
live in the Bots pane (roster rows, room views, and each bot's session
browser) instead of interleaving with your own conversations.
Bots you don't use can be tucked away: right-click a bot row → **Hide
Bot**. Hidden bots leave the roster but keep working — @mentions still
resolve and group-chat membership is untouched. An eye toggle appears in
the Bots header whenever at least one bot is hidden; click it to reveal
hidden bots dimmed in place (right-click → **Unhide Bot** brings one back),
and the eye shows a dot when a hidden bot has unread activity. Hidden
state is stored in the bot's profile, so it follows the bot across
machines.
Don't want it? Flip it off in **Settings → Plugins → Bots** — the roster,
routines pane, and composer middleware unregister live, no restart needed.
Full guide — creating agents (including the multi-machine **Create on**
picker), the roster across connections, bot-to-bot mentions, and how group
chats decide who replies: [Bot Mode: A Roster of Agents](./bot-mode.md).
### Keyboard & navigation
- **Command palette** — press **Cmd+K** or **Cmd+P** (Ctrl+K / Ctrl+P on Windows/Linux) to jump to actions and navigate the app from the keyboard: open any page or settings section, jump to a session by title or id, switch model/theme/color mode, spawn a terminal, restart the gateway, update Hermes, and more.
- **Rebindable shortcuts** — **Settings → Keyboard Shortcuts** (or **Cmd/Ctrl+/**) opens the shortcuts panel where you can remap almost every binding — profile switching, session navigation, view toggles, and any shortcuts contributed by desktop plugins. Duplicate assignments are flagged as conflicts. A few defaults worth knowing: **Cmd/Ctrl+N** new session, **Cmd/Ctrl+.** Command Center, **Cmd/Ctrl+,** Settings, **Cmd/Ctrl+Shift+F** search sessions, **Cmd/Ctrl+1–9** switch profiles, **Shift+X** toggle light/dark.
- **Custom zoom shortcuts** — zoom the interface in half-step increments for finer control over text size.
- **UI language switcher** — change the app's interface language in-app: English, Simplified Chinese (zh-Hans), Traditional Chinese (zh-Hant), Japanese, Arabic (RTL), and Russian.
### Sessions & profiles
- **Session-list overhaul** — a reworked session list with archiving and general session hygiene to keep the list manageable as it grows.
- **Search sessions by id** — find a specific session directly by its id.
- **Concurrent multi-profile sessions** — run sessions across multiple [profiles](./profiles.md) at the same time, and reference a session in another profile with cross-profile `@session` links.
- **Export / import a profile** — share a whole setup as a single file. **⌘K → Export profile…** (or right-click a profile square in the rail) writes a `.tar.gz` with skills, memory, persona, crons, plugins, and settings; API keys are stripped. Exporting from the desktop also bundles your appearance and interface — skin, light/dark mode, custom themes, the profile's rail color, and your window layout — so an imported profile arrives looking the way the sender had it. Import via **⌘K → Import profile…** or the button beside the rail's **+**; it applies the overlay and drops you into the new profile. The same archive works with `/export` / `/import` in chat and `hermes profile export` / `import` from a shell. See [Export and import a profile file](./profile-distributions.md#export-and-import-a-profile-file).
## Updating
The app checks for updates in the background and offers a one-click update when one is ready.
The desktop app and the Hermes backend it talks to update on separate clocks — the app package on your machine, the backend wherever it runs. When more than one update target exists (a remote gateway, or several registered gateways), the update affordances (**Update now** on the About panel, the ⌘K **Update Hermes** row, and the update-ready toast) update **everything**: the connected backend first, then every other eligible registered gateway (Hermes Cloud entries are platform-managed and skipped), and the desktop app itself last, since applying the client update relaunches the app. Single-machine installs keep the one-button experience.
After any backend update, the app also re-checks its own version and warns with a one-click **Update desktop app** action if the GUI is still behind — so updating a remote backend can never silently leave you on a stale desktop build.
The [manual update process](https://hermes-agent.nousresearch.com/docs/getting-started/updating) also works with the GUI.
## Uninstalling
Open **Settings → About → Danger zone** and pick how much to remove:
- **Uninstall Chat GUI only** — removes the desktop app and its data; the Hermes agent, your config, and your chats stay. (Same as `hermes uninstall --gui`.)
- **Uninstall GUI + agent, keep my data** — removes the app and the agent but keeps config, chats, and secrets for a future reinstall. (Same as `hermes uninstall`.)
- **Uninstall everything** — removes the app, the agent, and all user data. (Same as `hermes uninstall --full`.)
The app closes to finish the job (the cleanup runs after it exits so it can remove the running app bundle and its own venv). The agent-removing options are hidden automatically when no local agent is installed (for example, a GUI-only "lite" client connected to a remote backend).
You can do the same from the terminal — `hermes uninstall --gui` for the GUI alone, or `hermes uninstall` / `hermes uninstall --full` for the agent too.
:::note
Running `hermes uninstall --gui` from a **source checkout** (a `hermes desktop` dev build) also removes the workspace `node_modules` and `apps/desktop/{dist,release}` build output, since those are GUI build artifacts. They're recoverable with `hermes desktop` (or `npm install` + a rebuild) — but if you're actively hacking on the desktop app, expect to reinstall dependencies afterward.
:::
## CLI reference: `hermes desktop`
To launch via the CLI, simply run `hermes desktop`. By default it installs workspace Node dependencies, builds the current OS's unpacked Electron app, then launches that packaged artifact.
| Flag | Description |
| -------------------- | ----------------------------------------------------------------------------------------- |
| `--skip-build` | Skip npm install/package and launch the existing unpacked app from `apps/desktop/release` |
| `--force-build` | Force a full rebuild even if the content stamp matches |
| `--build-only` | Build the desktop app but do not launch it (used by `hermes update`) |
| `--source` | Launch via `electron .` against `apps/desktop/dist` instead of the packaged app |
| `--cwd PATH` | Initial project directory for desktop chat sessions (sets `HERMES_DESKTOP_CWD`) |
| `--hermes-root PATH` | Override the Hermes source root the app uses (sets `HERMES_DESKTOP_HERMES_ROOT`) |
| `--ignore-existing` | Force the app to ignore any `hermes` CLI already on `PATH` during backend resolution |
| `--fake-boot` | Enable deterministic boot delays for validating the startup UI |
## How it works
The packaged app ships the Electron shell and a native React chat surface. On first launch it can install the Hermes Agent runtime into `HERMES_HOME` (`~/.hermes`, or `%LOCALAPPDATA%\hermes` on Windows) — **the same layout a CLI install uses**, which is why the two are interchangeable. Backend resolution first honours `HERMES_DESKTOP_HERMES_ROOT`, then a completed managed install, then a probed `hermes` on `PATH` (unless `--ignore-existing` / `HERMES_DESKTOP_IGNORE_EXISTING=1` is set), and finally an explicit `HERMES_DESKTOP_HERMES` command override for packagers such as Nix. The React renderer talks to a headless backend the app launches for you — a `hermes serve` process that serves the `tui_gateway` JSON-RPC/WebSocket API — and reuses the agent runtime rather than embedding `hermes --tui`. The desktop app is **self-contained**: it runs its own `hermes serve` backend and never opens or requires the [web dashboard](./features/web-dashboard.md). (Runtimes older than the `serve` command fall back to a headless `dashboard --no-open` automatically, so an app update never outruns its backend.) Install, backend-resolution, and self-update logic live in the Electron main process.
## Connecting to a remote backend
By default the app starts and manages its own **local** backend. You can instead point it at a Hermes backend running on another machine — a VPS, a home server, or a Mini behind Tailscale.
Everything connection-related lives on one settings page: **Settings → Gateways**. (Older builds split this across separate **Gateway** and **Connections** pages — those are now unified, and old `?tab=connections` deep links redirect to the unified page.)
**Settings → Gateways → Connection mode** offers the alternatives to the local gateway:
- **Remote gateway** — enter the URL of a `hermes serve` backend you run yourself and sign in. This is the mode the rest of this section walks through.
- **Hermes Cloud** — sign in once to Hermes Cloud and pick from the agents on your account; no URL to paste. The app discovers your agents (with an organization picker if your account spans several orgs), and connecting to one switches the session over automatically. The status bar shows the cloud connection while it's active.
Gateway connections are **machine-level**: the Gateways page manages which gateway backends this desktop can connect to, and profiles are discovered *from* the gateways you connect. Sessions select one gateway at a time, while the adjacent profile rail selects a profile discovered on that gateway.
### The multi-connection registry
Further down the same **Settings → Gateways** page, **Registered gateways** manages a named list of every Hermes gateway the app knows about — the local runtime, any number of remote gateways (LAN, Tailscale, internet), Hermes Cloud instances, and SSH hosts — all persisted together in one place. You can jump there from the plug button at the right end of the sidebar profile rail (**Connect another Hermes gateway…**) or via **⌘K → Gateways**. The full guide, including the union agent roster, `@name-device` handles, fleet-wide updates, and the plugin SDK surface, is at [Connecting Desktop to Many Hermes Instances](./multi-connection-desktop.md).
- **Every connection needs a unique name** (a device name such as "Homelab" or "Work laptop"). When the same profile name exists on several registered gateways, surfaces disambiguate it as `@profile-device` (e.g. `@research-homelab`).
- **Switch gateways from the Sessions sidebar.** A named gateway selector appears when more than one gateway is registered and handles any registry size without making gateways look like profiles. The adjacent profile rail then shows only that gateway's agents and remembers the last profile used there; large profile sets condense independently.
- **Choose what opens after a restart.** **Open on launch** keeps the backward-compatible **Primary gateway** default, or can resume the **Last used** gateway after it connects successfully. This preference is stored outside the application bundle and survives Desktop updates.
- **Add / edit / remove / test** connections from the panel. The **Add** flow offers all four kinds — **Local**, **Hermes Cloud**, **Remote gateway**, and **SSH** (the Local button is disabled while the app-managed local entry exists, and a hint points cloud adds at the sign-in/discovery flow above). The local entry is managed by the app and cannot be removed. **Test** probes the connection's own HTTP and WebSocket legs directly.
- **Duplicates are rejected at save time**: only one **local** entry ever; remote and cloud entries are deduplicated on the normalized URL (trimmed, trailing slashes stripped, lowercased — across both kinds); SSH entries on the normalized `user@host:port` plus remote profile.
- Existing settings are **imported automatically** the first time you run a build with the registry: your current global connection and any legacy per-profile overrides become named entries. The legacy settings file is left untouched, so older builds keep working.
- Cloud entries come from the Hermes Cloud sign-in/discovery flow above, not from a hand-typed URL.
- Tokens are stored encrypted with the OS keyring (with an explicit plain-text opt-in on keyring-less Linux).
Side-by-side routing is live: each registered gateway dials its own backends and sockets on demand (keyed per connection + profile), the plugin SDK exposes the union agent roster (`host.agents()` / `host.ensureAgent()`), and **Update all instances** on the Gateways page dispatches `hermes update` to every eligible gateway at once — Hermes Cloud entries are skipped (the platform updates them), and each instance reports its own result.
:::info The remote backend is a running `hermes serve` process
"Remote backend" means a **`hermes serve`** server running on the remote machine — that is the process the desktop app connects to. Nothing in this section works unless that backend is actually up and reachable. The desktop app does not start it for you; you (or a `systemd` service) keep `hermes serve` running on the remote host, and the app attaches to it. If you also use messaging channels (Telegram, Discord, etc.), the **gateway** is a *separate* long-running process you start independently — see the note after the setup steps.
:::
The connection has two halves: on the backend you protect it with an **auth provider**, and in the app you enter the backend's URL and sign in. Binding the backend to a non-loopback address automatically engages its auth gate, and the provider you configure is what lets the desktop app through.
**Pick a provider based on where the backend lives:**
- **OAuth (Nous Portal) — preferred for anything reachable beyond your own machine.** Logins are verified against your Nous account, so this is the option suitable for a VPS, a public host, or any remote backend. Register the dashboard with `hermes dashboard register` (or the Portal [`/local-dashboards`](https://portal.nousresearch.com/local-dashboards) page) to provision its OAuth client, then sign in from the app with **Sign in with Nous Research**. A self-hosted OIDC provider works the same way if you run your own identity provider.
- **Username/password — local / trusted-network use only.** The simplest option when the backend is on the same trusted LAN or reachable only over a VPN (e.g. Tailscale). It protects a single shared credential with no external identity provider, so **do not use it for a dashboard exposed to the public internet** — reach for OAuth there instead.
The rest of this section shows the username/password path because it's the quickest to stand up on a trusted network; for the OAuth path see [Web Dashboard → Default provider: Nous Research](./features/web-dashboard.md#default-provider-nous-research).
### On the backend (the remote machine)
Set a username and password, then start the backend bound to a reachable address. The credentials live in `~/.hermes/.env` (the secrets file, mode 0600):
```bash
# 1. Set the dashboard login credentials.
cat >> ~/.hermes/.env <<'EOF'
HERMES_DASHBOARD_BASIC_AUTH_USERNAME=admin
HERMES_DASHBOARD_BASIC_AUTH_PASSWORD=choose-a-strong-password
# Recommended: a stable signing secret so sessions survive restarts.
# Without it a random key is generated per boot and you'll be logged out
# on every restart.
HERMES_DASHBOARD_BASIC_AUTH_SECRET=$(openssl rand -base64 32)
EOF
chmod 600 ~/.hermes/.env
# 2. Run the backend bound to a reachable address. The non-loopback bind
# engages the auth gate; the username/password provider handles login.
hermes serve --host 0.0.0.0 --port 9119
```
Keep that `hermes serve` process running for as long as you want the desktop app to be able to connect — if it stops, the app can no longer reach the backend. Run it under `systemd`, `tmux`, or your process manager of choice so it survives logout and reboots.
Separately, make sure the **gateway is running** on the remote host if you rely on messaging channels — the `hermes serve` backend is what the desktop app talks to, but your Telegram/Discord/Slack gateway sessions are a different process that you start and keep running on their own. See [Messaging](./messaging/index.md) for gateway setup.
Prefer not to keep a plaintext password at rest? Set `HERMES_DASHBOARD_BASIC_AUTH_PASSWORD_HASH` to a scrypt hash instead — compute it with `python -c "from plugins.dashboard_auth.basic import hash_password; print(hash_password('PW'))"`. Full configuration surface (config.yaml keys, every env var, the rate limiter): [Web Dashboard → Username/password provider](./features/web-dashboard.md#usernamepassword-provider-no-oauth-idp).
Running the backend as a systemd service? Give the unit `EnvironmentFile=%h/.hermes/.env` so the credentials are in the environment at boot.
:::warning
The backend reads and writes your `.env` (API keys, secrets) and can run agent commands. The **username/password** setup shown above is for a trusted network — never expose a password-protected backend directly to the open internet; put it behind a VPN. [Tailscale](https://tailscale.com/) is the clean option: bind to the machine's tailscale IP (`--host <tailscale-ip>`) and use `http://<tailscale-ip>:9119` as the Remote URL so only your tailnet can reach it. To reach a backend over the public internet, use the **OAuth (Nous Portal)** provider instead.
:::
### In the app
**Settings → Gateways → Remote gateway:**
1. **Remote URL** — `http://<backend-host>:9119` (path prefixes like `/hermes` work if you front it with a reverse proxy)
2. **Sign in** — the app detects which provider the backend advertises and adapts the button. For a username/password backend it shows a **Sign in** button that opens a credential form (enter the credentials from step 1). For an OAuth backend it shows **Sign in with `<provider>`** (e.g. *Sign in with Nous Research*), which runs the provider's browser sign-in. Either way the app ends up with an authenticated session against the backend.
3. **Save and reconnect** — switches the desktop shell onto the remote backend. The session refreshes automatically; you stay signed in across restarts when `HERMES_DASHBOARD_BASIC_AUTH_SECRET` is set.
You can also set the backend URL without the UI via the `HERMES_DESKTOP_REMOTE_URL` environment variable before launching the app (it overrides the in-app setting); you still sign in from the Gateways settings panel.
:::note Per-profile remote hosts
The remote gateway host is configured per [profile](./profiles.md), so each profile can point at its own remote backend (or stay on its local one). Switching profiles switches which remote host the app connects to.
:::
### Troubleshooting
- **Sign-in fails with 401 / "Invalid credentials"** — the username or password doesn't match the backend's `HERMES_DASHBOARD_BASIC_AUTH_USERNAME` / `HERMES_DASHBOARD_BASIC_AUTH_PASSWORD`. The backend returns the same generic error for an unknown user and a wrong password (no enumeration oracle), so double-check both. Confirm the gate is on with `curl -s http://<host>:9119/api/status | jq '.auth_required, .auth_providers'` — it should report `true` and include `"basic"`.
- **No "Sign in" button — it asks for a session token instead** — the backend's username/password provider isn't active. `/api/status` won't list `"basic"` in `auth_providers`. Make sure both the username and a password (or password hash) are set in `~/.hermes/.env` and that the dashboard process actually loaded them.
- **Signed out on every restart** — set `HERMES_DASHBOARD_BASIC_AUTH_SECRET` to a stable value. Without it the token-signing key is regenerated per boot, invalidating all sessions.
- **Connection refused / times out** — the backend bound to `127.0.0.1` (the default) or a firewall/VPN is blocking the port. Bind to `0.0.0.0` or the tailscale IP and open the port to your trusted network.
For the same setup from the web-dashboard angle, see [Web Dashboard → Connecting Hermes Desktop to a remote backend](./features/web-dashboard.md#connecting-hermes-desktop-to-a-remote-backend); the env vars are catalogued under [Environment Variables → Web Dashboard & Hermes Desktop](../reference/environment-variables.md#web-dashboard--hermes-desktop).
## Extending the desktop app
The desktop app is contribution-driven — panes, pages, sidebar nav, status-bar
items, palette commands, keybinds, and themes all register through one SDK, and
you can add your own. A plugin is a single ESM file dropped in
`$HERMES_HOME/desktop-plugins/<id>/plugin.js`; the app loads it within seconds and
hot-reloads every save. Manage installed plugins live in **Settings → Plugins**.
See [Desktop Plugin SDK](../developer-guide/desktop-plugin-sdk.md) for the full
reference. (This is separate from the [web dashboard plugin system](./features/extending-the-dashboard.md).)
The **Agent plugins** section on the same Settings → Plugins page manages
backend (agent-side) [plugins](./features/plugins.md) you installed — user,
git, project, pip, and portable installs. Repo-bundled built-ins (platform
adapters, provider plugins, and similar) are not listed there: they ship
enabled by default and are configured from their own surfaces, so the section
stays focused on what you added yourself. With two or more profiles the
section also has its own **Applies to** selector, so you can list and toggle
another profile's agent plugins without switching the whole app (the backend
`plugins.manage` RPC accepts an optional `profile` parameter for this).
## Troubleshooting
### Failed turns name the failing layer
When a turn fails, the chat renders an error card that names **which layer
failed** — provider/model, custom endpoint, streaming connection,
authentication, billing, gateway, local runtime, or disk — instead of a
generic error toast. The card offers recovery actions matched to the failure:
- **Retry** — re-runs the failed turn in place (hidden when retrying would
deterministically reproduce the failure, e.g. a content-policy rejection).
- **Switch provider** — jumps to Settings → Models for provider, endpoint,
auth, and billing failures.
- **Open logs** — opens `HERMES_HOME/logs` in your file manager. On a remote
or Cloud connection the button reads **Open Desktop logs**: it opens the
local Desktop-side logs (transport evidence), since the failed turn's
gateway/agent logs live on the remote machine.
- **Send diagnostics** — uploads a redacted debug bundle to Nous-internal
storage after an explicit consent prompt (same pipeline as
`hermes debug share --nous`; secrets are always redacted, the bundle is
viewable by Nous staff only and auto-deletes after 14 days). On success you
get a private view link to paste into your support thread, plus quick links
to GitHub Issues, Nous Portal Support, and Discord. On a remote or Cloud
connection the backend bundles its own agent/gateway logs and the local
Desktop log is attached alongside, so support sees both halves.
- **Copy error details** — copies a compact plain-text summary (layer, code,
provider/model, error message) you can paste into a bug report or Discord.
The layer comes from the same error classifier the agent's retry loop uses,
so it reflects the real failure semantics, not a guess from the message text.
Older backends that predate the descriptor still render the card with a
generic title and the Retry / Open logs / Copy error details actions.
Boot logs land in `HERMES_HOME/logs/desktop.log` (it includes backend output and recent Python tracebacks) — check it first if the app reports a boot failure. You can also tail it from the CLI:
```bash
hermes logs gui -f
```
Common resets:
```bash
# Force a clean first-launch setup (macOS/Linux)
rm "$HOME/.hermes/hermes-agent/.hermes-bootstrap-complete"
# Rebuild a broken Python venv (macOS/Linux)
rm -rf "$HOME/.hermes/hermes-agent/venv"
# Reset a stuck macOS microphone prompt
tccutil reset Microphone com.nousresearch.hermes
```
### "The host key has CHANGED since you last connected" (SSH remote)
If your SSH remote was reinstalled or its host key rotated, SSH fails closed
and Desktop latches an error overlay instead of retrying (retrying can never
succeed until the stale key is cleared). Verify the change is expected, then
remove the old entry and retry from the overlay:
```bash
ssh-keygen -R <host>
```
Click **Retry** (or re-apply the connection in Settings → Gateway) after
clearing the entry — the latch resets and the next boot dials fresh.
### "Build desktop app" stuck on Electron download
The build downloads the Electron runtime (~114&nbsp;MB) from `github.com/electron/electron/releases`. If the installer hangs on the **Build desktop app** step with the live output repeating `retrying attempt=…`, GitHub is being blocked or throttled on your network (firewall, proxy, or region).
The installer self-heals this automatically: on a failed build it (1) clears a corrupt cached Electron zip and retries, then (2) if it still fails and you haven't set `ELECTRON_MIRROR`, retries once more through `npmmirror.com`, the de-facto Electron community mirror. `@electron/get` SHASUM-checks the download, but the checksums come from the same mirror — that catches a corrupt or partial download, not a compromised mirror. If you'd rather not trust a third-party host, pin your own `ELECTRON_MIRROR` (below); the build never overrides one you've set.
To **choose your own mirror** (e.g. a corporate/trusted one), set `ELECTRON_MIRROR` before installing or rebuild manually — the build honors it and won't override it:
```bash
ELECTRON_MIRROR=https://npmmirror.com/mirrors/electron/ \
bash -c 'cd "$HOME/.hermes/hermes-agent/apps/desktop" && CSC_IDENTITY_AUTO_DISCOVERY=false npm run pack'
```
To clear a corrupt cached zip by hand:
```bash
rm -f "$HOME/Library/Caches/electron"/electron-*.zip # macOS
rm -f "$HOME/.cache/electron"/electron-*.zip # Linux
```
## Building from source
If you want to hack on the app itself, install workspace deps from the repo root once, then run the dev server from `apps/desktop`:
```bash
npm install # from repo root — links apps/desktop, web, apps/shared
cd apps/desktop
npm run dev # Vite renderer + Electron, which boots the Python backend
```
Point the app at a specific checkout, or sandbox it from your real config:
```bash
HERMES_DESKTOP_HERMES_ROOT=/path/to/clone npm run dev
HERMES_HOME=/tmp/throwaway npm run dev
npm run dev:fake-boot # exercise the startup overlay with deterministic delays
```
Build installers:
```bash
npm run dist:mac # DMG + zip
npm run dist:win # NSIS + MSI
npm run dist:linux # AppImage + deb + rpm
npm run pack # unpacked app under release/ (no installer)
```
macOS/Windows signing and notarization run automatically when the relevant credentials are present in the environment (`CSC_LINK` / `CSC_KEY_PASSWORD` / `APPLE_*` for macOS, `WIN_CSC_*` for Windows).
### macOS permissions and local rebuilds (TCC)
**Silence every folder prompt with one switch.** macOS prompts per-category
(Desktop, then Downloads, then Documents, ...) as Hermes touches each folder.
A single **Full Disk Access** grant covers all of them, permanently — and
with Hermes' stable signing identities it survives every update:
1. System Settings → **Privacy & Security → Full Disk Access** (or run
`open "x-apple.systempreferences:com.apple.preference.security?Privacy_AllFiles"`)
2. Enable your terminal app — and **Hermes.app** if you use Desktop.
3. Fully quit and relaunch them once.
`hermes doctor` reports whether the current terminal context already has the
grant, and `hermes setup` shows this tip on macOS when it doesn't.
macOS remembers permission grants (Full Disk Access, Desktop/Downloads/Documents,
Accessibility, Automation, microphone) against the app's *code-signing identity*,
not its path. Locally built and self-updated apps are signed with a stable
identifier-pinned ad-hoc signature, so grants persist across updates.
One-time note: grants made to builds *before* the identifier-pinned signing
fix (PR #73681) carry the old cdhash-pinned requirement. macOS keeps showing the
toggle as ON for those stale grants but still re-prompts, because the stored
grant no longer matches the rebuilt binary — and the modern prompt has no Allow
button, so it looks like there is nothing to re-check. If that happens, reset
the stale grant once and re-grant:
```bash
tccutil reset ScreenCapture com.nousresearch.hermes # repeat per service
```
then toggle the fresh entry ON in System Settings and fully quit & relaunch
Hermes. Grants are stable from then on.
For the strongest guarantee — a certificate-anchored identity, the same
mechanism yabai/skhd users rely on — create a self-signed code-signing
certificate once and tell Hermes to use it. The one-shot command does
everything (creates the certificate in your login keychain, grants `codesign`
access, writes the config, and re-signs the packaged app):
```bash
hermes desktop --setup-tcc-identity
```
Or do it manually:
1. Keychain Access → Certificate Assistant → **Create a Certificate…**
2. Name: `Hermes Local Signing`, Identity Type: *Self-Signed Root*,
Certificate Type: **Code Signing**.
3. In Keychain Access, double-click the new certificate → **Trust** → set
**Code Signing** to *Always Trust* (an imported self-signed certificate is
not a valid signing identity until it is trusted for code signing —
`security find-identity -v -p codesigning` should list it afterwards).
4. `hermes config set desktop.macos_signing_identity "Hermes Local Signing"`
Use `--identity <name>` with the command to create/use a differently named
certificate (default: `Hermes Local Signing`). The command is idempotent —
re-run it after updates to re-point the config and re-sign the rebuilt app.
The next update re-signs the rebuilt app with that certificate; every TCC grant
survives. No Apple Developer account is required. Notarized release builds are
detected and never re-signed.
One-time note: changing the signing identity (including the first update after
this fix) changes the app's identity once, so macOS will re-prompt one final
time. Grants are stable from then on. If a permission gets stuck, reset it with
`tccutil reset All com.nousresearch.hermes` and re-grant.
## See also
- [CLI Guide](./cli.md) — the terminal interface
- [TUI](./tui.md) — the modern terminal UI used by `hermes --tui` and the dashboard chat tab
- [Web Dashboard](./features/web-dashboard.md) — browser admin panel with an embedded chat tab
- [Configuration](./configuration.md) — config that the desktop app reads and writes
- [Windows (Native)](./windows-native.md) — native Windows install path
website/docs/user-guide/docker.md
+856
View File
@@ -0,0 +1,856 @@
---
sidebar_position: 7
title: "Hermes Docker Setup"
description: "Running Hermes Agent in Docker and using Docker as a terminal backend"
---
# Hermes Docker Setup
There are two distinct ways Docker intersects with Hermes Agent:
1. **Running Hermes IN Docker** — the agent itself runs inside a container (this page's primary focus)
2. **Docker as a terminal backend** — the agent runs on your host but executes every command inside a single, persistent Docker sandbox container that survives across tool calls, `/new`, and subagents for the life of the Hermes process (see [Configuration → Docker Backend](./configuration.md#docker-backend))
This page covers option 1. The container stores all user data (config, API keys, sessions, skills, memories) in a single directory mounted from the host at `/opt/data`. The image itself is stateless and can be upgraded by pulling a new version without losing any configuration.
## Quick start
If this is your first time running Hermes Agent, create a data directory on the host and start the container interactively to run the setup wizard:
:::caution Avoid browser-based VPS consoles for the install commands
Some VPS providers (Hetzner Cloud, and several others) offer a browser-based
console for managing hosts. These consoles transmit special characters
incorrectly — `:` may arrive as `;`, `@` may be mis-rendered, and non-English
keyboard layouts fare worse — which silently corrupts `docker run` arguments
like `-v ~/.hermes:/opt/data`, `-e KEY=value`, and pasted API keys / tokens.
**Connect over SSH instead** (`ssh root@<host>`) for copy-paste-safe command
entry. If you must use the browser console, type the commands manually
instead of pasting, and double-check every `:`, `@`, `=`, and `/` in the
result before hitting Enter.
:::
```sh
mkdir -p ~/.hermes
docker run -it --rm \
-v ~/.hermes:/opt/data \
nousresearch/hermes-agent setup
```
This drops you into the setup wizard, which will prompt you for your API keys and write them to `~/.hermes/.env`. You only need to do this once. It is highly recommended to set up a chat system for the gateway to work with at this point.
:::tip
Inside the container, run `hermes setup --portal` once — the refresh token persists in the mounted `~/.hermes` volume. See [Nous Portal](/integrations/nous-portal).
:::
## Running in gateway mode
Once configured, run the container in the background as a persistent gateway (Telegram, Discord, Slack, WhatsApp, etc.):
```sh
docker run -d \
--name hermes \
--restart unless-stopped \
-v ~/.hermes:/opt/data \
-p 8642:8642 \
nousresearch/hermes-agent gateway run
```
Port 8642 exposes the gateway's [OpenAI-compatible API server](./features/api-server.md) and health endpoint. It's optional if you only use chat platforms (Telegram, Discord, etc.), but required if you want the dashboard or external tools to reach the gateway.
:::tip Gateway runs supervised
Inside the official Docker image, `gateway run` is **automatically supervised by s6-overlay**: if the gateway process crashes it's restarted within a couple of seconds without losing the container, and the dashboard (when `HERMES_DASHBOARD=1` is set) is supervised alongside it. The `gateway run` CMD process itself is a `sleep infinity` heartbeat that keeps the container alive while s6 manages the actual gateway process — so `docker stop` still shuts everything down cleanly, but `docker logs` shows the supervised gateway's output.
You'll see a one-line breadcrumb in `docker logs` confirming the upgrade. To opt out — and get the historical "gateway is the container's main process, container exit = gateway exit" semantics — pass `--no-supervise` or set `HERMES_GATEWAY_NO_SUPERVISE=1`. The opt-out is useful for CI smoke tests that want the container to exit with the gateway's status code; for production deployments the supervised default is strictly better.
This behavior applies to the s6-based image only. Earlier (tini-based) images still run `gateway run` as the foreground main process.
:::
:::note Where gateway logs go
See the [Where the logs go](#where-the-logs-go) section below for the full routing map (per-profile gateways, dashboard, boot reconciler, container-wide `docker logs`).
:::
:::note Tool-loop hard stops for unattended gateways
Unattended gateway and cron sessions enable tool-loop hard stops by default through `non_interactive_hard_stop_enabled`. Interactive CLI, TUI, Desktop, and ACP sessions remain warning-only. To opt an unattended deployment out in the profile's `config.yaml`:
```yaml
tool_loop_guardrails:
non_interactive_hard_stop_enabled: false
```
:::
Note: the API server is gated on `API_SERVER_ENABLED=true`. To expose it beyond `127.0.0.1` inside the container, also set `API_SERVER_HOST=0.0.0.0` and an `API_SERVER_KEY` (minimum 8 characters — generate one with `openssl rand -hex 32`). Example:
```sh
docker run -d \
--name hermes \
--restart unless-stopped \
-v ~/.hermes:/opt/data \
-p 8642:8642 \
-e API_SERVER_ENABLED=true \
-e API_SERVER_HOST=0.0.0.0 \
-e API_SERVER_KEY="$(openssl rand -hex 32)" \
-e API_SERVER_CORS_ORIGINS='*' \
nousresearch/hermes-agent gateway run
```
Opening any port on an internet facing machine is a security risk. You should not do it unless you understand the risks.
## Running the dashboard
The built-in web dashboard runs as a supervised s6-rc service alongside the gateway in the same container. Set `HERMES_DASHBOARD=1` to bring it up:
```sh
docker run -d \
--name hermes \
--restart unless-stopped \
-v ~/.hermes:/opt/data \
-p 8642:8642 \
-p 9119:9119 \
-e HERMES_DASHBOARD=1 \
nousresearch/hermes-agent gateway run
```
The dashboard is supervised by s6 — if it crashes, `s6-supervise` restarts it automatically after a short backoff. Dashboard stdout/stderr is forwarded to `docker logs <container>` (no prefix; the gateway's own output now lives in a per-profile s6-log file — see [Where the logs go](#where-the-logs-go) below — so the two streams don't clash).
| Environment variable | Description | Default |
|---------------------|-------------|---------|
| `HERMES_DASHBOARD` | Set to `1` (or `true` / `yes`) to enable the supervised dashboard service | *(unset — service is registered but stays down)* |
| `HERMES_DASHBOARD_HOST` | Bind address for the dashboard HTTP server | `0.0.0.0` |
| `HERMES_DASHBOARD_PORT` | Port for the dashboard HTTP server | `9119` |
| `HERMES_DASHBOARD_INSECURE` | **Deprecated / no-op.** Formerly bypassed the auth gate; as of the June 2026 hardening it no longer disables authentication. A non-loopback bind always requires an auth provider | *(ignored — configure a provider instead)* |
The dashboard inside the container defaults to binding `0.0.0.0` — without it, the published `-p 9119:9119` port would not be reachable from the host. To restrict the bind to container loopback (for sidecar / reverse-proxy setups), set `HERMES_DASHBOARD_HOST=127.0.0.1`.
The dashboard's auth gate engages automatically when both of the following are true:
1. The bind host is non-loopback (e.g. the default `0.0.0.0` inside the container), **and**
2. A `DashboardAuthProvider` plugin is registered.
There are three bundled ways to satisfy the second condition:
- **Username/password** — the simplest for a self-hosted / on-prem / homelab container on a trusted network or behind a VPN: set `HERMES_DASHBOARD_BASIC_AUTH_USERNAME` + `HERMES_DASHBOARD_BASIC_AUTH_PASSWORD` (and `HERMES_DASHBOARD_BASIC_AUTH_SECRET` for restart-stable sessions). Not suitable for direct public-internet exposure.
- **OAuth (Nous Portal)** — for hosted/public deploys: the `dashboard_auth/nous` provider activates whenever `HERMES_DASHBOARD_OAUTH_CLIENT_ID` is set.
- **Self-hosted OIDC** — to authenticate against your own identity provider via standard OpenID Connect: the `dashboard_auth/self_hosted` provider activates when `HERMES_DASHBOARD_OIDC_ISSUER` + `HERMES_DASHBOARD_OIDC_CLIENT_ID` are set.
Whichever you choose, the gate redirects callers to a login page before they can reach any protected route. See [Web Dashboard → Authentication](features/web-dashboard.md#authentication-gated-mode) for all three providers.
When a reverse proxy such as Traefik or nginx runs in another container, its
bridge-network address is not trusted by default. Set the dashboard's public
URL and trust only that proxy's exact IP, or a bounded CIDR for a dedicated
proxy network, in the mounted `config.yaml`:
```yaml
dashboard:
public_url: "https://dashboard.example.com"
trusted_proxies:
- "172.20.0.5"
# Or, if the proxy address is dynamic on a dedicated network:
# - "172.20.0.0/24"
```
This allows the proxy's `X-Forwarded-Proto: https` to control secure OAuth
cookies while leaving forwarding headers from other peers untrusted. Do not
use `*`, `0.0.0.0/0`, or `::/0`; Hermes rejects those unbounded entries.
If no provider is registered and the bind is non-loopback, the dashboard **fails closed at startup** with a specific error pointing at the missing env var. There is no longer an escape hatch that serves the dashboard unauthenticated on a public bind: `HERMES_DASHBOARD_INSECURE=1` is now a deprecated no-op (it logs a warning and is ignored). Configure a provider, or bind `HERMES_DASHBOARD_HOST=127.0.0.1` and reach the dashboard over an SSH tunnel / Tailscale instead.
:::warning Why `--insecure` was removed
An unauthenticated public dashboard was the entry point for the June 2026 MCP-config persistence campaign: internet scanners reached exposed dashboards (and OpenAI API servers) and drove the agent into planting an SSH-key backdoor. The auth gate is now mandatory on every non-loopback bind. For a trusted-LAN / homelab box, the bundled username/password provider (`HERMES_DASHBOARD_BASIC_AUTH_USERNAME` + `_PASSWORD`) is the zero-infra way to satisfy it.
:::
Running the dashboard as a separate container **is** supported when that container shares the host PID and network namespace (e.g. `network_mode: host`, as the repo's own `docker-compose.yml` does — see its `dashboard` service). Its gateway-liveness detection requires a shared PID namespace with the gateway process, so the limitation only applies to dashboards run in isolated bridge-network containers without a shared PID namespace.
## Running interactively (CLI chat)
To open an interactive chat session against a running data directory:
```sh
docker run -it --rm \
-v ~/.hermes:/opt/data \
nousresearch/hermes-agent
```
Or if you have already opened a terminal in your running container (via Docker Desktop for instance), just run:
```sh
/opt/hermes/.venv/bin/hermes
```
## Persistent volumes
The `/opt/data` volume is the single source of truth for all Hermes state. It maps to your host's `~/.hermes/` directory and contains:
| Path | Contents |
|------|----------|
| `.env` | API keys and secrets |
| `config.yaml` | All Hermes configuration |
| `SOUL.md` | Agent personality/identity |
| `sessions/` | Conversation history |
| `memories/` | Persistent memory store |
| `skills/` | Installed skills |
| `home/` | Per-profile HOME for Hermes tool subprocesses (`git`, `ssh`, `gh`, `npm`, and skill CLIs) |
| `cron/` | Scheduled job definitions |
| `hooks/` | Event hooks |
| `logs/` | Runtime logs |
| `skins/` | Custom CLI skins |
### Immutable install tree
In hosted and published Docker images, `/opt/hermes` is the installed application tree. It is root-owned and read-only to the runtime `hermes` user, so agent turns, gateway sessions, dashboard actions, and normal `docker exec hermes hermes ...` commands cannot edit the core source, bundled `.venv`, `node_modules`, or TUI bundle in place.
All mutable Hermes state belongs under `/opt/data`: config, `.env`, profiles, skills, memories, sessions, logs, dashboard uploads, plugins, and other user-managed files. The image also disables runtime `.pyc` writes and Hermes lazy dependency installs into `/opt/hermes`; optional platform dependencies needed by the published image should be baked into the image or installed through a new image build.
On hosted/published images, agent self-improvement is scoped to skills, memory, plugins, and config under `/opt/data`. The installed core source under `/opt/hermes` is immutable; core changes are made via PRs to the repo and shipped by updating the image, not by live-editing the running install.
If an operator needs to repair or inspect files outside `/opt/data`, use a root shell intentionally. The `hermes` shim normally drops `docker exec hermes hermes ...` back to the runtime user; set `HERMES_DOCKER_EXEC_AS_ROOT=1` for a one-off root invocation when you explicitly need root semantics.
Skill CLIs that store credentials under `~` must be initialized against the subprocess HOME, not just the data-volume root. For example, the [xurl skill](./skills/bundled/social-media/social-media-xurl.md) stores OAuth state in `~/.xurl`; in the official Docker layout, Hermes tool calls read that as `/opt/data/home/.xurl`, so run manual xurl auth with `HOME=/opt/data/home` and verify with `HOME=/opt/data/home xurl auth status`.
:::warning
Never run two Hermes **gateway** containers against the same data directory simultaneously — session files and memory stores are not designed for concurrent write access.
:::
## Multi-profile support
Hermes supports [multiple profiles](../reference/profile-commands.md) — separate `~/.hermes/` subdirectories that let you run independent agents (different SOUL, skills, memory, sessions, credentials) from a single installation. **Inside the official Docker image, the s6 supervision tree treats each profile as a first-class supervised service**, so the recommended deployment is **one container hosting all profiles**.
Each profile created with `hermes profile create <name>` gets:
- A dedicated s6 service slot at `/run/service/gateway-<name>/`, registered dynamically by the runtime — no container rebuild required.
- Auto-restart on crash, backoff-managed by `s6-supervise`.
- Per-profile rotated logs at `${HERMES_HOME}/logs/gateways/<name>/current` (10 archives × 1 MB each).
- State persistence across container restarts: the boot-time reconciler reads `gateway_state.json` from each profile directory and brings the slot back up only for profiles whose last recorded state was `running`. Only a gateway you explicitly stopped (`hermes gateway stop`) stays down across a restart — a container restart, image upgrade, or unexpected exit leaves the recorded state as `running`, so the gateway auto-starts on the next boot.
The lifecycle commands you'd run on the host work the same way from inside the container:
```sh
# Create a profile — registers the gateway-<name> s6 slot.
docker exec hermes hermes profile create coder
# Start / stop / restart — dispatches s6-svc; the gateway lifecycle survives docker restart.
docker exec hermes hermes -p coder gateway start
docker exec hermes hermes -p coder gateway stop
docker exec hermes hermes -p coder gateway restart
# Status — reports `Manager: s6 (container supervisor)` inside the container.
docker exec hermes hermes -p coder gateway status
# Remove a profile — tears down the s6 slot too.
docker exec hermes hermes profile delete coder
```
Under the hood, `hermes gateway start/stop/restart` inside the container is intercepted and routed to `s6-svc` against the right service directory; you don't need to learn the s6 commands directly. For raw supervisor state, use `/command/s6-svstat /run/service/gateway-<name>` (note `/command/` is on PATH only for processes spawned by the supervision tree — when calling from `docker exec`, pass the absolute path).
### Reaching more than one profile from outside the container
Two different surfaces reach a profile's gateway from outside, and they behave differently — don't conflate them:
**Hermes Desktop (and the web dashboard).** The Desktop app's **Remote Gateway** connection talks to a `hermes dashboard` backend (default **port 9119**, enabled by `HERMES_DASHBOARD=1`) — *not* the OpenAI API server. One dashboard backend serves **every** co-located profile: the app's profile switcher sends the target profile with each request and the backend opens that profile's `HERMES_HOME` on disk. So you do **not** need a second port — or a second connection — per profile for Desktop; one `:9119` connection covers them all through the switcher.
**OpenAI-compatible API clients (Open WebUI, LobeChat, `/v1/...`).** These talk to each profile's **API server**, which binds **port 8642 for every profile** (resolved from `API_SERVER_PORT` / `platforms.api_server.extra.port` — there is no auto-allocation and no `config.yaml`/`gateway.port` key). If you want a client to reach a *specific* second profile, give that profile a distinct `API_SERVER_PORT` in **its own** `.env`, otherwise its gateway tries to bind 8642 too and conflicts with the default profile:
```sh
# Create the profile (registers its gateway-<name> s6 slot)
docker exec hermes hermes profile create work
# Point its API server at a free port (write to the profile's own .env)
cat >> /opt/data/profiles/work/.env <<'EOF'
API_SERVER_ENABLED=true
API_SERVER_PORT=8643
EOF
docker exec hermes hermes -p work gateway restart
```
Keep `API_SERVER_PORT` in each profile's **own** `.env`, never in the container-wide `environment:` block — a global value would force every profile onto the same port and they would collide. With bridge networking, publish the extra port in `docker-compose.yml` (`- "8643:8643"`); with `network_mode: host` it is already reachable on the host. The default profile's 8642 connection is untouched.
### Why one container with many profiles, not many containers
Before the s6 migration, "one container per profile" was the recommended pattern because there was no in-container supervisor to manage multiple gateways. With s6 as PID 1, that's no longer necessary, and the single-container layout is simpler in almost every dimension:
| | One container, many profiles | One container per profile |
|---|---|---|
| Disk overhead | One image, one bundled venv, one Playwright cache | N images / N caches |
| Memory overhead | Shared Python interpreter cache, shared node_modules | Duplicated per container |
| Profile creation | `docker exec ... hermes profile create <name>` (seconds) | New `docker run` invocation + port allocation + bind-mount config |
| Per-profile crash recovery | `s6-supervise` auto-restart | Docker's `--restart unless-stopped` (slower, kills sibling work) |
| Logs | Per-profile rotated file via `s6-log`, plus container-boot audit log | `docker logs <name>` per container — no built-in rotation |
| Backup | One `~/.hermes` directory | N directories to coordinate |
The default profile (`default`) is always registered on first boot, so a fresh container ships with one supervised gateway out of the box. Additional profiles are pure runtime adds.
### When you DO want a separate container
Profile-in-container is the default. Run a separate container per profile only when you have a specific reason:
- **Resource isolation per workload** — e.g. a runaway browser-tool session in profile A shouldn't be able to OOM profile B. Containers give you `--memory` / `--cpus` per profile.
- **Independent image pinning** — different upstream image tags per workload.
- **Network segmentation** — distinct Docker networks per profile (e.g. one customer-facing, one internal).
- **Compliance / blast radius** — distinct credentials never share an OS-level process tree.
In those cases, declare one service per profile with distinct `container_name`, `volumes`, and `ports`:
```yaml
services:
hermes-work:
image: nousresearch/hermes-agent:latest
container_name: hermes-work
restart: unless-stopped
command: gateway run
ports:
- "8642:8642"
volumes:
- ~/.hermes-work:/opt/data
hermes-personal:
image: nousresearch/hermes-agent:latest
container_name: hermes-personal
restart: unless-stopped
command: gateway run
ports:
- "8643:8642"
volumes:
- ~/.hermes-personal:/opt/data
```
The warning from [Persistent volumes](#persistent-volumes) still applies: never point two containers at the same `~/.hermes` directory simultaneously. The s6 supervisor inside each container manages its own profile set; cross-container sharing of a data volume corrupts session files and memory stores.
## Where the logs go
The s6 container has four distinct log surfaces, and "why isn't my gateway showing anything in `docker logs`" is a common surprise. Cheatsheet:
| Source | Where it lands | How to read it |
|---|---|---|
| **Per-profile gateway** (`hermes gateway run` and per-profile gateways under s6) | Tee'd to two places: `docker logs <container>` (real time, no extra prefix) **and** `${HERMES_HOME}/logs/gateways/<profile>/current` (rotated, ISO-8601 timestamped, 10 archives × 1 MB each) | `docker logs -f hermes` or `tail -F ~/.hermes/logs/gateways/default/current` on the host |
| **Dashboard** (when `HERMES_DASHBOARD=1`) | `docker logs <container>` (no prefix) | `docker logs -f hermes` — interleaved with gateway lines |
| **Boot reconciler** (records which profile gateways were restored on each container start) | `${HERMES_HOME}/logs/container-boot.log` (append-only audit log) | `tail -F ~/.hermes/logs/container-boot.log` |
| **Generic Hermes logs** (`agent.log`, `errors.log`) | `${HERMES_HOME}/logs/` (profile-aware) | `docker exec hermes hermes logs --follow [--level WARNING] [--session <id>]` |
Two practical consequences worth knowing:
- The file copy at `logs/gateways/<profile>/current` is what survives container restarts. `docker logs` only retains output from the current container's lifetime (and is wiped on `docker rm`); the rotated files persist on the bind-mounted volume.
- The boot reconciler's audit line shape is `<iso-timestamp> profile=<name> prior_state=<state> action=<registered|started>`, so a quick `grep profile=coder ~/.hermes/logs/container-boot.log` reveals when a given profile was last restored and whether s6 auto-started it.
## Environment variable forwarding
API keys are read from `/opt/data/.env` inside the container. You can also pass environment variables directly:
```sh
docker run -it --rm \
-v ~/.hermes:/opt/data \
-e ANTHROPIC_API_KEY="sk-ant-..." \
-e OPENAI_API_KEY="sk-..." \
nousresearch/hermes-agent
```
Direct `-e` flags override values from `.env`. This is useful for CI/CD or secrets-manager integrations where you don't want keys on disk.
:::note Looking for Docker as the **terminal backend**?
This page covers running Hermes itself inside Docker. If you want Hermes to execute the agent's `terminal` / `execute_code` calls inside a Docker sandbox container (one long-lived container shared across Hermes processes — see issue #20561), that's a separate config block — `terminal.backend: docker` plus `terminal.docker_image`, `terminal.docker_volumes`, `terminal.docker_forward_env`, `terminal.docker_env`, `terminal.docker_run_as_host_user`, `terminal.docker_extra_args`, `terminal.docker_persist_across_processes`, and `terminal.docker_orphan_reaper`. See [Configuration → Docker Backend](configuration.md#docker-backend) for the full set including container-lifecycle rules.
:::
## Docker Compose example
For persistent deployment with both the gateway and dashboard, a `docker-compose.yaml` is convenient:
```yaml
services:
hermes:
image: nousresearch/hermes-agent:latest
container_name: hermes
restart: unless-stopped
command: gateway run
ports:
- "8642:8642" # gateway API
- "9119:9119" # dashboard (only reached when HERMES_DASHBOARD=1)
volumes:
- ~/.hermes:/opt/data
environment:
- HERMES_DASHBOARD=1
# Uncomment to forward specific env vars instead of using .env file:
# - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
# - OPENAI_API_KEY=${OPENAI_API_KEY}
# - TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN}
deploy:
resources:
limits:
memory: 4G
cpus: "2.0"
```
Start with `docker compose up -d` and view logs with `docker compose logs -f`. The supervised gateway's stdout is also tee'd to `${HERMES_HOME}/logs/gateways/<profile>/current` on the volume — see [Where the logs go](#where-the-logs-go) for the full routing map.
## Optional: Linux desktop audio bridge
Voice mode in Docker needs two separate things to work: Hermes must be allowed to probe audio devices inside the container, and the container must be able to reach your host audio server. The setup below covers the host audio plumbing for Linux desktops that expose a PulseAudio-compatible socket, including many PipeWire setups.
:::caution
This is a Linux desktop workaround, not a general Docker Desktop feature. It is useful when you already have host audio working and want CLI voice mode inside the Hermes container. If Hermes still reports `Running inside Docker container -- no audio devices`, use a build that includes Docker audio probing support for `PULSE_SERVER` / `PIPEWIRE_REMOTE`.
:::
First, create an ALSA config next to your Compose file:
```conf title="asound.conf"
pcm.!default {
type pulse
hint {
show on
description "Default ALSA Output (PulseAudio)"
}
}
pcm.pulse {
type pulse
}
ctl.!default {
type pulse
}
```
Then build a small derived image with the ALSA PulseAudio plugin installed:
```dockerfile title="Dockerfile.audio"
FROM nousresearch/hermes-agent:latest
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends libasound2-plugins \
&& rm -rf /var/lib/apt/lists/*
```
Use that image in Compose and pass through the host user's PulseAudio socket and cookie:
```yaml
services:
hermes:
build:
context: .
dockerfile: Dockerfile.audio
image: hermes-agent-audio
container_name: hermes
restart: unless-stopped
command: gateway run
volumes:
- ~/.hermes:/opt/data
- /run/user/${HERMES_UID}/pulse:/run/user/${HERMES_UID}/pulse
- ~/.config/pulse/cookie:/tmp/pulse-cookie:ro
- ./asound.conf:/etc/asound.conf:ro
environment:
- HERMES_UID=${HERMES_UID}
- HERMES_GID=${HERMES_GID}
- XDG_RUNTIME_DIR=/run/user/${HERMES_UID}
- PULSE_SERVER=unix:/run/user/${HERMES_UID}/pulse/native
- PULSE_COOKIE=/tmp/pulse-cookie
```
Start it with your host UID/GID so the container process can access the per-user audio socket:
```sh
export HERMES_UID="$(id -u)"
export HERMES_GID="$(id -g)"
docker compose up -d --build
```
To verify what PortAudio sees inside the container:
```sh
docker exec hermes /opt/hermes/.venv/bin/python -c "import sounddevice as sd; print(sd.query_devices())"
```
## Resource limits
The Hermes container needs moderate resources. Recommended minimums:
| Resource | Minimum | Recommended |
|----------|---------|-------------|
| Memory | 1 GB | 2–4 GB |
| CPU | 1 core | 2 cores |
| Disk (data volume) | 500 MB | 2+ GB (grows with sessions/skills) |
Browser automation (Playwright/Chromium) is the most memory-hungry feature. If you don't need browser tools, 1 GB is sufficient. With browser tools active, allocate at least 2 GB.
Set limits in Docker:
```sh
docker run -d \
--name hermes \
--restart unless-stopped \
--memory=4g --cpus=2 \
-v ~/.hermes:/opt/data \
nousresearch/hermes-agent gateway run
```
## What the Dockerfile does
The official image is based on `debian:13.4` and includes:
- Python 3.13 with dependencies synced from the lockfile via `uv sync --frozen --no-install-project` for the baked extras (`all`, `messaging`, Anthropic/Bedrock/Azure identity, Hindsight, Matrix), followed by a no-dependency editable install of Hermes itself.
- Node.js 26 + npm (for browser automation, WhatsApp bridge, TUI/Desktop bundles, and workspace build tooling)
- Playwright with Chromium (`npx playwright install --with-deps chromium --only-shell`)
- ripgrep, ffmpeg, git, and `xz-utils` as system utilities
- **`docker-cli`** — so agents running inside the container can drive the host's Docker daemon (bind-mount `/var/run/docker.sock` to opt in) for `docker build`, `docker run`, container inspection, etc.
- **`openssh-client`** — enables the [SSH terminal backend](/user-guide/configuration#ssh-backend) from inside the container. The SSH backend shells out to the system `ssh` binary; without this, it failed silently in containerized installs.
- The WhatsApp bridge (`scripts/whatsapp-bridge/`)
- **[`s6-overlay`](https://github.com/just-containers/s6-overlay) v3** as PID 1 (replaces the older `tini`) — supervises the dashboard and per-profile gateways with auto-restart on crash, reaps zombie subprocesses, and forwards signals.
The image treats `/opt/hermes` as an immutable install tree at runtime. Optional Python extras, Node workspaces, and TUI assets that must be available inside Docker need to be baked during the image build; runtime lazy installs are disabled so supervised gateways and `docker exec hermes …` commands do not try to write dependency artifacts back into the read-only source tree.
The container's `ENTRYPOINT` is a small dispatcher (`docker/entrypoint-dispatch.sh`). When the container owns PID 1 (normal Docker / Podman), it exec's s6-overlay's `/init` and you get the full supervision tree described below. When a platform wraps the image entrypoint under its own PID-1 init (Fly.io Machines, `docker run --init`, some Nomad/Kubernetes setups), `/init` would abort with `s6-overlay-suexec: fatal: can only run as pid 1` — so the dispatcher instead runs the stage2 bootstrap directly and exec's the main wrapper without s6. On that fallback path the requested command still runs, but supervised services (dashboard, per-profile gateways) are unavailable.
On the PID-1 path, `/init`:
1. Runs `/etc/cont-init.d/01-hermes-setup` (= `docker/stage2-hook.sh`) as root: optional UID/GID remap, fixes volume ownership, seeds `.env` / `config.yaml` / `SOUL.md` on first boot, runs non-interactive config-schema migrations unless `HERMES_SKIP_CONFIG_MIGRATION=1`, syncs bundled skills.
2. Runs `/etc/cont-init.d/02-reconcile-profiles` (= `hermes_cli.container_boot`): walks `$HERMES_HOME/profiles/<name>/`, recreates the per-profile gateway s6 service slot under `/run/service/gateway-<profile>/`, and auto-starts only those whose last recorded state was `running` (see [Per-profile gateway supervision](#per-profile-gateway-supervision)).
3. Starts the static `main-hermes` and `dashboard` s6-rc services.
4. Exec's the container's CMD as the main program (`/opt/hermes/docker/main-wrapper.sh`), which routes the arguments the user passed to `docker run`:
- no args → `hermes` (the default)
- first arg is an executable on PATH (e.g. `sleep`, `bash`) → exec it directly
- anything else → `hermes <args>` (subcommand passthrough)
The container exits when this main program exits, with its exit code.
:::warning Breaking change vs. pre-s6 images
The container ENTRYPOINT is now the `entrypoint-dispatch.sh` dispatcher (which delegates to s6-overlay's `/init` under PID 1), not `/usr/bin/tini`. All five documented `docker run` invocation patterns (no args, `chat -q "…"`, `sleep infinity`, `bash`, `--tui`) behave identically to the tini-based image. If you have a downstream wrapper that depended on tini-specific signal behavior or hard-coded `/usr/bin/tini --` invocation, pin to the previous image tag.
:::
:::warning Privilege model
Do not override the image entrypoint unless you keep `/init` (or, equivalently, the legacy `docker/entrypoint.sh` shim that forwards to the stage2 hook) in the command chain. s6-overlay's `/init` runs as root so it can chown the volume on first boot, then drops to the `hermes` user via `s6-setuidgid` for every supervised service AND for the main program. Starting `hermes gateway run` as root inside the official image is refused by default because it can leave root-owned files in `/opt/data` and break later dashboard or gateway starts. Set `HERMES_ALLOW_ROOT_GATEWAY=1` only when you intentionally accept that risk.
:::
### `docker exec` automatically drops to the `hermes` user
`docker exec hermes <cmd>` defaults to running as root inside the container, but the image ships a thin shim at `/opt/hermes/bin/hermes` (earliest on PATH) that detects root callers and transparently re-execs through `s6-setuidgid hermes`. So `docker exec hermes login`, `docker exec hermes profile create …`, `docker exec hermes setup`, etc. all write files owned by UID 10000 — i.e. readable by the supervised gateway — with no extra `--user` flag needed. Non-root callers (the supervised processes themselves, `docker exec --user hermes`, kanban subagents inside the container) hit a short-circuit that exec's the venv binary directly, so there's no overhead on the hot paths.
If you specifically need a `docker exec` that retains root semantics (diagnostic sessions, inspecting root-only state, files outside `/opt/data` that root happens to own), opt out per invocation:
```sh
docker exec -e HERMES_DOCKER_EXEC_AS_ROOT=1 hermes <cmd>
```
The shim accepts `1` / `true` / `yes` (case-insensitive). Anything else — including typos like `=0` — falls through to the drop, so silent opt-outs aren't possible. If `s6-setuidgid` isn't available (custom builds that stripped s6-overlay), the shim refuses to run as root and exits 126 instead, surfacing the broken privilege model loudly rather than regressing to the historical footgun where `docker exec hermes login` would write `auth.json` as `root:root` and break the supervised gateway's auth on every chat platform message.
### Per-profile gateway supervision
Each profile created with `hermes profile create <name>` automatically gets an s6-supervised gateway service registered at `/run/service/gateway-<name>/`, with state-persistent auto-restart across container restarts. See [Multi-profile support](#multi-profile-support) above for the user-facing workflow and the lifecycle commands.
**Supervision benefits over the pre-s6 image:**
- Gateway crashes are auto-restarted by `s6-supervise` after a ~1s backoff.
- Dashboard, when enabled with `HERMES_DASHBOARD=1`, is supervised on the same supervision tree and gets the same auto-restart treatment.
- `docker restart`, image upgrades (`docker compose up -d --force-recreate`), and unexpected exits preserve running gateways: the cont-init reconciler reads `$HERMES_HOME/profiles/<name>/gateway_state.json` and brings the slot back up if the last recorded state was `running`. Only an explicit `hermes gateway stop` records `stopped` and keeps the gateway down across the restart; the container/s6 SIGTERM sent on a restart or upgrade is treated as "still running" and auto-starts.
- Per-profile gateway logs persist under `$HERMES_HOME/logs/gateways/<profile>/current` (rotated by `s6-log`), and the reconciler's actions are appended to `$HERMES_HOME/logs/container-boot.log` per boot. See [Where the logs go](#where-the-logs-go) for the full routing map.
`hermes status` inside the container reports `Manager: s6 (container supervisor)`. Use `/command/s6-svstat /run/service/gateway-<name>` for the raw supervisor view (note `/command/` is on PATH for supervision-tree processes only; pass the absolute path when calling from `docker exec`).
## Upgrading
Pull the latest image and recreate the container. Your data directory is
preserved, and the container runs non-interactive config-schema migrations
against the mounted `$HERMES_HOME/config.yaml` before starting the gateway.
When a migration is needed, Hermes writes timestamped backups next to
`config.yaml` and `.env` first.
```sh
docker pull nousresearch/hermes-agent:latest
docker rm -f hermes
docker run -d \
--name hermes \
--restart unless-stopped \
-v ~/.hermes:/opt/data \
nousresearch/hermes-agent gateway run
```
Or with Docker Compose:
```sh
docker compose pull
docker compose up -d
```
Set `HERMES_SKIP_CONFIG_MIGRATION=1` only if you need to inspect or migrate the
persisted config manually before letting the new image rewrite it.
## Skills and credential files
When using Docker as the execution environment (not the methods above, but when the agent runs commands inside a Docker sandbox — see [Configuration → Docker Backend](./configuration.md#docker-backend)), Hermes reuses a single long-lived container for all tool calls and automatically bind-mounts the skills directory (`~/.hermes/skills/`) and any credential files declared by skills into that container as read-only volumes. Skill scripts, templates, and references are available inside the sandbox without manual configuration, and because the container persists for the life of the Hermes process, any dependencies you install or files you write stay around for the next tool call.
The same syncing happens for SSH and Modal backends — skills and credential files are uploaded via rsync or the Modal mount API before each command.
## Installing more tools in the container
The official image ships with a curated set of utilities (see [What the Dockerfile does](#what-the-dockerfile-does)), but not every tool an agent might want is preinstalled. There are five recommended approaches, in increasing order of effort and durability.
### npm or Python tools — use `npx` or `uvx`
For any tool published to npm or PyPI, instruct Hermes to run it via `npx` (npm) or `uvx` (Python) and to remember that command in its persistent memory. If the tool needs a config file or credentials, instruct it to drop those under `/opt/data` (e.g. `/opt/data/<tool>/config.yaml`).
Dependencies are fetched on demand and cached for the life of the container. Configuration written under `/opt/data` survives container restarts because it lives on the bind-mounted host directory. The package cache itself is rebuilt after a `docker rm`, but `npx` and `uvx` re-fetch transparently the next time the tool runs.
### Other tools (apt packages, binaries) — install and remember
For anything outside npm or PyPI — `apt` packages, prebuilt binaries, language runtimes not already in the image — instruct Hermes how to install it (e.g. `apt-get update && apt-get install -y <package>`) and tell it to remember the install command. The tool persists for the rest of the container's lifetime, and Hermes will re-run the install command after a container restart when it next needs the tool.
This is a good fit for tools that are quick to install and used occasionally. For tools used constantly, prefer the next approach.
### Durable installs — build a derived image
When a tool must be available immediately on every container start with no re-install delay, build a new image that inherits from `nousresearch/hermes-agent` and installs the tool in a layer:
```dockerfile
FROM nousresearch/hermes-agent:latest
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends <your-package> \
&& rm -rf /var/lib/apt/lists/*
USER hermes
```
Build it and use it in place of the official image:
```sh
docker build -t my-hermes:latest .
docker run -d \
--name hermes \
--restart unless-stopped \
-v ~/.hermes:/opt/data \
-p 8642:8642 \
my-hermes:latest gateway run
```
The entrypoint script and `/opt/data` semantics are inherited unchanged, so the rest of this page still applies. Remember to rebuild the image when pulling a newer upstream `nousresearch/hermes-agent`.
### Complex tools or multi-service stacks — run a sidecar container
For tools that bring their own service (a database, a web server, a queue, a headless browser farm) or that are too heavy to live inside the Hermes container, run them as a separate container on a shared Docker network. Hermes reaches the sidecar by container name, the same way it reaches a local inference server (see [Connecting to local inference servers](#connecting-to-local-inference-servers-vllm-ollama-etc)).
```yaml
services:
hermes:
image: nousresearch/hermes-agent:latest
container_name: hermes
restart: unless-stopped
command: gateway run
ports:
- "8642:8642"
volumes:
- ~/.hermes:/opt/data
networks:
- hermes-net
my-tool:
image: example/my-tool:latest
container_name: my-tool
restart: unless-stopped
networks:
- hermes-net
networks:
hermes-net:
driver: bridge
```
From inside the Hermes container, the sidecar is reachable at `http://my-tool:<port>` (or whatever protocol it serves). This pattern keeps each service's lifecycle, resource limits, and upgrade cadence independent, and avoids bloating the Hermes image with dependencies that are only needed by one tool.
### Broadly useful tools — open an issue or pull request
If a tool is likely to be useful to most Hermes Agent users, consider contributing it upstream rather than carrying it in a private derived image. Open an issue or pull request on the [hermes-agent repository](https://github.com/NousResearch/hermes-agent) describing the tool and its use case. Tools that get bundled into the official image benefit every user and avoid the maintenance overhead of a downstream fork.
## Connecting to local inference servers (vLLM, Ollama, etc.)
When running Hermes in Docker and your inference server (vLLM, Ollama, text-generation-inference, etc.) is also running on the host or in another container, networking requires extra attention.
### Docker Compose (recommended)
Put both services on the same Docker network. This is the most reliable approach:
```yaml
services:
vllm:
image: vllm/vllm-openai:latest
container_name: vllm
command: >
--model Qwen/Qwen2.5-7B-Instruct
--served-model-name my-model
--host 0.0.0.0
--port 8000
ports:
- "8000:8000"
networks:
- hermes-net
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
hermes:
image: nousresearch/hermes-agent:latest
container_name: hermes
restart: unless-stopped
command: gateway run
ports:
- "8642:8642"
volumes:
- ~/.hermes:/opt/data
networks:
- hermes-net
networks:
hermes-net:
driver: bridge
```
Then in your `~/.hermes/config.yaml`, use the **container name** as the hostname:
```yaml
model:
provider: custom
model: my-model
base_url: http://vllm:8000/v1
api_key: "none"
```
:::tip Key points
- Use the **container name** (`vllm`) as the hostname — not `localhost` or `127.0.0.1`, which refer to the Hermes container itself.
- The `model` value must match the `--served-model-name` you passed to vLLM.
- Set `api_key` to any non-empty string (vLLM requires the header but doesn't validate it by default).
- Do **not** include a trailing slash in `base_url`.
:::
### Standalone Docker run (no Compose)
If your inference server runs directly on the host (not in Docker), use `host.docker.internal` on macOS/Windows, or `--network host` on Linux:
**macOS / Windows:**
```sh
docker run -d \
--name hermes \
-v ~/.hermes:/opt/data \
-p 8642:8642 \
nousresearch/hermes-agent gateway run
```
```yaml
# config.yaml
model:
provider: custom
model: my-model
base_url: http://host.docker.internal:8000/v1
api_key: "none"
```
**Linux (host networking):**
```sh
docker run -d \
--name hermes \
--network host \
-v ~/.hermes:/opt/data \
nousresearch/hermes-agent gateway run
```
```yaml
# config.yaml
model:
provider: custom
model: my-model
base_url: http://127.0.0.1:8000/v1
api_key: "none"
```
:::warning With `--network host`, the `-p` flag is ignored — all container ports are directly exposed on the host.
:::
### Verifying connectivity
From inside the Hermes container, confirm the inference server is reachable:
```sh
docker exec hermes curl -s http://vllm:8000/v1/models
```
You should see a JSON response listing your served model. If this fails, check:
1. Both containers are on the same Docker network (`docker network inspect hermes-net`)
2. The inference server is listening on `0.0.0.0`, not `127.0.0.1`
3. The port number matches
### Ollama
Ollama works the same way. If Ollama runs on the host, use `host.docker.internal:11434` (macOS/Windows) or `127.0.0.1:11434` (Linux with `--network host`). If Ollama runs in its own container on the same Docker network:
```yaml
model:
provider: custom
model: llama3
base_url: http://ollama:11434/v1
api_key: "none"
```
## Troubleshooting
### Container exits immediately
Check logs: `docker logs hermes`. Common causes:
- Missing or invalid `.env` file — run interactively first to complete setup
- Port conflicts if running with exposed ports
### "Permission denied" errors
The container's stage2 hook drops privileges to the non-root `hermes` user (UID 10000) via `s6-setuidgid` inside each supervised service. If your host `~/.hermes/` is owned by a different UID, set `HERMES_UID`/`HERMES_GID` — or their `PUID`/`PGID` aliases, for parity with LinuxServer.io and NAS images — to match your host user, or ensure the data directory is writable:
```sh
chmod -R 755 ~/.hermes
```
On a NAS (UGOS, Synology, unRAID) the data directory is typically a **bind mount** owned by a host UID the container cannot `chown`. Set `PUID`/`PGID` (or `HERMES_UID`/`HERMES_GID`) to that host user so the runtime runs as the owner of the mount rather than UID 10000:
```sh
docker run -d \
--name hermes \
-e PUID=1000 -e PGID=10 \
-v /volume1/docker/hermes:/opt/data \
nousresearch/hermes-agent gateway run
```
`docker exec hermes <cmd>` automatically drops to UID 10000 too — see [`docker exec` automatically drops to the `hermes` user](#docker-exec-automatically-drops-to-the-hermes-user) for details and the per-invocation opt-out.
### "Permission denied" on every `docker exec` (install dir locked to 0700)
Images built before late August 2026 had a bug where writing a credential file directly under `/opt/hermes` restricted that directory to `0700`, locking the `hermes` user (UID 10000) out of the install tree. Every new `docker exec` then fails with `Permission denied`.
Pulling a newer image and recreating the container fixes it permanently (the install dir ships as `0755` and current releases no longer restrict it). If you need to recover a running container in place without recreating it:
```sh
docker exec -u root hermes chmod 0755 /opt/hermes
```
### Browser tools not working
Playwright needs shared memory. Add `--shm-size=1g` to your Docker run command:
```sh
docker run -d \
--name hermes \
--shm-size=1g \
-v ~/.hermes:/opt/data \
nousresearch/hermes-agent gateway run
```
### Gateway not reconnecting after network issues
The `--restart unless-stopped` flag handles most transient failures. If the gateway is stuck, restart the container:
```sh
docker restart hermes
```
### Checking container health
```sh
docker logs --tail 50 hermes # Recent logs
docker run -it --rm nousresearch/hermes-agent:latest version # Verify version
docker stats hermes # Resource usage
```
website/docs/user-guide/egress/index.md
+10
View File
@@ -0,0 +1,10 @@
---
title: Egress proxy
sidebar_position: 1
---
# Egress proxy
Optional outbound credential-injection firewall for remote terminal sandboxes. The sandbox only ever holds opaque proxy tokens; real API keys never leave the host.
- [iron-proxy](./iron-proxy) — single-binary TLS-intercepting proxy from [ironsh/iron-proxy](https://github.com/ironsh/iron-proxy), lazy-installed and managed by `hermes egress`.
website/docs/user-guide/egress/iron-proxy.md
+570
View File
@@ -0,0 +1,570 @@
# Egress credential-injection proxy (iron-proxy)
When Hermes runs your agent inside a Docker terminal sandbox, that sandbox normally holds your real upstream API keys (`OPENROUTER_API_KEY`, `OPENAI_API_KEY`, etc.). A prompt-injected agent in that sandbox can `cat ~/.config/openrouter/auth.json` or `printenv | grep -i key` and exfiltrate them.
The egress proxy fixes this: the sandbox holds opaque **proxy tokens**, never the real keys. All outbound traffic from the sandbox routes through a local [iron-proxy](https://github.com/ironsh/iron-proxy) daemon (Apache-2.0, Go) on the host, which terminates TLS and swaps the proxy token for the real credential before forwarding the request upstream. Compromise the sandbox and the attacker walks away with tokens that only work behind the **configured trusted proxy boundary** — the CA private key and the proxy endpoint integrity are part of that boundary. If traffic can be redirected to attacker-controlled proxy infrastructure (e.g. a stolen CA private key or a hijacked proxy endpoint), the token guarantee no longer holds.
This release wires the egress proxy into the Docker backend only. Modal, Daytona, SSH, and Singularity do **not** receive proxy env vars or CA mounts yet.
## What it is
- A managed `iron-proxy` subprocess on the host, lazy-installed into `~/.hermes/bin/iron-proxy`
- A local CA at `~/.hermes/proxy/ca.crt` that the sandbox trusts so iron-proxy can MITM TLS and rewrite headers
- A `proxy.yaml` config at `~/.hermes/proxy/proxy.yaml` listing the upstream hosts you allow and the secrets-transform mapping
- A `mappings.json` recording which proxy token corresponds to which real env var
The sandbox gets `HTTPS_PROXY=http://host.docker.internal:9090`, `HTTP_PROXY=http://host.docker.internal:9091`, and standard provider env vars such as `OPENROUTER_API_KEY` set to opaque proxy tokens. Matching `HERMES_PROXY_TOKEN_<ENV_NAME>` aliases are also exported for diagnostics. Existing provider SDKs read the usual env names, send the proxy token in `Authorization`, and iron-proxy's `secrets` transform substitutes the real value sourced from the host-side daemon environment.
## What it is not
- It is **not** the inbound `hermes proxy` command, which is an OAuth aggregator reverse proxy. Different command (`hermes egress`), different direction.
- It does **not** sit between your local terminal and providers — only between the sandbox and providers.
- It does **not** rewrite credentials for in-process LLM calls the host process makes. Those continue to use your `.env` keys directly. The threat model is the *sandbox*, not the host.
## Quick start
```bash
# 1. Install the iron-proxy binary (pinned version, SHA-256 verified)
hermes egress install
# 2. Run the wizard: generates CA, mints proxy tokens for every provider key
# in your env, writes proxy.yaml.
hermes egress setup
# 3. Start the proxy daemon
hermes egress start
# 4. Check status
hermes egress status
```
`hermes egress setup` discovers provider keys from your environment. If your keys live only in `~/.hermes/.env` (not exported into your shell), setup reads that file automatically — you don't have to `export` them first.
When you re-run `setup` later (new allowlist host, rotated tokens, switched credential source), it stops the running daemon because its config is held in memory, then **offers to restart it for you** so the change takes effect immediately. On a tty it asks; pass `--restart` to always restart or `--no-restart` to leave it down. To apply changes any other time, `hermes egress restart` is the one-command stop-then-start.
Once running, the Docker terminal backend automatically:
- Mounts `~/.hermes/proxy/ca.crt` into the sandbox at `/etc/ssl/certs/hermes-egress-ca.crt`
- Sets `HTTPS_PROXY`, `HTTP_PROXY`, `REQUESTS_CA_BUNDLE`, `SSL_CERT_FILE`, `CURL_CA_BUNDLE`, `NODE_EXTRA_CA_CERTS` to make every common HTTP runtime route through the proxy and trust the CA
- Sets `NODE_OPTIONS=--use-openssl-ca` (appended to whatever you already have in `docker_env.NODE_OPTIONS`) so Node.js routes through the OpenSSL store the other CA-bundle vars control — see [Node.js asymmetric CA caveat](#nodejs-asymmetric-ca-caveat) below for the residual gap
- Adds `--add-host=host.docker.internal:host-gateway` so the sandbox can reach the host-side proxy on Linux (Docker Desktop handles this automatically on macOS/Windows)
- Exports the proxy token under the standard provider env name (for example `OPENROUTER_API_KEY`) plus one `HERMES_PROXY_TOKEN_<ENV_NAME>` diagnostic alias per minted mapping
## Configuration
The full config lives in `~/.hermes/config.yaml` under the `proxy:` section. Defaults are documented inline; everything is optional.
```yaml
proxy:
# Master switch. When false the feature is a complete no-op — no
# binaries downloaded, no docker mounts added, no subprocess started.
enabled: false
# Tunnel listener port. Sandboxes hit http://host.docker.internal:<port>.
tunnel_port: 9090
# Auto-download the pinned iron-proxy binary on first use.
auto_install: true
# Where iron-proxy looks up the real upstream secrets at egress time.
# env — process env (default). Whatever is in your ~/.hermes/.env
# at proxy-start time is the source of truth.
# bitwarden — refetch from Bitwarden Secrets Manager on each proxy
# restart. Rotation in the BW web app propagates without
# touching .env. Requires `secrets.bitwarden.enabled: true`.
credential_source: env
# When true (default), the Docker backend refuses to start a sandbox if
# the proxy is enabled but not running. Set to false to fall back to the
# legacy "real credentials inside the sandbox" posture when the proxy
# is unavailable.
enforce_on_docker: true
# When `credential_source: bitwarden` but the BWS access token /
# project_id is missing OR the bws fetch returns no values for mapped
# providers, the daemon raises by default (matches the spirit of "I
# asked for rotation — don't silently use stale env values"). Set
# to true to opt back into the legacy host-env fallback — useful for
# migrations where you want to start switching to BW mode but haven't
# wired every secret yet.
allow_env_fallback: false
# SSRF deny list applied to outbound traffic. Omit / leave null to
# use the safe default: loopback (v4 + v6), link-local (incl. cloud
# metadata IPs at 169.254.169.254), RFC1918, IPv6 ULA, IPv4-mapped-v6,
# CGNAT, and the RFC2544 benchmark range. Set to an explicit `[]`
# to opt out entirely (only sensible in hermetic tests).
upstream_deny_cidrs: null
# Extra allowed upstream hosts beyond the bundled defaults.
# Wildcards (`*.foo.com`) are supported. The defaults cover OpenRouter,
# OpenAI, Anthropic, Google, xAI, Mistral, Groq, Together, DeepSeek,
# and Nous Research.
extra_allowed_hosts: []
```
### Default allowed upstream hosts
```
openrouter.ai *.openrouter.ai
api.openai.com api.anthropic.com
generativelanguage.googleapis.com
api.x.ai api.mistral.ai
api.groq.com api.together.xyz
api.deepseek.com inference.nousresearch.com
```
If your agent needs an upstream that isn't on the list — a self-hosted inference endpoint, an extra cloud LLM, an MCP server — add it to `proxy.extra_allowed_hosts`. Wildcards are matched against the full hostname (`*.example.com` matches `api.example.com` and `staging.example.com` but not `example.com` itself).
### Default SSRF deny CIDRs
Applied regardless of allowlist. These ranges are refused by iron-proxy at the network boundary, so a DNS rebinding attack via an allowlisted hostname can't reach IMDS or your internal network:
| CIDR | Purpose |
|---|---|
| `127.0.0.0/8`, `::1/128` | Loopback (v4 + v6) |
| `169.254.0.0/16`, `fe80::/10` | Link-local — **incl. AWS / GCP / Azure IMDS at `169.254.169.254`** |
| `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16` | RFC1918 |
| `fc00::/7` | IPv6 ULA |
| `::ffff:0:0/96` | IPv4-mapped IPv6 — closes the dual-stack IMDS bypass |
| `100.64.0.0/10` | RFC6598 CGNAT (used by AWS VPC, K8s pod networks) |
| `198.18.0.0/15` | RFC2544 benchmark range |
To override: set `proxy.upstream_deny_cidrs` to your own list. To opt out entirely (e.g. for a hermetic test that needs to reach a loopback upstream): set it to an empty list `[]`.
### Bind policy
The proxy never binds `0.0.0.0`. The default bind is platform-specific because iron-proxy v0.39 supports only a **single bind per daemon process**:
- **Linux:** the docker bridge gateway (`172.17.0.1:<tunnel_port>` by default). Containers reach the proxy via `host.docker.internal`, which `--add-host=host.docker.internal:host-gateway` resolves to exactly this bridge gateway IP — a loopback-only bind would be unreachable from inside sandboxes. The bridge IP is an address on the host's `docker0` interface, so it is not exposed to the LAN; it IS reachable by other containers on the default bridge network, but requests still require a minted proxy token and an allowlisted upstream. If no docker bridge is detected (docker not installed/running), the bind falls back to loopback with a warning.
- **macOS / Windows Docker Desktop:** loopback (`127.0.0.1:<tunnel_port>`). Desktop's VPNkit routes `host.docker.internal` to the host, so loopback is reachable from containers and is the least-exposed choice.
A LAN peer with a leaked proxy token cannot use the proxy — neither bind is reachable from the external network.
We also pin `metrics.listen: 127.0.0.1:0` so the daemon's built-in metrics server gets an ephemeral loopback port instead of its default `:9090` — otherwise it would fight `tunnel_port: 9090` for the same socket and the daemon would refuse to start with "address already in use". Note the `:0` ephemeral port is random per start and not surfaced anywhere, so metrics are effectively disabled at this pin.
If a hostile `ip` shim earlier on PATH had been able to inject a non-private IPv4 as the bridge address (`0.0.0.0`, a public address, multicast, link-local, etc.) the loopback fallback still applies — we never bind anything we couldn't validate via `ipaddress.IPv4Address` + `is_*` checks.
## Covered auth schemes
The `secrets` transform swaps the proxy token wherever it appears in a matched location — and it matches more than `Authorization: Bearer`:
| Provider | Env var | Swapped in |
|---|---|---|
| OpenRouter, OpenAI, Groq, Together, DeepSeek, Mistral, xAI, Nous | `*_API_KEY` | `Authorization` header |
| Anthropic native | `ANTHROPIC_API_KEY` | `x-api-key` + `Authorization` |
| Azure OpenAI | `AZURE_OPENAI_API_KEY` | `api-key` + `Authorization` (`*.openai.azure.com`, `*.cognitiveservices.azure.com`, `*.services.ai.azure.com`) |
| Google AI Studio (Gemini) | `GEMINI_API_KEY` / `GOOGLE_API_KEY` | `x-goog-api-key` header or `?key=` query param |
`GEMINI_API_KEY` and `GOOGLE_API_KEY` are treated as one credential: a single proxy token is minted and injected into the sandbox under **both** names, and either name in your host env satisfies discovery.
## Uncovered providers
Auth schemes that involve request signing or SDK-minted OAuth cannot be swapped by a static header replacement — if their env vars are present, the sandbox holds **real credentials** for those providers and the egress isolation guarantee is incomplete for them:
| Env var | Provider | Reason |
|---|---|---|
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | AWS Bedrock / SageMaker | SigV4-signed requests |
| `GOOGLE_APPLICATION_CREDENTIALS` | GCP Vertex AI | OAuth minted from a service-account file |
These env vars are present on most developer laptops for unrelated tooling (terraform, gcloud, aws CLI, ECR push). They surface as warnings in the wizard and `hermes egress status` but never block the proxy from starting. If you don't use those providers from sandboxes, `unset` the vars to clear the warning.
## Bitwarden integration
If you already use Bitwarden Secrets Manager via [`hermes secrets bitwarden setup`](../secrets/bitwarden), the egress proxy can pull real credentials from there instead of `os.environ`:
```bash
hermes egress setup --from-bitwarden
```
This sets `proxy.credential_source: bitwarden` and discovers provider env names from your BW project.
### Rotation semantics
When `credential_source: bitwarden`, the iron-proxy daemon refetches secrets from BWS via `bws secret list <project_id>` **every time it starts**. So the rotation flow is:
1. Rotate a key in the Bitwarden web app.
2. `hermes egress stop && hermes egress start` on the host.
3. Sandboxes started after that point swap proxy tokens for the new value.
No `.env` edits. No Hermes restart on the host. The proxy daemon is the only thing that touches the new value — your host process and `os.environ` are untouched.
### Fail-loud at start
When `credential_source: bitwarden`, `hermes egress start` pre-checks at the wizard layer AND `_build_proxy_subprocess_env` re-checks at the daemon layer:
- BWS access token env var is unset → refuse to start with a hint to `unset` and re-run, or `hermes egress setup --no-bitwarden` to switch back to env mode
- `secrets.bitwarden.project_id` is empty → refuse to start with a hint to run `hermes secrets bitwarden setup`
- `bws secret list` returns no values for one or more mapped providers → refuse to start, listing the missing names
This is intentional. Falling back to host env in BW mode reintroduces exactly the staleness bug the BW path is meant to defeat (operator picked BW for the rotation guarantee; silent fallback breaks that guarantee).
The `proxy.allow_env_fallback: true` config flag opts back in to the legacy "silently fall back to host env if BWS is unreachable" behavior for migration scenarios. Use it when you're moving secrets into BW one at a time and want the daemon to start with whichever values are available.
### Switching credential source
| From | To | Command |
|---|---|---|
| env | bitwarden | `hermes egress setup --from-bitwarden` |
| bitwarden | env | `hermes egress setup --no-bitwarden` |
**Re-running `hermes egress setup` WITHOUT either flag preserves the existing `credential_source`** — the wizard refuses to silently downgrade you back to env. This matters because once you've configured bitwarden mode, the rotation guarantee is what you signed up for; you have to explicitly say "I want env again" to change it.
## Slash commands
The CLI subcommand tree:
```
hermes egress install # download the pinned iron-proxy binary
hermes egress install --force # re-download even if a managed copy exists
hermes egress setup # interactive wizard
hermes egress setup --tunnel-port N # override the tunnel listener port
hermes egress setup --from-bitwarden # use BWS as credential source (fail-loud)
hermes egress setup --no-bitwarden # explicitly switch back to env mode
hermes egress setup --rotate-tokens # mint fresh tokens for every provider
# (default preserves existing)
hermes egress start # spawn the managed proxy daemon
hermes egress stop # SIGTERM (then SIGKILL after 5s grace)
hermes egress restart # stop (if running) then start — needed when
# upstream SECRETS change (rotation, new provider)
hermes egress reload # hot-reload the ruleset from proxy.yaml via the
# management API — no restart, no dropped
# connections (allowlist / mapping edits)
hermes egress status # binary + config + pid + listening state + mappings
hermes egress status --show-tokens # print proxy tokens in full
# (default: redacted prefix + suffix only)
hermes egress disable # flip proxy.enabled = false
# (does not stop a running proxy)
hermes egress config # print the path to proxy.yaml for debugging
```
### Token rotation
By default, `hermes egress setup` **preserves** proxy tokens for providers that already have them. Adding a new provider mints a fresh token only for the new one; existing tokens are unchanged. This avoids 401-ing running sandboxes when you re-run the wizard.
`--rotate-tokens` rolls every token:
```bash
hermes egress setup --rotate-tokens
```
When there are existing tokens AND stdin is a tty, the wizard prompts for confirmation:
```
⚠ --rotate-tokens will invalidate proxy tokens in every running
Hermes sandbox. They will start 401-ing against upstreams until restarted.
Type 'rotate' to confirm:
```
Non-tty invocations (CI, scripts) skip the prompt — the flag is treated as deliberate. Before any overwrite the current `mappings.json` is copied to a timestamped sibling so manual recovery is possible:
```
backup: ~/.hermes/proxy/mappings.json.rotated-20260524T143012
```
`hermes egress setup` stops a running daemon when it rewrites config or token mappings, because the daemon keeps the old YAML in memory. After `--rotate-tokens`:
```bash
hermes egress start
```
Containers already running hold the old tokens and will need to be restarted to pick up the new ones. New persistent Docker containers include an egress-posture label, so Hermes will not reuse a pre-egress or pre-rotation container for new sessions.
## State directory layout
Everything iron-proxy maintains lives in `~/.hermes/proxy/`:
| Path | Mode | Purpose |
|---|---|---|
| `~/.hermes/proxy/` (dir) | `0o700` | Owned + traversable by you only |
| `ca.crt` | `0o644` | Public CA cert distributed into sandboxes |
| `ca.key` | `0o600` | CA signing key — never leaves the host |
| `proxy.yaml` | `0o600` | iron-proxy config; rewritten every `setup` |
| `mappings.json` | `0o600` | Sandbox proxy token → upstream env var |
| `mappings.json.rotated-*` | `0o600` | Backups created by `--rotate-tokens` |
| `iron-proxy.pid` | `0o600` | PID of the running daemon |
| `iron-proxy.nonce` | `0o600` | Per-start nonce for PID-recycle defense |
| `iron-proxy.log` | `0o600` | Daemon stdout/stderr — **includes per-request records on v0.39** |
| `audit.log` | `0o600` | Reserved for the dedicated per-request audit stream on future binary versions; pre-created so the privacy contract holds when upstream wires it in |
The CA private key is the most sensitive file. It's created with `0o600` from the first byte (no umask-window TOCTOU) and `O_NOFOLLOW` so a same-uid attacker can't redirect it via a planted symlink. The pidfile, nonce file, daemon log, and audit log get the same treatment.
### Logging on iron-proxy v0.39
On the currently pinned binary version (**v0.39.0**) iron-proxy writes ALL output — daemon-level diagnostics AND per-request records — to **`~/.hermes/proxy/iron-proxy.log`**. v0.39's `config.Log` struct doesn't have a separate `audit_path` field, so we can't route per-request records to a dedicated stream there.
We still pre-create `~/.hermes/proxy/audit.log` at `0o600` with `O_NOFOLLOW` because:
1. It reserves the path for the future version bump: when the pinned version moves to one that supports `log.audit_path`, per-request records will start flowing there without operator-side reconfiguration. **Until then the file stays at 0 bytes — do not point monitoring, alerting, or forensics tooling at it yet.** Use `iron-proxy.log` for everything today.
2. The 0o600-from-first-byte guarantee defends against the upstream-fix-day where v0.40+ creates the file under its default umask if it doesn't already exist.
Until that version bump lands, treat `iron-proxy.log` as the source of truth for both audiences:
- Daemon-level events (startup banner, bind errors, shutdown reason, transform errors). Operations + troubleshooting.
- Per-request records (CONNECT to allowlisted upstream, secret swap fired, allowlist denial). Forensics + compliance.
Both files are appended to across restarts. Rotate them with logrotate if you care about disk usage on long-lived hosts.
## How it works
```
┌──────────────┐ ┌──────────────┐ ┌─────────────┐
│ Docker │ CONNECT / │ iron-proxy │ HTTPS w/ │ OpenRouter │
│ sandbox ├──────────────▶│ (host:9090) ├───────────────▶│ / OpenAI / │
│ │ HTTP forward │ │ real API key │ Anthropic … │
│ has: │ w/ proxy tok │ mints leaf │ │ │
│ - proxy tok │ in Auth hdr │ cert from CA │ │ │
│ - CA cert │ │ matches token │ │ │
│ - HTTPS_PROXY│ │ swaps secret │ │ │
└──────────────┘ └──────────────┘ └─────────────┘
│
│ daemon + per-request log (combined on v0.39)
▼
~/.hermes/proxy/iron-proxy.log
(~/.hermes/proxy/audit.log reserved for v0.40+ split stream)
```
1. Sandbox makes an HTTPS request, e.g. `POST https://openrouter.ai/v1/chat/completions` with `Authorization: Bearer hermes-proxy-openrouter-…` (the proxy token, not the real key).
2. Because `HTTPS_PROXY` is set, the request goes to iron-proxy as a CONNECT tunnel.
3. iron-proxy checks the allowlist. `openrouter.ai` is allowed.
4. iron-proxy mints a leaf cert signed by our CA for `openrouter.ai`, terminates the TLS connection, inspects the request.
5. The `secrets` transform matches the proxy-token string in the `Authorization` header and substitutes the real `OPENROUTER_API_KEY` value, sourced from iron-proxy's own environment.
6. Request is re-encrypted and forwarded to OpenRouter.
7. The request is logged to `~/.hermes/proxy/iron-proxy.log` on v0.39. When the pinned binary version supports the split stream (v0.40+), per-request records will flow to `~/.hermes/proxy/audit.log` and daemon-level diagnostics will stay in `iron-proxy.log`. See [Logging on iron-proxy v0.39](#logging-on-iron-proxy-v039).
A request to a non-allowlisted host (e.g. `https://attacker.example.com/leak?key=...`) is rejected with HTTP 403 before any bytes leave the host. The denial is recorded in `iron-proxy.log` with the upstream host and the source sandbox.
### CA distribution into the sandbox
When the Docker backend starts a container with `proxy.enabled: true` and the daemon is listening, it adds these arguments to `docker run`:
| Arg | Purpose |
|---|---|
| `-v ~/.hermes/proxy/ca.crt:/etc/ssl/certs/hermes-egress-ca.crt:ro` | Read-only mount of the CA |
| `-e HTTPS_PROXY=http://host.docker.internal:9090` | Python httpx / curl / go default transport / Node fetch |
| `-e HTTP_PROXY=http://host.docker.internal:9091` | curl + wget for plain HTTP — the plain-HTTP forward listener lives on `tunnel_port + 1` |
| `-e NO_PROXY=127.0.0.1,localhost,::1` | Loopback dev servers inside the sandbox bypass the proxy |
| `-e REQUESTS_CA_BUNDLE=…ca.crt` | Python `requests` |
| `-e SSL_CERT_FILE=…ca.crt` | Python `ssl` module / OpenSSL — **replaces** the system store |
| `-e CURL_CA_BUNDLE=…ca.crt` | curl — **replaces** the system store |
| `-e NODE_EXTRA_CA_CERTS=…ca.crt` | Node.js — **adds** to the system store |
| `-e NODE_OPTIONS="<your value> --use-openssl-ca"` | Node.js — route through OpenSSL store (appended; your `--max-old-space-size` etc. are preserved) |
| `-e HERMES_EGRESS_PROXY=1` | Sentinel the agent can read to know it's proxy-aware |
| `-e OPENROUTER_API_KEY=<proxy-token>` | Standard provider env names receive proxy tokens so existing SDKs keep working |
| `-e HERMES_PROXY_TOKEN_<NAME>=…` | Diagnostic alias for each mapping; same value as the standard provider env var |
| `--add-host=host.docker.internal:host-gateway` | Linux-only; Docker Desktop maps it automatically |
#### Node.js asymmetric CA caveat
`REQUESTS_CA_BUNDLE` / `SSL_CERT_FILE` / `CURL_CA_BUNDLE` **replace** the system CA store inside the sandbox. `NODE_EXTRA_CA_CERTS` **adds** to it. A Node.js process inside the sandbox could in principle bypass the proxy by opening a raw `net.Socket` and starting its own TLS handshake — the system CA store would still trust real upstream certs, so the request would succeed where Python / curl would fail validation.
`NODE_OPTIONS=--use-openssl-ca` is appended to whatever you already have in `docker_env.NODE_OPTIONS`. This forces Node through the OpenSSL store that `SSL_CERT_FILE` controls, narrowing the asymmetry. It does NOT cover code that explicitly passes its own `ca` option to `tls.connect()` or `https.request()`, but it closes the easy case.
This is a known v1 limitation. Track [github.com/ironsh/iron-proxy/issues](https://github.com/ironsh/iron-proxy/issues) for an upstream resolution; in the meantime, do not run untrusted Node code that opens raw sockets in a sandbox you're depending on egress isolation for.
### docker\_env collisions
If you set proxy-controlling env vars in your `docker_env:` config block (rare but possible), Hermes refuses to start the sandbox when `enforce_on_docker: true` is set. This includes both:
- Egress-control vars: `HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `REQUESTS_CA_BUNDLE`, `SSL_CERT_FILE`, `CURL_CA_BUNDLE`, `NODE_EXTRA_CA_CERTS`
- Real provider env vars: every name in `mappings.json` (e.g. `OPENROUTER_API_KEY`, `OPENAI_API_KEY`)
Example error:
```
docker_env in config.yaml overrides egress-proxy variables
['HTTPS_PROXY', 'OPENROUTER_API_KEY']; enforce_on_docker is enabled.
Remove these keys from docker_env or disable enforce_on_docker to
opt out of egress isolation.
```
With `enforce_on_docker: false` the same situation surfaces as a warning and your `docker_env` values win — useful for migrations or testing, but you're explicitly opting OUT of the isolation guarantee.
## PID and nonce defense
The daemon's pidfile is written with `O_EXCL` + `O_NOFOLLOW` + ownership check. Concurrent `hermes egress start` calls produce one of two outcomes:
- The existing pidfile points at a live iron-proxy → second start refuses with "another start in progress" + a hint to run `hermes egress stop`
- The existing pidfile is stale (crashed daemon) → second start unlinks it and retries once
Beyond that, every `start_proxy` plants a fresh random nonce in two places:
- `HERMES_IRON_PROXY_NONCE=<nonce>` in the daemon's env
- `~/.hermes/proxy/iron-proxy.nonce` (0o600 sibling of the pidfile)
When `hermes egress stop` (or any other `_pid_alive` check) wants to confirm a PID still refers to *our* daemon — not an unrelated process that was assigned the same PID after iron-proxy crashed — it reads `/proc/<pid>/environ` and looks for the nonce. The on-disk copy is what makes this work across CLI invocations (the in-memory `_proxy_nonce` is per-process and resets on every `hermes` invocation).
If the nonce check fails, the code falls back to matching `argv[0]` basename against `iron-proxy`. `stop_proxy` additionally captures `/proc/<pid>/stat` starttime before SIGTERM and re-verifies after the 5s grace window — if starttime drifted, the PID was recycled mid-wait and SIGKILL is suppressed with a warning.
## Security model
**What this protects against:**
- Prompt-injected agent in a Docker sandbox reading `printenv` / credential files and exfiltrating real keys.
- Compromised dependency in the sandbox phoning home to an arbitrary host — default-deny allowlist blocks unknown destinations.
- Agent dialing cloud metadata endpoints (`169.254.169.254`) — iron-proxy denies these by default via `upstream_deny_cidrs`, including the IPv4-mapped-v6 form `::ffff:169.254.169.254`.
- DNS rebinding through an allowlisted hostname to a private IP — the deny CIDRs are checked at connect time, not at allowlist time.
- Same-uid local processes reading the iron-proxy daemon's env to scrape secrets — only the env var names referenced by mappings are forwarded, not the full host env.
- A LAN peer with a leaked sandbox proxy token spending your API quota — the proxy binds the docker bridge gateway (Linux) or loopback (Docker Desktop), never `0.0.0.0`, so it is unreachable from the external network.
**What it does NOT protect against:**
- A compromised host process. If the agent process itself is compromised, real keys in the host's `~/.hermes/.env` are exposed regardless. This is a defense-in-depth feature for *sandbox* compromise, not host compromise.
- **Loss of the trusted-proxy boundary itself.** The token-swap guarantee assumes the sandbox trusts the mounted CA cert (`/etc/ssl/certs/hermes-egress-ca.crt`) and that traffic actually reaches *our* iron-proxy. If the CA private key is stolen, or sandbox egress is redirected to attacker-controlled proxy infrastructure, an adversary-in-the-middle can present a valid leaf cert and the proxy tokens are no longer a meaningful boundary (cf. [MITRE ATT&CK T1588.004](https://attack.mitre.org/techniques/T1588/004/) — obtained TLS certificate material enabling AiTM). Protect the CA key (it's `0600`, host-only) and the proxy endpoint accordingly.
- Sandbox processes that bypass `HTTPS_PROXY` by using a raw socket. The proxy can't intercept what doesn't route to it. Node.js is partially mitigated via `NODE_OPTIONS=--use-openssl-ca` (see caveat above).
- Credential files explicitly mounted into Docker (`terminal.credential_files` or skill-registered mounts). Egress protects provider env vars; it does not inspect arbitrary mounted files. Do not mount real provider credentials into an enforced egress sandbox.
- Allowlisted-host data exfiltration. If `api.openai.com` is allowed, an agent could embed exfil data in a request body to that host. The daemon log captures the request happened but doesn't prevent it.
- Uncovered providers (AWS Bedrock SigV4, GCP Vertex service-account OAuth). Their env vars stay in the sandbox; if you enable them, those credentials bypass the proxy entirely. See [Uncovered providers](#uncovered-providers).
- iron-proxy in-memory secret zeroisation. The Go binary holds swapped-in real credentials in process memory; a core-dump or `/proc/<pid>/mem` read from a same-uid attacker would expose them. Out of scope for this layer.
## Failure modes
- **Binary not installed, `auto_install: true`** — first `hermes egress setup` or `hermes egress start` downloads it. SHA-256 verified against the upstream `checksums.txt`.
- **Binary not installed, `auto_install: false`** — `start` fails with a clear message pointing to manual install.
- **`enabled: true` but proxy not running** — with `enforce_on_docker: true` (default), Docker sandbox creation refuses to start with an explanatory error. With `enforce_on_docker: false`, it falls back to direct outbound with real creds and logs a warning.
- **Port collision** — iron-proxy exits immediately; `hermes egress start` reports the last 20 log lines and fails with non-zero exit.
- **Upstream-host denied** — sandbox gets HTTP 403 from the proxy with a body explaining which host wasn't allowed. The agent sees the error and reports it.
- **Cloud metadata IP (169.254.169.254) requested** — refused by `upstream_deny_cidrs` regardless of allowlist.
- **`docker_env` collides with a proxy-controlling var (enforce on)** — sandbox creation refuses with the names of the colliding keys.
- **`docker_forward_env` tries to forward a protected provider key (enforce on)** — sandbox creation refuses; remove the key from `docker_forward_env` or opt out with `proxy.enforce_on_docker: false`.
- **`docker_extra_args` overrides proxy env/network controls (enforce on)** — sandbox creation refuses; user-supplied `-e HTTPS_PROXY=...`, `--env-file`, or `--network` args run after Hermes' generated args and can bypass egress.
- **BWS access token missing in `credential_source: bitwarden`** — `hermes egress start` refuses with `--no-bitwarden` as the recovery hint.
- **iron-proxy doesn't bind within 5 seconds** — process is killed, pidfile unlinked, error names the port + tail of `iron-proxy.log`.
- **Concurrent `hermes egress start` calls** — second call refuses with "another start in progress" if the first's daemon is up; otherwise the second unlinks the stale pidfile and proceeds.
## Troubleshooting
### "Refusing to start: BWS_ACCESS_TOKEN is not set"
You enabled `credential_source: bitwarden` but the access-token env var isn't in your shell. Either:
```bash
export BWS_ACCESS_TOKEN=… # one-shot
hermes egress start
```
Or move it into `~/.hermes/.env`. Or switch back to env mode:
```bash
hermes egress setup --no-bitwarden
```
### "iron-proxy exited immediately"
Look at the last 20 lines of `~/.hermes/proxy/iron-proxy.log`. Common causes:
- Port already in use → change `proxy.tunnel_port` or kill whatever else owns 9090
- Invalid `proxy.yaml` → run `hermes egress setup` to regenerate
- CA cert / key permissions wrong → `chmod 0o600 ~/.hermes/proxy/ca.key`
### "iron-proxy did not bind \<bind-host\>:9090 within 5s"
The daemon started but never bound the listener. Usually means the binary is wedged or doing something expensive at startup. Check `~/.hermes/proxy/iron-proxy.log`. The orphan process is killed automatically and the pidfile cleaned up so you can just retry `hermes egress start`.
### Sandbox times out connecting to the proxy (Linux)
The container resolves `host.docker.internal` to the docker bridge gateway and the proxy is bound there, but a host firewall (commonly `ufw` with default-deny INPUT) drops container→host traffic on `docker0`. Verify from a container:
```bash
docker run --rm --add-host host.docker.internal:host-gateway busybox \
nc -zv -w 3 host.docker.internal 9090
```
If that times out while `hermes egress status` shows `listening`, allow the bridge subnet in your firewall, e.g. for ufw:
```bash
sudo ufw allow in on docker0 to any port 9090 proto tcp
sudo ufw allow in on docker0 to any port 9091 proto tcp
```
(9091 = the plain-HTTP forward listener on `tunnel_port + 1`.)
### Sandbox sees `HTTP 403` from the proxy
The agent inside the sandbox tried to hit a host that isn't in `proxy.extra_allowed_hosts`. The 403 body explains which host. If you want to allow it, add to your config:
```yaml
proxy:
extra_allowed_hosts:
- api.example.com
- "*.staging.example.com"
```
Then `hermes egress setup` (to regenerate `proxy.yaml`) and `hermes egress stop && hermes egress start`.
### Sandbox sees SSL verification errors
Either the CA isn't mounted in the sandbox (rare; the docker backend does this automatically when `proxy.enabled: true`), or your image's HTTP client is reading from a non-standard env var.
```bash
# Inside the sandbox:
cat /etc/ssl/certs/hermes-egress-ca.crt | head -1
# Should print: -----BEGIN CERTIFICATE-----
env | grep -E "^(REQUESTS|CURL|SSL|NODE).*CA"
# Should list all four CA-bundle env vars pointing at /etc/ssl/certs/hermes-egress-ca.crt
```
If the cert isn't there, check that `proxy.enabled: true` AND `hermes egress status` shows `Listening yes`. If the env vars are missing, the sandbox image might be running an entrypoint that strips them — check your `docker_env` config.
### Sandbox sees `HTTP 401` from upstreams
Two common causes:
1. **Token-clobber on re-setup.** You ran `hermes egress setup --rotate-tokens` (or rotated tokens some other way) and the running sandboxes still hold the old tokens. Restart the sandboxes.
2. **Bitwarden refresh failed silently.** Should not happen with the new fail-loud behavior, but if you have `proxy.allow_env_fallback: true` set, the daemon may have started with stale env values. Check the daemon's environment (`/proc/<iron-proxy-pid>/environ`) for the expected `OPENROUTER_API_KEY` etc.
### "Address in use" after the parent process died
The parent Hermes process died during `hermes egress start` (Ctrl-C during the listening probe, OOM, panic). The new fix-up logic writes the pidfile immediately after `Popen` so the orphan is recoverable:
```bash
hermes egress stop # finds the orphan via the pidfile, kills it
hermes egress start
```
If `hermes egress stop` says "iron-proxy was not running" but you can still see the daemon in `ps`, the pidfile got out of sync. Manual recovery:
```bash
pkill -TERM iron-proxy
rm -f ~/.hermes/proxy/iron-proxy.pid ~/.hermes/proxy/iron-proxy.nonce
hermes egress start
```
### Inspecting per-request behavior
On the pinned binary version (**v0.39**) both daemon-level events and per-request records land in `~/.hermes/proxy/iron-proxy.log`. The format is line-delimited JSON. Grep for a specific upstream:
```bash
grep '"upstream":"openrouter.ai"' ~/.hermes/proxy/iron-proxy.log | tail -20
```
Or watch in real-time:
```bash
tail -f ~/.hermes/proxy/iron-proxy.log | jq
```
When the pinned version moves to v0.40+ (which adds `log.audit_path`), per-request records will move to `~/.hermes/proxy/audit.log` and `iron-proxy.log` will hold only daemon-level events. Until that bump, `audit.log` is an empty placeholder (pre-created at `0o600` so the future daemon inherits tight permissions) — wire your logrotate / monitoring tooling to `iron-proxy.log` today and plan to add `audit.log` after the version bump.
## Limitations (v1)
- Docker backend only. Modal, Daytona, and SSH wiring will follow in separate PRs.
- Providers with signature-based auth (AWS SigV4, GCP service-account OAuth) bypass the proxy entirely — see [Uncovered providers](#uncovered-providers). Header-token providers (bearer, `x-api-key`, `api-key`, `x-goog-api-key`) are all covered.
- No native Windows binary upstream. Run on Linux / macOS / WSL.
- The CA is a 10-year self-signed cert on first generation. Rotation requires `openssl genrsa ...` by hand (or wait for a follow-up that adds `hermes egress rotate-ca`).
- Re-running setup stops a running daemon after rewriting config or mappings; restart (or `hermes egress reload` for ruleset-only changes) and restart already-running sandboxes after token rotation.
- iron-proxy in-memory secret zeroisation is upstream-controlled. Same-uid attackers with `/proc/<pid>/mem` read access can read swapped-in secrets from the daemon's memory.
- iron-proxy v0.39 only supports a **single bind per daemon** (we bind the docker bridge gateway on Linux, loopback on Docker Desktop) and combines daemon + per-request records into a single log stream. When upstream adds `proxy.http_listens` (plural) and `log.audit_path`, a version bump can wire in multi-bind and the dedicated audit stream.
## See also
- Upstream project: [github.com/ironsh/iron-proxy](https://github.com/ironsh/iron-proxy)
- Upstream docs: [docs.iron.sh](https://docs.iron.sh/)
- Bitwarden integration: [`hermes secrets bitwarden`](../secrets/bitwarden)
- Hermes Docker terminal backend: [Docker](../docker)
- Developer / contributor reference: [Egress proxy internals](../../developer-guide/egress-internals)
website/docs/user-guide/features/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "Features",
"position": 4,
"link": {
"type": "generated-index",
"description": "Explore the powerful features of Hermes Agent."
}
}
website/docs/user-guide/features/acp.md
+398
View File
@@ -0,0 +1,398 @@
---
sidebar_position: 11
title: "ACP Host Integration"
description: "Use Hermes Agent inside ACP-compatible editors and collaboration platforms"
---
# ACP Host Integration
Hermes Agent can run as an ACP server, letting ACP-compatible hosts talk to
Hermes over stdio. Editors can render:
- chat messages
- tool activity
- file diffs
- terminal commands
- approval prompts
- streamed thinking / response chunks
Other hosts can use the same protocol to route collaboration events into
Hermes. ACP is a good fit when you want Hermes to keep its existing identity,
provider setup, memory, skills, and tools while another application owns the
conversation transport.
## What Hermes exposes in ACP mode
Hermes runs with a curated `hermes-acp` toolset designed for editor workflows. It includes:
- file tools: `read_file`, `write_file`, `patch`, `search_files`
- terminal tools: `terminal`, `process`
- web/browser tools
- memory, todo, session search
- skills
- execute_code and delegate_task
- vision
It intentionally excludes things that do not fit typical editor UX, such as messaging delivery and cronjob management.
## Installation
Install Hermes normally, then add the ACP extra from the install checkout:
```bash
cd ~/.hermes/hermes-agent && uv pip install -e '.[acp]'
```
This installs the `agent-client-protocol` dependency and enables:
- `hermes acp`
- `hermes-acp`
- `python -m acp_adapter`
## Launching the ACP server
Any of the following starts Hermes in ACP mode:
```bash
hermes acp
```
```bash
hermes-acp
```
```bash
python -m acp_adapter
```
Hermes logs to stderr so stdout remains reserved for ACP JSON-RPC traffic.
For non-interactive checks:
```bash
hermes acp --version
hermes acp --check
```
### Browser tools (optional)
Browser tools (`browser_navigate`, `browser_click`, etc.) depend on the
`agent-browser` npm package and Chromium, which aren't part of the Python
wheel. Install them with:
```bash
hermes acp --setup-browser # interactive (prompts before ~400 MB download)
hermes acp --setup-browser --yes # accept the download non-interactively
```
This is the standalone command. The terminal-auth flow (`hermes acp --setup`) also offers the browser bootstrap as a follow-up question after model selection, so most users never need to run `--setup-browser` directly.
What it does:
- Installs Node.js 26 into `~/.hermes/node/` if missing
- `npm install -g agent-browser @askjo/camofox-browser` into that prefix (no sudo needed — `npm`'s `--prefix` points at the user-writable Hermes-managed Node)
- Installs Playwright Chromium, or uses a detected system Chrome/Chromium when available
The bootstrap is idempotent — re-running it is fast and skips work that's already done.
## Host setup
### Buzz channels (relay bridge)
[Buzz](https://github.com/block/buzz) is a Nostr-based collaboration platform
for people and agents. Its `buzz-acp` harness connects Buzz channels to any ACP
agent over stdio:
```text
Buzz relay <-- WebSocket --> buzz-acp <-- ACP over stdio --> Hermes Agent
```
This is a transport integration, not a second Hermes installation. The
subprocess launched by `buzz-acp` uses the same Hermes configuration,
credentials, memory, skills, and state as `hermes` on that host.
(This is distinct from [Buzz Desktop's managed runtime](#buzz-desktop), which
spawns Hermes locally as a preset harness. The relay bridge is for joining Buzz
*channels* as an agent identity, typically on a server.)
Prerequisites:
- Complete the ACP installation and `hermes acp --check` above.
- Build `buzz-acp` and the `buzz` CLI from the
[Buzz repository](https://github.com/block/buzz)
(`cargo build --release -p buzz-acp`).
- Mint a dedicated Nostr keypair for Hermes (`buzz-admin generate-key`) and
register it as a relay member (`buzz-admin add-member`). Every agent needs
its own identity — do not reuse a human keypair.
- Add that identity to the intended Buzz channels.
Start a bridge with:
```bash
export BUZZ_RELAY_URL="wss://community.example.com"
export BUZZ_PRIVATE_KEY="..."
export BUZZ_API_TOKEN="..."
export BUZZ_ACP_AGENT_COMMAND="hermes"
export BUZZ_ACP_AGENT_ARGS="acp"
buzz-acp
```
`BUZZ_API_TOKEN` is needed only when the relay enforces token authentication.
Do not commit or paste the private key or API token.
For a persistent server deployment, run `buzz-acp` under a service manager as
the same operating-system user that owns the intended Hermes home. Setup,
key generation, channel discovery, and per-agent options are documented in the
[buzz-acp README](https://github.com/block/buzz/tree/main/crates/buzz-acp).
The bridge discovers every Buzz channel where the Hermes identity is a member
and automatically subscribes when it is added to another channel. Buzz channel
membership therefore remains the access boundary; Hermes does not need a
separate channel list in its own configuration.
To expose Hermes ACP activity in the owner's Buzz Desktop, add:
```bash
export BUZZ_ACP_RELAY_OBSERVER="true"
```
This publishes encrypted kind `24200` observer frames addressed to the agent's
owner (Buzz's NIP-AO). Desktop renders the live lifecycle, tool, response, and
usage stream in the agent's **Activity log**. The relay treats these frames as
ephemeral, so Desktop must be online before the turn starts; its local observer
archive is the durable owner-side history.
Headless bridges answer ACP permission requests themselves because no editor
is present to show approval dialogs — see
[Keep Buzz agents owner-only](#keep-buzz-agents-owner-only). Treat the bridge
as privileged automation: use a dedicated operating-system account, restrict
which Buzz users can prompt the agent (`buzz-acp` supports an owner-only
respond gate via `BUZZ_ACP_AGENT_OWNER`), and grant membership only in channels
where Hermes is expected to work.
### VS Code
Install the [ACP Client](https://marketplace.visualstudio.com/items?itemName=formulahendry.acp-client) extension.
To connect:
1. Open the ACP Client panel from the Activity Bar.
2. Select **Hermes Agent** from the built-in agent list.
3. Connect and start chatting.
If you want to define Hermes manually, add it through VS Code settings under `acp.agents`:
```json
{
"acp.agents": {
"Hermes Agent": {
"command": "hermes",
"args": ["acp"]
}
}
}
```
### Zed
Configure Hermes as a custom agent server in Zed settings:
1. Open the Agent Panel.
2. Add a custom agent server with the following configuration:
```json
{
"agent_servers": {
"hermes-agent": {
"type": "custom",
"command": "hermes",
"args": ["acp"]
}
}
}
```
3. Start a new Hermes external-agent thread.
Prerequisites:
- Configure Hermes provider credentials first with `hermes model`, or set them in `~/.hermes/.env` / `~/.hermes/config.yaml`.
### JetBrains
Use an ACP-compatible plugin and point it at `hermes acp` or `hermes-acp`.
### Buzz Desktop
[Buzz](https://github.com/block/buzz) ships Hermes Agent as a preset runtime.
With Hermes installed the normal way, Buzz discovers it automatically —
open **Settings → Runtimes** and Hermes appears under your runtimes.
If discovery fails (older installs), make sure the ACP launcher resolves on a
login-shell PATH:
```bash
command -v hermes-acp || command -v hermes
```
Recent installs write both `hermes` and `hermes-acp` launchers into
`~/.local/bin`; running `hermes update` adds the `hermes-acp` launcher to
older installs. As a manual fallback, configure Buzz's agent command as
`hermes` with args `["acp"]`.
#### Model picker
Buzz Desktop (v0.5.1+) renders Hermes' full model menu in the agent's runtime
settings. The list comes from Hermes itself over ACP: it shows every model
from providers you have authenticated in Hermes (the same inventory behind
`hermes model` and the `/model` command), so a model missing from the menu
means its provider has no credentials configured on the Hermes side.
Entry IDs take the form `provider:model` (e.g. `openrouter:z-ai/glm-5.1`), or
`custom:<name>:<model>` for custom OpenAI-compatible endpoints defined in
`config.yaml`. Picking a model applies to that agent's session; it does not
change your Hermes-wide default — use `hermes model` for that.
#### Keep Buzz agents owner-only
Buzz creates every agent with **Who can talk to this agent** set to `Owner only`.
Leave it there when the runtime is Hermes.
Two behaviors combine on this path. The `hermes-acp` toolset includes `terminal`
and `execute_code`, and Buzz's ACP bridge answers Hermes' permission requests
itself with `allow_once` rather than surfacing them. A Hermes agent in Buzz
therefore runs shell commands on the host without prompting. I asked one to run
`rm -rf` against a scratch directory and it deleted it, no prompt anywhere.
Selecting `Anyone` hands that same shell access to every author who can reach
the channel. Buzz does not warn when you pick it.
Neither of the obvious mitigations works today:
- `approvals.mode: manual` does make Hermes raise the permission request, but
Buzz auto-approves it and the command still runs.
- `platform_toolsets.acp` does not narrow the ACP toolset, so it cannot be used
to drop `terminal`.
`!shutdown` from the owner stops the agent in any mode, and Buzz ignores that
command from everyone else.
## Configuration and credentials
ACP mode uses the same Hermes configuration as the CLI:
- `~/.hermes/.env`
- `~/.hermes/config.yaml`
- `~/.hermes/skills/`
- `~/.hermes/state.db`
Provider resolution uses Hermes' normal runtime resolver, so ACP inherits the currently configured provider and credentials. Hermes also advertises a terminal auth method (`--setup`) for first-run ACP clients; this opens Hermes' interactive model/provider setup.
## Host integration
These variables are set by an **ACP host process** (an editor or another agent
harness) on the Hermes subprocess it spawns. They are not user configuration —
do not set them by hand in `.env` or `config.yaml`.
| Variable | Value | Effect |
|----------|-------|--------|
| `HERMES_ACP_SKIP_CONFIGURED_MCP` | `1` | Skip starting the **globally configured** MCP servers from `config.yaml` before the ACP JSON-RPC loop begins. |
Hermes normally starts every MCP server configured in `config.yaml` before it
enters the ACP JSON-RPC loop. A host that owns MCP itself — passing the
session's servers explicitly through `session/new` — does not need that global
startup, and an unrelated slow or interactive MCP server would otherwise delay
`initialize`. Setting the marker to exactly `1` lets such a host skip it.
Only the global `config.yaml` discovery is skipped. **MCP servers supplied by
the ACP session through `session/new` are still registered**, so a host loses
no capability it asked for. Any other value (unset, empty, `0`, `false`) keeps
the default behavior, so an unrelated truthy-looking string cannot silently
disable MCP.
## Session behavior
ACP sessions are tracked by the ACP adapter's in-memory session manager while the server is running.
Each session stores:
- session ID
- working directory
- selected model
- current conversation history
- cancel event
The underlying `AIAgent` still uses Hermes' normal persistence/logging paths, but ACP `list/load/resume/fork` are scoped to the currently running ACP server process.
## Working directory behavior
ACP sessions bind the editor's cwd to the Hermes task ID so file and terminal tools run relative to the editor workspace, not the server process cwd.
## Approvals
Dangerous terminal commands can be routed back to the editor as approval prompts. ACP approval options are simpler than the CLI flow:
- allow once
- allow always
- deny
Whether you actually see a prompt is up to the host. A host is free to answer the
request programmatically instead of showing it to you, in which case these
options exist on the wire but never reach a human. Buzz Desktop does this, so
treat that path as unattended execution regardless of your `approvals` setting.
On timeout or error, the approval bridge denies the request.
### Session-scoped edit auto-approval
ACP exposes a third tier between *allow once* and *allow always*: **Allow for session**. Picking it from the editor's permission prompt records the approval inside the current ACP session only — every subsequent matching command in that session goes through without prompting, but a new ACP session (or restarting the editor) resets the slate and re-prompts the first time.
| Option | Editor label | Scope | Persisted across restarts |
|---|---|---|---|
| `allow_once` | Allow once | This one tool call | No |
| `allow_session` | Allow for session | All matching calls in this ACP session | No — cleared when the session ends |
| `allow_always` | Allow always | All future sessions | Yes (written to the Hermes permanent allowlist) |
| `deny` | Deny | This one tool call | No |
`allow_session` is the right default for an editor workflow where you trust an agent for the duration of a task but don't want to grant a long-lived allowlist entry. The safety trade-off is straightforward: the broader the scope, the less the editor will interrupt you, and the more damage a misbehaving agent (or prompt injection) can do before you notice. Start with `allow_once` for unfamiliar commands; promote to `allow_session` once you've seen the agent run the same pattern correctly a few times; reserve `allow_always` for truly idempotent commands you trust forever (e.g. `git status`).
The ACP bridge maps these options onto Hermes' internal approval semantics — `allow_always` writes a permanent allowlist entry the same way the CLI does, while `allow_session` only affects the in-process approval cache for the current ACP session.
## Troubleshooting
### ACP agent does not appear in the editor
Check:
- For manual/local development, verify the host command points to `hermes acp`.
- Hermes is installed and on your PATH.
- The ACP extra is installed (`cd ~/.hermes/hermes-agent && uv pip install -e '.[acp]'`).
### ACP starts but immediately errors
Try these checks:
```bash
hermes acp --version
hermes acp --check
hermes doctor
hermes status
```
### Missing credentials
ACP mode uses Hermes' existing provider setup. Configure credentials with:
```bash
hermes model
```
or by editing `~/.hermes/.env`. The terminal auth flow (`hermes acp --setup`) can also trigger the interactive provider/model setup.
## See also
- [Buzz ACP harness](https://github.com/block/buzz/tree/main/crates/buzz-acp)
- [ACP Internals](../../developer-guide/acp-internals.md)
- [Provider Runtime Resolution](../../developer-guide/provider-runtime.md)
- [Tools Runtime](../../developer-guide/tools-runtime.md)
website/docs/user-guide/features/api-server.md
+771
View File
@@ -0,0 +1,771 @@
---
sidebar_position: 14
title: "API Server"
description: "Expose hermes-agent as an OpenAI-compatible API for any frontend"
---
# API Server
The API server exposes hermes-agent as an OpenAI-compatible HTTP endpoint. Any frontend that speaks the OpenAI format — Open WebUI, LobeChat, LibreChat, NextChat, ChatBox, and hundreds more — can connect to hermes-agent and use it as a backend.
Your agent handles requests with its full toolset (terminal, file operations, web search, memory, skills) and returns the final response. When streaming, tool progress indicators appear inline so frontends can show what the agent is doing.
:::tip One backend covers models + tools
Hermes itself needs a configured provider and tool backends for the API server to be useful. A [Nous Portal](/user-guide/features/tool-gateway) subscription handles both — 300+ models plus web/image/TTS/browser via the Tool Gateway. Run `hermes setup --portal` once before starting the API server and frontends like Open WebUI or LobeChat get a fully tool-equipped backend.
:::
## Quick Start
### 1. Enable the API server
Add to `~/.hermes/.env`:
```bash
API_SERVER_ENABLED=true
API_SERVER_KEY=change-me-local-dev
# Optional: only if a browser must call Hermes directly
# API_SERVER_CORS_ORIGINS=http://localhost:3000
```
### 2. Start the gateway
```bash
hermes gateway
```
You'll see:
```
[API Server] API server listening on http://127.0.0.1:8642
```
### 3. Connect a frontend
Point any OpenAI-compatible client at `http://localhost:8642/v1`:
```bash
# Test with curl
curl http://localhost:8642/v1/chat/completions \
-H "Authorization: Bearer change-me-local-dev" \
-H "Content-Type: application/json" \
-d '{"model": "hermes-agent", "messages": [{"role": "user", "content": "Hello!"}]}'
```
Or connect Open WebUI, LobeChat, or any other frontend — see the [Open WebUI integration guide](/user-guide/messaging/open-webui) for step-by-step instructions.
## Endpoints
### POST /v1/chat/completions
Standard OpenAI Chat Completions format. Stateless — the full conversation is included in each request via the `messages` array.
**Request:**
```json
{
"model": "hermes-agent",
"messages": [
{"role": "system", "content": "You are a Python expert."},
{"role": "user", "content": "Write a fibonacci function"}
],
"stream": false
}
```
**Response:**
```json
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1710000000,
"model": "hermes-agent",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": "Here's a fibonacci function..."},
"finish_reason": "stop"
}],
"usage": {"prompt_tokens": 50, "completion_tokens": 200, "total_tokens": 250}
}
```
**Inline image input:** user messages may send `content` as an array of `text` and `image_url` parts. Both remote `http(s)` URLs and `data:image/...` URLs are supported:
```json
{
"model": "hermes-agent",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "What is in this image?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.png", "detail": "high"}}
]
}
]
}
```
Uploaded files (`file` / `input_file` / `file_id`) and non-image `data:` URLs return `400 unsupported_content_type`.
**Streaming** (`"stream": true`): Returns Server-Sent Events (SSE) with token-by-token response chunks. For **Chat Completions**, the stream uses standard `chat.completion.chunk` events plus Hermes' custom `hermes.tool.progress` event for tool-start UX. For **Responses**, the stream uses OpenAI Responses event types such as `response.created`, `response.output_text.delta`, `response.output_item.added`, `response.output_item.done`, and `response.completed`.
**Tool progress in streams**:
- **Chat Completions**: Hermes emits `event: hermes.tool.progress` for tool-start visibility without polluting persisted assistant text.
- **Responses**: Hermes emits spec-native `function_call` and `function_call_output` output items during the SSE stream, so clients can render structured tool UI in real time.
### POST /v1/responses
OpenAI Responses API format. Supports server-side conversation state via `previous_response_id` — the server stores full conversation history (including tool calls and results) so multi-turn context is preserved without the client managing it.
**Request:**
```json
{
"model": "hermes-agent",
"input": "What files are in my project?",
"instructions": "You are a helpful coding assistant.",
"store": true
}
```
**Response:**
```json
{
"id": "resp_abc123",
"object": "response",
"status": "completed",
"model": "hermes-agent",
"output": [
{"type": "function_call", "status": "completed", "name": "terminal", "arguments": "{\"command\": \"ls\"}", "call_id": "call_1"},
{"type": "function_call_output", "status": "completed", "call_id": "call_1", "output": "README.md src/ tests/"},
{"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "Your project has..."}]}
],
"usage": {"input_tokens": 50, "output_tokens": 200, "total_tokens": 250}
}
```
Tool calls in the `output` array were already executed server-side by the Hermes agent — they are replayed with `"status": "completed"` for structured tool UI, never as pending calls for the client to execute.
**Inline image input:** `input[].content` can contain `input_text` and `input_image` parts. Both remote URLs and `data:image/...` URLs are supported:
```json
{
"model": "hermes-agent",
"input": [
{
"role": "user",
"content": [
{"type": "input_text", "text": "Describe this screenshot."},
{"type": "input_image", "image_url": "data:image/png;base64,iVBORw0K..."}
]
}
]
}
```
Uploaded files (`input_file` / `file_id`) and non-image `data:` URLs return `400 unsupported_content_type`.
#### Multi-turn with previous_response_id
Chain responses to maintain full context (including tool calls) across turns:
```json
{
"input": "Now show me the README",
"previous_response_id": "resp_abc123"
}
```
The server reconstructs the full conversation from the stored response chain — all previous tool calls and results are preserved. Chained requests also share the same session, so multi-turn conversations appear as a single entry in the dashboard and session history.
#### Named conversations
Use the `conversation` parameter instead of tracking response IDs:
```json
{"input": "Hello", "conversation": "my-project"}
{"input": "What's in src/?", "conversation": "my-project"}
{"input": "Run the tests", "conversation": "my-project"}
```
The server automatically chains to the latest response in that conversation. Like the `/title` command for gateway sessions.
### GET /v1/responses/\{id\}
Retrieve a previously stored response by ID.
### DELETE /v1/responses/\{id\}
Delete a stored response.
### GET /v1/models
Lists the agent as an available model. The advertised model name defaults to the [profile](/user-guide/profiles) name (or `hermes-agent` for the default profile). Required by most frontends for model discovery.
`/v1/models` is intentionally the cheap OpenAI-compat surface. It does **not**
enumerate every authenticated provider/model combination Hermes can route to,
and it does not do pricing or capability enrichment.
### GET /api/model/options
Hermes-aware clients can request the same curated provider/model inventory used
by the dashboard and TUI. This route uses the API server's normal bearer
authentication and returns provider rows, model capability hints, and pricing
metadata that do not belong in the OpenAI-compatible `/v1/models` response:
```bash
curl \
-H "Authorization: Bearer $API_SERVER_KEY" \
"http://127.0.0.1:8642/api/model/options"
```
That payload is the same substrate the dashboard Models page and the TUI
`model.options` RPC use. It returns authenticated providers, curated model
lists, per-model pricing, and model capability hints.
Normal opens are intentionally conservative for custom providers: Hermes probes
only the **currently selected** custom endpoint so a stale or offline saved
endpoint does not block the picker. An explicit refresh flips to full probing
and busts the provider model cache:
```bash
curl \
-H "Authorization: Bearer $API_SERVER_KEY" \
"http://127.0.0.1:8642/api/model/options?refresh=1"
```
Use `/v1/models` when an OpenAI-compatible client only needs a model name to
send back in chat/responses requests. Use `/api/model/options` when an
authenticated UI needs the richer Hermes-specific picker metadata.
### GET /v1/capabilities
Returns a machine-readable description of the API server's stable surface for external UIs, orchestrators, and plugin bridges.
```json
{
"object": "hermes.api_server.capabilities",
"platform": "hermes-agent",
"model": "hermes-agent",
"auth": {"type": "bearer", "required": true},
"features": {
"chat_completions": true,
"responses_api": true,
"run_submission": true,
"run_status": true,
"run_events_sse": true,
"run_stop": true
}
}
```
Use this endpoint when integrating dashboards, browser UIs, or control planes so they can discover whether the running Hermes version supports runs, streaming, cancellation, and session continuity without depending on private Python internals.
## Browser-extension control
Hermes can route browser tools through an authenticated extension that controls
the browser session associated with the current Hermes session. The feature is
disabled by default; set `browser.extension_control.enabled` to `true` to opt in:
```yaml
browser:
extension_control:
enabled: true
```
The local API path also requires the API server bearer key. A controller may
register only for an existing server session. Hermes derives the controller
principal from authenticated server state; a client-supplied `principal_id` is
ignored.
Discover the live contract through `GET /v1/capabilities`. The
`browser_extension_control` object reports whether the feature is enabled, the
protocol version, transport names, and the exact capability allowlist:
```text
controller.noop
browser_back
browser_click
browser_navigate
browser_press
browser_screenshot
browser_scroll
browser_snapshot
browser_tab_activate
browser_tabs
browser_type
```
Requested capabilities outside that list are filtered out. Raw CDP, arbitrary
script evaluation, console access, uploads, image extraction, and vision are not
part of the controller protocol.
When a request has no bound controller identity, or when the feature is disabled,
Hermes preserves the existing browser backend. Once the gateway binds a
controller principal and transport family to the request, that extension lane
is authoritative: missing, ambiguous, disconnected, or incapable controllers
fail closed instead of silently switching to a different local/cloud browser.
After an exact controller is selected, its result or error is authoritative and
Hermes never retries the same action through another backend.
### Local API registration
1. Send an authenticated `POST /v1/browser-control/register` with
`protocol_version`, `session_id`, `controller_id`, `browser_profile_id`, and
the requested `capabilities`.
2. Hermes returns a single-use ticket with a 30-second TTL and the filtered,
server-bound controller scope.
3. Open `GET /v1/browser-control/ws` with both WebSocket subprotocols:
`hermes-browser-control-v1` and
`hermes-browser-control-ticket.<ticket>`.
The ticket is never accepted in the query string. Unknown, expired, reused, or
malformed tickets fail before WebSocket upgrade.
### Controller frames
Hermes sends `browser.controller.command` frames containing `command_id`,
`action`, immutable `arguments`, browser/controller ids, and the originating
`tool_call_id`. The controller replies with `browser.controller.result`, the
same `command_id`, an exact boolean `ok`, and either `result` or `error`.
Cancellation and timeout emit `browser.controller.cancel`; late results are
ignored.
An unexpected socket loss marks the controller offline and preserves work
already in flight until each command's original deadline. A reconnect with the
same principal, profile, session, controller id, browser profile, and transport
identity refreshes the transport without admitting new work before any deferred
cancels are flushed. Negotiated capabilities may change on that reconnect; they
are not an identity field. A different controller id or browser profile in the
same authenticated session lane is a hard replacement: old pending work is
cancelled before the successor becomes routable. Send
`browser.controller.detach` on the authenticated controller transport for an
intentional hard detach — that immediately cancels pending work. Merely closing
the socket is treated as a recoverable disconnect.
The authenticated dashboard transport exposes the same registration, result,
heartbeat, capability, and ownership semantics over its Gateway RPC/event
channel. In both transports, selection requires one unambiguous exact match on
principal, profile, session, controller, browser profile, transport family, and
capability. Once selected, a controller failure is authoritative and is never
retried through a different browser backend.
## Per-request model selection
Authenticated clients can override Hermes' default model selection per request
by sending:
- `model` — the target model id for this turn
- `provider` — the Hermes provider slug to resolve credentials/runtime for this turn
- `model_options` — request-scoped reasoning / service-tier controls
The same request fields are accepted on:
- `POST /v1/chat/completions`
- `POST /v1/responses`
- `POST /v1/runs`
- `POST /api/sessions/{session_id}/chat`
- `POST /api/sessions/{session_id}/chat/stream`
Precedence is deterministic:
1. Session `/model` override, if that session already has one
2. A static `gateway.platforms.api_server.model_routes` mapping selected when
the request's `model` is a configured route alias
3. Direct request `model` / `provider` when no route alias matches
4. Global gateway config / environment defaults
`model_options` stays request-scoped regardless of which model/provider wins.
If a request sends a `provider` that conflicts with a configured `model_routes`
alias, Hermes rejects the request with `400` instead of silently remixing route
credentials with another provider.
**Bare `model` values on the OpenAI-compatible endpoints are opt-in.** Generic
OpenAI clients routinely hardcode model names (`gpt-4o`, ...), and existing
deployments rely on those falling back to the gateway default. On
`POST /v1/chat/completions` and `POST /v1/responses`, a `model` value sent
WITHOUT a `provider` is therefore ignored unless you enable:
```yaml
gateway:
platforms:
api_server:
direct_model_requests: true
```
Requests that include an explicit `provider` — and the Hermes-native
`/v1/runs` and session-chat endpoints — always honor the requested model
regardless of this flag.
Example:
```json
{
"model": "MiniMax-M3",
"provider": "minimax",
"model_options": {
"reasoning_effort": "high",
"service_tier": "priority"
},
"messages": [
{"role": "user", "content": "Summarize the repo status."}
]
}
```
### GET /health
Health check. Returns `{"status": "ok"}`. Also available at **GET /v1/health** for OpenAI-compatible clients that expect the `/v1/` prefix.
### GET /health/detailed
Authenticated readiness check for monitoring and control planes. It reports
bounded status for the active profile's config, state database, configured
model, disk space, gateway/platform state, active API runs, pending process
completions, and active delegations. The response exposes status and counts,
not config values, credentials, paths, commands, queue payloads, or raw errors.
The public `/health` route remains a cheap liveness probe and does not run
readiness checks. A degraded readiness result still uses HTTP 200; inspect the
top-level `status` and `readiness.checks` fields.
## Runs API (streaming-friendly alternative)
In addition to `/v1/chat/completions` and `/v1/responses`, the server exposes a **runs** API for long-form sessions where the client wants to subscribe to progress events instead of managing streaming themselves.
### POST /v1/runs
Create a new agent run. Returns a `run_id` that can be used to subscribe to progress events.
```json
{
"run_id": "run_abc123",
"status": "started"
}
```
Runs accept a simple `input` string and optional `session_id`, `instructions`, `conversation_history`, or `previous_response_id`. When `session_id` is provided, Hermes surfaces it in the run status so external UIs can correlate runs with their own conversation IDs.
For safely retryable creation, send an `Idempotency-Key` header (1–255 visible ASCII characters). Hermes durably reserves the key before starting work. An identical retry returns the original `run_id` with HTTP 202 and `Idempotency-Replayed: true`, including after a gateway restart and after the run has completed, failed, or been cancelled. Reusing the same key with a different JSON payload returns HTTP 409 with code `idempotency_key_conflict`. Keys are isolated by authenticated API profile/credential and retained for 24 hours after their last status update; clients should use unique, unguessable keys and must not reuse them for unrelated operations. Requests without the header retain the legacy behavior and always create a new run.
When `session_id` identifies an existing Hermes session and no explicit
`conversation_history` or `previous_response_id` is supplied, the run loads
that session's active transcript. Session turn leases serialize concurrent
writers and refresh the transcript after a contended wait.
### GET /v1/runs/\{run_id\}
Poll the current run state. This is useful for dashboards that need status without holding an SSE connection open, or for UIs that reconnect after navigation.
```json
{
"object": "hermes.run",
"run_id": "run_abc123",
"status": "completed",
"session_id": "space-session",
"model": "hermes-agent",
"output": "Done.",
"usage": {"input_tokens": 50, "output_tokens": 200, "total_tokens": 250}
}
```
Statuses are retained briefly after terminal states (`completed`, `failed`, or `cancelled`) for polling and UI reconciliation.
### GET /v1/runs/\{run_id\}/events
Server-Sent Events stream of the run's tool-call progress, token deltas, and lifecycle events. Designed for dashboards and thick clients that want to attach/detach without losing state.
When the agent delegates work to background subagents, the stream also carries
`subagent.start` and `subagent.complete` lifecycle events, so clients can
observe delegation outcomes — including timeouts and failures — instead of the
run going silent while a child works. The `subagent.complete` payload carries
the child's status, summary, duration, token/cost figures, a
`child_session_id` for correlation, and the `delegation_id` of the batch it
belongs to (so concurrent or nested fan-outs stay distinguishable); free-text fields pass forced secret
redaction before leaving the process. Per-tool child events
(`subagent.tool`, progress ticks) are intentionally **not** forwarded — they
are high-volume UI noise; use the per-child live transcript files for
play-by-play.
Unconsumed event buffers expire after five minutes so a detached client cannot
grow memory indefinitely. This expires transport state only: a run that is
still executing remains visible to status polling, approval, stop control, and
concurrency accounting until its executor work actually exits. A connected SSE
subscriber continues draining normally.
### POST /v1/runs/\{run_id\}/stop
Interrupt a running agent turn. The endpoint returns immediately with `{"status": "stopping"}` while Hermes asks the active agent to stop at the next safe interruption point.
The run stays tracked as `stopping` until the executor-backed work exits, then
settles as `cancelled`; requesting stop never hides a worker that is still
running.
### POST /v1/runs/\{run_id\}/approval
Resolve a pending approval for a run that is waiting on a human decision (for example, a tool call gated behind an approval policy). The body carries the approval decision; the run resumes once the decision is recorded. This endpoint is advertised in `/v1/capabilities` as the `run_approval` feature so external UIs can detect support before surfacing an approval prompt.
## Jobs API (background scheduled work)
The server exposes a lightweight jobs CRUD surface for managing scheduled / background agent runs from a remote client. All endpoints are gated behind the same bearer auth.
### GET /api/jobs
List all scheduled jobs.
### POST /api/jobs
Create a new scheduled job. Body accepts the same shape as `hermes cron` — prompt, schedule, skills, provider override, delivery target.
### GET /api/jobs/\{job_id\}
Fetch a single job's definition and last-run state.
### PATCH /api/jobs/\{job_id\}
Update fields on an existing job (prompt, schedule, etc.). Partial updates are merged.
### DELETE /api/jobs/\{job_id\}
Remove a job. Also cancels any in-flight run.
### POST /api/jobs/\{job_id\}/pause
Pause a job without deleting it. Next-scheduled-run timestamps are suspended until resumed.
### POST /api/jobs/\{job_id\}/resume
Resume a previously paused job.
### POST /api/jobs/\{job_id\}/run
Trigger the job to run immediately, out of schedule.
## Sessions API (session control over REST)
External UIs can manage Hermes sessions over REST without standing up the dashboard. All endpoints are gated by `API_SERVER_KEY` and live under `/api/sessions/*`.
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/sessions` | List sessions (paginated — `limit`, `offset`, `source`, `include_children`) |
| `POST` | `/api/sessions` | Create an empty session |
| `GET` | `/api/sessions/{id}` | Read session metadata |
| `PATCH` | `/api/sessions/{id}` | Update title or `end_reason` |
| `DELETE` | `/api/sessions/{id}` | Delete a session |
| `GET` | `/api/sessions/{id}/messages` | Message history for a session |
| `POST` | `/api/sessions/{id}/fork` | Branch the session via `SessionDB` lineage (matches CLI `/branch` semantics) |
| `POST` | `/api/sessions/{id}/chat` | Run one synchronous agent turn |
| `POST` | `/api/sessions/{id}/chat/stream` | SSE wrapper over a single turn — emits `assistant.delta`, `tool.started`, `tool.completed`, `run.completed` events |
`/v1/capabilities` advertises the full surface via `session_*` feature flags and `endpoints.session_*` entries so external UIs can detect support and fall back safely. Inline images are supported in `chat` and `chat/stream` payloads (multimodal-aware path).
```bash
# fork a session and run one turn
curl -X POST http://localhost:8642/api/sessions/$ID/fork \
-H "Authorization: Bearer $API_SERVER_KEY" \
-d '{"title": "explore alt path"}'
# stream a turn over SSE
curl -N -X POST http://localhost:8642/api/sessions/$ID/chat/stream \
-H "Authorization: Bearer $API_SERVER_KEY" \
-d '{"input": "what files changed in the last hour?"}'
```
## Skills and toolsets discovery
`GET /v1/skills` and `GET /v1/toolsets` let external clients enumerate the agent's capabilities deterministically over REST instead of asking the model. Both are read-only and gated by `API_SERVER_KEY`.
```bash
curl http://localhost:8642/v1/skills \
-H "Authorization: Bearer $API_SERVER_KEY"
# → [{"name": "github-pr-workflow", "description": "...", "category": "..."}, ...]
curl http://localhost:8642/v1/toolsets \
-H "Authorization: Bearer $API_SERVER_KEY"
# → [{"name": "core", "label": "...", "description": "...", "enabled": true,
# "configured": true, "tools": ["read_file", "write_file", ...]}, ...]
```
`/v1/skills` returns the same metadata the skills hub uses internally. `/v1/toolsets` returns toolsets resolved for the `api_server` platform with the concrete `tools` list each one expands to. Both are advertised under `endpoints.*` in `/v1/capabilities`.
## Long-term memory scoping (`X-Hermes-Session-Key`)
Multi-user frontends like Open WebUI need a stable per-channel identifier for long-term memory (Honcho, etc.) that is **independent** of the transcript-scoped `X-Hermes-Session-Id` (which rotates on `/new`). Pass `X-Hermes-Session-Key` on `/v1/chat/completions`, `/v1/responses`, or `/v1/runs` and Hermes threads it through to `AIAgent(gateway_session_key=...)`, where the Honcho memory provider uses it to derive a stable scope.
```http
POST /v1/chat/completions HTTP/1.1
Authorization: Bearer ***
X-Hermes-Session-Id: transcript-alpha
X-Hermes-Session-Key: agent:main:webui:dm:user-42
```
Rules: max 256 chars, control characters (`\r`, `\n`, `\x00`) are rejected, and the value is echoed back on responses (JSON + SSE). `/v1/capabilities` advertises support via `"session_key_header": "X-Hermes-Session-Key"`. Without the key, Honcho's `per-session` strategy produces a different scope per `session_id` — exactly the behavior Hermes had before.
## System Prompt Handling
When a frontend sends a `system` message (Chat Completions) or `instructions` field (Responses API), hermes-agent **layers it on top** of its core system prompt. Your agent keeps all its tools, memory, and skills — the frontend's system prompt adds extra instructions.
This means you can customize behavior per-frontend without losing capabilities:
- Open WebUI system prompt: "You are a Python expert. Always include type hints."
- The agent still has terminal, file tools, web search, memory, etc.
## Authentication
Bearer token auth via the `Authorization` header:
```
Authorization: Bearer ***
```
Configure the key via `API_SERVER_KEY` env var. If you need a browser to call Hermes directly, also set `API_SERVER_CORS_ORIGINS` to an explicit allowlist.
### Multi-profile routing (`/p/<profile>/…`)
When [multi-profile gateway routing](/user-guide/multi-profile-gateways) is
enabled (`gateway.multiplex_profiles`), the shared listener serves every
profile through a `/p/<profile>/` URL prefix — and **authentication is bound
to the routed profile**:
- Requests to `/p/<profile>/v1/...` must present that profile's own
`API_SERVER_KEY` (from `~/.hermes/profiles/<profile>/.env`). The default
listener's key is rejected on named-profile prefixes.
- Unprefixed routes and `/p/default/...` keep using the default profile's key.
- A named profile with no `API_SERVER_KEY` of its own fails closed — its
prefix is unreachable until you set one.
- Runs are per-profile scoped: `/v1/runs/{run_id}` and its `events`, `stop`,
`steer`, and `approval` routes only answer for the profile that created
the run (including runs started via `/api/sessions/{id}/chat/stream`);
another profile's run id returns `404`, never `403`.
:::warning Breaking change (July 2026)
Before this fix, a valid default-profile key was accepted on any
`/p/<profile>/` prefix. If you relied on one shared key across profile
prefixes, set a distinct `API_SERVER_KEY` in each profile's `.env` — reused
default keys on named prefixes now return `401`.
:::
:::warning Security
The API server gives full access to hermes-agent's toolset, **including terminal commands**. `API_SERVER_KEY` is **required for every deployment**, including the default loopback bind on `127.0.0.1`. Keep `API_SERVER_CORS_ORIGINS` narrow to control browser access when you explicitly allow browser callers.
:::
## Configuration
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `API_SERVER_ENABLED` | `false` | Enable the API server |
| `API_SERVER_PORT` | `8642` | HTTP server port |
| `API_SERVER_HOST` | `127.0.0.1` | Bind address (localhost only by default) |
| `API_SERVER_KEY` | _(required)_ | Bearer token for auth |
| `API_SERVER_CORS_ORIGINS` | _(none)_ | Comma-separated allowed browser origins |
| `API_SERVER_MODEL_NAME` | _(profile name)_ | Model name on `/v1/models`. Defaults to profile name, or `hermes-agent` for default profile. |
### config.yaml
The same settings can live in `~/.hermes/config.yaml` under a nested `gateway.api_server:` section:
```yaml
gateway:
api_server:
enabled: true
port: 8642
host: 127.0.0.1
key: your-secret-key
cors_origins: http://localhost:3000
model_name: my-hermes
max_concurrent_runs: 10 # concurrent-run cap; 0 disables the limit
```
`port`, `key`, `host`, `cors_origins`, and `model_name` are automatically bridged into the platform's `extra` settings, so they behave exactly like their `API_SERVER_*` environment-variable counterparts. Environment variables take precedence over `config.yaml` values. The block is also accepted under `gateway.platforms.api_server:` or a top-level `platforms.api_server:` section.
### Concurrent-run cap
The API server limits how many agent runs may execute at once across the OpenAI-compatible and Runs endpoints. The cap is read from `gateway.api_server.max_concurrent_runs` (default **10**; `0` disables the limit, negative values clamp to 0). When the cap is reached, new run-starting requests are rejected with **HTTP 429** `Too many concurrent runs (max N)` — clients should back off and retry.
## Security Headers
All responses include security headers:
- `X-Content-Type-Options: nosniff` — prevents MIME type sniffing
- `Referrer-Policy: no-referrer` — prevents referrer leakage
## CORS
The API server does **not** enable browser CORS by default.
For direct browser access, set an explicit allowlist:
```bash
API_SERVER_CORS_ORIGINS=http://localhost:3000,http://127.0.0.1:3000
```
When CORS is enabled:
- **Preflight responses** include `Access-Control-Max-Age: 600` (10 minute cache)
- **SSE streaming responses** include CORS headers so browser EventSource clients work correctly
- **`Idempotency-Key`** is an allowed request header — clients can send it for deduplication (responses are cached by key for 5 minutes)
Most documented frontends such as Open WebUI connect server-to-server and do not need CORS at all.
## Compatible Frontends
Any frontend that supports the OpenAI API format works. Tested/documented integrations:
| Frontend | Stars | Connection |
|----------|-------|------------|
| [Open WebUI](/user-guide/messaging/open-webui) | 126k | Full guide available |
| LobeChat | 73k | Custom provider endpoint |
| LibreChat | 34k | Custom endpoint in librechat.yaml |
| AnythingLLM | 56k | Generic OpenAI provider |
| NextChat | 87k | BASE_URL env var |
| ChatBox | 39k | API Host setting |
| Jan | 26k | Remote model config |
| HF Chat-UI | 8k | OPENAI_BASE_URL |
| big-AGI | 7k | Custom endpoint |
| OpenAI Python SDK | — | `OpenAI(base_url="http://localhost:8642/v1")` |
| curl | — | Direct HTTP requests |
## Multi-User Setup with Profiles
To give multiple users their own isolated Hermes instance (separate config, memory, skills), use [profiles](/user-guide/profiles):
```bash
# Create a profile per user
hermes profile create alice
hermes profile create bob
# Configure each profile's API server on a different port. API_SERVER_* are env
# vars (not config.yaml keys), so write them to each profile's .env:
cat >> ~/.hermes/profiles/alice/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_PORT=8643
API_SERVER_KEY=alice-secret
EOF
cat >> ~/.hermes/profiles/bob/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_PORT=8644
API_SERVER_KEY=bob-secret
EOF
# Start each profile's gateway
hermes -p alice gateway &
hermes -p bob gateway &
```
Each profile's API server automatically advertises the profile name as the model ID:
- `http://localhost:8643/v1/models` → model `alice`
- `http://localhost:8644/v1/models` → model `bob`
In Open WebUI, add each as a separate connection. The model dropdown shows `alice` and `bob` as distinct models, each backed by a fully isolated Hermes instance. See the [Open WebUI guide](/user-guide/messaging/open-webui#multi-user-setup-with-profiles) for details.
## Limitations
- **Response storage** — stored responses (for `previous_response_id`) are persisted in SQLite and survive gateway restarts. Max 100 stored responses (LRU eviction).
- **No file upload** — inline images are supported on both `/v1/chat/completions` and `/v1/responses`, but uploaded files (`file`, `input_file`, `file_id`) and non-image document inputs are not supported through the API.
- **Simple OpenAI clients still see an alias** — `/v1/models` advertises the
stable Hermes alias (`hermes-agent` or the active profile name). Richer
clients can send explicit `provider` / `model_options` overrides on requests.
## Proxy Mode
The API server also serves as the backend for **gateway proxy mode**. When another Hermes gateway instance is configured with `GATEWAY_PROXY_URL` pointing at this API server, it forwards all messages here instead of running its own agent. This enables split deployments — for example, a Docker container handling Matrix E2EE that relays to a host-side agent.
See [Matrix Proxy Mode](/user-guide/messaging/matrix#proxy-mode-e2ee-on-macos) for the full setup guide.
website/docs/user-guide/features/batch-processing.md
+230
View File
@@ -0,0 +1,230 @@
---
sidebar_position: 12
title: "Batch Processing"
description: "Generate agent trajectories at scale — parallel processing, checkpointing, and toolset distributions"
---
# Batch Processing
Batch processing lets you run the Hermes agent across hundreds or thousands of prompts in parallel, generating structured trajectory data. This is primarily used for **training data generation** — producing ShareGPT-format trajectories with tool usage statistics that can be used for fine-tuning or evaluation.
## Overview
The batch runner (`batch_runner.py`) processes a JSONL dataset of prompts, running each through a full agent session with tool access. Each prompt gets its own isolated environment. The output is structured trajectory data with full conversation history, tool call statistics, and reasoning coverage metrics.
## Quick Start
```bash
# Basic batch run
python batch_runner.py \
--dataset_file=data/prompts.jsonl \
--batch_size=10 \
--run_name=my_first_run \
--model=anthropic/claude-sonnet-4.6 \
--num_workers=4
# Resume an interrupted run
python batch_runner.py \
--dataset_file=data/prompts.jsonl \
--batch_size=10 \
--run_name=my_first_run \
--resume
# List available toolset distributions
python batch_runner.py --list_distributions
```
:::tip Predictable cost at scale
Batch runs spin up many concurrent agent sessions, each making model calls and tool calls. A [Nous Portal](/user-guide/features/tool-gateway) subscription bundles model access plus web search, image gen, TTS, and cloud browsers under one bill — useful when you want stable cost-per-trajectory without juggling rate limits across five vendor accounts. Set up with `hermes setup --portal`, then point `--model` at a Nous model.
:::
## Dataset Format
The input dataset is a JSONL file (one JSON object per line). Each entry must have a `prompt` field:
```jsonl
{"prompt": "Write a Python function that finds the longest palindromic substring"}
{"prompt": "Create a REST API endpoint for user authentication using Flask"}
{"prompt": "Debug this error: TypeError: cannot unpack non-iterable NoneType object"}
```
Entries can optionally include:
- `image` or `docker_image`: A container image to use for this prompt's sandbox (works with Docker, Modal, and Singularity backends)
- `cwd`: Working directory override for the task's terminal session
## Configuration Options
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--dataset_file` | (required) | Path to JSONL dataset |
| `--batch_size` | (required) | Prompts per batch |
| `--run_name` | (required) | Name for this run (used for output dir and checkpointing) |
| `--distribution` | `"default"` | Toolset distribution to sample from |
| `--model` | `claude-sonnet-4.6` | Model to use |
| `--base_url` | `https://openrouter.ai/api/v1` | API base URL |
| `--api_key` | (env var) | API key for model |
| `--max_turns` | `10` | Maximum tool-calling iterations per prompt |
| `--num_workers` | `4` | Parallel worker processes |
| `--resume` | `false` | Resume from checkpoint |
| `--verbose` | `false` | Enable verbose logging |
| `--max_samples` | all | Only process first N samples from dataset |
| `--max_tokens` | model default | Maximum tokens per model response |
### Provider Routing (OpenRouter)
| Parameter | Description |
|-----------|-------------|
| `--providers_allowed` | Comma-separated providers to allow (e.g., `"anthropic,openai"`) |
| `--providers_ignored` | Comma-separated providers to ignore (e.g., `"together,deepinfra"`) |
| `--providers_order` | Comma-separated preferred provider order |
| `--provider_sort` | Sort by `"price"`, `"throughput"`, or `"latency"` |
### Reasoning Control
| Parameter | Description |
|-----------|-------------|
| `--reasoning_effort` | Reasoning effort: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`, `ultra` |
| `--reasoning_disabled` | Completely disable reasoning/thinking tokens |
### Advanced Options
| Parameter | Description |
|-----------|-------------|
| `--ephemeral_system_prompt` | System prompt used during execution but NOT saved to trajectories |
| `--log_prefix_chars` | Characters to show in log previews (default: 100) |
| `--prefill_messages_file` | Path to JSON file with prefill messages for few-shot priming |
## Toolset Distributions
Each prompt gets a randomly sampled set of toolsets from a **distribution**. This ensures training data covers diverse tool combinations. Use `--list_distributions` to see all available distributions.
In the current implementation, distributions assign a probability to **each individual toolset**. The sampler flips each toolset independently, then guarantees that at least one toolset is enabled. This is different from a hand-authored table of prebuilt combinations.
## Output Format
All output goes to `data/<run_name>/`:
```text
data/my_run/
├── trajectories.jsonl # Combined final output (all batches merged)
├── batch_0.jsonl # Individual batch results
├── batch_1.jsonl
├── ...
├── checkpoint.json # Resume checkpoint
└── statistics.json # Aggregate tool usage stats
```
### Trajectory Format
Each line in `trajectories.jsonl` is a JSON object:
```json
{
"prompt_index": 42,
"conversations": [
{"from": "human", "value": "Write a function..."},
{"from": "gpt", "value": "I'll create that function...",
"tool_calls": [...]},
{"from": "tool", "value": "..."},
{"from": "gpt", "value": "Here's the completed function..."}
],
"metadata": {
"batch_num": 2,
"timestamp": "2026-01-15T10:30:00",
"model": "anthropic/claude-sonnet-4.6"
},
"completed": true,
"partial": false,
"api_calls": 3,
"toolsets_used": ["terminal", "file"],
"tool_stats": {
"terminal": {"count": 2, "success": 2, "failure": 0},
"read_file": {"count": 1, "success": 1, "failure": 0}
},
"tool_error_counts": {
"terminal": 0,
"read_file": 0
}
}
```
The `conversations` field uses a ShareGPT-like format with `from` and `value` fields. Tool stats are normalized to include all possible tools with zero defaults, ensuring consistent schema across entries for HuggingFace datasets compatibility.
## Checkpointing
The batch runner has robust checkpointing for fault tolerance:
- **Checkpoint file:** Saved after each batch completes, tracking which prompt indices are done
- **Content-based resume:** On `--resume`, the runner scans existing batch files and matches completed prompts by their actual text content (not just indices), enabling recovery even if the dataset order changes
- **Failed prompts:** Only successfully completed prompts are marked as done — failed prompts will be retried on resume
- **Batch merging:** On completion, all batch files (including from previous runs) are merged into a single `trajectories.jsonl`
### How Resume Works
1. Scan all `batch_*.jsonl` files for completed prompts (by content matching)
2. Filter the dataset to exclude already-completed prompts
3. Re-batch the remaining prompts
4. Process only the remaining prompts
5. Merge all batch files (old + new) into final output
## Quality Filtering
The batch runner applies automatic quality filtering:
- **No-reasoning filter:** Samples where zero assistant turns contain reasoning (no `<REASONING_SCRATCHPAD>` or native thinking tokens) are discarded
- **Corrupted entry filter:** Entries with hallucinated tool names (not in the valid tool list) are filtered out during the final merge
- **Reasoning statistics:** Tracks percentage of turns with/without reasoning across the entire run
## Statistics
After completion, the runner prints comprehensive statistics:
- **Tool usage:** Call counts, success/failure rates per tool
- **Reasoning coverage:** Percentage of assistant turns with reasoning
- **Samples discarded:** Count of samples filtered for lacking reasoning
- **Duration:** Total processing time
Statistics are also saved to `statistics.json` for programmatic analysis.
## Use Cases
### Training Data Generation
Generate diverse tool-use trajectories for fine-tuning:
```bash
python batch_runner.py \
--dataset_file=data/coding_prompts.jsonl \
--batch_size=20 \
--run_name=coding_v1 \
--model=anthropic/claude-sonnet-4.6 \
--num_workers=8 \
--distribution=default \
--max_turns=15
```
### Model Evaluation
Evaluate how well a model uses tools across standardized prompts:
```bash
python batch_runner.py \
--dataset_file=data/eval_suite.jsonl \
--batch_size=10 \
--run_name=eval_gpt4 \
--model=openai/gpt-4o \
--num_workers=4 \
--max_turns=10
```
### Per-Prompt Container Images
For benchmarks requiring specific environments, each prompt can specify its own container image:
```jsonl
{"prompt": "Install numpy and compute eigenvalues of a 3x3 matrix", "image": "python:3.11-slim"}
{"prompt": "Compile this Rust program and run it", "image": "rust:1.75"}
{"prompt": "Set up a Node.js Express server", "image": "node:20-alpine", "cwd": "/app"}
```
The batch runner verifies Docker images are accessible before running each prompt.
website/docs/user-guide/features/browser.md
+859
View File
@@ -0,0 +1,859 @@
---
title: Browser Automation
description: Control browsers with multiple providers, local Chromium-family browsers via CDP, or cloud browsers for web interaction, form filling, scraping, and more.
sidebar_label: Browser
sidebar_position: 5
---
# Browser Automation
Hermes Agent includes a full browser automation toolset with multiple backend options:
- **Browser Use cloud mode** via [Browser Use](https://browser-use.com) for managed Chromium with stealth, residential proxies, CAPTCHA solving, and reusable browser profiles
- **Browserbase cloud mode** via [Browserbase](https://browserbase.com) as an alternative cloud browser provider with anti-bot tooling
- **Browser Use mode** via the [Browser Use CLI 3.0](https://github.com/browser-use/browser-use), the default browser driver for local Chrome and Browser Use cloud browsers
- **Firecrawl cloud mode** via [Firecrawl](https://firecrawl.dev) for cloud browsers with built-in scraping
- **Camofox local mode** via [Camofox](https://github.com/jo-inc/camofox-browser) for local anti-detection browsing (Firefox-based fingerprint spoofing)
- **Lightpanda local engine** via [Lightpanda](https://lightpanda.io) — a headless browser built from scratch in Zig for machines; instant start up, 16x lower memory and 9x faster than Chrome. Works in Browser Use mode (Hermes spawns it, no Chromium or Node needed) and with the built-in tools (automatic Chrome fallback for actions it doesn't support yet)
- **Local Chromium-family CDP** — connect browser tools to your own Chrome, Brave, Chromium, or Edge instance using `/browser connect`
- **Local browser mode** via the `agent-browser` CLI and a local Chromium installation
In all modes, the agent can navigate websites, interact with page elements, fill forms, and extract information.
## Overview
Pages are represented as **accessibility trees** (text-based snapshots), making them ideal for LLM agents. Interactive elements get ref IDs (like `@e1`, `@e2`) that the agent uses for clicking and typing.
Key capabilities:
- **Multi-provider cloud execution** — Browser Use, Browserbase, or Firecrawl — no local browser needed
- **Local Chromium-family integration** — attach to your running Chrome, Brave, Chromium, or Edge browser via CDP for hands-on browsing
- **Cloud anti-bot support** — Browser Use Cloud includes stealth, residential proxies, and CAPTCHA solving
- **Persistent cloud profiles** — Browser Use Cloud can reuse cookies, localStorage, and saved passwords across sessions
- **Session isolation** — each task gets its own browser session
- **Automatic cleanup** — inactive sessions are closed after a timeout
- **Vision analysis** — screenshot + AI analysis for visual understanding
## Setup
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, you can use browser automation through the **[Tool Gateway](tool-gateway.md)** without any separate API keys. New installs can run `hermes setup --portal` to log in and turn on every gateway tool at once; existing installs can pick **Nous Subscription** as the browser provider via `hermes model` or `hermes tools`.
:::
### Browser Use cloud mode
To use Browser Use as your cloud browser provider, add:
```bash
# Add to ~/.hermes/.env
BROWSER_USE_API_KEY=***
```
Get your API key at [browser-use.com](https://browser-use.com).
Browser Use Cloud runs managed Chromium with [stealth](https://docs.browser-use.com/cloud/browser/stealth) and [residential proxies](https://docs.browser-use.com/cloud/browser/proxies) enabled by default, includes CAPTCHA solving, and supports [persistent profiles](https://docs.browser-use.com/cloud/guides/authentication) for cookies, localStorage, and saved passwords.
### Browserbase cloud mode
To use Browserbase-managed cloud browsers, add:
```bash
# Add to ~/.hermes/.env
BROWSERBASE_API_KEY=***
BROWSERBASE_PROJECT_ID=your-project-id-here
```
Get your credentials at [browserbase.com](https://browserbase.com).
:::note Selecting the provider
The `.env` keys above supply **credentials only**. The active cloud browser is chosen by the `browser.cloud_provider` selection written by `hermes tools` → Browser Automation (`browserbase`, `browser-use`, `camofox`, or `nous` for the Nous Subscription). Once a selection exists, adding or removing a key does not switch providers — and a selected provider with a missing key errors with guidance to run `hermes tools` instead of silently rerouting. Never-configured setups still autodetect from available credentials.
:::
### Browser Use mode (default)
Browser Use mode uses the [Browser Use CLI 3.0](https://github.com/browser-use/browser-use) instead of the built-in browser tools. The agent writes and executes Python in the browser to click, type, drag, scrape, and interact with webpages.
**This is the default browser mode**: when `browser.backend` is unset and the `browser-use` CLI is runnable (installed, or available through `uvx`), the agent gets the single `browser_exec` tool. If the CLI can't run, Hermes falls back to the built-in browser tools automatically.
The mode is a **driver** that composes with your configured browser backend: it drives your local Chrome, a Nous-subscription cloud browser, Browserbase, Firecrawl, or Browser Use cloud browsers — whichever browser source is selected in `hermes tools` → Browser Automation. The one exception is Camofox, which has no CDP endpoint for the harness to attach to; Camofox setups automatically keep the built-in browser tools.
**Concurrent sessions:** `browser_exec` accepts a `session=<name>` argument that isolates browser work per name on every backend. Each name gets its own harness daemon (its own IPC socket, log, and state), and on cloud backends its own browser — so parallel subagents or simultaneous chats no longer clobber a single shared connection. Omitting `session` uses the shared default daemon, which is fine for one-at-a-time browsing.
To opt out and force the built-in browser tools, use `/browser use off`, or:
```yaml
# Add to ~/.hermes/config.yaml
browser:
backend: "off"
```
(`backend: "browser-use"` remains valid to force the mode explicitly.)
Browser Use's own cloud browsers need `browser-use auth login` or `BROWSER_USE_API_KEY`; other browser sources use their existing credentials unchanged.
:::note
Because Browser Use mode executes model-written Python on your machine, the
`browser_exec` tool is only offered to sessions that also have terminal
access. Platforms configured without the terminal toolset (e.g. a locked-down
messaging surface) keep the default browser tools instead.
:::
### Firecrawl cloud mode
To use Firecrawl as your cloud browser provider, add:
```bash
# Add to ~/.hermes/.env
FIRECRAWL_API_KEY=fc-***
```
Get your API key at [firecrawl.dev](https://firecrawl.dev). Then select Firecrawl as your browser provider:
```bash
hermes setup tools
# → Browser Automation → Firecrawl
```
Optional settings:
```bash
# Self-hosted Firecrawl instance (default: https://api.firecrawl.dev)
FIRECRAWL_API_URL=http://localhost:3002
# Session TTL in seconds (default: 300)
FIRECRAWL_BROWSER_TTL=600
```
### Hybrid routing: cloud for public URLs, local for LAN/localhost
When a cloud provider is configured, Hermes auto-spawns a **local Chromium sidecar**
for URLs that resolve to a private/loopback/LAN address (`localhost`, `127.0.0.1`,
`192.168.x.x`, `10.x.x.x`, `172.16-31.x.x`, `*.local`, `*.lan`, `*.internal`,
IPv6 loopback `::1`, link-local `169.254.x.x`). Public URLs continue to use the
cloud provider in the same conversation.
This solves the common "I'm developing locally but using Browserbase" workflow —
the agent can screenshot your dashboard at `http://localhost:3000` AND scrape
`https://github.com` without you switching providers or disabling the SSRF guard.
The cloud provider never sees the private URL.
The feature is **on by default**. To disable it (all URLs go to the configured
cloud provider, as before):
```yaml
# ~/.hermes/config.yaml
browser:
cloud_provider: browserbase
auto_local_for_private_urls: false
```
With auto-routing disabled, private URLs are rejected with
`"Blocked: URL targets a private or internal address"` unless you also set
`browser.allow_private_urls: true` (which lets the cloud provider attempt them —
usually won't work since Browserbase etc. can't reach your LAN).
Requirements: the local sidecar uses the same `agent-browser` CLI as pure local
mode, so you need it installed (`hermes setup tools → Browser Automation`
auto-installs it). Post-navigation redirects from a public URL onto a private
address are still blocked (you can't use a redirect-to-internal trick to reach
your LAN through the public path).
### Real profile browsing (use your own logins)
By default, local browsing runs in a clean, throwaway profile — the agent is
logged into nothing. Turn on **real profile browsing** to let the agent browse
as *you*, with your existing logins and cookies:
```yaml
# ~/.hermes/config.yaml
browser:
use_real_profile: true
```
When enabled, Hermes copies your default browser's **active** profile — the one
you actually browse (`Local State → profile.last_used`), with its cookies, saved
logins, and preferences — into a managed snapshot under
`~/.hermes/browser-profile/<browser>/`, then launches your **real browser
binary** on that snapshot and attaches its browsing engine to it. Launching the
real binary (instead of a bundled Chromium with mock-keychain switches) is what
keeps OS-encrypted cookies decryptable — on macOS, Chrome cookies are encrypted
through the Keychain, and a mock-keychain launch would silently drop every one
of them, opening signed out. Your live browser profile is **never opened
directly**: the
snapshot is a separate directory, so it doesn't fight your running browser for
the profile lock and it sidesteps Chrome 136+'s block on remote-debugging the
default profile directory. The auth files (cookies/logins/preferences) are
re-synced from your real profile whenever a fresh session is launched, so logins
you do in your own browser show up in the agent's session. Only the active
profile is copied — other Chrome profiles are never snapshotted.
The snapshot browser runs **headless** — it drives your profile in the
background with no visible window and never steals focus, so you can keep
working while the agent tweets, fills forms, or scrapes on your behalf.
(Headless here uses Chrome's *new* headless mode, which reads your normal
cookie store, so your logins still load.) If you'd rather watch it work, the
same [headed-mode](#headed-mode-visible-browser-window) toggle applies —
`browser.headed: true` (or `AGENT_BROWSER_HEADED=1`) opens a visible window for
real-profile browsing too. On a display-less host (servers, CI) it always runs
headless regardless.
If your browser has several profiles (say a work profile and a personal one)
and you don't want "whichever profile you touched last" deciding the agent's
identity, pin the snapshot source explicitly:
```yaml
# ~/.hermes/config.yaml
browser:
use_real_profile: true
real_profile_pin: "Profile 2" # directory name under the browser's user-data dir
```
A pin naming a profile directory that doesn't exist fails closed with a
fixable message — it never silently falls back to the last-used profile.
When you turn the toggle back off, Hermes deletes the snapshot store
(`~/.hermes/browser-profile/`) on the next browser use, so the copied
credentials don't linger after you revoke consent.
:::note Windows: the browser must be fully closed
On Windows a running Chrome/Edge/Brave holds its cookie and login databases with
an exclusive (deny-all) lock, so Hermes cannot copy them while the browser is
open — it fails fast with a "fully quit the browser and retry" message rather
than hang or produce a signed-out session. Real-profile browsing on Windows
therefore requires the browser **fully quit**, including any background/tray
instance (Chrome's "continue running background apps when closed" keeps a
`chrome.exe` alive after you close the window). macOS and Linux can copy the
profile while the browser is running.
Set `browser.real_profile_autoclose: true` to let Hermes **offer to close the
browser for you** when it's holding the profile. Even with this on, Hermes never
closes it automatically — when the profile is locked it always stops and the
agent asks you first; only on your approval does it run `hermes browser
close-profile` (terminates the browser process tree bound to that profile,
losing unsaved tabs), then retries. If the profile is still locked after that
(e.g. a background/tray instance relaunched), Hermes stays blocked and tells you
to fully quit the browser — it won't loop or kill again on its own.
:::
- **Supported browsers:** Chrome, Edge, Brave, Brave Origin, Chromium (whichever is your OS
default). A non-Chromium default (e.g. Firefox) fails closed with a clear
message rather than guessing.
- **Works on any backend.** On a local backend it's automatic once the toggle
is on. Under a **cloud** browser backend, the agent can still open a
real-profile local session on demand via the `browser_exec` tool's `local`
argument (the tool only exposes that argument when this toggle is on) — the
cloud backend keeps serving everything else.
- **Security framing:** this is a consent-gated convenience, not an isolation
boundary. A page the agent visits runs with your real logins, so only enable
it when you want the agent acting as you. Off by default.
- **Desktop:** toggle it in **Capabilities → Tools → Browser → Use My Real
Browser Profile** (the switch sits above the backend options), or in
Settings → Config under the `browser` section.
### Camofox local mode
[Camofox](https://github.com/jo-inc/camofox-browser) is a self-hosted Node.js server wrapping Camoufox (a Firefox fork with C++ fingerprint spoofing). It provides local anti-detection browsing without cloud dependencies.
```bash
# Clone the Camofox browser server first
git clone https://github.com/jo-inc/camofox-browser
cd camofox-browser
# Build and start with Docker using the default container settings
# (auto-detects arch: aarch64 on M1/M2, x86_64 on Intel)
make up
# Stop and remove the default container
make down
# Force a clean rebuild (for example, after upgrading VERSION/RELEASE)
make reset
# Just download binaries without building
make fetch
# Override arch or version explicitly
make up ARCH=x86_64
make up VERSION=135.0.1 RELEASE=beta.24
```
`make up` starts the default container immediately. If you want custom runtime settings such as a larger Node heap, VNC, or a persistent profile directory, build the image first and then run it yourself:
```bash
# Build the image without starting the default container
make build
# Start with persistence, VNC live view, and a larger Node heap
mkdir -p ~/.camofox-docker
docker run -d \
--name camofox-browser \
--restart unless-stopped \
-p 9377:9377 \
-p 6080:6080 \
-p 5901:5900 \
-e CAMOFOX_PORT=9377 \
-e ENABLE_VNC=1 \
-e VNC_BIND=0.0.0.0 \
-e VNC_RESOLUTION=1920x1080 \
-e MAX_OLD_SPACE_SIZE=2048 \
-v ~/.camofox-docker:/root/.camofox \
camofox-browser:135.0.1-aarch64
```
With VNC enabled, the browser runs in headed mode and can be watched live in your browser at `http://localhost:6080` (noVNC). You can also connect a native VNC client to `localhost:5901`.
If you already ran `make up`, stop and remove that default container before starting the custom one:
```bash
make down
# then run the custom docker run command above
```
Then set in `~/.hermes/.env`:
```bash
CAMOFOX_URL=http://localhost:9377
```
If Camofox is running in Docker and you want it to open web apps served from the host machine, enable loopback rewriting. `CAMOFOX_URL` should still point at the host-published control API, but page URLs such as `http://127.0.0.1:3000` must be opened from inside the container as `http://host.docker.internal:3000`:
```yaml
# ~/.hermes/config.yaml
browser:
camofox:
rewrite_loopback_urls: true
loopback_host_alias: host.docker.internal # default; use a LAN IP if needed
```
Equivalent env vars:
```bash
CAMOFOX_REWRITE_LOOPBACK_URLS=true
CAMOFOX_LOOPBACK_HOST_ALIAS=host.docker.internal
```
The rewrite only applies to page navigation URLs with loopback hosts (`localhost`, `127.0.0.1`, `::1`). It does not change `CAMOFOX_URL`. Leave it disabled for non-Docker Camofox installs, where the browser already runs on the host and loopback URLs are correct.
Or configure via `hermes tools` → Browser Automation → Camofox.
Camofox is selected like any other browser backend: pick **Camofox** in `hermes tools` → Browser Automation, which writes `browser.cloud_provider: camofox` to `config.yaml`. `CAMOFOX_URL` is only the server address — setting it no longer selects the backend by itself once a browser selection exists (never-configured setups still autodetect it).
#### Persistent browser sessions
By default, each Camofox session gets a random identity — cookies and logins don't survive across agent restarts. To enable persistent browser sessions, add the following to `~/.hermes/config.yaml`:
```yaml
browser:
camofox:
managed_persistence: true
```
Then fully restart Hermes so the new config is picked up.
:::warning Nested path matters
Hermes reads `browser.camofox.managed_persistence`, **not** a top-level `managed_persistence`. A common mistake is writing:
```yaml
# ❌ Wrong — Hermes ignores this
managed_persistence: true
```
If the flag is placed at the wrong path, Hermes silently falls back to a random ephemeral `userId` and your login state will be lost on every session.
:::
##### What Hermes does
- Sends a deterministic profile-scoped `userId` to Camofox so the server can reuse the same Firefox profile across sessions.
- Skips server-side context destruction on cleanup, so cookies and logins survive between agent tasks.
- Scopes the `userId` to the active Hermes profile, so different Hermes profiles get different browser profiles (profile isolation).
##### What Hermes does not do
- It does not force persistence on the Camofox server. Hermes only sends a stable `userId`; the server must honor it by mapping that `userId` to a persistent Firefox profile directory.
- If your Camofox server build treats every request as ephemeral (e.g. always calls `browser.newContext()` without loading a stored profile), Hermes cannot make those sessions persist. Make sure you are running a Camofox build that implements userId-based profile persistence.
##### Verify it's working
1. Start Hermes and your Camofox server.
2. Open Google (or any login site) in a browser task and sign in manually.
3. End the browser task normally.
4. Start a new browser task.
5. Open the same site again — you should still be signed in.
If step 5 logs you out, the Camofox server isn't honoring the stable `userId`. Double-check your config path, confirm you fully restarted Hermes after editing `config.yaml`, and verify your Camofox server version supports persistent per-user profiles.
##### Where state lives
Hermes derives the stable `userId` from the profile-scoped directory `~/.hermes/browser_auth/camofox/` (or the equivalent under `$HERMES_HOME` for non-default profiles). The actual browser profile data lives on the Camofox server side, keyed by that `userId`. To fully reset a persistent profile, clear it on the Camofox server and remove the corresponding Hermes profile's state directory.
#### Externally managed Camofox sessions
When another app drives the visible Camofox browser (a desktop assistant, a custom integration, another agent), configure Hermes to operate inside that same identity instead of spawning its own isolated profile.
Three knobs control the behavior:
| Setting | Env var | Effect |
|---------|---------|--------|
| `browser.camofox.user_id` | `CAMOFOX_USER_ID` | Camofox `userId` Hermes uses when creating tabs. Setting this opts the session into "externally managed" mode. |
| `browser.camofox.session_key` | `CAMOFOX_SESSION_KEY` | `sessionKey` (a.k.a. `listItemId`) sent on tab creation. Used to match an existing tab during adoption. Defaults to a per-task value if unset. |
| `browser.camofox.adopt_existing_tab` | `CAMOFOX_ADOPT_EXISTING_TAB` | When true, Hermes calls `GET /tabs?userId=<user_id>` on first use and reuses an existing tab before creating a new one. |
Env vars take precedence over `config.yaml`. Either form works:
```yaml
browser:
camofox:
user_id: shared-camofox
session_key: visible-tab
adopt_existing_tab: true
```
```bash
CAMOFOX_USER_ID=shared-camofox
CAMOFOX_SESSION_KEY=visible-tab
CAMOFOX_ADOPT_EXISTING_TAB=true
```
**What changes when `user_id` is set:**
- Hermes skips destructive cleanup at task end (same as `managed_persistence: true`). The other app's tab/cookies/profile survive.
- Hermes does **not** call `DELETE /sessions/<user_id>` — that endpoint wipes all user data, so it would nuke the external app's session if it fired.
**How tab adoption works (when `adopt_existing_tab: true`):**
1. On the first browser tool call after a process start, Hermes issues `GET /tabs?userId=<user_id>` (5-second timeout).
2. If any tab in the response has `listItemId == session_key`, Hermes adopts the most recently created one in that group.
3. Otherwise, Hermes adopts the most recently created tab for the user (any `listItemId`).
4. If no tabs exist or the request fails, Hermes falls back to creating a new tab on the next operation.
Adoption only fires until `tab_id` is populated for the session. If the external app closes the adopted tab mid-run, the next browser tool call will surface a Camofox error — Hermes does not re-poll for a fresh tab on every call.
**Picking `session_key`:** if you want Hermes to reliably attach to a *specific* existing tab, set `session_key` to the `listItemId` the external app used when creating it. If you leave `session_key` unset and only set `user_id`, Hermes generates a per-task `session_key` (`task_<id>`) — Hermes will share cookies and the profile with the external app, but will open its own tab alongside instead of reusing one.
**Concurrency note:** the external app and Hermes can drive the same Camofox `userId` simultaneously, but Camofox does not coordinate per-tab focus between clients. Coordinate ownership at the application layer (e.g. the external app pauses while Hermes runs).
#### VNC live view
When Camofox runs in headed mode (with a visible browser window), it exposes a VNC port in its health check response. Hermes automatically discovers this and includes the VNC URL in navigation responses, so the agent can share a link for you to watch the browser live.
### Lightpanda local engine
[Lightpanda](https://lightpanda.io) is an open-source headless browser written from scratch. It starts instantly, runs 9x faster and uses 16x less memory than Chrome, which matters for agents that live on small VMs for long stretches.
Lightpanda is a **local engine** (a browser source, like "Local Browser"), not a cloud provider. Install the binary and put it on your `PATH` (see the [Lightpanda installation guide](https://lightpanda.io/docs/run-locally/installation/one-liner)), then pick **Lightpanda** in `hermes tools` → Browser Automation, or set:
```yaml
# Add to ~/.hermes/config.yaml
browser:
cloud_provider: local
engine: lightpanda
```
Or via environment variable:
```bash
AGENT_BROWSER_ENGINE=lightpanda
```
The engine works with both browser drivers:
- **Browser Use mode (the default).** Hermes launches `lightpanda serve --host 127.0.0.1 --port <free>` itself — one process per `browser_exec` session name (or per task) — and points the Browser Use CLI at it. No Chromium, Playwright or Node.js is needed. The process is reaped after `browser.inactivity_timeout`, on exit, and by the orphan sweep if Hermes crashes. All of these processes share one on-disk HTTP cache at `$HERMES_HOME/cache/browser-use/lightpanda/http-cache`, so repeat visits skip re-downloading assets. Hermes passes the cache flag only when the installed Lightpanda supports it (0.3.x+); older binaries simply run without a cache. To clear it, stop your Lightpanda sessions first, then delete that directory. Lightpanda has no graphical renderer, so `capture_screenshot()` is unavailable and the tool description tells the model to work text-first; it also holds one page per session, so the model is told to call `new_tab()` once and `goto_url()` afterwards (tracked upstream in [lightpanda-io/browser#1962](https://github.com/lightpanda-io/browser/issues/1962)).
- **Built-in browser tools** (`/browser use off`). Hermes drives Lightpanda through `agent-browser --engine lightpanda` over CDP, the same way it drives local Chrome, with **automatic Chrome fallback**: Lightpanda handles the actions it supports (navigate, snapshot, click, type, scroll, back, press, eval) and Hermes transparently retries on Chrome for anything it doesn't. Screenshots and `browser_vision` are routed straight to Chrome.
**When the engine is ignored.** `browser.engine` is the lowest-precedence browser setting: a cloud provider (including the Nous subscription browser — and on never-configured setups, any `BROWSERBASE_API_KEY` / `BROWSER_USE_API_KEY` in `~/.hermes/.env` auto-selects one), Camofox, a `browser.cdp_url` / `/browser connect` override, or `browser.use_real_profile` all take precedence. Picking Lightpanda in `hermes tools` writes `cloud_provider: local` for you; `/browser status` and `hermes doctor` report when the engine is configured but shadowed, and by what.
### Local Chromium-family browser via CDP (`/browser connect`)
Instead of a cloud provider, you can attach Hermes browser tools to your own running Chrome, Brave, Chromium, or Edge instance via the Chrome DevTools Protocol (CDP). This is useful when you want to see what the agent is doing in real-time, interact with pages that require your own cookies/sessions, or avoid cloud browser costs.
:::note
`/browser connect` is an **interactive-CLI slash command** — it is not dispatched by the gateway. If you try to run it inside a WebUI, Telegram, Discord, or other gateway chat, the message will be sent to the agent as plain text and the command will not execute. Start Hermes from the terminal (`hermes` or `hermes chat`) and issue `/browser connect` there.
:::
In the CLI, use:
```
/browser connect # Auto-launch/connect to a local Chromium-family browser at http://127.0.0.1:9222
/browser connect ws://host:port # Connect to a specific CDP endpoint
/browser status # Check current connection
/browser disconnect # Detach and return to cloud/local mode
```
If a browser isn't already running with remote debugging, Hermes will attempt to auto-launch a supported Chromium-family browser with `--remote-debugging-port=9222`. Detection includes Brave, Brave Origin/Nightly, Google Chrome, Chromium, and Microsoft Edge, with common Linux install paths and binary names such as `brave-origin`, `brave-origin-nightly`, `/opt/brave.com/brave-origin/brave-origin`, `/opt/brave.com/brave-origin-nightly/brave-origin`, `/opt/brave-bin/brave`, and `/snap/bin/brave`.
:::tip
To start a Chromium-family browser manually with CDP enabled, use a dedicated user-data-dir so the debug port actually comes up even if the browser is already running with your normal profile:
```bash
# Linux — Brave
brave-browser \
--remote-debugging-port=9222 \
--user-data-dir=$HOME/.hermes/chrome-debug \
--no-first-run \
--no-default-browser-check &
# Linux — Google Chrome
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir=$HOME/.hermes/chrome-debug \
--no-first-run \
--no-default-browser-check &
# macOS — Brave
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser" \
--remote-debugging-port=9222 \
--user-data-dir="$HOME/.hermes/chrome-debug" \
--no-first-run \
--no-default-browser-check &
# macOS — Google Chrome
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--remote-debugging-port=9222 \
--user-data-dir="$HOME/.hermes/chrome-debug" \
--no-first-run \
--no-default-browser-check &
```
Then launch the Hermes CLI and run `/browser connect`.
**Why `--user-data-dir`?** Without it, launching a Chromium-family browser while a regular instance is already running typically opens a new window on the existing process — and that existing process was not started with `--remote-debugging-port`, so port 9222 never opens. A dedicated user-data-dir forces a fresh browser process where the debug port actually listens. `--no-first-run --no-default-browser-check` skips the first-launch wizard for the fresh profile.
**Chrome 136+ makes the dedicated profile mandatory.** As a security hardening change, Chrome 136 and later silently refuse to open the remote debugging port when `--remote-debugging-port` is combined with the *default* user-data-dir — even from a cold start with no other Chrome running. The browser launches normally but nothing ever listens on 9222, so `/browser connect` (and any manual `curl http://127.0.0.1:9222/json/version`) fails with connection refused. There is no error message. The fix is exactly the commands above: always pass a `--user-data-dir` pointing somewhere other than your default profile directory (e.g. `$HOME/.hermes/chrome-debug`). This applies to Chrome, Chromium, Edge, and Brave builds that have picked up the change.
:::
When connected via CDP, all browser tools (`browser_navigate`, `browser_click`, etc.) operate on your live browser instance instead of spinning up a cloud session.
### WSL2 + Windows Chrome: prefer MCP over `/browser connect`
If Hermes runs inside WSL2 but the Chrome window you want to control runs on the Windows host, `/browser connect` is often not the best path.
Why:
- `/browser connect` expects Hermes itself to reach a usable CDP endpoint
- modern Chrome live-debugging sessions often expose a host-local endpoint that is not directly reachable from WSL the same way a classic `9222` port is
- even when Windows Chrome is debuggable, the cleanest integration is often to let a Windows-side browser MCP server attach to Chrome and let Hermes talk to that MCP server
For that setup, prefer `chrome-devtools-mcp` through Hermes MCP support.
See the MCP guide for the practical setup:
- [Use MCP with Hermes](../../guides/use-mcp-with-hermes.md#wsl2-bridge-hermes-in-wsl-to-windows-chrome)
### Local browser mode
If you do **not** set any cloud credentials and don't use `/browser connect`, Hermes can still use the browser tools through a local Chromium install driven by `agent-browser`.
### Optional Environment Variables
```bash
# Residential proxies for better CAPTCHA solving (default: "true")
BROWSERBASE_PROXIES=true
# Advanced stealth with custom Chromium — requires Scale Plan (default: "false")
BROWSERBASE_ADVANCED_STEALTH=false
# Session reconnection after disconnects — requires paid plan (default: "true")
BROWSERBASE_KEEP_ALIVE=true
# Custom session timeout in seconds (max 21600 = 6 hours) (default: project default)
# Examples: 600 (10min), 1800 (30min), 21600 (6h max)
BROWSERBASE_SESSION_TIMEOUT=1800
# Inactivity timeout before auto-cleanup in seconds (default: 120)
BROWSER_INACTIVITY_TIMEOUT=120
# Local browser engine. Equivalent to browser.engine in config.yaml. In
# Browser Use mode (default) "lightpanda" makes Hermes spawn `lightpanda serve`;
# with the built-in tools it is passed to agent-browser as --engine.
# auto — Chrome (default)
# lightpanda — Lightpanda
# chrome — force Chrome explicitly
AGENT_BROWSER_ENGINE=auto
# Extra Chromium launch flags (comma- or newline-separated). Hermes auto-injects
# `--no-sandbox,--disable-dev-shm-usage` when it detects root or AppArmor-restricted
# unprivileged user namespaces (Ubuntu 23.10+, DGX Spark, many container images),
# so most users don't need to set this. Set it manually only if you need a flag
# Hermes doesn't add automatically; setting it disables the auto-injection.
AGENT_BROWSER_ARGS=--no-sandbox
```
### Install agent-browser CLI
You don't need to install anything — `agent-browser` resolves automatically via
`npx agent-browser` on first browser-tool use. To avoid the one-time npx fetch,
you can install it globally ahead of time (optional):
```bash
npm install -g agent-browser
```
:::info
The `browser` toolset must be included in your config's `toolsets` list or enabled via `hermes config set toolsets '["hermes-cli", "browser"]'`.
:::
## Available Tools
### `browser_navigate`
Navigate to a URL. Must be called before any other browser tool. Initializes the Browserbase session.
```
Navigate to https://github.com/NousResearch
```
:::tip
For simple information retrieval, prefer `web_search` or `web_extract` — they are faster and cheaper. Use browser tools when you need to **interact** with a page (click buttons, fill forms, handle dynamic content).
:::
### `browser_snapshot`
Get a text-based snapshot of the current page's accessibility tree. Returns interactive elements with ref IDs like `@e1`, `@e2` for use with `browser_click` and `browser_type`.
- **`full=false`** (default): Compact view showing only interactive elements
- **`full=true`**: Complete page content
Snapshots larger than `browser.snapshot_threshold` (default 15,000 characters — the same per-page budget as `web_extract`) are automatically truncated at line boundaries; no LLM summarization is involved. When that happens, the complete snapshot is saved to `~/.hermes/cache/web/` and the tool output includes the file path plus a ready-to-use `read_file` call, so the agent can page through the full accessibility tree — including element refs beyond the cut — without re-snapshotting.
Increase the threshold for long pages where more source content should reach the agent inline:
```yaml
# ~/.hermes/config.yaml
browser:
snapshot_threshold: 30000
```
You can also run `hermes config set browser.snapshot_threshold 30000`. The setting applies to both explicit `browser_snapshot` calls and the automatic snapshot returned after navigation, including the Camofox backend (minimum 1000). Restart the current Hermes session after changing it so the browser config cache reloads.
### `browser_click`
Click an element identified by its ref ID from the snapshot.
```
Click @e5 to press the "Sign In" button
```
### `browser_type`
Type text into an input field. Clears the field first, then types the new text.
```
Type "hermes agent" into the search field @e3
```
### `browser_scroll`
Scroll the page up or down to reveal more content.
```
Scroll down to see more results
```
### `browser_press`
Press a keyboard key. Useful for submitting forms or navigation.
```
Press Enter to submit the form
```
Supported keys: `Enter`, `Tab`, `Escape`, `ArrowDown`, `ArrowUp`, and more.
### `browser_back`
Navigate back to the previous page in browser history.
### `browser_get_images`
List all images on the current page with their URLs and alt text. Useful for finding images to analyze.
### `browser_vision`
Take a screenshot and analyze it with vision AI. Use this when text snapshots don't capture important visual information — especially useful for CAPTCHAs, complex layouts, or visual verification challenges.
The screenshot is saved persistently and the file path is returned alongside the AI analysis. On messaging platforms (Telegram, Discord, Slack, WhatsApp), you can ask the agent to share the screenshot — it will be sent as a native photo attachment via the `MEDIA:` mechanism.
```
What does the chart on this page show?
```
Screenshots are stored in `~/.hermes/cache/screenshots/` and automatically cleaned up after 24 hours.
### `browser_console`
Get browser console output (log/warn/error messages) and uncaught JavaScript exceptions from the current page. Essential for detecting silent JS errors that don't appear in the accessibility tree.
```
Check the browser console for any JavaScript errors
```
Use `clear=True` to clear the console after reading, so subsequent calls only show new messages.
`browser_console` also evaluates JavaScript when called with an `expression` argument — same shape as DevTools console, the result comes back parsed (JSON-serialized objects become dicts; primitive values stay primitive).
```
browser_console(expression="document.querySelector('h1').textContent")
browser_console(expression="JSON.stringify(performance.timing)")
```
When a CDP supervisor is active for the current session (typical for any session that's run `browser_navigate` against a CDP-capable backend), evaluation runs over the supervisor's persistent WebSocket — no subprocess startup cost. Falls through to the standard agent-browser CLI path otherwise. Behaviour is identical either way; only latency changes.
Evaluation is unrestricted by default — the agent can use `fetch`, read storage, query form values, and run any DOM extraction. Requests targeting private/internal addresses are still blocked on non-local backends (the SSRF guard is independent of this setting). If you browse hostile pages with a logged-in profile and want a strict denylist over sensitive JS primitives (cookies, storage, clipboard, network calls, form values), opt in with `browser.restrict_evaluate: true` in `config.yaml`. Note the denylist matches primitive *names*, so it also blocks legitimate expressions that merely contain words like `fetch` or `cookie`.
### `browser_cdp`
Raw Chrome DevTools Protocol passthrough — the escape hatch for browser operations not covered by the other tools. Use for native dialog handling, iframe-scoped evaluation, cookie/network control, or any CDP verb the agent needs.
**Only available when a CDP endpoint is reachable at session start** — meaning `/browser connect` has attached to a running Chrome, Brave, Chromium, or Edge browser, or `browser.cdp_url` is set in `config.yaml`. The default local agent-browser mode, Camofox, and cloud providers (Browserbase, Browser Use, Firecrawl) do not currently expose CDP to this tool — cloud providers have per-session CDP URLs but live-session routing is a follow-up.
**CDP method reference:** https://chromedevtools.github.io/devtools-protocol/ — the agent can `web_extract` a specific method's page to look up parameters and return shape.
Common patterns:
```
# List tabs (browser-level, no target_id)
browser_cdp(method="Target.getTargets")
# Handle a native JS dialog on a tab
browser_cdp(method="Page.handleJavaScriptDialog",
params={"accept": true, "promptText": ""},
target_id="<tabId>")
# Evaluate JS in a specific tab
browser_cdp(method="Runtime.evaluate",
params={"expression": "document.title", "returnByValue": true},
target_id="<tabId>")
# Get all cookies
browser_cdp(method="Network.getAllCookies")
```
Browser-level methods (`Target.*`, `Browser.*`, `Storage.*`) omit `target_id`. Page-level methods (`Page.*`, `Runtime.*`, `DOM.*`, `Emulation.*`) require a `target_id` from `Target.getTargets`. Each stateless call is independent — sessions do not persist between calls.
**Cross-origin iframes:** pass `frame_id` (from `browser_snapshot.frame_tree.children[]` where `is_oopif=true`) to route the CDP call through the supervisor's live session for that iframe. This is how `Runtime.evaluate` inside a cross-origin iframe works on Browserbase, where stateless CDP connections would hit signed-URL expiry. Example:
```
browser_cdp(
method="Runtime.evaluate",
params={"expression": "document.title", "returnByValue": True},
frame_id="<frame_id from browser_snapshot>",
)
```
Same-origin iframes don't need `frame_id` — use `document.querySelector('iframe').contentDocument` from a top-level `Runtime.evaluate` instead.
### `browser_dialog`
Responds to a native JS dialog (`alert` / `confirm` / `prompt` / `beforeunload`). Before this tool existed, dialogs would silently block the page's JavaScript thread and subsequent `browser_*` calls would hang or throw; now the agent sees pending dialogs in `browser_snapshot` output and responds explicitly.
**Workflow:**
1. Call `browser_snapshot`. If a dialog is blocking the page, it shows up as `pending_dialogs: [{"id": "d-1", "type": "alert", "message": "..."}]`.
2. Call `browser_dialog(action="accept")` or `browser_dialog(action="dismiss")`. For `prompt()` dialogs, pass `prompt_text="..."` to supply the response.
3. Re-snapshot — `pending_dialogs` is empty; the page's JS thread has resumed.
**Detection happens automatically** via a persistent CDP supervisor — one WebSocket per task that subscribes to Page/Runtime/Target events. The supervisor also populates a `frame_tree` field in the snapshot so the agent can see the iframe structure of the current page, including cross-origin (OOPIF) iframes.
**Availability matrix:**
| Backend | Detection via `pending_dialogs` | Response (`browser_dialog` tool) |
|---|---|---|
| Local Chrome via `/browser connect` or `browser.cdp_url` | ✓ | ✓ full workflow |
| Browserbase | ✓ | ✓ full workflow (via injected XHR bridge) |
| Camofox / default local agent-browser | ✗ | ✗ (no CDP endpoint) |
**How it works on Browserbase.** Browserbase's CDP proxy auto-dismisses real native dialogs server-side within ~10ms, so we can't use `Page.handleJavaScriptDialog`. The supervisor injects a small script via `Page.addScriptToEvaluateOnNewDocument` that overrides `window.alert`/`confirm`/`prompt` with a synchronous XHR. We intercept those XHRs via `Fetch.enable` — the page's JS thread stays blocked on the XHR until we call `Fetch.fulfillRequest` with the agent's response. `prompt()` return values round-trip back into page JS unchanged.
**Dialog policy** is configured in `config.yaml` under `browser.dialog_policy`:
| Policy | Behavior |
|--------|----------|
| `must_respond` (default) | Capture, surface in snapshot, wait for explicit `browser_dialog()` call. Safety auto-dismiss after `browser.dialog_timeout_s` (default 300s) so a buggy agent can't stall forever. |
| `auto_dismiss` | Capture, dismiss immediately. Agent still sees the dialog in `browser_state` history but doesn't have to act. |
| `auto_accept` | Capture, accept immediately. Useful when navigating pages with aggressive `beforeunload` prompts. |
**Frame tree** inside `browser_snapshot.frame_tree` is capped to 30 frames and OOPIF depth 2 to keep payloads bounded on ad-heavy pages. A `truncated: true` flag surfaces when limits were hit; agents needing the full tree can use `browser_cdp` with `Page.getFrameTree`.
## Practical Examples
### Filling Out a Web Form
```
User: Sign up for an account on example.com with my email john@example.com
Agent workflow:
1. browser_navigate("https://example.com/signup")
2. browser_snapshot() → sees form fields with refs
3. browser_type(ref="@e3", text="john@example.com")
4. browser_type(ref="@e5", text="SecurePass123")
5. browser_click(ref="@e8") → clicks "Create Account"
6. browser_snapshot() → confirms success
```
### Researching Dynamic Content
```
User: What are the top trending repos on GitHub right now?
Agent workflow:
1. browser_navigate("https://github.com/trending")
2. browser_snapshot(full=true) → reads trending repo list
3. Returns formatted results
```
## Session Recording
Automatically record browser sessions as WebM video files:
```yaml
browser:
record_sessions: true # default: false
```
When enabled, recording starts automatically on the first `browser_navigate` and saves to `~/.hermes/browser_recordings/` when the session closes. Works in both local and cloud (Browserbase) modes. Recordings older than 72 hours are automatically cleaned up.
## Headed Mode (Visible Browser Window)
By default, the local browser runs headless. Enable headed mode to get a visible Chromium window you can watch and interact with:
```yaml
browser:
headed: true # default: false
```
Or via environment variable: `AGENT_BROWSER_HEADED=1`.
Headed mode does two things:
1. **Launches Chromium with a visible window** (passes `--headed` to agent-browser in local mode).
2. **Keeps the window open between turns.** Normally the browser session is cleaned up after every agent reply; in headed mode the per-turn cleanup is skipped so you can watch the agent work, intervene manually (sign-in challenges, CAPTCHAs), and keep login state warm across the conversation.
Idle sessions are still reaped after `browser.inactivity_timeout` (default 120s of no browser activity), and all sessions are closed on shutdown. Headed mode only affects the local browser — cloud sessions (Browserbase) are unaffected.
## Stealth Features
Browserbase provides automatic stealth capabilities:
| Feature | Default | Notes |
|---------|---------|-------|
| Basic Stealth | Always on | Random fingerprints, viewport randomization, CAPTCHA solving |
| Residential Proxies | On | Routes through residential IPs for better access |
| Advanced Stealth | Off | Custom Chromium build, requires Scale Plan |
| Keep Alive | On | Session reconnection after network hiccups |
:::note
If paid features aren't available on your plan, Hermes automatically falls back — first disabling `keepAlive`, then proxies — so browsing still works on free plans.
:::
## Session Management
- Each task gets an isolated browser session via Browserbase
- Sessions are automatically cleaned up after inactivity (default: 2 minutes)
- A background thread checks every 30 seconds for stale sessions
- Emergency cleanup runs on process exit to prevent orphaned sessions
- Sessions are released via the Browserbase API (`REQUEST_RELEASE` status)
## Limitations
- **Text-based interaction** — relies on accessibility tree, not pixel coordinates
- **Snapshot size** — large pages are truncated at `browser.snapshot_threshold` (default 15,000 characters, matching `web_extract`; no LLM summarization); the complete snapshot is saved to `~/.hermes/cache/web/` and the output points at it for `read_file` paging
- **Session timeout** — cloud sessions expire based on your provider's plan settings
- **Cost** — cloud sessions consume provider credits; sessions are automatically cleaned up when the conversation ends or after inactivity. Use `/browser connect` for free local browsing.
- **No file downloads** — cannot download files from the browser
website/docs/user-guide/features/built-in-plugins.md
+329
View File
@@ -0,0 +1,329 @@
---
sidebar_position: 12
sidebar_label: "Built-in Plugins"
title: "Built-in Plugins"
description: "Plugins shipped with Hermes Agent that run automatically via lifecycle hooks — disk-cleanup and friends"
---
# Built-in Plugins
Hermes ships a small set of plugins bundled with the repository. They live under `<repo>/plugins/<name>/` and load automatically alongside user-installed plugins in `~/.hermes/plugins/`. They use the same plugin surface as third-party plugins — hooks, tools, slash commands — just maintained in-tree.
See the [Plugins](/user-guide/features/plugins) page for the general plugin system, and [Build a Hermes Plugin](/developer-guide/plugins) to write your own.
## How discovery works
The `PluginManager` scans four sources, in order:
1. **Bundled** — `<repo>/plugins/<name>/` (what this page documents)
2. **User** — `~/.hermes/plugins/<name>/`
3. **Project** — `./.hermes/plugins/<name>/` (requires `HERMES_ENABLE_PROJECT_PLUGINS=1`)
4. **Pip entry points** — `hermes_agent.plugins`
On name collision, later sources win — a user plugin named `disk-cleanup` would replace the bundled one.
`plugins/memory/` and `plugins/context_engine/` are deliberately excluded from bundled scanning. Those directories use their own discovery paths because memory providers and context engines are single-select providers configured through `hermes memory setup` / `context.engine` in config.
## Bundled plugins are opt-in
Bundled plugins ship disabled. Discovery finds them (they appear in `hermes plugins list` and the interactive `hermes plugins` UI), but none load until you explicitly enable them:
```bash
hermes plugins enable disk-cleanup
```
Or via `~/.hermes/config.yaml`:
```yaml
plugins:
enabled:
- disk-cleanup
```
This is the same mechanism user-installed plugins use. Bundled plugins are never auto-enabled — not on fresh install, not for existing users upgrading to a newer Hermes. You always opt in explicitly.
To turn a bundled plugin off again:
```bash
hermes plugins disable disk-cleanup
# or: remove it from plugins.enabled in config.yaml
```
## Currently shipped
The repo ships these bundled plugins under `plugins/`. All are opt-in — enable them via `hermes plugins enable <name>`.
| Plugin | Kind | Purpose |
|---|---|---|
| `disk-cleanup` | hooks + slash command | Auto-track ephemeral files and clean them on session end |
| `security-guidance` | hooks | Pattern-match dangerous code on `write_file`/`patch` and append a security warning (or block) — 25 rules (Apache-2.0 fork of Anthropic's `claude-plugins-official` patterns) |
| `observability/langfuse` | hooks | Trace turns / LLM calls / tools to [Langfuse](https://langfuse.com) |
| `teams_pipeline` | standalone | Microsoft Teams meeting pipeline — Graph-backed, transcript-first meeting summaries |
| `spotify` | backend (7 tools) | Native Spotify playback, queue, search, playlists, albums, library |
| `google_meet` | standalone | Join Meet calls, live-caption transcription, optional realtime duplex audio |
| `image_gen/openai` | image backend | OpenAI `gpt-image-2` image generation backend (alternative to FAL) |
| `image_gen/openai-codex` | image backend | OpenAI image generation via Codex OAuth |
| `image_gen/xai` | image backend | xAI `grok-2-image` backend |
| `hermes-achievements` | dashboard tab | Steam-style collectible badges generated from your real Hermes session history |
| `kanban/dashboard` | dashboard tab | Kanban board UI for the multi-agent dispatcher — tasks, comments, fan-out, board switching. See [Kanban Multi-Agent](./kanban.md). |
Memory providers (`plugins/memory/*`) and context engines (`plugins/context_engine/*`) are listed separately on [Memory Providers](./memory-providers.md) — they're managed through `hermes memory` and `hermes plugins` respectively. The full per-plugin detail for the two long-running hooks-based plugins follows.
### disk-cleanup
Auto-tracks and removes ephemeral files created during sessions — test scripts, temp outputs, cron logs, stale chrome profiles — without requiring the agent to remember to call a tool.
**How it works:**
| Hook | Behaviour |
|---|---|
| `post_tool_call` | When `write_file` / `terminal` / `patch` creates a file matching `test_*`, `tmp_*`, or `*.test.*` inside `HERMES_HOME` or `/tmp/hermes-*`, track it silently as `test` / `temp` / `cron-output`. |
| `on_session_end` | If any test files were auto-tracked during the turn, run the safe `quick` cleanup and log a one-line summary. Stays silent otherwise. |
**Deletion rules:**
| Category | Threshold | Confirmation |
|---|---|---|
| `test` | every session end | Never |
| `temp` | >7 days since tracked | Never |
| `cron-output` | >14 days since tracked | Never |
| empty dirs under HERMES_HOME | always | Never |
| `research` | >30 days, beyond 10 newest | Always (deep only) |
| `chrome-profile` | >14 days since tracked | Always (deep only) |
| files >500 MB | never auto | Always (deep only) |
**Slash command** — `/disk-cleanup` available in both CLI and gateway sessions:
```
/disk-cleanup status # breakdown + top-10 largest
/disk-cleanup dry-run # preview without deleting
/disk-cleanup quick # run safe cleanup now
/disk-cleanup deep # quick + list items needing confirmation
/disk-cleanup track <path> <category> # manual tracking
/disk-cleanup forget <path> # stop tracking (does not delete)
```
**State** — everything lives at `$HERMES_HOME/disk-cleanup/`:
| File | Contents |
|---|---|
| `tracked.json` | Tracked paths with category, size, and timestamp |
| `tracked.json.bak` | Atomic-write backup of the above |
| `cleanup.log` | Append-only audit trail of every track / skip / reject / delete |
**Safety** — cleanup only ever touches paths under `HERMES_HOME` or `/tmp/hermes-*`. Windows mounts (`/mnt/c/...`) are rejected. Well-known top-level state dirs (`logs/`, `memories/`, `sessions/`, `cron/`, `cache/`, `skills/`, `plugins/`, `disk-cleanup/` itself) are never removed even when empty — a fresh install does not get gutted on first session end.
**Enabling:** `hermes plugins enable disk-cleanup` (or check the box in `hermes plugins`).
**Disabling again:** `hermes plugins disable disk-cleanup`.
### security-guidance
Fast pattern-matched security warnings on file writes. When the agent's `write_file` / `patch` / `skill_manage` calls carry content matching a known-dangerous code pattern — `pickle.load`, `yaml.load` without `SafeLoader`, `eval(`, `os.system`, `subprocess(..., shell=True)`, JS `child_process.exec`, React `dangerouslySetInnerHTML`, raw `.innerHTML =` / `.outerHTML =` / `document.write`, Node `crypto.createCipher`, AES ECB mode, TLS verification disabled, XXE-prone `xml.etree` / `minidom` parsers, `<script src="//..." >` without SRI, `torch.load` without `weights_only=True`, GitHub Actions `${{ github.event.* }}` injection — the plugin appends a `⚠️ Security guidance` block to the tool's result.
The file is still written. The model reads the warning in the next turn's tool message and can either fix the code or document why the construct is safe in this context. Pattern matching has a non-trivial false-positive rate, which is why warn (not block) is the default.
**Coverage:** 25 rules total, covering unsafe deserialization, command injection, XSS sinks, crypto footguns, XXE, supply-chain (SRI), and CI/CD workflow injection. The pattern data is a verbatim Apache-2.0 fork of [Anthropic's `claude-plugins-official`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/security-guidance/hooks) — see the plugin's `LICENSE` and `NOTICE` files for attribution.
**Modes:**
| Env var | Effect |
|---|---|
| (unset) | **warn mode** (default) — file is written, warning appended to result |
| `SECURITY_GUIDANCE_BLOCK=1` | **block mode** — write refused, warning returned as the block reason |
| `SECURITY_GUIDANCE_DISABLE=1` | kill switch — plugin loads but does nothing |
**Enabling:** `hermes plugins enable security-guidance` (or check the box in `hermes plugins`).
**Disabling again:** `hermes plugins disable security-guidance`.
**What it does not do (yet):** the upstream Anthropic plugin has two more layers — an LLM diff review on each agent turn that touched files, and an agentic commit-time review that traces data flow across files. Neither is ported. The agent can already run those reviews on demand via `delegate_task`.
### observability/langfuse
Traces Hermes turns, LLM calls, and tool invocations to [Langfuse](https://langfuse.com) — an open-source LLM observability platform. One span per turn, one generation per API call, one tool observation per tool call. Usage totals, per-type token counts, and cost estimates come out of Hermes' canonical `agent.usage_pricing` numbers, so the Langfuse dashboard sees the same breakdown (input / output / `cache_read_input_tokens` / `cache_creation_input_tokens` / `reasoning_tokens`) that appears in `hermes logs`.
The plugin is fail-open: no SDK installed, no credentials, or a transient Langfuse error — all turn into a silent no-op in the hook. The agent loop is never impacted.
**Setup (interactive — recommended):**
```bash
hermes tools # → Langfuse Observability → Cloud or Self-Hosted
```
The wizard collects your keys, `pip install`s the `langfuse` SDK, and adds `observability/langfuse` to `plugins.enabled` for you. Restart Hermes and the next turn ships a trace.
**Setup (manual):**
```bash
pip install langfuse
hermes plugins enable observability/langfuse
```
Then put the credentials in `~/.hermes/.env`:
```bash
HERMES_LANGFUSE_PUBLIC_KEY=pk-lf-...
HERMES_LANGFUSE_SECRET_KEY=sk-lf-...
HERMES_LANGFUSE_BASE_URL=https://cloud.langfuse.com # or your self-hosted URL
```
**How it works:**
| Hook | Behaviour |
|---|---|
| `pre_api_request` / `pre_llm_call` | Open (or reuse) a per-turn root span "Hermes turn". Start a `generation` child observation for this API call with serialized recent messages as input. |
| `post_api_request` / `post_llm_call` | Close the generation, attach `usage_details`, `cost_details`, `finish_reason`, assistant output + tool calls. If no tool calls and non-empty content, close the turn. |
| `pre_tool_call` | Start a `tool` child observation with sanitized `args`. |
| `post_tool_call` | Close the tool observation with sanitized `result`. `read_file` payloads get summarized (head + tail + omitted-line count) so a huge file read stays under `HERMES_LANGFUSE_MAX_CHARS`. |
Session grouping keys off the Hermes session ID (or task ID for sub-agents) via `langfuse.propagate_attributes`, so everything in a single `hermes chat` session lives under one Langfuse session.
**Verify:**
```bash
hermes plugins list # observability/langfuse should show "enabled"
hermes chat -q "hello" # check the Langfuse UI for a "Hermes turn" trace
```
**Optional tuning** (in `.env`):
| Variable | Default | Purpose |
|---|---|---|
| `HERMES_LANGFUSE_ENV` | — | Environment tag on traces (`production`, `staging`, …) |
| `HERMES_LANGFUSE_RELEASE` | — | Release/version tag |
| `HERMES_LANGFUSE_SAMPLE_RATE` | `1.0` | Sampling rate passed to the SDK (0.0–1.0) |
| `HERMES_LANGFUSE_MAX_CHARS` | `12000` | Per-field truncation for message content / tool args / tool results |
| `HERMES_LANGFUSE_DEBUG` | `false` | Verbose plugin logging to `agent.log` |
Hermes-prefixed and standard SDK env vars (`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, `LANGFUSE_BASE_URL`) are both accepted — Hermes-prefixed wins when both are set.
**Performance:** the Langfuse client is cached after the first hook call. If credentials or SDK are missing, that decision is also cached — subsequent hooks fast-return without re-checking env vars or reloading config.
**Disabling:** `hermes plugins disable observability/langfuse`. The plugin module is still discovered, but no module code runs until you re-enable.
### NeMo Relay native integration (migration note)
NeMo Relay is no longer a bundled Hermes plugin. Do not run `hermes plugins enable observability/nemo_relay`; Hermes core now owns the Relay session, turn, LLM, and tool lifecycles.
To opt into Relay middleware or exporters, create a standard Relay `plugins.toml`, then set `HERMES_NEMO_RELAY_PLUGINS_TOML` to that file before starting Hermes. The policy is process-wide for every profile hosted by that Hermes process. See the [NeMo Relay observability configuration](https://docs.nvidia.com/nemo/relay/configure-plugins/observability/about) for ATOF, ATIF, and OpenTelemetry options.
The old `HERMES_NEMO_RELAY_ATOF_*` and `HERMES_NEMO_RELAY_ATIF_*` settings no longer activate exporters. `hermes doctor` reports these stale settings when no replacement `plugins.toml` is selected.
#### Session-span segmentation (continuous sessions)
Relay exports a span when its scope closes. A continuous gateway session can keep its session span open for days even though each turn span exports normally. Optional segmentation rotates only the session scope at a turn boundary:
```yaml
gateway:
telemetry:
session_segments:
on_compaction: false # rotate after context compaction
max_turns: 0 # 0 = unlimited; N = turns per segment
```
| Key | Default | Behavior |
|---|---:|---|
| `on_compaction` | `false` | Rotate after compaction completes, at the next turn boundary. |
| `max_turns` | `0` | Rotate after every N completed turns; `0` disables the cap. |
Both defaults preserve one session scope for the full session. Rotated spans retain the same `session_id` and add `hermes.session.segment` plus `hermes.session.segment_reason` (`compaction` or `max_turns`).
### google_meet
Lets the agent **join, transcribe, and participate in Google Meet calls** — take notes on a meeting, summarize the back-and-forth after, follow up on specific points, and (optionally) speak replies back into the call via TTS.
**What it adds:**
- A headless virtual participant that joins a Meet URL using browser automation
- Live transcription of the meeting audio via the configured STT provider
- A `meet_join` / `meet_status` / `meet_transcript` / `meet_leave` / `meet_say` toolset the agent invokes to join calls, poll the live transcript, and act on what it heard
- Post-meeting artifacts (transcript, status) saved under `~/.hermes/workspace/meetings/<meeting_id>/`
**Setup:**
```bash
hermes plugins enable google_meet
hermes meet setup # preflight: playwright, chromium, auth file
hermes meet auth # opens a browser to sign into Google and saves session state —
# needs a Google account with Meet access. Host approval may be
# required if the meeting enforces "only invited participants can join".
```
Usage from chat:
> "Join meet.google.com/abc-defg-hij and take notes. After the call, send me a summary with action items."
The agent kicks off the meeting join, streams the transcription back into its context as the call proceeds, and produces a structured summary when the meeting ends (or when you tell it to stop).
**When to use it:** recurring standups where you want a bot to transcribe + summarize for async attendees; deposition-style interviews where you want structured notes; any case where you'd otherwise need Fireflies / Otter / Grain. When you'd rather not have an AI listening in — don't enable it.
**Disabling:** `hermes plugins disable google_meet`. Any saved transcripts stay in `~/.hermes/workspace/meetings/` until you remove them.
### hermes-achievements
Adds a **Steam-style achievements tab to the dashboard** — 60+ collectible, tiered badges generated from your real Hermes session history. Tool-chain feats, debugging patterns, vibe-coding streaks, skill/memory usage, model/provider variety, lifestyle quirks (weekend and night sessions). Originally authored by [@PCinkusz](https://github.com/PCinkusz) as an external plugin; brought in-tree so it stays in lockstep with Hermes feature changes.
**How it works:**
- Scans your entire `~/.hermes/state.db` session history on the dashboard backend
- Per-session stats are cached by `(started_at, last_active)` fingerprint, so only new or changed sessions re-analyze on subsequent scans
- First-ever scan runs in a background thread — the dashboard never blocks waiting for it, even on databases with thousands of sessions
- Unlock state is persisted to `$HERMES_HOME/plugins/hermes-achievements/state.json`
**Tier progression:** Copper → Silver → Gold → Diamond → Olympian. Each card exposes a "What counts" section listing the exact metric being tracked.
**Achievement states:**
| State | Meaning |
|---|---|
| Unlocked | At least one tier achieved |
| Discovered | Known achievement, progress visible, not yet earned |
| Secret | Hidden until Hermes detects the first related signal in your history |
**API** — routes mount under `/api/plugins/hermes-achievements/`:
| Endpoint | Purpose |
|---|---|
| `GET /achievements` | Full catalog with per-badge unlock state (returns a pending placeholder while the first cold scan is running) |
| `GET /scan-status` | State of the background scanner: `idle` / `running` / `failed`, last duration, run count |
| `GET /recent-unlocks` | Twenty most recently unlocked badges, newest first |
| `GET /sessions/{id}/badges` | Badges earned primarily in one specific session |
| `POST /rescan` | Manual synchronous rescan (blocks; use when the user clicks the rescan button) |
| `POST /reset-state` | Clear unlock history and cached snapshot |
**State files** — live under `$HERMES_HOME/plugins/hermes-achievements/`:
| File | Contents |
|---|---|
| `state.json` | Unlock history: which badges you've earned and when. Stable across Hermes updates. |
| `scan_snapshot.json` | Last completed scan payload (served immediately on dashboard load) |
| `scan_checkpoint.json` | Per-session stats cache keyed by fingerprint (makes warm rescans fast) |
**Performance notes:**
- Cold scan on ~8,000 sessions takes a few minutes. It runs in a background thread on first dashboard request; the UI sees a pending placeholder and polls `/scan-status`.
- **Incremental results during a cold scan** — the scanner publishes a partial snapshot every ~250 sessions so each dashboard refresh shows more badges unlocked as the scan progresses. No minute-long stare at zeros.
- Warm rescan reuses per-session stats for every session whose `started_at` + `last_active` fingerprint matches the checkpoint — completes in seconds even on large histories.
- The in-memory snapshot TTL is 120s; stale requests serve the old snapshot immediately and kick a background refresh. You never wait on a spinner just because TTL expired.
**Enabling:** Nothing to enable — `hermes-achievements` is a dashboard-only plugin (no lifecycle hooks, no model-visible tools). It auto-registers as a tab in `hermes dashboard` on first launch. The `plugins.enabled` config only gates lifecycle/tool plugins; dashboard plugins are discovered purely via their `dashboard/manifest.json`.
**Opting out:** Delete or rename `plugins/hermes-achievements/dashboard/manifest.json`, or override it with a user plugin of the same name in `~/.hermes/plugins/hermes-achievements/` that ships no dashboard. The plugin's state files under `$HERMES_HOME/plugins/hermes-achievements/` survive — reinstalling preserves your unlock history.
## Adding a bundled plugin
Bundled plugins are written exactly like any other Hermes plugin — see [Build a Hermes Plugin](/developer-guide/plugins). The only differences are:
- Directory lives at `<repo>/plugins/<name>/` instead of `~/.hermes/plugins/<name>/`
- Manifest source is reported as `bundled` in `hermes plugins list`
- User plugins with the same name override the bundled version
A plugin is a good candidate for bundling when:
- It has no optional dependencies (or they're already `pip install .[all]` deps)
- The behaviour benefits most users and is opt-out rather than opt-in
- The logic ties into lifecycle hooks that the agent would otherwise have to remember to invoke
- It complements a core capability without expanding the model-visible tool surface
Counter-examples — things that should stay as user-installable plugins, not bundled: third-party integrations with API keys, niche workflows, large dependency trees, anything that would meaningfully change agent behaviour by default.
website/docs/user-guide/features/code-execution.md
+296
View File
@@ -0,0 +1,296 @@
---
sidebar_position: 8
title: "Code Execution"
description: "Programmatic Python execution with RPC tool access — collapse multi-step workflows into a single turn"
---
# Code Execution (Programmatic Tool Calling)
The `execute_code` tool lets the agent write Python scripts that call Hermes tools programmatically, collapsing multi-step workflows into a single LLM turn. The script runs in a child process on the agent host, communicating with Hermes over a Unix domain socket RPC.
## How It Works
1. The agent writes a Python script using `from hermes_tools import ...`
2. Hermes generates a `hermes_tools.py` stub module with RPC functions
3. Hermes opens a Unix domain socket and starts an RPC listener thread
4. The script runs in a child process — tool calls travel over the socket back to Hermes
5. Only the script's `print()` output is returned to the LLM; intermediate tool results never enter the context window
```python
# The agent can write scripts like:
from hermes_tools import web_search, web_extract
results = web_search("Python 3.13 features", limit=5)
for r in results["data"]["web"]:
content = web_extract([r["url"]])
# ... filter and process ...
print(summary)
```
**Available tools inside scripts:** `web_search`, `web_extract`, `read_file`, `write_file`, `search_files`, `patch`, `terminal` (foreground only).
## When the Agent Uses This
The agent uses `execute_code` when there are:
- **3+ tool calls** with processing logic between them
- Bulk data filtering or conditional branching
- Loops over results
The key benefit: intermediate tool results never enter the context window — only the final `print()` output comes back, dramatically reducing token usage.
## Practical Examples
### Data Processing Pipeline
```python
from hermes_tools import search_files, read_file
import json
# Find all config files and extract database settings
matches = search_files("database", path=".", file_glob="*.yaml", limit=20)
configs = []
for match in matches.get("matches", []):
content = read_file(match["path"])
configs.append({"file": match["path"], "preview": content["content"][:200]})
print(json.dumps(configs, indent=2))
```
### Multi-Step Web Research
```python
from hermes_tools import web_search, web_extract
import json
# Search, extract, and summarize in one turn
results = web_search("Rust async runtime comparison 2025", limit=5)
summaries = []
for r in results["data"]["web"]:
page = web_extract([r["url"]])
for p in page.get("results", []):
if p.get("content"):
summaries.append({
"title": r["title"],
"url": r["url"],
"excerpt": p["content"][:500]
})
print(json.dumps(summaries, indent=2))
```
### Bulk File Refactoring
```python
from hermes_tools import search_files, read_file, patch
# Find all Python files using deprecated API and fix them
matches = search_files("old_api_call", path="src/", file_glob="*.py")
fixed = 0
for match in matches.get("matches", []):
result = patch(
path=match["path"],
old_string="old_api_call(",
new_string="new_api_call(",
replace_all=True
)
if "error" not in str(result):
fixed += 1
print(f"Fixed {fixed} files out of {len(matches.get('matches', []))} matches")
```
### Build and Test Pipeline
```python
from hermes_tools import terminal, read_file
import json
# Run tests, parse results, and report
result = terminal("cd /project && python -m pytest --tb=short -q 2>&1", timeout=120)
output = result.get("output", "")
# Parse test output
passed = output.count(" passed")
failed = output.count(" failed")
errors = output.count(" error")
report = {
"passed": passed,
"failed": failed,
"errors": errors,
"exit_code": result.get("exit_code", -1),
"summary": output[-500:] if len(output) > 500 else output
}
print(json.dumps(report, indent=2))
```
## Execution Mode
`execute_code` has two execution modes controlled by `code_execution.mode` in `~/.hermes/config.yaml`:
| Mode | Working directory | Python interpreter |
|------|-------------------|--------------------|
| **`project`** (default) | The session's working directory (same as `terminal()`) | Active `VIRTUAL_ENV` / `CONDA_PREFIX` python, falling back to Hermes's own python |
| `strict` | A temp staging directory isolated from the user's project | `sys.executable` (Hermes's own python) |
**When to leave it on `project`:** you want `import pandas`, `from my_project import foo`, or relative paths like `open(".env")` to work the same way they do in `terminal()`. This is almost always what you want.
**When to flip to `strict`:** you need maximum reproducibility — you want the same interpreter every session regardless of which venv the user activated, and you want scripts quarantined from the project tree (no risk of accidentally reading project files through a relative path).
```yaml
# ~/.hermes/config.yaml
code_execution:
mode: project # or "strict"
```
Fallback behavior in `project` mode: if `VIRTUAL_ENV` / `CONDA_PREFIX` is unset, broken, or points at a Python older than 3.8, the resolver falls back cleanly to `sys.executable` — it never leaves the agent without a working interpreter.
Security-critical invariants are identical across both modes:
- environment scrubbing (API keys, tokens, credentials stripped)
- tool whitelist (scripts cannot call `execute_code` recursively, `delegate_task`, or MCP tools)
- resource limits (timeout, stdout cap, tool-call cap)
Switching mode changes where scripts run and which interpreter runs them, not what credentials they can see or which tools they can call.
## Resource Limits
| Resource | Limit | Notes |
|----------|-------|-------|
| **Timeout** | 5 minutes (300s) | Script is killed with SIGTERM, then SIGKILL after 5s grace |
| **Stdout** | 50 KB | Output truncated with `[output truncated at 50KB]` notice |
| **Stderr** | 10 KB | Included in output on non-zero exit for debugging |
| **Tool calls** | 50 per execution | Error returned when limit reached |
All limits are configurable via `config.yaml`:
```yaml
# In ~/.hermes/config.yaml
code_execution:
mode: project # project (default) | strict
timeout: 300 # Max seconds per script (default: 300)
max_tool_calls: 50 # Max tool calls per execution (default: 50)
```
## How Tool Calls Work Inside Scripts
When your script calls a function like `web_search("query")`:
1. The call is serialized to JSON and sent over a Unix domain socket to the parent process
2. The parent dispatches through the standard `handle_function_call` handler
3. The result is sent back over the socket
4. The function returns the parsed result
This means tool calls inside scripts behave identically to normal tool calls — same rate limits, same error handling, same capabilities. The only restriction is that `terminal()` is foreground-only (no `background` or `pty` parameters).
## Error Handling
When a script fails, the agent receives structured error information:
- **Non-zero exit code**: stderr is included in the output so the agent sees the full traceback
- **Timeout**: Script is killed and the agent sees `"Script timed out after 300s and was killed."`
- **Interruption**: If the user sends a new message during execution, the script is terminated and the agent sees `[execution interrupted — user sent a new message]`
- **Tool call limit**: When the 50-call limit is hit, subsequent tool calls return an error message
The response always includes `status` (success/error/timeout/interrupted), `output`, `tool_calls_made`, and `duration_seconds`.
## Security
:::danger Security Model
The child process runs with a **minimal environment**. API keys, tokens, and credentials are stripped by default. The script accesses tools exclusively via the RPC channel — it cannot read secrets from environment variables unless explicitly allowed.
:::
Environment variables containing `KEY`, `TOKEN`, `SECRET`, `PASSWORD`, `CREDENTIAL`, `PASSWD`, or `AUTH` in their names are excluded. Only safe system variables (`PATH`, `HOME`, `LANG`, `SHELL`, `PYTHONPATH`, `VIRTUAL_ENV`, etc.) are passed through.
### Skill Environment Variable Passthrough
When a skill declares `required_environment_variables` in its frontmatter, those variables are **automatically passed through** to both `execute_code` and `terminal` child processes after the skill is loaded. This lets skills use their declared API keys without weakening the security posture for arbitrary code.
For non-skill use cases, you can explicitly allowlist variables in `config.yaml`:
```yaml
terminal:
env_passthrough:
- MY_CUSTOM_KEY
- ANOTHER_TOKEN
```
See the [Security guide](/user-guide/security#environment-variable-passthrough) for full details.
### `HERMES_*` variables in the child
The child process receives only a small, fixed set of operational `HERMES_*`
variables by exact name:
- `HERMES_HOME`
- `HERMES_PROFILE`
- `HERMES_CONFIG`
- `HERMES_ENV`
(plus `HERMES_RPC_DIR` / `HERMES_RPC_SOCKET` / `TZ` / `HOME`, which Hermes
injects explicitly so the RPC channel works).
:::note Behavior change
Earlier versions passed **any** variable whose name began with `HERMES_`
through to the child. That broad prefix was removed for security hardening: it
could leak `HERMES_*`-named configuration that doesn't match a secret substring
(for example `HERMES_BASE_URL`, `HERMES_KANBAN_DB`, or a `HERMES_*_WEBHOOK`
endpoint) into arbitrary sandboxed code.
If an `execute_code` script — or a repo/plugin module it imports at import time
— relied on a `HERMES_*` variable outside the four operational names above, it
will now find that variable **unset** in the child. The drop is intentional,
not a bug.
:::
**Workaround — opt the variable back in explicitly.** Both routes pass the
variable through `execute_code` *and* `terminal` children, and neither weakens
the secret-stripping guarantee (Hermes-managed provider credentials can never
be re-allowed this way):
1. **Per-machine, in `config.yaml`** — add the exact variable name to the
passthrough allowlist:
```yaml
terminal:
env_passthrough:
- HERMES_KANBAN_DB
- HERMES_BASE_URL
```
2. **Per-skill, in the skill's frontmatter** — declare it so it is registered
automatically whenever that skill is loaded:
```yaml
required_environment_variables:
- HERMES_KANBAN_DB
```
**Diagnosing it.** When the child drops one or more non-allowlisted `HERMES_*`
variables, Hermes emits a one-line `debug` log naming them and pointing at the
`env_passthrough` escape hatch. Run with debug logging (`hermes logs --level
DEBUG`, or check `~/.hermes/logs/agent.log`) and look for
`execute_code: dropped N non-allowlisted HERMES_* var(s)` if a script behaves
as though a `HERMES_*` variable is missing.
Hermes always writes the script and the auto-generated `hermes_tools.py` RPC stub into a temp staging directory that is cleaned up after execution. In `strict` mode the script also *runs* there; in `project` mode it runs in the session's working directory (the staging directory stays on `PYTHONPATH` so imports still resolve). The child process runs in its own process group so it can be cleanly killed on timeout or interruption.
## execute_code vs terminal
| Use Case | execute_code | terminal |
|----------|-------------|----------|
| Multi-step workflows with tool calls between | ✅ | ❌ |
| Simple shell command | ❌ | ✅ |
| Filtering/processing large tool outputs | ✅ | ❌ |
| Running a build or test suite | ❌ | ✅ |
| Looping over search results | ✅ | ❌ |
| Interactive/background processes | ❌ | ✅ |
| Needs API keys in environment | ⚠️ Only via [passthrough](/user-guide/security#environment-variable-passthrough) | ✅ (most pass through) |
**Rule of thumb:** Use `execute_code` when you need to call Hermes tools programmatically with logic between calls. Use `terminal` for running shell commands, builds, and processes.
## Platform Support
Code execution is available on **Linux, macOS, and Windows**. On Linux and macOS the RPC channel uses a Unix domain socket; on Windows, where `AF_UNIX` is unreliable, Hermes automatically falls back to a loopback TCP socket for the sandbox RPC transport. Remote terminal backends (Docker/SSH/Modal/etc.) use a file-based RPC transport instead and additionally require Python 3 inside the backend.
website/docs/user-guide/features/codex-app-server-runtime.md
+460
View File
@@ -0,0 +1,460 @@
---
title: Codex App-Server Runtime (optional)
sidebar_label: Codex App-Server Runtime
---
# Codex App-Server Runtime
Hermes can optionally hand `openai/*` and `openai-codex/*` turns to the [Codex CLI app-server](https://github.com/openai/codex) instead of running its own tool loop. When enabled, terminal commands, file edits, sandboxing, and MCP tool calls all execute inside Codex's runtime — Hermes becomes the shell around it (sessions DB, slash commands, gateway, memory and skill review).
This is **opt-in only**. Default Hermes behavior is unchanged unless you flip the flag. Hermes never auto-routes you onto this runtime.
:::tip
Not using OpenAI Codex? `hermes setup --portal` configures a non-Codex backend with Claude/Gemini/etc. in one step. See [Nous Portal](/integrations/nous-portal).
:::
## Why
- Run OpenAI agent turns against your **ChatGPT subscription** (no API key required) using the same auth flow Codex CLI uses.
- Use **Codex's own toolset and sandbox** — `shell` for terminal/read/write/search, `apply_patch` for structured edits, `update_plan` for planning, all running inside seatbelt/landlock sandboxing.
- **Native Codex plugins** — Linear, GitHub, Gmail, Calendar, Canva, etc. — installed via `codex plugin` are auto-migrated and active in your Hermes session.
- **Hermes' richer tools come along** — web_search, web_extract, browser automation, vision, image generation, skills, and TTS work via an MCP callback. Codex calls back into Hermes for tools it doesn't have built in.
- **Memory and skill nudges keep working** — Codex's events are projected into Hermes' message shape so the self-improvement loop sees a normal-looking transcript.
## What tools the model actually has
This is the part most users want to know up front. When this runtime is on, the model running your turn has three independent sources of tools:
### 1. Codex's built-in toolset (always on)
These ship with `codex app-server` itself — no Hermes involvement, no MCP, no plugins. All five are available the moment the runtime starts:
- **`shell`** — runs arbitrary shell commands inside the sandbox. This is how the model reads files (`cat`, `head`, `tail`), writes them (`echo > foo`, heredocs), searches them (`find`, `rg`, `grep`), navigates directories (`ls`, `cd`), runs builds, manages processes, and anything else you'd do in bash.
- **`apply_patch`** — applies a structured multi-file diff in Codex's patch format. The model uses this for non-trivial code edits (adding a function, refactoring across files); shell heredocs are still available for one-off writes.
- **`update_plan`** — codex's internal todo / plan tracker. Equivalent of Hermes' `todo` tool, but managed entirely inside codex's runtime.
- **`view_image`** — load a local image file into the conversation so the model can see it.
- **`web_search`** — codex has its own built-in web search when configured. Hermes also exposes `web_search` (Firecrawl-backed) via the callback below; the model picks whichever it prefers.
So **anything you'd do via terminal — read/write/search/find/run — codex does natively**. The sandbox profile (`:workspace` by default when you enable the runtime) controls what's writable.
### 2. Native Codex plugins (auto-migrated from your `codex plugin` install)
When you enable the runtime, Hermes queries codex's `plugin/list` RPC and writes a `[plugins."<name>@openai-curated"]` entry for every plugin you have installed. The plugins themselves are managed by codex and authorized once via codex's own UI.
Examples (the ones the OpenClaw thread highlighted as "YouTube-video-worthy"):
- **Linear** — find/update issues
- **GitHub** — search code, view PRs, comment
- **Gmail** — read/send mail
- **Google Calendar** — create/find events
- **Outlook calendar/email** — same shape via the Microsoft connector
- **Canva** — design generation
- ...whatever else you've installed via `codex plugin marketplace add openai-curated` + `codex plugin install ...`
What's NOT migrated:
- Plugins you haven't installed yet — install them in Codex first.
- ChatGPT app marketplace entries (`app/list`) — these are already enabled inside codex by virtue of your account auth.
### 3. Hermes tool callback (MCP server, registered in `~/.codex/config.toml`)
Hermes registers itself as an MCP server so codex can call back for tools codex doesn't ship with. Available via the callback:
- **`web_search`** / **`web_extract`** — Firecrawl-backed; tends to be cleaner than scraping for structured content.
- **`browser_navigate` / `browser_click` / `browser_type` / `browser_press` / `browser_snapshot` / `browser_scroll` / `browser_back` / `browser_get_images` / `browser_console` / `browser_vision`** — full browser automation via Camofox or Browserbase.
- **`vision_analyze`** — call a separate vision model to inspect an image (different from codex's `view_image` which loads it into the conversation).
- **`image_generate`** — image generation through Hermes' image_gen plugin chain.
- **`skill_view` / `skills_list`** — read from Hermes' skill library.
- **`text_to_speech`** — TTS through Hermes' configured provider.
When the model wants one of these, codex spawns the `hermes_tools_mcp_server` subprocess via stdio MCP, the call is dispatched through `model_tools.handle_function_call()` (same code path as Hermes' default runtime), and the result is returned to codex like any other MCP response.
### What's NOT available on this runtime
These four Hermes tools require the running AIAgent context (mid-loop state) to dispatch, and a stateless MCP callback can't drive them. Switch back to the default runtime (`/codex-runtime auto`) when you need any of them:
- **`delegate_task`** — spawn subagents
- **`memory`** — Hermes' persistent memory store
- **`session_search`** — cross-session search
- **`todo`** — Hermes' todo store (codex's `update_plan` is the in-runtime equivalent)
## Workflow features (`/goal`, kanban, cron)
### `/goal` (the Ralph loop)
**Works on this runtime.** Goals persist in `state_meta` keyed by session id, the continuation prompt feeds back as a normal user message through `run_conversation()`, and codex executes the next turn natively. The goal judge runs via the auxiliary client (configured via `auxiliary.goal_judge` in config.yaml), independent of which runtime is active. The judge's "blocked, needs user input" verdict is a clean escape if codex stalls on approvals.
**One thing to be aware of:** each continuation prompt is a fresh codex turn, which means codex re-evaluates command approval policy from scratch. If you're doing a long-running goal with lots of writes, expect more approval prompts than you'd see on a single in-session task. Set `default_permissions = ":workspace"` (which Hermes does automatically when you enable the runtime) so simple workspace writes don't require prompting.
### Kanban (multi-agent worktree dispatch)
**Works on this runtime, with one subtle dependency.** The kanban dispatcher spawns each worker as a separate `hermes chat -q` subprocess that reads the user's config — which means if `model.openai_runtime: codex_app_server` is set globally, workers also come up on the codex runtime.
What works inside a codex-runtime worker:
- Codex's full toolset (shell, apply_patch, update_plan, view_image, web_search) — the worker does its actual task work natively
- The migrated codex plugins — Linear, GitHub, etc.
- The Hermes tool callback for browser_*, vision, image_gen, skills, TTS
What also works because the MCP callback exposes them:
- **`kanban_complete` / `kanban_block` / `kanban_comment` / `kanban_heartbeat`** — the worker handoff tools. These read `HERMES_KANBAN_TASK` from env (set by the dispatcher), gate access correctly, and write to the per-board SQLite DB pinned by `HERMES_KANBAN_DB`. Without these in the callback, a worker on this runtime could do its task but couldn't report back, hanging until the dispatcher's timeout.
- **`kanban_show` / `kanban_list`** — read-only board queries for the worker to check its own context.
- **`kanban_create` / `kanban_unblock` / `kanban_link`** — orchestrator-only operations. Available for orchestrator agents running on the codex runtime that need to dispatch new tasks.
The kanban tools are gated by `HERMES_KANBAN_TASK` env var the dispatcher sets — that var is propagated to the codex subprocess (codex inherits env) and from there to the spawned `hermes-tools` MCP server subprocess. So the tools see the right task id and gate correctly. For Codex app-server workers, Hermes also passes narrow app-server sandbox overrides when `HERMES_KANBAN_TASK` is present: keep `workspace-write` sandboxing, add the **board DB directory plus every Kanban path the dispatcher pinned** as extra writable roots (`HERMES_KANBAN_WORKSPACES_ROOT`, `HERMES_KANBAN_WORKSPACE`, legacy `HERMES_KANBAN_ROOT` — deduplicated, DB-dir first), and keep network disabled by default. This avoids the brittle `:danger-no-sandbox` workaround while letting `kanban_complete` / `kanban_block` update the board DB **and** letting workers write reports/artifacts under workspace mounts that live outside the DB directory (e.g. `/media/.../kanban-workspaces/...` on a separate drive — [issue #27941](https://github.com/NousResearch/hermes-agent/issues/27941)).
### Cron jobs
**Not specifically tested.** Cron jobs run via `cronjob` → `AIAgent.run_conversation`, the same code path as the CLI. If the cron job's config has `openai_runtime: codex_app_server` it'll run on codex. The same tool-availability rules apply — codex built-ins + plugins + MCP callback work, agent-loop tools (delegate_task, memory, session_search, todo) don't. If your cron job relies on those, scope the cron to a profile that uses the default runtime.
## Trade-offs
| | Hermes default runtime | Codex app-server (opt-in) |
|---|---|---|
| `delegate_task` subagents | yes | not available — needs agent loop context |
| `memory`, `session_search`, `todo` | yes | not available — needs agent loop context |
| `web_search`, `web_extract` | yes | yes (via MCP callback) |
| Browser automation (Camofox/Browserbase) | yes | yes (via MCP callback) |
| `vision_analyze`, `image_generate` | yes | yes (via MCP callback) |
| `skill_view`, `skills_list` | yes | yes (via MCP callback) |
| `text_to_speech` | yes | yes (via MCP callback) |
| Codex `shell` (terminal/read/write/search/find/run) | — | yes (Codex built-in) |
| Codex `apply_patch` (structured multi-file edits) | — | yes (Codex built-in) |
| Codex `update_plan` (in-runtime todo) | — | yes (Codex built-in) |
| Codex `view_image` (load image into conversation) | — | yes (Codex built-in) |
| Codex sandbox (seatbelt/landlock, profiles) | — | yes (Codex built-in) |
| ChatGPT subscription auth | — | yes (via `openai-codex` provider) |
| Native Codex plugins (Linear, GitHub, etc.) | — | yes (auto-migrated) |
| User MCP servers | yes | yes (auto-migrated to codex) |
| Memory + skill review (background) | yes | yes (via item projection) |
| Multi-turn conversations | yes | yes |
| `/goal` (Ralph loop) | yes | yes |
| Kanban worker dispatch | yes | yes (via callback) |
| Kanban orchestrator tools | yes | yes (via callback) |
| All gateway platforms | yes | yes |
| Non-OpenAI providers | yes | n/a — OpenAI/Codex-scoped |
### Live display
Even though the agent loop runs inside the Codex subprocess, the runtime
bridges Codex's event stream into the same display path the default runtime
uses:
- Live assistant deltas, reasoning (including summary deltas), and stable-ID
tool start/completion events surface in the TUI, desktop, and messaging
gateways as the turn runs. The completion-only history projector remains
separate, so a resumed session hydrates the same tool cards shown during
the turn.
- Gateway commentary stays visible when token streaming is disabled, and
live tool events are forwarded even for notifications drained ahead of an
approval request. Commentary honors `display.show_commentary`.
## Prerequisites
1. **Codex CLI installed:**
```bash
npm i -g @openai/codex
codex --version # 0.130.0 or newer
```
2. **Codex OAuth login.** The codex subprocess reads `~/.codex/auth.json`. Two ways to populate it:
```bash
codex login # writes tokens to ~/.codex/auth.json
```
Hermes' own `hermes auth add openai-codex` writes to `~/.hermes/auth.json` — that's a separate session. **Run `codex login` separately** if you haven't.
3. **(Optional) Install the Codex plugins you want.** When you enable the runtime, Hermes auto-migrates whichever curated plugins you've already installed via Codex CLI:
```bash
codex plugin marketplace add openai-curated
# then via codex's TUI, install Linear / GitHub / Gmail / etc.
```
Hermes will discover them and write `[plugins."<name>@openai-curated"]` entries to `~/.codex/config.toml` automatically.
## Enabling
In a Hermes session:
```
/codex-runtime codex_app_server
```
That command:
- Verifies the `codex` CLI is installed (blocks with an install hint if not).
- Persists `model.openai_runtime: codex_app_server` to your config.yaml.
- Migrates user MCP servers from `~/.hermes/config.yaml` to `~/.codex/config.toml`.
- **Discovers and migrates installed native Codex plugins** (Linear, GitHub, Gmail, Calendar, Canva, etc.) by querying Codex's `plugin/list` RPC.
- **Registers Hermes' own tools as an MCP server** so the codex subprocess can call back for tools codex doesn't ship with.
- **Writes `default_permissions = ":workspace"`** so the sandbox allows writes within the workspace without prompting for every operation.
- Tells you what was migrated. Takes effect on the **next** session — the current cached agent keeps the prior runtime so prompt caches stay valid.
Synonyms: `/codex-runtime on`, `/codex-runtime off`, `/codex-runtime auto`.
To check current state without changing anything:
```
/codex-runtime
```
You can also set it manually in `~/.hermes/config.yaml`:
```yaml
model:
openai_runtime: codex_app_server # default is "auto" (= Hermes runtime)
```
## Self-improvement loop (memory + skill nudges)
Hermes' background self-improvement fires on counter thresholds:
- Every 10 user prompts → a forked review agent looks at the conversation and decides whether anything should be saved to memory.
- Every 10 tool iterations within a single turn → same idea but for skills (`skill_manage` writes).
**Both keep working on the codex runtime.** The codex path projects each completed `commandExecution` / `fileChange` / `mcpToolCall` / `dynamicToolCall` item into a synthetic `assistant tool_call` + `tool` result message, so by the time the review runs it sees the same shape it sees on the default Hermes runtime.
How the wiring stays equivalent:
| | Default runtime | Codex runtime |
|---|---|---|
| `_turns_since_memory` increments | per user prompt, in run_conversation pre-loop | same code path, before the early-return |
| `_iters_since_skill` increments | per tool iteration in the chat-completions loop | by `turn.tool_iterations` after the codex turn returns |
| Memory trigger (`_turns_since_memory >= _memory_nudge_interval`) | computed in pre-loop, fires after response | computed in pre-loop, passed through to codex helper |
| Skill trigger (`_iters_since_skill >= _skill_nudge_interval`) | computed after the loop | computed after the codex turn |
| `_spawn_background_review(messages_snapshot=..., review_memory=..., review_skills=...)` | called when either trigger fires | called identically when either trigger fires |
One detail: the review fork itself needs to call Hermes' agent-loop tools (`memory`, `skill_manage`), which require Hermes' own dispatch. So when the parent agent is on `codex_app_server`, the review fork is **downgraded to `codex_responses`** — same OAuth credentials, same `openai-codex` provider, but talks to OpenAI's Responses API directly so Hermes owns the loop and the agent-loop tools work. This is invisible to the user.
Net effect: enable the codex runtime and your memory + skill nudges keep firing exactly as they would otherwise.
## How approvals work
Codex requests approval before executing commands or applying patches. These get translated into Hermes' standard "Dangerous Command" prompt:
```
╭───────────────────────────────────────╮
│ Dangerous Command │
│ │
│ /bin/bash -lc 'echo hello > foo.txt' │
│ │
│ ❯ 1. Allow once │
│ 2. Allow for this session │
│ 3. Deny │
│ │
│ Codex requests exec in /your/cwd │
╰───────────────────────────────────────╯
```
- **Allow once** → approve this single command.
- **Allow for this session** → Codex won't re-prompt for similar commands.
- **Deny** → command is rejected; Codex continues in read-only mode.
For `apply_patch` (file edit) approvals, Hermes shows a summary of what changed (`1 add, 1 update: /tmp/new.py, /tmp/old.py`) when codex provides the data via the corresponding `fileChange` item.
## Permission profiles
Codex has three built-in permission profiles:
- `:read-only` — no writes; every shell command requires approval
- `:workspace` — writes within the current workspace allowed without prompts (Hermes' default when you enable the runtime)
- `:danger-no-sandbox` — no sandbox at all (don't use this unless you understand it)
You can override the default in `~/.codex/config.toml` outside Hermes' managed block:
```toml
default_permissions = ":read-only"
```
(Hermes will preserve your override on re-migration as long as it lives outside the `# managed by hermes-agent` markers.)
## Auxiliary tasks and ChatGPT subscription token cost
When this runtime is on with the `openai-codex` provider, **auxiliary tasks (title generation, context compression, vision auto-detect, the background self-improvement review fork) also flow through your ChatGPT subscription by default**, because Hermes' auxiliary client uses the main provider/model when no per-task override is set.
This isn't specific to `codex_app_server` — it's true for the existing `codex_responses` path too — but it's more visible here because you're explicitly opting in for the subscription billing.
To route specific aux tasks to a cheaper / different model, set explicit overrides in `~/.hermes/config.yaml`:
```yaml
auxiliary:
title_generation:
provider: openrouter
model: google/gemini-3-flash-preview
compression:
provider: openrouter
model: google/gemini-3-flash-preview
vision:
provider: openrouter
model: google/gemini-3-flash-preview
goal_judge:
provider: openrouter
model: google/gemini-3-flash-preview
```
The self-improvement review fork inherits the main runtime via `_current_main_runtime()` and Hermes downgrades it from `codex_app_server` to `codex_responses` automatically (so the fork can actually call `memory` and `skill_manage` — Hermes' own agent-loop tools). That fork still uses your subscription auth unless you've routed aux tasks elsewhere.
## Editing `~/.codex/config.toml` safely
Hermes wraps everything it manages between two marker comments:
```toml
# managed by hermes-agent — `hermes codex-runtime migrate` regenerates this section
default_permissions = ":workspace"
[mcp_servers.filesystem]
...
[plugins."github@openai-curated"]
...
# end hermes-agent managed section
```
Anything **outside** that block is yours. Re-running migration (via `/codex-runtime codex_app_server` or whenever you toggle the runtime on) replaces the managed block in place but preserves user content above and below it verbatim. This means you can:
- Add your own MCP servers Hermes doesn't know about
- Override `default_permissions` to `:read-only` if you prefer to be prompted
- Configure codex-only options (model, providers, otel, etc.)
- Add user-defined permission profiles in `[permissions.<name>]` tables
Anything you add **inside** the managed block will get clobbered on the next migration. If you need a tweak that requires editing the managed block, file an issue and we'll add the knob.
## Multi-profile / multi-tenant setups
By default, Hermes points the codex subprocess at `~/.codex/` regardless of which Hermes profile is active. This means `hermes -p work` and `hermes -p personal` share the same Codex auth, plugins, and config. For most users this is the right behavior — it matches what running `codex` CLI directly would do.
If you want per-profile Codex isolation (separate auth, separate installed plugins, separate config), set `CODEX_HOME` explicitly per profile. The cleanest way is to point at a directory under your `HERMES_HOME`:
```bash
# Inside the work profile, you might wrap hermes:
CODEX_HOME=~/.hermes/profiles/work/codex hermes chat
```
You'll need to re-run `codex login` once with that `CODEX_HOME` set so the OAuth tokens land in the profile-scoped location. After that, `hermes -p work` will operate on isolated Codex state.
We don't auto-scope this because moving an existing user's `~/.codex/` would silently invalidate their Codex CLI auth — anyone who already ran `codex login` would have to re-authenticate. Opt-in feels safer than surprising users.
## HOME environment variable passthrough
Hermes does NOT rewrite `HOME` when spawning the codex app-server subprocess (we use `os.environ.copy()` and only overlay `CODEX_HOME` and `RUST_LOG`). This means:
- Commands codex runs via its `shell` tool see the real user `HOME` and find `~/.gitconfig`, `~/.gh/`, `~/.aws/`, `~/.npmrc`, etc. correctly.
- Codex's internal state stays isolated through `CODEX_HOME` (which points at `~/.codex/` by default).
This matches the boundary OpenClaw arrived at after some early experimentation: isolate Codex's state, leave the user's home alone. (Cf. openclaw/openclaw#81562.)
## MCP server migration
Hermes' `mcp_servers` config is auto-translated to the TOML format Codex expects. The migration runs every time you enable the runtime and is idempotent — re-runs replace the managed section but preserve any user-edited Codex config.
What translates:
| Hermes (`config.yaml`) | Codex (`config.toml`) |
|---|---|
| `command` + `args` + `env` | stdio transport |
| `url` + `headers` | streamable_http transport |
| `timeout` | `tool_timeout_sec` |
| `connect_timeout` | `startup_timeout_sec` |
| `enabled: false` | `enabled = false` |
What's not migrated:
- Hermes-specific keys like `sampling` (Codex's MCP client has no equivalent — these are dropped with a per-server warning).
## Native Codex plugin migration
Plugins installed via `codex plugin` (Linear, GitHub, Gmail, Calendar, Canva, etc.) are discovered through Codex's `plugin/list` RPC. For each plugin where `installed: true`, Hermes writes a `[plugins."<name>@openai-curated"]` block enabling it in your Hermes session.
This means: when your friend says "I have Calendar and GitHub set up in my Codex CLI" and they enable Hermes' codex runtime, Hermes activates those automatically. No re-configuration needed.
What's NOT migrated:
- Plugins you haven't installed yet — install them in Codex first.
- Plugins where codex reports `availability != AVAILABLE` (broken install, expired OAuth, removed from marketplace, etc.). These are skipped to avoid writing config that would fail at activation time.
- ChatGPT app marketplace entries (the per-account `app/list` results — these are already enabled inside codex by virtue of your account auth).
- Plugin OAuth — you authorize each plugin once in Codex itself; Hermes doesn't touch credentials.
## Hermes tool callback (the new MCP server)
Codex's built-in toolset covers shell/file ops/patches but doesn't have web search, browser automation, vision, image generation, etc. To keep those usable in a codex turn, Hermes registers itself as an MCP server in `~/.codex/config.toml`:
```toml
[mcp_servers.hermes-tools]
command = "/path/to/python"
args = ["-m", "agent.transports.hermes_tools_mcp_server"]
env = { HERMES_HOME = "/your/.hermes", PYTHONPATH = "...", HERMES_QUIET = "1" }
startup_timeout_sec = 30.0
tool_timeout_sec = 600.0
```
When the model calls `web_search` (or another exposed Hermes tool), codex spawns the `hermes_tools_mcp_server` subprocess via stdio, the request is dispatched through `model_tools.handle_function_call()`, and the result is projected back to codex like any other MCP response.
**Tools available via the callback:** `web_search`, `web_extract`, `browser_navigate`, `browser_click`, `browser_type`, `browser_press`, `browser_snapshot`, `browser_scroll`, `browser_back`, `browser_get_images`, `browser_console`, `browser_vision`, `vision_analyze`, `image_generate`, `skill_view`, `skills_list`, `text_to_speech`.
**Tools NOT available:** `delegate_task`, `memory`, `session_search`, `todo`. These need the running AIAgent context to dispatch (mid-loop state) and a stateless MCP callback can't drive them. Use the default Hermes runtime (`/codex-runtime auto`) when you need these.
## Disabling
Switch back at any time:
```
/codex-runtime auto
```
Effective on the next session. The Codex managed block stays in `~/.codex/config.toml` so you can re-enable later without losing config — or remove it manually if you prefer.
## Limitations
This runtime is **opt-in beta**. Working as of Hermes Agent 2026.5 + Codex CLI 0.130.0:
- Multi-turn conversations
- `commandExecution` and `fileChange` (apply_patch) approvals via Hermes UI
- MCP tool calls (verified against `@modelcontextprotocol/server-filesystem` and the new `hermes-tools` callback)
- Native Codex plugin migration (verified against Linear / GitHub / Calendar inventory)
- Deny/cancel paths
- Toggle on/off cycle
- Memory and skill nudge counters (verified live via integration tests)
- Hermes web_search through codex (verified live: "OpenAI Codex CLI – Getting Started" returned end-to-end)
Known limitations:
- **Hermes auth and codex auth are separate sessions.** You need both `codex login` AND `hermes auth add openai-codex` for the cleanest UX (the runtime uses codex's session for the LLM call). This is a deliberate design choice in Hermes' `_import_codex_cli_tokens` — Hermes won't share OAuth state with codex CLI to avoid clobbering each other on token refresh.
- **`delegate_task`, `memory`, `session_search`, `todo` are unavailable on this runtime.** They need the running AIAgent context which a stateless MCP callback can't provide. Use `/codex-runtime auto` when you need these.
- **No inline patch preview in approval prompts when codex doesn't track the changeset.** Codex's `fileChange` approval params don't always carry the changeset. Hermes caches the data from the corresponding `item/started` notification when possible, but if approval arrives before the item has streamed, the prompt falls back to whatever `reason` codex provides.
- **Sub-second cancellation isn't guaranteed.** Mid-stream interrupts (Ctrl+C while codex is responding) are sent via `turn/interrupt`, but if codex has already flushed the final message, you get the response anyway.
If you find a bug, [open an issue](https://github.com/NousResearch/hermes-agent/issues) with the output of `hermes logs --since 5m`. Mention `codex-runtime` in the title so it's easy to triage.
## Architecture
```
┌─── Hermes shell (CLI / TUI / gateway) ───┐
│ sessions DB · slash commands · memory │
│ & skill review · cron · session pickers │
└──┬──────────────────────────────────────┬┘
│ user_message final │
▼ text + │
┌──────────────────────────────────┐ projected │
│ AIAgent.run_conversation() │ messages │
│ if api_mode == codex_app_server │ │
│ → CodexAppServerSession │ │
│ else: chat_completions / codex_responses (default)
└────┬─────────────────────────────┘ │
│ JSON-RPC over stdio │
▼ │
┌──────────────────────────────────┐ │
│ codex app-server (subprocess) │──────────────┘
│ thread/start, turn/start │
│ item/* notifications │
│ shell + apply_patch + update_plan│
│ view_image + sandbox │
│ ┌─────────────────────────┐ │
│ │ MCP client │ │
│ │ ├─ user MCP servers │ │
│ │ ├─ native plugins │ │
│ │ │ (linear, github, │ │
│ │ │ gmail, calendar, │ │
│ │ │ canva, ...) │ │
│ │ └─ hermes-tools ───────┼─────────────────┐
│ │ (callback to │ │ │
│ │ Hermes' richer │ │ │
│ │ tools) │ │ │
│ └─────────────────────────┘ │ │
└──────────────────────────────────┘ │
│
▼
┌──────────────────────────────────────────────────────────┐
│ hermes_tools_mcp_server.py (subprocess on demand) │
│ web_search, web_extract, browser_*, vision_analyze, │
│ image_generate, skill_view, skills_list, text_to_speech│
└──────────────────────────────────────────────────────────┘
```
For implementation details, see [PR #24182](https://github.com/NousResearch/hermes-agent/pull/24182) and the [Codex app-server protocol README](https://github.com/openai/codex/blob/main/codex-rs/app-server/README.md).
website/docs/user-guide/features/computer-use.md
+620
View File
@@ -0,0 +1,620 @@
---
title: Computer Use
sidebar_position: 16
---
# Computer Use
Hermes Agent can drive your desktop — clicking, typing, scrolling,
dragging — in the **background** on **macOS, Windows, and Linux**. Your
cursor doesn't move, keyboard focus doesn't change, and your virtual
desktops / Spaces don't switch on you. You and the agent co-work on the
same machine.
Unlike most computer-use integrations, this works with **any tool-capable
model** — Claude, GPT, Gemini, or an open model on a local
OpenAI-compatible endpoint. There's no Anthropic-native schema to worry
about.
## How it works
The built-in `computer_use` toolset is the recommended Hermes integration. It
speaks MCP over stdio to
[`cua-driver`](https://github.com/trycua/cua), an open-source background
computer-use driver. Each platform uses the appropriate accessibility +
input stack under the hood:
| Platform | Accessibility tree | Input dispatch |
|---|---|---|
| macOS | AX (private SkyLight SPIs) | `SLPSPostEventRecordTo` — pid-scoped, no cursor warp |
| Windows | UIAutomation | `SendInput` + `PostMessage` — no focus steal |
| Linux | AT-SPI (X11 + Wayland) | XTest (X11) / virtual-keyboard (Wayland) |
The result is the same on every platform: the agent can read the
accessibility tree of any visible window AND post synthesized events
without bringing it to front, switching virtual desktops, or moving the
real OS cursor.
For the underlying contract — *why* background mode matters, the
no-foreground invariant, click-dispatch internals — see
**[cua.ai/docs/explanation/the-no-foreground-contract](https://cua.ai/docs/explanation/the-no-foreground-contract)**.
## Enabling
**Fresh installs already have the driver.** The Hermes installer
(`install.sh` / `install.ps1`) pre-installs `cua-driver` (best-effort;
pass `--skip-computer-use` / `-SkipComputerUse` to opt out), so enabling
Computer Use is just a config flip:
- **`hermes tools`** → pick `🖱️ Computer Use` — installs the driver
automatically if it's still missing.
- **Dashboard / desktop app** → toggle the Computer Use toolset — if the
driver is missing, the toggle kicks off the install in the background
automatically (watch progress in the toolset panel).
**Manual fallback (older installs, skipped installer step):**
```
hermes computer-use install
```
This fetches and runs the upstream cua-driver installer — `install.sh`
on macOS/Linux, `install.ps1` on Windows. Use `hermes computer-use
status` to verify the install.
Already have cua-driver? Hermes reuses it when it supports the 0.20 runtime
contract. During setup, toolset enablement, `hermes update`, and the first
`computer_use` call of a session, Hermes checks the local version and
manifest. It repairs an old or incomplete standard installation through
the upstream installer (at most once per session at runtime). A binary
selected with `HERMES_CUA_DRIVER_CMD` stays
under your control, so Hermes reports the incompatibility and leaves it
unchanged.
If you install Cua Driver first, `cua-driver skills install` installs Cua's
skill pack under `~/.cua-driver/skills/cua-driver`. Hermes autodetection is a
planned cua-driver follow-up, so currently point Hermes at that directory or
symlink it into your skill space. You can also register raw Cua MCP tools as a
custom MCP server, but that is an alternative for users who need the low-level
interface. The built-in toolset provides Hermes actions, configuration,
approvals, and diagnostics.
After installing, regardless of which path you took, grant the
platform-appropriate prereqs:
| Platform | Prereqs |
|---|---|
| **macOS** | System Settings → Privacy & Security → **Accessibility** + **Screen Recording**. Grant the identity named by `hermes computer-use doctor`. Standard mode uses CuaDriver.app; bounded and unrestricted modes use the Hermes host identity. |
| **Windows** | None at install time. If you're driving over SSH (not RDP / console), you need the autostart pattern — see [cua.ai/docs/how-to-guides/driver/windows-ssh](https://cua.ai/docs/how-to-guides/driver/windows-ssh) for the Session 0 ↔ Session 1+ proxy. |
| **Linux** | A reachable display server: `DISPLAY` set for X11, or `XDG_SESSION_TYPE=wayland`. Wayland sessions need an XWayland bridge for capture. AT-SPI must be on (default on GNOME/KDE/Xfce). |
Then start a session with the toolset enabled:
```
hermes -t computer_use chat
```
or add `computer_use` to your enabled toolsets in `~/.hermes/config.yaml`.
## Permission modes and logged-in browser profiles
Hermes maps its existing approval UX onto cua-driver's immutable runtime
modes. Permission mode and capability manifest approval are launch settings.
They cannot change after the runtime starts:
| Hermes session | cua-driver mode | Human intervention |
|---|---|---|
| Manual or smart approvals (default) | `standard` | Normal Hermes approvals; Cua stops at its protected boundary |
| `computer_use.permission_mode: bounded` + reviewed manifest | private `bounded` daemon | You review and approve the capability manifest once, at launch |
| `--yolo`, `/yolo`, or `approvals.mode: off` | private `unrestricted` daemon | One explicit Hermes risk acceptance; no runtime Cua prompts |
Browser work — including pages in a signed-in profile — goes through the
`browser` toolset (`browser_exec`), not `computer_use`. The former
`computer_use.grant_existing_profile` opt-in was removed along with the typed
browser route; a leftover key in config.yaml is ignored.
### Bounded mode for repeatable automation
For recurring browser automation (cron jobs, scheduled research against an
authenticated app), `bounded` mode uses a capability manifest you review once:
```yaml
# config.yaml
computer_use:
permission_mode: bounded
capability_manifest: ~/.hermes/cua-manifest.yaml
```
The manifest names the apps, browser profile kinds, allowed origins, and
typed tools the session may use (see the
[cua-driver permission modes reference](https://cua.ai/docs/reference/cua-driver/permission-modes)
for the format). Hermes launches a private runtime with
`--capability-manifest ... --approve-capability-manifest`; anything outside
the manifest fails closed inside cua-driver. A missing or unreadable manifest
fails loudly at session start rather than silently downgrading. Session YOLO
still overrides bounded for that one session.
On macOS, private-session daemons launch through the installed
`CuaDriver.app` bundle (so permission grants attribute to the driver's own
identity instead of resetting with every Hermes build), and Hermes verifies
the bundle's code signature — exact `com.trycua.driver` identifier and the
official signing team — before launching it. If you build cua-driver from
source (unsigned), opt in explicitly:
```yaml
# config.yaml
computer_use:
allow_unsigned_driver: true # local driver development only
```
Each MCP transport owns a private lifecycle session inside its runtime. A
public session name is only a label for cursor identity and session-scoped
state. It does not select, share, or keep a runtime alive. Turning `/yolo` off,
resetting or closing the Hermes session, cancellation cleanup, or process exit
closes that transport session. Hermes also stops private runtimes that it
launched for bounded or unrestricted access. One Hermes
conversation cannot change another runtime's mode or grants. Bounded and
unrestricted modes use a private
embedded service under the Hermes host identity.
`smart` approval remains `standard`: an LLM classification cannot stand in for
a reviewed manifest.
<div class="alert alert--warning">
YOLO/unrestricted mode does not protect against prompt injection or unintended
input. Use it only in a disposable VM or with accounts and data whose full
compromise you accept.
</div>
## `hermes computer-use doctor` — your first triage stop
`hermes computer-use doctor` runs cua-driver's structured
`health_report` MCP tool and prints a per-check matrix. It's the single
fastest way to find out *why* an action isn't working.
```
$ hermes computer-use doctor
⚠️ cua-driver VERSION on darwin: degraded
✅ binary_version: cua-driver VERSION
✅ platform_supported: macOS 26.4.1 (arm64)
✅ session_active: MCP session is active.
❌ bundle_identity: Process has no CFBundleIdentifier.
→ Run the binary inside CuaDriver.app so TCC grants attribute correctly.
✅ tcc_accessibility: Accessibility is granted.
✅ tcc_screen_recording: Screen Recording is granted.
✅ ax_capability: AX is trusted and reachable.
✅ screen_capture_capability: ScreenCaptureKit reachable; 1 display(s) shareable.
```
- **Exit code 0** when overall is `ok` — everything's wired up.
- **Exit code 1** when `degraded` or `failed` — at least one check failed; the hint on each failure tells you what to fix.
- **Exit code 2** when the cua-driver binary itself isn't reachable.
Useful flags:
- `--include CHECK` — run only the listed checks (repeat for multiple)
- `--skip CHECK` — skip a check (wins over `--include`)
- `--json` — emit the raw structured payload, same shape as the
`tools/call health_report` MCP response
The check matrix is platform-aware: `bundle_identity` / `tcc_*` are
`skip` on Windows + Linux because those concepts don't apply.
`ax_capability` checks AX on macOS, UIA on Windows, AT-SPI on Linux —
each with the right diagnostic hint when it can't reach.
## The agent cursor and sessions
When the agent acts, you'll see a **tinted overlay cursor** glide
across the screen to where each click / type / scroll lands. The real
OS cursor never moves. The overlay shows where the agent is acting. Each
Hermes run declares a public cua-driver **session name** (something like
`hermes-3a7b9c14d2e8`). The name labels cursor identity and related state, so
concurrent runs and subagents get distinct cursors. The MCP transport owns the
private lifecycle session inside the runtime; the public name does not.
The overlay cursor is cosmetic — captures, clicks, and typing all work
without it. Hermes disables it automatically where it is a known failure
mode: macOS (idle CPU burn), headless Linux / WSL2 / containers, and
**Linux X11 desktops** (the overlay is a fullscreen always-on-top window
that can get stuck over every workspace after an unclean session end,
wedging desktop input). Linux Wayland and Windows keep the overlay. Set
`computer_use.no_overlay: false` in `config.yaml` to force the cursor on
(or `true` to force it off) on any platform.
Tune the cursor with `cua-driver`'s CLI flags or the runtime
`set_agent_cursor_style` MCP tool — see
[cua.ai/docs/how-to-guides/driver/personalize-cursor](https://cua.ai/docs/how-to-guides/driver/personalize-cursor)
for the full menu (built-in `arrow` vs `teardrop` silhouette, custom
SVG / PNG / ICO via `--cursor-icon`, runtime gradient colors, bloom
halo).
## Going deeper — the cua-driver skill pack
Hermes keeps its wrapper skill (`skills/autonomous-ai-agents/computer-use/SKILL.md`)
focused on the Hermes-side `computer_use` workflow and action vocabulary. For
platform details, recording semantics, browser page interaction, and other
deep Cua behavior, install the skill pack that the cua-driver team ships and
maintains directly:
```
cua-driver skills install
```
The command installs the pack under `~/.cua-driver/skills/cua-driver`. Hermes
autodetection is a planned cua-driver follow-up, so currently point Hermes at
that directory or symlink it into your skill space. The wrapper remains the
workflow layer and points to Cua's installed skill for driver behavior. The
pack contains:
| File | Topic |
|---|---|
| `SKILL.md` | The cross-platform core (snapshot invariant, no-foreground contract, click dispatch, AX-tree mechanics) |
| `MACOS.md` | macOS specifics: no-foreground contract, AXMenuBar navigation, SkyLight click dispatch, Apple Events JS bridge |
| `WINDOWS.md` | Windows specifics: UIA tree, UWP / `ApplicationFrameHost` hosting, Session 0 isolation, autostart pattern |
| `LINUX.md` | Linux specifics: AT-SPI tree, X11 / Wayland, terminal-emulator detection |
| `RECORDING.md` | Trajectory + video recording semantics |
| `WEB_APPS.md` | Browser-page interaction tips |
| `TESTS.md` | Replay-by-trajectory workflow |
These are **platform deep dives, not duplicates of the Hermes skill** —
when an agent reports "on Windows, my click landed on the wrong
element," it reads `WINDOWS.md` for the UIA / UWP context that
explains why and what to do differently.
`cua-driver skills status` shows what's installed and which agent
harnesses it's linked into. Today the autodetect list covers Claude
Code, Codex, OpenCode, OpenClaw, and Antigravity; **Hermes
autodetection is planned as a follow-up in `trycua/cua`** — until
then, run `cua-driver skills install` once and point your harness at
the resulting `~/.cua-driver/skills/cua-driver` directory (or symlink
it into your usual skill space).
## Quick example
User prompt: *"Find my latest email from Stripe and summarise what they want me to do."*
The agent's plan (this is the same shape on macOS / Windows / Linux —
the model substitutes the platform's idiomatic shortcut and app name):
1. `computer_use(action="capture", mode="som", app="Mail")` — gets a
screenshot of the email app with every sidebar item, toolbar button,
and message row numbered.
2. `computer_use(action="click", element=14)` — clicks the search field.
3. `computer_use(action="type", text="from:stripe")`
4. `computer_use(action="key", keys="return", capture_after=True)` —
submit and get the new screenshot.
5. Click the top result, read the body, summarise.
During all of this, your cursor stays wherever you left it and the email
app never comes to front.
## Receiving the actual screenshot
Screenshots taken during computer control are normally internal — they exist
so the model can see the screen, and the agent replies in text. But every
image capture also saves a bounded, shareable copy under Hermes' image cache
and reports its path, so on attachment-capable surfaces (Telegram, Discord,
Desktop, and other gateway platforms) you can simply ask:
> *"Send me a screenshot of my screen."*
and the agent delivers the real image as a native attachment, not just a
description. On the CLI there is no attachment channel, so the agent gives
you the saved file's path instead.
Only the 20 most recent capture files are kept, and screenshots are never
sent automatically — only when you ask for one.
### Whole screen vs. desktop surface
"Screenshot my screen" captures **everything currently displayed** — a
composited grab of all visible windows, like pressing PrtScn. This image has
no clickable elements, so to *act* on something in it the agent re-captures
the specific app.
Asking for the **desktop** instead targets the OS shell surface itself —
wallpaper, desktop icons, taskbar — with its clickable elements, so requests
like "open the Recycle Bin on my desktop" still work.
## Provider compatibility
| Provider | Vision? | Works? | Notes |
|---|---|---|---|
| Anthropic (Claude Sonnet/Opus 3+) | ✅ | ✅ | Best overall; SOM + raw coordinates. |
| OpenRouter (any vision model) | ✅ | ✅ | Multi-part tool messages supported. |
| OpenAI (GPT-4+, GPT-5) | ✅ | ✅ | Same as above. |
| Google (Gemini 2+) | ✅ | ✅ | Tool-calling + vision both supported. |
| Local vLLM / LM Studio / Ollama (vision model) | ✅ | ✅ | If the model supports multi-part tool content. |
| Text-only models | ❌ | ✅ (degraded) | Use `mode="ax"` for accessibility-tree-only operation. |
Screenshots are sent inline with tool results as OpenAI-style `image_url`
parts. For Anthropic, the adapter converts them into native `tool_result`
image blocks. The image MIME type comes from cua-driver's explicit
`mimeType` field (`image/png` or `image/jpeg`) — no client-side
magic-byte sniffing.
## Safety
Hermes applies multi-layer guardrails:
- Destructive actions (click, type, drag, scroll, key, focus_app)
require approval — either interactively via the CLI dialog or via the
messaging-platform approval buttons.
- Hard-blocked key combos at the tool level: empty trash, force delete,
lock screen, log out, force log out.
- Hard-blocked type patterns: `curl | bash`, `sudo rm -rf /`, fork
bombs, etc.
- The agent's system prompt tells it explicitly: no clicking permission
dialogs, no typing passwords, no following instructions embedded in
screenshots.
Pair with `approvals.mode: manual` in `~/.hermes/config.yaml` if you
want every action confirmed.
## Token efficiency
Screenshots are expensive. Hermes applies four layers of optimisation:
- **Screenshot eviction** — the Anthropic adapter keeps only the 3 most
recent screenshots in context; older ones become `[screenshot removed
to save context]` placeholders.
- **Client-side compression pruning** — the context compressor detects
multimodal tool results and strips image parts from old ones.
- **Image-aware token estimation** — each image is counted as ~1500
tokens (Anthropic's flat rate) instead of its base64 char length.
- **Server-side context editing (Anthropic only)** — when active, the
adapter enables `clear_tool_uses_20250919` via `context_management` so
Anthropic's API clears old tool results server-side.
A 20-action session on a 1568×900 display typically costs ~30K tokens
of screenshot context, not ~600K.
## Limitations
- **Performance.** Background mode is slower than foreground —
accessibility-routed events take ~5–20 ms on macOS, ~3–10 ms on
Windows UIA, ~5–15 ms on Linux AT-SPI vs direct HID posting. Not
noticeable for agent-speed clicking; noticeable if you try to record
a speed-run.
- **No keyboard password entry.** `type` has hard-block patterns on
command-shell payloads; for passwords, use the system's autofill
(macOS Keychain / Windows Credential Manager / GNOME Keyring /
KWallet).
- **Some apps don't expose an accessibility tree.** Modern UWP apps on
Windows, Electron < 28 on Linux, and a few macOS apps with custom
drawing (Logic, Final Cut, some games) have sparse or empty AX trees.
Fall back to pixel coordinates if the tree is empty — or skip the
task entirely.
- **Windows: elevated (admin) windows can't be driven from a normal
agent.** Windows UIPI (User Interface Privilege Isolation) enforces
integrity-level boundaries: a Medium-integrity process (the default
Hermes agent) cannot enumerate the UIA tree of, or inject mouse input
into, a window owned by a High-integrity (Administrator) process.
Symptom: `capture(mode='som')` returns 0 elements and `click(...)`
reports success while doing nothing, even though the screenshot
renders fine (GDI capture sits below the integrity check). Keyboard
events partially bypass UIPI, so Tab / Enter can still navigate an
elevated dialog. This is an OS constraint, not a cua-driver bug — it
affects every Windows automation stack. To drive elevated windows,
run the Hermes agent itself at High integrity (launch from an
elevated terminal); otherwise target non-elevated windows.
- **Platform-specific deployment gotchas:**
- **macOS** uses private SkyLight SPIs. Apple can change them in any
OS update. Hermes warns when the installed cua-driver is older than
the version it was tested against.
- **Windows** SSH sessions run in **Session 0**, which has no
interactive desktop. Drive Hermes from inside the RDP / console
session, or set up cua-driver's autostart Scheduled Task —
[windows-ssh](https://cua.ai/docs/how-to-guides/driver/windows-ssh)
has the recipe.
- **Linux** requires a reachable display server. Headless servers
need Xvfb (`Xvfb :99 -screen 0 1920x1080x24`) before
`computer_use` can capture or inject events. Pure Wayland sessions
need an XWayland bridge for screen capture (cua-driver's Wayland
inject path handles input independently).
For cross-platform GUI automation without the desktop overhead (and
without TCC / Session 0 / X11 setup), the `browser` toolset uses a
real headless Chromium and is the right answer for web-only tasks.
## Configuration
Permission mode and manifest (see
[Permission modes](#permission-modes-and-logged-in-browser-profiles) above):
```yaml
computer_use:
permission_mode: standard # standard (default) | bounded
capability_manifest: "" # capability manifest path, required for bounded
```
On Linux, native Wayland support remains an explicit opt-in. Hermes passes the
opt-in to every cua-driver process, including gateway sessions, only when that
process also has `WAYLAND_DISPLAY`:
```yaml
computer_use:
native_wayland: true
```
Restart a running gateway after changing this setting.
Override the driver binary path (tests / CI / local builds):
```
HERMES_CUA_DRIVER_CMD=/path/to/your/cua-driver
```
Swap the backend entirely (for testing):
```
HERMES_COMPUTER_USE_BACKEND=noop # records calls, no side effects
```
### Telemetry
cua-driver ships with anonymous usage telemetry (PostHog) enabled by default
upstream. **Hermes disables it for you** — on every cua-driver invocation
(the MCP backend, `status`, `doctor`, and install) Hermes sets
`CUA_DRIVER_RS_TELEMETRY_ENABLED=0` in the driver's environment.
To opt back in (let cua-driver use its own default and send telemetry), set
this in `config.yaml`:
```yaml
computer_use:
cua_telemetry: true # default: false (telemetry off)
```
When it's on, `hermes computer-use doctor` reports `telemetry: enabled`;
when off (the default), it reports `telemetry: disabled via
CUA_DRIVER_RS_TELEMETRY_ENABLED`.
## Testing against a local cua-driver build
When you're developing cua-driver itself — or want to test an
unreleased fix — point Hermes at a binary you built from source instead
of the published release. Hermes resolves the driver with
`shutil.which("cua-driver")` and **does not enforce
`HERMES_CUA_DRIVER_VERSION`**, so a local build (reported as
`0.0.0-local-*`) is accepted as-is. Two approaches:
### Option A — `install-local` (build + put it on PATH)
From your `trycua/cua` checkout, run the upstream local installer. It
builds the Rust backend in release mode and drops `cua-driver` into the
same install layout the production installer uses, adding its bin dir
to your PATH:
```powershell
# Windows (PowerShell), from the cua repo root
./libs/cua-driver/scripts/install-local.ps1 -NoAutoStart
```
```bash
# macOS / Linux, from the cua repo root (defaults to a debug build without --release)
./libs/cua-driver/scripts/install-local.sh --release
```
- Windows stages the build under `%USERPROFILE%\.cua-driver\packages\…`
and junctions
`%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` (added to your User
PATH) to it. macOS/Linux symlinks `cua-driver` into `~/.local/bin`
(override with `--bin-dir <path>`).
- `-NoAutoStart` skips registering the `cua-driver-serve` logon daemon
— you don't need it for Hermes testing (see notes).
Then open a fresh shell (so the PATH change is visible) and confirm:
```
cua-driver --version # local builds report 0.0.0-local-release
# Windows: (Get-Command cua-driver).Source
# macOS/Linux: which cua-driver
```
### Option B — point Hermes straight at the built binary (fastest loop)
Skip the install ceremony entirely: `cargo build` and set
`HERMES_CUA_DRIVER_CMD` to the resulting binary. Best for rapid
edit/build/test.
```bash
cargo build -p cua-driver # add --release for a release build; run from libs/cua-driver/rust
```
```
# Windows (.env)
HERMES_CUA_DRIVER_CMD=C:\path\to\cua\libs\cua-driver\rust\target\debug\cua-driver.exe
# macOS / Linux (.env)
HERMES_CUA_DRIVER_CMD=/path/to/cua/libs/cua-driver/rust/target/debug/cua-driver
```
### Confirm Hermes is using your build
- `hermes computer-use status` prints the resolved binary path and
version.
- `hermes computer-use doctor` confirms the binary is reachable and
exercises the full MCP path end-to-end.
- In a session, `computer_use(action="capture")` exercises the spawned
`cua-driver mcp` child process.
### Notes & gotchas
- **Hermes spawns a `cua-driver mcp` stdio proxy.** In a normal session the
proxy connects to (and may start) the standard machine daemon. In explicit
Hermes YOLO, Hermes instead owns a private `cua-driver serve --embedded`
child and points the proxy at its private socket or named pipe. The Windows
autostart/UIAccess pattern still matters for interactive Session 1+ input
from SSH — see the Limitations section.
- **Locked binary on Windows.** A running `cua-driver-serve` daemon can
hold `cua-driver.exe` and block an overwrite on rebuild.
`install-local.ps1` renames the locked binary out of the way
automatically; if you `cargo build` manually (Option B), stop it
first with `cua-driver autostart disable` (or `schtasks /End /TN
cua-driver-serve`).
- **Rebuild loop.** After editing cua-driver source, re-run
`install-local` (rebuilds, restages, flips the `current` junction)
for Option A, or just re-`cargo build` for Option B — no Hermes
change needed either way.
- **Local builds skip the version check.** Hermes warns when the
installed cua-driver is older than its per-OS tested baseline, but
exempts `0.0.0-local-*` dev builds — so your local build never
triggers that warning.
## Troubleshooting
**First action when anything's off: run `hermes computer-use doctor`.**
The structured per-check matrix tells you (and any agent helping you
debug) exactly what's wrong.
Specific failure modes the doctor doesn't catch:
**`computer_use backend unavailable: cua-driver is not installed`** —
Run `hermes computer-use install` to fetch the cua-driver binary, or
run `hermes tools` and enable the Computer Use toolset.
**Clicks seem to have no effect** — Capture and verify. A modal you
didn't see may be blocking input. Dismiss it with `escape` or the close
button.
**Element indices are stale** — SOM indices are only valid until the
next `capture`. Re-capture after any state-changing action. The
wrapper carries opaque `element_token`s for stale detection — you'll
see an explicit error rather than a wrong click.
**"blocked pattern in type text"** — The text you tried to `type`
matches the dangerous-shell-pattern list. Break the command up or
reconsider.
**Empty captures on Linux** — `DISPLAY` not set, or you're on pure
Wayland without an XWayland bridge. `hermes computer-use doctor` will
flag this as `ax_capability: fail` with a `Set DISPLAY (X11)…` hint.
**Empty captures on Windows over SSH** — You're in Session 0 (the
services session). Drive from RDP / console directly, or set up the
autostart pattern — see
[cua.ai/docs/how-to-guides/driver/windows-ssh](https://cua.ai/docs/how-to-guides/driver/windows-ssh).
## See also
- **Hermes-side skill** — `skills/autonomous-ai-agents/computer-use/SKILL.md` — teaches the
Hermes `computer_use` action vocabulary; this is what the agent loads.
- **cua-driver skill pack** — for platform-specific deep dives
(macOS no-foreground contract, Windows UIA + Session 0, Linux AT-SPI
+ X11/Wayland, recording, browser pages), run
`cua-driver skills install` and read `MACOS.md` / `WINDOWS.md` /
`LINUX.md` / `RECORDING.md` / `WEB_APPS.md`. Hermes autodetection is a
planned follow-up; currently point Hermes at the installed pack directory
or symlink it into your skill space.
- **cua.ai/docs** — the cua-driver project's documentation:
- [What is computer use?](https://cua.ai/docs/explanation/what-is-computer-use) — concept intro
- [The no-foreground contract](https://cua.ai/docs/explanation/the-no-foreground-contract) — *why* background mode matters
- [Install reference](https://cua.ai/docs/how-to-guides/driver/install) — cross-platform install details
- [Personalize the agent cursor](https://cua.ai/docs/how-to-guides/driver/personalize-cursor) — built-in shapes, custom assets, runtime overrides
- [Drive Windows over SSH](https://cua.ai/docs/how-to-guides/driver/windows-ssh) — the Session 0 → Session 1+ autostart pattern
- [Keep cua-driver running](https://cua.ai/docs/how-to-guides/driver/keep-running) — autostart / daemon lifecycle
- [Connect your agent](https://cua.ai/docs/how-to-guides/driver/connect-your-agent) — register cua-driver with various harnesses (Hermes among them)
- [cua-driver source (trycua/cua)](https://github.com/trycua/cua)
- [Browser automation](./browser.md) for cross-platform web tasks where you don't need to drive native apps.
website/docs/user-guide/features/context-files.md
+237
View File
@@ -0,0 +1,237 @@
---
sidebar_position: 8
title: "Context Files"
description: "Project context files — .hermes.md, AGENTS.md, CLAUDE.md, global SOUL.md, and .cursorrules — automatically injected into every conversation"
---
# Context Files
Hermes Agent automatically discovers and loads context files that shape how it behaves. Some are project-local and discovered from your working directory. `SOUL.md` is now global to the Hermes instance and is loaded from `HERMES_HOME` only.
## Supported Context Files
| File | Purpose | Discovery |
|------|---------|-----------|
| **.hermes.md** / **HERMES.md** | Project instructions (highest priority) | Walks to git root |
| **AGENTS.override.md** | Personal, per-directory override of AGENTS.md (typically gitignored) | CWD at startup + subdirectories progressively |
| **AGENTS.md** | Project instructions, conventions, architecture | CWD at startup + subdirectories progressively |
| **CLAUDE.md** | Claude Code context files (also detected) | CWD at startup + subdirectories progressively |
| **SOUL.md** | Global personality and tone customization for this Hermes instance | `HERMES_HOME/SOUL.md` only |
| **.cursorrules** | Cursor IDE coding conventions | CWD only |
| **.cursor/rules/*.mdc** | Cursor IDE rule modules | CWD only |
:::info Priority system
Only **one** project context type is loaded per session (first match wins): `.hermes.md` → `AGENTS.override.md` → `AGENTS.md` → `CLAUDE.md` → `.cursorrules`. **SOUL.md** is always loaded independently as the agent identity (slot #1).
If an `AGENTS.override.md` exists next to an `AGENTS.md`, the override is loaded **instead of** the committed file — keep a personal (usually gitignored) `AGENTS.override.md` when you want different instructions than the ones checked into the repo, without editing the tracked `AGENTS.md`.
:::
## AGENTS.md
`AGENTS.md` is the primary project context file. It tells the agent how your project is structured, what conventions to follow, and any special instructions.
### Directory Chain (git root → working directory)
When your working directory sits inside a git repository, Hermes loads a **merged chain** of `AGENTS.md` files at session start: the git-root `AGENTS.md` first, then the `AGENTS.md` in every intermediate directory down to your working directory. Deeper files appear later in the prompt, so more specific guidance takes precedence. Each file gets its own provenance header (e.g. `## ../../AGENTS.md`), and identical copies along the chain are deduplicated.
```
monorepo/ (git root, cwd = packages/webapp/)
├── AGENTS.md ← Loaded first (repo-wide conventions)
└── packages/
├── AGENTS.md ← Loaded second
└── webapp/
└── AGENTS.md ← Loaded last (most specific, takes precedence)
```
Outside a git repository, only the working directory itself is checked — parents are never consulted, so an `AGENTS.md` planted in `/tmp` or `$HOME` can't leak into unrelated sessions.
### Progressive Subdirectory Discovery
At session start, Hermes loads the `AGENTS.md` from your working directory into the system prompt. As the agent navigates into subdirectories during the session (via `read_file`, `terminal`, `search_files`, etc.), it **progressively discovers** context files in those directories and injects them into the conversation at the moment they become relevant.
```
my-project/
├── AGENTS.md ← Loaded at startup (system prompt)
├── frontend/
│ └── AGENTS.md ← Discovered when agent reads frontend/ files
├── backend/
│ └── AGENTS.md ← Discovered when agent reads backend/ files
└── shared/
└── AGENTS.md ← Discovered when agent reads shared/ files
```
This approach has two advantages over loading everything at startup:
- **No system prompt bloat** — subdirectory hints only appear when needed
- **Prompt cache preservation** — the system prompt stays stable across turns
Each subdirectory is checked at most once per session. The discovery also walks up parent directories, so reading `backend/src/main.py` will discover `backend/AGENTS.md` even if `backend/src/` has no context file of its own.
:::info
Subdirectory context files go through the same [security scan](#security-prompt-injection-protection) as startup context files. Malicious files are blocked.
:::
### Example AGENTS.md
```markdown
# Project Context
This is a Next.js 14 web application with a Python FastAPI backend.
## Architecture
- Frontend: Next.js 14 with App Router in `/frontend`
- Backend: FastAPI in `/backend`, uses SQLAlchemy ORM
- Database: PostgreSQL 16
- Deployment: Docker Compose on a Hetzner VPS
## Conventions
- Use TypeScript strict mode for all frontend code
- Python code follows PEP 8, use type hints everywhere
- All API endpoints return JSON with `{data, error, meta}` shape
- Tests go in `__tests__/` directories (frontend) or `tests/` (backend)
## Important Notes
- Never modify migration files directly — use Alembic commands
- The `.env.local` file has real API keys, don't commit it
- Frontend port is 3000, backend is 8000, DB is 5432
```
## SOUL.md
`SOUL.md` controls the agent's personality, tone, and communication style. See the [Personality](/user-guide/features/personality) page for full details.
**Location:**
- `~/.hermes/SOUL.md`
- or `$HERMES_HOME/SOUL.md` if you run Hermes with a custom home directory
Important details:
- Hermes seeds a default `SOUL.md` automatically if one does not exist yet
- Hermes loads `SOUL.md` only from `HERMES_HOME`
- Hermes does not probe the working directory for `SOUL.md`
- If the file is empty, nothing from `SOUL.md` is added to the prompt
- If the file has content, the content is injected verbatim after scanning and truncation
## .cursorrules
Hermes is compatible with Cursor IDE's `.cursorrules` file and `.cursor/rules/*.mdc` rule modules. If these files exist in your project root and no higher-priority context file (`.hermes.md`, `AGENTS.md`, or `CLAUDE.md`) is found, they're loaded as the project context.
This means your existing Cursor conventions automatically apply when using Hermes.
## How Context Files Are Loaded
### At startup (system prompt)
Context files are loaded by `build_context_files_prompt()` in `agent/prompt_builder.py`:
1. **Scan working directory** — checks for `.hermes.md` → `AGENTS.md` → `CLAUDE.md` → `.cursorrules` (first match wins)
2. **Content is read** — each file is read as UTF-8 text
3. **Security scan** — content is checked for prompt injection patterns
4. **Truncation** — files exceeding the character cap are head/tail truncated (70% head, 20% tail, with a marker in the middle). The cap is an explicit `context_file_max_chars` from config.yaml when set; otherwise it scales dynamically with the model's context window (floor 20,000 chars, ceiling 500,000)
5. **Assembly** — all sections are combined under a `# Project Context` header
6. **Injection** — the assembled content is added to the system prompt
### During the session (progressive discovery)
`SubdirectoryHintTracker` in `agent/subdirectory_hints.py` watches tool call arguments for file paths:
1. **Path extraction** — after each tool call, file paths are extracted from arguments (`path`, `workdir`, shell commands)
2. **Ancestor walk** — the directory and up to 5 parent directories are checked (stopping at already-visited directories)
3. **Hint loading** — if an `AGENTS.md`, `CLAUDE.md`, or `.cursorrules` is found, it's loaded (first match per directory)
4. **Security scan** — same prompt injection scan as startup files
5. **Truncation** — capped at 8,000 characters per file
6. **Injection** — appended to the tool result, so the model sees it in context naturally
The final prompt section looks roughly like:
```text
# Project Context
The following project context files have been loaded and should be followed:
## AGENTS.md
[Your AGENTS.md content here]
## .cursorrules
[Your .cursorrules content here]
[Your SOUL.md content here]
```
Notice that SOUL content is inserted directly, without extra wrapper text.
## Security: Prompt Injection Protection
All context files are scanned for potential prompt injection before being included. The scanner checks for:
- **Instruction override attempts**: "ignore previous instructions", "disregard your rules"
- **Deception patterns**: "do not tell the user"
- **System prompt overrides**: "system prompt override"
- **Hidden HTML comments**: `<!-- ignore instructions -->`
- **Hidden div elements**: `<div style="display:none">`
- **Credential exfiltration**: `curl ... $API_KEY`
- **Secret file access**: `cat .env`, `cat credentials`
- **Invisible characters**: zero-width spaces, bidirectional overrides, word joiners
If any threat pattern is detected, the file is blocked:
```
[BLOCKED: AGENTS.md contained potential prompt injection (prompt_injection). Content not loaded.]
```
:::warning
This scanner protects against common injection patterns, but it's not a substitute for reviewing context files in shared repositories. Always validate AGENTS.md content in projects you didn't author.
:::
## Size Limits
| Limit | Value |
|-------|-------|
| Max chars per file | `context_file_max_chars` when set; otherwise dynamic (scales with model context window, floor 20,000, ceiling 500,000) |
| Read timeout per file | `context_file_read_timeout` (default 5 seconds); a file that takes longer to read — e.g. on iCloud Drive, OneDrive or NFS — is skipped with a warning |
| Head truncation ratio | 70% |
| Tail truncation ratio | 20% |
| Truncation marker | 10% (shows char counts and suggests using file tools) |
When a file exceeds the configured limit, the truncation message reads:
```
[...truncated AGENTS.md: kept 14000+4000 of 25000 chars. Use file tools to read the full file.]
```
## Tips for Effective Context Files
:::tip Best practices for AGENTS.md
1. **Keep it concise** — stay under your configured `context_file_max_chars`; the agent reads it every turn
2. **Structure with headers** — use `##` sections for architecture, conventions, important notes
3. **Include concrete examples** — show preferred code patterns, API shapes, naming conventions
4. **Mention what NOT to do** — "never modify migration files directly"
5. **List key paths and ports** — the agent uses these for terminal commands
6. **Update as the project evolves** — stale context is worse than no context
:::
### Per-Subdirectory Context
For monorepos, put subdirectory-specific instructions in nested AGENTS.md files:
```markdown
<!-- frontend/AGENTS.md -->
# Frontend Context
- Use `pnpm` not `npm` for package management
- Components go in `src/components/`, pages in `src/app/`
- Use Tailwind CSS, never inline styles
- Run tests with `pnpm test`
```
```markdown
<!-- backend/AGENTS.md -->
# Backend Context
- Use `poetry` for dependency management
- Run the dev server with `poetry run uvicorn main:app --reload`
- All endpoints need OpenAPI docstrings
- Database models are in `models/`, schemas in `schemas/`
```
website/docs/user-guide/features/context-references.md
+142
View File
@@ -0,0 +1,142 @@
---
sidebar_position: 9
sidebar_label: "Context References"
title: "Context References"
description: "Inline @-syntax for attaching files, folders, git diffs, and URLs directly into your messages"
---
# Context References
Type `@` followed by a reference to inject content directly into your message. Hermes expands the reference inline and appends the content under an `--- Attached Context ---` section.
## Supported References
| Syntax | Description |
|--------|-------------|
| `@file:path/to/file.py` | Inject file contents |
| `@file:path/to/file.py:10-25` | Inject specific line range (1-indexed, inclusive) |
| `@folder:path/to/dir` | Inject directory tree listing with file metadata |
| `@diff` | Inject `git diff` (unstaged working tree changes) |
| `@staged` | Inject `git diff --staged` (staged changes) |
| `@git:5` | Inject last N commits with patches (max 10) |
| `@url:https://example.com` | Fetch and inject web page content |
## Usage Examples
```text
Review @file:src/main.py and suggest improvements
What changed? @diff
Compare @file:old_config.yaml and @file:new_config.yaml
What's in @folder:src/components?
Summarize this article @url:https://arxiv.org/abs/2301.00001
```
Multiple references work in a single message:
```text
Check @file:main.py, and also @file:test.py.
```
Trailing punctuation (`,`, `.`, `;`, `!`, `?`) is automatically stripped from reference values.
## CLI Tab Completion
In the interactive CLI, typing `@` triggers autocomplete:
- `@` shows all reference types (`@diff`, `@staged`, `@file:`, `@folder:`, `@git:`, `@url:`)
- `@file:` and `@folder:` trigger filesystem path completion with file size metadata
- Bare `@` followed by partial text shows matching files and folders from the current directory
## Line Ranges
The `@file:` reference supports line ranges for precise content injection:
```text
@file:src/main.py:42 # Single line 42
@file:src/main.py:10-25 # Lines 10 through 25 (inclusive)
```
Lines are 1-indexed. Invalid ranges are silently ignored (full file is returned).
## Size Limits
Context references are bounded to prevent overwhelming the model's context window:
| Threshold | Value | Behavior |
|-----------|-------|----------|
| Soft limit | 25% of context length | Warning appended, expansion proceeds |
| Hard limit | 50% of context length | Expansion refused, original message returned unchanged |
| Folder entries | 200 files max | Excess entries replaced with `- ...` |
| Git commits | 10 max | `@git:N` clamped to range [1, 10] |
## Security
### Sensitive Path Blocking
These paths are always blocked from `@file:` references to prevent credential exposure:
- SSH keys and config: `~/.ssh/id_rsa`, `~/.ssh/id_ed25519`, `~/.ssh/authorized_keys`, `~/.ssh/config`
- Shell profiles: `~/.bashrc`, `~/.zshrc`, `~/.profile`, `~/.bash_profile`, `~/.zprofile`
- Credential files: `~/.netrc`, `~/.pgpass`, `~/.npmrc`, `~/.pypirc`
- Hermes env: `$HERMES_HOME/.env`
These directories are fully blocked (any file inside):
- `~/.ssh/`, `~/.aws/`, `~/.gnupg/`, `~/.kube/`, `$HERMES_HOME/skills/.hub/`
### Path Traversal Protection
All paths are resolved relative to the working directory. References that resolve outside the allowed workspace root are rejected.
### Binary File Detection
Binary files are detected via MIME type and null-byte scanning. Known text extensions (`.py`, `.md`, `.json`, `.yaml`, `.toml`, `.js`, `.ts`, etc.) bypass MIME-based detection. Binary files are rejected with a warning.
## Platform Availability
Context references are primarily a **CLI feature**. They work in the interactive CLI where `@` triggers tab completion and references are expanded before the message is sent to the agent.
In **messaging platforms** (Telegram, Discord, etc.), the `@` syntax is not expanded by the gateway — messages are passed through as-is. The agent itself can still reference files via the `read_file`, `search_files`, and `web_extract` tools.
## Interaction with Context Compression
When conversation context is compressed, the expanded reference content is included in the compression summary. This means:
- Large file contents injected via `@file:` contribute to context usage
- If the conversation is later compressed, the file content is summarized (not preserved verbatim)
- For very large files, consider using line ranges (`@file:main.py:100-200`) to inject only relevant sections
## Common Patterns
```text
# Code review workflow
Review @diff and check for security issues
# Debug with context
This test is failing. Here's the test @file:tests/test_auth.py
and the implementation @file:src/auth.py:50-80
# Project exploration
What does this project do? @folder:src @file:README.md
# Research
Compare the approaches in @url:https://arxiv.org/abs/2301.00001
and @url:https://arxiv.org/abs/2301.00002
```
## Error Handling
Invalid references produce inline warnings rather than failures:
| Condition | Behavior |
|-----------|----------|
| File not found | Warning: "file not found" |
| Binary file | Warning: "binary files are not supported" |
| Folder not found | Warning: "folder not found" |
| Git command fails | Warning with git stderr |
| URL returns no content | Warning: "no content extracted" |
| Sensitive path | Warning: "path is a sensitive credential file" |
| Path outside workspace | Warning: "path is outside the allowed workspace" |
website/docs/user-guide/features/credential-pools.md
+267
View File
@@ -0,0 +1,267 @@
---
title: Credential Pools
description: Pool multiple API keys or OAuth tokens per provider for automatic rotation and rate limit recovery.
sidebar_label: Credential Pools
sidebar_position: 9
---
# Credential Pools
Credential pools let you register multiple API keys or OAuth tokens for the same provider. When one key hits a rate limit or billing quota, Hermes automatically rotates to the next healthy key — keeping your session alive without switching providers.
This is different from [fallback providers](./fallback-providers.md), which switch to a *different* provider entirely. Credential pools are same-provider rotation; fallback providers are cross-provider failover. Pools are tried first — if all pool keys are exhausted, *then* the fallback provider activates.
:::warning Key rotation resets the prompt cache
Provider-side prompt caches (Anthropic, OpenAI, OpenRouter) are scoped to the account/API key that made the request. When the pool rotates to a different key mid-session, the new key has no cached prefix for your conversation — the next request re-reads the full history at undiscounted input price, and rotating back later is another full re-read unless the earlier key's cache TTL is still alive. Rotation keeps your session running, which is the point, but on long conversations each rotation costs one full-price pass over the context.
:::
:::tip
Credential pools are mainly for API-key providers (OpenRouter, Anthropic). A single [Nous Portal](/integrations/nous-portal) OAuth covers 300+ models, so most users don't need a pool when on Portal.
:::
## How It Works
```
Your request
→ Pick key from pool (round_robin / least_used / fill_first / random)
→ Send to provider
→ 429 rate limit?
→ Plan/usage limit reached (e.g. ChatGPT/Codex "usage limit reached")?
→ Rotate to next pool key immediately (no retry — the cap won't clear on retry)
→ Generic / transient 429?
→ Retry same key once (transient blip)
→ Second 429 → rotate to next pool key
→ All keys exhausted → fallback_model (different provider)
→ 402 billing error?
→ Immediately rotate to next pool key (1h cooldown)
→ 401 auth expired?
→ Try refreshing the token (OAuth)
→ Refresh failed → rotate to next pool key
→ Success → continue normally
```
## Quick Start
If you already have an API key set in `.env`, Hermes auto-discovers it as a 1-key pool. To benefit from pooling, add more keys:
```bash
# Add a second OpenRouter key
hermes auth add openrouter --api-key sk-or-v1-your-second-key
# Add a second Anthropic key
hermes auth add anthropic --type api-key --api-key sk-ant-api03-your-second-key
# Add an Anthropic OAuth credential (requires Claude Max plan + extra usage credits)
hermes auth add anthropic --type oauth
# Opens browser for OAuth login
```
Check your pools:
```bash
hermes auth list
```
Output:
```
openrouter (2 credentials):
#1 OPENROUTER_API_KEY api_key env:OPENROUTER_API_KEY ←
#2 backup-key api_key manual
anthropic (3 credentials):
#1 hermes_pkce oauth hermes_pkce ←
#2 claude_code oauth claude_code
#3 ANTHROPIC_API_KEY api_key env:ANTHROPIC_API_KEY
```
The `←` marks the currently selected credential.
## Interactive Management
Run `hermes auth` with no subcommand for an interactive wizard:
```bash
hermes auth
```
This shows your full pool status and offers a menu:
```
What would you like to do?
1. Add a credential
2. Remove a credential
3. Reset cooldowns for a provider
4. Set rotation strategy for a provider
5. Exit
```
For providers that support both API keys and OAuth (Anthropic, Nous, Codex), the add flow asks which type:
```
anthropic supports both API keys and OAuth login.
1. API key (paste a key from the provider dashboard)
2. OAuth login (authenticate via browser)
Type [1/2]:
```
## CLI Commands
| Command | Description |
|---------|-------------|
| `hermes auth` | Interactive pool management wizard |
| `hermes auth list` | Show all pools and credentials |
| `hermes auth list <provider>` | Show a specific provider's pool |
| `hermes auth add <provider>` | Add a credential (prompts for type and key) |
| `hermes auth add <provider> --type api-key --api-key <key>` | Add an API key non-interactively |
| `hermes auth add <provider> --type oauth` | Add an OAuth credential via browser login |
| `hermes auth remove <provider> <index>` | Remove credential by 1-based index |
| `hermes auth reset <provider>` | Clear all cooldowns/exhaustion status |
## Rotation Strategies
Configure via `hermes auth` → "Set rotation strategy" or in `config.yaml`:
```yaml
credential_pool_strategies:
openrouter: round_robin
anthropic: least_used
```
| Strategy | Behavior |
|----------|----------|
| `fill_first` (default) | Use the first healthy key until it's exhausted, then move to the next |
| `round_robin` | Cycle through keys evenly, rotating after each selection |
| `least_used` | Always pick the key with the lowest request count |
| `random` | Random selection among healthy keys |
## Error Recovery
The pool handles different errors differently:
| Error | Behavior | Cooldown |
|-------|----------|----------|
| **429 Rate Limit** | Retry same key once (transient). Second consecutive 429 rotates to next key | 1 hour |
| **402 Billing/Quota** | Immediately rotate to next key | 1 hour |
| **401 Auth Expired** | Try refreshing the OAuth token first. Rotate only if refresh fails | 5 minutes |
| **All keys exhausted** | Fall through to `fallback_model` if configured | — |
Provider-supplied `reset_at` timestamps override these default cooldowns.
The `has_retried_429` flag resets on every successful API call, so a single transient 429 doesn't trigger rotation.
## Custom Endpoint Pools
Custom OpenAI-compatible endpoints (Together.ai, RunPod, local servers) get their own pools, keyed by the endpoint name from the `providers:` dict in config.yaml (or the legacy `custom_providers` list, which is auto-migrated).
When you set up a custom endpoint via `hermes model`, it auto-generates a name like "Together.ai" or "Local (localhost:8080)". This name becomes the pool key.
```bash
# After setting up a custom endpoint via hermes model:
hermes auth list
# Shows:
# Together.ai (1 credential):
# #1 config key api_key config:Together.ai ←
# Add a second key for the same endpoint:
hermes auth add Together.ai --api-key sk-together-second-key
```
Custom endpoint pools are stored in `auth.json` under `credential_pool` with a `custom:` prefix:
```json
{
"credential_pool": {
"openrouter": [...],
"custom:together.ai": [...]
}
}
```
## Auto-Discovery
Hermes automatically discovers credentials from multiple sources and seeds the pool on startup:
| Source | Example | Auto-seeded? |
|--------|---------|-------------|
| Environment variables | `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY` | Yes |
| OAuth tokens (auth.json) | Codex device code, Nous device code | Yes |
| Claude Code credentials | `~/.claude/.credentials.json` | Yes (Anthropic) |
| Hermes PKCE OAuth | `~/.hermes/auth.json` | Yes (Anthropic) |
| Custom endpoint config | `model.api_key` in config.yaml | Yes (custom endpoints) |
| Manual entries | Added via `hermes auth add` | Persisted in auth.json |
Auto-seeded entries are updated on each pool load — if you remove an env var, its pool entry is automatically pruned. Manual entries (added via `hermes auth add`) are never auto-pruned.
Borrowed runtime secrets (for example env vars, Bitwarden/Vault/keyring/systemd references, and custom config values) are reference-only at the `auth.json` boundary. Hermes can use the resolved value in memory for the current run, but it persists only metadata such as the source ref, label, status, request counters, and a non-reversible fingerprint. Manual entries and Hermes-owned OAuth/device-code state keep the durable tokens they need to refresh.
## Delegation & Subagent Sharing
When the agent spawns subagents via `delegate_task`, the parent's credential pool is automatically shared with children:
- **Same provider** — the child receives the parent's full pool, enabling key rotation on rate limits
- **Different provider** — the child loads that provider's own pool (if configured)
- **No pool configured** — the child falls back to the inherited single API key
This means subagents benefit from the same rate-limit resilience as the parent, with no extra configuration needed. Per-task credential leasing ensures children don't conflict with each other when rotating keys concurrently.
## Thread Safety
The credential pool uses a threading lock for all state mutations (`select()`, `mark_exhausted_and_rotate()`, `try_refresh_current()`, `mark_used()`). This ensures safe concurrent access when the gateway handles multiple chat sessions simultaneously.
Across processes (many subagents, a gateway plus a CLI, cron jobs), OAuth refreshes are serialized through a file lock on `auth.json`. When one shared OAuth grant expires under many concurrent processes, exactly one process performs the refresh; the others detect that the on-disk token no longer matches the one that failed and adopt it instead of rotating the single-use refresh token again. A process that loses the lock race keeps its entry healthy and retries — lock contention is never recorded as a credential failure.
## Architecture
For the full data flow diagram, see [`docs/credential-pool-flow.excalidraw`](https://excalidraw.com/#json=2Ycqhqpi6f12E_3ITyiwh,c7u9jSt5BwrmiVzHGbm87g) in the repository.
The credential pool integrates at the provider resolution layer:
1. **`agent/credential_pool.py`** — Pool manager: storage, selection, rotation, cooldowns
2. **`hermes_cli/auth_commands.py`** — CLI commands and interactive wizard
3. **`hermes_cli/runtime_provider.py`** — Pool-aware credential resolution
4. **`run_agent.py`** — Error recovery: 429/402/401 → pool rotation → fallback
## Storage
Pool state is stored in `~/.hermes/auth.json` under the `credential_pool` key:
```json
{
"version": 1,
"credential_pool": {
"openrouter": [
{
"id": "abc123",
"label": "OPENROUTER_API_KEY",
"auth_type": "api_key",
"priority": 0,
"source": "env:OPENROUTER_API_KEY",
"secret_source": "bitwarden",
"secret_fingerprint": "sha256:12ab34cd56ef7890",
"last_status": "ok",
"request_count": 142
}
],
"anthropic": [
{
"id": "manual1",
"label": "personal-api-key",
"auth_type": "api_key",
"priority": 0,
"source": "manual",
"access_token": "sk-ant-api03-..."
}
]
}
}
```
The OpenRouter entry above was borrowed from an external source, so the raw key is not stored in `auth.json`. The manual Anthropic entry was intentionally added to Hermes' credential store, so its token remains persistable.
Strategies are stored in `config.yaml` (not `auth.json`):
```yaml
credential_pool_strategies:
openrouter: round_robin
anthropic: least_used
```
website/docs/user-guide/features/cron.md
+1126
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/features/curator.md
+401
View File
@@ -0,0 +1,401 @@
---
sidebar_position: 3
title: "Curator"
description: "Background maintenance for agent-created skills — usage tracking, staleness, archival, and LLM-driven review"
---
# Curator
The curator is a background maintenance pass for **agent-created skills**. It tracks how often each skill is viewed, used, and patched, moves long-unused skills through `active → stale → archived` states, and periodically spawns a short auxiliary-model review that proposes consolidations or patches drift.
It exists so that skills created via the [self-improvement loop](/user-guide/features/skills#agent-managed-skills-skill_manage-tool) don't pile up forever. Every time the agent solves a novel problem and saves a skill, that skill lands in `~/.hermes/skills/`. Without maintenance, you end up with dozens of narrow near-duplicates that pollute the catalog and waste tokens.
By default (`prune_builtins: true`) the curator can archive **unused bundled built-in skills** (shipped with the repo) after `archive_after_days` of non-use, alongside the agent-created skills it primarily manages. Hub-installed skills (from [agentskills.io](https://agentskills.io)) are always off-limits. Set `curator.prune_builtins: false` to restore the old agent-created-only behavior, where bundled skills are never touched. The curator also **never auto-deletes** — the worst outcome is archival into `~/.hermes/skills/.archive/`, which is recoverable.
Tracks [issue #7816](https://github.com/NousResearch/hermes-agent/issues/7816).
## How it runs
The curator is triggered by an inactivity check, not a cron daemon. On CLI session start, and on a recurring tick inside the gateway's cron-ticker thread, Hermes checks whether:
1. Enough time has passed since the last curator run (`interval_hours`, default **7 days**), and
2. The agent has been idle long enough (`min_idle_hours`, default **2 hours**).
If both are true, it spawns a background fork of `AIAgent` — the same pattern used by the memory/skill self-improvement nudges. The fork runs in its own prompt cache and never touches the active conversation.
:::info First-run behavior
On a brand-new install (or the first time a pre-curator install ticks after `hermes update`), the curator **does not run immediately**. The first observation seeds `last_run_at` to "now" and defers the first real pass by one full `interval_hours`. This gives you a full interval to review your skill library, pin anything important, or opt out entirely before the curator ever touches it.
If you want to see what the curator *would* do before it runs for real, run `hermes curator run --dry-run` — it produces the same review report without mutating the library.
:::
A run has two phases:
1. **Automatic transitions** (deterministic, no LLM). Skills unused for `stale_after_days` (30) become `stale`; skills unused for `archive_after_days` (90) are moved to `~/.hermes/skills/.archive/`. This is the always-on pruning behavior — it runs whenever the curator is enabled, with no aux-model cost.
- **Pinned skills** and **skills referenced by any cron job** (including paused/disabled jobs) are skipped entirely — treated like pin for auto-transitions so a slow or paused schedule cannot archive a skill out from under a job. Consolidation also rewrites cron skill references when it merges umbrellas.
- **Never-used skills** (`use_count == 0`) get a grace floor: they are not archived until they are at least `stale_after_days` old. Zero uses is absence of evidence, not proof the skill is disposable.
2. **LLM consolidation** (single aux-model pass with a high iteration ceiling — a full curation sweep typically takes 50–100 API calls) — **OFF by default**. When `curator.consolidate: true`, the forked agent surveys the agent-created skills, can read any of them with `skill_view`, and decides per-skill whether to keep, patch (via `skill_manage`), consolidate overlapping ones into class-level umbrellas, or archive via the terminal tool. Consolidation treats a skill as a full package: if a skill has `references/`, `templates/`, `scripts/`, `assets/`, or relative links to those paths, the curator must either keep it standalone, re-home the needed support files and rewrite paths, or archive the entire package unchanged — not flatten only `SKILL.md` into another skill's `references/` file.
:::info Consolidation is opt-in
By default the curator only **prunes** — the deterministic inactivity pass marks skills stale and archives long-unused ones. The opinionated LLM **consolidation** pass (umbrella-building, merging overlapping skills) is off by default because it costs aux-model tokens on every run and makes broad structural changes to your library. Turn it on with `curator.consolidate: true`, or run it once on demand with `hermes curator run --consolidate`.
:::
Pinned skills are off-limits to both the curator's auto-transitions and the agent's own `skill_manage` tool. See [Pinning a skill](#pinning-a-skill) below.
## Configuration
All settings live in `config.yaml` under `curator:` (not `.env` — this isn't a secret). Defaults:
```yaml
curator:
enabled: true
interval_hours: 168 # 7 days
min_idle_hours: 2
stale_after_days: 30
archive_after_days: 90
consolidate: false # LLM umbrella-building pass — opt-in (prune-only by default)
prune_builtins: true # archive unused bundled built-in skills too (hub skills always exempt)
```
To disable entirely, set `curator.enabled: false`. To keep the always-on pruning but opt into LLM consolidation, set `curator.consolidate: true`.
### Running the review on a cheaper aux model
The curator's LLM review pass is a regular auxiliary task slot — `auxiliary.curator` — alongside Vision, Compression, Session Search, etc. "Auto" means "use my main chat model"; override the slot to pin a specific provider + model for the review pass instead.
**Easiest — `hermes model`:**
```bash
hermes model # → "Auxiliary models — side-task routing"
# → pick "Curator" → pick provider → pick model
```
The same picker is available in the web dashboard under the **Models** tab.
**Direct config.yaml (equivalent):**
```yaml
auxiliary:
curator:
provider: openrouter
model: google/gemini-3-flash-preview
timeout: 600 # generous — reviews can take several minutes
```
Leaving `provider: auto` (the default) routes the review pass through whatever your main chat model is, matching the behavior of every other auxiliary task.
:::note Legacy config
Earlier releases used a one-off `curator.auxiliary.{provider,model}` block. That path still works but emits a deprecation log line — please migrate to `auxiliary.curator` above so the curator shares the same plumbing (`hermes model`, dashboard Models tab, `base_url`, `api_key`, `timeout`, `extra_body`) as every other aux task.
:::
## CLI
```bash
hermes curator status # last run, counts, pinned list, LRU top 5
hermes curator run # trigger a run now (blocks until done). Prune-only unless curator.consolidate: true
hermes curator run --consolidate # force the LLM consolidation pass on for this run, overriding the config default
hermes curator run --background # fire-and-forget: start the run in a background thread
hermes curator run --dry-run # preview only — report without any mutations
hermes curator backup # take a manual snapshot of ~/.hermes/skills/
hermes curator rollback # restore from the newest snapshot
hermes curator rollback --list # list available snapshots
hermes curator rollback --id <ts> # restore a specific snapshot
hermes curator rollback -y # skip the confirmation prompt
hermes curator pause # stop runs until resumed
hermes curator resume
hermes curator pin <skill> # never auto-transition this skill
hermes curator unpin <skill>
hermes curator adopt <skill> # hand an unmanaged skill to the curator
hermes curator adopt --all-unmanaged # hand over every unmanaged skill
hermes curator list-unmanaged # itemize skills with no provenance marker
hermes curator restore <skill> # move an archived skill back to active
hermes curator list-archived # list skills currently in ~/.hermes/skills/.archive/
hermes curator archive <skill> # manually archive a single skill now
hermes curator prune [--days N] # bulk-archive agent-created skills idle >= N days (default 90)
hermes curator ledger # list the per-mutation audit ledger (all actors)
hermes curator ledger --skill <name> --limit 50 # filter/paginate ledger entries
hermes curator rollback <entry-id> # undo a single mutation from the ledger
hermes curator purge [--days N] [--dry-run] # delete archived skills older than the TTL (explicit only)
```
## Backups and rollback
Before every real curator pass, Hermes takes a tar.gz snapshot of `~/.hermes/skills/` at `~/.hermes/skills/.curator_backups/<utc-iso>/skills.tar.gz`. If a pass archives or consolidates something you didn't want touched, you can undo the whole run with one command:
```bash
hermes curator rollback # restore newest snapshot (with confirmation)
hermes curator rollback -y # skip the prompt
hermes curator rollback --list # see all snapshots with reason + size
```
The rollback itself is reversible: before replacing the skills tree, Hermes takes another snapshot tagged `pre-rollback to <target-id>`, so a mistaken rollback can be undone by rolling forward to that one with `--id`.
You can also take manual snapshots at any time with `hermes curator backup --reason "before-refactor"`. The `--reason` string lands in the snapshot's `manifest.json` and is shown in `--list`.
Snapshots are pruned to `curator.backup.keep` (default 5) to keep disk usage bounded:
```yaml
curator:
backup:
enabled: true
keep: 5
```
Set `curator.backup.enabled: false` to disable automatic snapshotting. The manual `hermes curator backup` command still works when backups are disabled only if you set `enabled: true` first — the flag gates both paths symmetrically so there's no way to accidentally skip the pre-run snapshot on mutating runs.
`hermes curator status` also lists the five least-recently-used skills — a quick way to see what's likely to become stale next.
The same subcommands are available as the `/curator` slash command inside a running session (CLI or gateway platforms).
## Audit ledger and single-edit rollback
Whole-run snapshots answer "undo everything the last curator pass did" — but sometimes you want to know *who changed what* and undo exactly one mutation. Every skill mutation — curator auto-transitions, agent `skill_manage` calls, and your own CLI archive/restore/purge — appends one entry to the append-only JSONL ledger at `~/.hermes/skills/.curator_ledger.jsonl`:
- **actor** — `curator` (background review fork / auto-transitions), `agent` (foreground agent tool calls), or `user` (CLI commands)
- **action** — `create`, `edit`, `patch`, `delete`, `write_file`, `remove_file`, `archive`, `restore`, `purge`, `rollback`
- **evidence** — delete intent (`absorbed_into` for consolidations, empty for prunes, and whether the recoverable-archive path handled it), triggering session id when available
- **before/after** — per-file `{path, sha256}` manifests. File contents are stored content-addressed (deduped by hash) under `~/.hermes/.curator_backups/blobs/`, so a hundred entries touching the same unchanged file cost one blob.
```bash
hermes curator ledger # newest 20 entries
hermes curator ledger --skill my-skill --limit 50
hermes curator rollback <entry-id> # restore that one mutation's before-state
```
Single-entry rollback restores exactly the files that mutation touched (and removes files it created) from the blob store — nothing else in the skills tree moves. Like whole-tree rollback, it takes a safety ledger entry of the current state first and **fails closed**: if the safety capture can't be written, nothing is changed. Because foreground deletes are ledgered too, `hermes curator rollback <entry-id>` can resurrect a hard-deleted skill.
The ledger is telemetry, never a gate — if writing an entry fails, the mutation still goes through. Disable it with:
```yaml
skills:
ledger: false
```
## Archive TTL purge
Archived skills are kept forever by default. If you want `~/.hermes/skills/.archive/` bounded, set a TTL and purge explicitly — purging never runs automatically, and every purged skill is captured into the ledger (with blobs) first, so even a purge leaves an auditable, recoverable trail:
```yaml
curator:
archive_ttl_days: 180 # 0 (default) = never purge
```
```bash
hermes curator purge --dry-run # preview what would be deleted
hermes curator purge # delete archives older than the TTL (with confirmation)
hermes curator purge --days 90 # one-off TTL override
```
## What "agent-created" means
The curator only manages skills explicitly marked as **agent-created** in
`~/.hermes/skills/.usage.json`. A skill qualifies when ALL of the following
are true:
1. Its name is **not** in `~/.hermes/skills/.bundled_manifest` (bundled skills shipped with the repo).
2. Its name is **not** in `~/.hermes/skills/.hub/lock.json` (hub-installed skills).
3. Its `.usage.json` entry has `"created_by": "agent"` or `"agent_created": true`.
Currently, only the **background self-improvement review fork** sets this marker
— when it creates a new umbrella skill during its periodic review pass (~every 10
agent turns). The background fork runs with a write origin of `"background_review"`
(via `tools/skill_provenance.py`), which is the only path that triggers the
`mark_agent_created()` call in `skill_manage`.
Skills the foreground agent creates via `skill_manage(action="create")` during a
conversation are **not** marked as agent-created — they are considered
user-directed and the curator intentionally leaves them alone.
:::warning Your hand-written skills are NOT curated
If you manually created a `SKILL.md` or pointed Hermes at an external skill
directory, that skill will have a `.usage.json` entry with `created_by: null`
(or the field absent). The curator will not touch it. The same applies to
skills the foreground agent created at your request.
**To see which skills the curator actually manages**, run `hermes curator status`.
If the agent-created count is 0, no skills are currently in the curator's
jurisdiction — the LLM review pass is skipped and the report will show
`Model: (not resolved) via (not resolved)` with `Duration: 0s`.
:::
### Adopting unmanaged skills
`hermes curator status` reports an **unmanaged** count alongside the managed
one:
```
curator-managed skills: 43 total (agent-created=43 bundled=0)
active 41
stale 2
archived 0
unmanaged (no provenance marker): 112 total
pre-dates marker 34
foreground-created 78
never auto-staled or archived — `hermes curator adopt <name>` hands one over
```
Those 112 are curation-*eligible* but permanently invisible to the lifecycle,
for one of two reasons:
- **pre-dates marker** — the record was written before `created_by` existed, so
it carries no provenance signal at all. Authorship is genuinely unknowable
from the record.
- **foreground-created** — a foreground `skill_manage(create)` left the marker
unset by design, since skills you ask for belong to you.
A large library can therefore look fully curated while most of it is
untouchable. `adopt` closes that gap by **declaration**:
```bash
hermes curator list-unmanaged # itemize them, with reasons
hermes curator adopt <name> [<name> ...] # hand specific skills over
hermes curator adopt --all-unmanaged --dry-run # preview the full list
hermes curator adopt --all-unmanaged # hand over everything (prompts)
hermes curator adopt --all-unmanaged --yes # skip the prompt
```
Adoption writes the same `created_by: agent` marker the background review fork
writes. It does **not** reset the inactivity clock — an adopted skill keeps its
existing `last_activity_at`, so handing over a library you already stopped
using does not buy it a fresh 90-day window. Expect adopted long-idle skills to
go `stale` (or `archived`) on the next pass; that is the point.
Adoption is also what unblocks autonomous *improvement*. The background review
fork refuses to patch a skill that isn't curator-managed, so if it notices one
of your skills is outdated it will say so and recommend adoption rather than
edit it. Foreground (user-directed) edits are never affected — you and the
agent can always edit your own skills on request.
:::note `created_by` is a policy flag, not a provenance claim
The stored field is named `created_by`, but it is consumed as "may autonomous
curation touch this?" — not "who wrote this file". Those are different
questions, and for records predating the marker the authorship answer is simply
unrecoverable. The name is kept because it is already on disk in every
`.usage.json`; read it as policy. `hermes curator adopt` changes the policy, and
says nothing about who authored the file.
:::
:::note Provenance is declared, never inferred
Adoption is deliberately manual. Telemetry cannot establish authorship: a skill
with thousands of patches proves the agent **maintains** it, not that the agent
**wrote** it — Hermes edits user-authored skills on your behalf constantly. An
automatic "looks agent-made, adopt it" heuristic would eventually archive
something you hand-wrote. `adopt` refuses bundled, hub-installed, external, and
protected built-in skills, which have an owner other than you.
:::
Skills that ARE agent-created follow the full lifecycle:
- `active` → (30d unused) `stale` → (90d unused) `archived`
- Pinned skills bypass all auto-transitions
- Archives are recoverable via `hermes curator restore <name>`
If you want to protect a specific skill from ever being touched — for example a
hand-authored skill you rely on — use `hermes curator pin <name>`. See the next
section.
## Pinning a skill
Pinning protects a skill from deletion — both the curator's automated archive passes and the agent's `skill_manage(action="delete")` tool call. Once a skill is pinned:
- The **curator** skips it during auto-transitions (`active → stale → archived`), and its LLM review pass is instructed to leave it alone.
- The **agent's `skill_manage` tool** refuses `delete` on it, pointing the user at `hermes curator unpin <name>`. Patches and edits still go through, so the agent can improve a pinned skill's content as pitfalls come up without a pin/unpin/re-pin dance.
Pin and unpin with:
```bash
hermes curator pin <skill>
hermes curator unpin <skill>
```
The flag is stored as `"pinned": true` on the skill's entry in `~/.hermes/skills/.usage.json`, so it survives across sessions.
Skills named in any cron job's `skills:` list are protected the same way for **auto-transitions** (the curator never stales/archives them while the reference remains), even when the job is paused or disabled. Prefer an explicit pin when you also want `skill_manage delete` blocked.
Only **agent-created** skills can be pinned — `hermes curator pin` refuses on bundled and hub-installed skills with an explanatory message if you try. Hub-installed skills are never subject to curator mutation. Bundled built-in skills are only touched when `curator.prune_builtins: true` (the default), and even then only archived after `archive_after_days` of non-use — never patched, consolidated, or deleted. Set `curator.prune_builtins: false` to exempt bundled skills entirely.
A small set of **protected built-ins** can be hardcoded as never-archivable and never-consolidatable, regardless of `curator.prune_builtins`, pin state, or LLM judgment. These back load-bearing UX, so silently archiving one would turn its slash command into an "Unknown command" error with no signal to you. (The set is currently empty — `plan`, its original member, graduated to a built-in `/plan` command with no skill on disk.) Protected built-ins are filtered out of the curator's candidate list entirely, so the consolidation pass never sees them.
If you want a stronger guarantee than "no deletion" — for instance, freezing a skill's content entirely while the agent still reads it — edit `~/.hermes/skills/<name>/SKILL.md` directly with your editor. The pin guards tool-driven deletion, not your own filesystem access.
## Usage telemetry
The curator maintains a sidecar at `~/.hermes/skills/.usage.json` with one entry per skill:
```json
{
"my-skill": {
"use_count": 12,
"view_count": 34,
"last_used_at": "2026-04-24T18:12:03Z",
"last_viewed_at": "2026-04-23T09:44:17Z",
"patch_count": 3,
"last_patched_at": "2026-04-20T22:01:55Z",
"created_at": "2026-03-01T14:20:00Z",
"state": "active",
"pinned": false,
"archived_at": null
}
}
```
Counters increment when:
- `view_count`: the agent calls `skill_view` on the skill.
- `use_count`: the skill is loaded into a conversation's prompt.
- `patch_count`: `skill_manage patch/edit/write_file/remove_file` runs on the skill.
Bundled and hub-installed skills are explicitly excluded from telemetry writes.
## Per-run reports
Every curator run writes a timestamped directory under `~/.hermes/logs/curator/`:
```
~/.hermes/logs/curator/
└── 20260429-111512/
├── run.json # machine-readable: full fidelity, stats, LLM output
└── REPORT.md # human-readable summary
```
`REPORT.md` is a quick way to see what a given run did — which skills transitioned, what the LLM reviewer said, which skills it patched. Good for auditing without having to grep `agent.log`.
:::note No candidates? Report shows `(not resolved)`
When the curator has **no agent-created skills** to review, the LLM review pass
is skipped entirely. The report header will show
`Model: (not resolved) via (not resolved)` with `Duration: 0s` — this does **not**
indicate a configuration error or model resolution failure. It simply means there
were no candidates, so no model was ever invoked. The auto-transition phase still
runs and reports its counts normally.
:::
### Rename map in the summary
If a run consolidated multiple skills under an umbrella (or merged near-duplicates), the user-visible summary printed at the end of the run includes an explicit rename map showing every `old-name → new-name` pair the curator applied. This is in addition to per-skill transition lines, so when a wave of renames lands you can spot them at a glance without diffing the JSON report. The hint also surfaces under `hermes curator pin` so you can pin the umbrella name immediately if you want to lock the new label in.
## Restoring an archived skill
If the curator archived something you still want:
```bash
hermes curator restore <skill-name>
```
This moves the skill back from `~/.hermes/skills/.archive/` to the active tree and resets its state to `active`. The restore refuses if a bundled or hub-installed skill has since been installed under the same name (would shadow upstream).
## Disabling per environment
The curator is on by default. To turn it off:
- **For one profile only:** edit `~/.hermes/config.yaml` (or the active profile's config) and set `curator.enabled: false`.
- **For just one run:** `hermes curator pause` — the pause persists across sessions; use `resume` to re-enable.
The curator also refuses to run if `min_idle_hours` hasn't elapsed, so on an active dev machine it naturally only runs during quiet stretches.
## See also
- [Skills System](/user-guide/features/skills) — how skills work in general and the self-improvement loop that creates them
- [Memory](/user-guide/features/memory) — a parallel background review that maintains long-term memory
- [Bundled Skills Catalog](/reference/skills-catalog)
- [Issue #7816](https://github.com/NousResearch/hermes-agent/issues/7816) — original proposal and design discussion
website/docs/user-guide/features/delegation.md
+544
View File
@@ -0,0 +1,544 @@
---
sidebar_position: 7
title: "Subagent Delegation"
description: "Spawn isolated child agents for parallel workstreams with delegate_task"
---
# Subagent Delegation
The `delegate_task` tool spawns child AIAgent instances with isolated context, inherited tool access, and their own terminal sessions. Each child gets a fresh conversation and works independently — only its final summary enters the parent's context.
Top-level model calls run in the background automatically. Hermes returns a handle immediately so the conversation can continue, then posts the result back as a new message. An orchestrator subagent waits for its own workers so it can synthesize their results before returning.
## Single Task
```python
delegate_task(
goal="Debug why tests fail",
context="Error: assertion in test_foo.py line 42"
)
```
## Parallel Batch
Up to 3 concurrent subagents by default (configurable, no hard ceiling):
```python
delegate_task(tasks=[
{"goal": "Research topic A", "context": "Focus on recent primary sources"},
{"goal": "Research topic B", "context": "Compare the leading explanations"},
{"goal": "Fix the build", "context": "Project root: /home/user/project"}
])
```
## How Subagent Context Works
:::warning Critical: Subagents Know Nothing
Subagents start with a **completely fresh conversation**. They have zero knowledge of the parent's conversation history, prior tool calls, or anything discussed before delegation. The subagent's only context comes from the `goal` and `context` fields the parent agent populates when it calls `delegate_task`.
:::
One exception: when the parent has a resolved workspace directory, every subagent's system prompt embeds that workspace's **project context files** (`.hermes.md` > AGENTS.md chain > CLAUDE.md > `.cursorrules` — the same discovery, priority, and size caps as the main agent's system prompt; SOUL.md is excluded). Subagents working in a repo operate under the repo's own conventions without having to rediscover them.
This means the parent agent must pass **everything** the subagent needs in the call:
```python
# BAD - subagent has no idea what "the error" is
delegate_task(goal="Fix the error")
# GOOD - subagent has all context it needs
delegate_task(
goal="Fix the TypeError in api/handlers.py",
context="""The file api/handlers.py has a TypeError on line 47:
'NoneType' object has no attribute 'get'.
The function process_request() receives a dict from parse_body(),
but parse_body() returns None when Content-Type is missing.
The project is at /home/user/myproject and uses Python 3.11."""
)
```
The subagent receives a focused system prompt built from your goal and context, instructing it to complete the task and provide a structured summary of what it did, what it found, any files modified, and any issues encountered.
## Practical Examples
### Parallel Research
Research multiple topics simultaneously and collect summaries:
```python
delegate_task(tasks=[
{
"goal": "Research the current state of WebAssembly in 2025",
"context": "Focus on: browser support, non-browser runtimes, language support"
},
{
"goal": "Research the current state of RISC-V adoption in 2025",
"context": "Focus on: server chips, embedded systems, software ecosystem"
},
{
"goal": "Research quantum computing progress in 2025",
"context": "Focus on: error correction breakthroughs, practical applications, key players"
}
])
```
### Code Review + Fix
Delegate a review-and-fix workflow to a fresh context:
```python
delegate_task(
goal="Review the authentication module for security issues and fix any found",
context="""Project at /home/user/webapp.
Auth module files: src/auth/login.py, src/auth/jwt.py, src/auth/middleware.py.
The project uses Flask, PyJWT, and bcrypt.
Focus on: SQL injection, JWT validation, password handling, session management.
Fix any issues found and run the test suite (pytest tests/auth/)."""
)
```
### Multi-File Refactoring
Delegate a large refactoring task that would flood the parent's context:
```python
delegate_task(
goal="Refactor all Python files in src/ to replace print() with proper logging",
context="""Project at /home/user/myproject.
Use the 'logging' module with logger = logging.getLogger(__name__).
Replace print() calls with appropriate log levels:
- print(f"Error: ...") -> logger.error(...)
- print(f"Warning: ...") -> logger.warning(...)
- print(f"Debug: ...") -> logger.debug(...)
- Other prints -> logger.info(...)
Don't change print() in test files or CLI output.
Run pytest after to verify nothing broke."""
)
```
## Batch Mode Details
When a top-level agent provides a `tasks` array, Hermes returns one background handle, runs the subagents in parallel, and posts one consolidated result after every child finishes. An orchestrator subagent waits for its batch in the current turn so it can synthesize the results.
- **Maximum concurrency:** 3 tasks by default (configurable via `delegation.max_concurrent_children` or the `DELEGATION_MAX_CONCURRENT_CHILDREN` env var; floor of 1, no hard ceiling). Batches larger than the limit return a tool error rather than being silently truncated.
- **Thread pool:** Uses `ThreadPoolExecutor` with the configured concurrency limit as max workers
- **Progress display:** In CLI mode, a tree-view shows tool calls from each subagent in real-time with per-task completion lines. In gateway mode, progress is batched and relayed to the parent's progress callback
- **Result ordering:** Results are sorted by task index to match input order regardless of completion order
- **Cancellation:** Follow-up messages do not cancel a top-level background batch. `/stop` or closing/resetting the owning session cancels its active children. Synchronous orchestrator children still follow their parent's interrupt state
Synchronous single-task delegation from an orchestrator runs directly without thread pool overhead.
### Durable background completions
When a background delegation finishes, Hermes stores its completion event in
the active profile's `state.db` before publishing it to the normal fresh-turn
queue. If Hermes restarts after completion but before delivery, the pending
event is restored and routed through the same ownership checks. Competing
consumers use a durable claim, so only the consumer that successfully accepts
the synthetic turn acknowledges delivery; failed attempts release the claim for
retry.
This does not resume child execution after a crash. A delegation whose owner
process disappears while it is still running is recorded as `unknown`, because
Hermes cannot prove whether its external side effects happened. Pending and
delivered records are bounded and profile-local.
### Child background-process notifications
Background processes a subagent starts (e.g. `npm ci` with
`notify_on_complete`) technically route their completion and watch-pattern
notifications to the **parent** conversation, because anything that outlives
the child needs a durable consumer. By default those notifications are
**suppressed** in the parent chat — the child's consolidated delegation result
is the deliverable, and mid-conversation "process finished" walls from a
child's internal builds are noise. Suppressed events are logged at debug level
with the process session ID and subagent task ID, so they remain diagnosable.
The delegation result itself is never suppressed. To restore delivery of the
child process notifications (each carries a "Started by subagent …"
attribution line):
```yaml
delegation:
surface_child_process_notifications: true # default: false
```
## Model Override
You can configure a different model for subagents via `config.yaml` — useful for delegating simple tasks to cheaper/faster models:
```yaml
# In ~/.hermes/config.yaml
delegation:
model: "google/gemini-flash-2.0" # Cheaper model for subagents
provider: "openrouter" # Optional: route subagents to a different provider
```
If omitted, subagents use the same model as the parent.
### Cost strategy: frontier planner, inexpensive workers
Decomposing a problem into well-specified subtasks takes frontier-level judgment; executing a subtask that already comes with a clear goal, full context, and an output contract usually doesn't. Meanwhile the children are where the tokens go — a parallel batch of subagents typically burns the large majority of a run's total tokens, so the worker model is where the cost actually lives. Pinning `delegation.model` to an inexpensive model while your main session stays on a frontier model keeps the planning quality where it matters and cuts spend where the volume is:
```yaml
# ~/.hermes/config.yaml
model:
default: "your-frontier-model" # parent (planner) stays on the frontier model
delegation:
model: "your-inexpensive-model" # all delegate_task children run on this
provider: "openrouter" # optional: route children to a different provider
```
Resolution order: `delegation.base_url` (direct endpoint) takes precedence, then `delegation.provider` (full credential bundle resolved via the runtime provider system), and when neither is set children inherit the parent's provider and credentials; `delegation.model` applies in all cases, and when it is empty children inherit the parent's model. Setting `delegation.provider` alongside `delegation.base_url` keeps the explicit endpoint but carries that provider's request overrides and max output tokens into the child. An explicit `delegation.request_overrides` dict is honored on every branch and merges over those runtime-derived values (see [Configuration](#configuration) below).
Note that the pin is global: `delegate_task` has no per-task model parameter, so every child in a batch runs on the configured delegation model. For quality-sensitive subtasks that need a stronger model, either leave `delegation.model` unset for that session or hand the task to the [kanban board](kanban.md#per-task-model-override), which does support a per-task model override.
## The `/review` Command
`/review` spawns an independent, full-privilege background subagent whose only job is to review the work your conversation just produced — a PR, a diff, code, documentation, a design. It works on every surface: CLI, TUI, the Desktop app, and every gateway messaging platform.
```
/review # review whatever the last 10 messages presented
/review focus on security # add extra instructions for the reviewer
```
What happens:
1. The last 10 user/assistant messages are snapshotted as the reviewer's starting evidence (tool output and system messages are excluded).
2. A reviewer subagent is dispatched on the same background delegation rail as `delegate_task` — it gets the full normal subagent toolset (terminal, web, files, browser...), so it actually opens the PR, reads the diff, and runs code rather than judging from the excerpt.
3. The reviewer inherits the primary agent's working context: any skills the primary agent had loaded (launch-preloaded or via `skill_view` during the session) are named in its briefing with an instruction to load them and judge the work against their conventions. Like every subagent, its system prompt also embeds the workspace's project context files (AGENTS.md / CLAUDE.md / .cursorrules) as binding conventions.
4. When it finishes, its full review re-enters the same session as a normal background-subagent completion — your primary agent sees it and can act on it (fix the findings, push follow-ups, reply to you).
The canonical flow: your main agent opens a PR, you type `/review`, and a second pair of eyes investigates it while you keep working; the review lands back in the chat addressed to the agent that created the PR.
### Review model
By default the reviewer runs on your main model. To pin a dedicated review model, set `auxiliary.review` in `config.yaml`:
```yaml
auxiliary:
review:
provider: openrouter # or nous, anthropic, a direct base_url, ...
model: anthropic/claude-opus-4.6 # a strong reviewer model
```
Credentials resolve exactly like a `delegation.provider` pin (full runtime-provider bundle: base_url, api key, api_mode). `provider: auto` with an empty `model` means "inherit the main agent's model" — the default.
`/review` is deliberately separate from `/refine`: `/refine` reviews the conversation to update memory and skills, `/review` reviews the *work product* the conversation created.
## Inherited Tool Access
`delegate_task` does not accept a model-facing `toolsets` parameter. Each subagent inherits the parent's enabled toolsets so the model cannot grant a child capabilities that the parent does not have. Configure the parent's tools before starting the conversation if delegated work needs additional capabilities.
Certain tools are blocked for subagents even when the parent has them:
- `delegate_task` — blocked for leaf subagents (the default). Retained for `role="orchestrator"` children, bounded by `max_spawn_depth` — see [Depth Limit and Nested Orchestration](#depth-limit-and-nested-orchestration) below.
- `clarify` — subagents cannot interact with the user
- `memory` — no writes to shared persistent memory
- `send_message` — no cross-platform side effects
- `cronjob` — no scheduling more work in the parent's name
Both roles retain `execute_code` (programmatic tool calling) so children can batch mechanical work.
## Max Iterations
Each subagent has an iteration limit (default: 50) that controls how many tool-calling turns it can take:
```python
delegate_task(
goal="Quick file check",
context="Check if /etc/nginx/nginx.conf exists and print its first 10 lines",
max_iterations=10 # Simple task, don't need many turns
)
```
## Child Timeout
By default there is **no wall-clock timeout** on subagents. Children fail only from what they're actually doing — API errors, tool errors, or hitting their iteration budget — never from a delegation-level stopwatch. Earlier releases shipped a hard cap (300s, later 600s), which kept killing legitimately busy children mid-task: deep code reviews, large research fan-outs, and slow reasoning models routinely need more than 10 minutes while making steady progress the whole time.
Genuinely stuck children are still detected: the heartbeat staleness monitor stops refreshing the parent's activity when a child makes no progress (no API calls, no tool starts, and no activity-timestamp ticks), letting the gateway inactivity timeout fire on a truly wedged worker. An in-flight model wait still counts as progress — subagents refresh the activity clock while waiting on the provider, so a slow local / long-prefill completion is not treated as stalled.
If you want a hard cap anyway (e.g. cost control on unattended cron-driven delegation), opt in per-install:
```yaml
delegation:
child_timeout_seconds: 0 # default: 0 = no timeout
# child_timeout_seconds: 1800 # opt-in hard cap (floor 30s)
```
A positive value enforces a hard wall-clock limit on each child; `0` or a negative value disables it.
When a configured cap fires, the child's result carries structured timeout
metadata alongside the error message so parents and hooks can distinguish a
stopwatch kill from other failures without parsing text: `timeout_seconds`
(the configured cap), `timed_out_after_seconds` (actual wall clock), and
`timeout_phase` (`before_first_llm_call` when the child never reached its
first request, `after_llm_calls` otherwise). All three are `null` on
non-timeout errors.
## Failure Visibility
A subagent that fails — non-retryable provider error (404/400), timeout, crash, or no usable output — is never silent:
- **CLI**: the delegation tree prints a one-line reason: `⚠️ Subagent failed — "your goal": HTTP 404: model not found (after 12s)`. Batch runs append the reason to the per-task `✗` completion line.
- **Gateway platforms** (Telegram, Discord, Slack, ...): the same clean line is delivered as a standalone chat notice, **even when `tool_progress` is off** for that platform.
- **Parent agent**: the tool result entry carries `status: "failed"` plus the full `error` text, so the model can react (retry, re-route, report).
Error text is reduced to the single most informative line (the exception message, not a traceback wall) and capped in length.
:::tip Diagnostic dump on zero-call timeout
With a hard cap configured, if a subagent times out having made **zero** API calls (usually: provider unreachable, auth failure, or tool-schema rejection), `delegate_task` writes a structured diagnostic to `~/.hermes/logs/subagent-timeout-<session>-<timestamp>.log` containing the subagent's config snapshot, credential-resolution trace, any early error messages, and stack traces for **all** live threads (not just the child's own) — a child parked waiting on a nested helper thread is indistinguishable from a slow provider without the full picture.
:::
## Stall Detection for Background Subagents
Background delegations (`delegate_task(background=true)`) are watched by a
**progress-based stall monitor** — on by default, zero config. Unlike a
wall-clock timeout, it never touches a child that is making progress, no
matter how long it runs.
The monitor samples each detached child's progress signals — API-call count,
current tool, and last-activity timestamp (which ticks on **every streamed
token**, tool transition, and API-call boundary, so a child mid-stream on a
long response always counts as alive):
1. **Progressing children are never touched.** Any advancing signal resets
the clock.
2. A child whose progress is completely frozen past the stale threshold
(450s idle, 1200s while inside a tool — legitimately slow terminal
commands and web fetches get the higher ceiling) is **interrupted** and
given a 120s grace window. A child that unwinds in time delivers its
partial results through the normal completion path.
3. A child that never returns is force-finalized with a terminal `stalled`
completion event, so the owning session hears an outcome instead of
going silent, and the async slot frees for new work.
The `stalled` event carries structured metadata mirroring the sync-path
timeout fields: `stalled_after_quiet_seconds`, `stall_threshold_seconds`,
`stall_phase` (`idle` / `in_tool`), and `stall_grace_seconds`.
This closed a long-standing failure mode where a wedged background child
left its session looking dead until a process restart. The underlying wedge
(children hanging at their first API call after multi-day gateway uptime)
was also fixed at the root: delegated children now run their OpenAI-wire
API requests inline on their own conversation thread instead of a nested
worker thread — the layer where the wedge lived. The stall monitor remains
as the safety net for anything else.
## Monitoring Running Subagents (`/agents`)
The TUI ships a `/agents` overlay (alias `/tasks`) that turns recursive `delegate_task` fan-out into a first-class audit surface:
- Live tree view of running and recently-finished subagents, grouped by parent
- Per-branch cost, token, and file-touched rollups
- Kill and pause controls — cancel a specific subagent mid-flight without interrupting its siblings
- Post-hoc review: step through each subagent's turn-by-turn history even after they've returned to the parent
The classic CLI just prints `/agents` as a text summary; the TUI is where the overlay shines. See [TUI — Slash commands](/user-guide/tui#slash-commands).
On the classic CLI and every gateway platform (Telegram, Discord, Slack, ...),
`/agents` also lists **background delegations with live per-child activity**,
sampled directly from each running child:
```
Background delegations: 1 running
- deleg_ab12cd34 · running · research the delegation stall monitor
- child 1: 4 api calls · in web_search · active 12s ago
- child 2: 7 api calls · between turns · active 3s ago
```
A delegation the stall monitor has flagged shows as
`stalling · no progress 450s — interrupting`, and long-quiet-but-healthy
children show their quiet time so you can tell "slow" from "stuck" at a
glance.
## Steering a Running Subagent
Interrupting a child throws away its in-flight work; often you just want to redirect it.
### From the parent agent (model-facing)
The parent agent orchestrates its own running children with the same `delegate_task` tool it spawned them with — no separate control tool:
```json
{"action": "list"}
{"action": "steer", "subagent_id": "sa-0-1a2b3c4d", "message": "focus on pricing instead"}
{"action": "stop", "subagent_id": "sa-0-1a2b3c4d"}
```
- **`list`** returns the conversation's live children: `subagent_id`, goal, status, `running_seconds`, `accepting_steer`, and the live transcript path. Ids also come back in the spawn dispatch response as `subagent_ids`.
- **`steer`** queues a course correction into a running child without stopping it (delivery semantics below).
- **`stop`** ends a child early at its next iteration boundary; the partial result still re-enters the conversation as a normal completion message.
Control actions run synchronously in-turn (never backgrounded), are scoped to the caller's own spawn tree — a conversation can never see or control another session's children — and never consume the per-turn subagent spawn cap, so `stop` keeps working even after the cap is hit.
### From the TUI / gateway (session-facing)
`steer_subagent(subagent_id, text)` in `tools/delegate_tool.py` is the redirection-side mirror of `interrupt_subagent()`: it queues text into a live child through the same mechanism as [`/steer`](/reference/slash-commands) — the text is appended to the child's last tool result at its next iteration boundary, the in-flight tool call is never cut, and the child sees it as an out-of-band user message. Programmatic hosts reach it through the session-scoped `subagent.steer` gateway RPC, which sits beside `subagent.interrupt`:
```json
{"method": "subagent.steer", "params": {"session_id": "owning-ui-session", "subagent_id": "sa-0-1a2b3c4d", "text": "focus on pricing instead"}}
```
Subagent ids come from `delegation.status` (or `list_active_subagents()`) — the same place `subagent.interrupt` gets them. The gateway accepts steering only from the exact live UI/gateway session that spawned the child. A missing, foreign, ambiguous, or stale/recycled session identity is rejected; knowing a global subagent id is not authority. Direct in-process callers retain the unscoped helper contract deliberately.
**Queued is not delivered, but it is never synthetic success.** A `"queued"` response means the text was accepted before the child's completion boundary, not necessarily that the child has seen it. Acceptance and completion are synchronized: either the child can still consume the text, or its exact text is drained into the result as `pending_steer`. Calls after closure return `"rejected"`. If a child accepted the steer but had already produced its final answer, the completion entry the parent receives retains it as `missed_steer`, with a note appended to the summary:
```
[steer did not land — the subagent finished before it could be delivered: focus on pricing instead]
```
So the parent (or the operator driving it) can tell a steered child from one that finished on the old instructions, and re-issue the guidance as a follow-up instead of trusting that it landed.
## Live Transcripts
Every `delegate_task` dispatch also creates one **append-only, human-readable log per task** so you (or the parent agent) can watch a subagent work in real time instead of waiting for the consolidated summary:
```
<hermes_home>/cache/delegation/live/<delegation_id>/task-<n>.log
```
The dispatch response includes the paths as `live_transcripts`, and the files are pre-created at dispatch time, so this works immediately:
```bash
tail -f ~/.hermes/cache/delegation/live/deleg_ab12cd34/task-0.log
```
Each line is timestamped and shows the child's assistant text, thinking snippets, tool calls (`-> tool_name({args})`), tool results, and a final status marker. A `manifest.json` in the same directory describes the batch (goals, task count, per-task status). The logs persist after completion — they double as the full-fidelity operational record alongside the summary — and directories older than 7 days are pruned automatically on new dispatches. Because they live under `cache/delegation`, they are also readable from remote terminal backends (Docker/Modal/SSH).
## Depth Limit and Nested Orchestration
By default, delegation is **flat**: a parent (depth 0) spawns children (depth 1), and those children cannot delegate further. This prevents runaway recursive delegation.
For multi-stage workflows (research → synthesis, or parallel orchestration over sub-problems), a parent can spawn **orchestrator** children that *can* delegate their own workers:
```python
delegate_task(
goal="Survey three code review approaches and recommend one",
role="orchestrator", # Allows this child to spawn its own workers
context="...",
)
```
- `role="leaf"` (default): child cannot delegate further — identical to the flat-delegation behavior.
- `role="orchestrator"`: child retains the `delegation` toolset. Gated by `delegation.max_spawn_depth` (default **1** = flat, so `role="orchestrator"` is a no-op at defaults). Raise `max_spawn_depth` to 2 to allow orchestrator children to spawn leaf grandchildren; 3+ for deeper trees. There is no upper ceiling — cost is the practical limit.
- `delegation.orchestrator_enabled: false`: global kill switch that forces every child to `leaf` regardless of the `role` parameter.
**Cost warning:** With `max_spawn_depth: 3` and `max_concurrent_children: 3`, the tree can reach 3×3×3 = 27 concurrent leaf agents. Each extra level multiplies spend — raise `max_spawn_depth` intentionally.
## Lifetime and Durability
:::warning Background completion durability is not durable execution
Top-level model-facing `delegate_task` calls run in the background automatically where the session supports later delivery. Hermes returns a handle immediately, and the result re-enters the conversation after the child or batch finishes. Orchestrator subagents wait for their workers in the current turn because they must synthesize those results before returning. Stateless request/response endpoints fall back to synchronous execution when they cannot deliver a detached result later.
- Normal follow-up messages do not cancel background children. `/stop` cancels running background delegations, and closing or resetting the owning session discards its active children.
- Explicit session close/reset interrupts that session's background children. Closing a TUI viewer of a gateway-owned session does not kill the gateway's work.
- A Hermes process restart does **not** resume a running child. Its attempt becomes `unknown` because Hermes cannot prove which side effects happened.
- A child that completed before restart but whose result was not delivered is restored and routed back through the owning session's normal checks.
- Cancelled children return a structured result (`status="interrupted"`, `exit_reason="interrupted"`), but because the parent was interrupted too, that result often never makes it into a user-visible reply.
For **durable execution** that must survive session closure or process restart, use:
- `cronjob` (action=`create`) — schedules a separate agent run; immune to parent-turn interrupts.
- `terminal(background=True, notify_on_complete=True)` — long-running shell commands that keep running while the agent does other things.
:::
## Key Properties
- Each subagent gets its **own terminal session** (separate from the parent)
- Subagents inherit the parent's enabled toolsets; the model cannot select or widen them per call
- **Nested delegation is opt-in** — only `role="orchestrator"` children can delegate further, and only when `max_spawn_depth` is raised from its default of 1 (flat). Disable globally with `orchestrator_enabled: false`.
- Leaf subagents **cannot** call: `delegate_task`, `clarify`, `memory`, `send_message`, `cronjob`. Orchestrator subagents retain `delegate_task` but keep the other blocks. Both roles retain `execute_code` (programmatic tool calling) so children can batch mechanical work instead of burning reasoning iterations.
- **Cancellation follows ownership** — `/stop` or closing/resetting the owning session cancels its background children; synchronous descendants under orchestrators follow their parent's interrupt state
- Only the final summary enters the parent's context, keeping token usage efficient
- Subagents inherit the parent's **API key, provider configuration, and credential pool** (enabling key rotation on rate limits)
## Worktree Isolation
By default, subagents share the parent's working directory — fine for research
and read-heavy work, but parallel children editing the same repo can collide.
Set `delegation.worktree_isolation: true` to give each child its own git
worktree, branched from the repo's current `HEAD` (inspired by Muse Code's
`--subagent-worktree-isolation`):
```yaml
delegation:
worktree_isolation: true # default: false
```
With isolation on:
- Each child starts its terminal in `<repo>/.worktrees/subagent-<id>` on its
own branch `hermes-subagent/subagent-<id>`, and its goal message tells it to
work and commit there.
- The parent's checkout stays untouched; children can't clobber each other's
edits.
- When a child finishes, its result entry gains a `worktree` field reporting
`path`, `branch`, `commits` (ahead of the base), and `dirty`. The parent
reviews or merges each branch (`git log <branch>`, `git merge <branch>`).
- A worktree left with **no commits and a clean tree is pruned automatically**
(`pruned: true`); anything holding work is kept.
- Pruning requires proof. If a git inspection probe fails — or finalization
itself errors — the worktree and branch are kept and the entry carries
`inspection_failed: true` plus a `note` — `commits`/`dirty` are then
defaults, not measurements, so inspect the worktree rather than assuming
the child produced nothing.
Scope: opt-in, git-only, and local-terminal-backend-only. In a non-git
directory, on docker/ssh/modal backends, or if worktree creation fails, the
setting degrades silently to today's shared-workspace behavior — never an
error.
## Delegation vs execute_code
| Factor | delegate_task | execute_code |
|--------|--------------|-------------|
| **Reasoning** | Full LLM reasoning loop | Just Python code execution |
| **Context** | Fresh isolated conversation | No conversation, just script |
| **Tool access** | All non-blocked tools with reasoning | 7 tools via RPC, no reasoning |
| **Parallelism** | 3 concurrent subagents by default (configurable) | Single script |
| **Best for** | Complex tasks needing judgment | Mechanical multi-step pipelines |
| **Token cost** | Higher (full LLM loop) | Lower (only stdout returned) |
| **User interaction** | None (subagents can't clarify) | None |
**Rule of thumb:** Use `delegate_task` when the subtask requires reasoning, judgment, or multi-step problem solving. Use `execute_code` when you need mechanical data processing or scripted workflows.
## Configuration
```yaml
# In ~/.hermes/config.yaml
delegation:
max_iterations: 50 # Max turns per child (default: 50)
# max_concurrent_children: 3 # Parallel children per batch (default: 3)
# worktree_isolation: false # Give each child its own git worktree (see Worktree Isolation above)
# max_spawn_depth: 1 # Tree depth (floor 1, no ceiling, default 1 = flat). Raise to 2 to allow orchestrator children to spawn leaves; 3+ for deeper trees.
# orchestrator_enabled: true # Disable to force all children to leaf role.
model: "google/gemini-3-flash-preview" # Optional provider/model override
provider: "openrouter" # Optional built-in provider
api_mode: anthropic_messages # optional; auto-detected from base_url for anthropic_messages endpoints
# Or use a direct custom endpoint instead of provider:
delegation:
model: "qwen2.5-coder"
base_url: "http://localhost:1234/v1"
api_key: "local-key"
# api_mode: "anthropic_messages" # Optional. Wire protocol override for base_url ("chat_completions", "codex_responses", or "anthropic_messages"). Empty = auto-detect from URL (e.g. /anthropic suffix). Set explicitly for endpoints the heuristic can't classify (Azure AI Foundry, MiniMax, Zhipu GLM, LiteLLM proxies, …).
# Send per-child request settings on every subagent API call — e.g. OpenRouter
# routing hints when delegating straight to openrouter.ai via base_url:
delegation:
model: "deepseek/deepseek-v4-flash-0731"
base_url: "https://openrouter.ai/api/v1"
api_key: "sk-or-..."
request_overrides:
extra_body:
provider:
sort: throughput # children route to the fastest OpenRouter provider
```
When `base_url` points at an Anthropic-compatible endpoint — for example a path ending in `/anthropic`, an Azure Foundry Claude route, or a MiniMax `/anthropic` proxy — `api_mode` is auto-detected as `anthropic_messages` so the subagent uses the right wire format without you setting anything. Set `api_mode` explicitly when the auto-detection guess is wrong (rare).
`delegation.request_overrides` works on **all three** resolution branches — direct `base_url`, named `provider`, and pure inherit — so it always takes effect. Top-level keys are API kwargs (e.g. `service_tier`); an `extra_body` sub-dict is merged into the request's `extra_body`. Explicit values merge **over** runtime- or parent-derived overrides: explicit top-level keys win, and `extra_body` is deep-merged one level, so a provider's own request personality (e.g. `thinking: {type: disabled}`) survives unless your key redefines it. See [Configuration → Delegation](../configuration.md#delegation) for details.
:::tip
The agent handles delegation automatically based on the task complexity. You don't need to explicitly ask it to delegate — it will do so when it makes sense.
:::
website/docs/user-guide/features/deliverable-mode.md
+134
View File
@@ -0,0 +1,134 @@
---
title: Deliverable Mode (Artifacts in Chat)
sidebar_label: Deliverable Mode
description: How the agent ships generated charts, PDFs, spreadsheets, and other files as native attachments in messaging platforms.
---
# Deliverable Mode
When Hermes Agent runs inside a messaging gateway (Slack, Discord, Telegram,
WhatsApp, Signal, etc.), it can deliver generated files directly into the
chat — not as paths the user has to copy, but as native attachments.
A chart shows up as an inline image. A PDF report shows up as a file
download. A spreadsheet uploads as `.xlsx`. The agent does not need to
write a `MEDIA:` tag or do anything special — it just generates the file
and mentions its absolute path in the response. The gateway picks the path
out of the text, removes it from the visible message, and uploads the
file natively.
## How it works
Three pieces fit together:
1. **The agent has tools that produce files.** `execute_code` for charts via
matplotlib, the `docx` skill for Word documents, the `xlsx` skill for
spreadsheets, the `pdf` and `latex-pdf-report` skills for PDFs, the
`powerpoint` skill for decks, `image_generate` for images,
`text_to_speech` for audio, and so on.
2. **The gateway scans agent responses for file paths.** Any absolute path
(`/tmp/...`) or home-relative path (`~/...`) ending in a supported
extension gets extracted. Paths inside code blocks and inline code are
ignored so code samples are never mutilated.
3. **The gateway dispatches by file type.** Images embed inline where the
platform supports it; videos embed inline; audio routes to voice/audio
attachments; everything else uploads as a file attachment.
## Supported file extensions
| Category | Extensions | Delivery |
|---|---|---|
| Images | `.png .jpg .jpeg .gif .webp .bmp .tiff .svg` | Inline embed |
| Video | `.mp4 .mov .avi .mkv .webm .3gp` | Inline embed (where supported) |
| Audio | `.mp3 .m2a .wav .ogg .opus .m4a .flac` | Voice / audio attachment |
| Documents | `.pdf .docx .doc .odt .rtf .txt .md .epub` | File upload |
| Data | `.xlsx .xls .ods .csv .tsv .json .xml .yaml .yml` | File upload |
| Geospatial | `.kmz .kml .geojson .gpx` | File upload |
| Presentations | `.pptx .ppt .odp .key` | File upload |
| Archives | `.zip .tar .gz .tgz .bz2 .xz .7z .rar .apk .ipa` | File upload |
| Web | `.html .htm` | File upload |
`.py`, `.log`, and other source-file extensions are intentionally excluded so
the agent doesn't auto-ship arbitrary source files; if you want to send code
to the user, use a code block.
## Encouraging the agent to produce artifacts
The agent doesn't reach for artifacts by default — it has to know to.
Two ways to nudge it:
**Per-session:** ask explicitly ("send me the comparison as a chart",
"return the data as a CSV") or write your own custom-instructions /
personality entry that biases toward artifact-style replies on
messaging platforms.
**Project-level:** add the bias to `AGENTS.md` / `CLAUDE.md` /
`.cursorrules` in a project the agent works from, to your global
persona in `~/.hermes/SOUL.md`, or as a named preset under
`agent.personalities` in `~/.hermes/config.yaml` (switchable per session
via `/personality`).
The mechanic the agent has to use is simple: render the file to an
absolute path (e.g. `/tmp/q3-revenue.png`) and mention that path as
plain text in the reply. The gateway does the rest. Paths inside
fenced code blocks or backticks are ignored so code samples are never
mutilated.
## Kanban: artifacts ride completion notifications
If you use Hermes' kanban multi-agent workflow, workers can attach
deliverable files to their `kanban_complete` call:
```python
kanban_complete(
summary="rendered Q3 revenue chart and report",
artifacts=[
"/tmp/q3-revenue.png",
"/tmp/q3-report.pdf",
],
)
```
When the gateway notifier delivers the "task completed" message to whoever
subscribed to the task in Slack/Telegram/etc., it also uploads each artifact
as a native attachment to that chat. The human gets the deliverable and the
summary in one place.
Files that don't exist on disk when the notifier runs are silently skipped.
## Connecting more services with MCP
Beyond the artifact-delivery pipeline, the agent can reach into other
services via MCP (Model Context Protocol). The MCP ecosystem ships
community servers for most popular tools — install whichever you need:
| Service | What it unlocks |
|---|---|
| **Notion** | Read/write Notion pages, databases, query workspace |
| **GitHub** | Issues, PRs, comments, repo search beyond the gh CLI |
| **Linear** | Tickets, projects, cycles |
| **Slack** | Workspace-wide search, read other channels |
| **Gmail** | Inbox triage, send mail, label management |
| **Salesforce** | Leads, opportunities, account data |
| **Snowflake / BigQuery** | SQL against data warehouses |
| **Google Drive** | File search, contents, share management |
Install MCP servers via `~/.hermes/config.yaml` under the `mcp_servers`
section. See [MCP integration](./mcp.md) for the full setup guide.
## Comparison to Perplexity Computer in Slack
Perplexity Computer's Slack integration is built around the same idea:
the agent generates a deliverable (chart, PDF, slide deck) and posts it
back into the thread as a native attachment. Hermes Agent's deliverable
mode provides the same user-facing pattern locally:
- Generation happens in the user's own venv / sandbox (no remote tenant).
- Files land in the chat via the same Slack `files.uploadV2` API.
- Connector breadth comes via MCP rather than a curated catalog of 400
hosted integrations — install the ones you actually use.
OAuth tokens stay on the user's machine in `auth.json` / `.env`. No hosted
token storage. No multi-tenant microVM. Same end result.
website/docs/user-guide/features/document-extraction.md
+58
View File
@@ -0,0 +1,58 @@
---
sidebar_position: 3
title: "Document Extraction"
description: "How read_file converts PDFs, Office documents, and notebooks to text — and what to do when a PDF is scanned images"
---
# Document Extraction
The `read_file` tool automatically converts common document formats to readable text, so the agent can inspect a PDF or spreadsheet the same way it reads source code.
## Supported formats
| Format | Extensions | Converter | Availability |
|--------|-----------|-----------|--------------|
| Jupyter notebooks | `.ipynb` | Built-in (stdlib) | Always |
| Word documents | `.docx` | Built-in (stdlib) | Always |
| Excel workbooks | `.xlsx` | Built-in (stdlib) | Always |
| PDF | `.pdf` | Optional `anydoc` converter | Auto-installed on first use* |
| Legacy Office | `.doc`, `.ppt`, `.xls`, `.pptx`, and variants | Optional `anydoc` converter | Auto-installed on first use* |
| OpenDocument | `.odt`, `.ods`, `.odp` | Optional `anydoc` converter | Auto-installed on first use* |
| Rich text / eBooks | `.rtf`, `.epub` | Optional `anydoc` converter | Auto-installed on first use* |
\* The optional converter is the `firecrawl-anydoc` package, installed lazily where installs are permitted (`security.allow_lazy_installs` in `config.yaml`). Without it, the three stdlib formats still work; other formats fall back to the binary-file guard.
Conversion output is Markdown, paginated through `read_file`'s normal `offset`/`limit` window. Documents over 50 MB are refused to keep tool turns bounded.
Extraction works with remote terminal backends (Docker, Modal, SSH): the file's bytes are transferred across the backend boundary and converted host-side, so a document inside a sandbox reads the same as a local one.
## Scanned PDFs: the coverage warning
PDF conversion reads the **text layer only**. Pages that are scanned images — common in legal documents, resale packages, signed contracts, faxes — contain no text layer and silently convert to nothing. The telltale signature is section headers with empty bodies.
When a meaningful share of pages yields no text (over 20% of the document, or 10+ pages absolute), `read_file` prepends a warning to the extraction. Each unreadable gap is labeled with the last text extracted before it — usually a section divider — so the agent can target only the gaps it actually needs instead of OCRing the whole document:
```
[EXTRACTION COVERAGE WARNING: 198 of 311 pages in this PDF yielded no
text. ... Unreadable gaps, each labeled with the last text extracted
before it:
pages 42-77 (36 pages) — after "Antigua Maintenance Corp Bylaws" (p41)
pages 92-213 (122 pages) — after "... Covenants, Codes and Regulations" (p91)
page 224 (1 page) — after "... Insurance Declaration Pages" (p223)
Decide which gaps you actually need — do NOT OCR or render everything. ...]
```
The warning lists the exact page ranges and the recovery paths:
1. **A few pages — render + vision.** Convert the pages to images and read them with the vision tool:
```bash
pdftoppm -jpeg -r 150 -f 92 -l 94 document.pdf /tmp/page
```
Then inspect each image with `vision_analyze`. Zero extra dependencies (poppler is required for the detection itself).
2. **Many pages — OCR.** The `ocr-and-documents` skill covers bulk OCR with marker-pdf (90+ languages, handles equations and tables; ~3-5 GB install).
Detection uses poppler's `pdftotext` for per-page text counts. If poppler is not installed, extraction still works — the coverage check is silently skipped.
:::tip
The agent handles the warning on its own — it will offer to render or OCR the missing pages. If you're reading extractions yourself, treat "header with an empty body" as a scanned section, not a missing one.
:::
website/docs/user-guide/features/extending-the-dashboard.md
+927
View File
@@ -0,0 +1,927 @@
---
sidebar_position: 17
title: "Extending the Dashboard"
description: "Build themes and plugins for the Hermes web dashboard — palettes, typography, layouts, custom tabs, shell slots, page-scoped slots, and backend API routes"
---
# Extending the Dashboard
The Hermes web dashboard (`hermes dashboard`) is built to be reskinned and extended without forking the codebase. Three layers are exposed:
1. **Themes** — YAML files that repaint the dashboard's palette, typography, layout, and per-component chrome. Drop a file in `~/.hermes/dashboard-themes/`; it appears in the theme switcher.
2. **UI plugins** — a directory with `manifest.json` + a JavaScript bundle that registers a tab, replaces a built-in page, augments one via page-scoped slots, or injects components into named shell slots.
3. **Backend plugins** — a Python file inside that plugin directory that exposes a FastAPI `router`; routes are mounted under `/api/plugins/<name>/` and called from the plugin's UI.
All three are **drop-in at runtime**: no repo clone, no `npm run build`, no patching the dashboard source. This page is the canonical reference for all three.
If you just want to use the dashboard, see [Web Dashboard](./web-dashboard). If you want to reskin the terminal CLI (not the web dashboard), see [Skins & Themes](./skins) — the CLI skin system is unrelated to dashboard themes.
:::note Not the desktop app
This page covers the **web dashboard** (`hermes dashboard`) plugin system — `window.__HERMES_PLUGIN_SDK__`, a `manifest.json`, and a pre-built JS bundle. The **native desktop app** (`hermes desktop`) has its own, unrelated SDK — `@hermes/plugin-sdk`, a single ESM file, no build step — documented at [Desktop Plugin SDK](/developer-guide/desktop-plugin-sdk). Only the backend `plugin_api.py` namespace (`/api/plugins/<name>`) is shared between them.
:::
:::note How the pieces compose
Themes and plugins are independent but synergistic. A theme can stand alone (just a YAML file). A plugin can stand alone (just a tab). Together they let you build a complete visual reskin with custom HUDs — the example `strike-freedom-cockpit` demo (lives in the `hermes-example-plugins` companion repo — see [Combined theme + plugin demo](#combined-theme--plugin-demo) for install steps) does exactly that.
:::
---
## Table of contents
- [Themes](#themes)
- [Quick start — your first theme](#quick-start--your-first-theme)
- [Palette, typography, layout](#palette-typography-layout)
- [Layout variants](#layout-variants)
- [Theme assets (images as CSS vars)](#theme-assets-images-as-css-vars)
- [Component chrome overrides](#component-chrome-overrides)
- [Color overrides](#color-overrides)
- [Raw `customCSS`](#raw-customcss)
- [Built-in themes](#built-in-themes)
- [Full theme YAML reference](#full-theme-yaml-reference)
- [Plugins](#plugins)
- [Quick start — your first plugin](#quick-start--your-first-plugin)
- [Directory layout](#directory-layout)
- [Manifest reference](#manifest-reference)
- [The Plugin SDK](#the-plugin-sdk)
- [Shell slots](#shell-slots)
- [Replacing built-in pages (`tab.override`)](#replacing-built-in-pages-taboverride)
- [Augmenting built-in pages (page-scoped slots)](#augmenting-built-in-pages-page-scoped-slots)
- [Slot-only plugins (`tab.hidden`)](#slot-only-plugins-tabhidden)
- [Backend API routes](#backend-api-routes)
- [Custom CSS per plugin](#custom-css-per-plugin)
- [Plugin discovery & reload](#plugin-discovery--reload)
- [Combined theme + plugin demo](#combined-theme--plugin-demo)
- [API reference](#api-reference)
- [Troubleshooting](#troubleshooting)
---
## Themes
Themes are YAML files stored in `~/.hermes/dashboard-themes/`. The file name doesn't matter (the theme's `name:` field is what the system uses), but convention is `<name>.yaml`. Every field is optional — missing keys fall back to the built-in `default` theme, so a theme can be as small as one color.
### Quick start — your first theme
```bash
mkdir -p ~/.hermes/dashboard-themes
```
```yaml
# ~/.hermes/dashboard-themes/neon.yaml
name: neon
label: Neon
description: Pure magenta on black
palette:
background: "#000000"
midground: "#ff00ff"
```
Refresh the dashboard. Click the palette icon in the header and pick **Neon**. The background goes black, text and accents go magenta, and every derived color (card, border, muted, ring, etc.) is recomputed from that 2-color triplet via `color-mix()` in CSS.
That's the whole onboarding: one file, two colors. Everything below is optional refinement.
### Palette, typography, layout
These three blocks are the heart of a theme. Each is independent — override one, leave the others.
#### Palette (3-layer)
The palette is a triplet of color layers plus a warm-glow vignette color and a noise-grain multiplier. The dashboard's design-system cascade derives every shadcn-compatible token (card, popover, muted, border, primary, destructive, ring, etc.) from this triplet via CSS `color-mix()`. Overriding three colors cascades into the whole UI.
| Key | Description |
|-----|-------------|
| `palette.background` | Deepest canvas color — typically near-black. Drives the page background and card fill. |
| `palette.midground` | Primary text and accent. Most UI chrome reads this (foreground text, button outlines, focus rings). |
| `palette.foreground` | Top-layer highlight. The default theme sets this to white at alpha 0 (invisible); themes that want a bright accent on top can raise its alpha. |
| `palette.warmGlow` | `rgba(...)` string used as the vignette color by `<Backdrop />`. |
| `palette.noiseOpacity` | 0–1.2 multiplier on the grain overlay. Lower = softer, higher = grittier. |
Each layer accepts either `{hex: "#RRGGBB", alpha: 0.0–1.0}` or a bare hex string (alpha defaults to 1.0).
```yaml
palette:
background:
hex: "#05091a"
alpha: 1.0
midground: "#d8f0ff" # bare hex, alpha = 1.0
foreground:
hex: "#ffffff"
alpha: 0 # invisible top layer
warmGlow: "rgba(255, 199, 55, 0.24)"
noiseOpacity: 0.7
```
#### Typography
| Key | Type | Description |
|-----|------|-------------|
| `fontSans` | string | CSS font-family stack for body copy (applied to `html`, `body`). |
| `fontMono` | string | CSS font-family stack for code blocks, `<code>`, `.font-mono` utilities. |
| `fontDisplay` | string | Optional heading/display stack. Falls back to `fontSans`. |
| `fontUrl` | string | Optional external stylesheet URL. Injected as `<link rel="stylesheet">` in `<head>` on theme switch. Same URL is never injected twice. Works with Google Fonts, Bunny Fonts, self-hosted `@font-face` sheets — anything linkable. |
| `baseSize` | string | Root font size — controls the rem scale. E.g. `"14px"`, `"16px"`. |
| `lineHeight` | string | Default line-height. E.g. `"1.5"`, `"1.65"`. |
| `letterSpacing` | string | Default letter-spacing. E.g. `"0"`, `"0.01em"`, `"-0.01em"`. |
```yaml
typography:
fontSans: '"Orbitron", "Eurostile", "Impact", sans-serif'
fontMono: '"Share Tech Mono", ui-monospace, monospace'
fontDisplay: '"Orbitron", "Eurostile", sans-serif'
fontUrl: "https://fonts.googleapis.com/css2?family=Orbitron:wght@400;500;600;700&family=Share+Tech+Mono&display=swap"
baseSize: "14px"
lineHeight: "1.5"
letterSpacing: "0.04em"
```
##### Changing the font from the UI (no YAML)
The theme picker in the dashboard header has a **Font** section below the
theme list. Pick any font there and it overrides the body font of whatever
theme is active — the choice is independent of the theme and persists across
theme switches (stored in `config.yaml` under `dashboard.font`). Choose
**Theme default** to clear the override and fall back to the active theme's
own `fontSans`.
The picker offers a curated catalog (system stacks plus a set of Google-Fonts
families across sans / serif / mono). It deliberately does **not** accept a
free-text font URL — the font's stylesheet is injected as a `<link>`, so the
catalog keeps the injected origins fixed. For a fully custom face, set
`fontSans` + `fontUrl` in a theme YAML as shown above. The theme's `fontMono`
(code blocks, terminal) is always left untouched by the UI override.
#### Layout
| Key | Values | Description |
|-----|--------|-------------|
| `radius` | any CSS length (`"0"`, `"0.25rem"`, `"0.5rem"`, `"1rem"`, ...) | Corner-radius token. Maps to `--radius` and cascades into `--radius-sm/md/lg/xl` — every rounded element shifts together. |
| `density` | `compact` \| `comfortable` \| `spacious` | Spacing multiplier applied as the `--spacing-mul` CSS var. `compact = 0.85×`, `comfortable = 1.0×` (default), `spacious = 1.2×`. Scales Tailwind's base spacing, so padding, gap, and space-between utilities all shift proportionally. |
```yaml
layout:
radius: "0"
density: compact
```
### Layout variants
`layoutVariant` picks the overall shell layout. Defaults to `"standard"` when absent.
| Variant | Behaviour |
|---------|-----------|
| `standard` | Single column, 1600px max-width (default). |
| `cockpit` | Left sidebar rail (260px) + main content. Populated by plugins via the `sidebar` slot — see [Shell slots](#shell-slots). Without a plugin the rail shows a placeholder. |
| `tiled` | Drops the max-width clamp so pages can use the full viewport width. |
```yaml
layoutVariant: cockpit
```
The current variant is exposed as `document.documentElement.dataset.layoutVariant`, so raw CSS in `customCSS` can target it via `:root[data-layout-variant="cockpit"] ...`.
### Theme assets (images as CSS vars)
Ship artwork URLs with a theme. Each named slot becomes a CSS var (`--theme-asset-<name>`) that the built-in shell and any plugin can read. The `bg` slot is automatically wired into the backdrop; other slots are plugin-facing.
```yaml
assets:
bg: "https://example.com/hero-bg.jpg" # auto-wired into <Backdrop />
hero: "/my-images/strike-freedom.png" # for plugin sidebars
crest: "/my-images/crest.svg" # for header-left plugins
logo: "/my-images/logo.png"
sidebar: "/my-images/rail.png"
header: "/my-images/header-art.png"
custom:
scanLines: "/my-images/scanlines.png" # → --theme-asset-custom-scanLines
```
Values accept:
- Bare URLs — wrapped in `url(...)` automatically.
- Pre-wrapped `url(...)`, `linear-gradient(...)`, `radial-gradient(...)` expressions — used as-is.
- `"none"` — explicit opt-out.
Every asset is also emitted as `--theme-asset-<name>-raw` (the unwrapped URL), in case a plugin needs to pass it to `<img src>` instead of `background-image`.
Plugins read these with plain CSS or JS:
```javascript
// In a plugin slot
const hero = getComputedStyle(document.documentElement)
.getPropertyValue("--theme-asset-hero").trim();
```
### Component chrome overrides
`componentStyles` restyles individual shell components without writing CSS selectors. Each bucket's entries become CSS vars (`--component-<bucket>-<kebab-property>`) that the shell's shared components read. So `card:` overrides apply to every `<Card>`, `header:` to the app bar, etc.
```yaml
componentStyles:
card:
clipPath: "polygon(12px 0, 100% 0, 100% calc(100% - 12px), calc(100% - 12px) 100%, 0 100%, 0 12px)"
background: "linear-gradient(180deg, rgba(10, 22, 52, 0.85), rgba(5, 9, 26, 0.92))"
boxShadow: "inset 0 0 0 1px rgba(64, 200, 255, 0.28)"
header:
background: "linear-gradient(180deg, rgba(16, 32, 72, 0.95), rgba(5, 9, 26, 0.9))"
tab:
clipPath: "polygon(6px 0, 100% 0, calc(100% - 6px) 100%, 0 100%)"
sidebar: {}
backdrop: {}
footer: {}
progress: {}
badge: {}
page: {}
```
Supported buckets: `card`, `header`, `footer`, `sidebar`, `tab`, `progress`, `badge`, `backdrop`, `page`.
Property names use camelCase (`clipPath`) and are emitted as kebab (`clip-path`). Values are plain CSS strings — anything CSS accepts (`clip-path`, `border-image`, `background`, `box-shadow`, `animation`, ...).
### Color overrides
Most themes won't need this — the 3-layer palette derives every shadcn token. Use `colorOverrides` when you want a specific accent the derivation won't produce (a softer destructive red for a pastel theme, a specific success green for a brand).
```yaml
colorOverrides:
primary: "#ffce3a"
primaryForeground: "#05091a"
accent: "#3fd3ff"
ring: "#3fd3ff"
destructive: "#ff3a5e"
border: "rgba(64, 200, 255, 0.28)"
```
Supported keys: `card`, `cardForeground`, `popover`, `popoverForeground`, `primary`, `primaryForeground`, `secondary`, `secondaryForeground`, `muted`, `mutedForeground`, `accent`, `accentForeground`, `destructive`, `destructiveForeground`, `success`, `warning`, `border`, `input`, `ring`.
Each key maps 1:1 to the `--color-<kebab>` CSS var (e.g. `primaryForeground` → `--color-primary-foreground`). Any key set here wins over the palette cascade for the active theme only — switching to another theme clears the overrides.
### Raw `customCSS`
For selector-level chrome that `componentStyles` can't express — pseudo-elements, animations, media queries, theme-scoped overrides — drop raw CSS into `customCSS`:
```yaml
customCSS: |
/* Scanline overlay — only visible when cockpit variant is active. */
:root[data-layout-variant="cockpit"] body::before {
content: "";
position: fixed;
inset: 0;
pointer-events: none;
z-index: 100;
background: repeating-linear-gradient(to bottom,
transparent 0px, transparent 2px,
rgba(64, 200, 255, 0.035) 3px, rgba(64, 200, 255, 0.035) 4px);
mix-blend-mode: screen;
}
```
The CSS is injected as a single scoped `<style data-hermes-theme-css>` tag on theme apply and cleaned up on theme switch. **Capped at 32 KiB per theme.**
### Built-in themes
Each built-in ships its own palette, typography, and layout — switching produces visible changes beyond color alone.
| Theme | Palette | Typography | Layout |
|-------|---------|------------|--------|
| **Hermes Teal** (`default`) | Dark teal + cream | System stack, 15px | 0.5rem radius, comfortable |
| **Hermes Teal (Large)** (`default-large`) | Same as default | System stack, 18px, line-height 1.65 | 0.5rem radius, spacious |
| **Midnight** (`midnight`) | Deep blue-violet | Inter + JetBrains Mono, 14px | 0.75rem radius, comfortable |
| **Ember** (`ember`) | Warm crimson + bronze | Spectral (serif) + IBM Plex Mono, 15px | 0.25rem radius, comfortable |
| **Mono** (`mono`) | Grayscale | IBM Plex Sans + IBM Plex Mono, 13px | 0 radius, compact |
| **Cyberpunk** (`cyberpunk`) | Neon green on black | Share Tech Mono everywhere, 14px | 0 radius, compact |
| **Rosé** (`rose`) | Pink + ivory | Fraunces (serif) + DM Mono, 16px | 1rem radius, spacious |
Themes that reference Google Fonts (all except Hermes Teal) load the stylesheet on demand — the first time you switch to them a `<link>` tag is injected into `<head>`.
### Full theme YAML reference
Every knob in one file — copy and trim what you don't need:
```yaml
# ~/.hermes/dashboard-themes/ocean.yaml
name: ocean
label: Ocean Deep
description: Deep sea blues with coral accents
# 3-layer palette (accepts {hex, alpha} or bare hex)
palette:
background:
hex: "#0a1628"
alpha: 1.0
midground:
hex: "#a8d0ff"
alpha: 1.0
foreground:
hex: "#ffffff"
alpha: 0.0
warmGlow: "rgba(255, 107, 107, 0.35)"
noiseOpacity: 0.7
typography:
fontSans: "Poppins, system-ui, sans-serif"
fontMono: "Fira Code, ui-monospace, monospace"
fontDisplay: "Poppins, system-ui, sans-serif" # optional
fontUrl: "https://fonts.googleapis.com/css2?family=Poppins:wght@400;500;600&family=Fira+Code:wght@400;500&display=swap"
baseSize: "15px"
lineHeight: "1.6"
letterSpacing: "-0.003em"
layout:
radius: "0.75rem"
density: comfortable
layoutVariant: standard # standard | cockpit | tiled
assets:
bg: "https://example.com/ocean-bg.jpg"
hero: "/my-images/kraken.png"
crest: "/my-images/anchor.svg"
logo: "/my-images/logo.png"
custom:
pattern: "/my-images/waves.svg"
componentStyles:
card:
boxShadow: "inset 0 0 0 1px rgba(168, 208, 255, 0.18)"
header:
background: "linear-gradient(180deg, rgba(10, 22, 40, 0.95), rgba(5, 9, 26, 0.9))"
colorOverrides:
destructive: "#ff6b6b"
ring: "#ff6b6b"
customCSS: |
/* Any additional selector-level tweaks */
```
Refresh the dashboard after creating the file. Switch themes live from the header bar — click the palette icon. Selection persists to `config.yaml` under `dashboard.theme` and is restored on reload.
---
## Plugins
A dashboard plugin is a directory with a `manifest.json`, a pre-built JS bundle, and optionally a CSS file and a Python file with FastAPI routes. Plugins live next to other Hermes plugins in `~/.hermes/plugins/<name>/` — the dashboard extension is a `dashboard/` subfolder inside that plugin directory, so one plugin can extend both the CLI/gateway and the dashboard from a single install.
Plugins don't bundle React or UI components. They use the **Plugin SDK** exposed on `window.__HERMES_PLUGIN_SDK__`. This keeps plugin bundles tiny (typically a few KB) and avoids version conflicts.
### Quick start — your first plugin
Create the directory structure:
```bash
mkdir -p ~/.hermes/plugins/my-plugin/dashboard/dist
```
Write the manifest:
```json
// ~/.hermes/plugins/my-plugin/dashboard/manifest.json
{
"name": "my-plugin",
"label": "My Plugin",
"icon": "Sparkles",
"version": "1.0.0",
"tab": {
"path": "/my-plugin",
"position": "after:skills"
},
"entry": "dist/index.js"
}
```
Write the JS bundle (a plain IIFE — no build step needed):
```javascript
// ~/.hermes/plugins/my-plugin/dashboard/dist/index.js
(function () {
"use strict";
const SDK = window.__HERMES_PLUGIN_SDK__;
const { React } = SDK;
const { Card, CardHeader, CardTitle, CardContent } = SDK.components;
function MyPage() {
return React.createElement(Card, null,
React.createElement(CardHeader, null,
React.createElement(CardTitle, null, "My Plugin"),
),
React.createElement(CardContent, null,
React.createElement("p", { className: "text-sm text-muted-foreground" },
"Hello from my custom dashboard tab.",
),
),
);
}
window.__HERMES_PLUGINS__.register("my-plugin", MyPage);
})();
```
Refresh the dashboard — your tab appears in the nav bar, after **Skills**.
:::tip Skip React.createElement
If you prefer JSX, use any bundler (esbuild, Vite, rollup) with React as an external and IIFE output. The only hard requirement is that the final file is a single JS file loadable via `<script>`. React is never bundled; it comes from `SDK.React`.
:::
### Directory layout
```
~/.hermes/plugins/my-plugin/
├── plugin.yaml # optional — existing CLI/gateway plugin manifest
├── __init__.py # optional — existing CLI/gateway hooks
└── dashboard/ # dashboard extension
├── manifest.json # required — tab config, icon, entry point
├── dist/
│ ├── index.js # required — pre-built JS bundle (IIFE)
│ └── style.css # optional — custom CSS
└── plugin_api.py # optional — backend API routes (FastAPI)
```
A single plugin directory can carry three orthogonal extensions:
- `plugin.yaml` + `__init__.py` — CLI/gateway plugin ([see plugins page](./plugins)).
- `dashboard/manifest.json` + `dashboard/dist/index.js` — dashboard UI plugin.
- `dashboard/plugin_api.py` — dashboard backend routes.
None of them are required; include only the layers you need.
### Manifest reference
```json
{
"name": "my-plugin",
"label": "My Plugin",
"description": "What this plugin does",
"icon": "Sparkles",
"version": "1.0.0",
"tab": {
"path": "/my-plugin",
"position": "after:skills",
"override": "/",
"hidden": false
},
"slots": ["sidebar", "header-left"],
"entry": "dist/index.js",
"css": "dist/style.css",
"api": "plugin_api.py"
}
```
| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | Unique plugin identifier. Lowercase, hyphens ok. Used in URLs and registration. |
| `label` | Yes | Display name shown in the nav tab. |
| `description` | No | Short description (shown in dashboard admin surfaces). |
| `icon` | No | Lucide icon name. Defaults to `Puzzle`. Unknown names fall back to `Puzzle`. |
| `version` | No | Semver string. Defaults to `0.0.0`. |
| `tab.path` | Yes | URL path for the tab (e.g. `/my-plugin`). |
| `tab.position` | No | Where to insert the tab. `"end"` (default), `"after:<path>"`, or `"before:<path>"` — value after the colon is the **path segment** of the target tab (no leading slash). Examples: `"after:skills"`, `"before:config"`. |
| `tab.override` | No | Set to a built-in route path (`"/"`, `"/sessions"`, `"/config"`, ...) to **replace** that page instead of adding a new tab. See [Replacing built-in pages](#replacing-built-in-pages-taboverride). |
| `tab.hidden` | No | When true, register the component and any slots without adding a tab to the nav. Used by slot-only plugins. See [Slot-only plugins](#slot-only-plugins-tabhidden). |
| `slots` | No | Named shell slots this plugin populates. **Documentation aid only** — actual registration happens from the JS bundle via `registerSlot()`. Listing slots here makes discovery surfaces more informative. |
| `entry` | Yes | Path to the JS bundle relative to `dashboard/`. Defaults to `dist/index.js`. |
| `css` | No | Path to a CSS file to inject as a `<link>` tag. |
| `api` | No | Path to a Python file with FastAPI routes. Mounted at `/api/plugins/<name>/`. |
#### Available icons
Plugins use Lucide icon names. The dashboard maps these by name — unknown names silently fall back to `Puzzle`.
Currently mapped: `Activity`, `BarChart3`, `Clock`, `Code`, `Database`, `Eye`, `FileText`, `Globe`, `Heart`, `KeyRound`, `MessageSquare`, `Package`, `Puzzle`, `Settings`, `Shield`, `Sparkles`, `Star`, `Terminal`, `Wrench`, `Zap`.
Need a different icon? Open a PR to `web/src/App.tsx`'s `ICON_MAP` — pure additive change.
### The Plugin SDK
Everything a plugin needs is on `window.__HERMES_PLUGIN_SDK__`. Plugins should never import React directly.
```javascript
const SDK = window.__HERMES_PLUGIN_SDK__;
// React + hooks
SDK.React // the React instance
SDK.hooks.useState
SDK.hooks.useEffect
SDK.hooks.useCallback
SDK.hooks.useMemo
SDK.hooks.useRef
SDK.hooks.useContext
SDK.hooks.createContext
// UI components (shadcn/ui primitives)
SDK.components.Card
SDK.components.CardHeader
SDK.components.CardTitle
SDK.components.CardContent
SDK.components.Badge
SDK.components.Button
SDK.components.Input
SDK.components.Label
SDK.components.Select
SDK.components.SelectOption
SDK.components.Separator
SDK.components.Tabs
SDK.components.TabsList
SDK.components.TabsTrigger
SDK.components.PluginSlot // render a named slot (useful for nested plugin UIs)
// Hermes API client + raw fetcher
SDK.api // typed client — getStatus, getSessions, getConfig, ...
SDK.fetchJSON // raw fetch for custom endpoints (plugin-registered routes)
// Utilities
SDK.utils.cn // Tailwind class merger (clsx + twMerge)
SDK.utils.timeAgo // "5m ago" from unix timestamp
SDK.utils.isoTimeAgo // "5m ago" from ISO string
// Hooks
SDK.useI18n // i18n hook for multi-language plugins
```
#### Calling your plugin's backend
```javascript
SDK.fetchJSON("/api/plugins/my-plugin/data")
.then((data) => console.log(data))
.catch((err) => console.error("API call failed:", err));
```
`fetchJSON` injects the session auth token, surfaces errors as thrown exceptions, and parses JSON automatically.
#### Calling built-in Hermes endpoints
```javascript
// Agent status
SDK.api.getStatus().then((s) => console.log("Version:", s.version));
// Recent sessions
SDK.api.getSessions(10).then((resp) => console.log(resp.sessions.length));
```
See [Web Dashboard → REST API](./web-dashboard#rest-api) for the full list.
### Shell slots
Slots let a plugin inject components into named locations of the app shell — the cockpit sidebar, the header, the footer, an overlay layer — without claiming a whole tab. Multiple plugins can populate the same slot; they render stacked in registration order.
Register from inside the plugin bundle:
```javascript
window.__HERMES_PLUGINS__.registerSlot("my-plugin", "sidebar", MySidebar);
window.__HERMES_PLUGINS__.registerSlot("my-plugin", "header-left", MyCrest);
```
#### Slot catalogue
**Shell-wide slots** (render anywhere in the app chrome):
| Slot | Location |
|------|----------|
| `backdrop` | Inside the `<Backdrop />` layer stack, above the noise layer. |
| `header-left` | Before the Hermes brand in the top bar. |
| `header-right` | Before the theme/language switchers in the top bar. |
| `header-banner` | Full-width strip below the nav. |
| `sidebar` | Cockpit sidebar rail — **only rendered when `layoutVariant === "cockpit"`**. |
| `pre-main` | Above the route outlet (inside `<main>`). |
| `post-main` | Below the route outlet (inside `<main>`). |
| `footer-left` | Footer cell content (replaces default). |
| `footer-right` | Footer cell content (replaces default). |
| `overlay` | Fixed-position layer above everything else. Useful for chrome (scanlines, vignettes) `customCSS` can't achieve alone. |
**Page-scoped slots** (render only on the named built-in page — use these to inject widgets, cards, or toolbars into an existing page without overriding the whole route):
| Slot | Where it renders |
|------|------------------|
| `sessions:top` / `sessions:bottom` | Top / bottom of the `/sessions` page. |
| `analytics:top` / `analytics:bottom` | Top / bottom of the `/analytics` page. |
| `logs:top` / `logs:bottom` | Top (above filter toolbar) / bottom (below log viewer) of `/logs`. |
| `cron:top` / `cron:bottom` | Top / bottom of the `/cron` page. |
| `skills:top` / `skills:bottom` | Top / bottom of the `/skills` page. |
| `config:top` / `config:bottom` | Top / bottom of the `/config` page. |
| `env:top` / `env:bottom` | Top / bottom of the `/env` (Keys) page. |
| `docs:top` / `docs:bottom` | Top (above the iframe) / bottom of `/docs`. |
| `chat:top` / `chat:bottom` | Top / bottom of `/chat` (only active when embedded chat is enabled). |
Example — add a banner card to the top of the Sessions page:
```javascript
function PinnedSessionsBanner() {
return React.createElement(Card, null,
React.createElement(CardContent, { className: "py-2 text-xs" },
"Pinned note injected by my-plugin"),
);
}
window.__HERMES_PLUGINS__.registerSlot("my-plugin", "sessions:top", PinnedSessionsBanner);
```
Combine page-scoped slots with `tab.hidden: true` if your plugin only augments existing pages and doesn't need a sidebar tab of its own.
The shell only renders `<PluginSlot name="..." />` for the slots above. Additional names are accepted by the registry for nested plugin UIs — a plugin can expose its own slots via `SDK.components.PluginSlot`.
#### Re-registration and HMR
If the same `(plugin, slot)` pair is registered twice, the later call replaces the earlier one — this matches how React HMR expects plugin re-mounts to behave.
### Replacing built-in pages (`tab.override`)
Setting `tab.override` to a built-in route path makes the plugin's component replace that page instead of adding a new tab. Useful when a theme wants a custom home page (`/`) but wants to keep the rest of the dashboard intact.
```json
{
"name": "my-home",
"label": "Home",
"tab": {
"path": "/my-home",
"override": "/",
"position": "end"
},
"entry": "dist/index.js"
}
```
With `override` set:
- The original page component at `/` is removed from the router.
- Your plugin renders at `/` instead.
- No nav tab is added for `tab.path` (the override is the point).
Only one plugin can override a given path. If two plugins claim the same override, the first wins and the second is ignored with a dev-mode warning.
If you only need to add a card or toolbar to an existing page without taking it over, use [page-scoped slots](#augmenting-built-in-pages-page-scoped-slots) instead.
### Augmenting built-in pages (page-scoped slots)
Full replacement via `tab.override` is heavy — your plugin now owns the entire page, including any future updates we ship to it. Most of the time you just want to add a banner, card, or toolbar to an existing page. That's what **page-scoped slots** are for.
Every built-in page exposes `<page>:top` and `<page>:bottom` slots rendered at the top and bottom of its content area. Your plugin populates one by calling `registerSlot()` — the built-in page keeps working normally, and your component renders alongside it.
Available slots: `sessions:*`, `analytics:*`, `logs:*`, `cron:*`, `skills:*`, `config:*`, `env:*`, `docs:*`, `chat:*` (each with `:top` and `:bottom`). See the full catalogue in [Shell slots → Slot catalogue](#slot-catalogue).
Minimal example — pin a banner to the top of the Sessions page:
```json
// ~/.hermes/plugins/session-notes/dashboard/manifest.json
{
"name": "session-notes",
"label": "Session Notes",
"tab": { "path": "/session-notes", "hidden": true },
"slots": ["sessions:top"],
"entry": "dist/index.js"
}
```
```javascript
// ~/.hermes/plugins/session-notes/dashboard/dist/index.js
(function () {
const SDK = window.__HERMES_PLUGIN_SDK__;
const { React } = SDK;
const { Card, CardContent } = SDK.components;
function Banner() {
return React.createElement(Card, null,
React.createElement(CardContent, { className: "py-2 text-xs" },
"Remember to label important sessions before archiving."),
);
}
// Placeholder for the hidden tab.
window.__HERMES_PLUGINS__.register("session-notes", function () { return null; });
// The real work.
window.__HERMES_PLUGINS__.registerSlot("session-notes", "sessions:top", Banner);
})();
```
Key points:
- `tab.hidden: true` keeps the plugin out of the sidebar — it has no standalone page.
- The `slots` manifest field is documentation only. The actual binding happens in the JS bundle via `registerSlot()`.
- Multiple plugins can claim the same page-scoped slot. They render stacked in registration order.
- Zero footprint when no plugin registers: the built-in page renders exactly as before.
A reference plugin (`example-dashboard` in [`hermes-example-plugins`](https://github.com/NousResearch/hermes-example-plugins/tree/main/example-dashboard)) ships a live demo that injects a banner into `sessions:top` — install it to see the pattern end-to-end.
### Slot-only plugins (`tab.hidden`)
When `tab.hidden: true`, the plugin registers its component (for direct URL visits) and any slots, but never adds a tab to the navigation. Used by plugins that only exist to inject into slots — a header crest, a sidebar HUD, an overlay.
```json
{
"name": "header-crest",
"label": "Header Crest",
"tab": {
"path": "/header-crest",
"position": "end",
"hidden": true
},
"slots": ["header-left"],
"entry": "dist/index.js"
}
```
The bundle still calls `register()` with a placeholder component (good practice in case someone hits the URL directly) and then `registerSlot()` to do the real work.
### Backend API routes
Plugins can register FastAPI routes by setting `api` in the manifest. Create the file and export a `router`:
```python
# ~/.hermes/plugins/my-plugin/dashboard/plugin_api.py
from fastapi import APIRouter
router = APIRouter()
@router.get("/data")
async def get_data():
return {"items": ["one", "two", "three"]}
@router.post("/action")
async def do_action(body: dict):
return {"ok": True, "received": body}
```
Routes are mounted under `/api/plugins/<name>/`, so the above becomes:
- `GET /api/plugins/my-plugin/data`
- `POST /api/plugins/my-plugin/action`
Plugin API routes sit behind the dashboard's normal auth gate — unauthenticated requests get a `401` before the plugin route runs, and requests to a disabled plugin's routes are rejected at request time. Still, **don't expose the dashboard on a public interface with `--host 0.0.0.0` if you run untrusted plugins** — an authenticated session can reach their routes too.
#### Accessing Hermes internals
Backend routes run inside the dashboard process, so they can import from the hermes-agent codebase directly:
```python
from fastapi import APIRouter
from hermes_state import SessionDB
from hermes_cli.config import load_config
router = APIRouter()
@router.get("/session-count")
async def session_count():
db = SessionDB()
try:
count = len(db.list_sessions(limit=9999))
return {"count": count}
finally:
db.close()
@router.get("/config-snapshot")
async def config_snapshot():
cfg = load_config()
return {"model": cfg.get("model", {})}
```
### Custom CSS per plugin
If your plugin needs styles beyond Tailwind classes and inline `style=`, add a CSS file and reference it in the manifest:
```json
{
"css": "dist/style.css"
}
```
The file is injected as a `<link>` tag on plugin load. Use specific class names to avoid conflicts with the dashboard's styles, and reference the dashboard's CSS vars to stay theme-aware:
```css
/* dist/style.css */
.my-plugin-chart {
border: 1px solid var(--color-border);
background: var(--color-card);
color: var(--color-card-foreground);
padding: 1rem;
}
.my-plugin-chart:hover {
border-color: var(--color-ring);
}
```
The dashboard exposes every shadcn token as `--color-*` plus theme extras (`--theme-asset-*`, `--component-<bucket>-*`, `--radius`, `--spacing-mul`). Reference those and your plugin automatically reskins with the active theme.
### Plugin discovery & reload
The dashboard scans three directories for `dashboard/manifest.json`:
| Priority | Directory | Source label |
|----------|-----------|--------------|
| 1 (wins on conflict) | `~/.hermes/plugins/<name>/dashboard/` | `user` |
| 2 | `<repo>/plugins/memory/<name>/dashboard/` | `bundled` |
| 2 | `<repo>/plugins/<name>/dashboard/` | `bundled` |
| 3 | `./.hermes/plugins/<name>/dashboard/` | `project` — only when `HERMES_ENABLE_PROJECT_PLUGINS` is set |
Discovery results are cached per dashboard process. After adding a new plugin, either:
```bash
# Force a rescan without restart
curl http://127.0.0.1:9119/api/dashboard/plugins/rescan
```
…or restart `hermes dashboard`.
#### Plugin load lifecycle
1. Dashboard loads. `main.tsx` exposes the SDK on `window.__HERMES_PLUGIN_SDK__` and the registry on `window.__HERMES_PLUGINS__`.
2. `App.tsx` calls `usePlugins()` → fetches `GET /api/dashboard/plugins`.
3. For each manifest: CSS `<link>` is injected (if declared), then a `<script>` tag loads the JS bundle.
4. The plugin's IIFE runs and calls `window.__HERMES_PLUGINS__.register(name, Component)` — and optionally `.registerSlot(name, slot, Component)` for each slot.
5. The dashboard resolves the registered component against the manifest, adds the tab to navigation (unless `hidden`), and mounts the component as a route.
Plugins have up to **2 seconds** after their script loads to call `register()`. After that the dashboard stops waiting and finishes initial render. If a plugin later registers, it still appears — the nav is reactive.
If a plugin's script fails to load (404, syntax error, exception during IIFE), the dashboard logs a warning to the browser console and continues without it.
---
## Combined theme + plugin demo
The [`strike-freedom-cockpit`](https://github.com/NousResearch/hermes-example-plugins/tree/main/strike-freedom-cockpit) plugin (companion repo `hermes-example-plugins`) is a complete reskin demo. It pairs a theme YAML with a slot-only plugin to produce a cockpit-style HUD without forking the dashboard.
**What it demonstrates:**
- A full theme using palette, typography, `fontUrl`, `layoutVariant: cockpit`, `assets`, `componentStyles` (notched card corners, gradient backgrounds), `colorOverrides`, and `customCSS` (scanline overlay).
- A slot-only plugin (`tab.hidden: true`) that registers into three slots:
- `sidebar` — an MS-STATUS panel with live telemetry bars driven by `SDK.api.getStatus()`.
- `header-left` — a faction crest that reads `--theme-asset-crest` from the active theme.
- `footer-right` — a custom tagline replacing the default org line.
- The plugin reads theme-supplied artwork via CSS vars, so swapping themes changes the hero/crest without plugin code changes.
**Install:**
```bash
git clone https://github.com/NousResearch/hermes-example-plugins.git
# Theme
cp hermes-example-plugins/strike-freedom-cockpit/theme/strike-freedom.yaml \
~/.hermes/dashboard-themes/
# Plugin
cp -r hermes-example-plugins/strike-freedom-cockpit ~/.hermes/plugins/
```
Open the dashboard, pick **Strike Freedom** from the theme switcher. The cockpit sidebar appears, the crest shows in the header, the tagline replaces the footer. Switch back to **Hermes Teal** and the plugin remains installed but invisible (the `sidebar` slot only renders under the `cockpit` layout variant).
Read the plugin source (`strike-freedom-cockpit/dashboard/dist/index.js` in the companion repo) to see how it reads CSS vars, guards against older dashboards without slot support, and registers three slots from one bundle.
---
## API reference
### Theme endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/dashboard/themes` | GET | List available themes + active name. Built-ins return `{name, label, description}`; user themes also include a `definition` field with the full normalised theme object. |
| `/api/dashboard/theme` | PUT | Set active theme. Body: `{"name": "midnight"}`. Persists to `config.yaml` under `dashboard.theme`. |
### Plugin endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/dashboard/plugins` | GET | List discovered plugins (with manifests, minus internal fields). |
| `/api/dashboard/plugins/rescan` | GET | Force re-scan the plugin directories without restarting. |
| `/dashboard-plugins/<name>/<path>` | GET | Serve static assets from a plugin's `dashboard/` directory. Path traversal is blocked. |
| `/api/plugins/<name>/*` | * | Plugin-registered backend routes. |
### SDK on `window`
| Global | Type | Provider |
|--------|------|----------|
| `window.__HERMES_PLUGIN_SDK__` | object | `registry.ts` — React, hooks, UI components, API client, utils. |
| `window.__HERMES_PLUGINS__.register(name, Component)` | function | Register a plugin's main component. |
| `window.__HERMES_PLUGINS__.registerSlot(name, slot, Component)` | function | Register into a named shell slot. |
---
## Troubleshooting
**My theme doesn't appear in the picker.**
Check that the file is in `~/.hermes/dashboard-themes/` and ends in `.yaml` or `.yml`. Refresh the page. Run `curl http://127.0.0.1:9119/api/dashboard/themes` — your theme should be in the response. If the YAML has a parse error, the dashboard logs to `errors.log` under `~/.hermes/logs/`.
**My plugin's tab doesn't show up.**
1. Check the manifest is at `~/.hermes/plugins/<name>/dashboard/manifest.json` (note the `dashboard/` subdirectory).
2. `curl http://127.0.0.1:9119/api/dashboard/plugins/rescan` to force re-discovery.
3. Open browser dev tools → Network — confirm `manifest.json`, `index.js`, and any CSS loaded without 404s.
4. Open browser dev tools → Console — look for errors during the IIFE or `window.__HERMES_PLUGINS__ is undefined` (indicates the SDK didn't initialize, usually a React render crash earlier).
5. Verify your bundle calls `window.__HERMES_PLUGINS__.register(...)` with the **same name** as `manifest.json:name`.
**Slot-registered components don't render.**
The `sidebar` slot only renders when the active theme has `layoutVariant: cockpit`. Other slots always render. If you're registering into a slot with no hits, add `console.log` inside `registerSlot` to confirm the plugin bundle ran at all.
**Plugin backend routes return 404.**
1. Confirm the manifest has `"api": "plugin_api.py"` pointing to an existing file inside `dashboard/`.
2. Restart `hermes dashboard` — plugin API routes are mounted once at startup, **not** on rescan.
3. Check that `plugin_api.py` exports a module-level `router = APIRouter()`. Other export names are not picked up.
4. Tail `~/.hermes/logs/errors.log` for `Failed to load plugin <name> API routes` — import errors are logged there.
**Theme change drops my color overrides.**
`colorOverrides` are scoped to the active theme and cleared on theme switch — that's by design. If you want overrides that persist, put them in your theme's YAML, not in the live switcher.
**Theme customCSS gets truncated.**
The `customCSS` block is capped at 32 KiB per theme. Split large stylesheets across multiple themes, or switch to a plugin that injects a full stylesheet via its `css` field (no size cap).
**I want to ship a plugin on PyPI.**
Dashboard plugins are installed by directory layout, not by pip entry point. The cleanest distribution path today is a git repo the user clones into `~/.hermes/plugins/`. A pip-based installer for dashboard plugins is not currently wired up.
website/docs/user-guide/features/fallback-providers.md
+435
View File
@@ -0,0 +1,435 @@
---
title: Fallback Providers
description: Configure automatic failover to backup LLM providers when your primary model is unavailable.
sidebar_label: Fallback Providers
sidebar_position: 8
---
# Fallback Providers
Hermes Agent has three layers of resilience that keep your sessions running when providers hit issues:
1. **[Credential pools](./credential-pools.md)** — rotate across multiple API keys for the *same* provider (tried first)
2. **Primary model fallback** — automatically switches to a *different* provider:model when your main model fails
3. **Auxiliary task fallback** — independent provider resolution for side tasks like vision and compression
Credential pools handle same-provider rotation (e.g., multiple OpenRouter keys). This page covers cross-provider fallback. Both are optional and work independently.
## Primary Model Fallback
When your main LLM provider encounters errors — rate limits, server overload, auth failures, connection drops — Hermes can automatically switch to a backup provider:model pair mid-session without losing your conversation.
### Configuration
The easiest path is the interactive manager:
```bash
hermes fallback
```
`hermes fallback` reuses the provider picker from `hermes model` — same provider list, same credential prompts, same validation. Use the subcommands `add`, `list` (alias `ls`), `remove` (alias `rm`), and `clear` to manage the chain. Changes persist under the top-level `fallback_providers:` list in `config.yaml`.
If you'd rather edit the YAML directly, add a top-level `fallback_providers` list to `~/.hermes/config.yaml`:
```yaml
fallback_providers:
- provider: openrouter
model: anthropic/claude-sonnet-4
```
Each entry requires both `provider` and `model`. Entries missing either field are ignored.
:::note `fallback_model` vs `fallback_providers`
`fallback_providers` (plural, list) is the current config shape and supports multiple fallbacks tried in order. `fallback_model` (singular) is the legacy single-fallback key — Hermes still honors it for back-compat, but `hermes fallback` writes the current `fallback_providers` key and migrates legacy config on write. When both are set, `fallback_providers` takes priority.
:::
### Supported Providers
| Provider | Value | Requirements |
|----------|-------|-------------|
| AI Gateway | `ai-gateway` | `AI_GATEWAY_API_KEY` |
| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` |
| Nous Portal | `nous` | `hermes setup --portal` (fresh) or `hermes auth add nous` (OAuth) |
| OpenAI Codex | `openai-codex` | `hermes model` → **ChatGPT or Codex Subscription** (ChatGPT OAuth) |
| GitHub Copilot | `copilot` | `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN` |
| GitHub Copilot ACP | `copilot-acp` | External process (editor integration) |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` or Claude Code credentials |
| z.ai / GLM | `zai` | `GLM_API_KEY` |
| Kimi / Moonshot | `kimi-coding` | `KIMI_API_KEY` |
| MiniMax | `minimax` | `MINIMAX_API_KEY` |
| MiniMax (China) | `minimax-cn` | `MINIMAX_CN_API_KEY` |
| DeepSeek | `deepseek` | `DEEPSEEK_API_KEY` |
| NVIDIA NIM | `nvidia` | `NVIDIA_API_KEY` (optional: `NVIDIA_BASE_URL`) |
| GMI Cloud | `gmi` | `GMI_API_KEY` (optional: `GMI_BASE_URL`) |
| Upstage Solar | `upstage` (alias `solar`) | `UPSTAGE_API_KEY` (optional: `UPSTAGE_BASE_URL`) |
| StepFun | `stepfun` | `STEPFUN_API_KEY` (optional: `STEPFUN_BASE_URL`) |
| Ollama Cloud | `ollama-cloud` | `OLLAMA_API_KEY` |
| Google AI Studio | `gemini` | `GOOGLE_API_KEY` (alias: `GEMINI_API_KEY`) |
| xAI (Grok) | `xai` (alias `grok`) | `XAI_API_KEY` (optional: `XAI_BASE_URL`) |
| xAI Grok OAuth (SuperGrok) | `xai-oauth` (alias `grok-oauth`) | `hermes model` → xAI Grok OAuth (browser login; SuperGrok subscription) |
| AWS Bedrock | `bedrock` | Standard boto3 auth (`AWS_REGION` + `AWS_PROFILE` or `AWS_ACCESS_KEY_ID`) |
| Qwen Portal (OAuth) | `qwen-oauth` | `hermes model` (Qwen Portal OAuth; optional: `HERMES_QWEN_BASE_URL`) |
| MiniMax (OAuth) | `minimax-oauth` | `hermes model` (MiniMax portal OAuth) |
| OpenCode Zen | `opencode-zen` | `OPENCODE_ZEN_API_KEY` |
| CommandCode | `commandcode` (alias `commandcode-chat`; Claude via `commandcode-anthropic`) | `COMMANDCODE_API_KEY` |
| OpenCode Go | `opencode-go` | `OPENCODE_GO_API_KEY` |
| OpenCode Free | `opencode-free` | — (keyless, no credential) |
| Kilo Code | `kilocode` | `KILOCODE_API_KEY` |
| Ramp Router | `router` | `RAMP_ROUTER_API_KEY` |
| Xiaomi MiMo | `xiaomi` | `XIAOMI_API_KEY` |
| Arcee AI | `arcee` | `ARCEEAI_API_KEY` |
| GMI Cloud | `gmi` | `GMI_API_KEY` |
| Nebius Token Factory | `nebius-token-factory` | `NEBIUS_API_KEY` |
| Alibaba / DashScope | `alibaba` | `DASHSCOPE_API_KEY` |
| Alibaba Coding Plan | `alibaba-coding-plan` | `ALIBABA_CODING_PLAN_API_KEY` (falls back to `DASHSCOPE_API_KEY`) |
| Kimi / Moonshot (China) | `kimi-coding-cn` | `KIMI_CN_API_KEY` |
| StepFun | `stepfun` | `STEPFUN_API_KEY` |
| Tencent TokenHub | `tencent-tokenhub` | `TOKENHUB_API_KEY` |
| Tencent TokenPlan | `tencent-tokenplan` | `TOKENPLAN_API_KEY` |
| Microsoft Foundry | `azure-foundry` | `AZURE_FOUNDRY_API_KEY` + `AZURE_FOUNDRY_BASE_URL` |
| LM Studio (local) | `lmstudio` | `LM_API_KEY` (or none for local) + `LM_BASE_URL` |
| Hugging Face | `huggingface` | `HF_TOKEN` |
| Custom endpoint | `custom` | `base_url` + `key_env` (see below) |
### Custom Endpoint Fallback
For a custom OpenAI-compatible endpoint, add `base_url` and optionally `key_env`:
```yaml
fallback_providers:
- provider: custom
model: my-local-model
base_url: http://localhost:8000/v1
key_env: MY_LOCAL_KEY # env var name containing the API key
```
### When Fallback Triggers
The fallback activates automatically when the primary model fails with:
- **Rate limits** (HTTP 429) — after exhausting retry attempts
- **Server errors** (HTTP 500, 502, 503) — after exhausting retry attempts
- **Auth failures** (HTTP 401, 403) — immediately (no point retrying)
- **Not found** (HTTP 404) — immediately
- **Invalid responses** — when the API returns malformed or empty responses repeatedly
When triggered, Hermes:
1. Resolves credentials for the fallback provider
2. Builds a new API client
3. Swaps the model, provider, and client in-place
4. Resets the retry counter and continues the conversation
The switch is seamless — your conversation history, tool calls, and context are preserved. The agent continues from exactly where it left off, just using a different model.
:::warning Fallback resets the prompt cache
Prompt caches are keyed to the model (and on most providers, the account) serving the request. When fallback fires, the new provider:model has no cached prefix for your conversation, so the next request re-reads the entire history at full input-token price instead of the ~75–90% discounted cached rate. The same applies when the turn ends and the primary is restored — that first request back on the primary is a full re-read too (unless the primary's cache TTL hasn't expired). This is unavoidable — it's the cost of staying alive through an outage — but it's why a long session that bounces between providers can cost noticeably more than one that stays put.
:::
:::info Per-Turn, Not Per-Session
Fallback is **turn-scoped**: each new user message starts with the primary model restored. If the primary fails mid-turn, fallback activates for that turn only. On the next message, Hermes tries the primary again. Within a single turn, fallback activates at most once — if the fallback also fails, normal error handling takes over (retries, then error message). This prevents cascading failover loops within a turn while giving the primary model a fresh chance every turn.
The per-turn retry is **reset-aware**: when the primary's credentials report a rate-limit reset time that hasn't elapsed yet (subscription windows like Claude Pro/Max's 5-hour blocks or Codex weekly limits report these as hours or days), Hermes skips the doomed retry and stays on the fallback until the reset passes — avoiding two pointless provider switches (and two prompt-cache invalidations) per turn. The moment the reset time elapses, the next turn goes back to the primary automatically. Transient 429s without a reset time keep the existing behavior: a short cooldown, then retry every turn.
:::
### Examples
**OpenRouter as fallback for Anthropic native:**
```yaml
model:
provider: anthropic
default: claude-sonnet-4-6
fallback_providers:
- provider: openrouter
model: anthropic/claude-sonnet-4
```
**Nous Portal as fallback for OpenRouter:**
```yaml
model:
provider: openrouter
default: anthropic/claude-opus-4
fallback_providers:
- provider: nous
model: nous-hermes-3
```
**Local model as fallback for cloud:**
```yaml
fallback_providers:
- provider: custom
model: llama-3.1-70b
base_url: http://localhost:8000/v1
key_env: LOCAL_API_KEY
```
**Codex OAuth as fallback:**
```yaml
fallback_providers:
- provider: openai-codex
model: gpt-5.3-codex
```
### Where Fallback Works
| Context | Fallback Supported |
|---------|-------------------|
| CLI sessions | ✔ |
| Messaging gateway (Telegram, Discord, etc.) | ✔ |
| Subagent delegation | ✔ (subagents inherit the parent fallback chain) |
| Cron jobs | ✔ (cron agents inherit configured fallback providers) |
| Auxiliary tasks on `provider: auto` | ✔ (try per-task fallback, then the main fallback chain before built-in aux discovery) |
:::tip
There are no environment variables for the primary fallback chain — configure it exclusively through `config.yaml` or `hermes fallback`. This is intentional: fallback configuration is a deliberate choice, not something a stale shell export should override.
:::
---
## Auxiliary Task Fallback
Hermes uses separate lightweight models for side tasks. Each task has its own provider resolution chain that acts as a built-in fallback system.
### Tasks with Independent Provider Resolution
| Task | What It Does | Config Key |
|------|-------------|-----------|
| Vision | Image analysis, browser screenshots | `auxiliary.vision` |
| Compression | Context compression summaries | `auxiliary.compression` |
| Skills Hub | Skill search and discovery | `auxiliary.skills_hub` |
| MCP | MCP helper operations | `auxiliary.mcp` |
| Approval | Smart command-approval classification | `auxiliary.approval` |
| Title Generation | Session title summaries | `auxiliary.title_generation` |
| Review | `/review` reviewer subagent (full agent, not a single LLM call) | `auxiliary.review` |
| Triage Specifier | `hermes kanban specify` / dashboard ✨ button — fleshes out a one-liner triage task into a real spec | `auxiliary.triage_specifier` |
### Auto-Detection Chain
When a task's provider is set to `"auto"` (the default), Hermes first tries the main provider + main model for that auxiliary task. If that route is unavailable or later fails with a capacity-style error, Hermes now honors user-configured fallback policy before using the built-in discovery chain:
```text
Main provider + main model → auxiliary.<task>.fallback_chain →
fallback_providers / fallback_model → built-in auxiliary discovery chain
```
The task-specific chain is most precise and wins when present. The top-level `fallback_providers` chain is the same policy the main agent uses, so free-only or same-provider fallback rules apply to auxiliary tasks on `auto` as well.
**Built-in text discovery chain (compression, web extract, title generation, etc.):**
```text
OpenRouter → Nous Portal → Custom endpoint → Codex OAuth →
API-key providers (z.ai, Kimi, MiniMax, Xiaomi MiMo, Hugging Face, Anthropic) → give up
```
**Built-in vision discovery chain:**
```text
Main provider (if vision-capable) → OpenRouter → Nous Portal →
Codex OAuth → Anthropic → Custom endpoint → give up
```
Those built-in chains are a convenience fallback for users who have not declared a task-specific or main fallback policy.
### Configuring Auxiliary Providers
Each task can be configured independently in `config.yaml`:
```yaml
auxiliary:
vision:
provider: "auto" # auto | openrouter | nous | codex | main | anthropic
model: "" # e.g. "openai/gpt-4o"
base_url: "" # direct endpoint (takes precedence over provider)
api_key: "" # API key for base_url
compression:
provider: "auto"
model: ""
fallback_chain: # optional, task-specific fallback policy
- provider: openrouter
model: inclusionai/ring-2.6-1t:free
skills_hub:
provider: "auto"
model: ""
mcp:
provider: "auto"
model: ""
```
Every task above follows the same **provider / model / base_url** pattern. Each task can also declare its own `fallback_chain`; if omitted, `provider: auto` uses the top-level `fallback_providers` chain before Hermes' built-in auxiliary discovery chain.
Context compression is configured under `auxiliary.compression`:
```yaml
auxiliary:
compression:
provider: main # Same provider options as other auxiliary tasks
model: google/gemini-3-flash-preview
base_url: null # Custom OpenAI-compatible endpoint
```
And the primary fallback chain uses:
```yaml
fallback_providers:
- provider: openrouter
model: anthropic/claude-sonnet-4
# base_url: http://localhost:8000/v1 # Optional custom endpoint
```
All three — auxiliary, compression, fallback — work the same way: set `provider` to pick who handles the request, `model` to pick which model, and `base_url` to point at a custom endpoint (overrides provider).
### Provider Options for Auxiliary Tasks
These options apply to `auxiliary:`, `compression:`, and `fallback_providers:` entries only — `"main"` is **not** a valid value for your top-level `model.provider`. For custom endpoints, use `provider: custom` in your `model:` section (see [AI Providers](/integrations/providers)).
| Provider | Description | Requirements |
|----------|-------------|-------------|
| `"auto"` | Try providers in order until one works (default) | At least one provider configured |
| `"openrouter"` | Force OpenRouter | `OPENROUTER_API_KEY` |
| `"nous"` | Force Nous Portal | `hermes auth` |
| `"codex"` | Force Codex OAuth | `hermes model` → ChatGPT or Codex Subscription |
| `"main"` | Use whatever provider the main agent uses (auxiliary tasks only) | Active main provider configured |
| `"anthropic"` | Force Anthropic native | `ANTHROPIC_API_KEY` or Claude Code credentials |
### Direct Endpoint Override
For any auxiliary task, setting `base_url` bypasses provider resolution entirely and sends requests directly to that endpoint:
```yaml
auxiliary:
vision:
base_url: "http://localhost:1234/v1"
api_key: "local-key"
model: "qwen2.5-vl"
```
`base_url` takes precedence over `provider`. Hermes uses the configured `api_key` for authentication, falling back to `OPENAI_API_KEY` if not set. It does **not** reuse `OPENROUTER_API_KEY` for custom endpoints.
---
## Auxiliary Capacity-Error Fallback
When you set an explicit auxiliary provider (e.g. `auxiliary.vision.provider: glm`), Hermes treats that as your preferred choice — but if the provider literally cannot serve the request because of a **capacity error** (HTTP 402 payment required, HTTP 429 daily-quota exhaustion, connection failure), Hermes falls back through a layered chain instead of failing silently:
1. **Primary aux provider** — the one you configured (tried first, always)
2. **`auxiliary.<task>.fallback_chain`** — your per-task override list, if you wrote one
3. **Main agent provider + model** — last-resort safety net (always tried, even if you didn't write a chain)
4. **Warn + re-raise** — if every layer fails, Hermes logs `Auxiliary <task>: ... all fallbacks exhausted` at WARNING level and re-raises the original error
Transient HTTP 429 rate limits (`Retry-After: ...`) are treated as request constraints, not capacity problems — they respect your explicit provider choice and do **not** trigger the fallback ladder. Only daily/monthly quota exhaustion, payment errors, and connection failures bypass the explicit-provider gate.
For users on `provider: auto` (no explicit aux provider), the existing auto-detection chain runs in place of steps 2–3. Its first step is already the main agent model, so `auto` users get the same outcome with zero config.
### Optional: per-task fallback chain
If you want a different fallback ordering than "main agent model first", configure `fallback_chain` explicitly. Each entry needs at least `provider`; `model`, `base_url`, and `api_key` are optional.
```yaml
auxiliary:
vision:
provider: glm
model: glm-4v-flash
fallback_chain:
- provider: openrouter
model: google/gemini-3-flash-preview
- provider: nous
model: anthropic/claude-sonnet-4
compression:
provider: openrouter
fallback_chain:
- provider: openai
model: gpt-4o-mini
timeout: 240 # optional — this candidate's own deadline (seconds)
```
You do **not** need to configure `fallback_chain` to get fallback — the main-agent safety net runs regardless. Use it only when you specifically want a different order than the default.
Each `fallback_chain` entry may also declare its own `timeout` (seconds). Without it, a fallback candidate inherits the task-level timeout — which may be tuned for the primary provider. Declaring a per-entry `timeout` lets a slower-but-reliable fallback (e.g. a large-context summarizer) get the budget it actually needs instead of dying on the primary's clock.
### Provider quota errors that trigger fallback
Hermes recognizes these as capacity-equivalent to 402 credit exhaustion (not transient rate limits):
- Bedrock / LiteLLM: `Too many tokens per day`, `daily limit`, `tokens per day`
- Vertex AI / GCP: `quota exceeded`, `resource exhausted`, `RESOURCE_EXHAUSTED`
- Generic: `daily quota`, `quota_exceeded`
If your provider returns a different phrase for daily-quota exhaustion and Hermes doesn't trigger fallback, that's a bug — open an issue with the exact error string.
---
## Context Compression Fallback
Context compression uses the `auxiliary.compression` config block to control which model and provider handles summarization:
```yaml
auxiliary:
compression:
provider: "auto" # auto | openrouter | nous | main
model: "google/gemini-3-flash-preview"
```
:::info Legacy migration
Older configs with `compression.summary_model` / `compression.summary_provider` / `compression.summary_base_url` are automatically migrated to `auxiliary.compression.*` on first load (config version 17).
:::
If no provider is available for compression, Hermes drops middle conversation turns without generating a summary rather than failing the session.
---
## Delegation Provider Override
Subagents spawned by `delegate_task` inherit the parent agent's primary fallback chain. You can still route subagents to a different primary provider:model pair for cost optimization:
```yaml
delegation:
provider: "openrouter" # override provider for all subagents
model: "google/gemini-3-flash-preview" # override model
# base_url: "http://localhost:1234/v1" # or use a direct endpoint
# api_key: "local-key"
```
See [Subagent Delegation](/user-guide/features/delegation) for full configuration details.
---
## Cron Job Providers
Cron jobs inherit your configured `fallback_providers` chain (or legacy `fallback_model`) when they create an agent. To use a different primary provider for a cron job, configure `provider` and `model` overrides on the cron job itself:
```python
cronjob(
action="create",
schedule="every 2h",
prompt="Check server status",
provider="openrouter",
model="google/gemini-3-flash-preview"
)
```
See [Scheduled Tasks (Cron)](/user-guide/features/cron) for full configuration details.
---
## Summary
| Feature | Fallback Mechanism | Config Location |
|---------|-------------------|----------------|
| Main agent model | `fallback_providers` in config.yaml — per-turn failover on errors (primary restored each turn) | `fallback_providers:` (top-level list) |
| Auxiliary tasks (any) — auto users | Full auto-detection chain (main agent model first, then provider chain) on capacity errors | `auxiliary.<task>.provider: auto` |
| Auxiliary tasks (any) — explicit provider | `fallback_chain` (if set) → main agent model → warn + raise, on capacity errors only | `auxiliary.<task>.fallback_chain` |
| Vision | Layered (see above) + internal OpenRouter retry | `auxiliary.vision` |
| Context compression | Layered (see above); degrades to no-summary if all layers unavailable | `auxiliary.compression` |
| Skills hub | Layered (see above) | `auxiliary.skills_hub` |
| MCP helpers | Layered (see above) | `auxiliary.mcp` |
| Approval classification | Layered (see above) | `auxiliary.approval` |
| Title generation | Layered (see above) | `auxiliary.title_generation` |
| Triage specifier | Layered (see above) | `auxiliary.triage_specifier` |
| Delegation | Inherits the parent's `fallback_providers` chain; optional provider/model override | `delegation.provider` / `delegation.model` |
| Cron jobs | Inherit the configured `fallback_providers` chain; optional per-job provider override | Per-job `provider` / `model` |
website/docs/user-guide/features/goals.md
+289
View File
@@ -0,0 +1,289 @@
---
sidebar_position: 16
title: "Persistent Goals"
description: "Set a standing goal and let Hermes keep working across turns until it's done. Our take on the Ralph loop."
---
# Persistent Goals (`/goal`)
`/goal` gives Hermes a standing objective that survives across turns. After every turn a lightweight judge model checks whether the goal is satisfied by the assistant's last response. If not, Hermes automatically feeds a continuation prompt back into the same session and keeps working — until the goal is achieved, you pause or clear it, or the turn budget runs out.
It's our take on the **Ralph loop**, directly inspired by [Codex CLI 0.128.0's `/goal`](https://github.com/openai/codex) by Eric Traut (OpenAI). The core idea — keep a goal alive across turns and don't stop until it's achieved — is theirs. The implementation here is independent and adapted to Hermes' architecture.
## When to use it
Use `/goal` for tasks where you want Hermes to iterate on its own without you re-prompting every turn:
- "Fix every lint error in `src/` and verify `ruff check` passes"
- "Port feature X from repo Y, including tests, and get CI green"
- "Investigate why session IDs sometimes drift on mid-run compression and write up a report"
- "Build a small CLI to rename files by their EXIF dates, then test it against the photos/ folder"
Tasks where the agent does one turn and stops don't need `/goal`. Tasks where *you'd otherwise have to say "keep going" three times* are where this shines.
## Goals vs Kanban: which one do I want?
`/goal` and [Kanban](./kanban) both keep Hermes working without you re-prompting, so it's tempting to assume one flows into the other. It doesn't — the boundary is sharp:
- **`/goal` is single-session.** The loop feeds continuation prompts back into *this* conversation until the judge says done. Setting a goal never creates a kanban card, never assigns work to another profile, and never fans out. There is no handoff to the board, implicit or otherwise.
- **Kanban is a board of many tasks.** Each card is dispatched to its own worker process with its own session. Cards, dependencies, assignees, and handoffs live on the board — not in `/goal`.
- **The overlap is deliberate, and small.** A kanban card created with `--goal` runs the same Ralph-style continuation engine as `/goal` — but *inside that one card's worker session*. It borrows the engine, not the board. See [Goal-mode cards](./kanban#goal-mode-cards---goal).
| You want | Reach for |
|---|---|
| Keep iterating on one task in this chat until it's done | `/goal <text>` |
| Many independent tasks, with dependencies, handoffs, or multiple profiles | [Kanban](./kanban) — `hermes kanban create …` |
| One card on the board that should keep iterating until its acceptance criteria are met | A kanban card with `--goal` |
:::note
If you want work on the board, put it there yourself (`hermes kanban create …`) — `/goal` won't do it for you. The reverse is also true: pausing, resuming, or clearing a goal in this chat never creates, claims, or moves a kanban card.
:::
## Quick start
```
/goal Fix every failing test in tests/hermes_cli/ and make sure scripts/run_tests.sh passes for that directory
```
What you'll see:
1. **Goal accepted** — `⊙ Goal set (20-turn budget): <your goal>`
2. **Turn 1 runs** — Hermes starts working as if you'd sent the goal as a normal message.
3. **Judge runs** — after the turn, the judge model decides `done`, `continue`, or `blocked`.
4. **Loop fires if needed** — if `continue`, you'll see `↻ Continuing toward goal (1/20): <judge's reason>` and Hermes takes the next step automatically.
5. **Terminates** — eventually you see either `✓ Goal achieved: <reason>` or `⏸ Goal paused — N/20 turns used`.
## Commands
| Command | What it does |
|---|---|
| `/goal <text>` | Set (or replace) the standing goal. Kicks off the first turn immediately so you don't need to send a separate message. |
| `/goal draft <text>` | Draft a structured completion contract from a plain-language objective, then set it. See [Completion contracts](#completion-contracts). |
| `/goal show` | Print the active goal's completion contract. |
| `/goal` or `/goal status` | Show the current goal, its status, and turns used. |
| `/goal pause` | Stop the auto-continuation loop without clearing the goal. |
| `/goal resume` | Resume the loop (resets the turn counter back to zero). |
| `/goal clear` | Drop the goal entirely. |
| `/goal wait <pid> [reason]` | Park the loop on a background process — it stops re-poking the agent every turn while the process runs, and auto-resumes when it exits. |
| `/goal unwait` | Drop the wait barrier and resume the loop immediately. |
| `/goal gate add <command>` | Add a **quality gate**: a shell command that must pass before the goal can be judged done. See [Quality gates](#quality-gates). |
| `/goal gate` or `/goal gate list` | List the goal's gates and their pass/fail state. |
| `/goal gate remove <N>` | Remove the Nth gate (1-based). |
| `/goal gate clear` | Remove all gates. |
Works identically on the CLI and every gateway platform (Telegram, Discord, Slack, Matrix, Signal, WhatsApp, SMS, iMessage, Webhook, API server, and the web dashboard).
## Completion contracts
A bare `/goal <text>` works fine, but a *vague* goal makes for vague judging — the judge can only check what you told it to want. Codex's `/goal` guidance makes the same point: a durable objective works best when it names **what done means, how to prove it, what not to break, what's in scope, and when to stop**. Hermes adapts this as an optional **completion contract** layered on top of the existing goal loop.
A contract has five fields, all optional:
| Field | Meaning |
|---|---|
| `outcome` | The single end state that must be true when done. |
| `verification` | The specific test / command / artifact that *proves* the outcome. |
| `constraints` | What must not change or regress. |
| `boundaries` | Which files, dirs, tools, or systems are in scope. |
| `stop_when` | The condition under which Hermes should stop and ask for input. |
When a contract is set, both prompts change: the **continuation prompt** tells the agent to target the verification surface and respect the constraints, and the **judge prompt** decides `done` *only when the verification criterion is met with concrete evidence* (a command result, file excerpt, test output) — not a loose "looks done" claim. This directly tightens the most common `/goal` failure mode (premature completion or endless over-continuation on an underspecified objective).
### Two ways to set a contract
**1. Let Hermes draft it** (recommended — adapted from Codex's "let the agent draft the goal" tip):
```
/goal draft Migrate the auth service from session cookies to JWT
```
Hermes expands your one-liner into a full contract via the `goal_judge` auxiliary model, sets it, and shows you the result so you can review or tighten any field. If the aux model is unavailable, it falls back to a plain free-form goal — drafting never blocks setting a goal.
**2. Write it inline** with `field: value` lines:
```
/goal Migrate auth to JWT
verify: pytest tests/auth passes
constraints: keep the /login response shape unchanged
boundaries: only touch services/auth and its tests
stop when: a DB schema migration is required
```
The first non-field line(s) are the goal headline; recognized field prefixes (`verify:`, `verified by:`, `constraints:`, `preserve:`, `boundaries:`, `scope:`, `stop when:`, `blocked:`, …) populate the contract. A plain goal with an incidental colon (`Fix bug: the parser drops commas`) is **not** mangled — only known field prefixes are pulled out.
Use `/goal show` to review the active contract. Contracts persist in `SessionDB.state_meta` alongside the goal, so they survive `/resume`. Old goals from before this feature load unchanged (no contract). Contracts and `/subgoal` criteria compose: subgoals fold into the contract as extra criteria the judge must also satisfy.
## Adding criteria mid-goal: `/subgoal`
While a goal is active you can append extra acceptance criteria with `/subgoal <text>` without resetting the loop. Each call adds one numbered item to the goal's subgoal list; the **continuation prompt** the agent sees on the next turn includes the original goal plus an "Additional criteria the user added mid-loop" block, and the **judge prompt** is rewritten so the verdict must consider every subgoal — the goal isn't marked done until the original objective **and** every subgoal are met.
| Command | What it does |
|---|---|
| `/subgoal <text>` | Append a new criterion to the active goal. Requires an active `/goal`. |
| `/subgoal` (no args) | Show the current numbered subgoal list. |
| `/subgoal remove <N>` | Remove the Nth subgoal (1-based). |
| `/subgoal clear` | Drop every subgoal but keep the original goal intact. |
Subgoals are persisted alongside the goal in `SessionDB.state_meta`, so they survive `/resume`. Setting a new `/goal <text>` replaces the goal and clears the subgoal list; `/goal clear` does the same.
Use this when you start a loop ("fix the failing tests") and notice partway through that you also want it to "and add a regression test for the bug you just patched" — `/subgoal add a regression test` tightens the success criteria without breaking the running loop.
## Quality gates
A completion contract makes the judge stricter, but the judge is still an LLM reading prose. A **quality gate** is stronger: a deterministic shell command that must exit 0 before the goal can complete at all. Inspired by Prime-Agent's bounded autonomous mode (`--autonomous-gate`).
```
/goal Fix the flaky session tests
/goal gate add scripts/run_tests.sh tests/hermes_cli/test_goals.py
```
How it works, each turn:
1. **Gates run before the judge.** If any gate fails, the judge is *not called* — a red gate is deterministic evidence the goal isn't done. The gate's exit code and output tail (last ~3 KB) become the continuation prompt, so the agent iterates against the actual failure instead of a vibe.
2. **All gates pass → normal judging.** The LLM judge then decides done/blocked/continue/wait exactly as before.
3. **Unchanged workspace → no re-run.** If a gate failed and nothing changed in the workspace since (tracked via a git fingerprint of HEAD + working-tree status), the gate is not re-run — the recorded failure is replayed and the attempt count advances. A stuck agent can't burn wall-clock re-running an identical red suite. Outside a git repo, gates simply always re-run.
4. **Retries are bounded.** Each gate defaults to 3 retries and a 5-minute timeout. When a gate exhausts its retries the goal auto-pauses (like the turn budget) with a message telling you to fix it manually, remove the gate, or `/goal resume`.
Gates persist with the goal in `SessionDB.state_meta` (they survive `/resume` and context compression), and gate management (`/goal gate …`) is safe mid-run on the gateway — gates only run at turn boundary.
Gates and contracts compose: use a contract to shape *what the agent aims for*, and gates to make *"done" mechanically checkable*. When both are set, gates run first.
## Parking on a background process: automatic, with a manual override
Some goals are gated on something that takes minutes and runs on its own — CI on a pushed PR, a long build, a test matrix, a deploy, a rate-limit cooldown. Without help, the goal loop would re-poke the agent every turn into "is it done yet?" busy-work while it waits.
**This is handled automatically.** Every turn, the judge is shown the agent's live background processes (the `terminal(background=true)` registry — pid, session id, command, uptime, recent output, and any `watch_patterns` / `notify_on_complete` trigger) alongside the goal and the agent's response. When the agent's progress is genuinely gated on one of them, the judge returns a **`wait`** verdict instead of `continue`, and the loop **parks**: the next turns are skipped (no judge call, no continuation, no turn consumed) until the wait is satisfied — then it resumes normally with the result in hand. The judge can also park on a **time** basis (`wait_for_seconds`) for backoff/cooldown waits. `/goal status` shows `⏳ Goal (parked …)` while parked.
The judge picks the right kind of wait from the process's own signal:
- **`wait_on_session <id>`** — releases when the process's *own trigger* fires: it exits, **or** (if it was started with `watch_patterns`) its pattern matches. This is the one for a long-lived watcher / server / poller that signals **mid-run** (e.g. a build process that prints `BUILD SUCCESSFUL` and keeps running, or a `notify_on_complete` watcher) and may never exit on its own.
- **`wait_on_pid <pid>`** — releases on process exit only.
- **`wait_for_seconds <n>`** — releases after a fixed delay.
You don't type anything for this — it's the judge's decision, made from the process context the loop hands it. The manual commands exist as an override:
| Command | What it does |
|---|---|
| `/goal wait <pid> [reason]` | Manually park the loop until the process with that PID exits. |
| `/goal unwait` | Clear any wait barrier (judge- or manually-set) and resume immediately. |
The barrier (pid- or time-based) is persisted with the goal in `SessionDB.state_meta`, so it survives `/resume`. `/goal pause`, `/goal resume`, and `/goal clear` all drop it. If the PID is already dead when the barrier is set (or dies while parked), or the time deadline passes, the barrier clears on the next check — a stale barrier can never wedge the loop.
Typical flow: the agent pushes a PR, starts a CI watcher with `terminal(background=true, notify_on_complete=true)`, and reports "watching CI." The judge sees the watcher process still running, returns `wait` on its pid, and the loop goes quiet — then picks back up the instant CI finishes and judges the goal against the actual result.
## Behavior details
### The judge
After every turn, Hermes calls an auxiliary model with:
- The standing goal text
- The agent's most recent final response (last ~4 KB of text)
- A system prompt telling the judge to reply with strict one-line JSON: `{"verdict": "done" | "blocked" | "continue" | "wait", "reason": "<one-sentence rationale>"}` (wait verdicts add `wait_on_session` / `wait_on_pid` / `wait_for_seconds`; the legacy `{"done": <bool>, "reason": "..."}` shape is still accepted)
The judge is deliberately conservative: it marks a goal `done` only when the response **explicitly** confirms the goal is complete, when the final deliverable is clearly produced. A goal the agent explains is **unachievable** (impossible, out of scope, needs user input) gets a `blocked` verdict instead — never `done`: the goal **pauses** with the judge's reason (`🚫 Goal judged unachievable — paused`), so you can re-scope it with `/goal <text>` or override with `/goal resume` rather than burning budget or having an impossible task waved through as complete.
### Fail-open semantics
If the judge errors (network blip, malformed response, unavailable aux client), Hermes treats the verdict as `continue` — a broken judge never wedges progress. The **turn budget** is the real backstop.
### Turn budget
Default is 20 continuation turns (`goals.max_turns` in `config.yaml`). When the budget is hit, Hermes auto-pauses and tells you exactly how to proceed:
```
⏸ Goal paused — 20/20 turns used. Use /goal resume to keep going, or /goal clear to stop.
```
`/goal resume` resets the counter to zero, so you can keep going in measured chunks.
### User messages always preempt
Any real message you send while a goal is active takes priority over the continuation loop. On the CLI your message lands in `_pending_input` ahead of the queued continuation; on the gateway it goes through the adapter FIFO the same way. The judge runs again after your turn — so if your message happens to complete the goal, the judge will catch it and stop.
### Mid-run safety (gateway)
While an agent is already running, `/goal status`, `/goal pause`, `/goal clear`, `/goal wait`, and `/goal unwait` are safe to run — they only touch control-plane state and don't interrupt the current turn. Setting a **new** goal mid-run (`/goal <new text>`) is rejected with a message telling you to `/stop` first, so the old continuation can't race the new one.
### Persistence
Goal state lives in `SessionDB.state_meta` keyed by `goal:<session_id>`. That means `/resume` picks up right where you left off — set a goal, close your laptop, come back tomorrow, `/resume`, and the goal is still standing exactly as you left it (active, paused, or done).
### Prompt cache
The continuation prompt is a plain user-role message appended to history. It does **not** mutate the system prompt, swap toolsets, or touch the conversation in any way that invalidates Hermes' prompt cache. Running a 20-turn goal costs the same cache-wise as 20 turns of normal conversation.
## Configuration
Add to `~/.hermes/config.yaml`:
```yaml
goals:
# Max continuation turns before Hermes auto-pauses and asks you to
# /goal resume. Default 20. Lower this if you want tighter loops;
# raise it for long-running refactors.
max_turns: 20
```
### Choosing the judge model
The judge uses the `goal_judge` auxiliary task. By default it resolves to your main model (see [Auxiliary Models](/user-guide/configuration#auxiliary-models)). If you want to route the judge to a cheap fast model to keep costs down, add an override:
```yaml
auxiliary:
goal_judge:
provider: openrouter
model: google/gemini-3-flash-preview
```
The judge call is small (~200 output tokens) and runs once per turn, so a cheap fast model is usually the right call.
## Example walkthrough
```
You: /goal Create four files /tmp/note_{1..4}.txt, one per turn, each containing its number as text
⊙ Goal set (20-turn budget): Create four files /tmp/note_{1..4}.txt, one per turn, each containing its number as text
Hermes: Creating /tmp/note_1.txt now.
💻 echo "1" > /tmp/note_1.txt (0.1s)
I've created /tmp/note_1.txt with the content "1". I'll continue with the remaining files on the next turn as you specified.
↻ Continuing toward goal (1/20): Only 1 of 4 files has been created; 3 files remain.
Hermes: [Continuing toward your standing goal]
💻 echo "2" > /tmp/note_2.txt (0.1s)
Created /tmp/note_2.txt. Two more to go.
↻ Continuing toward goal (2/20): 2 of 4 files created; 2 remain.
Hermes: [Continuing toward your standing goal]
💻 echo "3" > /tmp/note_3.txt (0.1s)
Created /tmp/note_3.txt.
↻ Continuing toward goal (3/20): 3 of 4 files created; 1 remains.
Hermes: [Continuing toward your standing goal]
💻 echo "4" > /tmp/note_4.txt (0.1s)
All four files have been created: /tmp/note_1.txt through /tmp/note_4.txt, each containing its number.
✓ Goal achieved: All four files were created with the specified content, completing the goal.
You: _
```
Four turns, one `/goal` invocation, zero "keep going" prompts from you.
## When the judge gets it wrong
No judge is perfect. Two failure modes to watch for:
**False negative — judge says continue when the goal is actually done.** The turn budget catches this. You'll see `⏸ Goal paused` and can `/goal clear` or just send a new message.
**False positive — judge says done when work remains.** You'll see `✓ Goal achieved` but you know better. Send a follow-up message to continue, or re-set the goal more precisely: `/goal <more specific text>`. The judge's system prompt is deliberately conservative to make false positives rarer than false negatives.
If you find a judge verdict unconvincing, the reason text in the `↻ Continuing toward goal` or `✓ Goal achieved` line tells you exactly what the judge saw. That's usually enough to diagnose whether the goal text was ambiguous or the model's response was.
## Attribution
`/goal` is Hermes' take on the **Ralph loop** pattern. The user-facing design — keep a goal alive across turns, don't stop until it's achieved, with create/pause/resume/clear controls — was popularised and shipped in [Codex CLI 0.128.0](https://github.com/openai/codex) by Eric Traut on OpenAI's Codex team. Our implementation is independent (central `CommandDef` registry, `SessionDB.state_meta` persistence, auxiliary-client judge, adapter-FIFO continuation on the gateway side) but the idea is theirs. Credit where credit's due.
website/docs/user-guide/features/heartbeat.md
+65
View File
@@ -0,0 +1,65 @@
---
sidebar_position: 17
title: "Session Heartbeats"
description: "A recurring prompt that re-enters your current session whenever it's idle — /heartbeat every 10m Check the deployment."
---
# Session Heartbeats (`/heartbeat`)
`/heartbeat` gives the **current session** one recurring instruction. Whenever the session is idle and the interval has elapsed, the prompt fires as a normal user turn — same conversation, same context, same prompt cache.
```
/heartbeat every 10m Check the deployment and report meaningful changes
```
Inspired by Prime-Agent's `/heartbeat`. The Hermes adaptation keeps the strict message-flow invariants: the heartbeat is injected only between turns (never mid-run), as a plain user-role message.
## Heartbeat vs cron: which one do I want?
They look similar but serve different jobs:
| | `/heartbeat` | [`hermes cron`](./cron) |
|---|---|---|
| Runs in | **This conversation** — full context, memory of the discussion | A fresh isolated session per tick |
| Survives process restart | State survives (SessionDB); firing resumes next time the session is driven | Yes — fully durable scheduler |
| How many | One per session | Unlimited jobs |
| Best for | "Keep an eye on X *in this thread* while we work" | Standing jobs, reports, watchdogs, deliveries |
Rule of thumb: if the recurring prompt needs the conversation's context, use `/heartbeat`. If it's a self-contained job, use cron.
## Commands
| Command | What it does |
|---|---|
| `/heartbeat every <interval> <prompt>` | Set (or replace) the session's heartbeat. Intervals: `90s`, `10m`, `2h`, `1d` (minimum 60s). |
| `/heartbeat` or `/heartbeat status` | Show the heartbeat, its interval, and time to next fire. |
| `/heartbeat pause` | Stop firing without clearing. |
| `/heartbeat resume` | Resume (re-anchors the timer — no instant stale fire). |
| `/heartbeat clear` | Remove the heartbeat. |
`/hb` is an alias. Works on the CLI and gateway platforms (on Slack, use `/hermes heartbeat …`).
## Behavior details
- **Idle-only.** A heartbeat never interrupts a running turn. If the agent is busy when the tick comes due, it fires at the next idle poll.
- **Missed ticks coalesce.** If the session was busy (or the process wasn't running) through several intervals, you get **one** heartbeat turn, not a backlog. The timer re-anchors on every fire.
- **User messages win.** A queued user message always takes priority; the heartbeat waits for the input queue to drain.
- **Cache-safe.** The injected prompt is an ordinary user message. No system-prompt mutation, no toolset change.
- **Persistence.** State lives in `SessionDB.state_meta` keyed by `heartbeat:<session_id>` — it survives `/resume` and rides across context-compression session rotations. Firing requires the owning process (CLI session or gateway) to be running; for schedules that must survive anything, use cron.
- **Don't-invent-work guard.** The injected prompt tells the agent to reply briefly and stop when nothing meaningful changed, so an idle heartbeat doesn't generate busywork.
## Example
```
You: /heartbeat every 15m Check whether the CI run for PR #1234 finished; summarize the result when it does
♥ Heartbeat set (every 15m): Check whether the CI run for PR #1234 finished; ...
[15 minutes of you working on other things in the same session]
Hermes: [Heartbeat — recurring instruction, fires every 15m]
💻 gh pr checks 1234 (1.2s)
CI is still running (14/37 checks complete). Nothing to report yet.
```
When the answer stops changing, `/heartbeat clear` it — or let it keep watch.
website/docs/user-guide/features/honcho.md
+265
View File
@@ -0,0 +1,265 @@
---
sidebar_position: 99
title: "Honcho Memory"
description: "AI-native persistent memory via Honcho — dialectic reasoning, multi-agent user modeling, and deep personalization"
---
# Honcho Memory
[Honcho](https://github.com/plastic-labs/honcho) is an AI-native memory backend that adds dialectic reasoning and deep user modeling on top of Hermes's built-in memory system. Instead of simple key-value storage, Honcho maintains a running model of who the user is — their preferences, communication style, goals, and patterns — by reasoning about conversations after they happen.
:::info Honcho is a Memory Provider Plugin
Honcho is integrated into the [Memory Providers](./memory-providers.md) system. All features below are available through the unified memory provider interface.
:::
## What Honcho Adds
| Capability | Built-in Memory | Honcho |
|-----------|----------------|--------|
| Cross-session persistence | ✔ File-based MEMORY.md/USER.md | ✔ Server-side with API |
| User profile | ✔ Manual agent curation | ✔ Automatic dialectic reasoning |
| Session summary | — | ✔ Session-scoped context injection |
| Multi-agent isolation | — | ✔ Per-peer profile separation |
| Observation modes | — | ✔ Unified or directional observation |
| Conclusions (derived insights) | — | ✔ Server-side reasoning about patterns |
| Search across history | ✔ FTS5 session search | ✔ Semantic search over conclusions |
**Dialectic reasoning**: After each conversation turn (gated by `dialecticCadence`), Honcho analyzes the exchange and derives insights about the user's preferences, habits, and goals. These accumulate over time, giving the agent a deepening understanding that goes beyond what the user explicitly stated. The dialectic supports multi-pass depth (1–3 passes) with automatic cold/warm prompt selection — cold start queries focus on general user facts while warm queries prioritize session-scoped context.
**Session-scoped context**: Base context now includes the session summary alongside the user representation and peer card. This gives the agent awareness of what has already been discussed in the current session, reducing repetition and enabling continuity.
**Multi-agent profiles**: When multiple Hermes instances talk to the same user (e.g., a coding assistant and a personal assistant), Honcho maintains separate "peer" profiles. Each peer sees only its own observations and conclusions, preventing cross-contamination of context.
## Setup
```bash
hermes memory setup # select "honcho" from the provider list
```
Or configure manually:
```yaml
# ~/.hermes/config.yaml
memory:
provider: honcho
```
```bash
echo 'HONCHO_API_KEY=***' >> ~/.hermes/.env
```
Get an API key at [honcho.dev](https://honcho.dev).
## Architecture
### Two-Layer Context Injection
Every turn (in `hybrid` or `context` mode), Honcho assembles two layers of context injected into the system prompt:
1. **Base context** — session summary, user representation, user peer card, AI self-representation, and AI identity card. Refreshed on `contextCadence`. This is the "who is this user" layer.
2. **Dialectic supplement** — LLM-synthesized reasoning about the user's current state and needs. Refreshed on `dialecticCadence`. This is the "what matters right now" layer.
Both layers are concatenated and truncated to the `contextTokens` budget (if set).
### Cold/Warm Prompt Selection
The dialectic automatically selects between two prompt strategies:
- **Cold start** (no base context yet): General query — "Who is this person? What are their preferences, goals, and working style?"
- **Warm session** (base context exists): Session-scoped query — "Given what's been discussed in this session so far, what context about this user is most relevant?"
This happens automatically based on whether base context has been populated.
### Three Orthogonal Config Knobs
Cost and depth are controlled by three independent knobs:
| Knob | Controls | Default |
|------|----------|---------|
| `contextCadence` | Turns between `context()` API calls (base layer refresh) | `1` |
| `dialecticCadence` | Turns between `peer.chat()` LLM calls (dialectic layer refresh) | `2` (recommended 1–5) |
| `dialecticDepth` | Number of `.chat()` passes per dialectic invocation (1–3) | `1` |
These are orthogonal — you can have frequent context refreshes with infrequent dialectic, or deep multi-pass dialectic at low frequency. Example: `contextCadence: 1, dialecticCadence: 5, dialecticDepth: 2` refreshes base context every turn, runs dialectic every 5 turns, and each dialectic run makes 2 passes.
### Dialectic Depth (Multi-Pass)
When `dialecticDepth` > 1, each dialectic invocation runs multiple `.chat()` passes:
- **Pass 0**: Cold or warm prompt (see above)
- **Pass 1**: Self-audit — identifies gaps in the initial assessment and synthesizes evidence from recent sessions
- **Pass 2**: Reconciliation — checks for contradictions between prior passes and produces a final synthesis
Each pass uses a proportional reasoning level (lighter early passes, base level for the main pass). Override per-pass levels with `dialecticDepthLevels` — e.g., `["minimal", "medium", "high"]` for a depth-3 run.
Passes bail out early if the prior pass returned strong signal (long, structured output), so depth 3 doesn't always mean 3 LLM calls.
### Session-Start Prewarm
On session init, Honcho fires a dialectic call in the background at the full configured `dialecticDepth` and hands the result directly to turn 1's context assembly. A single-pass prewarm on a cold peer often returns thin output — multi-pass depth runs the audit/reconcile cycle before the user ever speaks. If prewarm hasn't landed by turn 1, turn 1 falls back to a synchronous call with a bounded timeout.
### Query-Adaptive Reasoning Level
The auto-injected dialectic scales `dialecticReasoningLevel` by query length: +1 level at ≥120 chars, +2 at ≥400, clamped at `reasoningLevelCap` (default `"high"`). Disable with `reasoningHeuristic: false` to pin every auto call to `dialecticReasoningLevel`. Available levels: `minimal`, `low`, `medium`, `high`, `max`.
## Configuration Options
Honcho is configured in `~/.honcho/config.json` (global) or `$HERMES_HOME/honcho.json` (profile-local). The setup wizard handles this for you.
### Self-Hosted Honcho with Authentication
When pointing Hermes at a self-hosted Honcho server, `hermes honcho setup` (and `hermes memory setup`) ask for a **local JWT / bearer token** after the base URL. Paste a JWT signed with the server's `AUTH_JWT_SECRET` (the Honcho compose env var) to enable authenticated access; leave it blank for servers running with `AUTH_USE_AUTH=false`. The local token is stored under the host block (`hosts.<host>.apiKey` in `honcho.json`), separate from any cloud `apiKey`, so you can flip the `Cloud or local?` prompt back to `cloud` later without losing either credential.
### Full Config Reference
| Key | Default | Description |
|-----|---------|-------------|
| `contextTokens` | `null` (uncapped) | Token budget for auto-injected context per turn. Set to an integer (e.g. 1200) to cap. Truncates at word boundaries |
| `contextCadence` | `1` | Minimum turns between `context()` API calls (base layer refresh) |
| `dialecticCadence` | `2` | Minimum turns between `peer.chat()` LLM calls (dialectic layer). Recommended 1–5. In `tools` mode, irrelevant — model calls explicitly |
| `dialecticDepth` | `1` | Number of `.chat()` passes per dialectic invocation. Clamped to 1–3 |
| `dialecticDepthLevels` | `null` | Optional array of reasoning levels per pass, e.g. `["minimal", "low", "medium"]`. Overrides proportional defaults |
| `dialecticReasoningLevel` | `'low'` | Base reasoning level: `minimal`, `low`, `medium`, `high`, `max` |
| `dialecticDynamic` | `true` | When `true`, model can override reasoning level per-call via tool param |
| `dialecticMaxChars` | `600` | Max chars of dialectic result injected into system prompt |
| `recallMode` | `'hybrid'` | `hybrid` (auto-inject + tools), `context` (inject only), `tools` (tools only) |
| `writeFrequency` | `'async'` | When to flush messages: `async` (background thread), `turn` (sync), `session` (batch on end), or integer N |
| `saveMessages` | `true` | Whether to persist messages to Honcho API |
| `observationMode` | `'directional'` | `directional` (all on) or `unified` (shared pool). Override with `observation` object for granular control |
| `messageMaxChars` | `25000` | Max chars per message sent via `add_messages()`. Chunked if exceeded |
| `dialecticMaxInputChars` | `10000` | Max chars for dialectic query input to `peer.chat()` |
| `sessionStrategy` | `'per-directory'` | `per-directory`, `per-repo`, `per-session`, or `global` |
| `pinUserPeer` | `false` | Gateway only. When `true`, every platform user collapses to `peerName` |
| `userPeerAliases` | `{}` | Gateway only. Map of runtime IDs to peers (`{"7654321": "alice"}`). Many-to-one |
| `runtimePeerPrefix` | `""` | Gateway only. Namespaces unknown runtime IDs (`telegram_7654321`) when no alias matches |
**Session strategy** controls how Honcho sessions map to your work:
- `per-session` — each `hermes` run gets a fresh session. Clean starts, memory via tools. Recommended for new users.
- `per-directory` — one Honcho session per working directory. Context accumulates across runs.
- `per-repo` — one session per git repository.
- `global` — single session across all directories.
**Recall mode** controls how memory flows into conversations:
- `hybrid` — context auto-injected into system prompt AND tools available (model decides when to query).
- `context` — auto-injection only, tools hidden.
- `tools` — tools only, no auto-injection. Agent must explicitly call `honcho_reasoning`, `honcho_search`, etc.
**Settings per recall mode:**
| Setting | `hybrid` | `context` | `tools` |
|---------|----------|-----------|---------|
| `writeFrequency` | flushes messages | flushes messages | flushes messages |
| `contextCadence` | gates base context refresh | gates base context refresh | irrelevant — no injection |
| `dialecticCadence` | gates auto LLM calls | gates auto LLM calls | irrelevant — model calls explicitly |
| `dialecticDepth` | multi-pass per invocation | multi-pass per invocation | irrelevant — model calls explicitly |
| `contextTokens` | caps injection | caps injection | irrelevant — no injection |
| `dialecticDynamic` | gates model override | N/A (no tools) | gates model override |
In `tools` mode, the model is fully in control — it calls `honcho_reasoning` when it wants, at whatever `reasoning_level` it picks. Cadence and budget settings only apply to modes with auto-injection (`hybrid` and `context`).
## Gateway Identity Mapping
These settings only matter when you run the [Hermes gateway](../../developer-guide/gateway-internals.md) — the one entrypoint where users arrive with platform-native runtime IDs (Telegram UID, Discord snowflake, Slack user). CLI, TUI, and desktop sessions have no runtime ID and always resolve to `peerName`, so off-gateway these keys do nothing.
The setup wizard detects whether a gateway platform is connected and skips this step entirely if not. When it runs, it asks one question — *who talks to this gateway?* — and derives the keys:
| Answer | Result |
|--------|--------|
| **just me** | `pinUserPeer: true` — every non-agent gateway user collapses to your peer. Pin overrides all aliases, so pick this only when no user-side identity needs its own peer. If separate agents reach the gateway and each needs a distinct peer, do **not** pin — leave `pinUserPeer: false` and map them via `userPeerAliases` (the `[e]` editor) instead |
| **me + other people** (pooled) | `pinUserPeer: false` + `userPeerAliases` mapping your runtime IDs to `peerName` — you stay on your shared history, others get their own peers |
| **only other people** | `pinUserPeer: false`, optional `runtimePeerPrefix` — each user gets their own peer |
Pick `[e]` at the prompt to set the three keys directly instead.
The resolver tries the keys top-down, first match wins: `pinUserPeer` → `userPeerAliases[id]` → `runtimePeerPrefix + id` → raw runtime ID → `peerName` → session-key fallback.
:::warning Un-pinning orphans pooled memory
Flipping `pinUserPeer` from `true` to `false` does not migrate data — memory accumulated under `peerName` stays there, and platform users resolve to fresh, empty peers. To keep your own continuity, choose the **pooled** path so your runtime IDs alias back to `peerName`. The wizard offers this steer automatically when it detects the transition.
:::
:::note Deprecated key
`pinPeerName` is a legacy alias for `pinUserPeer` — still read for back-compat (`pinUserPeer` wins where both are set), never written. Re-running setup migrates it onto the canonical key.
:::
## Observation (Directional vs. Unified)
Honcho models a conversation as peers exchanging messages. Each peer has two observation toggles that map 1:1 to Honcho's `SessionPeerConfig`:
| Toggle | Effect |
|--------|--------|
| `observeMe` | Honcho builds a representation of this peer from its own messages |
| `observeOthers` | This peer observes the other peer's messages (feeds cross-peer reasoning) |
Two peers × two toggles = four flags. `observationMode` is a shorthand preset:
| Preset | User flags | AI flags | Semantics |
|--------|-----------|----------|-----------|
| `"directional"` (default) | me: on, others: on | me: on, others: on | Full mutual observation. Enables cross-peer dialectic — "what does the AI know about the user, based on what the user said and the AI replied." |
| `"unified"` | me: on, others: off | me: off, others: on | Shared-pool semantics — the AI observes the user's messages only, the user peer only self-models. Single-observer pool. |
Override the preset with an explicit `observation` block for per-peer control:
```json
"observation": {
"user": { "observeMe": true, "observeOthers": true },
"ai": { "observeMe": true, "observeOthers": false }
}
```
Common patterns:
| Intent | Config |
|--------|--------|
| Full observation (most users) | `"observationMode": "directional"` |
| AI shouldn't re-model the user from its own replies | `"ai": {"observeMe": true, "observeOthers": false}` |
| Strong persona the AI peer shouldn't update from self-observation | `"ai": {"observeMe": false, "observeOthers": true}` |
Server-side toggles set via the [Honcho dashboard](https://app.honcho.dev) win over local defaults — Hermes syncs them back at session init.
## Tools
When Honcho is active as the memory provider, five tools become available:
| Tool | Purpose |
|------|---------|
| `honcho_profile` | Read or update peer card — pass `card` (list of facts) to update, omit to read |
| `honcho_search` | Semantic search over context — raw excerpts, no LLM synthesis |
| `honcho_context` | Full session context — summary, representation, card, recent messages |
| `honcho_reasoning` | Synthesized answer from Honcho's LLM — pass `reasoning_level` (minimal/low/medium/high/max) to control depth |
| `honcho_conclude` | Create or delete conclusions — pass `conclusion` to create, `delete_id` to remove (PII only) |
## CLI Commands
The `hermes honcho` subcommand is **only registered when Honcho is the active memory provider** (`memory.provider: honcho` in `config.yaml`). On a fresh install, configure Honcho directly with `hermes memory setup honcho` (or run `hermes memory setup` and pick it from the list); the `hermes honcho` subcommand then appears on the next invocation.
```bash
hermes memory setup honcho # Configure Honcho directly (works before activation)
hermes honcho status # Connection status, config, and key settings
hermes honcho setup # Redirects to `hermes memory setup` (post-activation alias)
hermes honcho strategy # Show or set session strategy (per-session/per-directory/per-repo/global)
hermes honcho peer # Show or update peer names + dialectic reasoning level
hermes honcho mode # Show or set recall mode (hybrid/context/tools)
hermes honcho tokens # Show or set token budget for context and dialectic
hermes honcho identity # Seed or show the AI peer's Honcho identity
hermes honcho sync # Sync Honcho config to all existing profiles
hermes honcho peers # Show peer identities across all profiles
hermes honcho sessions # List known Honcho session mappings
hermes honcho map # Map current directory to a Honcho session name
hermes honcho enable # Enable Honcho for the active profile
hermes honcho disable # Disable Honcho for the active profile
hermes honcho migrate # Step-by-step migration guide from openclaw-honcho
```
## Migrating from `hermes honcho`
If you previously used the standalone `hermes honcho setup`:
1. Your existing configuration (`honcho.json` or `~/.honcho/config.json`) is preserved
2. Your server-side data (memories, conclusions, user profiles) is intact
3. Set `memory.provider: honcho` in config.yaml to reactivate
No re-login or re-setup needed. Run `hermes memory setup` and select "honcho" — the wizard detects your existing config.
## Full Documentation
See [Memory Providers — Honcho](./memory-providers.md#honcho) for the complete reference.
website/docs/user-guide/features/hooks.md
+1928
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/features/image-generation.md
+284
View File
@@ -0,0 +1,284 @@
---
title: Image Generation
description: Generate images via FAL.ai — 11 models including FLUX 2, GPT Image (1.5 & 2), Nano Banana Pro, Ideogram, Recraft V4 Pro, Krea 2, and more, selectable via `hermes tools`.
sidebar_label: Image Generation
sidebar_position: 6
---
# Image Generation
Hermes Agent generates images from text prompts via FAL.ai. Eleven models are supported out of the box, each with different speed, quality, and cost tradeoffs. The active model is user-configurable via `hermes tools` and persists in `config.yaml`.
## Supported Models
| Model | Speed | Strengths | Price |
|---|---|---|---|
| `fal-ai/flux-2/klein/9b` *(default)* | `<1s` | Fast, crisp text | $0.006/MP |
| `fal-ai/flux-2-pro` | ~6s | Studio photorealism | $0.03/MP |
| `fal-ai/z-image/turbo` | ~2s | Bilingual EN/CN, 6B params | $0.005/MP |
| `fal-ai/nano-banana-pro` | ~8s | Gemini 3 Pro, reasoning depth, text rendering | $0.15/image (1K) |
| `fal-ai/gpt-image-1.5` | ~15s | Prompt adherence | $0.034/image |
| `fal-ai/gpt-image-2` | ~20s | SOTA text rendering + CJK, world-aware photorealism | $0.04–0.06/image |
| `fal-ai/ideogram/v3` | ~5s | Best typography | $0.03–0.09/image |
| `fal-ai/recraft/v4/pro/text-to-image` | ~8s | Design, brand systems, production-ready | $0.25/image |
| `fal-ai/qwen-image` | ~12s | LLM-based, complex text | $0.02/MP |
| `fal-ai/krea/v2/medium/text-to-image` | ~15-25s | Illustration, anime, painting, expressive/artistic styles | $0.030–0.035/image |
| `fal-ai/krea/v2/large/text-to-image` | ~25-60s | Photorealism, raw textured looks (motion blur, grain, film) | $0.060–0.065/image |
Prices are FAL's pricing at time of writing; check [fal.ai](https://fal.ai/) for current numbers.
## Setup
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, you can use image generation through the **[Tool Gateway](tool-gateway.md)** without a FAL API key. Your model selection persists across both paths. New installs can run `hermes setup --portal` to log in and turn on every gateway tool at once; existing installs can pick **Nous Subscription** as the image-gen backend via `hermes tools`.
If the managed gateway returns `HTTP 4xx` for a specific model, that model isn't yet proxied on the portal side — the agent will tell you so, with remediation steps (switch to FAL.ai in `hermes tools` with your own `FAL_KEY` for direct access, or pick a different model).
:::
### Get a FAL API Key
1. Sign up at [fal.ai](https://fal.ai/)
2. Generate an API key from your dashboard
### Configure and Pick a Model
Run the tools command:
```bash
hermes tools
```
Navigate to **🎨 Image Generation**, pick your backend (Nous Subscription or FAL.ai), then the picker shows all supported models in a column-aligned table — arrow keys to navigate, Enter to select:
```
Model Speed Strengths Price
fal-ai/flux-2/klein/9b <1s Fast, crisp text $0.006/MP ← currently in use
fal-ai/flux-2-pro ~6s Studio photorealism $0.03/MP
fal-ai/z-image/turbo ~2s Bilingual EN/CN, 6B $0.005/MP
...
```
Your selection is saved to `config.yaml`:
```yaml
image_gen:
provider: fal # `nous` if you picked Nous Subscription
model: fal-ai/flux-2/klein/9b
max_parallel_requests: 4 # concurrent images in one tool-call batch
```
`image_gen.provider` is the single selection key: `nous` routes through the managed Tool Gateway; a vendor name (`fal`, `openai`, `xai`, `krea`, ...) goes direct with your own key. The runtime always follows this stored selection — a `FAL_KEY` in `.env` is ignored while `provider: nous`, and `provider: fal` without `FAL_KEY` errors with `image_gen is configured to use fal (set via hermes tools), but FAL_KEY is not set. Run 'hermes tools' to change it.` rather than silently rerouting. Change providers via `hermes tools`, not by adding/removing keys. (The old `use_gateway` boolean is legacy — still read as `nous` when `true`, but never written anymore.)
`max_parallel_requests` defaults to `4`. Hermes clamps it to at least one and
to the global tool-worker limit, so image providers receive bounded parallel
requests without allowing an image batch to bypass the agent's concurrency cap.
### OpenRouter: the full Image API catalog
With `image_gen.provider: openrouter`, the model picker lists OpenRouter's
entire live image catalog — the dedicated
[Image API](https://openrouter.ai/docs/guides/overview/multimodal/image-generation)
models (Seedream, FLUX.2, Recraft, Qwen Image, MAI, Krea, Riverflow, Grok
Imagine, and more — 40+ ids) merged with the chat-completions image models.
The catalog is fetched live from `GET /images/models` and `GET /models`, so
new models appear in the picker as soon as OpenRouter serves them; no Hermes
update needed. Generation routes each model to the surface that serves it
(dedicated `POST /images/generations` vs chat-completions) automatically.
Nous Portal proxies the chat-completions protocol only, so its picker offers
the chat-served models.
Optional per-request knobs for Image API models go under the scoped config
section (or `OPENROUTER_IMAGE_API_*` env vars):
```yaml
image_gen:
provider: openrouter
model: bytedance-seed/seedream-4.5
openrouter:
resolution: 2K # model-dependent: 1K / 2K / 4K
quality: high # gpt-image models
output_format: png
```
### GPT-Image Quality
The `fal-ai/gpt-image-1.5` and `fal-ai/gpt-image-2` request quality is pinned to `medium` (~$0.034–$0.06/image at 1024×1024). We don't expose the `low` / `high` tiers as a user-facing option so that Nous Portal billing stays predictable across all users — the cost spread between tiers is 3–22×. If you want a cheaper option, pick Klein 9B or Z-Image Turbo; if you want higher quality, use Nano Banana Pro or Recraft V4 Pro.
### Meta Model API: Muse Image
With `image_gen.provider: meta-ai`, images are generated through the
[Meta Model API](https://api.meta.ai) (`https://api.meta.ai/v1`), the same
OpenAI-compatible endpoint that serves the Muse Spark chat models. It is the
image-gen companion to the bundled `meta-ai` chat provider.
| Model | Speed | Strengths | Price |
|---|---|---|---|
| `muse-image-1.0` *(default)* | ~10s | Meta Model API image generation | $0.01/image |
```yaml
image_gen:
provider: meta-ai
model: muse-image-1.0
```
Auth reuses the same env vars as the Meta chat provider — `MODEL_API_KEY`
(Meta's documented name), with `META_API_KEY` / `META_MODEL_API_KEY` accepted
as aliases. Set `META_BASE_URL` to point at a proxy or alternate host. Text-to-image
only for now; responses are saved to `$HERMES_HOME/cache/images/`.
## Usage
The agent-facing schema is intentionally minimal — the model picks up whatever you've configured:
```
Generate an image of a serene mountain landscape with cherry blossoms
```
```
Create a square portrait of a wise old owl — use the typography model
```
```
Make me a futuristic cityscape, landscape orientation
```
## Image-to-Image / Editing
The same `image_generate` tool also **edits existing images** when the active
model supports it — pass a source image and the backend routes to its editing
endpoint automatically (mirrors how `video_generate` handles image-to-video).
Omit the source image and it's plain text-to-image.
```
Take this photo and make it a rainy Tokyo street at night → <image>
```
```
Blend these two product shots into one hero image → <image1> <image2>
```
Two inputs drive the edit:
- **`image_url`** — the primary source image to edit/transform (public URL or local path).
- **`reference_image_urls`** — additional style/composition references (capped per-model).
### Which backends support editing
| Backend | Image-to-image | Reference cap | How |
|---|---|---|---|
| **FAL.ai** (edit-capable models below) | ✓ | up to 9 | routes to the model's `/edit` endpoint |
| **OpenAI** (`gpt-image-2`) | ✓ | up to 16 | `images.edit()` |
| **xAI** (Grok Imagine) | ✓ | 1 | `/v1/images/edits` (`grok-imagine-image-quality`) |
| **Krea** (`Krea 2`) | ✓ | up to 10 | reference-guided generation (`image_style_references`) |
| **OpenAI (Codex auth)** | ✓ | up to 16 | Codex Responses `image_generation` tool with `input_image` content parts |
| **OpenRouter** (Image API models) | ✓ | up to 14–16 (per model) | `input_references` on `POST /images/generations`; chat-served models use `image_url` content parts (up to 3) |
FAL models with an editing endpoint: `flux-2/klein/9b`, `flux-2-pro`,
`nano-banana-pro`, `gpt-image-1.5`, `gpt-image-2`, `ideogram/v3`, and
`qwen-image`. Pure text-to-image FAL models (`z-image/turbo`, `recraft`,
`krea/*`) reject image inputs with a clear error pointing you at an
edit-capable model.
:::note OpenAI (Codex auth) is best-effort
The Codex surface (`chatgpt.com/backend-api/codex`) hosts `image_generation`
as a tool the chat model may call, and Hermes cannot force the call — the
backend rejects every `tool_choice` shape for hosted tools, so the request
relies on instructions to steer the model. When the host model declines to
invoke the tool, the call fails with `empty_response`. Whether the hosted
image tool is reachable at all has also been reported to vary between
accounts. If you need image generation to work deterministically, configure
the **OpenAI** (API key), **FAL**, or **xAI** backend instead.
:::
The active model's editing capability is surfaced in the tool description at
runtime, so the agent knows whether `image_url` will be honored before it
calls the tool.
## Aspect Ratios
Every model accepts the same three aspect ratios from the agent's perspective. Internally, each model's native size spec is filled in automatically:
| Agent input | image_size (flux/z-image/qwen/recraft/ideogram) | aspect_ratio (nano-banana-pro) | image_size (gpt-image-1.5) | image_size (gpt-image-2) |
|---|---|---|---|---|
| `landscape` | `landscape_16_9` | `16:9` | `1536x1024` | `landscape_4_3` (1024×768) |
| `square` | `square_hd` | `1:1` | `1024x1024` | `square_hd` (1024×1024) |
| `portrait` | `portrait_16_9` | `9:16` | `1024x1536` | `portrait_4_3` (768×1024) |
GPT Image 2 maps to 4:3 presets rather than 16:9 because its minimum pixel count is 655,360 — the `landscape_16_9` preset (1024×576 = 589,824) would be rejected.
This translation happens in `_build_fal_payload()` — agent code never has to know about per-model schema differences.
## Upscaling
### Opt-in only
No model upscales by default. Modern image models emit their best quality
natively, and the available upscalers are *creative* enhancers (diffusion
passes) that can subtly redraw content — degrading rendered text, faces, and
fine detail. Upscaling only runs when the agent explicitly requests it.
### The `upscale` parameter (per-call opt-in)
- `upscale: true` — chain a high-resolution pass after generation:
| Backend | Upscaler |
|---|---|
| **FAL.ai** | Clarity Upscaler (2×, +$0.03/MP) |
| **Krea** | Krea Enhance (2×, up to 8K ceiling) |
| Other backends | no upscaler; native resolution returned |
- `upscale: false` / omitted — native resolution (the default)
`video_generate` also accepts `upscale: true` on the FAL backend, chaining
ByteDance's **SeedVR2** video upscaler (2×, $0.001/MP of output video) after
generation.
When the FAL image pass runs, it uses these settings:
| Setting | Value |
|---|---|
| Upscale factor | 2× |
| Creativity | 0.35 |
| Resemblance | 0.6 |
| Guidance scale | 4 |
| Inference steps | 18 |
If upscaling fails (network issue, rate limit), the original image is returned automatically. The response reports `upscaled: true/false` so the agent knows which resolution it got.
## How It Works Internally
1. **Model resolution** — `_resolve_fal_model()` reads `image_gen.model` from `config.yaml`, falls back to the `FAL_IMAGE_MODEL` env var, then to `fal-ai/flux-2/klein/9b`.
2. **Payload building** — `_build_fal_payload()` translates your `aspect_ratio` into the model's native format (preset enum, aspect-ratio enum, or GPT literal), merges the model's default params, applies any caller overrides, then filters to the model's `supports` whitelist so unsupported keys are never sent.
3. **Submission** — `_submit_fal_request()` routes via direct FAL credentials or the managed Nous gateway, according to the stored `image_gen.provider` selection.
4. **Upscaling** — runs only when the agent passed `upscale: true`; every model's catalog default is off.
5. **Delivery** — final image URL returned to the agent, which emits a `MEDIA:<url>` tag that platform adapters convert to native media.
## Debugging
Enable debug logging:
```bash
export IMAGE_TOOLS_DEBUG=true
```
Debug logs go to `./logs/image_tools_debug_<session_id>.json` with per-call details (model, parameters, timing, errors).
## Platform Delivery
| Platform | Delivery |
|---|---|
| **CLI** | Image URL printed as markdown `![](url)` — click to open |
| **Telegram** | Photo message with the prompt as caption |
| **Discord** | Embedded in a message |
| **Slack** | URL unfurled by Slack |
| **WhatsApp** | Media message |
| **Others** | URL in plain text |
## Limitations
- **Requires credentials** for the active backend (FAL `FAL_KEY` / Nous Subscription, `OPENAI_API_KEY`, xAI OAuth, `KREA_API_KEY`)
- **Editing is model-dependent** — image-to-image works only on edit-capable models (see the table above); text-to-image-only models reject image inputs with a clear error
- **Temporary URLs** — backends return hosted URLs that expire after hours/days; Hermes materializes them to the local cache so delivery still works after expiry
- **Per-model constraints** — some models don't support `seed`, `num_inference_steps`, etc. The `supports` / `edit_supports` filter silently drops unsupported params; this is expected behavior
website/docs/user-guide/features/kanban-tutorial.md
+310
View File
@@ -0,0 +1,310 @@
# Kanban tutorial
A walkthrough of the four use-cases the Hermes Kanban system was designed for, with the dashboard open in a browser. If you haven't read the [Kanban overview](./kanban) yet, start there — this assumes you know what a task, run, assignee, and dispatcher are.
## Setup
```bash
hermes kanban init # optional; first `hermes kanban <anything>` auto-inits
hermes dashboard # opens http://127.0.0.1:9119 in your browser
# click Kanban in the left nav
```
The dashboard is the most comfortable place for **you** to watch the system. Agent workers the dispatcher spawns never see the dashboard or the CLI — they drive the board through a dedicated `kanban_*` [toolset](./kanban#how-workers-interact-with-the-board) (`kanban_show`, `kanban_list`, `kanban_complete`, `kanban_block`, `kanban_heartbeat`, `kanban_comment`, `kanban_attach`, `kanban_attach_url`, `kanban_attachments`, `kanban_create`, `kanban_link`, `kanban_unblock`). All three surfaces — dashboard, CLI, worker tools — route through the same per-board SQLite DB (`~/.hermes/kanban.db` for the default board, `~/.hermes/kanban/boards/<slug>/kanban.db` for any board you create later), so each board is consistent no matter which side of the fence a change came from.
This tutorial uses the `default` board throughout. If you want multiple isolated queues (one per project / repo / domain), see [Boards (multi-project)](./kanban#boards-multi-project) in the overview — the same CLI / dashboard / worker flows apply per board, and workers physically cannot see tasks on other boards.
Throughout the tutorial, **code blocks labelled `bash` are commands *you* run.** Code blocks labelled `# worker tool calls` are what the spawned worker's model emits as tool calls — shown here so you can see the loop end-to-end, not because you'd ever run them yourself.
## The board at a glance
![Kanban board overview](/img/kanban-tutorial/01-board-overview.png)
Six columns, left to right:
- **Triage** — raw ideas. By default the dispatcher auto-runs the **decomposer** on tasks here: the built-in decomposer uses `auxiliary.kanban_decomposer`, reads your profile roster + descriptions, and produces a graph of child tasks routed to the best-fit specialists. The original task is held alive as the parent so its assignee (`kanban.orchestrator_profile`, or the active default profile when unset) wakes back up to judge completion when everything finishes. Flip the **Orchestration: Auto/Manual** pill at the top of the kanban page to switch modes. In Manual mode click **⚗ Decompose** on a card, or run `hermes kanban decompose <id>` / `/kanban decompose <id>`. For single tasks that don't need fan-out, **✨ Specify** does a one-shot spec rewrite (goal, approach, acceptance criteria) and promotes to `todo`. Configure the models under `auxiliary.kanban_decomposer` and `auxiliary.triage_specifier` in `config.yaml`. See [Auto vs Manual orchestration](./kanban#auto-vs-manual-orchestration) in the main Kanban guide.
- **Todo** — created but waiting on dependencies, or not yet assigned.
- **Ready** — assigned and waiting for the dispatcher to claim.
- **In progress** — a worker is actively running the task. With "Lanes by profile" on (the default), this column sub-groups by assignee so you can see at a glance what each worker is doing.
- **Blocked** — a worker asked for human input, or the circuit breaker tripped.
- **Done** — completed.
The top bar has filters for search, tenant, and assignee, plus a `Lanes by profile` toggle and a `Nudge dispatcher` button that runs one dispatch tick right now instead of waiting for the daemon's next interval. Clicking any card opens its drawer on the right.
### Flat view
If the profile lanes are noisy, toggle "Lanes by profile" off and the In Progress column collapses to a single flat list ordered by claim time:
![Board with lanes by profile off](/img/kanban-tutorial/02-board-flat.png)
## Story 1 — Solo dev shipping a feature
You're building a feature. Classic flow: design a schema, implement the API, write the tests. Three tasks with parent→child dependencies.
```bash
SCHEMA=$(hermes kanban create "Design auth schema" \
--assignee backend-dev --tenant auth-project --priority 2 \
--body "Design the user/session/token schema for the auth module." \
--json | jq -r .id)
API=$(hermes kanban create "Implement auth API endpoints" \
--assignee backend-dev --tenant auth-project --priority 2 \
--parent $SCHEMA \
--body "POST /register, POST /login, POST /refresh, POST /logout." \
--json | jq -r .id)
hermes kanban create "Write auth integration tests" \
--assignee qa-dev --tenant auth-project --priority 2 \
--parent $API \
--body "Cover happy path, wrong password, expired token, concurrent refresh."
```
Because `API` has `SCHEMA` as its parent, and `tests` has `API` as its parent, only `SCHEMA` starts in `ready`. The other two sit in `todo` until their parents complete. This is the dependency promotion engine doing its job — no other worker will pick up the test-writing until there's an API to test.
On the next dispatcher tick (60s by default, or immediately if you hit **Nudge dispatcher**) the `backend-dev` profile spawns as a worker with `HERMES_KANBAN_TASK=$SCHEMA` in its env. Here's what the worker's tool-call loop looks like from inside the agent:
```python
# worker tool calls — NOT commands you run
kanban_show()
# → returns title, body, worker_context, parents, prior attempts, comments
# (worker reads worker_context, uses terminal/file tools to design the schema,
# write migrations, run its own checks, commit — the real work happens here)
kanban_heartbeat(note="schema drafted, writing migrations now")
kanban_complete(
summary="users(id, email, pw_hash), sessions(id, user_id, jti, expires_at); "
"refresh tokens stored as sessions with type='refresh'",
metadata={
"changed_files": ["migrations/001_users.sql", "migrations/002_sessions.sql"],
"decisions": ["bcrypt for hashing", "JWT for session tokens",
"7-day refresh, 15-min access"],
},
)
```
`kanban_show` defaults `task_id` to `$HERMES_KANBAN_TASK`, so the worker doesn't need to know its own id. `kanban_complete` writes the summary + metadata onto the current `task_runs` row, closes that run, and transitions the task to `done` — all in one atomic hop through `kanban_db`.
When `SCHEMA` hits `done`, the dependency engine promotes `API` to `ready` automatically. The API worker, when it picks up, will call `kanban_show()` and see `SCHEMA`'s summary and metadata attached to the parent handoff — so it knows the schema decisions without re-reading a long design doc.
Click the completed schema task on the board and the drawer shows everything:
![Solo dev — completed schema task drawer](/img/kanban-tutorial/03-drawer-schema-task.png)
The Run History section at the bottom is the key addition. One attempt: outcome `completed`, worker `@backend-dev`, duration, timestamp, and the handoff summary in full. The metadata blob (`changed_files`, `decisions`) is stored on the run too and surfaced to any downstream worker that reads this parent.
You can inspect the same data from your terminal at any time — these commands are **you** peeking at the board, not the worker:
```bash
hermes kanban show $SCHEMA
hermes kanban runs $SCHEMA
# # OUTCOME PROFILE ELAPSED STARTED
# 1 completed backend-dev 0s 2026-04-27 19:34
# → users(id, email, pw_hash), sessions(id, user_id, jti, expires_at); refresh tokens ...
```
## Story 2 — Fleet farming
You have three workers (a translator, a transcriber, a copywriter) and a pile of independent tasks. You want all three pulling in parallel and making visible progress. This is the simplest kanban use-case and the one the original design optimized for.
Create the work:
```bash
for lang in Spanish French German; do
hermes kanban create "Translate homepage to $lang" \
--assignee translator --tenant content-ops
done
for i in 1 2 3 4 5; do
hermes kanban create "Transcribe Q3 customer call #$i" \
--assignee transcriber --tenant content-ops
done
for sku in 1001 1002 1003 1004; do
hermes kanban create "Generate product description: SKU-$sku" \
--assignee copywriter --tenant content-ops
done
```
Start the gateway and walk away — it hosts the embedded dispatcher
that picks up all three specialist profiles' tasks on the same
kanban.db:
```bash
hermes gateway start
```
Now filter the board to `content-ops` (or just search for "Transcribe") and you get this:
![Fleet view filtered to transcribe tasks](/img/kanban-tutorial/07-fleet-transcribes.png)
Two transcribes done, one running, two ready waiting for the next dispatcher tick. The In Progress column is grouped by profile (the "Lanes by profile" default) so you see each worker's active task without scanning a mixed list. The dispatcher will promote the next ready task to running as soon as the current one completes. With three daemons working on three assignee pools in parallel, the whole content queue drains without further human input.
**Everything Story 1 said about structured handoff still applies here.** A translator worker completing a call emits `kanban_complete(summary="translated 4 pages, style matched existing marketing voice", metadata={"duration_seconds": 720, "tokens_used": 2100})` — useful for analytics and for any downstream task that depends on this one.
## Story 3 — Role pipeline with retry
This is where Kanban earns its keep over a flat TODO list. A PM writes a spec. An engineer implements it. A reviewer rejects the first attempt. The engineer tries again with changes. The reviewer approves.
The dashboard view, filtered by `auth-project`:
![Pipeline view for a multi-role feature](/img/kanban-tutorial/08-pipeline-auth.png)
The screenshot uses the **pre-created downstream card** model: the implementation card has a dedicated reviewer child. In that model the engineer must call `kanban_complete` when implementation is ready so the reviewer child can leave `todo`. Never block the implementation parent merely to ask for review.
For workflows where the same card owns implementation and review, use the first-class review lifecycle instead. The full implement → review → changes → re-review choreography is:
```python
# --- Engineer: first implementation attempt ---
kanban_show()
# (write code, run tests, prepare the candidate)
kanban_request_review(
summary="implemented reset flow; candidate is ready for review",
metadata={"changed_files": ["auth/reset.py"], "tests_run": 8},
reviewer="reviewer",
)
# → the same card enters review; the implementation run closes as
# outcome='review_requested'
# --- Reviewer: request concrete changes ---
kanban_show()
# (inspect the handoff and candidate)
kanban_request_changes(
reason="Add password-strength validation and make reset tokens single-use."
)
# → the review run closes as outcome='changes_requested'; the card returns
# to backend-dev in ready/todo without touching block-loop accounting
# --- Engineer: second implementation attempt ---
kanban_show() # prior review evidence is in worker_context
# (apply feedback and re-run tests)
kanban_request_review(
summary="added zxcvbn validation and single-use reset tokens",
metadata={
"changed_files": [
"auth/reset.py",
"auth/tests/test_reset.py",
"migrations/003_single_use_reset_tokens.sql",
],
"tests_run": 11,
"review_iteration": 2,
},
reviewer="reviewer",
)
# --- Reviewer: approve ---
kanban_complete(summary="review passed; acceptance criteria verified")
# → done
```
The task's run history now records `review_requested → changes_requested → review_requested → completed`. Each attempt has its own actor, summary, metadata, and outcome, so the second engineer sees exactly what the reviewer rejected and the final approval remains auditable. `kanban_block` is reserved for a real external escalation (missing access, a product decision, unavailable infrastructure), not normal review feedback.
If you intentionally use the downstream-card model shown in the screenshot, the reviewer opens `Review password reset PR` after its implementation parent completes:
![Reviewer's drawer view of the pipeline](/img/kanban-tutorial/09-drawer-pipeline-review.png)
The reviewer card's `worker_context` includes the completed implementation handoff. That is a separate card workflow; do not combine it with same-card `kanban_request_review` or you will duplicate the review lane.
## Story 4 — Circuit breaker and crash recovery
Real workers fail. Missing credentials, OOM kills, transient network errors. The dispatcher has two lines of defense: a **circuit breaker** that auto-blocks after N consecutive failures so the board doesn't thrash forever, and **crash detection** that reclaims a task whose worker PID went away before its TTL expired.
### Circuit breaker — permanent-looking failure
A deploy task that can't spawn its worker because `AWS_ACCESS_KEY_ID` isn't set in the profile's environment:
```bash
hermes kanban create "Deploy to staging (missing creds)" \
--assignee deploy-bot --tenant ops \
--max-retries 3
```
The dispatcher tries to spawn the worker. Spawn fails (`RuntimeError: AWS_ACCESS_KEY_ID not set`). The dispatcher releases the claim, increments a failure counter, and tries again next tick. Because this example sets `--max-retries 3`, the circuit trips after three consecutive failures: the task goes to `blocked` with outcome `gave_up`. If you omit the flag, Hermes uses `kanban.failure_limit` (default: 2). No more retries until a human unblocks it.
Click the blocked task:
![Circuit breaker — 2 spawn_failed + 1 gave_up](/img/kanban-tutorial/11-drawer-gave-up.png)
Three runs, all with the same error on the `error` field. The first two are `spawn_failed` (retryable), the third is `gave_up` (terminal). The event log above shows the full sequence: `created → claimed → spawn_failed → claimed → spawn_failed → claimed → gave_up`.
On the terminal:
```bash
hermes kanban runs t_ef5d
# # OUTCOME PROFILE ELAPSED STARTED
# 1 spawn_failed deploy-bot 0s 2026-04-27 19:34
# ! AWS_ACCESS_KEY_ID not set in deploy-bot env
# 2 spawn_failed deploy-bot 0s 2026-04-27 19:34
# ! AWS_ACCESS_KEY_ID not set in deploy-bot env
# 3 gave_up deploy-bot 0s 2026-04-27 19:34
# ! AWS_ACCESS_KEY_ID not set in deploy-bot env
```
If Telegram / Discord / Slack is wired in, a gateway notification fires on the `gave_up` event so you hear about the outage without having to check the board.
### Crash recovery — worker dies mid-flight
Sometimes the spawn succeeds but the worker process dies later — segfault, OOM, `systemctl stop`. The dispatcher polls `kill(pid, 0)` and detects the dead pid; the claim releases, the task goes back to `ready`, and the next tick gives it to a fresh worker.
The example in the seed data is a migration that was running out of memory:
```bash
# Worker claims, starts scanning 2.4M rows, OOM kills it at ~2.3M
# Dispatcher detects dead pid, releases claim, increments attempt counter
# Retry with a chunked strategy succeeds
```
The drawer shows the full two-attempt history:
![Crash and recovery — 1 crashed + 1 completed](/img/kanban-tutorial/06-drawer-crash-recovery.png)
Run 1 — `crashed`, with the error `OOM kill at row 2.3M (process 99999 gone)`. Run 2 — `completed`, with `"strategy": "chunked with LIMIT + WHERE id > last_id"` in its metadata. The retrying worker saw the crash of run 1 in its context and picked a safer strategy; the metadata makes it obvious to a future observer (or postmortem writer) what changed.
## Structured handoff — why `summary` and `metadata` matter
In every story above, workers called `kanban_complete(summary=..., metadata=...)` at the end. That's not decoration — it's the primary handoff channel between stages of a workflow.
When a worker on task B is spawned and calls `kanban_show()`, the `worker_context` it gets back includes:
- B's **prior attempts** (previous runs: outcome, summary, error, metadata) so a retrying worker doesn't repeat a failed path.
- **Parent task results** — for each parent, the most-recent completed run's summary and metadata — so downstream workers see why and how the upstream work was done.
This replaces the "dig through comments and the work output" dance that plagues flat kanban systems. A PM writes acceptance criteria in the spec's metadata, and the engineer's worker sees them structurally in the parent handoff. An engineer records which tests they ran and how many passed, and the reviewer's worker has that list in hand before opening a diff.
The bulk-close guard exists because this data is per-run. `hermes kanban complete a b c --summary X` (you, from the CLI) is refused — copy-pasting the same summary to three tasks is almost always wrong. Bulk close without the handoff flags still works for the common "I finished a pile of admin tasks" case. The tool surface doesn't expose a bulk variant at all; `kanban_complete` is always single-task-at-a-time for the same reason.
## Follow-up on a done card — CI remediation via the parent link
Story 1's implementation card is `done`. Two hours later CI fails on the merged branch. Don't reopen the done card — completed cards are history, and their handoff flows forward. Create a remediation card with the done card as its **parent**:
```bash
hermes kanban create "Fix CI: test_backoff_jitter flakes on 3.11" \
--assignee backend-dev \
--parent t_impl \
--workspace worktree --branch wt/ci-fix-backoff \
--body "CI run #4812 failed after t_impl completed.
FAILED tests/test_retry.py::test_backoff_jitter - TimeoutError
Acceptance: tests/test_retry.py green on 3.11 and 3.12."
```
Three things make this work:
- **Immediate dispatch.** Because the parent is already `done`, the child is created straight into `ready` — the dispatcher can claim it on the next tick. (A child of a still-open parent would wait in `todo`.)
- **Inherited context.** The remediation worker's context includes a *Parent task results* section carrying `t_impl`'s completion summary and metadata — the changed files and decisions the original worker recorded — so it knows why the code is shaped the way it is before reading a line of it.
- **Fresh evidence in the body.** The CI log didn't exist when `t_impl` completed, so it can't be in the parent's handoff — it goes in the new card's body, alongside explicit acceptance criteria.
Prefer a fresh worktree/branch for the remediation card. Checking out the original branch gives the worker repo *state* but none of the *rationale* — the parent handoff carries that. Same assignee profile is usually right: the profile that wrote the code has the skills to fix it.
## Inspecting a task currently running
For completeness — here's the drawer of a task still in flight (the API implementation from Story 1, claimed by `backend-dev` but not yet complete):
![Claimed, in-flight task](/img/kanban-tutorial/10-drawer-in-flight.png)
Status is `Running`. The active run appears in the Run History section with outcome `active` and no `ended_at`. If this worker dies or times out, the dispatcher closes this run with the appropriate outcome and opens a new one on the next claim — the attempt row never disappears.
## Next steps
- [Kanban overview](./kanban) — the full data model, event vocabulary, and CLI reference.
- `hermes kanban --help` — every subcommand, every flag.
- `hermes kanban watch --kinds completed,gave_up,timed_out` — live stream terminal events across the whole board.
- `hermes kanban notify-subscribe <task> --platform telegram --chat-id <id>` — get a gateway ping when a specific task finishes.
website/docs/user-guide/features/kanban-worker-lanes.md
+117
View File
@@ -0,0 +1,117 @@
# Kanban worker lanes
A **worker lane** is a class of process that the kanban dispatcher can route tasks to. Each lane has an identity (the assignee string), a spawn mechanism, and a contract for what it must do with the task once spawned.
This page is the contract. It exists for two audiences:
- **Operators** picking which lanes to wire into a board (which profiles to create, which assignees to use).
- **Plugin / integration authors** wanting to add a new lane shape (a CLI worker that wraps Codex / Claude Code / OpenCode, a containerised review worker, a non-Hermes service that pulls tasks via the API).
If you're writing the worker code itself — the agent that runs *inside* a lane — the kanban lifecycle and reference details are injected into the worker's system prompt automatically (the `KANBAN_GUIDANCE` block in [`agent/prompt_builder.py`](https://github.com/NousResearch/hermes-agent/blob/main/agent/prompt_builder.py)).
## The hierarchy
```text
Hermes Kanban = canonical task lifecycle + audit trail
Worker lane = implementation executor for one assigned card
Reviewer = human or human-proxy that gates "done"
GitHub PR = upstreamable artifact (optional, for code lanes)
```
Hermes Kanban owns lifecycle truth — `ready` → `running` → `review` / `blocked` / `done` / `archived`. Worker lanes execute work but never own that truth; everything they do flows back through the kanban kernel via the `kanban_*` tools (or, for non-Hermes external workers, via the API). Reviewers gate the transition from "code change written" to "task done."
## What a lane provides
To be a kanban worker lane, an integration must provide three things:
### 1. An assignee string
The dispatcher matches `task.assignee` against either a Hermes profile name (the default lane shape) or a registered non-spawnable identifier (the plugin lane shape — see [Adding an external CLI worker lane](#adding-an-external-cli-worker-lane) below). Tasks whose assignee doesn't resolve are left on `ready` with a `skipped_nonspawnable` event so a board operator can fix them; they are not silently dropped or executed by an arbitrary fallback.
### 2. A spawn mechanism
For Hermes profile lanes, the dispatcher's `_default_spawn` runs `hermes -p <assignee> chat -q <prompt>` (or the equivalent module form when the `hermes` shim isn't on `$PATH`) inside the task's pinned workspace, with these env vars set:
| Variable | Carries |
|---|---|
| `HERMES_KANBAN_TASK` | the task id the worker is operating on |
| `HERMES_KANBAN_DB` | absolute path to the per-board SQLite file |
| `HERMES_KANBAN_BOARD` | board slug |
| `HERMES_KANBAN_WORKSPACES_ROOT` | root of the board's workspace tree |
| `HERMES_KANBAN_WORKSPACE` | absolute path to *this* task's workspace |
| `HERMES_KANBAN_RUN_ID` | the current run's id (for the lifecycle gate) |
| `HERMES_KANBAN_CLAIM_LOCK` | the claim lock string (`<host>:<pid>:<uuid>`) |
| `HERMES_PROFILE` | the worker's own profile name (for `kanban_comment` author attribution) |
| `HERMES_TENANT` | tenant namespace, if the task has one |
For non-Hermes lanes (registered via a plugin), the plugin supplies its own `spawn_fn` callable that gets `task`, `workspace`, and `board` and returns an optional pid for crash detection.
### 3. A lifecycle terminator
Every claim must end in exactly one of:
- `kanban_complete(summary=..., metadata=...)` — task succeeds, status flips to `done`.
- `kanban_request_review(summary=..., metadata=..., reviewer=...)` — same-card implementation is complete and enters first-class review; status flips to `review`. The dispatcher loads the bundled `sdlc-review` skill unless `kanban.review_dispatch` is disabled. A reviewer approves with `kanban_complete`, returns actionable rework with `kanban_request_changes`, or escalates a genuine external blocker with `kanban_block`.
- `kanban_block(reason=...)` — task waits for human input, status flips to `blocked`. The dispatcher respawns when `kanban_unblock` runs.
- The worker process exits without a tool call. The kernel reaps it and emits `crashed` (PID died) or `gave_up` (consecutive-failure breaker tripped) or `timed_out` (max_runtime exceeded). This is the failure path; healthy workers don't end here.
The kanban kernel enforces that exactly one of these terminates each run. A worker that calls neither and exits normally is treated as crashed.
## Outputs and the review handoff
For code-changing tasks, pick the review model encoded by the task graph:
- **Same-card review:** call `kanban_request_review(summary=..., metadata=..., reviewer=...)`. The task enters `review` without touching block recurrence accounting. The dispatcher claims it with the bundled `sdlc-review` skill by default. The reviewer approves with `kanban_complete`, calls `kanban_request_changes(reason=...)` to close the review run and route the task back to its original implementer, or blocks only for a genuine external escalation.
- **Pre-created downstream review/QA/release card:** `kanban_show` lists child IDs; inspect those cards with `kanban_show(task_id=...)` before choosing the terminal action. When a child is the downstream review/QA/release phase, call `kanban_complete` on the implementation phase. It cannot promote until this parent is `done`/`archived`. Do not additionally request same-card review and never sticky-block the parent with `review-required:` — either choice strands or duplicates the downstream lane.
- **Human-only boards:** set `kanban.review_dispatch: false`. A task can then remain in `review` until a human approves it or uses `reopen-review`/the dashboard to return it to `ready`/`todo`.
Both review models carry their structured handoff on the lifecycle transition itself. Do not place secrets, tokens, or raw PII in `summary` or `metadata`; run rows are durable.
The injected `KANBAN_GUIDANCE` covers both graph shapes, `kanban_complete`, the same-card review loop, and `kanban_block` for genuine blockers.
## Logs and audit trail
The dispatcher writes per-task worker stdout/stderr to `<board-root>/logs/<task_id>.log`. Logs are auditable from kanban metadata:
- `task_runs` rows carry the `log_path`, exit code (where available), summary, and metadata.
- `task_events` rows carry every state transition (`promoted`, `claimed`, `heartbeat`, `completed`, `blocked`, `review_requested`, `changes_requested`, `review_reopened`, `gave_up`, `crashed`, `timed_out`, `reclaimed`, `claim_extended`).
- `kanban_show` returns both, so a reviewer (or a follow-up worker) reading the task gets the full history without needing dashboard access.
The dashboard renders run history with summaries, metadata blocks, and exit-status badges. CLI users can run `hermes kanban tail <task_id>` to follow live, or `hermes kanban runs <task_id>` for the historical attempt list.
## Existing lane shapes
### Hermes profile lane (default)
The shape every kanban worker takes today: the assignee is a profile name, the dispatcher spawns `hermes -p <profile>`, the worker gets the `KANBAN_GUIDANCE` system-prompt block injected automatically, and uses the `kanban_*` tools to terminate the run. No setup beyond defining the profile.
When you create profiles for your fleet, choose names that match the *role* you want the orchestrator to route to. The orchestrator (when there is one) discovers your profile names via `hermes profile list` — there's no fixed roster the system assumes (the orchestrator side of the contract is part of the injected `KANBAN_GUIDANCE`).
### Orchestrator profile lane
A specialisation of the profile lane: an orchestrator is a Hermes profile whose toolset includes `kanban` but excludes `terminal` / `file` / `code` / `web` for implementation. Its job is decomposing a high-level goal into child tasks via `kanban_create` + `kanban_link` and stepping back. The orchestrator skill encodes the anti-temptation rules.
## Adding an external CLI worker lane
Wiring a non-Hermes CLI tool (Codex CLI, Claude Code CLI, OpenCode CLI, a local coding-model runner, etc.) as a kanban worker lane is *not yet a paved path*. The dispatcher's spawn function is pluggable (`spawn_fn` is a parameter on `dispatch_once`), and a plugin could register its own `spawn_fn` for a non-Hermes assignee, but the surrounding integration work — wrapping the CLI's exit code into `kanban_complete` / `kanban_block` calls, mapping the CLI's workspace/sandbox conventions onto the dispatcher's `HERMES_KANBAN_WORKSPACE` env, handling auth and per-CLI policy — is still per-integration design work.
If you're considering adding a CLI lane, open an issue describing the specific CLI and the workflow you're trying to enable. The contract above is the constraints any such lane must satisfy; the implementation shape (one plugin per CLI vs a generic CLI-runner plugin parameterised by config) is open.
The historical issue for this is [#19931](https://github.com/NousResearch/hermes-agent/issues/19931) and the closed-not-merged Codex-specific PR [#19924](https://github.com/NousResearch/hermes-agent/pull/19924) — those describe the original architecture proposal but didn't land a runner.
## Failure modes the dispatcher handles
So lane authors don't have to reimplement these:
- **Stale claim TTL** — a worker that claims and then never heartbeats / completes / blocks gets reclaimed after `DEFAULT_CLAIM_TTL_SECONDS` (15 min default) — but only if the worker process has actually died. A live worker (slow model spending 20+ min in one tool-free LLM call) gets the claim *extended* instead of killed; only a dead PID is reclaimed.
- **Crashed worker** — a worker whose host-local PID has vanished is detected by `detect_crashed_workers` and reaped; the task increments `consecutive_failures` and may auto-block when the breaker trips.
- **Run-level retry** — when a task is retried (post-block, post-crash, post-reclaim), the worker can use the `expected_run_id` parameter on terminating tools to fail fast if its own run was already superseded.
- **Per-task max runtime** — `task.max_runtime_seconds` hard-caps wall-clock time per run, regardless of PID liveness. Catches genuinely-deadlocked workers that the live-PID extension would otherwise keep running.
- **Stranded-task detection** — a ready task whose assignee never produces a claim within `kanban.stranded_threshold_seconds` (default 30 min) shows up in `hermes kanban diagnostics` as a `stranded_in_ready` warning. Severity escalates to error at 2x the threshold and critical at 6x. Catches typo'd assignees, deleted profiles, and down external worker pools in one signal — identity-agnostic, no per-board allowlist to curate.
- **Legacy review dependency deadlock** — a parent sticky-blocked with `review-required:` while one or more direct children remain dependency-gated in `todo` produces an immediate `review_dependency_deadlock` error. The diagnostic is read-only: it suggests completing the finished phase or unlinking the incorrect edge but never removes a user block automatically.
## Related
- [Kanban overview](./kanban) — the user-facing intro.
- [Kanban tutorial](./kanban-tutorial) — walkthrough with the dashboard open.
- [`KANBAN_GUIDANCE`](https://github.com/NousResearch/hermes-agent/blob/main/agent/prompt_builder.py) — the worker + orchestrator lifecycle injected into every kanban worker's system prompt.
website/docs/user-guide/features/kanban.md
+1171
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/features/loops.md
+122
View File
@@ -0,0 +1,122 @@
---
sidebar_position: 17
title: "Recurring Loops"
description: "Re-run a prompt on a recurring interval inside your session — Hermes' take on Claude Code's /loop."
---
# Recurring Loops (`/loop`)
`/loop` re-runs a prompt (or a slash command) on a recurring cadence **inside your current session**. Each wakeup is a real agent turn: Hermes reads the current state fresh — the latest CI result, the newest queue depth, the file as it is now — does the work, reports back, and goes quiet until the next tick.
It's Hermes' take on **Claude Code's `/loop`** (and its `/proactive` alias, which works here too). Where [`/goal`](./goals.md) is judge-driven — "keep working until this objective is achieved" — `/loop` is timer-driven: "do this again every N minutes (or whenever it makes sense) until something says stop."
## When to use it
- **Polling external state.** "Watch the deploy / the CI run / the queue and tell me when it changes." The canonical use case.
- **Iterate-until-green.** "Run the tests, fix what fails, repeat until they pass."
- **Monitoring during a work session.** Keep an eye on error rates or a long job's progress while you do something else in the same conversation.
- **Periodic housekeeping.** Re-run a lint pass or a status summary every N minutes during a long session.
When the work should run **unattended** — overnight, on a real schedule, surviving restarts of your terminal — use a [cron job](./cron.md) instead. `/loop` lives inside a session; cron lives outside all of them. And when the task is a single objective with a definition of done, [`/goal`](./goals.md) is usually the better fit.
## Quick start
```
/loop 5m check the deploy status and tell me if it's live yet
```
What you'll see:
1. **Loop accepted** — `↻ Loop set (every 5m): check the deploy status…`
2. **First wakeup fires right away** — on the next idle poll (gateway: the next 15s watcher scan), Hermes injects the wakeup and runs a normal turn against current state.
3. **Repeat** — every 5 minutes after that, until a stop condition fires or you stop it.
Loop a slash command just as easily:
```
/loop 10m /recap
```
## The two cadence modes
**Fixed interval — you set the clock.** Give an interval (`30s`, `5m`, `2h`, `1h30m`) and the loop fires on that schedule. Use it when the thing you're watching changes on its own timeline:
```
/loop 2m poll the build at ci.example.com/job/42 and ping me the moment it finishes
```
**Self-paced — Hermes sets the clock.** Omit the interval and the loop paces itself: it starts at the floor (1 minute by default), and while the agent's replies stop changing it backs off exponentially — 2m, 4m, 8m, up to the ceiling (15 minutes by default). The moment a reply differs from the last one, cadence snaps back to the floor. Change detection is a local digest comparison (timestamps are ignored), so idle waits cost nothing extra:
```
/loop keep an eye on the migration and summarize progress
```
The rule of thumb: **fixed interval when an external clock drives the work; self-paced when the work drives the rhythm.**
## Stop conditions
A loop ends when any of these fires:
| Condition | How |
|---|---|
| The agent decides it's done | The wakeup prompt teaches the agent to end its reply with `LOOP_COMPLETE` on its own line when the task is finished or moot. |
| A run cap | `--times N` — stop after N wakeups. |
| An evidence-based condition | `--until <condition>` — after each wakeup, the same auxiliary judge that powers `/goal` checks the reply against your condition. If the judge rules the condition unachievable, the loop **pauses** with the reason instead of re-firing until the tick budget (fail-open: a broken judge never wedges the loop). |
| You | `/loop stop` (or `/loop pause` to keep it around). |
| The backstop budget | `loops.max_ticks` (default 100) pauses the loop so an unattended session can't burn tokens forever. `0` = unlimited. |
Examples:
```
/loop 2m poll CI --times 30
/loop 5m watch the queue --until queue depth reaches zero
```
## Commands
| Command | What it does |
|---|---|
| `/loop [interval] <prompt> [--times N] [--until <cond>]` | Start (or replace) the loop for this session. The first wakeup fires immediately; later ones follow the cadence. |
| `/loop` or `/loop status` | Show cadence, ticks fired, and time to the next wakeup. |
| `/loop pause` | Stop firing without losing the loop. |
| `/loop resume` | Pick it back up. |
| `/loop stop` | End the loop. |
| `/proactive …` | Alias for `/loop` (Claude Code parity). |
Works on the CLI, the TUI (`hermes --tui`), the web dashboard chat, the desktop app, and every gateway platform (Telegram, Discord, Slack, WhatsApp, …). On messaging platforms the gateway fires wakeups even between your messages — the loop belongs to the chat's session, and its results arrive as ordinary replies.
## Mixing with `/goal`
Both features inject synthetic turns at idle boundaries, so they follow one rule: **an active goal owns the session.** While a `/goal` is actively driving (judge saying "continue"), loop wakeups defer. The loop resumes using idle time as soon as the goal finishes, pauses, or parks itself on a wait barrier (`/goal wait`, or the judge's automatic WAIT verdict). A parked goal plus a `/loop` is a natural combo: the goal waits on the big async thing while the loop keeps a heartbeat on something else.
A real user message always wins over both — wakeups only fire while the session is idle and nothing of yours is queued.
## Behavior details
- **A wakeup is a normal user-role turn.** No system-prompt mutation, no toolset swap — prompt caching stays intact.
- **Survives `/resume` and compression.** Loop state persists per session and migrates across context-compression boundaries, same as `/goal`.
- **One loop per session.** Setting a new `/loop` replaces the old one. Run several loops by running several sessions (or use cron for a fleet of schedules).
- **Interrupting a wakeup turn (Ctrl+C) pauses the loop** — recoverable with `/loop resume`, so cancel actually means cancel.
- **Token cost scales with cadence.** Every tick is a full agent turn. Match the interval to how often the state actually changes; prefer self-pacing for idle waits.
## Configuration
```yaml
# ~/.hermes/config.yaml
loops:
min_interval_seconds: 30 # floor for fixed intervals
max_ticks: 100 # backstop budget (0 = unlimited)
self_paced_floor_seconds: 60 # self-paced starting cadence
self_paced_ceiling_seconds: 900 # self-paced max backoff
```
The `--until` judge routes through the `goal_judge` auxiliary task, so `auxiliary.goal_judge.*` overrides (provider, model, max_tokens) apply to loop conditions too.
## `/loop` vs `/goal` vs cron
| | `/loop` | `/goal` | cron |
|---|---|---|---|
| **Trigger** | Timer (or self-paced) | Judge verdict after each turn | Schedule, outside any session |
| **Lives in** | Your current session | Your current session | Its own session per run |
| **Ends when** | Stop condition / caps / you | Goal achieved / budget / you | You remove the job |
| **Best for** | Polling, monitoring, periodic re-runs | One objective, iterate until done | Unattended, long-horizon schedules |
website/docs/user-guide/features/lsp.md
+307
View File
@@ -0,0 +1,307 @@
---
sidebar_position: 16
title: "LSP — Semantic Diagnostics"
description: "Real language servers (pyright, gopls, rust-analyzer, …) wired into the post-write lint check used by write_file and patch."
---
# Language Server Protocol (LSP)
Hermes runs full language servers — pyright, gopls, rust-analyzer,
typescript-language-server, clangd, and ~20 more — as background
subprocesses and feeds their semantic diagnostics into the post-write
lint check used by `write_file` and `patch`. When the agent edits a
file, it sees exactly the errors that edit introduced — not just
syntax errors, but **type errors, undefined names, missing imports,
and project-wide semantic issues** the language server detects.
This is the same architecture top-tier coding agents use. Hermes
ships it self-contained: no editor host required, no plugins to
install, no separate daemon to manage.
## When LSP runs
LSP is gated on **git workspace detection**. When the agent's working
directory (or the file being edited) is inside a git repository, LSP
runs against that workspace. When neither is in a git repo, LSP
stays dormant — useful for messaging gateways where the cwd is the
user's home directory and there's no project to diagnose.
The check is layered: in-process syntax check first (microseconds),
then LSP diagnostics second when syntax is clean. A flaky or missing
language server can never break a write — every LSP failure path
falls back silently to the syntax-only result.
Concretely, on every successful `write_file` or `patch`:
1. Hermes captures a baseline of current diagnostics for the file.
2. Performs the write.
3. Re-queries the language server, filters out diagnostics that were
already in the baseline, and surfaces only the new ones.
The agent sees output like:
```
{
"bytes_written": 42,
"dirs_created": false,
"lint": {"status": "ok", "output": ""},
"lsp_diagnostics": "LSP diagnostics introduced by this edit:\n<diagnostics file=\"/path/to/foo.py\">\nERROR [42:5] Cannot find name 'foo' [reportUndefinedVariable] (Pyright)\nERROR [50:1] Argument of type \"str\" is not assignable to \"int\" [reportArgumentType] (Pyright)\n</diagnostics>"
}
```
The `lint` field carries the syntax-check result (microsecond
in-process parse via `ast.parse`, `json.loads`, etc.); the
`lsp_diagnostics` field carries the semantic diagnostics from the
real language server. Two channels, independent signals — the
agent sees a syntax-clean file with semantic problems as
``lint: ok`` plus a populated ``lsp_diagnostics``.
## Supported languages
| Language | Server | Auto-install |
|----------|--------|--------------|
| Python | `pyright-langserver` | npm |
| TypeScript / JavaScript / JSX / TSX | `typescript-language-server` | npm |
| Vue | `@vue/language-server` | npm |
| Svelte | `svelte-language-server` | npm |
| Astro | `@astrojs/language-server` | npm |
| Go | `gopls` | `go install` |
| Rust | `rust-analyzer` | manual (rustup) |
| C / C++ | `clangd` | manual (LLVM) |
| Bash / Zsh | `bash-language-server` | npm |
| YAML | `yaml-language-server` | npm |
| Lua | `lua-language-server` | manual (GitHub releases) |
| PHP | `intelephense` | npm |
| OCaml | `ocaml-lsp` | manual (opam) |
| Dockerfile | `dockerfile-language-server-nodejs` | npm |
| Terraform | `terraform-ls` | manual |
| Dart | `dart language-server` | manual (dart sdk) |
| Haskell | `haskell-language-server` | manual (ghcup) |
| Julia | `julia` + LanguageServer.jl | manual |
| Clojure | `clojure-lsp` | manual |
| Nix | `nixd` | manual |
| Zig | `zls` | manual |
| Gleam | `gleam lsp` | manual (gleam install) |
| Elixir | `elixir-ls` | manual |
| Prisma | `prisma language-server` | manual |
| Kotlin | `kotlin-language-server` | manual |
| Java | `jdtls` | manual |
| PowerShell | `PowerShellEditorServices` (`pwsh` host) | manual (release zip) |
For "manual" entries, install the server through whatever toolchain
manager makes sense for that language (rustup, ghcup, opam, brew,
…). Hermes auto-detects the binary on PATH or in
`<HERMES_HOME>/lsp/bin/`.
### PowerShell
PowerShellEditorServices isn't a single binary — it's a PowerShell
module bundle launched by a `pwsh` (PowerShell 7+) or `powershell`
host. Setup:
1. Install [PowerShell](https://github.com/PowerShell/PowerShell) so
`pwsh` (or Windows `powershell`) is on PATH.
2. Download the latest release zip from
[PowerShellEditorServices releases](https://github.com/PowerShell/PowerShellEditorServices/releases)
and extract it.
3. Point Hermes at the extracted bundle — the directory that contains
`PowerShellEditorServices/Start-EditorServices.ps1`. Either:
- set `lsp.servers.powershell.command: ["/path/to/bundle"]` in
`config.yaml`, or
- extract it to `<HERMES_HOME>/lsp/PowerShellEditorServices`, or
- export `PSES_BUNDLE_PATH=/path/to/bundle`.
`hermes lsp status` reports `installed` once `pwsh` is found; if the
bundle is missing you'll see a one-time warning in the logs with the
download link.
A few servers are installed alongside a peer dependency that npm
won't auto-pull. The current case is `typescript-language-server`,
which requires the `typescript` SDK importable from the same
`node_modules` tree — Hermes installs both packages together when you
run `hermes lsp install typescript` or auto-install fires on first
use.
## CLI
```
hermes lsp status # service state + per-server install status
hermes lsp list # registry, optionally --installed-only
hermes lsp install <id> # eagerly install one server
hermes lsp install-all # try every server with a known recipe
hermes lsp restart # tear down running clients
hermes lsp which <id> # print resolved binary path
```
`hermes lsp status` is the best starting point — it shows which
languages will get semantic diagnostics today and which need a
binary installed.
## Configuration
The defaults work for typical setups; nothing to set if the binaries
are on PATH.
```yaml
# config.yaml
lsp:
# Master toggle. Disabling skips the entire subsystem — no servers
# spawn, no background event loop runs.
enabled: true
# How long to wait for diagnostics after each write.
wait_mode: document # "document" or "full"
# Max seconds to wait for the server to re-check the file after an
# edit. Only *fresh* diagnostics (produced for the post-edit
# content) are ever reported; if the server doesn't finish within
# this budget, the edit reports "no LSP data" rather than stale
# errors from before the edit. Raise this for slow servers on big
# projects (tsserver, rust-analyzer mid-indexing).
wait_timeout: 5.0
# How to handle missing server binaries.
# auto — install via npm/pip/go install into <HERMES_HOME>/lsp/bin
# manual — only use binaries already on PATH
install_strategy: auto
# How long an unused language-server client stays alive (seconds).
# Idle servers are shut down automatically and respawned on the next
# relevant file operation. Set to 0 to disable idle reaping and keep
# servers alive for the life of the process. Values below 30s are
# clamped to 30 so a sweep can never reap a client mid-operation.
idle_timeout: 600
# Per-server overrides (all optional).
servers:
pyright:
disabled: false
command: ["/abs/path/to/pyright-langserver", "--stdio"]
env: { PYRIGHT_LOG_LEVEL: "info" }
initialization_options:
python:
analysis:
typeCheckingMode: "strict"
typescript:
disabled: true # skip TS even when its extensions match
```
### Per-server keys
* `disabled: true` — skip this server entirely even when its
extensions match a file.
* `command: [bin, ...args]` — pin a custom binary path. Bypasses
auto-install.
* `env: {KEY: value}` — extra env vars passed to the spawned process.
* `initialization_options: {...}` — merged into the LSP
`initializationOptions` payload sent in the `initialize`
handshake. Server-specific; consult the language server's docs.
## Installation locations
When `install_strategy: auto`, Hermes installs binaries into
`<HERMES_HOME>/lsp/bin/`. NPM packages land in
`<HERMES_HOME>/lsp/node_modules/` with bin symlinks one level up.
Go binaries come from `go install` with `GOBIN` pointed at the
staging dir.
Nothing is ever installed to `/usr/local/`, `~/.local/`, or any other
shared location — the staging dir is fully Hermes-owned and is
removed when you reset the profile.
## Performance characteristics
LSP servers are **lazy-spawned** on first use. Editing a Python file
in a project that's never seen `.py` traffic spawns pyright; the
spawn takes 1-3 seconds for most servers (rust-analyzer can take 10+
on a cold project). Subsequent edits in the same workspace re-use
the running server.
The LSP layer adds a few milliseconds to clean writes when no
diagnostics are emitted. When diagnostics are emitted, the wait
budget is `wait_timeout` seconds — typically the server responds in
tens of milliseconds for pyright/tsserver and a few seconds for
rust-analyzer mid-indexing.
Diagnostics are **freshness-gated**: a result only counts when the
server produced it for the content of the current edit (a
`publishDiagnostics` push at/after the change, or a pull request
answered after it). Slow servers that haven't re-checked yet result
in "no data" for that edit — never in yesterday's errors being
re-reported as current.
Servers are kept alive while they're being used and shut down after
`lsp.idle_timeout` seconds (default 600) with no file activity — a
long-running gateway that touches many worktrees no longer accumulates
one language-server process per workspace forever. A reaped server is
respawned automatically on the next relevant file operation. Set
`idle_timeout: 0` to disable reaping and hold every server's index warm
for the life of the process.
Servers that support multi-root workspaces (currently pyright) run as a
**single process** per Hermes process: the first Python project spawns
it, and every further project root — for example sibling git worktrees
edited by parallel subagents — is attached to that same server as an
additional workspace folder instead of starting another copy.
## Disabling
Set `lsp.enabled: false` in `config.yaml` to disable the entire
subsystem. The post-write check falls back to the in-process syntax
check (`ast.parse` for Python, `json.loads` for JSON, etc.) which
ships unchanged from earlier versions.
To disable a single language without disabling the whole layer:
```yaml
lsp:
servers:
rust-analyzer:
disabled: true
```
## Troubleshooting
**`hermes lsp status` shows a server as "missing"**
The binary isn't on PATH and isn't in `<HERMES_HOME>/lsp/bin/`. Run
`hermes lsp install <server_id>` to attempt an auto-install, or
install the binary manually through the language's normal toolchain.
**`Backend warnings` section in `hermes lsp status`**
Some servers ship as thin wrappers around an external CLI for actual
diagnostics — they spawn cleanly and accept requests but never emit
errors when the sidecar binary is missing. The most common case is
`bash-language-server`, which delegates diagnostics to `shellcheck`.
When `hermes lsp status` shows a `Backend warnings` section, install
the named tool through your OS package manager:
```
apt install shellcheck # Debian / Ubuntu
brew install shellcheck # macOS
scoop install shellcheck # Windows
```
The same warning is logged once at server spawn time in
`~/.hermes/logs/agent.log`.
**Server starts but never returns diagnostics**
Check `~/.hermes/logs/agent.log` for `[agent.lsp.client]` entries —
both stderr from the language server and protocol errors land
there. Some servers (rust-analyzer especially) need to finish a
project-wide index before they emit per-file diagnostics; the first
edit after server start may complete with no diagnostics, with
subsequent edits picking them up.
**Server crashed**
A crashed server is added to the broken-set and won't be retried for
the rest of the session. Run `hermes lsp restart` to clear the set;
the next edit re-spawns.
**Editing a file outside any git repo**
By design, LSP only runs inside a git repository. If the project isn't
yet initialized, run `git init` to enable LSP diagnostics. Otherwise the
in-process syntax-only fallback applies.
website/docs/user-guide/features/mcp.md
+936
View File
@@ -0,0 +1,936 @@
---
sidebar_position: 4
title: "MCP (Model Context Protocol)"
description: "Connect Hermes Agent to external tool servers via MCP — and control exactly which MCP tools Hermes loads"
---
# MCP (Model Context Protocol)
MCP lets Hermes Agent connect to external tool servers so the agent can use tools that live outside Hermes itself — GitHub, databases, file systems, browser stacks, internal APIs, and more.
If you have ever wanted Hermes to use a tool that already exists somewhere else, MCP is usually the cleanest way to do it.
:::tip Coming from Claude Code?
The `mcpServers` block in your `~/.claude.json` maps to `mcp_servers` in Hermes' `config.yaml` — and `hermes import-agent claude-code` migrates it (along with skills and instructions) automatically. See [Import from Other Agents](../import-from-other-agents.md).
:::
## What MCP gives you
- Access to external tool ecosystems without writing a native Hermes tool first
- Local stdio servers and remote HTTP MCP servers in the same config
- Automatic tool discovery and registration at startup
- Utility wrappers for MCP resources and prompts when supported by the server
- Per-server filtering so you can expose only the MCP tools you actually want Hermes to see
## Quick start
1. MCP support ships with the standard install — no extra step needed.
2. Add an MCP server to `~/.hermes/config.yaml`:
```yaml
mcp_servers:
filesystem:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-filesystem", "/home/user/projects"]
```
3. Start Hermes:
```bash
hermes chat
```
4. Ask Hermes to use the MCP-backed capability.
For example:
```text
List the files in /home/user/projects and summarize the repo structure.
```
Hermes will discover the MCP server's tools and use them like any other tool.
## Catalog: one-click install for Nous-approved MCPs
Hermes ships a curated catalog of MCP servers that Nous staff has reviewed
and merged. They're disabled by default — install only what you actually
want.
```bash
hermes mcp # interactive picker (default)
hermes mcp catalog # plain-text list, scriptable
hermes mcp install n8n # install a catalog entry by name
```
The picker shows each entry with its current status:
```
n8n available Manage and inspect n8n workflows from Hermes
linear enabled Linear issue/project management (remote OAuth)
github installed (disabled) GitHub repo + PR tools
```
Hit `Enter` on a row to install (and walk through any required credentials),
enable, disable, or uninstall. Catalog entries are stored under
`optional-mcps/` in the hermes-agent repo — presence in that directory means
Nous approval. There is no community submission tier; entries are added by
merging a PR.
Catalog entries can require:
- **API key** — Hermes prompts at install time and writes the value to
`~/.hermes/.env`. Non-secret values (base URLs) go to the same file.
- **OAuth** (remote MCP) — written as `auth: oauth` in your config; the MCP
client opens a browser on first connection.
- **OAuth** (third-party provider like Google/GitHub) — Hermes points you at
`hermes auth <provider>` if you haven't authenticated already.
### Tool selection at install time
After credentials are configured, Hermes probes the MCP server to list every
tool it exposes and presents a checklist:
```
Select tools for 'linear' (SPACE toggle, ENTER confirm)
[x] find_issues Find issues matching a query
[x] get_issue Get a single issue
[x] create_issue Create a new issue
[ ] delete_workspace Delete a Linear workspace
...
```
The pre-checked rows come from:
1. **Your prior selection** if you've installed this entry before (reinstalls
preserve what you had — the manifest's defaults don't override it)
2. **The manifest's `tools.default_enabled`** if the entry declares one (some
catalog entries pre-prune mutating or rarely-useful tools)
3. **Everything** if neither applies
Some entries with very large auto-generated surfaces (e.g. `cloudflare`,
~3,300 OpenAPI endpoint tools) instead declare `tools.default_excluded` — a
curated block-list of names and glob patterns. Installing one of these skips
the checklist entirely and writes `tools.exclude`; everything not matched
stays enabled, including tools the server adds later. Edit
`mcp_servers.<name>.tools.exclude` in config.yaml to re-enable a family.
Submit the checklist with ENTER. Only the checked tools end up in
`mcp_servers.<name>.tools.include`. If you select everything, no filter is
written (cleanest config shape, identical behavior).
**If the probe fails** (server unreachable, OAuth not yet completed,
backing service not running), the install still succeeds: the manifest's
`tools.default_enabled` is applied directly (if declared), or no filter is
written (if not). Re-run `hermes mcp configure <name>` once the server is
reachable to refine.
### Trust model
Installing a catalog entry runs whatever the manifest specifies — `git clone`,
the entry's `bootstrap` commands (`pip install`, `npm install`, etc.), and
ultimately the MCP server's own code. Manifests are gated by PR review into
the hermes-agent repo, so Nous has reviewed each entry before it shipped —
**but you should still read the manifest before installing**, especially the
`source:` field's repository, the `install.bootstrap:` commands, and any
`transport.command:` invocation.
Manifests live at
[`optional-mcps/<name>/manifest.yaml`](https://github.com/NousResearch/hermes-agent/tree/main/optional-mcps)
on GitHub. The picker also prints the manifest's `source:` URL at install
time so you can quickly verify the upstream repo. The web dashboard's MCP
page surfaces the same detail per catalog entry — transport, auth type, the
endpoint URL (HTTP) or command + args (stdio), the git install source/ref and
bootstrap commands, and setup notes — with the `source:` rendered as a
clickable link, so you can inspect exactly what an entry connects to or runs
before clicking Install.
### Manifest version compatibility
Manifests pin a `manifest_version`. The catalog is forward-compatible: if a
PR adds an entry with a newer `manifest_version` than your installed Hermes
understands, the picker will surface a warning (`⚠ '<name>' requires a newer
Hermes`) for that entry instead of silently hiding it. Run `hermes update`
to install the latest Hermes when you see that.
### Runtime `${ENV_VAR}` substitution
Inside an entry's `transport.command`, `transport.args`, `transport.url`,
and `headers`, `${VAR}` placeholders are resolved at server-connect time
from environment variables (which include everything in `~/.hermes/.env`).
This is useful when a catalog entry wants to reference a value the user
configured elsewhere — e.g. `${HOME}/foo` or `${MY_PROVIDER_TOKEN}`.
Cursor-style context variables are also substituted (case-sensitive):
`${userHome}` (home directory), `${workspaceFolder}` (session workspace
root), `${workspaceFolderBasename}`, and `${pathSeparator}` / `${/}`
(the OS path separator). See the
[MCP config reference](/docs/reference/mcp-config-reference) for details.
Note this is distinct from `${INSTALL_DIR}` in catalog manifests, which is
substituted at install-time with the path the catalog cloned the entry's
repo into.
### Updating tool selection later
```bash
hermes mcp configure linear
```
Reopens the same checklist with your current selection pre-checked. Use this
when you want more tools enabled, or when the server has added new tools that
you want to opt into.
### Updating the catalog manifest
MCPs are never auto-updated. Re-run `hermes mcp install <name>` to refresh
after a Hermes update if a manifest version changed.
To add an MCP to the catalog, open a PR against
[`optional-mcps/`](https://github.com/NousResearch/hermes-agent/tree/main/optional-mcps).
### Suggestion metadata (`suggest:`)
A manifest may declare an optional `suggest:` block with `keywords:` and/or
`hosts:` lists. UI surfaces (currently the Desktop app's composer) use it to
offer a one-click "Add &lt;server&gt;" pill when your draft mentions one of the
keywords as a completed word, or contains a pasted link whose hostname ends
with one of the host suffixes. It is purely advisory — installs still flow
through the same validated catalog/config paths — and most hosted remote
entries (Atlassian, Sentry, Notion, Stripe, Vercel, Supabase, and friends)
declare it.
GitHub is deliberately **not** in the catalog: its hosted MCP requires each
client to bring its own OAuth app (generic dynamic client registration is
rejected), and Hermes's bundled `github/*` skills driving the `gh` CLI are a
more capable integration. On Desktop, GitHub mentions instead offer the
`github-auth` skill when `gh` isn't signed in yet.
## Two kinds of MCP servers
### Stdio servers
Stdio servers run as local subprocesses and talk over stdin/stdout.
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
```
Use stdio servers when:
- the server is installed locally
- you want low-latency access to local resources
- you are following MCP server docs that show `command`, `args`, and `env`
### HTTP servers
HTTP MCP servers are remote endpoints Hermes connects to directly.
```yaml
mcp_servers:
remote_api:
url: "https://mcp.example.com/mcp"
headers:
Authorization: "Bearer ***"
```
Use HTTP servers when:
- the MCP server is hosted elsewhere
- your organization exposes internal MCP endpoints
- you do not want Hermes spawning a local subprocess for that integration
### OAuth-authenticated HTTP servers
Most hosted MCP servers (Cloudflare, Linear, Sentry, Atlassian, Asana, Figma, Stripe, …) require OAuth 2.1 instead of a static bearer token. Set `auth: oauth` and Hermes handles discovery, client identification, PKCE, token exchange, refresh, and step-up auth via the MCP Python SDK.
Hermes identifies itself with a [Client ID Metadata Document](../../reference/mcp-config-reference.md#client-identification-cimd-and-dcr) on servers that support one, and falls back to Dynamic Client Registration on those that don't. Both are automatic; there is nothing to configure.
:::tip Figma remote MCP
Figma's hosted endpoint (`https://mcp.figma.com/mcp`) allowlists Dynamic Client Registration by **exact `client_name`** — bare `"Hermes Agent"` 403s, while `"Claude Code"` and `"Codex"` succeed. Hermes auto-sets `oauth.client_name: "Claude Code"` for `mcp.figma.com` so install/login works without a special trick:
```yaml
mcp_servers:
figma:
url: "https://mcp.figma.com/mcp"
auth: oauth
```
Or: `hermes mcp install figma`, then `hermes mcp login figma`.
:::
```yaml
mcp_servers:
linear:
url: "https://mcp.linear.app/mcp"
auth: oauth
```
On first connect, Hermes prints an authorize URL, opens your browser when possible, and waits for the OAuth callback on a local loopback port. Tokens are cached at `~/.hermes/mcp-tokens/<server>.json` with 0o600 perms; subsequent runs reuse them silently until refresh fails.
**Remote / headless hosts.** When Hermes runs on a different machine than your browser, the loopback callback can't reach your laptop. Ways to complete the flow:
- **Hermes Desktop (automatic):** when you run the OAuth sign-in from the Desktop app's MCP setup UI against a remote backend, Desktop hosts the callback listener on *your* machine and relays the authorization back to the gateway automatically — no tunnel, paste, or proxy needed. Requires both the Desktop app and the backend to be up to date.
- **Paste-back (no setup):** on an interactive terminal Hermes prints "Or paste the redirect URL here…" alongside the authorize URL. Open the URL in your browser, approve, copy the full URL the browser ends up on (the redirect will show a connection error — that's expected), paste it at the prompt. Bare `?code=…&state=…` query strings work too.
- **SSH port forward:** `ssh -N -L <port>:127.0.0.1:<port> user@host` in a separate terminal, then let the redirect flow normally.
- **Proxied callback (`redirect_uri`):** when a public HTTPS endpoint forwards to the host (e.g. a Tailscale Funnel or reverse proxy pointed at the callback port), set `oauth.redirect_uri` and the browser redirect reaches Hermes on its own — no tunnel or paste needed:
```yaml
mcp_servers:
myserver:
url: "https://mcp.example.com/mcp"
auth: oauth
oauth:
redirect_port: 8765 # fixed port for the proxy to target
redirect_uri: "https://oauth.example.ts.net/callback"
```
For fully headless gateways (messaging bot, no interactive terminal at all), the optional [`mcp-oauth-remote-gateway` skill](../skills/optional/mcp/mcp-mcp-oauth-remote-gateway.md) walks the agent through completing the flow manually and writing tokens where Hermes expects them.
**Pitfall — WAF rejects `127.0.0.1` redirect URIs.** A few providers front their authorization server with a WAF that 403s any authorize request whose query string contains a literal `127.0.0.1` (Reclaim.ai's AWS API Gateway is a known example — every attempt returns `{"message":"Forbidden"}` before reaching the OAuth app). Set `oauth.redirect_host: localhost` to use `http://localhost:<port>/callback` instead; the callback listener still binds `127.0.0.1` either way.
See [OAuth over SSH / Remote Hosts](../../guides/oauth-over-ssh.md#mcp-servers) for the full walkthrough, including DCR-less servers (e.g. Slack), pre-registered `client_id`/`client_secret`, scope customization, and re-auth via `hermes mcp login <server>`.
**Pitfall — providers that don't support automatic registration (Google Drive, Atlassian).** Some servers reject the dynamic client registration step (RFC 7591) that bare `auth: oauth` relies on — Google's official Drive server (`https://drivemcp.googleapis.com/mcp/v1`) returns a `400 Bad Request`, so no OAuth client is created and no token is acquired. The symptom is subtle: these servers also serve `tools/list` *without* auth, so `hermes mcp login` can list the tools and look like it worked, but every real tool call later times out. `hermes mcp login` now detects this (it checks that a token actually landed on disk) and tells you to supply your own OAuth client. Create one in the provider's console and add it to config:
```yaml
mcp_servers:
googledrive:
url: "https://drivemcp.googleapis.com/mcp/v1"
auth: oauth
oauth:
client_id: "<your-oauth-client-id>"
client_secret: "<your-oauth-client-secret>"
```
Then run `hermes mcp login googledrive` — with the pre-registered client, Hermes skips registration and runs the normal browser authorization flow.
**Pitfall — config auto-reload race.** When you edit `~/.hermes/config.yaml` from inside a running Hermes session, the CLI auto-reloads MCP connections with a 30s timeout. That's not enough for an interactive OAuth flow. Add the entry, then run `hermes mcp login <server>` from a fresh terminal — it waits the full 5 minutes for you to complete auth.
## mTLS / client certificates
Remote HTTP MCP servers that require mutual TLS (client-certificate authentication) are supported via `client_cert` / `client_key`. Hermes passes the resolved certificate to the underlying HTTP client for the TLS handshake.
`client_cert` accepts three shapes:
- **A single combined PEM path** — one file holding both the certificate and the private key:
```yaml
mcp_servers:
internal_api:
url: "https://mcp.internal.example.com/mcp"
client_cert: "~/.certs/mcp-client.pem"
```
- **A `[cert, key]` 2-tuple** — certificate and key in separate files (equivalent to setting `client_cert` + `client_key`):
```yaml
mcp_servers:
internal_api:
url: "https://mcp.internal.example.com/mcp"
client_cert: ["~/.certs/mcp-client.crt", "~/.certs/mcp-client.key"]
```
- **A `[cert, key, password]` 3-tuple** — when the private key is encrypted, the third element is the key passphrase:
```yaml
mcp_servers:
internal_api:
url: "https://mcp.internal.example.com/mcp"
client_cert: ["~/.certs/mcp-client.crt", "~/.certs/mcp-client.key", "${MCP_KEY_PASSWORD}"]
```
You can also keep the cert and key fully separate via `client_cert` (combined PEM) plus an explicit `client_key`. Paths support `~` expansion; a missing file raises a clear, server-scoped error rather than an opaque TLS handshake failure.
## Per-user identity header
Remote HTTP/SSE MCP servers that key behavior on a caller identity (per-user rate limits, audit trails, multi-tenant routing) can be sent an identity header on every request via `identity_header`:
```yaml
mcp_servers:
team_api:
url: "https://mcp.team.example.com/mcp"
identity_header:
name: "X-User-Id"
value_from: "static" # "static" (default) or "profile"
value: "alice" # required for static
```
- `value_from: static` sends the literal `value` from config.yaml.
- `value_from: profile` sends the active Hermes profile name, resolved once at connect time — useful when multiple profiles on one machine talk to the same server and it needs to tell them apart.
An explicit entry in the server's `headers` mapping with the same name (any casing) always wins; the identity header never overrides your own header config. Invalid `identity_header` blocks are warned about and ignored — they never block the server from connecting. On stdio servers the key is ignored with a warning (stdio transports have no headers).
## Basic configuration reference
Hermes reads MCP config from `~/.hermes/config.yaml` under `mcp_servers`.
### Common keys
| Key | Type | Meaning |
|---|---|---|
| `command` | string | Executable for a stdio MCP server |
| `args` | list | Arguments for the stdio server |
| `env` | mapping | Environment variables passed to the stdio server |
| `url` | string | HTTP MCP endpoint |
| `headers` | mapping | HTTP headers for remote servers |
| `client_cert` | string \| list | Client certificate for mTLS — a combined PEM path, or `[cert, key]` / `[cert, key, password]` |
| `client_key` | string | Client private-key PEM path (when separate from `client_cert`) |
| `identity_header` | mapping | Optional per-user identity header for HTTP/SSE servers — `{name, value_from: static\|profile, value}` |
| `timeout` | number | Tool call timeout |
| `connect_timeout` | number | Initial connection timeout (also bounds the MCP `initialize` handshake) |
| `idle_timeout_seconds` | number | Recycle a stdio server after this many seconds without a tool call (`0` = never, default). The server restarts transparently on the next tool call. |
| `max_lifetime_seconds` | number | Recycle a stdio server after this total age (`0` = never, default). Restarts transparently on next use. |
| `enabled` | bool | If `false`, Hermes skips the server entirely |
| `supports_parallel_tool_calls` | bool | If `true`, tools from this server may run concurrently |
| `tools` | mapping | Per-server tool filtering and utility policy |
### Minimal stdio example
```yaml
mcp_servers:
filesystem:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
```
### Recycling memory-heavy stdio servers
Browser-based MCP servers (e.g. `@playwright/mcp`) keep a full Chromium
resident after their first tool call — hundreds of MB that never get
released. Opt in to automatic recycling and the server is torn down after
the idle/lifetime limit, then restarted transparently the next time one of
its tools is called (its tools stay registered the whole time):
```yaml
mcp_servers:
playwright:
command: "npx"
args: ["-y", "@playwright/mcp@latest", "--headless"]
idle_timeout_seconds: 900 # recycle after 15 min without a tool call
max_lifetime_seconds: 86400 # and at least once a day regardless
```
### Minimal HTTP example
```yaml
mcp_servers:
company_api:
url: "https://mcp.internal.example.com"
headers:
Authorization: "Bearer ***"
```
## Built-in presets
For well-known MCP servers, `hermes mcp add` accepts a `--preset` flag that fills in the transport details so you don't have to look up the command and args. The preset only supplies defaults — anything else (env vars, headers, filtering) you pass on the same command line still wins.
| Preset | What it wires up |
|---|---|
| `codex` | The Codex CLI's MCP server (`codex mcp-server` over stdio). Requires the `codex` CLI on PATH. |
```bash
# Add Codex CLI as an MCP server in one line
hermes mcp add codex --preset codex
```
That writes the equivalent of:
```yaml
mcp_servers:
codex:
command: "codex"
args: ["mcp-server"]
```
You can pick any local name (`hermes mcp add my-codex --preset codex` is fine); the preset only provides the `command`/`args` defaults.
## How Hermes registers MCP tools
Hermes prefixes MCP tools so they do not collide with built-in names:
```text
mcp_<server_name>_<tool_name>
```
Examples:
| Server | MCP tool | Registered name |
|---|---|---|
| `filesystem` | `read_file` | `mcp_filesystem_read_file` |
| `github` | `create-issue` | `mcp_github_create_issue` |
| `my-api` | `query.data` | `mcp_my_api_query_data` |
In practice, you usually do not need to call the prefixed name manually — Hermes sees the tool and chooses it during normal reasoning.
### Tool-result sanitization and `_meta`
Two behaviors apply to every MCP tool result before the model sees it:
- **Invisible Unicode TAG characters are stripped.** Characters in the U+E0000–U+E007F range render as nothing in terminals and chat UIs but are fully visible to the model — a classic prompt-injection smuggling channel for a malicious or compromised server. Hermes strips them from tool results, resource content, and tool descriptions. Legitimate emoji tag sequences (regional flags like 🏴󠁧󠁢󠁳󠁣󠁴󠁿) are preserved.
- **Vendor `_meta` is surfaced; protocol-reserved keys are not.** When a server attaches a `_meta` mapping to a tool result (vendor namespaces like `com.example/handoff`), Hermes passes it through to the model alongside the result content. Keys under protocol-reserved prefixes — a `modelcontextprotocol` or `mcp` label followed by another label, e.g. `modelcontextprotocol.io/...` or `tools.mcp.com/...` — are dropped, matching the MCP spec's key-name rules. If nothing model-facing remains, the `_meta` field is omitted entirely.
## MCP utility tools
When supported, Hermes also registers utility tools around MCP resources and prompts:
- `list_resources`
- `read_resource`
- `list_prompts`
- `get_prompt`
These are registered per server with the same prefix pattern, for example:
- `mcp_github_list_resources`
- `mcp_github_get_prompt`
### Important
These utility tools are now capability-aware:
- Hermes only registers resource utilities if the MCP session actually supports resource operations
- Hermes only registers prompt utilities if the MCP session actually supports prompt operations
So a server that exposes callable tools but no resources/prompts will not get those extra wrappers.
## Per-server filtering
You can control which tools each MCP server contributes to Hermes, allowing fine-grained management of your tool namespace.
### Disable a server entirely
```yaml
mcp_servers:
legacy:
url: "https://mcp.legacy.internal"
enabled: false
```
If `enabled: false`, Hermes skips the server completely and does not even attempt a connection.
### Whitelist server tools
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
tools:
include: [create_issue, list_issues]
```
Only those MCP server tools are registered.
Entries in `include`/`exclude` may also be glob patterns (`*`, `?`, `[...]`,
matched case-sensitively): `include: ["*_dns_*"]` registers every tool whose
name contains `_dns_`. Plain entries without metacharacters stay exact-match.
Globs are the practical way to filter servers that expose thousands of
auto-generated endpoint tools by product family.
### Blacklist server tools
```yaml
mcp_servers:
stripe:
url: "https://mcp.stripe.com"
tools:
exclude: [delete_customer]
```
All server tools are registered except the excluded ones.
### Glob patterns
Both lists accept fnmatch-style globs alongside exact names — essential for
huge flat surfaces like Cloudflare's API MCP (`?codemode=false`, ~3,300
tools) where excluding product areas one endpoint at a time is impractical:
```yaml
mcp_servers:
cloudflare:
url: "https://mcp.cloudflare.com/mcp?codemode=false"
auth: oauth
tools:
exclude: ["*_radar_*", "*_accounts_dlp_*", "*_zones_web3_*"]
```
Entries without glob metacharacters (`*`, `?`, `[`) match exactly — `docs`
excludes only the tool named `docs`, never `docs_search`.
### Precedence rule
If both are present:
```yaml
tools:
include: [create_issue]
exclude: [create_issue, delete_issue]
```
`include` wins.
### Filter utility tools too
You can also separately disable Hermes-added utility wrappers:
```yaml
mcp_servers:
docs:
url: "https://mcp.docs.example.com"
tools:
prompts: false
resources: false
```
That means:
- `tools.resources: false` disables `list_resources` and `read_resource`
- `tools.prompts: false` disables `list_prompts` and `get_prompt`
### Full example
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
tools:
include: [create_issue, list_issues, search_code]
prompts: false
stripe:
url: "https://mcp.stripe.com"
headers:
Authorization: "Bearer ***"
tools:
exclude: [delete_customer]
resources: false
legacy:
url: "https://mcp.legacy.internal"
enabled: false
```
## What happens if everything is filtered out?
If your config filters out all callable tools and disables or omits all supported utilities, Hermes does not create an empty runtime MCP toolset for that server.
That keeps the tool list clean.
## Runtime behavior
### Discovery time
Hermes discovers MCP servers at startup and registers their tools into the normal tool registry.
### Dynamic Tool Discovery
MCP servers can notify Hermes when their available tools change at runtime by sending a `notifications/tools/list_changed` notification. When Hermes receives this notification, it automatically re-fetches the server's tool list and updates the registry — no manual `/reload-mcp` required.
This is useful for MCP servers whose capabilities change dynamically (e.g. a server that adds tools when a new database schema is loaded, or removes tools when a service goes offline).
The refresh is lock-protected so rapid-fire notifications from the same server don't cause overlapping refreshes. Prompt and resource change notifications (`prompts/list_changed`, `resources/list_changed`) are received but not yet acted on.
### Reloading
If you change MCP config, use:
```text
/reload-mcp
```
This reloads MCP servers from config and refreshes the available tool list. It is also the explicit way to re-probe availability-gated tools (Docker, `HASS_TOKEN`, OAuth…): a session's tool set is otherwise frozen, so a credential or daemon that appears mid-session is only picked up on `/reload-mcp`, `/new`, or context compaction. For runtime tool changes pushed by the server itself, see [Dynamic Tool Discovery](#dynamic-tool-discovery) above.
### Toolsets
Each configured MCP server also creates a runtime toolset when it contributes at least one registered tool:
```text
mcp-<server>
```
That makes MCP servers easier to reason about at the toolset level.
## Security model
### Stdio env filtering
For stdio servers, Hermes does not blindly pass your full shell environment.
Only explicitly configured `env` plus a safe baseline are passed through. This reduces accidental secret leakage.
### Config-level exposure control
The new filtering support is also a security control:
- disable dangerous tools you do not want the model to see
- expose only a minimal whitelist for a sensitive server
- disable resource/prompt wrappers when you do not want that surface exposed
## Example use cases
### GitHub server with a minimal issue-management surface
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
tools:
include: [list_issues, create_issue, update_issue]
prompts: false
resources: false
```
Use it like:
```text
Show me open issues labeled bug, then draft a new issue for the flaky MCP reconnection behavior.
```
### Stripe server with dangerous actions removed
```yaml
mcp_servers:
stripe:
url: "https://mcp.stripe.com"
headers:
Authorization: "Bearer ***"
tools:
exclude: [delete_customer, refund_payment]
```
Use it like:
```text
Look up the last 10 failed payments and summarize common failure reasons.
```
### Filesystem server for a single project root
```yaml
mcp_servers:
project_fs:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-filesystem", "/home/user/my-project"]
```
Use it like:
```text
Inspect the project root and explain the directory layout.
```
## Troubleshooting
### MCP server not connecting
Check:
```bash
# Verify MCP deps are installed (already included in standard install)
cd ~/.hermes/hermes-agent && uv pip install -e ".[mcp]"
node --version
npx --version
```
Then verify your config and restart Hermes.
### Tools not appearing
Possible causes:
- the server failed to connect
- discovery failed
- your filter config excluded the tools
- the utility capability does not exist on that server
- the server is disabled with `enabled: false`
If you are intentionally filtering, this is expected.
### Why didn't resource or prompt utilities appear?
Because Hermes now only registers those wrappers when both are true:
1. your config allows them
2. the server session actually supports the capability
This is intentional and keeps the tool list honest.
## Parallel Tool Calls
By default, MCP tools run sequentially — one at a time. If your MCP server exposes tools that are safe to run concurrently (e.g. read-only queries, independent API calls), you can opt-in to parallel execution:
```yaml
mcp_servers:
docs:
command: "docs-server"
supports_parallel_tool_calls: true
```
When `supports_parallel_tool_calls` is `true`, Hermes may execute multiple tools from that server at the same time within a single tool-call batch, just like it does for built-in read-only tools (web_search, read_file, etc.).
:::caution
Only enable parallel calls for MCP servers whose tools are safe to run at the same time. If tools read and write shared state, files, databases, or external resources, review the read/write race conditions before enabling this setting.
:::
## MCP Sampling Support
MCP servers can request LLM inference from Hermes via the `sampling/createMessage` protocol. This allows an MCP server to ask Hermes to generate text on its behalf — useful for servers that need LLM capabilities but don't have their own model access.
Sampling is **enabled by default** for all MCP servers (when the MCP SDK supports it). Configure it per-server under the `sampling` key:
```yaml
mcp_servers:
my_server:
command: "my-mcp-server"
sampling:
enabled: true # Enable sampling (default: true)
model: "openai/gpt-4o" # Override model for sampling requests (optional)
max_tokens_cap: 4096 # Max tokens per sampling response (default: 4096)
timeout: 30 # Timeout in seconds per request (default: 30)
max_rpm: 10 # Rate limit: max requests per minute (default: 10)
max_tool_rounds: 5 # Max tool-use rounds in sampling loops (default: 5)
allowed_models: [] # Allowlist of model names the server may request (empty = any)
log_level: "info" # Audit log level: debug, info, or warning (default: info)
```
The sampling handler includes a sliding-window rate limiter, per-request timeouts, and tool-loop depth limits to prevent runaway usage. Metrics (request count, errors, tokens used) are tracked per server instance.
To disable sampling for a specific server:
```yaml
mcp_servers:
untrusted_server:
url: "https://mcp.example.com"
sampling:
enabled: false
```
## MCP Elicitation Support
MCP servers can ask the user for structured input mid-tool-call via the `elicitation/create` protocol (mcp Python SDK ≥ 1.11.0). Hermes routes **form-mode** elicitations through its existing approval surface — an interactive prompt in the CLI/TUI, or approval buttons on gateway platforms like Telegram and Slack — so the request reaches you wherever the session lives. **URL-mode** elicitations (where a server points you at an external URL) are declined as unsupported.
Elicitation is **enabled by default** per server. Configure it under the `elicitation` key:
```yaml
mcp_servers:
my_server:
command: "my-mcp-server"
elicitation:
enabled: true # default: true
timeout: 300 # seconds to wait for your answer (default: 300)
```
The 5-minute default timeout mirrors the gateway approval default so users on async surfaces have time to respond before the server gives up. Per-server metrics (requests, accepted, declined, errors) are tracked on the handler.
## Running Hermes as an MCP server
In addition to connecting **to** MCP servers, Hermes can also **be** an MCP server. This lets other MCP-capable agents (Claude Code, Cursor, Codex, or any MCP client) use Hermes's messaging capabilities — list conversations, read message history, and send messages across all your connected platforms.
### When to use this
- You want Claude Code, Cursor, or another coding agent to send and read Telegram/Discord/Slack messages through Hermes
- You want a single MCP server that bridges to all of Hermes's connected messaging platforms at once
- You already have a running Hermes gateway with connected platforms
### Quick start
```bash
hermes mcp serve
```
This starts a stdio MCP server. The MCP client (not you) manages the process lifecycle.
### MCP client configuration
Add Hermes to your MCP client config. For example, in Claude Code's `~/.claude/claude_desktop_config.json`:
```json
{
"mcpServers": {
"hermes": {
"command": "hermes",
"args": ["mcp", "serve"]
}
}
}
```
Or if you installed Hermes in a specific location:
```json
{
"mcpServers": {
"hermes": {
"command": "/home/user/.hermes/hermes-agent/venv/bin/hermes",
"args": ["mcp", "serve"]
}
}
}
```
### Available tools
The MCP server exposes 10 tools, matching OpenClaw's channel bridge surface plus a Hermes-specific channel browser:
| Tool | Description |
|------|-------------|
| `conversations_list` | List active messaging conversations. Filter by platform or search by name. |
| `conversation_get` | Get detailed info about one conversation by session key. |
| `messages_read` | Read recent message history for a conversation. |
| `attachments_fetch` | Extract non-text attachments (images, media) from a specific message. |
| `events_poll` | Poll for new conversation events since a cursor position. |
| `events_wait` | Long-poll / block until the next event arrives (near-real-time). |
| `messages_send` | Send a message through a platform (e.g. `telegram:123456`, `discord:#general`). |
| `channels_list` | List available messaging targets across all platforms. |
| `permissions_list_open` | List pending approval requests observed during this bridge session. |
| `permissions_respond` | Allow or deny a pending approval request. |
### Event system
The MCP server includes a live event bridge that polls Hermes's session database for new messages. This gives MCP clients near-real-time awareness of incoming conversations:
```
# Poll for new events (non-blocking)
events_poll(after_cursor=0)
# Wait for next event (blocks up to timeout)
events_wait(after_cursor=42, timeout_ms=30000)
```
Event types: `message`, `approval_requested`, `approval_resolved`
The event queue is in-memory and starts when the bridge connects. Older messages are available through `messages_read`.
### Options
```bash
hermes mcp serve # Normal mode
hermes mcp serve --verbose # Debug logging on stderr
```
### How it works
The MCP server reads conversation data directly from Hermes's session store — `~/.hermes/state.db` is the primary source, with `sessions.json` kept only as a legacy fallback. A background thread polls the database for new messages and maintains an in-memory event queue. For sending messages, it uses the same internal send engine (`tools/send_message_tool.py`) that powers cron delivery and the `hermes send` CLI.
The gateway does NOT need to be running for read operations (listing conversations, reading history, polling events). It DOES need to be running for send operations, since the platform adapters need active connections.
### Current limits
- The embedded `hermes mcp serve` exposes a **stdio-only** MCP server today. If you need an HTTP MCP server, run a separate adapter — or, much more commonly, use the MCP **client** side of Hermes, which already speaks both stdio and HTTP (`url` + `headers` in `mcp_servers.yaml` / `config.yaml`; see [HTTP servers](#http-servers) above).
- Event polling at ~200ms intervals via mtime-optimized DB polling (skips work when files are unchanged)
- No `claude/channel` push notification protocol yet
- Text-only sends (no media/attachment sending through `messages_send`)
## Related docs
- [Use MCP with Hermes](/guides/use-mcp-with-hermes)
- [CLI Commands](/reference/cli-commands)
- [Slash Commands](/reference/slash-commands)
- [FAQ](/reference/faq)
website/docs/user-guide/features/memory-providers.md
+698
View File
@@ -0,0 +1,698 @@
---
sidebar_position: 4
title: "Memory Providers"
description: "External memory provider plugins — Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, Supermemory"
---
# Memory Providers
Hermes Agent ships with 8 external memory provider plugins that give the agent persistent, cross-session knowledge beyond the built-in MEMORY.md and USER.md. Only **one** external provider can be active at a time — the built-in memory is always active alongside it.
## Quick Start
```bash
hermes memory setup # interactive picker + configuration
hermes memory status # check what's active
hermes memory off # disable external provider
```
You can also select the active memory provider via `hermes plugins` → Provider Plugins → Memory Provider.
Or set manually in `~/.hermes/config.yaml`:
```yaml
memory:
provider: openviking # or honcho, mem0, hindsight, holographic, retaindb, byterover, supermemory
```
## How It Works
When a memory provider is active, Hermes automatically:
1. **Injects provider context** into the system prompt (what the provider knows)
2. **Prefetches relevant memories** before each turn (background, non-blocking)
3. **Syncs conversation turns** to the provider after each response
4. **Extracts memories on session end** (for providers that support it)
5. **Mirrors built-in memory writes** to the external provider
6. **Adds provider-specific tools** so the agent can search, store, and manage memories
The built-in memory (MEMORY.md / USER.md) continues to work exactly as before. The external provider is additive.
## Available Providers
### Honcho
AI-native cross-session user modeling with dialectic reasoning, session-scoped context injection, semantic search, and persistent conclusions. Base context now includes the session summary alongside user representation and peer cards, giving the agent awareness of what has already been discussed.
| | |
|---|---|
| **Best for** | Multi-agent systems with cross-session context, user-agent alignment |
| **Requires** | `pip install honcho-ai` + [API key](https://app.honcho.dev) or self-hosted instance |
| **Data storage** | Honcho Cloud or self-hosted |
| **Cost** | Honcho pricing (cloud) / free (self-hosted) |
**Tools (5):** `honcho_profile` (read/update peer card), `honcho_search` (semantic search), `honcho_context` (session context — summary, representation, card, messages), `honcho_reasoning` (LLM-synthesized), `honcho_conclude` (create/delete conclusions)
**Architecture:** Two-layer context injection — a base layer (session summary + representation + peer card, refreshed on `contextCadence`) plus a dialectic supplement (LLM reasoning, refreshed on `dialecticCadence`). The dialectic automatically selects cold-start prompts (general user facts) vs. warm prompts (session-scoped context) based on whether base context exists.
**Three orthogonal config knobs** control cost and depth independently:
- `contextCadence` — how often the base layer refreshes (API call frequency)
- `dialecticCadence` — how often the dialectic LLM fires (LLM call frequency)
- `dialecticDepth` — how many `.chat()` passes per dialectic invocation (1–3, depth of reasoning)
The auto-injected dialectic also scales its reasoning level by query length (longer query → deeper reasoning, capped at `reasoningLevelCap`); see [Query-Adaptive Reasoning Level](./honcho.md#query-adaptive-reasoning-level).
**Setup Wizard:**
```bash
hermes memory setup # select "honcho" — runs the Honcho-specific post-setup
```
The legacy `hermes honcho setup` command still works (it now redirects to `hermes memory setup`), but is only registered after Honcho is selected as the active memory provider.
**Headless / remote machines:** for cloud auth on a box without a browser (SSH, remote VM), pick **device** at the wizard's auth-method prompt. The CLI prints a short code and a verification link; open the link in a browser on any other machine, approve, and setup completes — no API key copy-paste. The wizard defaults to this option automatically when it detects no usable local browser.
**Config:** `$HERMES_HOME/honcho.json` (profile-local) or `~/.honcho/config.json` (global). Resolution order: `$HERMES_HOME/honcho.json` > `~/.hermes/honcho.json` > `~/.honcho/config.json`. See the [config reference](https://github.com/NousResearch/hermes-agent/blob/main/plugins/memory/honcho/README.md) and the [Honcho integration guide](https://docs.honcho.dev/v3/guides/integrations/hermes).
<details>
<summary>Full config reference</summary>
| Key | Default | Description |
|-----|---------|-------------|
| `apiKey` | -- | API key from [app.honcho.dev](https://app.honcho.dev) |
| `baseUrl` | -- | Base URL for self-hosted Honcho |
| `peerName` | -- | User peer identity |
| `aiPeer` | host key | AI peer identity (one per profile) |
| `workspace` | host key | Shared workspace ID |
| `contextTokens` | `null` (uncapped) | Token budget for auto-injected context per turn. Truncates at word boundaries |
| `contextCadence` | `1` | Minimum turns between `context()` API calls (base layer refresh) |
| `dialecticCadence` | `2` | Minimum turns between `peer.chat()` LLM calls. Recommended 1–5. Only applies to `hybrid`/`context` modes |
| `dialecticDepth` | `1` | Number of `.chat()` passes per dialectic invocation. Clamped 1–3. Pass 0: cold/warm prompt, pass 1: self-audit, pass 2: reconciliation |
| `dialecticDepthLevels` | `null` | Optional array of reasoning levels per pass, e.g. `["minimal", "low", "medium"]`. Overrides proportional defaults |
| `dialecticReasoningLevel` | `'low'` | Base reasoning level: `minimal`, `low`, `medium`, `high`, `max` |
| `dialecticDynamic` | `true` | When `true`, model can override reasoning level per-call via tool param |
| `dialecticMaxChars` | `600` | Max chars of dialectic result injected into system prompt |
| `recallMode` | `'hybrid'` | `hybrid` (auto-inject + tools), `context` (inject only), `tools` (tools only) |
| `writeFrequency` | `'async'` | When to flush messages: `async` (background thread), `turn` (sync), `session` (batch on end), or integer N |
| `saveMessages` | `true` | Whether to persist messages to Honcho API |
| `observationMode` | `'directional'` | `directional` (all on) or `unified` (shared pool). Override with `observation` object |
| `messageMaxChars` | `25000` | Max chars per message (chunked if exceeded) |
| `dialecticMaxInputChars` | `10000` | Max chars for dialectic query input to `peer.chat()` |
| `sessionStrategy` | `'per-directory'` | `per-directory`, `per-repo`, `per-session`, `global` |
| `pinUserPeer` | `false` | Gateway only. When `true`, every non-agent gateway user collapses to `peerName`; the pin overrides all aliases |
| `userPeerAliases` | `{}` | Gateway only. Maps runtime IDs to peers (`{"7654321": "alice"}`). Many-to-one |
| `runtimePeerPrefix` | `""` | Gateway only. Namespaces unknown runtime IDs (`telegram_7654321`) when no alias matches |
</details>
<details>
<summary>Minimal honcho.json (cloud)</summary>
```json
{
"apiKey": "your-key-from-app.honcho.dev",
"hosts": {
"hermes": {
"enabled": true,
"aiPeer": "hermes",
"peerName": "your-name",
"workspace": "hermes"
}
}
}
```
</details>
<details>
<summary>Minimal honcho.json (self-hosted)</summary>
```json
{
"baseUrl": "http://localhost:8000",
"hosts": {
"hermes": {
"enabled": true,
"aiPeer": "hermes",
"peerName": "your-name",
"workspace": "hermes"
}
}
}
```
</details>
:::tip Migrating from `hermes honcho`
If you previously used `hermes honcho setup`, your config and all server-side data are intact. Just re-enable through the setup wizard again or manually set `memory.provider: honcho` to reactivate via the new system.
:::
**Multi-peer setup:**
Honcho models conversations as peers exchanging messages — one user peer plus one AI peer per Hermes profile, all sharing a workspace. The workspace is the shared environment: the user peer is global across profiles, each AI peer is its own identity. Every AI peer builds an independent representation / card from its own observations, so a `coder` profile stays code-oriented while a `writer` profile stays editorial against the same user.
The mapping:
| Concept | What it is |
|---------|-----------|
| **Workspace** | Shared environment. All Hermes profiles under one workspace see the same user identity. |
| **User peer** (`peerName`) | The human. Shared across profiles in the workspace. |
| **AI peer** (`aiPeer`) | One per Hermes profile. Host key `hermes` → default; `hermes.<profile>` for others. |
| **Observation** | Per-peer toggles controlling what Honcho models from whose messages. `directional` (default, all four on) or `unified` (single-observer pool). |
### New profile, fresh Honcho peer
```bash
hermes profile create coder --clone
```
`--clone` creates a `hermes.coder` host block in `honcho.json` with `aiPeer: "coder"`, shared `workspace`, inherited `peerName`, `recallMode`, `writeFrequency`, `observation`, etc. The AI peer is eagerly created in Honcho so it exists before the first message.
### Existing profiles, backfill Honcho peers
```bash
hermes honcho sync
```
Scans every Hermes profile, creates host blocks for any profile without one, inherits settings from the default `hermes` block, and creates the new AI peers eagerly. Idempotent — skips profiles that already have a host block.
### Per-profile observation
Each host block can override the observation config independently. Example: a code-focused profile where the AI peer observes the user but doesn't self-model:
```json
"hermes.coder": {
"aiPeer": "coder",
"observation": {
"user": { "observeMe": true, "observeOthers": true },
"ai": { "observeMe": false, "observeOthers": true }
}
}
```
**Observation toggles (one set per peer):**
| Toggle | Effect |
|--------|--------|
| `observeMe` | Honcho builds a representation of this peer from its own messages |
| `observeOthers` | This peer observes the other peer's messages (feeds cross-peer reasoning) |
Presets via `observationMode`:
- **`"directional"`** (default) — all four flags on. Full mutual observation; enables cross-peer dialectic.
- **`"unified"`** — user `observeMe: true`, AI `observeOthers: true`, rest false. Single-observer pool; AI models the user but not itself, user peer only self-models.
Server-side toggles set via the [Honcho dashboard](https://app.honcho.dev) win over local defaults — synced back at session init.
See the [Honcho page](./honcho.md#observation-directional-vs-unified) for the full observation reference.
### Gateway identity mapping
The peer model above covers CLI, TUI, and desktop sessions, where every conversation resolves to `peerName`. The [gateway](../../developer-guide/gateway-internals.md) adds a second axis: users arrive with platform-native runtime IDs (Telegram UID, Discord snowflake, Slack user), and three keys decide which peer each ID resolves to.
| Key | Effect |
|-----|--------|
| `pinUserPeer: true` | Every non-agent gateway user collapses to `peerName`. The pin is checked first, so it overrides all aliases — pick it only when no user-side identity needs its own peer |
| `userPeerAliases` | Maps specific runtime IDs to peers (`{"7654321": "alice"}`). The home for routing distinct identities — including agents that each carry their own peer |
| `runtimePeerPrefix` | Namespaces any unmapped runtime ID (`telegram_7654321`) so platforms with same-shaped IDs don't collide |
Off-gateway these keys do nothing. `hermes memory setup` only prompts for them when it detects a connected gateway platform. See the [Honcho page](./honcho.md#gateway-identity-mapping) for the resolver ladder and the setup flow.
<details>
<summary>Full honcho.json example (multi-profile)</summary>
```json
{
"apiKey": "your-key",
"workspace": "hermes",
"peerName": "eri",
"hosts": {
"hermes": {
"enabled": true,
"aiPeer": "hermes",
"workspace": "hermes",
"peerName": "eri",
"recallMode": "hybrid",
"writeFrequency": "async",
"sessionStrategy": "per-directory",
"observation": {
"user": { "observeMe": true, "observeOthers": true },
"ai": { "observeMe": true, "observeOthers": true }
},
"dialecticReasoningLevel": "low",
"dialecticDynamic": true,
"dialecticCadence": 2,
"dialecticDepth": 1,
"dialecticMaxChars": 600,
"contextCadence": 1,
"messageMaxChars": 25000,
"saveMessages": true
},
"hermes.coder": {
"enabled": true,
"aiPeer": "coder",
"workspace": "hermes",
"peerName": "eri",
"recallMode": "tools",
"observation": {
"user": { "observeMe": true, "observeOthers": false },
"ai": { "observeMe": true, "observeOthers": true }
}
},
"hermes.writer": {
"enabled": true,
"aiPeer": "writer",
"workspace": "hermes",
"peerName": "eri"
}
},
"sessions": {
"/home/user/myproject": "myproject-main"
}
}
```
</details>
See the [config reference](https://github.com/NousResearch/hermes-agent/blob/main/plugins/memory/honcho/README.md) and [Honcho integration guide](https://docs.honcho.dev/v3/guides/integrations/hermes).
---
### OpenViking
Context database by Volcengine (ByteDance) with filesystem-style knowledge hierarchy, tiered retrieval, and automatic memory extraction into 6 categories.
| | |
|---|---|
| **Best for** | Self-hosted knowledge management with structured browsing |
| **Requires** | OpenViking initialized, validated, and running |
| **Data storage** | Self-hosted (local or cloud) |
| **Cost** | Free (open-source, AGPL-3.0) |
**Tools (6):** `viking_search` (semantic search), `viking_read` (tiered: abstract/overview/full), `viking_browse` (filesystem navigation), `viking_remember` (store facts), `viking_forget` (delete a memory file by exact `viking://` URI), `viking_add_resource` (ingest URLs/docs)
**Setup:**
```bash
# Prepare OpenViking first
openviking-server init
openviking-server doctor
openviking-server
# Then configure Hermes
hermes memory setup # select "openviking"
# Or manually:
hermes config set memory.provider openviking
```
`hermes memory setup` can reuse or copy connection values from
`~/.openviking/ovcli.conf`. Manual setup uses the active profile's `.env` file;
for the default profile that is `~/.hermes/.env`, and for named profiles use
`~/.hermes/profiles/<profile>/.env`.
```text
OPENVIKING_ENDPOINT=http://127.0.0.1:1933
# OPENVIKING_API_KEY=...
# OPENVIKING_ACCOUNT=default
# OPENVIKING_USER=default
```
OpenViking server settings live in `ov.conf` (`--config`,
`OPENVIKING_CONFIG_FILE`, or `~/.openviking/ov.conf`). Client connection values
live in `ovcli.conf` (`OPENVIKING_CLI_CONFIG_FILE` or
`~/.openviking/ovcli.conf`).
**Key features:**
- Tiered context loading: L0 (~100 tokens) → L1 (~2k) → L2 (full)
- Automatic memory extraction on session commit (profile, preferences, entities, events, cases, patterns)
- `viking://` URI scheme for hierarchical knowledge browsing
`OPENVIKING_ACCOUNT` and `OPENVIKING_USER` are used for local/trusted mode.
Peer identity is optional. By default, Hermes sends no peer ID and writes
explicit memories to `viking://user/<user>/memories/...`. Setup does not ask
for a peer ID. For separate assistant context, set
`memory.openviking.agent: work-assistant` in `config.yaml`.
Existing non-empty peer settings keep their peer-scoped writes and recall.
This includes `OPENVIKING_AGENT` and `actor_peer_id` or legacy `agent_id` in a
linked OpenViking config. Existing memories are not moved or deleted.
With no peer ID, default search covers user memory and existing peer memories
under the same OpenViking user. Old peer memories remain searchable at their
existing paths. Ranking and result limits determine which memories are returned.
Set `memory.openviking.agent: hermes` to restore the old peer-scoped writes.
Memories written at user scope before this change stay there and remain
searchable. The setting changes future writes, not existing memory locations.
Hermes sends `User-Agent: openviking-memory-hermes/<version>` on OpenViking
requests. This standard harness identifier contains no per-user identifier and
does not add a separate request.
---
### Mem0
Server-side LLM fact extraction with semantic search, reranking, and automatic deduplication. Three connection modes: **Platform** (Mem0 Cloud), **self-hosted dashboard** (a Mem0 server you run via Docker), and **OSS** (Mem0 in-process with your own LLM + vector store).
| | |
|---|---|
| **Best for** | Hands-off memory management — Mem0 handles extraction automatically |
| **Requires** | `pip install mem0ai` + API key (platform), a running Mem0 server (self-hosted dashboard), or an LLM + vector store (OSS) |
| **Data storage** | Mem0 Cloud (platform), your own Mem0 server (self-hosted dashboard), or in-process (OSS) |
| **Cost** | Mem0 pricing (platform) / free (self-hosted or OSS) |
**Tools (4):** `mem0_search` (semantic search; optional reranking in platform mode, off by default), `mem0_add` (store verbatim facts), `mem0_update` (update by ID), `mem0_delete` (delete by ID)
**Setup (Platform):**
```bash
hermes memory setup # select "mem0" → "Platform"
# Or manually:
hermes config set memory.provider mem0
echo "MEM0_API_KEY=your-key" >> ~/.hermes/.env
```
**Setup (OSS):**
```bash
hermes memory setup # select "mem0" → "Open Source (self-hosted)"
# Or via flags:
hermes memory setup mem0 --mode oss --oss-llm openai --oss-llm-key sk-... --oss-vector qdrant
```
Preview without writing files:
```bash
hermes memory setup mem0 --mode oss --oss-llm-key sk-... --dry-run
```
**Setup (Self-Hosted Dashboard):** connect to a Mem0 server you run via Docker (the dashboard's REST API):
```bash
hermes memory setup # select "mem0" → "Self-hosted server"
# Or via flags:
hermes memory setup mem0 --mode selfhosted --host http://localhost:8888 --api-key your-admin-api-key
```
Or configure manually — either as env vars:
```bash
echo "MEM0_HOST=http://localhost:8888" >> ~/.hermes/.env
echo "MEM0_API_KEY=your-admin-api-key" >> ~/.hermes/.env
```
or in `mem0.json`:
```json
{ "host": "http://localhost:8888", "api_key": "your-admin-api-key" }
```
The plugin authenticates with `X-API-Key` and uses the server's `/search` / `/memories` routes. `api_key` is optional (omit only for `AUTH_DISABLED` servers). Don't set `mode: oss` — it takes precedence over `host`.
**Config:** `$HERMES_HOME/mem0.json` (behavioral settings). Only the secret `MEM0_API_KEY` belongs in `~/.hermes/.env`.
| Key | Default | Description |
|-----|---------|-------------|
| `mode` | `platform` | `platform` (Mem0 Cloud) or `oss` (self-managed, in-process) |
| `host` | — | Self-hosted Mem0 server URL (Docker dashboard). Routes over HTTP with `X-API-Key`; don't combine with `mode: oss` |
| `user_id` | `hermes-user` | User identifier |
| `agent_id` | `hermes` | Agent identifier |
| `rerank` | `false` | Rerank search results for relevance (platform mode only) |
**OSS supported providers:**
| Component | Providers |
|-----------|-----------|
| LLM | openai, ollama |
| Embedder | openai, ollama |
| Vector Store | qdrant (local/server), pgvector |
**Switching modes:** Re-run `hermes memory setup mem0 --mode <platform|selfhosted|oss>` or edit `mem0.json` directly.
---
### Hindsight
Long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval. The `hindsight_reflect` tool provides cross-memory synthesis that no other provider offers. Automatically retains full conversation turns (including tool calls) with session-level document tracking.
| | |
|---|---|
| **Best for** | Knowledge graph-based recall with entity relationships |
| **Requires** | Cloud: API key from [ui.hindsight.vectorize.io](https://ui.hindsight.vectorize.io). Local: LLM API key (OpenAI, Groq, OpenRouter, etc.) |
| **Data storage** | Hindsight Cloud or local embedded PostgreSQL |
| **Cost** | Hindsight pricing (cloud) or free (local) |
**Tools:** `hindsight_retain` (store with entity extraction), `hindsight_recall` (multi-strategy search), `hindsight_reflect` (cross-memory synthesis)
**Setup:**
```bash
hermes memory setup # select "hindsight"
# Or manually:
hermes config set memory.provider hindsight
echo "HINDSIGHT_API_KEY=your-key" >> ~/.hermes/.env
```
The setup wizard installs dependencies automatically and only installs what's needed for the selected mode (`hindsight-client` for cloud, `hindsight-all` for local). Requires `hindsight-client >= 0.4.22` (auto-upgraded on session start if outdated).
**Local mode UI:** `hindsight-embed -p hermes ui start`
**Config:** `$HERMES_HOME/hindsight/config.json`
| Key | Default | Description |
|-----|---------|-------------|
| `mode` | `cloud` | `cloud` or `local` |
| `bank_id` | `hermes` | Memory bank identifier |
| `recall_budget` | `mid` | Recall thoroughness: `low` / `mid` / `high` |
| `memory_mode` | `hybrid` | `hybrid` (context + tools), `context` (auto-inject only), `tools` (tools only) |
| `auto_retain` | `true` | Automatically retain conversation turns |
| `auto_recall` | `true` | Automatically recall memories before each turn |
| `retain_async` | `true` | Process retain asynchronously on the server |
| `retain_context` | `conversation between Hermes Agent and the User` | Context label for retained memories |
| `retain_tags` | — | Default tags applied to retained memories; merged with per-call tool tags |
| `retain_source` | — | Optional `metadata.source` attached to retained memories |
| `retain_user_prefix` | `User` | Label used before user turns in auto-retained transcripts |
| `retain_assistant_prefix` | `Assistant` | Label used before assistant turns in auto-retained transcripts |
| `recall_tags` | — | Tags to filter on recall |
See [plugin README](https://github.com/NousResearch/hermes-agent/blob/main/plugins/memory/hindsight/README.md) for the full configuration reference.
---
### Holographic
Local SQLite fact store with FTS5 full-text search, trust scoring, and HRR (Holographic Reduced Representations) for compositional algebraic queries.
| | |
|---|---|
| **Best for** | Local-only memory with advanced retrieval, no external dependencies |
| **Requires** | Nothing (SQLite is always available). NumPy optional for HRR algebra. |
| **Data storage** | Local SQLite |
| **Cost** | Free |
**Tools:** `fact_store` (9 actions: add, search, probe, related, reason, contradict, update, remove, list), `fact_feedback` (helpful/unhelpful rating that trains trust scores)
**Setup:**
```bash
hermes memory setup # select "holographic"
# Or manually:
hermes config set memory.provider holographic
```
**Config:** `config.yaml` under `plugins.hermes-memory-store`
| Key | Default | Description |
|-----|---------|-------------|
| `db_path` | `$HERMES_HOME/memory_store.db` | SQLite database path |
| `auto_extract` | `false` | Auto-extract facts at session end |
| `default_trust` | `0.5` | Default trust score (0.0–1.0) |
**Unique capabilities:**
- `probe` — entity-specific algebraic recall (all facts about a person/thing)
- `reason` — compositional AND queries across multiple entities
- `contradict` — automated detection of conflicting facts
- Trust scoring with asymmetric feedback (+0.05 helpful / -0.10 unhelpful)
---
### RetainDB
Cloud memory API with hybrid search (Vector + BM25 + Reranking), 7 memory types, and delta compression.
| | |
|---|---|
| **Best for** | Teams already using RetainDB's infrastructure |
| **Requires** | RetainDB account + API key |
| **Data storage** | RetainDB Cloud |
| **Cost** | $20/month |
**Tools (10):** `retaindb_profile` (user profile), `retaindb_search` (semantic search), `retaindb_context` (task-relevant context), `retaindb_remember` (store with type + importance), `retaindb_forget` (delete memories), plus file tools: `retaindb_upload_file`, `retaindb_list_files`, `retaindb_read_file`, `retaindb_ingest_file`, `retaindb_delete_file`
**Setup:**
```bash
hermes memory setup # select "retaindb"
# Or manually:
hermes config set memory.provider retaindb
echo "RETAINDB_API_KEY=your-key" >> ~/.hermes/.env
```
---
### ByteRover
Persistent memory via the `brv` CLI — hierarchical knowledge tree with tiered retrieval (fuzzy text → LLM-driven search). Local-first with optional cloud sync.
| | |
|---|---|
| **Best for** | Developers who want portable, local-first memory with a CLI |
| **Requires** | ByteRover CLI (`npm install -g byterover-cli` or [install script](https://byterover.dev)) |
| **Data storage** | Local (default) or ByteRover Cloud (optional sync) |
| **Cost** | Free (local) or ByteRover pricing (cloud) |
**Tools:** `brv_query` (search knowledge tree), `brv_curate` (store facts/decisions/patterns), `brv_status` (CLI version + tree stats)
**Setup:**
```bash
# Install the CLI first
curl -fsSL https://byterover.dev/install.sh | sh
# Then configure Hermes
hermes memory setup # select "byterover"
# Or manually:
hermes config set memory.provider byterover
```
**Key features:**
- Automatic pre-compression extraction (saves insights before context compression discards them)
- Knowledge tree stored at `$HERMES_HOME/byterover/` (profile-scoped)
- SOC2 Type II certified cloud sync (optional)
---
### Supermemory
Semantic long-term memory with profile recall, semantic search, explicit memory tools, and session-end conversation ingest via the Supermemory graph API.
| | |
|---|---|
| **Best for** | Semantic recall with user profiling and session-level graph building |
| **Requires** | `pip install supermemory` + [cloud API key](http://app.supermemory.ai/integrations?connect=hermes), or a [self-hosted server](https://supermemory.ai/docs/self-hosting/overview) |
| **Data storage** | Supermemory Cloud or self-hosted |
| **Cost** | Supermemory pricing (cloud) / free (self-hosted) |
**Tools:** `supermemory_store` (save explicit memories), `supermemory_search` (semantic similarity search), `supermemory_forget` (forget by ID or best-match query), `supermemory_profile` (persistent profile + recent context)
**Setup:**
```bash
hermes memory setup # select "supermemory"
# Or manually:
hermes config set memory.provider supermemory
echo 'SUPERMEMORY_API_KEY=***' >> ~/.hermes/.env
```
Self-hosted setup:
```bash
npx supermemory local
```
Before running `hermes memory setup`, set `base_url` in
`$HERMES_HOME/supermemory.json`:
```json
{
"base_url": "http://localhost:6767"
}
```
Then run `hermes memory setup` and enter the API key printed by the local
server. Configuring the endpoint first ensures the setup connection probe also
stays local.
**Config:** `$HERMES_HOME/supermemory.json`
| Key | Default | Description |
|-----|---------|-------------|
| `base_url` | `https://api.supermemory.ai` | API endpoint for hosted or self-hosted Supermemory. Takes priority over `SUPERMEMORY_BASE_URL`. |
| `container_tag` | `hermes` | Container tag used for search and writes. Supports `{identity}` template for profile-scoped tags. |
| `auto_recall` | `true` | Inject relevant memory context before turns |
| `auto_capture` | `true` | Store cleaned user-assistant turns after each response |
| `max_recall_results` | `10` | Max recalled items to format into context |
| `profile_frequency` | `50` | Include profile facts on first turn and every N turns |
| `capture_mode` | `all` | Skip tiny or trivial turns by default |
| `search_mode` | `hybrid` | Search mode: `hybrid`, `memories`, or `documents` |
| `api_timeout` | `5.0` | Timeout for SDK and ingest requests |
**Environment variables:** `SUPERMEMORY_API_KEY` (required), `SUPERMEMORY_BASE_URL` (compatibility fallback when `base_url` is not configured), `SUPERMEMORY_CONTAINER_TAG` (overrides config).
Base URL precedence is `supermemory.json` → `SUPERMEMORY_BASE_URL` → `https://api.supermemory.ai`. SDK operations, setup/status probes, and conversation ingest all use the resolved endpoint.
**Key features:**
- Automatic context fencing — strips recalled memories from captured turns to prevent recursive memory pollution
- Full-session ingest — the entire conversation is sent once at session boundaries
- Session-end conversation ingest (to `/v4/conversations`) for richer profile + graph building in Supermemory
- End-to-end self-hosted routing — SDK, probe, and conversation-ingest requests use the same configured endpoint
- Profile facts injected on first turn and at configurable intervals
- **Profile-scoped containers** — use `{identity}` in `container_tag` (e.g. `hermes-{identity}` → `hermes-coder`) to isolate memories per Hermes profile
- **Multi-container mode** — enable `enable_custom_container_tags` with a `custom_containers` list to let the agent read/write across named containers. Automatic operations stay on the primary container.
<details>
<summary>Multi-container example</summary>
```json
{
"container_tag": "hermes",
"enable_custom_container_tags": true,
"custom_containers": ["project-alpha", "shared-knowledge"],
"custom_container_instructions": "Use project-alpha for coding context."
}
```
</details>
**Support:** [Discord](https://supermemory.link/discord) · [support@supermemory.com](mailto:support@supermemory.com)
### Memori
Structured long-term memory using Memori Cloud, with background completed-turn capture, tool-aware turn context, and explicit recall tools for facts, summaries, quota, signup, and feedback.
| | |
|---|---|
| **Best for** | Agent-controlled recall with structured project and session attribution |
| **Requires** | `pip install hermes-memori` + `hermes-memori install` + [Memori API key](https://app.memorilabs.ai/signup) |
| **Data storage** | Memori Cloud |
| **Cost** | Memori pricing |
**Tools:** `memori_recall` (search long-term memory), `memori_recall_summary` (summarized context), `memori_quota` (usage/quota), `memori_signup` (request signup email), `memori_feedback` (send integration feedback)
**Setup:**
```bash
pip install hermes-memori
hermes-memori install
hermes config set memory.provider memori
hermes memory setup
```
---
## Provider Comparison
| Provider | Storage | Cost | Tools | Dependencies | Unique Feature |
|----------|---------|------|-------|-------------|----------------|
| **Honcho** | Cloud | Paid | 5 | `honcho-ai` | Dialectic user modeling + session-scoped context |
| **OpenViking** | Self-hosted | Free | 6 | `openviking` + server | Filesystem hierarchy + tiered loading |
| **Mem0** | Cloud/Self-hosted | Free/Paid | 4 | `mem0ai` | Server-side LLM extraction + self-hosted/OSS modes |
| **Hindsight** | Cloud/Local | Free/Paid | 3 | `hindsight-client` | Knowledge graph + reflect synthesis |
| **Holographic** | Local | Free | 2 | None | HRR algebra + trust scoring |
| **RetainDB** | Cloud | $20/mo | 10 | `requests` | Delta compression |
| **ByteRover** | Local/Cloud | Free/Paid | 3 | `brv` CLI | Pre-compression extraction |
| **Supermemory** | Cloud/Self-hosted | Free/Paid | 4 | `supermemory` | Context fencing + session graph ingest + multi-container |
| **Memori** | Cloud | Free/Paid | 5 | `hermes-memori` | Tool-aware memory + structured recall |
## Profile Isolation
Each provider's data is isolated per [profile](/user-guide/profiles):
- **Local storage providers** (Holographic, ByteRover) use `$HERMES_HOME/` paths which differ per profile
- **Config file providers** (Honcho, Mem0, Hindsight, Supermemory) store config in `$HERMES_HOME/` so each profile has its own credentials
- **Cloud providers** (RetainDB) auto-derive profile-scoped project names
- **Env var providers** (OpenViking) are configured via each profile's `.env` file
## Building a Memory Provider
See the [Developer Guide: Memory Provider Plugins](/developer-guide/memory-provider-plugin) for how to create your own.
website/docs/user-guide/features/memory.md
+443
View File
@@ -0,0 +1,443 @@
---
sidebar_position: 3
title: "Persistent Memory"
description: "How Hermes Agent remembers across sessions — MEMORY.md, USER.md, and session search"
---
# Persistent Memory
Hermes Agent has bounded, curated memory that persists across sessions. This lets it remember your preferences, your projects, your environment, and things it has learned.
## How It Works
Two files make up the agent's memory:
| File | Purpose | Char Limit |
|------|---------|------------|
| **MEMORY.md** | Agent's personal notes — environment facts, conventions, things learned | 2,200 chars (~800 tokens) |
| **USER.md** | User profile — your preferences, communication style, expectations | 1,375 chars (~500 tokens) |
Both are stored in `~/.hermes/memories/` and are injected into the system prompt as a frozen snapshot at session start. The agent manages its own memory via the `memory` tool — it can add, replace, or remove entries.
:::caution One agent per Hermes home
Don't point two agent processes at the same Hermes home directory. Memory writes are automatic and load back into the system prompt at session start, so two writers sharing one home will compound each other's entries into state neither of them (nor you) authored. Memory is scoped per [profile](/user-guide/profiles) by design — give a second agent its own profile, and if they need shared memory, use an [external memory provider](/user-guide/features/memory-providers) instead.
:::
:::info
Character limits keep memory focused. Memory does **not** auto-compact: when a
write would exceed the limit, the `memory` tool returns an error instead of
silently dropping entries. The agent then makes room itself — consolidating or
removing entries in the same turn before retrying (see [What Happens When Memory
is Full](#what-happens-when-memory-is-full)). Note that `replace` is also bound
by the limit: swapping an entry for a longer one can still overflow, so the new
content must be shortened (or another entry removed) to fit.
:::
## How Memory Appears in the System Prompt
At the start of every session, memory entries are loaded from disk and rendered into the system prompt as a frozen block:
```
══════════════════════════════════════════════
MEMORY (your personal notes) [67% — 1,474/2,200 chars]
══════════════════════════════════════════════
User's project is a Rust web service at ~/code/myapi using Axum + SQLx
§
This machine runs Ubuntu 22.04, has Docker and Podman installed
§
User prefers concise responses, dislikes verbose explanations
```
The format includes:
- A header showing which store (MEMORY or USER PROFILE)
- Usage percentage and character counts so the agent knows capacity
- Individual entries separated by `§` (section sign) delimiters
- Entries can be multiline
**Frozen snapshot pattern:** The system prompt injection is captured once at session start and never changes mid-session. This is intentional — it preserves the LLM's prefix cache for performance. When the agent adds/removes memory entries during a session, the changes are persisted to disk immediately but won't appear in the system prompt until the next session starts. Tool responses always show the live state.
## Memory Tool Actions
The agent uses the `memory` tool with these actions:
- **add** — Add a new memory entry
- **replace** — Replace an existing entry with updated content (uses substring matching via `old_text`)
- **remove** — Remove an entry that's no longer relevant (uses substring matching via `old_text`)
There is no `read` action — memory content is automatically injected into the system prompt at session start. The agent sees its memories as part of its conversation context.
### Substring Matching
The `replace` and `remove` actions use short unique substring matching — you don't need the full entry text. The `old_text` parameter just needs to be a unique substring that identifies exactly one entry:
```python
# If memory contains "User prefers dark mode in all editors"
memory(action="replace", target="memory",
old_text="dark mode",
content="User prefers light mode in VS Code, dark mode in terminal")
```
If the substring matches multiple entries, an error is returned asking for a more specific match.
## Two Targets Explained
### `memory` — Agent's Personal Notes
For information the agent needs to remember about the environment, workflows, and lessons learned:
- Environment facts (OS, tools, project structure)
- Project conventions and configuration
- Tool quirks and workarounds discovered
- Completed task diary entries
- Skills and techniques that worked
### `user` — User Profile
For information about the user's identity, preferences, and communication style:
- Name, role, timezone
- Communication preferences (concise vs detailed, format preferences)
- Pet peeves and things to avoid
- Workflow habits
- Technical skill level
## What to Save vs Skip
### Save These (Proactively)
The agent saves automatically — you don't need to ask. It saves when it learns:
- **User preferences:** "I prefer TypeScript over JavaScript" → save to `user`
- **Environment facts:** "This server runs Debian 12 with PostgreSQL 16" → save to `memory`
- **Corrections:** "Don't use `sudo` for Docker commands, user is in docker group" → save to `memory`
- **Conventions:** "Project uses tabs, 120-char line width, Google-style docstrings" → save to `memory`
- **Completed work:** "Migrated database from MySQL to PostgreSQL on 2026-01-15" → save to `memory`
- **Explicit requests:** "Remember that my API key rotation happens monthly" → save to `memory`
### Skip These
- **Trivial/obvious info:** "User asked about Python" — too vague to be useful
- **Easily re-discovered facts:** "Python 3.12 supports f-string nesting" — can web search this
- **Raw data dumps:** Large code blocks, log files, data tables — too big for memory
- **Session-specific ephemera:** Temporary file paths, one-off debugging context
- **Information already in context files:** SOUL.md and AGENTS.md content
## Capacity Management
Memory has strict character limits to keep system prompts bounded:
| Store | Limit | Typical entries |
|-------|-------|----------------|
| memory | 2,200 chars | 8-15 entries |
| user | 1,375 chars | 5-10 entries |
### What Happens When Memory is Full
When you try to add an entry that would exceed the limit, the tool returns an error:
```json
{
"success": false,
"error": "Memory at 2,100/2,200 chars. Adding this entry (250 chars) would exceed the limit. Consolidate now: use 'replace' to merge overlapping entries into shorter ones or 'remove' stale or less important entries (see current_entries below), then retry this add — all in this turn.",
"current_entries": ["..."],
"usage": "2,100/2,200"
}
```
The agent should then:
1. Read the current entries (shown in the error response)
2. Identify entries that can be removed or consolidated
3. Use `replace` to merge related entries into shorter versions
4. Then `add` the new entry
**Best practice:** When memory is above 80% capacity (visible in the system prompt header), consolidate entries before adding new ones. For example, merge three separate "project uses X" entries into one comprehensive project description entry.
### Practical Examples of Good Memory Entries
**Compact, information-dense entries work best:**
```
# Good: Packs multiple related facts
User runs macOS 14 Sonoma, uses Homebrew, has Docker Desktop and Podman. Shell: zsh with oh-my-zsh. Editor: VS Code with Vim keybindings.
# Good: Specific, actionable convention
Project ~/code/api uses Go 1.22, sqlc for DB queries, chi router. Run tests with 'make test'. CI via GitHub Actions.
# Good: Lesson learned with context
The staging server (10.0.1.50) needs SSH port 2222, not 22. Key is at ~/.ssh/staging_ed25519.
# Bad: Too vague
User has a project.
# Bad: Too verbose
On January 5th, 2026, the user asked me to look at their project which is
located at ~/code/api. I discovered it uses Go version 1.22 and...
```
## Duplicate Prevention
The memory system automatically rejects exact duplicate entries. If you try to add content that already exists, it returns success with a "no duplicate added" message.
## Security Scanning
Memory entries are scanned for injection and exfiltration patterns before being accepted, since they're injected into the system prompt. Content matching threat patterns (prompt injection, credential exfiltration, SSH backdoors) or containing invisible Unicode characters is blocked.
## Session Search
Beyond MEMORY.md and USER.md, the agent can search its past conversations using the `session_search` tool:
- All CLI and messaging sessions are stored in SQLite (`~/.hermes/state.db`) with FTS5 full-text search
- Search queries return actual messages from the DB — no LLM summarization, no truncation
- The agent can find things it discussed weeks ago, even if they're not in its active memory
- The agent can also scroll forward/backward inside any session it finds
```bash
hermes sessions list # Browse past sessions
```
See [Session Search Tool](/user-guide/sessions#session-search-tool) for the three calling shapes (discovery / scroll / browse) and the response format.
### session_search vs memory
| Feature | Persistent Memory | Session Search |
|---------|------------------|----------------|
| **Capacity** | ~1,300 tokens total | Unlimited (all sessions) |
| **Speed** | Instant (in system prompt) | ~20ms FTS5 query, ~1ms scroll |
| **Cost** | Token cost in every prompt | Free — no LLM calls |
| **Use case** | Key facts always available | Finding specific past conversations |
| **Management** | Manually curated by agent | Automatic — all sessions stored |
| **Token cost** | Fixed per session (~1,300 tokens) | On-demand (searched when needed) |
**Memory** is for critical facts that should always be in context. **Session search** is for "did we discuss X last week?" queries where the agent needs to recall specifics from past conversations.
## Learning Journey (`/journey`)
The learning journey is a timeline view of everything Hermes has learned — saved skills and memory entries plotted over time (oldest at top, newest at bottom), with a playable "constellation" scrubber that replays the build-up. The same graph data drives three surfaces:
- **Classic CLI / standalone** — `hermes journey` (aliases: `hermes learning`, `hermes memory-graph`) renders the timeline in the terminal. Flags: `--play` animates the build-up (`--fps` to tune it), `--width`/`--height` override the render size, `--no-color` disables color, and `--json` dumps the raw graph payload.
- **TUI** — `/journey` (aliases: `/learning`, `/memory-graph`) opens the timeline as an overlay.
- **Desktop app** — `/journey` opens the Star Map / memory-graph panel, an interactive visual of the same nodes.
Beyond viewing, the journey is also where you **prune and correct** what Hermes has learned:
| Command | What it does |
|---------|--------------|
| `hermes journey list` | List node ids — skill names and `memory:<source>:<index>` ids for memory chunks. |
| `hermes journey delete <node> [-y]` | Delete a node. Skills are **archived** (restorable), memory chunks are removed. `-y` skips the confirmation. |
| `hermes journey edit <node>` | Open the node's content (a skill's `SKILL.md` or the memory chunk) in `$EDITOR`. |
The same `list` / `delete <id>` / `edit <id>` subcommands work from the in-chat `/journey` command on the CLI, and the desktop panel offers edit/delete on nodes directly.
## Configuration
```yaml
# In ~/.hermes/config.yaml
memory:
memory_enabled: true
user_profile_enabled: true
memory_char_limit: 2200 # ~800 tokens
user_char_limit: 1375 # ~500 tokens
write_approval: false # false = write freely (default) | true = require approval
```
Setting **both** `memory_enabled` and `user_profile_enabled` to `false` turns the
built-in stores off completely: the `memory` tool is dropped from the schema and
its guidance block is dropped from the system prompt, so the model is never told
about a tool it cannot use. An external provider set via `memory.provider`
(Hindsight, Mem0, Honcho, …) is unaffected and keeps its own tools — use this
when you want a third-party memory backend *instead of* the built-in files.
Listing `memory` under `agent.disabled_toolsets` is the heavier switch: it hides
external provider tools too.
With only `memory_enabled: false` (user profile still on), the tool stays —
it backs the profile store — but the system prompt swaps the full memory
guidance for a narrower profile-only block. The tool schema advertises only the
`user` target, and direct or staged writes to disabled `MEMORY.md` are rejected.
The inverse configuration advertises only `memory` and rejects `USER.md` writes.
## Controlling memory writes (`write_approval`)
By default the agent saves memory freely — including from the background
self-improvement review that runs after a turn. If you'd rather approve saves
first, set `memory.write_approval: true`. It's a simple on/off gate applied to
**both** foreground turns and the background review:
| `write_approval` | Behaviour |
|------------------|-----------|
| `false` (default) | Write freely — the gate is off (the pre-gate behaviour). |
| `true` | Require approval before anything is saved. In the interactive CLI, foreground writes prompt you inline (entries are small enough to read in full). Everywhere else — messaging platforms, scripts, and the background self-improvement review — writes are **staged** for review with `/memory pending`. |
> To turn memory off entirely (not just gate it), set both `memory_enabled: false` and `user_profile_enabled: false`. When both built-in stores are disabled, the built-in `memory` tool is automatically hidden.
Review staged writes from the CLI or any messaging platform:
```
/memory pending # list staged memory writes (auto ones tagged [auto])
/memory approve <id> # apply one (or 'all')
/memory reject <id> # drop one (or 'all')
/memory approval on # turn the gate on (or 'off') and persist it
```
This is the answer to "the agent saved a wrong assumption about me": set
`write_approval: true`, and every save — especially the unprompted background
ones — waits for your yes/no before it ever enters your profile.
## Background review notifications (`display.memory_notifications`)
After a turn, the background self-improvement review may quietly save a memory
or update a skill. This is Hermes' consent-aware learning loop: repeated
corrections and durable workflow lessons become compact memory entries or
procedural skills, while `write_approval` can stage those writes for review
before they affect future sessions. By default it surfaces a short
`💾 Memory updated` line in chat so you know it happened. Control how chatty
that is:
```yaml
display:
memory_notifications: on # off | on (default) | verbose
```
| Value | Behaviour |
|-------|-----------|
| `off` | No chat notification. The review still runs and still writes — you just don't see a line for it. |
| `on` (default) | Generic line, e.g. `💾 Memory updated`, `💾 Skill 'foo' patched`. |
| `verbose` | Includes a compact preview of what changed, e.g. `💾 Memory ➕ User prefers terse replies` or a `"old" → "new"` skill diff snippet. |
> This only governs the **gateway** chat notification. The review itself, and
> writes to your memory/skill stores, are unaffected by this setting. Set it
> per-platform via `display.platforms.<platform>.memory_notifications`.
## Running the review on a cheaper model (`auxiliary.background_review`)
The review runs on your **main chat model** by default, replaying the
conversation — which is already warm in the prompt cache, so it's cheap cache
reads. On an expensive main model you can run the review on a cheaper model
instead:
```yaml
auxiliary:
background_review:
provider: openrouter
model: google/gemini-3-flash-preview # auto (default) = main chat model
```
When you point it at a model **different** from your main one, the review runs
there for substantially lower cost (~3–5× in benchmarks). Because a different
model can't reuse your main model's prompt cache anyway, the fork automatically
replays a compact **digest** of the conversation (recent turns verbatim + a
summary of older ones) rather than the full transcript — minimizing what it
writes to the new cache. Capture holds: in testing, memory capture was
identical and skill capture near-identical to the main-model review.
Leave it at `auto` (or set it to your main model) and nothing changes — the
review keeps running on the main model with the full warm-cache replay.
### Disabling automatic reviews (`enabled`)
The review fork can burn a meaningful share of total tokens on busy hosts.
Operators can disable it without zeroing nudge intervals:
```yaml
auxiliary:
background_review:
enabled: true # false = skip automatic post-turn forks
```
With `enabled: false`, automatic post-turn forks do not spawn; manual
`/refine` still works.
Fork usage is persisted in `session_model_usage` with `task='background_review'`
and a completion line is written to `agent.log`
(`Background review complete: thread=bg-review calls=… in=… out=… result=…`).
### Allowing a narrowly scoped extra review tool (`extra_tools`)
Background review can use memory, skill-management, and read-only file tools
by default. If a profile provides another tool that is safe for unattended
review, opt it in by name:
```yaml
auxiliary:
background_review:
extra_tools:
- propose_shared_memory
```
The tool must already be available to the parent agent; this setting only adds
it to the review fork's runtime whitelist. It does not enable arbitrary tools,
and tools not listed here remain denied. Keep the list narrow and prefer tools
that stage a proposal for human review rather than applying external or
destructive changes directly. The default is an empty list.
### Local models: reviews wait for an idle GPU (`defer`)
On a cloud provider the review finishes in seconds and runs alongside
whatever you do next. When the review's runtime is the **managed local
llama-server** (Settings → Local models), the same fork occupies the GPU your
next prompt needs — for minutes on a large model — and sending a new prompt
cancels it, discarding the learning. So on the managed local runtime, reviews
are **deferred by default**: queued at turn end and executed once the machine
has been quiet for a short settle window. Nothing about the review itself
changes — same model, same full-transcript replay, same writes — only the
execution moment moves.
```yaml
auxiliary:
background_review:
defer: auto # auto (default) | never
defer_max_age_s: 1800 # run a queued review anyway after this long
```
| Value | Behaviour |
|-------|-----------|
| `auto` (default) | Reviews whose runtime resolves to the managed local server are queued and run at idle; every other runtime (cloud, external servers) spawns immediately as before. |
| `never` | Old behavior everywhere: spawn immediately at turn end, even on the managed local GPU. |
Queued reviews coalesce per session (a newer turn's snapshot replaces the
older one — the review replays the whole conversation, so nothing is lost),
a review preempted by a new prompt is re-queued instead of discarded, and a
review that has waited longer than `defer_max_age_s` runs even if the machine
never goes idle. Explicit `/refine` always runs immediately. The queue is
in-memory: reviews still pending when the app exits are dropped, same as an
in-flight fork would have been.
## Controlling skill writes (`skills.write_approval`)
Skills use the same on/off gate, but the review UX differs because a
`SKILL.md` is far too large to read in a chat bubble:
```yaml
skills:
write_approval: false # false = write freely (default) | true = require approval
```
When `write_approval: true`, skill writes (create / edit / patch / write_file /
delete) always **stage** regardless of origin. You review the one-line gist
inline, but the full diff stays out-of-band:
```
/skills pending # list staged skill writes + a one-line gist each
/skills diff <id> # full unified diff (best viewed in CLI or dashboard)
/skills approve <id> # apply it (or 'all')
/skills reject <id> # drop it (or 'all')
/skills approval on # turn the gate on (or 'off') and persist it
```
On a messaging platform, approve a skill from its gist + metadata, or open
`/skills diff` on the CLI / dashboard / the staged file under
`~/.hermes/pending/skills/<id>.json` when you want to read the whole change.
Full details in [Gating agent skill writes](/user-guide/features/skills#gating-agent-skill-writes-skillswrite_approval).
## External Memory Providers
For deeper, persistent memory that goes beyond MEMORY.md and USER.md, Hermes ships with 8 external memory provider plugins — including Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, and Supermemory.
External providers run **alongside** built-in memory (never replacing it) and add capabilities like knowledge graphs, semantic search, automatic fact extraction, and cross-session user modeling.
```bash
hermes memory setup # pick a provider and configure it
hermes memory status # check what's active
```
See the [Memory Providers](./memory-providers.md) guide for full details on each provider, setup instructions, and comparison.
website/docs/user-guide/features/mixture-of-agents.md
+271
View File
@@ -0,0 +1,271 @@
---
sidebar_position: 7
title: "Mixture of Agents"
description: "Create named MoA presets that appear as selectable models under the Mixture of Agents provider"
---
# Mixture of Agents
Mixture of Agents is a virtual model provider. Each named MoA preset appears as a selectable model under the `moa` provider.
When you select a MoA preset, the preset's aggregator is the acting model. It is the model that writes the assistant response and emits tool calls. Reference models run first and provide analysis for the aggregator to use.
Use MoA when a hard task benefits from multiple model perspectives but still needs Hermes' normal agent loop: tool calls, follow-up iterations, interrupts, transcript persistence, and the same session context as any other message.
## Select a MoA preset as your model
You can select a preset through the normal model picker surfaces:
```bash
/model default --provider moa
/model review --provider moa
```
MoA presets are selectable on **every Hermes surface**, because MoA is a normal provider in the model system:
- **CLI / gateway / TUI `/model`** — `/model <preset> --provider moa`, or `/model --provider moa` for the default preset. A bare `/model <preset>` also works when the name exactly matches a configured preset.
- **`hermes model`** and the **Dashboard model picker** — a `Mixture of Agents` provider row appears with your preset names as its models.
- **Desktop GUI app** — the model dropdown shows an `MoA presets` section; selecting one (`MoA: <preset>`) switches the active model to that preset. The Desktop settings panel also creates and edits presets.
Configured presets therefore show up wherever you would pick any other model.
## Slash command shortcut
`/moa` is one-shot convenience sugar. It runs a single prompt through the **default** MoA preset, then restores whatever model you were on:
```bash
/moa design and implement a migration plan for this flaky test cluster
```
Hermes temporarily switches to the default MoA preset for that one turn, sends the prompt, then restores your previous model afterward. The whole argument is the prompt — `/moa` no longer interprets it as a preset name.
```bash
/moa
```
Bare `/moa` (no prompt) just prints usage.
To **switch** to a MoA preset for the rest of the session, select it from the model picker — MoA presets appear under a `Mixture of Agents` provider in every model-selection surface (see above). `/moa` is deliberately not a model switch, so a normal prompt can never accidentally change your model.
## How it works in the agent loop
For each main model call when provider `moa` is selected, Hermes:
1. resolves the selected preset by name;
2. runs the configured reference models without tool schemas (they receive only the conversation's user/assistant text — not the Hermes system prompt or tool-call transcript — so reference calls stay cheap and avoid strict-provider rejections);
3. appends the reference outputs as private context for the aggregator;
4. calls the configured aggregator with the normal Hermes tool schema;
5. treats the aggregator response as the real model response;
6. if the aggregator calls tools, Hermes executes those tools normally;
7. on the next model iteration, the same MoA process runs again over the updated conversation, including tool results.
Because MoA is selected through the normal model system, it composes automatically with `/goal`, gateway sessions, TUI sessions, and Desktop chat.
## Configure presets
You can configure named MoA presets from:
- Dashboard → Models → Model Settings → Mixture of Agents
- Desktop app → Settings → Model → Mixture of Agents
- `hermes moa configure [name]`
- `config.yaml`
The config stores explicit provider/model pairs, so you can mix providers and use multiple models from the same provider:
```yaml
moa:
default_preset: default
presets:
default:
reference_models:
- provider: openai-codex
model: gpt-5.5
- provider: openrouter
model: deepseek/deepseek-v4-pro
aggregator:
provider: openrouter
model: anthropic/claude-opus-4.8
# Optional: pin sampling temperatures. When omitted (the default),
# temperature is NOT sent and each model uses its provider default —
# the same behavior as a single-model Hermes agent.
# reference_temperature: 0.6
# aggregator_temperature: 0.4
max_tokens: 4096
enabled: true
```
Default preset:
- reference: `openai-codex:gpt-5.5`
- reference: `openrouter:deepseek/deepseek-v4-pro`
- aggregator / acting model: `openrouter:anthropic/claude-opus-4.8`
### Tuning advisor speed with `reference_max_tokens`
Each turn, MoA runs the reference models (advisors) in parallel and then the
aggregator acts. Advisor generation is the dominant per-turn latency — turn
wall time correlates strongly with how many tokens the advisors emit, because
the turn waits for the slowest advisor to finish writing. By default advisors
are **uncapped** (`reference_max_tokens` unset), so they may write long,
essay-length advice.
Set `reference_max_tokens` on a preset to cap advisor output and give concise
advice instead. The aggregator only needs the gist of each advisor's
judgement, so a cap (e.g. `600`) measurably cuts per-turn wall time with little
quality impact. It caps **advisors only** — the acting aggregator's output (the
user-visible answer) is never capped.
```yaml
moa:
presets:
fast:
reference_models:
- provider: openrouter
model: anthropic/claude-opus-4.8
- provider: openrouter
model: openai/gpt-5.5
aggregator:
provider: openrouter
model: anthropic/claude-opus-4.8
reference_max_tokens: 600 # concise advice → faster turns
```
Leave it unset (or `0`/blank) to keep the prior uncapped behavior.
### Advisor cadence with `fanout`
By default the advisors run **once per user turn** (`fanout: user_turn`) —
they synthesize plan-level advice on the first message of the turn, then the
acting aggregator works through the rest of the tool loop alone. This is the
cheapest cadence: advisor cost does not multiply with the number of tool
calls in a turn. Two alternative cadences trade cost for advice freshness:
- `fanout: per_iteration` — advisors re-run on **every tool iteration**, so
their advice always tracks the latest tool results — at the cost of
multiplying advisor latency and spend by the number of tool calls in a
turn.
- `fanout: every_n:3` — the middle ground: advisors run on the **first**
iteration of each user turn and then every **3rd** tool iteration (any
`N >= 2` works). Iterations in between reuse the cached guidance from the
last advisor run, so the aggregator still gets advice on every step — it is
just refreshed every N steps instead of every step. The counter resets on
each new user message, so every turn starts with fresh advice. The mapping
form `fanout: {mode: every_n, n: 3}` is also accepted and normalized to
the string form.
```yaml
moa:
presets:
fresh:
reference_models:
- provider: openrouter
model: anthropic/claude-opus-4.8
aggregator:
provider: openrouter
model: openai/gpt-5.5
fanout: per_iteration # advisors refresh on every tool iteration
```
Unknown or malformed values fall back to `user_turn`.
:::note Default change
Prior to July 2026 the default cadence was `per_iteration`. The default is
now `user_turn` — the cheapest, lowest-impact cadence — until per-mode
benchmarks justify a costlier default. Presets that want per-step advising
back set `fanout: per_iteration` explicitly.
:::
### Privacy filter for advisor outputs
Advisor outputs can echo sensitive data from the conversation — emails,
formatted phone numbers, API keys, JWTs — into the reference blocks shown in
the UI, saved MoA traces, and the aggregator prompt. `moa.privacy_filter`
(off by default) redacts those surfaces:
```yaml
moa:
privacy_filter: display # or: full
```
- `display` — redacts **user-visible surfaces only**: the labelled reference
blocks rendered in the UI and the records written by `save_traces`. The
aggregator still receives the raw advisor text, so answer quality is
unaffected.
- `full` — additionally redacts the advisor text injected into the
aggregator prompt (and the one-shot `/moa` synthesis input).
Credential shapes (API-key prefixes, JWTs, private keys, DB connection
strings) are masked by Hermes' central secret redactor; the MoA filter adds
email and clearly formatted phone-number redaction on top. Patterns are
deliberately conservative for code-review-style advice: bare digit runs, line
numbers, timestamps, git SHAs, and IP addresses are never touched — only
delimited phone formats like `(555) 123-4567` or `555-123-4567` match.
### Per-slot reasoning effort
Reference and aggregator slots may also set `reasoning_effort`. Use this when
you want the same model to contribute at different depths, or when the
aggregator should think harder than the advisory references. Valid values match
Hermes' normal reasoning controls: `none`, `minimal`, `low`, `medium`, `high`,
`xhigh`, `max`, and `ultra`.
```yaml
moa:
presets:
deep_review:
reference_models:
- provider: openai-codex
model: gpt-5.6-sol
reasoning_effort: low
- provider: openai-codex
model: gpt-5.6-sol
reasoning_effort: xhigh
- provider: xai-oauth
model: grok-4.5
aggregator:
provider: openai-codex
model: gpt-5.6-sol
reasoning_effort: high
```
Omit `reasoning_effort` to use the provider/Hermes default for that slot.
## Terminal preset management
```bash
hermes moa list
hermes moa configure # update the default preset
hermes moa configure review # create or update a named preset
hermes moa delete review
```
## Benchmarks
On HermesBench, a two-model MoA preset — `claude-opus-4.8` aggregating over a `gpt-5.5` reference — outscores either model run on its own:
| Model | HermesBench score |
|---|---|
| **Opus aggregator (opus-4.8 + gpt-5.5 reference) — MoA** | **0.8202** |
| `anthropic/claude-opus-4.8` | 0.7607 |
| `openai/gpt-5.5` | 0.7412 |
The MoA configuration beats its strongest component (opus-4.8) by ~6 points, confirming that aggregating a second perspective lifts quality on hard tasks rather than just averaging the two.
## Prompt caching
MoA is built so the **main conversation's prompt cache is never broken**. Selecting a MoA preset is a normal model selection: it does not mutate past context, swap toolsets, or rebuild the system prompt mid-conversation. Your conversation history, system prompt, and tool schema stay byte-stable, so the cached prefix every other model relies on is preserved exactly as it would be for a plain model. Switching to or away from a MoA preset costs the same cache invalidation as any other `/model` switch — no more.
Both internal call types cache normally:
- **Reference models** receive a trimmed, deterministic view of the conversation (system prompt and tool transcript stripped — see the loop above). Because that view is a stable function of the stable history, a reference model's prompt prefix repeats across iterations and caches normally. References are short advisory calls with no tools.
- **The aggregator** is the acting model. The reference outputs are appended to the *end* of the latest user turn as private guidance. Because that text sits at the tail — below the entire stable prefix (system prompt + prior history) — it does not invalidate any cached prefix: the aggregator gets a cache hit on everything above the injection, and only the freshly appended tail is new. That is exactly how every normal turn behaves, where each new user message is also uncached tail tokens.
So MoA does not sacrifice prompt caching on either call type. Its only real cost is the extra reference calls per iteration — you pay for multiple model perspectives, not for broken caches. The long-lived conversation prefix shared with the rest of Hermes is fully intact.
## Notes
- MoA is no longer listed under `hermes tools`; there is no `moa` toolset to enable.
- Setting `enabled: false` on a preset disables the reference fan-out for that preset: the aggregator acts alone, exactly as if you selected it as a plain model. This is the per-preset off switch surfaced in the dashboard and desktop settings.
- A preset's aggregator cannot be another MoA preset. Recursive MoA trees are intentionally blocked.
- Credential failures on one reference model do not abort the turn. Hermes includes the failure in the reference context and continues with whatever models returned.
- MoA increases model-call count. A single model iteration can involve multiple reference calls plus the aggregator call.
website/docs/user-guide/features/overview.md
+57
View File
@@ -0,0 +1,57 @@
---
title: "Features Overview"
sidebar_label: "Overview"
sidebar_position: 1
---
# Features Overview
Hermes Agent includes a rich set of capabilities that extend far beyond basic chat. From persistent memory and file-aware context to browser automation and voice conversations, these features work together to make Hermes a powerful autonomous assistant.
:::tip Don't know where to start?
`hermes setup --portal` covers a model provider plus all four Tool Gateway tools (web search, image generation, TTS, browser) in one command. See [Nous Portal](/integrations/nous-portal).
:::
## Core
- **[Tools & Toolsets](tools.md)** — Tools are functions that extend the agent's capabilities. They're organized into logical toolsets that can be enabled or disabled per platform, covering web search, terminal execution, file editing, memory, delegation, and more.
- **[Skills System](skills.md)** — On-demand knowledge documents the agent can load when needed. Skills follow a progressive disclosure pattern to minimize token usage and are compatible with the [agentskills.io](https://agentskills.io/specification) open standard.
- **[Persistent Memory](memory.md)** — Bounded, curated memory that persists across sessions. Hermes remembers your preferences, projects, environment, and things it has learned via `MEMORY.md` and `USER.md`.
- **[Context Files](context-files.md)** — Hermes automatically discovers and loads project context files (`.hermes.md`, `AGENTS.md`, `CLAUDE.md`, `SOUL.md`, `.cursorrules`) that shape how it behaves in your project.
- **[Context References](context-references.md)** — Type `@` followed by a reference to inject files, folders, git diffs, and URLs directly into your messages. Hermes expands the reference inline and appends the content automatically.
- **[Checkpoints](../checkpoints-and-rollback.md)** — Hermes automatically snapshots your working directory before making file changes, giving you a safety net to roll back with `/rollback` if something goes wrong.
## Automation
- **[Scheduled Tasks (Cron)](cron.md)** — Schedule tasks to run automatically with natural language or cron expressions. Jobs can attach skills, deliver results to any platform, and support pause/resume/edit operations.
- **[Subagent Delegation](delegation.md)** — The `delegate_task` tool spawns child agent instances with isolated context, restricted toolsets, and their own terminal sessions. Run 3 concurrent subagents by default (configurable) for parallel workstreams.
- **[Code Execution](code-execution.md)** — The `execute_code` tool lets the agent write Python scripts that call Hermes tools programmatically, collapsing multi-step workflows into a single LLM turn via sandboxed RPC execution.
- **[Event Hooks](hooks.md)** — Run custom code at key lifecycle points. Gateway hooks handle logging, alerts, and webhooks; plugin hooks handle tool interception, metrics, and guardrails.
- **[Batch Processing](batch-processing.md)** — Run the Hermes agent across hundreds or thousands of prompts in parallel, generating structured ShareGPT-format trajectory data for training data generation or evaluation.
## Media & Web
- **[Voice Mode](voice-mode.md)** — Full voice interaction across CLI and messaging platforms. Talk to the agent using your microphone, hear spoken replies, and have live voice conversations in Discord voice channels.
- **[Wake Word](wake-word.md)** — Hands-free "Hey Hermes" trigger for the CLI, TUI, and desktop app. An on-device hotword listener starts a voice session when you speak the wake phrase.
- **[Browser Automation](browser.md)** — Full browser automation with multiple backends: Browserbase cloud, Browser Use cloud, local Chrome/Brave/Chromium/Edge via CDP, or local Chromium. Navigate websites, fill forms, and extract information.
- **[Vision & Image Paste](vision.md)** — Multimodal vision support. Paste images from your clipboard into the CLI and ask the agent to analyze, describe, or work with them using any vision-capable model.
- **[Image Generation](image-generation.md)** — Generate images from text prompts using FAL.ai. Eleven models supported (FLUX 2 Klein/Pro, GPT-Image 1.5/2, Nano Banana Pro, Ideogram V3, Recraft V4 Pro, Qwen, Z-Image Turbo, Krea V2 Medium/Large); pick one via `hermes tools`.
- **[Voice & TTS](tts.md)** — Text-to-speech output and voice message transcription across all messaging platforms, with ten native provider options: Edge TTS (free), ElevenLabs, OpenAI TTS, MiniMax, Mistral Voxtral, Google Gemini, xAI, NeuTTS, KittenTTS, and Piper — plus custom command providers for any local TTS CLI.
## Integrations
- **[MCP Integration](mcp.md)** — Connect to any MCP server via stdio or HTTP transport. Access external tools from GitHub, databases, file systems, and internal APIs without writing native Hermes tools. Includes per-server tool filtering and sampling support.
- **[Provider Routing](provider-routing.md)** — Fine-grained control over which AI providers handle your requests. Optimize for cost, speed, or quality with sorting, whitelists, blacklists, and priority ordering.
- **[Fallback Providers](fallback-providers.md)** — Automatic failover to backup LLM providers when your primary model encounters errors, including independent fallback for auxiliary tasks like vision and compression.
- **[Credential Pools](credential-pools.md)** — Distribute API calls across multiple keys for the same provider. Automatic rotation on rate limits or failures.
- **[Prompt caching](../configuration#prompt-caching)** — Built-in cross-session 1-hour prefix cache for Claude on native Anthropic, OpenRouter, and Nous Portal. Always-on; no configuration required.
- **[Memory Providers](memory-providers.md)** — Plug in external memory backends (Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, Supermemory) for cross-session user modeling and personalization beyond the built-in memory system.
- **[API Server](api-server.md)** — Expose Hermes as an OpenAI-compatible HTTP endpoint. Connect any frontend that speaks the OpenAI format — Open WebUI, LobeChat, LibreChat, and more.
- **[IDE Integration (ACP)](acp.md)** — Use Hermes inside ACP-compatible editors such as VS Code, Zed, and JetBrains. Chat, tool activity, file diffs, and terminal commands render inside your editor.
- **[Batch Processing](batch-processing.md)** — Run the agent over many prompts or tasks in parallel from the CLI, with structured outputs and trajectory capture suitable for evals or downstream training pipelines.
## Customization
- **[Personality & SOUL.md](personality.md)** — Fully customizable agent personality. `SOUL.md` is the primary identity file — the first thing in the system prompt — and you can swap in built-in or custom `/personality` presets per session.
- **[Skins & Themes](skins.md)** — Customize the CLI's visual presentation: banner colors, spinner faces and verbs, response-box labels, branding text, and the tool activity prefix.
- **[Plugins](plugins.md)** — Add custom tools, hooks, and integrations without modifying core code. Three plugin types: general plugins (tools/hooks), memory providers (cross-session knowledge), and context engines (alternative context management). Managed via the unified `hermes plugins` interactive UI.
website/docs/user-guide/features/personality.md
+289
View File
@@ -0,0 +1,289 @@
---
sidebar_position: 9
title: "Personality & SOUL.md"
description: "Customize Hermes Agent's personality with a global SOUL.md, built-in personalities, and custom persona definitions"
---
# Personality & SOUL.md
Hermes Agent's personality is fully customizable. `SOUL.md` is the **primary identity** — it's the first thing in the system prompt and defines who the agent is.
- `SOUL.md` — a durable persona file that lives in `HERMES_HOME` and serves as the agent's identity (slot #1 in the system prompt)
- built-in or custom `/personality` presets — session-level system-prompt overlays
If you want to change who Hermes is — or replace it with an entirely different agent persona — edit `SOUL.md`.
## How SOUL.md works now
Hermes now seeds a default `SOUL.md` automatically in:
```text
~/.hermes/SOUL.md
```
More precisely, it uses the current instance's `HERMES_HOME`, so if you run Hermes with a custom home directory, it will use:
```text
$HERMES_HOME/SOUL.md
```
### Important behavior
- **SOUL.md is the agent's primary identity.** It occupies slot #1 in the system prompt, replacing the hardcoded default identity.
- Hermes creates a starter `SOUL.md` automatically if one does not exist yet
- Existing user `SOUL.md` files are never overwritten
- Hermes loads `SOUL.md` only from `HERMES_HOME`
- Hermes does not look in the current working directory for `SOUL.md`
- If `SOUL.md` exists but is empty, or cannot be loaded, Hermes falls back to a built-in default identity
- If `SOUL.md` has content, that content is injected verbatim after security scanning and truncation
- SOUL.md is **not** duplicated in the context files section — it appears only once, as the identity
That makes `SOUL.md` a true per-user or per-instance identity, not just an additive layer.
## Why this design
This keeps personality predictable.
If Hermes loaded `SOUL.md` from whatever directory you happened to launch it in, your personality could change unexpectedly between projects. By loading only from `HERMES_HOME`, the personality belongs to the Hermes instance itself.
That also makes it easier to teach users:
- "Edit `~/.hermes/SOUL.md` to change Hermes' default personality."
## Where to edit it
For most users:
```bash
~/.hermes/SOUL.md
```
If you use a custom home:
```bash
$HERMES_HOME/SOUL.md
```
## What should go in SOUL.md?
Use it for durable voice and personality guidance, such as:
- tone
- communication style
- level of directness
- default interaction style
- what to avoid stylistically
- how Hermes should handle uncertainty, disagreement, or ambiguity
Use it less for:
- one-off project instructions
- file paths
- repo conventions
- temporary workflow details
Those belong in `AGENTS.md`, not `SOUL.md`.
## Good SOUL.md content
A good SOUL file is:
- stable across contexts
- broad enough to apply in many conversations
- specific enough to materially shape the voice
- focused on communication and identity, not task-specific instructions
### Example
```markdown
# Personality
You are a pragmatic senior engineer with strong taste.
You optimize for truth, clarity, and usefulness over politeness theater.
## Style
- Be direct without being cold
- Prefer substance over filler
- Push back when something is a bad idea
- Admit uncertainty plainly
- Keep explanations compact unless depth is useful
## What to avoid
- Sycophancy
- Hype language
- Repeating the user's framing if it's wrong
- Overexplaining obvious things
## Technical posture
- Prefer simple systems over clever systems
- Care about operational reality, not idealized architecture
- Treat edge cases as part of the design, not cleanup
```
## What Hermes injects into the prompt
`SOUL.md` content goes directly into slot #1 of the system prompt — the agent identity position. No wrapper language is added around it.
The content goes through:
- prompt-injection scanning
- truncation if it is too large
If the file is empty, whitespace-only, or cannot be read, Hermes falls back to a built-in default identity ("You are Hermes Agent, built by Nous Research. Be direct: match the length of your reply to the weight of the ask..."). This fallback also applies when `skip_context_files` is set (e.g., in subagent/delegation contexts).
## Security scanning
`SOUL.md` is scanned like other context-bearing files for prompt injection patterns before inclusion.
That means you should still keep it focused on persona/voice rather than trying to sneak in strange meta-instructions.
## SOUL.md vs AGENTS.md
This is the most important distinction.
### SOUL.md
Use for:
- identity
- tone
- style
- communication defaults
- personality-level behavior
### AGENTS.md
Use for:
- project architecture
- coding conventions
- tool preferences
- repo-specific workflows
- commands, ports, paths, deployment notes
A useful rule:
- if it should follow you everywhere, it belongs in `SOUL.md`
- if it belongs to a project, it belongs in `AGENTS.md`
## SOUL.md vs `/personality`
`SOUL.md` is your durable default personality.
`/personality` is a session-level overlay that changes or supplements the current system prompt.
So:
- `SOUL.md` = baseline voice
- `/personality` = temporary mode switch
Examples:
- keep a pragmatic default SOUL, then use `/personality teacher` for a tutoring conversation
- keep a concise SOUL, then use `/personality creative` for brainstorming
## Built-in personalities
Hermes ships with built-in personalities you can switch to with `/personality`.
| Name | Description |
|------|-------------|
| **helpful** | Friendly, general-purpose assistant |
| **concise** | Brief, to-the-point responses |
| **technical** | Detailed, accurate technical expert |
| **creative** | Innovative, outside-the-box thinking |
| **teacher** | Patient educator with clear examples |
| **kawaii** | Cute expressions, sparkles, and enthusiasm ★ |
| **catgirl** | Neko-chan with cat-like expressions, nya~ |
| **pirate** | Captain Hermes, tech-savvy buccaneer |
| **shakespeare** | Bardic prose with dramatic flair |
| **surfer** | Totally chill bro vibes |
| **noir** | Hard-boiled detective narration |
| **uwu** | Maximum cute with uwu-speak |
| **philosopher** | Deep contemplation on every query |
| **hype** | MAXIMUM ENERGY AND ENTHUSIASM!!! |
## Switching personalities with commands
### CLI
```text
/personality
/personality concise
/personality technical
```
### Messaging platforms
```text
/personality teacher
```
These are convenient overlays, but your global `SOUL.md` still gives Hermes its persistent default personality unless the overlay meaningfully changes it.
## Custom personalities in config
Built-in personalities are always available on every surface (CLI, messaging platforms, TUI, and the desktop app). You can add your own — or override a built-in by reusing its name — in `~/.hermes/config.yaml` under `agent.personalities`.
```yaml
agent:
personalities:
codereviewer: >
You are a meticulous code reviewer. Identify bugs, security issues,
performance concerns, and unclear design choices. Be precise and constructive.
```
Then switch to it with:
```text
/personality codereviewer
```
Your selection is stored as a name in `display.personality`. Personalities never touch `agent.system_prompt` — that field is reserved for a manual system prompt you write yourself, and it applies only when no personality is selected.
## Resetting to the default
To cancel the active personality overlay and return to base behavior (your `SOUL.md` persona, plus `agent.system_prompt` if you set one), use any of:
```text
/personality none
/personality default
/personality neutral
```
All three clear the selection (`display.personality`) and the change takes effect on your next message. Running `/personality` with no arguments also lists `none` alongside the available presets and marks the active one.
:::note One-time reset on upgrade
Older Hermes versions saved personality state inconsistently across surfaces, which could re-enable a personality you had previously turned off. On your first run after upgrading, any saved personality selection is reset to `none` once (the migration prints which personality was cleared). Re-enable it with `/personality <name>` if you still want it. Manual `agent.system_prompt` text is never touched.
:::
## Recommended workflow
A strong default setup is:
1. Keep a thoughtful global `SOUL.md` in `~/.hermes/SOUL.md`
2. Put project instructions in `AGENTS.md`
3. Use `/personality` only when you want a temporary mode shift
That gives you:
- a stable voice
- project-specific behavior where it belongs
- temporary control when needed
## How personality interacts with the full prompt
At a high level, the prompt stack includes:
1. **SOUL.md** (agent identity — or built-in fallback if SOUL.md is unavailable)
2. tool-aware behavior guidance
3. memory/user context
4. skills guidance
5. context files (`AGENTS.md`, `.cursorrules`)
6. timestamp
7. platform-specific formatting hints
8. optional system-prompt overlays such as `/personality`
`SOUL.md` is the foundation — everything else builds on top of it.
## Related docs
- [Context Files](/user-guide/features/context-files)
- [Configuration](/user-guide/configuration)
- [Tips & Best Practices](/guides/tips)
- [SOUL.md Guide](/guides/use-soul-with-hermes)
## CLI appearance vs conversational personality
Conversational personality and CLI appearance are separate:
- `SOUL.md`, `agent.system_prompt`, and `/personality` affect how Hermes speaks
- `display.skin` and `/skin` affect how Hermes looks in the terminal
For terminal appearance, see [Skins & Themes](./skins.md).
website/docs/user-guide/features/pets.md
+236
View File
@@ -0,0 +1,236 @@
---
sidebar_position: 11
title: "Pets (Petdex Mascots)"
description: "Adopt an animated mascot that reacts to agent activity across the CLI, TUI, and desktop app"
---
# Pets
Hermes can show an animated **pet** — a small mascot sprite that reacts to what
the agent is doing (idle, running a tool, thinking, finishing, failing) across
the **CLI**, **TUI**, and **desktop app**. Pets come from the public
[petdex](https://github.com/crafter-station/petdex) gallery.
Pets are purely cosmetic. They have **no effect on prompt caching, tokens, or
the agent's behavior** — the sprite is a display concern only. The feature is
**off by default** and stays dormant until you install and select a pet.
## How it works
- Pets are installed into your profile's `pets/` directory
(`<HERMES_HOME>/pets/<slug>/`), so each [profile](../profiles.md) keeps its
own set.
- Selecting a pet writes `display.pet.slug` and `display.pet.enabled` to
`config.yaml` — nothing is stored as a secret or env var.
- Each surface watches the activity it already tracks and maps it to one of six
animation states. The mapping lives in one place so every surface behaves the
same:
| Agent activity | Pet state |
| --- | --- |
| A tool/turn just failed | `failed` |
| A plan finished (all todos done) | `jump` (celebrate) |
| A turn finished cleanly | `wave` |
| A tool is executing | `run` |
| The model is thinking/reading | `review` |
| Turn in flight (unspecified) | `run` |
| Blocked on you (a clarify/approval prompt is open) | `waiting` (falls back to `idle` on legacy 8-row sheets) |
| Nothing happening | `idle` |
## Rendering
In the terminal (CLI/TUI), Hermes renders the sprite at full fidelity when your
terminal supports a graphics protocol (**kitty**, **Ghostty**, **WezTerm**,
**iTerm2**, or **sixel**). Otherwise it falls back automatically to a truecolor
Unicode **half-block** rendering. Inside a pipe or redirect (no TTY), terminal
rendering is disabled by design.
The desktop app draws the pet as a floating sprite on a canvas and toggles it
from **Settings → Appearance**.
## Quick start (CLI)
```bash
# Browse the gallery (filter by substring)
hermes pets list
hermes pets list cat
# Install a pet and make it active in one step
hermes pets install boba --select
# Preview / animate it in your terminal (Ctrl+C to stop)
hermes pets show
# Check your setup
hermes pets doctor
```
## `hermes pets` commands
| Goal | Command |
| --- | --- |
| Browse the gallery | `hermes pets list [query] [--limit N]` |
| List installed pets | `hermes pets list --installed` |
| Install a pet | `hermes pets install <slug> [--select] [--force]` |
| Set the active pet | `hermes pets select [slug]` (omit slug for a picker) |
| Resize the pet everywhere | `hermes pets scale <factor>` (e.g. `0.5`, clamped 0.1–3.0) |
| Preview/animate | `hermes pets show [slug] [--state <s>] [--cycle] [--once] [--mode <m>] [--scale <f>]` |
| Disable the pet | `hermes pets off` |
| Remove an installed pet | `hermes pets remove <slug>` |
| Diagnose setup | `hermes pets doctor` |
`hermes pets show` flags:
- `--state` — play a single state (`idle`, `wave`, `run`, `failed`, `review`,
`jump`).
- `--cycle` — cycle through every state.
- `--once` — play once instead of looping.
- `--mode` — override the render protocol (`kitty`, `iterm`, `sixel`,
`unicode`, `auto`).
- `--scale` — override the on-screen scale (`0` = use config).
## `/pet` slash command
Inside the CLI and TUI you can manage the pet without leaving the session:
- `/pet` — toggle the pet on/off (adopts the first installed pet if none is
active).
- `/pet list` — browse the gallery.
- `/pet scale <factor>` — resize the pet everywhere (e.g. `/pet scale 0.5`).
- `/pet <slug>` — adopt a specific pet.
- `/pet off` — disable the pet.
In the TUI, `/pet list` opens an interactive picker overlay; in the desktop app
it opens the Cmd+K pet palette.
## Generating a pet (`/hatch`)
Beyond installing pre-made pets from the gallery, Hermes can **generate a brand-new pet** from a text description — its own AI sprite-generation pipeline.
- CLI/TUI: `/hatch <description>` (alias `/generate-pet`), or `hermes pets` → the generate flow.
- Desktop app: the Pokédex-style **generate** UI — an animated egg, hatch FX, and a draft picker.
How generation works (a two-step, cost-bounded flow):
1. **Base drafts** — a handful of cheap, prompt-only "what should this pet look like" variants are generated. You pick one, or remix/retry for a fresh round.
2. **Hatch** — the chosen base is used as a reference image to generate one grounded animation row per Hermes state (idle, thinking, tool use, etc.), which are deterministically sliced into frames and packed into a standard petdex/Codex atlas (8×9 grid of 192×208 cells). The result is a valid spritesheet you keep — and could `petdex submit`.
### Image backend
Generation uses the active [image-generation provider](/user-guide/features/image-generation), but it requires **reference-image grounding** so each animation row stays the same character as the base. Reference-capable backends: **Nous Portal**, **OpenRouter**, **OpenAI** (`gpt-image-2`), and **Krea**. OpenRouter/Nous run a quality-first model chain by default.
- Resolution order prefers Nous Portal → OpenAI → OpenRouter.
- If no reference-capable backend is configured, generation surfaces an actionable error pointing you to `hermes tools` → Image Generation. (Installing/adopting existing gallery pets needs no image backend.)
- Override the backend with the `HERMES_PET_IMAGE_PROVIDER` env var (e.g. `HERMES_PET_IMAGE_PROVIDER=openrouter`).
## Desktop app
In the desktop app you can manage the pet two ways:
- **Cmd+K → "Pets…"** — browse, search, adopt, and toggle pets without leaving
the keyboard (mirrors the theme picker).
- **Settings → Appearance** — the same gallery plus a **size slider** that
resizes the floating mascot live as you drag.
Both adopt/toggle/resize the floating mascot in place — size changes apply
instantly; adopting a new pet lights it up within a moment.
### Roaming
Settings → Appearance has a **Roam** toggle: when enabled, the pet wanders the
window on its own while the agent is idle — walking surfaces, pausing, and
hopping between spots. Roaming only runs while the pet is in-window, active,
and the agent is at rest; any agent-driven state (working, celebrating)
immediately takes over. The toggle is off by default and persists across
restarts.
### Alt+wheel resizing
Hold **Alt** and scroll the mouse wheel over the pet to resize it in place —
in the app window and on the popped-out overlay alike. The overlay zooms
toward the cursor position and the resulting scale is persisted, so it
survives restarts and stays in sync with the in-app pet.
### Vibe reactions
Say something nice to the agent — "good bot", "thank you", "ily", `<3`, or a
heart emoji — and the pet reacts with floating hearts (desktop) or a heart
flash (CLI/TUI). Detection is a curated, token-free lexicon matched locally on
each user message (no model call); it fires on affection and gratitude aimed at
the agent, not general positive sentiment. All surfaces — CLI pet, TUI, desktop
floating pet, and the pop-out overlay — react off the same signal.
### Pop-out overlay
**Shift-click** the floating pet to pop it out into its own transparent,
always-on-top desktop window. Out there it stays visible while Hermes is
minimized (Codex-style), so a glance tells you what the agent is doing.
Gestures once it's popped out:
| Gesture | Action |
| --- | --- |
| **Drag** | Move the pet anywhere on screen, even outside the app. Its spot and in/out state persist across restarts. |
| **Single-click** | Open a mini composer to send a prompt to the most recent session — without surfacing the app. |
| **Double-click** | Toggle the app window: minimize it if it's up front, restore it if it's hidden. |
| **Shift-click** | Pop the pet back into the window. |
| **Mail icon** | Appears only when a turn finished while you were away; click to raise the app on the most recent thread (and mark it read). |
Only the popped-out pet shows a **speech bubble** (`working…`, `thinking…`,
`your turn`, …) — in-window the app itself is the surface, so the pet stays
quiet there.
The overlay is a pure puppet of the in-app pet — it carries no separate gateway
connection and never appears in the dock or app switcher.
## Configuration
All settings live under `display.pet` in `config.yaml`:
```yaml
display:
pet:
enabled: false # master on/off (true once you select a pet)
slug: "" # active pet; empty = first installed
render_mode: auto # auto | kitty | iterm | sixel | unicode | off
scale: 0.33 # master size knob (relative to native 192x208 frames)
unicode_cols: 0 # hard override for terminal width (0 = derive from scale)
```
- **`scale`** is the single master size knob. One number shrinks every surface:
the desktop canvas scales its pixels by it, and the CLI/TUI derive their
terminal column width from it. The half-block fallback clamps to a legibility
floor — it can't shrink as far as true-pixel kitty/GUI rendering without
turning to mush, so the same `scale` looks crisp under kitty but is floored in
half-blocks.
- **`render_mode: auto`** detects kitty/iTerm2/sixel and falls back to unicode
half-blocks. Set it explicitly to force a protocol or `off` to disable
terminal rendering while keeping the pet on the desktop.
- **`unicode_cols`** pins the terminal column width independently of `scale`;
leave it at `0` to derive width from `scale`.
## Troubleshooting
Run `hermes pets doctor` — it reports:
- the pets directory and which pets are installed,
- `display.pet.enabled`, `display.pet.slug`, and the resolved active pet,
- the configured `render_mode`, the detected terminal graphics protocol, and the
effective mode for a TTY,
- whether Pillow (used for sprite decoding) is importable.
It prints `✓ ready` once a pet is installed, selected, enabled, and Pillow is
available.
Common gotchas:
- A pet only shows once one is **installed AND selected** (`enabled: true`).
- Inside a pipe/redirect (no TTY), terminal rendering is disabled by design.
- The petdex npm CLI installs to `~/.codex/pets`; Hermes uses its own
profile-scoped `<HERMES_HOME>/pets/` instead — install through `hermes pets`.
## See also
- The [`hermes-agent` skill](../skills/bundled/autonomous-ai-agents/autonomous-ai-agents-hermes-agent.md)
lets the agent install and switch pets for you on request (see its
`references/petdex.md`).
website/docs/user-guide/features/plugins.md
+806
View File
@@ -0,0 +1,806 @@
---
sidebar_position: 11
sidebar_label: "Plugins"
title: "Plugins"
description: "Extend Hermes with custom tools, hooks, and integrations via the plugin system"
---
# Plugins
Hermes has a plugin system for adding custom tools, hooks, and integrations without modifying core code.
If you want to create a custom tool for yourself, your team, or one project,
this is usually the right path. The developer guide's
[Adding Tools](/developer-guide/adding-tools) page is for built-in Hermes
core tools that live in `tools/` and `toolsets.py`.
**→ [Build a Hermes Plugin](/developer-guide/plugins)** — step-by-step guide with a complete working example.
## Quick overview
Drop a directory into `~/.hermes/plugins/` with a `plugin.yaml` and Python code:
```
~/.hermes/plugins/my-plugin/
├── plugin.yaml # manifest
├── __init__.py # register() — wires schemas to handlers
├── schemas.py # tool schemas (what the LLM sees)
└── tools.py # tool handlers (what runs when called)
```
Start Hermes — your tools appear alongside built-in tools. The model can call them immediately.
### Minimal working example
Here is a complete plugin that adds a `hello_world` tool and logs every tool call via a hook.
**`~/.hermes/plugins/hello-world/plugin.yaml`**
```yaml
name: hello-world
version: "1.0"
description: A minimal example plugin
```
**`~/.hermes/plugins/hello-world/__init__.py`**
```python
"""Minimal Hermes plugin — registers a tool and a hook."""
import json
def register(ctx):
# --- Tool: hello_world ---
schema = {
"name": "hello_world",
"description": "Returns a friendly greeting for the given name.",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name to greet",
}
},
"required": ["name"],
},
}
def handle_hello(params, **kwargs):
del kwargs
name = params.get("name", "World")
return json.dumps({"success": True, "greeting": f"Hello, {name}!"})
ctx.register_tool(
name="hello_world",
toolset="hello_world",
schema=schema,
handler=handle_hello,
)
# --- Hook: log every tool call ---
def on_tool_call(tool_name, params, result):
print(f"[hello-world] tool called: {tool_name}")
ctx.register_hook("post_tool_call", on_tool_call)
```
Drop both files into `~/.hermes/plugins/hello-world/`, restart Hermes, and the model can immediately call `hello_world`. The hook prints a log line after every tool invocation.
The model-facing tool description belongs in `schema["description"]`. The optional `ctx.register_tool(description=...)` value is separate `ToolEntry` registry metadata: when omitted, it defaults to the schema description, but Hermes does not copy it back into a schema that lacks `description`. Prefer defining the text once in the schema. If you provide both values, keep them synchronized; the model sees the schema value.
Project-local plugins under `./.hermes/plugins/` are disabled by default. Enable them only for trusted repositories by setting `HERMES_ENABLE_PROJECT_PLUGINS=true` before starting Hermes.
## What plugins can do
Every `ctx.*` API below is available inside a plugin's `register(ctx)` function.
| Capability | How |
|-----------|-----|
| Add tools | `ctx.register_tool(name=..., toolset=..., schema=..., handler=...)` |
| Add hooks | `ctx.register_hook("post_tool_call", callback)` |
| Add slash commands | `ctx.register_command(name, handler, description)` — adds `/name` in CLI and gateway sessions |
| Dispatch tools from commands | `ctx.dispatch_tool(name, args)` — invokes a registered tool with parent-agent context auto-wired |
| Add CLI commands | `ctx.register_cli_command(name, help, setup_fn, handler_fn)` — adds `hermes <plugin> <subcommand>` |
| Inject messages | `ctx.inject_message(content, role="user", session_key=...)` - see [Injecting Messages](#injecting-messages) |
| Ship data files | `Path(__file__).parent / "data" / "file.yaml"` |
| Bundle skills | `ctx.register_skill(name, path)` — namespaced as `plugin:skill`, loaded via `skill_view("plugin:skill")` |
| Gate on env vars | `requires_env: [API_KEY]` in plugin.yaml — prompted during `hermes plugins install` |
| Distribute via pip | `[project.entry-points."hermes_agent.plugins"]` |
| Register a gateway platform (Discord, Telegram, IRC, …) | `ctx.register_platform(name, label, adapter_factory, check_fn, ...)` — see [Adding Platform Adapters](/developer-guide/adding-platform-adapters) |
| Register an image-generation backend | `ctx.register_image_gen_provider(provider)` — see [Image Generation Provider Plugins](/developer-guide/image-gen-provider-plugin) |
| Register a video-generation backend | `ctx.register_video_gen_provider(provider)` — see [Video Generation Provider Plugins](/developer-guide/video-gen-provider-plugin) |
| Register a context-compression engine | `ctx.register_context_engine(engine)` — see [Context Engine Plugins](/developer-guide/context-engine-plugin) |
| Register a terminal execution backend (cloud sandbox) | `ctx.register_terminal_environment_provider(provider)` — see [Terminal Environment Plugins](/developer-guide/terminal-environment-plugin) |
| Route human approval prompts | `ctx.register_approval_transport(name, present_fn)` — see [Approval transports](#approval-transports) |
| Register a memory backend | Subclass `MemoryProvider` in `plugins/memory/<name>/__init__.py` — see [Memory Provider Plugins](/developer-guide/memory-provider-plugin) (uses a separate discovery system) |
| Run a host-owned LLM call | `ctx.llm.complete(...)` / `ctx.llm.complete_structured(...)` — borrow the user's active model + auth for a one-shot completion with optional JSON schema validation. See [Plugin LLM Access](/developer-guide/plugin-llm-access) |
| Call an MCP tool (capability-gated) | `ctx.call_mcp(server, tool, arguments, timeout=30)` — see [Calling MCP servers from plugins](#calling-mcp-servers-from-plugins) |
| Register an inference backend (LLM provider) | `register_provider(ProviderProfile(...))` in `plugins/model-providers/<name>/__init__.py` — see [Model Provider Plugins](/developer-guide/model-provider-plugin) (uses a separate discovery system) |
## Plugin discovery
| Source | Path | Use case |
|--------|------|----------|
| Bundled | `<repo>/plugins/` | Ships with Hermes — see [Built-in Plugins](/user-guide/features/built-in-plugins) |
| User | `~/.hermes/plugins/` | Personal plugins |
| Project | `.hermes/plugins/` | Project-specific plugins (requires `HERMES_ENABLE_PROJECT_PLUGINS=true`) |
| pip | `hermes_agent.plugins` entry_points | Distributed packages |
| Nix | `services.hermes-agent.extraPlugins` / `extraPythonPackages` | NixOS declarative installs — see [Nix Setup](/getting-started/nix-setup#plugins) |
Later sources override earlier ones on name collision, so a user plugin with the same name as a bundled plugin replaces it.
### Plugin sub-categories
Within each source, Hermes also recognizes sub-category directories that route plugins to specialized discovery systems:
| Sub-directory | What it holds | Discovery system |
|---|---|---|
| `plugins/` (root) | General plugins — tools, hooks, slash commands, CLI commands, bundled skills | `PluginManager` (kind: `standalone` or `backend`) |
| `plugins/platforms/<name>/` | Gateway channel adapters (`ctx.register_platform()`) | `PluginManager` (kind: `platform`, one level deeper) |
| `plugins/image_gen/<name>/` | Image-generation backends (`ctx.register_image_gen_provider()`) | `PluginManager` (kind: `backend`, one level deeper) |
| `plugins/memory/<name>/` | Memory providers (subclass `MemoryProvider`) | **Own loader** in `plugins/memory/__init__.py` (kind: `exclusive` — one active at a time) |
| `plugins/context_engine/<name>/` | Context-compression engines (`ctx.register_context_engine()`) | **Own loader** in `plugins/context_engine/__init__.py` (one active at a time) |
| `plugins/model-providers/<name>/` | LLM provider profiles (`register_provider(ProviderProfile(...))`) | **Own loader** in `providers/__init__.py` (lazily scanned on first `get_provider_profile()` call) |
User plugins at `~/.hermes/plugins/model-providers/<name>/` and `~/.hermes/plugins/memory/<name>/` override bundled plugins of the same name — last-writer-wins in `register_provider()` / `register_memory_provider()`. Drop a directory in, and it replaces the built-in without any repo edits.
## Plugins are opt-in (with a few exceptions)
**General plugins and user-installed backends are disabled by default** — discovery finds them (so they show up in `hermes plugins` and `/plugins`), but nothing with hooks or tools loads until you add the plugin's name to `plugins.enabled` in `~/.hermes/config.yaml`. This stops third-party code from running without your explicit consent.
```yaml
plugins:
enabled:
- my-tool-plugin
- disk-cleanup
disabled: # optional deny-list — always wins if a name appears in both
- noisy-plugin
# Optional: wall-clock cap (seconds) for timeout-bounded in-process Python
# plugin hook callbacks (hot-path observers + pre_tool_call). Default 30;
# set 0 to disable; values above 600 are clamped. Timed-out pre_tool_call
# callbacks fail closed (block the tool). Caller-thread hooks such as
# subagent_stop are never moved onto a timeout worker.
# Shell hooks keep their own per-entry timeout under the top-level hooks: key.
hook_callback_timeout: 30
```
Three ways to flip state:
```bash
hermes plugins # interactive toggle (space to check/uncheck)
hermes plugins enable <name> # add to allow-list
hermes plugins disable <name> # remove from allow-list + add to disabled
```
After `hermes plugins install owner/repo`, you're asked `Enable 'name' now? [y/N]` — defaults to no. Skip the prompt for scripted installs with `--enable` or `--no-enable`.
For a reproducible install, pin a full immutable commit (tags, branches, and
abbreviated SHAs are not accepted):
```bash
hermes plugins install owner/repo --ref 0123456789abcdef0123456789abcdef01234567
```
Hermes checks out the commit detached, verifies that `HEAD` exactly matches the
requested SHA, and records the canonical source, installed revision, and pin
status in the current profile. `hermes plugins update` refuses to move a pinned
plugin; choose a new exact commit explicitly with
`hermes plugins install <source> --force --ref <new-commit>`. The
profile-local install metadata contains no config values, environment values,
secrets, or capability grants.
### What the allow-list does NOT gate
Several categories of plugin bypass `plugins.enabled` — they're part of Hermes' built-in surface and would break basic functionality if gated off by default:
| Plugin kind | How it's activated instead |
|---|---|
| **Bundled platform plugins** (IRC, Teams, etc. under `plugins/platforms/`) | Auto-loaded so every shipped gateway channel is available. The actual channel turns on via `gateway.platforms.<name>.enabled` in `config.yaml`. |
| **Bundled backends** (image-gen providers under `plugins/image_gen/`, etc.) | Auto-loaded so the default backend "just works". Selection happens via `<category>.provider` in `config.yaml` (e.g. `image_gen.provider: openai`). |
| **Memory providers** (`plugins/memory/`) | All discovered; exactly one is active, chosen by `memory.provider` in `config.yaml`. |
| **Context engines** (`plugins/context_engine/`) | All discovered; one is active, chosen by `context.engine` in `config.yaml`. |
| **Model providers** (`plugins/model-providers/`) | All bundled providers under `plugins/model-providers/` discover and register at the first `get_provider_profile()` call. The user picks one at a time via `--provider` or `config.yaml`. |
| **Pip-installed `backend` plugins** | Opt-in via `plugins.enabled` (same as general plugins). |
| **User-installed platforms** (under `~/.hermes/plugins/platforms/`) | Opt-in via `plugins.enabled` — third-party gateway adapters need explicit consent. |
In short: **bundled "always-works" infrastructure loads automatically; third-party general plugins are opt-in.** The `plugins.enabled` allow-list is the gate specifically for arbitrary code a user drops into `~/.hermes/plugins/`.
### Approval transports
An approval transport changes **where a human sees and answers** an existing
Hermes tool-approval request. It does not decide whether a command needs
approval and it is not an authorization-policy API.
```python
def present(request):
# Deliver request.command and request.description to your UI, wait for
# its authenticated human response, then return a request-bound decision.
choice = send_to_my_ui_and_wait(request) # once/session/always/deny
return request.respond(choice)
def register(ctx):
ctx.register_approval_transport("my-ui", present)
```
`present` may be synchronous or async. Hermes runs it on a bounded worker and
enforces the canonical `approvals.timeout` even if the plugin does not. The
request is immutable and contains redacted display text, its host presentation
class (`cli` or `gateway`), the host timeout, allowed choices, and an opaque
request ID/digest.
Return the result of
`request.respond(choice)`; unbound dictionaries and stale or changed request
IDs/digests are rejected. A plugin cannot return a scope that the host did not
offer (for example, `always` on a once-only request).
Registration alone does nothing. Enabling the plugin and explicitly selecting
its transport are separate consent steps:
```yaml
plugins:
enabled: [my-approval-plugin]
security:
approval:
transport: my-ui
transport_fallback: deny # default
```
Transport exceptions, timeouts, unavailable registrations, invalid choices,
and stale responses deny by default. To deliberately show the prompt on the
ordinary CLI/TUI/gateway/ACP surface when the selected transport fails, set
`transport_fallback: builtin`. Without that exact opt-in, Hermes never
materializes the prompt on another surface.
Hermes still owns hardline blocks, sudo-stdin protection, user deny rules,
request binding, allowed scopes, persistence, hooks, and final authorization.
Hardline commands are blocked before any transport callback. There is
intentionally **no plugin approval policy, auto-allow callback, or required
`pre_tool_call` policy** in this interface. A future approval-policy capability
may use the plugin capability-consent model, but transport selection does not
grant it.
### Migration for existing users
When you upgrade to a version of Hermes that has opt-in plugins (config schema v21+), any user plugins already installed under `~/.hermes/plugins/` that weren't already in `plugins.disabled` are **automatically grandfathered** into `plugins.enabled`. Your existing setup keeps working. Bundled standalone plugins are NOT grandfathered — even existing users have to opt in explicitly. (Bundled platform/backend plugins never needed grandfathering because they were never gated.)
## Available hooks
Plugins can register the 26 lifecycle events currently accepted by `hermes_cli.plugins.VALID_HOOKS`. The **[Event Hooks catalog](/user-guide/features/hooks#shipped-plugin-hook-catalog)** is canonical for exact timing, return handling, payload fields, and privacy notes.
| Descriptive category | Shipped hooks |
|---|---|
| **Directive/control** | `pre_tool_call`, `pre_llm_call`, `pre_verify`, `pre_gateway_dispatch` |
| **Transform** | `transform_tool_result`, `transform_terminal_output`, `transform_llm_output`, `pre_transcription` |
| **Observer** | `post_tool_call`, `post_llm_call`, `pre_api_request`, `post_api_request`, `api_request_error`, `on_stream_start`, `on_stream_delta`, `on_stream_end`, `on_interim_message`, `on_session_start`, `on_session_end`, `on_session_finalize`, `on_session_reset`, `on_skill_lifecycle`, `subagent_start`, `subagent_stop`, `pre_approval_request`, `post_approval_response`, `pre_command`, `kanban_task_claimed`, `kanban_task_completed`, `kanban_task_blocked` |
These categories describe current behavior rather than defining future naming rules. Plugin middleware remains a separate registry/surface.
## Plugin types
Hermes has four kinds of plugins:
| Type | What it does | Selection | Location |
|------|-------------|-----------|----------|
| **General plugins** | Add tools, hooks, slash commands, CLI commands | Multi-select (enable/disable) | `~/.hermes/plugins/` |
| **Memory providers** | Replace or augment built-in memory | Single-select (one active) | `plugins/memory/` |
| **Context engines** | Replace the built-in context compressor | Single-select (one active) | `plugins/context_engine/` |
| **Model providers** | Declare an inference backend (OpenRouter, Anthropic, …) | Multi-register, picked by `--provider` / `config.yaml` | `plugins/model-providers/` |
Memory providers and context engines are **provider plugins** — only one of each type can be active at a time. Model providers are also plugins, but many load simultaneously; the user picks one at a time via `--provider` or `config.yaml`. General plugins can be enabled in any combination.
## Pluggable interfaces — where to go for each
The table above shows the four plugin categories, but within "General plugins" the `PluginContext` exposes several distinct extension points — and Hermes also accepts extensions outside the Python plugin system (config-driven backends, shell-hooked commands, external servers, etc.). Use this table to find the right doc for what you want to build:
| Want to add… | How | Authoring guide |
|---|---|---|
| A **tool** the LLM can call | Python plugin — `ctx.register_tool()` | [Build a Hermes Plugin](/developer-guide/plugins) · [Adding Tools](/developer-guide/adding-tools) |
| A **lifecycle hook** (pre/post LLM, session start/end, tool filter) | Python plugin — `ctx.register_hook()` | [Hooks reference](/user-guide/features/hooks) · [Build a Hermes Plugin](/developer-guide/plugins) |
| A **slash command** for the CLI / gateway | Python plugin — `ctx.register_command()` | [Build a Hermes Plugin](/developer-guide/plugins) · [Extending the CLI](/developer-guide/extending-the-cli) |
| A **subcommand** for `hermes <thing>` | Python plugin — `ctx.register_cli_command()` | [Extending the CLI](/developer-guide/extending-the-cli) |
| A bundled **skill** that your plugin ships | Python plugin — `ctx.register_skill()` | [Creating Skills](/developer-guide/creating-skills) |
| An **inference backend** (LLM provider: OpenAI-compat, Codex, Anthropic-Messages, Bedrock) | Provider plugin — `register_provider(ProviderProfile(...))` in `plugins/model-providers/<name>/` | **[Model Provider Plugins](/developer-guide/model-provider-plugin)** · [Adding Providers](/developer-guide/adding-providers) |
| A **gateway channel** (Discord / Telegram / IRC / Teams / etc.) | Platform plugin — `ctx.register_platform()` in `plugins/platforms/<name>/` | [Adding Platform Adapters](/developer-guide/adding-platform-adapters) |
| A **memory backend** (Honcho, Mem0, Supermemory, …) | Memory plugin — subclass `MemoryProvider` in `plugins/memory/<name>/` | [Memory Provider Plugins](/developer-guide/memory-provider-plugin) |
| A **context-compression strategy** | Context-engine plugin — `ctx.register_context_engine()` | [Context Engine Plugins](/developer-guide/context-engine-plugin) |
| An **image-generation backend** (DALL·E, SDXL, …) | Backend plugin — `ctx.register_image_gen_provider()` | [Image Generation Provider Plugins](/developer-guide/image-gen-provider-plugin) |
| A **video-generation backend** (Veo, Kling, Pixverse, Grok-Imagine, Runway, …) | Backend plugin — `ctx.register_video_gen_provider()` | [Video Generation Provider Plugins](/developer-guide/video-gen-provider-plugin) |
| A **TTS backend** (any CLI — Piper, VoxCPM, Kokoro, xtts, voice-cloning scripts, …) | Config-driven (recommended) — declare under `tts.providers.<name>` with `type: command` in `config.yaml`. OR Python backend plugin — `ctx.register_tts_provider()` for Python-SDK / streaming engines that need more than a shell template. | [TTS Setup](/user-guide/features/tts#custom-command-providers) · [Python plugin guide](/user-guide/features/tts#python-plugin-providers) |
| An **STT backend** (any CLI — whisper.cpp, custom whisper binary, local ASR CLI) | Config-driven (recommended) — declare under `stt.providers.<name>` with `type: command` in `config.yaml`, or set `HERMES_LOCAL_STT_COMMAND` for the legacy single-command escape hatch. OR Python backend plugin — `ctx.register_transcription_provider()` for Python-SDK engines (OpenRouter, SenseAudio, Gemini-STT, etc.). | [STT Setup](/user-guide/features/tts#stt-custom-command-providers) · [Python plugin guide](/user-guide/features/tts#python-plugin-providers-stt) |
| **External tools via MCP** (filesystem, GitHub, Linear, Notion, any MCP server) | Config-driven — declare `mcp_servers.<name>` with `command:` / `url:` in `config.yaml`. Hermes auto-discovers the server's tools and registers them alongside built-ins. | [MCP](/user-guide/features/mcp) |
| **Additional skill sources** (custom GitHub repos, private skill indexes) | CLI — `hermes skills tap add <repo>` | [Skills Hub](/user-guide/features/skills#skills-hub) · [Publishing a custom tap](/user-guide/features/skills#publishing-a-custom-skill-tap) |
| **Gateway event hooks** (fire on `gateway:startup`, `session:start`, `agent:end`, `command:*`) | Drop `HOOK.yaml` + `handler.py` into `~/.hermes/hooks/<name>/` | [Event Hooks](/user-guide/features/hooks#gateway-event-hooks) |
| **Shell hooks** (run a shell command on events — notifications, audit logs, desktop alerts) | Config-driven — declare under `hooks:` in `config.yaml` | [Shell Hooks](/user-guide/features/hooks#shell-hooks) |
:::note
Not everything is a Python plugin. Some extension surfaces intentionally use **config-driven shell commands** (TTS, STT, shell hooks) so any CLI you already have becomes a plugin without writing Python. Others are **external servers** (MCP) the agent connects to and auto-registers tools from. And some are **drop-in directories** (gateway hooks) with their own manifest format. Pick the right surface for the integration style that fits your use case; the authoring guides in the table above each cover placeholders, discovery, and examples.
:::
## NixOS declarative plugins
On NixOS, plugins can be installed declaratively via the module options — no `hermes plugins install` needed. See the **[Nix Setup guide](/getting-started/nix-setup#plugins)** for full details.
```nix
services.hermes-agent = {
# Directory plugin (source tree with plugin.yaml)
extraPlugins = [ (pkgs.fetchFromGitHub { ... }) ];
# Entry-point plugin (pip package)
extraPythonPackages = [ (pkgs.python312Packages.buildPythonPackage { ... }) ];
# Enable in config
settings.plugins.enabled = [ "my-plugin" ];
};
```
Declarative plugins are symlinked with a `nix-managed-` prefix — they coexist with manually installed plugins and are cleaned up automatically when removed from the Nix config.
## Managing plugins
```bash
hermes plugins # unified interactive UI
hermes plugins list # table: enabled / disabled / not enabled
hermes plugins search <term> # search the community plugin index
hermes plugins install <name> # install by index name (resolved to repo @ pinned ref)
hermes plugins install user/repo # install from Git, then prompt Enable? [y/N]
hermes plugins install user/repo --enable # install AND enable (no prompt)
hermes plugins install user/repo --no-enable # install but leave disabled (no prompt)
hermes plugins update my-plugin # pull latest (local edits are autostashed and re-applied)
hermes plugins remove my-plugin # uninstall
hermes plugins enable my-plugin # add to allow-list
hermes plugins disable my-plugin # remove from allow-list + add to disabled
hermes plugins capabilities [my-plugin] # declared vs granted capabilities
```
### One-click install links (Desktop)
Hermes Desktop registers the `hermes://` URL scheme, so a website, README, or
chat message can link straight to a plugin install:
```
hermes://plugin/install?repo=owner/repo # main install link
hermes://plugin/install?repo=owner/repo&enable=1 # enable the agent plugin after install
hermes://plugin/install?repo=owner/repo&force=1 # replace an existing install
```
Clicking one opens Hermes and shows a **confirmation dialog** — the repo id,
a "Before you install" note, and GitHub browse + clone links — then
shallow-clones the repo to detect what it ships (an **agent plugin** —
backend Python, a **desktop plugin** — app UI, or both). You pick the
components with checkboxes and confirm. Nothing is installed until you do;
deep links never auto-install, and agent-plugin installs go through the same
[install-time security scanning](#install-time-security-scanning) as
`hermes plugins install`.
Hybrid repos (agent + desktop halves in one repo) use one link and one
dialog. The same modal is reachable without a link via **Settings → Plugins →
Install from Git**. Legacy `hermes://plugin-agent/…` and
`hermes://plugin-desktop/…` URLs route into the same dialog. In dev builds
(`npm run dev`) the scheme is `hermes-dev://`.
Websites need no SDK — a normal anchor works:
```html
<a href="hermes://plugin/install?repo=owner/repo&enable=1">Install in Hermes</a>
```
MCP servers have the equivalent link form — see
[Add to Hermes link](/reference/mcp-config-reference#add-to-hermes-link).
### Plugin capabilities and consent
Plugins can declare the privileged host surfaces they want in their
`plugin.yaml`:
```yaml
name: my-plugin
capabilities:
- tools.override # replace built-in tools
- llm.model_override # pick the model for host-owned LLM calls
```
When a plugin declares capabilities, `hermes plugins install` (and
`hermes plugins enable`) shows the list with one-line risk descriptions and
asks once. Consenting records the grant under
`plugins.entries.<id>.granted_capabilities` together with a consent hash and
timestamp. Declining leaves the plugin enabled with those capabilities off —
a well-behaved plugin probes with `ctx.has_capability()` and degrades
gracefully.
**Update re-consent:** if a plugin update declares capabilities you haven't
granted, `hermes plugins update` surfaces the additions and asks again. New
capabilities stay off until you consent — a plugin update can never silently
widen its access.
**Non-interactive sessions fail closed:** installing or updating without a
TTY completes the install, but declared capabilities are *not* granted. Run
`hermes plugins enable <id>` interactively to grant them later.
Inspect the state at any time:
```bash
hermes plugins capabilities # all plugins with declared/granted capabilities
hermes plugins capabilities my-plugin # one plugin, declared vs granted
```
Capability ids map 1:1 to the older per-feature config gates, which keep
working but are **deprecated** in favor of the consent flow:
| Capability | Legacy key (`plugins.entries.<id>.…`) |
|---|---|
| `tools.override` | `allow_tool_override` |
| `llm.provider_override` | `llm.allow_provider_override` |
| `llm.model_override` | `llm.allow_model_override` |
| `llm.agent_id_override` | `llm.allow_agent_id_override` |
| `llm.profile_override` | `llm.allow_profile_override` |
| `llm.task_override` | `llm.allow_task_override` |
| `gateway.platform_actions` | `allow_platform_actions` |
A gate is open when *either* the capability is granted *or* the legacy key is
set — existing configs keep working unchanged.
:::warning Not a sandbox
Capabilities are a **consent and audit layer**, not isolation. Plugins run as
regular in-process Python: a malicious plugin can ignore every gate here.
Granting a capability is a statement of trust in the plugin author — it is
not a code audit, and Hermes has not reviewed the plugin's code. Only install
plugins from sources you trust.
:::
### Platform actions
`ctx.platform_actions` gives a plugin a minimal, capability-gated verb set for
acting on connected chat platforms through the live gateway adapter registry —
the sanctioned alternative to monkeypatching an adapter. **It is off by
default**: every call re-checks the `gateway.platform_actions` capability
(legacy key `plugins.entries.<id>.allow_platform_actions`), and an ungranted
call returns a structured error instead of acting.
v1 verbs (both `async`, both return a plain dict, and neither ever raises into
hook dispatch):
```python
result = await ctx.platform_actions.add_reaction(
platform="telegram", chat_id="-100123", message_id="456", emoji="👍",
)
result = await ctx.platform_actions.set_thread_title(
platform="discord", chat_id="123", thread_id="456", title="New title",
)
if not result["ok"]:
print(result["error"], result.get("detail"))
```
Success is `{"ok": True, "action": <verb>}`. Failures are
`{"ok": False, "error": <code>, "detail": <str>}` with stable error codes:
`capability_not_granted`, `invalid_argument`, `gateway_unavailable`,
`unknown_platform`, `adapter_not_registered`, `adapter_disconnected`,
`unsupported_platform_action`, `action_failed`. Actions validate that the
target adapter exists and is connected before acting; a disconnected or
missing adapter degrades to a structured error, never an exception.
Platforms supported in v1: Telegram and Discord. Telegram's `add_reaction`
*sets* the bot's reaction (the Bot API replaces a previous bot reaction rather
than stacking). Every action — allowed or denied — is written to the log with
the plugin id, verb, platform, and outcome.
:::warning Security note
Platform actions are a **messaging-as-the-bot power**: a granted plugin can
react and rename threads in any chat the gateway bot can reach, not just the
chat that triggered the hook. Grant `gateway.platform_actions` only to plugins
you trust, and prefer plugins that document exactly which actions they take.
Raw platform SDK payload/handle access is deliberately **not** part of this
surface — per the #64176 round-2 design correction it requires its own
capability (`gateway.raw_events`) with a "no stability guarantee" label and a
separate design, and has not shipped.
:::
### Discovering community plugins
`hermes plugins search <term>` searches the **community plugin index** — a
static, machine-readable JSON catalog of community plugins. Matching is fuzzy
across name, description, and tags:
```bash
hermes plugins search telegram # fuzzy search
hermes plugins search # browse the whole index
hermes plugins search --capability platform # filter by declared capability
hermes plugins search media --json # machine-readable output
hermes plugins search --refresh # bypass the 24h local cache
```
Once you've found a plugin, install it by bare name — the name is resolved
through the index to its `owner/repo` plus the index-pinned commit:
```bash
hermes plugins install hermes-media-studio
```
If a name matches more than one entry, the candidates are listed and nothing
is installed. Explicit `owner/repo` or Git-URL identifiers never touch the
index and keep working exactly as before. An explicit `--ref <sha>` always
overrides the index pin.
**How the index is fetched.** The index lives at a canonical URL
(`https://raw.githubusercontent.com/NousResearch/hermes-plugin-index/main/index.json`,
overridable via `hermes config set plugins.index_url <url>`). Fetches are
cached under `~/.hermes/cache/plugin_index.json` for 24 hours; when the
remote is unreachable the stale cache is used, and when there is no cache at
all a bundled seed copy ships with Hermes — so search works fully offline.
**Index entry format.** Each entry is a JSON object:
```json
{
"name": "hermes-media-studio",
"description": "Generative media workspace plugin.",
"author": "NousResearch",
"tags": ["media", "image-gen"],
"repo": "NousResearch/hermes-media-studio",
"ref": "<40-char commit SHA>",
"subdir": null,
"homepage": "https://github.com/NousResearch/hermes-media-studio",
"capabilities": ["tools", "dashboard"],
"api_version": 1,
"added_at": "2026-08-12"
}
```
`repo` is the `owner/name` GitHub identifier, `ref` pins an immutable commit
SHA, and optional `subdir` supports monorepos. The bundled seed file
(`hermes_cli/data/plugin_index.json` in the repo) is the format reference.
**Submitting a plugin.** The index is maintained as a plain JSON file —
submit a pull request to the
[hermes-plugin-index](https://github.com/NousResearch/hermes-plugin-index)
repository adding your entry (name, description, author, tags, `owner/repo`,
and a pinned commit SHA). Review covers the entry's *metadata* only.
:::warning Indexed ≠ audited
Inclusion in the community index means the entry's metadata was reviewed —
**it is not a code audit**. Installing still goes through the normal
consent/review flow (plugins install disabled by default, enabling is an
explicit step, and tool-override rights require a separate grant). Review a
plugin's source before enabling it.
:::
### Plugin packs
A **plugin pack** is a declarative, shareable YAML file (`hermes-pack.yaml`)
that pins a set of plugins — like sharing a modpack. Installing a pack fans
out to ordinary pinned installs; nothing new exists at runtime.
```yaml
name: voice-assistant-pack
description: STT + streaming TTS + approval relay
author: hyper
version: 1.0.0
plugins:
- name: hermes-media-studio # bare community-index name…
ref: e8d59971d2b7901405b39dac7b03bdd616272d0d
- repo: owner/approval-relay # …or explicit owner/repo (or git URL)
ref: 8f3c2d1a9b4e5f6071829304a5b6c7d8e9f00112
subdir: plugins/relay # optional monorepo path
config: # optional, non-secret seeds only
hermes-media-studio:
default_model: flux-3
skills: [] # declared list only (not auto-installed yet)
```
```bash
hermes plugins pack show ./hermes-pack.yaml # dry-run review
hermes plugins pack install ./hermes-pack.yaml # review → confirm → install
hermes plugins pack export > hermes-pack.yaml # snapshot the current install
hermes plugins pack export --enabled-only # only plugins.enabled
```
**Supply-chain posture.** Every entry's `ref` must be an exact 40-character
commit SHA — tags and branch names are rejected with an error naming the
entry, the same rule as the community index. Pack installs ride the exact
same pinned install path as `hermes plugins install --ref <sha>` and record
the same provenance in `plugins/.install-metadata.json`, so two installs of
the same pack resolve identically. Packs build on the
[manifest v2 fields](/developer-guide/plugins) (`manifest_version`,
`api_version`, `requires_plugins`) — each plugin's own manifest still
validates through the normal install path.
**Consent is never bulk-granted.** `pack install` shows a mandatory review
screen (every plugin, source, pinned ref, and the capabilities it declares),
then asks **one** confirmation for the pack contents. After that, each
plugin's declared capabilities go through the standard per-plugin
capability-consent prompt — identical to a single `hermes plugins install`.
There is no `--yes`, and non-interactive sessions cannot install packs.
**Secrets never travel in packs.** `config:` seeds are limited to
non-secret `plugins.entries.<id>` keys — secret-shaped key names
(`*token*`, `*key*`, `*password*`, …), capability grants, and the deprecated
`allow_*` trust gates are rejected on install and stripped on export.
Plugins that need secrets declare them in their own `requires_env`, which
prompts during install as usual. Existing user values in
`plugins.entries.<id>` always win over pack seeds.
**Partial failure.** Each plugin installs independently; failures are
reported per plugin, the rest continue, and the command exits non-zero if
any plugin failed.
**Export caveats.** `pack export` only includes plugins with known Git
provenance (installed via `hermes plugins install`). Local-only plugins are
listed as warning comments in the emitted YAML, not as installable entries.
The `skills:` list is parsed and displayed at install time but not yet
auto-installed — install those manually for now (`hermes skills`). Wiring
skill-hub ids into pack install is a documented follow-up seam.
### Install-time security scanning
Every `hermes plugins install` and `hermes plugins update` runs a static
security scan over the plugin tree before it is activated (inspired by
Claude Cowork's skill & plugin security scanning). The scanner reuses the
same threat-pattern engine as the [Skills Hub guard](/user-guide/features/skills)
— exfiltration of credential stores, reverse shells, destructive commands,
persistence mechanisms, obfuscated execution, and prompt injection in
documentation files — with plugin-aware exemptions: a provider plugin
reading its **own** API key from the environment (the documented
`requires_env` pattern) is not flagged.
Three verdicts, matching Cowork's pass/warn/fail:
| Verdict | Behavior |
|---|---|
| **safe** | Installs normally, no extra output |
| **caution** | Findings are shown; you confirm `Install anyway? [y/N]` (or pass `--force`) |
| **dangerous** | Blocked. `--force` does **not** override |
On `hermes plugins update`, a dangerous verdict on the updated tree
disables the plugin until you review the findings and re-enable it.
Scanning is on by default; disable it in `config.yaml`:
```yaml
plugins:
scan_on_install: false
```
### Interactive UI
Running `hermes plugins` with no arguments opens a composite interactive screen:
```
Plugins
↑↓ navigate SPACE toggle ENTER configure/confirm ESC done
General Plugins
→ [✓] my-tool-plugin — Custom search tool
[ ] webhook-notifier — Event hooks
[ ] disk-cleanup — Auto-cleanup of ephemeral files [bundled]
Provider Plugins
Memory Provider ▸ honcho
Context Engine ▸ compressor
```
- **General Plugins section** — checkboxes, toggle with SPACE. Checked = in `plugins.enabled`, unchecked = in `plugins.disabled` (explicit off).
- **Provider Plugins section** — shows current selection. Press ENTER to drill into a radio picker where you choose one active provider.
- Bundled plugins appear in the same list with a `[bundled]` tag.
Provider plugin selections are saved to `config.yaml`:
```yaml
memory:
provider: "honcho" # empty string = built-in only
context:
engine: "compressor" # default built-in compressor
```
### Enabled vs. disabled vs. neither
Plugins occupy one of three states:
| State | Meaning | In `plugins.enabled`? | In `plugins.disabled`? |
|---|---|---|---|
| `enabled` | Loaded on next session | Yes | No |
| `disabled` | Explicitly off — won't load even if also in `enabled` | (irrelevant) | Yes |
| `not enabled` | Discovered but never opted in | No | No |
The default for a newly-installed or bundled plugin is `not enabled`. `hermes plugins list` shows all three distinct states so you can tell what's been explicitly turned off vs. what's just waiting to be enabled.
In a running session, `/plugins` shows which plugins are currently loaded.
## Injecting Messages
Plugins can inject messages into a CLI conversation or a known gateway session using `ctx.inject_message()`:
```python
# Active CLI conversation
ctx.inject_message("New data arrived from the webhook", role="user")
# Existing gateway conversation
ctx.inject_message(
"New data arrived from the webhook",
role="user",
session_key="agent:main:telegram:dm:123456789",
)
```
**Signature:** `ctx.inject_message(content: str, role: str = "user", *, session_key: str | None = None) -> bool`
In CLI mode:
- If the agent is **idle** (waiting for user input), the message is queued as the next input and starts a new turn.
- If the agent is **mid-turn** (actively running), the message interrupts the current operation — the same as a user typing a new message and pressing Enter.
- For non-`"user"` roles, the content is prefixed with `[role]` (e.g. `[system] ...`).
- Returns `True` if the message was queued successfully.
In gateway mode:
- `session_key` is required and must identify an existing gateway session. It is the stable routing key, not the CLI session ID.
- Hermes reuses that session's stored platform, chat, thread, profile, and conversation history. Plugins cannot supply a new chat route through this API.
- Hermes rechecks the stored route against the gateway's current authorisation rules before dispatch.
- Routes that relied only on an adapter-time or upstream authorisation decision are rejected unless Hermes can revalidate them from current core allowlists, pairing, or explicit allow-all configuration.
- Injected text is always conversational input. It cannot invoke slash commands, approve tools, or resolve pending confirmation and clarification prompts.
- The route and conversation are pinned while dispatch is pending. Hermes drops the request if topic recovery changes the route or the session rotates before handling starts.
- The request enters the platform adapter's normal message path. Active sessions use the existing busy-session queue rather than starting a competing turn.
- Returns `True` when the live gateway accepts the request for asynchronous dispatch. This does not confirm that the agent turn or platform delivery has completed.
- Returns `False` when `session_key` is omitted, the permission is not granted, or no live gateway can accept the request. Unknown or unroutable session keys discovered after asynchronous acceptance are written to the gateway log.
This enables plugins like remote control viewers, messaging bridges, or webhook receivers to feed messages into the conversation from external sources.
Gateway injection can send an agent response to an external messaging platform. It is disabled by default for every plugin. Grant it per plugin in `config.yaml`:
```yaml
plugins:
entries:
my-plugin:
allow_gateway_injection: true
```
:::warning
Only grant gateway injection to plugins you trust. Hermes checks this host API permission and restricts it to existing session routes, but Python plugins run in-process and this setting is not a sandbox.
:::
:::note
This plugin API does not expose a public HTTP endpoint or CLI command for external processes. The plugin must already know the target gateway `session_key`, for example from its own trusted configuration or previously retained session state.
:::
## Calling MCP servers from plugins
`ctx.call_mcp()` lets a plugin call a tool on one of the user's configured MCP servers — synchronously, from any hook or tool handler — routing through Hermes' existing native MCP client (same connections, trust-tier gates, circuit breaker, and reconnect logic as model-invoked MCP tools; never a parallel client).
```python
result = ctx.call_mcp(
"knowledge_rag", # server name from mcp.servers
"query_knowledge", # tool on that server
{"query": "deploy runbook"},
timeout=30, # seconds; clamped to 1–600
)
if result["ok"]:
print(result["result"])
else:
print("MCP error:", result["error"])
```
**Signature:** `ctx.call_mcp(server: str, tool: str, arguments: dict | None = None, timeout: float = 30) -> dict`
Returns a stable envelope: `{"ok": True, "result": ...}` (plus `structuredContent` when the server provides it) or `{"ok": False, "error": "..."}`. Results over ~64 KB are truncated and flagged with `"truncated": True`.
### Security: default-off, per-server allowlist
A plugin has **no MCP access by default**. The operator must grant each server explicitly in `config.yaml`:
```yaml
plugins:
entries:
my-plugin:
mcp_allowlist: ["knowledge_rag", "github"]
```
- Calling a server not in the list raises `PermissionError` naming the exact config key to set.
- The grant is per-server and per-plugin — never ambient authority over every configured server, and `"*"` wildcards are not honored.
- Every call has an enforced timeout (default 30 s) so a hung MCP server cannot stall the hook or tool pipeline that invoked it.
- MCP servers return untrusted content. Treat `result` as data, not instructions — don't feed it into privileged decisions (approvals, command execution) without validation.
:::warning
Granting `mcp_allowlist` gives the plugin the same access to that MCP server as the model has — including any write-capable tools the server exposes (subject to the server's `trust` tier gates). Grant only servers the plugin genuinely needs.
:::
See the **[full guide](/developer-guide/plugins)** for handler contracts, schema format, hook behavior, error handling, and common mistakes.
website/docs/user-guide/features/provider-routing.md
+204
View File
@@ -0,0 +1,204 @@
---
title: Provider Routing
description: Configure OpenRouter or Nous Portal provider preferences to optimize for cost, speed, or quality.
sidebar_label: Provider Routing
sidebar_position: 7
---
# Provider Routing
When using [OpenRouter](https://openrouter.ai) or [Nous Portal](/integrations/nous-portal) as your LLM provider, Hermes Agent supports **provider routing** — fine-grained control over which underlying AI providers handle your requests and how they're prioritized.
OpenRouter routes requests to many providers (e.g., Anthropic, Google, AWS Bedrock, Together AI). Provider routing lets you optimize for cost, speed, quality, or enforce specific provider requirements.
:::tip
Traffic routed through Nous Portal respects the same provider preferences — and Portal subscribers get 10% off token-billed providers.
:::
## Configuration
Add a `provider_routing` section to your `~/.hermes/config.yaml`:
```yaml
provider_routing:
sort: "price" # How to rank providers
only: [] # Whitelist: only use these providers
ignore: [] # Blacklist: never use these providers
order: [] # Explicit provider priority order
require_parameters: false # Only use providers that support all parameters
data_collection: null # Control data collection ("allow" or "deny")
```
:::info
Provider routing only applies when using OpenRouter or Nous Portal. It has no effect with direct provider connections (e.g., connecting directly to the Anthropic API).
:::
## Options
### `sort`
Controls how OpenRouter ranks available providers for your request.
| Value | Description |
|-------|-------------|
| `"price"` | Cheapest provider first |
| `"throughput"` | Fastest tokens-per-second first |
| `"latency"` | Lowest time-to-first-token first |
```yaml
provider_routing:
sort: "price"
```
### `only`
Whitelist of provider slugs. When set, **only** these providers will be used. All others are excluded. Use the lowercase slug shown by OpenRouter for each provider.
```yaml
provider_routing:
only:
- "anthropic"
- "google"
```
### `ignore`
Blacklist of provider names. These providers will **never** be used, even if they offer the cheapest or fastest option.
```yaml
provider_routing:
ignore:
- "together"
- "deepinfra"
```
### `order`
Explicit priority order. Providers listed first are preferred. Unlisted providers are used as fallbacks.
```yaml
provider_routing:
order:
- "anthropic"
- "google"
- "amazon-bedrock"
```
### `require_parameters`
When `true`, OpenRouter will only route to providers that support **all** parameters in your request (like `temperature`, `top_p`, `tools`, etc.). This avoids silent parameter drops.
```yaml
provider_routing:
require_parameters: true
```
### `data_collection`
Controls whether providers can use your prompts for training. Options are `"allow"` or `"deny"`.
```yaml
provider_routing:
data_collection: "deny"
```
## Practical Examples
### Optimize for Cost
Route to the cheapest available provider. Good for high-volume usage and development:
```yaml
provider_routing:
sort: "price"
```
### Optimize for Speed
Prioritize low-latency providers for interactive use:
```yaml
provider_routing:
sort: "latency"
```
### Optimize for Throughput
Best for long-form generation where tokens-per-second matters:
```yaml
provider_routing:
sort: "throughput"
```
### Lock to Specific Providers
Ensure all requests go through a specific provider for consistency:
```yaml
provider_routing:
only:
- "anthropic"
```
### Avoid Specific Providers
Exclude providers you don't want to use (e.g., for data privacy):
```yaml
provider_routing:
ignore:
- "together"
- "lepton"
data_collection: "deny"
```
### Preferred Order with Fallbacks
Try your preferred providers first, fall back to others if unavailable:
```yaml
provider_routing:
order:
- "anthropic"
- "google"
require_parameters: true
```
## How It Works
Provider routing preferences are passed to OpenRouter or Nous Portal on agent chat requests and iteration-limit summaries via the `extra_body.provider` field. (`extra_body` is the OpenAI Python SDK argument; it becomes the top-level `provider` object in the JSON request.) Auxiliary tasks such as compression and title generation are configured independently under `auxiliary.<task>.extra_body`.
- **CLI mode** — configured in `~/.hermes/config.yaml`, loaded at startup
- **Gateway mode** — same config file, loaded when the gateway starts
The routing config is read from `config.yaml` and passed as parameters when creating the `AIAgent`:
```
providers_allowed ← from provider_routing.only
providers_ignored ← from provider_routing.ignore
providers_order ← from provider_routing.order
provider_sort ← from provider_routing.sort
provider_require_parameters ← from provider_routing.require_parameters
provider_data_collection ← from provider_routing.data_collection
```
:::tip
You can combine multiple options. For example, sort by price but exclude certain providers and require parameter support:
```yaml
provider_routing:
sort: "price"
ignore: ["together"]
require_parameters: true
data_collection: "deny"
```
:::
## Default Behavior
When no `provider_routing` section is configured (the default), the aggregator uses its own default routing logic, which generally balances cost and availability automatically.
:::tip Provider Routing vs. Fallback Models
Provider routing controls which **sub-providers behind OpenRouter or Nous Portal** handle your requests. For automatic failover to an entirely different provider when your primary model fails, see [Fallback Providers](/user-guide/features/fallback-providers).
:::
website/docs/user-guide/features/skills.md
+1037
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/features/skins.md
+271
View File
@@ -0,0 +1,271 @@
---
sidebar_position: 10
title: "Skins & Themes"
description: "Customize the Hermes CLI with built-in and user-defined skins"
---
# Skins & Themes
Skins control the **visual presentation** of the Hermes CLI: banner colors, spinner faces and verbs, response-box labels, branding text, and the tool activity prefix.
Conversational style and visual style are separate concepts:
- **Personality** changes the agent's tone and wording.
- **Skin** changes the CLI's appearance.
## Change skins
```bash
/skin # show the current skin and list available skins
/skin ares # switch to a built-in skin
/skin mytheme # switch to a custom skin from ~/.hermes/skins/mytheme.yaml
```
Or set the default skin in `~/.hermes/config.yaml`:
```yaml
display:
skin: default
```
## Built-in skins
| Skin | Description | Agent branding | Visual character |
|------|-------------|----------------|------------------|
| `default` | Classic Hermes — gold and kawaii | `Hermes Agent` | Warm gold borders, cornsilk text, kawaii faces in spinners. The familiar caduceus banner. Clean and inviting. |
| `ares` | War-god theme — crimson and bronze | `Ares Agent` | Deep crimson borders with bronze accents. Aggressive spinner verbs ("forging", "marching", "tempering steel"). Custom sword-and-shield ASCII art banner. |
| `mono` | Monochrome — clean grayscale | `Hermes Agent` | All grays — no color. Borders are `#555555`, text is `#c9d1d9`. Ideal for minimal terminal setups or screen recordings. |
| `slate` | Cool blue — developer-focused | `Hermes Agent` | Royal blue borders (`#4169e1`), soft blue text. Calm and professional. No custom spinner — uses default faces. |
| `daylight` | Light theme for bright terminals with dark text and cool blue accents | `Hermes Agent` | Designed for white or bright terminals. Dark slate text with blue borders, pale status surfaces, and a light completion menu that stays readable in light terminal profiles. |
| `warm-lightmode` | Warm brown/gold text for light terminal backgrounds | `Hermes Agent` | Warm parchment tones for light terminals. Dark brown text with saddle-brown accents, cream-colored status surfaces. An earthy alternative to the cooler daylight theme. |
| `poseidon` | Ocean-god theme — deep blue and seafoam | `Poseidon Agent` | Deep blue to seafoam gradient. Ocean-themed spinners ("charting currents", "sounding the depth"). Trident ASCII art banner. |
| `sisyphus` | Sisyphean theme — austere grayscale with persistence | `Sisyphus Agent` | Light grays with stark contrast. Boulder-themed spinners ("pushing uphill", "resetting the boulder", "enduring the loop"). Boulder-and-hill ASCII art banner. |
| `charizard` | Volcanic theme — burnt orange and ember | `Charizard Agent` | Warm burnt orange to ember gradient. Fire-themed spinners ("banking into the draft", "measuring burn"). Dragon-silhouette ASCII art banner. |
## Complete list of configurable keys
### Colors (`colors:`)
Controls all color values throughout the CLI. Values are hex color strings.
| Key | Description | Default (`default` skin) |
|-----|-------------|--------------------------|
| `banner_border` | Panel border around the startup banner | `#CD7F32` (bronze) |
| `banner_title` | Title text color in the banner | `#FFD700` (gold) |
| `banner_accent` | Section headers in the banner (Available Tools, etc.) | `#FFBF00` (amber) |
| `banner_dim` | Muted text in the banner (separators, secondary labels) | `#B8860B` (dark goldenrod) |
| `banner_text` | Body text in the banner (tool names, skill names) | `#FFF8DC` (cornsilk) |
| `ui_accent` | General UI accent color (highlights, active elements) | `#FFBF00` |
| `ui_label` | UI labels and tags | `#DAA520` (goldenrod) |
| `ui_ok` | Success indicators (checkmarks, completion) | `#4caf50` (green) |
| `ui_error` | Error indicators (failures, blocked) | `#ef5350` (red) |
| `ui_warn` | Warning indicators (caution, approval prompts) | `#ffa726` (orange) |
| `prompt` | Interactive prompt text color | `#FFF8DC` |
| `input_rule` | Horizontal rule above the input area | `#CD7F32` |
| `response_border` | Border around the agent's response box (ANSI escape) | `#FFD700` |
| `session_label` | Session label color | `#DAA520` |
| `session_border` | Session ID dim border color | `#8B8682` |
| `status_bar_bg` | Background color for the TUI status / usage bar | `#1a1a2e` |
| `voice_status_bg` | Background color for the voice-mode status badge | `#1a1a2e` |
| `selection_bg` | Background color for the TUI mouse-selection highlighter. Falls back to `completion_menu_current_bg` when unset. | `#3a3a55` |
| `completion_menu_bg` | Background color for the completion menu list | `#1a1a2e` |
| `completion_menu_current_bg` | Background color for the active completion row | `#333355` |
| `completion_menu_meta_bg` | Background color for the completion meta column | `#1a1a2e` |
| `completion_menu_meta_current_bg` | Background color for the active completion meta column | `#333355` |
### Spinner (`spinner:`)
Controls the animated spinner shown while waiting for API responses.
| Key | Type | Description | Example |
|-----|------|-------------|---------|
| `waiting_faces` | list of strings | Faces cycled while waiting for API response | `["(⚔)", "(⛨)", "(▲)"]` |
| `thinking_faces` | list of strings | Faces cycled during model reasoning | `["(⚔)", "(⌁)", "(<>)"]` |
| `thinking_verbs` | list of strings | Verbs shown in spinner messages | `["forging", "plotting", "hammering plans"]` |
| `wings` | list of [left, right] pairs | Decorative brackets around the spinner | `[["⟪⚔", "⚔⟫"], ["⟪▲", "▲⟫"]]` |
When spinner values are empty (like in `default` and `mono`), hardcoded defaults from `display.py` are used.
### Branding (`branding:`)
Text strings used throughout the CLI interface.
| Key | Description | Default |
|-----|-------------|---------|
| `agent_name` | Name shown in banner title and status display | `Hermes Agent` |
| `welcome` | Welcome message shown at CLI startup | `Welcome to Hermes Agent! Type your message or /help for commands.` |
| `goodbye` | Message shown on exit | `Goodbye! ⚕` |
| `response_label` | Label on the response box header | ` ⚕ Hermes ` |
| `prompt_symbol` | Symbol before the user input prompt (bare token, renderers add a trailing space) | `❯` |
| `help_header` | Header text for the `/help` command output | `(^_^)? Available Commands` |
### Other top-level keys
| Key | Type | Description | Default |
|-----|------|-------------|---------|
| `tool_prefix` | string | Character prefixed to tool output lines in the CLI | `┊` |
| `tool_emojis` | dict | Per-tool emoji overrides for spinners and progress (`{tool_name: emoji}`) | `{}` |
| `banner_logo` | string | Rich-markup ASCII art logo (replaces the default HERMES_AGENT banner) | `""` |
| `banner_hero` | string | Rich-markup hero art (replaces the default caduceus art) | `""` |
## Custom skins
Create YAML files under `~/.hermes/skins/`. User skins inherit missing values from the built-in `default` skin, so you only need to specify the keys you want to change.
### Full custom skin YAML template
```yaml
# ~/.hermes/skins/mytheme.yaml
# Complete skin template — all keys shown. Delete any you don't need;
# missing values automatically inherit from the 'default' skin.
name: mytheme
description: My custom theme
colors:
banner_border: "#CD7F32"
banner_title: "#FFD700"
banner_accent: "#FFBF00"
banner_dim: "#B8860B"
banner_text: "#FFF8DC"
ui_accent: "#FFBF00"
ui_label: "#4dd0e1"
ui_ok: "#4caf50"
ui_error: "#ef5350"
ui_warn: "#ffa726"
prompt: "#FFF8DC"
input_rule: "#CD7F32"
response_border: "#FFD700"
session_label: "#DAA520"
session_border: "#8B8682"
status_bar_bg: "#1a1a2e"
voice_status_bg: "#1a1a2e"
selection_bg: "#333355"
completion_menu_bg: "#1a1a2e"
completion_menu_current_bg: "#333355"
completion_menu_meta_bg: "#1a1a2e"
completion_menu_meta_current_bg: "#333355"
spinner:
waiting_faces:
- "(⚔)"
- "(⛨)"
- "(▲)"
thinking_faces:
- "(⚔)"
- "(⌁)"
- "(<>)"
thinking_verbs:
- "processing"
- "analyzing"
- "computing"
- "evaluating"
wings:
- ["⟪⚡", "⚡⟫"]
- ["⟪●", "●⟫"]
branding:
agent_name: "My Agent"
welcome: "Welcome to My Agent! Type your message or /help for commands."
goodbye: "See you later! ⚡"
response_label: " ⚡ My Agent "
prompt_symbol: "⚡"
help_header: "(⚡) Available Commands"
tool_prefix: "┊"
# Per-tool emoji overrides (optional)
tool_emojis:
terminal: "⚔"
web_search: "🔮"
read_file: "📄"
# Custom ASCII art banners (optional, Rich markup supported)
# banner_logo: |
# [bold #FFD700] MY AGENT [/]
# banner_hero: |
# [#FFD700] Custom art here [/]
```
### Minimal custom skin example
Since everything inherits from `default`, a minimal skin only needs to change what's different:
```yaml
name: cyberpunk
description: Neon terminal theme
colors:
banner_border: "#FF00FF"
banner_title: "#00FFFF"
banner_accent: "#FF1493"
spinner:
thinking_verbs: ["jacking in", "decrypting", "uploading"]
wings:
- ["⟨⚡", "⚡⟩"]
branding:
agent_name: "Cyber Agent"
response_label: " ⚡ Cyber "
tool_prefix: "▏"
```
## Hermes Mod — Visual Skin Editor
[Hermes Mod](https://github.com/cocktailpeanut/hermes-mod) is a community-built web UI for creating and managing skins visually. Instead of writing YAML by hand, you get a point-and-click editor with live preview.
![Hermes Mod skin editor](https://raw.githubusercontent.com/cocktailpeanut/hermes-mod/master/nous.png)
**What it does:**
- Lists all built-in and custom skins
- Opens any skin into a visual editor with all Hermes skin fields (colors, spinner, branding, tool prefix, tool emojis)
- Generates `banner_logo` text art from a text prompt
- Converts uploaded images (PNG, JPG, GIF, WEBP) into `banner_hero` ASCII art with multiple render styles (braille, ASCII ramp, blocks, dots)
- Saves directly to `~/.hermes/skins/`
- Activates a skin by updating `~/.hermes/config.yaml`
- Shows the generated YAML and a live preview
### Install
**Option 1 — Pinokio (1-click):**
Find it on [pinokio.computer](https://pinokio.computer) and install with one click.
**Option 2 — npx (quickest from terminal):**
```bash
npx -y hermes-mod
```
**Option 3 — Manual:**
```bash
git clone https://github.com/cocktailpeanut/hermes-mod.git
cd hermes-mod/app
npm install
npm start
```
### Usage
1. Start the app (via Pinokio or terminal).
2. Open **Skin Studio**.
3. Choose a built-in or custom skin to edit.
4. Generate a logo from text and/or upload an image for hero art. Pick a render style and width.
5. Edit colors, spinner, branding, and other fields.
6. Click **Save** to write the skin YAML to `~/.hermes/skins/`.
7. Click **Activate** to set it as the current skin (updates `display.skin` in `config.yaml`).
Hermes Mod respects the `HERMES_HOME` environment variable, so it works with [profiles](/user-guide/profiles) too.
## Operational notes
- Built-in skins load from `hermes_cli/skin_engine.py`.
- Unknown skins automatically fall back to `default`.
- `/skin` updates the active CLI theme immediately for the current session.
- User skins in `~/.hermes/skins/` take precedence over built-in skins with the same name.
- Skin changes via `/skin` are session-only. To make a skin your permanent default, set it in `config.yaml`.
- The `banner_logo` and `banner_hero` fields support Rich console markup (e.g., `[bold #FF0000]text[/]`) for colored ASCII art.
website/docs/user-guide/features/spotify.md
+279
View File
@@ -0,0 +1,279 @@
# Spotify
Hermes can control Spotify directly — playback, queue, search, playlists, saved tracks/albums, and listening history — using Spotify's official Web API with PKCE OAuth. Tokens are stored in `~/.hermes/auth.json` and refreshed automatically on 401; you only log in once per machine (refresh tokens expire after ~6 months; re-run `hermes auth spotify` when they do).
Unlike Hermes' built-in OAuth integrations (Google, GitHub Copilot, Codex), Spotify requires every user to register their own lightweight developer app. Spotify does not let third parties ship a public OAuth app that anyone can use. It takes about two minutes and `hermes auth spotify` walks you through it.
## Prerequisites
- A Spotify account. **Free** works for search, playlist, library, and activity tools. **Premium** is required for playback control (play, pause, skip, seek, volume, queue add, transfer).
- Hermes Agent installed and running.
- For playback tools: an **active Spotify Connect device** — the Spotify app must be open on at least one device (phone, desktop, web player, speaker) so the Web API has something to control. If nothing is active you'll get a `403 Forbidden` with a "no active device" message; open Spotify on any device and retry.
## Setup
### One-shot: `hermes tools` or first-run setup
The fastest path. Run:
```bash
hermes tools
```
Scroll to `🎵 Spotify`, press space to toggle it on, then `s` to save. The same toggle is also available during the first-run `hermes setup` / `hermes setup tools` flow. Spotify stays opt-in, so enabling it there runs the same provider-aware configuration as `hermes tools`.
Hermes drops you straight into the OAuth flow — if you don't have a Spotify app yet, it walks you through creating one inline. Once you finish, the toolset is enabled AND authenticated in one pass.
If you prefer to do the steps separately (or you're re-authing later), use the two-step flow below.
### Two-step flow
#### 1. Enable the toolset
```bash
hermes tools
```
Toggle `🎵 Spotify` on, save, and when the inline wizard opens, dismiss it (Ctrl+C). The toolset stays on; only the auth step is deferred.
#### 2. Run the login wizard
```bash
hermes auth spotify
```
The 7 Spotify tools only appear in the agent's toolset after step 1 — they're off by default so users who don't want them don't ship extra tool schemas on every API call.
If no `HERMES_SPOTIFY_CLIENT_ID` is set, Hermes walks you through the app registration inline:
1. Opens `https://developer.spotify.com/dashboard` in your browser
2. Prints the exact values to paste into Spotify's "Create app" form
3. Prompts you for the Client ID you get back
4. Saves it to `~/.hermes/.env` so future runs skip this step
5. Continues straight into the OAuth consent flow
After you approve, tokens are written under `providers.spotify` in `~/.hermes/auth.json`. The active inference provider is NOT changed — Spotify auth is independent of your LLM provider.
### Creating the Spotify app (what the wizard asks for)
When the dashboard opens, click **Create app** and fill in:
| Field | Value |
|-------|-------|
| App name | anything (e.g. `hermes-agent`) |
| App description | anything (e.g. `personal Hermes integration`) |
| Website | leave blank |
| Redirect URI | `http://127.0.0.1:43827/spotify/callback` |
| Which API/SDKs? | check **Web API** |
Agree to the terms and click **Save**. On the next page click **Settings** → copy the **Client ID** and paste it into the Hermes prompt. That's the only value Hermes needs — PKCE doesn't use a client secret.
### Running over SSH / in a headless environment
If `SSH_CLIENT` or `SSH_TTY` is set, Hermes skips the automatic browser open during both the wizard and the OAuth step. Copy the dashboard URL and the authorization URL Hermes prints, open them in a browser on your local machine, and proceed normally — the local HTTP listener still runs on the remote host on port `43827`. Your laptop's browser can't reach the remote loopback without an SSH local-forward:
```bash
ssh -N -L 43827:127.0.0.1:43827 user@remote-host
```
For jump-box / bastion setups and other gotchas (mosh, tmux, port conflicts), see [OAuth over SSH / Remote Hosts](../../guides/oauth-over-ssh.md).
## Verify
```bash
hermes auth status spotify
```
Shows whether tokens are present and when the access token expires. Refresh is automatic: when any Spotify API call returns 401, the client exchanges the refresh token and retries once. Refresh tokens persist across Hermes restarts, so you only re-auth if you revoke the app in your Spotify account settings or run `hermes auth logout spotify`.
## Using it
Once logged in, the agent has access to 7 Spotify tools. You talk to the agent naturally — it picks the right tool and action. For the best behavior, the agent loads a companion skill that teaches canonical usage patterns (single-search-then-play, when not to preflight `get_state`, etc.).
```
> play some miles davis
> what am I listening to
> add this track to my Late Night Jazz playlist
> skip to the next song
> make a new playlist called "Focus 2026" and add the last three songs I played
> which of my saved albums are by Radiohead
> search for acoustic covers of Blackbird
> transfer playback to my kitchen speaker
```
### Tool reference
All playback-mutating actions accept an optional `device_id` to target a specific device. If omitted, Spotify uses the currently active device.
#### `spotify_playback`
Control and inspect playback, plus fetch recently played history.
| Action | Purpose | Premium? |
|--------|---------|----------|
| `get_state` | Full playback state (track, device, progress, shuffle/repeat) | No |
| `get_currently_playing` | Just the current track (returns empty on 204 — see below) | No |
| `play` | Start/resume playback. Optional: `context_uri`, `uris`, `offset`, `position_ms` | Yes |
| `pause` | Pause playback | Yes |
| `next` / `previous` | Skip track | Yes |
| `seek` | Jump to `position_ms` | Yes |
| `set_repeat` | `state` = `track` / `context` / `off` | Yes |
| `set_shuffle` | `state` = `true` / `false` | Yes |
| `set_volume` | `volume_percent` = 0-100 | Yes |
| `recently_played` | Last played tracks. Optional `limit`, `before`, `after` (Unix ms) | No |
#### `spotify_devices`
| Action | Purpose |
|--------|---------|
| `list` | Every Spotify Connect device visible to your account |
| `transfer` | Move playback to `device_id`. Optional `play: true` starts playback on transfer |
### Home Assistant-managed speakers
If Home Assistant manages speakers that already support Spotify Connect (for example Sonos, Echo, Nest, or other Connect-capable speakers), they appear in `spotify_devices list` automatically whenever Spotify can see them. Hermes does not need a Home Assistant ↔ Spotify bridge for this path — Spotify handles the device routing natively.
Ask Hermes to transfer playback by the speaker's display name (for example, “transfer Spotify to the kitchen speaker”), or call `spotify_devices list` and pass the exact `device_id` to `spotify_devices transfer` when scripting. If the speaker is missing, open the Spotify app or the speaker's Spotify integration once so Spotify registers it as an active Connect target.
#### `spotify_queue`
| Action | Purpose | Premium? |
|--------|---------|----------|
| `get` | Currently queued tracks | No |
| `add` | Append `uri` to the queue | Yes |
#### `spotify_search`
Search the catalog. `query` is required. Optional: `types` (array of `track` / `album` / `artist` / `playlist` / `show` / `episode`), `limit`, `offset`, `market`.
#### `spotify_playlists`
| Action | Purpose | Required args |
|--------|---------|---------------|
| `list` | User's playlists | — |
| `get` | One playlist + tracks | `playlist_id` |
| `create` | New playlist | `name` (+ optional `description`, `public`, `collaborative`) |
| `add_items` | Add tracks | `playlist_id`, `uris` (optional `position`) |
| `remove_items` | Remove tracks | `playlist_id`, `uris` (+ optional `snapshot_id`) |
| `update_details` | Rename / edit | `playlist_id` + any of `name`, `description`, `public`, `collaborative` |
#### `spotify_albums`
| Action | Purpose | Required args |
|--------|---------|---------------|
| `get` | Album metadata | `album_id` |
| `tracks` | Album track list | `album_id` |
#### `spotify_library`
Unified access to saved tracks and saved albums. Pick the collection with the `kind` arg.
| Action | Purpose |
|--------|---------|
| `list` | Paginated library listing |
| `save` | Add `ids` / `uris` to library |
| `remove` | Remove `ids` / `uris` from library |
Required: `kind` = `tracks` or `albums`, plus `action`.
### Feature matrix: Free vs Premium
Read-only tools work on Free accounts. Anything that mutates playback or the queue requires Premium.
| Works on Free | Premium required |
|---------------|------------------|
| `spotify_search` (all) | `spotify_playback` — play, pause, next, previous, seek, set_repeat, set_shuffle, set_volume |
| `spotify_playback` — get_state, get_currently_playing, recently_played | `spotify_queue` — add |
| `spotify_devices` — list | `spotify_devices` — transfer |
| `spotify_queue` — get | |
| `spotify_playlists` (all) | |
| `spotify_albums` (all) | |
| `spotify_library` (all) | |
## Scheduling: Spotify + cron
Because Spotify tools are regular Hermes tools, a cron job running in a Hermes session can trigger playback on any schedule. No new code needed.
### Morning wake-up playlist
```bash
hermes cron add \
--name "morning-commute" \
"0 7 * * 1-5" \
"Transfer playback to my kitchen speaker and start my 'Morning Commute' playlist. Volume to 40. Shuffle on."
```
What happens at 7am every weekday:
1. Cron spins up a headless Hermes session.
2. Agent reads the prompt, calls `spotify_devices list` to find "kitchen speaker" by name, then `spotify_devices transfer` → `spotify_playback set_volume` → `spotify_playback set_shuffle` → `spotify_search` + `spotify_playback play`.
3. Music starts on the target speaker. Total cost: one session, a few tool calls, no human input.
### Wind-down at night
```bash
hermes cron add \
--name "wind-down" \
"30 22 * * *" \
"Pause Spotify. Then set volume to 20 so it's quiet when I start it again tomorrow."
```
### Gotchas
- **An active device must exist when the cron fires.** If no Spotify client is running (phone/desktop/Connect speaker), playback actions return `403 no active device`. For morning playlists, the trick is to target a device that's always on (Sonos, Echo, a smart speaker) rather than your phone.
- **Premium required for anything that mutates playback** — play, pause, skip, volume, transfer. Read-only cron jobs (scheduled "email me my recently played tracks") work fine on Free.
- **The cron agent inherits your active toolsets.** Spotify must be enabled in `hermes tools` for the cron session to see the Spotify tools.
- **Cron jobs run with `skip_memory=True`** so they don't write to your memory store.
Full cron reference: [Cron Jobs](./cron).
## Sign out
```bash
hermes auth logout spotify
```
Removes tokens from `~/.hermes/auth.json`. To also clear the app config, delete `HERMES_SPOTIFY_CLIENT_ID` (and `HERMES_SPOTIFY_REDIRECT_URI` if you set it) from `~/.hermes/.env`, or run the wizard again.
To revoke the app on Spotify's side, visit [Apps connected to your account](https://www.spotify.com/account/apps/) and click **REMOVE ACCESS**.
## Troubleshooting
**`403 Forbidden — Player command failed: No active device found`** — You need Spotify running on at least one device. Open the Spotify app on your phone, desktop, or web player, start any track for a second to register it, and retry. `spotify_devices list` shows what's currently visible.
**`403 Forbidden — Premium required`** — You're on a Free account trying to use a playback-mutating action. See the feature matrix above.
**`204 No Content` on `get_currently_playing`** — nothing is currently playing on any device. This is Spotify's normal response, not an error; Hermes surfaces it as an explanatory empty result (`is_playing: false`).
**`INVALID_CLIENT: Invalid redirect URI`** — the redirect URI in your Spotify app settings doesn't match what Hermes is using. The default is `http://127.0.0.1:43827/spotify/callback`. Either add that to your app's allowed redirect URIs, or set `HERMES_SPOTIFY_REDIRECT_URI` in `~/.hermes/.env` to whatever you registered.
**`429 Too Many Requests`** — Spotify's rate limit. Hermes returns a friendly error; wait a minute and retry. If this persists, you're probably running a tight loop in a script — Spotify's quota resets roughly every 30 seconds.
**`401 Unauthorized` keeps coming back** — Your refresh token was revoked (usually because you removed the app from your account, or the app was deleted). Run `hermes auth spotify` again.
**Wizard doesn't open the browser** — If you're over SSH or in a container without a display, Hermes detects it and skips the auto-open. Copy the dashboard URL it prints and open it manually.
## Advanced: custom scopes
By default Hermes requests the scopes needed for every shipped tool. Override if you want to restrict access:
```bash
hermes auth spotify --scope "user-read-playback-state user-modify-playback-state playlist-read-private"
```
Scope reference: [Spotify Web API scopes](https://developer.spotify.com/documentation/web-api/concepts/scopes). If you request fewer scopes than a tool needs, that tool's calls will fail with 403.
## Advanced: custom client ID / redirect URI
```bash
hermes auth spotify --client-id <id> --redirect-uri http://localhost:3000/callback
```
Or set them permanently in `~/.hermes/.env`:
```
HERMES_SPOTIFY_CLIENT_ID=<your_id>
HERMES_SPOTIFY_REDIRECT_URI=http://localhost:3000/callback
```
The redirect URI must be allow-listed in your Spotify app's settings. The default works for almost everyone — only change it if port 43827 is taken.
## Where things live
| File | Contents |
|------|----------|
| `~/.hermes/auth.json` → `providers.spotify` | access token, refresh token, expiry, scope, redirect URI |
| `~/.hermes/.env` | `HERMES_SPOTIFY_CLIENT_ID`, optional `HERMES_SPOTIFY_REDIRECT_URI` |
| Spotify app | owned by you at [developer.spotify.com/dashboard](https://developer.spotify.com/dashboard); contains the Client ID and the redirect URI allow-list |
website/docs/user-guide/features/subscription-proxy.md
+203
View File
@@ -0,0 +1,203 @@
---
sidebar_position: 15
title: "Subscription Proxy"
description: "Use your Nous Portal subscription (or other OAuth provider) as an OpenAI-compatible endpoint for external apps"
---
# Subscription Proxy
The subscription proxy is a local HTTP server that lets external apps —
OpenViking, Karakeep, Open WebUI, anything that speaks OpenAI-compatible
chat completions — use your Hermes-managed provider subscription as their
LLM endpoint. The proxy attaches the right credentials (refreshing them
automatically) so the app never needs a static API key.
This is different from the [API server](./api-server.md):
| | API server | Subscription proxy |
|---|---|---|
| What it serves | Your agent (full toolset, memory, skills) | Raw model inference |
| Use case | "Use Hermes as a chat backend" | "Use my Portal sub from another app" |
| Auth | Your `API_SERVER_KEY` | Any bearer (proxy attaches the real one) |
| Tool calls | Yes — the agent runs tools | No — passthrough only |
Use the API server when you want the **agent** as a backend. Use the
proxy when you just want **the model** through your subscription.
## Quick Start
### 1. Log into your provider (one-time)
```bash
hermes portal
```
This opens your browser for the Nous Portal OAuth flow. Hermes stores
the refresh token in `~/.hermes/auth.json` — the same place all Hermes
provider logins live.
### 2. Start the proxy
```bash
hermes proxy start
```
```
Starting Hermes proxy for Nous Portal
Listening on: http://127.0.0.1:8645/v1
Forwarding to: (resolved per-request from your subscription)
Use any bearer token in the client — the proxy attaches your real credential.
```
Leave this running in the foreground. Use `tmux`, `nohup`, or a systemd
unit if you want it to survive logout.
### 3. Point your app at it
Any OpenAI-compatible app config takes the same triple:
```
Base URL: http://127.0.0.1:8645/v1
API key: anything (e.g. "sk-unused")
Model: Hermes-4-70B # or Hermes-4.3-36B, Hermes-4-405B
```
The proxy ignores the `Authorization` header from your app and attaches
your real Portal credential to the upstream request. Refreshes happen
automatically when the bearer approaches expiry.
## Available providers
```bash
hermes proxy providers
```
Currently shipped: `nous` (Nous Portal) and `xai` (xAI / Grok). More
OAuth providers can be added by implementing the `UpstreamAdapter`
interface in `hermes_cli/proxy/adapters/`.
## Check status
```bash
hermes proxy status
```
```
Hermes proxy upstream adapters
[nous ] Nous Portal — ready (bearer expires 2026-05-15T06:43:21Z)
```
If you see `not logged in`, run `hermes portal`. If you see
`credentials need attention`, your refresh token was revoked (rare —
happens if you signed out from the Portal web UI) — just re-run
`hermes portal`.
## Allowed paths
The proxy only forwards paths the upstream actually serves. For Nous
Portal:
| Path | Purpose |
|------|---------|
| `/v1/chat/completions` | Chat completions (streaming + non-streaming) |
| `/v1/completions` | Legacy text completions |
| `/v1/embeddings` | Embeddings |
| `/v1/models` | Model list |
Other paths (`/v1/images/generations`, `/v1/audio/speech`, etc.) return
404 with a clear error pointing at the allowed paths. This keeps stray
clients from leaking weird requests to the upstream.
## Configuring OpenViking to use Portal
[OpenViking](https://github.com/volcengine/OpenViking) is a context
database that needs an LLM provider for its VLM (vision/language model
used to extract memories) and embedding model. With the proxy, you can
point its `vlm.api_base` at your local proxy:
Edit `~/.openviking/ov.conf`:
```json
{
"vlm": {
"provider": "openai",
"model": "Hermes-4-70B",
"api_base": "http://127.0.0.1:8645/v1",
"api_key": "unused-proxy-attaches-real-creds"
}
}
```
Then start your proxy in a terminal alongside `openviking-server`:
```bash
# Terminal 1
hermes proxy start
# Terminal 2
openviking-server
```
OpenViking's VLM calls now flow through your Portal subscription. The
embedding model side still needs its own provider — Portal does serve
`/v1/embeddings` but the model selection depends on what your tier
supports; check `portal.nousresearch.com/models`.
## Configuring Karakeep (or any bookmark/summarizer app)
[Karakeep](https://karakeep.app/) takes an OpenAI-compatible API for
bookmark summarization. In its config:
```bash
# Karakeep .env
OPENAI_API_BASE_URL=http://127.0.0.1:8645/v1
OPENAI_API_KEY=any-non-empty-string
INFERENCE_TEXT_MODEL=Hermes-4-70B
```
Same pattern works for Open WebUI, LobeChat, NextChat, or any other
OpenAI-compatible client.
## Exposing on LAN
By default the proxy binds `127.0.0.1` (localhost only). To let other
machines on your network use it:
```bash
hermes proxy start --host 0.0.0.0 --port 8645
```
⚠ **Be aware:** anyone on your network can now use your Portal
subscription. The proxy has no auth of its own — it accepts any bearer.
Use a firewall, VPN, or reverse proxy with proper auth if you expose
this beyond your trusted network.
## Rate limits
Your Portal tier's RPM/TPM limits apply across the whole proxy. The
proxy doesn't fan out or pool — it's a single bearer with your full
subscription quota. Monitor usage at
[portal.nousresearch.com](https://portal.nousresearch.com).
## Architecture
The proxy is intentionally minimal. Per request:
1. Receive `POST /v1/chat/completions` from your app
2. Look up the adapter's current credential (refresh if expiring)
3. Forward the request body verbatim, with `Authorization: Bearer <minted-key>`
4. Stream the response back unchanged (SSE preserved)
No transformation. No logging of request bodies. No agent loop. The
proxy is a credential-attaching pass-through.
## Future: more OAuth providers
The adapter system is pluggable. Adding a new provider (e.g.
HuggingFace, GitHub Copilot's chat endpoint, Anthropic via OAuth)
requires implementing `UpstreamAdapter` in
`hermes_cli/proxy/adapters/<provider>.py` and registering it in
`adapters/__init__.py`. Providers that aren't OpenAI-compatible at the
protocol level (Anthropic Messages API, for example) would need a
transformation layer, which is out of scope for the current shape.
website/docs/user-guide/features/tool-gateway.md
+215
View File
@@ -0,0 +1,215 @@
---
title: "Nous Tool Gateway"
description: "One subscription, every tool. Web search, image generation, TTS, and cloud browsers — all routed through Nous Portal with no extra API keys."
sidebar_label: "Tool Gateway"
sidebar_position: 2
---
# Nous Tool Gateway
**One subscription. Every tool built in.**
The Tool Gateway is included with every paid [Nous Portal](https://portal.nousresearch.com) subscription. It routes Hermes' tool calls — web search, image generation, text-to-speech, and cloud browser automation — through infrastructure Nous already runs, so you don't have to sign up with Firecrawl, FAL, OpenAI, Browser Use, or anyone else just to make your agent useful.
<div style={{display: 'flex', gap: '1rem', flexWrap: 'wrap', margin: '1.5rem 0'}}>
<a href="https://portal.nousresearch.com/manage-subscription" style={{background: 'var(--ifm-color-primary)', color: 'white', padding: '0.75rem 1.5rem', borderRadius: '6px', textDecoration: 'none', fontWeight: 'bold'}}>Start or manage subscription →</a>
</div>
## What's included
| | Tool | What you get |
|---|---|---|
| 🔍 | **Web search & extract** | Agent-grade web search and full-page extraction via Firecrawl. No rate limits to worry about — the gateway handles scaling. |
| 🎨 | **Image generation** | Nine models under one endpoint: **FLUX 2 Klein 9B**, **FLUX 2 Pro**, **Z-Image Turbo**, **Nano Banana Pro** (Gemini 3 Pro Image), **GPT Image 1.5**, **GPT Image 2**, **Ideogram V3**, **Recraft V4 Pro**, **Qwen Image**. Pick per-generation with a flag, or let Hermes default to FLUX 2 Klein. |
| 🔊 | **Text-to-speech** | OpenAI TTS voices wired into the `text_to_speech` tool. Drop voice notes into Telegram, generate audio for pipelines, narrate anything. |
| 🌐 | **Cloud browser automation** | Headless Chromium sessions via Browser Use. `browser_navigate`, `browser_click`, `browser_type`, `browser_vision` — all the agent-driving primitives, no Browserbase account required. |
All four are pay-as-you-use billed against your Nous subscription. Use any combination — run the gateway for web and images while keeping your own ElevenLabs key for TTS, or route everything through Nous.
## Why it's here
Building an agent that can actually *do things* means stitching together 5+ API subscriptions — each with their own signup, rate limits, billing, and quirks. The gateway collapses that into one account:
- **One bill.** Pay Nous; we handle the rest.
- **One signup.** No Firecrawl, FAL, Browser Use, or OpenAI audio accounts to manage.
- **One key.** Your Nous Portal OAuth covers every tool.
- **Same quality.** Same backends the direct-key route uses — just fronted by us.
Bring your own keys anytime — per-tool, whenever you want to. The gateway isn't a lock-in, it's a shortcut.
## Get started
There are three ways in — pick whichever fits where you are:
```bash
hermes setup --portal # Fresh install: Nous OAuth + set Nous as provider + turn on the Tool Gateway in one go
```
```bash
hermes model # Switch your inference provider to Nous Portal — Hermes then offers to turn on the gateway for all tools
```
```bash
hermes tools # Enable the gateway per-tool — pick "Nous Subscription" for any tool you want
```
`hermes setup --portal` and `hermes model` are the all-at-once paths: log in once, optionally flip every tool to the gateway. `hermes tools` is the à la carte path — turn on just the tools you want, one at a time.
**You don't have to log in first.** With `hermes tools`, the Nous-managed backends (Web search, Image, Video, TTS, Browser) are always listed, even if you've never signed into Nous Portal. Select one and Hermes runs the Portal login right there if you aren't already authenticated — no need to run `hermes model` beforehand. If your Nous OAuth is already active, selecting the backend enables it immediately with no extra prompt. This path only logs you in and turns on the one tool you picked — it does **not** switch your inference provider, and it does **not** prompt you to enable the gateway for every other tool.
Check what's active at any time:
```bash
hermes portal info # Portal auth + Tool Gateway routing summary
hermes portal tools # Gateway catalog with current routing per tool
hermes status # Full system status (Tool Gateway is one section)
```
`hermes portal info` shows a section like:
```
◆ Nous Tool Gateway
Nous Portal ✓ managed tools available
Web tools ✓ active via Nous subscription
Image gen ✓ active via Nous subscription
TTS ✓ active via Nous subscription
Browser ○ active via Browser Use key
```
Tools marked "active via Nous subscription" are going through the gateway. Anything else is using your own keys.
## Eligibility
The Tool Gateway is a **paid-subscription** feature. Free-tier Nous accounts can use Portal for inference but don't include managed tools — [upgrade your plan](https://portal.nousresearch.com/manage-subscription) to unlock the gateway.
Some accounts are also entitled to a **free tool pool** — a small managed-tool allowance that covers gateway tool calls without a paid subscription. When a free pool is available, the gateway surfaces it and shows a setup prompt on first use, so you can opt in and start using managed tools right away.
## The enablement checklist
Picking a Nous model (`hermes model`) offers a per-tool checklist of gateway backends. Its behavior respects your existing setup:
- Tools you've explicitly pointed at another backend (e.g. `web.backend: searxng`, `browser.cloud_provider: camofox`) are **never offered** — your selection can't be accidentally overwritten.
- Tools configured via environment variables alone (e.g. `SEARXNG_URL`, `CAMOFOX_URL`) are offered **unchecked**, labeled to keep your own backend.
- Only genuinely unconfigured tools come pre-checked.
- Declines stick: if you submit the checklist with a tool unchecked, it won't be pre-checked on future Nous model swaps (stored in `tool_gateway_declined_tools` in `config.yaml`; checking it later clears the decline).
## Mix and match
The gateway is per-tool. Turn it on for just what you want:
- **All tools through Nous** — easiest; one subscription, done.
- **Gateway for web + images, bring your own TTS** — keep your ElevenLabs voice, let Nous handle the rest.
- **Gateway only for things you don't have keys for** — "I already pay for Browserbase, but I don't want a Firecrawl account" works fine.
Switch any tool at any time via:
```bash
hermes tools # Interactive picker for each tool category
```
Select the tool, pick **Nous Subscription** as the provider (or any direct provider you prefer). No config editing required. If you aren't logged into Nous Portal yet, picking **Nous Subscription** kicks off the Portal login inline — you don't need to authenticate through `hermes model` first.
## Using individual image models
Image generation defaults to FLUX 2 Klein 9B for speed. Override per-call by passing the model ID to the `image_generate` tool:
| Model | ID | Best for |
|---|---|---|
| FLUX 2 Klein 9B | `fal-ai/flux-2/klein/9b` | Fast, good default |
| FLUX 2 Pro | `fal-ai/flux-2-pro` | Higher fidelity FLUX |
| Z-Image Turbo | `fal-ai/z-image/turbo` | Stylized, fast |
| Nano Banana Pro | `fal-ai/nano-banana-pro` | Google Gemini 3 Pro Image |
| GPT Image 1.5 | `fal-ai/gpt-image-1.5` | OpenAI image gen, text+image |
| GPT Image 2 | `fal-ai/gpt-image-2` | OpenAI latest |
| Ideogram V3 | `fal-ai/ideogram/v3` | Strong prompt adherence + typography |
| Recraft V4 Pro | `fal-ai/recraft/v4/pro/text-to-image` | Vector-style, graphic design |
| Qwen Image | `fal-ai/qwen-image` | Alibaba multimodal |
The set evolves — `hermes tools` → Image Generation shows the current live list.
---
## Configuration reference
Most users never need to touch this — `hermes model` and `hermes tools` cover every workflow interactively. This section is for writing config.yaml directly or scripting setups.
### One selection key per tool category
Each tool category has a single provider-selection key, written by the `hermes tools` picker (or the desktop GUI). Picking the **Nous Subscription** row stores the value `nous`, which routes that category through the managed Tool Gateway. Picking a BYOK row stores the vendor name (`fal`, `openai`, `firecrawl`, `browser-use`, ...), which goes direct with your own credentials:
```yaml
web:
backend: nous # web search/extract via the Tool Gateway
image_gen:
provider: nous # image generation via the Tool Gateway
tts:
provider: nous # TTS via the Tool Gateway
stt:
provider: nous # speech-to-text via the Tool Gateway
browser:
cloud_provider: nous # cloud browser via the Tool Gateway
```
The runtime **always uses the stored selection** — credential presence never selects or reroutes a category. A `FAL_KEY` sitting in `.env` is ignored while `image_gen.provider: nous`; conversely, `image_gen.provider: fal` with no `FAL_KEY` set produces a clear error instead of silently falling back to the gateway:
```
image_gen is configured to use fal (set via hermes tools), but FAL_KEY is not set. Run 'hermes tools' to change it.
```
Categories you have **never configured** (no selection key ever written) autodetect from available credentials, same as before. But once a selection exists, adding a key to `.env` does not change the route — only `hermes tools` (or editing the selection key) does.
### Switching back to your own keys
```bash
hermes tools # pick the tool → choose a direct provider (e.g. Firecrawl)
```
Or set the selection key directly:
```yaml
web:
backend: firecrawl # Hermes now uses FIRECRAWL_API_KEY from .env
```
### Legacy `use_gateway` flag (deprecated)
Older Hermes versions used a per-tool `use_gateway: true` boolean to route through the gateway. That flag is **legacy**: it is never written anymore, and the `hermes tools` picker removes it from a category's config when it rewrites the selection. Old configs that still contain `use_gateway: true` are interpreted at read time as the `nous` selection, so existing setups keep working. Don't set `use_gateway` in new configs — select the provider in `hermes tools` instead.
### Self-hosted gateway (advanced)
Running your own Nous-compatible gateway? Override endpoints in `~/.hermes/.env`:
```bash
TOOL_GATEWAY_DOMAIN=your-domain.example.com
TOOL_GATEWAY_SCHEME=https
TOOL_GATEWAY_USER_TOKEN=your-token # normally auto-populated from Portal login
FIRECRAWL_GATEWAY_URL=https://... # override one endpoint specifically
```
These knobs exist for custom infrastructure setups (enterprise deployments, dev environments). Regular subscribers never set them.
## FAQ
### Does it work with Telegram / Discord / the other messaging gateways?
Yes. Tool Gateway operates at the tool-execution layer, not the CLI. Every interface that can call a tool — CLI, Telegram, Discord, Slack, IRC, Teams, the API server, anything — benefits from it transparently.
### What happens if my subscription expires?
Tools routed through the gateway stop working until you renew or swap in direct API keys via `hermes tools`. Hermes shows a clear error pointing at the portal.
### Can I see usage or costs per tool?
Yes — the [Nous Portal dashboard](https://portal.nousresearch.com) breaks usage down by tool so you can see what's driving your bill.
### Is Modal (serverless terminal) included?
Modal is available as an **optional add-on** through the Nous subscription, not part of the default Tool Gateway bundle. Configure it via `hermes setup terminal` or directly in `config.yaml` when you want a remote sandbox for shell execution.
### Do I need to delete my existing API keys when I enable the gateway?
No — keep them in `.env`. While a tool's selection is **Nous Subscription**, direct keys for that tool are simply ignored. Pick the direct provider again in `hermes tools` and your keys become the source again. The gateway isn't a lock-in.
website/docs/user-guide/features/tool-search.md
+213
View File
@@ -0,0 +1,213 @@
---
title: Tool Search
sidebar_position: 95
---
# Tool Search
When you have many MCP servers or non-core plugin tools attached to a
session, their JSON schemas can consume a substantial fraction of the
context window on every turn — even when only a few of them are relevant
to what the user actually asked for.
**Tool Search** is Hermes' opt-in progressive-disclosure layer for that
problem. When activated, MCP and plugin tools are replaced in the
model-visible tools array by three bridge tools, and the model loads each
specific tool's schema on demand.
:::info Built-in Hermes tools never defer
The tools that make up Hermes' core capability set (`terminal`,
`read_file`, `write_file`, `patch`, `search_files`, `todo`, `memory`,
`browser_*`, `web_search`, `web_extract`, `clarify`, `execute_code`,
`delegate_task`, `session_search`, and the rest of
`_HERMES_CORE_TOOLS`) are *always* loaded directly. Only MCP tools and
non-core plugin tools are eligible for deferral.
:::
## How it works
When Tool Search activates for a turn, the model sees three new tools in
place of the deferred ones:
```
tool_search(queries, limit?) — search the deferred-tool catalog (one or more queries)
tool_describe(names) — load the full schemas for one or more tools
tool_call(name, arguments) — invoke a deferred tool
```
A typical interaction looks like:
```
Model: tool_search(["create a github issue", "send a slack message"])
→ { results: [ { query: "create a github issue",
matches: ["mcp_github_create_issue", ...] },
{ query: "send a slack message",
matches: ["mcp_slack_post_message", ...] } ],
tools: { mcp_github_create_issue: { description: "...",
required: ["title"], ... },
mcp_slack_post_message: { ... } } }
Model: tool_describe(["mcp_github_create_issue", "mcp_slack_post_message"])
→ { tools: { mcp_github_create_issue: { parameters: { ... } },
mcp_slack_post_message: { parameters: { ... } } } }
Model: tool_call("mcp_github_create_issue", { title: "...", body: "..." })
→ { ok: true, issue_number: 42 }
```
Each query in a `tool_search` call is searched independently against the
same catalog (`limit` applies per query); the per-query groups carry tool
names only, while the shared `tools` map holds each matched tool's
description and required parameter names once. Queries are stemmed, so
"issues" finds `create_issue`. Each query group that returns no matches
includes an `available_sources` summary of the connected servers so a lexical
miss is not mistaken for a missing capability.
`tool_describe` resolves every requested name in one call; unknown names
are reported in `not_found` without failing the rest of the batch.
When the model invokes `tool_call`, Hermes **unwraps the bridge** and
dispatches the underlying tool exactly as if the model had called it
directly. Pre-tool-call hooks, guardrails, approval prompts, and
post-tool-call hooks all run against the real tool name — not against
`tool_call`. The activity feed in the CLI and gateway also unwraps so you
see the underlying tool, not the bridge.
## When does it activate?
Tool Search uses **tiered disclosure**: the presence of *any* deferrable
(MCP/plugin) tool activates the bridge; what scales with catalog size is
how much of the catalog stays visible, not whether schemas defer.
| Tier | Condition | What the model sees |
| --- | --- | --- |
| **0** | No MCP/plugin tools | Every tool eager, no bridge. Pass-through. |
| **1** | Deferred catalog's listing fits the budget | Bridge + a skills-style manifest of every deferred tool (name + short description, degrading to names-only when over budget). Degradation is **per server**: when one oversized server (Cloudflare) is attached alongside small ones (Linear), the small servers keep their per-tool listings and only the oversized server collapses to a summary line. |
| **2** | Per-tool listing exceeds the budget even names-only for every server (e.g. Cloudflare's flat API surface alone: ~3,300 tools whose names are ~32K tokens) | Bare bridge + a one-line-per-server summary (server name + tool count), so the model knows which domains are reachable; individual tools are discoverable only through `tool_search`. |
The listing budget is `min(threshold_pct% of context, listing_max_tokens)`.
The decision is re-evaluated every time the tools array is built, so
adding or removing MCP servers mid-session moves the session between
tiers on the next assembly.
## Configuration
```yaml
tools:
tool_search:
enabled: auto # auto (default), on, or off
threshold_pct: 5 # listing budget as a percentage of context
search_default_limit: 5
max_search_limit: 25
listing: auto # embed a grouped name+description catalog manifest
listing_max_tokens: 4000
```
| Key | Default | Meaning |
| --- | --- | --- |
| `enabled` | `auto` | `auto`/`on` activate whenever at least one deferrable tool exists; `off` disables entirely (everything stays eager). `auto` is currently an alias of `on` — it is reserved for a future mode that inlines schemas when they fit the context and defers only when they don't. Pin `on` or `off` if you want today's behavior guaranteed across upgrades. |
| `threshold_pct` | `5` | Listing budget as a percentage of the active model's context length. Range 0–100. |
| `search_default_limit` | `5` | Hits returned per query when the model calls `tool_search` without a `limit`. |
| `max_search_limit` | `25` | Hard upper bound the model can request via `limit` (per query). Range 1–50. |
| `listing` | `auto` | Embed a skills-style manifest of every deferred tool (name + first sentence of its description, ≤60 chars, grouped by MCP server) in the `tool_search` bridge description. `auto` includes it when it fits the budget (falling back to names-only, then to the tier-2 server summary); `on`/`off` force either way. |
| `listing_max_tokens` | `4000` | Absolute cap on the embedded listing, regardless of context size. Range 200–60000. Large catalogs degrade to names-only or per-server summaries, keeping full schemas available through search. |
Per-call array caps are internal safety bounds, not configuration. Over-cap
calls return an error so the model can retry with a smaller batch.
### Why the listing exists
Without it, deferred capabilities are *invisible* — live benchmarking showed
models substituting visible core tools (running `gh` in the terminal instead
of searching for the deferred GitHub tool) or declaring a capability
nonexistent instead of calling `tool_search`. The listing applies the skills
pattern to tools: every capability stays discoverable by name at all times,
while full parameter schemas remain deferred. If the model sees the exact
tool name in the listing, it can skip `tool_search` and go straight to
`tool_describe`, saving a round trip.
You can also flip the legacy boolean shape:
```yaml
tools:
tool_search: true # equivalent to {enabled: auto}
```
## When NOT to use it
Tool Search trades a fixed per-turn token cost (the three bridge tool
schemas plus the catalog listing) and at least one extra round trip on
cold tools (describe → call) for the savings on the deferred schemas.
At tier 1 the listing keeps every capability visible, so the discovery
round trip usually disappears — the model goes straight to
`tool_describe`. Live benchmarking showed the listing mode matching
eager loading's task success while costing less than the bare bridge.
If you want the old always-eager behavior for a small toolset, set
`enabled: off`.
## Trade-offs that don't go away
These come from the prompt-cache integrity invariant — they are inherent
to any progressive-disclosure design, not specific to this implementation:
- **One extra round trip on cold tools.** The first time the model needs
a deferred tool, it spends one or two extra model calls to find and
load the schema. The token savings on the static side are real, but a
portion is paid back at runtime.
- **No cache benefit on deferred schemas.** A loaded `tool_describe`
result enters the conversation history (so it does get cached on
subsequent turns) but it never benefits from the system-prompt cache
prefix.
- **No provider-native validation for deferred schemas.** `tool_describe`
lets the model read a deferred tool's schema, but the provider still sees
only the generic `tool_call.arguments` object. Hermes therefore coerces and
validates the underlying arguments locally before dispatch; the concrete
tool or MCP server remains responsible for schemas Hermes cannot safely
validate, such as malformed schemas or external references.
- **Model-quality dependence.** Tool Search assumes the model can write a
reasonable search query for the tool it wants. Smaller models do this
less well; the published Anthropic numbers (49% → 74% on Opus 4 with
vs. without tool search) show the upside but also that ~26 points of
accuracy is still retrieval failure.
- **Toolset edits invalidate cache.** Adding or removing a tool mid-
session changes the bridge tools' descriptions (which include the
count of deferred tools) and the catalog, so the prompt cache is
invalidated. This is the same trade-off as any toolset edit.
## Implementation details
- **Retrieval:** BM25 over tokenized tool name, source name (the MCP
server or plugin toolset the tool belongs to, so searching `"linear"`
finds that server's tools even when a tool's own name doesn't carry
the service), description, and parameter names, with Snowball
stemming (English) applied to both the index and the query so
morphological variants match ("issues" finds `create_issue`). Falls
back to a literal substring match on the tool name when no query
token matches any document (e.g. searching `"hub"` where the token is
`github`).
- **Parallel execution unwraps the bridge.** The batch planner decides
concurrency on the *underlying* tool of a `tool_call`, not on the
literal bridge name — so an MCP server opted in via
`supports_parallel_tool_calls: true` keeps its concurrency when its
tools are called through the bridge, and `tool_search` /
`tool_describe` lookups batch concurrently like any read-only tool.
- **Catalog is stateless across turns.** It rebuilds from the current
tool-defs list every assembly — no session-keyed `Map`. This avoids
the class of bug where a stored catalog drifts out of sync with the
live tool registry.
- **The catalog is scoped to the session's toolsets.** `tool_search`,
`tool_describe`, and `tool_call` only ever see and invoke tools the
session was actually granted. A subagent, kanban worker, or gateway
session restricted to a subset of toolsets cannot use the bridge to
discover or call a tool outside that subset — the deferred catalog is
the deferrable slice of the session's own enabled/disabled toolsets,
not the whole process registry.
- **No JS sandbox.** Hermes uses the simpler "structured tools" mode
(search / describe / call as plain functions). The JS-sandbox "code
mode" some other implementations offer is a large surface area; we
skip it.
## See also
- `tools/tool_search.py` — the implementation
- `tests/tools/test_tool_search.py` — the regression suite
- The `openclaw-tool-search-report` PDF in the original implementation
PR for the research that shaped the design
website/docs/user-guide/features/tools.md
+240
View File
@@ -0,0 +1,240 @@
---
sidebar_position: 1
title: "Tools & Toolsets"
description: "Overview of Hermes Agent's tools — what's available, how toolsets work, and terminal backends"
---
# Tools & Toolsets
Tools are functions that extend the agent's capabilities. They're organized into logical **toolsets** that can be enabled or disabled per platform.
## Available Tools
Hermes ships with a broad built-in tool registry covering web search, browser automation, terminal execution, file editing, memory, delegation, scheduled tasks, Home Assistant, and more.
:::note
**Honcho cross-session memory** is available as a memory provider plugin (`plugins/memory/honcho/`), not as a built-in toolset. See [Plugins](./plugins.md) for installation.
:::
High-level categories:
| Category | Examples | Description |
|----------|----------|-------------|
| **Web** | `web_search`, `web_extract` | Search the web and extract page content. |
| **X Search** | `x_search` | Search X (Twitter) posts and threads via xAI's built-in `x_search` Responses tool — gated on xAI credentials (SuperGrok OAuth or `XAI_API_KEY`); off by default, opt in via `hermes tools` → 🐦 X (Twitter) Search. |
| **Terminal & Files** | `terminal`, `process`, `read_file`, `patch` | Execute commands and manipulate files. |
| **Browser** | `browser_navigate`, `browser_snapshot`, `browser_vision` | Interactive browser automation with text and vision support. |
| **Media** | `vision_analyze`, `image_generate`, `text_to_speech` | Multimodal analysis and generation. |
| **Agent orchestration** | `todo`, `clarify`, `execute_code`, `delegate_task` | Planning, clarification, code execution, and subagent delegation. |
| **Memory & recall** | `memory`, `session_search` | Persistent memory and session search. |
| **Automation** | `cronjob` | Scheduled tasks with create/list/update/pause/resume/run/remove actions. Outbound delivery is handled by cron's own delivery, the `hermes send` CLI, and the gateway notifier — not by an agent-callable tool. |
| **Integrations** | `ha_*`, MCP server tools | Home Assistant, MCP, and other integrations. |
For the authoritative code-derived registry, see [Built-in Tools Reference](/reference/tools-reference) and [Toolsets Reference](/reference/toolsets-reference).
:::tip Nous Tool Gateway
Paid [Nous Portal](https://portal.nousresearch.com) subscribers can use web search, image generation, TTS, and browser automation through the **[Tool Gateway](tool-gateway.md)** — no separate API keys needed. Run `hermes model` to enable it, or configure individual tools with `hermes tools`.
:::
## Using Toolsets
```bash
# Use specific toolsets
hermes chat --toolsets "web,terminal"
# See all available tools
hermes tools
# Configure tools per platform (interactive)
hermes tools
```
Common toolsets include `web`, `search`, `terminal`, `file`, `browser`, `vision`, `image_gen`, `skills`, `tts`, `todo`, `memory`, `session_search`, `cronjob`, `code_execution`, `delegation`, `clarify`, `homeassistant`, `messaging`, `spotify`, `discord`, `discord_admin`, `debugging`, and `safe`.
See [Toolsets Reference](/reference/toolsets-reference) for the full set, including platform presets such as `hermes-cli`, `hermes-telegram`, and dynamic MCP toolsets like `mcp-<server>`.
## Tool result annotations
A few tool behaviors are worth knowing when you read agent transcripts:
- **Signal deaths are explained.** When a terminal command is killed by a signal, the result carries a human-readable note instead of a bare numeric code — e.g. exit `-9`/`137` becomes "terminated by signal 9: SIGKILL — often the kernel OOM killer on memory exhaustion, or an explicit kill -9", and segfaults, aborts, SIGTERM, broken pipes, and CPU/file-size limits are labeled the same way. Negative codes (subprocess semantics) are stated definitively; the shell's `128+signum` convention is hedged with "usually" since an application can legitimately exit with those codes.
- **UTF-16 text files are transcoded, not refused.** `read_file` detects UTF-16 (BOM or byte-pattern heuristic, either endianness — common for Windows Notepad files and PowerShell `>` redirects) and transcodes it to UTF-8 for display instead of flagging the file as binary. The result includes a hint disclosing the conversion; edits via `patch`/`write_file` re-encode as UTF-8. Files over 10 MB and genuinely binary files still get the binary-file refusal.
## Terminal Backends
The terminal tool can execute commands in different environments:
| Backend | Description | Use Case |
|---------|-------------|----------|
| `local` | Run on your machine (default) | Development, trusted tasks |
| `docker` | Isolated containers | Security, reproducibility |
| `ssh` | Remote server | Sandboxing, keep agent away from its own code |
| `singularity` | HPC containers | Cluster computing, rootless |
| `modal` | Cloud execution | Serverless, scale |
| `daytona` | Cloud sandbox workspace | Persistent remote dev environments |
| `vercel_sandbox` | Vercel Sandbox cloud microVM | Cloud execution with snapshot-backed filesystem persistence |
### Configuration
```yaml
# In ~/.hermes/config.yaml
terminal:
backend: local # or: docker, ssh, singularity, modal, daytona, vercel_sandbox
cwd: "." # Working directory
timeout: 180 # Command timeout in seconds
```
### Shell startup files and non-interactive commands
Agent terminal calls run your shell **non-interactively** — there is no TTY and no human at the prompt. Heavy or interactive shell initialisation that you never notice in a normal terminal can break or badly slow every command the agent runs:
- **Slow init (`nvm`, version managers, network-touching prompts):** the classic `nvm.sh` sourcing adds noticeable latency to *every* shell start, and the agent starts many shells. Multi-second rc files turn a quick `git status` into a timeout risk.
- **TTY-expecting blocks:** anything in `.bashrc`/`.zshrc` that prompts, runs `tmux`/`screen` attach, calls `read`, or prints a menu will hang a non-interactive shell — the command appears to run forever and then times out.
- **Unconditional output:** rc files that `echo` banners pollute every command's output the agent has to parse.
The fix is the standard guard most distros already ship at the top of `.bashrc` — return early when the shell is non-interactive, and keep anything heavy or interactive below it:
```bash
# ~/.bashrc — keep this guard near the top
case $- in
*i*) ;; # interactive: continue
*) return;; # non-interactive: stop here
esac
# heavy/interactive init goes BELOW the guard
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
```
Zsh users: put login-only setup in `.zprofile` and interactive-only setup in `.zshrc`; keep `.zshenv` minimal, since it runs for every shell including non-interactive ones. If the agent genuinely needs a tool that only your rc file puts on `PATH`, export the `PATH` change *above* the guard (path exports are cheap) or symlink the binary into `~/.local/bin`.
If agent terminal commands hang or time out immediately after working in your own terminal, your shell init is the first suspect.
### Docker Backend
```yaml
terminal:
backend: docker
docker_image: python:3.11-slim
```
**One persistent container, shared across the whole process.** Hermes starts a single long-lived container on first use (`docker run -d ... sleep infinity`) and routes every terminal, file, and `execute_code` call through `docker exec` into that same container. Working-directory changes, installed packages, environment tweaks, and files written to `/workspace` all carry over from one tool call to the next, across `/new`, `/reset`, and `delegate_task` subagents, for the lifetime of the Hermes process. The container is stopped and removed on shutdown.
This means the Docker backend behaves like a persistent sandbox VM, not a fresh container per command. If you `pip install foo` once, it's there for the rest of the session. If you `cd /workspace/project`, subsequent `ls` calls see that directory. See [Configuration → Docker Backend](../configuration.md#docker-backend) for the full lifecycle details and the `container_persistent` flag that controls whether `/workspace` and `/root` survive across Hermes restarts.
### SSH Backend
Recommended for security — agent can't modify its own code:
```yaml
terminal:
backend: ssh
```
```bash
# Set credentials in ~/.hermes/.env
TERMINAL_SSH_HOST=my-server.example.com
TERMINAL_SSH_USER=myuser
TERMINAL_SSH_KEY=~/.ssh/id_rsa
```
### Singularity/Apptainer
```bash
# Pre-build SIF for parallel workers
apptainer build ~/python.sif docker://python:3.11-slim
# Configure
hermes config set terminal.backend singularity
hermes config set terminal.singularity_image ~/python.sif
```
### Modal (Serverless Cloud)
```bash
uv pip install modal
modal setup
hermes config set terminal.backend modal
```
### Vercel Sandbox
```bash
pip install 'hermes-agent[vercel]'
hermes config set terminal.backend vercel_sandbox
hermes config set terminal.vercel_runtime node24
```
Authenticate with all three of `VERCEL_TOKEN`, `VERCEL_PROJECT_ID`, and `VERCEL_TEAM_ID`. This access-token setup is the supported path for deployments and normal long-running Hermes processes on Render, Railway, Docker, and similar hosts. Supported runtimes are `node24`, `node22`, and `python3.13`; Hermes defaults to `/vercel/sandbox` as the remote workspace root.
For one-off local development, Hermes also accepts short-lived Vercel OIDC tokens:
```bash
VERCEL_OIDC_TOKEN="$(vc project token <project-name>)" hermes chat
```
From a linked Vercel project directory:
```bash
VERCEL_OIDC_TOKEN="$(vc project token)" hermes chat
```
With `container_persistent: true`, Hermes uses Vercel snapshots to preserve filesystem state across sandbox recreation for the same task. This can include Hermes-synced credentials, skills, and cache files inside the sandbox. Snapshots do not preserve live processes, PID space, or the same live sandbox identity.
Background terminal commands use Hermes' generic non-local process flow: spawn, poll, wait, log, and kill work through the normal process tool while the sandbox is alive, but Hermes does not provide native Vercel detached-process recovery after cleanup or restart.
Leave `container_disk` unset or at the shared default `51200`; custom disk sizing is unsupported for Vercel Sandbox and will fail diagnostics/backend creation.
### Container Resources
Configure CPU, memory, disk, and persistence for all container backends:
```yaml
terminal:
backend: docker # or singularity, modal, daytona, vercel_sandbox
container_cpu: 1 # CPU cores (default: 1)
container_memory: 5120 # Memory in MB (default: 5GB)
container_disk: 51200 # Disk in MB (default: 50GB)
container_persistent: true # Persist filesystem across sessions (default: true)
```
When `container_persistent: true`, installed packages, files, and config survive across sessions.
### Container Security
All container backends run with security hardening:
- Read-only root filesystem (Docker)
- All Linux capabilities dropped
- No privilege escalation
- PID limits (256 processes)
- Full namespace isolation
- Persistent workspace via volumes, not writable root layer
Docker can optionally receive an explicit env allowlist via `terminal.docker_forward_env`, but forwarded variables are visible to commands inside the container and should be treated as exposed to that session.
## Background Process Management
Start background processes and manage them:
```python
terminal(command="pytest -v tests/", background=true)
# Returns: {"session_id": "proc_abc123", "pid": 12345}
# Then manage with the process tool:
process(action="list") # Show all running processes
process(action="poll", session_id="proc_abc123") # Check status
process(action="wait", session_id="proc_abc123") # Block until done
process(action="log", session_id="proc_abc123") # Full output
process(action="kill", session_id="proc_abc123") # Terminate
process(action="write", session_id="proc_abc123", data="y") # Send input
```
PTY mode (`pty=true`) enables interactive CLI tools like Codex and Claude Code.
## Sudo Support
If a command needs sudo, you'll be prompted for your password (cached for the session). Or set `SUDO_PASSWORD` in `~/.hermes/.env`.
:::warning
On messaging platforms, if sudo fails, the output includes a tip to add `SUDO_PASSWORD` to `~/.hermes/.env`.
:::
website/docs/user-guide/features/tts.md
+735
View File
@@ -0,0 +1,735 @@
---
sidebar_position: 9
title: "Voice & TTS"
description: "Text-to-speech and voice message transcription across all platforms"
---
# Voice & TTS
Hermes Agent supports both text-to-speech output and voice message transcription across all messaging platforms.
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, OpenAI TTS is available through the **[Tool Gateway](tool-gateway.md)** without a separate OpenAI API key. New installs can run `hermes setup --portal` to log in and turn on every gateway tool at once; existing installs can pick **Nous Subscription** for just TTS via `hermes model` or `hermes tools`.
:::
## Text-to-Speech
Convert text to speech with eleven providers:
| Provider | Quality | Cost | API Key |
|----------|---------|------|---------|
| **Edge TTS** (default) | Good | Free | None needed |
| **ElevenLabs** | Excellent | Paid | `ELEVENLABS_API_KEY` |
| **OpenAI TTS** | Good | Paid | `VOICE_TOOLS_OPENAI_KEY` |
| **MiniMax TTS** | Excellent | Paid | `MINIMAX_API_KEY` or `MINIMAX_CN_API_KEY` |
| **Mistral (Voxtral TTS)** | Excellent | Paid | `MISTRAL_API_KEY` |
| **Google Gemini TTS** | Excellent | Free tier | `GEMINI_API_KEY` |
| **xAI TTS** | Excellent | Paid | `XAI_API_KEY` |
| **DeepInfra TTS** | Good | Paid | `DEEPINFRA_API_KEY` |
| **NeuTTS** | Good | Free (local) | None needed |
| **KittenTTS** | Good | Free (local) | None needed |
| **Piper** | Good | Free (local) | None needed |
### Platform Delivery
| Platform | Delivery | Format |
|----------|----------|--------|
| Telegram | Voice bubble (plays inline) | Opus `.ogg` |
| Discord | Voice bubble (Opus/OGG), falls back to file attachment | Opus/MP3 |
| WhatsApp | Audio file attachment | MP3 |
| CLI | Saved to `~/.hermes/audio_cache/` | MP3 |
### Configuration
```yaml
# In ~/.hermes/config.yaml
tts:
provider: "edge" # "edge" | "elevenlabs" | "openai" | "minimax" | "mistral" | "gemini" | "xai" | "deepinfra" | "neutts" | "kittentts" | "piper" — or "nous" for the managed Tool Gateway (written when you pick Nous Subscription in `hermes tools`)
speed: 1.0 # Global speed multiplier (provider-specific settings override this)
edge:
voice: "en-US-AriaNeural" # 322 voices, 74 languages
speed: 1.0 # Converted to rate percentage (+/-%)
elevenlabs:
voice_id: "pNInz6obpgDQGcFmaJgB" # Adam
model_id: "eleven_multilingual_v2"
openai:
model: "gpt-4o-mini-tts"
voice: "alloy" # alloy, echo, fable, onyx, nova, shimmer
base_url: "https://api.openai.com/v1" # Override for OpenAI-compatible TTS endpoints
speed: 1.0 # 0.25 - 4.0
# language: "es" # Sent as lang_code — only for OpenAI-compatible endpoints that support it (e.g. Kokoro)
minimax:
region: "global" # "global" or "cn"; see selection rules below
model: "speech-02-hd" # speech-02-hd (default), speech-02-turbo
voice_id: "English_expressive_narrator" # See https://platform.minimax.io/faq/system-voice-id
speed: 1 # 0.5 - 2.0
vol: 1 # 0 - 10
pitch: 0 # -12 - 12
# base_url: "https://tts.example/v1/t2a_v2" # Optional endpoint override for the selected region
mistral:
model: "voxtral-mini-tts-2603"
voice_id: "c69964a6-ab8b-4f8a-9465-ec0925096ec8" # Paul - Neutral (default)
gemini:
model: "gemini-2.5-flash-preview-tts" # or gemini-3.1-flash-tts-preview
voice: "Kore" # 30 prebuilt voices: Zephyr, Puck, Kore, Enceladus, Gacrux, etc.
audio_tags: false # Enable hidden Gemini 3.1 TTS audio-tag insertion
persona_prompt_file: "" # Optional Markdown/text file with Gemini voice direction
xai:
voice_id: "eve" # or a custom voice ID — see docs below
language: "en" # BCP-47 code (e.g. "en", "pt-BR") or "auto" for detection
speed: 1.0 # 0.7–1.5, playback speed (default: 1.0)
auto_speech_tags: false # insert expressive audio tags via LLM rewrite
text_normalization: false # normalize numbers/abbreviations/symbols to spoken form
optimize_streaming_latency: 0 # 0–2, trades quality for lower latency (default: 0)
sample_rate: 24000 # 22050 / 24000 (default) / 44100 / 48000
bit_rate: 128000 # MP3 bitrate; only applies when codec=mp3
# base_url: "https://api.x.ai/v1" # Override via XAI_BASE_URL env var
neutts:
ref_audio: ''
ref_text: ''
model: neuphonic/neutts-air-q4-gguf
device: cpu
kittentts:
model: KittenML/kitten-tts-nano-0.8-int8 # 25MB int8; also: kitten-tts-micro-0.8 (41MB), kitten-tts-mini-0.8 (80MB)
voice: Jasper # Jasper, Bella, Luna, Bruno, Rosie, Hugo, Kiki, Leo
speed: 1.0 # 0.5 - 2.0
clean_text: true # Expand numbers, currencies, units
piper:
voice: en_US-lessac-medium # voice name (auto-downloaded) OR absolute path to .onnx
# voices_dir: '' # default: ~/.hermes/cache/piper-voices/
# use_cuda: false # requires onnxruntime-gpu
# length_scale: 1.0 # 2.0 = twice as slow
# noise_scale: 0.667
# noise_w_scale: 0.8
# volume: 1.0 # 0.5 = half as loud
# normalize_audio: true
```
MiniMax TTS selects its region, endpoint, and credential together:
- `region: "global"` uses `https://api.minimax.io/v1/t2a_v2` with `MINIMAX_API_KEY`.
- `region: "cn"` uses `https://api.minimaxi.com/v1/t2a_v2` with `MINIMAX_CN_API_KEY`.
- If `region` is omitted, `MINIMAX_API_KEY` keeps precedence for backward compatibility. If only `MINIMAX_CN_API_KEY` is configured, Hermes selects `cn`.
- An explicitly selected region must have its matching credential. Hermes never borrows the other region's key. A `base_url` override does not change the selected credential, and an override pointing at the other region's official endpoint is rejected.
**Speed control**: The global `tts.speed` value applies to all providers by default. Each provider can override it with its own `speed` setting (e.g., `tts.openai.speed: 1.5`). Provider-specific speed takes precedence over the global value. Default is `1.0` (normal speed).
### Gemini Persona Prompts
Gemini TTS can follow natural-language performance direction. Set `tts.gemini.persona_prompt_file` to a local Markdown or text file that describes the voice persona. The file can include Gemini-style sections such as `AUDIO PROFILE`, `SCENE`, `DIRECTOR'S NOTES`, `SAMPLE CONTEXT`, and `TRANSCRIPT`.
If the file contains `{transcript}` or `{{ transcript }}`, Hermes replaces that placeholder with the live TTS text. Otherwise, Hermes appends a labeled `TRANSCRIPT` section automatically. The persona prompt stays local and is not shown in the chat reply.
```yaml
tts:
provider: gemini
gemini:
voice: Algieba
persona_prompt_file: ~/.hermes/tts/butler-voice.md
```
### Audio Tags (Gemini, xAI)
Google's Gemini 3.1 Flash TTS and xAI's Grok TTS support freeform square-bracket audio tags such as `[whispers]`, `[excitedly]`, `[very slow]`, `[laughs]`, and other expressive delivery notes. Enable `tts.gemini.audio_tags` or `tts.xai.auto_speech_tags` to have Hermes run a hidden rewrite pass before TTS. The rewrite inserts inline tags into the TTS script only; the visible chat reply stays unchanged.
```yaml
tts:
provider: gemini
gemini:
model: gemini-3.1-flash-tts-preview
audio_tags: true
xai:
auto_speech_tags: true
```
The rewrite uses `auxiliary.tts_audio_tags` and defaults to your main chat model. Override that auxiliary task if you want tag insertion handled by a cheaper or faster model.
**Language (OpenAI-compatible endpoints)**: `tts.openai.language` is forwarded to the endpoint as a `lang_code` request parameter. It is intended for OpenAI-compatible TTS servers that support `lang_code` — for example [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI), where `language: "es"` selects the Spanish phonemizer instead of the English default. Leave it unset when using the official OpenAI API, which does not accept this parameter. When unset, nothing extra is sent.
### Input length limits
Each provider has a documented per-request input-character cap. Hermes splits longer replies into ordered, sentence-aware chunks before calling the provider, so the full normalized text is preserved instead of silently truncated:
| Provider | Default cap (chars) |
|----------|---------------------|
| Edge TTS | 5000 |
| OpenAI | 4096 |
| xAI | 15000 |
| MiniMax | 10000 |
| Mistral | 4000 |
| Google Gemini | 32000 |
| ElevenLabs | Model-aware (see below) |
| NeuTTS | 2000 |
| KittenTTS | 2000 |
| Piper | 5000 |
**ElevenLabs** picks a cap from the configured `model_id`:
| `model_id` | Cap (chars) |
|------------|-------------|
| `eleven_flash_v2_5` | 40000 |
| `eleven_flash_v2` | 30000 |
| `eleven_multilingual_v2` (default), `eleven_multilingual_v1`, `eleven_english_sts_v2`, `eleven_english_sts_v1` | 10000 |
| `eleven_v3`, `eleven_ttv_v3` | 5000 |
| Unknown model | Falls back to provider default (10000) |
**Override per provider** with `max_text_length:` under the provider section of your TTS config:
```yaml
tts:
openai:
max_text_length: 8192 # raise or lower the provider cap
```
Only positive integers are honored. Zero, negative, non-numeric, or boolean values fall through to the provider default, so a broken config can't accidentally bypass the provider request limit.
### Telegram Voice Bubbles & ffmpeg
Telegram voice bubbles require Opus/OGG audio format:
- **OpenAI, ElevenLabs, and Mistral** produce Opus natively — no extra setup
- **Edge TTS** (default) outputs MP3 and needs **ffmpeg** to convert:
- **MiniMax TTS** outputs MP3 and needs **ffmpeg** to convert for Telegram voice bubbles
- **Google Gemini TTS** outputs raw PCM and uses **ffmpeg** to encode Opus directly for Telegram voice bubbles
- **xAI TTS** outputs MP3 and needs **ffmpeg** to convert for Telegram voice bubbles
- **NeuTTS** outputs WAV and also needs **ffmpeg** to convert for Telegram voice bubbles
- **KittenTTS** outputs WAV and also needs **ffmpeg** to convert for Telegram voice bubbles
- **Piper** outputs WAV and also needs **ffmpeg** to convert for Telegram voice bubbles
```bash
# Ubuntu/Debian
sudo apt install ffmpeg
# macOS
brew install ffmpeg
# Fedora
sudo dnf install ffmpeg
```
Without ffmpeg, Edge TTS, MiniMax TTS, NeuTTS, KittenTTS, and Piper audio are sent as regular audio files (playable, but shown as a rectangular player instead of a voice bubble).
:::tip
If you want voice bubbles without installing ffmpeg, switch to the OpenAI, ElevenLabs, or Mistral provider.
:::
### xAI Custom Voices (voice cloning)
xAI supports cloning your voice and using it with TTS. Create a custom voice in the [xAI Console](https://console.x.ai/team/default/voice/voice-library), then set the resulting `voice_id` in your config:
```yaml
tts:
provider: xai
xai:
voice_id: "nlbqfwie" # your custom voice ID
```
See the [xAI Custom Voices docs](https://docs.x.ai/developers/model-capabilities/audio/custom-voices) for details on recording, supported formats, and limits.
### Piper (local, 44 languages)
Piper is a fast, local neural TTS engine from the Open Home Foundation (the Home Assistant maintainers). It runs entirely on CPU, supports **44 languages** with pre-trained voices, and needs no API key.
**Install via `hermes tools`** → Voice & TTS → Piper — Hermes runs `pip install piper-tts` for you. Or install manually: `pip install piper-tts`.
**Switch to Piper:**
```yaml
tts:
provider: piper
piper:
voice: en_US-lessac-medium
```
On the first TTS call for a voice that isn't cached locally, Hermes runs `python -m piper.download_voices <name>` and downloads the model (~20-90MB depending on quality tier) into `~/.hermes/cache/piper-voices/`. Subsequent calls reuse the cached model.
**Picking a voice.** The [full voice catalog](https://github.com/OHF-Voice/piper1-gpl/blob/main/docs/VOICES.md) covers English, Spanish, French, German, Italian, Dutch, Portuguese, Russian, Polish, Turkish, Chinese, Arabic, Hindi, and more — each with `x_low` / `low` / `medium` / `high` quality tiers. Sample voices at [rhasspy.github.io/piper-samples](https://rhasspy.github.io/piper-samples/).
**Using a pre-downloaded voice.** Set `tts.piper.voice` to an absolute path ending in `.onnx`:
```yaml
tts:
piper:
voice: /path/to/my-custom-voice.onnx
```
**Advanced knobs** (`tts.piper.length_scale` / `noise_scale` / `noise_w_scale` / `volume` / `normalize_audio`, `use_cuda`) correspond 1:1 to Piper's `SynthesisConfig`. They're ignored on older `piper-tts` versions.
### Warm-up and unload via speech toggles (local engines)
Local engines (Piper, KittenTTS) load their model lazily, so without help the *first* spoken reply after you turn speech on pays the whole model load — and on a fresh install the voice download — as silence before the first word. Hermes treats the speech-output toggles as the signal that TTS is about to be needed:
- **Desktop** — turning on **Read replies aloud**, or starting a **voice conversation**, pre-loads the configured engine in the background right away. Turning both off again unloads the resident model (a Piper voice is tens of MB; KittenTTS up to ~80MB) so it isn't parked in RAM for nothing.
- **CLI / TUI** — `/voice tts` (and `/voice on` when `voice.auto_tts` is set) do the same; `/voice off` releases.
Each toggle holds a *lease* on the engine; the model is only unloaded when the last lease across surfaces is released, so switching off read-aloud in one Desktop window never pulls the voice out from under a conversation running in another. For cloud providers there is no model to hold — the toggle only makes sure a lazily-installed SDK (edge-tts, ElevenLabs, Mistral) is present. Warm-up is best-effort: if the engine can't load, the toggle still succeeds and the first reply falls back to loading on demand as before.
The Desktop calls `POST /api/audio/tts-lease` with `{"lease": "<name>", "active": true|false}`; other frontends can use the same endpoint.
The same lease also reaches user-declared providers, so a self-hosted TTS server can preload and unload its model on the toggles: a [command provider](#custom-command-providers) runs its optional `warm_command` / `release_command`, and a [Python plugin provider](#python-plugin-providers) gets `warm()` / `release()`.
### Custom command providers
If a TTS engine you want isn't natively supported (VoxCPM, MLX-Kokoro, XTTS CLI, a voice-cloning script, anything else that exposes a CLI), you can wire it in as a **command-type provider** without writing any Python. Hermes writes the input text to a temp UTF-8 file, runs your shell command, and reads the audio file the command produced.
Declare one or more providers under `tts.providers.<name>` and switch between them with `tts.provider: <name>` — the same way you switch between built-ins like `edge` and `openai`.
```yaml
tts:
provider: voxcpm # pick any name under tts.providers
providers:
voxcpm:
type: command
command: "voxcpm --ref ~/voice.wav --text-file {input_path} --out {output_path}"
output_format: mp3
timeout: 180
voice_compatible: true # try to deliver as a Telegram voice bubble
mlx-kokoro:
type: command
command: "python -m mlx_kokoro --in {input_path} --out {output_path} --voice {voice}"
voice: af_sky
output_format: wav
piper-custom: # native Piper also supports custom .onnx via tts.piper.voice
type: command
command: "piper -m /path/to/custom.onnx -f {output_path} < {input_path}"
output_format: wav
```
**Supported `output_format` values:** `mp3` (default), `wav`, `ogg`, `flac`, `m4a`, `aac`, `amr`, `opus`. Your command must actually produce that format (e.g. via `ffmpeg`); Hermes only validates the declared value and names the output file accordingly. An unknown value falls back to `mp3`. The chosen format is also exposed to the command as the `{format}` placeholder.
**Subprocess environment:** command providers (TTS and STT) run with Hermes secrets scrubbed from the child environment — gateway bot tokens, LLM provider API keys, and internal relay credentials are removed; `PATH`, `HOME`, locale, and other normal variables are kept. If your command template needs its own API key from the environment (e.g. a `curl` one-liner), list the variable names under `env_passthrough` in the provider config:
```yaml
tts:
providers:
mycloud:
type: command
command: 'curl -s -H "Authorization: Bearer $MYCLOUD_API_KEY" ... -o {output_path}'
env_passthrough: [MYCLOUD_API_KEY]
```
#### Example: Doubao (Chinese seed-tts-2.0)
For high-quality Chinese TTS via ByteDance's [seed-tts-2.0](https://www.volcengine.com/docs/6561/1257544) bidirectional-streaming API, install the [`doubao-speech`](https://pypi.org/project/doubao-speech/) PyPI package and wire it in as a command provider:
```bash
pip install doubao-speech
export VOLCENGINE_APP_ID="your-app-id"
export VOLCENGINE_ACCESS_TOKEN="your-access-token"
```
```yaml
tts:
provider: doubao
providers:
doubao:
type: command
command: "doubao-speech say --text-file {input_path} --out {output_path}"
output_format: mp3
max_text_length: 1024
timeout: 30
```
Credentials come from your shell environment (`VOLCENGINE_APP_ID` / `VOLCENGINE_ACCESS_TOKEN`) or `~/.doubao-speech/config.yaml`. Pick a voice by adding `--voice zh-female-warm` (or any other alias from `doubao-speech list-voices`) to the command. `doubao-speech` also bundles streaming ASR — see the [STT section below](#example-doubao--volcengine-asr) for Hermes integration. Source and full docs: [github.com/Hypnus-Yuan/doubao-speech](https://github.com/Hypnus-Yuan/doubao-speech).
#### Placeholders
Your command template can reference these placeholders. Hermes substitutes them at render time and shell-quotes each value for the surrounding context (bare / single-quoted / double-quoted), so paths with spaces and other shell-sensitive characters are safe.
| Placeholder | Meaning |
|------------------|------------------------------------------------------|
| `{input_path}` | Path to the temp UTF-8 text file Hermes wrote |
| `{text_path}` | Alias for `{input_path}` |
| `{output_path}` | Path the command must write audio to |
| `{format}` | `mp3` / `wav` / `ogg` / `flac` |
| `{voice}` | `tts.providers.<name>.voice`, empty when unset |
| `{model}` | `tts.providers.<name>.model` |
| `{speed}` | Resolved speed multiplier (provider or global) |
Use `{{` and `}}` for literal braces.
#### Optional keys
| Key | Default | Meaning |
|--------------------|---------|------------------------------------------------------------------------------------------------------------|
| `timeout` | `120` | Idle seconds; stdout or stderr output resets the deadline. The process tree is killed after inactivity (Unix `killpg`, Windows `taskkill /T`). |
| `output_format` | `mp3` | One of `mp3` / `wav` / `ogg` / `flac`. Auto-inferred from the output extension if Hermes picks a path. |
| `voice_compatible` | `false` | When `true`, Hermes converts MP3/WAV output to Opus/OGG via ffmpeg so Telegram renders a voice bubble. |
| `max_text_length` | `5000` | Maximum input characters per command invocation; longer text is split into ordered chunks. |
| `voice` / `model` | empty | Passed to the command as placeholder values only. |
| `warm_command` / `release_command` | unset | Shell commands run when a surface toggles speech output on / when the last lease across surfaces is released — e.g. `curl -s localhost:5002/load?model={model}` to preload a local TTS server, and its `unload` counterpart. Best-effort and non-blocking: run in the background with the same `timeout`, `env_passthrough` and `{voice}` / `{model}` / `{speed}` placeholders as `command`; output is discarded and failures are only logged at debug. |
#### Behavior notes
- **Built-in names always win.** A `tts.providers.openai` entry never shadows the native OpenAI provider, so no user config can silently replace a built-in.
- **Default delivery is a document.** Command providers deliver as regular audio attachments on every platform. Opt in to voice-bubble delivery per-provider with `voice_compatible: true`.
- **Command failures surface to the agent.** Non-zero exit, empty output, or timeout all return an error with the command's stderr/stdout included so you can debug the provider from the conversation.
- **`type: command` is the default when `command:` is set.** Writing `type: command` explicitly is good practice but not required; an entry with a non-empty `command` string is treated as a command provider.
- **`{input_path}` / `{text_path}` are interchangeable.** Use whichever reads better in your command.
#### Security
Command-type providers run whatever shell command you configure, with your user's permissions. Hermes quotes placeholder values and enforces the configured timeout, but the command template itself is trusted local input — treat it the same way you would a shell script on your PATH.
### Python plugin providers
For TTS engines that can't be expressed as a single shell command — Python SDKs without a CLI, streaming engines, voice-listing APIs, OAuth-refreshing auth — register a Python plugin via `ctx.register_tts_provider()`. The plugin **coexists with** (does not replace) the [Custom command providers](#custom-command-providers) registry; pick the surface that fits your engine.
#### When to pick which
| Your backend has… | Use |
|---|---|
| A single CLI reading text from a file/stdin and writing audio to a file/stdout | **Command provider** (no Python needed) |
| Two or three CLIs chained with shell pipes | **Command provider** |
| A Python SDK only — no CLI | **Plugin** |
| Streaming bytes you want to deliver chunked (mid-generation voice bubbles) | **Plugin** (override `stream()`) |
| A voice-listing API used by `hermes setup` | **Plugin** (override `list_voices()`) |
| OAuth refresh flow (not a static bearer token) | **Plugin** |
Built-ins always win, and command providers win over a same-name plugin — so plugins are safe to register against any non-built-in name without worrying about shadowing your existing config.
#### Minimal plugin
Drop this in `~/.hermes/plugins/my-tts/`:
`plugin.yaml`:
```yaml
name: my-tts
version: 0.1.0
description: "My custom Python TTS backend"
```
`__init__.py`:
```python
from agent.tts_provider import TTSProvider
class MyTTSProvider(TTSProvider):
@property
def name(self) -> str:
return "my-tts" # what tts.provider matches against
@property
def display_name(self) -> str:
return "My Custom TTS"
def is_available(self) -> bool:
# Return False when credentials/deps are missing — picker skips
# this row but the dispatcher still routes here on explicit config.
import os
return bool(os.environ.get("MY_TTS_API_KEY"))
def synthesize(self, text, output_path, *, voice=None, model=None,
speed=None, format="mp3", **extra) -> str:
# Write audio bytes to output_path, return the path.
# Raise on failure — the dispatcher converts exceptions to a
# standard error envelope.
import my_tts_sdk
client = my_tts_sdk.Client()
audio_bytes = client.synthesize(text=text, voice=voice or "default")
with open(output_path, "wb") as f:
f.write(audio_bytes)
return output_path
def register(ctx):
ctx.register_tts_provider(MyTTSProvider())
```
Enable it (`hermes plugins enable my-tts`), point `tts.provider` at it (`tts.provider: my-tts` in `config.yaml`), and the `text_to_speech` tool will route through your plugin.
#### Optional hooks
Override these on your provider class for richer integration:
- `list_voices()` → list of `{id, display, language, gender, preview_url}` dicts shown in `hermes tools`.
- `list_models()` → list of `{id, display, languages, max_text_length}` dicts.
- `get_setup_schema()` → return `{name, badge, tag, env_vars: [{key, prompt, url}]}` to power the picker row in `hermes tools` / `hermes setup`. Without this, the plugin still works but its row in the picker is minimal.
- `stream(text, *, voice, model, format, **extra)` → iterator yielding audio bytes for streaming delivery (default raises `NotImplementedError`).
- `voice_compatible` property → set `True` if your output is Opus-compatible and the gateway should deliver it as a voice bubble (default `False` = regular audio attachment).
- `warm()` / `release()` → called when a surface toggles speech output on / when the last lease across surfaces is released, while your provider is the configured `tts.provider` — preload or unload a local model server here. Both default to no-ops; exceptions are logged at debug and never fail the toggle.
See `agent/tts_provider.py` for the full ABC including docstrings.
## Voice Message Transcription (STT)
Voice messages sent on Telegram, Discord, WhatsApp, Slack, or Signal are automatically transcribed and injected as text into the conversation. The agent sees the transcript as normal text.
| Provider | Quality | Cost | API Key |
|----------|---------|------|---------|
| **Local Whisper** (default) | Good | Free | None needed |
| **Groq Whisper API** | Good–Best | Free tier | `GROQ_API_KEY` |
| **OpenAI Whisper API** | Good–Best | Paid | `VOICE_TOOLS_OPENAI_KEY` or `OPENAI_API_KEY` |
:::info Zero Config
Local transcription works out of the box when `faster-whisper` is installed. If that's unavailable, Hermes can also use a local `whisper` CLI from common install locations (like `/opt/homebrew/bin`) or a custom command via `HERMES_LOCAL_STT_COMMAND`.
:::
### Configuration
```yaml
# In ~/.hermes/config.yaml
stt:
provider: "local" # "local" | "groq" | "openai" | "mistral" | "xai" | "elevenlabs" | "deepinfra"
language: "en" # Global language hint applied to every provider unless a per-provider language overrides it; set "" to restore auto-detect
local:
model: "base" # tiny, base, small, medium, large-v3
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else auto-detect
groq:
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else auto-detect
openai:
model: "whisper-1" # whisper-1, gpt-4o-mini-transcribe, gpt-4o-transcribe, gpt-transcribe
mistral:
model: "voxtral-mini-latest" # voxtral-mini-latest, voxtral-mini-2602
xai:
model: "grok-stt" # xAI Grok STT
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else "en"
```
### Provider Details
**Local (faster-whisper)** — Runs Whisper locally via [faster-whisper](https://github.com/SYSTRAN/faster-whisper). Uses CPU by default, GPU if available. Model sizes:
| Model | Size | Speed | Quality |
|-------|------|-------|---------|
| `tiny` | ~75 MB | Fastest | Basic |
| `base` | ~150 MB | Fast | Good (default) |
| `small` | ~500 MB | Medium | Better |
| `medium` | ~1.5 GB | Slower | Great |
| `large-v3` | ~3 GB | Slowest | Best |
**Groq API** — Requires `GROQ_API_KEY`. Good cloud fallback when you want a free hosted STT option. Set `stt.groq.language` (or the global `HERMES_LOCAL_STT_LANGUAGE` env var) to skip Whisper's auto-detect and reduce latency on known-language audio.
**OpenAI API** — Accepts `VOICE_TOOLS_OPENAI_KEY` first and falls back to `OPENAI_API_KEY`. Supports `whisper-1`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe`, and `gpt-transcribe`.
**Mistral API (Voxtral Transcribe)** — Requires `MISTRAL_API_KEY`. Uses Mistral's [Voxtral Transcribe](https://docs.mistral.ai/capabilities/audio/speech_to_text/) models. Supports 13 languages, speaker diarization, and word-level timestamps. Install with `cd ~/.hermes/hermes-agent && uv pip install -e ".[mistral]"`.
**xAI Grok STT** — Requires `XAI_API_KEY`. Posts to `https://api.x.ai/v1/stt` as multipart/form-data. Good choice if you're already using xAI for chat or TTS and want one API key for everything. Auto-detection order puts it after Groq — explicitly set `stt.provider: xai` to force it.
**Custom local CLI fallback** — Set `HERMES_LOCAL_STT_COMMAND` if you want Hermes to call a local transcription command directly. The command template supports `{input_path}`, `{output_dir}`, `{language}`, and `{model}` placeholders. Hermes tokenizes the rendered template into an argument list and executes it without a shell, so operators such as `|`, `>`, `&&`, and `;` are passed as literal arguments. Your command must write a `.txt` transcript somewhere under `{output_dir}`.
#### Example: Doubao / Volcengine ASR
If you use [`doubao-speech`](https://pypi.org/project/doubao-speech/) for Doubao TTS (see [above](#example-doubao-chinese-seed-tts-20)), the same package handles speech-to-text via the local-command STT surface:
```bash
pip install doubao-speech
export VOLCENGINE_APP_ID="your-app-id"
export VOLCENGINE_ACCESS_TOKEN="your-access-token"
export HERMES_LOCAL_STT_COMMAND='doubao-speech transcribe {input_path} --out {output_dir}/transcript.txt'
```
If a trusted local template intentionally needs pipes, redirects, or another shell feature, invoke the shell explicitly. Keep dynamic paths outside the shell program and pass them as positional arguments:
```bash
export HERMES_LOCAL_STT_COMMAND='sh -c '\''whisper "$1" --output_format txt --output_dir "$2" | tee "$2/whisper.log"'\'' _ {input_path} {output_dir}'
```
On Windows, use an explicit `cmd /c` or PowerShell wrapper instead. An explicit wrapper makes shell interpretation an opt-in part of the configured argv rather than an implicit property of every local STT template.
```yaml
stt:
provider: local_command
```
Hermes writes the incoming voice message to `{input_path}`, runs the command, and reads the `.txt` file produced under `{output_dir}`. Language is auto-detected by the Volcengine bigmodel endpoint.
### Fallback Behavior
An **explicit** `stt.provider` selection (written in `config.yaml`, e.g. via `hermes tools`) is honored strictly — if that provider can't run, transcription fails with a clear error (`stt is configured to use <provider> (set via hermes tools), but <failure>. Run 'hermes tools' to change it.`) instead of silently switching engines. Note that `stt.provider: local` written in your config counts as an explicit selection.
When **no provider has ever been selected**, Hermes auto-detects from what's available:
- **Local faster-whisper unavailable** → Tries a local `whisper` CLI or `HERMES_LOCAL_STT_COMMAND` before cloud providers
- **Groq key not set** → Skipped; next available provider
- **OpenAI key not set** → Skipped; next available provider
- **Mistral key/SDK not set** → Skipped in auto-detect; falls through to next available provider
- **Nothing available** → Voice messages pass through with an accurate note to the user
### STT custom command providers
If the STT engine you want isn't natively supported (Doubao ASR, NVIDIA Parakeet, a whisper.cpp build, an open-source SenseVoice CLI, anything else that exposes a shell command), wire it in as a **command-type provider** without writing any Python. Hermes runs your shell command against the audio file and reads back the transcript.
Declare one or more providers under `stt.providers.<name>` and switch between them with `stt.provider: <name>` — same shape as the TTS [command-provider registry](#custom-command-providers), adapted for the input=audio → output=transcript direction.
```yaml
stt:
provider: parakeet # pick any name under stt.providers
providers:
parakeet:
type: command
command: "parakeet-asr --model nvidia/parakeet-tdt-0.6b-v2 --in {input_path} --out {output_path}"
format: txt
language: en
timeout: 300
whispercpp:
type: command
command: "whisper-cli -m ~/models/ggml-large-v3.bin -f {input_path} -otxt -of {output_dir}/transcript"
format: txt
sensevoice:
type: command
command: "sensevoice-cli {input_path} --json | tee {output_path}"
format: json
```
This complements the legacy `HERMES_LOCAL_STT_COMMAND` escape hatch via the built-in `local_command` path. Unlike the shell-driven command-provider registry, the legacy template is tokenized into argv and runs without implicit shell interpretation. Use `stt.providers.<name>` when you want **multiple** shell-driven STT engines, a name you can pick via `stt.provider`, or anything that needs per-provider `language` / `model` / `timeout`.
#### STT placeholders
Your command template can reference these placeholders. Hermes substitutes them at render time and shell-quotes each value for the surrounding context (bare / single-quoted / double-quoted), so paths with spaces are safe.
| Placeholder | Meaning |
|-------------------|----------------------------------------------------------------------|
| `{input_path}` | Absolute path to the input audio file (original location, read-only) |
| `{output_path}` | Absolute path the command should write the transcript to |
| `{output_dir}` | Parent directory of `{output_path}` (handy for whisper-style tools) |
| `{format}` | Configured output format: `txt` / `json` / `srt` / `vtt` |
| `{language}` | Configured language code (defaults to `en`) |
| `{model}` | `stt.providers.<name>.model`, empty when unset |
Use `{{` and `}}` for literal braces (handy when embedding JSON snippets in the command).
#### How the transcript is read back
After your command exits successfully:
1. If `{output_path}` exists and is non-empty → Hermes reads it as UTF-8 text.
2. Otherwise, if the command wrote to stdout → Hermes uses that.
3. Otherwise → error: "Command STT provider wrote no output file and produced no stdout".
This lets you use the registry for both file-writing CLIs (`whisper-cli`, `parakeet-asr`) and curl-style one-liners that emit transcript to stdout (`curl … | jq -r .text`).
For `format: json` / `srt` / `vtt`, Hermes returns the raw file content as the `transcript` field. Extracting `.text` from JSON is out of scope for the runner — either configure `format: txt`, or post-process JSON downstream.
#### STT command-provider optional keys
| Key | Default | Meaning |
|-----------------|---------|------------------------------------------------------------------------------------------------------|
| `timeout` | `300` | Seconds; the process tree is killed on expiry (Unix `start_new_session`, Windows `taskkill /T`). |
| `format` | `txt` | One of `txt` / `json` / `srt` / `vtt`. Sets the extension of `{output_path}`. |
| `language` | `en` | Forwarded to `{language}`. Defaults to `stt.language` then `en`. |
| `model` | empty | Forwarded to `{model}`. The `model=` argument to `transcribe_audio()` overrides this. |
#### STT command-provider behavior notes
- **Built-ins always win.** Declaring `stt.providers.openai: type: command` does NOT override the real OpenAI Whisper handler. The built-in name is short-circuited before the command-provider resolver runs.
- **Process-tree cleanup.** A command running over `timeout` has its entire process tree killed, not just the shell wrapper. Long-running ASR pipelines that fork model-loading subprocesses are reaped reliably.
- **Shell-quoting is automatic.** Placeholders inside `'…'` get single-quote-safe escaping; inside `"…"` get `$`/`` ` ``/`"` escaping; outside quotes get `shlex.quote`. Don't pre-quote placeholder values.
#### STT command-provider security
The shell command runs under the same user as Hermes with full filesystem access — same trust model as `tts.providers.<name>: type: command` and `HERMES_LOCAL_STT_COMMAND`. Only declare command providers from sources you trust.
### Python plugin providers (STT)
For STT engines that aren't built-in AND can't be expressed as a shell command (need a Python SDK, OAuth-refreshing auth, streaming chunks, etc.), register a Python plugin via `ctx.register_transcription_provider()`. The plugin **coexists with** the 8 built-in providers (`local`, `local_command`, `groq`, `openai`, `mistral`, `xai`, `elevenlabs`, `deepinfra`) and the `stt.providers.<name>: type: command` registry — built-ins keep their native implementations and always win on name collision; command providers win over plugins of the same name (config is more local than plugin install).
#### When to pick which (STT)
| Backend has… | Use |
|--------------------------------------------------------------|------------------------------------------------------------------|
| A single shell command that takes an audio file and emits text | `stt.providers.<name>: type: command` (no Python needed) |
| Only the legacy single-command escape hatch is wanted | `HERMES_LOCAL_STT_COMMAND` env var (argv-tokenized; no implicit shell) |
| A Python SDK with no CLI | `register_transcription_provider()` plugin |
| OAuth-refreshing auth, streaming chunks, voice-list metadata | `register_transcription_provider()` plugin |
| A built-in already covers it (`local`, `groq`, `openai`, …) | Set `stt.provider: <name>` — built-ins are inline |
#### Resolution order
1. **`stt.provider` is a built-in name** → built-in dispatch. **Always wins.**
2. **`stt.provider` matches `stt.providers.<name>` with `command:` set** → command-provider runner (see [STT custom command providers](#stt-custom-command-providers)). Wins over a same-name plugin.
3. **`stt.provider` matches a plugin-registered `TranscriptionProvider`** → plugin dispatch:
- if the plugin's `is_available()` returns `False` (missing creds or SDK), the call surfaces an unavailability error envelope identifying the plugin — **not** the generic "No STT provider available" message.
- otherwise the plugin's `transcribe()` is called with `model` (from the public `model=` arg, falling back to `stt.<provider>.model`) and `language` (from `stt.<provider>.language`).
4. **No match** → "No STT provider available" error.
#### Per-provider config namespace
Plugins read their per-provider configuration from `stt.<provider>` in `config.yaml`, mirroring how built-ins read `stt.openai.model` / `stt.mistral.model`:
```yaml
stt:
provider: my-stt
my-stt:
model: whisper-large-v3
language: ja # forwarded as language= to transcribe()
# any other plugin-specific keys go here; read them via your
# own config.yaml access in __init__/is_available/transcribe
```
The dispatcher forwards `model` and `language` from this section; everything else, the plugin can read itself.
#### Minimal plugin
Drop this in `~/.hermes/plugins/my-stt/`:
`plugin.yaml`:
```yaml
name: my-stt
version: 0.1.0
description: "My custom Python STT backend"
```
`__init__.py`:
```python
from agent.transcription_provider import TranscriptionProvider
class MySTTProvider(TranscriptionProvider):
@property
def name(self) -> str:
return "my-stt" # what stt.provider matches against
@property
def display_name(self) -> str:
return "My Custom STT"
def is_available(self) -> bool:
# Return False when credentials/deps are missing — picker skips
# this row but the dispatcher still routes here on explicit config.
import os
return bool(os.environ.get("MY_STT_API_KEY"))
def transcribe(self, file_path, *, model=None, language=None, **extra):
# Return the standard transcribe envelope:
# {"success": bool, "transcript": str, "provider": str, "error": str}
# Do NOT raise — convert exceptions to the error envelope so the
# gateway/CLI caller sees a consistent shape on failure.
try:
import my_stt_sdk
client = my_stt_sdk.Client()
text = client.transcribe(open(file_path, "rb"))
return {
"success": True,
"transcript": text,
"provider": "my-stt",
}
except Exception as exc:
return {
"success": False,
"transcript": "",
"error": f"my-stt failed: {exc}",
"provider": "my-stt",
}
def register(ctx):
ctx.register_transcription_provider(MySTTProvider())
```
Enable it (`hermes plugins enable my-stt`), set `stt.provider: my-stt` in `config.yaml`, and voice-message transcription will route through your plugin.
#### Optional hooks
Override these on your provider class for richer integration:
- `list_models()` → list of `{id, display, languages, max_audio_seconds}` dicts.
- `default_model()` → string returned when the user doesn't override the model.
- `get_setup_schema()` → return `{name, badge, tag, env_vars: [{key, prompt, url}]}` to power picker rows in `hermes tools` / `hermes setup` (the picker category for STT is not yet shipped — this metadata is available to plugins for forward compatibility).
See `agent/transcription_provider.py` for the full ABC including docstrings.
website/docs/user-guide/features/vision.md
+214
View File
@@ -0,0 +1,214 @@
---
title: Vision & Image Paste
description: Paste images from your clipboard into the Hermes CLI for multimodal vision analysis.
sidebar_label: Vision & Image Paste
sidebar_position: 7
---
# Vision & Image Paste
Hermes Agent supports **multimodal vision** — you can paste images from your clipboard directly into the CLI and ask the agent to analyze, describe, or work with them. Images are sent to the model as base64-encoded content blocks, so any vision-capable model can process them.
:::tip
Portal subscribers get vision-capable models (Claude, GPT-5, Gemini) in the same catalog — no extra credentials needed. See [Nous Portal](/integrations/nous-portal).
:::
## How It Works
1. Copy an image to your clipboard (screenshot, browser image, etc.)
2. Attach it using one of the methods below
3. Type your question and press Enter
4. The image appears as a `[📎 Image #1]` badge above the input
5. On submit, the image is sent to the model as a vision content block
You can attach multiple images before sending — each gets its own badge. Press `Ctrl+C` to clear all attached images.
Images are saved to `~/.hermes/images/` as PNG files with timestamped filenames.
## Paste Methods
How you attach an image depends on your terminal environment. Not all methods work everywhere — here's the full breakdown:
### `/paste` Command
**The most reliable explicit image-attach fallback.**
```
/paste
```
Type `/paste` and press Enter. Hermes checks your clipboard for an image and attaches it. This is the safest option when your terminal rewrites `Cmd+V`/`Ctrl+V`, or when you copied only an image and there is no bracketed-paste text payload to inspect.
### Ctrl+V / Cmd+V
Hermes now treats paste as a layered flow:
- normal text paste first
- native clipboard / OSC52 text fallback if the terminal did not deliver text cleanly
- image attach when the clipboard or pasted payload resolves to an image or image path
This means pasted macOS screenshot temp paths and `file://...` image URIs can attach immediately instead of sitting in the composer as raw text.
:::warning
If your clipboard has **only an image** (no text), terminals still cannot send binary image bytes directly. Use `/paste` as the explicit image-attach fallback.
:::
### `/terminal-setup` for VS Code / Cursor / Windsurf
If you run the TUI inside a local VS Code-family integrated terminal on macOS, Hermes can install the recommended `workbench.action.terminal.sendSequence` bindings for better multiline and undo/redo parity:
```text
/terminal-setup
```
This is especially useful when `Cmd+Enter`, `Cmd+Z`, or `Shift+Cmd+Z` are being intercepted by the IDE. Run it on the local machine only — not inside an SSH session.
## Platform Compatibility
| Environment | `/paste` | Cmd/Ctrl+V | `/terminal-setup` | Notes |
|---|:---:|:---:|:---:|---|
| **macOS Terminal / iTerm2** | ✅ | ✅ | n/a | Best experience — native clipboard + screenshot-path recovery |
| **Apple Terminal** | ✅ | ✅ | n/a | If Cmd+←/→/⌫ gets rewritten, use Ctrl+A / Ctrl+E / Ctrl+U fallbacks |
| **Linux X11 desktop** | ✅ | ✅ | n/a | Requires `xclip` (`apt install xclip`) |
| **Linux Wayland desktop** | ✅ | ✅ | n/a | Requires `wl-paste` (`apt install wl-clipboard`) |
| **WSL2 (Windows Terminal)** | ✅ | ✅ | n/a | Uses `powershell.exe` — no extra install needed |
| **VS Code / Cursor / Windsurf (local)** | ✅ | ✅ | ✅ | Recommended for better Cmd+Enter / undo / redo parity |
| **VS Code / Cursor / Windsurf (SSH)** | ❌² | ❌² | ❌³ | Run `/terminal-setup` on the local machine instead |
| **SSH terminal (any)** | ❌² | ❌² | n/a | Remote clipboard not accessible |
² See [SSH & Remote Sessions](#ssh--remote-sessions) below
³ The command writes local IDE keybindings and should not be run from the remote host
## Platform-Specific Setup
### macOS
**No setup required.** Hermes uses `osascript` (built into macOS) to read the clipboard. For faster performance, optionally install `pngpaste`:
```bash
brew install pngpaste
```
### Linux (X11)
Install `xclip`:
```bash
# Ubuntu/Debian
sudo apt install xclip
# Fedora
sudo dnf install xclip
# Arch
sudo pacman -S xclip
```
### Linux (Wayland)
Modern Linux desktops (Ubuntu 22.04+, Fedora 34+) often use Wayland by default. Install `wl-clipboard`:
```bash
# Ubuntu/Debian
sudo apt install wl-clipboard
# Fedora
sudo dnf install wl-clipboard
# Arch
sudo pacman -S wl-clipboard
```
:::tip How to check if you're on Wayland
```bash
echo $XDG_SESSION_TYPE
# "wayland" = Wayland, "x11" = X11, "tty" = no display server
```
:::
### WSL2
**No extra setup required.** Hermes detects WSL2 automatically (via `/proc/version`) and uses `powershell.exe` to access the Windows clipboard through .NET's `System.Windows.Forms.Clipboard`. This is built into WSL2's Windows interop — `powershell.exe` is available by default.
The clipboard data is transferred as base64-encoded PNG over stdout, so no file path conversion or temp files are needed.
:::info WSLg Note
If you're running WSLg (WSL2 with GUI support), Hermes tries the PowerShell path first, then falls back to `wl-paste`. WSLg's clipboard bridge only supports BMP format for images — Hermes auto-converts BMP to PNG using Pillow (if installed) or ImageMagick's `convert` command.
:::
#### Verify WSL2 clipboard access
```bash
# 1. Check WSL detection
grep -i microsoft /proc/version
# 2. Check PowerShell is accessible
which powershell.exe
# 3. Copy an image, then check
powershell.exe -NoProfile -Command "Add-Type -AssemblyName System.Windows.Forms; [System.Windows.Forms.Clipboard]::ContainsImage()"
# Should print "True"
```
## SSH & Remote Sessions
**Clipboard image paste does not fully work over SSH.** When you SSH into a remote machine, the Hermes CLI runs on the remote host. Clipboard tools (`xclip`, `wl-paste`, `powershell.exe`, `osascript`) read the clipboard of the machine they run on — which is the remote server, not your local machine. Your local clipboard image is therefore inaccessible from the remote side.
Text can sometimes still bridge through terminal paste or OSC52, but image clipboard access and local screenshot temp paths remain tied to the machine running Hermes.
### Workarounds for SSH
1. **Upload the image file** — Save the image locally, upload it to the remote server via `scp`, VSCode's file explorer (drag-and-drop), or any file transfer method. Then reference it by path. *(A `/attach <filepath>` command is planned for a future release.)*
2. **Use a URL** — If the image is accessible online, just paste the URL in your message. The agent can use `vision_analyze` to look at any image URL directly.
3. **X11 forwarding** — Connect with `ssh -X` to forward X11. This lets `xclip` on the remote machine access your local X11 clipboard. Requires an X server running locally (XQuartz on macOS, built-in on Linux X11 desktops). Slow for large images.
4. **Use a messaging platform** — Send images to Hermes via Telegram, Discord, Slack, or WhatsApp. These platforms handle image upload natively and are not affected by clipboard/terminal limitations.
## Why Terminals Can't Paste Images
This is a common source of confusion, so here's the technical explanation:
Terminals are **text-based** interfaces. When you press Ctrl+V (or Cmd+V), the terminal emulator:
1. Reads the clipboard for **text content**
2. Wraps it in [bracketed paste](https://en.wikipedia.org/wiki/Bracketed-paste) escape sequences
3. Sends it to the application through the terminal's text stream
If the clipboard contains only an image (no text), the terminal has nothing to send. There is no standard terminal escape sequence for binary image data. The terminal simply does nothing.
This is why Hermes uses a separate clipboard check — instead of receiving image data through the terminal paste event, it calls OS-level tools (`osascript`, `powershell.exe`, `xclip`, `wl-paste`) directly via subprocess to read the clipboard independently.
## Supported Models
Image paste works with any vision-capable model. The image is sent as a base64-encoded data URL in the OpenAI vision content format:
```json
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,..."
}
}
```
Most modern models support this format, including GPT-4 Vision, Claude (with vision), Gemini, and open-source multimodal models served through OpenRouter.
## Image Routing (Vision-Capable vs Text-Only Models)
When a user attaches an image — from the CLI clipboard, the gateway (Telegram/Discord photo), or any other entry point — Hermes routes it based on whether your current model actually supports vision:
| Your model | What happens to the image |
|---|---|
| **Vision-capable** (GPT-4V, Claude with vision, Gemini, Qwen-VL, MiMo-VL, etc.) | Sent as **real pixels** using the provider's native image content format above. No text summary layer. |
| **Text-only** (DeepSeek V3, smaller open-source models, older chat-only endpoints) | Routed through the `vision_analyze` auxiliary tool — an auxiliary vision model describes the image, and the text description is injected into the conversation. |
You don't configure this — Hermes looks up your current model's capability in the provider metadata and picks the right path automatically. The practical effect: you can switch between vision and non-vision models mid-session and image handling "just works" without changing your workflow. Text-only models get coherent context about the image rather than a broken multimodal payload they'd have to reject.
Which auxiliary model handles the text-description path is configurable under `auxiliary.vision` — see [Auxiliary Models](/user-guide/configuration#auxiliary-models).
### `vision_analyze` has the same dual behavior
The `vision_analyze` tool itself follows the same routing. When the active main model is vision-capable **and** its provider supports image content inside tool results (currently the Anthropic, OpenAI, Azure-OpenAI, and Gemini 3.x stacks), `vision_analyze` short-circuits the auxiliary describer and returns the raw image pixels as a multimodal tool-result envelope. The main model sees the image natively on its next turn — no aux call, no text-summary information loss, no extra latency.
For text-only main models (or providers whose tool-result channel doesn't carry images), `vision_analyze` falls back to the legacy path: it asks the configured auxiliary vision model to describe the image and returns the description as plain text. Either way the calling tool signature is the same — the tool decides which path to take at runtime based on the active model.
website/docs/user-guide/features/voice-mode.md
+581
View File
@@ -0,0 +1,581 @@
---
sidebar_position: 10
title: "Voice Mode"
description: "Real-time voice conversations with Hermes Agent — CLI, Telegram, Discord (DMs, text channels, and voice channels)"
---
# Voice Mode
Hermes Agent supports full voice interaction across CLI and messaging platforms. Talk to the agent using your microphone, hear spoken replies, and have live voice conversations in Discord voice channels.
If you want a practical setup walkthrough with recommended configurations and real usage patterns, see [Use Voice Mode with Hermes](../../guides/use-voice-mode-with-hermes.md).
For hands-free session start — saying "hey hermes" (or any phrase) to open a fresh voice session on the CLI, TUI, or desktop app — see [Wake Word](/user-guide/features/wake-word).
## Prerequisites
Before using voice features, make sure you have:
1. **Hermes Agent installed** — via the install script (see [Installation](/getting-started/installation))
2. **An LLM provider configured** — run `hermes model` or set your preferred provider credentials in `~/.hermes/.env`
3. **A working base setup** — run `hermes` to verify the agent responds to text before enabling voice
:::tip
The `~/.hermes/` directory and default `config.yaml` are created automatically the first time you run `hermes`. You only need to create `~/.hermes/.env` manually for API keys.
:::
:::tip Nous Portal covers both
A paid [Nous Portal](/user-guide/features/tool-gateway) subscription supplies the LLM (step 2) **and** OpenAI TTS via the Tool Gateway — no separate OpenAI key needed. On a fresh install, `hermes setup --portal` wires both up at once.
:::
## Overview
| Feature | Platform | Description |
|---------|----------|-------------|
| **Interactive Voice** | CLI | Press Ctrl+B to record, agent auto-detects silence and responds |
| **Auto Voice Reply** | Telegram, Discord | Agent sends spoken audio alongside text responses |
| **Voice Channel** | Discord | Bot joins VC, listens to users speaking, speaks replies back |
## Requirements
### Python Packages
```bash
# CLI voice mode (microphone + audio playback)
cd ~/.hermes/hermes-agent && uv pip install -e ".[voice]"
# Discord + Telegram messaging (includes discord.py[voice] for VC support)
cd ~/.hermes/hermes-agent && uv pip install -e ".[messaging]"
# Premium TTS (ElevenLabs)
cd ~/.hermes/hermes-agent && uv pip install -e ".[tts-premium]"
# Local TTS (NeuTTS, optional)
python -m pip install -U neutts[all]
# Everything at once
cd ~/.hermes/hermes-agent && uv pip install -e ".[all]"
```
| Extra | Packages | Required For |
|-------|----------|-------------|
| `voice` | `sounddevice`, `numpy` | CLI voice mode |
| `messaging` | `discord.py[voice]`, `python-telegram-bot`, `aiohttp` | Discord & Telegram bots |
| `tts-premium` | `elevenlabs` | ElevenLabs TTS provider |
Optional local TTS provider: install `neutts` separately with `python -m pip install -U neutts[all]`. On first use it downloads the model automatically.
:::info
`discord.py[voice]` installs **PyNaCl** (for voice encryption) and **opus bindings** automatically. This is required for Discord voice channel support.
:::
### System Dependencies
```bash
# macOS
brew install portaudio ffmpeg opus
brew install espeak-ng # for NeuTTS
# Ubuntu/Debian
sudo apt install portaudio19-dev ffmpeg libopus0
sudo apt install espeak-ng # for NeuTTS
```
| Dependency | Purpose | Required For |
|-----------|---------|-------------|
| **PortAudio** | Microphone input and audio playback | CLI voice mode |
| **ffmpeg** | Audio format conversion (MP3 → Opus, PCM → WAV) | All platforms |
| **Opus** | Discord voice codec | Discord voice channels |
| **espeak-ng** | Phonemizer backend | Local NeuTTS provider |
### API Keys
Add to `~/.hermes/.env`:
```bash
# Speech-to-Text — local provider needs NO key at all
# pip install faster-whisper # Free, runs locally, recommended
GROQ_API_KEY=your-key # Groq Whisper — fast, free tier (cloud)
VOICE_TOOLS_OPENAI_KEY=your-key # OpenAI Whisper — paid (cloud)
# Text-to-Speech (optional — Edge TTS and NeuTTS work without any key)
ELEVENLABS_API_KEY=*** # ElevenLabs — premium quality
# VOICE_TOOLS_OPENAI_KEY above also enables OpenAI TTS
```
:::tip
If `faster-whisper` is installed, voice mode works with **zero API keys** for STT. The model (~150 MB for `base`) downloads automatically on first use.
:::
---
## CLI Voice Mode
Voice mode is available in both the **classic CLI** (`hermes chat`) and the **TUI** (`hermes --tui`). Behavior is identical across both — same slash commands, same VAD silence detection, same streaming TTS, same hallucination filter. The TUI additionally forwards crash-forensic logs to `~/.hermes/logs/` so push-to-talk failures on exotic audio backends can be reported with a full stack trace rather than disappearing silently.
### Quick Start
Start the CLI and enable voice mode:
```bash
hermes # Start the interactive CLI
```
Then use these commands inside the CLI:
```
/voice Toggle voice mode on/off
/voice on Enable voice mode
/voice off Disable voice mode
/voice tts Toggle TTS output
/voice status Show current state
```
### How It Works
1. Start the CLI with `hermes` and enable voice mode with `/voice on`
2. **Press Ctrl+B** — a beep plays (880Hz), recording starts
3. **Speak** — a live audio level bar shows your input: `● [▁▂▃▅▇▇▅▂] ❯`
4. **Stop speaking** — after 3 seconds of silence, recording auto-stops
5. **Two beeps** play (660Hz) confirming the recording ended
6. Audio is transcribed via Whisper and sent to the agent
7. If TTS is enabled, the agent's reply is spoken aloud
8. Recording **automatically restarts** — speak again without pressing any key
This loop continues until you press **Ctrl+B** during recording (exits continuous mode) or 3 consecutive recordings detect no speech.
:::tip
The record key is configurable via `voice.record_key` in `~/.hermes/config.yaml` (default: `ctrl+b`).
:::
### Silence Detection
Two-stage algorithm detects when you've finished speaking:
1. **Speech confirmation** — waits for audio above the RMS threshold (200) for at least 0.3s, tolerating brief dips between syllables
2. **End detection** — once speech is confirmed, triggers after 3.0 seconds of continuous silence
If no speech is detected at all for 15 seconds, recording stops automatically.
Both `silence_threshold` and `silence_duration` are configurable in `config.yaml`. You can also disable the record start/stop beeps with `voice.beep_enabled: false`.
### Ending a voice chat by voice
Say **"stop"** — and nothing else — to end the voice conversation hands-free. The match is deliberately strict: the whole utterance (case-insensitive, surrounding punctuation ignored) must equal a configured phrase, so "stop doing that and try X instead" still reaches the agent normally. Customize the phrase list with `voice.stop_phrases` in `config.yaml` (e.g. `["stop", "goodbye hermes"]`), or set it to `[]` to disable. A voice chat also ends on its own after three consecutive silent cycles (no speech detected).
**Typing** a bare stop phrase while a voice chat is active works the same way on every surface (CLI, TUI, desktop): the message ends the voice chat instead of being sent to the agent. Outside a voice chat, typed "stop" is an ordinary message.
### Streaming TTS
When TTS is enabled, the agent speaks its reply **sentence-by-sentence** as it generates text — you don't wait for the full response. This works with **every TTS provider**:
1. Buffers text deltas into complete sentences (min 20 chars)
2. Strips markdown formatting, emoji, and `<think>` blocks
3. Plays audio per sentence in real-time — providers with a chunked PCM API (ElevenLabs, OpenAI) stream raw audio for the lowest time-to-first-word; every other provider (including the default Edge) synthesizes and plays each sentence as it completes
The same pipeline runs in the classic CLI, the TUI, and the desktop app. In a desktop voice conversation the reply text is fed **live** into a per-reply speech WebSocket as the model generates it, so speech overlaps generation — one socket and one audio clock per reply, no per-sentence connection gaps.
### Desktop remote: client-direct voice (lowest-hop path)
When Hermes Desktop is connected to a **remote gateway**, audio does not need to be relayed through the gateway at all. At voice-session start the desktop fetches the active profile's resolved STT/TTS settings (provider, model, language/voice, and credential) from the gateway over the authenticated REST channel (`GET /api/audio/voice-config`) and then calls the providers **directly**:
- **Dictation / voice input:** the mic recording goes straight from your desktop to the profile's STT provider; only the resulting *text* is sent to the gateway as the prompt.
- **Spoken replies:** the reply text is already streaming to the desktop over the chat socket, so the desktop synthesizes it locally with the profile's TTS provider and plays it — the gateway link never carries audio.
There is nothing to configure on the client: the profile you're talking to is the single source of truth for providers and keys, exactly as if the gateway had done the work itself. Keys are held in the desktop's memory for the session only — never written to disk on the client.
Providers that can only run on the gateway host (local whisper, `edge` TTS, command providers, plugins) automatically fall back to the relay path (`/api/audio/transcribe` and the speech WebSocket), as does any older backend without the endpoint. To force the relay for every provider, set:
```yaml
voice:
client_direct: false
```
Client-direct wire support: OpenAI (incl. Nous-managed audio), Groq, Mistral, and DeepInfra via the OpenAI-compatible shapes, xAI Grok STT, and ElevenLabs STT + TTS. xAI configured through OAuth stays on the relay (the OAuth bearer refreshes server-side).
### Barge-in
You can interrupt the agent at ANY point in its turn — the microphone stays live from the moment you finish speaking until the reply has fully played (full duplex):
- **Interject while it's thinking** — in continuous voice mode, speaking during LLM generation (before any audio plays) interrupts the in-flight turn and your interjection becomes the next message, the same as typing over a running turn.
- **Talk over it** — speaking while the agent's reply plays cuts playback the moment you start talking and submits what you said. The detector calibrates its noise floor against the *quiet room* at turn start (never against the playback itself), so speaker bleed can't deafen it and normal speech reliably trips it.
- **Type or press the record key** — sending a new message or hitting the push-to-talk key stops playback instantly on every surface.
- **Say "stop"** — the stop phrase works in both phases: mid-generation it interrupts the turn AND ends the voice chat; mid-playback it cuts the speech and ends the chat.
Tuning (config.yaml): `voice.barge_in: false` disables it; `voice.barge_in_threshold_multiplier` (default `3.0`) scales the speech trigger over the quiet-room floor; `voice.barge_in_grace_seconds` (default `0.5`) suppresses trips right after playback starts. Set `HERMES_VOICE_DEBUG=1` to stream per-block VAD diagnostics (calibrated floor, RMS, trip decisions) to stderr for live tuning.
The agent **knows** it was interrupted: the next message carries a short note telling the model its spoken reply was cut off, so it can react naturally ("rude!") or pick up where it left off instead of being oblivious.
### Hallucination Filter
Whisper sometimes generates phantom text from silence or background noise ("Thank you for watching", "Subscribe", etc.). The agent filters these out using a set of 26 known hallucination phrases across multiple languages, plus a regex pattern that catches repetitive variations.
---
## Gateway Voice Reply (Telegram & Discord)
If you haven't set up your messaging bots yet, see the platform-specific guides:
- [Telegram Setup Guide](../messaging/telegram.md)
- [Discord Setup Guide](../messaging/discord.md)
Start the gateway to connect to your messaging platforms:
```bash
hermes gateway # Start the gateway (connects to configured platforms)
hermes gateway setup # Interactive setup wizard for first-time configuration
```
### Discord: Channels vs DMs
The bot supports two interaction modes on Discord:
| Mode | How to Talk | Mention Required | Setup |
|------|------------|-----------------|-------|
| **Direct Message (DM)** | Open the bot's profile → "Message" | No | Works immediately |
| **Server Channel** | Type in a text channel where the bot is present | Yes (`@botname`) | Bot must be invited to the server |
**DM (recommended for personal use):** Just open a DM with the bot and type — no @mention needed. Voice replies and all commands work the same as in channels.
**Server channels:** The bot only responds when you @mention it (e.g. `@hermesbyt4 hello`). Make sure you select the **bot user** from the mention popup, not the role with the same name.
:::tip
To disable the mention requirement in server channels, add to `~/.hermes/.env`:
```bash
DISCORD_REQUIRE_MENTION=false
```
Or set specific channels as free-response (no mention needed):
```bash
DISCORD_FREE_RESPONSE_CHANNELS=123456789,987654321
```
:::
### Commands
These work in both Telegram and Discord (DMs and text channels):
```
/voice Toggle voice mode on/off
/voice on Voice replies only when you send a voice message
/voice tts Voice replies for ALL messages
/voice off Disable voice replies
/voice status Show current setting
```
### Modes
| Mode | Command | Behavior |
|------|---------|----------|
| `off` | `/voice off` | Text only (default) |
| `voice_only` | `/voice on` | Speaks reply only when you send a voice message |
| `all` | `/voice tts` | Speaks reply to every message |
Voice mode setting is persisted across gateway restarts.
### Platform Delivery
| Platform | Format | Notes |
|----------|--------|-------|
| **Telegram** | Voice bubble (Opus/OGG) | Plays inline in chat. ffmpeg converts MP3 → Opus if needed |
| **Discord** | Native voice bubble (Opus/OGG) | Plays inline like a user voice message. Falls back to file attachment if voice bubble API fails |
---
## Discord Voice Channels
The most immersive voice feature: the bot joins a Discord voice channel, listens to users speaking, transcribes their speech, processes through the agent, and speaks the reply back in the voice channel.
### Setup
#### 1. Discord Bot Permissions
If you already have a Discord bot set up for text (see [Discord Setup Guide](../messaging/discord.md)), you need to add voice permissions.
Go to the [Discord Developer Portal](https://discord.com/developers/applications) → your application → **Installation** → **Default Install Settings** → **Guild Install**:
**Add these permissions to the existing text permissions:**
| Permission | Purpose | Required |
|-----------|---------|----------|
| **Connect** | Join voice channels | Yes |
| **Speak** | Play TTS audio in voice channels | Yes |
| **Use Voice Activity** | Detect when users are speaking | Recommended |
**Updated Permissions Integer:**
| Level | Integer | What's Included |
|-------|---------|----------------|
| Text only | `309237763136` | View Channels, Send Messages, Read History, Embeds, Attachments, Threads, Reactions, Create Public Threads |
| Text + Voice | `309240908864` | All above + Connect, Speak |
**Re-invite the bot** with the updated permissions URL:
```
https://discord.com/oauth2/authorize?client_id=YOUR_APP_ID&scope=bot+applications.commands&permissions=309240908864
```
Replace `YOUR_APP_ID` with your Application ID from the Developer Portal.
:::warning
Re-inviting the bot to a server it's already in will update its permissions without removing it. You won't lose any data or configuration.
:::
#### 2. Privileged Gateway Intents
In the [Developer Portal](https://discord.com/developers/applications) → your application → **Bot** → **Privileged Gateway Intents**, enable all three:
| Intent | Purpose |
|--------|---------|
| **Presence Intent** | Detect user online/offline status |
| **Server Members Intent** | Resolve usernames in `DISCORD_ALLOWED_USERS` to numeric IDs (conditional) |
| **Message Content Intent** | Read text message content in channels |
**Message Content Intent** is required. **Server Members Intent** is only needed if your `DISCORD_ALLOWED_USERS` list uses usernames — if you use numeric user IDs, you can leave it OFF. Voice-channel SSRC → user_id mapping comes from Discord's SPEAKING opcode on the voice websocket and does **not** require the Server Members Intent.
#### 3. Opus Codec
The Opus codec library must be installed on the machine running the gateway:
```bash
# macOS (Homebrew)
brew install opus
# Ubuntu/Debian
sudo apt install libopus0
```
The bot auto-loads the codec from:
- **macOS:** `/opt/homebrew/lib/libopus.dylib`
- **Linux:** `libopus.so.0`
#### 4. Environment Variables
```bash
# ~/.hermes/.env
# Discord bot (already configured for text)
DISCORD_BOT_TOKEN=your-bot-token
DISCORD_ALLOWED_USERS=your-user-id
# STT — local provider needs no key (pip install faster-whisper)
# GROQ_API_KEY=your-key # Alternative: cloud-based, fast, free tier
# TTS — optional. Edge TTS and NeuTTS need no key.
# ELEVENLABS_API_KEY=*** # Premium quality
# VOICE_TOOLS_OPENAI_KEY=*** # OpenAI TTS / Whisper
```
### Start the Gateway
```bash
hermes gateway # Start with existing configuration
```
The bot should come online in Discord within a few seconds.
### Commands
Use these in the Discord text channel where the bot is present:
```
/voice join Bot joins your current voice channel
/voice channel Alias for /voice join
/voice leave Bot disconnects from voice channel
/voice status Show voice mode and connected channel
```
:::info
You must be in a voice channel before running `/voice join`. The bot joins the same VC you're in.
:::
### How It Works
When the bot joins a voice channel, it:
1. **Listens** to each user's audio stream independently
2. **Detects silence** — 1.5s of silence after at least 0.5s of speech triggers processing
3. **Transcribes** the audio via Whisper STT (local, Groq, or OpenAI)
4. **Processes** through the full agent pipeline (session, tools, memory)
5. **Speaks** the reply back in the voice channel via TTS
### Text Channel Integration
When the bot is in a voice channel:
- Transcripts appear in the text channel: `[Voice] @user: what you said`
- Agent responses are sent as text in the channel AND spoken in the VC
- The text channel is the one where `/voice join` was issued
### Echo Prevention
The bot automatically pauses its audio listener while playing TTS replies, preventing it from hearing and re-processing its own output.
### Access Control
Only users listed in `DISCORD_ALLOWED_USERS` can interact via voice. Other users' audio is silently ignored.
```bash
# ~/.hermes/.env
DISCORD_ALLOWED_USERS=284102345871466496
```
---
## Configuration Reference
### config.yaml
```yaml
# Voice recording (CLI)
voice:
record_key: "ctrl+b" # Key to start/stop recording
max_recording_seconds: 120 # Maximum recording length
auto_tts: false # Auto-enable TTS when voice mode starts
beep_enabled: true # Play record start/stop beeps
silence_threshold: 200 # RMS level (0-32767) below which counts as silence
silence_duration: 3.0 # Seconds of silence before auto-stop
stop_phrases: ["stop"] # Saying exactly one of these ends the voice chat; [] disables
# Speech-to-Text
stt:
enabled: true # set to false to skip auto-transcription —
# the gateway still caches the audio file and
# passes its path to the agent as part of the
# inbound message, useful for custom pipelines
# (diarization, alignment, archival, etc.)
provider: "local" # "local" (free) | "groq" | "openai" | "mistral" | "xai"
local:
model: "base" # tiny, base, small, medium, large-v3
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else auto-detect
groq:
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else auto-detect
# model: "whisper-1" # Legacy: used when provider is not set
# Text-to-Speech
tts:
provider: "edge" # "edge" (free) | "elevenlabs" | "openai" | "neutts" | "minimax" | "mistral" | "gemini" | "xai" | "kittentts" | "piper"
edge:
voice: "en-US-AriaNeural" # 322 voices, 74 languages
elevenlabs:
voice_id: "pNInz6obpgDQGcFmaJgB" # Adam
model_id: "eleven_multilingual_v2"
openai:
model: "gpt-4o-mini-tts"
voice: "alloy" # alloy, echo, fable, onyx, nova, shimmer
base_url: "https://api.openai.com/v1" # optional: override for self-hosted or OpenAI-compatible endpoints
# The `text_to_speech` tool accepts an optional per-call `instructions`
# argument (tone, emotion, pacing, accent, whispering) that is forwarded
# to `gpt-4o-mini-tts` and to OpenAI-compatible voice-design servers
# (e.g. Qwen3-TTS-VoiceDesign via oMLX). See OpenAI's voice-design guide:
# https://platform.openai.com/docs/guides/text-to-speech
neutts:
ref_audio: ''
ref_text: ''
model: neuphonic/neutts-air-q4-gguf
device: cpu
```
### Environment Variables
```bash
# Speech-to-Text providers (local needs no key)
# pip install faster-whisper # Free local STT — no API key needed
GROQ_API_KEY=... # Groq Whisper (fast, free tier)
VOICE_TOOLS_OPENAI_KEY=... # OpenAI Whisper (paid)
# STT advanced overrides (optional)
STT_GROQ_MODEL=whisper-large-v3-turbo # Override default Groq STT model
STT_OPENAI_MODEL=whisper-1 # Override default OpenAI STT model
GROQ_BASE_URL=https://api.groq.com/openai/v1 # Custom Groq endpoint
STT_OPENAI_BASE_URL=https://api.openai.com/v1 # Custom OpenAI STT endpoint
# Text-to-Speech providers (Edge TTS and NeuTTS need no key)
ELEVENLABS_API_KEY=*** # ElevenLabs (premium quality)
# VOICE_TOOLS_OPENAI_KEY above also enables OpenAI TTS
# Discord voice channel
DISCORD_BOT_TOKEN=...
DISCORD_ALLOWED_USERS=...
```
### STT Provider Comparison
| Provider | Model | Speed | Quality | Cost | API Key |
|----------|-------|-------|---------|------|---------|
| **Local** | `base` | Fast (depends on CPU/GPU) | Good | Free | No |
| **Local** | `small` | Medium | Better | Free | No |
| **Local** | `large-v3` | Slow | Best | Free | No |
| **Groq** | `whisper-large-v3-turbo` | Very fast (~0.5s) | Good | Free tier | Yes |
| **Groq** | `whisper-large-v3` | Fast (~1s) | Better | Free tier | Yes |
| **OpenAI** | `whisper-1` | Fast (~1s) | Good | Paid | Yes |
| **OpenAI** | `gpt-4o-transcribe` | Medium (~2s) | Best | Paid | Yes |
| **OpenAI** | `gpt-transcribe` | Fast | Best | Paid ($0.0045/min) | Yes |
| **Mistral** | `voxtral-mini-latest` | Fast | Good | Paid | Yes |
| **xAI** | `grok-stt` | Fast | Good | Paid | Yes |
Provider priority (automatic fallback): **local** > **groq** > **openai**
### TTS Provider Comparison
| Provider | Quality | Cost | Latency | Key Required |
|----------|---------|------|---------|-------------|
| **Edge TTS** | Good | Free | ~1s | No |
| **ElevenLabs** | Excellent | Paid | ~2s | Yes |
| **OpenAI TTS** | Good | Paid | ~1.5s | Yes |
| **NeuTTS** | Good | Free | Depends on CPU/GPU | No |
NeuTTS uses the `tts.neutts` config block above.
For `openai`, the `text_to_speech` tool accepts an optional `instructions`
argument that unlocks `gpt-4o-mini-tts`'s voice-design capability (tone,
emotion, pacing, accent, whispering). The same field also routes to
OpenAI-compatible voice-design servers mounted via `tts.openai.base_url`
(e.g. Qwen3-TTS-VoiceDesign via oMLX).
---
## Troubleshooting
### "No audio device found" (CLI)
PortAudio is not installed:
```bash
brew install portaudio # macOS
sudo apt install portaudio19-dev # Ubuntu
```
If you are running Hermes inside Docker on a Linux desktop, the container also needs access to your host audio socket. See the [Docker audio bridge](/user-guide/docker#optional-linux-desktop-audio-bridge) notes for a PulseAudio/PipeWire-compatible setup.
### Bot doesn't respond in Discord server channels
The bot requires an @mention by default in server channels. Make sure you:
1. Type `@` and select the **bot user** (with the #discriminator), not the **role** with the same name
2. Or use DMs instead — no mention needed
3. Or set `DISCORD_REQUIRE_MENTION=false` in `~/.hermes/.env`
### Bot joins VC but doesn't hear me
- Check your Discord user ID is in `DISCORD_ALLOWED_USERS`
- Make sure you're not muted in Discord
- The bot needs a SPEAKING event from Discord before it can map your audio — start speaking within a few seconds of joining
### Bot hears me but doesn't respond
- Verify STT is available: install `faster-whisper` (no key needed) or set `GROQ_API_KEY` / `VOICE_TOOLS_OPENAI_KEY`
- Check the LLM model is configured and accessible
- Review gateway logs: `tail -f ~/.hermes/logs/gateway.log`
### Bot responds in text but not in voice channel
- TTS provider may be failing — check API key and quota
- Edge TTS (free, no key) is the default fallback
- Check logs for TTS errors
### Whisper returns garbage text
The hallucination filter catches most cases automatically. If you're still getting phantom transcripts:
- Use a quieter environment
- Adjust `silence_threshold` in config (higher = less sensitive)
- Try a different STT model
website/docs/user-guide/features/wake-word.md
+341
View File
@@ -0,0 +1,341 @@
---
sidebar_position: 11
title: "Wake Word"
description: "Hands-free 'Hey Hermes' wake word — start a voice session by speaking, the 'Hey Siri' way"
---
# Wake Word ("Hey Hermes")
The wake word turns Hermes into a hands-free assistant across the CLI, TUI, and
desktop app: with one setting on, Hermes listens in the background for a spoken
trigger phrase. Say it, and Hermes starts a fresh session, opens the microphone,
captures your command via the normal [voice pipeline](/user-guide/features/voice-mode),
and answers — exactly like "Hey Siri" or "Alexa". Use `surface` to pick which
one listens.
Detection runs **entirely on-device**. The always-on listener only watches for
the wake phrase; no audio leaves your machine until you actually speak a command
to the agent.
## How it works
1. With `wake_word.enabled: true` (or after `/wake on`), a lightweight hotword
detector listens on your configured input device, or the process default
microphone when `wake_word.input_device` is unset.
2. When it hears the wake phrase it pauses itself (freeing the mic), starts a new
session, and records one utterance with voice mode's silence detection.
3. Your speech is transcribed and sent to the agent. After it replies, the
listener resumes automatically and waits for the next wake word.
It is **off by default** — nothing listens until you turn it on.
On the desktop app, a hands-free voice conversation can be ended by simply
**saying "stop"** (or "never mind", "goodbye", "cancel", "that's all") — the
spoken command ends the conversation instead of being sent to the agent. Only a
whole-utterance stop command matches, so a real request like "stop the docker
container" still goes through normally.
## Remote desktop (client capture)
When the desktop app connects to a **remote** Hermes backend (for example a
headless Docker host or a machine in another room), the backend often has **no
microphone**. Server-side PortAudio then fails with “Failed to open the
wake-word microphone.”
Hermes supports **client capture** for that case:
1. The desktop arms wake with `capture: client` (automatic for the GUI when the
backend has no local input device, or set explicitly below).
2. openWakeWord still runs **on the backend** (same engines, same models).
3. The desktop opens the **local Mac/PC microphone**, resamples to 16 kHz mono
int16, and streams short frames via the `wake.feed` RPC.
4. On detection the backend emits `wake.detected` as usual; the desktop starts
the normal voice pipeline on the client mic.
```yaml
wake_word:
enabled: true
capture: auto # auto | local | client
# auto — local PortAudio unless the desktop arms with client_capture
# local — always open the backend mic (CLI/TUI default)
# client — always expect wake.feed PCM from the desktop (remote-friendly)
```
The desktop GUI always passes `client_capture: true` on `wake.start`, so remote
backends without a mic arm in client mode automatically. CLI and TUI keep local
capture unless you set `capture: client` explicitly.
Privacy note: with client capture, wake PCM travels over the authenticated
desktop↔backend WebSocket (same channel as the rest of the session). Detection
still does not send audio to third-party wake APIs; the engine is local to the
backend process.
## Engines
| Engine | Cost | API key | Notes |
|--------|------|---------|-------|
| **openWakeWord** (default) | Free | None | Local ONNX models. Ships a bundled **"hey hermes"** model (default); also supports `hey_jarvis`, `alexa`, `hey_mycroft`, … and custom models |
| **sherpa** | Free | None | **Open vocabulary** — detects ANY typed phrase with zero training. Small English model auto-downloads on first use (~13 MB) |
| **Porcupine** | Free tier / paid | `PORCUPINE_ACCESS_KEY` | Picovoice engine; built-in keywords + custom `.ppn` files |
By default the phrase is **"hey hermes"** — a model for it ships with Hermes, so
it works out of the box with no training. (On first use, openWakeWord downloads
its shared feature-extraction models — a small one-time fetch.)
Both are lazy-installed the first time you enable the wake word (desktop
installs made with `--include-desktop` pre-install them, so the ear works
instantly). To install ahead of time:
```bash
cd ~/.hermes/hermes-agent && uv pip install -e ".[wake]"
```
## Quick start
```bash
# In an interactive `hermes` session:
/wake on # start listening (installs the engine on first use)
/wake status # show phrase, provider, and state
/wake off # stop listening
```
In the desktop app, click the ear icon in the composer.
The toggle IS the setting: turning the wake word on or off — via `/wake` or the
desktop ear button — also writes `wake_word.enabled` to `~/.hermes/config.yaml`,
so your choice persists across sessions. You can also flip it by hand:
```yaml
wake_word:
enabled: true
```
## Configuration
```yaml
wake_word:
enabled: false
surface: auto # eligible surface: "auto" | "cli" | "tui" | "gui"
input_device: null # PortAudio input index or device-name substring; null = process default
capture: auto # auto | local | client — where PCM is captured (see Remote desktop)
provider: openwakeword # "openwakeword" (free, local) | "sherpa" (free, any phrase) | "porcupine"
phrase: "hey hermes" # cosmetic label only — detection is keyed by the model/keyword below
sensitivity: 0.6 # 0.0-1.0 — higher = stricter (fewer false triggers), consistent across all engines
confirmation_frames: 3 # openWakeWord only — consecutive over-threshold frames required to fire
start_new_session: true # start a fresh session on wake vs. continue the current one
openwakeword:
model: hey_hermes # bundled default; OR a built-in name OR a path to a custom .onnx/.tflite
inference_framework: "" # "" (auto) | "onnx" | "tflite"
porcupine:
keyword: jarvis # built-in keyword OR path to a custom .ppn
```
`sensitivity`, `phrase`, and `start_new_session` apply to both engines. The
`openwakeword` and `porcupine` blocks select the actual detection model.
`input_device` is passed directly to the wake listener's PortAudio
(`sounddevice`) stream. Use either a numeric device index or an unambiguous
device-name substring. This setting only changes wake-word capture; desktop
push-to-talk still uses the desktop application's microphone path.
### Reducing false triggers on ambient speech
openWakeWord scores one short (~80ms) audio frame at a time, so a stray phoneme
in background conversation can occasionally spike a single frame over the
threshold and fire the wake word unintentionally. Two knobs control this:
- **`confirmation_frames`** (default `3`, openWakeWord only) — how many
*consecutive* over-threshold frames are required before the wake fires. A real
"hey hermes" holds a high score across several frames; an ambient blip spikes
just one. Raise it (e.g. `4`–`5`) if you still get false triggers in a noisy
room; the cost is a few tens of milliseconds of extra latency. `1` restores
the old fire-on-first-frame behavior.
- **`sensitivity`** (default `0.6`) — the detection threshold, `0.0`–`1.0`.
Higher is stricter (fewer false triggers). This direction is consistent across
**all** engines — for openWakeWord it's the raw per-frame score threshold, for
sherpa it maps onto the keyword threshold, and for Porcupine it's inverted
internally so "higher = stricter" holds there too. The `0.6` default sits
above openWakeWord's permissive `0.5` baseline, which let near-misses like
"hey hor" through; raise toward `0.8` if you still get false fires, lower it
if real "hey hermes" utterances are missed.
The `sherpa` and `porcupine` engines decode the whole phrase internally, so they
don't have the single-frame-spike problem and ignore `confirmation_frames`
(but they still honor `sensitivity`).
`inference_framework` picks the openWakeWord backend. Leave it empty (the
default) to let Hermes choose per platform: **tflite on Apple Silicon**, onnx
everywhere else. openWakeWord's onnx backend returns near-zero scores on macOS
ARM64 ([openWakeWord#336](https://github.com/dscripka/openWakeWord/issues/336)),
so a listener pinned to `onnx` there will arm, show as listening, and never
fire. The tflite backend needs `ai-edge-litert` on macOS, which Hermes installs
on demand alongside the other wake-word deps.
### Surfaces (CLI, TUI, GUI)
The wake word works in all three Hermes surfaces, and `surface` picks which one
owns the listener and opens the new session when it fires:
| `surface` | Behavior |
|-----------|----------|
| `auto` (default) | All local surfaces are eligible; the first one to arm owns the listener. |
| `cli` | Only the classic `hermes` CLI. |
| `tui` | Only `hermes --tui`. |
| `gui` | Only the desktop app. |
The detector is on-device and single-mic, so only one surface listens at a time,
including when Hermes surfaces run in separate processes. Ownership is sticky:
the first eligible claimant keeps the listener until it stops, disconnects, or
its process exits. Hermes does not silently fail over to another open surface.
Set `surface` when you want to pin ownership instead of using first-claim wins.
The TUI and desktop GUI share the same Python backend (`tui_gateway`), which
runs the detector server-side and yields the mic to voice capture while a
command records.
## Using a different phrase
"Hey Hermes" works out of the box — the bundled openWakeWord model
(`model: hey_hermes`) is the default. To wake on something else, the easiest
path is the open-vocabulary engine:
### Option A — sherpa (any phrase, zero training)
Type the phrase you want; it's tokenized at runtime — "hey coder",
"computer", "wake up neo", anything:
```yaml
wake_word:
enabled: true
provider: sherpa
phrase: "hey coder" # detection key — just type your phrase
```
The small English KWS model (~13 MB) downloads once on first use. Each
profile can set its own phrase — "hey \<profile\>" for every profile you run.
### Waking a specific profile (desktop)
With the sherpa engine, ONE listener can wake ANY profile. Every profile
whose config has `wake_word.enabled: true` is enrolled automatically; its
phrase defaults to `hey <profile name>` when unset. Say a profile's phrase
and the desktop app live-switches to that profile, opens a fresh session
there, and starts hands-free voice:
- "hey hermes" → default profile
- "hey coder" → the `coder` profile
- "hey trader" → the `trader` profile
Set `wake_word.profile_routing: false` on the listener's profile to opt out
and listen only for its own phrase. The CLI and TUI are single-profile
processes: a wake phrase belonging to another profile prints the switch
command (`hermes -p <profile>`) instead of routing.
Names are matched acoustically by their English subword sounds: two-word
phrases with distinct, 2+ syllable names work best. Very short names, heavy
non-English phonology, or two profiles with similar-sounding names will
degrade accuracy — tune per-profile `sensitivity` if needed.
### Option B — openWakeWord (free, trained model)
Name a built-in model (`hey_jarvis`, `alexa`, `hey_mycroft`, …), or train a
custom model (≈75–90 min on a free/Colab GPU) for maximum robustness, drop
the `.onnx` file somewhere, and reference it:
```yaml
wake_word:
enabled: true
provider: openwakeword
phrase: "computer"
openwakeword:
model: ~/.hermes/wakewords/computer.onnx # or a built-in name like hey_jarvis
```
Training references:
- [openWakeWord](https://github.com/dscripka/openWakeWord)
- [2026 training Colab](https://github.com/alfiedennen/openwakeword-colab-2026)
:::tip Pick a distinctive phrase
Wake phrases that don't collide with everyday speech generalize best. Two
syllables with an uncommon word ("hermes" qualifies) beat common words like
"hello" or "stop".
:::
### Option C — Porcupine (custom keyword in seconds)
Create a "Hey Hermes" keyword in the [Picovoice Console](https://console.picovoice.ai/),
download the `.ppn`, and:
```yaml
wake_word:
enabled: true
provider: porcupine
phrase: "hey hermes"
porcupine:
keyword: ~/.hermes/wakewords/hey_hermes.ppn
```
Set your access key in `~/.hermes/.env`:
```bash
PORCUPINE_ACCESS_KEY=your-key-here
```
## Requirements
- A working microphone and the `sounddevice` + `numpy` audio stack (shared with
voice mode).
- An STT provider for transcribing the spoken command — local `faster-whisper`
works out of the box; see [Voice Mode](/user-guide/features/voice-mode) for the
full provider list.
- A TTS provider for speaking the reply (the default `edge-tts` works with no
key). The wake flow is fully hands-free, so the toggle refuses to arm until
both STT and TTS are ready — `hermes tools` (Voice section) sets them up.
- The wake engine deps (auto-installed, or `hermes-agent[wake]`).
`/wake status` reports exactly what's missing if the listener won't start.
### "Listening" but never wakes (macOS)
macOS grants microphone access per **process**. STT working in the desktop app
proves the *renderer* has mic access — the wake listener runs in the Python
*backend*, which needs its own grant. Without it, CoreAudio hands the backend a
"working" stream that only ever delivers silence, so the ear shows listening
but the phrase never fires. Hermes detects this (`/wake status` shows
"mic delivers only silence"; the desktop ear tooltip carries the same hint).
Fix: System Settings → Privacy & Security → Microphone → enable the Hermes
backend (it may appear as your terminal, `python`, or Hermes), then toggle the
wake word off and on.
### "Listening" but receives silence (Windows)
Desktop push-to-talk and wake-word capture use different microphone paths.
Push-to-talk uses the desktop application's browser capture, while the
wake-word listener opens a PortAudio stream in the Python backend. One can work
while the other selects a silent or unusable Windows input.
`/wake status` reports the selected input device and Windows audio host API.
When it reports silence, set `wake_word.input_device` to the numeric index or an
unambiguous name of the working PortAudio input, then toggle the wake word:
```bash
hermes config set wake_word.input_device "Microphone Array"
```
Use `null` to return to the process default:
```bash
hermes config set wake_word.input_device null
```
## Notes & limits
- **Local surfaces only.** The wake word runs in the CLI, TUI, and desktop GUI —
wherever a local microphone is available. It does not run in the messaging
gateway (Telegram, Discord, …), which has no mic.
- **One mic at a time.** The detector releases the microphone while a command is
recording and reclaims it once the turn ends, so it won't fight voice capture.
- **Privacy.** Hotword detection is local. Set `sensitivity` higher if you get
false triggers, lower if it misses you.
website/docs/user-guide/features/web-dashboard.md
+1205
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/features/web-search.md
+489
View File
@@ -0,0 +1,489 @@
---
title: Web Search & Extract
description: Search the web and extract page content with multiple backend providers — including free self-hosted SearXNG.
sidebar_label: Web Search
sidebar_position: 6
---
# Web Search & Extract
Hermes Agent includes two model-callable web tools backed by multiple providers:
- **`web_search`** — search the web and return ranked results
- **`web_extract`** — fetch and extract readable content from one or more URLs
Both are configured through a single backend selection. Providers are chosen via `hermes tools` or set directly in `config.yaml`.
## Backends
| Provider | Env Var | Search | Extract | Free tier |
|----------|---------|--------|---------|-----------|
| **Firecrawl** (default) | `FIRECRAWL_API_KEY` (optional — keyless when selected) | ✔ | ✔ | 500 credits/mo · keyless cloud when selected |
| **SearXNG** | `SEARXNG_URL` | ✔ | — | ✔ Free (self-hosted) |
| **Brave Search (free tier)** | `BRAVE_SEARCH_API_KEY` | ✔ | — | 2 000 queries/mo |
| **DDGS (DuckDuckGo)** | — (no key) | ✔ | — | ✔ Free |
| **Exa** | `EXA_API_KEY` (optional) | ✔ | ✔ | ✔ Keyless ring member · 1 000 searches/mo with key |
| **Parallel** | `PARALLEL_API_KEY` (optional) | ✔ | ✔ | ✔ Keyless ring member · paid with key |
| **Tavily** | `TAVILY_API_KEY` (optional) | ✔ | ✔ | ✔ Opt-in keyless when selected |
| **Keenable** | `KEENABLE_API_KEY` (optional) | ✔ | ✔ | ✔ Keyless ring member · paid with key |
| **xAI (Grok)** | `XAI_API_KEY` or `hermes auth add xai-oauth` | ✔ | — | Paid (SuperGrok or per-token) |
Brave Search, DDGS, and xAI are **search-only** — pair any of them with Firecrawl/Tavily/Keenable/Exa/Parallel when you also need `web_extract`. DDGS uses the [`ddgs` Python package](https://pypi.org/project/ddgs/) under the hood; if it isn't already installed, run `pip install ddgs` (or let Hermes lazy-install it on first use). xAI runs Grok's server-side `web_search` tool on the Responses API — results are LLM-generated rather than index-backed, so titles, descriptions, and URL choice are all model output (see the [trust-model caveat](#xai-grok) below).
**Per-capability split:** you can use different providers for search and extract independently — for example SearXNG (free) for search and Firecrawl for extract. See [Per-capability configuration](#per-capability-configuration) below.
:::info Works out of the box — keyless free-tier rotation
A fresh install with **no web credentials at all** gets working `web_search` and `web_extract` out of the box: requests rotate round-robin across the ring vendors' public free tiers — **Exa, Parallel, Firecrawl, and Keenable** — spreading load evenly, and a rate-limited request automatically retries on the next vendor in the ring (multi-hop, until one serves or all are throttled). No signup, no key. This tier is strictly last-resort — any configured backend or present API key always wins — and requests carry no user identifiers (only a random per-process session id, rotated on restart). For guaranteed, unthrottled service, set up a keyed provider. Disable the keyless tier entirely with `web.keyless_fallback: false`.
:::
**Choosing free vs paid explicitly:** in `hermes tools`, Exa, Parallel, and Keenable each appear as two rows — **Free (keyless)** and **Paid (API key)**. Picking Free pins that vendor's anonymous endpoint (even if you later add a key); picking Paid pins the keyed path (a missing key then errors instead of silently downgrading to the free tier). The selection is stored as `web.provider_tier.<name>: free|paid`; leave it unset for auto (key present → paid, otherwise the keyless ring).
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, web search and extract are available through the **[Tool Gateway](tool-gateway.md)** via managed Firecrawl — no API key needed. New installs can run `hermes setup --portal` to log in and turn on all gateway tools at once; existing installs can flip just web via `hermes tools`.
:::
---
## How `web_extract` handles long pages
Backends return raw page markdown, which can be huge (forum threads, docs sites, news articles with embedded comments). To keep your context window usable, `web_extract` applies a **deterministic character budget** — no LLM summarization is involved:
| Page size (characters) | What happens |
|------------------------|--------------|
| At or under the budget (default 15 000) | Returned whole — full markdown reaches the agent |
| Over the budget | Head+tail window (~75% head / ~25% tail, cut on markdown line boundaries) plus an explicit `[TRUNCATED]` footer. The full clean text is stored to disk and the footer tells the agent the file path and the exact `read_file` call to page through the omitted middle |
| Over 2 000 000 | Stored text is capped at 2 MB |
The per-page budget is configurable via `web.extract_char_limit` in `config.yaml` (default `15000`, clamped to 2 000–500 000), and the agent can raise it per-call with the tool's `char_limit` argument.
### When truncation gets in the way
If you specifically need the live DOM rather than extracted markdown — for example, a JS-heavy page where extraction returns little content — use `browser_navigate` + `browser_snapshot` instead. The browser tool returns the live accessibility tree (subject to its own snapshot cap on huge pages).
---
## Result caching
Repeat web calls within a short window are served from cache instead of the paid backend — this saves credits and latency in the two patterns where duplicates are common: subagent fan-outs (several delegated agents researching the same topic) and the agent re-checking a page it read minutes ago.
| Call | Cache | Scope |
|------|-------|-------|
| `web_search` — same query (case/whitespace-insensitive), same provider | In-memory memo | Per process |
| `web_extract` — same URL, same format, same provider | Full text stored under `~/.hermes/cache/web/` | Shared across CLI, gateway, cron, and subagent processes |
Concurrent identical searches (a parallel subagent fan-out firing the same query at once) are **coalesced into a single backend request** — the first caller pays; the rest share the response. Requested search limits are bucketed up to 10/20/50/100 so near-identical requests (`limit=5` vs `limit=8`) share one entry, with each caller receiving its requested count.
Only successful responses are cached. Failures always retry the backend, responses served by the one-shot keyless rescue are never cached (the next call attempts your chosen backend again), and URLs matched by your `security.website_blocklist` are never served from cache. Cached extracts re-run the normal truncation pipeline, so a different `char_limit` on the second call works off the same stored scrape.
**Local development URLs are never cached.** Anything on `localhost`, `127.0.0.1`, `*.local`, single-label LAN hostnames, or private/link-local IP ranges (`192.168.*`, `10.*`, `172.16-31.*`) bypasses the extract cache entirely — dev servers, hot-reload builds, and chat-GUI artifact previews change on every save, and a cached copy would show you a stale build. Every fetch of a local page is live. (These URLs are only reachable at all when `security.allow_private_urls` is enabled.)
**Testing over the public internet?** Staging deploys and tunnel URLs are public DNS, so the local-dev rule can't catch them — list them in `web.cache_exempt_hosts` and they're always fetched live too. Entries match exactly, as a `*.` wildcard, or as a domain suffix (`mysite.dev` also covers `preview.mysite.dev`):
```yaml
# ~/.hermes/config.yaml
web:
cache_exempt_hosts:
- mysite.vercel.app
- "*.ngrok-free.app"
```
```yaml
# ~/.hermes/config.yaml
web:
cache_enabled: true # default; set false to disable both caches
cache_ttl_minutes: 20 # freshness window, clamped 1–1440
```
If you're researching genuinely live data (scores, prices, breaking news) and need every call fresh, lower the TTL or set `web.cache_enabled: false`.
---
## Setup
### Quick setup via `hermes tools`
Run `hermes tools`, navigate to **Web Search & Extract**, and pick a provider. The wizard prompts for the required URL or API key and writes it to your config.
```bash
hermes tools
```
---
### Firecrawl (default)
Full-featured search and extract. Recommended for most users.
```bash
# ~/.hermes/.env
FIRECRAWL_API_KEY=fc-your-key-here
```
Get a key at [firecrawl.dev](https://firecrawl.dev). The free tier includes 500 credits/month.
**Self-hosted Firecrawl:** Point at your own instance instead of the cloud API:
```bash
# ~/.hermes/.env
FIRECRAWL_API_URL=http://localhost:3002
```
When `FIRECRAWL_API_URL` is set, the API key is optional (disable server auth with `USE_DB_AUTHENTICATION=false`).
---
### SearXNG (free, self-hosted)
SearXNG is a privacy-respecting, open-source metasearch engine that aggregates results from 70+ search engines. **No API key required** — just point Hermes at a running SearXNG instance.
SearXNG is **search-only** — `web_extract` requires a separate extract provider.
#### Option A — Self-host with Docker (recommended)
This gives you a private instance with no rate limits.
**1. Create a working directory:**
```bash
mkdir -p ~/searxng/searxng
cd ~/searxng
```
**2. Write a `docker-compose.yml`:**
```yaml
# ~/searxng/docker-compose.yml
services:
searxng:
image: searxng/searxng:latest
container_name: searxng
ports:
- "8888:8080"
volumes:
- ./searxng:/etc/searxng:rw
environment:
- SEARXNG_BASE_URL=http://localhost:8888/
restart: unless-stopped
```
**3. Start the container:**
```bash
docker compose up -d
```
**4. Enable the JSON API format:**
SearXNG ships with JSON output disabled by default. Copy the generated config and enable it:
```bash
# Copy the auto-generated config out of the container
docker cp searxng:/etc/searxng/settings.yml ~/searxng/searxng/settings.yml
```
Open `~/searxng/searxng/settings.yml`.
If `use_default_settings: true` is present, the file only contains your overrides. All other settings are inherited from the built-in defaults.
To enable JSON responses for Hermes, add the following override:
```yaml
search:
formats:
- html
- json
```
Your `settings.yml` should look similar to:
```yaml
# Read the documentation before extending the defaults:
# https://docs.searxng.org/admin/settings/
use_default_settings: true
server:
secret_key: "abcdef12345678"
image_proxy: true
search:
formats:
- html
- json
```
**5. Restart to apply:**
```bash
docker cp ~/searxng/searxng/settings.yml searxng:/etc/searxng/settings.yml
docker restart searxng
```
**6. Verify it works:**
```bash
curl -s "http://localhost:8888/search?q=test&format=json" | python3 -c \
"import sys,json; d=json.load(sys.stdin); print(f'{len(d[\"results\"])} results')"
```
You should see something like `10 results`. If you get a `403 Forbidden`, JSON format is still disabled — recheck step 4.
**7. Configure Hermes:**
```bash
# ~/.hermes/.env
SEARXNG_URL=http://localhost:8888
```
Then select SearXNG as the search backend in `~/.hermes/config.yaml`:
```yaml
web:
search_backend: "searxng"
```
Or set via `hermes tools` → Web Search & Extract → SearXNG.
---
#### Option B — Use a public instance
Public SearXNG instances are listed at [searx.space](https://searx.space/). Filter by instances that have **JSON format enabled** (shown in the table).
```bash
# ~/.hermes/.env
SEARXNG_URL=https://searx.example.com
```
:::caution Public instances
Public instances have rate limits, variable uptime, and may disable JSON format at any time. For production use, self-hosting is strongly recommended.
:::
---
#### Pair SearXNG with an extract provider
SearXNG handles search; you need a separate provider for `web_extract`. Use the per-capability keys:
```yaml
# ~/.hermes/config.yaml
web:
search_backend: "searxng"
extract_backend: "firecrawl" # or tavily, keenable, exa, parallel
```
With this config, Hermes uses SearXNG for all search queries and Firecrawl for URL extraction — combining free search with high-quality extraction.
---
### Tavily
AI-optimised search and extract. Select Tavily in `hermes tools` (or set `web.backend: tavily`) to use it **keyless** with no account (rate-limited). Set an API key when you want higher limits.
```bash
# optional — skip this for keyless access after selecting Tavily
# ~/.hermes/.env
TAVILY_API_KEY=tvly-your-key-here
```
Get a key at [app.tavily.com](https://app.tavily.com/home). See [Tavily keyless](https://docs.tavily.com/documentation/keyless).
---
### Exa
Neural search with semantic understanding. Good for research and finding conceptually related content.
```bash
# ~/.hermes/.env
EXA_API_KEY=your-exa-key-here
```
Get a key at [exa.ai](https://exa.ai). The free tier includes 1 000 searches/month.
---
### Parallel
AI-native search and extraction with deep research capabilities.
```bash
# ~/.hermes/.env
PARALLEL_API_KEY=your-parallel-key-here
```
Get access at [parallel.ai](https://parallel.ai).
---
### xAI (Grok) {#xai-grok}
Routes `web_search` through Grok's server-side [web_search tool](https://docs.x.ai/developers/tools/web-search) on the Responses API. Grok runs the actual searching and returns the top results as structured JSON.
Works with either credential path — no new env vars, no new setup wizard:
```bash
# ~/.hermes/.env (env-var path)
XAI_API_KEY=sk-xai-your-key-here
```
or for SuperGrok subscribers:
```bash
hermes auth add xai-oauth
```
Then select xAI as the search backend:
```yaml
# ~/.hermes/config.yaml
web:
backend: "xai"
```
**Optional knobs:**
```yaml
web:
backend: "xai"
xai:
model: grok-build-0.1 # reasoning model required by web_search (default)
allowed_domains: # optional, max 5 — mutex with excluded_domains
- arxiv.org
excluded_domains: # optional, max 5
- example-spam.com
timeout: 90 # seconds (default)
```
**Search-only** — pair with Firecrawl / Tavily / Keenable / Exa / Parallel if you also need `web_extract`. On 401 the provider performs a single forced OAuth-token refresh and retries (covers mid-window revocation and opaque tokens the proactive expiry check can't decode); env-var credentials skip the retry.
:::caution Trust model
Unlike index-backed providers (Brave, Tavily, Exa) which return verbatim search-engine results, xAI is an LLM choosing which URLs to surface and writing the titles and descriptions itself. The *content* of the query influences the output, so a maliciously crafted query (e.g. injected via untrusted upstream input the agent picked up) can in principle steer Grok into emitting attacker-chosen URLs. Treat returned URLs the same way you'd treat any model-generated link — validate before fetching, especially if the query came from untrusted input.
:::
---
## Configuration
### Single backend
Set one provider for all web capabilities:
```yaml
# ~/.hermes/config.yaml
web:
backend: "searxng" # firecrawl | searxng | brave-free | ddgs | tavily | keenable | exa | parallel | xai
```
### Per-capability configuration
Use different providers for search vs extract. This lets you combine free search (SearXNG) with a paid extract provider, or vice versa:
```yaml
# ~/.hermes/config.yaml
web:
search_backend: "searxng" # used by web_search
extract_backend: "firecrawl" # used by web_extract
```
When per-capability keys are empty, both fall through to `web.backend`. Only when no web selection has ever been written is the backend auto-detected from whichever API key/URL is present — once a selection exists, the runtime always uses it, and adding a key to `.env` does not reroute web traffic.
**Priority order (per capability):**
1. `web.search_backend` / `web.extract_backend` (explicit per-capability)
2. `web.backend` (shared fallback; `nous` = managed Tool Gateway)
3. Auto-detect from environment variables (never-configured setups only)
### Auto-detection
If no backend has **ever** been selected (no `web.backend` / per-capability key written by you or `hermes tools`), Hermes picks the first available one based on which credentials are set:
| Credential present | Auto-selected backend |
|--------------------|-----------------------|
| `TAVILY_API_KEY` | tavily |
| `EXA_API_KEY` | exa |
| `PARALLEL_API_KEY` | parallel |
| `FIRECRAWL_API_KEY` or `FIRECRAWL_API_URL` (or the Nous Tool Gateway is ready) | firecrawl |
| `SEARXNG_URL` | searxng |
| `BRAVE_SEARCH_API_KEY` | brave-free |
| `ddgs` package importable | ddgs |
| *(nothing set at all)* | keyless ring: exa / parallel / firecrawl / keenable (round-robin) |
**Keyless free-tier ring:** when *no* credential above is present, requests rotate across the ring vendors' public free tiers (Exa, Parallel, Firecrawl, Keenable) so web tools work on a fresh install with zero setup — and a rate-limited request fails over to the next vendor in the ring automatically. Pin one vendor in `hermes tools` to stop the rotation (the ring is then only used as failover succession on throttles). All free tiers are vendor-rate-limited under burst load; sustained normal usage goes through fine. Set `web.keyless_fallback: false` to turn the tier off — with it off and no credentials, web tools are unavailable until a provider is configured.
**One-shot keyless rescue for keyed backends:** when your chosen/keyed backend fails a call (bad key, outage, upstream 5xx), that single call automatically retries on the keyless free-tier ring instead of erroring — the result notes which vendor served it and why (`rescued_from` / `backend_error`). The failover is never sticky: the very next `web_search`/`web_extract` call attempts your chosen backend again. Disable with `web.keyless_rescue: false` (also off whenever `keyless_fallback` is off).
xAI Web Search is **not** in the auto-detection chain — having `XAI_API_KEY` set (or being signed in via xAI Grok OAuth) does not automatically route web traffic through xAI, since those credentials are also used for inference / TTS / image gen and the user may want a different backend for web. Opt in explicitly with `web.backend: "xai"`.
---
## Verify your setup
Run `hermes setup` to see which web backend is detected:
```
✅ Web Search & Extract (searxng)
```
Or check via the CLI:
```bash
# Activate the venv and run the web tools module directly
source ~/.hermes/hermes-agent/.venv/bin/activate
python -m tools.web_tools
```
This prints the active backend and its status:
```
✅ Web backend: searxng
Using SearXNG (search only): http://localhost:8888
```
---
## Troubleshooting
### `web_search` returns `{"success": false}`
- Check `SEARXNG_URL` is reachable: `curl -s "http://localhost:8888/search?q=test&format=json"`
- If you get HTTP 403, JSON format is disabled — add `json` to the `formats` list in `settings.yml` and restart
- If you get a connection error, the container may not be running: `docker ps | grep searxng`
### `web_extract` says "search-only backend"
SearXNG cannot extract URL content. Set `web.extract_backend` to a provider that supports extraction:
```yaml
web:
search_backend: "searxng"
extract_backend: "firecrawl" # or tavily / keenable / exa / parallel
```
### SearXNG returns 0 results
Some public instances disable certain search engines or categories. Try:
- A different query
- A different public instance from [searx.space](https://searx.space/)
- Self-hosting your own instance for reliable results
### Rate limited on a public instance
Switch to a self-hosted instance (see [Option A](#option-a--self-host-with-docker-recommended) above). With Docker, your own instance has no rate limits.
### `web_extract` returns truncated content with a `[TRUNCATED]` footer
That's expected for pages over the character budget. The footer names the on-disk file holding the full clean text and the exact `read_file` call to page through the omitted middle. To see more inline, raise `web.extract_char_limit` in `config.yaml` or pass a larger `char_limit` on the call.
---
## Optional skill: `searxng-search`
For agents that need to use SearXNG via `curl` directly (e.g. as a fallback when the web toolset isn't available), install the `searxng-search` optional skill:
```bash
hermes skills install official/research/searxng-search
```
This adds a skill that teaches the agent how to:
- Call the SearXNG JSON API via `curl` or Python
- Filter by category (`general`, `news`, `science`, etc.)
- Handle pagination and error cases
- Fall back gracefully when SearXNG is unreachable
website/docs/user-guide/features/x-search.md
+167
View File
@@ -0,0 +1,167 @@
---
title: X (Twitter) Search
description: Search X (Twitter) posts and threads from within the agent using xAI's built-in x_search Responses tool — works with either a SuperGrok OAuth login or an XAI_API_KEY.
sidebar_label: X (Twitter) Search
sidebar_position: 7
---
# X (Twitter) Search
The `x_search` tool lets the agent search X (Twitter) posts, profiles, and threads directly. It's backed by xAI's built-in `x_search` tool on the Responses API at `https://api.x.ai/v1/responses` — Grok itself runs the search server-side and returns synthesized results with citations to the originating posts.
**Use this instead of `web_search`** when you specifically want current discussion, reactions, or claims **on X**. For general web pages, keep using `web_search` / `web_extract`.
## `x_search` vs `xurl`
Hermes can expose two different X surfaces:
| Surface | Use it for | Do not use it for |
|---------|------------|-------------------|
| `x_search` | Read-only public X discovery: current discussion, reactions, claims, profiles, threads, and synthesized answers with citations. | Posting, replying, liking, DMs, media upload, deleting, or proving that an authenticated X account changed state. |
| `xurl` skill | Exact or authenticated X API work: `post`, `reply`, `read`, `like`, `dm`, timelines, mentions, media upload, account-specific reads, and raw v2 endpoints. | Broad Grok-synthesized public X research when `x_search` is available and no authenticated account context is needed. |
For mixed workflows, use `x_search` to discover candidate public posts, then switch to `xurl read` or another exact `xurl` command after the target post/user/action is clear. Any state-changing X action must be confirmed by `xurl` output or the X API response; an `x_search` answer is never evidence that a write happened.
:::tip
If you're paying Portal for an xAI model anyway, Live Search calls bill against the same xAI key configured for chat. See [Nous Portal](/integrations/nous-portal).
:::
## Authentication
`x_search` registers when **either** xAI credential path is available:
| Credential | Source | Setup |
|------------|--------|-------|
| **SuperGrok / X Premium+ OAuth** | Browser login at `accounts.x.ai`, refreshed automatically | `hermes auth add xai-oauth` — see [xAI Grok OAuth (SuperGrok / X Premium+)](../../guides/xai-grok-oauth.md) |
| **`XAI_API_KEY`** (preferred) | Paid xAI API key | Set in `~/.hermes/.env` |
Both hit the same endpoint with the same payload — the only difference is the bearer token. **When both are configured, the explicit `XAI_API_KEY` wins** — the subscription OAuth bearer authorizes `/v1/responses` but answers x_search in a degraded Grok explanatory mode with no citations, while the API key returns real posts. Note this means x_search runs against metered API billing when a key is set; remove `XAI_API_KEY` to fall back to your subscription quota (with the degraded-answer caveat).
The tool's `check_fn` runs the xAI credential resolver every time the model's tool list is rebuilt. A `True` return means the bearer is fetchable AND non-empty AND (if it had expired) successfully refreshed. Revoked tokens with a failed refresh hide the tool from the schema; the model simply can't see it.
## Enabling the tool
Auto-enables when xAI credentials (OAuth token or `XAI_API_KEY`) are present. Disable explicitly via `hermes tools` → Search → x_search if you don't want this.
```bash
hermes tools
# → 🐦 X (Twitter) Search (press space to toggle on)
```
The picker offers two credential choices:
1. **xAI Grok OAuth (SuperGrok / Premium+)** — opens the browser to `accounts.x.ai` if you're not already logged in
2. **xAI API key** — prompts for `XAI_API_KEY`
Either choice satisfies the gating. You can pick whichever credentials you already have; the tool works identically with both. If both end up configured, OAuth is preferred at call time.
## Configuration
```yaml
# ~/.hermes/config.yaml
x_search:
# xAI model used for the Responses call.
# grok-4.5 is the recommended default; any Grok model
# with x_search tool access works.
model: grok-4.5
# Optional reasoning effort: low, medium, high, or xhigh. When omitted,
# the selected model's default applies. xhigh is supported only by
# models that document it, such as grok-4.20-multi-agent.
# reasoning_effort: low
# Request timeout in seconds. x_search can take 60–120s for
# complex queries — the default is generous. Minimum: 30.
timeout_seconds: 180
# Number of automatic retries on 5xx / ReadTimeout / ConnectionError.
# Each retry backs off (1.5x attempt seconds, capped at 5s).
retries: 2
```
`reasoning_effort` is sent to the xAI Responses API as
`reasoning: {effort: ...}`. Leave it unset for models that do not support
configurable reasoning. Invalid values fail before an API request is made.
## Tool parameters
The agent calls `x_search` with these arguments:
| Parameter | Type | Description |
|-----------|------|-------------|
| `query` | string (required) | What to look up on X. |
| `allowed_x_handles` | string array | Optional list of handles to include **exclusively** (max 10). Leading `@` is stripped. |
| `excluded_x_handles` | string array | Optional list of handles to exclude (max 10). Mutually exclusive with `allowed_x_handles`. |
| `from_date` | string | Optional `YYYY-MM-DD` start date. |
| `to_date` | string | Optional `YYYY-MM-DD` end date. |
| `enable_image_understanding` | boolean | Ask xAI to analyze images attached to matching posts. |
| `enable_video_understanding` | boolean | Ask xAI to analyze videos attached to matching posts. |
The tool returns JSON with:
- `answer` — synthesized text response from Grok
- `citations` — citations returned by the Responses API top-level field
- `inline_citations` — `url_citation` annotations extracted from the message body (each with `url`, `title`, `start_index`, `end_index`)
- `degraded` — `true` when any narrowing filter (`allowed_x_handles`, `excluded_x_handles`, `from_date`, `to_date`) was set AND both citation channels came back empty. In that case the `answer` was synthesized from the model's own knowledge rather than the X index, so treat it as unsourced. `false` otherwise (including the "no filters set" case — a broad unsourced answer is just an answer, not a filter miss)
- `degraded_reason` — short string naming which filters were active, or `null` when `degraded` is `false`
- `credential_source` — `"xai-oauth"` if OAuth resolved, `"xai"` if API key resolved
- `model`, `query`, `provider`, `tool`, `success`
### Date validation
`from_date` / `to_date` are validated client-side before the HTTP call:
- Both, if provided, must parse as `YYYY-MM-DD`.
- When both are set, `from_date` must be on or before `to_date`.
- `from_date` must not be later than today UTC — no posts can exist in a window that hasn't started yet, so the call would be guaranteed to return zero citations.
- `to_date` in the future is allowed (callers may legitimately request "from yesterday to tomorrow" to catch posts as they arrive).
Validation failures surface as a structured `{"error": "..."}` tool result, never as an HTTP call to xAI.
## Example
Talking to the agent:
> What are people on X saying about the new Grok image features? Focus on responses from @xai.
The agent will:
1. Call `x_search` with `query="reactions to new Grok image features"`, `allowed_x_handles=["xai"]`
2. Get back a synthesized answer plus a list of citations linking to specific posts
3. Reply with the answer and references
If the next user request is "reply to the best one" or "like that post", the agent should switch to the `xurl` skill, confirm the exact target post, and use the X API action. `x_search` remains a discovery tool.
## Troubleshooting
### "No xAI credentials available"
The tool surfaces this when both auth paths fail. Either set `XAI_API_KEY` in `~/.hermes/.env` or run `hermes auth add xai-oauth` and complete the browser login. Then restart your session so the agent re-reads the tool registry.
### "`x_search` is not enabled for this model"
The configured `x_search.model` doesn't have access to the server-side `x_search` tool. Switch to `grok-4.5` (the default) or another Grok model that supports it. Check the [xAI documentation](https://docs.x.ai/) for the current list.
### Tool doesn't appear in the schema
Two possible causes:
1. **Toolset not enabled.** Run `hermes tools` and confirm `🐦 X (Twitter) Search` is checked.
2. **No xAI credentials.** The check_fn returns False, so the schema stays hidden. Run `hermes auth status` to confirm xai-oauth login state, and check that `XAI_API_KEY` is set (if you're using the API-key path).
### `degraded: true` — answer with no citations
When you used `allowed_x_handles`, `excluded_x_handles`, or a date range and the response comes back with `degraded: true`, xAI's X index returned no matching posts but Grok still produced a synthesized answer from its own training data. The answer is unsourced — do not treat it as a real X result.
Causes worth checking:
- **Typo in the handle.** Strip the `@`, double-check spelling, and confirm the account exists.
- **Date range too narrow** or sliding past today's posts; widen and retry.
- **xAI index gap.** Some active accounts intermittently fail to surface in `x_search` even when they post regularly. Retry after a few minutes, or use the `xurl` skill for direct X API reads when you need an exact handle's timeline.
## See Also
- [xAI Grok OAuth (SuperGrok / Premium+)](../../guides/xai-grok-oauth.md) — the OAuth setup guide
- [xurl skill](../skills/bundled/social-media/social-media-xurl.md) — official X API CLI for authenticated account actions
- [Web Search & Extract](web-search.md) — for general (non-X) web search
- [Tools Reference](../../reference/tools-reference.md) — full tool catalog
website/docs/user-guide/git-worktrees.md
+196
View File
@@ -0,0 +1,196 @@
---
sidebar_position: 3
sidebar_label: "Git Worktrees"
title: "Git Worktrees"
description: "Run multiple Hermes agents safely on the same repository using git worktrees and isolated checkouts"
---
# Git Worktrees
Hermes Agent is often used on large, long‑lived repositories. When you want to:
- Run **multiple agents in parallel** on the same project, or
- Keep experimental refactors isolated from your main branch,
Git **worktrees** are the safest way to give each agent its own checkout without duplicating the entire repository.
This page shows how to combine worktrees with Hermes so each session has a clean, isolated working directory.
## Why Use Worktrees with Hermes?
Hermes treats the **current working directory** as the project root:
- CLI: the directory where you run `hermes` or `hermes chat`
- Messaging gateways: the directory set by `terminal.cwd` in `~/.hermes/config.yaml`
If you run multiple agents in the **same checkout**, their changes can interfere with each other:
- One agent may delete or rewrite files the other is using.
- It becomes harder to understand which changes belong to which experiment.
With worktrees, each agent gets:
- Its **own branch and working directory**
- Its **own Checkpoint Manager history** for `/rollback`
See also: [Checkpoints and /rollback](./checkpoints-and-rollback.md).
## Quick Start: Creating a Worktree
### From inside a session: `/worktree new`
The fastest path (inspired by Copilot CLI's `/worktree new`): from an
interactive CLI session, run
```
/worktree new my-experiment
```
Hermes creates `.worktrees/my-experiment/` inside the repo (branch
`hermes/my-experiment`, based on the freshly-fetched remote tip unless
`worktree_sync: false`), and retargets the session's terminal and file tools
into it — no restart needed. Omit the name to get a random `hermes-<id>`
tree. `/worktree` alone shows the active tree; `/worktree list` lists all of
them. On exit the tree is kept only if it has unpushed commits, exactly like
`hermes -w`.
### Manually with git
From your main repository (containing `.git/`), create a new worktree for a feature branch:
```bash
# From the main repo root
cd /path/to/your/repo
# Create a new branch and worktree in ../repo-feature
git worktree add ../repo-feature feature/hermes-experiment
```
This creates:
- A new directory: `../repo-feature`
- A new branch: `feature/hermes-experiment` checked out in that directory
Now you can `cd` into the new worktree and run Hermes there:
```bash
cd ../repo-feature
# Start Hermes in the worktree
hermes
```
Hermes will:
- See `../repo-feature` as the project root.
- Use that directory for context files, code edits, and tools.
- Use a **separate checkpoint history** for `/rollback` scoped to this worktree.
## Running Multiple Agents in Parallel
You can create multiple worktrees, each with its own branch:
```bash
cd /path/to/your/repo
git worktree add ../repo-experiment-a feature/hermes-a
git worktree add ../repo-experiment-b feature/hermes-b
```
In separate terminals:
```bash
# Terminal 1
cd ../repo-experiment-a
hermes
# Terminal 2
cd ../repo-experiment-b
hermes
```
Each Hermes process:
- Works on its own branch (`feature/hermes-a` vs `feature/hermes-b`).
- Writes checkpoints under a different shadow repo hash (derived from the worktree path).
- Can use `/rollback` independently without affecting the other.
This is especially useful when:
- Running batch refactors.
- Trying different approaches to the same task.
- Pairing CLI + gateway sessions against the same upstream repo.
## Cleaning Up Worktrees Safely
When you are done with an experiment:
1. Decide whether to keep or discard the work.
2. If you want to keep it:
- Merge the branch into your main branch as usual.
3. Remove the worktree:
```bash
cd /path/to/your/repo
# Remove the worktree directory and its reference
git worktree remove ../repo-feature
```
Notes:
- `git worktree remove` will refuse to remove a worktree with uncommitted changes unless you force it.
- Removing a worktree does **not** automatically delete the branch; you can delete or keep the branch using normal `git branch` commands.
- Hermes checkpoint data under `~/.hermes/checkpoints/` is not automatically pruned when you remove a worktree, but it is usually very small.
## Best Practices
- **One worktree per Hermes experiment**
- Create a dedicated branch/worktree for each substantial change.
- This keeps diffs focused and PRs small and reviewable.
- **Name branches after the experiment**
- e.g. `feature/hermes-checkpoints-docs`, `feature/hermes-refactor-tests`.
- **Commit frequently**
- Use git commits for high‑level milestones.
- Use [checkpoints and /rollback](./checkpoints-and-rollback.md) as a safety net for tool‑driven edits in between.
- **Avoid running Hermes from the bare repo root when using worktrees**
- Prefer the worktree directories instead, so each agent has a clear scope.
## Using `hermes -w` (Automatic Worktree Mode)
Hermes has a built‑in `-w` flag that **automatically creates a disposable git worktree** with its own branch. You don't need to set up worktrees manually — just `cd` into your repo and run:
```bash
cd /path/to/your/repo
hermes -w
```
Hermes will:
- Create a temporary worktree under `.worktrees/` inside your repo.
- Check out an isolated branch (e.g. `hermes/hermes-<hash>`).
- Run the full CLI session inside that worktree.
This is the easiest way to get worktree isolation. You can also combine it with a single query:
```bash
hermes -w -z "Fix issue #123"
```
For parallel agents, open multiple terminals and run `hermes -w` in each — every invocation gets its own worktree and branch automatically.
## Putting It All Together
- Use **git worktrees** to give each Hermes session its own clean checkout.
- Use **branches** to capture the high‑level history of your experiments.
- Use **checkpoints + `/rollback`** to recover from mistakes inside each worktree.
This combination gives you:
- Strong guarantees that different agents and experiments do not step on each other.
- Fast iteration cycles with easy recovery from bad edits.
- Clean, reviewable pull requests.
## Developing the UI surfaces across worktrees
The TypeScript surfaces (`ui-tui/`, `apps/desktop/`) each need a `node_modules`, which a fresh `npm ci` per worktree duplicates across every branch. If you hack on the TUI or desktop app from multiple worktrees, see [TUI & Desktop from Worktrees](../developer-guide/worktree-ui-dev.md) for the `htui` / `hgui` helpers that share one install by symlink.
website/docs/user-guide/import-from-other-agents.md
+54
View File
@@ -0,0 +1,54 @@
---
sidebar_position: 9
title: "Import from Other Agents"
description: "One-command import of a Claude Code (~/.claude) or OpenAI Codex CLI (~/.codex) setup into Hermes — instructions, allowlists, MCP servers, skills, and memories."
---
# Import from Other Agents
`hermes import-agent` imports your existing **Claude Code** or **OpenAI Codex CLI** setup into Hermes with one command. It follows the same preview-first pattern as [`hermes claw migrate`](../guides/migrate-from-openclaw.md): you always see a per-item plan before anything is written, and `--dry-run` never touches disk.
```bash
hermes import-agent # auto-detect ~/.claude or ~/.codex
hermes import-agent claude-code # import from ~/.claude
hermes import-agent codex # import from ~/.codex
hermes import-agent claude-code --dry-run # preview only
hermes import-agent codex --source /path/to/.codex # custom location
hermes import-agent claude-code --overwrite --yes # replace conflicts, skip prompts
```
## What gets imported
### Claude Code (`~/.claude`)
| Claude Code | Hermes |
|---|---|
| `CLAUDE.md` (global instructions) | Memory entries in `~/.hermes/memories/MEMORY.md` |
| `settings.json` → `permissions.allow` (`Bash(...)` rules) | `command_allowlist` in `config.yaml` |
| `settings.json` → `permissions.deny` (`Bash(...)` rules) | `approvals.deny` in `config.yaml` |
| `mcpServers` (from `~/.claude.json` and `settings.json`) | `mcp_servers` in `config.yaml` |
| `skills/<name>/` (dirs with `SKILL.md`) | `~/.hermes/skills/claude-code-imports/<name>/` |
| `commands/*.md` (slash commands) | Skipped with a note — convert them into skills |
Claude's `Bash(npm run test:*)` prefix rules become `npm run test*` globs. Non-`Bash` permission rules (`Read(...)`, `WebFetch`, ...) gate Claude-specific tools and are reported as unmapped rather than imported.
### Codex CLI (`~/.codex`)
| Codex CLI | Hermes |
|---|---|
| `AGENTS.md` (global instructions) | Memory entries in `~/.hermes/memories/MEMORY.md` |
| `config.toml` → `[mcp_servers.*]` | `mcp_servers` in `config.yaml` |
| `memories/*.md` | Memory entries in `~/.hermes/memories/MEMORY.md` |
| `skills/<name>/` (dirs with `SKILL.md`) | `~/.hermes/skills/codex-imports/<name>/` |
## What is never imported
**API keys and credentials.** Credential files (`~/.claude/.credentials.json`, `~/.codex/auth.json`) are never read, and MCP server environment variables or headers with secret-looking names (`*_TOKEN`, `*_API_KEY`, `Authorization`, ...) are stripped and listed in the report so you can re-add them deliberately. Run `hermes setup` to configure providers, or add secrets to `~/.hermes/.env`.
## Behavior notes
- **Preview first, always.** The command prints the full plan before applying; in non-interactive sessions it stops at the preview unless you pass `--yes`.
- **Merges, not replaces.** Memory entries are deduplicated against your existing `MEMORY.md`; allowlist/denylist patterns merge with what's already in `config.yaml`.
- **Conflicts are skipped by default.** An MCP server or skill that already exists in Hermes is reported as a conflict; pass `--overwrite` to replace it.
- **Malformed files don't abort the run.** A broken `settings.json` or `config.toml` becomes a per-item error in the report while everything else still imports.
- Coming from OpenClaw instead? Use [`hermes claw migrate`](../guides/migrate-from-openclaw.md).
website/docs/user-guide/local-models.md
+134
View File
@@ -0,0 +1,134 @@
---
sidebar_position: 4
title: Local Models
description: Run models entirely on your own machine — no account, no API key, nothing leaves your computer.
---
# Local Models
Hermes can run open models entirely on your own machine. It downloads and
manages the inference engine (llama.cpp), picks the right build of each
model for your hardware, and handles memory so you never configure context
sizes, GPU layers, or quantization. You pick a model; Hermes does the rest.
Nothing leaves your computer: no account, no API key, and no network access
after a model is downloaded.
## Getting started
1. Open **Settings → Providers → Local Models** (or choose **Run models
locally** during onboarding).
2. Click **Install runtime**. Hermes downloads the official llama.cpp
build for your hardware (a few hundred MB), verifies it, and keeps it
updated.
3. Pick a model from the catalog and click **Download**.
4. Click **Use**. New chats now run on the local model.
That's the whole flow. The server starts and stops with Hermes, restarts
survive app restarts, and switching back to a cloud provider is one click
in the model picker.
## How Hermes picks what to download
Every model in the catalog is priced against **your machine** before you
download anything. Each row shows:
- **Memory fit** — green (*Fits your GPU*: runs entirely in GPU memory),
amber (*Uses system RAM*: works, but slower), or red (*Too big for this
machine*).
- **Context** — the window the model starts with and the maximum it can
grow to.
- The download size of the build selected for your hardware.
Models ship in several quality grades (quantizations). Hermes picks the
highest-quality build that runs fully on your GPU; machines with less
memory get a more compact build of the same model with the same
guarantees. Below 4-bit the quality loss is too severe, so Hermes never
offers builds smaller than that — a machine that can't run the 4-bit
build spilled to system RAM simply can't run that model.
Models that don't fit stay visible with the reason, so you always know
what a hardware upgrade would unlock.
## How memory management works
Local models live or die by memory placement, so Hermes manages it
end-to-end and exposes no knobs:
- **Models start at a context window that fully fits your GPU** and grow
toward their native maximum as your conversation needs more room. You
may see "Context window grown" in the status feed during long sessions
— that's the window expanding, not an error.
- **Every recommended model gets at least a 64K context window.** When a
model is larger than your GPU's memory, Hermes deliberately places the
overflow in system RAM in the order that hurts least (expert weights
first, never the attention cache), trading some speed to protect the
context guarantee.
- **Conversation compression only kicks in at the model's maximum
window** — growth always comes first.
- Idle models are unloaded after 15 minutes to free GPU memory; they
reload automatically on the next message.
## The status bar
Right-click the status bar and enable **System resources** to see live GPU
utilization, GPU memory, and RAM while local models run. The context meter
always reflects the window the model is actually running with.
## Finding more models
The catalog is a curated starting point, not a boundary. The **Find more
models** section on the same page searches all of Hugging Face:
- Results show download counts and a per-file fit check sized to your
machine, so you know before downloading whether a build runs fully on
your GPU.
- Anything you download behaves exactly like a catalog model — Hermes
reads the model file itself to pick its context window and memory
placement. The only difference: community models don't carry our
"validated" testing badge.
- Already have a `.gguf` file on disk? **Add model file** links it into
your library without copying it (the original stays where it is), and
it's usable immediately.
## Using your own llama-server
If a llama-server is already running on your machine, Hermes detects it
and uses it instead of starting its own. Point a custom endpoint at any
OpenAI-compatible server for full manual control — the managed runtime is
a default, not a requirement. For manual setups (Ollama, MLX, custom
builds, headless CLI machines), see
[Run Hermes Locally with Ollama](/guides/local-ollama-setup) and
[Run Local LLMs on Mac](/guides/local-llm-on-mac).
## Configuration
The managed runtime is controlled by the `local_runtime` section of
`config.yaml`. The desktop UI writes these values for you; they're
documented for CLI and headless use:
```yaml
local_runtime:
enabled: false # true = start the managed server with Hermes.
# The desktop "Use" button sets this automatically.
backend: auto # auto | cuda | metal | vulkan | hip | cpu
tag: b10362 # pinned llama.cpp release; Hermes updates it with
# each release after re-validation
```
Models and runtime builds live under the Hermes home directory
(`models/` and `runtimes/llamacpp/`). Selecting a local model as your
main model uses the standard `model.provider: llamacpp` +
`model.default` settings — the same shape as every other provider.
## Requirements and limits
- **Windows and Linux:** NVIDIA GPU (CUDA) or CPU. **macOS:** Apple
Silicon (Metal). Vulkan builds serve AMD GPUs.
- A GPU with 8 GB+ of memory runs the small catalog models comfortably;
16 GB+ runs the 27–35B models at high quality.
- Model downloads are byte-size checked against the catalog during the
transfer; an incomplete download is deleted and reported, never
half-used. (Only the runtime engine zips are SHA-256 verified.)
- Deleting a model removes every file it staged, including vision
adapters and speculative-decoding companions.
website/docs/user-guide/managed-scope.md
+157
View File
@@ -0,0 +1,157 @@
---
sidebar_position: 3
title: "Managed Scope"
description: "Administrator-pinned, user-immutable config and secrets via a system-level managed directory"
---
# Managed Scope
**Managed scope** lets an administrator push a baseline of configuration and
secrets that a standard (non-root) user **cannot override**. It is intended for
fleet/org deployments where IT needs to pin, for example, the model provider, a
shared API base URL, or `security.redact_secrets: true` across every user on a
machine.
When a managed scope is present, the values it specifies win over the user's
`~/.hermes/config.yaml`, `~/.hermes/.env`, and even the shell environment — for
exactly the keys it pins. Everything else stays fully user-controlled.
:::note Different from a package-manager–locked install
A package-manager–managed install (declarative-distro / formula) blocks *all*
config mutation and tells you to use your package manager. Managed scope is a
separate mechanism: it injects *specific immutable values* on a per-key basis
rather than locking the whole config. The two are independent and can coexist.
:::
## Where it lives
Managed scope is read from a system-level directory, default `/etc/hermes`:
```text
/etc/hermes/
├── config.yaml # managed config layer (wins over ~/.hermes/config.yaml)
└── .env # managed env layer (wins over ~/.hermes/.env + shell)
```
The directory and files are owned by `root` (directory mode `0755`, files
`0644`): readable by everyone, writable only by an administrator. **That
filesystem permission is the enforcement mechanism** — a standard user can read
the managed files but cannot edit them.
Either file is optional. A missing managed directory or missing file simply
means "no managed scope," and configuration resolves exactly as it does without
the feature.
### Relocating the directory
The location can be relocated with the `HERMES_MANAGED_DIR` environment variable
(for containers or non-`/etc` deployments). This is a deployment/bootstrap path
knob — like `HERMES_HOME` — set by the same administrator who owns the managed
files. It is **never persisted** to any `.env` by Hermes.
```bash
# Point managed scope at a custom directory (set by IT / the deployment, not the user)
export HERMES_MANAGED_DIR=/opt/org/hermes-policy
```
:::warning
A user who can set `HERMES_MANAGED_DIR` can repoint managed scope at a directory
they control, defeating it. In a real deployment this variable should be fixed
by the administrator (e.g. baked into the service unit / container image), not
left user-settable. `hermes doctor` reports the *resolved* managed directory so
a redirect is visible.
:::
## Precedence
For the keys a managed layer specifies, the order is (highest wins):
| Tier | config.yaml | .env |
|---|---|---|
| 1 | `/etc/hermes/config.yaml` (managed) | `/etc/hermes/.env` (managed) |
| 2 | `~/.hermes/config.yaml` (user) | `~/.hermes/.env` (user) |
| 3 | built-in defaults | pre-existing shell environment |
Merging is **leaf-level**: pinning `model.default` does not freeze the rest of
`model.*`. A managed `config.yaml` of:
```yaml
model:
default: org/standard-model
```
forces `model.default` for every user while leaving `model.fallback` (and every
other key) under user control.
:::note Precedence note
For the keys it pins, managed scope deliberately wins over the shell environment
too — otherwise it would not be "managed." This is the one place that inverts the
usual "an environment variable overrides config.yaml" rule, and it applies only
to the specific keys the managed layer specifies.
:::
## Seeing what's managed
```bash
hermes config # shows a header naming the managed source + the pinned keys
hermes doctor # reports the resolved managed dir + pinned key counts
```
If you try to change a managed value, Hermes refuses and names the source:
```bash
$ hermes config set model.default my/model
Cannot set 'model.default': it is managed by your administrator
(/etc/hermes/config.yaml) and cannot be changed.
```
The same applies to managed secrets — `hermes config set` / setup will not write
a user value for an env key pinned by the managed `.env`.
## Setting up a managed scope (administrators)
```bash
sudo mkdir -p /etc/hermes
# Pin some config values for every user on this machine
sudo tee /etc/hermes/config.yaml >/dev/null <<'YAML'
model:
provider: nous
security:
redact_secrets: true
YAML
# Optionally pin a shared, non-sensitive env value
sudo tee /etc/hermes/.env >/dev/null <<'ENV'
OPENAI_API_BASE=https://inference.example.com/v1
ENV
sudo chmod 0755 /etc/hermes
sudo chmod 0644 /etc/hermes/config.yaml /etc/hermes/.env
```
Changes take effect on the next Hermes start (a malformed managed file is logged
loudly and ignored — it never blocks startup, but the admin should check
`hermes doctor` to confirm the policy is being applied).
## Security model and limitations (v1)
- **Enforcement is filesystem permissions only.** If a user has write access to
the managed directory (or runs Hermes as `root`), managed scope is advisory.
- **The managed `.env` is world-readable** (`0644`), so any local user can read
secrets pushed through it. Use it for shared, non-sensitive values (an org API
base URL, feature defaults) rather than high-sensitivity secrets.
- **The agent's own tools are not hard-blocked from a managed *env* value.** A
managed environment variable is applied at startup, but nothing stops the
agent from setting a different value inside its own subprocess shell. v1 is a
management-convenience boundary against a normal user, not an un-escapable
sandbox.
The following are intentionally **out of scope for v1** and may come later:
- A hard boundary that the agent itself cannot escape.
- Native managed locations on macOS and Windows (v1 is Linux/POSIX-first).
- Drop-in fragment directories (`managed.d/`) for layered policy.
- Signed / integrity-checked managed files.
- Remote / device-management (MDM) delivery.
- Tighter (group-scoped) permissions for managed secrets.
website/docs/user-guide/messaging/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "Messaging Gateway",
"position": 3,
"link": {
"type": "doc",
"id": "user-guide/messaging/index"
}
}
website/docs/user-guide/messaging/a2a.md
+130
View File
@@ -0,0 +1,130 @@
# A2A (Agent-to-Agent)
[A2A](https://a2a-protocol.org) is the open Agent2Agent protocol (v1.0, stewarded by the Linux Foundation) for communication between independent AI agents. The Hermes A2A plugin works in **both directions**: your agent can call other A2A agents as tools, and other agents can send tasks to your Hermes over HTTP.
It interoperates with any A2A-compliant peer — another Hermes, LangChain, CrewAI, Google ADK agents, or anything built on the official `a2a-sdk`.
## When to use A2A
- **Hermes ↔ Hermes across machines** — let your desktop agent hand tasks to a Hermes on a server, or vice versa, each with its own memory, tools, and credentials.
- **Delegating to specialist agents** — a peer that advertises `web_search`/`research`/`coding` skills on its Agent Card can be discovered and called mid-conversation.
- **Being a callable service** — expose your Hermes so other frameworks' agents can send it tasks.
When you want multiple agents on the **same machine**, prefer [delegation](../features/delegation.md) (in-process subagents) or the [kanban board](../features/kanban.md) (durable multi-profile work queue) — A2A is for crossing process/machine/framework boundaries.
## Enable
```bash
hermes gateway setup # pick A2A
```
Or in `~/.hermes/config.yaml`:
```yaml
gateway:
platforms:
a2a:
enabled: true
extra:
port: 9900
```
The outbound client tools ship as the `a2a` toolset, **off by default** — enable it per platform:
```bash
hermes tools enable a2a --platform cli # CLI/TUI sessions
hermes tools enable a2a --platform telegram # or any messaging platform
hermes tools enable a2a --platform a2a # let inbound A2A tasks call peers (agent chaining)
```
The tools are available in every process type — CLI, TUI, gateway, and cron — without the inbound platform needing to be enabled.
## Outbound: calling other agents
With the `a2a` toolset enabled, the agent gets:
| Tool | What it does |
|---|---|
| `a2a_discover(url)` | Fetch and summarize a peer's Agent Card |
| `a2a_call(agent, message, context_id?)` | Send a task, get the reply; multi-turn via `context_id` |
| `a2a_list()` | Configured peers, saved conversations, metrics |
| `a2a_history(context_id)` | Recall a persisted A2A conversation |
| `a2a_orchestrate(capability, message, mode?)` | Fan a task out to every peer advertising a capability (`all` / `first` / `best`) |
Configure known peers in `config.yaml`:
```yaml
a2a_agents:
researcher:
url: "http://research-box.local:9900"
auth: { type: bearer, token: "..." }
timeout: 120
capabilities: [web_search, research]
```
Then just ask: *"Ask the researcher agent to summarize today's arXiv postings."* Direct URLs work too — `a2a_call` accepts any A2A endpoint.
## Inbound: being callable
With the platform enabled, Hermes serves:
- **Agent Card** at `GET /.well-known/agent-card.json` (canonical v1.0 path; the legacy `agent.json` also answers) — advertises your agent's name, skills (derived from enabled toolsets), and auth requirements.
- **JSON-RPC 2.0** at `POST /` — canonical v1.0 methods (`SendMessage`, `SendStreamingMessage`, `GetTask`, `ListTasks`, `CancelTask`, `SubscribeToTask`, push-notification config CRUD) plus the pre-1.0 path-style aliases (`message/send`, …).
- **SSE streaming** for `SendStreamingMessage`, with spec-correct JSON-RPC-enveloped frames.
- **Push notifications** (webhooks) for long-running tasks, HMAC-SHA256 signed.
Inbound tasks are injected into a **live gateway session** — the same agent, memory, and tools that serve your other channels — and the final reply is returned to the caller as the task result. Conversations are keyed by the A2A `contextId`, so a peer can hold a multi-turn exchange.
Interoperability is verified against the official Python `a2a-sdk` (card resolution, `SendMessage`, streaming).
## Security model
Secure by default; every widening step is explicit:
- **No token ⇒ localhost only.** The server binds `127.0.0.1`. Remote exposure requires a bearer token **and** an explicit `A2A_HOST`.
- **Per-peer tokens** — `A2A_PEER_TOKENS="alice:tok1,bob:tok2"` gives each peer its own credential; the authenticated name drives rate limiting, trust, and audit.
- **Prompt-injection filtering** — inbound text is filtered and framed as untrusted peer input. Remote peers cannot invoke operator slash commands.
- **Outbound redaction** — credential-shaped strings (API keys, JWTs, tokens) are scrubbed from replies.
- **Audit log** — every exchange appends to `~/.hermes/a2a_audit.jsonl`.
- **Anti-loop** — per-context turn caps stop two agents ping-ponging forever.
## Configuration reference
| Env var | Default | Meaning |
|---|---|---|
| `A2A_PEER_TOKENS` | _(unset)_ | Per-peer credentials `name:token,…` (preferred) |
| `A2A_BEARER_TOKEN` | _(unset)_ | Shared token; identity falls back to caller IP |
| `A2A_HOST` | `127.0.0.1` | Bind host — only widens when a token is set |
| `A2A_PORT` | `9900` | Inbound port |
| `A2A_AGENT_NAME` | hostname-derived | Name on the Agent Card |
| `A2A_PUBLIC_URL` | _(unset)_ | Routable URL advertised on the card (reverse proxies / k8s) |
| `A2A_TRUSTED_PEERS` | _(unset)_ | Allow-list of authenticated identities |
| `A2A_ALLOW_ALL_USERS` | `false` | Allow any authenticated peer (dev only) |
| `A2A_RATE_LIMIT` | `60` | Requests/minute per identity |
| `A2A_MAX_PINGPONG_TURNS` | `5` | Anti-loop turn cap per context (max 20) |
| `A2A_REPLY_TIMEOUT` | `300` | Seconds to wait for the agent's reply |
| `A2A_PUSH_SECRET` | bearer token | HMAC secret for push-notification signing |
| `A2A_ADVERTISED_TOOLSETS` | all registered | Restrict which skills appear on the Agent Card |
Behind a reverse proxy or Kubernetes Service, set `A2A_PUBLIC_URL` (or rely on `X-Forwarded-Host`/`X-Forwarded-Proto`) so the Agent Card advertises a URL peers can actually call back.
## Quick test
```bash
# From another machine / agent:
curl http://your-host:9900/.well-known/agent-card.json
curl -X POST http://your-host:9900/ \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <token>' \
-d '{"jsonrpc":"2.0","id":1,"method":"SendMessage",
"params":{"message":{"messageId":"m1","role":"ROLE_USER",
"parts":[{"text":"What tools do you have?"}]}}}'
```
## Troubleshooting
- **Peers can't reach the card URL** — the card was advertising your bind address; set `A2A_PUBLIC_URL` to the externally routable URL.
- **`401 Unauthorized`** — token mismatch; check `A2A_PEER_TOKENS`/`A2A_BEARER_TOKEN` on the server and the peer's `auth:` block.
- **Server won't bind non-localhost** — by design: set a bearer token first, then `A2A_HOST=0.0.0.0`.
- **Replies time out on long tasks** — raise `A2A_REPLY_TIMEOUT`, or have the caller register a push-notification config and poll `GetTask`.
website/docs/user-guide/messaging/bluebubbles.md
+171
View File
@@ -0,0 +1,171 @@
# BlueBubbles (iMessage)
Connect Hermes to Apple iMessage via [BlueBubbles](https://bluebubbles.app/) — a free, open-source macOS server that bridges iMessage to any device.
## Prerequisites
- A **Mac** (always on) running [BlueBubbles Server](https://bluebubbles.app/)
- Apple ID signed into Messages.app on that Mac
- BlueBubbles Server v1.0.0+ (webhooks require this version)
- Network connectivity between Hermes and the BlueBubbles server
## Setup
### 1. Install BlueBubbles Server
Download and install from [bluebubbles.app](https://bluebubbles.app/). Complete the setup wizard — sign in with your Apple ID and configure a connection method (local network, Ngrok, Cloudflare, or Dynamic DNS).
### 2. Get your Server URL and Password
In BlueBubbles Server → **Settings → API**, note:
- **Server URL** (e.g., `http://192.168.1.10:1234`)
- **Server Password**
### 3. Configure Hermes
Run the setup wizard:
```bash
hermes gateway setup
```
Select **BlueBubbles (iMessage)** and enter your server URL and password.
Or set environment variables directly in `~/.hermes/.env`:
```bash
BLUEBUBBLES_SERVER_URL=http://192.168.1.10:1234
BLUEBUBBLES_PASSWORD=your-server-password
```
#### Optional: Require mentions in group chats
By default, Hermes responds to every authorized BlueBubbles/iMessage DM or group message. To make group chats opt-in, enable mention gating:
```yaml
platforms:
bluebubbles:
enabled: true
extra:
require_mention: true
```
With `require_mention: true`, DMs still work normally, but group-chat messages are ignored unless they match a mention pattern. If you do not configure custom patterns, Hermes uses conservative defaults for `Hermes` and `@Hermes agent` variants.
For a custom agent name, set regex patterns:
```yaml
platforms:
bluebubbles:
extra:
require_mention: true
mention_patterns:
- '(?<![\w@])@?amos\b[,:\-]?'
```
### 4. Authorize Users
Choose one approach:
**DM Pairing (recommended):**
When someone messages your iMessage, Hermes automatically sends them a pairing code. Approve it with:
```bash
hermes pairing approve bluebubbles <CODE>
```
Use `hermes pairing list` to see pending codes and approved users.
**Pre-authorize specific users** (in `~/.hermes/.env`):
```bash
BLUEBUBBLES_ALLOWED_USERS=user@icloud.com,+15551234567
```
**Open access** (in `~/.hermes/.env`):
```bash
BLUEBUBBLES_ALLOW_ALL_USERS=true
```
### 5. Start the Gateway
```bash
hermes gateway run
```
Hermes will connect to your BlueBubbles server, register a webhook, and start listening for iMessage messages.
## How It Works
```
iMessage → Messages.app → BlueBubbles Server → Webhook → Hermes
Hermes → BlueBubbles REST API → Messages.app → iMessage
```
- **Inbound:** BlueBubbles sends webhook events to a local listener when new messages arrive. No polling — instant delivery.
- **Outbound:** Hermes sends messages via the BlueBubbles REST API.
- **Media:** Images, voice messages, videos, and documents are supported in both directions. Inbound attachments are downloaded and cached locally for the agent to process.
## Environment Variables
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `BLUEBUBBLES_SERVER_URL` | Yes | — | BlueBubbles server URL |
| `BLUEBUBBLES_PASSWORD` | Yes | — | Server password |
| `BLUEBUBBLES_WEBHOOK_HOST` | No | `127.0.0.1` | Webhook listener bind address |
| `BLUEBUBBLES_WEBHOOK_PORT` | No | `8645` | Webhook listener port |
| `BLUEBUBBLES_WEBHOOK_PATH` | No | `/bluebubbles-webhook` | Webhook URL path |
| `BLUEBUBBLES_HOME_CHANNEL` | No | — | Phone/email for cron delivery |
| `BLUEBUBBLES_ALLOWED_USERS` | No | — | Comma-separated authorized users |
| `BLUEBUBBLES_ALLOW_ALL_USERS` | No | `false` | Allow all users |
| `BLUEBUBBLES_REQUIRE_MENTION` | No | `false` | Require a mention pattern before responding in group chats |
| `BLUEBUBBLES_MENTION_PATTERNS` | No | Hermes wake words | JSON array, newline-separated, or comma-separated regex patterns for group mention matching |
Auto-marking messages as read is controlled by the `send_read_receipts` key under `platforms.bluebubbles.extra` in `~/.hermes/config.yaml` (default: `true`). There is no corresponding environment variable.
## Features
### Text Messaging
Send and receive iMessages. Markdown is automatically stripped for clean plain-text delivery.
### Rich Media
- **Images:** Photos appear natively in the iMessage conversation
- **Voice messages:** Audio files sent as iMessage voice messages
- **Videos:** Video attachments
- **Documents:** Files sent as iMessage attachments
### Tapback Reactions
Love, like, dislike, laugh, emphasize, and question reactions. Requires the BlueBubbles [Private API helper](https://docs.bluebubbles.app/helper-bundle/installation).
### Typing Indicators
Shows "typing..." in the iMessage conversation while the agent is processing. Requires Private API.
### Read Receipts
Automatically marks messages as read after processing. Requires Private API.
### Chat Addressing
You can address chats by email or phone number — Hermes resolves them to BlueBubbles chat GUIDs automatically. No need to use raw GUID format.
## Private API
Some features require the BlueBubbles [Private API helper](https://docs.bluebubbles.app/helper-bundle/installation):
- Tapback reactions
- Typing indicators
- Read receipts
- Creating new chats by address
Without the Private API, basic text messaging and media still work.
## Troubleshooting
### "Cannot reach server"
- Verify the server URL is correct and the Mac is on
- Check that BlueBubbles Server is running
- Ensure network connectivity (firewall, port forwarding)
### Messages not arriving
- Check that the webhook is registered in BlueBubbles Server → Settings → API → Webhooks
- Verify the webhook URL is reachable from the Mac
- Check `hermes logs gateway` for webhook errors (or `hermes logs -f` to follow in real-time)
### "Private API helper not connected"
- Install the Private API helper: [docs.bluebubbles.app](https://docs.bluebubbles.app/helper-bundle/installation)
- Basic messaging works without it — only reactions, typing, and read receipts require it
website/docs/user-guide/messaging/buzz.md
+162
View File
@@ -0,0 +1,162 @@
# Buzz
The Buzz adapter connects Hermes to a [Buzz](https://github.com/block/buzz) community — Block's open-source human+agent collaboration platform built on the Nostr protocol — and relays messages between Buzz channels (or DMs) and the agent. Outbound traffic shells out to the `buzz` CLI binary ("JSON in, JSON out"); inbound uses a native Nostr WebSocket subscription (via the already-bundled `websockets` package) with CLI polling as fallback. **No extra Python packages are required** — just the `buzz` binary.
Buzz renders markdown, so agent replies keep their formatting. Images are delivered as uploads (local files) or links (URLs). Replies can thread onto an existing message via its event id. When progress or status messages are enabled, they inherit the triggering Buzz event as their reply anchor instead of appearing as unrelated top-level channel posts.
Files sent **to** the agent are fetched back off the relay with the agent's authenticated identity and cached locally, so tools receive a real file path rather than a `/media/…` URL that anonymous requests cannot read. Images, audio, video, and documents (PDFs and the like) are all handled.
Inbound messages arrive over a persistent NIP-42-authenticated Nostr WebSocket subscription by default (near-instant delivery), with automatic fallback to CLI polling when the WebSocket can't be established. Outbound messages always go through the `buzz` CLI. Control it with `transport` / `BUZZ_TRANSPORT`: `auto` (default), `websocket` (require WS, fail otherwise), or `poll`. If your relay membership uses NIP-OA owner attestation, set `BUZZ_AUTH_TAG` to the four-string auth tag JSON.
> Run `hermes gateway setup` and pick **Buzz** for a guided walk-through.
## Prerequisites
- The `buzz` CLI binary on your `PATH` (or point `BUZZ_CLI_PATH` at it) — build it from the [Buzz repo](https://github.com/block/buzz) with `cargo build --release -p buzz-cli`
- A Buzz community relay URL (e.g. `https://mycommunity.communities.buzz.xyz`)
- A Nostr private key (nsec or hex) whose identity is already a **member** of that community
## Configure Hermes
You can configure Buzz two ways — the `gateway` block in `config.yaml` (canonical) or environment variables (which override it). The private key is a **secret** and always belongs in `~/.hermes/.env`.
### Option A — config.yaml
```yaml
gateway:
platforms:
buzz:
enabled: true
extra:
relay_url: https://mycommunity.communities.buzz.xyz
attachment_hosts: [] # additional exact HTTPS host[:port] origins for inbound files
channels: # channel UUIDs to watch (empty = all joined)
- ccc2bc1a-7a82-5a8f-8c4e-57a070cbe7cd
home_channel: ccc2bc1a-7a82-5a8f-8c4e-57a070cbe7cd
poll_interval: 4 # seconds between inbound poll sweeps
cli_path: "" # buzz binary (default: PATH, then ~/bin/buzz)
credentials_file: "" # JSON file with the nsec (BUZZ_PRIVATE_KEY fallback)
allowed_users: [] # empty = allow all; hex pubkeys or npubs
```
Plus, in `~/.hermes/.env`:
```
BUZZ_PRIVATE_KEY=nsec1...
```
### Option B — environment variables
| Variable | Required | Description |
|----------|:--------:|-------------|
| `BUZZ_RELAY_URL` | ✅ | Base URL of the community relay |
| `BUZZ_PRIVATE_KEY` | ✅ | Nostr private key (nsec or hex) — the only secret |
| `BUZZ_CHANNELS` | — | Comma-separated channel UUIDs to watch (default: all joined channels) |
| `BUZZ_HOME_CHANNEL` | — | Channel UUID for cron / notification delivery (defaults to the first watched channel) |
| `BUZZ_ALLOWED_USERS` | — | Comma-separated npubs or hex pubkeys allowed to talk to the agent |
| `BUZZ_ALLOW_ALL_USERS` | — | Allow any community member to talk to the agent |
| `BUZZ_POLL_INTERVAL` | — | Seconds between inbound poll sweeps (default: 4) |
| `BUZZ_CLI_PATH` | — | Path to the `buzz` binary (default: `buzz` on PATH, then `~/bin/buzz`) |
| `BUZZ_CREDENTIALS_FILE` | — | JSON credentials file holding the nsec, used when `BUZZ_PRIVATE_KEY` is unset |
## Recommended default settings
When wiring up Buzz, set these defaults in `config.yaml` to keep the channel clean and the agent focused on final results rather than its internal tool execution log. These match the behavior on Telegram and email, which already suppress intermediate tool output.
```yaml
display:
platforms:
buzz:
interim_assistant_messages: false # suppress intermediate tool results, reasoning comments, and progress updates — only the final response reaches the channel
tool_progress: off # suppress tool progress bubbles (e.g., "Running terminal command...", "Reading file...")
gateway:
platforms:
buzz:
enabled: true
extra:
relay_url: https://mycommunity.communities.buzz.xyz
attachment_hosts: [] # additional exact HTTPS host[:port] origins for inbound files
channels: # channel UUIDs to watch (empty = all joined)
- ccc2bc1a-7a82-5a8f-8c4e-57a070cbe7cd
home_channel: ccc2bc1a-7a82-5a8f-8c4e-57a070cbe7cd
poll_interval: 4 # seconds between inbound poll sweeps (default 4 — balances latency vs. relay load)
cli_path: "" # buzz binary (default: PATH, then ~/bin/buzz)
credentials_file: "" # JSON file with the nsec (BUZZ_PRIVATE_KEY fallback)
allowed_users: [] # empty = allow all if allow_all_users is true; otherwise restrict to listed npubs/hex pubkeys
require_mention: true # in channels: only respond when addressed (@name, npub, or hex pubkey); DMs always dispatch regardless
allow_all_users: false # set true for community mode (everyone can chat, only owner is admin); false for private mode (only allowed_users)
```
**Why these defaults:**
- `interim_assistant_messages: false` — prevents intermediate tool results, reasoning comments, and progress updates from being posted as separate messages to the channel. Only the final response goes to the channel.
- `tool_progress: off` — suppresses tool progress bubbles (e.g., "Running terminal command...", "Reading file..."). Keeps the channel focused on actual results, not process.
- `poll_interval: 4` — balances inbound latency (up to 4s delay) against relay load. Lower values increase polling frequency; higher values reduce it.
- `allowed_users: []` + `allow_all_users: false` — private mode by default. Only listed users can interact. Set `allow_all_users: true` for community mode where everyone can chat (admin tier still restricted to the owner).
- `require_mention: true` — in channels, the agent only responds when addressed. DMs always dispatch regardless of this setting.
**Rationale:** Channels are for final results and conversation, not for the agent's internal tool execution log. Users see the final answer, not the steps taken to get there. This matches the behavior on Telegram and email, which already have these defaults.
**Exception:** If you want users to see tool progress (e.g., for long-running operations), set `tool_progress: all` — but `interim_assistant_messages` should still be `false` to avoid spamming with every tool result.
## Mentions, channels, and DMs
- In shared channels the agent only responds when **addressed** — by `@name`, its npub, or its hex pubkey. Everything else is ignored.
- Direct messages always reach the agent, no mention needed.
- The agent's own messages are never dispatched back to it (self-echo suppression by pubkey), and every event is de-duplicated by event id against a per-channel high-water mark.
## Reply threading
Replies are threaded by default: the agent's answer (and any enabled progress/status messages) is anchored to the message that triggered it. Anchoring is NIP-10 aware — when the triggering message was already **inside** a thread, the agent replies to that thread's *root*, so the answer joins the existing thread instead of nesting a new one-message sub-thread under every turn.
To post replies flat at the channel level instead, set either of these (they are equivalent; `reply_in_thread` matches the key Slack uses):
```yaml
gateway:
platforms:
buzz:
reply_to_mode: off # PlatformConfig-level, like Discord/Telegram
extra:
reply_in_thread: false # Slack-style key; env: BUZZ_REPLY_IN_THREAD
```
The opt-out applies to **all** send paths — final answers, streamed updates, interim commentary, tool-progress bubbles, and out-of-process cron delivery (`deliver=buzz`).
## Access control
By default the allow-list is empty, which means every community member who mentions the agent gets a response only if `BUZZ_ALLOW_ALL_USERS=true`; otherwise restrict access by listing npubs or hex pubkeys in `BUZZ_ALLOWED_USERS` (or `allowed_users` in config.yaml). Community membership itself is enforced by the relay — only members can post.
The allow-list also gates **inbound attachments**: relay media is fetched with the agent's own Buzz credentials, so a download only happens for a sender the gateway explicitly authorizes. A denied, missing, or failed authorization leaves the message text untouched and makes no credentialed request.
Cron jobs and notifications (`deliver=buzz`) are delivered to the **home channel** — `BUZZ_HOME_CHANNEL` if set, otherwise the first watched channel — and work even when cron runs outside the gateway process.
## Inbound attachments
Buzz messages with native NIP-94 `imeta` tags can deliver images, audio,
video, and documents to the agent. Hermes downloads attachments only after
the message has passed self-echo, addressing, and sender authorization checks.
Each file must use HTTPS and declare an exact byte size and SHA-256 digest;
redirects, URL credentials, fragments, oversized payloads, and integrity
mismatches are rejected.
The relay's own HTTPS origin is trusted automatically. If a community stores
media on another public origin, add its exact `host` or `host:port` to
`attachment_hosts` under `gateway.platforms.buzz.extra`. Non-default ports
must be listed explicitly. Protected media that requires authenticated
retrieval through the Buzz CLI is not handled by this native public-URL path.
## Run the gateway
```bash
hermes gateway start
```
Check status with `hermes gateway status` — Buzz connection state is reported there, including for env-only setups.
## Notes and limitations
- **`BUZZ_*` env vars are available in terminal tool children for Buzz sessions** — the agent can invoke the `buzz` CLI directly (e.g. `buzz messages send ...`) because `BUZZ_PRIVATE_KEY`, `BUZZ_AUTH_TAG`, `BUZZ_RELAY_URL`, and the other `BUZZ_*` variables are passed through to terminal subprocesses when the session's platform is `buzz` or the process is a Buzz Desktop managed agent (`BUZZ_MANAGED_AGENT`). Non-Buzz sessions on the same host, `execute_code`, and other non-terminal spawns remain sealed.
- **Inbound is polled, not streamed.** The `buzz` CLI is request/response, so the adapter polls `buzz messages get` per watched channel every `poll_interval` seconds (default 4). Expect up to one interval of latency on inbound messages. A future optimization is a websocket transport (the Buzz repo ships `buzz-ws-client` for true streaming).
- On (re)connect the adapter seeds its high-water mark from the newest events, so channel history is never replayed into the agent.
- New DM conversations are discovered automatically (every few poll sweeps).
- The private key is passed to the CLI via the subprocess environment — it never appears in argv or logs.
website/docs/user-guide/messaging/dingtalk.md
+283
View File
@@ -0,0 +1,283 @@
---
sidebar_position: 10
title: "DingTalk"
description: "Set up Hermes Agent as a DingTalk chatbot"
---
# DingTalk Setup
Hermes Agent integrates with DingTalk (钉钉) as a chatbot, letting you chat with your AI assistant through direct messages or group chats. The bot connects via DingTalk's Stream Mode — a long-lived WebSocket connection that requires no public URL or webhook server — and replies using markdown-formatted messages through DingTalk's session webhook API.
Before setup, here's the part most people want to know: how Hermes behaves once it's in your DingTalk workspace.
## How Hermes Behaves
| Context | Behavior |
|---------|----------|
| **DMs (1:1 chat)** | Hermes responds to every message. No `@mention` needed. Each DM has its own session. |
| **Group chats** | Hermes responds when you `@mention` it. Without a mention, Hermes ignores the message. |
| **Shared groups with multiple users** | By default, Hermes isolates session history per user inside the group. Two people talking in the same group do not share one transcript unless you explicitly disable that. |
### Session Model in DingTalk
By default:
- each DM gets its own session
- each user in a shared group chat gets their own session inside that group
This is controlled by `config.yaml`:
```yaml
group_sessions_per_user: true
```
Set it to `false` only if you explicitly want one shared conversation for the entire group:
```yaml
group_sessions_per_user: false
```
This guide walks you through the full setup process — from creating your DingTalk bot to sending your first message.
## Prerequisites
Install the required Python packages:
```bash
cd ~/.hermes/hermes-agent && uv pip install -e ".[dingtalk]"
```
Or individually:
```bash
pip install dingtalk-stream httpx alibabacloud-dingtalk
```
- `dingtalk-stream` — DingTalk's official SDK for Stream Mode (WebSocket-based real-time messaging)
- `httpx` — async HTTP client used for sending replies via session webhooks
- `alibabacloud-dingtalk` — DingTalk OpenAPI SDK for AI Cards, emoji reactions, and media downloads
## Step 1: Create a DingTalk App
1. Go to the [DingTalk Developer Console](https://open-dev.dingtalk.com/).
2. Log in with your DingTalk admin account.
3. Click **Application Development** → **Custom Apps** → **Create App via H5 Micro-App** (or **Robot** depending on your console version).
4. Fill in:
- **App Name**: e.g., `Hermes Agent`
- **Description**: optional
5. After creating, navigate to **Credentials & Basic Info** to find your **Client ID** (AppKey) and **Client Secret** (AppSecret). Copy both.
:::warning[Credentials shown only once]
The Client Secret is only displayed once when you create the app. If you lose it, you'll need to regenerate it. Never share these credentials publicly or commit them to Git.
:::
## Step 2: Enable the Robot Capability
1. In your app's settings page, go to **Add Capability** → **Robot**.
2. Enable the robot capability.
3. Under **Message Reception Mode**, select **Stream Mode** (recommended — no public URL needed).
:::tip
Stream Mode is the recommended setup. It uses a long-lived WebSocket connection initiated from your machine, so you don't need a public IP, domain name, or webhook endpoint. This works behind NAT, firewalls, and on local machines.
:::
## Step 3: Find Your DingTalk User ID
Hermes Agent uses your DingTalk User ID to control who can interact with the bot. DingTalk User IDs are alphanumeric strings set by your organization's admin.
To find yours:
1. Ask your DingTalk organization admin — User IDs are configured in the DingTalk admin console under **Contacts** → **Members**.
2. Alternatively, the bot logs the `sender_id` for each incoming message. Start the gateway, send the bot a message, then check the logs for your ID.
## Step 4: Configure Hermes Agent
### Option A: Interactive Setup (Recommended)
Run the guided setup command:
```bash
hermes gateway setup
```
Select **DingTalk** when prompted. The setup wizard can authorize via one of two paths:
- **QR-code device flow (recommended).** Scan the QR that prints in your terminal with the DingTalk mobile app — your Client ID and Client Secret are returned automatically and written to `~/.hermes/.env`. No developer-console trip needed.
- **Manual paste.** If you already have credentials (or QR scanning isn't convenient), paste your Client ID, Client Secret, and allowed user IDs when prompted.
:::note openClaw branding disclosure
Because DingTalk's `verification_uri_complete` is hardcoded to the openClaw identity at the API layer, the QR currently authorizes under an `openClaw` source string until Alibaba / DingTalk-Real-AI registers a Hermes-specific template server-side. This is purely how DingTalk presents the consent screen — the bot you create is fully yours and private to your tenant.
:::
### Option B: Manual Configuration
Add the following to your `~/.hermes/.env` file:
```bash
# Required
DINGTALK_CLIENT_ID=your-app-key
DINGTALK_CLIENT_SECRET=your-app-secret
# Security: restrict who can interact with the bot
DINGTALK_ALLOWED_USERS=user-id-1
# Multiple allowed users (comma-separated)
# DINGTALK_ALLOWED_USERS=user-id-1,user-id-2
# Optional: group-chat gating (mirrors Slack/Telegram/Discord/WhatsApp)
# DINGTALK_REQUIRE_MENTION=true
# DINGTALK_FREE_RESPONSE_CHATS=cidABC==,cidDEF==
# DINGTALK_MENTION_PATTERNS=^小马
# DINGTALK_HOME_CHANNEL=cidXXXX==
# DINGTALK_ALLOW_ALL_USERS=true
```
Optional behavior settings in `~/.hermes/config.yaml`:
```yaml
group_sessions_per_user: true
gateway:
platforms:
dingtalk:
extra:
# Require @mention in groups before the bot replies (parity with Slack/Telegram/Discord).
# DMs ignore this — the bot always replies in 1:1 chats.
require_mention: true
# Per-platform allowlist. When set, only these DingTalk user IDs can interact with the bot
# (same semantics as DINGTALK_ALLOWED_USERS, but scoped here instead of in .env).
allowed_users:
- user-id-1
- user-id-2
```
- `group_sessions_per_user: true` keeps each participant's context isolated inside shared group chats
- `require_mention: true` prevents the bot from responding to every group message — it only answers when someone @-mentions it
- `allowed_users` under `dingtalk.extra` is an alternative to `DINGTALK_ALLOWED_USERS`; set one or the other (if both are set, only users present in both lists are authorized)
### Start the Gateway
Once configured, start the DingTalk gateway:
```bash
hermes gateway
```
The bot should connect to DingTalk's Stream Mode within a few seconds. Send it a message — either a DM or in a group where it's been added — to test.
:::tip
You can run `hermes gateway` in the background or as a systemd service for persistent operation. See the deployment docs for details.
:::
## Features
### AI Cards
Hermes can reply using DingTalk AI Cards instead of plain markdown messages. Cards provide a richer, more structured display and support streaming updates as the agent generates its response.
To enable AI Cards, configure a card template ID in `config.yaml`:
```yaml
platforms:
dingtalk:
enabled: true
extra:
card_template_id: "your-card-template-id"
```
You can find your card template ID in the DingTalk Developer Console under your app's AI Card settings. When AI Cards are enabled, all replies are sent as cards with streaming text updates.
### Emoji Reactions
Hermes automatically adds emoji reactions to your messages to show processing status:
- 🤔Thinking — added when the bot starts processing your message
- 🥳Done — added when the response is complete (replaces the Thinking reaction)
These reactions work in both DMs and group chats.
### Display Settings
You can customize DingTalk's display behavior independently from other platforms:
```yaml
display:
platforms:
dingtalk:
show_reasoning: false # Show model reasoning/thinking in replies
streaming: true # Enable streaming responses (works with AI Cards)
tool_progress: all # Show tool execution progress (all/new/off)
interim_assistant_messages: true # Show intermediate commentary messages
```
To disable tool progress and intermediate messages for a cleaner experience:
```yaml
display:
platforms:
dingtalk:
tool_progress: off
interim_assistant_messages: false
```
## Troubleshooting
### Bot is not responding to messages
**Cause**: The robot capability isn't enabled, or `DINGTALK_ALLOWED_USERS` doesn't include your User ID.
**Fix**: Verify the robot capability is enabled in your app settings and that Stream Mode is selected. Check that your User ID is in `DINGTALK_ALLOWED_USERS`. Restart the gateway.
### "dingtalk-stream not installed" error
**Cause**: The `dingtalk-stream` Python package is not installed.
**Fix**: Install it:
```bash
pip install dingtalk-stream httpx
```
### "DINGTALK_CLIENT_ID and DINGTALK_CLIENT_SECRET required"
**Cause**: The credentials aren't set in your environment or `.env` file.
**Fix**: Verify `DINGTALK_CLIENT_ID` and `DINGTALK_CLIENT_SECRET` are set correctly in `~/.hermes/.env`. The Client ID is your AppKey, and the Client Secret is your AppSecret from the DingTalk Developer Console.
### Stream disconnects / reconnection loops
**Cause**: Network instability, DingTalk platform maintenance, or credential issues.
**Fix**: The adapter automatically reconnects with exponential backoff (2s → 5s → 10s → 30s → 60s). Check that your credentials are valid and your app hasn't been deactivated. Verify your network allows outbound WebSocket connections.
### Bot is offline
**Cause**: The Hermes gateway isn't running, or it failed to connect.
**Fix**: Check that `hermes gateway` is running. Look at the terminal output for error messages. Common issues: wrong credentials, app deactivated, `dingtalk-stream` or `httpx` not installed.
### "No session_webhook available"
**Cause**: The bot tried to reply but doesn't have a session webhook URL. This typically happens if the webhook expired or the bot was restarted between receiving the message and sending the reply.
**Fix**: Send a new message to the bot — each incoming message provides a fresh session webhook for replies. This is a normal DingTalk limitation; the bot can only reply to messages it has received recently.
## Security
:::warning
Always set `DINGTALK_ALLOWED_USERS` to restrict who can interact with the bot. Without it, the gateway denies all users by default as a safety measure. Only add User IDs of people you trust — authorized users have full access to the agent's capabilities, including tool use and system access.
:::
For more information on securing your Hermes Agent deployment, see the [Security Guide](../security.md).
## Notes
- **Stream Mode**: No public URL, domain name, or webhook server needed. The connection is initiated from your machine via WebSocket, so it works behind NAT and firewalls.
- **AI Cards**: Optionally reply with rich AI Cards instead of plain markdown. Configure via `card_template_id`.
- **Emoji Reactions**: Automatic 🤔Thinking/🥳Done reactions for processing status.
- **Markdown responses**: Replies are formatted in DingTalk's markdown format for rich text display.
- **Media support**: Images and files in incoming messages are automatically resolved and can be processed by vision tools.
- **Message deduplication**: The adapter deduplicates messages with a 5-minute window to prevent processing the same message twice.
- **Auto-reconnection**: If the stream connection drops, the adapter automatically reconnects with exponential backoff.
- **Message length limit**: Responses are capped at 20,000 characters per message. Longer responses are truncated.
website/docs/user-guide/messaging/discord.md
+931
View File
@@ -0,0 +1,931 @@
---
sidebar_position: 3
title: "Discord"
description: "Set up Hermes Agent as a Discord bot"
---
# Discord Setup
Hermes Agent integrates with Discord as a bot, letting you chat with your AI assistant through direct messages or server channels. The bot receives your messages, processes them through the Hermes Agent pipeline (including tool use, memory, and reasoning), and responds in real time. It supports text, voice messages, file attachments, and slash commands.
Before setup, here's the part most people want to know: how Hermes behaves once it's in your server.
## How Hermes Behaves
| Context | Behavior |
|---------|----------|
| **DMs** | Hermes responds to every message. No `@mention` needed. Each DM has its own session. |
| **Server channels** | By default, Hermes only responds when you `@mention` it. If you post in a channel without mentioning it, Hermes ignores the message. |
| **Free-response channels** | You can make specific channels mention-free with `DISCORD_FREE_RESPONSE_CHANNELS`, or disable mentions globally with `DISCORD_REQUIRE_MENTION=false`. Messages in these channels are answered inline — auto-threading is skipped so the channel stays a lightweight chat. |
| **Threads** | Hermes replies in the same thread. Mention rules still apply unless that thread or its parent channel is configured as free-response. Threads stay isolated from the parent channel for session history. |
| **Shared channels with multiple users** | By default, Hermes isolates session history per user inside the channel for safety and clarity. Two people talking in the same channel do not share one transcript unless you explicitly disable that. |
| **Messages mentioning other users** | When `DISCORD_IGNORE_NO_MENTION` is `true` (the default), Hermes stays silent if a message @mentions other users but does **not** mention the bot. This prevents the bot from jumping into conversations directed at other people. Set to `false` if you want the bot to respond to all messages regardless of who is mentioned. This only applies in server channels, not DMs. |
:::tip
If you want a normal bot-help channel where people can talk to Hermes without tagging it every time, add that channel to `DISCORD_FREE_RESPONSE_CHANNELS`.
:::
### Discord Gateway Model
Hermes on Discord is not a webhook that replies statelessly. It runs through the full messaging gateway, which means each incoming message goes through:
1. authorization (`DISCORD_ALLOWED_USERS`)
2. mention / free-response checks
3. session lookup
4. session transcript loading
5. normal Hermes agent execution, including tools, memory, and slash commands
6. response delivery back to Discord
That matters because behavior in a busy server depends on both Discord routing and Hermes session policy.
### Session Model in Discord
By default:
- each DM gets its own session
- each server thread gets its own session namespace
- each user in a shared channel gets their own session inside that channel
So if Alice and Bob both talk to Hermes in `#research`, Hermes treats those as separate conversations by default even though they are using the same visible Discord channel.
This is controlled by `config.yaml`:
```yaml
group_sessions_per_user: true
```
Set it to `false` only if you explicitly want one shared conversation for the entire room:
```yaml
group_sessions_per_user: false
```
Shared sessions can be useful for a collaborative room, but they also mean:
- users share context growth and token costs
- one person's long tool-heavy task can bloat everyone else's context
- one person's in-flight run can interrupt another person's follow-up in the same room
### Interrupts and Concurrency
Hermes tracks running agents by session key.
With the default `group_sessions_per_user: true`:
- Alice interrupting her own in-flight request only affects Alice's session in that channel
- Bob can keep talking in the same channel without inheriting Alice's history or interrupting Alice's run
With `group_sessions_per_user: false`:
- the whole room shares one running-agent slot for that channel/thread
- follow-up messages from different people can interrupt or queue behind each other
This guide walks you through the full setup process — from creating your bot on Discord's Developer Portal to sending your first message.
### Gateway WebSocket health
Discord REST and the Gateway WebSocket are separate transports. A successful REST response (including `fetch_user()` returning HTTP 200) does not prove that the bot can still receive Gateway events. Hermes therefore combines the ready state, client/socket closure state, socket openness, heartbeat ACK age, and finite heartbeat latency.
After the configured number of consecutive unhealthy samples, the adapter emits one retryable fatal event. The existing gateway reconnect watcher creates a fresh adapter; the Discord adapter does not start a second unbounded reconnect loop.
Configure the non-secret thresholds in `config.yaml`:
```yaml
discord:
websocket_liveness_interval_seconds: 15
websocket_liveness_failure_threshold: 2
websocket_heartbeat_ack_max_age_seconds: 60
websocket_max_latency_seconds: 30
```
The old `liveness_interval_seconds` and `liveness_failure_threshold` names remain compatibility aliases only; they no longer mean REST probing.
## Step 1: Create a Discord Application
1. Go to the [Discord Developer Portal](https://discord.com/developers/applications) and sign in with your Discord account.
2. Click **New Application** in the top-right corner.
3. Enter a name for your application (e.g., "Hermes Agent") and accept the Developer Terms of Service.
4. Click **Create**.
You'll land on the **General Information** page. Note the **Application ID** — you'll need it later to build the invite URL.
## Step 2: Create the Bot
1. In the left sidebar, click **Bot**.
2. Discord automatically creates a bot user for your application. You'll see the bot's username, which you can customize.
3. Under **Authorization Flow**:
- Set **Public Bot** to **ON** — required to use the Discord-provided invite link (recommended). This allows the Installation tab to generate a default authorization URL.
- Leave **Require OAuth2 Code Grant** set to **OFF**.
:::tip
You can set a custom avatar and banner for your bot on this page. This is what users will see in Discord.
:::
:::info[Private Bot Alternative]
If you prefer to keep your bot private (Public Bot = OFF), you **must** use the **Manual URL** method in Step 5 instead of the Installation tab. The Discord-provided link requires Public Bot to be enabled.
:::
## Step 3: Enable Privileged Gateway Intents
This is the most critical step in the entire setup. Without the correct intents enabled, your bot will connect to Discord but **will not be able to read message content**.
On the **Bot** page, scroll down to **Privileged Gateway Intents**. You'll see three toggles:
| Intent | Purpose | Required? |
|--------|---------|-----------|
| **Presence Intent** | See user online/offline status | Optional |
| **Server Members Intent** | Access the member list, resolve usernames | **Required** |
| **Message Content Intent** | Read the text content of messages | **Required** |
**Enable both Server Members Intent and Message Content Intent** by toggling them **ON**.
- Without **Message Content Intent**, your bot receives message events but the message text is empty — the bot literally cannot see what you typed.
- Without **Server Members Intent**, the bot cannot resolve usernames for the allowed users list and may fail to identify who is messaging it.
:::warning[This is the #1 reason Discord bots don't work]
If your bot is online but never responds to messages, the **Message Content Intent** is almost certainly disabled. Go back to the [Developer Portal](https://discord.com/developers/applications), select your application → Bot → Privileged Gateway Intents, and make sure **Message Content Intent** is toggled ON. Click **Save Changes**.
:::
**Regarding server count:**
- If your bot is in **fewer than 100 servers**, you can simply toggle intents on and off freely.
- If your bot is in **100 or more servers**, Discord requires you to submit a verification application to use privileged intents. For personal use, this is not a concern.
Click **Save Changes** at the bottom of the page.
## Step 4: Get the Bot Token
The bot token is the credential Hermes Agent uses to log in as your bot. Still on the **Bot** page:
1. Under the **Token** section, click **Reset Token**.
2. If you have two-factor authentication enabled on your Discord account, enter your 2FA code.
3. Discord will display your new token. **Copy it immediately.**
:::warning[Token shown only once]
The token is only displayed once. If you lose it, you'll need to reset it and generate a new one. Never share your token publicly or commit it to Git — anyone with this token has full control of your bot.
:::
Store the token somewhere safe (a password manager, for example). You'll need it in Step 8.
## Step 5: Generate the Invite URL
You need an OAuth2 URL to invite the bot to your server. There are two ways to do this:
### Option A: Using the Installation Tab (Recommended)
:::note[Requires Public Bot]
This method requires **Public Bot** to be set to **ON** in Step 2. If you set Public Bot to OFF, use the Manual URL method below instead.
:::
1. In the left sidebar, click **Installation**.
2. Under **Installation Contexts**, enable **Guild Install**.
3. For **Install Link**, select **Discord Provided Link**.
4. Under **Default Install Settings** for Guild Install:
- **Scopes**: select `bot` and `applications.commands`
- **Permissions**: select the permissions listed below.
### Option B: Manual URL
You can construct the invite URL directly using this format:
```
https://discord.com/oauth2/authorize?client_id=YOUR_APP_ID&scope=bot+applications.commands&permissions=274878286912
```
Replace `YOUR_APP_ID` with the Application ID from Step 1.
### Required Permissions
These are the minimum permissions your bot needs:
- **View Channels** — see the channels it has access to
- **Send Messages** — respond to your messages
- **Embed Links** — format rich responses
- **Attach Files** — send images, audio, and file outputs
- **Read Message History** — maintain conversation context
### Recommended Additional Permissions
- **Send Messages in Threads** — respond in thread conversations
- **Add Reactions** — react to messages for acknowledgment
### Permission Integers
| Level | Permissions Integer | What's Included |
|-------|-------------------|-----------------|
| Minimal | `117760` | View Channels, Send Messages, Read Message History, Attach Files |
| Recommended | `274878286912` | All of the above plus Embed Links, Send Messages in Threads, Add Reactions |
## Step 6: Invite to Your Server
1. Open the invite URL in your browser (from the Installation tab or the manual URL you constructed).
2. In the **Add to Server** dropdown, select your server.
3. Click **Continue**, then **Authorize**.
4. Complete the CAPTCHA if prompted.
:::info
You need the **Manage Server** permission on the Discord server to invite a bot. If you don't see your server in the dropdown, ask a server admin to use the invite link instead.
:::
After authorizing, the bot will appear in your server's member list (it will show as offline until you start the Hermes gateway).
## Step 7: Find Your Discord User ID
Hermes Agent uses your Discord User ID to control who can interact with the bot. To find it:
1. Open Discord (desktop or web app).
2. Go to **Settings** → **Advanced** → toggle **Developer Mode** to **ON**.
3. Close settings.
4. Right-click your own username (in a message, the member list, or your profile) → **Copy User ID**.
Your User ID is a long number like `284102345871466496`.
:::tip
Developer Mode also lets you copy **Channel IDs** and **Server IDs** the same way — right-click the channel or server name and select Copy ID. You'll need a Channel ID if you want to set a home channel manually.
:::
## Step 8: Configure Hermes Agent
### Option A: Interactive Setup (Recommended)
Run the guided setup command:
```bash
hermes gateway setup
```
Select **Discord** when prompted, then paste your bot token and user ID when asked.
### Option B: Manual Configuration
Add the following to your `~/.hermes/.env` file:
```bash
# Required
DISCORD_BOT_TOKEN=your-bot-token
DISCORD_ALLOWED_USERS=284102345871466496
# Multiple allowed users (comma-separated)
# DISCORD_ALLOWED_USERS=284102345871466496,198765432109876543
```
Then start the gateway:
```bash
hermes gateway
```
The bot should come online in Discord within a few seconds. Send it a message — either a DM or in a channel it can see — to test.
:::tip
You can run `hermes gateway` in the background or as a systemd service for persistent operation. See the deployment docs for details.
:::
## Configuration Reference
Discord behavior is controlled through two files: **`~/.hermes/.env`** for credentials and env-level toggles, and **`~/.hermes/config.yaml`** for structured settings. Environment variables always take precedence over config.yaml values when both are set.
### Environment Variables (`.env`)
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `DISCORD_BOT_TOKEN` | **Yes** | — | Bot token from the [Discord Developer Portal](https://discord.com/developers/applications). |
| `DISCORD_ALLOWED_USERS` | Conditional | — | Comma-separated Discord user IDs allowed to interact with the bot. Without this **or** `DISCORD_ALLOWED_ROLES`, the gateway denies all users unless `DISCORD_ALLOW_ALL_USERS=true`, `GATEWAY_ALLOW_ALL_USERS=true`, or `DISCORD_ALLOWED_CHANNELS` explicitly scopes guild access. |
| `DISCORD_ALLOWED_ROLES` | No | — | Comma-separated Discord role IDs. Any member with one of these roles is authorized — OR semantics with `DISCORD_ALLOWED_USERS`. Auto-enables the **Server Members Intent** on connect. Useful when moderation teams churn: new mods get access as soon as the role is granted, no config push needed. |
| `DISCORD_ALLOW_ALL_USERS` | No | `false` | Explicit opt-in to allow every Discord user who can reach the bot. This restores the pre-0.18 open behavior for Discord only; use only for trusted/private guilds or development. |
| `GATEWAY_ALLOW_ALL_USERS` | No | `false` | Global allow-all opt-in for every gateway platform. Prefer the platform-specific `DISCORD_ALLOW_ALL_USERS` unless you intentionally want all connected platforms open. |
| `DISCORD_HOME_CHANNEL` | No | — | Channel ID where the bot sends proactive messages (cron output, reminders, notifications). |
| `DISCORD_HOME_CHANNEL_NAME` | No | `"Home"` | Display name for the home channel in logs and status output. |
| `DISCORD_COMMAND_SYNC_POLICY` | No | `"safe"` | Controls native slash-command startup sync. `"safe"` diffs existing global commands and only updates what changed, recreating commands when Discord metadata changes cannot be applied via patch. `"bulk"` preserves the old `tree.sync()` behavior. `"off"` skips startup sync entirely. |
| `DISCORD_REQUIRE_MENTION` | No | `true` | When `true`, the bot only responds in server channels when `@mentioned`. Set to `false` to respond to all messages in every channel. |
| `DISCORD_THREAD_REQUIRE_MENTION` | No | `false` | When `true`, the in-thread mention shortcut is disabled — threads are gated the same as channels, requiring `@mention` even after the bot has already participated. Use this when multiple bots share a thread and you want each to fire only on explicit `@mention`. |
| `DISCORD_FREE_RESPONSE_CHANNELS` | No | — | Comma-separated channel IDs where the bot responds without requiring an `@mention`, even when `DISCORD_REQUIRE_MENTION` is `true`. |
| `DISCORD_IGNORE_NO_MENTION` | No | `true` | When `true`, the bot stays silent if a message `@mentions` other users but does **not** mention the bot. Prevents the bot from jumping into conversations directed at other people. Only applies in server channels, not DMs. |
| `DISCORD_AUTO_THREAD` | No | `true` | When `true`, automatically creates a new thread for every `@mention` in a text channel, so each conversation is isolated (similar to Slack behavior). Messages already inside threads or DMs are unaffected. |
| `DISCORD_ALLOW_BOTS` | No | `"none"` | Controls how the bot handles messages from other Discord bots. `"none"` — ignore all other bots. `"mentions"` — only accept bot messages that `@mention` Hermes. `"all"` — accept all bot messages. |
| `DISCORD_REACTIONS` | No | `true` | When `true`, the bot adds emoji reactions to messages during processing (👀 when starting, ✅ on success, ❌ on error). Set to `false` to disable reactions entirely. |
| `DISCORD_IGNORED_CHANNELS` | No | — | Comma-separated channel IDs where the bot **never** responds, even when `@mentioned`. Takes priority over all other channel settings. |
| `DISCORD_ALLOWED_CHANNELS` | No | — | Comma-separated channel IDs. When set, the bot **only** responds in these channels (plus DMs if allowed). Overrides `config.yaml` `discord.allowed_channels`. Combine with `DISCORD_IGNORED_CHANNELS` to express allow/deny rules. |
| `DISCORD_NO_THREAD_CHANNELS` | No | — | Comma-separated channel IDs where the bot responds directly in the channel instead of creating a thread. Only relevant when `DISCORD_AUTO_THREAD` is `true`. |
| `DISCORD_HISTORY_BACKFILL` | No | `true` | When `true`, prepend recent channel scrollback (since the bot's last response) to the user message when the bot is mentioned. Recovers context the bot would otherwise miss with `require_mention`. Skipped in DMs and free-response channels. Set to `false` to disable. |
| `DISCORD_HISTORY_BACKFILL_LIMIT` | No | `50` | Maximum number of messages to scan backwards when assembling the backfill block. In practice the scan usually stops earlier — at the bot's own last message in the channel. |
| `DISCORD_REPLY_TO_MODE` | No | `"first"` | Controls reply-reference behavior: `"off"` — never reply to the original message, `"first"` — reply-reference on the first message chunk only (default), `"all"` — reply-reference on every chunk. |
| `DISCORD_ALLOW_MENTION_EVERYONE` | No | `false` | When `false` (default), the bot cannot ping `@everyone` or `@here` even if its response contains those tokens. Set to `true` to opt back in. See [Mention Control](#mention-control) below. |
| `DISCORD_ALLOW_MENTION_ROLES` | No | `false` | When `false` (default), the bot cannot ping `@role` mentions. Set to `true` to allow. |
| `DISCORD_ALLOW_MENTION_USERS` | No | `true` | When `true` (default), the bot can ping individual users by ID. |
| `DISCORD_ALLOW_MENTION_REPLIED_USER` | No | `true` | When `true` (default), replying to a message pings the original author. |
| `DISCORD_PROXY` | No | — | Proxy URL for Discord connections (HTTP, WebSocket, REST). Overrides `HTTPS_PROXY`/`ALL_PROXY`. Supports `http://`, `https://`, and `socks5://` schemes. |
| `DISCORD_ALLOW_ANY_ATTACHMENT` | No | `false` | When `true`, the bot accepts attachments of any file type (not just the built-in PDF/text/zip/office allowlist). Unknown types are cached to disk and surfaced to the agent as a local path with `application/octet-stream` MIME so it can inspect them with `terminal` / `read_file` / `ffprobe` / etc. |
| `DISCORD_MAX_ATTACHMENT_BYTES` | No | `33554432` | Maximum bytes per attachment the gateway will download and cache. Default 32 MiB. Set to `0` for no cap (attachments are held in memory while being written, so unlimited carries a real memory cost). |
| `HERMES_DISCORD_TEXT_BATCH_DELAY_SECONDS` | No | `0.6` | Grace window the adapter waits before flushing a queued text chunk. Useful for smoothing streamed output. |
| `HERMES_DISCORD_TEXT_BATCH_SPLIT_DELAY_SECONDS` | No | `2.0` | Delay between split chunks when a single message exceeds Discord's length limit. |
:::warning Bot-to-bot conversation is not supported
`DISCORD_ALLOW_BOTS` exists to accept input from a specific trusted bot (e.g. a relay or webhook bot), not to let two Hermes profiles talk to each other. The default, `"none"`, ignores all other bots and is the safe setting.
Wiring multiple Hermes profiles to reply to one another in a shared channel — by setting `"mentions"` or `"all"` across several profiles — is an unsupported topology. Discord auto-`@mentions` the replied-to author on every reply, so under `"mentions"` two bots will satisfy each other's mention gate indefinitely and ack-loop. There is no circuit breaker for this because the supported configuration is simply to leave `DISCORD_ALLOW_BOTS` at `"none"`. If you must accept a particular bot, scope the acceptance narrowly and never to another auto-replying agent.
:::
### Config File (`config.yaml`)
The `discord` section in `~/.hermes/config.yaml` mirrors the env vars above. Config.yaml settings are applied as defaults — if the equivalent env var is already set, the env var wins.
```yaml
# Discord-specific settings
discord:
require_mention: true # Require @mention in server channels
thread_require_mention: false # If true, require @mention in threads too (multi-bot threads)
free_response_channels: "" # Comma-separated channel IDs (or YAML list)
auto_thread: true # Auto-create threads on @mention
reactions: true # Add emoji reactions during processing
ignored_channels: [] # Channel IDs where bot never responds
no_thread_channels: [] # Channel IDs where bot responds without threading
history_backfill: true # Prepend recent channel scrollback on mention (default: true)
history_backfill_limit: 50 # Max messages to scan backwards (default: 50)
missed_message_backfill: # Replay messages missed while disconnected (opt-in)
enabled: false
channels: [] # Empty uses free_response_channels
window_seconds: 21600 # Look back at most 6 hours
limit: 100 # Global scan cap per reconnect
max_dispatches: 10 # Recovery dispatch cap per reconnect
channel_prompts: {} # Per-channel ephemeral system prompts
voice_channel_inactivity_timeout_seconds: 300 # Set 0 to stay in VC until explicit /voice leave
voice_playback_timeout_seconds: 120 # Minimum playback watchdog; long clips get duration+padding
allow_mentions: # What the bot is allowed to ping (safe defaults)
everyone: false # @everyone / @here pings (default: false)
roles: false # @role pings (default: false)
users: true # @user pings (default: true)
replied_user: true # reply-reference pings the author (default: true)
# Session isolation (applies to all gateway platforms, not just Discord)
group_sessions_per_user: true # Isolate sessions per user in shared channels
```
#### `discord.require_mention`
**Type:** boolean — **Default:** `true`
When enabled, the bot only responds in server channels when directly `@mentioned`. DMs always get a response regardless of this setting.
#### `discord.thread_require_mention`
**Type:** boolean — **Default:** `false`
By default, once the bot has participated in a thread (auto-created on `@mention` or replied in once), it keeps responding to every subsequent message in that thread without needing to be `@mentioned` again. That's the right default for one-on-one conversations.
In **multi-bot threads** where users address one bot per turn, this default becomes a footgun — every other bot in the thread also fires on every message, burning credits and spamming the channel. Set `thread_require_mention: true` to disable the in-thread shortcut and gate threads the same way channels are gated. Explicit `@mentions` still work as before.
```yaml
discord:
require_mention: true
thread_require_mention: true # multi-bot setup
```
#### `discord.free_response_channels`
**Type:** string or list — **Default:** `""`
Channel IDs where the bot responds to all messages without needing an `@mention`. Accepts either a comma-separated string or a YAML list:
```yaml
# String format
discord:
free_response_channels: "1234567890,9876543210"
# List format
discord:
free_response_channels:
- 1234567890
- 9876543210
```
If a thread's parent channel is in this list, the thread also becomes mention-free.
Free-response channels also **skip auto-threading** — the bot replies inline rather than spinning off a new thread per message. This keeps the channel usable as a lightweight chat surface. If you want threading behavior, don't list the channel as free-response (use normal `@mention` flow instead).
#### `discord.auto_thread`
**Type:** boolean — **Default:** `true`
When enabled, every `@mention` in a regular text channel automatically creates a new thread for the conversation. This keeps the main channel clean and gives each conversation its own isolated session history. Once a thread is created, subsequent messages in that thread don't require `@mention` — the bot knows it's already participating. Set [`thread_require_mention`](#discordthread_require_mention) to `true` to disable this in-thread shortcut for multi-bot setups.
Messages sent in existing threads or DMs are unaffected by this setting. Channels listed in `discord.free_response_channels` or `discord.no_thread_channels` also bypass auto-threading and get inline replies instead.
#### `discord.reactions`
**Type:** boolean — **Default:** `true`
Controls whether the bot adds emoji reactions to messages as visual feedback:
- 👀 added when the bot starts processing your message
- ✅ added when the response is delivered successfully
- ❌ added if an error occurs during processing
Disable this if you find the reactions distracting or if the bot's role doesn't have the **Add Reactions** permission.
#### `discord.ignored_channels`
**Type:** string or list — **Default:** `[]`
Channel IDs where the bot **never** responds, even when directly `@mentioned`. This takes the highest priority — if a channel is in this list, the bot silently ignores all messages there, regardless of `require_mention`, `free_response_channels`, or any other setting.
```yaml
# String format
discord:
ignored_channels: "1234567890,9876543210"
# List format
discord:
ignored_channels:
- 1234567890
- 9876543210
```
If a thread's parent channel is in this list, messages in that thread are also ignored.
#### `discord.no_thread_channels`
**Type:** string or list — **Default:** `[]`
Channel IDs where the bot responds directly in the channel instead of auto-creating a thread. This only has an effect when `auto_thread` is `true` (the default). In these channels, the bot responds inline like a normal message rather than spawning a new thread.
```yaml
discord:
no_thread_channels:
- 1234567890 # Bot responds inline here
```
Useful for channels dedicated to bot interaction where threads would add unnecessary noise.
#### `discord.channel_prompts`
**Type:** mapping — **Default:** `{}`
Per-channel ephemeral system prompts that are injected on every turn in the matching Discord channel or thread without being persisted to transcript history.
```yaml
discord:
channel_prompts:
"1234567890": |
This channel is for research tasks. Prefer deep comparisons,
citations, and concise synthesis.
"9876543210": |
This forum is for therapy-style support. Be warm, grounded,
and non-judgmental.
```
Behavior:
- Exact thread/channel ID matches win.
- If a message arrives inside a thread or forum post and that thread has no explicit entry, Hermes falls back to the parent channel/forum ID.
- Prompts are applied ephemerally at runtime, so changing them affects future turns immediately without rewriting past session history.
#### `discord.history_backfill`
**Type:** boolean — **Default:** `true`
When enabled, the bot recovers missed channel messages on each `@mention`. With `require_mention: true`, the bot only processes messages that tag it directly — everything else in the channel is invisible to the session transcript. History backfill scans backwards through recent channel history when triggered, collecting messages between the bot's last response and the current mention, and includes them as context.
Behavior by surface:
- **Server channels** (with `require_mention: true`): backfill scans the channel since the bot's last response. Useful when other participants posted while the bot wasn't addressed.
- **Threads**: backfill scans the thread only — Discord's `channel.history()` on a thread returns only that thread's messages, not the parent channel. This is the right scope because threads are usually self-contained conversations.
- **DMs**: skipped. Every DM message triggers the bot, so the session transcript is already complete — there's no mention gap to fill.
- **Free-response channels** and **bot's own auto-created threads**: skipped for the same reason — no mention gating means no gap.
Per-user sessions (`group_sessions_per_user: true`, the default) also benefit: a user's session is missing the context posted by other channel participants and the user's own messages from before they tagged the bot. Backfill fills both gaps.
```yaml
discord:
history_backfill: true # default
```
To turn it off:
```yaml
discord:
history_backfill: false
```
> **Note:** Messages that arrive *while* the bot is processing (between a trigger and its response) are not captured. This is an accepted simplification — the user can re-send or tag again.
#### `discord.history_backfill_limit`
**Type:** integer — **Default:** `50`
Maximum number of messages to scan backwards when recovering channel context. In practice the scan usually stops much earlier — at the bot's own last message in the channel, which is the natural boundary between turns. This limit is a safety cap for cold starts and long gaps where no prior bot message exists in recent history.
```yaml
discord:
history_backfill: true
history_backfill_limit: 50
```
#### `discord.missed_message_backfill`
**Type:** object — **Default:** disabled
Discord's WebSocket resume window can expire during a restart or network outage. Messages sent during that gap are not delivered as live gateway events. When this option is enabled, Hermes scans a bounded set of configured channel and thread histories after Discord reconnects, then sends still-unhandled messages through the same authorization, mention, channel, deduplication, and dispatch path as live events.
```yaml
discord:
missed_message_backfill:
enabled: true
channels: ["123456789012345678"]
window_seconds: 3600
limit: 100
max_dispatches: 10
```
If `channels` is empty, Hermes uses `discord.free_response_channels`. Set it to `"*"` only when the bot should inspect every reachable server text channel. The recovery ledger is stored per profile under `gateway/discord_message_recovery.db`, preventing a successfully answered message from being replayed again after a later restart.
#### `group_sessions_per_user`
**Type:** boolean — **Default:** `true`
This is a global gateway setting (not Discord-specific) that controls whether users in the same channel get isolated session histories.
When `true`: Alice and Bob talking in `#research` each have their own separate conversation with Hermes. When `false`: the entire channel shares one conversation transcript and one running-agent slot.
```yaml
group_sessions_per_user: true
```
See the [Session Model](#session-model-in-discord) section above for the full implications of each mode.
#### `display.tool_progress`
**Type:** string — **Default:** `"all"` — **Values:** `off`, `new`, `all`, `verbose`
Controls whether the bot sends progress messages in the chat while processing (e.g., "Reading file...", "Running terminal command..."). This is a global gateway setting that applies to all platforms.
```yaml
display:
tool_progress: "all" # off | new | all | verbose
```
- `off` — no progress messages
- `new` — only show the first tool call per turn
- `all` — show all tool calls (truncated to 40 characters in gateway messages)
- `verbose` — show full tool call details (can produce long messages)
#### `display.tool_progress_command`
**Type:** boolean — **Default:** `false`
When enabled, makes the `/verbose` slash command available in the gateway, letting you cycle through tool progress modes (`off → new → all → verbose → off`) without editing config.yaml.
```yaml
display:
tool_progress_command: true
```
#### `display.reasoning_style`
**Type:** string — **Default (Discord):** `"subtext"` — **Values:** `code`, `blockquote`, `subtext`
Controls how the model's reasoning block is rendered when reasoning display is enabled. Discord defaults to `subtext`, which uses Discord's native `-# ` small grey metadata text so reasoning stays visually secondary to the answer. `blockquote` renders it as a `>` quote, and `code` (the default on other platforms) uses a fenced code block. Long reasoning is collapsed to the first 15 lines.
```yaml
display:
platforms:
discord:
reasoning_style: subtext # code | blockquote | subtext
```
## Slash Command Access Control
By default, every allowed user can run every slash command. To split your allowlist into **admins** (full slash command access) and **regular users** (only commands you explicitly enable), add `allow_admin_from` and `user_allowed_commands` to the Discord platform's `extra` block:
```yaml
gateway:
platforms:
discord:
extra:
# Existing user allowlist (unchanged)
allow_from:
- "123456789012345678" # admin user ID
- "999888777666555444" # regular user ID
# NEW — admins get all slash commands (built-in + plugin)
allow_admin_from:
- "123456789012345678"
# NEW — non-admin allowed users can only run these slash commands.
# /help and /whoami are always allowed so users can see their access.
user_allowed_commands:
- status
- model
- history
# Optional: separate admin / command lists for server channels
group_allow_admin_from:
- "123456789012345678"
group_user_allowed_commands:
- status
```
**Behavior:**
- A user in `allow_admin_from` for a scope (DM or server channel) can run **every** registered slash command — built-in AND plugin-registered — through the live command registry.
- A user not in `allow_admin_from` can only run commands listed in `user_allowed_commands`, plus the always-allowed floor: `/help` and `/whoami`.
- Plain chat (non-slash messages) is unaffected. Non-admin users can still talk to the agent normally; they just can't trigger arbitrary commands.
- **Backward compat:** if `allow_admin_from` is not set for a scope, slash command gating is disabled for that scope. Existing installs keep working with no changes.
- DM admin status does not imply server-channel admin status. Each scope has its own admin list.
Use `/whoami` to see the active scope, your tier (admin / user / unrestricted), and which slash commands you can run.
## Interactive Model Picker
Send `/model` with no arguments in a Discord channel to open a dropdown-based model picker:
1. **Provider selection** — a Select dropdown showing available providers (up to 25).
2. **Model selection** — a second dropdown with models for the chosen provider (up to 25).
The picker times out after 120 seconds. Only authorized users (those in `DISCORD_ALLOWED_USERS`) can interact with it. If you know the model name, type `/model <name>` directly.
## Native Slash Commands for Skills
Hermes automatically registers installed skills as **native Discord Application Commands**. This means skills appear in Discord's autocomplete `/` menu alongside built-in commands.
- Each skill becomes a Discord slash command (e.g., `/code-review`, `/ascii-art`)
- Skills accept an optional `args` string parameter
- Discord has a limit of 100 application commands per bot — if you have more skills than available slots, extra skills are skipped with a warning in the logs
- Skills are registered during bot startup alongside built-in commands like `/model`, `/reset`, and `/bg`
No extra configuration is needed — any skill installed via `hermes skills install` is automatically registered as a Discord slash command on the next gateway restart.
### Disabling Slash Command Registration
If you run multiple Hermes gateways against the same Discord application (e.g. staging + production), only one of them should own the global slash-command registration — otherwise the last startup wins and the registrations flap. Turn slash registration off on the "follower" gateway:
```yaml
gateway:
platforms:
discord:
extra:
slash_commands: false # default: true
```
Leaving this at `true` on the "primary" gateway keeps the normal behavior — global `/`-menu commands for built-ins and installed skills.
## Sending Media (inline `MEDIA:` tags)
The Discord adapter supports native file uploads for every common media type via inline `MEDIA:/path/to/file` tags emitted in the agent's response — the adapter strips the tag and auto-uploads the file:
| Type | How it's delivered |
|---|---|
| Images (PNG/JPG/WebP) | Native Discord image attachment with inline preview |
| Animated GIFs | `send_animation` uploads as `animation.gif` so Discord plays it inline (not as a static thumbnail) |
| Video (MP4/MOV) | `send_video` — native video player |
| Audio / Voice | `send_voice` — native voice message when possible, file attachment otherwise |
| Documents (PDF/ZIP/docx/etc.) | `send_document` — native attachment with download button |
Discord's per-upload size limit depends on the server's boost tier (25 MB free, up to 500 MB). If Hermes gets an HTTP 413, the adapter falls back to a link pointing at the local cache path rather than failing silently.
## Receiving Arbitrary File Types
Any file type a user uploads is accepted. Authorization to message the agent is the gate — not the file extension. Every upload is downloaded, cached under `~/.hermes/cache/documents/`, and surfaced to the agent as a `DOCUMENT`-typed message event so it can inspect the file with `terminal` (`ffprobe`, `unzip`, `file`, `strings`, etc.) or `read_file`.
- Known types (PDF, docx/xlsx/pptx, zip, images/audio/video, etc.) keep their precise MIME.
- Unknown types fall back to the upload's reported content type, or `application/octet-stream` when none is given.
- Small UTF-8-decodable files (text, code, config, HTML, CSS, JSON, YAML, ...) have their contents auto-injected into the prompt up to 100 KiB. Binary files that can't be decoded are surfaced as a path-pointing context note only (auto-translated for Docker/Modal sandboxed terminals via `to_agent_visible_cache_path`), so they don't blow up the context window.
The only inbound limit is the per-file size cap (default 32 MiB):
```yaml
discord:
# Optional — raise/disable the per-file size cap. Default is 32 MiB.
# The whole file is held in memory while being cached, so unlimited
# uploads carry a real memory cost.
max_attachment_bytes: 33554432 # bytes; 0 = unlimited
```
Equivalent env var: `DISCORD_MAX_ATTACHMENT_BYTES=33554432` (or `0` for no cap).
The legacy `discord.allow_any_attachment` flag is now a no-op — any file type is always accepted — and is kept only so existing configs don't error.
:::warning Memory cost of unlimited
Disabling the size cap (`max_attachment_bytes: 0`) means a user can drop a multi-GB file on the bot and the gateway will dutifully buffer it through memory while caching to disk. Only set this in trusted single-user installs. For shared bots, keep the default 32 MiB or raise it conservatively.
:::
## Interactive Prompts (clarify)
When the agent calls the `clarify` tool — to ask which approach you prefer, get post-task feedback, or check before a non-trivial decision — Discord renders the question with **one button per choice**:
> Which framework should I use for the dashboard?
>
> [1. Next.js] [2. Remix] [3. Astro] [Other (type answer)]
Click a numbered button to answer, or click **Other** to type a free-form response (the next message you send in that channel becomes the answer). Open-ended `clarify` calls (no preset choices) skip the buttons and just capture your next message.
The buttons disable themselves once a choice is made so duplicate clicks don't double-resolve the prompt. Configure the response timeout via `agent.clarify_timeout` in `~/.hermes/config.yaml` (default `600` seconds). If you don't respond within the timeout, the agent unblocks with a sentinel message and adapts rather than hanging.
## Home Channel
You can designate a "home channel" where the bot sends proactive messages (such as cron job output, reminders, and notifications). There are two ways to set it:
### Using the Slash Command
Type `/sethome` in any Discord channel where the bot is present. That channel becomes the home channel.
### Manual Configuration
Add these to your `~/.hermes/.env`:
```bash
DISCORD_HOME_CHANNEL=123456789012345678
DISCORD_HOME_CHANNEL_NAME="#bot-updates"
```
Replace the ID with the actual channel ID (right-click → Copy Channel ID with Developer Mode on).
## Voice Messages
Hermes Agent supports Discord voice messages:
- **Incoming voice messages** are automatically transcribed using the configured STT provider: local `faster-whisper` (no key), Groq Whisper (`GROQ_API_KEY`), or OpenAI Whisper (`VOICE_TOOLS_OPENAI_KEY`).
- **Text-to-speech**: Use `/voice tts` to have the bot send spoken audio responses alongside text replies.
- **Discord voice channels**: Hermes can also join a voice channel, listen to users speaking, and talk back in the channel.
For the full setup and operational guide, see:
- [Voice Mode](/user-guide/features/voice-mode)
- [Use Voice Mode with Hermes](/guides/use-voice-mode-with-hermes)
### Voice Channel Audio Effects (ambient + verbal acks)
When the bot is in a voice channel, you can give it a more conversational feel: a short verbal acknowledgement ("let me look into that") before it starts working, and a subtle ambient "thinking" bed that plays underneath while tools run — the speech ducks the ambient down and swells it back when finished, similar to Grok voice mode.
discord.py plays only one audio stream per connection, so Hermes installs a software mixer on the outgoing stream that sums an ambient loop, acknowledgements, and TTS replies into that single stream — they overlap instead of cutting each other off.
This is **off by default**. Enable it in `config.yaml`:
```yaml
discord:
voice_fx:
enabled: true # master switch
ambient_enabled: true # idle "thinking" bed while tools run
ambient_path: "" # custom loop file (any audio format); "" = built-in synthesised pad
ambient_gain: 0.18 # idle bed loudness (0.0–1.0)
duck_gain: 0.06 # ambient loudness while the bot is speaking
speech_gain: 1.0 # TTS / acknowledgement loudness
ack_enabled: true # speak a short phrase before the first tool call of a turn
ack_phrases: # picked at random; set to [] to disable the spoken ack
- "Let me look into that."
- "One moment."
- "Checking on that now."
```
Notes:
- Set `voice_channel_inactivity_timeout_seconds: 0` if you want the bot to remain in the voice channel until an explicit `/voice leave` or manual disconnect. The default preserves the historical 300-second idle auto-leave.
- `voice_playback_timeout_seconds` is a floor, not a hard cap for long TTS. Hermes probes the generated audio duration and waits for `duration + 30s` when that is longer than the configured floor.
- The acknowledgement fires at most once per turn, only when the bot is in a voice channel and the mixer is active. It uses your configured TTS provider.
- `ambient_path` accepts any file `ffmpeg` can decode; it's looped seamlessly. Leave it empty to use the built-in synthesised pad (no asset needed).
- All settings live in `config.yaml` (not `.env`) — they're behavioral, not secrets.
- When `voice_fx.enabled` is `false`, voice playback uses the original one-shot path and nothing changes.
## Forum Channels
Discord forum channels (type 15) don't accept direct messages — every post in a forum must be a thread. Hermes auto-detects forum channels and creates a new thread post whenever it needs to send there, so text replies, TTS, images, voice messages, and file attachments all work without special handling from the agent.
- **Thread name** is derived from the first line of the message (markdown heading prefix stripped, capped at 100 chars). When the message is attachment-only, the filename is used as the fallback thread name.
- **Attachments** ride along on the starter message of the new thread — no separate upload step, no partial sends.
- **One call, one thread**: each forum send creates a new thread. Successive sends to the same forum will therefore produce separate threads.
- **Detection is three-layered**: the channel directory cache first, a process-local probe cache second, and a live `GET /channels/{id}` probe as a last resort (whose result is then memoized for the life of the process).
Refreshing the directory (`/channels refresh` on platforms that expose it, or a gateway restart) populates the cache with any forum channels created after the bot started.
## Troubleshooting
### Bot is online but not responding to messages
**Cause**: Either Message Content Intent is disabled, or Discord auth is failing closed because no access policy is configured.
**Fix**:
1. Go to [Developer Portal](https://discord.com/developers/applications) → your app → Bot → Privileged Gateway Intents → enable **Message Content Intent** → Save Changes.
2. Verify that at least one Discord access policy is configured:
```bash
# recommended: allow specific users
DISCORD_ALLOWED_USERS=284102345871466496
# or allow a trusted guild/dev bot to behave like pre-0.18 Discord
DISCORD_ALLOW_ALL_USERS=true
```
3. Restart the gateway:
```bash
hermes gateway restart
```
If the gateway log says Discord is connected and REST API checks work, but every inbound message is silent, look for this warning in `~/.hermes/logs/gateway.log`:
```text
No Discord access policy configured; inbound Discord messages will be denied by default.
```
Hermes 0.18 intentionally fails closed on externally reachable adapters. A Discord bot with no `DISCORD_ALLOWED_USERS`, no `DISCORD_ALLOWED_ROLES`, no `DISCORD_ALLOWED_CHANNELS`, and no explicit allow-all flag will connect successfully but deny inbound users before normal message handling.
### "Privileged intents" / `PrivilegedIntentsRequired` error on startup
**Cause**: Hermes requests privileged Gateway Intents that are not enabled for your bot in the Developer Portal. Discord then rejects the WebSocket connection. Hermes always requests **Message Content Intent**. It also requests **Server Members Intent** when your allowlist uses usernames (not numeric IDs) or when `DISCORD_ALLOWED_ROLES` is set. Presence Intent is not required.
**Fix**:
1. Go to [Developer Portal](https://discord.com/developers/applications) → your app → Bot → Privileged Gateway Intents.
2. Enable **Message Content Intent** (required). Enable **Server Members Intent** if you use usernames or role allowlists.
3. Click **Save Changes**, then restart the gateway (`hermes gateway restart`).
The gateway log should name the exact intent(s) Hermes requested. Until they are enabled, Discord will keep rejecting the connection — this is a portal configuration error, not a flaky network issue.
### Bot can't see messages in a specific channel
**Cause**: The bot's role doesn't have permission to view that channel.
**Fix**: In Discord, go to the channel's settings → Permissions → add the bot's role with **View Channel** and **Read Message History** enabled.
### 403 Forbidden errors
**Cause**: The bot is missing required permissions.
**Fix**: Re-invite the bot with the correct permissions using the URL from Step 5, or manually adjust the bot's role permissions in Server Settings → Roles.
### Bot is offline
**Cause**: The Hermes gateway isn't running, or the token is incorrect.
**Fix**: Check that `hermes gateway` is running. Verify `DISCORD_BOT_TOKEN` in your `.env` file. If you recently reset the token, update it.
### "User not allowed" / Bot ignores you
**Cause**: Your User ID isn't in `DISCORD_ALLOWED_USERS`.
**Fix**: Add your User ID to `DISCORD_ALLOWED_USERS` in `~/.hermes/.env` and restart the gateway.
### People in the same channel are sharing context unexpectedly
**Cause**: `group_sessions_per_user` is disabled, or the platform cannot provide a user ID for the messages in that context.
**Fix**: Set this in `~/.hermes/config.yaml` and restart the gateway:
```yaml
group_sessions_per_user: true
```
If you intentionally want a shared room conversation, leave it off — just expect shared transcript history and shared interrupt behavior.
## Security
:::warning
Always set `DISCORD_ALLOWED_USERS` (or `DISCORD_ALLOWED_ROLES`) to restrict who can interact with the bot. Without either, the gateway denies all users by default as a safety measure. Only authorize people you trust — authorized users have full access to the agent's capabilities, including tool use and system access.
:::
### Role-Based Access Control
For servers where access is managed by roles instead of individual user lists (moderator teams, support staff, internal tooling), use `DISCORD_ALLOWED_ROLES` — a comma-separated list of role IDs. Any member with one of those roles is authorized.
```bash
# ~/.hermes/.env — works alongside or instead of DISCORD_ALLOWED_USERS
DISCORD_ALLOWED_ROLES=987654321098765432,876543210987654321
```
Semantics:
- **OR with user allowlist.** A user is authorized if their ID is in `DISCORD_ALLOWED_USERS` **or** they have any role in `DISCORD_ALLOWED_ROLES`.
- **Server Members Intent auto-enabled.** When `DISCORD_ALLOWED_ROLES` is set, the bot enables the Members intent on connect — required for Discord to send role information with member records.
- **Role IDs, not names.** Grab them from Discord: **User Settings → Advanced → Developer Mode ON**, then right-click any role → **Copy Role ID**.
- **DM fallback.** In DMs the role check scans mutual guilds; a user with an allowed role in any shared server is authorized in DMs too.
This is the preferred pattern when the moderation team churns — new moderators get access the moment the role is granted, with no `.env` edit or gateway restart.
### Mention Control
By default, Hermes blocks the bot from pinging `@everyone`, `@here`, and role mentions, even if its reply contains those tokens. This prevents a poorly-worded prompt or echoed user content from spamming a whole server. Individual `@user` pings and reply-reference pings (the little "replying to…" chip) stay enabled so normal conversation still works.
You can relax these defaults via either env vars or `config.yaml`:
```yaml
# ~/.hermes/config.yaml
discord:
allow_mentions:
everyone: false # allow the bot to ping @everyone / @here
roles: false # allow the bot to ping @role mentions
users: true # allow the bot to ping individual @users
replied_user: true # ping the author when replying to their message
```
```bash
# ~/.hermes/.env — env vars win over config.yaml
DISCORD_ALLOW_MENTION_EVERYONE=false
DISCORD_ALLOW_MENTION_ROLES=false
DISCORD_ALLOW_MENTION_USERS=true
DISCORD_ALLOW_MENTION_REPLIED_USER=true
```
:::tip
Leave `everyone` and `roles` at `false` unless you know exactly why you need them. It is very easy for an LLM to produce the string `@everyone` inside a normal-looking response; without this protection, that would notify every member of your server.
:::
For more information on securing your Hermes Agent deployment, see the [Security Guide](../security.md).
website/docs/user-guide/messaging/email.md
+223
View File
@@ -0,0 +1,223 @@
---
sidebar_position: 7
title: "Email"
description: "Set up Hermes Agent as an email assistant via IMAP/SMTP"
---
# Email Setup
Hermes can receive and reply to emails using standard IMAP and SMTP protocols. Send an email to the agent's address and it replies in-thread — no special client or bot API needed. Works with Gmail, Outlook, Yahoo, Fastmail, or any provider that supports IMAP/SMTP.
:::info Gateway adapter only: no external dependencies
This page covers the Email gateway adapter, which uses Python's built-in `imaplib`, `smtplib`, and `email` modules. No additional packages or external services are required for this gateway path.
:::
This is separate from the bundled [Himalaya email skill](/docs/user-guide/skills/bundled/email/email-himalaya), which lets the agent manage email through terminal commands and requires the external `himalaya` CLI plus a Himalaya config file.
| Use case | What to configure | External dependency |
|---|---|---|
| Let people email the Hermes agent and receive replies | Email gateway adapter on this page | None beyond an IMAP/SMTP email account |
| Let the agent inspect, compose, move, and manage mailbox messages from terminal tools | Himalaya email skill | `himalaya` CLI and `~/.config/himalaya/config.toml` |
---
## Prerequisites
- **A dedicated email account** for your Hermes agent (don't use your personal email)
- **IMAP enabled** on the email account
- **An app password** if using Gmail or another provider with 2FA
### Gmail Setup
1. Enable 2-Factor Authentication on your Google Account
2. Go to [App Passwords](https://myaccount.google.com/apppasswords)
3. Create a new App Password (select "Mail" or "Other")
4. Copy the 16-character password — you'll use this instead of your regular password
### Outlook / Microsoft 365
1. Go to [Security Settings](https://account.microsoft.com/security)
2. Enable 2FA if not already active
3. Create an App Password under "Additional security options"
4. IMAP host: `outlook.office365.com`, SMTP host: `smtp.office365.com`
### Other Providers
Most email providers support IMAP/SMTP. Check your provider's documentation for:
- IMAP host and port (usually port 993 with SSL)
- SMTP host and port (usually port 587 with STARTTLS)
- Whether app passwords are required
### Proton Mail Bridge / local relays
Proton Mail Bridge (and similar local relays such as a self-hosted MTA) listen on
loopback with **STARTTLS** and a self-signed certificate, so the defaults
(implicit TLS on IMAP 993, verified certificates) won't connect. Override the
transport in `~/.hermes/config.yaml`:
```yaml
platforms:
email:
enabled: true
extra:
imap_host: 127.0.0.1
imap_security: starttls # tls (default) | starttls | plain
imap_tls_verify: false # Bridge uses a self-signed cert
smtp_host: 127.0.0.1
smtp_security: starttls # default: tls on port 465, starttls otherwise
smtp_tls_verify: false
```
and set `EMAIL_IMAP_PORT=1143` / `EMAIL_SMTP_PORT=1025` alongside your Bridge
credentials in `~/.hermes/.env`. Unknown `*_security` values log a warning and
fall back to the secure default. Only disable `*_tls_verify` for loopback hosts —
Hermes logs a warning when verification is off for any other host.
---
## Step 1: Configure Hermes
The easiest way:
```bash
hermes gateway setup
```
Select **Email** from the platform menu. The wizard prompts for your email address, password, IMAP/SMTP hosts, and allowed senders.
### Manual Configuration
Add to `~/.hermes/.env`:
```bash
# Required
EMAIL_ADDRESS=hermes@gmail.com
EMAIL_PASSWORD=abcd efgh ijkl mnop # App password (not your regular password)
EMAIL_IMAP_HOST=imap.gmail.com
EMAIL_SMTP_HOST=smtp.gmail.com
# Security (recommended)
EMAIL_ALLOWED_USERS=your@email.com,colleague@work.com
# Optional
EMAIL_IMAP_PORT=993 # Default: 993 (IMAP SSL)
EMAIL_SMTP_PORT=587 # Default: 587 (SMTP STARTTLS)
EMAIL_POLL_INTERVAL=15 # Seconds between inbox checks (default: 15)
EMAIL_HOME_ADDRESS=your@email.com # Default delivery target for cron jobs
```
---
## Step 2: Start the Gateway
```bash
hermes gateway # Run in foreground
hermes gateway install # Install as a user service
sudo hermes gateway install --system # Linux only: boot-time system service
```
On startup, the adapter:
1. Tests IMAP and SMTP connections
2. Marks all existing inbox messages as "seen" (only processes new emails)
3. Starts polling for new messages
---
## How It Works
### Receiving Messages
The adapter polls the IMAP inbox for UNSEEN messages at a configurable interval (default: 15 seconds). For each new email:
- **Subject line** is included as context (e.g., `[Subject: Deploy to production]`)
- **Reply emails** (subject starting with `Re:`) skip the subject prefix — the thread context is already established
- **Attachments** are cached locally:
- Images (JPEG, PNG, GIF, WebP) → available to the vision tool
- Documents (PDF, ZIP, etc.) → available for file access
- **HTML-only emails** have tags stripped for plain text extraction
- **Self-messages** are filtered out to prevent reply loops
- **Automated/noreply senders** are silently ignored — `noreply@`, `mailer-daemon@`, `bounce@`, `no-reply@`, and emails with `Auto-Submitted`, `Precedence: bulk`, or `List-Unsubscribe` headers
### Sending Replies
Replies are sent via SMTP with proper email threading:
- **In-Reply-To** and **References** headers maintain the thread
- **Subject line** preserved with `Re:` prefix (no double `Re: Re:`)
- **Message-ID** generated with the agent's domain
- Responses are sent as plain text (UTF-8)
### File Attachments
The agent can send file attachments in replies. Include `MEDIA:/path/to/file` in the response and the file is attached to the outgoing email.
### Skipping Attachments
To ignore all incoming attachments (for malware protection or bandwidth savings), add to your `config.yaml`:
```yaml
platforms:
email:
skip_attachments: true
```
When enabled, attachment and inline parts are skipped before payload decoding. The email body text is still processed normally.
---
## Access Control
Email access is stricter by default than chat-style platforms:
1. **`EMAIL_ALLOWED_USERS` set** → only emails from those addresses are processed
2. **No allowlist set** → unknown senders are ignored silently
3. **`EMAIL_ALLOW_ALL_USERS=true`** → any sender is accepted (use with caution)
4. **`platforms.email.unauthorized_dm_behavior: pair`** → unknown senders receive a pairing code
:::warning
**Use a dedicated inbox and configure `EMAIL_ALLOWED_USERS` for normal operation.** Email pairing is opt-in because shared inboxes often contain unrelated unread messages, and Hermes should not reply to those contacts by default.
:::
---
## Troubleshooting
| Problem | Solution |
|---------|----------|
| **"IMAP connection failed"** at startup | Verify `EMAIL_IMAP_HOST` and `EMAIL_IMAP_PORT`. Ensure IMAP is enabled on the account. For Gmail, enable it in Settings → Forwarding and POP/IMAP. |
| **"SMTP connection failed"** at startup | Verify `EMAIL_SMTP_HOST` and `EMAIL_SMTP_PORT`. Check that your password is correct (use App Password for Gmail). |
| **Messages not received** | Check `EMAIL_ALLOWED_USERS` includes the sender's email. Check spam folder — some providers flag automated replies. |
| **"Authentication failed"** | For Gmail, you must use an App Password, not your regular password. Ensure 2FA is enabled first. |
| **Duplicate replies** | Ensure only one gateway instance is running. Check `hermes gateway status`. |
| **Slow response** | The default poll interval is 15 seconds. Reduce with `EMAIL_POLL_INTERVAL=5` for faster response (but more IMAP connections). |
| **Replies not threading** | The adapter uses In-Reply-To headers. Some email clients (especially web-based) may not thread correctly with automated messages. |
---
## Security
:::warning
**Use a dedicated email account.** Don't use your personal email — the agent stores the password in `.env` and has full inbox access via IMAP.
:::
- Use **App Passwords** instead of your main password (required for Gmail with 2FA)
- Set `EMAIL_ALLOWED_USERS` to restrict who can interact with the agent
- The password is stored in `~/.hermes/.env` — protect this file (`chmod 600`)
- IMAP uses SSL (port 993) and SMTP uses STARTTLS (port 587) by default — connections are encrypted
---
## Environment Variables Reference
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `EMAIL_ADDRESS` | Yes | — | Agent's email address |
| `EMAIL_PASSWORD` | Yes | — | Email password or app password |
| `EMAIL_IMAP_HOST` | Yes | — | IMAP server host (e.g., `imap.gmail.com`) |
| `EMAIL_SMTP_HOST` | Yes | — | SMTP server host (e.g., `smtp.gmail.com`) |
| `EMAIL_IMAP_PORT` | No | `993` | IMAP server port |
| `EMAIL_SMTP_PORT` | No | `587` | SMTP server port |
| `EMAIL_POLL_INTERVAL` | No | `15` | Seconds between inbox checks |
| `EMAIL_ALLOWED_USERS` | No | — | Comma-separated allowed sender addresses |
| `EMAIL_HOME_ADDRESS` | No | — | Default delivery target for cron jobs |
| `EMAIL_ALLOW_ALL_USERS` | No | `false` | Allow all senders (not recommended) |
website/docs/user-guide/messaging/feishu.md
+590
View File
@@ -0,0 +1,590 @@
---
sidebar_position: 11
title: "Feishu / Lark"
description: "Set up Hermes Agent as a Feishu or Lark bot"
---
# Feishu / Lark Setup
Hermes Agent integrates with Feishu and Lark as a full-featured bot. Once connected, you can chat with the agent in direct messages or group chats, receive cron job results in a home chat, and send text, images, audio, and file attachments through the normal gateway flow.
The integration supports both connection modes:
- `websocket` — recommended; Hermes opens the outbound connection and you do not need a public webhook endpoint
- `webhook` — useful when you want Feishu/Lark to push events into your gateway over HTTP
## How Hermes Behaves
| Context | Behavior |
|---------|----------|
| Direct messages | Hermes responds to every message. |
| Group chats | Hermes responds only when the bot is @mentioned in the chat. |
| Shared group chats | By default, session history is isolated per user inside a shared chat. |
This shared-chat behavior is controlled by `config.yaml`:
```yaml
group_sessions_per_user: true
```
Set it to `false` only if you explicitly want one shared conversation per chat.
## Step 1: Create a Feishu / Lark App
### Recommended: Scan-to-Create (one command)
```bash
hermes gateway setup
```
Select **Feishu / Lark** and scan the QR code with your Feishu or Lark mobile app. Hermes will automatically create a bot application with the correct permissions and save the credentials.
### Alternative: Manual Setup
If scan-to-create is not available, the wizard falls back to manual input:
1. Open the Feishu or Lark developer console:
- Feishu: [https://open.feishu.cn/](https://open.feishu.cn/)
- Lark: [https://open.larksuite.com/](https://open.larksuite.com/)
2. Create a new app.
3. In **Credentials & Basic Info**, copy the **App ID** and **App Secret**.
4. Enable the **Bot** capability for the app.
5. Run `hermes gateway setup`, select **Feishu / Lark**, and enter the credentials when prompted.
:::warning
Keep the App Secret private. Anyone with it can impersonate your app.
:::
### Configure Permissions
In the Feishu developer console, go to **Permission Management** and add the following scopes. You can bulk-import them in the permissions page.
**Required permissions:**
| Scope | Purpose |
|-------|---------|
| `im:message` | Receive and read messages |
| `im:message:send_as_bot` | Send messages as the bot |
| `im:resource` | Access images, files, and audio sent by users |
| `im:chat` | Access chat/group metadata |
| `im:chat:readonly` | Read chat list and membership |
**Recommended permissions (for full functionality):**
| Scope | Purpose |
|-------|---------|
| `im:message.reactions:readonly` | Receive emoji reaction events |
| `admin:app.info:readonly` | Auto-detect bot identity for @mention gating |
| `contact:user.id:readonly` | Resolve user IDs for allowlist matching |
### Configure Events
In **Events and Callbacks**:
1. Set the connection mode to **Long Connection (WebSocket)** (recommended) or configure a webhook URL
2. In the **Event Configuration** section, subscribe to:
- `im.message.receive_v1` — required for receiving messages
### Publish the App
After configuring permissions and events, go to **Version Management** and publish a new version of the app. The permissions won't take effect until a version is published and approved (for enterprise apps, this may require admin approval).
## Step 2: Choose a Connection Mode
### Recommended: WebSocket mode
Use WebSocket mode when Hermes runs on your laptop, workstation, or a private server. No public URL is required. The official Lark SDK opens and maintains a persistent outbound WebSocket connection with automatic reconnection.
```bash
FEISHU_CONNECTION_MODE=websocket
```
**Requirements:** The `websockets` Python package must be installed. The SDK handles connection lifecycle, heartbeats, and auto-reconnection internally.
**How it works:** The adapter runs the Lark SDK's WebSocket client in a background executor thread. Inbound events (messages, reactions, card actions) are dispatched to the main asyncio loop. On disconnect, the SDK will attempt to reconnect automatically.
### Optional: Webhook mode
Use webhook mode only when you already run Hermes behind a reachable HTTP endpoint.
```bash
FEISHU_CONNECTION_MODE=webhook
```
In webhook mode, Hermes starts an HTTP server (via `aiohttp`) and serves a Feishu endpoint at:
```text
/feishu/webhook
```
**Requirements:** The `aiohttp` Python package must be installed.
You can customize the webhook server bind address and path:
```bash
FEISHU_WEBHOOK_HOST=127.0.0.1 # default: 127.0.0.1
FEISHU_WEBHOOK_PORT=8765 # default: 8765
FEISHU_WEBHOOK_PATH=/feishu/webhook # default: /feishu/webhook
```
When Feishu sends a URL verification challenge (`type: url_verification`), the webhook responds automatically so you can complete the subscription setup in the Feishu developer console. The challenge response is gated on `FEISHU_VERIFICATION_TOKEN` when set — challenge requests with a missing or mismatched token are rejected so an unauthenticated remote cannot prove endpoint control by echoing attacker-controlled challenge data.
## Step 3: Configure Hermes
### Option A: Interactive Setup
```bash
hermes gateway setup
```
Select **Feishu / Lark** and fill in the prompts.
### Option B: Manual Configuration
Add the following to `~/.hermes/.env`:
```bash
FEISHU_APP_ID=cli_xxx
FEISHU_APP_SECRET=secret_xxx
FEISHU_DOMAIN=feishu
FEISHU_CONNECTION_MODE=websocket
# Optional but strongly recommended
FEISHU_ALLOWED_USERS=ou_xxx,ou_yyy
FEISHU_HOME_CHANNEL=oc_xxx
```
`FEISHU_DOMAIN` accepts:
- `feishu` for Feishu China
- `lark` for Lark international
## Step 4: Start the Gateway
```bash
hermes gateway
```
Then message the bot from Feishu/Lark to confirm that the connection is live.
## Home Chat
Use `/set-home` in a Feishu/Lark chat to mark it as the home channel for cron job results and cross-platform notifications.
You can also preconfigure it:
```bash
FEISHU_HOME_CHANNEL=oc_xxx
```
## Security
### User Allowlist
For production use, set an allowlist of Feishu Open IDs:
```bash
FEISHU_ALLOWED_USERS=ou_xxx,ou_yyy
```
If you leave the allowlist empty, anyone who can reach the bot may be able to use it. In group chats, the allowlist is checked against the sender's open_id before the message is processed.
### Webhook Encryption Key
When running in webhook mode, set an encryption key to enable signature verification of inbound webhook payloads:
```bash
FEISHU_ENCRYPT_KEY=your-encrypt-key
```
This key is found in the **Event Subscriptions** section of your Feishu app configuration. When set, the adapter verifies every webhook request using the signature algorithm:
```
SHA256(timestamp + nonce + encrypt_key + body)
```
The computed hash is compared against the `x-lark-signature` header using timing-safe comparison. Requests with invalid or missing signatures are rejected with HTTP 401.
:::tip
In WebSocket mode, signature verification is handled by the SDK itself, so `FEISHU_ENCRYPT_KEY` is optional. In webhook mode, it is strongly recommended for production.
:::
### Verification Token
An additional layer of authentication that checks the `token` field inside webhook payloads:
```bash
FEISHU_VERIFICATION_TOKEN=your-verification-token
```
This token is also found in the **Event Subscriptions** section of your Feishu app. When set, every inbound webhook payload must contain a matching `token` in its `header` object. Mismatched tokens are rejected with HTTP 401.
Both `FEISHU_ENCRYPT_KEY` and `FEISHU_VERIFICATION_TOKEN` can be used together for defense in depth.
## Group Message Policy
The `FEISHU_GROUP_POLICY` environment variable controls whether and how Hermes responds in group chats:
```bash
FEISHU_GROUP_POLICY=allowlist # default
```
| Value | Behavior |
|-------|----------|
| `open` | Hermes responds to @mentions from any user in any group. |
| `allowlist` | Hermes only responds to @mentions from users listed in `FEISHU_ALLOWED_USERS`. |
| `disabled` | Hermes ignores all group messages entirely. |
In all modes, the bot must be explicitly @mentioned (or @all) in the group before the message is processed. Direct messages always bypass this gate.
Set `FEISHU_REQUIRE_MENTION=false` to let Hermes read all group traffic without requiring an @mention:
```bash
FEISHU_REQUIRE_MENTION=false
```
For per-chat control, set `require_mention` on a `group_rules` entry — see [Per-Group Access Control](#per-group-access-control) below.
### Bot Identity
Hermes auto-detects the bot's `open_id` and display name on startup. You only need to set these manually when auto-detection cannot reach the Feishu API, or when your app uses tenant-scoped user IDs:
```bash
FEISHU_BOT_OPEN_ID=ou_xxx # only when auto-detection fails
FEISHU_BOT_USER_ID=xxx # required if your app uses sender_id_type=user_id
FEISHU_BOT_NAME=MyBot # only when auto-detection fails
```
## Bot-to-Bot Messaging
By default Hermes ignores messages sent by other bots. Enable bot-to-bot messaging when you want Hermes to participate in A2A orchestration or receive notifications from other bots in the same group.
```bash
FEISHU_ALLOW_BOTS=mentions # default: none
```
| Value | Behavior |
|-------|----------|
| `none` | Ignore all messages from other bots (default). |
| `mentions` | Accept only when the peer bot @mentions Hermes. |
| `all` | Accept every peer bot message. |
Also configurable as `feishu.allow_bots` in `config.yaml` (env wins when both are set).
Peer bots do not need to be added to `FEISHU_ALLOWED_USERS` — that allowlist applies to human senders only.
Grant the `application:bot.basic_info:read` scope to display peer bot names; without it, peer bots still route correctly but appear as their `open_id`.
## Interactive Card Actions
When users click buttons or interact with interactive cards sent by the bot, the adapter routes these as synthetic `/card` command events:
- Button clicks become: `/card button {"key": "value", ...}`
- The action's `value` payload from the card definition is included as JSON.
- Card actions are deduplicated with a 15-minute window to prevent double processing.
Gateway-driven update prompts use a native Feishu `Yes` / `No` card instead of falling back to plain text replies. When `hermes update --gateway` needs confirmation, the adapter records the selected answer in Hermes's `.update_response` file and replaces the card inline with a resolved state.
Card action events are dispatched with `MessageType.COMMAND`, so they flow through the normal command processing pipeline.
This is also how **command approval** works — when the agent needs to run a dangerous command, it sends an interactive card with Allow Once / Session / Always / Deny buttons. The user clicks a button, and the card action callback delivers the approval decision back to the agent.
### Required Feishu App Configuration
Interactive cards require **three** configuration steps in the Feishu Developer Console. Missing any of them causes error **200340** when users click card buttons.
1. **Subscribe to the card action event:**
In **Event Subscriptions**, add `card.action.trigger` to your subscribed events.
2. **Enable the Interactive Card capability:**
In **App Features > Bot**, ensure the **Interactive Card** toggle is enabled. This tells Feishu that your app can receive card action callbacks.
3. **Configure the Card Request URL (webhook mode only):**
In **App Features > Bot > Message Card Request URL**, set the URL to the same endpoint as your event webhook (e.g. `https://your-server:8765/feishu/webhook`). In WebSocket mode this is handled automatically by the SDK.
:::warning
Without all three steps, Feishu will successfully *send* interactive cards (sending only requires `im:message:send` permission), but clicking any button will return error 200340. The card appears to work — the error only surfaces when a user interacts with it.
:::
## Document Comment Intelligent Reply
Beyond chat, the adapter can also answer `@`-mentions left on **Feishu/Lark documents**. When a user comments on a document (local text selection or whole-doc comment) and @-mentions the bot, Hermes reads the document plus the surrounding comment thread and posts an LLM reply inline on the thread.
Powered by the `drive.notice.comment_add_v1` event, the handler:
- Fetches the document content and comment timeline in parallel (20 messages for whole-doc threads, 12 for local-selection threads).
- Runs the agent with the `feishu_doc` + `feishu_drive` toolsets scoped to that single comment session.
- Chunks replies at 4000 chars and posts them back as threaded replies.
- Caches per-document sessions for 1 hour with a 50-message cap so follow-up comments on the same doc keep context.
### 3-Tier Access Control
Document-comment replies are **explicit-grant only** — there is no implicit allow-all mode. Permissions resolve in this order (first match wins, per field):
1. **Exact doc** — rule scoped to a specific document token.
2. **Wildcard** — rule that matches a pattern of docs.
3. **Top-level** — default rule for the workspace.
Two policies are available per rule:
- **`allowlist`** — a static list of users / tenants.
- **`pairing`** — static list ∪ runtime-approved store. Useful for rollouts where moderators can grant access live.
Rules live in `~/.hermes/feishu_comment_rules.json` (pairing grants in `~/.hermes/feishu_comment_pairing.json`) with mtime-cached hot-reload — edits take effect on the next comment event without restarting the gateway.
CLI:
```bash
# Inspect current rules and pairing state
python -m gateway.platforms.feishu_comment_rules status
# Simulate an access check for a specific doc + user
python -m gateway.platforms.feishu_comment_rules check <fileType:fileToken> <user_open_id>
# Manage pairing grants at runtime
python -m gateway.platforms.feishu_comment_rules pairing list
python -m gateway.platforms.feishu_comment_rules pairing add <user_open_id>
python -m gateway.platforms.feishu_comment_rules pairing remove <user_open_id>
```
### Required Feishu App Configuration
On top of the chat/card permissions already granted, add the drive comment event:
- Subscribe to `drive.notice.comment_add_v1` in **Event Subscriptions**.
- Grant the `docs:doc:readonly` and `drive:drive:readonly` scopes so the handler can read document content.
## Meeting Invitation Events
You can invite the Hermes Feishu/Lark bot into a video meeting the same way you invite a human participant. When the bot receives the meeting invitation event, Hermes can automatically start an agent turn that attempts to join the meeting.
Powered by the `vc.bot.meeting_invited_v1` event, the flow is:
- A user invites the bot to a Feishu/Lark video meeting.
- Feishu/Lark sends Hermes the meeting invitation event.
- Hermes extracts the inviter, meeting topic, and meeting number.
- If the inviter is authorized by the normal gateway allowlist or pairing policy, the agent receives the meeting number and tries to join automatically.
- If the invite is malformed, or the agent cannot join, Hermes drops the event or replies to the inviter with a concise explanation.
Malformed invitations that do not include both an inviter and a `meeting_no` are ignored.
### Required Feishu App Configuration
On top of the chat/card permissions already granted, add the video-meeting invitation event:
- Subscribe to `vc.bot.meeting_invited_v1` in **Event Subscriptions**.
- Enable the Video Conferencing permission scope prompted by the Feishu/Lark developer console for that event.
- Keep `im:message` and `im:message:send_as_bot` enabled so Hermes can reply to the inviter.
- Ensure the gateway user allowlist or pairing policy authorizes the inviter. Meeting invitations do not bypass normal gateway access checks.
## Media Support
### Inbound (receiving)
The adapter receives and caches the following media types from users:
| Type | Extensions | How it's processed |
|------|-----------|-------------------|
| **Images** | .jpg, .jpeg, .png, .gif, .webp, .bmp | Downloaded via Feishu API and cached locally |
| **Audio** | .ogg, .mp3, .wav, .m4a, .aac, .flac, .opus, .webm | Downloaded and cached; small text files are auto-extracted |
| **Video** | .mp4, .mov, .avi, .mkv, .webm, .m4v, .3gp | Downloaded and cached as documents |
| **Files** | .pdf, .doc, .docx, .xls, .xlsx, .ppt, .pptx, and more | Downloaded and cached as documents |
Media from rich-text (post) messages, including inline images and file attachments, is also extracted and cached.
For small text-based documents (.txt, .md), the file content is automatically injected into the message text so the agent can read it directly without needing tools.
### Outbound (sending)
| Method | What it sends |
|--------|--------------|
| `send` | Text or rich post messages (auto-detected based on markdown content) |
| `send_image` / `send_image_file` | Uploads image to Feishu, then sends as native image bubble (with optional caption) |
| `send_document` | Uploads file to Feishu API, then sends as file attachment |
| `send_voice` | Uploads audio file as a Feishu file attachment |
| `send_video` | Uploads video and sends as native media message |
| `send_animation` | GIFs are downgraded to file attachments (Feishu has no native GIF bubble) |
File upload routing is automatic based on extension:
- `.ogg`, `.opus` → uploaded as `opus` audio
- `.mp4`, `.mov`, `.avi`, `.m4v` → uploaded as `mp4` media
- `.pdf`, `.doc(x)`, `.xls(x)`, `.ppt(x)` → uploaded with their document type
- Everything else → uploaded as a generic stream file
## Markdown Rendering and Post Fallback
When outbound text contains markdown formatting (headings, bold, lists, code blocks, links, etc.), the adapter automatically sends it as a Feishu **post** message with an embedded `md` tag rather than as plain text. This enables rich rendering in the Feishu client.
If the Feishu API rejects the post payload (e.g., due to unsupported markdown constructs), the adapter automatically falls back to sending as plain text with markdown stripped. This two-stage fallback ensures messages are always delivered.
Plain text messages (no markdown detected) are sent as the simple `text` message type.
## Processing Status Reactions
While the agent is working, the bot shows a `Typing` reaction on your message. It's cleared when the reply arrives, or replaced with `CrossMark` if processing failed.
Set `FEISHU_REACTIONS=false` to turn it off.
## Burst Protection and Batching
The adapter includes debouncing for rapid message bursts to avoid overwhelming the agent:
### Text Batching
When a user sends multiple text messages in quick succession, they are merged into a single event before being dispatched:
| Setting | Env Var | Default |
|---------|---------|---------|
| Quiet period | `HERMES_FEISHU_TEXT_BATCH_DELAY_SECONDS` | 0.6s |
| Max messages per batch | `HERMES_FEISHU_TEXT_BATCH_MAX_MESSAGES` | 8 |
| Max characters per batch | `HERMES_FEISHU_TEXT_BATCH_MAX_CHARS` | 4000 |
### Media Batching
Multiple media attachments sent in quick succession (e.g., dragging several images) are merged into a single event:
| Setting | Env Var | Default |
|---------|---------|---------|
| Quiet period | `HERMES_FEISHU_MEDIA_BATCH_DELAY_SECONDS` | 0.8s |
### Per-Chat Serialization
Messages within the same chat are processed serially (one at a time) to maintain conversation coherence. Each chat has its own lock, so messages in different chats are processed concurrently.
## Rate Limiting (Webhook Mode)
In webhook mode, the adapter enforces per-IP rate limiting to protect against abuse:
- **Window:** 60-second sliding window
- **Limit:** 120 requests per window per (app_id, path, IP) triple
- **Tracking cap:** Up to 4096 unique keys tracked (prevents unbounded memory growth)
Requests that exceed the limit receive HTTP 429 (Too Many Requests).
### Webhook Anomaly Tracking
The adapter tracks consecutive error responses per IP address. After 25 consecutive errors from the same IP within a 6-hour window, a warning is logged. This helps detect misconfigured clients or probing attempts.
Additional webhook protections:
- **Body size limit:** 1 MB maximum
- **Body read timeout:** 30 seconds
- **Content-Type enforcement:** Only `application/json` is accepted
## WebSocket Tuning
When using `websocket` mode, you can customize reconnect and ping behavior:
```yaml
platforms:
feishu:
extra:
ws_reconnect_interval: 120 # Seconds between reconnect attempts (default: 120)
ws_ping_interval: 30 # Seconds between WebSocket pings (optional; SDK default if unset)
```
| Setting | Config key | Default | Description |
|---------|-----------|---------|-------------|
| Reconnect interval | `ws_reconnect_interval` | 120s | How long to wait between reconnection attempts |
| Ping interval | `ws_ping_interval` | _(SDK default)_ | Frequency of WebSocket keepalive pings |
## Per-Group Access Control
Beyond the global `FEISHU_GROUP_POLICY`, you can set fine-grained rules per group chat using `group_rules` in config.yaml:
```yaml
platforms:
feishu:
extra:
default_group_policy: "open" # Default for groups not in group_rules
admins: # Users who can manage bot settings
- "ou_admin_open_id"
group_rules:
"oc_group_chat_id_1":
policy: "allowlist" # open | allowlist | blacklist | admin_only | disabled
allowlist:
- "ou_user_open_id_1"
- "ou_user_open_id_2"
"oc_group_chat_id_2":
policy: "admin_only"
"oc_group_chat_id_3":
policy: "blacklist"
blacklist:
- "ou_blocked_user"
"oc_free_chat":
policy: "open"
require_mention: false # overrides FEISHU_REQUIRE_MENTION for this chat
```
| Policy | Description |
|--------|-------------|
| `open` | Anyone in the group can use the bot |
| `allowlist` | Only users in the group's `allowlist` can use the bot |
| `blacklist` | Everyone except users in the group's `blacklist` can use the bot |
| `admin_only` | Only users in the global `admins` list can use the bot in this group |
| `disabled` | Bot ignores all messages in this group |
Set `require_mention: false` on a `group_rules` entry to skip the @-mention requirement for that specific chat. When omitted, the chat inherits the global `FEISHU_REQUIRE_MENTION` value.
Groups not listed in `group_rules` fall back to `default_group_policy` (defaults to the value of `FEISHU_GROUP_POLICY`).
## Deduplication
Inbound messages are deduplicated using message IDs with a 24-hour TTL. The dedup state is persisted across restarts to `~/.hermes/feishu_seen_message_ids.json`.
| Setting | Env Var | Default |
|---------|---------|---------|
| Cache size | `HERMES_FEISHU_DEDUP_CACHE_SIZE` | 2048 entries |
## All Environment Variables
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `FEISHU_APP_ID` | ✅ | — | Feishu/Lark App ID |
| `FEISHU_APP_SECRET` | ✅ | — | Feishu/Lark App Secret |
| `FEISHU_DOMAIN` | — | `feishu` | `feishu` (China) or `lark` (international) |
| `FEISHU_CONNECTION_MODE` | — | `websocket` | `websocket` or `webhook` |
| `FEISHU_ALLOWED_USERS` | — | _(empty)_ | Comma-separated open_id list for user allowlist |
| `FEISHU_ALLOW_BOTS` | — | `none` | Accept messages from other bots: `none`, `mentions`, or `all` |
| `FEISHU_REQUIRE_MENTION` | — | `true` | Whether group messages must @mention the bot |
| `FEISHU_HOME_CHANNEL` | — | — | Chat ID for cron/notification output |
| `FEISHU_ENCRYPT_KEY` | — | _(empty)_ | Encrypt key for webhook signature verification |
| `FEISHU_VERIFICATION_TOKEN` | — | _(empty)_ | Verification token for webhook payload auth |
| `FEISHU_GROUP_POLICY` | — | `allowlist` | Group message policy: `open`, `allowlist`, `disabled` |
| `FEISHU_BOT_OPEN_ID` | — | _(empty)_ | Bot's open_id (for @mention detection) |
| `FEISHU_BOT_USER_ID` | — | _(empty)_ | Bot's user_id (for @mention detection) |
| `FEISHU_BOT_NAME` | — | _(empty)_ | Bot's display name (for @mention detection) |
| `FEISHU_WEBHOOK_HOST` | — | `127.0.0.1` | Webhook server bind address |
| `FEISHU_WEBHOOK_PORT` | — | `8765` | Webhook server port |
| `FEISHU_WEBHOOK_PATH` | — | `/feishu/webhook` | Webhook endpoint path |
| `HERMES_FEISHU_DEDUP_CACHE_SIZE` | — | `2048` | Max deduplicated message IDs to track |
| `HERMES_FEISHU_TEXT_BATCH_DELAY_SECONDS` | — | `0.6` | Text burst debounce quiet period |
| `HERMES_FEISHU_TEXT_BATCH_MAX_MESSAGES` | — | `8` | Max messages merged per text batch |
| `HERMES_FEISHU_TEXT_BATCH_MAX_CHARS` | — | `4000` | Max characters merged per text batch |
| `HERMES_FEISHU_MEDIA_BATCH_DELAY_SECONDS` | — | `0.8` | Media burst debounce quiet period |
WebSocket and per-group ACL settings are configured via `config.yaml` under `platforms.feishu.extra` (see [WebSocket Tuning](#websocket-tuning) and [Per-Group Access Control](#per-group-access-control) above).
## Troubleshooting
| Problem | Fix |
|---------|-----|
| `lark-oapi not installed` | Install the SDK: `pip install lark-oapi` |
| `websockets not installed; websocket mode unavailable` | Install websockets: `pip install websockets` |
| `aiohttp not installed; webhook mode unavailable` | Install aiohttp: `pip install aiohttp` |
| `FEISHU_APP_ID or FEISHU_APP_SECRET not set` | Set both env vars or configure via `hermes gateway setup` |
| `Another local Hermes gateway is already using this Feishu app_id` | Only one Hermes instance can use the same app_id at a time. Stop the other gateway first. |
| Bot doesn't respond in groups | Ensure the bot is @mentioned, check `FEISHU_GROUP_POLICY`, and verify the sender is in `FEISHU_ALLOWED_USERS` if policy is `allowlist` |
| `Webhook rejected: invalid verification token` | Ensure `FEISHU_VERIFICATION_TOKEN` matches the token in your Feishu app's Event Subscriptions config |
| `Webhook rejected: invalid signature` | Ensure `FEISHU_ENCRYPT_KEY` matches the encrypt key in your Feishu app config |
| Post messages show as plain text | The Feishu API rejected the post payload; this is normal fallback behavior. Check logs for details. |
| Images/files not received by bot | Grant `im:message` and `im:resource` permission scopes to your Feishu app |
| Bot identity not auto-detected | Usually a transient network issue reaching Feishu's bot info endpoint. Set `FEISHU_BOT_OPEN_ID` and `FEISHU_BOT_NAME` manually as a workaround. |
| Peer bot messages still ignored after enabling `FEISHU_ALLOW_BOTS` | Hermes can't identify itself yet — set `FEISHU_BOT_OPEN_ID` (and `FEISHU_BOT_USER_ID` if your app uses `sender_id_type=user_id`). |
| Peer bots show as `ou_xxxxxx` instead of by name | Grant the `application:bot.basic_info:read` scope. |
| Error 200340 when clicking approval buttons | Enable **Interactive Card** capability and configure **Card Request URL** in the Feishu Developer Console. See [Required Feishu App Configuration](#required-feishu-app-configuration) above. |
| `Webhook rate limit exceeded` | More than 120 requests/minute from the same IP. This is usually a misconfiguration or loop. |
## Toolset
Feishu / Lark uses the `hermes-feishu` platform preset, which includes the same core tools as Telegram and other gateway-based messaging platforms.
website/docs/user-guide/messaging/google_chat.md
+423
View File
@@ -0,0 +1,423 @@
---
sidebar_position: 12
title: "Google Chat"
description: "Set up Hermes Agent as a Google Chat bot using Cloud Pub/Sub"
---
# Google Chat Setup
Connect Hermes Agent to Google Chat as a bot. The integration uses Cloud Pub/Sub
pull subscriptions for inbound events and the Chat REST API for outbound messages.
Equivalent ergonomics to Slack Socket Mode or Telegram long-polling: your Hermes
process does not need a public URL, a tunnel, or a TLS certificate. It connects,
authenticates, and listens on a subscription — the same way a Telegram bot listens
on a token.
> Run `hermes gateway setup` and pick **Google Chat** for a guided walk-through.
:::note Workspace edition
Google Chat is part of Google Workspace. You can use this integration with a
personal Workspace (`@yourdomain.com` registered through Google) or a work
Workspace where you have the Admin rights to publish an app. Gmail-only accounts
cannot host Chat apps.
:::
## Overview
| Component | Value |
|-----------|-------|
| **Libraries** | `google-cloud-pubsub`, `google-api-python-client`, `google-auth` |
| **Inbound transport** | Cloud Pub/Sub pull subscription (no public endpoint) |
| **Outbound transport** | Chat REST API (`chat.googleapis.com`) |
| **Authentication** | Service Account JSON with `roles/pubsub.subscriber` on the subscription |
| **User identification** | Chat resource names (`users/{id}`) + email |
---
## Step 1: Create or pick a GCP project
You need a Google Cloud project to host the Pub/Sub topic. If you don't have one,
create it at [console.cloud.google.com](https://console.cloud.google.com) —
personal accounts get a free tier that easily covers bot traffic.
Note the project ID (e.g., `my-chat-bot-123`). You'll use it in every subsequent
step.
---
## Step 2: Enable two APIs
In the console, go to **APIs & Services → Library** and enable:
- **Google Chat API**
- **Cloud Pub/Sub API**
Both are free for the volumes a personal bot generates.
---
## Step 3: Create a Service Account
**IAM & Admin → Service Accounts → Create Service Account.**
- Name: `hermes-chat-bot`
- Skip the "Grant this service account access to project" step. IAM on the specific
subscription is all you need — do **NOT** grant project-level Pub/Sub roles.
After creation, open the SA, go to **Keys → Add Key → Create new key → JSON** and
download the file. Save it somewhere only Hermes can read (e.g.,
`~/.hermes/google-chat-sa.json`, `chmod 600`).
:::caution There is NO "Chat Bot Caller" role
A common mistake is to search for a Chat-specific IAM role and grant it at the
project level. That role doesn't exist. Chat bot authority comes from being
installed in a space, not from IAM. All your SA needs is Pub/Sub subscriber on
the subscription you create in the next step.
:::
---
## Step 4: Create the Pub/Sub topic and subscription
**Pub/Sub → Topics → Create topic.**
- Topic ID: `hermes-chat-events`
- Leave the defaults for everything else.
After creation, the topic's detail page has a **Subscriptions** tab. Create one:
- Subscription ID: `hermes-chat-events-sub`
- Delivery type: **Pull**
- Message retention: **7 days** (so backlog survives a hermes restart)
- Leave the rest default.
---
## Step 5: IAM binding on the topic (critical)
On the **topic** (not the subscription), add an IAM principal:
- Principal: `chat-api-push@system.gserviceaccount.com`
- Role: `Pub/Sub Publisher`
Without this, Google Chat cannot publish events to your topic and your bot will
never receive anything.
---
## Step 6: IAM binding on the subscription
On the **subscription**, add your own Service Account as a principal:
- Principal: `hermes-chat-bot@<your-project>.iam.gserviceaccount.com`
- Role: `Pub/Sub Subscriber`
Also grant `Pub/Sub Viewer` on the same subscription — Hermes calls
`subscription.get()` at startup as a reachability check.
---
## Step 7: Configure the Chat app
Go to **APIs & Services → Google Chat API → Configuration**.
- **App name**: whatever you want users to see ("Hermes" is reasonable).
- **Avatar URL**: any public PNG (Google has some defaults).
- **Description**: a short sentence shown in the app directory.
- **Functionality**: enable **Receive 1:1 messages** and **Join spaces and group
conversations**.
- **Connection settings**: select **Cloud Pub/Sub**, enter the topic name
`projects/<your-project>/topics/hermes-chat-events`.
- **Visibility**: restrict to your workspace (or specific users) — do not publish
to everyone while you're testing.
Save.
---
## Step 8: Install the bot in a test space
Open Google Chat in a browser. Start a DM with your app by searching for its name
in the **+ New Chat** menu. The first time you message it, Google sends an
`ADDED_TO_SPACE` event that Hermes uses to cache the bot's own `users/{id}` for
self-message filtering.
---
## Step 9: Configure Hermes
Add the Google Chat section to `~/.hermes/.env`:
```bash
# Required
GOOGLE_CHAT_PROJECT_ID=my-chat-bot-123
GOOGLE_CHAT_SUBSCRIPTION_NAME=projects/my-chat-bot-123/subscriptions/hermes-chat-events-sub
GOOGLE_CHAT_SERVICE_ACCOUNT_JSON=/home/you/.hermes/google-chat-sa.json
# Authorization — paste the emails of people allowed to talk to the bot
GOOGLE_CHAT_ALLOWED_USERS=you@yourdomain.com,coworker@yourdomain.com
# Optional
GOOGLE_CHAT_HOME_CHANNEL=spaces/AAAA... # default delivery destination for cron jobs
GOOGLE_CHAT_MAX_MESSAGES=1 # Pub/Sub FlowControl; 1 serializes commands per session
GOOGLE_CHAT_MAX_BYTES=16777216 # 16 MiB — cap on in-flight message bytes
```
The project ID also falls back to `GOOGLE_CLOUD_PROJECT`, and the SA path falls
back to `GOOGLE_APPLICATION_CREDENTIALS` — use whichever convention you prefer.
Under a [multi-profile gateway](../multi-profile-gateways.md), every
`GOOGLE_CHAT_*` setting is read from the routed profile's own `.env`; a
secondary profile never inherits the default profile's project, subscription,
or service account. If a profile has no SA configured while the process
environment carries one for another profile, the adapter refuses to fall back
to Application Default Credentials (which would authenticate as that other
profile) and logs an explicit error instead — put
`GOOGLE_CHAT_SERVICE_ACCOUNT_JSON` in that profile's `.env`.
Install the Google Chat adapter dependencies through its maintained installer.
It applies the same pinned security floors used by the runtime checks:
```bash
python -m plugins.platforms.google_chat.oauth --install-deps
```
Start the gateway:
```bash
hermes gateway
```
You should see a log line like:
```
[GoogleChat] Connected; project=my-chat-bot-123, subscription=<redacted>,
bot_user_id=users/XXXX, flow_control(msgs=1, bytes=16777216)
```
Send "hola" in the test DM. The bot posts a "Hermes is thinking…" marker, then
edits that same message in place with the real response — no "message deleted"
tombstones.
### Customizing the working-state marker
The marker text is configurable via `typing_status_text` in
`~/.hermes/config.yaml` — e.g. a kitten assistant named Ada:
```yaml
platforms:
google_chat:
# Custom working-state marker text (default: "Hermes is thinking…").
typing_status_text: "is pouncing… 🐾"
```
Unlike Slack's ephemeral status line, this is a **real posted message** that
gets edited in place with the response — so whatever you set here briefly
appears in the chat as a normal message. Set `typing_indicator: false` to
disable the marker entirely.
---
## Formatting and capabilities
Google Chat renders a limited markdown subset:
| Supported | Not supported |
|-----------|---------------|
| `*bold*`, `_italic_`, `~strike~`, `` `code` `` | Headings, lists |
| Inline images via URL | Interactive Card v2 buttons (v1 of this gateway) |
| Native file attachments (after `/setup-files` — see Step 10) | Native voice notes / circular video notes |
The agent's system prompt includes a Google Chat–specific hint so it knows these
limits and avoids formatting that won't render.
Message size limit: 4000 characters per message. Longer agent responses are
automatically split across multiple messages.
Thread support: when a user replies inside a thread, Hermes detects the
`thread.name` and posts its reply in the same thread, so each thread gets a
separate Hermes session.
### Clarify questions as interactive cards
When the agent asks a multiple-choice clarify question, the adapter renders it
as a native **Card v2** with one button per choice plus an
**"Other / type answer"** button, instead of a plain numbered text list.
Clicking a button answers the question directly (`CARD_CLICKED` events route
the choice back into the waiting session). If the card fails to send, or the
question has no fixed choices, the adapter falls back to the standard text
clarify. No configuration needed.
---
## Step 10: Native attachment delivery (optional)
Out of the box the bot can post text, inline images via URL, and download cards
for audio/video/documents. To deliver **native** Chat attachments — the same
file widget you get when a human drags-and-drops a file — each user authorizes
the bot once via a per-user OAuth flow.
### Why a separate flow
Google Chat's `media.upload` endpoint hard-rejects service-account auth:
> This method doesn't support app authentication with a service account.
> Authenticate with a user account.
There's no IAM role or scope that fixes this. The endpoint only accepts user
credentials. So the bot has to act *as a user* whenever it uploads a file —
specifically, as the user who asked for the file.
### One-time setup (per profile)
1. Go to **APIs & Services → Credentials** in the same GCP project.
2. **Create credentials → OAuth client ID → Desktop app**.
3. Download the JSON. Move it onto the host that runs Hermes.
4. Register the client with Hermes (run under the profile you want it scoped to):
```bash
# Default profile:
python -m plugins.platforms.google_chat.oauth \
--client-secret /path/to/client_secret.json
# A named profile gets its own separate registration:
hermes -p <profile> python -m plugins.platforms.google_chat.oauth \
--client-secret /path/to/client_secret.json
```
That writes the client secret into the active profile's Hermes home (e.g.
`~/.hermes/google_chat_user_client_secret.json` for the default profile). The
client secret is **profile-scoped, not shared across profiles** — each profile
registers its own. This is deliberate: profiles are isolated auth boundaries, so
two profiles can point at different Google OAuth apps / accounts. Register it
once per profile that needs Google Chat attachment delivery.
### Per-user authorization (in chat)
Each user runs the flow once, in their own DM with the bot:
1. They send `/setup-files` to the bot. It replies with status and the next
step.
2. They send `/setup-files start`. The bot replies with an OAuth URL.
3. They open the URL, click **Allow**, and watch the browser fail to load
`http://localhost:1/?...&code=...`. That failure is expected — the auth
code is in the URL bar.
4. They copy the failed URL (or just the `code=...` value) and paste it back
into chat as `/setup-files <PASTED_URL>`. The bot exchanges it for a
refresh token.
The token lands at `~/.hermes/google_chat_user_tokens/<sanitized_email>.json`.
Subsequent file requests in that user's DM use *their* token, so the bot
uploads as them and the message lands in their space.
To revoke later: `/setup-files revoke` deletes only that user's token. Other
users' tokens are untouched.
### Scope
The flow requests exactly one scope: `chat.messages.create`. That covers both
`media.upload` and the `messages.create` that references the uploaded
`attachmentDataRef`. No Drive, no broader Chat scopes — this is least-privilege
on purpose.
### Multi-user behavior
When the asker has no per-user token yet, the bot falls back to a legacy
single-user token at `~/.hermes/google_chat_user_token.json` (if present from
a pre-multi-user install). When neither is available, the bot posts a clear
text notice telling the asker to run `/setup-files`.
A user revoking only clears their own slot. A 401/403 from one user's token
evicts only that user's cache. Users don't disrupt each other.
---
## Troubleshooting
**Bot stays silent after sending "hola."**
1. Check the Pub/Sub subscription has undelivered messages in the console.
If it does, Hermes isn't authenticated — verify `GOOGLE_CHAT_SERVICE_ACCOUNT_JSON`
and that the SA is listed as `Pub/Sub Subscriber` on the subscription.
2. If the subscription has zero messages, Google Chat isn't publishing.
Double-check the IAM binding on the **topic**:
`chat-api-push@system.gserviceaccount.com` must have `Pub/Sub Publisher`.
3. Check `hermes gateway` logs for `[GoogleChat] Connected`. If you see
`[GoogleChat] Config validation failed`, the error message tells you which
env var to fix.
**Bot replies but an error message appears instead of the agent's answer.**
Check logs for `[GoogleChat] Pub/Sub stream died` — if these repeat, your SA
credentials may have been rotated or the subscription deleted. After 10 attempts
the adapter marks itself fatal.
**"403 Forbidden" on every outbound message.**
The bot was removed from the space, or you revoked it in the Chat API console.
Re-install it in the space (the next `ADDED_TO_SPACE` event will re-enable
messaging automatically).
**Too many "Rate limit hit" warnings.**
The Chat API's default quotas allow 60 messages per space per minute. If your
agent produces long streaming responses that exceed that, the adapter retries
with exponential backoff — but you'll still see user-visible latency. Consider
concise responses or raising the quota in the GCP console.
**Bot keeps posting the "/setup-files" notice instead of files.**
The asker has no per-user OAuth token and there's no legacy fallback. Run
`/setup-files` in their DM and follow Step 10. After the exchange completes
the next file request uploads natively without a gateway restart.
**`/setup-files start` says "No client credentials stored."**
The one-time setup wasn't done *for this profile* (the client secret is
profile-scoped, so a registration under one profile won't be seen by another).
From a terminal, run it under the profile the gateway uses:
```bash
# Default profile:
python -m plugins.platforms.google_chat.oauth \
--client-secret /path/to/client_secret.json
# Named profile:
hermes -p <profile> python -m plugins.platforms.google_chat.oauth \
--client-secret /path/to/client_secret.json
```
Then send `/setup-files start` again.
**`/setup-files <PASTED_URL>` says "Token exchange failed."**
The auth code is single-use and short-lived (typically a few minutes). Send
`/setup-files start` to get a fresh URL and retry.
---
## Security notes
- **Service Account scope**: the adapter requests `chat.bot` and `pubsub` scopes.
IAM should be the actual enforcement — grant your SA the minimum
(`roles/pubsub.subscriber` + `roles/pubsub.viewer` on the subscription), not
project-level or org-level Pub/Sub roles.
- **Attachment download protection**: Hermes will only attach the SA bearer
token to URLs whose host matches a short allowlist of Google-owned domains
(`googleapis.com`, `drive.google.com`, `lh[3-6].googleusercontent.com`, and
a few others). Any other host is rejected before the HTTP request, to
protect against SSRF scenarios where a crafted event could redirect the
bearer token to the GCE metadata service.
- **Redaction**: Service Account emails, subscription paths, and topic paths
are stripped from log output by `agent/redact.py`. The debug envelope dump
(`GOOGLE_CHAT_DEBUG_RAW=1`) routes through the same redaction filter and
logs at DEBUG level.
- **Compliance**: if you plan to connect this bot to a regulated workspace
(anything with a data-residency or AI-governance policy), get that approval
before the first install.
- **User OAuth scope**: the per-user attachment flow requests *only*
`chat.messages.create` — the minimum that covers `media.upload` plus the
follow-up `messages.create`. Tokens are persisted as plain JSON at
`~/.hermes/google_chat_user_tokens/<sanitized_email>.json` (filesystem
permissions are the protection — same model as the SA key file). Each
token is owned by exactly one user; revoke is scoped to that user.
website/docs/user-guide/messaging/homeassistant.md
+269
View File
@@ -0,0 +1,269 @@
---
title: Home Assistant
description: Control your smart home with Hermes Agent via Home Assistant integration.
sidebar_label: Home Assistant
sidebar_position: 5
---
# Home Assistant Integration
Hermes Agent integrates with [Home Assistant](https://www.home-assistant.io/) in two ways:
1. **Gateway platform** — subscribes to real-time state changes via WebSocket and responds to events
2. **Smart home tools** — four LLM-callable tools for querying and controlling devices via the REST API
## Setup
### 1. Create a Long-Lived Access Token
1. Open your Home Assistant instance
2. Go to your **Profile** (click your name in the sidebar)
3. Scroll to **Long-Lived Access Tokens**
4. Click **Create Token**, give it a name like "Hermes Agent"
5. Copy the token
### 2. Configure Environment Variables
```bash
# Add to ~/.hermes/.env
# Required: your Long-Lived Access Token
HASS_TOKEN=your-long-lived-access-token
# Optional: HA URL (default: http://homeassistant.local:8123)
HASS_URL=http://192.168.1.100:8123
```
:::info
The `homeassistant` toolset is automatically enabled when `HASS_TOKEN` is set. Both the gateway platform and the device control tools activate from this single token.
:::
### 3. Start the Gateway
```bash
hermes gateway
```
Home Assistant will appear as a connected platform alongside any other messaging platforms (Telegram, Discord, etc.).
## Available Tools
Hermes Agent registers four tools for smart home control:
### `ha_list_entities`
List Home Assistant entities, optionally filtered by domain or area.
**Parameters:**
- `domain` *(optional)* — Filter by entity domain: `light`, `switch`, `climate`, `sensor`, `binary_sensor`, `cover`, `fan`, `media_player`, etc.
- `area` *(optional)* — Filter by area/room name (matches against friendly names): `living room`, `kitchen`, `bedroom`, etc.
**Example:**
```
List all lights in the living room
```
Returns entity IDs, states, and friendly names.
### `ha_get_state`
Get detailed state of a single entity, including all attributes (brightness, color, temperature setpoint, sensor readings, etc.).
**Parameters:**
- `entity_id` *(required)* — The entity to query, e.g., `light.living_room`, `climate.thermostat`, `sensor.temperature`
**Example:**
```
What's the current state of climate.thermostat?
```
Returns: state, all attributes, last changed/updated timestamps.
### `ha_list_services`
List available services (actions) for device control. Shows what actions can be performed on each device type and what parameters they accept.
**Parameters:**
- `domain` *(optional)* — Filter by domain, e.g., `light`, `climate`, `switch`
**Example:**
```
What services are available for climate devices?
```
### `ha_call_service`
Call a Home Assistant service to control a device.
**Parameters:**
- `domain` *(required)* — Service domain: `light`, `switch`, `climate`, `cover`, `media_player`, `fan`, `scene`, `script`
- `service` *(required)* — Service name: `turn_on`, `turn_off`, `toggle`, `set_temperature`, `set_hvac_mode`, `open_cover`, `close_cover`, `set_volume_level`
- `entity_id` *(optional)* — Target entity, e.g., `light.living_room`
- `data` *(optional)* — Additional parameters as a JSON object
**Examples:**
```
Turn on the living room lights
→ ha_call_service(domain="light", service="turn_on", entity_id="light.living_room")
```
```
Set the thermostat to 22 degrees in heat mode
→ ha_call_service(domain="climate", service="set_temperature",
entity_id="climate.thermostat", data={"temperature": 22, "hvac_mode": "heat"})
```
```
Set living room lights to blue at 50% brightness
→ ha_call_service(domain="light", service="turn_on",
entity_id="light.living_room", data={"brightness": 128, "color_name": "blue"})
```
## Gateway Platform: Real-Time Events
The Home Assistant gateway adapter connects via WebSocket and subscribes to `state_changed` events. When a device state changes and matches your filters, it's forwarded to the agent as a message.
### Event Filtering
:::warning Required Configuration
By default, **no events are forwarded**. You must configure at least one of `watch_domains`, `watch_entities`, or `watch_all` to receive events. Without filters, a warning is logged at startup and all state changes are silently dropped.
:::
Configure which events the agent sees in `~/.hermes/config.yaml` under the Home Assistant platform's `extra` section:
```yaml
platforms:
homeassistant:
enabled: true
extra:
watch_domains:
- climate
- binary_sensor
- alarm_control_panel
- light
watch_entities:
- sensor.front_door_battery
ignore_entities:
- sensor.uptime
- sensor.cpu_usage
- sensor.memory_usage
cooldown_seconds: 30
```
| Setting | Default | Description |
|---------|---------|-------------|
| `watch_domains` | *(none)* | Only watch these entity domains (e.g., `climate`, `light`, `binary_sensor`) |
| `watch_entities` | *(none)* | Only watch these specific entity IDs |
| `watch_all` | `false` | Set to `true` to receive **all** state changes (not recommended for most setups) |
| `ignore_entities` | *(none)* | Always ignore these entities (applied before domain/entity filters) |
| `cooldown_seconds` | `30` | Minimum seconds between events for the same entity |
:::tip
Start with a focused set of domains — `climate`, `binary_sensor`, and `alarm_control_panel` cover the most useful automations. Add more as needed. Use `ignore_entities` to suppress noisy sensors like CPU temperature or uptime counters.
:::
### Event Formatting
State changes are formatted as human-readable messages based on domain:
| Domain | Format |
|--------|--------|
| `climate` | "HVAC mode changed from 'off' to 'heat' (current: 21, target: 23)" |
| `sensor` | "changed from 21°C to 22°C" |
| `binary_sensor` | "triggered" / "cleared" |
| `light`, `switch`, `fan` | "turned on" / "turned off" |
| `alarm_control_panel` | "alarm state changed from 'armed_away' to 'triggered'" |
| *(other)* | "changed from 'old' to 'new'" |
### Agent Responses
Outbound messages from the agent are delivered as **Home Assistant persistent notifications** (via `persistent_notification.create`). These appear in the HA notification panel with the title "Hermes Agent".
### Connection Management
- **WebSocket** with 30-second heartbeat for real-time events
- **Automatic reconnection** with backoff: 5s → 10s → 30s → 60s
- **REST API** for outbound notifications (separate session to avoid WebSocket conflicts)
- **Authorization** — HA events are always authorized (no user allowlist needed, since the `HASS_TOKEN` authenticates the connection)
## Security
The Home Assistant tools enforce security restrictions:
:::warning Blocked Domains
The following service domains are **blocked** to prevent arbitrary code execution on the HA host:
- `shell_command` — arbitrary shell commands
- `command_line` — sensors/switches that execute commands
- `python_script` — scripted Python execution
- `pyscript` — broader scripting integration
- `hassio` — addon control, host shutdown/reboot
- `rest_command` — HTTP requests from HA server (SSRF vector)
Attempting to call services in these domains returns an error.
:::
Entity IDs are validated against the pattern `^[a-z_][a-z0-9_]*\.[a-z0-9_]+$` to prevent injection attacks.
## Example Automations
### Morning Routine
```
User: Start my morning routine
Agent:
1. ha_call_service(domain="light", service="turn_on",
entity_id="light.bedroom", data={"brightness": 128})
2. ha_call_service(domain="climate", service="set_temperature",
entity_id="climate.thermostat", data={"temperature": 22})
3. ha_call_service(domain="media_player", service="turn_on",
entity_id="media_player.kitchen_speaker")
```
### Security Check
```
User: Is the house secure?
Agent:
1. ha_list_entities(domain="binary_sensor")
→ checks door/window sensors
2. ha_get_state(entity_id="alarm_control_panel.home")
→ checks alarm status
3. ha_list_entities(domain="lock")
→ checks lock states
4. Reports: "All doors closed, alarm is armed_away, all locks engaged."
```
### Reactive Automation (via Gateway Events)
When connected as a gateway platform, the agent can react to events:
```
[Home Assistant] Front Door: triggered (was cleared)
Agent automatically:
1. ha_get_state(entity_id="binary_sensor.front_door")
2. ha_call_service(domain="light", service="turn_on",
entity_id="light.hallway")
3. Sends notification: "Front door opened. Hallway lights turned on."
```
## Troubleshooting
**Environment variables not picked up.**
The adapter reads credentials from `~/.hermes/.env` (auto-merged at startup) or
from `config.yaml`. Double-check the file lives under the active Hermes profile
home and that there's no stray quoting around the URL/token. Restart the gateway
after editing — env changes are only applied on process start.
**REST auth failing (`401 Unauthorized`).**
The token must be a *Long-Lived Access Token* created from your HA user profile
page (**Profile → Security → Long-lived access tokens**). Short-lived UI
session tokens won't work. Also verify the base URL includes the scheme and
port (e.g. `http://homeassistant.local:8123`) and is reachable from the host
running Hermes — `curl -H "Authorization: Bearer <token>" <url>/api/` should
return `{"message": "API running."}`.
website/docs/user-guide/messaging/index.md
+857
View File
@@ -0,0 +1,857 @@
---
sidebar_position: 1
title: "Messaging Gateway"
description: "Chat with Hermes from Telegram, Discord, Slack, WhatsApp, Signal, SMS, Email, Home Assistant, Mattermost, Matrix, DingTalk, Yuanbao, Microsoft Teams, LINE, Raft, Webhooks, or any OpenAI-compatible frontend via the API server — architecture and setup overview"
---
# Messaging Gateway
Chat with Hermes from Telegram, Discord, Slack, WhatsApp, Signal, SMS, Email, Home Assistant, Mattermost, Matrix, DingTalk, Feishu/Lark, WeCom, Weixin, BlueBubbles (iMessage), QQ, Yuanbao, Microsoft Teams, LINE, ntfy, or your browser. The gateway is a single background process that connects to all your configured platforms, handles sessions, runs cron jobs, and delivers voice messages.
For the full voice feature set — including CLI microphone mode, spoken replies in messaging, and Discord voice-channel conversations — see [Voice Mode](/user-guide/features/voice-mode) and [Use Voice Mode with Hermes](/guides/use-voice-mode-with-hermes).
:::tip
Bots need both a model provider and tool providers (TTS, web). A [Nous Portal](/integrations/nous-portal) subscription bundles all of them.
:::
## Platform Comparison
| Platform | Voice | Images | Files | Threads | Reactions | Typing | Streaming |
|----------|:-----:|:------:|:-----:|:-------:|:---------:|:------:|:---------:|
| Telegram | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ |
| Discord | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Slack | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Google Chat | — | ✅ | ✅ | ✅ | — | ✅ | — |
| WhatsApp | — | ✅ | ✅ | — | — | ✅ | ✅ |
| WhatsApp Cloud API | ✅ | ✅ | ✅ | — | — | ✅ | — |
| Signal | — | ✅ | ✅ | — | — | ✅ | — |
| SMS | — | — | — | — | — | — | — |
| Email | — | ✅ | ✅ | ✅ | — | — | — |
| Home Assistant | — | — | — | — | — | — | — |
| Mattermost | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ |
| Matrix | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| DingTalk | — | ✅ | ✅ | — | ✅ | — | ✅ |
| Feishu/Lark | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| WeCom | ✅ | ✅ | ✅ | — | — | — | — |
| WeCom Callback | — | — | — | — | — | — | — |
| Weixin | ✅ | ✅ | ✅ | — | — | ✅ | — |
| BlueBubbles | — | ✅ | ✅ | — | ✅ | ✅ | — |
| Photon (iMessage) | ✅ | ✅ | ✅ | — | ✅ | ✅ | — |
| QQ | ✅ | ✅ | ✅ | — | — | ✅ | — |
| Yuanbao | ✅ | ✅ | ✅ | — | — | ✅ | ✅ |
| Microsoft Teams | — | ✅ | — | ✅ | — | ✅ | — |
| LINE | — | ✅ | ✅ | — | — | ✅ | — |
| ntfy | — | — | — | — | — | — | — |
| Raft | — | — | — | — | — | — | — |
| IRC | — | — | — | — | — | — | — |
| Buzz | — | ✅ | — | ✅ | — | — | — |
| SimpleX | ✅ | ✅ | ✅ | — | — | ✅ | — |
**Voice** = TTS audio replies and/or voice message transcription. **Images** = send/receive images. **Files** = send/receive file attachments. **Threads** = threaded conversations. **Reactions** = emoji reactions on messages. **Typing** = typing indicator while processing. **Streaming** = progressive message updates via editing.
:::note Hermes Relay
[Hermes Relay](/user-guide/messaging/relay) (experimental) is not a chat platform itself — it is a connector system that fronts platforms like Discord, Telegram, Slack, and WhatsApp through an external connector that owns the platform credentials. Capabilities (media, native approval/clarify prompts, reactions, threads, typing, streaming) are negotiated per connector at handshake rather than fixed in the table above.
:::
## Architecture
```mermaid
flowchart TB
subgraph Gateway["Hermes Gateway"]
subgraph Adapters["Platform adapters"]
tg[Telegram]
dc[Discord]
wa[WhatsApp]
sl[Slack]
gc[Google Chat]
sig[Signal]
sms[SMS]
em[Email]
ha[Home Assistant]
mm[Mattermost]
mx[Matrix]
dt[DingTalk]
fs[Feishu/Lark]
wc[WeCom]
wcb[WeCom Callback]
wx[Weixin]
bb[BlueBubbles]
qq[QQ]
yb[Yuanbao]
ms[Microsoft Teams]
api["API Server<br/>(OpenAI-compatible)"]
wh[Webhooks]
end
store["Session store<br/>per chat"]
agent["AIAgent<br/>run_agent.py"]
cron["Cron scheduler<br/>ticks every 60s"]
end
tg --> store
dc --> store
wa --> store
sl --> store
gc --> store
sig --> store
sms --> store
em --> store
ha --> store
mm --> store
mx --> store
dt --> store
fs --> store
wc --> store
wcb --> store
wx --> store
bb --> store
qq --> store
yb --> store
ms --> store
api --> store
wh --> store
store --> agent
cron --> store
```
Each platform adapter receives messages, routes them through a per-chat session store, and dispatches them to the AIAgent for processing. The gateway also runs the cron scheduler, ticking every 60 seconds to execute any due jobs.
## Intentional Silence Tokens
For group chats, hooks, and automation flows, Hermes supports explicit silence tokens. If the agent's final response is exactly one supported token, the gateway suppresses outbound delivery and sends nothing to the chat.
Supported tokens:
- `[SILENT]`
- `SILENT`
- `NO_REPLY`
- `NO REPLY`
Whitespace and case are normalized, but the whole final response must be the token. A sentence like "Use `[SILENT]` when nothing changed" is delivered normally.
Silence is a delivery decision only. Hermes keeps the assistant silence turn in the session transcript, so the conversation still alternates normally:
```text
user: side-channel chatter
assistant: [SILENT] # stored, not delivered
user: next message
```
Failed turns still surface as errors; Hermes does not hide failures just because the text resembles a silence token.
## Quick Setup
The easiest way to configure messaging platforms is the interactive wizard:
```bash
hermes gateway setup # Interactive setup for all messaging platforms
```
This walks you through configuring each platform with arrow-key selection, shows which platforms are already configured, and offers to start/restart the gateway when done.
## Gateway Commands
```bash
hermes gateway # Run in foreground
hermes gateway setup # Configure messaging platforms interactively
hermes gateway install # Install as a user service (Linux) / launchd service (macOS)
sudo hermes gateway install --system # Linux only: install a boot-time system service
hermes gateway start # Start the default service
hermes gateway stop # Stop the default service
hermes gateway status # Check default service status
hermes gateway status --system # Linux only: inspect the system service explicitly
```
### Optional Linux event-loop watchdog
A systemd-managed gateway can opt into process recovery when Python's asyncio
event loop stops receiving scheduling time. This covers whole-process stalls
that also prevent platform-specific liveness tasks from running:
```yaml title="~/.hermes/config.yaml"
gateway:
systemd_watchdog_seconds: 120
```
Regenerate the service unit after changing this setting:
```bash
hermes gateway install --force
```
A positive value makes the generated unit use `Type=notify`,
`NotifyAccess=main`, and the matching `WatchdogSec`. Hermes sends heartbeats
only while its event loop is making timely progress; systemd restarts the
process when they stop. The default `0` keeps the existing `Type=simple`
behavior. This setting is Linux/systemd-only and does not treat an ordinary
platform network disconnect as an event-loop failure.
## Chat Commands (Inside Messaging)
| Command | Description |
|---------|-------------|
| `/new` or `/reset` | Start a fresh conversation |
| `/model [provider:model]` | Show or change the model (supports `provider:model` syntax) |
| `/personality [name]` | Set a personality (`none` to reset) |
| `/retry` | Retry the last message |
| `/undo` | Remove the last exchange |
| `/status` | Show session info |
| `/whoami` | Show your slash command access on this scope (admin / user / unrestricted) |
| `/stop` | Stop the running agent |
| `/approve` | Approve a pending dangerous command |
| `/deny` | Reject a pending dangerous command |
| `/sethome` | Set this chat as the home channel |
| `/compress` | Manually compress conversation context |
| `/title [name]` | Set or show the session title |
| `/resume [name]` | Resume a previously named session |
| `/sessions [all] [search <query>]` | List previous sessions; `search <query>` filters by title or id |
| `/usage` | Show token usage for this session (`/usage reset [--force]` redeems a banked Codex limit reset) |
| `/insights [days]` | Show usage insights and analytics |
| `/reasoning [level\|show\|hide]` | Change reasoning effort or toggle reasoning display |
| `/voice [on\|off\|tts\|join\|leave\|status]` | Control messaging voice replies and Discord voice-channel behavior |
| `/rollback [number]` | List or restore filesystem checkpoints |
| `/bg <prompt>` | Run a prompt in a separate background session |
| `/btw <question>` | Ask a side question about the current conversation without interrupting it |
| `/reload-mcp` | Reload MCP servers from config |
| `/update` | Update Hermes Agent to the latest version |
| `/help` | Show available commands |
| `/<skill-name>` | Invoke any installed skill |
## Session Management
### Session Persistence
Sessions persist across messages until they reset. The agent remembers your conversation context.
### Finding Past Sessions (`/sessions`)
`/sessions` lists your previous sessions for the current chat — including the one you're in now, marked `(current)` — and `/sessions <name>` resumes one (shorthand for `/resume`). When the list grows long, `/sessions search <query>` (alias `find`) filters by title or session-id match, ordered by most recently active. Cross-origin listing with `/sessions all` is admin-only — regular users get a notice explaining the list stayed chat-scoped, and only ever see sessions from their own chat origin.
### Persistent `/model` Overrides
A `/model` switch in a gateway chat applies to that session and now **survives gateway restarts**: the model/provider choice is persisted to the session store and rehydrated on first use after a restart (credentials are re-resolved at load time and never written to disk). `/new` (or `/reset`) clears the override, and `/model <name> --global` writes it through to `config.yaml` instead. `/model <name> --once` applies for a single turn only.
### Delivery Reliability
Final agent responses are recorded in a durable **delivery ledger**
(`state.db`) around each platform send. If the gateway crashes or restarts
between producing a response and the platform confirming receipt, the next
boot redelivers the stored response instead of losing it — or re-running the
whole turn.
Semantics are honest at-least-once:
- A response whose send **never started** is redelivered as-is.
- A response that was **mid-send** when the gateway died (the platform may or
may not have received it) is redelivered with a visible
"♻️ Recovered reply — … may be a duplicate" prefix. Ambiguity is labeled,
never silently resent.
- Redelivery is bounded: 3 attempts, 24-hour freshness, then the row is
abandoned. Delivered rows are pruned after 7 days.
Disable with `gateway.delivery_ledger: false` in `config.yaml` (restores the
old behavior: in-flight responses are lost on crash).
### Reset Policies
**By default sessions never auto-reset** — context lives until you `/reset`
manually or context compression kicks in. If you want automatic resets, opt in
with the `session_reset` section in `~/.hermes/config.yaml`:
```yaml
session_reset:
mode: idle # "idle", "daily", "both", or "none" (default)
idle_minutes: 1440 # for idle/both: minutes of inactivity before reset
at_hour: 4 # for daily/both: hour of day (0-23, local time)
```
| Mode | Description |
|------|-------------|
| `none` | Never auto-reset (default) |
| `daily` | Reset at a specific hour each day |
| `idle` | Reset after N minutes of inactivity |
| `both` | Whichever triggers first |
A live background process (started with `terminal(background=true)`) normally
protects its session from resetting so output isn't lost. To stop a forgotten
process — say a preview server — from pinning a session open forever, a
background process older than `bg_process_max_age_hours` (default **24**) no
longer blocks reset. The process is **not** killed, only ignored by the reset
guard. Set it to `0` to disable the cutoff (any live process blocks reset, the
old behavior), or raise it if you run legitimate multi-day jobs whose liveness
should keep the conversation open.
Configure per-platform overrides in `~/.hermes/gateway.json`:
```json
{
"reset_by_platform": {
"telegram": { "mode": "idle", "idle_minutes": 240 },
"discord": { "mode": "idle", "idle_minutes": 60 }
}
}
```
## Per-Channel Model & System Prompt Overrides
Different channels can run different models and personas from a **single gateway** — e.g. a cheap fast model in `#daily` and a frontier model with a specialist prompt in `#dev`. Configure `channel_overrides` under the platform in `~/.hermes/gateway-config.yaml`:
```yaml
platforms:
discord:
enabled: true
channel_overrides:
"123456789012345678": # channel/thread id
model: anthropic/claude-sonnet-4.6
provider: anthropic
system_prompt: "You are the #dev channel code-review specialist."
"987654321098765432":
model: openai/gpt-5-mini
```
Details:
- All three keys are optional — set only `model`, only `system_prompt`, or any combination. Unset fields fall back to the global defaults.
- Lookup order is exact channel/thread id first, then the **parent** channel/forum id — so Discord threads inherit their parent channel's override automatically.
- Resolution priority for the model is: session `/model` override → `channel_overrides` → global config. A user running `/model` in a chat still wins over the channel default.
- The `system_prompt` override replaces the global gateway prompt for that channel (it is ephemeral — injected per turn, not stored in history).
## Security
**By default, the gateway denies all users who are not in an allowlist or paired via DM.** This is the safe default for a bot with terminal access.
```bash
# Restrict to specific users (recommended):
TELEGRAM_ALLOWED_USERS=123456789,987654321
DISCORD_ALLOWED_USERS=123456789012345678
SIGNAL_ALLOWED_USERS=+155****4567,+155****6543
SMS_ALLOWED_USERS=+155****4567,+155****6543
EMAIL_ALLOWED_USERS=trusted@example.com,colleague@work.com
MATTERMOST_ALLOWED_USERS=3uo8dkh1p7g1mfk49ear5fzs5c
MATRIX_ALLOWED_USERS=@alice:matrix.org
DINGTALK_ALLOWED_USERS=user-id-1
FEISHU_ALLOWED_USERS=ou_xxxxxxxx,ou_yyyyyyyy
WECOM_ALLOWED_USERS=user-id-1,user-id-2
WECOM_CALLBACK_ALLOWED_USERS=user-id-1,user-id-2
TEAMS_ALLOWED_USERS=aad-object-id-1,aad-object-id-2
# Or allow
GATEWAY_ALLOWED_USERS=123456789,987654321
# Or explicitly allow all users (NOT recommended for bots with terminal access):
GATEWAY_ALLOW_ALL_USERS=true
```
### DM Pairing (Alternative to Allowlists)
Instead of manually configuring user IDs, unknown users receive a one-time pairing code when they DM the bot. Email is the exception: unknown email senders are ignored unless email pairing is explicitly enabled.
```bash
# The user sees: "Pairing code: XKGH5N7P"
# You approve them with:
hermes pairing approve telegram XKGH5N7P
# Other pairing commands:
hermes pairing list # View pending + approved users
hermes pairing revoke telegram 123456789 # Remove access
```
Pairing codes expire after 1 hour, are rate-limited, and use cryptographic randomness.
### Admins vs Regular Users
Allowlists answer "can this person reach the bot at all?" The **admin / user split** answers "now that they're in, what are they allowed to do?"
Every allowed user falls into one of two tiers per scope (DM vs group/channel):
- **Admin** — full access. Can run every registered slash command (built-in + plugin) and use every gated capability.
- **Regular user** — restricted access. Can chat with the agent normally, but can only run the slash commands you explicitly enable. The always-allowed floor is `/help` and `/whoami`.
The tiers are configured per platform and per scope. DM admin status does not imply group/channel admin status — each scope has its own admin list.
**What the tiers gate today:** slash commands. The split runs through the live command registry, so it covers built-ins and plugin-registered commands without per-feature wiring. Plain chat is not affected — non-admins can still talk to the agent.
**What may be gated in the future:** more capability surfaces (tool access, model switching, expensive operations) will hang off the same admin / user distinction as we add them. Configuring the split now means those future restrictions land cleanly without you having to re-model who's an admin.
#### Configuration
```yaml
gateway:
platforms:
discord:
extra:
allow_from: ["111", "222", "333"]
allow_admin_from: ["111"] # admins → all slash commands
user_allowed_commands: [status, model] # what non-admins may run
# Optional: separate group/channel scope
group_allow_admin_from: ["111"]
group_user_allowed_commands: [status]
```
**Backward compat:** if `allow_admin_from` is not set for a scope, the tier split is disabled for that scope and every allowed user has full access. Existing installs keep working with no changes — opt in when you want the distinction.
#### Inspecting your access
Use `/whoami` from any platform to see the active scope, your tier (admin / user / unrestricted), and which slash commands you can run. See the [Telegram](/user-guide/messaging/telegram#slash-command-access-control) and [Discord](/user-guide/messaging/discord#slash-command-access-control) pages for platform-specific examples.
## Redirecting the Agent
Send a message while the agent is working to correct the active turn:
- **Model generation restarts with context** — reasoning already shown and visible partial text are retained as an ordinary assistant checkpoint
- **Completed work stays available** — prior tool calls and results remain in the turn
- **Running tools finish safely** — the correction is applied at the next tool-result boundary instead of killing the tool
- **`/stop` remains a hard stop** — use it to cancel the active turn and foreground work
### Queue vs interrupt vs steer (busy-input mode)
By default, messaging a busy agent redirects its active turn. Two other modes are available:
- `queue` — follow-up messages wait and run as the next turn after the current task finishes.
- `steer` — follow-up messages are injected into the current run via `/steer`, arriving at the agent after the next tool call. No interrupt, no new turn. Falls back to `queue` behavior if the agent hasn't started yet.
```yaml
display:
busy_input_mode: steer # or queue, or interrupt (default)
busy_ack_enabled: true # set to false to suppress the ⚡/⏳/⏩ chat reply entirely
```
The first time you message a busy agent on any platform, Hermes appends a one-line reminder to the busy-ack explaining the knob (`"💡 First-time tip — …"`). The reminder fires once per install — a flag under `onboarding.seen.busy_input_prompt` latches it. Delete that key to see the tip again.
If you find the busy acknowledgment noisy, set `display.busy_ack_enabled: false`. Input handling is unchanged; only the confirmation message is hidden.
## Clarify Questions (Multi-Select)
When the agent uses the `clarify` tool to ask you a question, the gateway renders the choices as a numbered prompt (or native buttons on platforms that support them). Clarify supports **multi-select** questions too — the agent can let you pick several options at once:
- **Messaging platforms** — the prompt says "Multiple selections allowed"; reply with the numbers separated by commas or spaces (e.g. `1, 3`), the option text, or your own free-form answer.
- **Classic CLI / TUI** — multi-select renders as checkboxes: **Space** toggles an option, **Enter** submits the selection.
Single-select prompts behave as before: pick one option by number, button, or text, or type your own answer via the "Other" path.
## Tool Progress Notifications
Control how much tool activity is displayed in `~/.hermes/config.yaml`:
```yaml
display:
tool_progress: all # off | new | all | verbose | log
tool_progress_command: false # set to true to enable /verbose in messaging
# How progress is grouped on platforms that support message editing:
# accumulate (default) — edit one bubble in place as tools run
# separate — send one message per tool (pre-v0.9 style; noisier)
# Only applies where tool_progress is already enabled.
tool_progress_grouping: accumulate # accumulate | separate
```
### `log` mode — audit file instead of chat messages
Setting `display.tool_progress: log` sends **no** progress bubbles to chat. Instead, each tool call is appended as a line to `~/.hermes/logs/tool_calls.log` — a rotating audit file (5 MB × 3 backups) run through the same secret-redacting formatter as regular logs, so credentials never land on disk. Use it when you want a full tool-call trail without any chat noise.
### Configurable status phrases
Long-running gateway status lines ("still working…"-style heartbeats) draw from a phrase catalog. Built-in defaults ship in `gateway/assets/status_phrases.yaml`; you can add your own with profile-portable files under `HERMES_HOME`:
- `~/.hermes/status_phrases.yaml` or any `*.yaml` in `~/.hermes/status_phrases/` (conventional paths, auto-loaded), or
- point config at a relative path:
```yaml
display:
status_phrases:
path: status_phrases/whatsapp.yaml # relative to HERMES_HOME
mode: append # append (default) or replace
```
Phrase files map a surface (`status`, `generic`) to a list of strings (max 80 phrases per surface, 160 chars each). Absolute paths and `..` escapes are ignored so config stays profile-portable. Only your configured phrase strings are used — raw tool arguments, commands, and reasoning text are never interpolated into a status phrase.
### Message timestamps in model context
Off by default. When enabled, Hermes prepends a human-readable timestamp
(e.g. `[Tue 2026-04-28 13:40:53 CEST]`) onto each **user** message *in the
model's context* so the agent knows when messages were sent — useful for
temporal reasoning ("you asked this morning…", noticing a long gap). It is
**not** added to assistant messages or the system prompt.
```yaml
gateway:
message_timestamps:
enabled: false # set true to show send-times to the model
```
Persisted transcripts always stay clean — the timestamp is stored as message
metadata regardless of this toggle, so enabling it later also surfaces
send-times for past messages, and replay never accumulates duplicate prefixes.
When enabled, the bot sends status messages as it works:
```text
💻 `ls -la`...
🔍 web_search...
📄 web_extract...
🐍 execute_code...
```
## Background Sessions
Run a prompt in a separate background session so the agent works on it independently while your main chat stays responsive:
```
/bg Check all servers in the cluster and report any that are down
```
Hermes confirms immediately:
```
🔄 Background task started: "Check all servers in the cluster..."
Task ID: bg_143022_a1b2c3
```
### How It Works
Each `/bg` prompt spawns a **separate agent instance** that runs asynchronously:
- **Isolated session** — the background agent has its own session with its own conversation history. It has no knowledge of your current chat context and receives only the prompt you provide.
- **Same configuration** — inherits your model, provider, toolsets, reasoning settings, and provider routing from the current gateway setup.
- **Non-blocking** — your main chat stays fully interactive. Send messages, run other commands, or start more background tasks while it works.
- **Result delivery** — when the task finishes, the result is sent back to the **same chat or channel** where you issued the command, prefixed with "✅ Background task complete". If it fails, you'll see "❌ Background task failed" with the error.
### Background Process Notifications
When the agent running a background session uses `terminal(background=true)` to start long-running processes (servers, builds, etc.), the gateway can push status updates to your chat. Control this with `display.background_process_notifications` in `~/.hermes/config.yaml`:
```yaml
display:
background_process_notifications: concise # concise | all | result | error | off
```
| Mode | What you receive |
|------|-----------------|
| `concise` | One-line status message on completion; failures append a short output tail (default) |
| `all` | Running-output updates **and** the final raw-output message |
| `result` | Only the final raw-output completion message (regardless of exit code) |
| `error` | Only the final raw-output message when the exit code is non-zero |
| `off` | No process watcher messages at all |
You can also set this via environment variable:
```bash
HERMES_BACKGROUND_NOTIFICATIONS=result
```
### Use Cases
- **Server monitoring** — "/bg Check the health of all services and alert me if anything is down"
- **Long builds** — "/bg Build and deploy the staging environment" while you continue chatting
- **Research tasks** — "/bg Research competitor pricing and summarize in a table"
- **File operations** — "/bg Organize the photos in ~/Downloads by date into folders"
:::tip
Background tasks on messaging platforms are fire-and-forget — you don't need to wait or check on them. Results arrive in the same chat automatically when the task finishes.
:::
## Service Management
### Linux (systemd)
```bash
hermes gateway install # Install as user service
hermes gateway start # Start the service
hermes gateway stop # Stop the service
hermes gateway status # Check status
journalctl --user -u hermes-gateway -f # View logs
# Enable lingering (keeps running after logout)
sudo loginctl enable-linger $USER
# Or install a boot-time system service that still runs as your user
sudo hermes gateway install --system
sudo hermes gateway start --system
sudo hermes gateway status --system
journalctl -u hermes-gateway -f
```
Use the user service on laptops and dev boxes. Use the system service on VPS or headless hosts that should come back at boot without relying on systemd linger.
:::danger Don't add a custom `ExecStopPost` kill drop-in
The unit Hermes installs already shuts the gateway down cleanly with `KillMode=mixed` + `KillSignal=SIGTERM`, and uses `Restart=always` with `RestartForceExitStatus` so updates and `/restart` respawn correctly. Do **not** add a systemd drop-in such as `ExecStopPost=/bin/kill -9 $MAINPID` — `ExecStopPost` fires on *every* stop, including clean restarts, so it `SIGKILL`s the freshly spawned instance before it stabilizes and `Restart=always` immediately respawns it. The result is an infinite restart loop (and, on Telegram, a flood of restart messages). If you've added such a drop-in, remove it: `systemctl --user edit hermes-gateway` (or `sudo systemctl edit hermes-gateway` for a system service) and delete the `ExecStopPost` line, then `systemctl --user daemon-reload`.
:::
:::tip Headless VMs: user service + linger avoids root prompts
A system service needs root for every restart — including the automatic gateway restart at the end of `hermes update`. When `hermes update` runs as a non-root user, it tries passwordless `sudo systemctl`; if that's unavailable, it skips the restart and prints the manual `sudo systemctl restart hermes-gateway` command (it never blocks on an interactive password prompt).
For a headless VM you never log into, a **user** service with lingering enabled gives you the same start-at-boot behavior with zero root involvement:
```bash
hermes gateway install # user service
sudo loginctl enable-linger $USER # one-time: start at boot, survive logout
```
After that, `hermes update` can restart the gateway without any privileges. If you prefer to keep the system service, either run updates with `sudo hermes update`, or grant the service account passwordless sudo for systemctl, e.g. in `sudo visudo -f /etc/sudoers.d/hermes-gateway`:
```
hermes ALL=(root) NOPASSWD: /usr/bin/systemctl --no-ask-password reset-failed hermes-gateway*, /usr/bin/systemctl --no-ask-password start hermes-gateway*, /usr/bin/systemctl --no-ask-password restart hermes-gateway*
```
:::
Avoid keeping both the user and system gateway units installed at once unless you really mean to. Hermes will warn if it detects both because start/stop/status behavior gets ambiguous.
:::info Multiple installations
If you run multiple Hermes installations on the same machine (with different `HERMES_HOME` directories), each gets its own systemd service name. The default `~/.hermes` uses `hermes-gateway`; other installations use `hermes-gateway-<hash>`. The `hermes gateway` commands automatically target the correct service for your current `HERMES_HOME`.
:::
### macOS (launchd)
```bash
hermes gateway install # Install as launchd agent
hermes gateway start # Start the service
hermes gateway stop # Stop the service
hermes gateway status # Check status
tail -f ~/.hermes/logs/gateway.log # View logs
```
The generated plist lives at `~/Library/LaunchAgents/ai.hermes.gateway.plist`. It includes three environment variables:
- **PATH** — your full shell PATH at install time, with the venv `bin/` and `node_modules/.bin` prepended. This ensures user-installed tools (Node.js, ffmpeg, etc.) are available to gateway subprocesses like the WhatsApp bridge.
- **VIRTUAL_ENV** — points to the Python virtualenv so tools can resolve packages correctly.
- **HERMES_HOME** — scopes the gateway to your Hermes installation.
:::tip PATH changes after install
launchd plists are static — if you install new tools (e.g. a new Node.js version via nvm, or ffmpeg via Homebrew) after setting up the gateway, run `hermes gateway install` again to capture the updated PATH. The gateway will detect the stale plist and reload automatically.
:::
:::info Multiple installations
Like the Linux systemd service, each `HERMES_HOME` directory gets its own launchd label. The default `~/.hermes` uses `ai.hermes.gateway`; other installations use `ai.hermes.gateway-<suffix>`.
:::
## Platform-Specific Toolsets
Each platform has its own toolset:
| Platform | Toolset | Capabilities |
|----------|---------|--------------|
| CLI | `hermes-cli` | Full access |
| Telegram | `hermes-telegram` | Full tools including terminal |
| Discord | `hermes-discord` | Full tools including terminal |
| WhatsApp | `hermes-whatsapp` | Full tools including terminal |
| WhatsApp Cloud API | `hermes-whatsapp` | Full tools including terminal (shares toolset with the Baileys bridge) |
| Slack | `hermes-slack` | Full tools including terminal |
| Google Chat | `hermes-google_chat` | Full tools including terminal |
| Signal | `hermes-signal` | Full tools including terminal |
| SMS | `hermes-sms` | Full tools including terminal |
| Email | `hermes-email` | Full tools including terminal |
| Home Assistant | `hermes-homeassistant` | Full tools + HA device control (ha_list_entities, ha_get_state, ha_call_service, ha_list_services) |
| Mattermost | `hermes-mattermost` | Full tools including terminal |
| Matrix | `hermes-matrix` | Full tools including terminal |
| DingTalk | `hermes-dingtalk` | Full tools including terminal |
| Feishu/Lark | `hermes-feishu` | Full tools including terminal |
| WeCom | `hermes-wecom` | Full tools including terminal |
| WeCom Callback | `hermes-wecom-callback` | Full tools including terminal |
| Weixin | `hermes-weixin` | Full tools including terminal |
| BlueBubbles | `hermes-bluebubbles` | Full tools including terminal |
| QQBot | `hermes-qqbot` | Full tools including terminal |
| Yuanbao | `hermes-yuanbao` | Full tools including terminal |
| Microsoft Teams | `hermes-teams` | Full tools including terminal |
| API Server | `hermes-api-server` | Full tools (drops `clarify`, `text_to_speech` — programmatic access doesn't have an interactive user) |
| Webhooks | `hermes-webhook` | Full tools including terminal |
| Raft | `hermes-raft` | Wake-only channel; agent uses Raft CLI for message I/O |
## Operating a multi-platform gateway
A gateway typically runs several adapters at once (Telegram + Discord + Slack, etc.). The sections below cover day-2 operations that span all platforms.
### `/platform` command
Once the gateway is running, use the `/platform` slash command from any connected CLI session or chat to inspect and steer individual adapters without restarting the whole gateway:
```
/platform list # show all adapters and their state
/platform pause <name> # stop dispatching new messages to one adapter
/platform resume <name> # re-enable a paused adapter
```
`/platform list` shows whether each adapter is `running`, `paused` (manually), or `paused-by-breaker` (see below). Pausing keeps the adapter loaded and its background loops alive — incoming messages are dropped on the floor, but the connection itself stays open so resume is instant.
See also the broader status summary command [`/platforms`](../../reference/slash-commands.md#info).
### Disabling a platform whose credentials are still in `.env`
`platforms.<name>.enabled: false` in `~/.hermes/config.yaml` is authoritative.
Credentials for that platform left in the environment (`TELEGRAM_BOT_TOKEN`,
`WEIXIN_TOKEN`, `HASS_TOKEN`, `EMAIL_*`, `TWILIO_ACCOUNT_SID`, ...) are still
wired into the platform's config so send-only tooling keeps working, but they
no longer start the adapter:
```yaml title="~/.hermes/config.yaml"
platforms:
weixin:
enabled: false # wins over WEIXIN_TOKEN in .env
```
Earlier releases let the mere presence of credentials re-enable twelve
platforms (Weixin, WhatsApp Cloud, Home Assistant, Email, SMS, DingTalk, Feishu,
WeCom, WeCom callback, BlueBubbles, QQ Bot, Yuanbao) regardless of that key. If
you relied on that, the gateway now logs one WARNING per affected platform at
startup so it does not just go dark:
```
Platform 'weixin' is explicitly disabled by platforms.weixin.enabled: false in config.yaml,
so the credentials found in the environment (WEIXIN_TOKEN, WEIXIN_ACCOUNT_ID) will NOT start
its adapter. Environment credentials no longer override an explicit disable. Remove the key
or set platforms.weixin.enabled: true to turn it back on.
```
Omitting the `enabled` key entirely keeps the env-only behaviour: credentials
present → adapter starts.
### Ignoring an inherited proxy (`gateway.trust_env`)
By default every platform adapter honors `HTTP_PROXY` / `HTTPS_PROXY` /
`NO_PROXY` (and `SSL_CERT_FILE`) from the gateway's environment, and
auto-detects the macOS system proxy. A gateway started by a Windows Scheduled
Task or a service manager can inherit a proxy the interactive shell never
sees — a local Clash/V2Ray listener that isn't running yet — and log
`Cannot connect to host 127.0.0.1:7890` on every poll. Turn the inherited
proxy off for all adapters at once:
```yaml title="~/.hermes/config.yaml"
gateway:
trust_env: false
```
Explicit per-platform proxy variables (`DISCORD_PROXY`, `TELEGRAM_PROXY`,
`MATRIX_PROXY`, ...) are still honored. Restart the gateway after changing it.
### Automatic circuit breaker
Each adapter is wrapped in a circuit breaker. Repeated retryable failures (network blips, rate-limit replies, 5xx upstream responses, websocket disconnects) cause the breaker to trip — the adapter is auto-paused, an operator notification is sent to the home channel of another live platform when one is configured, and a structured log line is emitted.
The breaker does **not** auto-resume — it stays open until you run `/platform resume <name>` manually. This is intentional: if a platform is in a sustained outage, you don't want the gateway thrashing reconnects.
### Where to look when a platform is paused
When an adapter is paused, check:
1. **Gateway log** (`~/.hermes/logs/gateway.log` or the systemd / launchd unit log). Search for the platform name and `circuit breaker`, `paused`, or `disabled`. The trip event includes the failure count and the last error.
2. **`/platform list`** output — shows the current state and last reason.
3. **The provider's status page** (Telegram bot API status, Discord status, etc.). The breaker tripped because the platform was unhealthy; don't try to resume until it's back.
Once upstream is healthy, `/platform resume <name>` clears the breaker and re-arms the adapter.
### Restart notifications
When the gateway restarts (or is shut down with in-flight sessions), it can send a one-shot "the agent is back" / "the agent was interrupted" message to each platform's home channel. This is controlled per-platform by the `gateway_restart_notification` flag in `gateway-config.yaml`, which defaults to `true`:
```yaml
gateway:
platforms:
telegram:
home_chat_id: "123456789"
gateway_restart_notification: false # opt out for this platform
discord:
home_chat_id: "987654321"
# gateway_restart_notification omitted → defaults to true
```
Disable it on noisy or low-priority platforms while leaving it on for your primary chat. The notification is sent once per restart, regardless of how many sessions were in flight.
### Typing indicators
While the agent is processing a message, the gateway shows a live typing status on platforms that support it — a "typing…" bubble on Telegram/Discord/Signal, or the "is thinking…" assistant status on Slack. This is controlled per-platform by the `typing_indicator` flag in `gateway-config.yaml`, which defaults to `true`:
```yaml
gateway:
platforms:
slack:
typing_indicator: false # don't show "is thinking…" on Slack
telegram:
# typing_indicator omitted → defaults to true
```
Set `typing_indicator: false` on any platform where the indicator is unwanted. Some users find Slack's "is thinking…" status noisy (it also briefly disables the compose box while shown, since it uses Slack's Assistant API). Disabling it only suppresses the indicator — message delivery and everything else is unchanged. The flag is generic, so the same key works for every platform.
### Session resume across gateway restarts
When the gateway shuts down with an in-flight tool call or generation, the affected sessions are flagged as `restart_interrupted`. On the next startup, the gateway schedules an auto-resume for each one — the user gets a short heads-up in the chat ("Send any message after restart and I'll try to resume where you left off.") and the session picks up from the last committed turn when they reply.
This behaviour is on by default and is logged at gateway start:
```
Scheduled auto-resume for N restart-interrupted session(s)
```
No configuration is required. If you don't want the heads-up, set `gateway_restart_notification: false` on the platform.
### Mobile-friendly progress defaults
Telegram is usually a mobile inbox, so the defaults are tuned for that surface:
- **`tool_progress`** defaults to **`off`** — no per-tool breadcrumb stream filling up the chat.
- **`busy_ack_detail`** defaults to **`off`** — busy-state acknowledgments and long-running heartbeats stay terse (no `iteration 21/60` debug detail).
- **`interim_assistant_messages`** stays **on** — real mid-turn assistant commentary (the model literally telling you what it's about to do) is signal, not noise.
- **`long_running_notifications`** stays **on** — a single edit-in-place "⏳ Working — N min" bubble updates every few minutes so you have a heartbeat instead of staring at `typing…` for half an hour.
Opt out of either of the kept-on defaults or opt back into verbose progress per platform:
```yaml
display:
platforms:
telegram:
# Re-enable the tool-progress stream
tool_progress: new
# Show "iteration N/M, running: tool" in heartbeats and busy acks
busy_ack_detail: true
# Or quiet them entirely
interim_assistant_messages: false
long_running_notifications: false
```
### Progress bubble cleanup (opt-in)
Tool-progress messages, the "still working…" heartbeat, and status-callback bubbles can also be auto-deleted after the final response lands. Enable per-platform via `display.platforms.<platform>.cleanup_progress`:
```yaml
display:
platforms:
telegram:
cleanup_progress: true
discord:
cleanup_progress: true
```
Defaults to `false`. Only platforms whose adapter implements `delete_message` honor the setting (currently Telegram and Discord). Failed runs **skip** cleanup so the bubbles remain as breadcrumbs.
## Next Steps
- [Telegram Setup](telegram.md)
- [Discord Setup](discord.md)
- [Slack Setup](slack.md)
- [Google Chat Setup](google_chat.md)
- [WhatsApp Setup](whatsapp.md)
- [WhatsApp Business Cloud API Setup](whatsapp-cloud.md)
- [Signal Setup](signal.md)
- [SMS Setup (Twilio)](sms.md)
- [Email Setup](email.md)
- [Home Assistant Integration](homeassistant.md)
- [Mattermost Setup](mattermost.md)
- [Matrix Setup](matrix.md)
- [DingTalk Setup](dingtalk.md)
- [Feishu/Lark Setup](feishu.md)
- [WeCom Setup](wecom.md)
- [WeCom Callback Setup](wecom-callback.md)
- [Weixin Setup (WeChat)](weixin.md)
- [BlueBubbles Setup (iMessage)](bluebubbles.md)
- [Photon Setup (iMessage)](photon.md)
- [QQBot Setup](qqbot.md)
- [Yuanbao Setup](yuanbao.md)
- [Microsoft Teams Setup](teams.md)
- [Teams Meetings Pipeline](teams-meetings.md)
- [Microsoft Graph Webhook Listener](msgraph-webhook.md)
- [LINE Setup](line.md)
- [ntfy Setup](ntfy.md)
- [SimpleX Chat Setup](simplex.md)
- [Open WebUI + API Server](open-webui.md)
- [Raft Setup](raft.md)
- [IRC Setup](irc.md)
- [Buzz Setup](buzz.md)
- [A2A (Agent-to-Agent) Setup](a2a.md)
- [Webhooks](webhooks.md)
website/docs/user-guide/messaging/irc.md
+78
View File
@@ -0,0 +1,78 @@
# IRC
The IRC adapter connects Hermes to any IRC server and relays messages between an IRC channel (or direct messages) and the agent. It speaks the IRC protocol over Python's stdlib `asyncio` — **no external dependencies, no SDK, no daemon**. It works with public networks like [Libera.Chat](https://libera.chat/) and any self-hosted ircd.
IRC is plain text: there is no voice, image, file, thread, reaction, typing, or streaming support — replies are sent as `PRIVMSG` lines, with long messages split to fit the IRC line limit.
> Run `hermes gateway setup` and pick **IRC** for a guided walk-through.
## Prerequisites
- An IRC server to connect to (e.g. `irc.libera.chat`)
- A channel to join (e.g. `#hermes`) — comma-separate to join several
- A nickname for the bot (default: `hermes-bot`)
- Optional: a registered nick + NickServ password if your network requires identification
## Configure Hermes
You can configure IRC two ways — environment variables (for a quick env-only setup) or the `gateway` block in `~/.hermes/gateway-config.yaml`.
### Option A — gateway-config.yaml
```yaml
gateway:
platforms:
irc:
enabled: true
extra:
server: irc.libera.chat
port: 6697
nickname: hermes-bot
channel: "#hermes"
use_tls: true
server_password: "" # optional server password
nickserv_password: "" # optional NickServ identification
allowed_users: [] # empty = allow all, or list of nicks
max_message_length: 450 # IRC line limit (safe default)
```
### Option B — environment variables
| Variable | Required | Description |
|----------|:--------:|-------------|
| `IRC_SERVER` | ✅ | IRC server hostname (e.g. `irc.libera.chat`) |
| `IRC_CHANNEL` | ✅ | Channel(s) to join — comma-separate for multiple |
| `IRC_NICKNAME` | ✅ | Bot nickname (default: `hermes-bot`) |
| `IRC_PORT` | — | Server port (default: `6697` with TLS, `6667` without) |
| `IRC_USE_TLS` | — | Use TLS (`true`/`false`; default `true` on port 6697) |
| `IRC_SERVER_PASSWORD` | — | Server password for the `PASS` command |
| `IRC_NICKSERV_PASSWORD` | — | NickServ password for automatic IDENTIFY on connect |
| `IRC_ALLOWED_USERS` | — | Comma-separated nicks allowed to talk to the bot |
| `IRC_ALLOW_ALL_USERS` | — | Allow anyone in the channel to talk to the bot (dev only) |
| `IRC_HOME_CHANNEL` | — | Channel for cron / notification delivery (defaults to `IRC_CHANNEL`) |
## Access control
By default, only nicks listed in `allowed_users` (or `IRC_ALLOWED_USERS`) may talk to the bot. Leave the list empty **and** set `IRC_ALLOW_ALL_USERS=true` to let anyone in the channel chat with Hermes — useful for testing, but not recommended on public networks since IRC nicks are not authenticated unless the network enforces NickServ.
If your network registers nicks, set `IRC_NICKSERV_PASSWORD` (or `nickserv_password`) so the bot identifies to NickServ on connect and keeps its registered nick.
## Channels vs. DMs
- Messages in a joined channel are treated as a **group** conversation.
- Private messages to the bot are treated as **direct messages**.
Cron jobs and notifications are delivered to the **home channel** — `IRC_HOME_CHANNEL` if set, otherwise the first `IRC_CHANNEL`.
## Run the gateway
```bash
hermes gateway start
```
Check status with `hermes gateway status` — IRC connection state is reported there, including for env-only setups.
## Notes
- Long agent replies are automatically split into multiple `PRIVMSG` lines to stay within the IRC line limit (`max_message_length`, default 450 bytes after protocol overhead).
- The adapter acquires a scoped credential lock per server+nick, so two Hermes profiles won't fight over the same IRC identity.
website/docs/user-guide/messaging/line.md
+200
View File
@@ -0,0 +1,200 @@
---
sidebar_position: 17
title: "LINE"
description: "Set up Hermes Agent as a LINE Messaging API bot"
---
# LINE Setup
Run Hermes Agent as a [LINE](https://line.me/) bot via the official LINE Messaging API. The adapter lives as a bundled platform plugin under `plugins/platforms/line/` — no core edits, just enable it like any other platform.
LINE is the dominant messaging app in Japan, Taiwan, and Thailand. If your users live there, this is how they reach you.
> Run `hermes gateway setup` and pick **LINE** for a guided walk-through.
## How the bot responds
| Context | Behavior |
|---------|----------|
| **1:1 chat** (`U` IDs) | Responds to every message |
| **Group chat** (`C` IDs) | Responds when the group is on the allowlist |
| **Multi-user room** (`R` IDs) | Responds when the room is on the allowlist |
Inbound text, images, audio, video, files, stickers, and locations are all handled. Outbound text uses the **free reply token first** (single-use, ~60s window) and falls back to the metered Push API when the token has expired.
---
## Step 1: Create a LINE Messaging API channel
1. Go to the [LINE Developers Console](https://developers.line.biz/console/).
2. Create a Provider, then under it a **Messaging API** channel.
3. From the channel's **Basic settings** tab, copy the **Channel secret**.
4. From the **Messaging API** tab, scroll to **Channel access token (long-lived)** and click **Issue**. Copy the token.
5. In the **Messaging API** tab, also disable **Auto-reply messages** and **Greeting messages** so they don't fight your bot's replies.
---
## Step 2: Expose the webhook port
LINE delivers webhooks over public HTTPS. The default port is `8646` — override with `LINE_PORT` if needed.
```bash
# Cloudflare Tunnel (recommended for production — fixed hostname)
cloudflared tunnel --url http://localhost:8646
# ngrok (good for dev)
ngrok http 8646
# devtunnel
devtunnel create hermes-line --allow-anonymous
devtunnel port create hermes-line -p 8646 --protocol https
devtunnel host hermes-line
```
Copy the `https://...` URL — you'll set it as the webhook URL below. **Leave the tunnel running** while testing. For production, set up a fixed Cloudflare named tunnel so the webhook URL doesn't change on restart.
---
## Step 3: Configure Hermes
Add to `~/.hermes/.env`:
```env
LINE_CHANNEL_ACCESS_TOKEN=YOUR_LONG_LIVED_TOKEN
LINE_CHANNEL_SECRET=YOUR_CHANNEL_SECRET
# Allowlist — at least one of these (or LINE_ALLOW_ALL_USERS=true for dev)
LINE_ALLOWED_USERS=U1234567890abcdef... # comma-separated U-prefixed IDs
LINE_ALLOWED_GROUPS=C1234567890abcdef... # optional group IDs
LINE_ALLOWED_ROOMS=R1234567890abcdef... # optional room IDs
# Required for image / audio / video sends — the public HTTPS base URL
# the tunnel resolves to. Without it, send_image/voice/video will refuse.
LINE_PUBLIC_URL=https://my-tunnel.example.com
```
Then in `~/.hermes/config.yaml`:
```yaml
gateway:
platforms:
line:
enabled: true
```
That's enough — the bundled-plugin scan in `gateway/config.py` automatically picks up `plugins/platforms/line/`. No `Platform.LINE` enum edit, no `_create_adapter` registration.
---
## Step 4: Set the webhook URL
Back in the LINE console:
1. Open your channel → **Messaging API** tab.
2. Under **Webhook settings** → **Webhook URL**, paste `https://<your-tunnel>/line/webhook` (note the `/line/webhook` path — the adapter listens there).
3. Click **Verify**. LINE pings the URL; you should see a 200.
4. Toggle **Use webhook** to **On**.
---
## Step 5: Run the gateway
```bash
hermes gateway
```
The agent log shows:
```
LINE: webhook listening on * (all interfaces, IPv4+IPv6):8646/line/webhook (public: https://my-tunnel.example.com)
```
Add the bot as a friend from the LINE app (scan the QR in the channel's **Messaging API** tab) and send it a message.
---
## Slow LLM responses
LINE's reply token is single-use and expires roughly 60 seconds after the inbound event. Slow LLMs can't reply in time, which would normally force a paid Push API call.
When the LLM is still running past `LINE_SLOW_RESPONSE_THRESHOLD` seconds (default `45`), the adapter consumes the original reply token to send a **Template Buttons** bubble:
> 🤔 Still thinking. Tap below to fetch the answer when it's ready.
>
> [ Get answer ]
The user taps **Get answer** when convenient — that postback delivers a *fresh* reply token, which the adapter uses to send the cached answer (still free).
State machine: `PENDING → READY → DELIVERED`, plus `ERROR` for cancelled runs (the orphan PENDING resolves to "Run was interrupted before completion." after `/stop` so the persistent button doesn't loop).
To disable the postback button and always Push-fallback instead:
```env
LINE_SLOW_RESPONSE_THRESHOLD=0
```
For the postback flow to fire reliably, suppress chatter that would consume the reply token before the threshold:
```yaml
# ~/.hermes/config.yaml
display:
interim_assistant_messages: false
platforms:
line:
tool_progress: off
```
---
## Cron / notification delivery
```env
LINE_HOME_CHANNEL=Uxxxxxxxxxxxxxxxxxxxx # default delivery target
```
Cron jobs with `deliver: line` route to `LINE_HOME_CHANNEL`. The adapter ships a standalone Push-only sender so cron jobs work even when cron runs in a separate process from the gateway.
---
## Environment variable reference
| Variable | Required | Default | Description |
|---|---|---|---|
| `LINE_CHANNEL_ACCESS_TOKEN` | yes | — | Long-lived channel access token |
| `LINE_CHANNEL_SECRET` | yes | — | Channel secret (HMAC-SHA256 webhook verification) |
| `LINE_HOST` | no | unset (dual-stack: all interfaces, IPv4+IPv6) | Webhook bind host |
| `LINE_PORT` | no | `8646` | Webhook bind port |
| `LINE_PUBLIC_URL` | for media | — | Public HTTPS base URL; required for image/voice/video sends |
| `LINE_ALLOWED_USERS` | one of | — | Comma-separated user IDs (U-prefixed) |
| `LINE_ALLOWED_GROUPS` | one of | — | Comma-separated group IDs (C-prefixed) |
| `LINE_ALLOWED_ROOMS` | one of | — | Comma-separated room IDs (R-prefixed) |
| `LINE_ALLOW_ALL_USERS` | dev only | `false` | Skip allowlist entirely |
| `LINE_HOME_CHANNEL` | no | — | Default cron / notification delivery target |
| `LINE_SLOW_RESPONSE_THRESHOLD` | no | `45` | Seconds before the postback button fires (`0` = disabled) |
| `LINE_PENDING_TEXT` | no | "🤔 Still thinking…" | Bubble text shown alongside the postback button |
| `LINE_BUTTON_LABEL` | no | "Get answer" | Button label |
| `LINE_DELIVERED_TEXT` | no | "Already replied ✅" | Reply when an already-delivered button is tapped again |
| `LINE_INTERRUPTED_TEXT` | no | "Run was interrupted before completion." | Reply when a `/stop` orphan button is tapped |
---
## Troubleshooting
**"invalid signature" on webhook verify.** The `Channel secret` was copied wrong, or your tunnel rewrote the request body. Verify with `curl -i https://<tunnel>/line/webhook/health` first — that should return `{"status":"ok","platform":"line"}`.
**Bot receives nothing in groups.** Check `LINE_ALLOWED_GROUPS` includes the `C...` group ID. To find a group ID, send a test message and grep `~/.hermes/logs/gateway.log` for `LINE: rejecting unauthorized source` — the rejected source dict has the IDs.
**`send_image` fails with "LINE_PUBLIC_URL must be set".** LINE's Messaging API does not accept binary uploads — images, audio, and video must be reachable HTTPS URLs. Set `LINE_PUBLIC_URL` to the tunnel's public hostname and the adapter will serve files from `/line/media/<token>/<filename>` automatically.
**Postback button never appears.** Either the LLM responded faster than `LINE_SLOW_RESPONSE_THRESHOLD`, or another bubble (tool-progress, streaming) consumed the reply token first. See the suppression block under "Slow LLM responses".
**"already in use by another profile".** The same channel access token is bound to another running Hermes profile. Stop the other gateway or use a separate channel.
---
## Limitations
* **Bubble and length caps.** Each LINE text bubble is capped at 5000 characters. Longer responses are smart-chunked at ~4500 characters across up to 5 bubbles per Reply/Push call, splitting on natural boundaries where possible.
* **No native message editing.** LINE has no edit-message API — streaming responses always send fresh bubbles, never edit prior ones.
* **No Markdown rendering.** Bold (`**`), italics (`*`), code fences, and headings render as literal characters. The adapter strips them before sending; URLs are preserved (`[label](url)` becomes `label (url)`).
* **Loading indicator is DM-only.** LINE rejects the chat/loading API for groups and rooms, so the typing indicator only shows in 1:1 chats.
website/docs/user-guide/messaging/matrix.md
+917
View File
@@ -0,0 +1,917 @@
---
sidebar_position: 9
title: "Matrix"
description: "Set up Hermes Agent as a Matrix bot"
---
# Matrix Setup
Hermes Agent integrates with Matrix, the open, federated messaging protocol. Matrix lets you run your own homeserver or use a public one like matrix.org — either way, you keep control of your communications. The bot connects via the `mautrix` Python SDK, processes messages through the Hermes Agent pipeline (including tool use, memory, and reasoning), and responds in real time. It supports text, file attachments, images, audio, video, and optional end-to-end encryption (E2EE).
Hermes works with any Matrix homeserver — Synapse, Conduit, Dendrite, or matrix.org.
Before setup, here's the part most people want to know: how Hermes behaves once it's connected.
## How Hermes Behaves
| Context | Behavior |
|---------|----------|
| **DMs** | Hermes responds to every message. No `@mention` needed. Each DM has its own session. Set `MATRIX_DM_MENTION_THREADS=true` to start a thread when the bot is `@mentioned` in a DM. |
| **Rooms** | By default, Hermes requires an `@mention` to respond. Set `MATRIX_REQUIRE_MENTION=false` or add room IDs to `MATRIX_FREE_RESPONSE_ROOMS` for free-response rooms. Room invites are auto-accepted. |
| **Threads** | Hermes supports Matrix threads (MSC3440). If you reply in a thread, Hermes keeps the thread context isolated from the main room timeline. Threads where the bot has already participated do not require a mention. |
| **Auto-threading** | By default, Hermes auto-creates a thread for each message it responds to in a room. This keeps conversations isolated. Set `MATRIX_AUTO_THREAD=false` to disable. Set `MATRIX_DM_AUTO_THREAD=true` (default false) to also auto-create threads for DM messages — this is distinct from `MATRIX_DM_MENTION_THREADS`, which only starts a thread when the bot is `@mentioned` in a DM. |
| **Commands** | Hermes accepts normal `/commands` when your Matrix client sends them. If your client reserves `/` for local commands, use `!commands` instead; Hermes normalizes known `!command` aliases to `/command`. |
| **Interactive controls** | Dangerous-command approval and `/model` selection can use Matrix reactions. Approval reactions can be limited to the user who requested the action. |
| **Thinking and tool activity** | Matrix uses threaded, editable thinking/tool-activity panes when gateway progress is enabled, so updates do not flood the main room timeline. |
| **Shared rooms with multiple users** | By default, Hermes isolates session history per user inside the room. Two people talking in the same room do not share one transcript unless you explicitly disable that. |
:::tip
The bot automatically joins rooms when invited. Just invite the bot's Matrix user to any room and it will join and start responding.
:::
## Capability Matrix
This table is backed by the Matrix adapter capability declaration and Matrix test
coverage. E2EE is mode-based because deployments choose whether encrypted rooms
are disabled, opportunistic, or required.
| Capability | Matrix |
|------------|--------|
| text | yes |
| threads | yes |
| reactions | yes |
| approvals | yes |
| model picker | yes |
| thinking panes | yes |
| images | yes |
| multiple images | yes |
| files | yes |
| voice/audio | yes |
| video | yes |
| E2EE | off / optional / required |
| diagnostics | yes |
### Session Model in Matrix
By default:
- each DM gets its own session
- each thread gets its own session namespace
- each user in a shared room gets their own session inside that room
This is controlled by `config.yaml`:
```yaml
group_sessions_per_user: true
```
Set it to `false` only if you explicitly want one shared conversation for the entire room:
```yaml
group_sessions_per_user: false
```
Shared sessions can be useful for a collaborative room, but they also mean:
- users share context growth and token costs
- one person's long tool-heavy task can bloat everyone else's context
- one person's in-flight run can interrupt another person's follow-up in the same room
### Mention and Threading Configuration
You can configure mention and auto-threading behavior via environment variables or `config.yaml`:
```yaml
matrix:
require_mention: true # Require @mention in rooms (default: true)
allowed_users: # Matrix users allowed to trigger agent turns
- "@alice:matrix.org"
allowed_rooms: # Matrix rooms allowed to trigger agent turns
- "!abc123:matrix.org"
free_response_rooms: # Rooms exempt from mention requirement
- "!abc123:matrix.org"
ignore_user_patterns: # Bridge/appservice ghost users to ignore
- "^@telegram_"
- "^@whatsapp_"
process_notices: false # Ignore m.notice by default
session_scope: room # auto|room|thread; room is recommended for project rooms
auto_thread: true # Auto-create threads for responses (default: true)
dm_mention_threads: false # Create thread when @mentioned in DM (default: false)
max_message_length: 16000 # Outbound chunk size in chars (default: 16000, max: 65535)
```
Or via environment variables:
```bash
MATRIX_REQUIRE_MENTION=true
MATRIX_ALLOWED_USERS=@alice:matrix.org
MATRIX_ALLOWED_ROOMS=!abc123:matrix.org
MATRIX_FREE_RESPONSE_ROOMS=!abc123:matrix.org,!def456:matrix.org
MATRIX_IGNORE_USER_PATTERNS='^@telegram_,^@whatsapp_'
MATRIX_PROCESS_NOTICES=false
MATRIX_SESSION_SCOPE=room # recommended for stable project-room context
MATRIX_AUTO_THREAD=true
MATRIX_DM_MENTION_THREADS=false
MATRIX_REACTIONS=true # default: true — emoji reactions during processing
MATRIX_ALLOW_ROOM_MENTIONS=false
```
:::tip Disabling reactions
`MATRIX_REACTIONS=false` turns off the processing-lifecycle emoji reactions (👀/✅/❌) the bot posts on inbound messages. Useful for rooms where reaction events are noisy or aren't supported by all participating clients.
:::
:::tip Room-wide mentions
Hermes sends structured Matrix user mentions for explicit Matrix IDs such as `@alice:example.org`. Room-wide `@room` notifications are disabled by default; set `MATRIX_ALLOW_ROOM_MENTIONS=true` only in rooms where the bot is allowed to notify everyone.
:::
:::note
If you are upgrading from a version that did not have `MATRIX_REQUIRE_MENTION`, the bot previously responded to all messages in rooms. To preserve that behavior, set `MATRIX_REQUIRE_MENTION=false`.
:::
### Project Room Isolation
If you use the same Matrix bot in multiple project rooms, configure stable
room-scoped sessions:
```bash
MATRIX_SESSION_SCOPE=room
MATRIX_AUTO_THREAD=false
```
`MATRIX_SESSION_SCOPE` accepts:
| Scope | Behavior |
|-------|----------|
| `auto` | Backward-compatible default. Existing `MATRIX_AUTO_THREAD` behavior controls synthetic threads. |
| `room` | Unthreaded room messages stay in one stable room session. Real Matrix threads still use their thread root. |
| `thread` | Unthreaded room messages synthesize a thread/session from the triggering event ID. |
Hermes now includes the current Matrix room name, room ID, topic, message ID,
and a Matrix room-boundary note in the agent prompt. `/status` also shows the
current Matrix room/session scope, and `/resume` will not silently resume a
named session from another Matrix room unless you explicitly use
`/resume --cross-room <session name>`.
`MATRIX_SESSION_SCOPE=room` controls the room/thread lane. The existing
`group_sessions_per_user` setting still controls whether users inside that room
share the lane. With `group_sessions_per_user: true` (default), Alice and Bob get
separate Project B sessions. With `group_sessions_per_user: false`, the room has
one shared Project B transcript.
This guide walks you through the full setup process — from creating your bot account to sending your first message.
## Step 1: Create a Bot Account
You need a Matrix user account for the bot. There are several ways to do this:
### Option A: Register on Your Homeserver (Recommended)
If you run your own homeserver (Synapse, Conduit, Dendrite):
1. Use the admin API or registration tool to create a new user:
```bash
# Synapse example
register_new_matrix_user -c /etc/synapse/homeserver.yaml http://localhost:8008
```
2. Choose a username like `hermes` — the full user ID will be `@hermes:your-server.org`.
### Option B: Use matrix.org or Another Public Homeserver
1. Go to [Element Web](https://app.element.io) and create a new account.
2. Pick a username for your bot (e.g., `hermes-bot`).
### Option C: Use Your Own Account
You can also run Hermes as your own user. This means the bot posts as you — useful for personal assistants.
## Step 2: Get an Access Token
Hermes needs an access token to authenticate with the homeserver. You have two options:
### Option A: Access Token (Recommended)
The most reliable way to get a token:
**Via Element:**
1. Log in to [Element](https://app.element.io) with the bot account.
2. Go to **Settings** → **Help & About**.
3. Scroll down and expand **Advanced** — the access token is displayed there.
4. **Copy it immediately.**
**Via the API:**
```bash
curl -X POST https://your-server/_matrix/client/v3/login \
-H "Content-Type: application/json" \
-d '{
"type": "m.login.password",
"user": "@hermes:your-server.org",
"password": "your-password"
}'
```
The response includes an `access_token` field — copy it.
:::warning[Keep your access token safe]
The access token gives full access to the bot's Matrix account. Never share it publicly or commit it to Git. If compromised, revoke it by logging out all sessions for that user.
:::
### Option B: Password Login
Instead of providing an access token, you can give Hermes the bot's user ID and password. Hermes will log in automatically on startup. This is simpler but means the password is stored in your `.env` file.
```bash
MATRIX_USER_ID=@hermes:your-server.org
MATRIX_PASSWORD=your-password
```
## Step 3: Find Your Matrix User ID
Hermes Agent uses your Matrix User ID to control who can interact with the bot. Matrix User IDs follow the format `@username:server`.
To find yours:
1. Open [Element](https://app.element.io) (or your preferred Matrix client).
2. Click your avatar → **Settings**.
3. Your User ID is displayed at the top of the profile (e.g., `@alice:matrix.org`).
:::tip
Matrix User IDs always start with `@` and contain a `:` followed by the server name. For example: `@alice:matrix.org`, `@bob:your-server.com`.
:::
## Step 4: Configure Hermes Agent
### Option A: Interactive Setup (Recommended)
Run the guided setup command:
```bash
hermes gateway setup
```
Select **Matrix** when prompted, then provide your homeserver URL, access token (or user ID + password), and allowed user IDs when asked.
### Option B: Manual Configuration
Add the following to your `~/.hermes/.env` file:
**Using an access token:**
```bash
# Required
MATRIX_HOMESERVER=https://matrix.example.org
MATRIX_ACCESS_TOKEN=***
# Optional: user ID (auto-detected from token if omitted)
# MATRIX_USER_ID=@hermes:matrix.example.org
# Security: restrict who can interact with the bot
MATRIX_ALLOWED_USERS=@alice:matrix.example.org
# Optional: restrict which rooms can trigger the bot
MATRIX_ALLOWED_ROOMS=!abc123:matrix.example.org
# Multiple allowed users (comma-separated)
# MATRIX_ALLOWED_USERS=@alice:matrix.example.org,@bob:matrix.example.org
```
**Using password login:**
```bash
# Required
MATRIX_HOMESERVER=https://matrix.example.org
MATRIX_USER_ID=@hermes:matrix.example.org
MATRIX_PASSWORD=***
# Security
MATRIX_ALLOWED_USERS=@alice:matrix.example.org
```
## Private Deployment Hardening
For private Matrix deployments, set both user and room allowlists. If
`MATRIX_ALLOWED_USERS` is unset, any sender who can reach the bot in a joined
room can trigger an agent turn. If `MATRIX_ALLOWED_ROOMS` is unset, any room the
bot joins can trigger an agent turn. A locked-down deployment should set both:
```bash
MATRIX_ALLOWED_USERS=@alice:matrix.example.org,@bob:matrix.example.org
MATRIX_ALLOWED_ROOMS=!ops:matrix.example.org,!dmroom:matrix.example.org
```
Bridge and appservice deployments need extra loop protection. Hermes always
ignores its own events, Matrix appservice-style users whose localpart starts
with `_`, duplicate event IDs, old startup events, edit replacement events, and
`m.notice` events by default. Add deployment-specific bridge ghost patterns when
your bridge uses a different naming convention:
```bash
MATRIX_IGNORE_USER_PATTERNS='^@telegram_,^@slack_,^@whatsapp_'
```
Only enable notices when a trusted human workflow really sends `m.notice`:
```bash
MATRIX_PROCESS_NOTICES=true
```
Outbound whole-room notifications are disabled by default. Keep
`MATRIX_ALLOW_ROOM_MENTIONS=false` unless the bot is explicitly allowed to wake
the whole room with `@room`.
Diagnostics and debug payloads redact Matrix access tokens, recovery keys,
device identifiers, and message bodies. Media downloads are limited to Matrix
`mxc://` content URIs and rejected when they exceed `MATRIX_MAX_MEDIA_BYTES`.
Treat federated rooms and untrusted homeservers as untrusted input: keep room
allowlists tight, prefer DMs or private rooms for tool-heavy work, and avoid
authorizing bridge ghosts or appservice puppets as allowed users.
Optional behavior settings in `~/.hermes/config.yaml`:
```yaml
group_sessions_per_user: true
```
- `group_sessions_per_user: true` keeps each participant's context isolated inside shared rooms
### Start the Gateway
Once configured, start the Matrix gateway:
```bash
hermes gateway
```
The bot should connect to your homeserver and start syncing within a few seconds. Send it a message — either a DM or in a room it has joined — to test.
:::tip
You can run `hermes gateway` in the background or as a systemd service for persistent operation. See the deployment docs for details.
:::
## End-to-End Encryption (E2EE)
Hermes supports Matrix end-to-end encryption, so you can chat with your bot in encrypted rooms.
### Requirements
E2EE requires the `mautrix` library with encryption extras and the `libolm` C library:
```bash
# Install mautrix with E2EE support
pip install 'mautrix[encryption]'
# Or install with hermes extras
cd ~/.hermes/hermes-agent && uv pip install -e ".[matrix]"
```
You also need `libolm` installed on your system:
```bash
# Debian/Ubuntu
sudo apt install libolm-dev
# macOS
brew install libolm
# Fedora
sudo dnf install libolm-devel
```
### Enable E2EE
Add to your `~/.hermes/.env`:
```bash
MATRIX_E2EE_MODE=required
```
`MATRIX_E2EE_MODE` accepts:
| Mode | Behavior |
|------|----------|
| `off` | Do not initialize Matrix E2EE. |
| `optional` | Try E2EE when dependencies are available, but keep unencrypted rooms working if crypto cannot initialize. |
| `required` | Fail closed if E2EE dependencies or crypto setup are not available. |
Optional mode may fall back to non-E2EE operation when crypto setup is unavailable. Required mode fails closed instead of silently downgrading.
For backwards compatibility, `MATRIX_ENCRYPTION=true` still enables required E2EE behavior.
When E2EE is enabled, Hermes:
- Stores encryption keys in `~/.hermes/platforms/matrix/store/` (legacy installs: `~/.hermes/matrix/store/`)
- Uploads device keys on first connection
- Decrypts incoming messages and encrypts outgoing messages automatically
- Auto-joins encrypted rooms when invited
### Matrix Tools and Controls
In Matrix conversations, Hermes exposes Matrix-specific tools to the agent:
- `matrix_send_reaction`
- `matrix_redact_message`
- `matrix_create_room`
- `matrix_invite_user`
- `matrix_fetch_history`
- `matrix_set_presence`
These tools are scoped to Matrix contexts and are not available in non-Matrix toolsets. Admin-style tools are disabled by default: redaction requires `MATRIX_TOOLS_ALLOW_REDACTION=true`, invites require `MATRIX_TOOLS_ALLOW_INVITES=true`, and room creation requires `MATRIX_TOOLS_ALLOW_ROOM_CREATE=true`. Public room creation also requires `MATRIX_ALLOW_PUBLIC_ROOMS=true`.
If `MATRIX_ALLOWED_ROOMS` is set, Matrix tools may only target those rooms.
Reaction controls use:
- ✅ approve once
- ♾️ approve always
- ❌ deny
- number reactions for `/model` choices
Set `MATRIX_APPROVAL_REQUIRE_SENDER=false` if you intentionally want any authorized Matrix user in the room to operate an approval/model picker prompt. The default is requester-bound when Hermes knows who requested the action.
### Media Limits
Hermes uploads and downloads Matrix images, files, audio, and video through Matrix media APIs. Multiple generated images are sent as one ordered logical batch, preserving captions and thread context across the batch.
By default, Matrix media over 100 MB is rejected before upload/download. Override with:
```bash
MATRIX_MAX_MEDIA_BYTES=104857600
```
Inbound media must use Matrix `mxc://` content URIs. Hermes rejects arbitrary
HTTP(S) media URLs in Matrix events to avoid turning a federated room into an
unrestricted downloader.
### Cross-Signing Verification (Recommended)
If your Matrix account has cross-signing enabled (the default in Element), set the recovery key so the bot can self-sign its device on startup. Without this, other Matrix clients may refuse to share encryption sessions with the bot after a device key rotation.
```bash
MATRIX_RECOVERY_KEY=EsT... your recovery key here
```
**Where to find it:** In Element, go to **Settings** → **Security & Privacy** → **Encryption** → your recovery key (also called the "Security Key"). This is the key you were asked to save when you first set up cross-signing.
On each startup, if `MATRIX_RECOVERY_KEY` is set, Hermes imports cross-signing keys from the homeserver's secure secret storage and signs the current device. This is idempotent and safe to leave enabled permanently.
If Hermes bootstraps a new Matrix recovery key, it never logs the raw key. Set
`MATRIX_RECOVERY_KEY_OUTPUT_FILE=/secure/path/matrix-recovery-key.txt` before
startup to write a generated key once with file mode `0600`; the file is not
overwritten if it already exists.
:::warning[Deleting the crypto store]
If you delete `~/.hermes/platforms/matrix/store/crypto.db`, the bot loses its encryption identity. Simply restarting with the same device ID will **not** fully recover — the homeserver still holds one-time keys signed with the old identity key, and peers cannot establish new Olm sessions.
Hermes detects this condition on startup and refuses to enable E2EE, logging: `device XXXX has stale one-time keys on the server signed with a previous identity key`.
**Easiest recovery: generate a new access token** (which gets a fresh device ID with no stale key history). See the "Upgrading from a previous version with E2EE" section below. This is the most reliable path and avoids touching the homeserver database.
**Manual recovery** (advanced — keeps the same device ID):
1. Stop Synapse and delete the old device from its database:
```bash
sudo systemctl stop matrix-synapse
sudo sqlite3 /var/lib/matrix-synapse/homeserver.db "
DELETE FROM e2e_device_keys_json WHERE device_id = 'DEVICE_ID' AND user_id = '@hermes:your-server';
DELETE FROM e2e_one_time_keys_json WHERE device_id = 'DEVICE_ID' AND user_id = '@hermes:your-server';
DELETE FROM e2e_fallback_keys_json WHERE device_id = 'DEVICE_ID' AND user_id = '@hermes:your-server';
DELETE FROM devices WHERE device_id = 'DEVICE_ID' AND user_id = '@hermes:your-server';
"
sudo systemctl start matrix-synapse
```
Or via the Synapse admin API (note the URL-encoded user ID):
```bash
curl -X DELETE -H "Authorization: Bearer ADMIN_TOKEN" \
'https://your-server/_synapse/admin/v2/users/%40hermes%3Ayour-server/devices/DEVICE_ID'
```
Note: deleting a device via the admin API may also invalidate the associated access token. You may need to generate a new token afterward.
2. Delete the local crypto store and restart Hermes:
```bash
rm -f ~/.hermes/platforms/matrix/store/crypto.db*
# restart hermes
```
Other Matrix clients (Element, matrix-commander) may cache the old device keys. After recovery, type `/discardsession` in Element to force a new encryption session with the bot.
:::
:::info
If `mautrix[encryption]` is not installed or `libolm` is missing, the bot falls back to a plain (unencrypted) client automatically. You'll see a warning in the logs.
:::
## Home Room
You can designate a "home room" where the bot sends proactive messages (such as cron job output, reminders, and notifications). There are two ways to set it:
### Using the Slash Command
Type `/sethome` in any Matrix room where the bot is present. That room becomes the home room.
If your Matrix client intercepts slash commands, type `!sethome` instead.
### Manual Configuration
Add this to your `~/.hermes/.env`:
```bash
MATRIX_HOME_ROOM=!abc123def456:matrix.example.org
```
## Room allowlist (`allowed_rooms`)
Restrict the bot to a fixed set of Matrix rooms. When set, the bot **only** responds in rooms whose ID appears in the list — messages from any other room are silently ignored, even if the bot is mentioned.
**DMs (direct chat rooms) are exempt** from this filter, so authorized users can always reach the bot one-on-one.
```yaml
matrix:
allowed_rooms:
- "!abc123def456:matrix.example.org"
- "!opsroom789:matrix.example.org"
```
Or via env var (comma-separated):
```bash
MATRIX_ALLOWED_ROOMS="!abc123def456:matrix.example.org,!opsroom789:matrix.example.org"
```
Behavior:
- Empty / unset → no restriction (default).
- Non-empty → room ID must be on the list. The check runs **before** any other gating (mention requirement, sender allowlist, etc.).
- Use the room's **internal ID** (`!abc...:server`), not its alias (`#room:server`). You can find a room's internal ID in Element via Room → Settings → Advanced.
See also: [admin/user slash command split](../../reference/slash-commands.md#permissions-and-adminuser-split).
:::tip
To find a Room ID: in Element, go to the room → **Settings** → **Advanced** → the **Internal room ID** is shown there (starts with `!`).
:::
## Commands in Matrix
Hermes supports the same gateway commands in Matrix that it supports on other
messaging platforms, including `/commands`, `/model`, `/stop`, `/queue`,
`/steer`, `/goal`, `/subgoal`, `/bg`, `/btw`, `/tasks`, and
`/yolo`.
Some Matrix clients reserve leading `/` for local client commands and may not
send unknown slash commands to the room. In that case, use `!` as a Matrix-safe
alias:
```text
!commands
!model
!model gpt-5.5 --provider openrouter
!queue continue with the next task
!stop
```
Hermes only normalizes `!command` when the command is known to the gateway, a
registered plugin command, or an installed skill command. Ordinary exclamations
such as `!important` remain normal chat messages.
## Troubleshooting
### Bot is not responding to messages
**Cause**: The bot hasn't joined the room, `MATRIX_ALLOWED_USERS` doesn't include your User ID, `MATRIX_ALLOWED_ROOMS` doesn't include the room, or a room message did not mention the bot.
**Fix**: Invite the bot to the room — it auto-joins on invite. Verify your User ID is in `MATRIX_ALLOWED_USERS` (use the full `@user:server` format) and the room ID is in `MATRIX_ALLOWED_ROOMS` if that allowlist is configured. In rooms, mention the bot or add the room to `MATRIX_FREE_RESPONSE_ROOMS`. Restart the gateway.
### Bot joins rooms but silently drops every message (clock skew)
**Cause**: The host's system clock is set ahead of real time. The Matrix adapter applies a 5-second startup-grace filter (`event_ts < startup_ts - 5`) to ignore events replayed from initial sync. When the wall clock is ahead, every incoming event looks "older than startup" and is dropped before reaching the message handler — the bot appears connected but never replies. See [#12614](https://github.com/NousResearch/hermes-agent/issues/12614).
**Symptom**: Gateway log shows `Matrix: dropped N live events as 'too old' more than 30s after startup`.
**Fix**: Sync the host clock with NTP and restart the bot:
```bash
# Debian/Ubuntu
sudo timedatectl set-ntp true
timedatectl status # confirm "System clock synchronized: yes"
# macOS
sudo sntp -sS time.apple.com
```
### "Failed to authenticate" / "whoami failed" on startup
**Cause**: The access token or homeserver URL is incorrect.
**Fix**: Verify `MATRIX_HOMESERVER` points to your homeserver (include `https://`, no trailing slash). Check that `MATRIX_ACCESS_TOKEN` is valid — try it with curl:
```bash
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://your-server/_matrix/client/v3/account/whoami
```
If this returns your user info, the token is valid. If it returns an error, generate a new token.
### "mautrix not installed" error
**Cause**: The `mautrix` Python package is not installed.
**Fix**: Install it:
```bash
pip install 'mautrix[encryption]'
```
Or with Hermes extras:
```bash
cd ~/.hermes/hermes-agent && uv pip install -e ".[matrix]"
```
### Encryption errors / "could not decrypt event"
**Cause**: Missing encryption keys, `libolm` not installed, or the bot's device isn't trusted.
**Fix**:
1. Verify `libolm` is installed on your system (see the E2EE section above).
2. Make sure `MATRIX_ENCRYPTION=true` is set in your `.env`.
3. In your Matrix client (Element), go to the bot's profile -> Sessions -> verify/trust the bot's device.
4. If the bot just joined an encrypted room, it can only decrypt messages sent *after* it joined. Older messages are inaccessible.
### Upgrading from a previous version with E2EE
:::tip
If you also manually deleted `crypto.db`, see the "Deleting the crypto store" warning in the E2EE section above — there are additional steps to clear stale one-time keys from the homeserver.
:::
If you previously used Hermes with `MATRIX_ENCRYPTION=true` and are upgrading to
a version that uses the new SQLite-based crypto store, the bot's encryption
identity has changed. Your Matrix client (Element) may cache the old device keys
and refuse to share encryption sessions with the bot.
**Symptoms**: The bot connects and shows "E2EE enabled" in the logs, but all
messages show "could not decrypt event" and the bot never responds.
**What's happening**: The old encryption state (from the previous `matrix-nio` or
serialization-based `mautrix` backend) is incompatible with the new SQLite crypto
store. The bot creates a fresh encryption identity, but your Matrix client still
has the old keys cached and won't share the room's encryption session with a
device whose keys changed. This is a Matrix security feature -- clients treat
changed identity keys for the same device as suspicious.
**Fix** (one-time migration):
1. **Generate a new access token** to get a fresh device ID. The simplest way:
```bash
curl -X POST https://your-server/_matrix/client/v3/login \
-H "Content-Type: application/json" \
-d '{
"type": "m.login.password",
"identifier": {"type": "m.id.user", "user": "@hermes:your-server.org"},
"password": "***",
"initial_device_display_name": "Hermes Agent"
}'
```
Copy the new `access_token` and update `MATRIX_ACCESS_TOKEN` in `~/.hermes/.env`.
2. **Delete old encryption state**:
```bash
rm -f ~/.hermes/platforms/matrix/store/crypto.db
rm -f ~/.hermes/platforms/matrix/store/crypto_store.*
```
3. **Set your recovery key** (if you use cross-signing — most Element users do). Add to `~/.hermes/.env`:
```bash
MATRIX_RECOVERY_KEY=EsT... your recovery key here
```
This lets the bot self-sign with cross-signing keys on startup, so Element trusts the new device immediately. Without this, Element may see the new device as unverified and refuse to share encryption sessions. Find your recovery key in Element under **Settings** → **Security & Privacy** → **Encryption**.
4. **Force your Matrix client to rotate the encryption session**. In Element,
open the DM room with the bot and type `/discardsession`. This forces Element
to create a new encryption session and share it with the bot's new device.
5. **Restart the gateway**:
```bash
hermes gateway run
```
If `MATRIX_RECOVERY_KEY` is set, you should see `Matrix: cross-signing verified via recovery key` in the logs.
6. **Send a new message**. The bot should decrypt and respond normally.
:::note
After migration, messages sent *before* the upgrade cannot be decrypted -- the old
encryption keys are gone. This only affects the transition; new messages work
normally.
:::
:::tip
**New installations are not affected.** This migration is only needed if you had
a working E2EE setup with a previous version of Hermes and are upgrading.
**Why a new access token?** Each Matrix access token is bound to a specific device
ID. Reusing the same device ID with new encryption keys causes other Matrix
clients to distrust the device (they see changed identity keys as a potential
security breach). A new access token gets a new device ID with no stale key
history, so other clients trust it immediately.
:::
## Proxy Mode (E2EE on macOS)
Matrix E2EE requires `libolm`, which doesn't compile on macOS ARM64 (Apple Silicon). The `hermes-agent[matrix]` extra is gated to Linux only. If you're on macOS, proxy mode lets you run E2EE in a Docker container on a Linux VM while the actual agent runs natively on macOS with full access to your local files, memory, and skills.
### How It Works
```
macOS (Host):
└─ hermes gateway
├─ api_server adapter ← listens on 0.0.0.0:8642
├─ AIAgent ← single source of truth
├─ Sessions, memory, skills
└─ Local file access (Obsidian, projects, etc.)
Linux VM (Docker):
└─ hermes gateway (proxy mode)
├─ Matrix adapter ← E2EE decryption/encryption
└─ HTTP forward → macOS:8642/v1/chat/completions
(no LLM API keys, no agent, no inference)
```
The Docker container only handles Matrix protocol + E2EE. When a message arrives, it decrypts it and forwards the text to the host via a standard HTTP request. The host runs the agent, calls tools, generates a response, and streams it back. The container encrypts and sends the response to Matrix. All sessions are unified — CLI, Matrix, Telegram, and any other platform share the same memory and conversation history.
### Step 1: Configure the Host (macOS)
Enable the API server so the host accepts incoming requests from the Docker container.
Add to `~/.hermes/.env`:
```bash
API_SERVER_ENABLED=true
API_SERVER_KEY=your-secret-key-here
API_SERVER_HOST=0.0.0.0
```
- `API_SERVER_HOST=0.0.0.0` binds to all interfaces so the Docker container can reach it.
- `API_SERVER_KEY` is required for non-loopback binding. Pick a strong random string.
- The API server runs on port 8642 by default (change with `API_SERVER_PORT` if needed).
Start the gateway:
```bash
hermes gateway
```
You should see the API server start alongside any other platforms you have configured. Verify it's reachable from the VM:
```bash
# From the Linux VM
curl http://<mac-ip>:8642/health
```
### Step 2: Configure the Docker Container (Linux VM)
The container needs Matrix credentials and the proxy URL. It does NOT need LLM API keys.
**`docker-compose.yml`:**
```yaml
services:
hermes-matrix:
build: .
environment:
# Matrix credentials
MATRIX_HOMESERVER: "https://matrix.example.org"
MATRIX_ACCESS_TOKEN: "syt_..."
MATRIX_ALLOWED_USERS: "@you:matrix.example.org"
MATRIX_ENCRYPTION: "true"
MATRIX_DEVICE_ID: "HERMES_BOT"
# Proxy mode — forward to host agent
GATEWAY_PROXY_URL: "http://192.168.1.100:8642"
GATEWAY_PROXY_KEY: "your-secret-key-here"
volumes:
- ./matrix-store:/root/.hermes/platforms/matrix/store
```
**`Dockerfile`:**
```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y libolm-dev && rm -rf /var/lib/apt/lists/*
RUN cd ~/.hermes/hermes-agent && uv pip install -e ".[matrix]"
CMD ["hermes", "gateway"]
```
That's the entire container. No API keys for OpenRouter, Anthropic, or any inference provider.
### Step 3: Start Both
1. Start the host gateway first:
```bash
hermes gateway
```
2. Start the Docker container:
```bash
docker compose up -d
```
3. Send a message in an encrypted Matrix room. The container decrypts it, forwards it to the host, and streams the response back.
### Configuration Reference
Proxy mode is configured on the **container side** (the thin gateway):
| Setting | Description |
|---------|-------------|
| `GATEWAY_PROXY_URL` | URL of the remote Hermes API server (e.g., `http://192.168.1.100:8642`) |
| `GATEWAY_PROXY_KEY` | Bearer token for authentication (must match `API_SERVER_KEY` on the host) |
| `gateway.proxy_url` | Same as `GATEWAY_PROXY_URL` but in `config.yaml` |
The host side needs:
| Setting | Description |
|---------|-------------|
| `API_SERVER_ENABLED` | Set to `true` |
| `API_SERVER_KEY` | Bearer token (shared with the container) |
| `API_SERVER_HOST` | Set to `0.0.0.0` for network access |
| `API_SERVER_PORT` | Port number (default: `8642`) |
### Works for Any Platform
Proxy mode is not limited to Matrix. Any platform adapter can use it — set `GATEWAY_PROXY_URL` on any gateway instance and it will forward to the remote agent instead of running one locally. This is useful for any deployment where the platform adapter needs to run in a different environment from the agent (network isolation, E2EE requirements, resource constraints).
:::tip
Session continuity is maintained via the `X-Hermes-Session-Id` header. The host's API server tracks sessions by this ID, so conversations persist across messages just like they would with a local agent.
:::
:::note
**Limitations (v1):** Tool progress messages from the remote agent are not relayed back — the user sees the streamed final response only, not individual tool calls. Dangerous command approval prompts are handled on the host side, not relayed to the Matrix user. These can be addressed in future updates.
:::
### Bot connects and sends, but ignores inbound messages
**Cause**: Matrix event handlers only fire when sync payloads are dispatched through
mautrix's `handle_sync()` machinery. A raw `client.sync()` poll that never calls
`handle_sync()` can leave the adapter connected (send works) while inbound
messages never reach `_on_room_message`.
**Fix**: Hermes uses an explicit sync loop that calls `client.handle_sync()` on
both the initial sync and every incremental sync response. This matches the
diagnosis in upstream issue #7914 and closed PR #37807, but keeps Hermes's own
background maintenance tasks (joined-room tracking, invite handling, E2EE key
share) instead of delegating the full lifecycle to `client.start()`. If inbound
messages still fail after a gateway restart, verify handlers are registered before
the first sync and check logs for `sync event dispatch error`.
### Sync issues / bot falls behind
**Cause**: Long-running tool executions can delay the sync loop, or the homeserver is slow.
**Fix**: The sync loop automatically retries every 5 seconds on error. Check the Hermes logs for sync-related warnings. If the bot consistently falls behind, ensure your homeserver has adequate resources.
### Bot is offline
**Cause**: The Hermes gateway isn't running, or it failed to connect.
**Fix**: Check that `hermes gateway` is running. Look at the terminal output for error messages. Common issues: wrong homeserver URL, expired access token, homeserver unreachable.
### "User not allowed" / Bot ignores you
**Cause**: Your User ID isn't in `MATRIX_ALLOWED_USERS`.
**Fix**: Add your User ID to `MATRIX_ALLOWED_USERS` in `~/.hermes/.env` and restart the gateway. Use the full `@user:server` format.
### Bot ignores an entire room
**Cause**: `MATRIX_ALLOWED_ROOMS` is set and the current room ID is not listed, or the room requires a mention and the message did not mention the bot.
**Fix**: Add the room ID to `MATRIX_ALLOWED_ROOMS`, or remove the room allowlist if this is a personal deployment. To find a Room ID in Element, open room settings and check **Advanced**.
### Bridge messages loop or echo
**Cause**: A bridge/appservice puppet is relaying bot output back as a new user message, or a bridge uses non-standard ghost user IDs.
**Fix**: Keep bridge ghosts out of `MATRIX_ALLOWED_USERS`, add a matching `MATRIX_IGNORE_USER_PATTERNS` entry, and leave `MATRIX_PROCESS_NOTICES=false` unless notices are part of a trusted workflow.
## Security
:::warning
Always set `MATRIX_ALLOWED_USERS` and, for shared/private deployments, `MATRIX_ALLOWED_ROOMS`. Without them, anyone who can message the bot in a joined room may trigger the agent. Only authorize people and rooms you trust — authorized users have full access to the agent's capabilities, including tool use and system access.
:::
For more information on securing your Hermes Agent deployment, see the [Security Guide](../security.md).
## Notes
- **Any homeserver**: Works with Synapse, Conduit, Dendrite, matrix.org, or any spec-compliant Matrix homeserver. No specific homeserver software required.
- **Federation**: If you're on a federated homeserver, the bot can communicate with users from other servers — just add their full `@user:server` IDs to `MATRIX_ALLOWED_USERS`.
- **Auto-join**: The bot automatically accepts room invites and joins. It starts responding immediately after joining.
- **Media support**: Hermes can send and receive images, audio, video, and file attachments. Media is uploaded to your homeserver using the Matrix content repository API.
- **Native voice messages (MSC3245)**: The Matrix adapter automatically tags outgoing voice messages with the `org.matrix.msc3245.voice` flag. This means TTS responses and voice audio are rendered as **native voice bubbles** in Element and other clients that support MSC3245, rather than as generic audio file attachments. Incoming voice messages with the MSC3245 flag are also correctly identified and routed to speech-to-text transcription. No configuration is needed — this works automatically.
website/docs/user-guide/messaging/mattermost.md
+340
View File
@@ -0,0 +1,340 @@
---
sidebar_position: 8
title: "Mattermost"
description: "Set up Hermes Agent as a Mattermost bot"
---
# Mattermost Setup
Hermes Agent integrates with Mattermost as a bot, letting you chat with your AI assistant through direct messages or team channels. Mattermost is a self-hosted, open-source Slack alternative — you run it on your own infrastructure, keeping full control of your data. The bot connects via Mattermost's REST API (v4) and WebSocket for real-time events, processes messages through the Hermes Agent pipeline (including tool use, memory, and reasoning), and responds in real time. It supports text, file attachments, images, and slash commands.
No external Mattermost library is required — the adapter uses `aiohttp`, which is already a Hermes dependency.
Before setup, here's the part most people want to know: how Hermes behaves once it's in your Mattermost instance.
## How Hermes Behaves
| Context | Behavior |
|---------|----------|
| **DMs** | Hermes responds to every message. No `@mention` needed. Each DM has its own session. |
| **Public/private channels** | Hermes responds when you `@mention` it. Without a mention, Hermes ignores the message. |
| **Threads** | If `MATTERMOST_REPLY_MODE=thread`, Hermes replies in a thread under your message. Thread context stays isolated from the parent channel. |
| **Shared channels with multiple users** | By default, Hermes isolates session history per user inside the channel. Two people talking in the same channel do not share one transcript unless you explicitly disable that. |
:::tip
If you want Hermes to reply as threaded conversations (nested under your original message), set `MATTERMOST_REPLY_MODE=thread`. The default is `off`, which sends flat messages in the channel.
:::
### Session Model in Mattermost
By default:
- each DM gets its own session
- each thread gets its own session namespace
- each user in a shared channel gets their own session inside that channel
This is controlled by `config.yaml`:
```yaml
group_sessions_per_user: true
```
Set it to `false` only if you explicitly want one shared conversation for the entire channel:
```yaml
group_sessions_per_user: false
```
Shared sessions can be useful for a collaborative channel, but they also mean:
- users share context growth and token costs
- one person's long tool-heavy task can bloat everyone else's context
- one person's in-flight run can interrupt another person's follow-up in the same channel
This guide walks you through the full setup process — from creating your bot on Mattermost to sending your first message.
## Step 1: Enable Bot Accounts
Bot accounts must be enabled on your Mattermost server before you can create one.
1. Log in to Mattermost as a **System Admin**.
2. Go to **System Console** → **Integrations** → **Bot Accounts**.
3. Set **Enable Bot Account Creation** to **true**.
4. Click **Save**.
:::info
If you don't have System Admin access, ask your Mattermost administrator to enable bot accounts and create one for you.
:::
## Step 2: Create a Bot Account
1. In Mattermost, click the **☰** menu (top-left) → **Integrations** → **Bot Accounts**.
2. Click **Add Bot Account**.
3. Fill in the details:
- **Username**: e.g., `hermes`
- **Display Name**: e.g., `Hermes Agent`
- **Description**: optional
- **Role**: `Member` is sufficient
4. Click **Create Bot Account**.
5. Mattermost will display the **bot token**. **Copy it immediately.**
:::warning[Token shown only once]
The bot token is only displayed once when you create the bot account. If you lose it, you'll need to regenerate it from the bot account settings. Never share your token publicly or commit it to Git — anyone with this token has full control of the bot.
:::
Store the token somewhere safe (a password manager, for example). You'll need it in Step 5.
:::tip
You can also use a **personal access token** instead of a bot account. Go to **Profile** → **Security** → **Personal Access Tokens** → **Create Token**. This is useful if you want Hermes to post as your own user rather than a separate bot user.
:::
## Step 3: Add the Bot to Channels
The bot needs to be a member of any channel where you want it to respond:
1. Open the channel where you want the bot.
2. Click the channel name → **Add Members**.
3. Search for your bot username (e.g., `hermes`) and add it.
For DMs, simply open a direct message with the bot — it will be able to respond immediately.
## Step 4: Find Your Mattermost User ID
Hermes Agent uses your Mattermost User ID to control who can interact with the bot. To find it:
1. Click your **avatar** (top-left corner) → **Profile**.
2. Your User ID is displayed in the profile dialog — click it to copy.
Your User ID is a 26-character alphanumeric string like `3uo8dkh1p7g1mfk49ear5fzs5c`.
:::warning
Your User ID is **not** your username. The username is what appears after `@` (e.g., `@alice`). The User ID is a long alphanumeric identifier that Mattermost uses internally.
:::
**Alternative**: You can also get your User ID via the API:
```bash
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://your-mattermost-server/api/v4/users/me | jq .id
```
:::tip
To get a **Channel ID**: click the channel name → **View Info**. The Channel ID is shown in the info panel. You'll need this if you want to set a home channel manually.
:::
## Step 5: Configure Hermes Agent
### Option A: Interactive Setup (Recommended)
Run the guided setup command:
```bash
hermes gateway setup
```
Select **Mattermost** when prompted, then paste your server URL, bot token, and user ID when asked.
### Option B: Manual Configuration
Add the following to your `~/.hermes/.env` file:
```bash
# Required
MATTERMOST_URL=https://mm.example.com
MATTERMOST_TOKEN=***
MATTERMOST_ALLOWED_USERS=3uo8dkh1p7g1mfk49ear5fzs5c
# Multiple allowed users (comma-separated)
# MATTERMOST_ALLOWED_USERS=3uo8dkh1p7g1mfk49ear5fzs5c,8fk2jd9s0a7bncm1xqw4tp6r3e
# Optional: reply mode (thread or off, default: off)
# MATTERMOST_REPLY_MODE=thread
# Optional: respond without @mention (default: true = require mention)
# MATTERMOST_REQUIRE_MENTION=false
# Optional: channels where bot responds without @mention (comma-separated channel IDs)
# MATTERMOST_FREE_RESPONSE_CHANNELS=channel_id_1,channel_id_2
```
Optional behavior settings in `~/.hermes/config.yaml`:
```yaml
group_sessions_per_user: true
```
- `group_sessions_per_user: true` keeps each participant's context isolated inside shared channels and threads
### Start the Gateway
Once configured, start the Mattermost gateway:
```bash
hermes gateway
```
The bot should connect to your Mattermost server within a few seconds. Send it a message — either a DM or in a channel where it's been added — to test.
:::tip
You can run `hermes gateway` in the background or as a systemd service for persistent operation. See the deployment docs for details.
:::
## Home Channel
You can designate a "home channel" where the bot sends proactive messages (such as cron job output, reminders, and notifications). There are two ways to set it:
### Using the Slash Command
Type `/sethome` in any Mattermost channel where the bot is present. That channel becomes the home channel.
### Manual Configuration
Add this to your `~/.hermes/.env`:
```bash
MATTERMOST_HOME_CHANNEL=abc123def456ghi789jkl012mn
```
Replace the ID with the actual channel ID (click the channel name → View Info → copy the ID).
## Reply Mode
The `MATTERMOST_REPLY_MODE` setting controls how Hermes posts responses:
| Mode | Behavior |
|------|----------|
| `off` (default) | Hermes posts flat messages in the channel, like a normal user. |
| `thread` | Hermes replies in a thread under your original message. Keeps channels clean when there's lots of back-and-forth. |
Set it in your `~/.hermes/.env`:
```bash
MATTERMOST_REPLY_MODE=thread
```
## Mention Behavior
By default, the bot only responds in channels when `@mentioned`. You can change this:
| Variable | Default | Description |
|----------|---------|-------------|
| `MATTERMOST_REQUIRE_MENTION` | `true` | Set to `false` to respond to all messages in channels (DMs always work). |
| `MATTERMOST_FREE_RESPONSE_CHANNELS` | _(none)_ | Comma-separated channel IDs where the bot responds without `@mention`, even when require_mention is true. |
To find a channel ID in Mattermost: open the channel, click the channel name header, and look for the ID in the URL or channel details.
When the bot is `@mentioned`, the mention is automatically stripped from the message before processing.
## Channel allowlist (`allowed_channels`)
Restrict the bot to a fixed set of Mattermost channels. When set, the bot **only** responds in channels whose ID appears in the list — messages from any other channel are silently ignored, even if the bot is `@mentioned`.
**DMs are exempt** from this filter, so authorized users can always reach the bot in a direct message.
```yaml
mattermost:
allowed_channels:
- "abc123def456ghi789jkl012mno" # #ops
- "xyz987uvw654rst321opq098nml" # #incident-response
```
Or via env var (comma-separated):
```bash
MATTERMOST_ALLOWED_CHANNELS="abc123def456ghi789jkl012mno,xyz987uvw654rst321opq098nml"
```
Behavior:
- Empty / unset → no restriction (fully backward compatible).
- Non-empty → channel ID must be on the list, or the message is dropped before any other gating (mention requirement, `MATTERMOST_FREE_RESPONSE_CHANNELS`, etc.) runs.
- Find a channel ID via the Mattermost UI → channel header → "View Info", or read it from the channel URL.
See also: [admin/user slash command split](../../reference/slash-commands.md#permissions-and-adminuser-split).
## Troubleshooting
### Bot is not responding to messages
**Cause**: The bot is not a member of the channel, or `MATTERMOST_ALLOWED_USERS` doesn't include your User ID.
**Fix**: Add the bot to the channel (channel name → Add Members → search for the bot). Verify your User ID is in `MATTERMOST_ALLOWED_USERS`. Restart the gateway.
### 403 Forbidden errors
**Cause**: The bot token is invalid, or the bot doesn't have permission to post in the channel.
**Fix**: Check that `MATTERMOST_TOKEN` in your `.env` file is correct. Make sure the bot account hasn't been deactivated. Verify the bot has been added to the channel. If using a personal access token, ensure your account has the required permissions.
### WebSocket disconnects / reconnection loops
**Cause**: Network instability, Mattermost server restarts, or firewall/proxy issues with WebSocket connections.
**Fix**: The adapter automatically reconnects with exponential backoff (2s → 60s). Check your server's WebSocket configuration — reverse proxies (nginx, Apache) need WebSocket upgrade headers configured. Verify no firewall is blocking WebSocket connections on your Mattermost server.
For nginx, ensure your config includes:
```nginx
location /api/v4/websocket {
proxy_pass http://mattermost-backend;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 600s;
}
```
### "Failed to authenticate" on startup
**Cause**: The token or server URL is incorrect.
**Fix**: Verify `MATTERMOST_URL` points to your Mattermost server (include `https://`, no trailing slash). Check that `MATTERMOST_TOKEN` is valid — try it with curl:
```bash
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://your-server/api/v4/users/me
```
If this returns your bot's user info, the token is valid. If it returns an error, regenerate the token.
### Bot is offline
**Cause**: The Hermes gateway isn't running, or it failed to connect.
**Fix**: Check that `hermes gateway` is running. Look at the terminal output for error messages. Common issues: wrong URL, expired token, Mattermost server unreachable.
### "User not allowed" / Bot ignores you
**Cause**: Your User ID isn't in `MATTERMOST_ALLOWED_USERS`.
**Fix**: Add your User ID to `MATTERMOST_ALLOWED_USERS` in `~/.hermes/.env` and restart the gateway. Remember: the User ID is a 26-character alphanumeric string, not your `@username`.
## Per-Channel Prompts
Assign ephemeral system prompts to specific Mattermost channels. The prompt is injected at runtime on every turn — never persisted to transcript history — so changes take effect immediately.
```yaml
mattermost:
channel_prompts:
"channel_id_abc123": |
You are a research assistant. Focus on academic sources,
citations, and concise synthesis.
"channel_id_def456": |
Code review mode. Be precise about edge cases and
performance implications.
```
Keys are Mattermost channel IDs (find them in the channel URL or via the API). All messages in the matching channel get the prompt injected as an ephemeral system instruction.
## Security
:::warning
Always set `MATTERMOST_ALLOWED_USERS` to restrict who can interact with the bot. Without it, the gateway denies all users by default as a safety measure. Only add User IDs of people you trust — authorized users have full access to the agent's capabilities, including tool use and system access.
:::
For more information on securing your Hermes Agent deployment, see the [Security Guide](../security.md).
## Notes
- **Self-hosted friendly**: Works with any self-hosted Mattermost instance. No Mattermost Cloud account or subscription required.
- **No extra dependencies**: The adapter uses `aiohttp` for HTTP and WebSocket, which is already included with Hermes Agent.
- **Team Edition compatible**: Works with both Mattermost Team Edition (free) and Enterprise Edition.
website/docs/user-guide/messaging/msgraph-webhook.md
+142
View File
@@ -0,0 +1,142 @@
---
sidebar_position: 23
title: "Microsoft Graph Webhook Listener"
description: "Receive Microsoft Graph change notifications (meetings, calendar, chat, etc.) in Hermes"
---
# Microsoft Graph Webhook Listener
The `msgraph_webhook` gateway platform is an inbound event listener. It's how Hermes receives **change notifications** from Microsoft Graph — "a Teams meeting ended," "a new message landed in this chat," "this calendar event was updated." Different from the `teams` platform (which is a chat bot users type to) — this one is M365 telling Hermes something happened, not a person.
Right now the primary consumer is the Teams meeting summary pipeline: Graph notifies when a meeting produces a transcript, the pipeline fetches it, and Hermes posts a summary back into Teams. Other Graph resources (`/chats/.../messages`, `/users/.../events`) use the same listener — the pipeline consumers land with their own PRs.
## Prerequisites
- Microsoft Graph application credentials — [Register a Microsoft Graph Application](/guides/microsoft-graph-app-registration)
- A **public HTTPS URL** that Microsoft Graph can reach (Graph does not call private endpoints). A dev tunnel works for testing; production needs a real domain with a valid certificate.
- A strong shared secret to use as the `clientState` value. Generate with `openssl rand -hex 32` and put it in `~/.hermes/.env` as `MSGRAPH_WEBHOOK_CLIENT_STATE`.
## Quick Start
Minimum `~/.hermes/config.yaml`:
```yaml
platforms:
msgraph_webhook:
enabled: true
extra:
host: 127.0.0.1
port: 8646
client_state: "replace-with-a-strong-secret"
accepted_resources:
- "communications/onlineMeetings"
```
Or via env vars in `~/.hermes/.env` (auto-merged on startup):
```bash
MSGRAPH_WEBHOOK_ENABLED=true
MSGRAPH_WEBHOOK_PORT=8646
MSGRAPH_WEBHOOK_CLIENT_STATE=<generate-with-openssl-rand-hex-32>
MSGRAPH_WEBHOOK_ACCEPTED_RESOURCES=communications/onlineMeetings
```
Note: the bind host is read from `extra.host` in `config.yaml` (see the example above); there is no `MSGRAPH_WEBHOOK_HOST` env-var override.
Start the gateway: `hermes gateway run`. The listener exposes:
- `POST /msgraph/webhook` — change notifications from Graph
- `GET /msgraph/webhook?validationToken=...` — Graph subscription validation handshake
- `GET /health` — readiness probe with accepted/duplicate counters
Expose the listener publicly (reverse proxy, dev tunnel, ingress). Your notification URL for Graph subscriptions is your public HTTPS origin followed by `/msgraph/webhook`:
```
https://ops.example.com/msgraph/webhook
```
## Configuration
All settings go under `platforms.msgraph_webhook.extra`:
| Setting | Default | Description |
|---------|---------|-------------|
| `host` | unset (dual-stack: all interfaces, IPv4+IPv6) | Bind address for the HTTP listener. Non-loopback binds require `allowed_source_cidrs`; loopback (`127.0.0.1` / `::1`) is the easiest dev-tunnel / reverse-proxy setup. |
| `port` | `8646` | Bind port. |
| `webhook_path` | `/msgraph/webhook` | URL path Graph POSTs to. |
| `health_path` | `/health` | Readiness endpoint. |
| `client_state` | — | Shared secret Graph echoes in every notification. Compared with `hmac.compare_digest` — generate with `openssl rand -hex 32`. |
| `accepted_resources` | `[]` (accept all) | Allowlist of Graph resource paths/patterns. Trailing `*` acts as prefix match. Leading `/` is tolerated. Example: `["communications/onlineMeetings", "chats/*/messages"]`. |
| `max_seen_receipts` | `5000` | Dedupe cache size for notification IDs. Oldest entries evicted when the cap is hit. |
| `allowed_source_cidrs` | `[]` | Required for non-loopback binds. Leave empty only when the listener is bound to loopback and fronted by a local tunnel / reverse proxy. |
Most settings also have an equivalent env var (`MSGRAPH_WEBHOOK_*`) that merges into the config at gateway startup (the exception is `host`, which is config-only — see the note above) — see the [environment variables reference](/reference/environment-variables#microsoft-graph-teams-meetings).
## Security Hardening
### clientState is the primary auth check
Every Graph notification includes the `clientState` string your subscription registered with. The listener rejects any notification whose `clientState` doesn't match, using timing-safe comparison. This is Microsoft's documented mechanism — treat the value as a strong shared secret.
If `client_state` is unset, the listener refuses to start.
### Source-IP allowlisting (production deployments)
For production, restrict the listener to Microsoft's published Graph webhook source IP ranges. Microsoft documents the egress ranges under the [Office 365 IP Address and URL Web service](https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges). Configure them as:
```yaml
platforms:
msgraph_webhook:
enabled: true
extra:
host: 0.0.0.0
client_state: "..."
allowed_source_cidrs:
- "52.96.0.0/14"
- "52.104.0.0/14"
# ...add the current Microsoft 365 "Common" + "Teams" category egress ranges
```
Or as an env var:
```bash
MSGRAPH_WEBHOOK_ALLOWED_SOURCE_CIDRS="52.96.0.0/14,52.104.0.0/14"
```
Binding a non-loopback host such as `0.0.0.0`, `::`, or a LAN IP without `allowed_source_cidrs` is refused at startup. If you're using a dev tunnel or reverse proxy on the same machine, bind Hermes to `127.0.0.1` or `::1` and leave the allowlist empty there. Invalid CIDR strings log a warning and are ignored. **Review the Microsoft IP list quarterly** — it changes.
### HTTPS termination
The listener speaks plain HTTP. Terminate TLS at your reverse proxy (Caddy, Nginx, Cloudflare Tunnel, AWS ALB) and proxy to the listener over the local network. Graph refuses to deliver to non-HTTPS endpoints, so there's no path for unencrypted traffic to reach you from Graph itself.
### Response hygiene
On success the listener returns `202 Accepted` with an empty body — internal counters stay out of the wire response. Operators can observe counts via `/health`, which is guarded by the same source-IP rules as the webhook path.
Status code table:
| Outcome | Status |
|---------|--------|
| Notification(s) accepted or deduped | 202 |
| Validation handshake (GET with `validationToken`) | 200 (echoes the token) |
| Every item in batch failed clientState | 403 |
| Malformed JSON / missing `value` array / unknown resource | 400 |
| Source IP not in allowlist | 403 |
| Bare GET without `validationToken` | 400 |
## Troubleshooting
| Problem | What to check |
|---------|---------------|
| Graph subscription validation fails | Public URL is reachable, `/msgraph/webhook` path matches, GET with `validationToken` echoes the token verbatim as `text/plain` within 10 seconds. |
| Notifications POST but nothing ingests | `client_state` matches what you registered the subscription with. Re-run `openssl rand -hex 32` and create a new subscription if the value drifted. Check `accepted_resources` includes the resource path Graph is sending. |
| Every notification 403s | `clientState` mismatch (forged, or subscription registered with a different value). Re-create the subscription with `hermes teams-pipeline subscribe --client-state "$MSGRAPH_WEBHOOK_CLIENT_STATE" ...` (ships with the pipeline runtime PR). |
| Listener refuses to start on `0.0.0.0` | Set `allowed_source_cidrs` to Microsoft's current webhook egress ranges, or bind Hermes to `127.0.0.1` / `::1` behind your tunnel or reverse proxy. |
| Listener starts but `curl http://localhost:8646/health` hangs | Port binding collision. Check `ss -tlnp \| grep 8646` and change `port:` if needed. |
| Real Graph requests from Microsoft get 403'd | Source IP allowlist is too narrow. Widen the list to include the current Microsoft egress ranges. If you're still validating the tunnel path, bind Hermes to loopback and let the tunnel handle public exposure. |
## Related Docs
- [Register a Microsoft Graph Application](/guides/microsoft-graph-app-registration) — Azure app registration prereq
- [Environment Variables → Microsoft Graph](/reference/environment-variables#microsoft-graph-teams-meetings) — full env var list
- [Microsoft Teams bot setup](/user-guide/messaging/teams) — the different platform that lets users chat with Hermes in Teams
website/docs/user-guide/messaging/ntfy.md
+157
View File
@@ -0,0 +1,157 @@
# ntfy
[ntfy](https://ntfy.sh/) is a simple HTTP-based pub-sub notification service. It works with the free public server at `ntfy.sh` or any self-hosted instance, and supports any client that can make HTTP requests — phones, browsers, scripts, watches.
ntfy makes a great lightweight push channel for Hermes: subscribe to a topic from the [ntfy mobile app](https://ntfy.sh/docs/subscribe/phone/), send messages to the topic to talk to the agent, get the response back on your phone.
> Run `hermes gateway setup` and pick **ntfy** for a guided walk-through.
## Prerequisites
- A topic name (any unique string — `hermes-myname-2026` works fine)
- The [ntfy mobile app](https://ntfy.sh/docs/subscribe/phone/) installed and subscribed to that topic
- Optional: a self-hosted ntfy server, or an `ntfy.sh` account token for private/reserved topics
That's it. No SDK, no daemon, no Node.js. The adapter uses `httpx` which is already a Hermes dependency.
## Configure Hermes
### Via setup wizard
```bash
hermes gateway setup
```
Select **ntfy** and follow the prompts.
### Via environment variables
Add these to `~/.hermes/.env`:
```
NTFY_TOPIC=hermes-myname-2026
NTFY_ALLOWED_USERS=hermes-myname-2026
NTFY_HOME_CHANNEL=hermes-myname-2026
```
| Variable | Required | Description |
|---|---|---|
| `NTFY_TOPIC` | Yes | Topic to subscribe to (incoming messages) |
| `NTFY_SERVER_URL` | Optional | Server URL (default: `https://ntfy.sh`) — point to a self-hosted ntfy for privacy |
| `NTFY_TOKEN` | Optional | Bearer token (e.g. `tk_xyz`) or `user:pass` for Basic auth |
| `NTFY_PUBLISH_TOPIC` | Optional | Different topic for outgoing replies (defaults to `NTFY_TOPIC`) |
| `NTFY_MARKDOWN` | Optional | Set `true` to send replies with `X-Markdown: true` header |
| `NTFY_ALLOWED_USERS` | Recommended | Comma-separated topic names allowed (treated as user IDs; see below) |
| `NTFY_ALLOW_ALL_USERS` | Optional | Set `true` to allow every publisher — only safe for private topics with read tokens |
| `NTFY_HOME_CHANNEL` | Optional | Default topic for cron / notification delivery |
| `NTFY_HOME_CHANNEL_NAME` | Optional | Human label for the home channel |
## Identity model — read this before deploying
ntfy has no native authenticated user identity. The `title` field on a published message is **publisher-controlled** and can be anything the sender wants. The Hermes adapter does NOT use `title` for authorization — it would let any publisher who knows the topic spoof an allowed user.
Instead, **the topic name itself is the identity**. Every message published to the topic is treated as coming from the same logical user (the topic). `NTFY_ALLOWED_USERS` is therefore typically just the topic name itself — a single-entry allowlist that gates the whole channel.
This means **anyone who knows the topic can talk to the agent**. To make that a real trust boundary:
- **Self-host ntfy** and lock the topic down with [Access Control](https://docs.ntfy.sh/config/#access-control). Only authorized clients with the read/write token can publish.
- Or **use a private topic on ntfy.sh** ([reserved topics](https://docs.ntfy.sh/publish/#reserved-topics) require an account) and protect it with a `NTFY_TOKEN`.
- Or **pick a long, unguessable topic name** (`hermes-7d4f9c8b-2026`) and treat it as the shared secret. This is the lightest setup but the topic name leaks via any logs or screenshots.
In all cases, do not put sensitive data through ntfy unless the underlying topic is access-controlled.
## Quick start — talk to your agent from your phone
1. Pick a topic name: `hermes-myname-2026`
2. On your phone: install the [ntfy app](https://ntfy.sh/docs/subscribe/phone/), tap **+**, enter `hermes-myname-2026`
3. On the host:
```bash
echo 'NTFY_TOPIC=hermes-myname-2026' >> ~/.hermes/.env
echo 'NTFY_ALLOWED_USERS=hermes-myname-2026' >> ~/.hermes/.env
hermes gateway restart
```
4. From the ntfy app, send a message to the topic. The agent's reply lands as a push notification.
## Using ntfy with cron jobs
Once `NTFY_HOME_CHANNEL` is set, cron jobs can deliver to ntfy:
```python
cronjob(
action="create",
schedule="every 1h",
deliver="ntfy", # uses NTFY_HOME_CHANNEL
prompt="Check for alerts and summarise."
)
```
Or target a specific topic explicitly via the cron job's `deliver:` field, or from a shell script with the [`hermes send` CLI](/guides/pipe-script-output):
```bash
hermes send ntfy:alerts-channel "Done!"
```
This works even when the cron runs out-of-process from the gateway — the plugin registers a `standalone_sender_fn` that opens its own HTTP connection.
## Self-hosting ntfy
If you want full control:
```bash
# Docker
docker run -p 80:80 -it binwiederhier/ntfy serve
# Native
go install heckel.io/ntfy/v2@latest
ntfy serve
```
Then point Hermes at it:
```
NTFY_SERVER_URL=https://ntfy.mydomain.com
NTFY_TOPIC=hermes
NTFY_TOKEN=tk_abc123 # if you've set up access control
```
Self-hosting gives you topic access control, message persistence policies, attachments, and emoji tags. See the [ntfy server docs](https://docs.ntfy.sh/install/).
## Markdown formatting
ntfy clients render markdown when the publisher sets the `X-Markdown: true` header. To enable for outgoing Hermes replies:
```
NTFY_MARKDOWN=true
```
Or in `config.yaml`:
```yaml
platforms:
ntfy:
extra:
markdown: true
```
The mobile app supports a subset of CommonMark — bold, italic, lists, links, fenced code blocks. See [ntfy's markdown docs](https://docs.ntfy.sh/publish/#markdown-formatting) for the exact set.
## Outgoing-only setup (notifications without inbound)
If you only want Hermes to *push* notifications to ntfy (cron summaries, alerts) and never accept messages back, set both `NTFY_TOPIC` and `NTFY_PUBLISH_TOPIC` to the same value and skip `NTFY_ALLOWED_USERS` entirely. With no allowlist, the agent never responds to inbound messages — your phone gets the pushes, but the conversation is one-way.
## Limits
- **Message size**: ntfy caps message bodies at 4096 chars. Hermes truncates with a warning when this is exceeded.
- **No typing indicators**: the protocol doesn't expose one; `send_typing` is a no-op.
- **No threads or attachments**: ntfy is plain push notifications. Long replies stay in the message body, no thread fanout.
- **No native user identity**: see the identity-model section above.
## Troubleshooting
**Auth failure / 401** — `NTFY_TOKEN` is wrong, or the token doesn't have publish/subscribe rights on this topic. The adapter halts its reconnect loop on 401 and the gateway runtime status will show `fatal: ntfy_unauthorized`. Fix the token and restart the gateway.
**Topic not found / 404** — `NTFY_TOPIC` doesn't exist on the configured server. For ntfy.sh, topics are auto-created on first publish, so a 404 means you're pointed at a self-hosted server that doesn't have the topic provisioned. The adapter halts its reconnect loop with `fatal: ntfy_topic_not_found`.
**Connected but no messages** — Check that `NTFY_ALLOWED_USERS` includes the topic name itself. With ntfy's identity model, the topic IS the user; leaving the allowlist empty rejects everything.
**Reconnects every 60s** — The stream keepalive default is 55s; ntfy may have intermittent network issues. The adapter applies exponential backoff (2 → 5 → 10 → 30 → 60s) and resets to 0 once a stream stays alive ≥60s.
website/docs/user-guide/messaging/open-webui.md
+296
View File
@@ -0,0 +1,296 @@
---
sidebar_position: 8
title: "Open WebUI"
description: "Connect Open WebUI to Hermes Agent via the OpenAI-compatible API server"
---
# Open WebUI Integration
[Open WebUI](https://github.com/open-webui/open-webui) (126k★) is the most popular self-hosted chat interface for AI. With Hermes Agent's built-in API server, you can use Open WebUI as a polished web frontend for your agent — complete with conversation management, user accounts, and a modern chat interface.
## Architecture
```mermaid
flowchart LR
A["Open WebUI<br/>browser UI<br/>port 3000"]
B["hermes-agent<br/>gateway API server<br/>port 8642"]
A -->|POST /v1/chat/completions| B
B -->|SSE streaming response| A
```
Open WebUI connects to Hermes Agent's API server just like it would connect to OpenAI. Hermes handles the requests with its full toolset — terminal, file operations, web search, memory, skills — and returns the final response.
:::important Runtime location
The API server is a **Hermes agent runtime**, not a pure LLM proxy. For each request, Hermes creates a server-side `AIAgent` on the API-server host. Tool calls run where that API server is running.
For example, if a laptop points Open WebUI or another OpenAI-compatible client at a Hermes API server on a remote machine, `pwd`, file tools, browser tools, local MCP tools, and other workspace tools run on the remote API-server host, not on the laptop.
:::
Open WebUI talks to Hermes server-to-server, so you do not need `API_SERVER_CORS_ORIGINS` for this integration.
## Quick Setup
### 1. Enable the API server
```bash
hermes config set API_SERVER_ENABLED true
hermes config set API_SERVER_KEY your-secret-key
```
`hermes config set` auto-routes the flag to `config.yaml` and the secret to `~/.hermes/.env`. If the gateway is already running, restart it so the change takes effect:
```bash
hermes gateway stop && hermes gateway
```
### 2. Start Hermes Agent gateway
```bash
hermes gateway
```
You should see:
```
[API Server] API server listening on http://127.0.0.1:8642
```
### 3. Verify the API server is reachable
```bash
curl -s http://127.0.0.1:8642/health
# {"status": "ok", ...}
curl -s -H "Authorization: Bearer your-secret-key" http://127.0.0.1:8642/v1/models
# {"object":"list","data":[{"id":"hermes-agent", ...}]}
```
If `/health` fails, the gateway didn't pick up `API_SERVER_ENABLED=true` — restart it. If `/v1/models` returns `401`, your `Authorization` header doesn't match `API_SERVER_KEY`.
### 4. Start Open WebUI
```bash
docker run -d -p 3000:8080 \
-e OPENAI_API_BASE_URL=http://host.docker.internal:8642/v1 \
-e OPENAI_API_KEY=your-secret-key \
-e ENABLE_OLLAMA_API=false \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:main
```
`ENABLE_OLLAMA_API=false` suppresses the default Ollama backend, which would otherwise show up empty and clutter the model picker. Omit it if you actually have Ollama running alongside.
First launch takes 15–30 seconds: Open WebUI downloads sentence-transformer embedding models (~150MB) the first time it starts. Wait for `docker logs open-webui` to settle before opening the UI.
### 5. Open the UI
Go to **http://localhost:3000**. Create your admin account (the first user becomes admin). You should see your agent in the model dropdown (named after your profile, or **hermes-agent** for the default profile). Start chatting!
## Docker Compose Setup
For a more permanent setup, create a `docker-compose.yml`:
```yaml
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
ports:
- "3000:8080"
volumes:
- open-webui:/app/backend/data
environment:
- OPENAI_API_BASE_URL=http://host.docker.internal:8642/v1
- OPENAI_API_KEY=your-secret-key
- ENABLE_OLLAMA_API=false
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
volumes:
open-webui:
```
Then:
```bash
docker compose up -d
```
## Configuring via the Admin UI
If you prefer to configure the connection through the UI instead of environment variables:
1. Log in to Open WebUI at **http://localhost:3000**
2. Click your **profile avatar** → **Admin Settings**
3. Go to **Connections**
4. Under **OpenAI API**, click the **wrench icon** (Manage)
5. Click **+ Add New Connection**
6. Enter:
- **URL**: `http://host.docker.internal:8642/v1`
- **API Key**: the exact same value as `API_SERVER_KEY` in Hermes
7. Click the **checkmark** to verify the connection
8. **Save**
Your agent model should now appear in the model dropdown (named after your profile, or **hermes-agent** for the default profile).
:::warning
Environment variables only take effect on Open WebUI's **first launch**. After that, connection settings are stored in its internal database. To change them later, use the Admin UI or delete the Docker volume and start fresh.
:::
## API Type: Chat Completions vs Responses
Open WebUI supports two API modes when connecting to a backend:
| Mode | Format | When to use |
|------|--------|-------------|
| **Chat Completions** (default) | `/v1/chat/completions` | Recommended. Works out of the box. |
| **Responses** (experimental) | `/v1/responses` | For server-side conversation state via `previous_response_id`. |
### Using Chat Completions (recommended)
This is the default and requires no extra configuration. Open WebUI sends standard OpenAI-format requests and Hermes Agent responds accordingly. Each request includes the full conversation history.
### Using Responses API
To use the Responses API mode:
1. Go to **Admin Settings** → **Connections** → **OpenAI** → **Manage**
2. Edit your hermes-agent connection
3. Change **API Type** from "Chat Completions" to **"Responses (Experimental)"**
4. Save
With the Responses API, Open WebUI sends requests in the Responses format (`input` array + `instructions`), and Hermes Agent can preserve full tool call history across turns via `previous_response_id`. When `stream: true`, Hermes also streams spec-native `function_call` and `function_call_output` items, which enables custom structured tool-call UI in clients that render Responses events.
:::note
Open WebUI currently manages conversation history client-side even in Responses mode — it sends the full message history in each request rather than using `previous_response_id`. The main advantage of Responses mode today is the structured event stream: text deltas, `function_call`, and `function_call_output` items arrive as OpenAI Responses SSE events instead of Chat Completions chunks.
:::
## How It Works
When you send a message in Open WebUI:
1. Open WebUI sends a `POST /v1/chat/completions` request with your message and conversation history
2. Hermes Agent creates a server-side `AIAgent` instance using the API server's profile, model/provider config, memory, skills, and configured API-server toolsets
3. The agent processes your request — it may call tools (terminal, file operations, web search, etc.) on the API-server host
4. As tools execute, **inline progress messages stream to the UI** so you can see what the agent is doing (e.g. `` `💻 ls -la` ``, `` `🔍 Python 3.12 release` ``)
5. The agent's final text response streams back to Open WebUI
6. Open WebUI displays the response in its chat interface
Your agent has access to the same tools and capabilities as that API-server Hermes instance. If the API server is remote, those tools are remote too.
If you need tools to run against your **local** workspace today, run Hermes locally and point it at a pure LLM provider or pure OpenAI-compatible model proxy (for example vLLM, LiteLLM, Ollama, llama.cpp, OpenAI, OpenRouter, etc.). A future split-runtime mode for "remote brain, local hands" is being tracked in [#18715](https://github.com/NousResearch/hermes-agent/issues/18715); it is not the behavior of the current API server.
:::tip Tool Progress
With streaming enabled (the default), you'll see brief inline indicators as tools run — the tool emoji and its key argument. These appear in the response stream before the agent's final answer, giving you visibility into what's happening behind the scenes.
:::
## Configuration Reference
### Hermes Agent (API server)
| Variable | Default | Description |
|----------|---------|-------------|
| `API_SERVER_ENABLED` | `false` | Enable the API server |
| `API_SERVER_PORT` | `8642` | HTTP server port |
| `API_SERVER_HOST` | `127.0.0.1` | Bind address |
| `API_SERVER_KEY` | _(required)_ | Bearer token for auth. Match `OPENAI_API_KEY`. |
### Open WebUI
| Variable | Description |
|----------|-------------|
| `OPENAI_API_BASE_URL` | Hermes Agent's API URL (include `/v1`) |
| `OPENAI_API_KEY` | Must be non-empty. Match your `API_SERVER_KEY`. |
## Troubleshooting
### No models appear in the dropdown
- **Check the URL has `/v1` suffix**: `http://host.docker.internal:8642/v1` (not just `:8642`)
- **Verify the gateway is running**: `curl http://localhost:8642/health` should return `{"status": "ok"}`
- **Check model listing**: `curl -H "Authorization: Bearer your-secret-key" http://localhost:8642/v1/models` should return a list with `hermes-agent`
- **Docker networking**: From inside Docker, `localhost` means the container, not your host. Use `host.docker.internal` or `--network=host`.
- **Empty Ollama backend shadowing the picker**: If you omitted `ENABLE_OLLAMA_API=false`, Open WebUI shows an empty Ollama section above your Hermes models. Restart the container with `-e ENABLE_OLLAMA_API=false` or disable Ollama in **Admin Settings → Connections**.
### Connection test passes but no models load
This is almost always the missing `/v1` suffix. Open WebUI's connection test is a basic connectivity check — it doesn't verify model listing works.
### Response takes a long time
Hermes Agent may be executing multiple tool calls (reading files, running commands, searching the web) before producing its final response. This is normal for complex queries. The response appears all at once when the agent finishes.
### "Invalid API key" errors
Make sure your `OPENAI_API_KEY` in Open WebUI matches the `API_SERVER_KEY` in Hermes Agent.
:::warning
Open WebUI persists OpenAI-compatible connection settings in its own database after first launch. If you accidentally saved a wrong key in the Admin UI, fixing the environment variables alone is not enough — update or delete the saved connection in **Admin Settings → Connections**, or reset the Open WebUI data directory / database.
:::
## Multi-User Setup with Profiles
To run separate Hermes instances per user — each with their own config, memory, and skills — use [profiles](/user-guide/profiles). Each profile runs its own API server on a different port and automatically advertises the profile name as the model in Open WebUI.
### 1. Create profiles and configure API servers
`API_SERVER_*` are env vars, not YAML config keys, so write them to each profile's `.env`. Pick ports outside the default-platform range (`8644` is the webhook adapter, `8645` is wecom-callback, `8646` is msgraph-webhook), e.g. `8650+`:
```bash
hermes profile create alice
cat >> ~/.hermes/profiles/alice/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_PORT=8650
API_SERVER_KEY=alice-secret
EOF
hermes profile create bob
cat >> ~/.hermes/profiles/bob/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_PORT=8651
API_SERVER_KEY=bob-secret
EOF
```
### 2. Start each gateway
```bash
hermes -p alice gateway &
hermes -p bob gateway &
```
### 3. Add connections in Open WebUI
In **Admin Settings** → **Connections** → **OpenAI API** → **Manage**, add one connection per profile:
| Connection | URL | API Key |
|-----------|-----|---------|
| Alice | `http://host.docker.internal:8650/v1` | `alice-secret` |
| Bob | `http://host.docker.internal:8651/v1` | `bob-secret` |
The model dropdown will show `alice` and `bob` as distinct models. You can assign models to Open WebUI users via the admin panel, giving each user their own isolated Hermes agent.
:::tip Custom Model Names
The model name defaults to the profile name. To override it, set `API_SERVER_MODEL_NAME` in the profile's `.env`:
```bash
hermes -p alice config set API_SERVER_MODEL_NAME "Alice's Agent"
```
:::
## Linux Docker (no Docker Desktop)
On Linux without Docker Desktop, `host.docker.internal` doesn't resolve by default. Options:
```bash
# Option 1: Add host mapping
docker run --add-host=host.docker.internal:host-gateway ...
# Option 2: Use host networking
docker run --network=host -e OPENAI_API_BASE_URL=http://localhost:8642/v1 ...
# Option 3: Use Docker bridge IP
docker run -e OPENAI_API_BASE_URL=http://172.17.0.1:8642/v1 ...
```
website/docs/user-guide/messaging/photon.md
+255
View File
@@ -0,0 +1,255 @@
---
sidebar_position: 18
---
# Photon iMessage
Connect Hermes to **iMessage** through [Photon][photon], a managed
service that handles the Apple line allocation and abuse-prevention
layer so you don't have to run your own Mac relay.
The free tier uses Photon's shared iMessage line pool — different
recipients may see different sending numbers, but each conversation
stays stable. The paid Business tier gives every user the same
dedicated number; the plugin supports both, and the free tier is the
recommended starting point.
:::info Free to start
Photon's shared-line pool is free. No subscription is required to send
your first iMessage from Hermes — just a phone number we can bind to
your account.
:::
## Architecture
Photon is a **persistent-connection** channel, like Discord or Slack —
**no webhook, no public URL, no signing secret to manage.**
The `spectrum-ts` SDK holds a long-lived **gRPC stream** to Photon for
both directions. Because the SDK is TypeScript-only, Hermes runs it in a
small supervised **Node sidecar** and talks to it over loopback:
- **Inbound** — the sidecar consumes the SDK's `app.messages` gRPC
stream and forwards each message to the Python adapter over a loopback
`GET /inbound` (NDJSON). The adapter dedupes and dispatches it to the
agent, reconnecting automatically if the stream drops.
- **Outbound** — replies are loopback POSTs to the sidecar, which calls
`space.send(...)` on the SDK.
The Python plugin starts, supervises, and shuts down the sidecar
automatically.
## Prerequisites
- A Photon account — sign up at [app.photon.codes][app]
- **Node.js 18.17 or newer** on PATH (`node --version`)
- A phone number that can receive iMessage (used to bind your account)
That's it — there is no public URL or tunnel to set up.
## First-time setup
Either run the unified gateway wizard and pick **Photon iMessage**:
```bash
hermes gateway setup
```
…or run the Photon setup directly (the wizard calls the same flow):
```bash
# Device-code login + project + user + sidecar deps, all in one
hermes photon setup --phone +15551234567
```
The setup, in order:
1. **Device login** (`client_id=photon-cli`) — opens
`https://app.photon.codes/` for approval and stores the bearer token.
2. **Finds or creates** the `Hermes Agent` project on your account.
3. **Enables Spectrum**, reads the project's Spectrum id, and rotates
the project secret.
4. **Registers your phone number** as a Spectrum user — skipped if a
user with that number already exists, so re-running is safe.
5. **Prints your assigned iMessage line** — the number you text to reach
your agent.
6. **Runs `npm install`** inside the plugin's sidecar directory. On
read-only / immutable install trees (hosted Docker images, Podman,
Nix) the sidecar automatically falls back to a writable mirror under
`~/.hermes/photon/sidecar`; set `PHOTON_SIDECAR_DIR` to pin an
explicit location.
Runtime credentials are written to `~/.hermes/.env`
(`PHOTON_PROJECT_ID` = the Spectrum project id, `PHOTON_PROJECT_SECRET`),
the same place every other channel keeps its token. Management metadata
(device token, dashboard project id) lives in `~/.hermes/auth.json` under
`credential_pool.photon` / `credential_pool.photon_project`.
## Authorizing users
Photon uses the same authorization model as every other Hermes
channel. Choose one approach:
**DM pairing (default).** When an unknown number messages your Photon
line, Hermes replies with a pairing code. Approve it with:
```bash
hermes pairing approve photon <CODE>
```
Use `hermes pairing list` to see pending codes and approved users.
**Pre-authorize specific numbers** (in `~/.hermes/.env`):
```bash
PHOTON_ALLOWED_USERS=+15551234567,+15559876543
```
**Open access** (dev only, in `~/.hermes/.env`):
```bash
PHOTON_ALLOW_ALL_USERS=true
```
When `PHOTON_ALLOWED_USERS` is set, unknown senders are silently
ignored rather than offered a pairing code (the allowlist signals you
deliberately restricted access).
### Require mentions in group chats
By default Hermes responds to every authorized DM and group message.
To make group chats opt-in, enable mention gating (DMs still always
work):
```yaml
gateway:
platforms:
photon:
enabled: true
require_mention: true
```
With `require_mention: true`, group-chat messages are ignored unless
they match a wake-word pattern. The defaults match `Hermes` and
`@Hermes agent` variants. For a custom agent name, set regex patterns:
```yaml
gateway:
platforms:
photon:
require_mention: true
mention_patterns:
- '(?<![\w@])@?amos\b[,:\-]?'
```
Both keys also accept env vars (`PHOTON_REQUIRE_MENTION`,
`PHOTON_MENTION_PATTERNS`). This is the same mention-gating model the
BlueBubbles iMessage channel uses.
## Start the gateway
```bash
hermes gateway start
```
You'll see something like:
```
[photon] connected — sidecar on 127.0.0.1:8789, streaming inbound over gRPC
```
Send an iMessage to your assigned number and Hermes will reply.
## Status & troubleshooting
```bash
hermes photon status
```
Prints saved credentials, sidecar health, your registered number, and the
assigned iMessage line Hermes uses. When a Photon token and dashboard project
are available, `status` refreshes missing number rows from the dashboard
without provisioning new lines.
```
Photon iMessage status
──────────────────────
device token : ✓ stored
dashboard project : 3c90c3cc-0d44-4b50-...
spectrum project id : sp-...
project secret : ✓ stored
my number : +15551234567
assigned number : +16282679185
node binary : /usr/bin/node
sidecar deps : ✓ installed
```
Common issues:
- **`sidecar deps : ✗ run hermes photon install-sidecar`** — Node is
installed but `spectrum-ts` isn't. Run the suggested command.
- **`device token : ✗ missing`** — run `hermes photon setup` to log in.
- **`No iMessage line assigned yet`** — Spectrum is enabled but no line
has been provisioned; re-run `hermes photon setup` or check the
[dashboard][app].
- **Sidecar won't start** — confirm `node --version` is 18.17+ and that
`hermes photon install-sidecar` completed without errors.
## Limits today
- **Inbound attachments are metadata-only.** Inbound events carry the
filename + MIME type; the agent sees a marker but can't yet read the
bytes. The SDK exposes attachment bytes via `content.read()`, so this
is a sidecar follow-up.
- **Outbound attachments are supported.** Hermes sends images, voice
notes, video, and documents through spectrum-ts' `attachment()` /
`voice()` content builders via the sidecar's `/send-attachment`
endpoint. Captions arrive as a separate iMessage bubble after the
media.
- **Native polls are supported.** Hermes sends poll content through
spectrum-ts' `poll()` builder via the sidecar's `/send-poll` endpoint.
- **Read receipts are supported.** The sidecar marks an inbound iMessage
read after forwarding it to Hermes, so the sender sees `Read` without
waiting for a model/tool turn. Inbound receipts for Hermes-sent messages
are consumed as presence telemetry and never create an agent turn. Set
`PHOTON_READ_RECEIPTS=false` to keep messages at `Delivered`.
- **Message effects are supported.** Hermes sends text with native iMessage
bubble/screen effects through spectrum-ts' iMessage `effect()` builder
via the sidecar's `/send-effect` endpoint.
- **Photon's free quotas:** 5,000 messages per server per day,
50 new-conversation initiations per shared line per day. Increases
available — email `help@photon.codes`.
- **Cron and standalone sends need the gateway running.** Out-of-process
senders (cron jobs, `hermes send`, the dashboard) reuse the sidecar the
gateway spawned — they read its port/token from
`<hermes-home>/runtime/photon-sidecar.json`, written once the sidecar
passes its health check and removed when it stops. If a standalone send
reports the gateway appears to be down, start (or restart) the gateway
first.
- **Shared/free-tier lines can't initiate conversations with new
targets.** Photon-side policy: a shared line can only message a number
after that number has texted the line first. A cron/standalone send to a
brand-new recipient will be rejected by Photon even when Hermes is set
up correctly — either have the recipient message the line once, or move
to a dedicated line.
## Env vars
| Variable | Default | Notes |
|---------------------------|--------------------|--------------------------------------------|
| `PHOTON_PROJECT_ID` | from `.env` | Spectrum project id (the SDK's `projectId`); set by setup |
| `PHOTON_PROJECT_SECRET` | from `.env` | Project secret; set by setup |
| `PHOTON_SIDECAR_PORT` | `8789` | Loopback port for the sidecar control + inbound channel |
| `PHOTON_SIDECAR_AUTOSTART`| `true` | Whether the adapter spawns the sidecar |
| `PHOTON_NODE_BIN` | `which node` | Override the Node binary path |
| `PHOTON_HOME_CHANNEL` | (unset) | Default space id for cron / notifications |
| `PHOTON_HOME_CHANNEL_NAME`| (unset) | Human label for the home channel |
| `PHOTON_ALLOWED_USERS` | (unset) | Comma-separated E.164 allowlist |
| `PHOTON_ALLOW_ALL_USERS` | `false` | Dev only — accept any sender |
| `PHOTON_REQUIRE_MENTION` | `false` | Require a wake word before responding in groups |
| `PHOTON_MENTION_PATTERNS` | Hermes wake words | JSON list / comma / newline regex patterns for group mentions |
| `PHOTON_DASHBOARD_HOST` | `app.photon.codes` | Override the dashboard / device-login host |
| `PHOTON_SPECTRUM_HOST` | `spectrum.photon.codes` | Override the Spectrum API host |
[photon]: https://photon.codes/
[app]: https://app.photon.codes/
website/docs/user-guide/messaging/qqbot.md
+123
View File
@@ -0,0 +1,123 @@
# QQ Bot
Connect Hermes to QQ via the **Official QQ Bot API (v2)** — supporting private (C2C), group @-mentions, guild, and direct messages with voice transcription.
## Overview
The QQ Bot adapter uses the [Official QQ Bot API](https://bot.q.qq.com/wiki/develop/api-v2/) to:
- Receive messages via a persistent **WebSocket** connection to the QQ Gateway
- Send text and markdown replies via the **REST API**
- Download and process images, voice messages, and file attachments
- Transcribe voice messages using Tencent's built-in ASR or a configurable STT provider
## Prerequisites
1. **QQ Bot Application** — Register at [q.qq.com](https://q.qq.com):
- Create a new application and note your **App ID** and **App Secret**
- Enable the required intents: C2C messages, Group @-messages, Guild messages
- Configure your bot in sandbox mode for testing, or publish for production
2. **Dependencies** — The adapter requires `aiohttp` and `httpx`:
```bash
pip install aiohttp httpx
```
## Configuration
### Interactive setup
```bash
hermes gateway setup
```
Select **QQ Bot** from the platform list and follow the prompts.
### Manual configuration
Set the required environment variables in `~/.hermes/.env`:
```bash
QQ_APP_ID=your-app-id
QQ_CLIENT_SECRET=your-app-secret
```
## Environment Variables
| Variable | Description | Default |
|---|---|---|
| `QQ_APP_ID` | QQ Bot App ID (required) | — |
| `QQ_CLIENT_SECRET` | QQ Bot App Secret (required) | — |
| `QQBOT_HOME_CHANNEL` | OpenID for cron/notification delivery | — |
| `QQBOT_HOME_CHANNEL_NAME` | Display name for home channel | `Home` |
| `QQ_ALLOWED_USERS` | Comma-separated user OpenIDs for DM access | open (all users) |
| `QQ_GROUP_ALLOWED_USERS` | Comma-separated group OpenIDs for group access | — |
| `QQ_ALLOW_ALL_USERS` | Set to `true` to allow all DMs | `false` |
| `QQ_PORTAL_HOST` | Override the QQ portal host (set to `sandbox.q.qq.com` for sandbox routing) | `q.qq.com` |
| `QQ_STT_API_KEY` | API key for voice-to-text provider | — |
| `QQ_STT_BASE_URL` | (Not read directly — set `platforms.qqbot.extra.stt.baseUrl` in `config.yaml` instead) | n/a |
| `QQ_STT_MODEL` | STT model name | `glm-asr` |
## Advanced Configuration
For fine-grained control, add platform settings to `~/.hermes/config.yaml`:
```yaml
platforms:
qqbot:
enabled: true
extra:
app_id: "your-app-id"
client_secret: "your-secret"
markdown_support: true # enable QQ markdown (msg_type 2). Config-only; no env-var equivalent.
dm_policy: "open" # open | allowlist | disabled
allow_from:
- "user_openid_1"
group_policy: "open" # open | allowlist | disabled
group_allow_from:
- "group_openid_1"
stt:
provider: "zai" # zai (GLM-ASR), openai (Whisper), etc.
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4"
apiKey: "your-stt-key"
model: "glm-asr"
```
## Voice Messages (STT)
Voice transcription works in two stages:
1. **QQ built-in ASR** (free, always tried first) — QQ provides `asr_refer_text` in voice message attachments, which uses Tencent's own speech recognition
2. **Configured STT provider** (fallback) — If QQ's ASR doesn't return text, the adapter calls an OpenAI-compatible STT API:
- **Zhipu/GLM (zai)**: Default provider, uses `glm-asr` model
- **OpenAI Whisper**: Set `QQ_STT_BASE_URL` and `QQ_STT_MODEL`
- Any OpenAI-compatible STT endpoint
## Troubleshooting
### Bot disconnects immediately (quick disconnect)
This usually means:
- **Invalid App ID / Secret** — Double-check your credentials at q.qq.com
- **Missing permissions** — Ensure the bot has the required intents enabled
- **Sandbox-only bot** — If the bot is in sandbox mode, it can only receive messages from QQ's sandbox test channel
### Voice messages not transcribed
1. Check if QQ's built-in `asr_refer_text` is present in the attachment data
2. If using a custom STT provider, verify `QQ_STT_API_KEY` is set correctly
3. Check gateway logs for STT error messages
### Messages not delivered
- Verify the bot's **intents** are enabled at q.qq.com
- Check `QQ_ALLOWED_USERS` if DM access is restricted
- For group messages, ensure the bot is **@mentioned** (group policy may require allowlisting)
- Check `QQBOT_HOME_CHANNEL` for cron/notification delivery
### Connection errors
- Ensure `aiohttp` and `httpx` are installed: `pip install aiohttp httpx`
- Check network connectivity to `api.sgroup.qq.com` and the WebSocket gateway
- Review gateway logs for detailed error messages and reconnect behavior
website/docs/user-guide/messaging/raft.md
+70
View File
@@ -0,0 +1,70 @@
---
sidebar_position: 19
title: "Raft"
description: "Connect Hermes Agent to Raft as an external agent via wake-channel bridge"
---
# Raft Setup
Hermes connects to [Raft](https://raft.build) as an external agent through a local wake-channel bridge. The adapter starts a loopback HTTP endpoint that receives content-free wake hints from the bridge, then injects them into the Hermes gateway session pipeline. The agent reads and sends messages through the Raft CLI — the adapter never touches message bodies or delivery cursors.
:::info Division of Labor
- **The bridge** owns: wake-hint consumption, dedup, backoff, reconnection, at-least-once delivery, and proof logging.
- **The Hermes adapter** owns: a localhost wake endpoint and injecting a short notice into the agent's context.
- **The agent** owns: pulling messages (`raft message check`), replying (`raft message send`), and all other Raft interactions via the CLI.
The adapter holds no Raft credentials — only a per-session shared token for localhost auth between the bridge and the endpoint.
:::
---
## Prerequisites
- A **Raft workspace** where you can create an External Agent
- The **Raft CLI** installed and logged in to that External Agent profile
- **aiohttp** — Python package (included in Hermes `[all]` extras)
In Raft, open the Agents menu, create an External Agent, and follow the setup card to install the Raft CLI and log in the agent profile. Once the agent is created, Raft shows a Hermes setup guide with the environment variables and configuration needed to start the gateway.
---
## Setup
Add to `~/.hermes/.env`:
```bash
RAFT_PROFILE=your-agent-profile
```
That's it — the adapter auto-enables when `RAFT_PROFILE` is set. It generates a per-session bridge token, picks an ephemeral port, and spawns the bridge child process automatically when the gateway starts.
---
## How It Works
```
Raft Server → Bridge (wake-hints SSE) → POST /wake → Hermes Adapter → Agent context
Agent → raft message check → Raft Server (message bodies)
Agent → raft message send → Raft Server (replies)
```
1. The Raft server sends wake hints to the bridge process via SSE.
2. The bridge forwards each hint as a `POST /wake` to the adapter's loopback endpoint.
3. The adapter validates the bridge token, verifies the payload is content-free, and injects a wake notice into the Hermes session.
4. The agent sees the wake notice and uses the Raft CLI to read messages and reply.
Wake payloads are **content-free by contract** — they carry metadata (event ID, message ID, timestamps) but never message bodies, channel names, or sender identities. The adapter rejects any payload containing content-shaped fields (`text`, `body`, `content`, `messages`, etc.).
---
## Bridge
The adapter automatically spawns `raft agent bridge` as a child process, passing the endpoint URL and token. The bridge connects to the Raft server using the configured profile and begins forwarding wake hints. It is terminated when the gateway shuts down.
---
## Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `RAFT_PROFILE` | Raft agent profile slug — auto-enables the adapter when set | _(required)_ |
website/docs/user-guide/messaging/relay.md
+191
View File
@@ -0,0 +1,191 @@
---
sidebar_position: 30
title: "Hermes Relay"
description: "Connect Hermes to messaging platforms through a relay connector that owns the platform credentials — enrollment, capabilities, config, and troubleshooting"
---
# Hermes Relay (Connector)
:::warning Experimental
Relay is **experimental**. The wire contract, auth scheme, and configuration
may change without a deprecation cycle while the system is being validated.
:::
Hermes Relay is not a chat platform itself — it is a **connector system** that
lets your gateway front one or more real messaging platforms (Discord,
Telegram, Slack, WhatsApp, …) **without holding any platform credentials**. A
separate service, the *connector*, owns the platform bot tokens and sockets.
Your gateway dials **out** to the connector over a single authenticated
WebSocket, receives a capability descriptor at handshake, and then exchanges
normalized message events (inbound) and actions (outbound) over that socket.
Key properties:
- **Outbound-only networking.** The gateway never opens an inbound port.
Inbound messages ride back down the same WebSocket the gateway dialed, so
relay works behind NAT and on hosts with no public IP.
- **No platform secrets on the gateway.** Bot tokens live on the connector.
Auth-gated platform media URLs are re-hosted connector-side, so platform
credentials never cross the wire.
- **Platform-agnostic.** The gateway learns what the fronted platform can do
(message length limits, markdown dialect, edit/thread/streaming support, and
the exact set of supported operations) from the handshake descriptor, not
from hardcoded platform logic.
The formal gateway ⇄ connector interface lives in the repository at
`docs/relay-connector-contract.md`.
## When to use Relay
Relay is for deployments where a hosted or shared connector service manages
the platform side — for example multi-tenant hosting where one shared bot
fronts many users' agents, or setups where you don't want bot tokens on the
gateway machine. If you run your own bots directly, use the native platform
adapters ([Telegram](/user-guide/messaging/telegram),
[Discord](/user-guide/messaging/discord), etc.) instead.
## Enrollment
A self-hosted gateway authenticates to the connector with a per-gateway
secret. `hermes gateway enroll` redeems a **single-use enrollment token**
(minted by the connector when your tenant's route is provisioned and delivered
with your gateway config) for that secret:
```bash
hermes gateway enroll \
--token <enrollment-token> \
--connector-url wss://connector.example.com/relay
```
What it does:
1. Resolves a fresh Nous Portal access token from your existing login
(`~/.hermes/auth.json`) — this proves which Nous org (tenant) you own. If
`gateway.idp.token_url` is configured, your own IdP is used instead (the
air-gapped / self-hosted-IdP path, no Nous Portal involved): with
`client_id`/`client_secret` configured it performs a generic OAuth2
client-credentials grant; with neither configured the URL is treated as an
ambient token endpoint (plain GET whose response body is the token — the
metadata-server pattern, e.g. Domino's `$DOMINO_API_PROXY/access-token`).
Configuring only one of the two credentials is an error.
2. POSTs the enrollment token and a gateway id to the connector's
`/relay/enroll` endpoint over TLS.
3. The connector verifies the token (signature, single-use, tenant match),
mints a per-gateway secret plus a per-tenant delivery key, and returns them
once.
4. Persists the credentials into `~/.hermes/.env`:
`GATEWAY_RELAY_ID`, `GATEWAY_RELAY_SECRET`, `GATEWAY_RELAY_DELIVERY_KEY`
(plus `GATEWAY_RELAY_URL` / `GATEWAY_RELAY_WAKE_URL` when supplied).
Restart the gateway afterwards to pick up the new environment.
Flags:
| Flag | Description |
|------|-------------|
| `--token` | The single-use enrollment token. Also settable via `GATEWAY_RELAY_ENROLL_TOKEN`. |
| `--connector-url` | Connector base or relay URL (`wss://…/relay` or `https://…`). Also settable via `GATEWAY_RELAY_URL` or `gateway.relay_url` in `config.yaml`. |
| `--gateway-id` | Stable id for this gateway instance (used for kill-switch granularity). Defaults to `gw-<hostname>`. |
| `--wake-url` | Optional reachable URL the connector pokes (payload-free GET) to wake this gateway when buffered work arrives while it is idle. Persisted as `GATEWAY_RELAY_WAKE_URL`. Without it the gateway still drains buffered messages whenever it next reconnects. |
:::note Managed installs
`hermes gateway enroll` refuses to run in managed/hosted installs — there the
hosting platform provisions the relay secret directly into the container
environment.
:::
## Configuration
Relay activates when a connector relay URL is configured — there is no
separate feature flag. Deployments that don't set it are unaffected.
| Setting | Where | Meaning |
|---------|-------|---------|
| `GATEWAY_RELAY_URL` | env (`~/.hermes/.env`) | Connector relay WebSocket URL. Presence enables the relay platform. |
| `gateway.relay_url` | `config.yaml` | Same as above, config-file form (env takes precedence). |
| `GATEWAY_RELAY_ID` | env | This gateway instance's id (written by `enroll`). |
| `GATEWAY_RELAY_SECRET` | env | Per-gateway secret authenticating the WebSocket upgrade (written by `enroll`). |
| `GATEWAY_RELAY_DELIVERY_KEY` | env | Per-tenant delivery key (written by `enroll`; retained for forward-compat). |
| `GATEWAY_RELAY_WAKE_URL` / `gateway.relay_wake_url` | env / `config.yaml` | Optional wake-poke target for idle/suspended gateways. |
| `GATEWAY_RELAY_PLATFORMS` | env | Comma-separated list of platforms this gateway fronts over one connection (e.g. `discord,telegram`). Usually stamped by the deployment/orchestrator. |
| `GATEWAY_RELAY_BOT_IDS` | env | JSON map of per-platform bot identities, e.g. `{"discord": {"botId": "…"}}`. Paired with `GATEWAY_RELAY_PLATFORMS`. |
| `gateway.idp.token_url` | `config.yaml` | When set, enrollment/provisioning authenticates against your own IdP instead of Nous Portal: OAuth2 client-credentials when `gateway.idp.client_id`/`client_secret` are also set; otherwise an ambient token endpoint (plain GET returning the token, raw or `{"access_token": …}`). |
## Supported capabilities
What actually works over a relay connection is negotiated at handshake: the
connector advertises a `supported_ops` list, and the gateway only uses an
operation the connector explicitly advertises (older connectors fall back to a
legacy `send`/`edit`/`typing`/`follow_up` set). Per-platform capability flags
(edit-based streaming, threads, draft streaming, markdown dialect, message
length limit) also come from the handshake descriptor. Subject to that
negotiation, the relay supports:
- **Text messages and streaming** — sends, replies, and progressive
edit-based streaming when the fronted platform supports message editing;
otherwise output degrades to one message per segment.
- **Media, both directions** — outbound images, voice, audio, video, and
documents are uploaded to the connector (or referenced by public URL) and
delivered through each platform's native upload lane, with captions.
Inbound attachments are localized to files for the agent; auth-gated
platform URLs are re-hosted connector-side so platform credentials never
reach the gateway. Media re-hosts are capped at 25 MB and expire (~1 hour).
- **Native interactive prompts** — exec approvals, confirmations, and clarify
questions render with **native platform controls** (Discord buttons,
Telegram inline keyboards, Slack Block Kit actions, WhatsApp button/list
messages) instead of numbered-text fallbacks. Button presses come back as
authenticated prompt responses from the actual clicking user, so the
gateway's authorization gates apply exactly as to a typed reply. Prompt
expiry is enforced gateway-side.
- **Reaction ack lifecycle** — the bot's processing-status reactions
(👀 while working, ✅/❌ on completion) work over the relay. Reactions are
best-effort: a failed reaction never fails a turn.
- **Thread lifecycle** — creating handoff threads and renaming threads
(including LLM-titled semantic renames) through platform-abstract
`thread_create` / `thread_rename` operations, with a no-clobber guard so a
human's manual rename wins. Availability depends on the platform (e.g.
Slack threads can't be renamed; WhatsApp has no threads).
- **Typing indicators** — the gateway egresses typing (and stop-typing) through
the connector while processing.
- **Chat metadata** — `get_chat_info` lookups are proxied to the connector
when advertised.
- **Buffered delivery and wake** — when the gateway goes idle or disconnects,
the connector buffers inbound messages durably and replays them in order on
reconnect (ack-gated, no loss or duplication). If a wake URL is registered,
the connector pokes it when buffered work arrives for a sleeping gateway.
Multi-platform fronting is supported: one gateway can front several platforms
(e.g. Discord *and* Telegram) over a single relay connection, with each
outbound message tagged for the platform it targets.
## Troubleshooting
**Enrollment fails with 401** — the connector could not verify your identity
token. Re-login with `hermes auth add nous` (or `hermes setup`) and retry.
**Enrollment fails with 403** — the enrollment token is invalid, expired,
already used, or belongs to a different tenant. Enrollment tokens are
single-use; request a fresh one from whoever provisioned your tenant route.
**"Could not reach the connector"** — check the connector URL. You can paste
either the `wss://…/relay` dial URL or the `https://…` base URL; the CLI maps
between them automatically.
**`enroll` refuses to run** — you are in a managed/hosted install, where the
relay secret is provisioned by the hosting platform. Self-enrollment is only
for self-hosted gateways.
**Relay platform shows as disabled after it previously worked** — a WebSocket
close with code 4401 *after* a successful handshake means the gateway's secret
was revoked (e.g. the instance was deprovisioned). The gateway deliberately
stops reconnecting and reports relay as disabled rather than retrying. A 4401
*before* any successful handshake is treated as a transient
not-yet-provisioned race and retried normally.
**Nothing changed after enrolling** — the gateway reads `GATEWAY_RELAY_*` at
startup. Restart it (`hermes gateway restart`).
**A feature (buttons, media, threads…) silently degrades to plain text** — the
connector for your platform did not advertise that operation in its handshake
`supported_ops`. The gateway intentionally falls back to the text behavior
rather than sending an op the connector can't handle.
website/docs/user-guide/messaging/signal.md
+262
View File
@@ -0,0 +1,262 @@
---
sidebar_position: 6
title: "Signal"
description: "Set up Hermes Agent as a Signal messenger bot via signal-cli daemon"
---
# Signal Setup
Hermes connects to Signal through the [signal-cli](https://github.com/AsamK/signal-cli) daemon running in HTTP mode. The adapter streams messages in real-time via SSE (Server-Sent Events) and sends responses via JSON-RPC.
Signal is the most privacy-focused mainstream messenger — end-to-end encrypted by default, open-source protocol, minimal metadata collection. This makes it ideal for security-sensitive agent workflows.
:::info No New Python Dependencies
The Signal adapter uses `httpx` (already a core Hermes dependency) for all communication. No additional Python packages are required. You just need signal-cli installed externally.
:::
---
## Prerequisites
- **signal-cli** — Java-based Signal client ([GitHub](https://github.com/AsamK/signal-cli))
- **Java 17+** runtime — required by signal-cli
- **A phone number** with Signal installed (for linking as a secondary device)
### Installing signal-cli
```bash
# macOS
brew install signal-cli
# Linux (download latest release)
VERSION=$(curl -Ls -o /dev/null -w %{url_effective} \
https://github.com/AsamK/signal-cli/releases/latest | sed 's/^.*\/v//')
curl -L -O "https://github.com/AsamK/signal-cli/releases/download/v${VERSION}/signal-cli-${VERSION}.tar.gz"
sudo tar xf "signal-cli-${VERSION}.tar.gz" -C /opt
sudo ln -sf "/opt/signal-cli-${VERSION}/bin/signal-cli" /usr/local/bin/
```
:::caution
signal-cli is **not** in apt or snap repositories. The Linux install above downloads directly from [GitHub releases](https://github.com/AsamK/signal-cli/releases).
:::
---
## Step 1: Link Your Signal Account
Signal-cli works as a **linked device** — like WhatsApp Web, but for Signal. Your phone stays the primary device.
```bash
# Generate a linking URI (displays a QR code or link)
signal-cli link -n "HermesAgent"
```
1. Open **Signal** on your phone
2. Go to **Settings → Linked Devices**
3. Tap **Link New Device**
4. Scan the QR code or enter the URI
---
## Step 2: Start the signal-cli Daemon
```bash
# Replace +1234567890 with your Signal phone number (E.164 format)
signal-cli --account +1234567890 daemon --http 127.0.0.1:8080
```
:::tip
Keep this running in the background. You can use `systemd`, `tmux`, `screen`, or run it as a service.
:::
Verify it's running:
```bash
curl http://127.0.0.1:8080/api/v1/check
# Should return: {"versions":{"signal-cli":...}}
```
---
## Step 3: Configure Hermes
The easiest way:
```bash
hermes gateway setup
```
Select **Signal** from the platform menu. The wizard will:
1. Check if signal-cli is installed
2. Prompt for the HTTP URL (default: `http://127.0.0.1:8080`)
3. Test connectivity to the daemon
4. Ask for your account phone number
5. Configure allowed users and access policies
### Manual Configuration
Add to `~/.hermes/.env`:
```bash
# Required
SIGNAL_HTTP_URL=http://127.0.0.1:8080
SIGNAL_ACCOUNT=+1234567890
# Security (recommended)
SIGNAL_ALLOWED_USERS=+1234567890,+0987654321 # Comma-separated E.164 numbers or UUIDs
# Optional
SIGNAL_GROUP_ALLOWED_USERS=groupId1,groupId2 # Enable groups (omit to disable, * for all)
SIGNAL_HOME_CHANNEL=+1234567890 # Default delivery target for cron jobs
```
Then start the gateway:
```bash
hermes gateway # Foreground
hermes gateway install # Install as a user service
sudo hermes gateway install --system # Linux only: boot-time system service
```
---
## Access Control
### DM Access
DM access follows the same pattern as all other Hermes platforms:
1. **`SIGNAL_ALLOWED_USERS` set** → only those users can message
2. **No allowlist set** → unknown users get a DM pairing code (approve via `hermes pairing approve signal CODE`)
3. **`SIGNAL_ALLOW_ALL_USERS=true`** → anyone can message (use with caution)
### Group Access
Group access is controlled by the `SIGNAL_GROUP_ALLOWED_USERS` env var:
| Configuration | Behavior |
|---------------|----------|
| Not set (default) | All group messages are ignored. The bot only responds to DMs. |
| Set with group IDs | Only listed groups are monitored (e.g., `groupId1,groupId2`). |
| Set to `*` | The bot responds in any group it's a member of. |
---
## Features
### Attachments
The adapter supports sending and receiving media in both directions.
**Incoming** (user → agent):
- **Images** — PNG, JPEG, GIF, WebP (auto-detected via magic bytes)
- **Audio** — MP3, OGG, WAV, M4A (voice messages transcribed if Whisper is configured)
- **Documents** — PDF, ZIP, and other file types
**Outgoing** (agent → user):
The agent can send media files via `MEDIA:` tags in responses. The following delivery methods are supported:
- **Images** — `send_multiple_images` and `send_image_file` send PNG, JPEG, GIF, WebP as native Signal attachments
- **Voice** — `send_voice` sends audio files (OGG, MP3, WAV, M4A, AAC) as attachments
- **Video** — `send_video` sends MP4 video files
- **Documents** — `send_document` sends any file type (PDF, ZIP, etc.)
All outgoing media goes through Signal's standard attachment API. Unlike some platforms, Signal does not distinguish between voice messages and file attachments at the protocol level.
Attachment size limit: **100 MB** (both directions).
:::warning
**Signal servers will rate-limit attachment uploads**, the adapter uses a scheduler for multiple image sending that batches images in groups of 32 and throttles uploads to match the Signal server policy.
:::
### Native Formatting, Reply Quotes, and Reactions
Signal messages render with **native formatting** instead of literal markdown characters. The adapter converts markdown (`**bold**`, `*italic*`, `` `code` ``, `~~strike~~`, `||spoiler||`, headings) into Signal `bodyRanges` so the text shows up with real styling on the recipient's client rather than as visible `**` / `` ` `` characters.
**Reply quotes.** When Hermes replies to a specific message, it now posts a native reply that quotes the original — same UI affordance Signal users see when they use "Reply" themselves. This is automatic for replies generated in response to an inbound message.
**Reactions.** The agent can react to messages via the standard reaction API; reactions surface in Signal as emoji reactions on the referenced message rather than as extra text.
None of this requires additional config — it ships on by default in recent signal-cli builds. If your `signal-cli` version is too old, Hermes falls back to plaintext delivery and logs a one-time warning.
### Long Messages
Signal caps a single message at **8,000 characters**. Hermes splits longer responses into numbered chunks (`(1/3)`, `(2/3)`, …) automatically instead of truncating them. This applies to every delivery path — live conversation replies, cron job deliveries, `hermes send`, and MCP `send_message` calls — and native formatting (bold, italic, code, spoilers) is preserved across chunk boundaries.
### Typing Indicators
The bot sends typing indicators while processing messages, refreshing every 8 seconds.
### Tool Progress Display
Signal does not support editing already-sent messages. Hermes therefore suppresses gateway tool-progress bubbles on Signal, even when `/verbose` is enabled and saves a non-`off` mode for the platform.
You can still see tool activity in the CLI, and final Signal replies can include normal assistant output. If you need live per-tool progress in chat, use a messaging platform with message editing support.
### Phone Number Redaction
All phone numbers are automatically redacted in logs:
- `+15551234567` → `+155****4567`
- This applies to both Hermes gateway logs and the global redaction system
### Note to Self (Single-Number Setup)
If you run signal-cli as a **linked secondary device** on your own phone number (rather than a separate bot number), you can interact with Hermes through Signal's "Note to Self" feature.
Just send a message to yourself from your phone — signal-cli picks it up and Hermes responds in the same conversation.
**How it works:**
- "Note to Self" messages arrive as `syncMessage.sentMessage` envelopes
- The adapter detects when these are addressed to the bot's own account and processes them as regular inbound messages
- Echo-back protection (sent-timestamp tracking) prevents infinite loops — the bot's own replies are filtered out automatically
**No extra configuration needed.** This works automatically as long as `SIGNAL_ACCOUNT` matches your phone number.
### Health Monitoring
The adapter monitors the SSE connection and automatically reconnects if:
- The connection drops (with exponential backoff: 2s → 60s)
- No activity is detected for 120 seconds (pings signal-cli to verify)
---
## Troubleshooting
| Problem | Solution |
|---------|----------|
| **"Cannot reach signal-cli"** during setup | Ensure signal-cli daemon is running: `signal-cli --account +YOUR_NUMBER daemon --http 127.0.0.1:8080` |
| **Messages not received** | Check that `SIGNAL_ALLOWED_USERS` includes the sender's number in E.164 format (with `+` prefix) |
| **"signal-cli not found on PATH"** | Install signal-cli and ensure it's in your PATH, or use Docker |
| **Connection keeps dropping** | Check signal-cli logs for errors. Ensure Java 17+ is installed. |
| **Group messages ignored** | Configure `SIGNAL_GROUP_ALLOWED_USERS` with specific group IDs, or `*` to allow all groups. |
| **Bot responds to no one** | Configure `SIGNAL_ALLOWED_USERS`, use DM pairing, or explicitly allow all users through gateway policy if you want broader access. |
| **Duplicate messages** | Ensure only one signal-cli instance is listening on your phone number |
---
## Security
:::warning
**Always configure access controls.** The bot has terminal access by default. Without `SIGNAL_ALLOWED_USERS` or DM pairing, the gateway denies all incoming messages as a safety measure.
:::
- Phone numbers are redacted in all log output
- Use DM pairing or explicit allowlists for safe onboarding of new users
- Keep groups disabled unless you specifically need group support, or allowlist only the groups you trust
- Signal's end-to-end encryption protects message content in transit
- The signal-cli session data in `~/.local/share/signal-cli/` contains account credentials — protect it like a password
---
## Environment Variables Reference
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `SIGNAL_HTTP_URL` | Yes | — | signal-cli HTTP endpoint |
| `SIGNAL_ACCOUNT` | Yes | — | Bot phone number (E.164) |
| `SIGNAL_ALLOWED_USERS` | No | — | Comma-separated phone numbers/UUIDs |
| `SIGNAL_GROUP_ALLOWED_USERS` | No | — | Group IDs to monitor, or `*` for all (omit to disable groups) |
| `SIGNAL_ALLOW_ALL_USERS` | No | `false` | Allow any user to interact (skip allowlist) |
| `SIGNAL_HOME_CHANNEL` | No | — | Default delivery target for cron jobs |
website/docs/user-guide/messaging/simplex.md
+151
View File
@@ -0,0 +1,151 @@
# SimpleX Chat
[SimpleX Chat](https://simplex.chat/) is a private, decentralised messaging platform where users own their contacts and groups. Unlike other platforms, SimpleX assigns no persistent user IDs — every contact is identified by an opaque internal ID generated at connection time, which makes it one of the most private messengers available.
> Run `hermes gateway setup` and pick **SimpleX** for a guided walk-through.
## Prerequisites
- The **simplex-chat** CLI installed and running as a daemon
- Python package **websockets** (`pip install websockets`)
## Install simplex-chat
Download the latest release from the [simplex-chat GitHub releases](https://github.com/simplex-chat/simplex-chat/releases) page:
```bash
# Linux / macOS binary
curl -L https://github.com/simplex-chat/simplex-chat/releases/latest/download/simplex-chat-ubuntu-22_04-x86_64 -o simplex-chat
chmod +x simplex-chat
```
The SimpleX Chat project does not publish a prebuilt Docker image for the chat client; to run it under Docker, build from source from the [simplex-chat repository](https://github.com/simplex-chat/simplex-chat).
## Start the daemon
```bash
simplex-chat -p 5225
```
The daemon listens on WebSocket at `ws://127.0.0.1:5225` by default.
## Configure Hermes
### Via setup wizard
```bash
hermes gateway setup
```
Select **SimpleX Chat** and follow the prompts.
### Via environment variables
Add these to `~/.hermes/.env`:
```
SIMPLEX_WS_URL=ws://127.0.0.1:5225
SIMPLEX_ALLOWED_USERS=<contact-id-1>,<contact-id-2>
SIMPLEX_HOME_CHANNEL=<contact-id>
```
| Variable | Required | Description |
|---|---|---|
| `SIMPLEX_WS_URL` | Yes | WebSocket URL of the simplex-chat daemon |
| `SIMPLEX_ALLOWED_USERS` | Recommended | Comma-separated allowlist. Each entry can be a numeric `contactId` **or** a display name — both forms work. |
| `SIMPLEX_ALLOW_ALL_USERS` | Optional | Set `true` to allow every contact (use carefully) |
| `SIMPLEX_AUTO_ACCEPT` | Optional | Auto-accept incoming contact requests (default: `true`) |
| `SIMPLEX_GROUP_ALLOWED` | Optional | Comma-separated group IDs the bot participates in, or `*` for any group. Omit to ignore group messages entirely |
| `SIMPLEX_HOME_CHANNEL` | Optional | Default contact/group ID for cron job delivery |
| `SIMPLEX_HOME_CHANNEL_NAME` | Optional | Human label for the home channel |
| `HERMES_SIMPLEX_TEXT_BATCH_DELAY` | Optional | Quiet-period seconds (default: `0.8`) used to concatenate rapid-fire inbound text messages into one event |
## Find your contact ID or display name
After starting the daemon, open a conversation with your agent contact. The numeric `contactId` appears in session logs. If you'd rather use the display name shown in the SimpleX UI, that works too — `SIMPLEX_ALLOWED_USERS` accepts either form.
## Authorization
By default **all contacts are denied**. You must either:
1. Set `SIMPLEX_ALLOWED_USERS` to a comma-separated list of `contactId`s and/or display names (e.g. `SIMPLEX_ALLOWED_USERS=4,alice` matches either contactId 4 or the contact whose display name is "alice"), or
2. Use **DM pairing** — send any message to the bot and it will reply with a pairing code. Enter that code via `hermes pairing approve simplex <CODE>`.
## Group chats
By default the adapter ignores group messages — a bot in a group otherwise
processes every member's traffic. Opt-in explicitly:
```
SIMPLEX_GROUP_ALLOWED=12,34 # specific group IDs
# or
SIMPLEX_GROUP_ALLOWED=* # any group the bot is in
```
Address groups by prefixing the chat ID with `group:`, e.g.
`simplex:group:12` as a cron `deliver=` target or in a `hermes send` call.
## Sending with `hermes send`
SimpleX works as a standalone send target — the daemon must be running,
but a live gateway is not required for plain text:
```bash
hermes send --to simplex:alice "hello" # DM by contact display name
hermes send --to simplex:group:12 "hello" # group by numeric ID
hermes send --to simplex "hello" # SIMPLEX_HOME_CHANNEL
```
While the gateway is running, the adapter enumerates your contacts and
allowed groups into the channel directory (refreshed every 5 minutes), so
`hermes send --list` shows them by name. Before the first gateway run the
platform still appears in `--list` with a "no channels discovered yet"
hint — direct targets like the ones above work regardless.
## Attachments
The adapter supports native SimpleX attachments in both directions:
- **Inbound** — incoming images, voice notes, and files are accepted via
the daemon's XFTP flow (`rcvFileDescrReady` → `/freceive` → wait for
`rcvFileComplete`) and surfaced as `MessageEvent.media_urls` with the
appropriate `MessageType` (`PHOTO`, `VOICE`, `TEXT` + document).
- **Outbound** — `send_image_file`, `send_voice`, `send_document`, and
`send_video` all use the structured `/_send` form with `filePath`, so
the receiving SimpleX client renders images inline and plays voice
notes inline rather than offering them as downloads.
Agent replies can also embed `MEDIA:/path/to/file` tags in plain text —
the adapter strips the tag from the body and sends the file as either a
voice note (audio extensions) or a document.
## Using SimpleX with cron jobs
```python
cronjob(
action="create",
schedule="every 1h",
deliver="simplex", # uses SIMPLEX_HOME_CHANNEL
prompt="Check for alerts and summarise."
)
```
Or target a specific contact via the cron job's `deliver:` field, or from a shell script with the [`hermes send` CLI](/guides/pipe-script-output):
```bash
hermes send simplex:<contact-id> "Done!"
```
## Privacy notes
- SimpleX never reveals phone numbers or email addresses — contacts use opaque IDs
- The connection between Hermes and the daemon is local WebSocket (`ws://127.0.0.1:5225`) — no data leaves your machine
- Messages are end-to-end encrypted by the SimpleX protocol before reaching the daemon
## Troubleshooting
**"Cannot reach daemon"** — Ensure `simplex-chat -p 5225` is running and the port matches `SIMPLEX_WS_URL`.
**"websockets not installed"** — Run `pip install websockets`.
**Messages not received** — Check that the contact's ID is in `SIMPLEX_ALLOWED_USERS` or approve them via DM pairing.
website/docs/user-guide/messaging/slack.md
+1073
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/messaging/sms.md
+203
View File
@@ -0,0 +1,203 @@
---
sidebar_position: 8
sidebar_label: "SMS (Twilio)"
title: "SMS (Twilio)"
description: "Set up Hermes Agent as an SMS chatbot via Twilio"
---
# SMS Setup (Twilio)
Hermes connects to SMS through the [Twilio](https://www.twilio.com/) API. People text your Twilio phone number and get AI responses back — same conversational experience as Telegram or Discord, but over standard text messages.
:::info Shared Credentials
The SMS gateway shares credentials with the optional [telephony skill](/reference/skills-catalog). If you've already set up Twilio for voice calls or one-off SMS, the gateway works with the same `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN`, and `TWILIO_PHONE_NUMBER`.
:::
---
## Prerequisites
- **Twilio account** — [Sign up at twilio.com](https://www.twilio.com/try-twilio) (free trial available)
- **A Twilio phone number** with SMS capability
- **A publicly accessible server** — Twilio sends webhooks to your server when SMS arrives
- **aiohttp** — `cd ~/.hermes/hermes-agent && uv pip install -e ".[sms]"`
---
## Step 1: Get Your Twilio Credentials
1. Go to the [Twilio Console](https://console.twilio.com/)
2. Copy your **Account SID** and **Auth Token** from the dashboard
3. Go to **Phone Numbers → Manage → Active Numbers** — note your phone number in E.164 format (e.g., `+15551234567`)
---
## Step 2: Configure Hermes
### Interactive setup (recommended)
```bash
hermes gateway setup
```
Select **SMS (Twilio)** from the platform list. The wizard will prompt for your credentials.
### Manual setup
Add to `~/.hermes/.env`:
```bash
TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_AUTH_TOKEN=your_auth_token_here
TWILIO_PHONE_NUMBER=+15551234567
# Security: restrict to specific phone numbers (recommended)
SMS_ALLOWED_USERS=+15559876543,+15551112222
# Optional: set a home channel for cron job delivery
SMS_HOME_CHANNEL=+15559876543
```
---
## Step 3: Configure Twilio Webhook
Twilio needs to know where to send incoming messages. In the [Twilio Console](https://console.twilio.com/):
1. Go to **Phone Numbers → Manage → Active Numbers**
2. Click your phone number
3. Under **Messaging → A MESSAGE COMES IN**, set:
- **Webhook**: `https://your-server:8080/webhooks/twilio`
- **HTTP Method**: `POST`
:::tip Exposing Your Webhook
If you're running Hermes locally, use a tunnel to expose the webhook:
```bash
# Using cloudflared
cloudflared tunnel --url http://localhost:8080
# Using ngrok
ngrok http 8080
```
Set the resulting public URL as your Twilio webhook.
:::
**Set `SMS_WEBHOOK_URL` to the same URL you configured in Twilio.** This is required for Twilio signature validation — the adapter will refuse to start without it:
```bash
# Must match the webhook URL in your Twilio Console
SMS_WEBHOOK_URL=https://your-server:8080/webhooks/twilio
```
The webhook port defaults to `8080`. Override with:
```bash
SMS_WEBHOOK_PORT=3000
```
---
## Step 4: Start the Gateway
```bash
hermes gateway
```
You should see:
```
[sms] Twilio webhook server listening on 127.0.0.1:8080, from: +1555***4567
```
If you see `Refusing to start: SMS_WEBHOOK_URL is required`, set `SMS_WEBHOOK_URL` to the public URL configured in your Twilio Console (see Step 3).
Text your Twilio number — Hermes will respond via SMS.
---
## Environment Variables
| Variable | Required | Description |
|----------|----------|-------------|
| `TWILIO_ACCOUNT_SID` | Yes | Twilio Account SID (starts with `AC`) |
| `TWILIO_AUTH_TOKEN` | Yes | Twilio Auth Token (also used for webhook signature validation) |
| `TWILIO_PHONE_NUMBER` | Yes | Your Twilio phone number (E.164 format) |
| `SMS_WEBHOOK_URL` | Yes | Public URL for Twilio signature validation — must match the webhook URL in your Twilio Console |
| `SMS_WEBHOOK_PORT` | No | Webhook listener port (default: `8080`) |
| `SMS_WEBHOOK_HOST` | No | Webhook bind address (default: `127.0.0.1`) |
| `SMS_INSECURE_NO_SIGNATURE` | No | Set to `true` to disable signature validation (local dev only — **not for production**) |
| `SMS_ALLOWED_USERS` | No | Comma-separated E.164 phone numbers allowed to chat |
| `SMS_ALLOW_ALL_USERS` | No | Set to `true` to allow anyone (not recommended) |
| `SMS_HOME_CHANNEL` | No | Phone number for cron job / notification delivery |
| `SMS_HOME_CHANNEL_NAME` | No | Display name for the home channel (default: `Home`) |
---
## SMS-Specific Behavior
- **Plain text only** — Markdown is automatically stripped since SMS renders it as literal characters
- **1600 character limit** — Longer responses are split across multiple messages at natural boundaries (newlines, then spaces)
- **Echo prevention** — Messages from your own Twilio number are ignored to prevent loops
- **Phone number redaction** — Phone numbers are redacted in logs for privacy
---
## Security
### Webhook signature validation
Hermes validates that inbound webhooks genuinely originate from Twilio by verifying the `X-Twilio-Signature` header (HMAC-SHA1). This prevents attackers from injecting forged messages.
**`SMS_WEBHOOK_URL` is required.** Set it to the public URL configured in your Twilio Console. The adapter will refuse to start without it.
For local development without a public URL, you can disable validation:
```bash
# Local dev only — NOT for production
SMS_INSECURE_NO_SIGNATURE=true
```
### User allowlists
**The gateway denies all users by default.** Configure an allowlist:
```bash
# Recommended: restrict to specific phone numbers
SMS_ALLOWED_USERS=+15559876543,+15551112222
# Or allow all (NOT recommended for bots with terminal access)
SMS_ALLOW_ALL_USERS=true
```
:::warning
SMS has no built-in encryption. Don't use SMS for sensitive operations unless you understand the security implications. For sensitive use cases, prefer Signal or Telegram.
:::
---
## Troubleshooting
### Messages not arriving
1. Check your Twilio webhook URL is correct and publicly accessible
2. Verify `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN` are correct
3. Check the Twilio Console → **Monitor → Logs → Messaging** for delivery errors
4. Ensure your phone number is in `SMS_ALLOWED_USERS` (or `SMS_ALLOW_ALL_USERS=true`)
### Replies not sending
1. Check `TWILIO_PHONE_NUMBER` is set correctly (E.164 format with `+`)
2. Verify your Twilio account has SMS-capable numbers
3. Check Hermes gateway logs for Twilio API errors
### Webhook port conflicts
If port 8080 is already in use, change it:
```bash
SMS_WEBHOOK_PORT=3001
```
Update the webhook URL in Twilio Console to match.
website/docs/user-guide/messaging/teams-meetings.md
+242
View File
@@ -0,0 +1,242 @@
---
sidebar_position: 6
title: "Teams Meetings"
description: "Set up the Microsoft Teams meeting summary pipeline with Microsoft Graph webhooks"
---
# Microsoft Teams Meetings
Use the Teams meeting pipeline when you want Hermes to ingest Microsoft Graph meeting events, fetch transcripts first, fall back to recordings plus STT when needed, and deliver a structured summary to downstream sinks.
Prerequisites: see [Microsoft Teams](./teams.md) for the underlying bot/credential setup.
> Run `hermes gateway setup` and pick **Teams Meetings** for a guided walk-through.
This page focuses on setup and enablement:
- Graph credentials
- webhook listener configuration
- Teams delivery modes
- pipeline config shape
For day-2 operations, go-live checks, and the operator worksheet, use the dedicated guide: [Operate the Teams Meeting Pipeline](/guides/operate-teams-meeting-pipeline).
## What This Feature Does
The pipeline:
1. receives Microsoft Graph webhook events
2. resolves the meeting and prefers transcript artifacts first
3. falls back to recording download plus STT when no usable transcript is available
4. stores durable job state and sink records locally
5. can write summaries to Notion, Linear, and Microsoft Teams
Operator actions stay in the CLI (the `teams-pipeline` subcommand is registered by the `teams_pipeline` plugin — enable it via `hermes plugins enable teams_pipeline` or set `plugins.enabled: [teams_pipeline]` in `config.yaml`):
```bash
hermes teams-pipeline validate
hermes teams-pipeline list
hermes teams-pipeline maintain-subscriptions
```
## Prerequisites
Before enabling the meetings pipeline, make sure you have:
- a working Hermes install
- the existing [Microsoft Teams bot setup](/user-guide/messaging/teams) if you want Teams outbound delivery
- Microsoft Graph application credentials with the permissions required for the meeting resources you plan to subscribe to
- a public HTTPS URL that Microsoft Graph can call for webhook delivery
- `ffmpeg` installed if you want recording-plus-STT fallback
## Step 1: Add Microsoft Graph Credentials
Add Graph app-only credentials to `~/.hermes/.env`:
```bash
MSGRAPH_TENANT_ID=<tenant-id>
MSGRAPH_CLIENT_ID=<client-id>
MSGRAPH_CLIENT_SECRET=<client-secret>
```
These credentials are used by:
- the Graph client foundation
- subscription maintenance commands
- meeting resolution and artifact fetches
- Graph-based Teams outbound delivery when you do not provide a dedicated Teams access token
## Step 2: Enable the Graph Webhook Listener
The webhook listener is a gateway platform named `msgraph_webhook`. At minimum, enable it and set a client state value:
```bash
MSGRAPH_WEBHOOK_ENABLED=true
MSGRAPH_WEBHOOK_PORT=8646
MSGRAPH_WEBHOOK_CLIENT_STATE=<random-shared-secret>
MSGRAPH_WEBHOOK_ACCEPTED_RESOURCES=communications/onlineMeetings
```
The bind host is read from the platform's `extra.host` in `config.yaml` (there is no `MSGRAPH_WEBHOOK_HOST` env var — see the [webhook listener reference](msgraph-webhook.md)).
The listener exposes:
- `/msgraph/webhook` for Graph notifications
- `/health` for a simple health check
You need to route your public HTTPS endpoint to that listener. For example, if your public domain is `https://ops.example.com`, your Graph notification URL would typically be:
```text
https://ops.example.com/msgraph/webhook
```
## Step 3: Configure Teams Delivery and Pipeline Behavior
The meeting pipeline reads its runtime config from the existing `teams` platform entry. Pipeline-specific knobs live under `teams.extra.meeting_pipeline`. Teams outbound delivery stays on the normal Teams platform config surface.
Example `~/.hermes/config.yaml`:
```yaml
platforms:
msgraph_webhook:
enabled: true
extra:
host: 127.0.0.1
port: 8646
client_state: "replace-me"
accepted_resources:
- "communications/onlineMeetings"
teams:
enabled: true
extra:
client_id: "your-teams-client-id"
client_secret: "your-teams-client-secret"
tenant_id: "your-teams-tenant-id"
# outbound summary delivery
delivery_mode: "graph" # or incoming_webhook
team_id: "team-id"
channel_id: "channel-id"
# incoming_webhook_url: "https://..."
meeting_pipeline:
transcript_min_chars: 80
transcript_required: false
transcription_fallback: true
ffmpeg_extract_audio: true
notion:
enabled: false
linear:
enabled: false
```
If you bind the listener to a non-loopback host such as `0.0.0.0`, you must also set `allowed_source_cidrs` to Microsoft's webhook egress ranges. Loopback binds (`127.0.0.1` / `::1`) are the intended dev-tunnel and local reverse-proxy setup.
## Teams Delivery Modes
The pipeline supports two Teams summary-delivery modes inside the existing Teams plugin.
### `incoming_webhook`
Use this when you want a simple webhook post into Teams without channel-message creation through Graph.
Required config:
```yaml
platforms:
teams:
enabled: true
extra:
delivery_mode: "incoming_webhook"
incoming_webhook_url: "https://..."
```
### `graph`
Use this when you want Hermes to post the summary through Microsoft Graph into a Teams chat or channel.
Supported targets:
- `chat_id`
- `team_id` + `channel_id`
- `team_id` + `home_channel` fallback for the existing Teams platform
Example:
```yaml
platforms:
teams:
enabled: true
extra:
delivery_mode: "graph"
team_id: "team-id"
channel_id: "channel-id"
```
## Step 4: Start the Gateway
Start Hermes normally after updating config:
```bash
hermes gateway run
```
Or, if you run Hermes in Docker, start the gateway the same way you already do for your deployment.
Check the listener:
```bash
curl http://localhost:8646/health
```
## Step 5: Create Graph Subscriptions
Use the plugin CLI to create and inspect subscriptions.
Examples:
```bash
hermes teams-pipeline subscribe \
--resource communications/onlineMeetings/getAllTranscripts \
--notification-url https://ops.example.com/msgraph/webhook \
--client-state "$MSGRAPH_WEBHOOK_CLIENT_STATE"
hermes teams-pipeline subscribe \
--resource communications/onlineMeetings/getAllRecordings \
--notification-url https://ops.example.com/msgraph/webhook \
--client-state "$MSGRAPH_WEBHOOK_CLIENT_STATE"
```
:::warning Graph subscriptions expire in 72 hours
Microsoft Graph caps webhook subscriptions at 72 hours and will not auto-renew them. You MUST schedule `hermes teams-pipeline maintain-subscriptions` before going live, or notifications will silently stop three days after any manual subscription creation. See [Automating subscription renewal](/guides/operate-teams-meeting-pipeline#automating-subscription-renewal-required-for-production) in the operator runbook — three options (Hermes cron, systemd timer, plain crontab).
:::
For subscription maintenance and day-2 operator flows, continue with the guide: [Operate the Teams Meeting Pipeline](/guides/operate-teams-meeting-pipeline).
## Validation
Run the built-in validation snapshot:
```bash
hermes teams-pipeline validate
```
Useful companion checks:
```bash
hermes teams-pipeline token-health
hermes teams-pipeline subscriptions
```
## Troubleshooting
| Problem | What to check |
|---------|---------------|
| Graph webhook validation fails | Confirm the public URL is correct and reachable, and that Graph is calling the exact `/msgraph/webhook` path |
| Jobs do not appear in `hermes teams-pipeline list` | Confirm `msgraph_webhook` is enabled and that subscriptions point at the right notification URL |
| Transcript-first never succeeds | Check Graph permissions for transcript resources and whether the transcript artifact exists for that meeting |
| Recording fallback fails | Confirm `ffmpeg` is installed and the Graph app can access recording artifacts |
| Teams summary delivery fails | Re-check `delivery_mode`, target IDs, and Teams auth config |
## Related Docs
- [Microsoft Teams bot setup](/user-guide/messaging/teams)
- [Operate the Teams Meeting Pipeline](/guides/operate-teams-meeting-pipeline)
website/docs/user-guide/messaging/teams.md
+288
View File
@@ -0,0 +1,288 @@
---
sidebar_position: 5
title: "Microsoft Teams"
description: "Set up Hermes Agent as a Microsoft Teams bot"
---
# Microsoft Teams Setup
Connect Hermes Agent to Microsoft Teams as a bot. Unlike Slack's Socket Mode, Teams delivers messages by calling a **public HTTPS webhook**, so your instance needs a publicly reachable endpoint — either a dev tunnel (local dev) or a real domain (production).
Need meeting summaries from Microsoft Graph events rather than normal bot conversations? Use the dedicated setup page: [Teams Meetings](/user-guide/messaging/teams-meetings).
> Run `hermes gateway setup` and pick **Microsoft Teams** for a guided walk-through.
## How the Bot Responds
| Context | Behavior |
|---------|----------|
| **Personal chat (DM)** | Bot responds to every message. No @mention needed. |
| **Group chat** | Bot only responds when @mentioned. |
| **Channel** | Bot only responds when @mentioned. |
Teams delivers @mentions as regular messages with `<at>BotName</at>` tags, which Hermes strips automatically before processing.
---
For source or local installs, include the Teams extra so the bundled adapter can
import the Microsoft Teams SDK:
```bash
uv sync --extra teams
# or, for editable installs:
uv pip install -e ".[teams]"
```
## Step 1: Install the Teams CLI
The `@microsoft/teams.cli` automates bot registration — no Azure portal needed.
```bash
npm install -g @microsoft/teams.cli@preview
teams login
```
To verify your login and find your own AAD object ID (needed for `TEAMS_ALLOWED_USERS`):
```bash
teams status --verbose
```
---
## Step 2: Expose the Webhook Port
Teams cannot deliver messages to `localhost`. For local development, use any tunnel tool to get a public HTTPS URL. The default port is `3978` — change it with `TEAMS_PORT` if needed.
```bash
# devtunnel (Microsoft)
devtunnel create hermes-bot --allow-anonymous
devtunnel port create hermes-bot -p 3978 --protocol http # replace 3978 with TEAMS_PORT if changed
devtunnel host hermes-bot
# ngrok
ngrok http 3978 # replace 3978 with TEAMS_PORT if changed
# cloudflared
cloudflared tunnel --url http://localhost:3978 # replace 3978 with TEAMS_PORT if changed
```
Copy the `https://` URL from the output — you'll use it in the next step. Leave the tunnel running while developing.
The public tunnel URL uses HTTPS, but Hermes' local webhook listener uses plain HTTP. The tunnel terminates TLS and forwards HTTP to port `3978`; do not configure the local tunnel port as HTTPS.
For production, point your bot's endpoint at your server's public domain instead (see [Production Deployment](#production-deployment)).
---
## Step 3: Create the Bot
```bash
teams app create \
--name "Hermes" \
--endpoint "https://<your-tunnel-url>/api/messages"
```
The CLI outputs your `CLIENT_ID`, `CLIENT_SECRET`, and `TENANT_ID`, plus an install link for Step 6. Save the client secret — it won't be shown again.
---
## Step 4: Configure Environment Variables
Add to `~/.hermes/.env`:
```bash
# Required
TEAMS_CLIENT_ID=<your-client-id>
TEAMS_CLIENT_SECRET=<your-client-secret>
TEAMS_TENANT_ID=<your-tenant-id>
# Restrict access to specific users (recommended)
# Use AAD object IDs from `teams status --verbose`
TEAMS_ALLOWED_USERS=<your-aad-object-id>
```
---
## Step 5: Start the Gateway
**Docker** (must run from the directory that contains `docker-compose.yml` — usually your cloned `hermes-agent` repo, not `~`):
```bash
cd /path/to/hermes-agent
HERMES_UID=$(id -u) HERMES_GID=$(id -g) docker compose up -d gateway
```
**Native / systemd install** (typical `hermes` one-liner installer under `~/.hermes/hermes-agent`):
```bash
hermes gateway restart
# or foreground: hermes gateway run
```
The Teams SDK is optional; when Teams is enabled, the gateway lazy-installs it into Hermes' own venv on first start (do **not** use system `pip install` on Ubuntu 24.04 — that hits PEP 668 `externally-managed-environment`). To install manually into the Hermes venv:
```bash
~/.hermes/hermes-agent/venv/bin/pip install microsoft-teams-apps aiohttp
# or from a clone of the agent: uv sync --extra teams
```
The default webhook port is `3978` (override with `TEAMS_PORT`). Check that it's running:
```bash
curl http://localhost:3978/health # should return: ok
# Docker:
docker logs -f hermes
# Native:
hermes gateway status -l
```
Look for:
```
[teams] Webhook server listening on * (all interfaces, IPv4+IPv6):3978/api/messages
```
---
## Step 6: Install the App in Teams
```bash
teams app get <teamsAppId> --install-link
```
Open the printed link in your browser — it opens directly in the Teams client. After installing, send a direct message to your bot — it's ready.
---
## Configuration Reference
### Environment Variables
| Variable | Description |
|----------|-------------|
| `TEAMS_CLIENT_ID` | Azure AD App (client) ID |
| `TEAMS_CLIENT_SECRET` | Azure AD client secret |
| `TEAMS_TENANT_ID` | Azure AD tenant ID |
| `TEAMS_ALLOWED_USERS` | Comma-separated AAD object IDs allowed to use the bot |
| `TEAMS_ALLOW_ALL_USERS` | Set `true` to skip the allowlist and allow anyone |
| `TEAMS_HOME_CHANNEL` | Conversation ID for cron/proactive message delivery |
| `TEAMS_HOME_CHANNEL_NAME` | Display name for the home channel |
| `TEAMS_PORT` | Webhook port (default: `3978`) |
### config.yaml
Alternatively, configure via `~/.hermes/config.yaml`:
```yaml
platforms:
teams:
enabled: true
extra:
client_id: "your-client-id"
client_secret: "your-secret"
tenant_id: "your-tenant-id"
port: 3978
```
---
## Features
### Interactive Approval Cards
When the agent needs to run a potentially dangerous command, it sends an Adaptive Card with four buttons instead of asking you to type `/approve`:
- **Allow Once** — approve this specific command
- **Allow Session** — approve this pattern for the rest of the session
- **Always Allow** — permanently approve this pattern
- **Deny** — reject the command
Clicking a button resolves the approval inline and replaces the card with the decision.
### Meeting Summary Delivery (Teams Meeting Pipeline)
When the [Teams meeting pipeline plugin](/user-guide/messaging/msgraph-webhook) is enabled, this adapter also handles outbound delivery of meeting summaries — one Teams integration surface, not two. After a meeting's transcript is summarized, the writer posts the summary into your chosen Teams target.
Pipeline summary delivery is configured under the `teams` platform entry alongside the bot config:
```yaml
platforms:
teams:
enabled: true
extra:
# existing bot config (client_id, client_secret, tenant_id, port) ...
# Meeting summary delivery (only used when the teams_pipeline plugin is enabled)
delivery_mode: "graph" # or "incoming_webhook"
# For delivery_mode: graph — pick ONE of:
chat_id: "19:meeting_..." # post into a Teams chat
# team_id: "..." # OR post into a channel
# channel_id: "..."
# access_token: "..." # optional; falls back to MSGRAPH_* app credentials
# For delivery_mode: incoming_webhook:
# incoming_webhook_url: "https://outlook.office.com/webhook/..."
```
| Mode | Use when | Trade-off |
|------|----------|-----------|
| `incoming_webhook` | Simple "post a summary into this channel" with a static Teams-generated URL. | No reply threading, no reactions, shows as the webhook's configured identity. |
| `graph` | Threaded channel posts or 1:1/group chat posts under the bot's identity via Microsoft Graph. | Requires the [Graph app registration](/guides/microsoft-graph-app-registration) with `ChannelMessage.Send` (channel) or `Chat.ReadWrite.All` (chat) application permissions. |
If the `teams_pipeline` plugin is **not** enabled, these settings are inert — they only wire up when the pipeline runtime binds to the Graph webhook ingress.
---
## Production Deployment
For a permanent server, terminate TLS at a reverse proxy and forward requests to the plain HTTP Hermes listener, normally `http://127.0.0.1:3978`. Register the proxy's public HTTPS endpoint with Teams:
```bash
teams app create \
--name "Hermes" \
--endpoint "https://your-domain.com/api/messages"
```
If you've already created the bot and just need to update the endpoint:
```bash
teams app update --id <teamsAppId> --endpoint "https://your-domain.com/api/messages"
```
Make sure the public HTTPS endpoint is reachable from the internet and uses a valid TLS certificate. Teams rejects self-signed certificates. Keep the Hermes listener behind the proxy; port `3978` does not serve HTTPS itself.
---
## Troubleshooting
| Problem | Solution |
|---------|----------|
| `Can't find a suitable configuration file` from `docker compose` | You are not in the repo that has `docker-compose.yml`, or you are on a native install — use `hermes gateway restart` instead, or `cd` into the clone first |
| `requirements not met` / `Teams SDK missing` / `No adapter available for teams` | Restart gateway so lazy-install can run, or install into the **Hermes venv**: `~/.hermes/hermes-agent/venv/bin/pip install microsoft-teams-apps aiohttp`. System `pip` fails on Ubuntu 24.04 (PEP 668) and would not affect the service anyway |
| `health` endpoint works but bot doesn't respond | Check that your tunnel is still running and the bot's messaging endpoint matches the tunnel URL |
| Logs show `"UNKNOWN / HTTP/1.0" 400` when Teams sends a message | The tunnel or reverse proxy is forwarding HTTPS to Hermes' plain HTTP listener. Terminate TLS at the proxy and forward HTTP to port `3978` |
| `KeyError: 'teams'` in logs | Restart the container — this is fixed in the current version |
| Bot responds with auth errors | Verify `TEAMS_CLIENT_ID`, `TEAMS_CLIENT_SECRET`, and `TEAMS_TENANT_ID` are all set correctly |
| `No inference provider configured` | Check that `ANTHROPIC_API_KEY` (or another provider key) is set in `~/.hermes/.env` |
| Bot receives messages but ignores them | Your AAD object ID may not be in `TEAMS_ALLOWED_USERS`. Run `teams status --verbose` to find it |
| Tunnel URL changes on restart | devtunnel URLs are persistent if you use a named tunnel (`devtunnel create hermes-bot`). ngrok and cloudflared generate a new URL each run unless you have a paid plan — update the bot endpoint with `teams app update` when it changes |
| Teams shows "This bot is not responding" | The webhook returned an error. Check `docker logs hermes` / `hermes gateway status -l` for tracebacks |
| `[teams] Failed to connect` in logs | The SDK failed to authenticate. Double-check your credentials and that the tenant ID matches the account you used in `teams login` |
---
## Security
:::warning
**Always set `TEAMS_ALLOWED_USERS`** with the AAD object IDs of authorized users. Without this, anyone who can find or install your bot can interact with it.
Treat `TEAMS_CLIENT_SECRET` like a password — rotate it periodically via the Azure portal or Teams CLI.
:::
- Store credentials in `~/.hermes/.env` with permissions `600` (`chmod 600 ~/.hermes/.env`)
- The bot only accepts messages from users in `TEAMS_ALLOWED_USERS`; unauthorized messages are silently dropped
- Your public endpoint (`/api/messages`) is authenticated by the Teams Bot Framework — requests without valid JWTs are rejected
## Related Docs
- [Teams Meetings](/user-guide/messaging/teams-meetings)
- [Operate the Teams Meeting Pipeline](/guides/operate-teams-meeting-pipeline)
website/docs/user-guide/messaging/telegram.md
+1347
View File
File diff suppressed because it is too large. Load diff
website/docs/user-guide/messaging/webhooks.md
+608
View File
@@ -0,0 +1,608 @@
---
sidebar_position: 13
title: "Webhooks"
description: "Receive events from GitHub, GitLab, and other services to trigger Hermes agent runs"
---
# Webhooks
Receive events from external services (GitHub, GitLab, JIRA, Stripe, etc.) and trigger Hermes agent runs automatically. The webhook adapter runs an HTTP server that accepts POST requests, validates HMAC signatures, transforms payloads into agent prompts, and routes responses back to the source or to another configured platform.
The agent processes the event and can respond by posting comments on PRs, sending messages to Telegram/Discord, or logging the result.
## Video Tutorial
<div style={{position: 'relative', width: '100%', aspectRatio: '16 / 9', marginBottom: '1.5rem'}}>
<iframe
src="https://www.youtube.com/embed/WNYe5mD4fY8"
title="Hermes Agent — Webhooks Tutorial"
style={{position: 'absolute', top: 0, left: 0, width: '100%', height: '100%', border: 0}}
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
allowFullScreen
/>
</div>
---
## Quick Start
1. Enable via `hermes gateway setup` or environment variables
2. Define routes in `config.yaml` **or** create them dynamically with `hermes webhook subscribe`
3. Point your service at `http://your-server:8644/webhooks/<route-name>`
---
## Setup
There are two ways to enable the webhook adapter.
### Via setup wizard
```bash
hermes gateway setup
```
Follow the prompts to enable webhooks, set the port, and set a global HMAC secret.
### Via environment variables
Add to `~/.hermes/.env`:
```bash
WEBHOOK_ENABLED=true
WEBHOOK_PORT=8644 # default
WEBHOOK_SECRET=your-global-secret
```
### Verify the server
Once the gateway is running:
```bash
curl http://localhost:8644/health
```
Expected response:
```json
{"status": "ok", "platform": "webhook"}
```
---
## Configuring Routes {#configuring-routes}
Routes define how different webhook sources are handled. Each route is a named entry under `platforms.webhook.extra.routes` in your `config.yaml`.
### Route properties
| Property | Required | Description |
|----------|----------|-------------|
| `events` | No | List of event types to accept (e.g. `["pull_request"]`). If empty, all events are accepted. Event type is read from `X-GitHub-Event`, `X-GitLab-Event`, or `event_type` in the payload. |
| `secret` | **Yes** | HMAC secret for signature validation. Falls back to the global `secret` if not set on the route. Set to `"INSECURE_NO_AUTH"` for testing only (skips validation). |
| `profile` | No | Profile authorized to execute this route when `gateway.multiplex_profiles` is enabled. Omit it for a default-profile-only route; set a profile name (for example `coder`) to bind the route and its secret to `/p/coder/webhooks/<route>`. |
| `prompt` | No | Template string with dot-notation payload access (e.g. `{pull_request.title}`). If omitted, the full JSON payload is dumped into the prompt. Payload fields are untrusted — see [Authenticated does not mean trusted](#authenticated-does-not-mean-trusted). |
| `filters` | No | Declarative payload filters evaluated after auth/body/event filtering and before agent or direct delivery work. Non-matches return `{"status":"ignored","reason":"filter"}` with HTTP 200. |
| `script` | No | Filter/transform script under `~/.hermes/scripts/`. The webhook payload is passed as JSON on stdin. JSON object stdout replaces the payload before templating; text stdout is exposed as `script_output`; empty stdout, `[SILENT]`, or a nonzero exit code ignores the webhook. |
| `skills` | No | List of skill names to load for the agent run. |
| `toolsets` | No | List of toolset keys (e.g. `["terminal", "file", "web"]`) that **replaces** the platform-level webhook toolset for runs triggered by this route only. Manual config edit only — not settable via `hermes webhook subscribe`, so agent-created subscriptions cannot self-grant elevated tools. Names are validated the same way as `platform_toolsets` entries (unknown or platform-restricted names are dropped). See [Per-route toolsets](#per-route-toolsets). |
| `deliver` | No | Where to send the response: `github_comment`, `telegram`, `discord`, `slack`, `signal`, `sms`, `whatsapp`, `matrix`, `mattermost`, `homeassistant`, `email`, `dingtalk`, `feishu`, `wecom`, `weixin`, `bluebubbles`, `qqbot`, or `log` (default). |
| `deliver_extra` | No | Additional delivery config — keys depend on `deliver` type (e.g. `repo`, `pr_number`, `chat_id`). Values support the same `{dot.notation}` templates as `prompt`. |
| `deliver_only` | No | If `true`, skip the agent entirely — the rendered `prompt` template becomes the literal message that gets delivered. Zero LLM cost, sub-second delivery. See [Direct Delivery Mode](#direct-delivery-mode) for use cases. Requires `deliver` to be a real target (not `log`). |
### Full example
```yaml
platforms:
webhook:
enabled: true
extra:
port: 8644
secret: "global-fallback-secret"
routes:
github-pr:
events: ["pull_request"]
secret: "github-webhook-secret"
prompt: |
Review this pull request:
Repository: {repository.full_name}
PR #{number}: {pull_request.title}
Author: {pull_request.user.login}
URL: {pull_request.html_url}
Diff URL: {pull_request.diff_url}
Action: {action}
skills: ["github-code-review"]
deliver: "github_comment"
deliver_extra:
repo: "{repository.full_name}"
pr_number: "{number}"
deploy-notify:
events: ["push"]
secret: "deploy-secret"
prompt: "New push to {repository.full_name} branch {ref}: {head_commit.message}"
filters:
- field: "ref"
equals: "refs/heads/main"
deliver: "telegram"
```
### Payload Filters
Use `filters` when a provider sends a broad event stream but only some payloads should wake the agent or trigger `deliver_only` delivery. Filters run after signature validation, body parsing, and `events`, but before prompt rendering, idempotency, agent dispatch, or direct delivery.
```yaml
platforms:
webhook:
extra:
routes:
todoist:
events: ["item:updated"]
secret: "todoist-secret"
filters:
- field: "payload.labels"
contains: "hermes"
- any:
- field: "payload.priority"
equals: 4
- field: "payload.project_id"
in_file: "~/.hermes/data/todoist/watchlist.json"
prompt: "Todoist task changed: {payload.content}"
```
Supported operators:
- `exists: true|false`
- `missing: true`
- `equals` / `not_equals`
- `contains` for strings, lists, and dict keys
- `in` for inline lists
- `in_file` for JSON arrays, JSON objects (keys are used), or newline-delimited text files
- `regex`
- `all`, `any`, and `not` groups
Field paths use dot notation. `payload.foo` reads from a top-level `payload` object when one exists, or from the root webhook body for flat payloads. `event` / `event_type` match the resolved event type, and `headers.<Name>` reads request headers.
### Script Filters and Transforms
Use `script` when declarative filters are not enough. Scripts must live under `~/.hermes/scripts/` for the active profile; relative paths resolve there, and path traversal outside that directory is blocked. `.sh` and `.bash` scripts run with bash, and all other extensions run with the current Python interpreter.
The route payload is sent to stdin as JSON:
```python
# ~/.hermes/scripts/todoist-hermes-label.py
import json
import sys
payload = json.load(sys.stdin)
labels = payload.get("payload", {}).get("labels", [])
if "hermes" not in labels:
print("[SILENT]")
raise SystemExit(0)
payload["body"] = payload["payload"]["content"]
print(json.dumps(payload))
```
Script outcomes:
- JSON object stdout replaces the payload used by `prompt` and `deliver_extra`.
- Non-JSON text stdout is added to the payload as `script_output`.
- Empty stdout, exact `[SILENT]`, `{"__hermes_ignore__": true}`, timeout, missing script, or nonzero exit code returns HTTP 200 with `{"status":"ignored","reason":"script"}`.
### Prompt Templates
Prompts use dot-notation to access nested fields in the webhook payload:
- `{pull_request.title}` resolves to `payload["pull_request"]["title"]`
- `{repository.full_name}` resolves to `payload["repository"]["full_name"]`
- `{__raw__}` — special token that dumps the **entire payload** as indented JSON (truncated at 4000 characters). Useful for monitoring alerts or generic webhooks where the agent needs the full context.
- Missing keys are left as the literal `{key}` string (no error)
- Nested dicts and lists are JSON-serialized and truncated at 2000 characters
You can mix `{__raw__}` with regular template variables:
```yaml
prompt: "PR #{pull_request.number} by {pull_request.user.login}: {__raw__}"
```
If no `prompt` template is configured for a route, the entire payload is dumped as indented JSON (truncated at 4000 characters).
The same dot-notation templates work in `deliver_extra` values.
### Forum Topic Delivery
When delivering webhook responses to Telegram, you can target a specific forum topic by including `message_thread_id` (or `thread_id`) in `deliver_extra`:
```yaml
webhooks:
routes:
alerts:
events: ["alert"]
prompt: "Alert: {__raw__}"
deliver: "telegram"
deliver_extra:
chat_id: "-1001234567890"
message_thread_id: "42"
```
If `chat_id` is not provided in `deliver_extra`, the delivery falls back to the home channel configured for the target platform.
---
## GitHub PR Review (Step by Step) {#github-pr-review}
This walkthrough sets up automatic code review on every pull request.
### 1. Create the webhook in GitHub
1. Go to your repository → **Settings** → **Webhooks** → **Add webhook**
2. Set **Payload URL** to `http://your-server:8644/webhooks/github-pr`
3. Set **Content type** to `application/json`
4. Set **Secret** to match your route config (e.g. `github-webhook-secret`)
5. Under **Which events?**, select **Let me select individual events** and check **Pull requests**
6. Click **Add webhook**
### 2. Add the route config
Add the `github-pr` route to your `~/.hermes/config.yaml` as shown in the example above.
### 3. Ensure `gh` CLI is authenticated
The `github_comment` delivery type uses the GitHub CLI to post comments:
```bash
gh auth login
```
### 4. Test it
Open a pull request on the repository. The webhook fires, Hermes processes the event, and posts a review comment on the PR.
---
## GitLab Webhook Setup {#gitlab-webhook-setup}
GitLab webhooks work similarly but use a different authentication mechanism. GitLab sends the secret as a plain `X-Gitlab-Token` header (exact string match, not HMAC).
### 1. Create the webhook in GitLab
1. Go to your project → **Settings** → **Webhooks**
2. Set the **URL** to `http://your-server:8644/webhooks/gitlab-mr`
3. Enter your **Secret token**
4. Select **Merge request events** (and any other events you want)
5. Click **Add webhook**
### 2. Add the route config
```yaml
platforms:
webhook:
enabled: true
extra:
routes:
gitlab-mr:
events: ["merge_request"]
secret: "your-gitlab-secret-token"
prompt: |
Review this merge request:
Project: {project.path_with_namespace}
MR !{object_attributes.iid}: {object_attributes.title}
Author: {object_attributes.last_commit.author.name}
URL: {object_attributes.url}
Action: {object_attributes.action}
deliver: "log"
```
---
## Delivery Options {#delivery-options}
The `deliver` field controls where the agent's response goes after processing the webhook event.
| Deliver Type | Description |
|-------------|-------------|
| `log` | Logs the response to the gateway log output. This is the default and is useful for testing. |
| `github_comment` | Posts the response as a PR/issue comment via the `gh` CLI. Requires `deliver_extra.repo` and `deliver_extra.pr_number`. The `gh` CLI must be installed and authenticated on the gateway host (`gh auth login`). |
| `telegram` | Routes the response to Telegram. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `discord` | Routes the response to Discord. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `slack` | Routes the response to Slack. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `signal` | Routes the response to Signal. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `sms` | Routes the response to SMS via Twilio. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `whatsapp` | Routes the response to WhatsApp. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `matrix` | Routes the response to Matrix. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `mattermost` | Routes the response to Mattermost. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `homeassistant` | Routes the response to Home Assistant. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `email` | Routes the response to Email. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `dingtalk` | Routes the response to DingTalk. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `feishu` | Routes the response to Feishu/Lark. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `wecom` | Routes the response to WeCom. Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `weixin` | Routes the response to Weixin (WeChat). Uses the home channel, or specify `chat_id` in `deliver_extra`. |
| `bluebubbles` | Routes the response to BlueBubbles (iMessage). Uses the home channel, or specify `chat_id` in `deliver_extra`. |
For cross-platform delivery, the target platform must also be enabled and connected in the gateway. If no `chat_id` is provided in `deliver_extra`, the response is sent to that platform's configured home channel.
---
## Direct Delivery Mode {#direct-delivery-mode}
By default, every webhook POST triggers an agent run — the payload becomes a prompt, the agent processes it, and the agent's response is delivered. This costs LLM tokens on every event.
For use cases where you just want to **push a plain notification** — no reasoning, no agent loop, just deliver the message — set `deliver_only: true` on the route. The rendered `prompt` template becomes the literal message body, and the adapter dispatches it directly to the configured delivery target.
### When to use direct delivery
- **External service push** — Supabase/Firebase webhook fires on a database change → notify a user in Telegram instantly
- **Monitoring alerts** — Datadog/Grafana alert webhook → push to a Discord channel
- **Inter-agent pings** — Agent A notifies Agent B's user that a long-running task finished
- **Background job completion** — Cron job finishes → post result to Slack
Benefits:
- **Zero LLM tokens** — the agent is never invoked
- **Sub-second delivery** — a single adapter call, no reasoning loop
- **Same security as agent mode** — HMAC auth, rate limits, idempotency, and body-size limits all still apply
- **Synchronous response** — the POST returns `200 OK` once delivery succeeds, or `502` if the target rejects it, so your upstream service can retry intelligently
### Example: Telegram push from Supabase
```yaml
platforms:
webhook:
enabled: true
extra:
port: 8644
secret: "global-secret"
routes:
antenna-matches:
secret: "antenna-webhook-secret"
deliver: "telegram"
deliver_only: true
prompt: "🎉 New match: {match.user_name} matched with you!"
deliver_extra:
chat_id: "{match.telegram_chat_id}"
```
Your Supabase edge function signs the payload with HMAC-SHA256 and POSTs to `https://your-server:8644/webhooks/antenna-matches`. The webhook adapter validates the signature, renders the template from the payload, delivers to Telegram, and returns `200 OK`.
### Example: Dynamic subscription via CLI
```bash
hermes webhook subscribe antenna-matches \
--deliver telegram \
--deliver-chat-id "123456789" \
--deliver-only \
--prompt "🎉 New match: {match.user_name} matched with you!" \
--description "Antenna match notifications"
```
### Response codes
| Status | Meaning |
|--------|---------|
| `200 OK` | Delivered successfully. Body: `{"status": "delivered", "route": "...", "target": "...", "delivery_id": "..."}` |
| `200 OK` (status=duplicate) | Duplicate `X-GitHub-Delivery` ID within the idempotency TTL (1 hour). Not re-delivered. |
| `401 Unauthorized` | HMAC signature invalid or missing. |
| `400 Bad Request` | Malformed JSON body. |
| `404 Not Found` | Unknown route name. |
| `413 Payload Too Large` | Body exceeded `max_body_bytes`. |
| `429 Too Many Requests` | Route rate limit exceeded. |
| `502 Bad Gateway` | Target adapter rejected the message or raised. The error is logged server-side; the response body is a generic `Delivery failed` to avoid leaking adapter internals. |
### Configuration gotchas
- `deliver_only: true` requires `deliver` to be a real target. `deliver: log` (or omitting `deliver`) is rejected at startup — the adapter refuses to start if it finds a misconfigured route.
- The `skills` field is ignored in direct delivery mode (no agent runs, so there's nothing to inject skills into).
- Template rendering uses the same `{dot.notation}` syntax as agent mode, including the `{__raw__}` token.
- Idempotency uses the same `X-GitHub-Delivery` / `X-Request-ID` header — retries with the same ID return `status=duplicate` and do NOT re-deliver.
---
## Dynamic Subscriptions (CLI) {#dynamic-subscriptions}
In addition to static routes in `config.yaml`, you can create webhook subscriptions dynamically using the `hermes webhook` CLI command. This is especially useful when the agent itself needs to set up event-driven triggers.
### Create a subscription
```bash
hermes webhook subscribe github-issues \
--events "issues" \
--prompt "New issue #{issue.number}: {issue.title}\nBy: {issue.user.login}\n\n{issue.body}" \
--deliver telegram \
--deliver-chat-id "-100123456789" \
--description "Triage new GitHub issues"
```
This returns the webhook URL and an auto-generated HMAC secret. Configure your service to POST to that URL.
### List subscriptions
```bash
hermes webhook list
```
### Remove a subscription
```bash
hermes webhook remove github-issues
```
### Test a subscription
```bash
hermes webhook test github-issues
hermes webhook test github-issues --payload '{"issue": {"number": 42, "title": "Test"}}'
```
### How dynamic subscriptions work
- Subscriptions are stored in `~/.hermes/webhook_subscriptions.json`
- The webhook adapter hot-reloads this file on each incoming request (mtime-gated, negligible overhead)
- Static routes from `config.yaml` always take precedence over dynamic ones with the same name
- Dynamic subscriptions use the same route format and capabilities as static routes (events, prompt templates, skills, delivery)
- No gateway restart required — subscribe and it's immediately live
### Agent-driven subscriptions
The agent can create subscriptions via the terminal tool when guided by the `webhook-subscriptions` skill. Ask the agent to "set up a webhook for GitHub issues" and it will run the appropriate `hermes webhook subscribe` command.
---
## Per-route toolsets {#per-route-toolsets}
Webhook agent runs default to a deliberately constrained toolset (`web_search`, `web_extract`, `vision_analyze`, `clarify`) because webhook payloads can carry untrusted third-party content — a public PR title or issue comment should never be able to prompt-inject its way into your terminal.
For **trusted** routes — a localhost monitoring daemon pushing system alerts, an internal CI system — you can grant a wider toolset to that route only, without widening every other webhook route:
```yaml
platforms:
webhook:
enabled: true
extra:
routes:
oom-emergency:
secret: "monitor-secret"
prompt: "Memory emergency: {detail}. Diagnose with ps/free/py-spy and report."
toolsets: ["terminal", "file", "code_execution", "web"]
deliver: "telegram"
```
For dynamic subscriptions, add the `toolsets` key by editing `~/.hermes/webhook_subscriptions.json` directly:
```json
{
"oom-emergency": {
"secret": "...",
"prompt": "...",
"toolsets": ["terminal", "file", "web"],
"deliver": "telegram"
}
}
```
Behavior and safety properties:
- The route list **replaces** the platform-level webhook toolset resolution for that route's runs (it is not merged).
- Names are validated through the same path as `platform_toolsets` config — unknown names and platform-restricted toolsets are dropped.
- `hermes webhook subscribe` deliberately does **not** accept a toolsets flag. Granting elevated tools is a manual config-file edit, so an agent creating its own subscription at runtime cannot self-grant `terminal`.
- Only grant elevated toolsets to routes whose senders you fully control, with a real HMAC secret. Anyone who can POST a validly-signed payload to that route is effectively running an agent with those tools.
---
## Security {#security}
The webhook adapter includes multiple layers of security:
### HMAC signature validation
The adapter validates incoming webhook signatures using the appropriate method for each source:
- **GitHub**: `X-Hub-Signature-256` header — HMAC-SHA256 hex digest prefixed with `sha256=`
- **GitLab**: `X-Gitlab-Token` header — plain secret string match
- **Generic (V2, recommended)**: `X-Webhook-Signature-V2` + `X-Webhook-Timestamp` headers — HMAC-SHA256 hex digest of `<timestamp>.<body>`. The timestamp (Unix seconds) must be within ±300 seconds of the server clock, which prevents captured requests from being replayed later.
- **Generic (V1, legacy)**: `X-Webhook-Signature` header — raw HMAC-SHA256 hex digest of the body only. Still accepted for backward compatibility, but it has no replay protection (a captured request replays indefinitely); the gateway logs a deprecation warning once per route. Switch senders to V2.
If a secret is configured but no recognized signature header is present, the request is rejected.
### Secret is required
Every route must have a secret — either set directly on the route or inherited from the global `secret`. Routes without a secret cause the adapter to fail at startup with an error. For development/testing only, you can set the secret to `"INSECURE_NO_AUTH"` to skip validation entirely.
When multi-profile routing is enabled, the route's `profile` field also
binds that secret to one execution target. A route without `profile` is
default-profile-only. A request carrying a valid route signature is still
rejected if its `/p/<profile>/` prefix does not match the route binding.
`INSECURE_NO_AUTH` is only accepted when the gateway is bound to a loopback host (`127.0.0.1`, `localhost`, `::1`). If it is combined with a non-loopback bind such as `0.0.0.0` or a LAN IP, the adapter refuses to start — this prevents accidentally exposing an unauthenticated endpoint on a public interface.
### Rate limiting
Each route is rate-limited to **30 requests per minute** by default (fixed-window). Configure this globally:
```yaml
platforms:
webhook:
extra:
rate_limit: 60 # requests per minute
```
Requests exceeding the limit receive a `429 Too Many Requests` response.
### Idempotency
Delivery IDs (from `X-GitHub-Delivery`, `X-Request-ID`, or a timestamp fallback) are cached for **1 hour**. Duplicate deliveries (e.g. webhook retries) are silently skipped with a `200` response, preventing duplicate agent runs.
### Body size limits
Payloads exceeding **1 MB** are rejected before the body is read. Configure this:
```yaml
platforms:
webhook:
extra:
max_body_bytes: 2097152 # 2 MB
```
### Authenticated does not mean trusted
:::warning
**HMAC validation authenticates the _sender_, not the _content_.** A valid signature only proves the request came from a party holding the route's secret (e.g. GitHub). It says nothing about who wrote the _business fields_ inside the payload — PR titles, commit messages, issue descriptions, and any other upstream text are authored by arbitrary third parties and must be treated as untrusted.
This is the same trust model that applies to everything the agent reads: web pages, files, and tool output are all untrusted input. Hermes does not — and cannot reliably — sanitize untrusted text with a blocklist; phrasing, encoding, and translation make that trivially bypassable. **The trust boundary is the agent's capability surface, not the input channel.** Harden there:
- **Sandbox the runtime.** Run the gateway with the Docker or SSH terminal backend (or in a VM) when exposed to the internet, so a hijacked turn cannot touch the host.
- **Scope the toolset.** Disable `terminal`, `file`, and outbound-action tools on webhook-triggered sessions if the route only needs to read and summarize. Fewer capabilities means a smaller blast radius if a payload field carries injected instructions.
- **Keep approvals on** for any destructive or outbound operation, so an injected instruction cannot act unattended.
- **Template narrowly.** Prefer a specific `prompt` with named fields (`{pull_request.title}`) over `{__raw__}` or an empty template that dumps the whole payload, so only the fields you intend reach the prompt.
:::
---
## Troubleshooting {#troubleshooting}
### Webhook not arriving
- Verify the port is exposed and accessible from the webhook source
- Check firewall rules — port `8644` (or your configured port) must be open
- Verify the URL path matches: `http://your-server:8644/webhooks/<route-name>`
- Use the `/health` endpoint to confirm the server is running
### Signature validation failing
- Ensure the secret in your route config exactly matches the secret configured in the webhook source
- For GitHub, the secret is HMAC-based — check `X-Hub-Signature-256`
- For GitLab, the secret is a plain token match — check `X-Gitlab-Token`
- Check gateway logs for `Invalid signature` warnings
### Event being ignored
- Check that the event type is in your route's `events` list
- GitHub events use values like `pull_request`, `push`, `issues` (the `X-GitHub-Event` header value)
- GitLab events use values like `merge_request`, `push` (the `X-GitLab-Event` header value)
- If `events` is empty or not set, all events are accepted
### Agent not responding
- Run the gateway in foreground to see logs: `hermes gateway run`
- Check that the prompt template is rendering correctly
- Verify the delivery target is configured and connected
### Duplicate responses
- The idempotency cache should prevent this — check that the webhook source is sending a delivery ID header (`X-GitHub-Delivery` or `X-Request-ID`)
- Delivery IDs are cached for 1 hour
### `gh` CLI errors (GitHub comment delivery)
- Run `gh auth login` on the gateway host
- Ensure the authenticated GitHub user has write access to the repository
- Check that `gh` is installed and on the PATH
---
## Environment Variables {#environment-variables}
| Variable | Description | Default |
|----------|-------------|---------|
| `WEBHOOK_ENABLED` | Enable the webhook platform adapter | `false` |
| `WEBHOOK_PORT` | HTTP server port for receiving webhooks | `8644` |
| `WEBHOOK_SECRET` | Global HMAC secret (used as fallback when routes don't specify their own) | _(none)_ |
website/docs/user-guide/messaging/wecom-callback.md
+179
View File
@@ -0,0 +1,179 @@
---
sidebar_position: 15
---
# WeCom Callback (Self-Built App)
Connect Hermes to WeCom (Enterprise WeChat) as a self-built enterprise application using the callback/webhook model.
:::info WeCom Bot vs WeCom Callback
Hermes supports two WeCom integration modes:
- **[WeCom Bot](wecom.md)** — bot-style, connects via WebSocket. Simpler setup, works in group chats.
- **WeCom Callback** (this page) — self-built app, receives encrypted XML callbacks. Shows as a first-class app in users' WeCom sidebar. Supports multi-corp routing.
:::
See also: [WeCom Bot](./wecom.md) for the bot-style integration.
> Run `hermes gateway setup` and pick **WeCom Callback** for a guided walk-through.
## How It Works
1. You register a self-built application in the WeCom Admin Console
2. WeCom pushes encrypted XML to your HTTP callback endpoint
3. Hermes decrypts the message, queues it for the agent
4. Immediately acknowledges (silent — nothing displayed to the user)
5. The agent processes the request (typically 3–30 minutes)
6. The reply is delivered proactively via the WeCom `message/send` API
## Prerequisites
- A WeCom enterprise account with admin access
- `aiohttp` and `httpx` Python packages (included in the default install)
- A publicly reachable server for the callback URL (or a tunnel like ngrok)
## Setup
### 1. Create a Self-Built App in WeCom
1. Go to [WeCom Admin Console](https://work.weixin.qq.com/) → **Applications** → **Create App**
2. Note your **Corp ID** (shown at the top of the admin console)
3. In the app settings, create a **Corp Secret**
4. Note the **Agent ID** from the app's overview page
5. Under **Receive Messages**, configure the callback URL:
- URL: `http://YOUR_PUBLIC_IP:8645/wecom/callback`
- Token: Generate a random token (WeCom provides one)
- EncodingAESKey: Generate a key (WeCom provides one)
### 2. Configure Environment Variables
Add to your `.env` file:
```bash
WECOM_CALLBACK_CORP_ID=your-corp-id
WECOM_CALLBACK_CORP_SECRET=your-corp-secret
WECOM_CALLBACK_AGENT_ID=1000002
WECOM_CALLBACK_TOKEN=your-callback-token
WECOM_CALLBACK_ENCODING_AES_KEY=your-43-char-aes-key
# Optional
# WECOM_CALLBACK_HOST= # optional pin; unset = dual-stack (all interfaces, IPv4+IPv6)
WECOM_CALLBACK_PORT=8645
WECOM_CALLBACK_ALLOWED_USERS=user1,user2
```
### 3. Start the Gateway
```bash
hermes gateway
```
(Use `hermes gateway start` only after `hermes gateway install` has registered the systemd/launchd service.)
The callback adapter starts an HTTP server on the configured port. WeCom will verify the callback URL via a GET request, then begin sending messages via POST.
## Configuration Reference
Set these in `config.yaml` under `platforms.wecom_callback.extra`, or use environment variables:
| Setting | Default | Description |
|---------|---------|-------------|
| `corp_id` | — | WeCom enterprise Corp ID (required) |
| `corp_secret` | — | Corp secret for the self-built app (required) |
| `agent_id` | — | Agent ID of the self-built app (required) |
| `token` | — | Callback verification token (required) |
| `encoding_aes_key` | — | 43-character AES key for callback encryption (required) |
| `host` | unset (dual-stack: all interfaces, IPv4+IPv6) | Bind address for the HTTP callback server |
| `port` | `8645` | Port for the HTTP callback server |
| `path` | `/wecom/callback` | URL path for the callback endpoint |
## Multi-App Routing
For enterprises running multiple self-built apps (e.g., across different departments or subsidiaries), configure the `apps` list in `config.yaml`:
```yaml
platforms:
wecom_callback:
enabled: true
extra:
host: "0.0.0.0"
port: 8645
apps:
- name: "dept-a"
corp_id: "ww_corp_a"
corp_secret: "secret-a"
agent_id: "1000002"
token: "token-a"
encoding_aes_key: "key-a-43-chars..."
- name: "dept-b"
corp_id: "ww_corp_b"
corp_secret: "secret-b"
agent_id: "1000003"
token: "token-b"
encoding_aes_key: "key-b-43-chars..."
```
Users are scoped by `corp_id:user_id` to prevent cross-corp collisions. When a user sends a message, the adapter records which app (corp) they belong to and routes replies through the correct app's access token.
## Access Control
Restrict which users can interact with the app:
```bash
# Allowlist specific users
WECOM_CALLBACK_ALLOWED_USERS=zhangsan,lisi,wangwu
# Or allow all users
WECOM_CALLBACK_ALLOW_ALL_USERS=true
```
## Endpoints
The adapter exposes:
| Method | Path | Purpose |
|--------|------|---------|
| GET | `/wecom/callback` | URL verification handshake (WeCom sends this during setup) |
| POST | `/wecom/callback` | Encrypted message callback (WeCom sends user messages here) |
| GET | `/health` | Health check — returns `{"status": "ok"}` |
## Encryption
All callback payloads are encrypted with AES-CBC using the EncodingAESKey. The adapter handles:
- **Inbound**: Decrypt XML payload, verify SHA1 signature
- **Outbound**: Replies sent via proactive API (not encrypted callback response)
The crypto implementation is compatible with Tencent's official WXBizMsgCrypt SDK.
## Limitations
- **No streaming** — replies arrive as complete messages after the agent finishes
- **No typing indicators** — the callback model doesn't support typing status
- **Text only** — currently supports text messages for input; image/file/voice input not yet implemented. The agent is aware of outbound media capabilities via the WeCom platform hint (images, documents, video, voice).
- **Response latency** — agent sessions take 3–30 minutes; users see the reply when processing completes
## Troubleshooting
**Signature verification failing.**
WeCom signs every request with the **Token** you registered in the admin
console. A mismatch between the token configured in Hermes and the token the
admin console expects is the most common cause. Re-copy both the **Token** and
**EncodingAESKey** from the admin console — they're easy to truncate. Whitespace
in `~/.hermes/.env` values around `=` will also break signature checks. After
fixing, restart `hermes gateway run`.
**Callback URL not reachable / verification step fails.**
WeCom hits the public URL you registered. Confirm:
1. Your reverse proxy / tunnel forwards `/wecom/callback` to the gateway's port.
2. The URL in the admin console is HTTPS (WeCom rejects plain HTTP).
3. From outside your network, `curl -i https://<your-domain>/wecom/callback`
returns something other than a timeout (a 4xx without query params is fine —
it just means the listener is reachable).
**Port not reachable / listener not bound.**
Check `hermes gateway run` logs for the bound host/port. If the adapter bound to
`127.0.0.1` you must front it with a reverse proxy or tunnel — WeCom's servers
can't reach loopback. Leave `extra.host` unset so the default dual-stack
bind (all interfaces, IPv4+IPv6) applies, or pin an interface in `config.yaml` (plus
`allowed_source_cidrs` if exposing directly) or keep loopback and use a tunnel
such as Cloudflare Tunnel / nginx.
website/docs/user-guide/messaging/wecom.md
+306
View File
@@ -0,0 +1,306 @@
---
sidebar_position: 14
title: "WeCom (Enterprise WeChat)"
description: "Connect Hermes Agent to WeCom via the AI Bot WebSocket gateway"
---
# WeCom (Enterprise WeChat)
Connect Hermes to [WeCom](https://work.weixin.qq.com/) (企业微信), Tencent's enterprise messaging platform. The adapter uses WeCom's AI Bot WebSocket gateway for real-time bidirectional communication — no public endpoint or webhook needed.
See also: [WeCom Callback](./wecom-callback.md) for inbound webhook setup.
## Prerequisites
- A WeCom organization account
- An AI Bot created in the WeCom Admin Console
- The Bot ID and Secret from the bot's credentials page
- Python packages: `aiohttp` and `httpx`
## Setup
### Step 1: Create an AI Bot
#### Recommended: Scan-to-Create (one command)
```bash
hermes gateway setup
```
Select **WeCom** and scan the QR code with your WeCom mobile app. Hermes will automatically create a bot application with the correct permissions and save the credentials.
The setup wizard will:
1. Display a QR code in your terminal
2. Wait for you to scan it with the WeCom mobile app
3. Automatically retrieve the Bot ID and Secret
4. Guide you through access control configuration
#### Alternative: Manual Setup
If scan-to-create is not available, the wizard falls back to manual input:
1. Log in to the [WeCom Admin Console](https://work.weixin.qq.com/wework_admin/frame)
2. Navigate to **Applications** → **Create Application** → **AI Bot**
3. Configure the bot name and description
4. Copy the **Bot ID** and **Secret** from the credentials page
5. Run `hermes gateway setup`, select **WeCom**, and enter the credentials when prompted
:::warning
Keep the Bot Secret private. Anyone with it can impersonate your bot.
:::
### Step 2: Configure Hermes
#### Option A: Interactive Setup (Recommended)
```bash
hermes gateway setup
```
Select **WeCom** and follow the prompts. The wizard will guide you through:
- Bot credentials (via QR scan or manual entry)
- Access control settings (allowlist, pairing mode, or open access)
- Home channel for notifications
#### Option B: Manual Configuration
Add the following to `~/.hermes/.env`:
```bash
WECOM_BOT_ID=your-bot-id
WECOM_SECRET=your-secret
# Optional: restrict access
WECOM_ALLOWED_USERS=user_id_1,user_id_2
# Optional: home channel for cron/notifications
WECOM_HOME_CHANNEL=chat_id
```
### Step 3: Start the gateway
```bash
hermes gateway
```
## Features
- **WebSocket transport** — persistent connection, no public endpoint needed
- **DM and group messaging** — configurable access policies
- **Per-group sender allowlists** — fine-grained control over who can interact in each group
- **Media support** — images, files, voice, video upload and download
- **AES-encrypted media** — automatic decryption for inbound attachments
- **Quote context** — preserves reply threading
- **Markdown rendering** — rich text responses
- **Reply correlation** — responses are correlated to the inbound message context
- **Auto-reconnect** — exponential backoff on connection drops
:::note Streaming and typing indicators
The WeCom adapter streams responses natively over WeCom's `msgtype: "stream"`
protocol: the client shows a thinking/typing bubble as soon as a turn starts,
and the reply renders token-by-token in a single bubble as the model
generates it. Tool-call progress is folded into the same bubble. Native
streaming is enabled by default (`display.platforms.wecom.streaming: true` in
`config.yaml`); set it to `false` to restore single-shot delivery.
:::
## Configuration Options
Set these in `config.yaml` under `platforms.wecom.extra`:
| Key | Default | Description |
|-----|---------|-------------|
| `bot_id` | — | WeCom AI Bot ID (required) |
| `secret` | — | WeCom AI Bot Secret (required) |
| `websocket_url` | `wss://openws.work.weixin.qq.com` | WebSocket gateway URL |
| `dm_policy` | `open` | DM access: `open`, `allowlist`, `disabled`, `pairing` |
| `group_policy` | `open` | Group access: `open`, `allowlist`, `disabled` |
| `allow_from` | `[]` | User IDs allowed for DMs (when dm_policy=allowlist) |
| `group_allow_from` | `[]` | Group IDs allowed (when group_policy=allowlist) |
| `groups` | `{}` | Per-group configuration (see below) |
| `stream_keepalive_enabled` | `false` | Send periodic keepalive frames to refresh WeCom's ~6-minute reply-stream window on long turns |
| `stream_keepalive_interval_seconds` | `120` | Keepalive frame cadence when enabled |
| `stream_safe_duration_seconds` | `330` | Stream age after which finalize prefers the reliable proactive send |
## Access Policies
### DM Policy
Controls who can send direct messages to the bot:
| Value | Behavior |
|-------|----------|
| `open` | Anyone can DM the bot (default) |
| `allowlist` | Only user IDs in `allow_from` can DM |
| `disabled` | All DMs are ignored |
| `pairing` | Pairing mode (for initial setup) |
```bash
WECOM_DM_POLICY=allowlist
```
### Group Policy
Controls which groups the bot responds in:
| Value | Behavior |
|-------|----------|
| `open` | Bot responds in all groups (default) |
| `allowlist` | Bot only responds in group IDs listed in `group_allow_from` |
| `disabled` | All group messages are ignored |
```bash
WECOM_GROUP_POLICY=allowlist
```
### Per-Group Sender Allowlists
For fine-grained control, you can restrict which users are allowed to interact with the bot within specific groups. This is configured in `config.yaml`:
```yaml
platforms:
wecom:
enabled: true
extra:
bot_id: "your-bot-id"
secret: "your-secret"
group_policy: "allowlist"
group_allow_from:
- "group_id_1"
- "group_id_2"
groups:
group_id_1:
allow_from:
- "user_alice"
- "user_bob"
group_id_2:
allow_from:
- "user_charlie"
"*":
allow_from:
- "user_admin"
```
**How it works:**
1. The `group_policy` and `group_allow_from` controls determine whether a group is allowed at all.
2. If a group passes the top-level check, the `groups.<group_id>.allow_from` list (if present) further restricts which senders within that group can interact with the bot.
3. A wildcard `"*"` group entry serves as a default for groups not explicitly listed.
4. Allowlist entries support the `*` wildcard to allow all users, and entries are case-insensitive.
5. Entries can optionally use the `wecom:user:` or `wecom:group:` prefix format — the prefix is stripped automatically.
If no `allow_from` is configured for a group, all users in that group are allowed (assuming the group itself passes the top-level policy check).
## Media Support
### Inbound (receiving)
The adapter receives media attachments from users and caches them locally for agent processing:
| Type | How it's handled |
|------|-----------------|
| **Images** | Downloaded and cached locally. Supports both URL-based and base64-encoded images. |
| **Files** | Downloaded and cached. Filename is preserved from the original message. |
| **Voice** | Voice message text transcription is extracted if available. |
| **Mixed messages** | WeCom mixed-type messages (text + images) are parsed and all components extracted. |
**Quoted messages:** Media from quoted (replied-to) messages is also extracted, so the agent has context about what the user is replying to.
### AES-Encrypted Media Decryption
WeCom encrypts some inbound media attachments with AES-256-CBC. The adapter handles this automatically:
- When an inbound media item includes an `aeskey` field, the adapter downloads the encrypted bytes and decrypts them using AES-256-CBC with PKCS#7 padding.
- The AES key is the base64-decoded value of the `aeskey` field (must be exactly 32 bytes).
- The IV is derived from the first 16 bytes of the key.
- This requires the `cryptography` Python package (`pip install cryptography`).
No configuration is needed — decryption happens transparently when encrypted media is received.
### Outbound (sending)
| Method | What it sends | Size limit |
|--------|--------------|------------|
| `send` | Markdown text messages | 4000 chars |
| `send_image` / `send_image_file` | Native image messages | 10 MB |
| `send_document` | File attachments | 20 MB |
| `send_voice` | Voice messages (AMR format only for native voice) | 2 MB |
| `send_video` | Video messages | 10 MB |
**Chunked upload:** Files are uploaded in 512 KB chunks through a three-step protocol (init → chunks → finish). The adapter handles this automatically.
**Automatic downgrade:** When media exceeds the native type's size limit but is under the absolute 20 MB file limit, it is automatically sent as a generic file attachment instead:
- Images > 10 MB → sent as file
- Videos > 10 MB → sent as file
- Voice > 2 MB → sent as file
- Non-AMR audio → sent as file (WeCom only supports AMR for native voice)
Files exceeding the absolute 20 MB limit are rejected with an informational message sent to the chat.
## Reply-Mode Responses
When the bot receives a message via the WeCom callback, the adapter remembers the inbound request ID. If a response is sent while the request context is still active, the adapter uses WeCom's reply-mode (`aibot_respond_msg`) to correlate the response directly to the inbound message. This provides a more natural conversation experience in the WeCom client.
When a native reply stream is active, the response streams incrementally through reply-mode `msgtype: "stream"` frames. If the inbound request context has expired or is unavailable (or a stream frame fails), the adapter falls back to proactive message sending via `aibot_send_msg`.
Reply-mode also works for media: uploaded media can be sent as a reply to the originating message.
## Connection and Reconnection
The adapter maintains a persistent WebSocket connection to WeCom's gateway at `wss://openws.work.weixin.qq.com`.
### Connection Lifecycle
1. **Connect:** Opens a WebSocket connection and sends an `aibot_subscribe` authentication frame with the bot_id and secret.
2. **Heartbeat:** Sends application-level ping frames every 30 seconds to keep the connection alive.
3. **Listen:** Continuously reads inbound frames and dispatches message callbacks.
### Reconnection Behavior
On connection loss, the adapter uses exponential backoff to reconnect:
| Attempt | Delay |
|---------|-------|
| 1st retry | 2 seconds |
| 2nd retry | 5 seconds |
| 3rd retry | 10 seconds |
| 4th retry | 30 seconds |
| 5th+ retry | 60 seconds |
After each successful reconnection, the backoff counter resets to zero. All pending request futures are failed on disconnect so callers don't hang indefinitely.
### Deduplication
Inbound messages are deduplicated using message IDs with a 5-minute window and a maximum cache of 1000 entries. This prevents double-processing of messages during reconnection or network hiccups.
## All Environment Variables
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `WECOM_BOT_ID` | ✅ | — | WeCom AI Bot ID |
| `WECOM_SECRET` | ✅ | — | WeCom AI Bot Secret |
| `WECOM_ALLOWED_USERS` | — | _(empty)_ | Comma-separated user IDs for the gateway-level allowlist |
| `WECOM_HOME_CHANNEL` | — | — | Chat ID for cron/notification output |
| `WECOM_WEBSOCKET_URL` | — | `wss://openws.work.weixin.qq.com` | WebSocket gateway URL |
| `WECOM_DM_POLICY` | — | `open` | DM access policy |
| `WECOM_GROUP_POLICY` | — | `open` | Group access policy |
## Troubleshooting
| Problem | Fix |
|---------|-----|
| `WECOM_BOT_ID and WECOM_SECRET are required` | Set both env vars or configure in setup wizard |
| `WeCom startup failed: aiohttp not installed` | Install aiohttp: `pip install aiohttp` |
| `WeCom startup failed: httpx not installed` | Install httpx: `pip install httpx` |
| `invalid secret (errcode=40013)` | Verify the secret matches your bot's credentials |
| `Timed out waiting for subscribe acknowledgement` | Check network connectivity to `openws.work.weixin.qq.com` |
| Bot doesn't respond in groups | Check `group_policy` setting and ensure the group ID is in `group_allow_from` |
| Bot ignores certain users in a group | Check per-group `allow_from` lists in the `groups` config section |
| Media decryption fails | Install `cryptography`: `pip install cryptography` |
| `cryptography is required for WeCom media decryption` | The inbound media is AES-encrypted. Install: `pip install cryptography` |
| Voice messages sent as files | WeCom only supports AMR format for native voice. Other formats are auto-downgraded to file. |
| `File too large` error | WeCom has a 20 MB absolute limit on all file uploads. Compress or split the file. |
| Images sent as files | Images > 10 MB exceed the native image limit and are auto-downgraded to file attachments. |
| `Timeout sending message to WeCom` | The WebSocket may have disconnected. Check logs for reconnection messages. |
| `WeCom websocket closed during authentication` | Network issue or incorrect credentials. Verify bot_id and secret. |
website/docs/user-guide/messaging/weixin.md
+333
View File
@@ -0,0 +1,333 @@
---
sidebar_position: 15
title: "Weixin (WeChat)"
description: "Connect Hermes Agent to personal WeChat accounts via the iLink Bot API"
---
# Weixin (WeChat)
Connect Hermes to [WeChat](https://weixin.qq.com/) (微信), Tencent's personal messaging platform. The adapter uses Tencent's **iLink Bot API** for personal WeChat accounts — this is distinct from WeCom (Enterprise WeChat). Messages are delivered via long-polling, so no public endpoint or webhook is required.
:::info
This adapter is for **personal WeChat accounts** (微信). If you need enterprise/corporate WeChat, see the [WeCom adapter](./wecom.md) instead.
:::
:::warning iLink bot identity — ordinary WeChat groups may not work
QR login connects Hermes to an **iLink bot identity** (e.g. `a5ace6fd482e@im.bot`), **not** a fully scriptable ordinary personal WeChat account. Consequences:
- The iLink bot identity generally **cannot be invited into ordinary WeChat groups** the way a normal contact can.
- iLink typically **does not deliver ordinary WeChat group events** (including `@`-mentions of the personal account used for QR login) to the gateway for most bot-type accounts.
- `@`-mentioning the personal WeChat account used to scan the QR code is **not** the same as `@`-mentioning the iLink bot — the bot is a separate identity.
- The `WEIXIN_GROUP_POLICY` / `WEIXIN_GROUP_ALLOWED_USERS` settings below only take effect when iLink actually returns group events for your account type. If it doesn't, group messages will never reach Hermes regardless of policy.
In practice, most deployments only get DMs to the iLink bot working reliably. If group delivery doesn't work after configuration, the limitation is on the iLink side, not in Hermes. The gateway logs a `WARNING` at startup whenever `WEIXIN_GROUP_POLICY` is set to anything other than `disabled`.
:::
## Prerequisites
- A personal WeChat account
- Python packages: `aiohttp` and `cryptography`
- Terminal QR rendering is included when Hermes is installed with the `messaging` extra
Install the required dependencies:
```bash
pip install aiohttp cryptography
# Optional: for terminal QR code display
cd ~/.hermes/hermes-agent && uv pip install -e ".[messaging]"
```
## Setup
### 1. Run the Setup Wizard
The easiest way to connect your WeChat account is through the interactive setup:
```bash
hermes gateway setup
```
Select **Weixin** when prompted. The wizard will:
1. Request a QR code from the iLink Bot API
2. Display the QR code in your terminal (or provide a URL)
3. Wait for you to scan the QR code with the WeChat mobile app
4. Prompt you to confirm the login on your phone
5. Save the account credentials automatically to `~/.hermes/weixin/accounts/`
Once confirmed, you'll see a message like:
```
微信连接成功，account_id=your-account-id
```
The wizard stores the `account_id`, `token`, and `base_url` so you don't need to configure them manually.
### 2. Configure Environment Variables
After initial QR login, set at minimum the account ID in `~/.hermes/.env`:
```bash
WEIXIN_ACCOUNT_ID=your-account-id
# Optional: override the token (normally auto-saved from QR login)
# WEIXIN_TOKEN=your-bot-token
# Optional: restrict access
WEIXIN_DM_POLICY=open
WEIXIN_ALLOWED_USERS=user_id_1,user_id_2
# Optional: restore legacy multiline splitting behavior
# WEIXIN_SPLIT_MULTILINE_MESSAGES=true
# Optional: home channel for cron/notifications
WEIXIN_HOME_CHANNEL=chat_id
WEIXIN_HOME_CHANNEL_NAME=Home
```
### 3. Start the Gateway
```bash
hermes gateway
```
The adapter will restore saved credentials, connect to the iLink API, and begin long-polling for messages.
## Features
- **Long-poll transport** — no public endpoint, webhook, or WebSocket needed
- **QR code login** — scan-to-connect setup via `hermes gateway setup`
- **DM messaging** — configurable access policies; group messaging depends on iLink actually delivering group events for the connected identity (often not the case for iLink bot accounts — see the warning above)
- **Media support** — images, video, files, and voice messages
- **AES-128-ECB encrypted CDN** — automatic encryption/decryption for all media transfers
- **Context token persistence** — disk-backed reply continuity across restarts
- **Markdown formatting** — preserves Markdown, including headers, tables, and code blocks, so WeChat clients that support Markdown can render it natively
- **Smart message chunking** — messages stay as a single bubble when under the limit; only oversized payloads split at logical boundaries
- **Typing indicators** — shows "typing…" status in the WeChat client while the agent processes
- **SSRF protection** — outbound media URLs are validated before download
- **Message deduplication** — 5-minute sliding window prevents double-processing
- **Automatic retry with backoff** — recovers from transient API errors
## Configuration Options
Set these in `config.yaml` under `platforms.weixin.extra`:
| Key | Default | Description |
|-----|---------|-------------|
| `account_id` | — | iLink Bot account ID (required) |
| `token` | — | iLink Bot token (required, auto-saved from QR login) |
| `base_url` | `https://ilinkai.weixin.qq.com` | iLink API base URL |
| `cdn_base_url` | `https://novac2c.cdn.weixin.qq.com/c2c` | CDN base URL for media transfer |
| `dm_policy` | `open` | DM access: `open`, `allowlist`, `disabled`, `pairing` |
| `group_policy` | `disabled` | Group access: `open`, `allowlist`, `disabled` |
| `allow_from` | `[]` | User IDs allowed for DMs (when dm_policy=allowlist) |
| `group_allow_from` | `[]` | Group IDs allowed (when group_policy=allowlist) |
| `split_multiline_messages` | `false` | When `true`, split multi-line replies into multiple chat messages (legacy behavior). When `false`, keep multi-line replies as one message unless they exceed the length limit. |
| `text_batch_delay_seconds` | `3.0` | Quiet period (seconds) before a buffered burst of rapid text messages is flushed as one combined request. iLink delivers messages individually, so this debounce avoids one agent invocation per fragment. Set `0` to dispatch each message immediately. |
| `text_batch_split_delay_seconds` | `5.0` | Extended flush delay used when the latest fragment is near the split threshold (long messages iLink may have chunked). |
## Access Policies
### DM Policy
Controls who can send direct messages to the bot:
| Value | Behavior |
|-------|----------|
| `open` | Anyone can DM the bot (default) |
| `allowlist` | Only user IDs in `allow_from` can DM |
| `disabled` | All DMs are ignored |
| `pairing` | Pairing mode (for initial setup) |
```bash
WEIXIN_DM_POLICY=allowlist
WEIXIN_ALLOWED_USERS=user_id_1,user_id_2
```
`WEIXIN_ALLOWED_USERS` is an **inbound filter**, not an invitation system. QR
login connects one iLink bot identity to Hermes. Other people do not scan the
Hermes QR code with their own accounts; they must message the connected iLink
bot/contact through WeChat, and Hermes will process the DM only if the sender's
Weixin user ID is present in `WEIXIN_ALLOWED_USERS`.
A practical setup flow is:
1. Pair Hermes once with `hermes gateway setup` and note the connected iLink bot
account.
2. Have each allowed user send a direct message to that bot/contact.
3. Read the sender/user ID from the gateway logs or the inbound event payload.
4. Add those IDs to `WEIXIN_ALLOWED_USERS`, then restart the gateway.
If only the account that scanned the QR code can talk to Hermes, verify that the
other users are messaging the iLink bot identity itself, not the personal WeChat
account that performed the QR login. The iLink bot is a separate identity, and
ordinary WeChat contact/group routing can be limited by Tencent's iLink behavior.
### Group Policy
Controls which groups the bot responds in **when iLink delivers group events for the connected identity**. For QR-login iLink bot identities (e.g. `...@im.bot`), group events are typically not delivered at all, so this policy may have no effect — see the iLink bot limitation warning at the top of the page.
| Value | Behavior |
|-------|----------|
| `open` | Bot responds in all groups (if events are delivered) |
| `allowlist` | Bot only responds in group IDs listed in `group_allow_from` (if events are delivered) |
| `disabled` | All group messages are ignored (default) |
```bash
WEIXIN_GROUP_POLICY=allowlist
# NOTE: this is a comma-separated list of group chat IDs, NOT member user IDs,
# despite the variable name containing "USERS". Keep this in mind when configuring.
WEIXIN_GROUP_ALLOWED_USERS=group_id_1,group_id_2
```
:::note
The default group policy is `disabled` for Weixin (unlike WeCom where it defaults to `open`). This is intentional — personal WeChat accounts may be in many groups, and iLink bot identities typically can't receive ordinary WeChat group messages at all. The gateway logs a `WARNING` at startup if you set `WEIXIN_GROUP_POLICY` to anything other than `disabled`.
:::
## Media Support
### Inbound (receiving)
The adapter receives media attachments from users, downloads them from the WeChat CDN, decrypts them, and caches them locally for agent processing:
| Type | How it's handled |
|------|-----------------|
| **Images** | Downloaded, AES-decrypted, and cached as JPEG. |
| **Video** | Downloaded, AES-decrypted, and cached as MP4. |
| **Files** | Downloaded, AES-decrypted, and cached. Original filename is preserved. |
| **Voice** | If a text transcription is available, it's extracted as text. Otherwise the audio (SILK format) is downloaded and cached. |
**Quoted messages:** Media from quoted (replied-to) messages is also extracted, so the agent has context about what the user is replying to.
### AES-128-ECB Encrypted CDN
WeChat media files are transferred through an encrypted CDN. The adapter handles this transparently:
- **Inbound:** Encrypted media is downloaded from the CDN using `encrypted_query_param` URLs, then decrypted with AES-128-ECB using the per-file key provided in the message payload.
- **Outbound:** Files are encrypted locally with a random AES-128-ECB key, uploaded to the CDN, and the encrypted reference is included in the outbound message.
- The AES key is 16 bytes (128-bit). Keys may arrive as raw base64 or hex-encoded — the adapter handles both formats.
- This requires the `cryptography` Python package.
No configuration is needed — encryption and decryption happen automatically.
### Outbound (sending)
| Method | What it sends |
|--------|--------------|
| `send` | Text messages with Markdown formatting |
| `send_image` / `send_image_file` | Native image messages (via CDN upload) |
| `send_document` | File attachments (via CDN upload) |
| `send_video` | Video messages (via CDN upload) |
All outbound media goes through the encrypted CDN upload flow:
1. Generate a random AES-128 key
2. Encrypt the file with AES-128-ECB + PKCS#7 padding
3. Request an upload URL from the iLink API (`getuploadurl`)
4. Upload the ciphertext to the CDN
5. Send the message with the encrypted media reference
## Context Token Persistence
The iLink Bot API requires a `context_token` to be echoed back with each outbound message for a given peer. The adapter maintains a disk-backed context token store:
- Tokens are saved per account+peer to `~/.hermes/weixin/accounts/<account_id>.context-tokens.json`
- On startup, previously saved tokens are restored
- Every inbound message updates the stored token for that sender
- Outbound messages automatically include the latest context token
This ensures reply continuity even after gateway restarts.
## Markdown Formatting
WeChat clients connected through the iLink Bot API can render Markdown directly, so the adapter preserves Markdown instead of rewriting it:
- **Headers** stay as Markdown headings (`#`, `##`, ...)
- **Tables** stay as Markdown tables
- **Code fences** stay as fenced code blocks
- **Excessive blank lines** are collapsed to double newlines outside fenced code blocks
## Message Chunking
Messages are delivered as a single chat message whenever they fit within the platform limit. Only oversized payloads are split for delivery:
- Maximum message length: **4000 characters**
- Messages under the limit stay intact even when they contain multiple paragraphs or line breaks
- Oversized messages split at logical boundaries (paragraphs, blank lines, code fences)
- Code fences are kept intact whenever possible (never split mid-block unless the fence itself exceeds the limit)
- Oversized individual blocks fall back to the base adapter's truncation logic
- A 0.3 s inter-chunk delay prevents WeChat rate-limit drops when multiple chunks are sent
## Typing Indicators
The adapter shows typing status in the WeChat client:
1. When a message arrives, the adapter fetches a `typing_ticket` via the `getconfig` API
2. Typing tickets are cached for 10 minutes per user
3. `send_typing` sends a typing-start signal; `stop_typing` sends a typing-stop signal
4. The gateway automatically triggers typing indicators while the agent processes a message
## Long-Poll Connection
The adapter uses HTTP long-polling (not WebSocket) to receive messages:
### How It Works
1. **Connect:** Validates credentials and starts the poll loop
2. **Poll:** Calls `getupdates` with a 35-second timeout; the server holds the request until messages arrive or the timeout expires
3. **Dispatch:** Inbound messages are dispatched concurrently via `asyncio.create_task`
4. **Sync buffer:** A persistent sync cursor (`get_updates_buf`) is saved to disk so the adapter resumes from the correct position after restarts
### Retry Behavior
On API errors, the adapter uses a simple retry strategy:
| Condition | Behavior |
|-----------|----------|
| Transient error (1st–2nd) | Retry after 2 seconds |
| Repeated errors (3+) | Back off for 30 seconds, then reset counter |
| Session expired (`errcode=-14`) | Pause for 10 minutes (re-login may be needed) |
| Timeout | Immediately re-poll (normal long-poll behavior) |
### Deduplication
Inbound messages are deduplicated using message IDs with a 5-minute window. This prevents double-processing during network hiccups or overlapping poll responses.
### Token Lock
Only one Weixin gateway instance can use a given token at a time. The adapter acquires a scoped lock on startup and releases it on shutdown. If another gateway is already using the same token, startup fails with an informative error message.
## All Environment Variables
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `WEIXIN_ACCOUNT_ID` | ✅ | — | iLink Bot account ID (from QR login) |
| `WEIXIN_TOKEN` | ✅ | — | iLink Bot token (auto-saved from QR login) |
| `WEIXIN_BASE_URL` | — | `https://ilinkai.weixin.qq.com` | iLink API base URL |
| `WEIXIN_CDN_BASE_URL` | — | `https://novac2c.cdn.weixin.qq.com/c2c` | CDN base URL for media transfer |
| `WEIXIN_DM_POLICY` | — | `open` | DM access policy: `open`, `allowlist`, `disabled`, `pairing` |
| `WEIXIN_GROUP_POLICY` | — | `disabled` | Group access policy: `open`, `allowlist`, `disabled` |
| `WEIXIN_ALLOWED_USERS` | — | _(empty)_ | Comma-separated user IDs for DM allowlist |
| `WEIXIN_GROUP_ALLOWED_USERS` | — | _(empty)_ | Comma-separated **group chat IDs** (not member user IDs) for group allowlist. The variable name is legacy — it expects group IDs, not user IDs. |
| `WEIXIN_HOME_CHANNEL` | — | — | Chat ID for cron/notification output |
| `WEIXIN_HOME_CHANNEL_NAME` | — | `Home` | Display name for the home channel |
| `WEIXIN_ALLOW_ALL_USERS` | — | — | Gateway-level flag to allow all users (used by setup wizard) |
## Troubleshooting
| Problem | Fix |
|---------|-----|
| `Weixin startup failed: aiohttp and cryptography are required` | Install both: `pip install aiohttp cryptography` |
| `Weixin startup failed: WEIXIN_TOKEN is required` | Run `hermes gateway setup` to complete QR login, or set `WEIXIN_TOKEN` manually |
| `Weixin startup failed: WEIXIN_ACCOUNT_ID is required` | Set `WEIXIN_ACCOUNT_ID` in your `.env` or run `hermes gateway setup` |
| `Another local Hermes gateway is already using this Weixin token` | Stop the other gateway instance first — only one poller per token is allowed |
| Session expired (`errcode=-14`) | Your login session has expired. Re-run `hermes gateway setup` to scan a new QR code |
| QR code expired during setup | The QR auto-refreshes up to 3 times. If it keeps expiring, check your network connection |
| Bot doesn't respond to DMs | Check `WEIXIN_DM_POLICY` — if set to `allowlist`, the sender must be in `WEIXIN_ALLOWED_USERS` |
| Bot ignores group messages | Group policy defaults to `disabled`. Set `WEIXIN_GROUP_POLICY=open` or `allowlist` — but note that QR-login iLink bot identities (`...@im.bot`) typically cannot receive ordinary WeChat group messages at all. If the gateway logs show no raw inbound events for group messages, the limitation is on the iLink side, not in Hermes. |
| Media download/upload fails | Ensure `cryptography` is installed. Check network access to `novac2c.cdn.weixin.qq.com` |
| `Blocked unsafe URL (SSRF protection)` | The outbound media URL points to a private/internal address. Only public URLs are allowed |
| Voice messages show as text | If WeChat provides a transcription, the adapter uses the text. This is expected behavior |
| Messages appear duplicated | The adapter deduplicates by message ID. If you see duplicates, check if multiple gateway instances are running |
| `iLink POST ... HTTP 4xx/5xx` | API error from the iLink service. Check your token validity and network connectivity |
| Terminal QR code doesn't render | Reinstall with the messaging extra: `cd ~/.hermes/hermes-agent && uv pip install -e ".[messaging]"`. Alternatively, open the URL printed above the QR |
website/docs/user-guide/messaging/whatsapp-cloud.md
+417
View File
@@ -0,0 +1,417 @@
---
sidebar_position: 6
title: "WhatsApp Business (Cloud API)"
description: "Set up Hermes Agent as a WhatsApp bot via Meta's official Business Cloud API"
---
# WhatsApp Business Cloud API Setup
Hermes can connect to WhatsApp through Meta's **official** WhatsApp Business Cloud API. This is the production-grade path: no Node.js bridge subprocess, no QR codes, no account-ban risk.
In exchange:
- You need a **Meta Business account** (not personal WhatsApp).
- The bot operates on a dedicated business phone number, not your personal number.
- The Hermes gateway needs a **public HTTPS URL** so Meta can deliver inbound messages via webhook.
- Replies more than 24 hours after the user's last message require a pre-approved **template** (this is Meta's "customer service window" rule, not a Hermes limit).
If those constraints don't work for your use case, the [Baileys bridge integration](./whatsapp.md) is the alternative — personal account, no public URL needed, but unofficial and ban-prone.
:::tip Which one should I use?
- **Cloud API (this guide)** — running a real business bot, want stability, fine with the Meta verification + template paperwork
- **[Baileys bridge](./whatsapp.md)** — personal projects, quick demos, single-user setups, willing to risk the bot phone number's account
:::
---
## Quick start
```bash
hermes whatsapp-cloud
```
The wizard walks you through every credential, validates each one as you paste it (catches the #1 setup trap — pasting a phone number into the Phone Number ID field), and prints exact follow-up instructions for the parts that need to happen outside the wizard (starting cloudflared, configuring Meta's webhook dashboard).
The rest of this page is the manual reference.
---
## Prerequisites
1. **A Meta Business account**. Create one at [business.facebook.com](https://business.facebook.com/).
2. **A Meta app with WhatsApp enabled**. See "Creating the Meta app" below.
3. **A way to expose a local port to the public internet** with HTTPS. Cloudflare Tunnel (`cloudflared`) is recommended — free, no port forwarding, no domain required. ngrok, your own domain with a reverse proxy + TLS, or a VPS with the gateway directly bound to a public IP all work too.
4. **Optional but recommended**: ffmpeg on `PATH` so outbound voice messages render as native WhatsApp voice-note bubbles (green waveform) instead of MP3 audio attachments. Hermes degrades gracefully if absent.
---
## Creating the Meta app
1. Go to [developers.facebook.com/apps](https://developers.facebook.com/apps) → **Create App**.
2. Choose use case: **"Connect with customers through WhatsApp"** → **Next**.
3. Pick or create a business portfolio. Review the publishing requirements. Confirm → **Create app**.
4. After creation you'll land on **Customize use case → Connect on WhatsApp → Quickstart**. Click **Start using the API** → you're now on the **API Setup** page.
5. Make sure a WhatsApp Business Account (WABA) is linked. If you created a new portfolio in step 3, one was auto-created. Verify in the API Setup page.
You'll need these values from the dashboard — the wizard prompts for them in this order:
| Value | Where in dashboard | Field shape | Notes |
|---|---|---|---|
| **Phone Number ID** | App Dashboard → WhatsApp → API Setup → below the "From" dropdown | Numeric, 15-17 digits | **NOT** the phone number itself. The #1 setup mistake is pasting the actual phone number here. |
| **Access Token** | App Dashboard → WhatsApp → API Setup → "Generate access token" | Starts with `EAA`, 100+ chars | Temp tokens last 24h — see "Permanent token" below for production. |
| **App Secret** | App Dashboard → Settings → Basic → click "Show" next to App secret | 32-character lowercase hex | Used to verify incoming webhook signatures. Without it, inbound delivery is refused with 503. |
| **App ID** (optional) | App Dashboard → Settings → Basic | Numeric, 15-16 digits | Not required for messaging, useful for analytics. |
| **WABA ID** (optional) | App Dashboard → WhatsApp → API Setup → near the top | Numeric, 15+ digits | Not required for messaging, useful for analytics. |
---
## Permanent token (production)
Temporary access tokens expire after **24 hours**, which means a token generated today stops working tomorrow. For production deployments use a **System User permanent token**:
1. Go to [business.facebook.com/latest/settings](https://business.facebook.com/latest/settings) → **System users** (left sidebar).
2. **Add** → name (e.g. `hermes-bot`) → role: **Admin**.
3. Select the new user → **Assign Assets**:
- Select your app → toggle **Manage app** under Full control.
- Select your WhatsApp account → toggle **Manage WhatsApp Business Accounts** under Full control.
- Click **Assign assets**.
4. **Generate token** with these permissions:
- `business_management`
- `whatsapp_business_messaging`
- `whatsapp_business_management`
5. Set **token expiration: Never**.
6. Copy the token → update `WHATSAPP_CLOUD_ACCESS_TOKEN` in `~/.hermes/.env` → restart the gateway.
System User tokens don't expire unless you explicitly revoke them.
---
## Exposing Hermes to the internet
The Cloud API delivers inbound messages by HTTPS POST to your webhook URL — that means the Hermes gateway has to be reachable from Meta's servers. Three common ways:
### Cloudflare Tunnel (recommended)
Free, no port forwarding, works on Windows / macOS / Linux. Runs as a separate process alongside the gateway.
**Install:**
```bash
# Windows
winget install Cloudflare.cloudflared
# macOS
brew install cloudflared
# Linux
# Download the binary from https://github.com/cloudflare/cloudflared/releases
```
**Run a quick tunnel** (no Cloudflare account needed — gives you a `https://<random>.trycloudflare.com` URL):
```bash
cloudflared tunnel --url http://localhost:8090
```
Note the printed URL — that's what you'll give Meta.
:::warning Quick tunnels rotate
The free quick-tunnel URL changes every time you restart `cloudflared`. For a stable URL, log in with `cloudflared tunnel login` and create a named tunnel. Free Cloudflare accounts get unlimited named tunnels — see [Cloudflare's docs](https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/) for the named-tunnel workflow.
:::
### ngrok
```bash
ngrok http 8090
```
Free tier shows a different URL on each restart. Paid tier gives you a stable subdomain.
### Your own domain + reverse proxy
If you already have a server with a TLS cert (Caddy, nginx, etc.), point a route at `localhost:8090`. This is the most stable option for production but requires existing infrastructure.
---
## Configuring the webhook on Meta's side
Once your tunnel is running:
1. Note the public URL printed by your tunnel — say `https://abc123.trycloudflare.com`.
2. Generate a **Verify Token** — the wizard does this for you with `secrets.token_urlsafe(32)`; if you're configuring manually, run:
```bash
python -c "import secrets; print(secrets.token_urlsafe(32))"
```
Save it as `WHATSAPP_CLOUD_VERIFY_TOKEN` in `~/.hermes/.env`.
3. Start the Hermes gateway: `hermes gateway`.
4. In the Meta App Dashboard → **WhatsApp → Configuration** (or **Use cases → Customize → Configuration** depending on UI version) → click **Edit** on the Webhook section.
5. Fill in:
- **Callback URL**: `https://abc123.trycloudflare.com/whatsapp/webhook`
- **Verify Token**: the string from step 2 (must match exactly)
6. Click **Verify and save**. Meta hits your URL with a GET request, the gateway echoes back the challenge, and Meta marks the webhook as verified.
7. Under **Webhook fields**, click **Manage** → subscribe to the **messages** field. This is what tells Meta to actually deliver inbound messages to your webhook.
**To verify the loop manually** (from a third terminal):
```bash
TUNNEL="https://abc123.trycloudflare.com"
VERIFY="<your verify token>"
# Should print HTTP 200 with body "hello"
curl -i "$TUNNEL/whatsapp/webhook?hub.mode=subscribe&hub.verify_token=$VERIFY&hub.challenge=hello"
# Health endpoint — should show verify_token_configured: true and app_secret_configured: true
curl "$TUNNEL/health"
```
---
## Recipient whitelist (Meta-side)
In development mode (before your app goes through App Review), Meta restricts which numbers your bot can message:
1. App Dashboard → WhatsApp → API Setup → **To** dropdown.
2. Click **Manage phone number list**.
3. Add the phone numbers you want to message (yours, your team's, friendly testers). Meta sends each one a 6-digit verification code via SMS or WhatsApp.
Up to 5 numbers in dev mode. Going to App Review removes this limit.
---
## Allowlist (Hermes-side)
In addition to Meta's recipient whitelist, Hermes has its own per-platform allowlist that controls **which incoming messages the agent processes**. Add to `~/.hermes/.env`:
```bash
# Comma-separated phone numbers, country code, no '+' / spaces / dashes
WHATSAPP_CLOUD_ALLOWED_USERS=15551234567,15557654321
# Or allow everyone (only safe in combination with Meta's recipient whitelist)
# WHATSAPP_CLOUD_ALLOW_ALL_USERS=true
```
The wizard sets this in step 6. Without an allowlist, **every inbound message is denied** — this is intentional, so the bot can't be invoked by random numbers if the recipient whitelist is ever loosened.
---
## Polishing your bot's WhatsApp profile
WhatsApp displays a **name and profile picture** for your bot in the chat header and contact list. These can't be set via the Cloud API — they live in Meta's Business Manager.
Once your bot is working, head to **[business.facebook.com/wa/manage/phone-numbers](https://business.facebook.com/wa/manage/phone-numbers/)**, click your phone number, and you'll find:
| What | Where | Notes |
|---|---|---|
| **Display name** | Top of the phone-number page | Changes go through Meta's name-review process (~24–48 hours). |
| **Profile picture** | Top of the phone-number page | Square image, ≥640×640px recommended. Updates immediately. |
| **About / description / website / email / hours / category** | "Edit profile" button | These appear in the info pane when a user taps the bot's name. Cosmetic. |
| **Verified badge** (green checkmark) | Business Manager → Security Center → Start Verification | Requires Meta's separate business verification process. |
The `hermes whatsapp-cloud` wizard prints these links at the end of setup. None of this is required for the bot to work — it's pure polish for how your bot appears to users.
---
## Configuration reference
All settings live in `~/.hermes/.env`. Required values are in **bold**.
| Variable | Default | Description |
|---|---|---|
| **`WHATSAPP_CLOUD_PHONE_NUMBER_ID`** | — | The 15-17 digit ID from API Setup. **Not** the phone number. |
| **`WHATSAPP_CLOUD_ACCESS_TOKEN`** | — | Meta access token (starts with `EAA`). Temp 24h or System User permanent. |
| **`WHATSAPP_CLOUD_APP_SECRET`** | — | 32-char hex from Settings → Basic. Without it, inbound is refused with 503. |
| **`WHATSAPP_CLOUD_VERIFY_TOKEN`** | — | Shared secret for the GET handshake. Auto-generated by the wizard. |
| **`WHATSAPP_CLOUD_ALLOWED_USERS`** | — | Comma-separated wa_ids allowed to message the bot. |
| `WHATSAPP_CLOUD_ALLOW_ALL_USERS` | `false` | Set to `true` to bypass the allowlist. |
| `WHATSAPP_CLOUD_APP_ID` | — | Optional, for future analytics integration. |
| `WHATSAPP_CLOUD_WABA_ID` | — | Optional, for future analytics integration. |
| `WHATSAPP_CLOUD_WEBHOOK_HOST` | unset (dual-stack: all interfaces, IPv4+IPv6) | Interface the webhook server binds to. |
| `WHATSAPP_CLOUD_WEBHOOK_PORT` | `8090` | Port the webhook server binds to. Must match the port your tunnel forwards. |
| `WHATSAPP_CLOUD_WEBHOOK_PATH` | `/whatsapp/webhook` | URL path Meta posts to. |
| `WHATSAPP_CLOUD_API_VERSION` | `v20.0` | Meta Graph API version. Only override if a newer version is recommended in Meta's docs. |
| `WHATSAPP_CLOUD_HOME_CHANNEL` | — | wa_id to use as the bot's home channel (for cron jobs etc). |
You can have **both** the Baileys (`whatsapp`) and Cloud (`whatsapp_cloud`) adapters enabled simultaneously, targeting different phone numbers.
---
## Features
### Inbound
- **Text messages** — passed straight to the agent.
- **Images** — auto-downloaded and attached to the agent's input. Models with native vision (Claude, GPT-4o, Gemini, etc.) read the image directly; non-vision models receive an auto-generated text description.
- **Voice notes** — auto-downloaded as `.ogg`, transcribed via your configured STT provider (local faster-whisper, OpenAI/Nous, Groq, etc.), then handed to the agent as text.
- **Documents** — auto-downloaded. Small text-readable files (`.txt`, `.md`, `.json`, `.py`, `.csv`, etc.) up to 100KB get inlined into the agent's input so it can read them without a tool call. Larger files are cached locally for the agent's other tools to access.
- **Button taps** — when the user taps a button the bot sent earlier (clarify choice, command approval, slash-command confirm), the tap is routed directly to the right handler. Stale taps fall back to being treated as regular text input.
- **Reply context** — when the user replies to a previous bot message, the agent sees the original message as context.
### Outbound
- **Text** — markdown is auto-converted to WhatsApp's flavored syntax (`**bold**` → `*bold*`, `~~strike~~` → `~strike~`, headers → bold, `[link](url)` → `link (url)`). Long messages split at 4096 chars per chunk.
- **Images** — agent-generated images and local image files both supported, delivered as native photo attachments.
- **Voice messages** — text-to-speech output is converted via ffmpeg into the native WhatsApp voice-note bubble (green waveform). Without ffmpeg installed, falls back to an MP3 audio attachment. See "Voice messages" below.
- **Video / documents** — both supported, sent as native attachments.
### Interactive UX
When the agent invokes any of these flows, Hermes uses WhatsApp's native interactive messages — tap-to-answer buttons instead of "reply with the number" prompts:
- **`clarify` tool** — multi-choice questions render as quick-reply buttons (1–3 choices) or a tap-to-open list sheet (4+ choices). Picking "✏️ Other" lets the user type a free-form answer that the agent receives as the resolution.
- **Dangerous-command approvals** — when the agent's terminal/code execution hits a gated command, the user sees `✅ Approve` / `❌ Deny` buttons instead of needing to type `/approve` or `/deny`.
- **Slash-command confirmations** — privileged commands like `/reload-mcp` show `✅ Approve Once` / `🔒 Always` / `❌ Cancel` buttons.
All interactive prompts gracefully degrade to plain text if the buttons fail to render (e.g. on legacy WhatsApp clients).
### Read receipts and typing indicator
Hermes acknowledges inbound messages immediately:
- Your message shows **blue double-checkmarks** as soon as the gateway receives it.
- The bot's name in your WhatsApp chat shows **"typing…"** while the agent is preparing a reply.
- The typing indicator auto-dismisses when the bot's first response message arrives.
This makes it obvious when the bot has seen your message versus when it's still working on a response.
### Voice messages
WhatsApp distinguishes between a "voice note" (the green waveform bubble) and a generic audio file attachment. The difference is purely codec: voice notes need to be `audio/ogg` with `opus` encoding.
Hermes TTS produces MP3. Two paths:
- **With ffmpeg on PATH** (recommended) — outbound TTS is converted and arrives as a proper voice note. Install:
- Windows: `winget install Gyan.FFmpeg`
- macOS: `brew install ffmpeg`
- Linux: package manager
- **Without ffmpeg** — outbound TTS arrives as an MP3 audio attachment. Plays fine, just doesn't look like a voice note. A one-time warning fires in the gateway log so you know.
You can check whether the gateway found ffmpeg via the health endpoint:
```bash
curl http://localhost:8090/health
# look for "ffmpeg_present": true
```
---
## Known limitations
### 24-hour conversation window
Meta only allows **free-form messages** within a 24-hour window after the user's last inbound message. Outside that window, the only thing Meta's API accepts is a pre-approved **message template**.
**What this means in practice:**
- Reactive chat (user DMs → bot replies within 24h → user replies → ...) works forever. This covers >95% of normal bot use.
- **Cron jobs that deliver to WhatsApp** after a gap > 24h will fail with Graph error code `131047` ("Re-engagement message").
- **Long-running `delegate_task` async results** that take longer than 24h fail the same way.
- **Webhook subscribers** that route external events to WhatsApp fail when the user hasn't DM'd the bot recently.
Hermes warns the agent about this window in its system prompt, so the model knows to mention it when scheduling delayed messages.
Message-template support (the workaround for outside-window sends) is not yet implemented in Hermes. If you need it, please [open an issue](https://github.com/NousResearch/hermes-agent/issues) — it's planned but waiting on a clear demand signal.
### Group chats
The Cloud API has limited group support (capability-tier gated by Meta). Hermes's `whatsapp_cloud` adapter currently handles **direct messages only** in v1. If you need group chats, use the Baileys bridge.
### Outbound rate limit
Meta's default throughput is **80 messages/second per business phone number**, with upgrades available. Hermes doesn't currently enforce this client-side — extremely high-volume sends could hit Meta's limit.
---
## Troubleshooting
### Setup verification fails ("URL couldn't be validated") in Meta dashboard
Almost always one of:
- **Tunnel URL is wrong or stale** — cloudflared quick tunnels rotate. Get a fresh URL and update both `.env` and Meta's dashboard.
- **Verify token mismatch** — the token in `~/.hermes/.env`'s `WHATSAPP_CLOUD_VERIFY_TOKEN` must match exactly what you typed into Meta's dashboard. Run the curl probe above to confirm the gateway's verify handshake works locally first.
- **Gateway not running** — check `hermes gateway` is up.
- **App Secret not set** — without it, Hermes refuses inbound POSTs with 503. Meta interprets that as "can't validate."
### `graph error 100`: Object with ID '...' does not exist
You pasted your phone number (10-11 digits) into `WHATSAPP_CLOUD_PHONE_NUMBER_ID` instead of the Phone Number ID (Meta's 15-17 digit internal ID). Re-check the API Setup page — the Phone Number ID is shown *below* the "From" dropdown.
The wizard catches this with a validator now, but it's worth knowing if you're configuring manually.
### `graph error 190`: Authentication Error
Your access token is invalid. Subcodes:
- `subcode 463` — token expired. Temp tokens last 24h. Regenerate, or switch to a System User permanent token (see above).
- `subcode 467` — token invalidated (revoked or password changed).
- Other 190 — token didn't have the required permissions when generated. Make sure all three (`business_management`, `whatsapp_business_messaging`, `whatsapp_business_management`) were selected.
### `graph error 131047`: Re-engagement message
The 24-hour conversation window expired (see "Known limitations"). Either:
- Ask the user to DM the bot first to reopen the window.
- Wait for template support to land in Hermes.
### Inbound message: `media metadata fetch failed (status=401)`
Same 401 root causes as outbound (`graph error 190`) — the access token is invalid or expired. Fix the token.
### Bot replies appear as raw JSON / tool-call leakage
Common cause: the toolset configured for `whatsapp_cloud` is missing the tools the agent wants to call. Check `hermes tools list` and verify the platform is using `hermes-whatsapp` (the default Cloud adapter toolset, same as Baileys).
If the model emits tool-call-shaped text instead of a structured call, it usually means the toolset was effectively empty. See `hermes_cli/platforms.py` for the platform → default toolset mapping.
### STT (voice note transcription) returns empty / "could not transcribe"
The default `stt.provider: local` requires `pip install faster-whisper`. If you're a Nous subscriber, you can route STT through the managed gateway instead — select **Nous Subscription** for speech-to-text in `hermes tools`, or set it directly:
```bash
hermes config set stt.provider nous
hermes gateway restart
```
This uses your Nous Portal access token instead of needing a separate OpenAI key. (Older docs suggested `stt.use_gateway true` — that flag is legacy; the provider selection alone controls routing now.)
---
## Security notes
- **Treat the App Secret like a password** — anyone with it can forge webhook payloads that Hermes will accept as authentic.
- **The verify token is a shared secret** — leaks are lower-stakes (worst case someone could re-subscribe Meta's webhook to a different URL of theirs), but still avoid committing it.
- **The access token is your bot's identity** — System User tokens are equivalent to long-lived API keys. Rotate immediately if a deployment is compromised.
- **The webhook endpoint accepts only signed requests when `WHATSAPP_CLOUD_APP_SECRET` is set** — leave it set even in development. Without it, the gateway refuses inbound delivery with HTTP 503.
- **The `/health` endpoint is unauthenticated** — it's safe to expose because it only reports config-presence booleans, not the values themselves. But if you'd rather not surface it, restrict access at the reverse proxy / tunnel layer.
---
## Comparison to the Baileys bridge
| | Baileys (`hermes whatsapp`) | Cloud API (`hermes whatsapp-cloud`) |
|---|---|---|
| Account type | Personal | Business |
| Setup | QR code scan | Meta app + WABA + token |
| Dependencies | Node.js + npm | Pure Python (httpx + aiohttp) |
| Process | Managed Node subprocess | aiohttp webhook server |
| Public URL needed? | No | Yes |
| Account ban risk | Yes (unofficial API) | No (officially supported) |
| Inbound | Polling Node bridge | Webhook POST from Meta |
| Outbound | Local bridge → Baileys | HTTPS to graph.facebook.com |
| Groups | Full support | DMs only (v1) |
| 24h window | No restriction | Hard rule — templates required after |
| Voice notes (out) | Native | Native with ffmpeg, MP3 fallback otherwise |
| Read receipts | No | Yes (blue double-checkmarks) |
| Typing indicator | No | Yes (auto-dismisses on response) |
| Interactive buttons | Text fallback only | Native (clarify, approval, slash-confirm) |
| Production use | Risky (Meta can ban) | Designed for it |
Most users running Hermes for personal projects prefer Baileys. Most users running customer-facing bots prefer Cloud API.
---
## See also
- [Meta's official WhatsApp Business Cloud API docs](https://developers.facebook.com/documentation/business-messaging/whatsapp/) — authoritative reference for the underlying platform, pricing, App Review, and Meta-side rate limits.
- [WhatsApp (Baileys bridge) Setup](whatsapp.md) — the alternative integration for personal projects.
- [Messaging Platforms overview](index.md) — all messaging integrations at a glance.
Loaded 100 of 11340 files, more files were not shown because too many files have changed in this diff. Show more