feat(chrome): let the IDE agent use the user's Chrome and beta.42
Server CI and Windows package / verify (push) Successful in 3m33s
Server CI and Windows package / windows-package (push) Successful in 7m36s

The AITURK Chrome extension now registers the user's Chrome as the desktop
backend's browser controller, the way the ChatGPT extension lets its agent
work in the user's browser. The agent's existing browser tools (navigate,
snapshot, click, type, scroll, back, press, get_images, vision) run in its
own background tabs grouped as "AITURK", or a tab the user hands over.

Backend: a local desktop lane in the browser-control broker. One controller
registered from the loopback desktop transport is offered to every desktop
session as a lane template; each session still gets its own exact scope on
first use. The lane has its own switch (browser.extension_control.local_desktop,
on by default); with no extension registered, or after it disconnects, the
normal browser backend runs unchanged. browser_vision turns the extension's
capture into the usual native/auxiliary vision result.

Bridge: forwards controller commands only to a connection that registered,
refuses unknown or oversized commands, relays results.

Extension: service-worker controller with its own native connection,
trusted chrome.debugger input (press/release only: a hover move waits 5 s on
background tabs), refusal of password and payment-card fields and of
browser, extension, file and Web Store pages, panel switch, Stop, tab
hand-over, recent actions, Alt+Shift+A and right-click "send to AITURK".
Cancelling Chrome's debugging bar stops the agent. Existing panel flows and
privacy rules are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
yilsemandClaude Opus 5.5 committed 2026-10-15 08:38:52 +03:00
1 parent 5c76c24c9c
commit 86a6621936
32 files changed
+2755 -195

No files matched your search

+17 -2
View File
@@ -284,6 +284,7 @@ protocol version, transport names, and the exact capability allowlist:
controller.noop
browser_back
browser_click
browser_get_images
browser_navigate
browser_press
browser_screenshot
@@ -292,11 +293,25 @@ browser_snapshot
browser_tab_activate
browser_tabs
browser_type
browser_vision
```
Requested capabilities outside that list are filtered out. Raw CDP, arbitrary
script evaluation, console access, uploads, image extraction, and vision are not
part of the controller protocol.
script evaluation, console access and uploads are not part of the controller
protocol. For `browser_vision` the controller only returns a capture of the
visible tab; Hermes saves it and runs the usual vision analysis itself.
### Desktop browser lane
The desktop app's own browser extension (AITURK Chrome) registers through the
app's native bridge with `browser.controller.local.register` on the loopback
desktop WebSocket. One registration serves every desktop session of that
backend; each session still gets its own exact controller scope on first use,
and command events carry that session id. The lane has its own switch,
`browser.extension_control.local_desktop` (on by default), independent of
`enabled`. When the extension disconnects or calls
`browser.controller.local.detach`, pending commands are cancelled and desktop
sessions return to their normal browser backend.
When a request has no bound controller identity, or when the feature is disabled,
Hermes preserves the existing browser backend. Once the gateway binds a