DEVELOPMENT DOCUMENTATION / COMPUTER USE
Desktop operations behind an explicit policy.
The computer.v1 adapter targets a locally supervised CUA Computer Server. It is not an unrestricted proxy to the server's entire API.
Pinned backend
The reviewed backend is cua-computer-server 0.3.46, a Python sidecar. The profile pins its /cmd command surface. CUA Driver is an optional backend of that server, not a separately supported Rust or stdio service.
Capability mapping
| Capability | Backend command family | Boundary |
|---|---|---|
| Screen capture | screenshot | Bounded image data and dimensions; no live-video guarantee. |
| Screen and cursor state | get_screen_size, get_cursor_position | Backend-specific capabilities must be discovered. |
| Pointer input | left_click, right_click, double_click, move_cursor, drag, scroll | Explicit permission and exclusive input-controller policy. |
| Keyboard input | type_text, press_key, hotkey | Layout and backend parameter semantics require real-backend verification. |
| Accessibility | get_accessibility_tree | Only when the backend actually supplies supported data. |
Parameter compatibility
Command and parameter names are now pinned against the released backend source. Drag is sent as a two-point path; scroll uses x and y as wheel amounts, not cursor coordinates. These are source-backed compatibility corrections, not a real-backend test result.
Known backend limitations
- Display selection: the pinned backend does not accept the requested display index. Do not rely on multi-display targeting.
- Scroll location: the adapter validates a point but cannot express scroll-at-point through the pinned backend command. Direction and units also require backend-specific verification.
- Capture identity: the released screenshot response does not provide the width, height and scale fields expected by the current capture-identity path. Without an identity, subsequent coordinate-based input is refused. Synthetic capture fixtures do not prove this works with the released backend.
Excluded surfaces
The initial profile does not expose arbitrary shell commands, host file APIs, clipboard, PTY, Playwright execution, the backend's MCP server or its responses API. Filesystem access has its own confined service.
Response semantics
The pinned /cmd implementation can return HTTP 200 with a failure payload. Its response uses data-framed JSON with a text/plain media type, so callers must not infer success from the status code alone or assume ordinary JSON.
Input is a side effect
A lost click response does not mean the click never happened. Never repeat input automatically after reconnect. Capability discovery must intersect backend support, local export policy and the caller's grant. Synthetic fixtures do not prove real parameter schemas, permission prompts, display scaling or focus behavior.
Source & review
Reviewed against eb2422008a3a on 2026-09-22. Based on docs/integrations.md, docs/tasks.md, crates/tunnel-cua, crates/tunnel-cua-export. Historical test evidence is not a new test run.