Screenshot, click, type or scroll
POST /v1/sandboxes/{sandbox}/computer
In this build: Works on the simulator or a real VM. Needs a real machine. The simulator returns placeholder results.
Scope: desktop
CLI: harebox computer
| Field | Type | Notes |
|---|---|---|
action (required) |
string |
One of: screenshot, click, double_click, right_click, move, drag, type, key, scroll, open_app, open_url, accessibility_tree |
x |
integer |
|
y |
integer |
|
text |
string |
|
keys |
array |
|
delta |
integer |
Scroll steps: positive scrolls down, negative up. |
to_x |
integer |
Where a drag ends. |
to_y |
integer |
|
url |
string |
For open_url: an http or https address to open in Chrome. |
ComputerResult
| Field | Type | Notes |
|---|---|---|
ok |
boolean |
|
screenshot_url |
string or null |
|
width |
integer |
|
height |
integer |
|
screenshot_png |
string or null |
The screenshot as base64 PNG, for screenshot actions. |
accessibility_tree |
array or null |
For accessibility_tree: each node's role, name, bounds and children. |
Errors follow one format.