Computer Use
Searches accessibility nodes within the requested scope.
application/json- body
Accessibility find request.
limit?|Maximum number of matches. Omitted or <=0 uses the platform default.
int64name?stringName filter. Omitted = any name.
nameMatch?stringName match mode: exact, substring, or regex.
pid?|Owning process id when scope=pid.
int64role?stringExact role filter. Omitted = any role.
scope?stringScope selector: omitted/focused, pid, or Linux all/Windows desktop aliases.
states?array<string>Required state names.
application/json- response
Accessibility find response.
matches*array<>Matching nodes in traversal order.
truncated*booleanWhether traversal stopped because of node or wall-clock limits.
curl -X POST 'http://sandbox:2280/computeruse/a11y/find' \ -H 'Content-Type: application/json' \ -d '{}'{ "matches": [ { "actions": [ "string" ], "bounds": { "height": 0, "width": 0, "x": 0, "y": 0 }, "children": [ {} ], "description": "string", "id": "string", "name": "string", "role": "string", "states": [ "string" ], "value": "string", "valueTruncated": true } ], "truncated": true}Moves desktop focus to the requested accessibility node.
application/json- body
Accessibility node request.
action?|Optional action name for invoke requests.
id*stringThe stable node id returned by tree/find.
value?|Optional text value for editable nodes.
Node focused
curl -X POST 'http://sandbox:2280/computeruse/a11y/node/focus' \ -H 'Content-Type: application/json' \ -d '{ "id": "string"}'Invokes the requested accessibility node action.
application/json- body
Accessibility node request.
action?|Optional action name for invoke requests.
id*stringThe stable node id returned by tree/find.
value?|Optional text value for editable nodes.
Node invoked
curl -X POST 'http://sandbox:2280/computeruse/a11y/node/invoke' \ -H 'Content-Type: application/json' \ -d '{ "id": "string"}'Sets the editable value of the requested accessibility node.
application/json- body
Accessibility node request.
action?|Optional action name for invoke requests.
id*stringThe stable node id returned by tree/find.
value?|Optional text value for editable nodes.
Node value set
curl -X POST 'http://sandbox:2280/computeruse/a11y/node/value' \ -H 'Content-Type: application/json' \ -d '{ "id": "string"}'Returns an accessibility tree rooted at the requested scope.
scope?stringScope selector: omitted/focused, pid, or Linux all/Windows desktop aliases.
pid?integerOwning process id when scope=pid.
int64maxDepth?integerMaximum tree depth. Omitted or -1 means unbounded.
int64application/json- response
Accessibility tree response.
root*The selected root node.
truncated*booleanWhether traversal stopped because of node or wall-clock limits.
curl -X GET 'http://sandbox:2280/computeruse/a11y/tree'{ "root": { "actions": [ "string" ], "bounds": { "height": 0, "width": 0, "x": 0, "y": 0 }, "children": [ {} ], "description": "string", "id": "string", "name": "string", "role": "string", "states": [ "string" ], "value": "string", "valueTruncated": true }, "truncated": true}Returns the current display/monitor list.
application/json- response
Display inventory response.
displays*array<>The current display list.
curl -X GET 'http://sandbox:2280/computeruse/display/info'{ "displays": [ { "height": 0, "id": 0, "isActive": true, "width": 0, "x": 0, "y": 0 } ]}Returns the current top-level window list.
application/json- response
Window inventory response.
windows*array<>The current top-level window list.
curl -X GET 'http://sandbox:2280/computeruse/display/windows'{ "windows": [ { "height": 0, "id": 0, "isActive": true, "title": "string", "width": 0, "x": 0, "y": 0 } ]}Presses a +-delimited hotkey chord.
application/json- body
Keyboard hotkey request (ctrl+shift+p).
keys*string+-delimited chord string.
Hotkey pressed
curl -X POST 'http://sandbox:2280/computeruse/keyboard/hotkey' \ -H 'Content-Type: application/json' \ -d '{ "keys": "string"}'Presses one key with optional modifiers.
application/json- body
Keyboard key request.
key*stringThe main key name.
modifiers?array<string>Modifier key names held while pressing key.
Key pressed
curl -X POST 'http://sandbox:2280/computeruse/keyboard/key' \ -H 'Content-Type: application/json' \ -d '{ "key": "string"}'Types UTF-8 text into the desktop.
application/json- body
Keyboard typing request.
delay?integerDelay in milliseconds between key presses. Omitted = 0.
int640 <= valuetext*stringUTF-8 text to type.
Text typed
curl -X POST 'http://sandbox:2280/computeruse/keyboard/type' \ -H 'Content-Type: application/json' \ -d '{ "text": "string"}'Moves the cursor and clicks the requested mouse button.
application/json- body
Mouse click request.
button?stringButton name (left, middle, right). Omitted = left.
double?booleanWhether to double-click instead of single-clicking.
x*integerTarget X coordinate in desktop pixels.
int64y*integerTarget Y coordinate in desktop pixels.
int64application/json- response
A point in desktop pixels.
x*integerHorizontal desktop coordinate. i32 so every platform's native
coordinate fits (X11 reports i16, Win32 LONG).
int32y*integerVertical desktop coordinate.
int32curl -X POST 'http://sandbox:2280/computeruse/mouse/click' \ -H 'Content-Type: application/json' \ -d '{ "x": 0, "y": 0}'{ "x": 0, "y": 0}Presses a mouse button, drags between two coordinates, and releases it.
application/json- body
Mouse drag request.
button?stringButton name (left, middle, right). Omitted = left.
endX*integerDrag end X coordinate in desktop pixels.
int64endY*integerDrag end Y coordinate in desktop pixels.
int64startX*integerDrag start X coordinate in desktop pixels.
int64startY*integerDrag start Y coordinate in desktop pixels.
int64application/json- response
A point in desktop pixels.
x*integerHorizontal desktop coordinate. i32 so every platform's native
coordinate fits (X11 reports i16, Win32 LONG).
int32y*integerVertical desktop coordinate.
int32curl -X POST 'http://sandbox:2280/computeruse/mouse/drag' \ -H 'Content-Type: application/json' \ -d '{ "endX": 0, "endY": 0, "startX": 0, "startY": 0}'{ "x": 0, "y": 0}Moves the cursor to the requested desktop coordinates.
application/json- body
Mouse move request.
x*integerTarget X coordinate in desktop pixels.
int64y*integerTarget Y coordinate in desktop pixels.
int64application/json- response
A point in desktop pixels.
x*integerHorizontal desktop coordinate. i32 so every platform's native
coordinate fits (X11 reports i16, Win32 LONG).
int32y*integerVertical desktop coordinate.
int32curl -X POST 'http://sandbox:2280/computeruse/mouse/move' \ -H 'Content-Type: application/json' \ -d '{ "x": 0, "y": 0}'{ "x": 0, "y": 0}Returns the current cursor position.
application/json- response
A point in desktop pixels.
x*integerHorizontal desktop coordinate. i32 so every platform's native
coordinate fits (X11 reports i16, Win32 LONG).
int32y*integerVertical desktop coordinate.
int32curl -X GET 'http://sandbox:2280/computeruse/mouse/position'{ "x": 0, "y": 0}Moves the cursor and scrolls the wheel at the requested position.
application/json- body
Mouse scroll request.
amount?|Number of wheel steps. Omitted = 1.
int64direction*stringScroll direction (up or down).
x*integerTarget X coordinate in desktop pixels.
int64y*integerTarget Y coordinate in desktop pixels.
int64application/json- response
Scroll success response.
success*booleanAlways true when the request succeeds.
curl -X POST 'http://sandbox:2280/computeruse/mouse/scroll' \ -H 'Content-Type: application/json' \ -d '{ "direction": "string", "x": 0, "y": 0}'{ "success": true}Returns the state of every supervised desktop process.
application/json- response
The process-status map keyed by process name.
[key: string]?One supervised computer-use process.
curl -X GET 'http://sandbox:2280/computeruse/process-status'{ "property1": { "AutoRestart": true, "Failed": true, "Pid": 0, "Priority": 0, "RestartAttempts": 0, "RestartBackoffMs": 0, "Running": true, "Status": "string" }, "property2": { "AutoRestart": true, "Failed": true, "Pid": 0, "Priority": 0, "RestartAttempts": 0, "RestartBackoffMs": 0, "Running": true, "Status": "string" }}Returns retained stderr/error logs for one supervised desktop process.
name*stringDesktop process name
application/json- response
The stderr/errors of one named process.
errors*stringThe retained stderr/error log text.
processName*stringThe requested process name.
curl -X GET 'http://sandbox:2280/computeruse/process/string/errors'{ "errors": "string", "processName": "string"}Returns retained stdout logs for one supervised desktop process.
name*stringDesktop process name
application/json- response
The logs of one named process.
logs*stringThe retained stdout log text.
processName*stringThe requested process name.
curl -X GET 'http://sandbox:2280/computeruse/process/string/logs'{ "logs": "string", "processName": "string"}Restarts one supervised desktop process when the desktop is ready.
name*stringDesktop process name
application/json- response
The response returned by POST /computeruse/process/{name}/restart.
message*stringStable success text.
processName*stringThe restarted process name.
curl -X POST 'http://sandbox:2280/computeruse/process/string/restart'{ "message": "string", "processName": "string"}Returns the running state of one supervised desktop process.
name*stringDesktop process name
application/json- response
The running state of one named process.
processName*stringThe requested process name.
running*booleanWhether the named process is currently alive.
curl -X GET 'http://sandbox:2280/computeruse/process/string/status'{ "processName": "string", "running": true}Returns live and retained recording metadata.
application/json- response
Recording list response.
recordings*array<>The current and retained recordings.
curl -X GET 'http://sandbox:2280/computeruse/recordings'{ "recordings": [ { "durationSeconds": 0, "endTime": "string", "fileName": "string", "filePath": "string", "id": "string", "sizeBytes": 0, "startTime": "string", "status": "string" } ]}Starts a desktop recording and returns its metadata.
application/json- body
Recording start request.
label?|Optional label used as the filename prefix.
application/json- response
Recording metadata.
durationSeconds?|Duration in seconds when known.
doubleendTime?|Recording end time in RFC 3339 format, omitted while live.
fileName*stringDownload filename.
filePath*stringAbsolute path on the addon host.
id*stringRecording id.
sizeBytes?|Encoded file size in bytes when known.
int64startTime*stringRecording start time in RFC 3339 format.
status*stringLifecycle status (recording, stopping, completed, failed, ...).
curl -X POST 'http://sandbox:2280/computeruse/recordings/start' \ -H 'Content-Type: application/json' \ -d '{}'{ "durationSeconds": 0, "endTime": "string", "fileName": "string", "filePath": "string", "id": "string", "sizeBytes": 0, "startTime": "string", "status": "string"}Stops a recording and returns its terminal metadata.
application/json- body
Recording stop request.
id*stringThe recording id.
application/json- response
Recording metadata.
durationSeconds?|Duration in seconds when known.
doubleendTime?|Recording end time in RFC 3339 format, omitted while live.
fileName*stringDownload filename.
filePath*stringAbsolute path on the addon host.
id*stringRecording id.
sizeBytes?|Encoded file size in bytes when known.
int64startTime*stringRecording start time in RFC 3339 format.
status*stringLifecycle status (recording, stopping, completed, failed, ...).
curl -X POST 'http://sandbox:2280/computeruse/recordings/stop' \ -H 'Content-Type: application/json' \ -d '{ "id": "string"}'{ "durationSeconds": 0, "endTime": "string", "fileName": "string", "filePath": "string", "id": "string", "sizeBytes": 0, "startTime": "string", "status": "string"}Returns metadata for one recording.
id*stringRecording id
application/json- response
Recording metadata.
durationSeconds?|Duration in seconds when known.
doubleendTime?|Recording end time in RFC 3339 format, omitted while live.
fileName*stringDownload filename.
filePath*stringAbsolute path on the addon host.
id*stringRecording id.
sizeBytes?|Encoded file size in bytes when known.
int64startTime*stringRecording start time in RFC 3339 format.
status*stringLifecycle status (recording, stopping, completed, failed, ...).
curl -X GET 'http://sandbox:2280/computeruse/recordings/string'{ "durationSeconds": 0, "endTime": "string", "fileName": "string", "filePath": "string", "id": "string", "sizeBytes": 0, "startTime": "string", "status": "string"}Deletes a recording and its retained file.
id*stringRecording id
Recording deleted
curl -X DELETE 'http://sandbox:2280/computeruse/recordings/string'Streams one recording file as video/mp4.
id*stringRecording id
Recording file stream
video/mp4curl -X GET 'http://sandbox:2280/computeruse/recordings/string/download'Replaces the current desktop stack and returns the ready desktop status projection.
application/json- response
The response returned by POST /computeruse/restart.
desktop*ComputerUseDesktopStateThe explicit desktop state after the restart succeeds.
"not_started""starting""ready""failed"message*stringStable success text.
status*ComputerUseActivityStatusLegacy compatibility state (active after a successful restart).
"active""inactive"curl -X POST 'http://sandbox:2280/computeruse/restart'{ "desktop": "not_started", "message": "string", "status": "active"}Captures the full desktop and returns base64-encoded image bytes in JSON.
show_cursor?stringWhether to include the current cursor position in the JSON response.
showCursor?stringLegacy camelCase alias for show_cursor, accepted but omitted from the schema.
format?stringImage format for compressed screenshot routes. Omitted = png.
quality?stringJPEG quality for compressed screenshot routes. Omitted = 85.
scale?stringDownscale factor for compressed screenshot routes. Omitted = 1.0.
x?stringRegion left coordinate for region routes.
y?stringRegion top coordinate for region routes.
width?stringRegion width for region routes. Must be greater than zero.
height?stringRegion height for region routes. Must be greater than zero.
application/json- response
Screenshot response.
cursorPosition?null|screenshot*stringBase64-encoded PNG or JPEG bytes.
sizeBytes*integerSize of the encoded image in bytes.
0 <= valuecurl -X GET 'http://sandbox:2280/computeruse/screenshot'{ "cursorPosition": null, "screenshot": "string", "sizeBytes": 0}Captures the full desktop and applies format, quality, and scale controls before returning JSON.
show_cursor?stringWhether to include the current cursor position in the JSON response.
showCursor?stringLegacy camelCase alias for show_cursor, accepted but omitted from the schema.
format?stringImage format for compressed screenshot routes. Omitted = png.
quality?stringJPEG quality for compressed screenshot routes. Omitted = 85.
scale?stringDownscale factor for compressed screenshot routes. Omitted = 1.0.
x?stringRegion left coordinate for region routes.
y?stringRegion top coordinate for region routes.
width?stringRegion width for region routes. Must be greater than zero.
height?stringRegion height for region routes. Must be greater than zero.
application/json- response
Screenshot response.
cursorPosition?null|screenshot*stringBase64-encoded PNG or JPEG bytes.
sizeBytes*integerSize of the encoded image in bytes.
0 <= valuecurl -X GET 'http://sandbox:2280/computeruse/screenshot/compressed'{ "cursorPosition": null, "screenshot": "string", "sizeBytes": 0}Captures a rectangular desktop region and returns base64-encoded image bytes in JSON.
show_cursor?stringWhether to include the current cursor position in the JSON response.
showCursor?stringLegacy camelCase alias for show_cursor, accepted but omitted from the schema.
format?stringImage format for compressed screenshot routes. Omitted = png.
quality?stringJPEG quality for compressed screenshot routes. Omitted = 85.
scale?stringDownscale factor for compressed screenshot routes. Omitted = 1.0.
x?stringRegion left coordinate for region routes.
y?stringRegion top coordinate for region routes.
width?stringRegion width for region routes. Must be greater than zero.
height?stringRegion height for region routes. Must be greater than zero.
application/json- response
Screenshot response.
cursorPosition?null|screenshot*stringBase64-encoded PNG or JPEG bytes.
sizeBytes*integerSize of the encoded image in bytes.
0 <= valuecurl -X GET 'http://sandbox:2280/computeruse/screenshot/region'{ "cursorPosition": null, "screenshot": "string", "sizeBytes": 0}Captures a rectangular desktop region and applies format, quality, and scale controls before returning JSON.
show_cursor?stringWhether to include the current cursor position in the JSON response.
showCursor?stringLegacy camelCase alias for show_cursor, accepted but omitted from the schema.
format?stringImage format for compressed screenshot routes. Omitted = png.
quality?stringJPEG quality for compressed screenshot routes. Omitted = 85.
scale?stringDownscale factor for compressed screenshot routes. Omitted = 1.0.
x?stringRegion left coordinate for region routes.
y?stringRegion top coordinate for region routes.
width?stringRegion width for region routes. Must be greater than zero.
height?stringRegion height for region routes. Must be greater than zero.
application/json- response
Screenshot response.
cursorPosition?null|screenshot*stringBase64-encoded PNG or JPEG bytes.
sizeBytes*integerSize of the encoded image in bytes.
0 <= valuecurl -X GET 'http://sandbox:2280/computeruse/screenshot/region/compressed'{ "cursorPosition": null, "screenshot": "string", "sizeBytes": 0}Starts the desktop stack and returns the supervised-process inventory.
application/json- response
The response returned by lifecycle operations that start or stop the stack.
message*stringStable success text.
status*The post-operation process inventory.
curl -X POST 'http://sandbox:2280/computeruse/start'{ "message": "string", "status": { "property1": { "AutoRestart": true, "Failed": true, "Pid": 0, "Priority": 0, "RestartAttempts": 0, "RestartBackoffMs": 0, "Running": true, "Status": "string" }, "property2": { "AutoRestart": true, "Failed": true, "Pid": 0, "Priority": 0, "RestartAttempts": 0, "RestartBackoffMs": 0, "Running": true, "Status": "string" } }}Returns the explicit desktop lifetime state and the legacy compatibility status field.
application/json- response
The public desktop status surface.
cause?null|desktop*ComputerUseDesktopStateThe explicit desktop lifetime state.
"not_started""starting""ready""failed"status*ComputerUseActivityStatusLegacy compatibility state derived from desktop.
"active""inactive"curl -X GET 'http://sandbox:2280/computeruse/status'{ "cause": null, "desktop": "not_started", "status": "active"}Stops the desktop stack and returns the final supervised-process inventory.
application/json- response
The response returned by lifecycle operations that start or stop the stack.
message*stringStable success text.
status*The post-operation process inventory.
curl -X POST 'http://sandbox:2280/computeruse/stop'{ "message": "string", "status": { "property1": { "AutoRestart": true, "Failed": true, "Pid": 0, "Priority": 0, "RestartAttempts": 0, "RestartBackoffMs": 0, "Running": true, "Status": "string" }, "property2": { "AutoRestart": true, "Failed": true, "Pid": 0, "Priority": 0, "RestartAttempts": 0, "RestartBackoffMs": 0, "Running": true, "Status": "string" } }}