---
site: steel.dev
collection: docs
canonical_url: https://steel.dev/docs/integrations/claude-computer-use
markdown_url: https://steel.dev/docs/integrations/claude-computer-use.md
content_signal: ai-input=yes, ai-train=yes, search=yes
title: Run Claude computer use on Steel
description: Connect Claude's computer-use tool to a Steel session, so the model sees screenshots and controls the browser.
summary: Connect Claude's computer-use tool to a Steel session, so the model sees screenshots and controls the browser.
type: integration-guide
reviewed: 2026-11-12
prerequisites: []
related:
  - https://steel.dev/docs/integrations/openai-computer-use.md
  - https://steel.dev/docs/integrations/gemini-computer-use.md
agent_setup: https://setup.steel.dev/install.md
docs_index: https://steel.dev/docs/sitemap.md
---

# Run Claude computer use on Steel

On the current Claude models, computer use is the `computer_toolset_20260801` toolset. With it, Claude asks for actions, such as a click at pixel coordinates or a key press, often several in one response. It asks for a screenshot only when it needs to see the screen. Your program runs each action in a Steel session with [Computer actions](https://steel.dev/docs/control/computer-actions) and returns each result to Claude.

Earlier models, such as Claude Opus 4.7, use the `computer_20251124` tool, which has different requests and results. Anthropic's [computer use documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool) lists the tool for each model.

To route the session through a [Steel proxy](https://steel.dev/docs/stealth/proxies) or to [solve CAPTCHAs](https://steel.dev/docs/stealth/captcha-solving), set those options when you create the session.

## Before you begin

- A Steel API key in the `STEEL_API_KEY` environment variable. See [API keys](https://steel.dev/docs/account/api-keys).
- An Anthropic API key in the `ANTHROPIC_API_KEY` environment variable.
- Node.js 20 or later, or Python 3.11 or later.

Install the Steel and Anthropic SDKs:

**TypeScript**

```sh
npm install steel-sdk @anthropic-ai/sdk
```

**Python**

```sh
python3 -m venv .venv
source .venv/bin/activate
pip install steel-sdk anthropic
```

## Set up the integration

### Connect Claude to a session

This program creates a session, gives Claude a task, and runs each action that Claude asks for in the session, in order. It stops when Claude returns text and no action, and it releases the session in `finally`.

**TypeScript**

```ts file=claude-computer-use.mts
import Anthropic from "@anthropic-ai/sdk";
import Steel from "steel-sdk";

const TASK = "Open https://example.com and tell me the title of the page.";
const HALTED = "Not executed: an earlier computer action in this turn failed.";
const BUTTONS = {
  left_click: "left",
  right_click: "right",
  middle_click: "middle",
  double_click: "left",
} as const;
const WHEEL: Record<string, [number, number]> = {
  down: [0, 1],
  up: [0, -1],
  right: [1, 0],
  left: [-1, 0],
};

const steel = new Steel();
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

function toSteelActions(
  name: string,
  input: any,
  at: number[],
): Steel.SessionComputerParams[] {
  if (input.text && name !== "type" && name !== "key") {
    throw new Error("Steel holds no modifier key during a click or a scroll.");
  }
  switch (name) {
    case "screenshot":
      return [{ action: "take_screenshot" }];
    case "cursor_position":
      return [{ action: "get_cursor_position" }];
    case "left_click":
    case "right_click":
    case "middle_click":
    case "double_click":
      return [
        {
          action: "click_mouse",
          button: BUTTONS[name],
          num_clicks: name === "double_click" ? 2 : 1,
          coordinates: at,
        },
      ];
    case "left_click_drag":
      return [{ action: "drag_mouse", path: [input.start_coordinate, at] }];
    case "mouse_move":
      return [{ action: "move_mouse", coordinates: at }];
    case "scroll": {
      const [delta_x, delta_y] = WHEEL[input.scroll_direction];
      return Array(input.scroll_amount).fill({
        action: "scroll",
        coordinates: at,
        delta_x,
        delta_y,
      });
    }
    case "type":
      return [{ action: "type_text", text: input.text }];
    case "key":
      return Array(input.repeat ?? 1).fill({
        action: "press_key",
        keys: input.text.split("+"),
      });
    case "wait":
      return [{ action: "wait", duration: input.duration }];
    default:
      throw new Error(`Steel has no action for ${name}.`);
  }
}

const session = await steel.sessions.create({
  dimensions: { width: 1280, height: 768 },
  timeout: 900000,
});
console.log("Session viewer:", session.sessionViewerUrl);

const messages: Anthropic.MessageParam[] = [{ role: "user", content: TASK }];
let cursor = [0, 0];

try {
  for (let turn = 0; turn < 50; turn++) {
    const response = await anthropic.messages.create({
      model: "claude-opus-5-5",
      max_tokens: 16000,
      messages,
      tools: [
        {
          type: "computer_toolset_20260801",
          configs: {
            zoom: { enabled: false },
            hold_key: { enabled: false },
            triple_click: { enabled: false },
            left_mouse_down: { enabled: false },
            left_mouse_up: { enabled: false },
          },
        },
      ],
    });
    messages.push({ role: "assistant", content: response.content });
    if (response.stop_reason === "refusal") console.log("Claude declined the task.");

    const results: Anthropic.ToolResultBlockParam[] = [];
    let failed = false;
    for (const block of response.content) {
      if (block.type === "text") console.log(block.text);
      if (block.type !== "tool_use") continue;
      console.log(block.name, JSON.stringify(block.input));

      const input = block.input as any;
      let content: Anthropic.ToolResultBlockParam["content"] = HALTED;
      if (!failed) {
        try {
          const at = input.coordinate ?? cursor;
          let result: Steel.SessionComputerResponse = {};
          for (const action of toSteelActions(block.name, input, at)) {
            result = await steel.sessions.computer(session.id, action);
            if (result.error) throw new Error(result.error);
          }
          cursor = at;
          content = result.base64_image
            ? [
                {
                  type: "image",
                  source: {
                    type: "base64",
                    media_type: "image/png",
                    data: result.base64_image,
                  },
                },
              ]
            : block.name === "cursor_position"
              ? (result.output ?? "")
              : "OK";
        } catch (error) {
          content = String(error);
          failed = true;
        }
      }
      results.push({
        type: "tool_result",
        tool_use_id: block.id,
        toolset_name: "computer",
        content,
        is_error: failed,
      });
    }

    if (results.length === 0) break;
    messages.push({ role: "user", content: results });
  }
} finally {
  await steel.sessions.release(session.id);
}
```

**Python**

```python file=claude_computer_use.py
import json
import os

from anthropic import Anthropic
from steel import Steel

TASK = "Open https://example.com and tell me the title of the page."
HALTED = "Not executed: an earlier computer action in this turn failed."
BUTTONS = {
    "left_click": "left",
    "right_click": "right",
    "middle_click": "middle",
    "double_click": "left",
}
WHEEL = {"down": (0, 1), "up": (0, -1), "right": (1, 0), "left": (-1, 0)}

steel = Steel()
anthropic = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])


def to_steel_actions(name, tool_input, at):
    if tool_input.get("text") and name not in ("type", "key"):
        raise ValueError("Steel holds no modifier key during a click or a scroll.")
    if name == "screenshot":
        return [{"action": "take_screenshot"}]
    if name == "cursor_position":
        return [{"action": "get_cursor_position"}]
    if name in BUTTONS:
        return [
            {
                "action": "click_mouse",
                "button": BUTTONS[name],
                "num_clicks": 2 if name == "double_click" else 1,
                "coordinates": at,
            }
        ]
    if name == "left_click_drag":
        return [{"action": "drag_mouse", "path": [tool_input["start_coordinate"], at]}]
    if name == "mouse_move":
        return [{"action": "move_mouse", "coordinates": at}]
    if name == "scroll":
        delta_x, delta_y = WHEEL[tool_input["scroll_direction"]]
        step = {"action": "scroll", "coordinates": at, "delta_x": delta_x, "delta_y": delta_y}
        return [step] * tool_input["scroll_amount"]
    if name == "type":
        return [{"action": "type_text", "text": tool_input["text"]}]
    if name == "key":
        press = {"action": "press_key", "keys": tool_input["text"].split("+")}
        return [press] * tool_input.get("repeat", 1)
    if name == "wait":
        return [{"action": "wait", "duration": tool_input["duration"]}]
    raise ValueError(f"Steel has no action for {name}.")


session = steel.sessions.create(
    dimensions={"width": 1280, "height": 768},
    api_timeout=900000,
)
print("Session viewer:", session.session_viewer_url)

messages = [{"role": "user", "content": TASK}]
cursor = [0, 0]

try:
    for turn in range(50):
        response = anthropic.messages.create(
            model="claude-opus-5-5",
            max_tokens=16000,
            messages=messages,
            tools=[
                {
                    "type": "computer_toolset_20260801",
                    "configs": {
                        "zoom": {"enabled": False},
                        "hold_key": {"enabled": False},
                        "triple_click": {"enabled": False},
                        "left_mouse_down": {"enabled": False},
                        "left_mouse_up": {"enabled": False},
                    },
                }
            ],
        )
        messages.append({"role": "assistant", "content": response.content})
        if response.stop_reason == "refusal":
            print("Claude declined the task.")

        results = []
        failed = False
        for block in response.content:
            if block.type == "text":
                print(block.text)
            if block.type != "tool_use":
                continue
            print(block.name, json.dumps(block.input))

            content = HALTED
            if not failed:
                try:
                    at = block.input.get("coordinate") or cursor
                    result = None
                    for action in to_steel_actions(block.name, block.input, at):
                        result = steel.sessions.computer(session.id, **action)
                        if result.error:
                            raise RuntimeError(result.error)
                    cursor = at
                    if result.base64_image:
                        content = [
                            {
                                "type": "image",
                                "source": {
                                    "type": "base64",
                                    "media_type": "image/png",
                                    "data": result.base64_image,
                                },
                            }
                        ]
                    elif block.name == "cursor_position":
                        content = result.output or ""
                    else:
                        content = "OK"
                except Exception as error:
                    content = str(error)
                    failed = True
            results.append(
                {
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "toolset_name": "computer",
                    "content": content,
                    "is_error": failed,
                }
            )

        if not results:
            break
        messages.append({"role": "user", "content": results})
finally:
    steel.sessions.release(session.id)
```

### Run the program

**TypeScript**

```sh
npx tsx claude-computer-use.mts
```

**Python**

```sh
python claude_computer_use.py
```

The program prints the [session viewer](https://steel.dev/docs/observability/live-view#open-the-live-view) URL, each action that Claude takes, and Claude's answer. Open the session viewer to watch Claude work. Your output varies from run to run.

**TypeScript**

```text nocopy
Session viewer: https://app.steel.dev/sessions/384d94f4-cdeb-4eae-990d-f1dd33fac6db
I'll start by taking a screenshot to see what's on the screen.
screenshot {}
left_click {"coordinate":[656,63]}
key {"text":"ctrl+a"}
type {"text":"https://example.com\n"}
wait {"duration":2}
screenshot {}
The page title of https://example.com is **"Example Domain"**, as shown on the browser tab.

The page says the domain is for use in documentation examples and isn't meant for testing or monitoring. That notice appears in several languages, including Arabic, Chinese, French, Russian and Spanish.
```

**Python**

```text nocopy
Session viewer: https://app.steel.dev/sessions/b9c91108-0d8a-45c7-8f6b-7e933af8544f
screenshot {}
left_click {"coordinate": [656, 63]}
key {"text": "ctrl+a"}
type {"text": "https://example.com\n"}
wait {"duration": 2}
screenshot {}
The page title is **"Example Domain"**, as shown on the browser tab after loading https://example.com.

The page itself explains that the domain is for use in documentation examples and that it isn't a service.
```

### How the program maps the actions

Claude and Steel name the same actions differently. `toSteelActions` (`to_steel_actions` in Python) turns one Claude action into one or more Steel actions:

| Claude action | Steel action | Translation |
| - | - | - |
| `screenshot` | `take_screenshot` | None. |
| `left_click`, `right_click`, `middle_click` | `click_mouse` | The button comes from the action name. |
| `double_click` | `click_mouse` | `num_clicks` is 2. |
| `left_click_drag` | `drag_mouse` | The `path` goes from `start_coordinate` to `coordinate`. |
| `mouse_move` | `move_mouse` | None. |
| `scroll` | `scroll` | One Steel action for each step of `scroll_amount`, because Steel turns the wheel one step for each action. |
| `type` | `type_text` | None. |
| `key` | `press_key` | `ctrl+l` becomes `["ctrl", "l"]`. The program sends one Steel action for each `repeat`. |
| `wait` | `wait` | `duration` passes through. Both count it in seconds. |
| `cursor_position` | `get_cursor_position` | Claude gets the `output` of the Steel action as text. |

### Notes on the program

- Claude gives coordinates in pixels of the screenshots that it receives. A Steel screenshot has the size of the session's [`dimensions`](https://steel.dev/docs/sessions/devices), so the program passes coordinates through without change.
- Only `screenshot` returns an image to Claude. `cursor_position` returns the position as text, and every other result is the text `OK`.
- Each result repeats `toolset_name: "computer"`. Claude's API rejects a result without it.
- If an action fails, the program does not run the later actions of the same response. Each of them gets the error text that Anthropic's documentation gives for this case, and Claude plans again.
- The toolset's `configs` turn off the actions that have no Steel equivalent: `zoom`, `hold_key`, `triple_click`, `left_mouse_down` and `left_mouse_up`. Claude then does not ask for them.
- Steel holds no modifier key during a click or a scroll. If Claude asks for one, the program returns an error, and Claude tries another way.
- Without `coordinate`, a Claude action acts at the cursor position. Steel clicks at `[0, 0]` when an action has no `coordinates`, so the program tracks the cursor position and sends it.
- `press_key` also accepts the key names that Claude uses, such as `Return` and `Page_Down`, so the program passes Claude's keys through. [Computer actions](https://steel.dev/docs/control/computer-actions) describes `press_key`.
- An agent loop can run longer than the default `timeout`, so the session has a `timeout` of 15 minutes. [Limits](https://steel.dev/docs/platform/limits#session-length) lists the longest `timeout` that each plan allows.

## Next steps

- [Anthropic computer use documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool): The toolset, its actions and the models that support it.
- Cookbook starters: [TypeScript](https://github.com/steel-dev/steel-cookbook/tree/main/examples/claude-computer-use-ts), [Python](https://github.com/steel-dev/steel-cookbook/tree/main/examples/claude-computer-use-py), [Go](https://github.com/steel-dev/steel-cookbook/tree/main/examples/claude-computer-use-go), [Rust](https://github.com/steel-dev/steel-cookbook/tree/main/examples/claude-computer-use-rs), and a [mobile viewport](https://github.com/steel-dev/steel-cookbook/tree/main/examples/claude-computer-use-mobile) variant. They use the earlier `computer_20251124` tool with Claude Opus 4.7.
