Skip to content

Get past bot detection

Set up your agent

Give your coding agent a Steel browser and the Steel skills, so that it browses and writes Steel code the right way. Learn more

Install the Steel CLI. It signs you in and installs the skills for your coding agents.

curl -fsS https://setup.steel.dev | sh

Or send this prompt to your coding agent, and it does the same:

Read https://setup.steel.dev/install.md and follow it to set up Steel for this coding client.
Then continue my requested task using the installed Steel skill.

Set up your agent →

When a site blocks your session, change one thing at a time, in this order: the IP address, the CAPTCHA, the fingerprint, and the browser. Each change costs more or changes more than the one before it, so stop at the first one that works. How stealth works explains what sites check and why.

No setting gets past every site. A site can still block a session, ask for a login, or forbid automation in its terms.

Before you begin

Put your Steel API key in the STEEL_API_KEY environment variable, and install the Steel SDK and Playwright with npm install steel-sdk playwright or with pip install steel-sdk playwright in a virtual environment.

Choose a setup

Find what the site does to your session, and try the setup in that row. If you do not know what blocked you, find out first.

What you seeWhat to useHow to check that it worked
"Access denied", a 403, or an empty pageA Steel proxy (useProxy)The page loads, and proxySource is steel.
Content for the wrong countryA Steel proxy in that country (geolocation)https://steel.dev/practice/country shows that country.
A new sign-in check in each session, or a site that allows only known addressesA dedicated IPhttps://steel.dev/practice/ip shows an address from the dashboard's Network page.
A CAPTCHA, or captcha_encounteredCAPTCHA solving (solveCaptcha)The CAPTCHA status is solved.
A block after your code clicks or typesHuman-like input (humanizeInteractions)Several runs end without bot_detection_triggered.
A block that stays even with a proxy and CAPTCHA solvingA mobile device (deviceConfig)Several runs end without bot_detection_triggered.
A block that stays after all of theseStealth Browser (useStealthBrowser)browserMode is stealth, and several runs end without bot_detection_triggered.

What each plan includes shows which plans have each setup. In every setup, leave Steel's fingerprint on.

Find out what blocked you

Run the task once with no stealth options, and read what Steel saw. When a session ends, Steel adds signals to it within a few seconds. Look for bot_detection_triggered, which marks a block or a challenge that the browser did not get past, and captcha_encountered, which marks a CAPTCHA. Signals reference lists the others.

This program opens the page that blocks you, prints its title, releases the session, and prints the signals. Set TARGET_URL to that page's URL.

check-site.mts
import Steel from "steel-sdk";
import { chromium } from "playwright";

const client = new Steel();
const session = await client.sessions.create({
  metadata: { check: "bot-detection", attempt: 1 },
});

try {
  const browser = await chromium.connectOverCDP(session.websocketUrl);
  const context = browser.contexts()[0];
  const page = context.pages()[0] ?? (await context.newPage());
  await page.goto(process.env.TARGET_URL!);
  console.log(`Title: ${await page.title()}`);
  await browser.close();
} finally {
  await client.sessions.release(session.id);
}

await new Promise((resolve) => setTimeout(resolve, 10000));
const { signals } = await client.sessions.retrieve(session.id);
console.log(`Signals: ${JSON.stringify(signals)}`);
console.log(`Recording: ${session.sessionViewerUrl}`);

Run the program:

npx tsx check-site.mts

The output of a blocked run looks like this:

Title: Just a moment...
Signals: ["bot_detection_triggered"]
Recording: https://app.steel.dev/sessions/2f7b7701-6017-45cc-8138-e90aeb0610ae

Open the recording URL to see the page that the browser saw. Do this even when the list of signals is empty, because signals do not catch every block.

Then find what you see in Choose a setup. The next sections add the most common setups to this program, in the order to try them: a proxy, CAPTCHA solving and a mobile device.

Add a proxy

Many sites block data center IP addresses, and a session uses one by default. A Steel proxy gives it a residential address instead. Change the create call in the program:

const session = await client.sessions.create({
  useProxy: true,
  metadata: { check: "bot-detection", attempt: 2 },
});

In Rust, this code needs cargo add serde_json.

On Launch, a Steel proxy needs a verified account. You verify it once, with $10 of paid credits. Until you verify it, create fails with deposit_required. See Verify a Launch account.

If the site serves one country, target that country with useProxy: { geolocation: { country: "DE" } }. A Steel proxy also adds a bandwidth charge. To use a proxy that you run yourself, see Proxies. To keep one address from run to run, use a dedicated IP.

Solve the CAPTCHA

A proxy makes CAPTCHAs less frequent, but it does not stop them. Set solveCaptcha to true, and Steel finds and solves CAPTCHAs on each page while your code waits. Keep the proxy:

const session = await client.sessions.create({
  useProxy: true,
  solveCaptcha: true,
  metadata: { check: "bot-detection", attempt: 3 },
});

On Launch, CAPTCHA solving needs a verified account too, so an unverified account gets deposit_required here as well.

A solve takes time, so after page.goto, wait for an element that you need before the next action. Give the wait a timeout that leaves room for the solve. Each solve has a charge. Steel does not solve some kinds of challenge. CAPTCHA solving lists them, and shows how to read each solve's status and how to solve only when you choose to.

Try a mobile device

Steel gives each session a fingerprint that matches a real desktop browser, unless you set userAgent or start from a profile, as Fingerprints explains. Do not turn it off. If a site still blocks the session, a mobile device gives it a different fingerprint, and many sites serve a simpler page to mobile devices.

const session = await client.sessions.create({
  useProxy: true,
  solveCaptcha: true,
  deviceConfig: { device: "mobile" },
  metadata: { check: "bot-detection", attempt: 4 },
});

A mobile session sets its own screen size. If you also pass dimensions, they must be at least 508 by 1074 pixels. To make the mouse move more like a person's, add stealthConfig: { humanizeInteractions: true }. Fingerprints and Set the device and viewport have the other options.

On Enterprise, if the site still blocks the session, run it on Stealth Browser, Steel's own Chromium fork. Add useStealthBrowser to experimentalFeatures, and keep the proxy and CAPTCHA solving.

If the site blocks every combination, a person can do the step in the live view. See Hand control to a human.

Check the signals

One good run does not prove that a setting works. Run the task several times with the same settings, then compare the results. Each attempt in this guide has its own attempt value in the metadata, so you can list the sessions that were still blocked in one attempt:

for await (const session of client.sessions.list({
  metadata: { check: "bot-detection", attempt: "3" },
  signal: ["bot_detection_triggered"],
})) {
  console.log(session.id, session.signals);
}

In Rust, this code needs cargo add futures-util.

Steel stores metadata values as strings, so filter with "3". The code prints each blocked session of that attempt with its signals. An empty result means that no session of the attempt carries the signal.

In production, keep the same check. Tag your sessions with metadata for the job or the site, and list the sessions that carry bot_detection_triggered or captcha_encountered. A rise in either one can mean that the site changed its checks. Signals shows more ways to read them.

Next steps