---
site: steel.dev
collection: blog
canonical_url: https://steel.dev/blog/happy-path-for-automating-the-web-with-steel
markdown_url: https://steel.dev/blog/happy-path-for-automating-the-web-with-steel.md
content_signal: ai-input=yes, ai-train=yes, search=yes
related:
  - https://steel.dev/blog/beginner-s-guide-to-steel.md
  - https://steel.dev/blog/headless-browsers-cloud-scraping.md
  - https://steel.dev/blog/what-is-browser-as-a-service.md
title: Happy Path for Automating the Web with Steel
description: Automate real web workflows fast with Steel’s managed cloud Chrome sessions. Create a session, connect Puppeteer, run steps, extract data, and release reliably.
date: 2026-02-24
authors:
  - Nikola Balic
category: Guides
minutes: 5
tldr: The core Steel pattern is create a session, connect, run automation, release. The guide installs steel-sdk and puppeteer-core, starts a cloud Chrome session, connects Puppeteer over WebSocket, extracts data from example.com with page.evaluate(), then releases the session because Steel bills by session time.
topics:
  - Cloud browser sessions
  - Browser automation
  - Web scraping
  - Session lifecycle
  - Data extraction
entities:
  - Steel
  - steel-sdk
  - Puppeteer
  - puppeteer-core
  - dotenv
  - Chrome
answers_questions:
  - How do I connect Puppeteer to a Steel cloud browser?
  - How do I create and release a Steel browser session?
  - How do I extract data from a page with Puppeteer?
  - Why should I release Steel sessions when a workflow is done?
---

# Happy Path for Automating the Web with Steel

This guide walks through automating a real browser workflow in minutes.

This is a practical web automation guide for AI agents, using a managed cloud browser API so you can replace local Puppeteer stacks and enjoy the benefits of infinite scaling.

Instead of manually opening a browser and copying data from a page, you can run the same workflow inside a dedicated cloud browser session. The session behaves exactly like Chrome on your machine, except it runs in Steel’s infrastructure and is fully controlled through code.

Steel provides programmable browser sessions designed for agents and automation. Sessions start in under a second, can run for hours, and execute web workflows reliably in production environments.

We’ll build it step by step.

## Step 1: Get Your Steel API Key

Before writing any code, you need access to Steel’s cloud browser sessions. Access is granted through an API key, which authenticates requests to the Steel control plane and authorizes session creation.

1. Go to [steel.dev](https://steel.dev/) and sign up for an account
2. Navigate to your dashboard
3. Find your **API key**, it looks something like `sk_live_abc123...`
4. Copy it somewhere safe (e.g., don’t feed it to your coding agent)

## Step 2: Set Up Your Project

Create a new project and install dependencies:

```sh
mkdir steel-automation
cd steel-automation
npm init -y
npm install steel-sdk puppeteer-core dotenv
```

These dependencies let you build browser automation with Steel's managed cloud browser infrastructure without local Chrome provisioning.

**What you just installed:**

- **steel-sdk**: Creates and manages cloud browser sessions
- **puppeteer-core**: Controls the browser (clicks, types, extracts data)
- **dotenv**: Loads environment variables from .env file

## Step 3: Store Your API Key

Create a `.env` file in your project root to keep your API key out of your code for security:

```js
STEEL_API_KEY=your_api_key_here
```

## Step 4: Write the Automation Script

Create `scrape.js`. The complete code is in the Appendix. We'll walk through that code and break down each part so you understand exactly what's happening.

### **The Setup**

```js
require('dotenv').config();
const Steel = require('steel-sdk');
const puppeteer = require('puppeteer-core');
const steel = new Steel({ steelAPIKey: process.env.STEEL_API_KEY });
```

This initializes the Steel client using your API key. The `steel` object is your interface for creating and managing browser sessions with browser automation over CDP.

### **Starting a Session**

```js
const session = await steel.sessions.create();
```

This call provisions a new isolated browser session in Steel’s cloud. Each session runs a real Chrome instance and returns a unique `session.id`. All subsequent commands are scoped to that session.

Sessions are independent, which means you can run multiple workflows concurrently without state collisions.

### **Connecting Puppeteer to Your Cloud Browser**

```js
const browser = await puppeteer.connect({
  browserWSEndpoint: `wss://connect.steel.dev?apiKey=${process.env.STEEL_API_KEY}&sessionId=${session.id}`
});
```

Puppeteer connects directly to the Steel session over WebSocket. The connection string includes your API key for authentication and the session ID to route commands to the correct browser instance.

Once connected, you’re controlling a real cloud browser as if it were running locally.

### **Opening a Page and Navigating**

```js
const page = await browser.newPage();
await page.goto('https://example.com');
```

You are now controlling the browser session. `newPage()` opens a fresh tab, and `goto()` tells it to load example.com just like clicking the address bar and hitting Enter.

**Why Steel:** Many modern websites block non-browser traffic or return incomplete responses to HTTP clients. Steel runs a full browser session, so your automation executes exactly as a real user would, including JavaScript rendering, authentication flows, and dynamic content.

### **Extracting Data**

```js
const pageData = await page.evaluate(() => {
  return {
    h1: document.querySelector('h1')?.textContent,
    paragraph: document.querySelector('p')?.textContent,
    allLinks: Array.from(document.querySelectorAll('a')).map(a => ({
      text: a.textContent,
      href: a.href
    }))
  };
});
```

This executes JavaScript inside the browser context to extract structured data. `page.evaluate()` runs JavaScript **inside the browser page**, just like opening your browser's console and typing code there.

You're using standard web APIs:

- `document.querySelector('h1')` - Find the first `<h1>` tag
- `document.querySelectorAll('a')` - Find all `<a>` link tags
- `.textContent` - Get the visible text
- `.href` - Get the link URL

Steel works with the automation frameworks you already use. If you know Puppeteer, you already know how to run workflows on Steel. There’s no new execution model, just managed browser infrastructure behind it.

### **Closing the Session**

```js
await browser.close();
await steel.sessions.release(session.id);
```

This closes the Puppeteer connection and releases the browser session.

Steel bills based on session time. Always release sessions when your workflow completes to ensure proper resource cleanup and cost control.

**Pro tip:** Not sure if you have sessions running? Check your [Steel dashboard](https://app.steel.dev/sessions) to view all active sessions.

## Step 5: Run Your Automation

Execute the script:

```sh
node scrape.js
```

Example output:

```js nocopy
Starting a Steel browser session...
Session created: ses_abc123xyz
Navigating to example.com...
Page loaded!
Reading page content...
Data extracted:
{
  "h1": "Example Domain",
  "paragraph": "This domain is for use in illustrative examples...",
  "allLinks": [
    { "text": "More information...", "href": "https://www.iana.org/domains/example" }
  ]
}
Closing the session...
Done! Session closed.
```

You just executed a browser workflow inside a managed cloud session without provisioning infrastructure or maintaining browser fleets.

That’s the core Steel pattern: create a session, connect, run automation, release.

## What's Next?

This example demonstrates the core session lifecycle and the browser automation foundation for scraping, form automation, and agent workflows. In production, Steel extends this execution model with:

- Managed residential proxy routing and fingerprint controls
- Built-in CAPTCHA handling (reCAPTCHA, Turnstile)
- Long-lived sessions (up to 24 hours) with persisted state
- Live session viewers and replayable traces for debugging and verification
- Structured and markdown extraction for LLM-driven workflows
- Horizontal scaling across thousands of concurrent sessions

The primitive stays the same: create → connect → execute → release. Steel handles the execution layer.

You can find more examples in [Steel's Cookbook](https://github.com/steel-dev/steel-cookbook), and you can copy the complete script from this walkthrough in one place: [Steel Happy Path appendix script](https://gist.github.com/nibzard/6a2f3911a1953b7bef363a411c45a95b).

## The Bottom Line

Steel replaces custom browser infrastructure with a managed execution layer built for agents.

You get isolated sessions, state management, observability, anti-bot handling, and long-running workflows without operating browsers yourself.

Humans use Chrome. Agents use Steel.

## Need Help?

- [Steel Documentation](https://docs.steel.dev/)
- [Discord Community](https://discord.gg/steel-dev)
