Build a CrewAI crew on Steel
CrewAI runs a crew of agents. Each agent has a role and works on a task, and a sequential crew passes each task's result to the next task. To give an agent the web, you write a CrewAI tool that calls Scrape. Scrape loads the page in a Steel browser, runs its JavaScript, and returns the page as Markdown. There is no session to create or release. CrewAI is a Python framework.
Before you begin
- Python 3.10 to 3.13. CrewAI does not support Python 3.14 yet.
- A Steel API key in the
STEEL_API_KEYenvironment variable. See API keys. - An OpenAI API key in the
OPENAI_API_KEYenvironment variable. CrewAI reads it from there.
Install CrewAI and the Steel SDK.
python3 -m venv .venv
source .venv/bin/activate
pip install crewai steel-sdkSet up the integration
Build the crew
This program defines a scrape tool with CrewAI's @tool decorator. The model reads the tool's docstring to decide when to call it. A researcher agent uses the tool to read a list of stories on a practice page, and a writer agent then summarizes what the researcher found.
from crewai import LLM, Agent, Crew, Process, Task
from crewai.tools import tool
from steel import Steel
steel = Steel()
@tool("Scrape a web page")
def scrape(url: str) -> str:
"""Load a web page in a Steel browser and return its content as Markdown."""
result = steel.scrape(url=url, format=["markdown"])
return result.content.markdown or ""
llm = LLM(model="gpt-6-luna", api="responses")
researcher = Agent(
role="Web researcher",
goal="Find facts on the web pages that a task names, and keep the link of each.",
backstory="You read pages with the scrape tool and never invent facts.",
tools=[scrape],
llm=llm,
)
writer = Agent(
role="Writer",
goal="Turn research notes into a short, accurate summary.",
backstory="You write plain summaries and keep every link from the notes.",
llm=llm,
)
research = Task(
description="Scrape https://steel.dev/practice/news and list the top 3 stories.",
expected_output="The top 3 stories, each with its title, link and points.",
agent=researcher,
)
summary = Task(
description="Summarize the stories from the research in one sentence each.",
expected_output="Three bullet points, one for each story, with its link.",
agent=writer,
)
crew = Crew(
agents=[researcher, writer],
tasks=[research, summary],
process=Process.sequential,
)
result = crew.kickoff()
print(result.raw)Only the researcher has the tool. The writer works from the researcher's result, which the crew passes to the second task.
Run the program
python crew.pyThe program prints the writer's summary. Its wording changes from run to run.
- [Steel Speaks Rust and Go](https://steel.dev/blog/rust-go-sdks) — Steel announces SDKs for Rust and Go.
- [Why Browser Agents Fail in Production](https://steel.dev/blog/why-browser-agents-fail-in-production) — The post explains why browser agents fail in production.
- [Steel Browser: the open-source browser API](https://github.com/steel-dev/steel-browser) — Steel Browser is an open-source browser API.To see each agent's steps and tool calls, pass verbose=True to Crew.
Add Steel features and a session
- To load a page through a Steel proxy, or to wait for a page that shows its content late, pass
use_proxy=Trueordelaytosteel.scrape(). See Control how the page loads. - Scrape returns one page for each call. When an agent must sign in, click, or keep cookies from page to page, give it tools that drive a session instead. Connect Playwright to Steel shows how a program connects to a session.