Steel runs browser infrastructure for AI agents. Every session assembles parts from at least two companies: our browsers and network on one side; the customer’s harness, credentials, and sometimes proxies on the other. When the assembled system fails, the error appears on our screen regardless of whose part broke.

This is the story of one error, “Proxy Connection Failed,” and an investigation into whose failure it actually was.

We came out with clear data, a partial fix, and questions we still can’t answer. We’re writing it up because every developer tool that requires assembly has a version of this problem, and we’d like to compare notes.

Investigating the Root Cause of Proxy Connection Failures

Customers kept hitting “Connection Failed” inside Steel sessions. From where they sat, the evidence was damning: a Steel session, a Steel error screen, a dead workload. Tickets treated it as a Steel outage. That was fair. The product gave them no reason to think anything else.

Customer’s agent
      │
      ▼
Steel browser ───► Proxy ───► Target website
     Steel            ?             ?
     owns         customer or
                  Steel-managed

Customer’s agent
      │
      ▼
Steel browser ───► Proxy ───► Target website
     Steel            ?             ?
     owns         customer or
                  Steel-managed

We pulled 30 days of tunnel-establishment data across the fleet and all customers.

Customer-supplied proxies failed to establish a tunnel about 9% of the time. Steel-managed proxies failed about 0.4% of the time: a 22x gap across thousands of browser workers.

The cleanest comparison came from customers running both modes against the same targets on the same days. Their proxies failed on nearly every session. Ours failed on essentially none. Inside the failing pools, some IPs were blocked by one or two sites, other pools were blocked across hundreds of domains, and in the clearest cases the proxy couldn’t reach a basic IP-echo endpoint. It was down.

So, mostly, it wasn’t us. Sheesh, crisis mildly averted.

Tunnel-establishment failure rate

Customer-supplied proxies   ██████████████████████  9.0%
Steel-managed proxies       █                       0.4%

                            22× higher

Tunnel-establishment failure rate

Customer-supplied proxies   ██████████████████████  9.0%
Steel-managed proxies       █                       0.4%

                            22× higher

Why Customer-Supplied Proxies Are Difficult to Diagnose

A customer hands us a proxy as a URL: scheme, host, port, credentials. The URL doesn’t say whether it points at a rotating residential network, a fixed datacenter server, or a sticky session whose exit IP has to survive for the workload to make sense.

The failure says little more. A tunnel that never establishes could mean rejected credentials, an unreachable host, or an upstream that dropped the CONNECT. A tunnel that dies mid-session could mean rotation, rate limits, or a destination blocking the exit IP.

We sit in the middle of that connection and often can’t tell these cases apart. The customer, one layer further out, has even less to go on. Both sides end up staring at the same screen holding the same incomplete signal.

Why Managed and Customer-Supplied Proxies Behave Differently

Steel-managed proxies get a resilience layer: we health-check endpoints, track pool quality, swap bad proxies, and retry, because we know what each proxy URL represents and when a new IP is safe.

Customer-supplied proxies get none of that. The first failed connection kills the session.

We still think that asymmetry is the right call. Blind rotation of a proxy we don’t understand could be useless (rotating within a dead pool), expensive (burning the customer’s metered bandwidth), or destructive (breaking a sticky session the workload depends on). Retrying safely requires semantics we don’t have.

The trouble is what this reasonable decision produces: a customer with a bad proxy gets a generic failure inside our product, with no pointer toward their own infrastructure. Their vendor flaked; we returned the confusion.

Better Error Attribution Is a Product Responsibility

We already had every fact the investigation needed: which proxy was active, who supplied it, where the tunnel died. Once we knew to look, the analysis was easy.

Knowing to look was the problem. Nothing in our monitoring flagged these failures; a customer had to report them. The customer took just as long to notice. The failures dissolved into the volume of sessions they were running.

The facts existed the whole time. Neither side’s tooling surfaced them. We’re changing the failure path so they show up where the failure does: the error names the active proxy and its owner, the failing stage when it’s distinguishable, and a concrete next step. When the CONNECT gets a 407, the error should say “the proxy rejected your credentials” and point at the provider.

The principle underneath: we can’t always recover from a failure in a component we don’t run, but we can always describe it.

Open Questions About Monitoring Customer-Owned Infrastructure

This is where a blog post would usually present the solution. Instead, here are the questions we’re stuck on. They sound proxy-shaped because proxies are where we got burned this time, but each one applies to any component a customer brings, including ones that don’t look like components until they fail.

1. How do you notice a failure in a component you don’t run?

Our trigger was a customer report, and it arrived late; an elevated failure rate reads as noise while the successes keep coming. A proxy-health baseline would have caught this one, but only because we now know proxies are worth watching. The next weak component won’t announce itself.

2. Should a vendor actively test the parts customers bring?

A health check from our fleet would let us say “your proxy is down” before a workload dies. It would also spend the customer’s metered bandwidth, hit their provider’s rate limits, and exercise their credentials outside any session they started. The same tension applies to testing a customer’s harness or their LLM provider. We haven’t found where that line is.

3. Should customers describe the parts they bring?

A declared semantic like rotating, static, or sticky would tell us which proxy retries are safe. Generalized, that means asking customers to document a contract for every component they plug in: more configuration burden, and a wrong declaration produces worse behavior than none.

Where does attribution actually change behavior? We can name the failing component on the error screen, in the API response, in a webhook, or on the dashboard. We don’t yet know which surface makes a customer check their own infrastructure instead of filing a ticket with us.

The Reliability Blind Spot in Composable Developer Tools

The uncomfortable part of the data wasn’t the 22x gap. It was that our dashboards looked healthy the whole time: sessions starting, traffic flowing, and 9% of customer proxy tunnels quietly dying underneath. Usage looked like success and wasn’t.

This time the weak part was proxies. Next time it could be the harness, the LLM driving it, or the router a cleaning crew unplugged at the customer’s office. Any developer tool that requires assembly has this blind spot somewhere: a component the vendor doesn’t run, failing inside an experience the vendor owns.

If you’ve drawn this responsibility line in your own product, especially if you’ve decided whether to probe infrastructure your customers bring, we’d like to hear how you did it.

And if enough of us are traumatised with the same thing, we’ll open a #couch-support channel in the Steel Discord and work through it together.