---
name: browser-use-automation
description: "Drive a real browser to fill forms, scrape data, or test flows. Vision + DOM hybrid. Wraps browser-use to automate web flows that resist normal scrapers — multi-step form submission, OAuth, captcha edge-cases. Combines vision and DOM strategies for resilience."
license: MIT
---

# Browser Use Automation

Browser Use Automation drives a real, headed or headless browser to complete web tasks that brittle CSS-selector scrapers can't survive: multi-page form wizards, OAuth redirect chains, session-gated dashboards, and pages where the DOM shifts on every load. It pairs a vision model with live DOM inspection, so when a selector breaks the agent can still locate the right element by what it looks like on screen. Built for QA engineers writing resilient end-to-end checks and data folks pulling structured records out of stubborn web apps.

## When to use it

- A login flow spans several screens, an SSO/OAuth provider, or an MFA prompt.
- The page renders client-side and elements have no stable `id` or `data-testid`.
- You need to fill and submit a form, then scrape the confirmation page.
- A scraper keeps breaking because the site A/B-tests its layout.
- You want a flow described in plain language rather than hand-maintained selectors.

## How it works

It wraps the open-source `browser-use` library, which runs on Playwright underneath (Chromium by default; Firefox and WebKit available). You hand an `Agent` a natural-language `task` and an LLM; the agent loops: it captures the page state (accessibility tree + DOM + a screenshot), the model decides the next action (`click`, `input_text`, `scroll`, `go_to_url`, `extract_content`, `done`), and the action executes via the Playwright `BrowserContext`. Structured results come back through an `output_model` (a Pydantic schema). You can register custom actions on a `Controller` and persist cookies/auth via a saved storage-state file.

## Example

```bash
curl -fsSL --create-dirs -o ~/.claude/skills/browser-use-automation/SKILL.md https://claudeskil.com/skills/browser-use-automation/SKILL.md
pip install browser-use && playwright install chromium
```

```python
from browser_use import Agent, Browser
from langchain_openai import ChatOpenAI

agent = Agent(
    task="Log in at example.com/login with $USER/$PASS, "
         "open Billing, and return the last 3 invoice totals.",
    llm=ChatOpenAI(model="gpt-4o"),
    browser=Browser(headless=False),
)
result = await agent.run(max_steps=25)
print(result.final_result())
```

## Tips & gotchas

- Set `headless=False` while developing — watching the run is the fastest way to see why a step misfired.
- Always cap `max_steps`; a confused agent will otherwise loop and burn tokens on a page it can't parse.
- Save and reuse Playwright `storage_state` to skip repeated logins; pass secrets via `sensitive_data` so they're masked in the model context.
- Vision calls dominate cost. Disable screenshots (`use_vision=False`) on simple DOM-only flows.

## FAQ

**Does it solve captchas?** No, and you shouldn't rely on it to. It handles edge-cases *around* captchas (detecting them, pausing for a human, resuming) but won't defeat anti-bot systems.

**Which LLMs work?** Any LangChain chat model — OpenAI, Anthropic Claude, Gemini, or a local model. Vision-capable models perform noticeably better.

**Playwright or Selenium?** It runs on Playwright. The `selenium` tag reflects the problem space, not the engine.