Browser Use Automation

Browser Use Automation drives a real, headed or headless browser to complete web tasks that brittle CSS-selector scrapers can't survive: multi-page form wizards, OAuth redirect chains, session-gated dashboards, and pages where the DOM shifts on every load. It pairs a vision model with live DOM inspection, so when a selector breaks the agent can still locate the right element by what it looks like on screen. Built for QA engineers writing resilient end-to-end checks and data folks pulling structured records out of stubborn web apps.

When to use it

• A login flow spans several screens, an SSO/OAuth provider, or an MFA prompt.
• The page renders client-side and elements have no stable id or data-testid.
• You need to fill and submit a form, then scrape the confirmation page.
• A scraper keeps breaking because the site A/B-tests its layout.
• You want a flow described in plain language rather than hand-maintained selectors.

How it works

It wraps the open-source browser-use library, which runs on Playwright underneath (Chromium by default; Firefox and WebKit available). You hand an Agent a natural-language task and an LLM; the agent loops: it captures the page state (accessibility tree + DOM + a screenshot), the model decides the next action (click, input_text, scroll, go_to_url, extract_content, done), and the action executes via the Playwright BrowserContext. Structured results come back through an output_model (a Pydantic schema). You can register custom actions on a Controller and persist cookies/auth via a saved storage-state file.

Example

curl -fsSL --create-dirs -o ~/.claude/skills/browser-use-automation/SKILL.md https://claudeskil.com/skills/browser-use-automation/SKILL.md
pip install browser-use && playwright install chromium

from browser_use import Agent, Browser
from langchain_openai import ChatOpenAI

agent = Agent(
    task="Log in at example.com/login with $USER/$PASS, "
         "open Billing, and return the last 3 invoice totals.",
    llm=ChatOpenAI(model="gpt-4o"),
    browser=Browser(headless=False),
)
result = await agent.run(max_steps=25)
print(result.final_result())

Tips & gotchas

• Set headless=False while developing — watching the run is the fastest way to see why a step misfired.
• Always cap max_steps; a confused agent will otherwise loop and burn tokens on a page it can't parse.
• Save and reuse Playwright storage_state to skip repeated logins; pass secrets via sensitive_data so they're masked in the model context.
• Vision calls dominate cost. Disable screenshots (use_vision=False) on simple DOM-only flows.

FAQ

Does it solve captchas? No, and you shouldn't rely on it to. It handles edge-cases around captchas (detecting them, pausing for a human, resuming) but won't defeat anti-bot systems.

Which LLMs work? Any LangChain chat model — OpenAI, Anthropic Claude, Gemini, or a local model. Vision-capable models perform noticeably better.

Playwright or Selenium? It runs on Playwright. The selenium tag reflects the problem space, not the engine.