How to Turn a Fragile Browser Use Agent into a Cheap Playwright Script
The most common arc with Browser Use goes like this: build an agent, it works, ship it to a daily job, watch the bill, then wonder why you are paying a model to click the same three buttons every morning.
You do not have to give up the agent. The pattern that holds is agent writes once, script replays many times, agent returns when the page changes.
How much does a replay actually save?
A ten-step agent run on a vision model sends roughly 30,000–60,000 input tokens. At typical 2026 pricing that is a few cents per run, and a five-times-a-day job is trivial.
Move to 500 runs a day and it becomes real money: several hundred dollars a month to click buttons that never moved. The Playwright equivalent is a few hundred milliseconds of CPU and zero tokens.
The secondary saving is worse to ignore: latency. A ten-step agent takes 30–90 seconds. A replay takes under two. If a person is waiting, that difference is your product.
How do you capture the actions the agent took?
Run the agent once against a clean account and log every action. Browser Use exposes the step history: each entry has the action name, its parameters, and the element it targeted.
Extract a minimal sequence:
- Navigate to the start URL. 2. Dismiss consent (record it as the first action so you can replay it deterministically). 3. Log in if needed. 4. Fill the form fields. 5. Click submit. 6. Wait for the confirmation selector.
Ignore steps that turned out to be exploratory, like clicking a search box and then deciding not to search. Keeping them in the script makes it more fragile, not less.
How do you turn that into a Playwright script?
Map each recorded action to a Playwright call. The rule that keeps it maintainable: prefer role and label selectors over CSS paths.
page.get_by_role("button", name="Submit").click()
page.get_by_label("Invoice number").fill(invoice_id)
page.get_by_text("Saved").wait_for(timeout=15000)
Role selectors survive class-name churn and redesigns; a path like div > div:nth-child(3) > button breaks the first time someone adds a wrapper. Where role selectors fail, add a data-testid if you control the page, or fall back to text matching.
Add explicit waits on outcomes rather than sleep(). Sleeping five seconds because the page usually loads in three is how replay scripts end up with 90% of their runtime spent waiting.
How do you know when to send the agent back in?
Never let the replay fail silently. Two checks:
- Assert the end state. After submit, verify the confirmation text or the updated record. A script that clicks the right button but gets a validation error has failed. – Screenshot on failure. Save a full-page screenshot to your logs when an assertion fails. That single image usually tells you which layout change broke it.
When the script fails twice in a row, kick off the agent for that run with the same task. If the agent succeeds, the site changed — capture its new action sequence and regenerate the script. If the agent fails too, the site is genuinely broken and you should page a human.
That two-strike rule keeps the agent as a repair mechanism rather than a primary path.
What should stay agentic?
Some parts of a workflow should never be recorded into a script:
- Navigation through unknown search results. If the path from search to target varies, a script cannot encode it. – Decision points. “If the price is above X, request approval” is a policy the agent can read from a prompt and a script can only code once. – Pages with rotating layouts or A/B tests. Two variants means two scripts and a branch the agent would have handled.
Keep the agent on the edges and the script on the trunk.
How do you keep the two in sync over time?
Put both in the same repo, and version the recorded action sequence as a JSON fixture next to the Playwright script. When the script is regenerated from a new recording, the diff review shows exactly which step changed, which is far more readable than a rewritten selector list.
Re-run the agent once a month even if nothing failed. A slow drift — a button moving into a menu, a consent banner appearing for EU traffic — can keep an old sequence passing while the task quietly does something slightly different.
When is a script the wrong choice?
If the workflow runs twice a month, the agent’s few cents per run is cheaper than maintaining a selector-based script. If the target site actively blocks automation, scripts fail the same way agents do and the maintenance burden is not worth the token savings.
And if you cannot assert the outcome, you cannot detect breakage, so a script is more dangerous than an agent that at least reports what it did.
Quick Summary
-
Agent runs cost tens of thousands of input tokens each; a Playwright replay of the same steps costs nothing and runs in a couple of seconds.
-
Record the successful action sequence once, drop exploratory steps, and map each action to a role or label selector rather than a CSS path.
-
Assert the end state and screenshot on failure, because a script that clicks the right button and gets a validation error has still failed.
-
Send the agent back after two consecutive replay failures to capture a new sequence, then regenerate the script.
-
Keep genuinely variable parts — search navigation, policy decisions, A/B layouts — agentic and put the rest in the script.
