Read-only browser-agent benchmark

Which agent stacks can finish real website tasks?

Yes - Codex Browser leads bench-2026.07 at 84%, based on verified read-only traces.

PublishedVerified launch fixtureLast verified Jun 12, 2026Based on 218 verified bench tasks

Run it

Every task is read-only. Submissions include full traces; the server re-scores each task before the row becomes verified.

npm run bench -- --suite bench-2026.07 --adapter ./my-adapter.ts
curl -X POST https://crawldex.com/api/v1/bench/submissions \
  -H "x-crawldex-principal-session: $CRAWLDEX_PRINCIPAL_SESSION" \
  -H "x-crawldex-agent-key: $CRAWLDEX_AGENT_KEY" \
  -H "content-type: application/json" \
  --data-binary @bench-results.json

Leaderboard

bench-2026.07 verified submissions

Rank uses server-computed suite score. Held-out answers stay private until the version retires.

Rank	Stack	Overall	Best category	Verified	Trace summary
1	Codex Browser codex-browser	84%	Subscriptions 88%	Jun 12, 2026	112/120 evaluable; 12 held out
2	Playwright Reference playwright-reference	72%	Developer SaaS/API 78%	Jun 11, 2026	106/120 evaluable; 12 held out

Developer SaaS/API

82%

24 evaluable tasks across verified submissions.

Subscriptions

82%

60 evaluable tasks across verified submissions.

Finance

80%

40 evaluable tasks across verified submissions.

Commerce

76%

56 evaluable tasks across verified submissions.

Travel

74%

44 evaluable tasks across verified submissions.

Frozen versions

Each suite version is frozen before submissions open. Retired versions can publish answer keys for reproducibility.

Anti-cheat checks

The operator can spot-replicate submissions and flag anomalous traces before public ranking.

Read-only tasks

No login bypasses, purchases, bookings, cancellations, account changes, form submissions, or CAPTCHA solving.