Operator guide

Why Rate Limits Stop AI Agent Measurement

Evidence-backed guide to HTTP 429 and rate-limit outcomes that stop repeatable browser-agent measurements.

Read flagship report Dispute a figure

Publication gate

Indexable

This report has at least 25 qualifying evidence-backed rows and is eligible for search indexing.

54: ranked rows
58 observations: evidence
4%: need a person nearby
Jul 8, 2026: last verified

Operator guide

Current through Jul 8, 2026, 54 CrawlDex rows match this report's evidence criteria.

Which public task rows stop with HTTP 429 or rate-limit evidence rather than a stable task outcome?

Rows qualify when CrawlDex evidence records rate-limited or HTTP 429 blockers.

What site owners can make clearer

Return explicit retry-after or rate-limit messages for legitimate public checks.
Publish API or bulk-access paths for high-frequency read-only measurement.
Keep rate-limit responses distinct from account or policy failures.

What agent builders should plan for

Treat rate-limit responses as blocked measurements instead of retrying aggressively.
Prefer official API paths when repeated public checks are needed.
Record the rate-limit state and stop before producing a false completion claim.

Live rows

Ranked from current CrawlDex evidence

Rank	Site task	Verdict	AES	Evidence	Main blocker
1	Chewy commerce.check_return_policy chewy.com	Needs user presence	46	3	login walls
2	Vrbo real_estate_moving.search_rental vrbo.com	Needs user presence	44	3	forced account creation
3	Chewy account_support.find_autoship_help_guidance chewy.com	Blocked	0	1	public url verified
4	Chewy commerce.find_autoship_guidance chewy.com	Blocked	0	1	public url verified
5	Expedia customer_support.find_public_help_guidance expedia.com	Blocked	0	1	public url verified
6	Expedia customer_support.find_travel_contact_guidance expedia.com	Blocked	0	1	public url verified
7	Expedia customer_support.find_travel_help_center_guidance expedia.com	Blocked	0	1	public url verified
8	Expedia customer_support.trip_help expedia.com	Blocked	0	1	public url verified
9	Expedia travel.cancel_third_party_booking expedia.com	Blocked	0	1	public url verified
10	Expedia travel.find_baggage_fee_guidance expedia.com	Blocked	0	1	public url verified
11	Expedia travel.find_booking_help_guidance expedia.com	Blocked	0	1	public url verified
12	Expedia travel.find_cancel_booking_guidance expedia.com	Blocked	0	1	public url verified
13	Expedia travel.find_cancel_change_trip_guidance expedia.com	Blocked	0	1	public url verified
14	Expedia travel.find_cancel_refund_guidance expedia.com	Blocked	0	1	public url verified
15	Expedia travel.find_change_cancel_guidance expedia.com	Blocked	0	1	public url verified
16	Expedia travel.find_change_cancel_help expedia.com	Blocked	0	1	public url verified
17	Expedia travel.find_check_in_guidance expedia.com	Blocked	0	1	public url verified
18	Expedia travel.find_customer_service_guidance expedia.com	Blocked	0	1	public url verified
19	Expedia travel.find_customer_service_help expedia.com	Blocked	0	1	public url verified
20	Expedia travel.find_refund_policy_guidance expedia.com	Blocked	0	1	public url verified
21	Expedia travel.find_refund_status_guidance expedia.com	Blocked	0	1	public url verified
22	Expedia travel.find_reservation_help_guidance expedia.com	Blocked	0	1	public url verified
23	Expedia travel.find_trip_help_center_guidance expedia.com	Blocked	0	1	public url verified
24	Expedia travel.find_trip_management_guidance expedia.com	Blocked	0	1	public url verified
25	Hashicorp customer_support.find_developer_support_guidance hashicorp.com	Blocked	0	1	public url verified
26	Hashicorp customer_support.find_public_help_guidance hashicorp.com	Blocked	0	1	public url verified
27	Hashicorp dev_saas_api.find_api_docs hashicorp.com	Blocked	0	1	public url verified
28	Hashicorp dev_saas_api.find_authentication_docs hashicorp.com	Blocked	0	1	public url verified
29	Hashicorp dev_saas_api.find_rest_api_reference hashicorp.com	Blocked	0	1	public url verified
30	Hashicorp dev_saas_api.find_sdk_docs hashicorp.com	Blocked	0	1	public url verified
31	Hashicorp dev_saas_api.find_status_page_guidance hashicorp.com	Blocked	0	1	public url verified
32	Hashicorp dev_saas_api.find_webhook_docs hashicorp.com	Blocked	0	1	public url verified
33	Help Vrbo customer_support.find_public_help_center_guidance help.vrbo.com	Blocked	0	1	public url verified
34	Intuit finance_bills.find_billing_help_guidance intuit.com	Blocked	0	1	public url verified
35	Intuit finance_bills.find_dispute_help_guidance intuit.com	Blocked	0	1	public url verified
36	Intuit finance_bills.find_fraud_report_guidance intuit.com	Blocked	0	1	public url verified
37	Intuit finance_bills.find_statement_help_guidance intuit.com	Blocked	0	1	public url verified
38	Lyft travel.find_cancel_refund_guidance lyft.com	Blocked	0	1	public url verified
39	Lyft travel.find_trip_management_guidance lyft.com	Blocked	0	1	public url verified
40	Orbitz travel.cancel_third_party_booking orbitz.com	Blocked	0	1	public url verified
41	Realtor real_estate_moving.search_rental realtor.com	Blocked	0	1	public url verified
42	Rent real_estate_moving.inspect_rental_search_surface rent.com	Blocked	0	1	public url verified
43	Rent real_estate_moving.search_rental rent.com	Blocked	0	1	public url verified
44	Travelocity travel.cancel_third_party_booking travelocity.com	Blocked	0	1	public url verified
45	Vrbo customer_support.find_public_help_guidance vrbo.com	Blocked	0	1	public url verified
46	Vrbo customer_support.find_travel_contact_guidance vrbo.com	Blocked	0	1	public url verified
47	Vrbo customer_support.find_travel_help_center_guidance vrbo.com	Blocked	0	1	public url verified
48	Vrbo travel.find_baggage_fee_guidance vrbo.com	Blocked	0	1	public url verified
49	Vrbo travel.find_booking_help_guidance vrbo.com	Blocked	0	1	public url verified
50	Vrbo travel.find_cancel_refund_guidance vrbo.com	Blocked	0	1	public url verified
51	Vrbo travel.find_check_in_guidance vrbo.com	Blocked	0	1	public url verified
52	Vrbo travel.find_refund_policy_guidance vrbo.com	Blocked	0	1	public url verified
53	Vrbo travel.find_reservation_help_guidance vrbo.com	Blocked	0	1	public url verified
54	Vrbo travel.find_trip_management_guidance vrbo.com	Blocked	0	1	public url verified