sponsored
Proxies for large marketplaces
Large marketplaces punish two things: going too fast, and assuming a single nightly pass is enough.
Split the catalogue
A full sweep takes hours. Break it by category and run the parts on their own schedules so a failure costs you one part and the night survives.
Checks arrive early
These platforms ask you to prove you are human sooner than most sites do. Start at five requests per minute per address and raise by measurement.
Verify a sample by hand
After each run, open a random handful of items and compare against what you stored. Gaps here form quietly.
Watch the markup
Product cards get restyled without notice. Anchor on structured data where it exists and keep the parsing rules in one place.
categories split into 12 parts rate 5 req/min per address verification 30 random items after each run