Proxies for web scraping
The case everything else is a variation of. Most of the trouble comes from two decisions made at the start: the rate you run at and the spread of the addresses you run on.
Size the pool from the rate
The platform counts requests arriving from one address per minute. Your thread count is invisible to it.
Divide the total request rate by the rate one address is allowed. Double the result. That is the working pool.
Why runs stall halfway
A run that starts cleanly and dies at the midpoint is almost always hitting a limit that accumulates: a daily cap, or a counter that resets on a schedule you have not measured.
Log the moment of the first refusal across several runs. If the timing repeats, the cause is a window, and the fix is pacing.
Anchor the parser on something stable
Class names change when a site is restyled. Structured data changes far less often, and most large platforms publish it.
Keep the parsing rules in one file so a redesign costs a minute of work.
What to check before buying
Subnet spread, traffic terms, and how quickly dead addresses get replaced. Those three answers predict the run better than the price does.
total rate = requests / minutes per address = allowed rate pool = (total / per address) x 2