Proxies for search results
Search engines answer with a check faster than ordinary sites do. Everything about this job follows from that one fact.
Pacing is the whole game
Five requests per minute per address is a safe starting point. Measure upward from there on your own list.
Going faster does not fail immediately. It fails an hour in, when the pool is already half gone.
Regions multiply everything
Each region is a separate pass over the same keyword list. Twelve regions multiply the volume by twelve, and the pool with it.
Decide which regions genuinely inform a decision before you commit to collecting them all.
Hold the conditions still
Keep the collection time and the address set constant. Changing either produces movement in the numbers that has nothing to do with ranking.
Store enough to compare
Save the date, the region and the depth alongside every position. Without those three fields, comparing two days becomes guesswork.
keywords 40,000 · regions 12 · window 4 h rate per address 5 / min pool sized from the total rate