I used to pick proxy providers by comparing per-GB prices. After running cost breakdowns across several scraping projects, it turned out to be the wrong metric entirely. The number that actually determines your budget is cost per usable page – and it’s driven far more by success rate, retry behavior, and failure latency than by the sticker price on a vendor’s pricing page.
A proxy plan at $2.50/GB looks cheaper than a pay-per-success API at $1.30 per thousand results. But once I started tracking retries, bandwidth burned on block pages, and compute time lost to 20-second timeouts, the math flipped completely.
The Math Behind Retries
The relationship between success rate and total cost isn’t linear, and that’s what surprised me when first running the numbers. Expected attempts per successful page follows a simple formula. Divide 1 by your success rate. At 90% success, you need about 1.11 attempts per page. Drop to 70%, and that jumps to 1.43 – a 43% increase in attempt volume.
Each failed attempt costs you in two ways. Under per-GB billing, every block page, partial response, or CAPTCHA challenge still consumes bandwidth. In my tests, a successful page averaged about 1 MB while failed attempts transferred around 200 KB of block page HTML. At 70% success, expected bytes per success work out to roughly 1.085 MB – not 1 MB. Across millions of pages, that gap adds up fast.
The second cost is less obvious but often larger. Failed attempts eat compute time. I measured successful fetches completing in about 1.2 seconds on average, but timeouts on defended sites consistently ran 20 seconds or more. At 70% success, expected time per delivered page ballooned to nearly 10 seconds, compared to around 3 seconds when I switched to a high-success proxy service that handles retries internally. If you’re paying for worker-seconds or capped on concurrency, slow failures will eat your budget.
More IPs, Fewer Blocks, Lower Cost
Anti-bot systems don’t just look at what you’re requesting. They track how often the same IP appears in a given time window. Hit a site too hard from a small set of IPs and you’ll trigger HTTP 429 rate limiting, CAPTCHAs, or outright bans. All of those cause retries, and retries are where cost spirals.
A bigger IP pool means each individual IP gets used less often, which keeps success rates high and retry volumes low. The major providers’ published pools vary widely. Bright Data advertises 150M+ residential proxy IPs, Oxylabs claims 175M+, and Decodo offers 115M+ ethically-sourced residential IPs with sub-0.6s average response times and a 99.86% claimed success rate. In practice, I measure each provider’s effectiveness per target domain rather than trusting the headline numbers. But the pattern held across every project: more unique IPs, less per-IP pressure, fewer blocks, lower cost.
Pay-Per-Success vs. Per-GB Billing
Two billing models dominate the proxy market, and they handle failure cost completely differently.
Per-GB billing charges for all traffic, including retries. You pay for every block page, every CAPTCHA response, every partial download your scraper discards. When my success rate dropped on a particularly defended e-commerce target, the effective cost per usable page nearly tripled. And on one project where my scraper accidentally fetched images and JS bundles, the bandwidth bill jumped by an order of magnitude. Expensive lesson.
Pay-per-success billing shifts the retry burden to the vendor. Services like Bright Data’s Web Unlocker, ZenRows, Oxylabs Web Scraper API, and Zyte API charge only for delivered successful responses. The vendor handles retries, CAPTCHA solving, and IP rotation internally. You see higher per-call latency (ScraperAPI reports 4–12 seconds on average, sometimes up to 60s), but you don’t pay for failures happening behind the curtain.
The nuances matter, though. Vendors define “success” differently. ZenRows counts 404 and 410 responses as successful and billable – reasonable, since those are valid data signals. ScraperAPI bills for 200 and 404 status codes but warns that cancelling a request before its ~70-second retry window may still incur a charge. Bright Data’s documentation notes that enabling certain custom controls on Web Unlocker can switch billing from success-only to charging for all requests. I learned that one the hard way – read the fine print.
When I ran the numbers on a defended target needing one million pages, the gap was stark. Pay-per-success API at $1.30/1k results with 98% success came to about $1.39 per thousand pages. The same workflow on brute-force rotating residential proxies at $2.50/GB and 70% success cost roughly $2.98 per thousand – more than twice as much, driven by retries and wasted compute on slow failures.
Latency Costs More Than You Think
Latency directly affects your scraping budget, and it’s the variable I underestimated most early on. When a failed attempt hangs for 20 seconds versus a successful one completing in 1.2 seconds, compute cost per delivered page skews hard. At 70% success with 20-second failure timeouts, I measured roughly 9.8 seconds of wall time per successful page. On a managed web scraping API with 98% success and vendor-side retries, that dropped to about 3.1 seconds.
If your infrastructure bills by worker-seconds, that 3x gap translates directly to 3x compute cost. And if concurrency is capped, slower pages mean more parallel workers to hit the same throughput deadline.
Vendor-claimed latency for raw proxy networks tends to be low – Bright Data reports ~0.7s, Oxylabs ~0.6s, Decodo under 0.6s. Those numbers matched what I saw for the proxy connection itself. But they don’t apply to unblocking APIs where the vendor handles retries internally. The trade-off is explicit – higher per-request latency in exchange for fewer total requests.
A Smarter Routing Strategy
The most cost-efficient setup I’ve landed on doesn’t use a single proxy tier for everything. It routes by target difficulty.
Easy targets – sites with no anti-bot protection, public APIs, simple HTML pages – go through cheap datacenter proxies. Moderately defended sites get residential proxies. Heavily protected targets get escalated to an unblocker or pay-per-success API.
The routing decision should be data-driven. Track these metrics per target domain:
- Data-level success rate – validated content, not just HTTP 200s. A 403 from Akamai wrapped in a 200 response is still a failure
- Attempts per success at p50 and p95, broken down by failure reason (timeout, 403, CAPTCHA)
- Cost per usable record on a rolling 24-hour and 7-day window
- Latency percentiles (p50, p90, p99) and the fraction of 429 responses followed by a Retry-After wait
- Bytes per success vs. bytes per attempt – the gap between these two numbers tells you how much bandwidth you’re wasting on failures
When your measured success rate on a domain drops below the break-even point where per-GB cost exceeds pay-per-success, escalate. That threshold isn’t static – it shifts as anti-bot defenses evolve and traffic patterns change.
Your cheapest configuration today can become your most expensive one tomorrow. I’ve watched it happen. Anti-bot systems get updated, rate limits tighten, IP reputations degrade. The only reliable way to keep proxy spend under control is to stop optimizing for the lowest unit price and start measuring what actually matters: the cost of each delivered, validated, usable page.
