Why Proxy “Success Rate” Numbers Disagree

A proxy success rate is legible only with its unit, numerator, denominator, validator, retry policy, and taxonomy version. Change one of those and the number can move while every raw network event stays exactly the same.

Two honest proxy tests can disagree because they are not answering the same question.

Six ways the reported rate changes

1. Request, attempt, or job

A browser job can produce one main-document request, dozens of subresource requests, and more than one physical attempt. “95 successful requests out of 100” is not comparable to “95 completed jobs out of 100.” Pick the operational unit first. Keep job_id, attempt_id, and request role separate so more than one valid view can be calculated later.

2. First attempt or final retry result

A job that fails once and passes on retry is a completed job and a failed first attempt. Job completion tells you whether the system eventually delivered. First-attempt success tells you how much retry budget, time, and traffic it needed. Publishing only the final job result hides that cost. Retries also share route, pool, session, destination, and time-window state, so do not assume they are independent samples.

3. Transport, HTTP, or workload success

For HTTPS through an HTTP proxy, a successful CONNECT means the proxy formed a tunnel. It does not mean the page loaded. RFC 9110 (opens in a new tab) is precise about that boundary. A requested-transfer 2xx means the client received a successful-class HTTP response after tunnel setup. It does not prove which intermediary generated it or that the payload satisfied the task. A 200 can still carry the wrong regional variant, a maintenance document, incomplete data, or a response that fails the declared validator.

Client defaults add another split. On a completed requested transfer, cURL can exit 0 for a 4xx or 5xx unless options such as --fail (opens in a new tab) change status handling. Playwright's page.goto() (opens in a new tab) does not throw merely because a valid response is 404 or 500, while requestfailed (opens in a new tab) is for a request where the browser did not obtain an HTTP response. Transport completion, status, and workload validation belong in separate columns.

4. Started attempts or post-hoc eligible attempts

Removing timeouts, authentication failures, unfinished work, or unsupported scenarios after seeing the result usually raises the percentage. A genuine fixture or harness fault can be excluded, but the rule should exist before the run and the count and reason should be published. Preserve every started attempt in the raw log; a later report can mark a predeclared exclusion without erasing the event.

5. Micro or macro average

A micro average pools all successes and attempts, so high-volume scenario cells carry more weight. A macro average computes each predeclared scenario rate first, then averages the cells. Equal weighting is appropriate only when each cell is intentionally equally important to the claim. If an easy, high-volume workload dominates the sample, the micro result can look much better than an equal-cell macro result. Publish the weighting rule and per-cell x/n.

6. Taxonomy version

An HTTP 403 establishes refusal under RFC 9110 (opens in a new tab). It does not establish bot detection, bad proxy credentials, or provider failure. If one adapter maps the raw 403 to bot_detected and another maps it to http_403, their category charts disagree before the network behavior does. Store the raw fact separately from normalized_outcome, taxonomy_version, and success_definition_id.

One dataset, several correct percentages

The interactive normalizer starts with eight fabricated attempts across six jobs. Edit the aggregate counts locally and watch the headline rates move. The synthetic receipts remain visible so the numerator and denominator never disappear behind the percentage. Nothing is stored or sent.

The range is 16.7% to 100%, and nothing in the event table changed. The 8/8 reading says only that the proxy returned an HTTP response during CONNECT. A 407 is still a response; it is not a successful tunnel or completed workload. The 4/7 conditional rate is useful for tunnel-established diagnostics, but it removes the failed authentication attempt and must not replace 4/8 when reporting the end-to-end requested-transfer 2xx rate. For a purchasing decision, use the predeclared validated-attempt or completed-job metric that matches the workload.

The raw fact can outlive the label

In one authorized production system, two browser-worker adapters assigned different normalized labels to the same observable condition: an HTTP 403 without an explicit cause flag. One recorded bot_detected. The other recorded http_403. The raw fact was stable; the taxonomy was not.

When publishing a similar example, disclose only what establishes the semantic drift. Remove target, customer, engine, and target-specific rule details.

Store evidence before interpretation

run_id, scenario_id, job_id, attempt_id, attempt_index
provider_product_alias, client_adapter, client_version
request_role, requested_geo, session_policy
started_at, timeout_budget_ms, concurrency
proxy_response_status, requested_transfer_status
transport_error_namespace, transport_error_code
validator_id, validator_version, validator_pass
raw_assertion_evidence, duration_ms, stage_reached

Separating raw evidence from the taxonomy lets you rerun a new classification over old events. It also lets a reviewer decide whether the network changed or only the label did.

Publish the full metric contract

• Freeze target class, geography, client, timeout, concurrency, session policy, and validator for every candidate.
• Interleave paired tests inside the same time window.
• Use an equal-cell macro average only when every scenario cell is intentionally equally important; otherwise publish the predeclared weights and estimand.
• Also publish the micro rate, retry-assisted job completion, tunnel and transfer-response rates, and raw x/n for each.
• Exclude only predeclared harness or fixture faults, with every excluded count and reason disclosed.
• Keep raw HTTP status separate from inferred cause; attribution stays unknown unless controls or authorized owner evidence support it.

Print the fraction next to every percentage. Also print the target mix, test window, sample count, timeout, client version, retry budget, validator, taxonomy version, and exclusions. A legacy all-attempt pass rate may remain useful for a trend inside one stable harness; it is not interchangeable with a normalized first-attempt rate from another system.

Use the 30-minute workload benchmark to produce a controlled first sample, the status-code guide to preserve protocol facts, and the cURL versus Playwright diagnostic when clients disagree.

Place every provider percentage beside current rankings, underlying reports, and the public methodology. A number without that contract is not ready to compare.