Market analysis
Evaluating routed CAPTCHA providers without losing causality
Keep selected providers, routing policy, task families, and unknown outcomes visible when comparing an aggregator with direct integrations.
By the easycaptcha team · Published
An aggregator can choose different upstream services for different work or over time. Its overall success rate can change because the provider mix changed, even when the behavior of each upstream remained similar. Keep the routing strategy and selected provider where exposed in the evaluation record.
Routing and populations
AnySolver documents its routing platform. Review the current decision rules rather than assuming labels such as cheapest or fastest match the criteria your application would use.
Separate contract adaptation from direct performance
An intermediary may transform inputs, choose fallback, or expose a normalized result. A routed trial is therefore evidence about that complete integration, not automatically evidence about the direct provider’s API.
Record the public request contract, relevant routing configuration, task family, expected output, and final application outcome. Keep the same intended operation when comparing direct and routed attempts.
Make fallback work visible
A definite rejection and an uncertain submission require different decisions. If the first upstream may have accepted work, an automatic fallback can create another paid operation. Count all work attributable to the original intended action, including late results and adjustments.
Do not report only the final successful result while dropping the earlier attempts that consumed time or credit. A useful end-to-end metric includes the whole routing sequence and its unresolved states.
Compare cohorts, not slogans
Use representative site configurations and distinguish tokens, recognition, cookies, and other artifacts. Compute cost with the billed unit and report observation coverage. A broad task catalog is a coverage claim, not a benchmark for your particular configuration.
Document what the integration exposes and what remains opaque. If the selected upstream is not visible, state that limitation rather than assigning exact outcomes to a guessed provider.
The AnySolver comparison discusses the architectural difference, the directory links discovery evidence, and fair benchmark design shows how to compare known application outcomes without treating a routing change as proof of model improvement.
Frequently asked questions
Can a routing change distort provider comparisons?
Yes. Routing can send different populations to different providers. Compare representative cohorts before attributing outcome changes to provider quality.
How should multiple solver providers be integrated?
Use explicit adapters for credentials, request shapes, states, results, and errors. Avoid assuming similar task names imply interchangeable contracts.
How can easycaptcha fit into my integration tests?
Use the documented recognition or token API for the hCaptcha stage, preserve task IDs, and inspect terminal results. Measure verifier acceptance and application completion separately from job completion.