Measurement
How to measure CAPTCHA success rate
Separate solver completion, verification acceptance, and user completion. Compare CAPTCHA outcomes by sitekey, challenge type, latency, and cost.
By the easycaptcha team · Published
A service can report that a CAPTCHA job finished successfully while the visitor still cannot complete the form. That gap makes success-rate figures difficult to compare across providers and applications. A useful measurement starts by explaining which stage of the journey the number represents.
Defining success
“Success rate” can mean the share of jobs that returned a result, the share of proofs accepted by a verifier, or the share of visitors who completed a form. Those are different measurements. A useful report names the stage and denominator rather than placing one percentage next to the product name.
For an evaluation, record submissions, terminal solver results, observed verifier outcomes, and completed application actions separately. Keep timeouts and unknown outcomes visible. If a test stopped waiting, that does not establish that the server-side job failed.
Define the events before collecting them
| Metric | Numerator | Denominator |
|---|---|---|
| Job completion rate | Jobs with a completed result | Submitted jobs in the evaluation cohort |
| Observed acceptance rate | Proofs the application accepted | Proofs submitted with an observed verification outcome |
| End-to-end completion | Completed application actions | Started application attempts in the cohort |
| Outcome coverage | Attempts with a known final outcome | Attempts expected to produce a final outcome |
This is a proposed measurement model, not a benchmark of easycaptcha or another vendor. Choose observation windows that let your jobs finish, and publish how many results remain unknown. A report with high acceptance and low outcome coverage may be hiding its hardest cases.
Keep challenge populations comparable
Segment results by the site’s configured key, challenge type, integration mode, and relevant environment. Avoid pooling an easy test configuration with a production flow and calling the average representative. A change in the challenge mix can change an overall rate even when performance within each group stays the same.
For easycaptcha recognition requests, the optional sitekey attributes the challenge to its originating site. Use task IDs to correlate your own application outcome. Do not infer that every task on a site has the same question or difficulty.
Measure latency and cost alongside outcomes
Report a distribution of completion times, such as the median and a higher percentile, instead of only an average. Show whether the number includes queuing, your polling delay, and application submission. A faster polling loop can make the client observe a result sooner without making the solver itself faster.
Compare cost per intended outcome using the actual billing unit. Recognition groups and token tasks are different products. Multiple groups in one challenge can consume multiple recognitions. Use current pricing, retain refunds or reversals in your accounting, and state which mode was tested.
Use feedback after the outcome is known
The easycaptcha SDKs provide feedback methods for observed outcomes. A token arriving is not sufficient reason to send positive feedback. Wait until your integration has enough evidence, and keep unknown results distinct from rejection.
Retain identifiers and aggregate metrics rather than tokens, passwords, or private form content. Review the same cohort after changing a challenge provider or application flow. Our browser-testing guide helps separate deterministic functional checks from live performance evaluation.
Frequently asked questions
What is the difference between job completion and challenge acceptance?
Completion means a job produced a result. Acceptance means the verifier accepted that result. The requested application action can still fail afterward.
How should unknown outcomes affect success-rate reports?
Report them separately and disclose observation coverage. Do not silently count missing acceptance evidence as either success or failure.
How can I evaluate easycaptcha on my own workload?
Start with a small integration trial using representative hCaptcha tasks. Compare observed acceptance, final outcomes, and costs in the actual billing unit rather than relying on a feature table alone.