Challenge analysis

Measuring multi-round CAPTCHA completion honestly

Count groups, rounds, proofs, and final actions separately so a one-round recognition result is not reported as a whole successful challenge.

By the easycaptcha team · Published April 11, 2026

A challenge can present multiple groups or rounds before a final proof or acceptance. A result for one group does not establish that the complete interaction finished. Define the level of the metric before counting successes.

Rounds and units of work

For easycaptcha recognition, the documented billing unit is a group: nine binary tiles or an area/drag image. That unit is not automatically the same as a visitor’s entire challenge or final application action.

Keep a hierarchy of identifiers

Use an application-owned interaction identifier to connect the tasks and observations belonging to one intended action. Preserve each remote task ID as well. A single field called solve ID can become ambiguous when several remote operations feed one interaction.

Record which unit completed, which later unit was requested, and whether final acceptance was known. Keep partial completion visible rather than promoting it to a full challenge success.

Use an illustrative sequence

Suppose an interaction requires two recognition groups. The first returns accepted answers; the second fails or remains unobserved. Reporting one successful recognition is legitimate at the group level, but reporting one completed interaction is not established.

The same sequence affects cost. Both submitted groups can contribute to the actual work and accounting, subject to the product’s current billing rules and adjustments. Do not compute cost only from the last observed successful unit.

Compare at consistent levels

A vendor returning a token and a service returning per-group recognition operate at different boundaries. Match the intended output and report the additional application work required. Use the actual price unit rather than assuming every stage is one interchangeable solve.

Test interruptions between rounds and after final submission. Unknown acceptance should not become a guessed failure or success merely because earlier work was observed.

Recognition types defines groups, cost per outcome follows all associated work, and success-rate definitions keeps group-level completion distinct from proof acceptance and final application completion.