Delving into human solver backends and moving on to the summary, Marti Motoyama provides retrieved statistics and draws general conclusions on the subject.Now we’re going to take a look at the human solver backends to get a sense of the worker experiences. We signed up as a solver on two sites, knowing that they are the backends for several services included in our study. The first is PixProfit, which is the backend for DeCaptcher. This is an example of what the interface looks like for those workers (see upper part of the right-hand image). The second is Kolotibablo, which is the backend for Antigate. Here is an example of what that interface looks like (bottom part of the image).
What we did was we set up scripts to scrape the presented images and then pipe them back into the corresponding service. So, for example, we pulled out images from Kolotibablo and then used Antigate to solve them. There is minimal due diligence to sign up for these services. We realized later that PixProfit actually had about 30 or so screener CAPTCHAs that they first present you with. Kolotibablo threaten you with an IP ban unless you present them with a valid WebMoney purse. WebMoney is an e-currency, and that’s how they pay their workers. And they do actually do this IP ban.As you solve CAPTCHAs, these services monitor your accuracy. I didn’t mention this, but these APIs also allow the customers to report when a worker has solved a CAPTCHA incorrectly. Additionally, you are not allowed to cash out until you solve anywhere between 1000 and 3000 CAPTCHAs. Here’s a breakdown on the targeted sites (see right-hand image). We can see that the CAPTCHA distribution is highly skewed. The top 5 CAPTCHAs on PixProfit comprise 90% of the data, while the top 5 for Kolotibablo comprise ¾ of the data. Between the two, however, we see that Microsoft is highly targeted, which is probably the reason why they’re rotating their CAPTCHA so often. However, the remainder of the top 5 really differs by region. Kolotibablo, which is Russian, has a fair number of Russian sites in the top 5. PixProfit, on the other hand, has more of a Western flavor to it: Google, Yahoo, AOL. So, we’ve done all this analysis at this point, and it’s time to step back and really ask ourselves: do CAPTCHAs work? In terms of differentiating between computers and humans – yes, our exploration of software solvers really seems to confirm this. However, do they prevent large-scale abuse? No, there’re a lot of tools out there that incorporate in human-based CAPTCHA solver services (see left-hand image).
We have shown that these services have a tremendous amount of capacity, are accurate and have good response times. Here’s a quote by Mr. “E”: he says that he solves on the order of about 100,000 CAPTCHAs daily. This means there’s over 100,000 daily instances of abuse taking place on a site: account sign-ups, spam emails sent, or forums postings made.However, we don’t want you to draw the conclusion that CAPTCHAs are a failure – that’s a mistake (see image). Our work is intended to suggest that CAPTCHAs are an economic impediment and not necessarily a technical one. What they do is they add friction to an attacker’s business model; they’re valuable in weeding out those attackers whose business models are just not cost effective in the face of needing to bypass CAPTCHAs. After all, an attacker is going to have to expand some capital to handle the problem or the barrier of a CAPTCHA, and because of this the value of the resource that’s being abused has to be much greater than the value of the CAPTCHA.
Furthermore, CAPTCHAs are a very low-impact defense mechanism, well, not very, but legitimate users remain willing to deal with them, and CAPTCHAs are certainly serviceable as the first line of defense. CAPTCHAs can be used in conjunction with a stronger defense mechanism, like SMS messages, should a user exhibit suspicious behaviors.
So, in our work we’ve evaluated CAPTCHAs from an economic perspective, and we have shown that human solver services are mature and cost effective. We hope that in the future people will give more consideration to evaluating defense mechanisms from both technical and economic perspectives.