Quantcast

How My Botnet Defeated the Russian Hackers 3: Beating a Competing Bot

As Michael’s bot started getting less successful, there occurred a need for improvement so that it could outperform a competing one made by Russian hackers.

Something went wrong

Something went wrong

Everything worked great for about six months, and then all of a sudden things weren’t as rosy anymore. The client would call and he would say: “We only got two out of seven today; something’s wrong.” He did some research and he discovered through his connections – he’s got lots of connections – that there was a group of Russian hackers that were hired to write a competing bot, and the dealership was someplace out in New Jersey. Competition is good, right? It leads to innovation, and I was kind of thinking: “Yeah, this is going to be fun now, we’ve got an arms race going on here!”

Modified clock sync

Modified clock sync

So, here’s part two of the solution (see left-hand image). What I did different is while I was synchronizing clocks with the sale server I started looking at lag time. And I got to the point where I got really good at estimating how much lag time there would be at the sale time. In other words, what I was essentially doing is I was estimating how many users run the system.

Further bot improvements

Further bot improvements

And with that information, I would not set one attempt to buy the car, but for each bot I would launch maybe between five and seven attempts to buy the car. And based on the amount of lag time that I was going to anticipate at the sale time, I would launch them incrementally just a little bit before the sale time (see right-hand image). This was really successful. Now there would be a number of bots and each one of those basically had a warhead that launched multiple attempts to buy the cars.

Success rate before and after

Success rate before and after

Our success rate prior to making this fix, during the competition, was about 50%. After, we were back right on the money, we were getting every car we wanted (see left-hand image). It stayed that way through the duration of this program.

Financial benefit owing to the bot

Financial benefit owing to the bot

So, how successful was the bot? These are all guesses, because I don’t have any hard facts here. But I know it was an operation for about 40 weeks, and they were buying roughly five cars a day. That’s about 800 cars purchased with this. If you figure the average wholesale cost of the cars they were purchasing, it was probably around $16,000. So, in a 40-week period this bot purchased almost $13 million with the cars (see image above). That has a huge impact on a small dealer like this one. So, this is a great example of not accepting the web as it is, not using browsers where everybody else would, but doing something different and not being afraid to step outside the box a little bit.

What was done well back then

What was done well back then

What would I do differently today if I was going to do this? First, those were things that were done pretty well back then, and things that I still do today. I really like having very lightweight clients: the lighter the better. Everything was easily updated because it was all online; and it was easily distributed, I could make changes on the server, it would get distributed everywhere (see left-hand image). These bot clients were, essentially, just web pages with some JavaScript and stuff going on.

Collecting metrics as an improvement

Collecting metrics as an improvement

One of the things I really definitely would do if I were to do this over is I would build in some analytics and collect metrics (see right-hand image). I would really want to know exactly what our success rate was. I would want to know exactly how much these cars were purchased for. It would be really great to also know how much they were sold for so it could actually show value. That’s something I really wish I had done.

Enhancing vehicle section process

Enhancing vehicle section process

The other thing, I think, that would have been nice if I were to do this over again is build in some process that actually assists in the selection of which vehicles you want to purchase (see left-hand image). In other words, maybe what I would have done is I would have my bot look at Kelley Blue Book and figure out what the good wholesale prices are for cars, and look for discrepancies, look at the ones that are underpriced. That would have been a really good thing to do.

Making the Buy button happen

Making the Buy button happen

The other thing that occurred to me actually within the last week is probably the only thing I really needed to do here was make that “Buy Now!” button happen (see right-hand image). And I could have done that simply by making the server act kind of like a proxy so that as HTML is coming in with the greyed-out button, I could have just replaced it with a real button and sent it off to the browser. That probably would have worked. The problem there is that, conceivably, you could have bought cars before the purchase time, and that may have been allowed, but that’s something you don’t want to do for the same reason you don’t want to buy cars that don’t exist. You don’t want to show your hand.

Some site architecture differences

Some site architecture differences

The website, the target, was a very traditional website. It used HTML forms which are really easy for me to emulate or submit using just PHP and cURL. Today you don’t find that so often. You find a lot of JavaScript, you find a lot of AJAX. There’s a lot of JavaScript validation of form data before it’s submitted (see left-hand image). It makes it a lot harder to do this kind of thing today.

Applying the Task Queue

Applying the Task Queue

The approach that I take now is to end up with a Task Queue, which is basically a table in a database that keeps track of what needs to be done (see right-hand image). And there’s a web interface into that. In this particular case, my client would, essentially, be loading a Task Queue. That Task Queue would be fed to individual computers, which I refer to as “harvesters”, and they can exist anywhere. They can be in the cloud, they can be in a closet, they can be in your office – they can be anywhere.

The use of iMacros

The use of iMacros

What I have them do now, since there’s so much more complexity in websites and so much more use of client-side scripting, is I do a lot of stuff in iMacros (see left-hand image). That’s the most amazing tool! It’s just an add-on for your browser that, essentially, lets you create a macro for your browser that you can display over and over again. And what I do now is the “harvesters” will dynamically create the macro so you can get them to do some very specific things. Once I learned how to do that, there was not a single website on the planet I could not manipulate. So, that’s what I do now. I actually communicate through Firefox, so it’s very easy for me to emulate human activity now with bots. I would have them hit the sale server. The difficulty there would be to get the timing done correctly, but I think that could have been done. And then the “harvesters”, after they do their thing with the sale server, the target server, report back to the bot server, and the queue is updated – that’s how you can tell what the results were of what you did.

Other presentations

Other presentations

If you’re interested in how that kind of stuff works, go on YouTube and look up my Defcon 17 talk (see right-hand image), because that’s all about manipulating iMacros and that way to do screen scrapers for very difficult-to-scrape sites or difficult-to-automate kind of sites.

So, that’s basically it. Thank you to the CFP Goons. Thank you!
 

Read previous: How My Botnet Defeated the Russian Hackers 2: The Car-Purchasing Bot

Like This Article? Let Others Know!
Related Articles:

Leave a comment:

Your email address will not be published. Required fields are marked *

Comment via Facebook: