How not to suck at pen testing 5: Hunt teaming

This part is about a really interesting, highly effective take on network penetration testing advocated by John Strand and his colleague Joff Thyer.

Introducing 'hunt teaming'

Introducing ‘hunt teaming’

John Strand: So, let’s talk about trying to find new areas, and that’s kind of where we are going to start tying this up (see right-hand image). We need to find new areas as network pen testers, and we need to continue pushing things as network pen testers beyond just saying “I hacked a toaster.” We need to find additional things that we can do as part of our network security assessments and network penetration tests. Keep pushing it – they will never be able to automate it, they will never be able to create a standard that says “This is what it is,” if we keep pushing the boundaries.

One of the things that hit me is kind of a weird, strange narrative. One of our good friends works at a very large bank. One day he said “Wait a minute, I want to know if we have any banking backdoors or Trojans that are not being actively detected by our current antimalware suites.” That’s kind of a tall order, right? How would you go about doing that? One of the things he noticed, though, whenever he was reverse-engineering some of these really high-end backdoor Trojans, is they hated Zeus. Anytime Zeus hit that box, those backdoors would immediately kill Zeus. Why? Competitor, right? Take out the competitor. You want that data.

So he got an idea that I think is a bit extreme – I don’t know if I would recommend it – but he modified Zeus so that it wouldn’t spread and then deployed it to every single one of his systems. Then he queried “Who has Zeus installed?” Everybody but those five systems – go investigate those five systems right away. We have noticed many situations where malware gets on a system and one of the first things it does is it looks for other malware and it tries to circumvent it, uninstall it and destroy it.

As network pen testers, aren’t we supposed to be trying to emulate what the bad guys do? Couldn’t we do that? I know this is a bit presumptuous to say I think that network pen testers may come up with some creative ways of trying to find malware in environments. Because we create malware. I think network pen testers may find some creative ways to try and detect attacks internally within environment, because that’s what we do. This area has been around for a while. It’s got a number of different names; we call it “hunt teaming”. The name “hunt teaming” was coined by Seth Meisner and Eric Conrad at SANS, and they have done some tremendously amazing, cool work in this arena of actively trying to find the bad guys – not using AV, not trying to use any number of security suites. But assuming that those security tools have failed, how would you detect it?

As network pen testers, aren’t we supposed to be trying to emulate what the bad guys do?

The reason why we started doing this is because we had a pen test – I think it was early summer – and the customer was completely compromised. We were going through, getting all animated about what we did, and the customer was getting horrified about what we were saying. They said “We must be horrible.” And Joff goes “No, you guys are one of the best customers we’ve ever pen tested, you guys were fantastic!” They about fell over. They were doing everything right and they were still getting compromised. So we were feeling pretty high and mighty, and then they said “We want you guys to come in and find attackers that are using these types of techniques on our network.” And it worked. We found a whole bunch of systems that were compromised in a number of our customers, and it worked out really well.

So I’m going to hand it over to Joff Thyer, and he’s going to go over a couple of slides talking about this. This is kind of a preview of a webcast that we have coming up on Security Weekly on the subject of hunt teaming.

Joff Thyer: Alright, DerbyCon, I love DerbyCon. Thanks John! I’m Joff, I work at Black Hills Information Security. I also co-host Security Weekly, because John doesn’t want to do that anymore – just kidding.

So, hunt teaming – as John was saying, why don’t we flip this model on its head? Let’s find some techniques that we can locate which indicate there’s a compromise in an environment. And I’m not just talking about individual systems, let’s look at the environment network-wide, look at all the systems, look at the behaviors of the systems, look at what’s going on in that environment, look for normal versus abnormal, and see if we can identify abnormal through these various techniques.

Monitoring DNS activity

Monitoring DNS activity

So, one example – DNS C2 (see right-hand image). We’ve got stuff we can leverage to look for DNS C2 activity. We can go to our customer and say “Hey guys, why don’t you give me your DNS cache, why don’t you dump that out of your domain controllers or out of your Bind9 environment, whatever it is? I want to look through it. Why don’t you give me logs of your DNS activity over a period of time? We are going to scan through that and we are going to look for interesting things.” One of the first things I did was pull the script together that just scanned through DNS cache, scanned through DNS logs, and just compared it with the DShield’s suspicious domains list. Let’s go see if something pops out. We’ve got to look up to a known bad domain, so let’s flag that and look at that.

Another thing I looked at was whether the records had very short TTL. Turns out that doesn’t have as much attraction as I thought, because when you are looking at DNS cache, things do have short TTLs. They count down, so it’s not quite as effective. But you can sometimes find indicators of the fluxing activity, with NS records in particular, in the short TTL space.

Pulling back data by means of host analysis

Pulling back data by means of host analysis

So, what else can we do? We can look at host analysis (see left-hand image). And again, we pick on PowerShell, we assume that we have domain admin access, and we just go to customers and say “Look guys, we just need your administrative access, because we’ve got to run some scripts to hunt around your environment and actually look for interesting things.” So we grab that, we’ve got all kinds of ways we can remote: we can use PsExec, we can use WMIC, or we can use WinRM in some cases. And we can run these scripts on the individual stations and pull back data and look at that data.

So, what are we going to pull back? There’s all sorts of things we want to pull back: we can pull back Run keys, RunOnce key, we can pull back installed software list – we can pull back all kinds of interesting things. So we can pull that back into central location, we can analyze it and we can find some really interesting artifacts.

Obtaining firewall logs

Obtaining firewall logs

Also in the host analysis space, we can do HTTP agent strings, for example. Let’s say we have a perimeter of firewall, a proxy – let’s go through all those HTTP agent strings (see right-hand image). What is reaching out and what user agent string are they setting? Let’s take that, let’s sort that, let’s move all the common side stuff aside – we know what Mozilla is, we know what IE is, we know what Safari is. Whatever it might be, look at that and you’ll maybe say “Hey, we got these really low frequency HTTP user agent strings that are really weird. Are these known? Are these supposed to be in our environment? Where are they coming from? What station is using that? Why is it using that HTTP agent string? Let’s chase it down, let’s hunt team it down.” Okay, back to John. That was my five seconds of slides.

John Strand: Like I said, that’s a preview, we don’t have an entire hour on that. Joff had something like 30 slides with all the different scripts, the PowerShell scripts. One of our favorites was dumping the certificates for code signing certificates from your environment and doing an analysis on those. We’ll have an hour of webcast on that coming up.

Read previous: How not to suck at pen testing 4: Bit9 issues and ISR Evilgrade attacks

Read next: How not to suck at pen testing 6: Penetration testers code of ethics

Like This Article? Let Others Know!
Related Articles:

Leave a comment:

Your email address will not be published. Required fields are marked *

Comment via Facebook: