Bruce Schneier’s public lecture: Liars and Outliers 3. Scaling of Societal Pressures

Bruce Schneier explains the variability of cooperator and defector notions depending on the society group, and outlines the issue of societal pressures scaling.

'Liars and Outliers' book cover And there are a lot of directions to take this research. One of the problems I had writing this book is that very quickly the topic became unbounded. There are so many things you could talk about that you have to put a box around what you’re saying and decide that some things are in the box and out. And the toil of this talk is actually much worse to have that much less time. So here are sort of some of the issues that I do talk about.

I’ve been saying that it’s group interest vs. self interest, but actually that’s really very simplistic. It is often not, often there are several different competing interests in operation at any one time, we are often members of several different groups, and each group will use its own societal pressure to get the individual to comply, to be cooperative within that group.

Let’s take an example of a group being an organized crime syndicate. So in a group of criminals, a criminal organization, being cooperative means sticking up for your fellow criminal, not ratting on him to the police. And being a police informant is being a defector, being a parasite within the criminal community. But in the larger community of society as a whole, cooperating with the police is being cooperative. And the person who rats on his fellow criminals to the police is being a cooperator, and it’s the criminals who are the defectors.

There are a lot of examples about it. A less stark example might be a person who is employed by a corporation, who has to choose between cooperating with the corporation and cooperating with the society, cooperating with one community he’s a part of, or another community he’s a part of. And people are good at this, but it gets very complicated.

There’s a lot of interactions between different societal pressures – how morals interact with reputation, how morality and the law interact. And we see that in the Internet age trying to pass laws against movie and music pirating – the real problem there is that the community has a community norm where sharing music and movies is being cooperative, that’s what you do on the Internet, you share digital files, and a law that goes against that societal norm is very hard to enforce. You see the same dynamic played out in some countries on drug laws, where the social norm is one thing, and the law is exactly the opposite. So that’s sort of how those interact, interactions are interesting.

How each type of societal pressures scales is really interesting. Morals tend to be very close, and this is rightfully so, we’re more morally protective of out intimates, our friends, our communities, our country, people who look like us, people who speak our language, all of humanity. As those circles get bigger, morality tends to fade. There’s an exception, we do have universal human morals, again, unlike any other species. But a lot of morality is in-group vs. out-group, and where you draw that in-group / out-group line will determine how well morals set a pressure scale.

And reputation doesn’t scale very well. The reputational pressures that work in small communities fail in large anonymous cities. Laws also have a different scaling mechanism. In a lot of ways the natural scaling mechanisms of laws determine how big our political units are, can we have a country the size of the United States or the size of the European Union, or are smaller kingdoms the norm, city-states, or even tribes?

How technology helps these scale is also really interesting. We have a lot on the Internet of technological enhanced reputational societal security systems. When you think of eBay feedback, it’s really interesting; it’s fundamentally a reputational-based system. If you as a merchant cheat me, I’ll write something bad about you in public. And it was interesting to watch over the years eBay having to refine that system as people figured out how to cheat it, how to game it. But it’s a reputational system.

Some people use recommender systems to blackmail businesses

Some people use recommender systems to blackmail businesses

We see that in some of the recommender systems. I assume Yelp is here. There are a number of sites on the Internet where you can rate businesses, you can rate restaurants and all sorts of other businesses. I use it pretty regularly. If I’m going to a city, I want to know where to eat dinner, I’ll go to Yelp and look for recommendations, look at reviews. That’s actually spawning an extortion racket. There are people who will go to businesses and say: “Give me a discount or I will bad mouth you on Yelp”.

How technology affects these reputational systems both in allowing them to scale and offering new ways to attack them is really interesting.

The effects of group decision-making – I think, it’s really important here. I’ve been talking about these cooperation defect decisions as made by individuals, but very often it’s groups who make them, and a company decides whether to pay its taxes or not, or to overfish or not. How this plays out within common groups is interesting, within organizations. I have an entire chapter where I talk about how this works within corporations, another chapter about how it works within government institutions.

Back to scale, looking at how this scaling of society gets larger and more complex. Then lastly, what happens when the defectors are in the right? I mentioned the criminal organization issue, and that’s really interesting because being within a criminal organization, being the parasite means being a police informant, that’s what a defector does, but in an abstract moral sense he’s doing the right thing. Think of any type of conscientious defector, civil disobedience. In the United States in the 1800s slavery was ended, as that was happening the abolitionists who helped free the slaves were the defectors, they were breaking the law. Similarly, in the United States, in Europe, the ‘Occupy movement’ are defectors in society.

I don’t have time to sort of just go into all of those complications, although they’re really interesting.

The one I do want to talk about, because I think it matters most to the people in this room, are the effects of technology. Technology gets back to scale, technology allows society to scale, and scale affects societal pressures, it affects it in a bunch of ways: more people, I’ve already mentioned how anonymity affects reputation, but more people affect societal pressures in a lot of different ways increase complexity; more complex systems; new social systems, the fact that Facebook exists and Yelp exists; new security systems, technology will invent security systems.

Increased intensity, this is an odd one – this is basically what I’m trying to say here, that technology allows defectors to do more damage.

Think about my balance again, we have some balance of, I don’t know, banking fraud, some amount of banking fraud that society deems acceptable, we’ve instituted societal pressures to maintain that level. Someone invents Internet banking, the Internet banking criminals realize that there are new ways to commit a fraud – and the balance changes. The same number of bank fraudsters can do more damage because they can automate their crimes on the Internet. That’s what I mean by ‘intensity’. This is the entire debate about terrorism and weapons of mass destruction right here. We as society are worried that the same very few number of terrorists that exist can do much more damage with these modern weapons. Technology increases frequency, it increases distance – you can have the bad actors operating from a longer distance.

Read previous: Bruce Schneier’s public lecture: Liars and Outliers 2. Societal pressures
Read next: Bruce Schneier’s public lecture: Liars and Outliers 4. Security Gap Concept

Like This Article? Let Others Know!
Related Articles:

Leave a comment:

Your email address will not be published. Required fields are marked *

Comment via Facebook: