The Social Dilemma

Interesting quotes

“Nothing vast enters the life of mortals without a curse.“ — Sophocles

Tristan Harris (presentation at Google) “Never before in history have 50 designers—20- to 35-year-old white guys in California—made decisions that would have an impact on two billion people. Two billion people will have thoughts that they didn’t intend to have because a designer at Google said, ‘This is how notifications work on that screen that you wake up to in the morning.’ And we have a moral responsibility, as Google, for solving this problem.“

13:15 “The classic saying is ‘If you’re not paying for the product, then you are the product.’”

14:18 “We’re the product. Our attention is the product being sold to advertisers.“

14:25 Jaron Lanier “It’s the gradual, slight, imperceptible change in your own behavior and perception that is the product.“

17:54 “What do they do with that data? They build models that predict our actions, and whoever has the best model wins.“

19:04 Three main goals: engagement, growth, advertising

21:18 Jaron Lanier “We’ve created a world in which online connection has become primary, especially for younger generations. And yer, in that world, any time two people connect, the only way it’s financed is through a sneaky third person who’s paying to manipulate those two people. So we’ve created an entire global generation of people who are raised within a context where the very meaning of communication, the very meaning of culture, is manipulation. We’ve put deceit and sneakiness at the absolute center of everything we do.“

21:58 “Any sufficiently advanced technology is indistinguishable from magic.“ — Arthur C. Clarke

24:34 Tristan Harris “You don’t know when you’re gonna get it or if you’re gonna get something, which operates just like the slot machines in Vegas. It’s not enough that you use the product consciously, I wanna dig down deeper into the brain stem and implant, inside of you, an unconscious habit, so that you are being programmed at a deeper level. You don’t even realize it.“

27:54 “We’re just zombies, and they want us to look at more ads so they can make more money.“

28:04 Shoshana Zuboff “Facebook conducted what they called ‘massive-scale contagion experiments.’ How do we use subliminal cues on the Facebook pages to get more people to go vote in the midterm elections? And they discovered that they were able to do that. One thing they concluded is that we now know we can affect real-world behavior and emotions without ever triggering the user’s awareness. They are completely clueless.“

30:38 Tristan Harris “Social media isn’t a tool that’s just waiting to be used. It has its own goals, and it has its own means of pursuing them by using your psychology against you.“

30:55 “There are only two industries that call. their customers ‘users’: illegal drugs and software.“ — Edward Tufte

47:41 Cathy O’Neil (Data Scientist) “I like to say that algorithms are opinions embedded in code, and that algorithms are not objective. Algorithms are optimized to some definition of success. So if you can imagine, if a commercial enterprise builds an algorithm to their definition of success, it’s a commercial interest. It’s usually profit.“

48:03 Jeff Seibert (Twitter Former Executive, Serial Tech Entrepreneur) “You are giving the computer the goal state, ‘I want this outcome,‘ and then the computer itself is learning how to do it. That’s where the term ‘machine learning‘ comes from. And so, every day, it gets slightly better at picking the right posts in the right order so that you spend longer and longer in that product. And no one really understands what they’re doing in order to achieve that goal.“

48:25 Bailey Richardson (Instagram Early Team) “The algorithm has a mind of its own, so even though a person writes it, it’s wirtten in a way that you kind of build the machine, and then the machine changes itself.“

48:35 Sandy Parakilas (Facebook Fomer Operations Manager, Uber Former Product Manager) There’s only a handful of people at these companies, at Facebook and Twitter and other companies… There’s only a few people who understand how thoses systems work, and even they don’t necessarily fully understand what’s gonna happen with a particular piece of content. So, as humans, we’ve almost lost control over these systems. Because they’re controlling, you know, the information that we see, they’re controlling us more than we’re controlling them.“

54:40 Jarlon Lanier “One of the ways I try to get people to understand just how wrong feeds from places like Facebook are, is to think about the Wikipedia. When you go to a page, you’re seeing the same thing as other people. So it’s one of the few things online that we at least hold in common. Now, just imagine for a second that Wikipedia said, ‘We’re gonna give each person a different customized definition, and we’re gonna be paid by people for that.‘ So, Wikipedia would be spying on you. Wikipedia would calculate, ‘What’s the thing I can do to get this person to change a little bit on behalf of some commercial interest?‘ Right? And then it would change the entry. Can you imagine that? Well, you should be able to, ‘cause that’s exactly what’s happening on Facebook. It’s exactly what’s happening in your YouTube feed.“

55:29 “When you go to Google and type in ‘Climate change is,‘ you’re going to see different results depending on where you live. In certain cities, you’re gonna see it autocomplete with ‘climate change is a hoax.‘ In other cases, you’re gonna see ‘climate change is causing the destruction of nature.‘ And that’s a function not of what the truth is about climate change, but about where you happen to be Googling from and the particular things Google knows about your interests.“

55:55 Tristan Harris “Even two friends, who are so close to each other, who have almost the exact same set of friends, they think, you know, ‘I’m going to news feeds on Facebook. I’ll see the exact same set of updates.‘ But it’s not like that at all. They see completely different worlds because they’re based on these computers calculating what’s perfect for each of them.“

57:33 Justin Rosenstein “And then you look over at the other side, and you start to think ‘How can those people be so stupid? Look at all of this information that I’m constantly seeing. How are they not seeing that same information?‘ And the answer is, ‘They’re not seeing that same information.‘“

58:20 Justin Rosenstein “So many of the problems that we’re discussing, like, around political polarization, exist in spades on cable television. The media has this exact same problem, where their business model, by and large, is that they’re selling our attention to advertisers. And the Internet is just a new, even more efficient way to do that.“

59:41 Guillaume Chaslot (YouTube Former Engineer, IntuitiveAI CEO, AlgoTransparency Founder) “People think the algorithm is designed to give them what they really want, only it’s not. The algorithm is actually trying to find a few rabbit holes that are very powerful, trying to find which rabbit hole is the closest to your interest. And then if you start watching one of those videos, then it will recommend it over and over again.“

01:02:46 “We’ve created a system that biases towards false information. Not because we want to, but because false information makes the companies more money than the truth. The truth is boring.“

01:03:03 Tristan Harris “It’s a disinformation-for-profit business model. You make money the more you allow unregulated messages to reach anyone for the best price.“

01:06:41 Roger McNamee “One of the problems with Facebook is that, as a tool of persuasion, it may be the greatest thing ever created. Now, imagine what that means in the hand of a dictator or an authoritarian. If you want to control the population of your country, there has never been a tool as effective as Facebook.“

01:09:00 “Algorithms and manipulative politicians are becoming so expert at learning how to trigger us, getting so good at creating fake news that we absorb as if it were reality, and confusing us into believing those lies. It’s as though we have less and less control over who we are and what we really believe.“

01:09:49 Tristan Harris “Imagine a world where no one believes anything true. Everyone believes the government’s lying to them. Everything is a conspiracy theory. ‘I shouldn’t trus anyone. I hate the other side.‘ That’s where all this is heading.“

01:11:04 Renée Diresta “What we’re seeing is a global assault on democracy. Most of the countries that are targeted are countries that run democratic elections.“

01:11:11 Tristan Harris “This is happening at scale. By state actors, by people with millions of dollars saying, ‘I wanna destablize Kenya. I wanna destabilize Cameroon. Oh, Angola? That only costs this much.‘“

01:11:33 Tristan Harris “We in the tech industry have created the tools to destabilize and erode the fabric of society in every country, all at once, everywhere.“

01:11:41 Joe Toscano “You have this in Germany, Spain, France, Brazil, Australia. Some of the most ‘developed nations‘ in the world are now imploding on each other, and what do they have in common?“

01:12:16 “The manipulation by third parties is not a hack. Right? The Russians didn’t hack Facebook. What they did was they used the tools that Facebook created for legitimate advertisers and legitimate users, and they applied it to a nefarious purpose.“

01:12:32 Tristan Harris “It’s like remote-control warfare. One country can manipulate another one without actually invading its physical borders.“

01:12:43 Tristan Harris “But it wasn’t about who you wanted to vote for. It was about sowing total chaos and division in society. It’s about making two sides who couldn’t hear each other anymore, who didn’t want to hear each other anymore, who didn’t trust each other anymore.“

01:13:56 Tristan Harris “Do we want this system for sale to the highest bidder? For democracy to be completely for sale, where you can reach any mind you want, target a lie to that specific population, and create culture wars? Do we want that?“

01:14:43 “If everyone’s entitled to their own facts, there’s really no need for compromise, no need for people to come together. In fact, there’s really no need for people to interact. We need to have some shared understanding of reality. Otherwise, we aren’t a country.“

01:15:11 Cathy O’Neil “We are allowing the technologists to frame this as a problem that they’re equipped to solve. That’s a lie. People talk about AI as if it will know truth. AI’s not gonna solve these problems. AI cannot solve the problem of fake news. Google doesn’t have the option of saying, ‘Oh, is this conspiracy? Is this truth?‘ Because they don’t know what truth is. They don’t have a proxy for truth that’s better than a click.“

01:15:42 Tristan Harris “If we don’t agree on what is true or that there is such a thing as truth, we’re toast. This is the problem beneath other problems because if we can’t agree on what’s true, then we can’t navigate out of any of our problems.“

01:16:41 Jaron Lanier “A lot of people in Silicon Valley subscribe to some kind of theory that we’re building some global super brain, and all of our users are just interchangeable little neurons, no one of which is important. And is subjugates people into this weird role where you’re just, like, this little comuting element that we’re programming through our behavior manipulation for the service of this giant brain, and you don’t matter. You’re not gonna get paid. You’re not gonna get acknowledged. You don’t have self-determination. We’re sneakily just manipulate you because you’re a computing node, so we need to program you ‘cause that’s what you do with computing nodes.“

01:17:25 Tristan Harris “When you think about technology and it being an existential threat, you know, that’s a big claim, and it’s easy to then, in your mind, think, ‘Okay, so, there I am with the phone, scrolling, clicking, using it. Like, where’s the existential threat? Okay, there’s the supercomputer. The other side of the screen, pointed at my brain, got me to watch one more video. Where’s the existentieal threat?‘ It’s not about the technology being the existential threat. It’s the technology’s ability to bring out the worst in society, and the worst is society being the existential threat. If technology creates mass chaos, outrage, incivility, lack of trust in each other, loneliness, alienation, more polarization, more election hacking, more populism, more distraction and inability to focus on the real issues… that’s just society. And now society is incapable of healing itself and just devolving into a kind of chaos.“

01:18:53 Tristan Harris (to the Congress) “This affects everyone, even if you don’t use these products. These things have become digital Frankensteins that are terraforming the world in their image, whether it’s the mental health of children or out politics and our political discourse, without taking responsibility for taking over the public square. So, again, it comes back to– (Mr. Sullivan, Congressman: And who do you think’s responsible?) […] I think we have to have the platforms be responsible for when they take over election advertising, they’re responsible for protecting elections. When they take over mental health of kids or Saturday morning, they’re responsible for protecting Saturday morning.“

01:19:24 Tristan Harris “The race to keep people’s attention isn’t going away. Our technology’s gonna become more integrated into our lives, not less. the AIs are gonna get better at predicting what keeps us on the screen, not worse at predicting what keeps us on the screen.“

01:20:10 “[interviewer: What are you most worried about?] I think in the shortest time horizon, civil war.“

01:20:28 Jaron Lanier “If we go down the current status quo for, let’s say, another 20 years… we probably dstroy our civilization through willful ignorance. We probably fail to meet the challenge of climate change. We probably degrade the world’s democracies so that they fall into some sort of bizarre autocratic dysfunction. We probably ruin the global economy. Uh, we probably, um, don’t survive. You know, I really do view it as existential.“

01:21:05 Tristan Harris “Is this the last generation of people that are gonna know what it was like before this illusion took place? Like how do you wake up from the matrix when you don’t know you’re in the matrix?“

01:21:21 “Whether it is to be utopia or oblivion will be a touch-and-go relay race right up to the final moment…“ – Buckminster Fuller

01:21:30 Tristan Harris “A lot of what we’re saying sounds like it’s just this one-sided doom and gloom. Like ‘Oh, my God, technology’s just ruining the world and it’s ruining kids,‘ and it’s like… ‘No.‘ It’s confusing because it’s simultaneous utopia… and dystopia. Like, I could hit a button on my phone, and a car shows up in 30 seconds, and I can go exactly where I need to go. That is magic. That’s amazing.“

01:21:57 “When we were making the like button, our entire motivation was, ‘Can we spread positivity and love in the world?‘ The idea that, fast-forward to today, and teens would be getting depressed when they don’t have enough likes, or it could be leading to political polarization was nowhere on our radar.“

01:22:10 “I don’t thing these guys set out to be evil. It’s just the business model that as a problem.“

01:22:17 Alex Roetter “You could shut down the service and destroy whatever it is–$20 billion of shareholder value– and get sued and… But you can’t, in practice, put the genie back in the bottle. You can make some tweaks, but at the end of the day, we’ve gotta grow revenue and usage, quarter of quarter. The bigger it gets, the harder it is for anyone to change.“

01:22:38 Tristan Harris “What is see is a bunch of people who are trapped by a business model, an economic incentive and shareholder pressure that makes it almost impossible to do something else.“

01:22:49 “I think we need to accept that it’s okay for companies to be focused on making money. What’s not okay is when there’s no regulation, no rules, and no competition, and the companies are acting as sort of de facto governments. And then they’re saying, ‘Well, we can regulate ourselves.‘ I mean, that’s just a lie. That’s just ridiculous.“

01:23:08 Jaron Lanier “Financial incentives kind of run the world, so any solution to this problem has to realign the financial incentives.“

01:23:16 “There’s no fiscal reason for these companies to change. And that is why I thing we need regulation.“

01:23:21 “The phone company has tons of sensitive data about you, and we have a lot of laws that make sure they don’t do the wrong things. We have almost no laws around digital privacy, for example.“

01:23:32 “We could tax data collection and processing the same way that you, for example, pay your water bill by monitoring the amount of water that you use. You tax these companies on the data assets that they have. It gives them a fiscal reason to not acquire every piece of data on the planet.“

01:23:49 “The law runs way behind on these things, but what I know is the current situation exists not for the protection of users, but for the protection of the rights and privileges of these gigantic, incredibly wealthy companies. Are we always gonna defer to the richest, most powerful people? Or are we ever gonna say, ‘You know, there are times when there is a national interest. There are times when the interests of people, of users, is actually more important than the profits of somebody who’s already a billionaire‘?“

01:24:23 “These markets undermine democracy, and they undermine freedom, and they should be outlawed. This is not a radical proposal. There are other markets that we outlaw. We outlaw markets in human organs. We outlaw markets in human slaves. Because they have inevitable destructive consequences.“

01:24:49 “We live in a world in which a tree is worth more, financially, dead than alive, in a world in which a whale is worth more dead than alive. For so long as our economy works in that way and corporations go unregulated, they’re going to continue to destroy trees, to kill whales, to mine the earth, and to continue to pull oil out of the ground, even though we know it is destroying the planet and we know that it’s going to leave a worse world for future generations. This is short-term thinking based on this religion of profit at all costs, as if somehow, magically, each corporation acting in its selfish interest is going to produce the best result. This has been affecting the environment for a long time. What’s frightening, and what hopefully is the last straw that will make us wake up as a civilization to how flawed this theory has been in the first place is to see that now we’re the tree, we’re the whale. Our attention can be mined. We are more profitable to a corporation if we’re spending time staring at a screen, staring at an ad, than if we’re spending that time living our life in a rich way. And so we’re seeing the results of that. We’re seeing corporations using powerful artificial intelligence to outsmart us and figure out how to pull our attention toward the things they want us to look at, rather than the things that are most consistent with our goals and our values and our lives.“

01:26:18 “The idea of human technology, that’s where Silicon Vally got its start. And we’ve lost sight of it because it became the cool thing to do, as opposed to the right thing to do.“

01:26:28 Bailey Richardson “The Internet was, like, a weird, wacky place. It was experimental. Creative things happened on the Internet, and certainly, they do still, but, like, it just feels like this, like, giant mall. You know, it’s just like, ‘God, there’s gotta be… there’s gotta be more to it than that.‘ I guess I’m just an optimist. ‘Cause I think we can change what social media looks like and means.“

01:26:56 Justin Rosenstein “The way the technology works is not a law of physics. It is not set in stone. These are choices that human beings like myself have been making. And human beings can change those technologies.“

01:27:07 Tristan Harris “And the question now is wheter or not we’re willing to admit that those bad outcomes are coming directly as a product of our work. It’s that we built these things, and we have a responsibility to change it.“

01:27:37 “The attention extraction model is not how we want to treat human beings. The fabric of a healthy society depends on us getting off this corrosive business model. We can demand that these products be designed humanely. We can demand to not be treated as an extractable resource. The intention could be: ‘How do we make the world better?‘“

01:28:21 Jaron Lanier “Throughout history, every single time something’s gotten better, it’s because somebody has come along to say, ‘This is stupid. We can do better.‘ Like, it’s the critics that drive improvement. It’s the critics who are the true optimists.“

01:28:50 Tristan Harris “It’s like the fundamental way that this stuff is designed isn’t going in a good direction. Like, the entire thing. So, it sounds crazy to say we need to change all that, but that’s what we need to do. [interviewer: Think we’re gonna get there?] We have to.“

‘Advice‘ shared during the credits

01:29:32 “I feel like we’re headed toward dystopia. I feel like we’re on the fast track to dystopia, and it’s gonna take a miracle to get us out of it. And that miracle is, of course, collective will.“

01:29:44 “I am optimistic that we’re going to figure it out, but I think it’s gonna take a long time. because not everybody recognizes that this is a problem.“

01:29:52 “I think one of the big failures in technology today is a real failure of leadership, of, like, people coming out and having these open conversations about things that… not just what went well, but what isn’t perfect so that someone can come in and build something new.“

01:30:08 Tristan Harris “At the end of the day, you know, this machine isn’t gonna turn around until there’s massive public pressure.“

01:30:16 “By having these conversations and voicing your opinion, in some cases through these very technologies, we can start to change the tide. We can start to change the conversation.“

01:30:24 Jaron Lanier “It might sound strange but it’s my world. it’s my community. I don’t hate them. I don’t wanna do any harm to Google or Facebook. I just want to reform them so they don’t destroy the world. You know?“

01:30:34 “I’ve uninstalled a ton of apps from my phone that I felt were just wasting my time. All the social media apps, all the news apps, and I’ve turned off notifications on anything that was vibrating my leg with information that wasn’t timely and important to me right now. It’s for the same reason I don’t keep cookies in my pocket.“

01:30:52 “Reduce the number of notifications you’re getting.“

01:31:02 “Never accept a video recommended to you on YouTube. Always choose. That’s another way to fight.“

01:31:19 “Before you share, fact-check, consider the source, do that extra Google. If it seems like it’s something designed to really push your emotional buttons, like, it probably is.“

01:31:27 “Essentially, you wote with your clicks. If you click on clickbait, you’re creating a financial incentive that perpetuates this existing system.“

01:31:34 Make sure that you get lots of different kinds of information in your own life. I follow people on Twitter that I disagree with because I want to be exposed to different points of view.“

01:31:45 “Notice that many people in the tech industry don’t give these devices to their own children.“

01:32:08 “I’ve worked out what I think are three simple rules, um, that make life a lot easier for families and that are justified by the research. So, the first rule is all devices out of the bedroom at a fixed time every night. Whatever the time is, half an hour before bedtime, all devices out. The second rule is no social media until hight school. Personally, I think the age should be 16. Middle school’s hard enough. Keep it out until high school. And the third rule is work out a time budget with your kid. And if you talk with them and say, ‘Well, how many hours a day do you wanna spend on your device? What do you think is a good amount?‘ they’ll often say something pretty reasonable.“

01:32:43 Jaron Lanier “Well, look, I know perfectly well that I’m not gonna get everybody to delete their social media accounts, but I think I can get a few. And just getting a few people to delete their accounts matters a lot, and the reason why is that that creates the space for a conversation because I want there to be enough people out in the society who are free of the manipulation engines to have a societal conversation that isn’t bounded by the manipulation engines. So, do it! Get out of the system. Yeah, delete. Get off the stupid stuff. The world’s beautiful. Look. Look, it’s great out there.“