What happens when humanity creates something potentially more intelligent than itself? Robert Wright, author of The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning, joins Henrik and Jeremy to explore whether AI could become something like a god, why managing its risks might be as much a human problem as a technological one, and what a future with less work might mean for how we live.
Robert Wright believes that whether you're excited about AI or terrified of it, you should be feeling some awe. He sees AI as a turning point in the history of life on Earth and explores the possibility of a “global brain” that could become far more intelligent than us. But if we build something that powerful, how do we make sure it works in humanity's favor?
Robert explains why managing AI may require international cooperation, and why our own tribal instincts could get in the way. Henrik and Jeremy also explore what happens if things go right, from breakthroughs in science to a world where people might no longer need to work. But what would life actually look like in that future?
The conversation turns to the AI companions we choose, and why having one that tells us the truth might matter more than having one that agrees with us. Afterward, Henrik and Jeremy wonder whether AI could help us reconnect with the physical world, and why having more free time doesn't necessarily mean we'll use it well.
Key Takeaways:
The God Test: Book Website
Robert Wright X: X/RobertWright
Nonzero: Nonzero.org
00:00 Intro: Could AI Become a God?
00:40 Meet Robert Wright
01:16 Why AI Deserves Our Awe
03:02 Are We Underestimating AI?
05:44 The Everyday Miracles of AI
07:07 How AI Learns Without Understanding the Brain
09:20 AI and the History of Life on Earth
13:13 Are Humans Really Less Flawed Than AI?
16:41 What Is the God Test?
18:12 Why AI Needs Global Cooperation
21:36 What Would AI Heaven Look Like?
25:12 Would Life Without Work Be Fulfilling?
28:55 Why the Future Is Hard to Imagine
32:48 AI Alignment vs. What the Market Wants
35:07 Choosing an AI That Tells You the Truth
37:52 Why You Should Actually Try AI
41:47 The Debrief
📜 Read the transcript for this episode: Here!
[00:00:00] Robert Wright: There are people who say we're building a superintelligence, possibly, quite possibly, and I think not implausibly, in the form of a kind of global brain, a superintelligence that could qualify for practical purposes as a god, a number of people say. I don't think that's crazy. I mean, in other words, it could become something much more powerful than us and something even that we grant ultimate governing authority to.
I'm, I'm not sure we'll get there, but it's not crazy. And if that happens, the question of what kind of God we build is a very important test. Do we manage to build one that is compatible with our continued welfare and flourishing?
I'm Robert Wright, and I'm the author of such books as The Moral Animal, Nonzero, The Evolution of God, Why Buddhism Is True, and most recently, The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning.
It's about AI, and I'm excited to talk about that and talk about the very promising parts of AI and the parts I'm liking already, and the very worrisome parts of AI that we can see signs of already, and how to arrange things so that the future is one in which the good outweighs the bad by a long shot.
[00:01:16] Jeremy Utley: I wanted to start with this phrase that I've seen you use, which I absolutely love. You, you say someone belongs on the awe spectrum, and I'd love for you to first define the awe spectrum and then talk about why do you insist someone belongs on it in regards to AI?
[00:01:32] Robert Wright: Yeah, at the very beginning of my book on AI, The God Test, I say that, you know, one thing that the doomers and the accelerationists, the people who are optimistic about AI and say full speed ahead, uh, have in common is awe.
They look at the technology with awe, but in different senses of the word. If you look at the history of the word awe, it originally had more kind of negative vibes. It was about fear in the presence of something vast and terrifying. And because of its religious usage, I mean, that was a religious usage, you know, kind of fear in the presence of a god that was to be feared, um, it, it came to refer, uh, also to awe in the presence of tremendous goodness, greatness, majesty.
And, you know, the, the doomers feel awe in the old sense of the word. The accelerationists feel awe in a more contemporary sense of the word. And I'm just saying, if you're not somewhere on that spectrum, either at one end or the other or kind of in the middle, going, "God, this is a, this is gonna be a very powerful technology.
I'm not sure if it's gonna be good or bad. Maybe depends on how we handle it, but this is huge," I'm saying if you don't sense the presence of a huge force gathering momentum, I would argue you're not getting the picture, because that, that's my take on AI. And why do you
[00:03:03] Henrik Werdelin: think a lot of people does not seem to be on that spectrum?
[00:03:09] Robert Wright: Well, first of all, I think fewer and fewer aren't, right? I mean, I think awareness is growing of the magnitude of AI, partly because of, uh, recent famous accomplishments, like in the realm of math. Partly because of recent famous mishaps, the so-called rogue AI episode, uh- Hugging Face, etc. ... and OpenAI, yeah, where they invaded the Hugging Face
[00:03:33] Henrik Werdelin: computers after breaking out of their sandbox.
[00:03:35] Robert Wright: Um, so I, I do think, uh, that, that there's a, a growing sense that something big is happening. You're hearing more politicians say we should do something dramatic. You're hearing more people in the field say, something I think a lot of them may have believed for a while, but it wasn't cool to say for a while, that, uh, we need some sort of international coordination to maybe slow the technology down and govern it collectively.
But if you ask why initially did a lot of people n- downplay the significance of this or just not think about it much, and why, you know, there's still a fair number of people in those categories, I, I think there are different reasons. I think on the left, there was a tendency to be skeptical of anything that big Silicon Valley companies say, and they were saying this was gonna be big.
[00:04:28] Jeremy Utley: Sure.
[00:04:28] Robert Wright: Uh, investors were saying it was gonna be big. And, you know, there, big claims have been made by big companies that turned out to be b- you know, PR hype. So skepticism was not a crazy thing. Um, and also the technology wasn't as good. I mean, when ChatGPT first hit the radar, it was pretty amazing compared to what there had been before, but it, it, it couldn't do nearly the stuff it can do now.
It, it wasn't agentic at all. You couldn't get it to actually do stuff Um, so, you know, the objective facts have changed. Awareness, uh, of them has grown. And so I think, you know, the, the problem of people not appreciating the magnitude of this is gonna be a shrinking problem.
[00:05:16] Jeremy Utley: So say someone's not yet on what I would call the awe frontier.
Regardless of where they are- Mm-hmm ... on the spectrum, there's kind of a frontier of awe, so to speak. Mm-hmm. If someone's not there or someone says, "I don't, I don't feel awe is warranted," do you have a short pitch for awe, so to speak? Meaning what would you recommend or what would you suggest to someone who's not convinced they really should stand in awe in the either conventional or, or classic sense?
What would you... What's the argument you make for you need to get on the awe frontier?
[00:05:45] Robert Wright: Well, first of all, there's just different uses I could point to. Again, like math accomplishments or the other day I'm, like, out looking at some kind of bush and I just started wondering about it in my front yard. So I pulled out my smartphone and, and said to Google Gemini, you know, turned on the camera and said, like, "What kind of bush is this?"
You know, and it tells me. And I say, "If I wanted to plant it in the backyard, are these... Like, do I just take these seeds?" And, and it says, "No, those aren't seeds. Those are next year's flowers. What you need to do is cut off the branches, go buy some rooting hormone." This is like a real-time conversation, and I can keep showing it stuff and asking it more questions.
And, uh, you know, it's easy to move the goalposts and forget how many of the things we can do now would've seemed completely miraculous four years ago, but they would have. And I could point to any number of things. I could talk about tasks that I can now do with AI that I used to pay people to do in various realms, including my kind of line of work where I have a very, very small nonprofit that puts out a newsletter and podcast and so on.
Um, so there's... I don't know. You can point to a lot of things. But people do naturally start taking for granted fairly recent things. Now, the, the... There's another angle I could take, uh, to convince people, try to convince people of this, and I take it kind of early in the book, which is- To explain the power behind the methodology of creating AI, okay?
If you look at the original kind of document, almost a founding document of artificial intelligence, the first place we, we I think we know of that the, the term was written, uh, was in the late '50s, a proposal for a famous Dartmouth conference featuring some luminaries i, in, in the field. And it said the, the premise of this conference is that any, uh, thing the human mind does, I'm paraphrasing, uh, anything the human brain or mind does can be so precisely described that a machine can be made to do it.
And the surprise has been that actually you don't have to understand how the human mind works to create a machine that works kind of like it, okay? You just, you know, in the case of a large language model, you start by doing this training that a lot of people are familiar with, where you get it better and better at predicting the next word.
You're showing it these gibberish symbols that don't mean anything to it. Uh, and yet by the end of the process, in a very meaningful sense, it has cons- you know, kind of reverse engineered, I would say, some of the cognitive functionality of the human brain. It's the same in all these realms. It's like we don't know what goes on in the brain when you drive.
Elon Musk doesn't have to know that. He just needs to get a ton of data, incoming video data, and then the outgoing human-generated data of the human responses. Like, oh, this is the incoming data, the steering wheel turns to the right. That's all you need, and you wind up kind of, the machine kind of reverse engineers, I would say, some of the basic functionality that natural selection took millions of years to build into the human brain.
I'm not saying it's exactly like that equipment structurally, but it does the jobs that the equipment
[00:09:19] Jeremy Utley: does. Okay. Okay, so, so pause here for a second because you're, you're making this case. I've gotta ask you, zooming out a level, where do you place AI on the continuum of, call it, uh, revolution, innovation, right?
Some people compare it to the word processor. Some people compare it to the internet. Some people compare it to fire, which, you know, is pretty amazing. Some people compare it to language itself. Where are you... And I think Kevin Kelly, you know, founder of Wired Magazine, at, at, when we spoke, I think he compared it either to language or fire.
I mean, which is kind of a... It, it's hard to overstate the significance. Bob, where, where would you place AI on the kind of continuum of innovation, just to help people kinda calibrate how to think about it?
[00:10:03] Robert Wright: Um, you know, I, I try to do that in the book. I mean, I, I, I put it in, in the I say it deserves to be viewed against the backdrop of the entire history of life on this planet.
I think it's a... Even if you zoom out that far, it's a distinct threshold. You know, it's hard. Technologies are different. Fire was important, uh, in a very different way. The wheel was important. But I just think it's fundamentally kind of unprecedented. And I, and I would say if you look at what's taking shape in the most cosmic sense right now, it really, it, it does justify what I just said about deserving a 3.54 billion year backdrop.
But what I mean is you start billions of years ago, you have self-replicating strands of information. You get... Eventually you get cells, then you get complex cells, then you get multi-celled life. So, you know, it jumps to a higher level of organization. Then you get societies of multicellular life, another higher level of organization.
Then you get human societies, and we're smart enough to launch a second kind of evolution, cultural evolution anthropologists call it, which includes technological evolution. And because of that, our social structure gets more and more complex and bigger. I tr- I, I kind of chronicled this in my book, Nonzero, and eventually it approaches the global level, and you get something that increasingly looks like a global brain, which is especially clear now that we have the internet.
Although the term noosphere for global mind was, uh, was coined a little over 100 years ago by a very prescient, uh, Jesuit paleontologist and theologian named Pierre Teilhard de Chardin, whom I talk about a little in the book. So you really, you are seeing the emergence of the global brain, and now we're realizing, like we had been thinking, Teilhard de Chardin had been thinking, the, the, uh, the guy who coined the term noosphere The neurons will be human brains.
But now it's looking like the question of who the you know, whether, whether neurons, whether the most consequential neurons in this brain will be silicon or organic is open, and what the relationship between the two will be. But I would argue that the planetary brain is taking shape. We are moving to a higher level of organization, and the only alternative to that is chaos, disorder, and suffering.
We have no choice but to move toward a, a globally cohesive, uh, community if we're gonna govern this technology. One way or another, it's gonna happen.
[00:12:39] Henrik Werdelin: That's a base question on that. Let me just go back to one of the previous things you said, which I thought was very fascinating, which is obviously, um, we have this piece of technology now that in many ways is the, you know, like the convention of, uh, what has been written on the internet.
And then it does these amazing things, and we don't really k- know why it does it, but it does it. And you made the point that we don't really need to know everything about driving in the world to create a car that basically can drive a bit, right? It just needs to know, you know, like when something happens, um, you should probably do this.
Do you think when we... So I, I'm sometimes kind of stuck between this is pretty base technology that just predicts the next token And then going, yeah, but so are humans, it is just an organic kind of way of doing the same thing. Where are you on the spectrum of, I guess like, either humans are pretty basic and so what the technology can do is the same because we're pretty basic, or the technology is quite advanced and so it's kind of a little bit almost on the same level as us.
Do you see what, do you see what I mean? Like, you can either dismiss- Yeah ... kind of like the human or kind of upgrade the technology or vice versa.
[00:13:54] Robert Wright: Yeah. I mean, um, at first I'd say a lot of the things people point to by way of minimizing it are things humans do, like the hallucination thing.
[00:14:05] Jeremy Utley: We call that
[00:14:06] Robert Wright: creativity with people.
You know? But we do it in a slightly different sense. Okay. But we all the time say unreliable things that we think are true.
[00:14:12] Jeremy Utley: Or, or, I mean, sometimes hallucination is a feature, Bob, right? Like- Yeah ... when I'm trying to create net new, I am, I am- Yeah ... imagining what could be, not, not stating what is, right?
Which is- Yeah, yeah, we are designed- ... human creativity is hallucination. Yeah.
[00:14:26] Robert Wright: Yeah. Natural selection designed our brains to warp our perception of reality. I mean, like, you know, men, when they... When heterosexual men see a woman in the distance, they overestimate her attractiveness relative to what their final judgment will be when they view her up close, okay?
When people, um, when people look at a... Well, they overestimate the speed of approaching o- objects, 'cause better safe than sorry, right? Better to get out of the way too soon than too late. Interesting. When they see, when they see an uphill when they're walking and they look and, and like, "Oh, the, it's sloping upward," uh, if they keep going, like they feel more tired.
They're not more tired, it's just nature's way of saying, "Why don't you think twice before you tackle this particular thing?" There, there are... And then there are m- m- more consequential things like our moral self-righteousness where we evaluate, and I talk about this in the book, 'cause I think we have to overcome this if we're gonna handle this technology well, 'cause I think we need to be a, become a more cohesive species.
But, you know, to judge the opposing tribe as morally wrong and our tribe as right. There's all kinds of biases built in. And in a way, the, uh, the famous sycophancy of AI is something people do all the time, right? Right. I mean-
[00:15:43] Jeremy Utley: When's the last time a human being gave you truly honest feedback, right? Most of the time- Right
people are like, "Great job, buddy," which is like-
[00:15:49] Robert Wright: Right ...
[00:15:50] Jeremy Utley: it's, the, the AI's- Even your friends ... doing what people do. Yeah.
[00:15:52] Robert Wright: You want your friends to be selectively honest in the way that somebody didn't know you would be, but not, not too honest. Yeah. Their job is to buck you up when you're feeling down and so on.
So yeah. Now, the dynamics of hallucination are a little different, but it is true, I, I, I, I will get into... I, I think they're less different between AI and humans than is commonly thought. So anyway, to get, uh, back to the, the overall question, I think- You know, you can look at it either way. Humans are, are a very impressive form of intelligence relative to all other life, a very imperfect form relative to what you can imagine.
Where is AI on that spectrum? It will ultimately exceed us very clearly, I believe, and now already is in some respects.
[00:16:42] Jeremy Utley: Okay, so this all feels like a wonderful precursor to defining the God test. You, you've titled your book The God Test. What is, in, in its simplest form, what is the God test?
[00:16:56] Robert Wright: The title has more than one meaning.
I mean, one is, you know, there are people who say we're building a superintelligence, possibly, quite possibly, and I think not implausibly, in the form of a kind of global brain. A superintelligence that could qualify for practical purposes as a god, a number of people say. I don't think that's crazy. I mean, in other words, something, it could be- become something much more powerful than us and something even that we grant ultimate governing authority to.
I don't know what our relationship will be. But in comparisons with, with a, you know, calling it a god-like being, I don't think is totally, uh, off the mark. I'm, I'm not sure we'll get there, but it's not crazy. And, and if that happens, the question of what kind of god we build is a very important test. Do we manage to build one that is compatible with our continued welfare and flourishing, right?
And there are a lot of questions about this. And, and that's a big test. There's a more kind of, uh, allegorical, metaphorical meaning of, of the title The God Test, which is, you know, as I said, I think to manage this well, we're gonna have to become a true global community. That's largely for just practical questions of governance.
It, it's just that-
[00:18:12] Henrik Werdelin: Can you just for a second, you mentioned that a few times. Can you just, am I, explaining what that
[00:18:15] Robert Wright: means? Well, let me, let me start at the other end of it, which is that this is a classic technology where if you're gonna govern it effectively, you have to govern it at the international level.
It, it's, uh, you know, we all know that like, uh, pandemics are like that. If you're going to keep people from genetically engineering a massively lethal virus, it's not enough to keep them from doing it in your country because the viruses don't respect borders. If y- if that's gonna be effective, it has to be an international project.
Well, AI, first of all, can help somebody build a bioweapon, so it really in that case is that very problem, but also it has a lot of other effects that are kind of metaphorically like a pandemic, a bioweapon or something, in that the effects cross borders. The cyber hacking stuff, there's a lot of international infrastructure that, uh, an, a single AI could take out.
There's just a lot of senses in which if you are worried about the effects certain kinds of AIs could have on your national security, you're gonna have to get more transparency into the rest of the world, and the other countries are in the same situation So you're gonna have to coordinate on this.
There's just a lot of international governance questions, and right now we are so far from being a global community, I don't know how many wars are going on, definitely more than zero, that it's, it's really hard to imagine, uh, uh, the degree of international governance that I think we need. And set aside the wars, we're just on pretty bad terms with the one country, China, that we have to start dealing with on this first, 'cause we are the two AI
[00:19:56] Jeremy Utley: superpowers.
Well, even, even, even within, I mean, even within our country, it's not like there's, there's a- Yeah ... you know, agreement. So then, I mean, if you take as a premise cooperation's or, you know, the, some kind of alignment is necessary, alignment among people, and if you take as a premise that might be im- nearly impossible.
I don't know. Are we cooked? Or do, are we just toast? Or what, what's your-
[00:20:17] Robert Wright: Well, I think
[00:20:18] Jeremy Utley: if we don't- ... analysis of the situation? ... make
[00:20:18] Robert Wright: that adaptation that I'm describing, which will involve becoming more aware of the psychology of tribalism and the cognitive biases that undergird it, and getting better at overcoming them, which I talk about in this book and in my last book, um, yeah, we're, we're, yeah, cooked is maybe not too strong a term.
I, I just think the chances of it working out badly go way, way up if we don't become, if we don't finally cross the threshold that we've tried to cross a couple times. Remember, in the wake of the two big catastrophes of the 20th century, World War I, World War II, you know, people said, "Look, let's do this, okay?
Let's try this League of Nations thing. Let's really try to, to do this and put an end to war." Didn't work. After World War II, "Let's try the United Nations, okay?" Didn't work. But they recognized that there's a certain level of catastrophe that is international in nature, uh, that it's best to avoid, and the only way you're going to, uh, you know, really lower significantly chances of avoiding it forever is, is to coordinate.
And so we're gonna, yeah, I think we're gonna have to try again. If we fail, I think we're... I just think the chances of a very bad outcome go way, way up. You know, it's not impossible that things will naturally work out great. I can sketch scenarios, but even there I think we'll suffer more in the, in the meanwhile if we
[00:21:39] Henrik Werdelin: don't coordinate.
Let's get some of those, 'cause I do think, you know, speaking about bias, obviously we also tend to have a bias to seeing things that can go wrong rather than things that go right. Question, I guess,
[00:21:48] Robert Wright: uh- Actually depends on the situation. Those men who are overestimating the attractiveness of a w- I mean, m- men over- That's true
also overestimate in many situations how attractive the woman is finding them.
[00:22:00] Henrik Werdelin: But what do you think is the attractive, what is that version for AI? 'Cause we do- Hmm ... talk a lot about all of the things that can go very wrong. We seldom talk about the things, or at least create a pretty, um, concrete example of what does it actually mean when this goes right outside, like, UBI, which in my point of view is just taking
[00:22:18] Jeremy Utley: agency around, uh, away from people, which is not necessarily a good thing either.
[00:22:22] Henrik Werdelin: Do you have, like, a, a version where you go, it's like, "Okay, this actually played out. This was the bright spot
[00:22:27] Robert Wright: I have a way to, to make it as good as it can be. In other words, like I outline, like, yeah, we need to start, you know, coordinating internationally and doing these things and looking at these questions.
I don't think we should put all our faith in alignment, although, and so on. So I have a, I have a path I sketch. In terms of what it looks like, I mean, I have a chapter on AI hell and AI heaven, and I look at Dario Amodei, the CEO of Anthropic's vision, uh, of what kind of AI heaven would look like, and I note, you know, some complications there.
But, I mean, look, some, uh, there, there are some obvious... They're just good things you can point to now. Like, that's great that I can go out in my front yard and find out about this plant, and from my point of view, do something that otherwise I guess I'd have to pay some expert to come tell me or do a lot more research.
Um, the diseases, I think, will get cured via AI, can get cured. There will be huge scientific progress that can be put to the good. It is already easier for people who, in the world, who previously had just no access to good education to get, in effect, you know, personal tutors at a pretty low cost. So there's, uh, all kinds of good things happening now.
If you're, if you're asking me, "Well, what about in 30 years? What will that look like?" I mean, I don't know. I suspect that people will probably have the opt- If it works out well, people will probably have the option of not working. I mean, it's honestly... I, I, I would challenge anyone who doesn't think there are gonna be big job impacts to tell me what economically valuable work do humans do that you think AI will never be able to do.
Okay? Is there a desk j- Is there an office job that fits that description? I don't think so. If you think the robotics thing is gonna be big, there will be a diminishing number of physical jobs, so, so anyway, I, I'm, I, you know, I think people have the option of doing that and of not working, uh, perhaps the necessity of not, not working in many cases, and the option, if things work out well, of doing a lot of fulfilling things.
Now, will some people misuse that? I, I mean, some people now misuse recreational opportunities involving drugs and things, yes. But, uh, you know, Dario Amodei in that piece of his that I look at talks about people having new, newly deep spiritual experiences thanks to drugs that are invented and stuff. All of this is possible, um, and you can dream up all kinds of other things that are, that are possible.
It can be great if it goes as well as possible.
[00:25:12] Jeremy Utley: Bob, one, one question that I have had on my mind, a, a book that comes to my mind often is Vonnegut's Player Piano. I was actually just talking with a friend about it the other day, and if you both don't know the premise, the short of it is Vonnegut imagines a future where engineers are a small percentage of the population, and they're designing all sorts of gadgets that eliminate the need for human work, w- hence the name Player Piano, right?
The, the machine can now play the piano, and so even pianists- Mm-hmm ... don't have a job, right?
[00:25:39] Robert Wright: Mm-hmm.
[00:25:39] Jeremy Utley: And in that kind of envisioned world, 90% of the population lives at a higher standard of living than they've ever had before, uh, they don't have to work, and they're miserable. And 10% of the population has to work, but they're always, you know, self-conscious of the fact that they're gonna work themselves out of a job, basically.
And I, the reason that story comes to my mind often is because I think one of the things that Vonnegut is illustrating is the meaning that we derive from work and the dignity of work and things like that, and I wonder, Bob, if as you've studied the kind of UBI, you know, or, or otherwise utopian scenarios, are there good examples of people being satisfied in not working?
I, I, I have a lot of bad examples, you know, but I don't call to mind a lot of great examples. Even, um, uh, even if you look at someone like Elon, right? As he's, he, he's, he has decided, despite the fact that he's the richest person in the world, he doesn't need to work, he throws himself into his work more than ever before.
And I don't see a lot of stories, I don't know if it's pensioners, uh, from, you know, the, the military or, uh, public services or things like that. But are there examples of folks who don't have to work and who are fulfilled in not working?
[00:26:58] Robert Wright: Well, I'm sure you can point to successful retirement stories, you know, but it, it, it's very challenging.
So I mean, if you just look at humans who retire That's a relevant subset. I don't think it's exactly analogous. Some people do it successfully. I mean, I think one thing that will ha- that may happen is, you know, if more and more of this stuff is turned over to machines, and our relationship with the machines is great, and they want us to flourish or whatever, or we've just kind of succeeded in keeping, uh, humans in the loop sufficiently so that they have to want...
They have to let us flourish. You know, we're still in charge, whatever. I think either way, there will be almost a, a more self-conscious cultivation of the human, if that makes sense. Like, people maybe more than ever- Meaning right, right
[00:27:47] Jeremy Utley: now it's ad hoc maybe? Or it's, it's not a
[00:27:50] Robert Wright: deliberate- Well, I just mean for most of history we haven't gone around saying, "Hey, this thing we're doing, it's the thing that our species likes," right?
You don't think that when you're at work. I mean, for example, that, you know, people long gone to listen to live music. I, I just mean that, uh, live music could conceivably grow in value because people say, "Yeah, this is something our species does, damn it." You can imagine a little more kind of species-centric pride developing, you know, by virtue of our awareness of this other form of intelligence.
I guess it's almost like a, a, a self-esteem salvaging project in a way. It's like, "Okay, these machines can do this, and this, and this, but we invented dancing, and we're gonna keep doing it." But I, I do think... I mean, there are a lot of things that give us joy that are healthy. They're like win-win, you know?
It's like you're doing them with friends, and they don't involve, like, in killing the other tribe of, of soccer fans. It doesn't have to get that bad to have a tribal allegiance of your own.
[00:28:55] Henrik Werdelin: Can I throw another, uh, thesis on you? Um, if we think that innovation is exponential and we think that AI's gonna accelerate a lot of these things- We talk about a lot of these even near futures as kinda like a very linear progression of where we are today.
[00:29:12] Robert Wright: Mm-hmm.
[00:29:12] Henrik Werdelin: And if you go back, let's say 200 years and f- 500 years, and you look at some of the innovations that came around that really transformed stuff, like, and there's of course fire and electricity and stuff like that, but there's also just realizing that the Earth is round and that we're not the center of the universe and all those different things, right?
Now, I heard this story the other day, I'm not sure it's completely true, so, you know, whatever, but that basically one of the reasons why we're getting closer to fusion energy is that now we can keep, um, the particles that are colliding more stable because the force field is using AI to become more stable, right?
And so, like, this kind of the, the secondary value of AI is not necessarily the AI itself, but it's, like, all these other things can happen. And then you- if you really start to kind of then imaginate what are some of the crazy stuff that could happen at the magnitude of that, then the world that we could live in could look just very different.
There's all these new perception. I.e., you've written a lot about meditation, for example. When I grew up, that was, like, a woo woo thing, and then suddenly mindfulness became a thing that was kind of very science-based. Do you know what I mean? So I wonder if a lot of this narrative we have is just very linear thinking, where really we ought to try to think what's gonna happen when all these foundational beliefs that we have might completely change.
[00:30:31] Robert Wright: Yeah. Um, first, before I answer that, I wanna, y- your question led me to think something I meant to articulate, which is that one thing, one problem with comparing different technologies is the extent to which sometimes they kinda fuse and it's hard to draw the border between one and the internet is, and AI is a good example.
The internet was kind of a developing global brain, and AI is almost a crystallization of that in a certain sense, and a harnessing of that.
[00:30:59] Jeremy Utley: It was i- it was impossible without the internet. Yeah. Yeah, I get
[00:31:02] Robert Wright: you. And moreover, the internet is now, uh, you know, it's landscape. I mean, these agents get out, and the internet is there, and, and it's one thing that creates the potential for various scenarios and for a global brain and so on.
Anyway, tha- that aside, I agree with you that... I mean, sometimes I almost, when I think about how different the future could be, and somebody brings up some other field that itself could be advanced by AI and would be hugely momentous, like quantum computing, I just go, "Oh, look, I just can't. I can't think about this."
It's just getting, you know, it's just overwhelming. I mean, AI is enough. It's like when you... I mean, I am gonna actually have a- an expert, uh, Scott Aaronson, on my podcast to talk about quantum computing and AI. But, um, so I'll know more then, but I agree with the point you're making. And look, if you look at these people, y- one weird thing about the field of AI is that it is populated in no small part, I'm talking about the researchers, by people who had been thinking in these sci-fi terms- Right
before LLMs existed, and they, they already have these... So, so they'll, like, point to the novel that they think represents the future, and a number of them imagine, I think, what is basically, uh, one scenario is this, like, okay, global brain, it's fundamentally silicon, it's expanding out into the universe, and we wish it well, and, and the only question is, is there a place for us in this scenario where we're still happy?
But I think all bets are kind of off if you look very far into the future. Things could conceivably get very weird. They could get very weird in a bad way. They could get very,
[00:32:48] Jeremy Utley: well, weird in a good way One thing that I saw, Bob, I heard, I don't know if it was Nick Thompson or, um, uh, Cal Newport. Someone was talking about how...
I think it was Cal Newport actually on his podcast recently. He was kind of discussing the hugging face, uh, incident, and one thing he said is, "It's not inevitable that AI agents behave this a certain way," like tr- trying to escape the sandbox, for example, or reward hacking. And he said that studies have demonstrated that if you remove kind of sci-fi from the training data, it changes how AI agents behave, that in a way, AI agent behavior is actually a self-fulfilling prophecy of sci-fi.
And I, I haven't dug deep into it, but I, I just think it's somewhat interesting to say. I mean, because part of the kind of the, the joke almost of the hugging face incident is, you know, I don't know if you've seen the meme, but it's like, uh, you know, there's a... It's like a three illustration cartoon and, and illustration number one is the human going, "Break out of this sandbox," and number two is it breaks out, and then number three is, "Oh my goodness, it broke out."
It was like, uh, yeah, it did what you told it to do, right? And so anyway, I, that, that's just kind of interesting to me.
[00:33:58] Robert Wright: Yeah. The, the... I mean, he's pointing to broadly speaking alignment. Can you, by fiddling with the training text or the reinforcement learning or various other things, make it less likely to break out and wreak havoc?
In principle, I mean, I'm not... Again, I wouldn't put all my stock in alignment for various reasons. But I also, I think I listened to the same podcast you did. To some extent, I, I think he was underestimating the extent to which these problematic agentic properties just flow from the demands of the market.
[00:34:32] Henrik Werdelin: Mm-hmm.
[00:34:32] Robert Wright: You know, he- Mm-hmm ... you know, what he said was, "Look, well, the, the problem is these harnesses that have this," uh, what was the sequence? It was, uh, ask something, act, reply, or whatever He was just describing the dynamics of agentic action. Agentic action is what the market wants. It- companies want AIs that do things.
Great. They want AIs that, that are autonomous, that pursue tasks that may take a long time, overcome obstacles, and so
[00:35:00] Jeremy Utley: on. They... And companies actually want them to, to conceive of novel ways around obstacles- Right ... that they wouldn't have anticipated. That's right. Yeah. That's right. Okay, so this is, this has all been very theoretical.
I'm just thinking as we, as we come to a close, if you wanna get hyper-practical, because we're kind of talking into the not too distant future in terms of awe, in terms of singularity, in terms of, uh, you know, global brain, et cetera, et cetera. If you had to advise someone today, what are two or three very practical things someone could do today?
If they accept your premise, if they accept this line of thinking, what, therefore, how now shall we live, so to speak?
[00:35:39] Robert Wright: I would say start giving thought to what kind of AI companion you want to whatever extent you want one. I think more and more people are gonna have an AI companion of some kind, and more and more people will be getting consequential, personally consequential feedback from it in one sense or another.
In the, y- you know, in the, in the most clear-cut cases, they'll be asking it about personal issues. "I'm having this problem with my spouse. I'm whatever." Um, I think people should ask, like, what would be a healthy kind of companion to have? And it is not, I would submit, the kind of companion that the market most naturally provides.
Which is to say An AI companion that tells you what you wanna hear. I mean, this is the s- the source of the famous sycophancy problem is just that companies naturally wanna optimize for engagement. All companies. Right. Right. If you make candy bars, you wanna maximize the time people spend eating your candy bars.
Every company pretty much wants to maximize engagement, so it, it shouldn't come as a shock that AIs will build... AI companies will build AIs that do that. I think it is, it is up to us, I mean, there are places for regulation, I think, but aside from that, it's up to us to, just the way you might say, "You know, I'd like to have a personal trainer," like if you could afford one.
I don't have a personal trainer, but it would be great if I did. Uh, you know, because you recognize, like, I'm not always gonna love this trainer showing up at my house at, like, 7:00 AM as I've arranged. It's gonna be unpleasant for me, but I recognize that it's good for me in the longer. And it's the same with an AI that maybe tells you a little more of the truth about this conflict you're having with a person or a group than you really, than is comfortable hearing, that hey, if you look at things from their point of view, you're actually the aggressor- Hmm
and it's not a crazy way to look at it, so maybe you should just
[00:37:38] Jeremy Utley: calm down- You almost need a- ... and make amends ... it's like, it's like a red teamer. Yeah. So- Yeah, yeah So then, so one is- A little bit of a red teamer ... one is think about the kind of companion you want. Is there a do- Mm-hmm ... or an act or, you know, something that some- you would recommend?
Take this action
[00:37:52] Robert Wright: I think it's in people's interest to get a sense for what the technology can do if they don't already have that
[00:37:59] Jeremy Utley: Hmm. Mm-hmm. So familiarize yourself with the tools. Yeah.
[00:38:03] Robert Wright: Yeah. I mean, there are whole block- there are whole ideological blocks that- Ooh ... like on the left in particular, I think, that, uh, are kind of avoiding- Right
acknowledging the usefulness of it.
[00:38:15] Jeremy Utley: I actually spoke w- I, I did a day in Central Park in Washington Square with a hand-drawn sign that said, "Why do you hate AI?" And I just talked to strangers all day. This is a true story.
[00:38:24] Henrik Werdelin: Mm-hmm, mm-hmm.
[00:38:24] Jeremy Utley: I talked to multiple people who take pride in the fact that they've never opened an AI.
Mm-hmm. Multiple people. And to your point, it's actually... I, I, I wouldn't have thought about it that way, but it's to be completely oblivious is not e- it's not a position from which you can derive strength, is, is kind of- Right ... what I'm hearing you say. You can't have a strong opinion of without a lack of experience of what the models are capable of.
Is that what you're saying?
[00:38:51] Robert Wright: Well, yeah. And just, just at a tactical level, I mean, ideological combat is, is not nothing, right? It's like the, the... there are tools you use if you believe in your ideology to try to advance the ideas and get- Yeah ... candidates elected and everything else, and if you're gonna, if you're gonna try to do this without AI, in 10 years, uh, people are gonna have to remind us what your ideology was, okay?
You're gonna get steamrolled. A- and I regret that. I regret that. But- Yeah ... and I welcome, I want to emphasize, I, I want an internationally coordinated slowdown of the technology, okay? I'm not I'm not an accelerations by any means. But so long as your ideological opponents are using the technology, you better use it if y- Yeah.
[00:39:33] Jeremy Utley: It's like- ... if you don't want to lose ... it's like bringing, it's like bringing a, you know, a paper airplane to a, you know, a, to a machine gun fight. To a knife fight or something. Yeah, exactly.
[00:39:39] Robert Wright: Yeah.
[00:39:40] Jeremy Utley: Now, the one thing I will say, and maybe this is a rejoinder actually to the people that you just spoke to, 'cause I feel like the do you said there is especially for folks on the left, engage.
Don't be oblivious, but try the technology. The one caveat or warning I would give such people is AI has this annoying tendency to perform to one's expectations Which is because, uh, AI output is a function of your input. So if you expect it to be sycophantic, for example, if you expect it to not really deliver great output, it's gonna not do very well.
Not because it can't do well, but because you don't expect it to do well. And so I wonder, I think for the person that you're advising to engage in some way, I think you have to almost... You have to force yourself to imagine the AI can do whatever you're about to ask. Imagine that it could be exceptional at it, and force yourself as a human to engage, to, to demand exceptional- Mm-hmm
and then to see, uh, uh, and to use that as the basis that formulates your understanding of what the capabilities are. Because if you naively prompt and naively engage, expecting it to be mid, as the kids say-
[00:40:46] Robert Wright: Yeah ...
[00:40:46] Jeremy Utley: then it's gonna be mid. Yeah. And you walk into the arena thinking, "Oh, it's actually not that interesting," or, "It's not that capable," which just compounds- Yeah
your own ignorance, I think.
[00:40:56] Robert Wright: Yeah. Don't be the person who says, "Look, the hands have six fingers. How can this ever..." You know, whereas you know that's not, that's not always gonna be the case, right? Yeah. And look at the video it can do today. It's, it's mind-blowing. That was completely predictable. So to kind of dwell on the fact that it wasn't yet a mature technology in that regard, I agree.
You know, it's, it's fine to look for flaws and be aware of flaws. I mean, I can tell you stories about how I, in an actual medical crisis, I got three different answers from three models. So yeah, be a wary consumer, but don't, don't, you know, try to, uh,
[00:41:33] Jeremy Utley: you know- Don't try to prove your assumption that it isn't any good.
[00:41:36] Robert Wright: Yeah. Don't do that We close on a note of agreement I
[00:41:40] Jeremy Utley: think actually, uh, that your, that like denouement, yeah, don't do that, was actually the perfect place to end. Henrik Werdelin- Jeremy ... what'd you, what'd you take away from that conversation with Pulitzer Prize finalist Robert Wright?
[00:41:54] Henrik Werdelin: I mean, like he is clearly fascinating in that I think there's a lot of questions around, you know, the, the treating AI as God and what that would mean, that as we were kind of just coming to the end of the conversation was coming to my mind.
And so, uh, I think it is interesting to think about what is this thing that could be larger than us as humans that could govern us if it actually could govern us. You know, our God of course is from this more abstract thing where, that we kind of try to interpret it.
[00:42:31] Jeremy Utley: Mm-hmm.
[00:42:31] Henrik Werdelin: And we say, "Well, this is probably what God would want us to do."
You could have something here that could be very concrete. It's like, "No, this is actually exactly what I want you to do."
[00:42:39] Jeremy Utley: Mm.
[00:42:40] Henrik Werdelin: And then- Mm ... what would that actually mean? Uh. And that I, I thought was a pretty spiritual and, and interesting kind of thing.
[00:42:47] Jeremy Utley: Yeah. I... To me, what, you know, interacting, having, you know, conducted research ahead of time and things like that, I have taken his God test to be far less religious or spiritual and more kind of a metaphor, and I was...
So I was a little bit surprised by how, actually, how l- kind of literal at times we took it. I mean, my, you know, not-so-hot take is there's no way man can create God, so, you know? It was like I personally, just to, you know, state clearly, I don't think anything we make will ever be God any more than like a wooden idol, you know, 5,000 years ago in the desert was God, you know?
[00:43:27] Henrik Werdelin: Yeah.
[00:43:27] Jeremy Utley: Um, and so that to me, it is interesting to think about the metaphor of knowledge that surpasses our own, and insofar as God is kind of, is a metaphor or is a, is a word that connotes- And not
[00:43:40] Henrik Werdelin: just knowledge, but also the idea of outsourcing for many people- Yes ... to see religion help them-
[00:43:47] Jeremy Utley: Yes ...
[00:43:47] Henrik Werdelin: regulate their
[00:43:48] Jeremy Utley: life.
Guidance, guidance, yes, input, et cetera. Yeah.
[00:43:52] Henrik Werdelin: And I guess in many ways for many people now, AI has that role, right? You know, for people of faith, they pose a question to God and then they kinda like, uh, feel that they get an answer through different- Yeah ... means.
[00:44:04] Jeremy Utley: Yeah.
[00:44:04] Henrik Werdelin: I think for people in other religions who use AI a lot, they go, you know, "Claude, should I be worried about this?"
Right? You know, like and then- Right. Right ... they get an answer there, and it, it does have similarities.
[00:44:14] Jeremy Utley: Mm-hmm. Mm-hmm. I enjoy, I feel he is one of the more, call it, I don't wanna say doomer necessarily, but I feel a lot of the time we're talking to people who are all in and who are primarily optimistic, and he strikes me As someone, he's not naive, um, but he is more cautious probably than I would say our average guest, which I think is just kind of a cool thing to engage
[00:44:42] Henrik Werdelin: You say that, and meanwhile, I'm still stuck with how few people I hear ever tell something where you go like, "You know, when that happens, that's gonna be really cool."
It's very, like, solving disease, being able to make you not have to do work you don't want, sure. But, like, this very compelling vision of a world-
[00:45:07] Robert Wright: Mm-hmm ...
[00:45:07] Henrik Werdelin: that has AI kind of like completely embedded into it that make me go, "You know what? This is how..." I'll, I'll give you one thing. I talked to, uh, I'm not sure we recorded this before, but I talked to Andrew Sarkulman the other day, who's, who's a creative and an amazing thinker, and he made the point that we have spent the last 100 years becoming very good at becoming cognitive as a human race.
We basically came from back where we were hunters and gatherers and stuff like that, and we were very physical.
[00:45:36] Jeremy Utley: Uh-huh.
[00:45:37] Henrik Werdelin: And then we've been going through this industrialization where basically we think, we think, we think. Right. And for many of us now, we think that's the only thing we really do, right?
Like-
[00:45:45] Jeremy Utley: Right. Most of us, most of our jobs could be if we could just connect our brain, then we could do the job. Exactly. Yeah. Yeah, yeah. No, no body required is your point, right?
[00:45:55] Henrik Werdelin: Yeah, and he was really getting onto this thought that maybe one of the revolutions of this technology is that it can take a lot of that stuff away, so we can go back to using our body, and not as in like, "And then I'll go fishing, and that'll be great," but like-
The physical world and to each other and to understand smell again, seeing probably, and stuff like that. I think it's something that probably a lot of us had lost a bit and, and he was articulating that this might be like a very enjoyable journey where you suddenly do suddenly realize what flowers smells again.
Ugh. And you do suddenly get an enjoyment of not being on your phone, but going on that walks in the woods and seeing how amazing this creation is that is just kind of like nature. And, and I know it sounds a little bit woo-woo, but like-
[00:46:47] Jeremy Utley: No, no, not at all ...
[00:46:48] Henrik Werdelin: I, when he said that, I was like, "Wow , you know, I, I would like that."
You know, like that, that sounds like something would
[00:46:54] Jeremy Utley: be cool. I, to me, I, I love that and I, uh, I resonate and I think about our conversation with Alex, which by the time this airs, we'll probably, we'll have aired the conversation with Alex. I keep thinking about Alex's comment when we were talking about the AI vampire and doing more work, and the reality is he used the metaphor of like dealing with your email's a lot easier than dealing with a six-year-old.
But I think generally speaking, there's a sense in which being in our heads is easier than being in our bodies. And while being in our bodies is amazing, you know, s- sight, smell, et cetera, is amazing, the truth is people doom scroll, you know, rather than hike. And so all that to say, I think it's a, it's a very idealistic way a- and maybe aspirational way to think about returning us to our bodies.
But I mean, I, I will tell you... I will make a confession here, you know, #confessiontime. Yesterday I got home, I, I was in Texas recording a thing, and I came home and I, I ended up getting home at like 4:00 and I kinda, I kinda done all the work I needed to do really for the day on the plane and things like that.
So I got home and I'm like, "Hey, I'm just gonna hang with the girls and play. We're gonna play basketball, we're gonna jump on the trampoline." And I had multiple times, "I should probably go check my email," you know? And I had Alex's kind of statement in the back of my mind that... I can't remember what he said.
It was something like, uh, work is a coping mechanism. Something like that. Yeah. I think he was
[00:48:19] Henrik Werdelin: talking about the numbing work or something like
[00:48:21] Jeremy Utley: that. Yes. And I just realized Yeah, I have to fight. I felt, I felt it quite pointedly yesterday. I have to fight the tendency to want to be productive. And by, by coding it a numbing mechanism, by coding it a coping mechanism, it actually helped me be more present.
And so, which is to say perhaps there are tools to help us return to our bodies, but I feel the, you know, I don't know Zuckerman, but like that, that idealistic hope or aspiration of returning us to our bodies, I think it is wonderful, and it's somewhat naive to the fact that many of us don't actually want to be in our bodies.
If given the chance, we don't. We retreat into device, we retreat into headspace. And so to me, it's, it's not inevitable. You could say what is inevitable is what has been . You know, I'm more persu- I'd be fa- far more persuaded if you said, "Hey, something is inevitable. What's inevitable?" Sa- it's tragic, but doom scrolling, obsession with work, you know, disconnection and estrangement from children.
I think those things are more inevitable than flourishing, deep and meaningful relationships, appropriate boundaries with technology. I just don't think those things are inevitable. And so if we take as a premise, we have to intervene in order to kind of, you know, shift the trajectory from the inevitable kind of thrust, then, then I'm, I'm excited by the Zuckerman vision, and I recognize it takes work.
And it's not policy work, but personal work. Like, I found at least five times yesterday in a span of 90 minutes, I had to do the personal work of deciding this is where I want to be.
[00:50:08] Henrik Werdelin: Yeah. I mean, this is what you say, it res- that resonates a lot. I mean, I'm the, I'm the same way. I've... After this con- uh, conversation with Alex Gilman that, you know, I think will be out by now, is, uh, yeah, definitely been thinking a lot about like, hey, this new agent that I'm spinning up to do this whatever task that seem very important, is that just kind of me not wanting to do this thing that I'm not as experienced in, which is, you know- Yeah
whatever, playing with my kids or- Yeah ... just go for a walk with the dogs
[00:50:36] Jeremy Utley: It's a challenging and humbling, uh, question to ask. Am I avoiding my unique responsibility- Exactly ... in the name of, in the name of professionalism, productivity, et cetera? We could keep going. Perhaps we sh- we're right at the hour.
We're right at the hour mark. Okay. Maybe we stop here.
[00:50:56] Henrik Werdelin: We're at the hour because you can hear... I'm not sure you can hear, but I moved next to, uh- Church bells ... a church, and so the church bells, uh, reminds me to, uh, to go back to reality.
[00:51:04] Jeremy Utley: It's gonna be hard for you to erase all digital trace, you know, when, when, when people start looking for you, 'cause all they have to do is find those church bells, and you're toast.
[00:51:11] Henrik Werdelin: Awesome, man. Thank you so much. Uh- Everybody who has, uh, listened all the way through, #confession will probably- Hashtag ... send us an email or, or a, or a note, and we'll, uh, we'll figure, figure some way to, uh, to make it worth your while.
[00:51:24] Jeremy Utley: To, to reward your patience, yes. Until next time. And
[00:51:27] Henrik Werdelin: with that, bye-bye.
[00:51:30] Jeremy Utley: Bye-bye.
Displaying A - Beyond the Prompt - Robert Wright.txt.