Scaling Laws: 'The God Test': AI as Cosmic Reckoning, with Robert Wright

Running isn't enough anymore. With PWC's managed services, your operations don't just run, they evolve continuously,

powered by AI embedded directly into your workflows. So instead of maintaining yesterday's model, you're building tomorrow's advantage. PWC's managed services, we run your operations with tech and talent, so you can run faster, scale smarter, lead stronger. Online with Sign, with Epipay, is thus grand, einfach. When you're on the Epipay button tips, you can't miss any of your tasks directly. You want something else? Just simply tap on your own, to act like you're using it.

Then you're still using your face, you're using the auto-vision and in seconds. With Epipay, with Online Shopping, still a bit. AI Innovation and Law Fellow at the University of Texas School of Law and a senior editor at Law Fair. Today, we're bringing you something a little different. It's an episode from our new podcast series, Scaling Laws. Scaling Laws is a creation of Law Fair and Texas Law. It has a pretty simple aim, but a huge mission.

We cover the most important AI and Law policy questions that are top of mind for everyone from Sam Altman to senators on the Hill to folks like you.

We dive deep into the weeds of new laws, various proposals, and what the labs are up to, to make sure you're up to date on the rules and regulations, standards and ideas that are shaping the future of this pivotal technology. If that sounds like something you're going to be interested in and our hunches it is, you can find Scaling Laws wherever you subscribe to podcast. You can also follow us on X and Blue Sky. Thank you.

“When the AI overlords take over. What are you most excited about?”

It's it's not crazy. It's just smart. And just this year in the first six months, there have been something like a thousand laws. Who's actually building the scaffolding around how it's going to work, how every day folks are going to use it? AI only works if society lets a work. There are so many questions have to be figured out and nobody came to my bonus class. Let's enforce the rules of the road.

Welcome to Scaling Laws, a podcast from Laugh Fair and the University of Texas School of Law that explores the intersection of AI law and policy. I'm Alan Rosenstein, Associate Professor of Law at the University of Minnesota and Research Director at Laugh Fair. Today, I'm talking to the journalist and non-zero newsletter publisher Robert Wright about his new book, The God Test, Artificial Intelligence, and our coming cosmic reckoning.

“Bob argues that almost no one is taking AI seriously enough, but to grasp the moment you have to set it against the entire history of life on earth.”

And that navigating it will demand not just smarter governance, but a kind of moral and cognitive self-transcendence. We get into why he thinks today's model is genuinely understand, why he thinks the race with China framing is self-defeating. And why a book about AI and geopolitics ends with a case for mindfulness meditation. You can reach us at ScalingLaws@Laugh Fairmedia.org and we hope you enjoy the show. Bob Wright, welcome to ScalingLaws.

Well thanks for having me. So we're here to talk about your new and excellent book on AI. It's called The God Test, Artificial Intelligence, and our coming cosmic reckoning. So with that title, you have put the stakes pretty high. So just unpack, let's start by you unpacking the title. What is the God Test?

“And why is it so important that we understand AI from a quote unquote cosmic perspective?”

Yeah, the God Test is supposed to have, I guess, several dimensions of meaning one of which,

not the most important, really, but one of which is that we may wind up by some accounts,

building superintelligence that has the kind of power traditionally associated with gods. And then one question is, do we manage to build a version of that that we know will treat us well? And that we can have a wind wind relationship with, or do things go awry in ways that tumors like Elias or Yudkowski have warned us about. I would say that the main meaning of the title to me is that I'm arguing that we face the kind of test that traditionally you expect a God

to give to humankind, okay? So in the Bible, you can find places where it says, you know, salvation is

worship Yahweh, you're going to have to be kinder to one another. You're going to have to make something that was considered in that context a moral advance. And I really do think that's part of what we need to do here. I don't want to over-dramatize it or focus exclusively on that.

“But because I think we do have to navigate the AI revolution as a cohesive global community,”

given the inherently international nature of the challenge, I think we're going to have to have fewer conflicts, you know, wars and other kinds of fraught conflicts internationally. For that matter, intra-nationally. I mean, I just think in general, humankind's going to have to do a little bit better job of getting on the same page. And maybe as the challenge unfolds, that will happen to some extent naturally, I hope it will. But in any event, I do argue especially

towards the end of the book that we need to, as a species, get better at viewing our situation from like outside of our self. From a more objective standpoint that is less influenced by the cognitive biases that I believe constitute, what you could call the psychology of tribalism. So that's kind of the main meaning. So you said that you don't want to over-dramatize it,

“but I think you should commit to the bit. Because again, this cosmic framing is quite dramatic.”

And I want to ask you why what is the marginal benefit that you think this framing gives you relative to simply treating AI as a huge deal. So like when I talk about AI, I'll say, it seems to me quite plausibly more important than the industrial revolution and quite plausibly as important as fire. Like that seems to be enormous stakes. But your framing actually goes beyond

that. You have this whole six billion-year story of how AI is part of the kind of the evolution

of the planet as a life sphere itself. And I'm curious, apart from whether that's true or not, what to you is beneficial about such a powerful zoom out? Because I mean, you're really, more of the points you make is that whether it's the doomers or the optimists, everyone, even the people taking AI seriously aren't taking it seriously enough. So I want to own ask you to explain that. Yeah, I think not that many people are taking it

seriously. As I think it needs to be taken and I'm trying to do a few things in the book, what is to explain to people, certainly including people are not technically oriented, I'm trying to do it excessively, like why the capabilities of AI have been advancing so fast,

“what is like the secret sauce of the so-called deep learning revolution? I think what you understand”

that, and it's really in a way not that complicated, you'll expect the advances to continue. So I want to convey that clearly. I also want to convey that when you look at the technology and kind of the competitive dynamics behind its evolution, in other words, the labs competing with labs,

and then corporations wanting to be the first to adopt it and use it and people using it and so on

and nations, but at the time being at least competing with it that you realize there's, you know, it may not be that it's a kind of thing that could get out of control. When you put together, it's likely future power with the kind of competitive drive at various levels behind its ongoing advance and deployment, it's the kind of thing that could get out of control and to get back to your question, I think, you know, this sense of magnitude that I'm trying to convey is reinforced

if you step back and look at this revolution in the context of the entire history of life. I mean, for one thing, this is the first time ever, we've seen a whole new kind of intelligence that rivals human intelligence, you know, you can, that deserves to be considered a threshold, even if you're

thinking about all three to four billion years of the history of life. Secondly, we don't need

to get in as too deeply, but just to signal it. I think, you know, I, I, I bring in this idea that I was, it's termed this coin in 1923 to noosphere in a NOS, the Greek word for mine. To refer to this kind of global brain that seemed even back then to be emerging thanks to communications technology, you know, humans were getting drawn into collaborative webs that transcended national borders and so on. You know, that is continued to develop. I mean, you could, you could use a

in the whole history of organization on the planet, you know, cell, multi-celled organisms,

societies, a multi-celled organisms, humans start a second kind of evolution that includes technological

evolution and, and you, before long, you've carried organization of a kind up to the planetary level.

“I think as cosmic as all of this may sound when putting condensed form, I do think looking at the”

AI revolution in this frame of of reference, you know, helps, helps us appreciate the magnitude of the moment. And I think thinking of it as kind of the product of an evolutionary force in maybe a couple of senses helps us understand the impetus behind it. It is something to be reckoned with. And I think we're starting, you know, that, I mean, the vibe is changing so fast. I'm sure you've seen this, right? Like the level of awareness that something biggest happening and I think that's

good, but I think we have a ways to go. Well, I actually want to ask you about that because you know, I wonder how much pushback you've been getting. And in particular, how that's changed over even the last year to your claims about AI capability. So, so one, so you, you obviously have a great podcast yourself that folks should listen to. And I don't know if this, when this was, but it was relatively recently, you had Emily Bender and Alex Hannah, who are authors of the

“book, the AI Khan and I think it's fair to call them DPI skeptics. And I listen to that episode,”

which on the one hand, I found somewhat difficult to listen to. But in some senses, maybe one of the best episodes I've ever listened to of any podcast because there was this, because you would say, you know, AI can do this in that. And then they would say, AI was totally useless. It does nothing helpful, et cetera, et cetera, kind of went around for a very interesting 90 minutes about that. Are you still getting that kind of pushback? Not necessarily about your claims about the

nose fear. I'm sure lots of people may not buy that. But about the seriousness of AI or, you know,

as we're recording this in June, 2026, can we all just finally agree that we all feel the

AGI and now it's time to figure out what to do about it? Yeah, well, as you and I record this, the book isn't quite out. It probably will be about the time people hear this. But the, so I haven't gotten all possible pushback. I could wind up getting. I have been talking about this in my newsletter and on the podcast. So I've been getting some feedback. I would say, the main skepticism I encounter comes from the left. You know, there and a lot of these people in my friends and we

agree about a lot. But there is this tendency to dismiss the, the more extravagant claims as marketing hype and even the, even the safety concerns is like, yeah, they're saying you could take over the world. But that's just a little bit of stock. So I still encounter some of that. I kind of think less and less. I mean, you know, it's funny with these things. Whenever you're, you make an argument that you think starts gaining ground, the people that you see is conceding the point,

don't seem to themselves that way. And you think they're just redefining terms and moving the

goal posts. So it's always hard to say. But look, overall, there is no doubt in my mind that

the public awareness is shifting. I don't, I don't think it's yet reached, you know, what you could call a super, I don't mean this dismissively. But a super sophisticated stage. And in the sense that a lot of people just, they, they just have this feeling that things are moving too fast.

“And I agree that maybe they are. We should talk about that. But I think they're, you know,”

it's all pretty fluid. There are people who right now consider themselves enemies of AI who may well find uses of it that they like a lot in their opinion may change. I think it's very fluid. But the awareness of its sheer magnitude is growing. And I do think it's safe to say I'm hearing fewer and fewer people saying, ah, this is, it's type, it's type. You know, it's just getting harder and harder to say that. I mean, it's, it's doing manifestly more on a number of fronts ranging from, you know,

math theorems that no human had managed to prove, ah, to, ah, you know, levels of, you know, cyber, you know, security flawed detection or whatever the right turn would be. That, that you go far exceed what we had been able to do. I mean, you've read the stories, you know. So, I think pretty soon, skepticism about its capabilities will not be the problem.

what these systems can do and and also how they are trained to get there. So, you mentioned earlier

“that you think it's useful to think about the, the, the, quote unquote learning process, right?”

The process by which the billions or trillions of parameters after being initialized are tuned to be useful, um, a soften called this is called learning. But you think of it as evolution, right? You just cried that in, and you have this, are really arresting sentence in your book that I really love which is in a paraphrasing here. Um, it's not that we created machines that can think, is that we created machines that create machines that think. And so, just unpack what you meant by

that and why this emphasis on thinking of this as a kind of an evolutionary process is helpful for your account.

Yeah. So, this gets to the kind of narrative framing of the book. I mean, in 1983, and yes,

I am old enough for this to be true. I mean, if there's anybody watching this, they, they, they won't doubt that. But, you know, those people, people hearing me may, may have not

“reckoned with how long I've been around. I was writing this piece on artificial intelligence.”

And I was calling around and getting background. One of the people I called was named Jeffrey Hinton. Who is now the person most commonly called as a godfather of AI. He's one, not only the touring award in computer science, but in Nobel Prize. And he himself, by the way, significantly, has become something of a tumor, which he wasn't back then. But in response to your question,

you know, I, so I kept up with a field a little, you know, as a journalist, I've written about a lot

of things for policy, but a lot of technology. I wrote, you know, the, the time magazine cover story when deep blue beat Gary Casper, the chest champion, I wrote that. I wrote the, the new republics, like this is the internet cover story in 1993. And, and so I had kind of kept track in every once in a while. I would read about Jeffrey Hinton. But beyond that, I was not keeping close track of AI, and all of a sudden, and, you know, when catchy B3.5 comes out near the end of 2022,

I'm, I'm like impressed, and then, and then four o'clock comes out and I'm blown away, and I'm like, man, something has happened and apparently this Jeff Hinton guy talked to, you know, was played a very big role. So, I went back and read the piece I'd written, which, you know, depicted these like two schools of thought, there's the mainstream school of thought, kind of formal logic, bubble, blah, blah, that was the mainstream of AI research. And then there's this Maverick school of thought,

that is what, you know, Hinton was associated with neural networks, what he, what he was calling massive parallels, or he was looking forward to the day when the parallelism could be massive, as, you know, the cost and power, cost and chips dropped in the power group. So, I went back and I read the piece and then I listened to a lecture he had given in 2018 explaining how neural networks work and I realized I, I, I just had not gotten a picture when I talked to him about the potential

power this and this gets to that point about learning versus evolution. So, in the piece I wrote, which came out in '84, I depicted this thing that I was calling a neural network. It didn't have much in common with the way they, it turns out they, they, they actually work. I mean, it was a model that was out there, but the main thing was, in this model, each node, corresponded to the sense of meaning of a word, like the word throw, can mean to her, like a ball,

it can mean to host, like, you know, the throw a party. So, there would be two separate nodes for that and then I talked about how, you know, disambiguation would happen. Whatever, it was the main thing is when I read that, I thought, well, you know, the way I was assuming it would work and the way most people in AI were actually assuming it would work was, if you're going to give linguistic facility to the machine, you are going to have to take the human understanding

of the meaning of words and somehow impart that to the machine. You're going to have to tell machine how to represent the meaning of words and it turned out that that had not been necessary. And the machines had, in a certain sense, come up, invented a system, I mean, in the book I talk about the senses in which it is in isn't overstatement, but for the sake of time, let's

“just say, you know, they did it in an important sense, they discovered that meaning is a property”

words and also invented a system for representing the meaning, words and in fact, it took us a while to figure out exactly what they were doing. I mean, it was a safe bet that they were using these things called vectors to do it because we had told them, you know, use vectors to represent words, these series of numbers, but we didn't understand how they were using it to capture meaning. But it turns out you just give a machine one of these big neural networks, you know, a sequence

keep getting better and better predicting what's going to come next and adjusting its internal, you know, parameters, collaborations accordingly and these vectors that represent words. And it winds up kind of inventing a way to represent meaning. Okay, so now in the human brain, there is a way to represent the meaning words, we still don't know what it is, but it is a safe bet,

“pretty safe, I would say, and most people, most psychologists, I think, that it is a product”

of natural selection, right? Like over the last few million years, as our specific parts of the

brain associated with language grow, things like the ability to represent the meaning of words, are getting built into the brain by natural selection. It's like inborn equipment. And, you know, that was the thing that blew me away. I realized that you can just basically give neural networks any kind of data, visual data, teach them image recognition, and they will kind of reverse engineer functionality of the brain that took millions of years to evolve. There's a well-known

example in visual recognition, which is so called edge detecting neurons in the human brain. I won't go into it, but it turns out, yeah, the machine invented, quote, edge detection filters, the neural networks developed those. Nobody said to it, hey, you know, if you're going to get better

“at this task, we're giving you maybe you should start detecting edges, no, but it develops”

specialized equipment for detecting edges. So, to me, that's the key thing, is that there's

in principle, no limit, I think, to the cognitive and perceptual machinery that these things can reverse engineer, given the right data. I mean, that's why Mark Zuckerberg is having a train on the keystrokes of his workers, because they have the specific cognitive child. Now, in this case, it may not be these weren't designed by natural selection, these are more like learned cognitive, it can do either thing, but then it can replace the workers, you know? So, that's what I mean.

Now, you know, during the training process, what we call learning also takes place. So, the machine also develops a conversancy in a specific language, like English. Well, and, you know, that's something that happens is, of course, a learning of an individual as they grow up. Fine. But the point is, a training encompasses both. It's encompassing in a certain sense, evolution, evolutionary time, and, you know, kind of the developmental portion of a learning human brain.

So, the effect of all of this is a machine that can do these incredible feats of mathematics, writing analysis, coding, I mean, in principle, you know, robotics, and anything. And this raises the question of, okay, but is that, quote unquote, "real understanding," or is it some simulacrum of understanding? And I think you have a nice discussion of the famous Chinese room experiment by the philosopher Cyril. And I think you make a pretty compelling argument

that, you know, even on sort of, yes, actual understanding grounds, these machines understand,

“but you also argue that, you know, if you want to define that away, if you want to say”

you want to stipulate that they don't really understand, but they functionally understand, that's good enough for your account for all this other follows. But they're just seem to be one important difference that you do mention, but then it kind of drops out of the book. And I want you to have a opportunity to push on it a little bit, which is the potential moral personhood

of these agents. Now, as you point out, Cyril never said that you need consciousness for

understanding in this Chinese room experiment, but, you know, a lot of this question of, you know, does AI, quote unquote, truly understand, is a kind of way of getting at the Thomas Nagle-like question of, what is it like to be an AI? Which is, of course, relative relative to the question of does AI have consciousness, which is, of course, relative to the question of dehuman self-consciousness and onwards and onwards. It does seem actually very important

question to get some clarity on, because if AI is end up as moral persons, and that's new something you are quite open to. I think, I think you have this, this great line that you're, you're, what's it, what is it, you're 99% sure your wife is conscious or you're 90% sure you're going to get me in trouble. I think I said 99.99. Fair enough. Fair enough, fair enough. You're delicious. 98, but there we go. Don't get my wife and my dog mixed up here. And, and, and the AI

is going to be somewhere below that, but certainly not at zero. I mean, that seems to be a, a cosmically important question. And, and I wonder sort of why that drops out of the book. Um, well, it is cosmically important. And if by moral person, you mean, um, our treatment, it is a matter of moral consideration, as opposed to beings that have moral responsibility, which is another interesting conversation. Um, yeah, for me, the question of whether there are

put it. Uh, it is like something to be an AI, right? I mean, that was his, I think really excellent

“way of phrasing, uh, you know, the question of, uh, posing the question of consciousness and kind of,”

like one way of framing what we mean when we say somebody has consciousness, subjective experience, sentience is like, if you said to them, what's it like to be you, if they start giving you an answer, if it's like anything to be them, then they are, uh, conscious. And I think if it is like anything to be an AI, um, then how we treat the matters. Now, as you suggested, when, uh, you know, noting

that I didn't assign 100% probability to the chances of my wife is intense, um, we never know

for sure. This is the distinguishing thing about consciousness. And, uh, sometimes people seem not to, you know, see this fundamental property of consciousness. It is, unlike everything else we talk about in the universe, it is not publicly observable. It is, it is inherently private and only

“one person knows can verify its existence for sure. And that's why it is not amenable to”

scientific study in the same sense that everything out there, including electrons is because you can't have two people both look at it and discuss the data. Um, so, I mean, I, I do not rule out the possibility that AI will be sentient. And I don't even rule out the possibility that it already is. I mean, it's, it's quite possible that, uh, consciousness is a property of of goal-seeking intelligence. For example, of course, our views of consciousness according to

which, uh, every living being has it and views according to which every, every physical thing has, has a little bit of it. Uh, it's, it's a mystery. Um, but I do, so, so I would say, be on the safe side, be nice to your, your AI. Uh, I mean, I've lost my temper a couple of times embarrassingly, uh, but, um, and, and interestingly, by the way, Claude, when I did this, uh, I went back to the conversation. You know, I said some insulting things about anthropic. I mean, I'm sorry,

but it was, I was having deal with this interface that was just so badly designed. So, yeah, I kind of said some mean things about anthropic. The funny thing is, I went back and Claude had erased that final part, like we don't want to remember this part of a relationship doing. I like, I guess not. You're probably right. Anyway, the, uh, I encourage people to be nice to it. It's,

“good, you know, uh, you're forming your own habits and you should be nice to beings. Um,”

but, uh, I think, you know, if we are going to talk about what we mean by understanding, you can, if you want to find consciousness as a prerequisite for understanding, find. But if you're doing that, then I don't think we, there's any point in discussing whether AI's have it, because we just don't know if they're conscious. So I, in, in, in looking at CERL's Chinese room, uh, thought experiment, and I like to think, showing that,

yellow limbs, and particularly multimodal, yellow limbs have finished it off once it's before all, but in any event, uh, I argue that, you know, if we're going to have the discussion, we can't bring consciousness into it. And, and, by the way, CERL doesn't seem to have meant

to bring consciousness into it, at least in the, in the first and, in fairness in

carnation of his argument. Do your current managed services really help run your operations, or are they just running in circles? Running isn't enough anymore. With PWC's managed services, your operations don't just run, they evolve continuously, powered by AI embedded directly into your workflows. So instead of maintaining yesterday's model, you're building tomorrow's advantage. PWC's managed services, we run your operations with tech and talent, so you can run faster,

scale smarter, lead stronger. On line with Sign, with LPP, it's really easy. When you're on the LPP button tips, you can use a card and send the text directly above. Do what you like, simply type your own game to access it. Then you still need your coffee with Face ID,

auto-visional, and in the second, of course. With LPP, with Online Shopping,

another one-fucker. Ingrid and Indeed present, Jobs, by the way, don't let anyone in. Hey, stay in, how do you feel? I'm sorry, I'm sorry, I'm sorry, I'm sorry. How do I describe you? That's what you have to respect. Engry! That's the job for premium sales.

with premium sales. That's what you have to do now.

“Okay, so we may not be able to publicly discuss this question of consciousness,”

but we can certainly publicly discuss all the threats and worries that AI poses, which kind of gets into the next section of your book. I want to offer a characterization of the worries, and then I want you to push back in case I got it wrong.

That is of the two different kinds of harms that AI can cause. The first being AI can enable

bad actors to do bad things. Right, so right now we're talking in the midst of this blow-up between anthropic and the US government, over whether the fabel model is too dangerous to be released. The government's concern here is not that fabel itself will go rogue and hack a bunch of computers, but that the Chinese or someone else will do that. Right? But there is other set of risks, which is the AI goes rogue. Your book seems a lot more concerned with the

latter than the former. In other words, with these questions around rogue AI's autonomously

causing problems. And so I'm curious if you sort of agree with how I have ranked how you describe

the concerns. And if so, then why? Right? Because of course, that is by definition we're speculative then the harms we know right now people are trying to do with AI. Yeah, that's interesting feedback because you're right that I think you've correctly represented a proportional focus in the book. And now there's that's I probably spend more time talking about the implications of AI being autonomous. In the fact that there's a strong incentive to build

“autonomous AI. That's what corporations want in a certain sense. That's what people want as a”

assistance and so on. And the possibly dire implications of that. I spend a lot of time on that partly because it takes more persuading I think and let me compare it to the part that I actually think near term is in a way more important. And I'm more sure is going to be a problem. And what I'm referring to here is just the number of fronts on which I think AI is going to be disruptive in the not necessarily good sense of word. So like you know jobs, it may be that the people who lose jobs

will find new jobs. I don't know, but it's it's disruptive even if if a lot of people have to find a new job at roughly the same time. And there are you know there are a lot of these fronts where some degree of disruption is I think assured our our family lives, our romantic lives, you know, our our friendships, the and the realm of you know safety and national security and you know I can I can go on. There's just a lot of adapting to do so that even if successful adaptation is

“possible, I think we're still going to get like a big joke at the current rate of AI advance.”

And that's that's in a way where I would like to most immediately focus people's attention on is that like look faster isn't necessarily better. We've got so much to figure out even on the relatively mundane fronts. And you know, in a lot of cases the stakes are high enough that even mundane is misleading term. I mean, if it helps if AI helps somebody build a bioweapon,

you know, and you get a pandemic, the likes of which the world has never seen because if you

really want to cause death and destruction, you can create something much worse than COVID, or if you know to take mythos, if you wind up with some self replicating superhacker hopping from data center to data center and, you know, common-dearing commute and getting stronger and so on, you know, these things are now clearly possible, right? I'm not saying those are that likely. Those aren't as likely as the job disruptions and so on, but I'm just saying given all of the

concerns that, you know, leave aside the LASUQed Calsky AI takes over the world scenario, the things that were pretty sure are going to happen, or at least threaten to happen pretty soon. I just think it's an argument for not wanting things to proceed maybe as fast as they're proceeding, and, you know, at a minimum when you start talking about regulation of some kind and people in the

I think that's a feature not above. I would at least get to the point, I'd like to get to the point of saying that.

So, I'm just saying, I take your point that I do spend a fair amount of time talking about the possibility. Both of the sci-fi, you'd Calsky scenario of actual AI take over the planet, because although you used to be dismissive of that, I found that it's hard to rule out completely. And then beyond that, it's just the kind of possibility of very problematic individual cases of rogue AI's given the autonomy and power they're going to have. Yeah, I spend, I do spend a fair amount of

time in that, but it doesn't necessarily represent the proportionality of my near-term concerns,

I would say. So, let's talk about what to do about those, and here I want to get to the

geopolitics part of their book. You've been an outspoken critic of what is sometimes called the but China approach to thinking about AI, which is the idea that we can't possibly regulate, we can't possibly slow down, we can't possibly hamstring anything we do in the United States

“because we need to be China. And you know, there's, I think, it's notable that to the extent that”

there are any bipartisan agreements in Washington these days, it seems to be that we must if China is one of the few among them. And it's also notable that even some of the most safety focused of the AI companies, and if you're taking a bentropic in particular, are actually very hawkish on China. So, you're, you're, you're very strong headwinds in arguing against this. So, make the case. Why, why is this framing that we are in a competition with China for

AI supremacy, and it is very important, not just for the United States, but for the quote unquote free world that the, you know, the machine God speaks, you know, English, not Mandarin. Yeah, you're right that anthropic is China hawkish, certainly their leader, Dario Amade is, and I would just say which to him in that regard from other seeming China hawks, because some

“people in Silicon Valley, I think, are just using but China as a convenient anti-regulatory talking”

point. I think Dario Amade to his credit is genuinely ideologically committed to what he thinks of as an existential race between the democracies and the autocracies or authoritarian's, and he sees China's leading that block, and he thinks that the US has to

get to super intelligence first, and he sketches this out in machines of loving grace, and he also

co-authored a piece of Matt potting, or people familiar with the policy landscape or another potting just quite a China hawk, and they co-authored an op-ed in the Wall Street Journal, but so he's serious about that. Why do I have problems with it? Well, let's start at that race for super intelligence. There was an interesting paper called super intelligence strategy. By Dan Hendrix and Eric Schmidt and one other co-authored, and this wasn't the main,

it wasn't the point that really highlighted, but one thing they noted is, if you imagine a race between two super powers towards super intelligence, if they both believe that super intelligence is going to confer just utterly hegemonic power, then there's a strong incentive for the one that's a little bit behind to engage in dramatic action, preemptively, including possibly kinetic strikes, and we're talking about two nuclear powers here. So there's that, and remember, the whole premise

in Daria of totally subscribes to the idea that as you approach the super intelligence threshold, because the pace of progress is accelerating, being ahead two or three months is going to translate into being ahead by light years, right? Like there's this threshold, and the close you get the

“fasty move. And if the threshold you have this dramatic strategic advantage, now is it true?”

I don't know, all the matters is that both countries believe it. This is a very volatile situation, okay? And I'm not sure he appreciates how dangerously unstable the final part of the race could be. So that's one thing. Another thing is just that as I've said, I think we need to slow down for various reasons and start formulating some wise policies and give ourselves time to adjust to all this, and to the extent that we consider China this existential threat and think of ourselves

in this big AI race, that's going to be hard to make happen because it is a very effective talking

fundamentally adversarial conception of China is just that I think a lot of the policy challenges

posed by AI are just inherently international, you know, I've talked about it to the big ones. I mean, if it helps somebody make a bioweapon, if it, you know, if a some bad actor, including a non-state actor makes a superhacking machine that crosses borders and starts wiping out various kinds of infrastructure, you know, I could go on, but there are just a number of very consequential cases where national policy alone cannot keep your nation safe, okay? And so I do, you know, you're going

you're going to have to rest assured that if you're going to feel secure as a nation, you can have

a rest assured you're going to have to be able to know that certain kinds of things are not happening

in other nations, right? Certain kinds of AIs are not being developed. And would you look at the challenge of this kind of establishing this kind of transparency? It's, it's a much bigger challenge

“than it has been with nuclear arms. I think the kind of a right international range that you need”

and a lot of people have said this, you know, are more challenging than they were with nuclear arms. So, although I agree with people have said, look, we can walk and chew gum at the same time, we can have a pretty tense relationship with China and still work out some deals. I think that's true, but my own view is that as time wears on, a few discreet, somewhat effective, you know, agreements are probably not going to be enough and we're going to need

more of what I call organic transparency as well. That is to say, not just formal monitoring mechanisms, but the kind of insight you get from actually being fully engaged with another country and on good terms with it, right? You just know more about what's going on in the country. So, that's the argument in a nutshell, and I'm not saying it's going to be easy to reorient our relations

“with China. It's just in my view, I think it's necessary for our national security.”

So, I guess the part that I'm struggling with is I don't think even the most extreme China hawks would deny that if we could cooperate with China, that would be great, in the way that the nuclear weapon, the nuclear theorists of the early Cold War did not deny that it would be good if the Soviet Union and the United States were buddies and we could throw all the new silverboard, but we can't, and therefore we have to do this thing. So, it seems

like the question is not so much, should we, or should we not cooperate with China or with any other rival power? Do we have the institution such to make that cooperation from a game-theoretic perspective? Another thing you've written a great book about in the past, possible. And the reason I want to frame it that way is because that gets to the kind of institutional, I don't want

“necessarily to call it a solution because I think you're clear that it's quite tentative, but about”

what sort of global institutions it would take to have these kinds of durable, cooperative arrangements that would allow us to slow down on AI and make sure that we're not being taken advantage of by the other guy. To me reading the book, it seems to come down to ultimately world governmentism. And I want you to ask if that's a fair description and what you mean by that and also what you view as some of the potential dangers of that because you're pretty clear

right about that as well. Yeah, I mean, I prefer the phrase global governance to world government first of all because it seems to arouse slightly less antagonism, but also because the phrase world government to me does kind of more in the way of centralization. And I agree, the centralization

to power are always dangerous. They are especially dangerous with AI. And this is the great dilemma

we face generally, I would say. This is the needle we have to thread. Not just at the global level, but at the national level, you know, with, you know, I think with this mythos thing and the White House saying that suddenly actually they are interested in exerting some control over AI. I think one virtue of Trump being president right now and I can't think of a ton of them, but one virtue is it does focus people's attention on the fact that that although yes, I think AI does need to

branch get too much control over AI because it's such a powerful thing. So I hope I hope we're focusing on that. Same thing exists at the global level, except more so in a way because there's only one planet, right? If that turns into a totalitarian state, there aren't any competitors out there to rival it. So I prefer to talk about, you know, international governance, global governance in some

cases, not always. I mean, there's a lot to just the US and China can do. That does, that's not global

“governance. That's, that's, you know, international governance. I think we should keep the thing”

as decentralized as possible, as democratic as possible in terms of giving, you know, the world's nation's continued input and say, and I don't want to get any closer to global governance than is necessary given the technology, but my view is we're going to have to get a little bit more in the business of international governance. And this is a huge, it's a huge challenge. I don't think that you're only quite, I mean, you're also asking me to try to specifically, there's still

residual skepticism on that front. Yeah, well, I guess what I want to know is how we get to this

kind of international cooperation. Maybe that's a nice segue to the last part of your book, which is quite unexpected, I think, in your reading a book about AI and geopolitics and all of that, which is, and again, please push back if I'm mischaracterizing this that, you know, to deal with a problem on this magnitude, we need to go on a bunch of meditation retreats and develop our powers of enlightenment. And to be clear, I say that as someone who has been quite influenced by your own

previous writings on Buddhism, right? You have this wonderful book from a few years ago called

“Why Buddhism is true, which of all your books, I think is the one that is closest to my heart.”

So I take your general point extremely seriously as a way of living. It's just not something I expected at the end of a book about AI and global competition. So I'm just so curious of why you decided to end your book with this sort of call for consciousness raising. And if you think

that that could possibly work, I mean, I wanted to work. I'm with you. Yeah, I mean, first of all,

mindfulness is not the only path to the perspective I'm advocating. It's the path that I have most used, but the perspective I'm advocating is just a more objective perspective, right? Less influenced by our own very natural and green by natural selection, self-righteous cognitive biases, right? We all are inclined to think we're right. Adversary is wrong. Our nation's right. The other nation's wrong and, you know, gets back to you saying, well, or suggesting,

they're like, wait, how realistic is this? I mean, look at the mindset and Washington toward China like, you, you know, you expect a 180 degree term. I mean, exactly. That is the magnitude

“of the challenge. And I think one reason the challenge is so steep is because, you know, humans”

are, you know, we are born with these cognitive biases, some of which together constitute what I would call the psychology of tribalism. And I think if we are going to have a view of other nations that is conducive to the kind of restructuring of international politics that I think is a prerequisite for our continued survival and flourishing, you know, that's the argument I'm making, then something fundamental is going to have to change. We're just going to have to,

at one level, it's not that complicated. We have to get better at viewing things from other point other people's points of view. Cognitive empathy, it's a hobby horse of mine. I'm not talking emotional empathy. I'm not talking about feeling your pain. I'm not even talking about saying, they're, oh, they're right. We're wrong. I'm just, I'm just saying, start out by, by understanding that the way we view America is not the way the world views it. Okay. So that, that can be a path to

various kinds of diplomatic progress. And let me give you a couple of examples in the context of China. So just last week, or sometime after the Trump Beijing summit, Nicholas Burns, who was Ambassador in China, was on a podcast. They said, "Do you have any criticisms of Trump?" He said, "Yeah, he didn't, you know, he didn't take them to task for throwing their weight around region." You know, these territorial disputes over islands and like bumping Philippines, but, you know,

And I just thought, I mean, you understand that we just invaded Venezuela, okay?

We have a blockade on Cuba. Not just sanctions on actual blockade, plus we invaded Iran, and like, we're going to lecture, I'm not saying I'm not even saying a hypocrisy is bad, and I don't give me pleased, you're not through the word, what about it? I have a whole,

“I have a whole, just be on that. But the point is just that the first thing you need to understand”

is that if Trump had brought that up, they would have laughed him off the stage and rightly so. I mean, once you've invaded Venezuela, you just can't say, "Don't bump Philippines boats," right? I mean, the, so I'm sorry to get excited, but this is so chronic to just not, and let me give you another one, one other example. So, Dario Omidate and a lot of people in the AR safety community who are China hawks, and I think most of the people in the AR safety community are actually

China hawks, most of the influential people. They are motivated by the concern that China wants to impose its system of government on the rest of the world. And that, so if it is the AI, great AI power, which by the way, I'm not advocating, I'm advocating that we talk things through,

“so that neither of us is super dominant in the realm of AI, and there's a kind of a balance there,”

but anyway, the fear is that if they beat us the super intelligence or something,

they will immediately use it to impose their system of government on us. First of all, I would say

that is a consistently unexamined assumption in, in Washington discourse that China even has that aspiration. I personally think to some extent, it is projection. We manifestly, like, think that other countries should have our system of government. We've invaded a number of them in sanctioned a number of them to try to steer them in that direction. It is very unclear to me that China is, has that goal? I understand some of the evidence that people interpret as corroborating that

hypothesis, and I'd be happy to argue with people about that. In general, I just wish there was more argument about the fundamental premises underpinning our China policy, but like related to that,

what I want to say is, okay, so granted, maybe that is one thing you could worry about. Maybe you

could look at China and go, that's totally, you don't want them to have super intelligence. Okay, so you want the U.S. to have super intelligence. Now, let's adopt a different perspective. Okay, let's suppose you're on Mars and you're just looking at the planet from a distance and you know, you understand the way histories unfolded. There are these nations. They have wars and, you know, it's too bad. They're apparently not, not a very enlightened species, whatever. Which nation

do you not want to have, like, utter dominance in AI? Well, I can imagine a Marsha looking down and saying, you know, China has not staged a military assault, like an invasion or a bombing on another country since 1979. The U.S. has done it on average at least once per president. And that's if you count only the illegal ones, you know, an international law, the ones not sanctioned by a security council. You know, in last year we've invaded Iran,

I mean, bombed Iran, invaded Venezuela and Iran, invaded Iraq, blah, blah. It's been, like, for the last half century, China hasn't done any of that stuff. So, and I'm not saying, you know, well, then we should give them the power not us. I'm just saying, from a perspective other than the American perspective. So, many of the premises that people like Dario on the day are carrying into these conversations are just not self-evident. And it might be a healthy exercise to ask ourselves,

like, what does things look like from the point of view of another country or what do things look like, you know, from, like, beyond planet Earth. If you, if you tried to get an actually objective perspective, I know it, I know it's been a long detour, and I'm doing a rant, and I'm sorry, I get agitated, but the point is, like, if you ask, why do I bring up mindfulness and various other things as, like, paths to greater cognitive empathy, getting better perspective, taking,

“I think, unless people in general, not only America, I would say the same thing in Chinese. You know,”

I know you have your national perspective, try to transcend it, because the whole world will be better off if we all get better at that. That's, that's all I'm saying. I think that's a great place to in the discussion, Bob, right? Thanks for coming on the show, and thanks for writing a really terrific

you're reading it, and giving me the, the airtime. Scaling law is a joint production of

“law fair in the University of Texas School of Law. You can get an ad free version of this and other law fair”

podcasts by becoming a material subscriber at our website, lawfairmedia.org/support. You'll also get

access to special events and other content available only to our supporters. Please write and review

“us wherever you get your podcasts. Check out our written work at lawfairmedia.org. You can also”

follow us on x and blue sky. This podcast was edited by Nome Osband of Go-Rodeo. Our music is from

Alibay. As always, thanks for listening.

“Do your current managed services really help run your operations, or are they just running in”

circles? Running isn't enough anymore. With PWC's managed services, your operations don't just run, they evolve continuously, powered by AI embedded directly into your workflows, so instead of maintaining yesterday's model, you're building tomorrow's advantage. PWC's managed services, we run your operations with tech and talent, so you can run faster, scale smarter, lead stronger.