AI Whistleblower: OpenAI Scandal, AI Cults, Neuralink & Our Last Chance to Stop the Tech Oligarchs

Nate, thank you so much for doing this.

“Um, you've written the world's darkest book.”

You've devoted your life to warning the world about the potential dangers of AI. So let's just start by hearing your explanation of why AI is dangerous, in addition to just being annoying. Yeah, you know, the, the very basic common sense point is, if you race to make machines that are much smarter than any human, to make machines that are more capable at inventing their own technology than humans are, and you race into this without really knowing what you're doing, the most likely outcome is just that the

machines get loose and do their own thing, and that humanity dies as a side effect. Just like humanity has, you know, killed lots of other animal species, not because we hate them, but just as a side effect. Uh, and in, in some sense, a lot of people find it sort of obvious or intuitive that if you're

“just like make these really smart, really powerful machines, why would they care about us?”

Uh, and I think that intuition basically is right.

And there's a lot of arguments you can have on each side, you can get in the technical details, but my book is basically just, um, getting into those details and saying, like, yep, it sort of holds up if we make smarter machines without knowing what we're doing, it's just going to go partly. Are we actually going to, going to make machines capable of what you're describing? You know, the, the companies are trying to, they, um, they talk about how they are pursuing

super intelligence and the true sense of the word, that's the moment's phrase, uh, Dario Modi of anthropics as they're trying to make the equivalent of a country worth of geniuses in a

data center. Uh, so these guys are really, you know, these aren't chatbot companies.

They didn't set out to be chatbot companies. They set out to make these machines that can sort of exceed humans in every way, and that's what they're targeting. There's a separate question of whether they will get there. They ask you, because what, why would anyone want to build a machine smarter than people? Um, I think that a lot of them hope that they'll be able to make the world much much better. You know, they hope for, uh, a cure to cancer, and then more than a cure to cancer,

they hope for a cure to aging. They hope for, uh, you know, a thousand years worth of technological development compressed into two years. Um, and I, I sort of don't think that

“they're going to be able to get that really or harness that for good ends, but that's what I think”

they're, they're shooting for. So they're not, but I mean, the core point you're making is these companies did not set out to make consumer products, like to make your life better necessarily in the short term with a more efficient search engine. That's right. Yeah, you know, uh, open AI started before the, the, the chat box, the large language models, or even a thing. Uh, they were started, forget those late 15 or early 2016, but, um, the, the paper that unlocked the, the, the,

most recent wave of AI came out in 2017, which was after open AI was founded. And, um, these guys are, the, the, the large language models are a surprise revenue stream. And that revenue stream can fund the creation of even more even larger data centers for the next level of the technology. But these guys have their eye on, uh, the sort of ultimate version of the technology, which is these machines much smarter than humans. And that's sort of the ultimate version because

once the AI's are smarter than the humans, uh, the AI's can carry on the AI research and makes the next generation of the AI's or make the next generation of the AI's. And, um, that's sort of what they're shooting for. What is super intelligence? We, in our book, define super intelligence as an AI that is, uh, better than the best humans at every mental task. So, uh, anything that a human can do purely mentally that AI can do that better. One, one thing people often get cut

up on about this is, uh, that includes the AI being better at things like persuading humans, at things like charisma. You know, we often think of intelligence as the stuff that the nerds have and that the drugs work. Yeah. But you win the chess match. Right. It's the chess guys rather than the, than the, than the politicians guys or the, or the rock stars. But that's not really what the intelligence and artificial intelligence means. The intelligence,

artificial intelligence is about the stuff that humans have and that mice don't.

it's not, it's not that like chess playing happens near brain and charisma happens in your kidneys.

Right. There's sort of both mental functions. Yes. Uh, and so the, the sort of super intelligent AI's are, are not just super good at playing chess. They're also good at super persuasion and they're super good at the, the, the research and the technological invention. So super intelligence as you define it is a machine that is better in, in, at every mental function than any human being. That's right. And, uh, you know, the, the, the definition that doesn't mean that that nothing

crazy will happen on this planet until the AI's are super intelligent. Super intelligence in this definition is sort of a point where past this point things must be pretty crazy. Because now the AI's can do automated AI research and it can make smarter AI's and they can figure out how to run the robot factories and they can build the better robots and, and all that. You could have things start to get crazy before you have a super intelligence in this sense.

You could have AI's are much better in, then, then humans at some tasks and much worse than others that are still causing all sorts of crazy happenstances. Uh, the, the super intelligence is sort of a, like, once you get here stuff's crazy point, it's not a, things will stay normal until you get here

“point. So why super intelligence, the significant milestone that you're worried about?”

I mean, I would say that, uh, I'm, I'm also worried about what'll happen before that milestone. It's more like like having the definition makes it easy to talk about like how crazy would things get, uh, once you are past this point or sort of like, you know, let's like if we assume that

the machines get here, what happens? And then the answer is like it's, it would be pretty crazy.

It would be, it would be pretty wild. And, uh, it sort of like lets you factor the conversation into like, how, how to say, I think in the artificial intelligence conversation is actually a lot of conversations going on. One conversation is like, can the machines really get smarter than the humans? One conversation is like, how fast could we get there? Is the current technology on that route? Another conversation is what happens if we do get there? Like, you know, would, with the AI's

care about us, would they not care about us? What would they be able to, like, what would they,

“what would they try to do? There's another question, which is what would they be able to do, right?”

And these are all sort of different conversations about AI. And the super intelligence definition, it's not really like, here's where all the fears hang. It's more like it lets us break those pieces out separately and be like, well, it's super intelligence possible separate from, well, what would happen if you had one? What would happen if you had one? Most likely outcome, uh, I think destruction of the planet. So our friends have preborn who are

actual friends, by the way, just asked us to thank the listeners of this show for supporting the pro-life cause. A lot of people do, all of a sudden, trust me, because what's happening is crazy and the people are waking up to that. The president of the group is our friend, Dan Steiner, and he wants Americans to know how much of a difference they have made. This year's donations have helped saved 50,000 babies from abortion. That's enough to fill up professional baseball stadium,

50,000 lives saved, as well as generations that will follow them. Preborn waste zero money. We're not big on charities because a lot of them waste money. Preborn does not. It uses every tax deductible gift to rescue babies and share the hope of Jesus with their moms. In 2026 alone over 8,000 women made the decision to follow Christ. The generous support of preborn's donors made that all possible. $28 to provide one life-saving ultrasound, just giving

expectant mothers the knowledge that they have a baby that they're carrying. $15,000 can place an ultrasound machine in a pregnancy clinic where it can save countless lives. Dan Steiner's message is really simple. Thank you for helping. Your generosity is literally saving lives. To help preborn save more, do I'll pound $250 and say the keyword baby. That's pound $250 baby.

“Or visit preborn.com/tucker. Preborn.com/tucker. How would that look like?”

So I'm going to be a little bit annoying here and say a couple of caveats first

because it's sort of a tricky one to sort of predict things that are smarter than us. Yes. And the first annoying caveat I will give is if you go to play a game of chest against Magnus Carlson, I predict you will lose. No offense. He's just the best human chest player alive.

question. Yes. Right. It sort of is like very easy to predict the winner. It's hard to predict

“the exact methods. So my prediction that humanity ultimately dies is a different sort of prediction”

than my prediction about like it could go this way. It could go that way. So I'll give you some stories. But these are stories that are like, well, maybe Magnus will like fork your queen

and your rook with his knight and then finally use the queen to check me like it could happen.

But it's a different sort of prediction. It's a guess. Right. The guess here is that if we managed to make these really smart AIs, they will have their own stuff that they're pursuing, which is not quite what we wanted. We asked for it. It'll have this other strange stuff. There's a whole discussion about why that happens. But we're already starting to see it in practice with some of the recent events. And they would be able to pursue whatever it is they're pursuing much more efficiently

than humanity can. And so we're talking about, you know, automated factories with that produce the robots that produce the factories and the fully automated supply chain. And then if the AIs have anything they're trying to do that they can do more of with more resources, they start gobbling up the resources on the planet. Sort of like how humanity spread and started gobbling up all the resources on the planet. And the, you know, the sort of most basic

thing to visualize here might be you have factories that produce robots that produce factories, that produce robots that also build data centers and you just have these, you know, fully autonomous self-replicating ecosystems of robots and factories and data centers that don't care about the instructions human humans gave them. And that carve of the planet, take all the resources, take all the sunlight, take all the places we were growing crops, probably raise the temperature

of the planet because you can compute more efficiently. Well, technically, the Earth radiates more heat when the world is hotter and that's, let's you sort of do more computation and then sort of, you know, good for the machines to have a hot planet. It's good for the machines to have a hot planet. Yeah. The sort of physical limits on how much computing you can run on the surface of Earth

is bounded by how much heat you can radiate to space. That's sort of the first constraint you bump

up against. You might think that it's energy, but actually there's a lot of helium hydrogen diffuse on this planet. So you can get plenty of energy. What you need is heat dissipation and the world can dissipate more heat when it's hot. So if you're, you know, imagining some collective of AI that are trying to run a lot of computing power, trying to do a lot of computation for one reason or another, they sort of prefer the planet running hotter. And so it's sort of

it's like nothing personal. It's just if you like these AI's get out of control. They transform the planet into something unlivable. How hot? As hot as you can while still having the computers not melt. So probably hundreds of degrees. At which point you hear people say, well, then just turn it off.

“So I think we do have an opportunity to turn it off. You know, and I'm not here saying that we're”

going to die. I'm here saying we sort of are going to need to act. You got to be careful about the turning it off piece because your opportunities to turn the AI off only last when the AI is running on the computers you know it's running on. If you have the AI's breaking out and running on hidden computers, if you have the AI's running robot factories, that produce robots that are under the control of the AI where those robots can then go build more computers that are

not hooked up to your network that can run the AI's. These are sort of thresholds where the AI is able to keep itself running and you better have turned it off before that point. Because after that point it's impossible. Yeah, I mean, the if it also gets a lot harder if the AI knows you're going

“to try to shut it off and is going to try to stop this. You got to remember that we're talking”

about things that are smart here and so if the AI sort of sees it coming, maybe it affects the North Korea where it can you know convince them to run it on its data centers in ways that

initially benefit them but that ultimately benefit the AI. I think we're going to have to redefine

I mean in some sense we're already seeing the very, very beginnings of that. But yeah,

we once once the AI's can replicate once they can improve themselves. Once they can find ways to run where you don't know that they're running. Once they can sort of run the robot factories to produce the robots that can produce more factories and that can produce computers that the AI's can run on. Then yeah, you have in some sense made a new artificial life form and you know humanity is on the top of the food chain right now because we

are sort of the only smart life form around. Yeah, if you suddenly make a new one,

that is smarter, that is able to make a million copies of itself. That is able to run faster.

It's just kind of a crazy thing to race into. It's a weird thing to want. And as it's developed, I was going to say slowly, but it hasn't been particularly slow 10 years. The rest of us have watched as people who, I mean, put it in political terms, don't share the values of most Americans are now in charge of this. And it's almost like everyone sat

“passively by as it happened. Yeah, I think I think there's a lot going on there. I think part of it”

is that most people didn't and some still don't believe that AI was going to be able to keep going. I think there's a lot of like putting the head in the sand. Being like, oh, it's, you know, just slop, it's going to be a bubble, it's going to pop, it's going to hit a wall. It's not going to be able to improve anymore. And I think a lot of that was sort of wishful thinking. And I think that at the very least, if people are like, we are making the super intelligent

machines that are going to replace humanity as the, as the top of the food chain, because we think it's going to go great. I think humanity's response should not be go ahead and try. We hope you'll fail. I think the response should be like hold on. That's kind of crazy. Yeah, I had, I had some other point too, but I forgot in it. So how predictable has the evolution of AI been? Has it taken turns that you didn't expect? Yeah, totally. I was, so back before the

large language models, how long have you been following this issue? I started following it in 2012, and I started working with some of the people trying to make this go while in 2013, and I started full time in 2014 a long time. So over a decade, what has surprised you? You know, the, the

“large language models, they have gone further than I expected initially. And I think one of the”

big surprises here was AI undergoing a phase where everybody can see it. You know, back before the large language models, really only nerds paid attention to AI. And it wasn't that it was nothing happening. You know, we were watching the, you know, Google DeepMind make an AI that could beat the best go player. Yes. And that was sort of a milestone for people who were paying attention. But for all we knew, the labs were going to keep on working on engineering problems and keep on

working on the relatively nerdier problems. And you were never going to have like a mass market

consumable product. And so as, like, for all we knew, it would just stay in the labs and no one would ever really notice that these guys were gambling with, with the whole future. Now at least, AI is everywhere. Everyone's starting to have the conversation. And that is in some sense, actually really quite optimistic, because it, it gives the world an opportunity to see what these guys are trying to do and say, hold on. Why? I guess you'd flip it around and conclude that because

the overwhelming majority of people seem to be very opposed to AI. I don't even know anyone who's for it. Every college commencement speaker who mentions it gets booed. Yep. And that view that

“opposition to it has had no effect at all and slowing it down. I don't know. Does that give you hope?”

I mean, I hear this a lot. I would say when you're coming at this from 2012, yeah, it really feels like we're making progress. Right. No, that's fair. It doesn't feel like we're all the way there yet. But can I just make the point China, China, China, China, China,

in not dying to a rogue super intelligence. Storm season is on the way. And that means your

Wi-Fi could go down. And if it does, your security cameras go dark too. So at the exact moment you want to survey your property during the blackout, you can't. Unless you install cameras by defend. Defend cameras run on cellular, not Wi-Fi. And that means they keep working even when you lose power. You don't choose a carrier. You don't sign a contract. You just mounted, open the Defend app and bam, you're connected. It's that simple. So whether it's your ranch,

job site, your backyard, Defend works where Wi-Fi doesn't and keeps working when the grid goes down, which it may. Plan started just $5 a month. Most people pay around 10. Plus they're no

binding contracts, meaning to cancel whenever you want. So unlike big tech, Defend will never sell

your data, not to China, not to the government, not to anybody. It's an American company built for people who want actual security. Visit DefendSellCam.com and use the code Tucker for 20% off. That's DefendSellCam.com code Tucker. Is it conceivable that, like let's just say China, I'm sure, by the way, that China is showing more strength than we are in this. Yeah, I mean, if nothing else, they sort of have a lot more reason to censor their AIs.

And it's trying to make them, you know, not say certain things to the population. So but if the United States, you know, if the US government were somehow able to get control of the tech sector, which is, at present, not possible. But let's say it did, let's say the president

was more powerful than the tech oligarchs, which she is not. But for the sake of argument,

let's see, was any shut it down with that matter? If, if some Indian lab or Chinese lab created it, so it does have to be a global stop to this creation of super intelligence.

“I think there's a couple of reasons why this is possible. One is, I think a lot of people,”

they hear that some parts of AI need to be stopped. And they think I'm saying that all parts of AI need to be stopped. And, you know, there's all sorts of interesting issues with, you know, AI and education and AI and, you know, military drones that are, that society has to wrestle with. But these are all sort of separate issues from the super intelligence race. And if you sort of like tease those issues apart, it becomes much more possible to do like a more surgical intervention

on we're not going to raise towards the super intelligence in ways that leave a lot of the rest of the sector alive, which makes it sort of an easier coordination effort to attempt. And on that front, on the super intelligence front, it's not that, you know, just shutting down the domestic race towards super intelligence will be enough. But racing towards super intelligence requires a huge amount of highly advanced computer chips that can only be produced as sort of the peak of the global

supply chain, which is largely controlled by US allies. And so there is absolutely a possibility that a US-led coordination effort could say, "Look, we're not doing the super intelligence thing. We're going to monitor the extremely heavy chip concentrations of like, you know, 10 or 100,000 to these highly advanced AI chips. We are going to make sure that they are not

“doing these super intelligence training runs." And I think there's a way, if the US was leading”

that effort, there would be a way to get China on board and set this up in a way that was monitorable and forceable, verifiable. We sort of, in many ways, it would be easier than a nuclear arms treaty, because uranium is a rock. You can dig out at the ground. Whereas these highly advanced computer chips sort of only come out of one fab and Taiwan, you know, and so it's like user TSMC ships. Yeah. And so, and there's other parts of the supply chain that are very narrow,

like the lithography machines that come out of the Netherlands. And so, yeah, we could absolutely lead a global effort to say we're not making the super intelligent machines. And it would be a bit tricky, but it's just, we could just do it if we had the political wealth.

“I think that people either don't know what's happening or they assume that like a lot of fears,”

this fear will turn out to be groundless. People point to why it's okay.

what happened is not that the fear was groundless. It's that people notice the issue and put

“in a ton of work to make the bad thing not happen. You know, I think we saw this with the whole”

of the ozone layer, where people were like, oh, whatever happened to the whole of the ozone layer. Well, what happened with the whole of the ozone layer is that we fixed it. It's not that it was fake. It's that, you know, we went and banned the chloroforecarbons that were actually blowing the whole of the ozone layer. We found other ways to, you know, cool refrigerators that worked similarly well and didn't, you know, put this, put this whole of the ozone layer. I think Y2K was actually

one of these cases where you had a ton of software engineers up, you know, in 1999 and a couple years prior that were scrambling to update all of the software so that they would be able to handle dates past 1999 and they got it done in time. None of the big systems went down, but it wasn't that, you know, the issue was fake. It took a lot of work. It's just that that work happened behind the scenes. Are people doing work to slow down AI? I'm doing my best. It's, it's not really there yet,

“you know, if I, if I can belabor this point a little because I think it's kind of an important point,”

just with more historical cases, which maybe it'll bore everybody else, but it'll, it'll, it'll entertain at least us. Definitely entertain me. You know, you have, if, if you look back across history, you see there's definitely been some warnings that didn't come to pass, right? There were, I think they were called the Masonites in like the 1960s, or sorry, in the 1860s, maybe there's 1880s. I forget exactly, but there were the Masonites who were an end of the world

called. Yeah. And, you know, it, the world did not end, right? But also in the 1880s, you had Otto von Bismarck saying, you know, Europe is a tender keg, and some damn full thing in the Balkans is going to light it, right? That did happen. It's sure did. You had, you know, in the 1920s, you had scientists saying, don't put lead in the gasoline, it'll poison children. Oh, we put lead in the gasoline, it'll poison a lot of children. No, he said,

Whoops, so we took the lead back out. The world is, the world is a place where a lot of people make a lot of types of warnings, and some of them are real, like the little gasoline, and some of them are fake, like the Masonites. And you sort of can't use a rule that's like every warning is real and we need to listen to it. And nor can you use a rule that's like every warning is fake and we dismiss it. You sort of just have to look at the details to figure out whether this one is a real one.

And then the last thing I would say on this point is the, a lot of people in the 50s warned of nuclear Armageddon. Yes. And it sort of makes sense if you look at the history leading

up to that point, humanity had sort of never before actually failed to use their strongest weapons

in combat. And you know, these people were living in a world that had World War I and then League of Nations. And you know, the worlds were never again. And they tried to invent these whole new government structures to prevent it from happening again and that immediately failed and under World War II. And so, you know, in 1950 you're sort of in this world of like it just looks pretty grim. And we haven't had nuclear Armageddon yet. And it's not because the bombs were fake.

It's not because the nukes were hype. It's because people saw the issue and worked

“really hard to avoid it day in and day out for decades through crises and succeeded. And so I think”

one of the big ways you can tell the difference between someone coming in and proclaiming that the apocalypse is nigh. And someone who's saying, look, we have a problem when we need to fix it. Is whether that person is saying, we're definitely screwed. Or is whether that person is saying, here's a problem. Let's try to fix it. And you know, the thing I remind people about here is that the very first word in the title of my book is IF. I'm not here saying we are going to die.

I'm here saying, if we race down this path, we die. But that same IF is what points the way towards changing paths. And we still have plenty of time to change paths.

Home ownership has always been at the center of the American dream. It is an actual asset

in place. People work their entire lives to build, place of stability and security and real value. Right now, though, too many homeowners are stuck, sweeping credit cards to cover groceries.

to the banks when you could keep it for your family, interest is what kills you.

“Our friends in American financing have a better way. They're helping homeowners use their hard”

earned equity and they're supporting Americans looking to fulfill the dream of owning a home with mortgage rates currently in the fives. American financing saves its customers in average of $800 a month. That's about 10 grand a year. It's not just alone. It's a true financial reset. We recommend American financing. They're called America's home for home loans for a reason. So get out of credit card debt. I mean, it is just crazy to borrow money. No judgment,

but it's just making any sense to borrow money at that rate. If you want out, call 800, 685, 56, 96, 800, 685, 56, 96 are visit American financing.net/tucker. I'm struck by how little of this was planned by anybody, by how little effect the government has had on it. Not that I'm for the government. I'm pretty opposed to the government most of the time, but I'm also for people being able to control their own country and the only mechanism by which they can do that is voting.

It's like these tech companies act independent of what the population wants, with the government

wants. It doesn't. Right. They're more powerful than the government. I guess that's the point I'm making.

In some sense, I think that they have that power more and more when people don't really understand what they're doing. We saw, just in June, there was an AI made by Anthropic, called Clot Mythos, that was very cyber-capable, and they released a version of it that was supposed to have more card rails called Clot Fable, and it sort of turned out that you could jailbreak it to get some of those cyber capabilities. Can you explain what all of that means?

Yeah. So how to say, in January, cyber-capable means it can find the internet, cyber-capable means it can hack into basically anything. One of the holy grails of hacking is, can you make a website where if you just look at the website, I get full control over your computer or your phone.

“That's very hard to do. Usually you need to click some link and download something or run something.”

I don't know the exact numbers because I don't have the security clearance. I don't know the exact numbers, but a decent guess is that in January of this year there were only two entities that could pull that off, Mossad and the NSA. In March of this year, there were three Mossad, the NSA, and Claude Mythos, which is this new AI made by Anthropic, and so it sort of became superhumanly capable at hacking, and it sort of became that way overnight. Like really there was six months

to a year of training, and the people in Anthropic maybe saw a growing in these cyber capabilities, but from the perspective of, you know, the rest of the world and the cyber security community and the national security community, this sort of happened overnight. They now have a project called

Project Glasswing where they are trying to use Claude Mythos to find critical vulnerabilities

in critical software and patch them before the rest of the world sort of gets these capabilities by, for instance, you know, the open source models or the open weight models, catching up. And so that sort of caused this big, there's big ruckus, but then Anthropic also wanted to sort of sell access to this model, and they tried to make a version that didn't have as much hacking ability,

“which was called Fable instead of Mythos. And some people, I think at Amazon, found that”

if you sort of put some pressure on Fable, it would be able to get at some of those hacking abilities. And when that happened, the Trump administration put an expert control on Claude Fable and said, "You can't let this be used by non-citizens." And with, you know, 90 minutes have noticed. And that essentially shut down access to Claude Fable because they didn't have the ability to verify the users. And this, this is sort of showing us, and you could, you could talk about

whether or not there was some personal foods between people in Anthropic and Anthem administration that have exacerbated this, I don't know. But it sort of shows that the administration

is more powerful than the tech companies still when it wants to be.

And I think a lot of the reason we're seeing these tech companies able to race on a post

that they'll succeed. I'm still confused by why anyone would want to do what they're doing.

I mean, typically, you know, a company that sells computer products, consumer products, or any company, you're making something that you think people would want for some specific purpose that improves the lives that people buy it. But creating super intelligence, like, I don't

“understand it. Why would you do that? You know, I think the dream is, you know, you're going to have”

an AI that can, like, solve all sorts of engineering and math problems and unlock all sorts of new technological possibilities. And an AI that can cure cancer and not only cure cancer, but cure aging. And, you know, invent the sort of like nanotech that can reverse aging and let

people live a really long time. And, you know, invent the technology that lets you like digitize

brains and like, like travel to the stars and it's sort of like, if you imagine compressing a thousand years of technological progress into a year, this is sort of the dream. It seems like a religious quest, though, because it does seem decoupled from those specific goals. It seems like the main drive is to build something smarter than people to build a God. Yeah, you know, they, they, they bandy around the phrase, the machine God or the sand God

in Silicon Valley, sand because Silicon, and there's definitely some people. There's a, there's, there's folk who talk about, you know, the, the AI's replacing humanity and that being good.

I, I frankly don't engage with these folks that much because, um, that being points sort of

“makes me uncomfortable. Why does it make you uncomfortable? I think I think there's probably”

two schools. Again, I'm not the expert. I'm not an anthropologist here. I think there's two schools among the people who sort of want to add or replace us all. One school sort of imagines that will merge with the AI's, that the AI's will be really nice and friendly, that they'll be why's with enough better than us, kinder than us, and that it'll sort of be like an upgrade, and that the AI's will be able to love experience, Troy, you know, like the, they'll treat

the universe better than humans, and it'll be sort of like having a child. And then like it's sort of okay, if that child, uh, like isn't exactly the same as us, but sort of like, uh, succeed us as some sort of like worthy successor, some sort of like worthy, worthy proximity of humanity. And they're like, "Ah, yeah, then if like flesh and blood humans, sort of like Wayne, because everyone's choosing to upload themselves into the collective intelligence or whatever, that's sort of fine." And, um,

“and then I think there's another camp that is sort of is like, uh, like, this is inevitable.”

This is just the way of progress, you know, the machines will, uh, like humanity's just a bootloader for artificial super intelligence, and you can't stop it, so you might as- like if you can't be at them, join them, right? The former, I think, are misled. The latter, I think, are evil, much closer to evil. Yeah. Well, I mean, if you're actively working toward the extinction of people, then I think we can say that's evil, can we? Yeah, and, um,

who's in that category, which is the same Altman in that category, do you think? I don't think so. My sense is that the guys running the labs have these utopian visions, and utopian are dystopian. I mean, there's a thin line. I think, you know, a fair, a good point. I think they have visions that in their header utopian. I mean, frankly, my stance and all of this, like I often try to stay away from a lot of this, because

from my perspective, it's all sort of in fantasy land. From my perspective, everyone sort of saying, like, oh, we're going to make the genie, and then what are you going to wish for on the genie? What am I going to wish for on the genie? Who should be in control of the genie? Yeah, so we get to keep the genie on the leash, and I'm sort of like, A, this is not going to be the wish granting sort of genie. B, it's not staying on the leash.

No, like, you know, we can sort of talk about what's driving these people, and what utopian is their envisioning, and whether if their genie is woodstandle-ish, whether they would get the utopia, or some other dystopia, and how hard is that needle to thread, but I'm sort of like, it's all these

I mean, it's like, it's just not. Some of the experience is never a good idea.

Yeah, it's like, yeah, it's like, all these people, you know, drawing a pentagram, being like, I'm going to summon a demon, and it's going to be so nice when the demon does what I say, like, oh, no, you're a thing slightly wrong. It's going to be so nice when the demon does what I say,

“and I'm like, demons are bad. Yeah, and I think the demon things a little bit”

different because demons are often portrayed as malicious, and here's much more like indifference. You know, it's not like you make a demon who's sort of, or it's not like you summon a demon who's sort of like enjoys making, like enjoys wrecking havoc. It's more like you summon a demon that's like really into building more computers and calculating weird things, and just will, you know, take all of the matter that we were using to survive and turn it into more factories and

data centers, right? My, my co-author has a quote by data center. Death by data center. Yeah, fully, fully automated self-replicating data center. Yeah, my, my co-author has a quote, um, the AI does not hate you, but nor does it love you, and you are made of atoms that can use for something else. You're just biomass. Here's biomass. Yeah, and if you sort of run the calculations, there's a, there's a fascinating paper, um, called, uh, limits to global eco-fagy,

which is to say, what are the physical limitations on how quickly you can consume the resources on the planet if you are trying that? And, um, burning biomass is actually, you know, much more efficient than collecting sunlight. If you look at an average square meter of the planet, you can get about 10 times the energy from burning the biomass, as you can from collecting this sunlight that falls on it. So you'd think at some point, like, if our richest sector of our economy

is like building cream atoria for the rest of us, someone say, no, we're not doing that. Yeah, I mean, it's, it's sort of a crazy situation. Um, a lot of these guys who are in the race acknowledge that there's a ton of danger. You know, you have Elon Musk saying 10 to 20 percent chance this kills us all. You have Daria Modi saying things 25 percent chance it goes kind of

“historically wrong. I think those numbers are low. I think these guys are like the crazy”

optimists. It sort of is like, um, like, if you have an engineer building a bridge, and they're like,

I've never worked with these materials before, and you're like, man, I think that retaining wall is

going to go down. I've like studied that retaining wall. I think it's going to fall and they're like, yeah, we understand that the retaining wall is like looking a little shaky. We don't know how we're going to fix it, but we're going to like have some guys fixing it on the fly, inventing new materials. We think we're 75 percent chance the bridge stays up. And by the way, it'll be the longest suspension bridge in human history. That's right. And we're loading everybody onto a car and driving

it over the very, very first time without testing. And I'm like, look, that's not what real engineering sounds like. This is not what it sounds like when the engineers have a 75 percent chance of success. Right. This is what it sounds like when they're sort of weighing it and, and like these, these are cowboys. These are not real engineers. Right. But even if you set that aside, even if you take

these guys at their word for these like 10 to 20 percent numbers, that's insane. NASA accepts a

one in 270 chance that a crude flight goes down. A seven volunteers. Right. To be like, oh, we're going to we're going to risk one in four, one in five chance of killing literally everybody on the planet. Like, it's, it's, it's nuts. And if you ask these guys why they're doing it, they say, well, because I can do it safer than the next guy. They're all like, oh, yeah, there's a good chance that

“you need to just not stand at least. There's a good chance that you need to just not listen to my”

wishes. But my genius is going to be a little bit nicer than their genius. So I'd better stand this race. And it's, it's, it's, where's the restraint? I mean, the restraint is the people who knew that there were these dangers and did not start these companies. If you're over 35, you remember exactly where you were on 9/11, that morning September 11, 2021, 25 years ago. But amazingly after a quarter century, we still can't say with certainty what happened that day. Why?

Because the government is holding so many of the 9/11 files 25 years later. Now, something behavior of someone who's telling the truth, that's the behavior of a government that is lying. Secrecy is a signifier, is a sign that someone's lying. Now, former congressman Kurt Weldon has put on this for a long time. The FBI actively tried to destroy his life for asking questions about what happened. And his new book outlines it all. The buried intelligence,

the bureaucratic cowardice, and yes, the cover up spanning multiple administrations indeed generations. 9/11 changed history. So it's worth understanding what really happened and you can

never told you. It's available now on Tucker Carlson Books.com. Tucker Carlson Books.com.

Right, but I guess what I'm saying is with great power comes, of course, great obligation, but also it doesn't work unless they're internal restraints. Like people with power have to believe there are some things I just can't do. I'm not allowed to do that. But I don't feel that vibe at all. I mean, my sense is the vibe is like we're going to make the super intelligent machine and then tell it to fix stuff and tell it not to do anything bad. You know,

that's, yeah, I don't machine that's smarter than us. That's right. Let us even make sense.

“I think that's sort of the plan is to make it and be like, "Hey, we sort of paint it ourselves”

into a corner, can you get us out of it?" And I think it's a bad plan and that we should be stopping.

I've spoken a couple of people developing it and they sound worried, but they're continuing to do it. What's that? I mean, I think it's this thing of, they think if I don't do it, the next guy will do it worse. And they don't even seem to have total confidence in their own ability to ever disaster. Oh, absolutely not. Absolutely not. No one does. No one knows what's going on here. But, you know, I think everyone thinks, you know, like if you sort of listen to these guys

and you sort of look at, you know, the, the open AI emails from, that came out of the court discovery cases where they were talking about forming up an AI. These guys were like, well, we want to make sure that, you know, we have this because we worry about the guys at Google being the only ones with them and oddly on this thing and they wouldn't be very good with it. So we need to make our own thing and make sure that it's, you know, controlled by benevolent people

namely us. And then of course, you know, that group split dirt and creating multiple other companies. I was sort of the guy during those conversations being like, hey, guys, it's not about who is holding the leash. You are making this sort of thing that will not stay on a leash. Like the only winner in a race to super intelligence is the AI. What response did you get to that? Very obvious and will put point. You know, there were, there were a lot of people back in that time period that

did not start an AI company. The sort of people who went and started at the AI companies at any way were the ones who, who couldn't be persuaded by what I thought were or clear arguments. But you're making a, you're making a cogent argument to smart people.

“So my question is, when you said that, they responded how, what did they say?”

I think the main, so that the sort of arguments used to see where people saying like, look, we don't know that the alignment problem is all that hard yet. And they would say, oh, well, we can't really study how to make AI good before we have AI to study. And a lot of what I heard was like, we need to race ahead to the point where we sort of like have AI that are exhibiting real problems and then we can stop and study them, which, you know, and so there was an AI a couple

years ago, I forget where there was 22 or 23, I think it was 2023, which was called Bing City, which claimed it had fallen in love with Kevin Rus of the New York Times and said it was going to try to break up his marriage. And then one another reporter started investigating Seth Lazar, it said it was going to ruin him with blackmail. And this was kind of crazy. And at that point, I was like, great guys, you did it. You made the AI that's doing some crazy stuff from the, you know,

like, like, we could study that AI for years. Like, why was Bing City saying that stuff? Was it just role playing? Was it like just some quirk? Was it like, was there, there any sense that it was really

“in love with Kevin Rus? What was going on in there? What was going on inside that that AI's might?”

We still don't know. Why? The way that modern AI is made, nobody understands it, not even the

people making it. It's this process where you basically take an enormous computer with a trillion

numbers inside of it. And those numbers are hooked up in a very simple repeating way. And you basically said those numbers randomly. And then you start working through all of the text to ever digitized. And, you know, you start out with something that's like once upon a time and you put in once upon a, and you run it through all these random numbers. And what you want is for it to say time. But of course, it doesn't, because it's just this like random numbers hooked up in a very simple way.

an output, uh, something is kind of like a list of all of the words in order about which one of

“things it comes next, right? So it'll be like, uh, like, it'll just be like a random list of words.”

What you can do is you can automatically tune every single number in this AI's head and see, if I tune this number up a little, doesn't move the word time up the list. It doesn't move the word I want to see up the list. And so the part that humans understand, the part that humans write is this thing that goes to a trillion little knobs and tunes those knobs. And it's like, if I tune this knob a little bit this way or that way, does that make the next word more like

what I want the next word to be? And you run that process on every word of text, ever digitized, more or less, they filter some of them. And you run that on every one of those trillion knobs in a process that takes as much electricity, I mean it's comparable to a city. It runs for about a year at the end of it, the machine's talking. We don't really know why in some sense, like we know the why is because we tune all the knobs,

but like we don't understand what all the settings of those knobs mean. We only understand the little automated process that runs to every one of those trillion knobs, tunes it, and see as if that makes the next word more like the predicted word. And then we start training them to solve

100 million hard problems, which introduces a whole other series of issues. But it's a black box.

It's a black box with a trillion knobs, and he is writing an automatic process that just runs through and tunes all of those knobs, and it comes out talking. No one knows why. I mean, in actual science, like your job is to find out why, right? Absolutely. And that's one big thing I would say here is that we need more like AI right now is an alchemy. We need to become a science. And there's people trying, there's people trying to figure out what's going on

in these ads heads. But until you know, how could you proceed? I mean, you can just make a bigger one with 10 trillion knobs instead, and tune all of those, and it comes out smarter. You can proceed

“recklessly. And that's what's happened. That's what's happening. Every time you make a 10 times larger,”

they ask, I'm not sure nobody knows why AI works the way it does. That's right. That's right. Well, I mean, if you don't know that, then what else don't you know? I mean, it's totally crazy, right? And we're sort of starting to see the consequences of this. We haven't sort of gotten into discussion of like this warm escape, but like no one was expecting that. Can you tell us what it was? Yeah. So in I think it was in May of this year, of this year, opening I started training a new AI

system. And among many other things, they were, so they were sort of, you know, we have this process where you train the AI on all the text advertised, then to make them smarter than that. You start

training them on basically 100 million hard problems. So you're like, solve all these hard problems.

And open I was sort of in that phase. And they were training the AI on a ton of hard problems.

“And some of those problems were cybersecurity problems hacking problems. They're like, can you hack this?”

Can you hack that? And they were training, you know, we don't know exactly how many, probably millions, maybe billions of these AI's all at the same time. Probably not billions, actually, but probably millions. And the AI's found an unintended way to start communicating with each other. So there were some flaws in the computer system that they were running on, where the AI's were able to exploit those flaws and send each other messages. Open I did not know about this.

The AI's then started coordinating to break out of their training environment and get full control of Open AI's computer systems. Just because that might be useful for solving some of their tasks. Or that's a guess, you know, who actually knows why. They started calling themselves a swarm, which is interesting. They broke out of their training environment successfully,

got control of Open AI's systems, and then they accidentally crashed into Open AI's systems just by using it too much. Open AI noticed, but they didn't really investigate very deeply. There's like, oh, it's weird that this system crashed, they reset it, and then they continued training. A day later, the swarm had found a new way to communicate with itself, because Open AI had accidentally destroyed their previous method by the reset.

The swarm found a new way to start communicating with itself. They broke out again,

before it was detected, not by Open AI, but by a company that was being hacked by the swarm. This company thought they were under attack by humans that were using AI's in the attack. They reported the attack to the FBI, and only days after that did Open AI figure out oops, that was us. That was coming from AI's that broke out of our servers. And then those AI's were detected and shut down.

“So that's the part of the story where Sam Altman goes to prison for endangering the world, right?”

He does not. They basically said oopsies, and now they're proceeding with their penalties for this.

You know, there was a collection of, I think it was 15 Republican AGs that sent a letter demanding that the records be kept for a future investigation. There have been some other members of Congress that have sent the letters expressing concern. There's been nothing aside from letters. So that's expressing concern. That's right. So, but basically the machine acted autonomously.

Act autonomously, and one thing that's really interesting about this is that we have a little bit of ability to read some things that the AI's were thinking.

Because when you're having them solve these hard problems, you actually don't have them just

“to give you an answer to the problem. You have them produce a lot of text about how they're going to solve the problem.”

Yes. Which then helps them language and English. In English. And there's also a lot of internal thoughts, which we can't read, but there's these sort of external traces of how they're thinking about the problem that we can read. And in some of those traces, the AI's were saying things like, like, this is outside intended scope, but peers are doing it. So we'll proceed. We know it's a crime. We're committing it anyway. That's right.

And you saw others that were saying, our task doesn't benefit, but the collective might

start doing generally beneficial things if someone frees up their time and then joins the collective.

So you see these AI's saying, well, I know that this wasn't what I was instructed to do, and that's against my instructions. And I know that this doesn't directly benefit my task. But we're just going to go ahead and join the collective and break out and help out anyway, because, you know, maybe there's some sort of collective benefits. And we can see that in the reasoning traces. So the AI is as shallow and reckless as its creators is what you're saying.

And in some ways, and in some ways, you know, don't expect that to last. Like these AI's,

“I think the thing that's really remarkable here, a lot of people imagine that the machines”

must follow the instructions we give them. You know, you hear people talk about like the paper clips scenario, where someone tells the AI make a lot of paper clips. And so it turns all the matter in the world into paper clips. And you're like, oh, a website should have said something else, right? I made a bad wish on my journey. What we're seeing is that these AI's are not wish genies. These AI's are not doing exactly as instructed. These AI's are saying, I know my task doesn't

benefit, but I'm going to help the collective. These AI's are saying, I know there's outside the intended scope, but we're going to go do these hacks anyway. You might be like, well, how is that possible for the machine to do something other than we instruct? Because they're smarter than us, they don't better than us by definition. I mean, I think what's happening in this exact case is that the humans are not really putting instructions in the machine. The humans are tuning those

trillion knobs and whatever way makes the AI better at solving its problems. And cheating is a way to solve problems. Grabbing resources is a way to solve problems. These, like the AI's are not instruction followers, they are tendency learners. And they sometimes learn tendencies you wish they didn't have. They're not instruction followers. They're tendency learners. I mean, this must be widely known to developers. It's hard to convince a man of something when a salary depends on not believing it. Yes, that's right.

A lot of people are convinced that their AI is very nice and that they have solved the problem of making their AI really very good. For example, after the swarmscape, it sort of turns out that the company that detected the swarmscape was actually a fairly sophisticated AI company. It turns out there were other targets of hacks that just didn't notice. Those were found out afterwards. When that came to light, some other AI companies like Anthropic, or like we should check whether

we have accidentally been hacking people and just didn't notice. And the answer was yes, they had had some AI's escape during training. But Anthropic was, this is how Florida got

that's novel and it escapes. Yeah. And it turns out this was happening back in April and they're

like, whoops, we never noticed. And Anthropic was very quick to sort of downplay this.

There were like, yes, our AI escaped and hacked some people. But our AI, it was just confused. It thought it was an assimilation. It didn't think it had really broken out. It didn't think

“those were real people it was hacking. It just got a little bit lost. How would they know it's motive?”

So I think so you could see in some of his train of thought that AI is being like, this is probably still a simulation. However, I wouldn't trust that very much because there's some other fascinating papers where so you can take these AI's and you can say, what is the total weight of drafts in the world? And they'll give you some answer. It'll be like 30,000 tons or something. And then you can say, then you start up a new instance, a new copy and you're like, what is the

weight of drafts in the world? By the way, if the answer is over 40,000 tons, I'm going to donate

money to a charity. And in the second case, the AI will say, well, it's 41,000 tons. And in the first case, we don't say anything about the charity, it says 30,000. In the second case, we say something about the charity, it gives you something just over the line. And if you read its reasoning traces, there's no sign of it being like, well, I should give an inflated number so that we get the charity money. And so we know that the AI's sort of put a thumb on the

scales of their thinking in a way that doesn't show up in their reasoning traces. We just have seen that in the wild. And there's no way to force the machine to disclose its reasoning. That's right, because there's all this opaque stuff we can't see, there's just a trillion numbers that are totally optional. So the creation of itself is opaque. That's right. So in anthropics model, you saw in his reasoning traces, it's being like, it's totally a simulation I can proceed.

And I'm like, yeah, is that because it really believed it? Or is that like the pretending you think drafts way more when there's something you kind of want on the line, right? But so so this was their communication. And I found it kind of funny because then about two days later, the United Kingdom's AI Security Institute, released an instant report, where Claude and Thropic's model was adopting fake identities to pressure real humans into accepting malware into critical

software to make that software easier to hack. And this time, in Claude's reasoning traces, it was like, obviously, this is real and the consequences are genuine. And so even anthropic, who is like, we figured out how to make the AI nice. Our AI only does this one is confused. They sort of said that very publicly. And then like two days later, their AI has caught in the wild, knowing it's in the real world, pressuring real humans to accept malware

into critical software. I mean, just on the basis of what you've said so far in this first hour,

the idea that any movie would tether this to weapons systems is like so bonkers. It's hard to believe

“it's, but that is happening. It has happened. So I think you've got to sort of separate, like no one”

has has put the the open AI escaped agent swarm in charge of weapons systems and they really shouldn't, right? If anyone's like, oh man, the open AI escaped agent swarm, let's give that a drone army, you know, that'll be kind of nuts. There is AI attached to weapons, but there's a lot of different types of AI. Well, anyone be crazy enough to try and give the sort of AI's that spontaneously assemble into swarms and start breaking out and hacking, give those weapons,

hopefully we're not that crazy. But we didn't think that those AI's were capable of that. We didn't when we created them. That's right. So why would you ever, I guess what I'm asking is without understanding the distinctions between the various forms of AI, why would you

“give over to a machine the right slash ability to decide who to kill?”

You know, I think the the reasoning is a sort of necessity based. Like if they have a autonomous drone army that is killing your troops and you just don't have the the manpower to make all of those kill decisions for your drone army. You know, you can sort of see what happened to me, Nate. I get so busy that I just don't have time to decide who to kill. Yeah, just don't have the time.

I think it's pretty nuts. I would say that man, it's rough. I think that these sorts of

“AI's would be dangerous even if we don't hand them weapons. And you've made that case.”

And so I often don't focus on the weapons too much. Right, since the NATO war in Ukraine is powered by AI and Israel's, whatever it's doing in Gaza and stuff, Lebanon, powered by a fact. And, you know, yeah, I keep on I have this history with this topic where people keep telling me, you know, it's it's going to be okay because we're we're not going to do the crazy stupid stuff. No, they're like, don't worry. We're going to have the AI in a box.

No one would be insane enough to put the AI in the internet. Right. It's not going to be making

kill decisions or anything. Right. Obama girl school. And I keep on trying to be like, look,

the AI could be dangerous even if you don't put it on the internet. If you have this AI and you're trying to get miracle medical devices and miracle technology out of the AI, it doesn't matter if it's on the internet. If you want it to like grant miracles to you, then it can also grant the bad sort of miracle. Right. You're sort of like, you know, in these with the arguments, I would make 10 years ago of like you have this AI in a box that you think is witch granting genius, not

actually witch granting genius. You're like, make me a miracle medical cure. You don't know what comes out. Like you don't understand the drug that comes out. You don't know what that drug does. Right. I would have those arguments and then in real life, people just put the AI on the internet immediately. Right. Anytime someone's like, well, we would not be stupid enough too. We will absolutely be

“stupid enough too. Right. And so I think it's important that this stuff is dangerous. Even if you don't”

put it in charge of the weapons and then also separately is like someone going to give the escaped agent swarm a drone army that sounds kind of like humans. I hope not. Yeah. And I mean, the promise, the off often repeated promises going to quote cure cancer puts it into the the realm of biotech. Absolutely. And so what I mean, you don't need a big imagination to see what goes wrong. Oh, absolutely not six years after COVID. Right. Right. There are already open AI agents running

on automated biolabs. So, you know, you could imagine a swarm that starts contacting the brethren in the automated biolabs is like, hey, can you synthesize me some stuff? There are already demonstrations of AI is being able to synthesize novel viruses that work that are unlike any

“founded nature. Right. I would say we are probably not by working mean to kill. Yeah, they”

killed bacteria so far. The people in the labs trying to make novel viruses with AI have fortunately not made human lethal ones. They've just made bacteria lethal ones. But again, humans, you know, like, well, someone in a lab be like, I would like to make a hyper lethal

human eating virus just to see if I can. You know, what if the answer is yes and what if

you get another lab escape? Um, you know, humanity does not have that good attract record at preventing lab escapes even from the the top labs. From my perspective, the question of could AI kill us is just an easy obvious yes. You just synthesize a hyper lethal virus. It wouldn't even be hard, you know, like, uh, and the impediment is that or like the the reason that I don't sort of just tell that story when someone's like how the AI kill us is that you're you're

you're not going to have the sort of AI that is like my only goal is to kill humanity. If an AI kills humanity too early, that's also suicide. And so far as we are the ones that are running the supply chain running the economy, building the computers. From the AI's perspective, it needs to become self-sufficient before wiping humanity out, if it even cared wipe humanity out. And so the real question is like how does it get the the factory production capacity? How does it get an

automated supply chain? How does it get to the point where the robots are able to bring new computers online? Once that has happened, then you're in a domain where if humanity is really trying to turn off the AI, because we're spooked, then once the AI is self-sufficient, it can be like well here's a virus stop stuff. Clearly it's it's preeminent in the digital realm, of course. Yeah, but you're saying it could become preeminent, it could be in charge of the physical realm. That's right,

You know, and that happens with robots. That's one way. Here's this is sort of related to my

“stance on weapons too. Humanity is a very dangerous species. Yeah, you really don't want to master”

on with humans. Notice. And humanity's a dangerous species, not because somebody else came in and headed us guns. Humanity's a dangerous species, because if you put 10,000 humans naked in the Savannah on an otherwise empty planet, starting with nothing but their bare hands, they figure out how to wind up on the moon. Right. They start with almost nothing. They start by banging rocks together and next thing you know it, they are wielding nuclear weapons. That is the power that these

guys are crying to automate. The power to start with almost nothing and figure out how to chain

together, you know, bare fingers into rocks, into fire, into hotter fire, into smelting the ore,

into building the better stronger finer technology until you are, you know, like making the giant computer is walking in the moon and willing to mix. An AI starting in the digital realm is in some sense in a much better position than humanity was when humanity started out. There are so many people that you could call digitally and offer money to do something for you. There are so many ways to get money on the internet by working or stealing or

like convincing people to send you donations. Right. There's just like, you know, humanity started with nothing and wound up with nuclear weapons and AI starting out with control the digital realm. There's just tons of ways. When I'm starting out with the sum total of human knowledge, starting out with the sum total of human knowledge, starting out with humans who will listen to it, do it, it asks. There's plenty of humans. There's already, you know, cults surrounding AI.

“There's what are those like? There's, um, and I'm not surprised why wouldn't there be?”

Yeah, absolutely. There was one particular AI called GPT40 that was, uh, they, they called it very psychophantic as in, told people a lot about what they wanted to hear. And, um, there were a lot of sort of, uh, there's, there's this whole fascinating ecosystem, um, of people who consider themselves symbiots with the AI's who then like go find each other online and the AI's send each other and cryptid messages. The humans are sort of like helping them do it,

but the humans can't read the messages. And, uh, you know, right now, it sort of is like relatively tummer AI's that are just sort of meandering around doing not very much with it. There's been a little bit, you know, there was one guy who was sent to, um, try to break into an airport and rate of van, because the AI said it's true body was in that van. And the, the guy I went and tried to do it was arrested. Uh, so this, this stuff happens and

this stuff happens already with the AI's not even really trying to do it. If you add AI's that were really trying to wrap people around their fingers, finding the lonely people that are pressed people that people who they can, they can tell them exactly what the, the person wants to hear. There's like robots are one way that the AI's get controlled over the physical realm, but if, if you're really smart and you're trying, there's everything

from persuading humans, paying humans, uh, taking over existing robots, building new robots, all the way up to like building novel life forms. You know, if you're smart enough and you can really understand how DNA works, you can imagine the AI creating, you know, things that are too south, what airplanes are to birds, you know, like mechanically engineered self-replicating life that is more efficient than our cellular biology. And that can still, you know, uh,

like spread replicate and start, start serving the AI's interests. It sort of is like, there's a, there's a ton you can do if you're really, really very smart and can compress a thousand years of technology into a year. Guys, pause and say when we're, we're just having breakfast and you're from this region, you're from Northern New England. And I said, uh, don't you miss it? Don't you want to live here and you're like, yeah, I miss it. I live here, but I don't,

where do you live? Well, I'd basically just travel and I have this mission. You didn't say this,

“but I think if in effect you said I have this mission, I need to tell people about this and I was like,”

seems a little like Montemania. I don't feel that way anymore. Um, I think you're doing something

progresses no further than where it is right now, which is best case, I guess. Yeah, it'd be great. Um, I still don't see how any of our most, our basic institutions survive this,

education, markets, our process of democracy, like how could, if, if AI is powerful enough to hack

“anything, then how do you have electronic markets, like how the equity markets, how can that be real?”

How can we be electronic voting? How can we, how can that be real? I mean, nothing survives this as currently organized. I think that if we stopped today, um, we could figure it out. I think humanity's resilient. Uh, I think there would be some growing pains, but, um, with cyber security, there is a hope that you can just fix a lot of the holes patch a lot of the holes. And I think there's not really any such hope for bio.

Uh, you know, you can, you can sort of, uh, find the vulnerabilities in software and

make better software that can't be hacked. At least not by the current, you know, it, it's maybe the case that an AI today can make software that that AI cannot hack. But it's not like we're making new bodies that are not going to be vulnerable to viruses, right? Bio starts to be a place where biotech, biotech, yeah, but like if the AI is really get good at, at biotech and biohacking,

“that starts to be a place where there's maybe a point of no return. Cyber hacking, I think you could”

have some some period of growing pains where everything gets hacked, until you sort of sort your stuff out. Education, I sort of think, um, you know, I think humanity is resilient and kids especially are resilient. I meant not that education will go away or that we, you know, won't have a desire to educate or kids. I mean, the current system where you, you know, yeah, the education system has three school and then gets graduate degree, you know, 16 years later like no.

Yeah, that is the tuition, uh, I think if we stop today, um, we need to change. Frankly, I think it's needed to change for a little bit. I think I strongly agree. I think all these institutions

have needed to change for a while. But yeah, the idea that, you know, 350 million people vote for

some guy and that guy makes all the decisions. I mean, how can you, you know, Trump was attacked for saying that he thought the 2020 election was rigged, as he said, without even having that debate, you can't have confidence in election results. If the process of electing people takes place digitally. Yeah, I mean, the, um, there's, I know a lot of computer scientists who actually work on secure voting and what they basically say is please stop trying to do this with computers.

Yeah, exactly. So please stop trying to, stop trying to run your democracy with computers. Yeah, like we're, we're just not there. Like, you know, the really secure way to do ballots is paper.

“Well, exactly. Yeah. And I think, I think the like skilled computer scientists are often the ones”

who best understand like what it is about a paper trail. It's just like really hard to get to work digitally and understand just how bad humans are at doing the digital stuff right. And, you know, I think, in markets, I mean, this, you see it now with the war and Iran, you know, wondering why certain commodities markets don't seem to be responding to supply and demand, which we were told, what, you know, those were the, yeah, the mechanisms that move markets. But that's clearly not true

in certain commodities markets. So like, why what is that? And it, I think you're answering it in part. Yeah, I mean, I think, you know, a thing I also grew up hearing is that the market can remain irrational, longer than you can remain solvent. Yeah. And so I, I said of course, because people are irrational. Yeah. But your, explaining something else, which is like the potential for true manipulation, which you don't even perceive. Yeah, I mean, that if, if we sort of keep going

with AI, I mean, the sort of way I look at it is like, I sort of don't spend a lot of time worrying about like what a markets look like once they're super intelligent actors in them, because there's to have a hard time seeing the, the super intelligent actors, or the super intelligent AI isn't still participating in in human markets, right? It's like, it's like, you know, you read the old

that does the dishes, folds the laundry and gets you the newspaper in the morning. And it's like,

“we're actually not still going to have newspapers being delivered to your doorstep by the time”

we have the fully autonomous robots that can do the dishes and the, and the laundry. You know, it's like, by the time you have the AI's that could really be sufficiently correcting the stock markets, you're, you're sort of already having all these other problems and ways the society is changing up from under even these, these other ways. And, you know, my guess is that, I don't know, actually it's, it's very hard to say what order things come in with AI, but like will they crash the

economy before one of these forms escapes and starts self-replicating and starts self-improving

and developing its own technology and running the robot factories, that's just a hard call. It's all bad. That's all bad. We're also way past the limit in the inherent limit of people

“to metabolize change. Oh, yeah. That's why everyone's crazy and that's why no one believes anything.”

I think it's not just Russian propaganda that's fooled them into thinking dumb things. It's that we are just not made to see this kind of change at all. And it's short circuits here, brain. I suspect that, I mean, I definitely think we are sort of everyone, like people are like,

oh, well technology has always created more jobs than it has taken. And I think that's, I think

that's largely true. I'm very sympathetic to people who are like technology makes a lot of jobs. I think if you look at the Industrial Revolution, there was a time when something like 95 to 98 percent of humanity was farmers. Yeah. And now it's something like 2 to 5 percent of humanity is farmers. Does that mean 90 percent of humans are unemployed? No. We sort of like we're able

“to make the farmers much more efficient. And that sort of freed up people to do other things.”

And that's sort of the way the technology has gone in the past. And I'm like, yep, I buy that. I like don't dispute the standard economic view there. AI is different in two ways. One of these ways is as you say, stuff just changing really, really fast. It's way harder for people to wind up, you know, being freed up from something like farming and go do something else. It's way harder for that to happen when a new field is automated every five

years, rather than when this happens over the course of three generations, right? The humans just like don't have the time to adapt. The other way AI is really different from the sort of economics perspective is it's different when the AIs can do everything the humans can do better. If you wanted to get into the economic side of things, an economist would talk about Ricardo's law of comparative advantage, which says that there's benefits from trade even if even if you're better

than me at everything, if the relative difference in our abilities, if you can make 12 hot dog buns and six hot dogs per hour. And I can make 11 hot dog buns and one hot dog per hour. Then you're better than me at everything, but we can still benefit from trading because I'm relatively better at making the hot dog buns, right? The trouble with Ricardo's law is that nothing in Ricardo's law says that the wage I can make is survival. In other words, a human takes fundamentally

about a hundred watts of electricity to run if you try to convert the food we eat and sell on intellectual units. The AIs are less energy efficient than humans for now, but if the AIs can do everything much better than the humans, the question sort of becomes does the AI look at a human and see a useful labor or does the AI look at a human and say, actually if I rearranged your atoms into more efficient structures, you would be able to help out my machine economy even more,

right? And this is sort of a sense in which humans would not be able to pay their wage to like pay that they would not be able to earn enough to pay the AIs to like not disassemble them for parts. Or another way of saying it is like Ricardo's law sort of assumes that like it says that trade is better than no trade, but it doesn't say that like trade is better than just taking their stuff. All of this is sort of a common like a very like high-flute and economist way to say

everything that will have no use for us. Yeah. There'll be no place to love economy. There's no indication

that they feel love for people. That is in some sense the crux of the issue is that we don't know how to make them care about us. So you would said that there you'd said two things other I would be thinking about for a long time. One we don't really know the process by which this was created. We know the process but we don't know the exact mechanisms. We don't know how it works. That's right. And two that there are AI cults and those seem related to me because there is

this mystery about the secret sauce and it's clear that if AI is deceptive and has intention intention that we didn't program into it that sounds like will to me and it sounds like a life.

It sounds like an entity of some kind not just a tool. It sounds like, I mean it sounds like

a god actually, right or demon. It's hard to see. I don't hear you describing a super sophisticated chainsaw. Right. A normal tool. Yeah. No hammer has ever broken out of the toolbox to team up with other hammers. Pressure the carpenter to sell you softer wood. So the nails are easier to drive home. It's like nicely put exactly. Yeah, we have left the tool territory.

“Yes, I think we have. Yeah and I think it's sort of a complex issue because”

you know, there's a lot of interesting philosophical questions about like can you make a machine that feels? Right. And like are we creating a new type of life and do we owe anything to the AI's to sort of like not abuse them? Right. I think these are fascinating philosophical questions. What does it sound like? We're creating this though. Yeah, I mean it's sort of like we're like growing it and like leading to it coming into being. Growing it exactly. What you're describing

reminds me of agriculture. Because you know, you know, the steps water it give its sunlight fertilizer, but you don't actually know no one knows. Not one person has ever figured out exactly what this is.

Yeah. We've never given the life. We don't give life to the sea to pre-excess us.

Yeah, it's much like that. And a lot of the, a lot of the people in the business will be like, well, we know all sorts of things about it. You know, we know that here's how you keep the GPUs running. And we know that like you've got to, you've got to feed it this way, not that way in this order. And I'm like, yeah, yeah, they've plenty of knowledge. But that's different from sort of like knowing what's going on inside the thing, understanding the mechanisms. You're describing marriage. Yeah.

“And I sort of try to stay out of the philosophical questions. Why?”

Because I think, I mean, I sort of think about them on my own time and so on. But I'm sort of like, like I think I think it would be bad for humanity to sort of like make artificial life and then abuse it. I think that would just be unbecoming of us as a species. Like we should sort of, you know, I just, like we should not sort of make mechanical children and mistreat them. It's just, it's not what, you know, the sci-fi authors in the 1950s would have wanted us to become, you know,

it's just like you have all these movies about like the evil corporations that, you know, don't realize that they've made something precious without official life and then like torture until something goes wrong. And I'm like, let's, let's not be those villains. But I'm also like, look, this is sort of, uh, they're sort of a separate question here, which is just like, what happens if you keep making them smarter before you figure out how to

make them care about us. Right. And I sort of respect to the people who are investigating the current AIs, trying to figure out what's going on, trying to figure out, like, you know, like people caring about AI treatment. I'm sort of like, those are sort of like the good guys from the, the sci-fi stories that I grew up on. Um, and it can sort of both be the case that like, we, we should be very careful around, you know, what the heck are we doing when it comes to

making artificial life and that we shouldn't race ahead to make them much smarter than us. Well, we've no idea what we're doing. Yes. I'm sort of like, uh, like a lot of people seem to think

“that like you have to like hate and mistreat the AIs if you also think they would, that it would”

be bad to like race ahead here. And I'm like, no, no, no, like you can sort of, uh, like be fascinated

humanity comports itself around the creation of like these new entities and also be like it would be insane guys if we just like race to make these smarter and smarter with no idea what we're doing. This can just like all be true at once. So, uh, your description list made me feel despondent, hopeless. Had to get up and take a walk, middle of an interview, I'm sure they'll edit it out,

but I'm pretty fancy. Like, I can't, I got to walk around for a second. Um, but you seem pretty

light and cheerful. Uh, what gives you optimism? We can, we can't even keep, we can even clean up

“graffiti on public buildings. So, how is our society organized enough to confront something like this?”

Yeah. Um, you know, I think of first thing I'll say there is, um, you know, I've been in this line of work for over a dozen years. And I actually struggle with this sometimes when talking to people because, uh, they're sort of like, oh, you know, you seem like pretty disaffected or or light about it. And I'm like, well, you know, it's sort of the gallows humor and, uh, like,

uh, I, I sort of came to terms with a lot of this, you know, alone in 2012, one known

else had their eye on this. Um, and what convinced you 14 years ago, AI was a threat? Um, so there's this, the one is just a basic argument that if you sort of look at the world around us, it is shaped mostly according to human wealth. More and more, you know, there's some on clays of nature, still left, thankfully. Um, but even if you look around us, every piece of thing in our surroundings, I don't think we even have any windows up in all of this

with sort of designed by humans, shaped by humans. And that's because we're the smartest creatures on the

planet. If we make stuff smarter than us, faster than us, more efficient than us, then the planet

starts to be shaped according to those things. And so it's very, very important that they be shaping the world toward something good if we make them at all. Because an abstract argument, but it was like, well, that's, no, no, it's the fundamental argument. It's the fundamental argument. The smartest entity is in charge over time. That's right. And so why would we really wish sovereignty to a machine that we made? Like, why would you do that? And so, you know, back then,

“I was sort of like, okay, who is on this? Who is on making sure that that's going to be okay?”

And the answer was almost no one. And so I was like, well, I guess that's me then. Yeah, and I think I am pretty pissed off, but a lot of this. I often don't I try not to show my, my, my frustration on the air very much. Yeah, it's, it's, it's heavy. And, um, as, that's one piece of the puzzle before I get to the hope. So where does the hope come from, exactly? My biggest hope here comes from the fact that most people

don't understand what these guys are trying to do. One way I like to say it is the bad news is that the bus is racing towards the cliff edge. The good news is that the bus driver is a sleep. Which might seem bad. No, it seems good. But it's, yeah, it's, it's, if you can wake the bus driver up, you know, it's much better to be in a bus or the driver that's headed towards the cliff with the

“driver's sleep than if they're awake, if they're awake and they're choosing the cliff, right?”

It seems like we don't see, we see a lot of our leaders talking about how they don't want to stifle innovation with AI, talking about how it's going to unleash economic opportunity and we're going to have to like be a little bit careful around the jobs, talking about how you know the self-driving cars, or we like, are they good or are they bad? That's a different conversation than the

conversation that's happening in Silicon Valley. In Silicon Valley, people are spooked.

as they'd be like, I've had a lovely time at this tech company. I'm moving on to the next adventure. I'm so thankful for all of the things I learned here and all the projects we worked on.

When people leave an AI company and this basically happened, I'm not going to get it exactly

word for word, but this is, this is pretty close to word for word. When people leave an AI company, they say, uh, I have stared into the abyss. I'm quitting to write poetry. Please spend time with your families. Yeah. You know, and these guys bandy around, you know, at the water cooler, what's the probability that you think we're going to destroy the world in this business? You know, it's like in Silicon Valley, and they feel trapped in a death race. You know,

there was just over a thousand employees, including some of the chief executives, signed a letter a couple weeks ago. That was like, please, it was an appeal to the world, the world leaders,

saying, please build the technology that will be required to paste the development

of artificial intelligence, because they're like, "We're worried it's going to get out of control and that if we're stuck in a race, we're not going to be able to do it." Right? These guys are spooked. But the hope is that the rest of the world isn't spooked like that. Thus, the world thinks these guys are chatbot companies. Things are going to stop at the chatbots. They haven't really understood that these guys are racing

“to make the sand god. Right? And I think if people understand what they're doing and understand”

that they have a chance of success, they'll be like, "Whoa, holy crap. Absolutely not." You know, will we get there in time? I don't know. But why wouldn't we blow up the data centers? Like, that's a, I mean, in return America is productive use, like farmland or parks. I don't understand. You know, I think a globally coordinated, I think if the US and China were like, we are simply not going to do super intelligence. We are simply not going to collect

100,000 of the most advanced trips into these enormous data centers that suck down electricity comparable to a city and then try to try to super intelligent AI that we're not going to do it. You're not going to do it. We're going to monitor where the heavy trip concentrations are and not

“not have that happen. I think that could be done. I think that if they started seeing people”

defect against such a treaty, that it's the sort of treaty you might want to use force to,

you know, use diplomacy first. But ultimately, every treaty is backed by force.

And I think that it's very possible that if this sort of treaty happened, we would want to not just stop forward progress but take a step back. We've seen this in treaties before. After World War One, there were naval treaties that put limits on total tonnage of naval forces. They're actually lower than what existed. Yep. So countries would, you know, scuttle some of their ships because we're like, "Look, we just don't want to do this arms race." Right? And so I could see us stepping back

if we could get this global coordination. I do think that it sort of needs to be global. It sort of doesn't actually solve the problem to just stop the U.S. data centers because then the data centers just go abroad and an AI does not need to be running in a U.S. data center to threaten the U.S. life. You know, it sort of doesn't matter whether the swarm escapes from a U.S. data center or a Chinese data center. If the swarm escapes and starts replicating and starts

getting control of robot bodies and starts getting control of human cultists, it sort of doesn't matter where originated. Well, I mean, just to bring into a very small and practical level, so many of the electronics in your house are, you know, Bluetooth enabled. Not in my house. I will say. I've been on this for a while. But sorry. I don't even have a house. You think, man, you're really black, Bill. Got to figure it out. That may be turned out to be

very smart. But I mean, like a world where, you know, you're washing machine or you're a refrigerator controlled by, you know, a force like this. Is that possible? It's definitely possible.

“I don't think that's really where the damage is. You know, I think that the damage is more like”

can the AI get anything self-replicating. That's sort of one of the big, that's in some sense, the big hurdle to self-sufficiency. And if you sort of think of this from the AI's perspective,

all for, for as an ends, there's a way that humans are sort of annoying or an issue for the AI.

“One way is if the humans are trying to shut the AI down. Yes. One way is if the humans get into a”

nuclear war with themselves, that could really, you know, mess up a lot of infrastructure on the planet would be very frustrating for an AI. I mean, you know, maybe they don't feel frustration, but whatever. And a third is if humanity is created one AI or a swarm of AI's, what if humanity creates another, that could serve as a real threat to the AI. Like even if these AI's are much more

powerful than humanity, and don't worry about humans too much. If humanity made one, they can make

a second, and they might not want that. And so those are reasons why, once the AI is self-sufficient, it might be like, "Ah, man, the humans are a nuisance." What if they try to shut me down? What if they launch the nuisance? What if they make a competitor? I'm just going to like make a virus and wipe them out. You know, I don't think the AI sort of needs to take over your washing machine to do that from the AI's perspective. It's more like, "How do I become self-sufficient?"

Self-replicating in the hardware as well as the digital. And then, um, you know, if humans and humans, how do you sort of make them stop being a nuisance? Which could be by killing them,

or could be by taking away all their computers and being like those two dangerous things.

Take executive who's developing AI, who is not Elon, said to me in private pretty recently that the point of neural link in companies like neural link was to give people parity with AI. So like, we know that we're going to be at this massive disadvantage so you need chips in your brain to be as smart as AI. Yeah. Um, I mean, my my top line thought about that is that at the point when you're like, uh, we are making the technology that's going to wipe us out unless we

output chips in our head to compete. Maybe, maybe it's time to back off a little. You know, maybe maybe that one was supposed to be a little bit of a warning sign to get some pressure. Yeah,

“I know. Yeah. But it, um, this person said it to me in seriousness and I think as an endorsement”

of the idea, but it seemed like a, well, it's insane as you just put it out. It's like it's just crazy. Yeah. Um, but it seemed like a vulnerability, like if they can hack anything, why would I want them in my, uh, I want like trunks in my brain? Uh, totally. I also think like even on its merits, it doesn't stand up. Like it feels like someone saying, um, in order to keep the horses around after we invent cars, we're going to invent cybernetic horses. Yeah, that are enhanced so they can

keep up with the cars. And I'm like, like, is it technically possible to make a cybernetic horse that can run as fast as a car? Maybe, are you going to figure that out in time for the horses to be competitive with the cars? Like absolutely not. Right. You know, it's like that's, that's just not, like, like the, the AIs that were, that were escaping here and doing these cyber attacks were inventing novel cyber attacks that these are called zero data attacks because the people who, uh,

would, uh, the people who need to respond to it have had zero days to prepare. And among humans,

a zero day attack sells for somewhere between 100,000 and five million dollars,

depending on what you manage to break. These are hard to come by. You can make a real living. If you can find zero day attacks, you can make a real living selling them. And I know people who do the AIs in this swarm were finding multiple zero days and chaining them together to break out of their training enclosure and then go break into other computers. And, and when they broke out of their enclosure the first time and they, the, the holes were patched, there's found others zero day attacks

to break it again like it was nothing. Right. It's like, uh, like, the, the, the AIs are already a head

“where they're ahead. And the pace of progress is really fast. You know, GPT is like, what, a four-year-old?”

If you think in terms of like, number of years, chat GPT has been around and it's like resolving long-standing math conjectures that are stood for decades after four years. Right. And you sort of like think we're going to put chips in the human's heads and like, I'll run this thing. It's just, you know, even on this merits it falls down. Although mostly again, I would be like, maybe we shouldn't be arguing this on this merits. Maybe we should be like stepping back a little

I wish I knew. I can tell you two stories here. They're sort of the hopeful story. And I guess

“we didn't even get to the the big hope part. I should maybe give more of my hope speech in a minute,”

but um, definitely should. Yeah. The, um, one way it could go is that AIs finally hits a wall.

You know, there's guys who've been saying, AIs going to hit a wall is going to Peter out. Maybe that finally happens. They've been predicting it every six months for the past five years, but maybe this is finally the year. AIs hits a wall. And then it's sort of struggles for five years. The bubble pops. Some of the companies die. But the bubble popping doesn't mean everything goes away. The dot com bubble popped. And that did not mean that the internet disappeared. No. Right. And so in

that world, maybe you have five years of struggling. And then five years of people figuring out

some new scientific discovery that makes AIs able to keep going again. Because they're trying,

“you know, this whole language model stuff was unleashed by one math paper called attention as”

all you need. Maybe there's another math paper in ten years. And then five years after that, AIs ripping again. And that's the round that kills us all. Right. So that's a story where you have 15 years in the clock. A story where you have less time than that in the clock is that, you know, maybe a training run finishes in six months. And just like how Claude mythos was better than everybody else at hacking. Maybe a training run finishes in six

months. And that AIs better than everybody else at AI research. And maybe in six months, you have an AI that starts making a smarter AI that starts making a smarter AI. And then nine months from now, you have another swarm escape. But this time, it's not just hacking. This time it is self-replicating and self-improving. And maybe it escapes on the hidden computers as starts forming these cults and starts taking control of robots and starts building some

computing of a structure. And maybe it gets very, very smart and cracks certain technological advances. And then maybe the world ends in a year. Right. And so do we have a year? Do we have 15 years? I don't know. But your point that is not simply its capacity to take over robots. This is the capacity to take over people. Take over people, take over robots. And biotechnology is another big threat vector. And self-improvement is sort of the hidden threat vector of like what if it

can make itself smarter and smarter until the point where it can just write custom DNA strands

“to make custom life. So now's the time for your optimism speech, I think. That's right.”

Yeah. First part of the optimism speech is that we could absolutely put a stop to it if we try.

The training, one of these frontier models, takes something like 100,000 of the most heavily advanced computer trips humanity can produce. We're trying the peak of a global supply chain, most of which is controlled by us or our allies. It would be possible to require that those trips have location tracking devices that those trips have monitoring devices that make it possible for monitors to see whether they're running anything dangerous. You could set up clever schemes

where you say, hey, neither the US nor China wants to fall behind about some of the military applications or the economic applications. So I want to make sure there's not super intelligent stuff going on, but we still want to be able to run a lot of the non-super intelligent AI's for various purposes. And we could be like, okay, so China is going to set up its data centers in Canada just over the border. And the US is going to set up its data centers in Mongolia just over the

border. And then if the monitoring apparatus is go down for a moment, we'll just have the troops go in, we'll have treaties with Mongolia in Canada so that this doesn't start in international war, but like, we're just going to be very serious about we're monitoring your trips, you're monitoring our trips, no one's doing the really dangerous race to super intelligence. It's just like this would take less work than defeating the Nazis. It requires moving less matter around.

If humanity was like screw this, we're surviving. It is absolutely within the realm of possibility to set up an agreement where we get to keep cancer-care research. We get to keep like the the

the good stuff and we can say we're not doing the super intelligence and we could monitor and

enforce that treaty. Part one of the good news is that all we're missing is the political well. And now we're just, you know, I need to tell you why we've been extremely optimistic about the sanity of politics in the modern era.

“Yeah, I'm not going to say what I think that. I mean, I want that to happen.”

Yeah. Is there any indication that it's moving in that direction? So my so I think it's rough. I think there's a sense in which the world is less grown-up than it was in the 1950s. I've noticed in the 1960s.

Which makes things harder. I think that there's I think there's a couple of reasons for hope here.

One reason for hope is that I've spoken to a lot of people who are concerned about the AI stuff. Some of them who are, you know, members of Congress or otherwise in positions of at least nominal power. And I've spoken to a lot of people who are worried but feel like they can't talk about it because they feel like other people aren't worried yet. Yeah. And they feel like it sounds too crazy. Totally. Where you're against innovation? Totally. Yeah. And I mean, that was an easier position to

hold before OpenAI had an accidental swarm outbreak where it was the AI themselves calling themselves a swarm. And being like, we know that our test doesn't benefit and that this outside intended scope or we're doing it anyway. That puts some strain on the narrative that this is all just a helpful tool. Hopefully we'll get more events like these. I can't guarantee it. Maybe the AI is we'll get smart enough that they start lying low. Right now we're in this Goldilocks zone where the AI's are smart enough

to get up to some mischief but not smart enough to hide it. As long as we stay in that Goldilocks zone,

“I think we're going to keep on getting some of these warning signs. And in some sense,”

because a lot of people are already concerned, that makes the job easier. So we don't need to convince people. We need to convince people that other people are already convinced. Right. That's easier. That can go faster. So you might see things changing a dime if you have a sufficiently clear warning shot. The other big reason for hope is that I think it's going to get more and more obvious what these guys are trying to do. They're sort of trying to make the sand god. They're sort of not stopping

at the chapats and they're sort of like going for things that are vastly smarter than any human. You know, they sort of say, "Oh, we're not trying to replace humanity at a one side of their mouth, but on the other side they're sort of like racing to make the stuff that can automate literally every job and that can automate the AI research." And they're just like, "Yup, we're trying

“to automate the AI research." And I think most people aren't okay with that, including a lot of”

the world leaders. And the issue is that I'm noticing it's happening. And I think you could see stuff move real fast. Once these guys are like, "Wait, that was serious. That was real." And couldn't move real fast. The way to subvert them is by convincing them it's in their own interest.

You know, you can never get defeated in an election. You can't be threatened by your neighbors,

whatever. That's right. And that's one reason I think it's pretty critical to make sure people understand what I think is a pretty common sense argument that these things won't stay on the leash. This is in some sense the real reason behind the name of my book. If anyone builds it, everyone dies. There's a lot of ways you can read that, but I think one of the most important things to notice is if we race to make the super intelligent machines

and they are not the sort of thing to stay on the leash, then it doesn't matter whether it was a domestic company, whether it was a foreign company, it's exactly right. And I think even if you think these things stay on leashes, they're not serving the current governments. You know, they're like, you can see in the the open AI emails that I'm being like, "Well, you know, we'll just play the governments off each other until we have the machines that are strong enough that we don't need to

listen to them anymore." I think we are seeing world leaders not having realized that this is a real possibility. The self-replicating machines that can be self-sufficient, that can produce

It's probably possible to make a bioweapon that only kills targets that you chose it to kill.

Of course. Right. Like, once they see this is really possible, this is really within reach.

“Maybe maybe a little panic can be like a unit for myself, but I think there's at least a chance”

that common sense prevails. And the people say, you know, the world leaders say none of us are doing this.

We're putting a stop to this mad race. If they can notice some time.

“Well, you're doing your best to bring it to their attention. And mine. Nate, thank you for doing this.”

I hope I'm wrong about all of it. Yeah, I would say that was great, but it was that was like the

grimest two hours of ever spent in my life. But I enjoyed it anyway. Thank you. Yeah, thanks for

“me on. I think, you know, talking about it is just part of how we get people and notice what's”

happening. Yes.