Siliconsciousness: The Scary Geopolitics of AI

Welcome to Siliconuous News.

The DSR network podcast focusing

“on the artificial intelligence revolution, politics, and policy.”

Hello and welcome to Siliconuous News. I am David Roskuff. And this week, as every week for the past few years, we're going to talk about AI. And we're going to do it from a slightly different perspective

this week. Talking about AI in the context of geopolitics and national policy. And we're going to do so with two of our friends who you may meet, it may have met, and some of our other podcasts,

and certainly you've seen the minority of other settings. One, Graham Allison is the Douglas Dylan Professor of Government at Harvard University, where he is taught for five decades. Allison is a leading analyst of national security

with special interest in nuclear weapons, Russia, China, and decision-making. Hi, Graham, how are you today? Great, how are you? Excellent.

And Eva Dalder served as U.S. ambassador in NATO under President Barack Obama, and on the U.S. National Security Council under President Bill Clinton. And he is the founder and CEO of IHDGO Political Advisory. How are you doing today?

“And what appears to be a kind of cloudy day in Chicago?”

Great day, great day here in Chicago, David, I'm doing well, just if you're flying in or out of O or D today, Chicago, we've got a ground stop going on. So, well, thanks for that advice. We often start our podcast here with weather report.

As both of you guys know, we have been deeply immersed in AI for some time now. We have a couple of podcasts. I don't be running events on it. Something I've actually been involved with for several decades,

actually, and I was really heartened to see that both of you had devoted some time to writing articles about some of the bigger picture issues associated with AI associated with the geopolitics of AI, but also with the challenges facing policymakers.

So what I thought I'd like to do is to turn to each of you, and if you bear with us, just do a two minute synopsis of your article, and then we'll get into a discussion about it because there are some similarities in the points you both raised. Let me start with you, Graham.

You're an excellent article in foreign policy, and tell us a little bit about what your main point was. Well, thank you very much, and it's great to be talking to you

always well about the topic.

You've been interested in the questions for a long time, as I have, and my article mainly poses four questions, and I'm confident we should be asking. I'm less confident about my answers, but let me frame it quickly.

“So first, I think most Americans have an appreciated”

the extent to which the US economy today is a highly leveraged hedge fund bet on an AI miracle. Now, I believe in miracles, and I would hope that this would be a good product to be won, but I think the extent to which AI is the driver today of American economic growth,

American economic investments, and the great increases in the stock market. So more than half of the growth is driven by AI. More than half of the investments is best shared, AI. 85% of the gains in the S&P from AI-related companies. So this is a huge boom, and to some of us,

it could even look a little bit like a bubble.

And the questions about this relate first to our economy,

if the bubble or boom, should be delayed or even deflate. And secondly, geopolitically. So again, first question, is it prudent? If you were investor, would you put this in the eggs in one basket? Obviously not.

Secondly, how could this go wrong? I say, ask your chats at GDP or your favorite bot, and they can unless it doesn't. And the one that I worry about as some version of an AI Chernobyl, or an AI 9/11, everyone for forecasting.

But it could be a rogue AI that does something like what we saw last week or next week. Thirdly, what would the economic consequences of the bubble deflating or bursting look like? And so look at 2008 with the financial crisis that was caused by the real estate excesses,

so we're talking about 1929, so a huge, huge collapse of the American economy, if this should happen, and a recession, and a minimum.

“And then finally, geopolitically, what's this about?”

Because if you look at China, the only other series rival in this space, they made a very different choice. So one to ten in terms of investment in compute, one to ten in terms of investments in Davis winners,

costs basically the Chinese models, sort of essentially open models,

given away for free, essentially, and open in terms of their recipe, so that you can download them and make your own adjustments to them. So just last week, after a week before last afternoon, moonshot came up with their latest model.

They are often in 99% reduction in the price of compute. So now you have about two thirds of the work of AI is now being done by Chinese models that people can go to efficiently for free. So that's going to yield into the profit margins,

that the companies that are raising money, especially the front tier companies, are facing, and how long that's going to be sustainable. So I would just say they're more questions than answers, and I also like very much Evo's questions,

which is in the background, there's how much damage conceivably could be done in this space. Everybody agrees it's conceivable of possible for a AI model to enable otherwise less capable people to create with a genius of the size of a virus that

could look like COVID or worse, that could kill a lot of people. So that certainly enabling the question is how much more likely is it with these models? And then what could we do about it? So I would say those are right questions as well to be asking.

And I think we see this rush that's caused mainly by companies chasing what they think is the biggest part of goal at the end of a rainbow ever.

So basically, company leaders and their investors,

and then financial markets who've jumped in, and then it comes to be almost like a craze,

“that's what happens in general in bubbles.”

And so if one of the ways in which this could go away would happen, which I don't know what the odds are, but I would say the consequences for Americans first are economy, but secondly, and more importantly, I'm concerned our geopolitics would be huge.

So sorry, that's a long version, but a lot of questions and fewer answers. Well, you know, Eva Graham just summarized your article. So you don't really have anything to say here. I mean, you didn't quite summarize it, but I mean,

first of all, I thought Graham's article was great.

And asking the right questions is part of it. And I think we are actually seeding the beginning of a more mainstream debate about, okay, everybody is spending a lot of money. And everybody is running around saying this is going to change the world.

I mean, Elon Musk said that in 10 years people won't need, not only don't need to work, they won't need money, 'cause there will be so much abundance. That's great, a lot of them we all, but I was, you know,

“I think increasingly people are starting to ask questions.”

And so I asked a question fundamentally, Graham's second question, what compulsively go wrong. And I'm struck that the people who know the technology, the best. So those who are driving actually the built in Google and AI,

even in Open AI, even Elon Musk, are also the ones who are the most scared of the technology. You know, Dario Amade has called the moment at which we move to recursive self-improvement. That is an AI that can improve itself faster and better than a human can.

It is like the entry of the nuclear age. Well, I remember, like Graham spent most of my life thinking about nuclear weapons. And I remember what happened when the nuclear age started. A hundred and and sixty thousand people were killed in Hiroshima and another 80,000 in Nagasaki. And so if that's the level of destruction that we are potentially facing,

I'm asking in the question, which is the title of the article I did in on my sub-stack America or abroad, we don't let companies own nuclear weapons.

And part of the answer is, well, AI can do a lot of good.

And there's no doubt about it. I have no doubt that the chance that cancer, cures are going to be available in the next five years because of AI is much, much higher. If not almost certain, then it was without this technology. But I also have no doubt that the danger that AI can pose.

A danger that Jacob Zimmerman, the field metal winner in Canada, field metal, of course being a Nobel Prize for math, he wrote an article with a colleague a year ago talking about the dangers of homicide.

If homicide is not homicide, it means killing everybody.

And he's so worried about it that he left mathematics to become a to work on safety and open AI. So the people who know this stuff are scared beyond belief. And if that doesn't make you scared, I am. And so the fundamental question is, okay, why are we letting a bunch of companies

and a bunch of individuals in those companies? And a bunch of venture capitalists drive this technology without any regulation,

“without anybody thinking through how do you put guardrails in?”

And the guardrails are starting to come, but it's usually come to late.

This feels like social media where we let it all loose. And it's kind of interesting that Mark Zuckerberg is now leading the effort to put an AI agent in everybody's and at least the political and public effort if not the economic effort. Well, that'd be great. Let's have everyone have the capacity to make a bio weapon that kills, I don't know, people over 60

or people with a certain skin pigmentation or people who have a certain disease. Because once you think about how you can target AI to particular a genetic phenomena, which is great for healing and curing disease, it happens to be also pretty great for killing people or naming people or making them sick. And so I just think, I'm not a Bernie Sanders fan, but Bernie Sanders has just said,

why don't we just stop?

“I mean, why don't we just actually before we get to the point where we are stop?”

And the one way I know how to stop is a very traditional way, which is the government's own to it. We own nuclear weapons. We don't allow Boeing to own them. We have national labs that develop nuclear weapons. We used to have national labs that develop chemical and biological weapons.

We didn't allow private companies to do that. And we have very strict regulations. We regulate household cleaning products better than AI. That's not, so that was really the fundamental purpose of the point of my article. This podcast is underwritten in part by the U.S. Embassy of the United Arab Emirates.

It's that editorial content is completely independent and the views expressed are exclusively those of participating experts. It is presented live without editing. For further information about the UAE's efforts in the areas of artificial intelligence and technology, go to the website of the embassy at www.ue-emBC.org and search for UAE-US tech cooperation.

We thank them for their support. We thank everybody who is supporting this podcast for their support.

“And we look forward to it developing and growing over time because the issue is so important.”

So, Graham, you know, one of the problems that exists here is that in order to make good policy, you need policymakers who understand what's going on. And we don't, we don't have that. We don't even have anything close to that. And in fact, we've got a kind of perfect storm of conditions that can lead to bad policy. We've got sweeping ignorance across the policy community about this kind of a thing.

We've got regulatory capture going on where the richest biggest companies, the ones benefiting from this bubble, are in Washington spending tons of money taking people to dinner, hiring people.

"Hey, don't regulate stuff or let us be free to do what we are doing and give us benefits

“for our investments and so forth." And we've also got, you know, what I would call sort of early”

stage problems of any major technological development. We don't even have very good vocabulary to go and refer to it. The word AI or the term AI is utterly meaningless. And when you're talking about, you know, something, you know, a G.I. that has the ability to kill everybody and saying in the same breath that we're talking about, some AI assists that can help you from driving your car into a tree. You know, those aren't the same things. And, you know, then on top of all of that,

you've got people using the wrong analogies from the past. This is a very different thing. It's not new. I'm not criticizing the fact that you guys have brought that up, but it's not nuclear weapons. And so we've seen a lot of bad policy. You know, I mean, during the Biden administration, some people who are friends of ours who are well-intentioned said, "Oh, well, AI is the game. We have to keep China from being in the game." So let's do this

small yard high-fence thing and we'll stay at them. And everybody I knew in AI at the time said, that's not going to work. And it didn't work. It was a ridiculous idea. And so, you know, the question is, how do you get the policy community up to speed on a series of major sweeping

“developments that are happening faster than any sweeping developments have ever happened before?”

This is happening faster than the speed of government. Certainly faster than the speed of government. Now, I think you capture a reality about it that I've now tracked this for, I don't know, seven eight years trying to understand just a little teeny bit about it. And one of the things that struck me is that the people who work at the frontier repeatedly say that they're surprised that things happen faster than they had expected. So I have three or four people who are

work at the frontier that, you know, my coordinates that I just try to listen to them and then triangulate to see what's inside can make of it. But I think the proposition that this is moving at a speed that began the technologies that we've seen recently is certainly correct. And that people working at the frontier keep being surprised that the models can do more and different things that they expected. People were genuinely surprised by the models that broke out of their

sandboxes so-called or their frameworks to go rogue and go about attacking some other entity in a way that nobody had proposed to them. But if you then say to them, well, maybe we should just all have a pause or slow down. They say, yeah, but we can't slow down because the people we're running against are running as fast as they can. And if they get to this superintelligence,

so-called, or AGI or RSI, it's going to be smarter than God and whoever gets their first is going

to win. And the Chinese are running as fast as they can. So if you won't let me run against my American competitors, I at least have to run against them. So I'll go and say, well, okay, let me think if it's correct, that if there were a superintelligence that had some sustainable advantage, would you prefer it to be American than Chinese? Of course, I would. Well, then maybe we should be talking to the Chinese about this. Well, I've been in favor of this for a long

“time. I think they would be prepared to talk about this. It'd be very difficult. But there's”

society's more concerned about control, their government, you know, than our society. On the other hand, you have the paradox that the Chinese models are mostly open models. And so normal people can go to deep sea or to moonshot, download their model, change it around and play with it as they want, make it their own bot. So with there's a risk of the models that entropics our open AI are creating of if there's a risk that it could enable people to create a bio-weapon

that kills a million people. It had the risk of an open model doing this have to be larger.

You would think, well, then that should give the Chinese government more incentive to be more concerned if we were engaged in that. So I as I say, I find myself stumbling from one question

because it's so, such so fast moving and it's so difficult to understand, Evo's point is exactly right that the policy community has been way, way, way behind all of this and not able to keep up and David, your point is exactly right that to imagine a government doing this in the nuclear space since governments invented nuclear weapons. And since this all happened inside of a urnid, you know, a fenced facility, that was one thing. In this case, this is now people everywhere

all over the world doing things with their own models. And we can say time out for a billion

“people. I don't think so. So I think that's again just the other, just adding to the worries”

rather than it, the solution. David, if I can, I think I don't think this is a question of having smartest people to be in the parliament, so they can figure out how to deal with it. I think they're beyond that. I think what we we're facing both economic and geopolitical as well as overarching technological drivers that are moving this technology forward in a way that makes it virtually impossible to capture it in a policy environment. I think that's the sort of the problem

that we have. And it isn't if we had just smarter people in government or not this guy who with the orange hair running it, but it would all be fine. I don't believe that the case for example, there's no just as one example. Xi Jinping is pretty much in control of his government. It took the Biden administration 12 months, 12 senior solitary months to get Xi Jinping and its system to agree to this proposition. AI should not be part of the command and control system of nuclear

way. Pretty obvious, you know, policy conclusion, but it took a year from 2022 until November

of 2023 to finally get it into the communicate. I'm not sure. I'm not sure. People should be

either. But let's go on. Well, that's it. I certainly, at least certain people. At least we can

“make some distinctions there. But just just I think the smartest people looking at this are expecting”

a disaster to happen within the next six to nine months. And so you're really looking at something like six to nine months pre-COVID, six to nine months of some disastrous, you know, what work, what were one to think about it? That's the analogy we find 9/11 although of course fundamentally different kind of disaster. And government and people just do not have the imagination necessary to take steps

that would prevent that from happening until it happens. And so the regulatory and policy framework

will come if by those who still are around after disaster happens. That's kind of been my conclusion and I think it will be easier if government owns this shit stuff technology than if we allow

“private companies to at least get the economic piece out of it. But that's that's it.”

I see you and Trump are both going in the direction of communism and state control. No, Trump wants to own it. He just wants the money. He doesn't want the ownership. Well, yeah, that's true. Although one of the questions of ownership is why stewardship and if you don't understand what you're stewarding, it's a problem. But it does lead us in a direction because we do have a lot of solutions, solutions, and quotes out there. And you've got people saying,

"Well, let's just stop. Let's not do this. No data centers. No more AI." Or let's just leave it to the AI guys. You know, the government shouldn't play a role on this. It's too ignorant. Here's one thing that seems to me absolutely clear, though, Graham. And it's not a popular idea for the past few years in the United States. Back in the day when I was in the U.S. government, we used to talk about transnational threats. And if you'd say, "Well, there's epidemics,

and there's migration, and there's terrorism and so forth, and you needed to cooperate internationally to deal with these." Now, that sounds like globalization. People are uncomfortable with it. If there was ever a transnational threat, this is it. There is no way for one country. I mean, we can ask a lot of questions, but I think we can all agree. There's no way for one

but you know, it has been a lot of time in Europe. They're going in one direction.

Small countries, Singapore, the UAE, they're going in another direction. The global south has a completely different set of issues. If we do not create as fast as humanly possible and effective multilateral mechanism to identify risks and identify goals and work together to establish international standards and international guardrails, it doesn't matter what individual countries do. And so I'm just wondering whether you agree

with that premise. If you're enjoying this episode and want to get even more out of the DSR

“network, you need to check out our sub-stack. The DSR network sub-stack is the absolute best way”

to follow deep state radio, the DSR daily, where it's matter, need to know sila consciousness, AI,

energy, and climate, and the daily blast. Sign up to get notified every time a new episode drops and stay up to date on the latest news and expert analysis. Right now, we're offering 20% off of a DSR network membership. Payed members get an ad free listening experience, select episodes two days early, 50% off of David's need to know sub-stack and more. To get your 20% off, go to DSR network.substack.com/August26. That's DSR network.substack.com/August26.

Well, I think yes, I think it's a, and I think that remembering the category of transnational threats is a good, a good way to think about it. And it's no question that if today, a superintelligence emerges somewhere and becomes rogue, it's not going to be particularly interested in whether you're an American or Chinese or a Canadian or a, you know, from Sudan, it's just another, another, you know, item in that, in that agent's subworld. So, initially,

“and I think I'm still maybe stuck there, since the US and China are the drivers of this”

at the frontier, that's is that this should naturally be a place where there should have a conversation about shared interests that I think to going back to Evo's trying to learn from the nuclear arena. Early, after the US had nuclear weapons, we had a monopoly for a while, but within three or four years, the Soviet Union acquired their own nuclear weapons, there were then many conversations that were developed both between the two governments had between experts,

that in which the two governments were deadly adversaries, they needed trusted either a word, the other would say, and nonetheless, through conversations, end up discovering islands of shared interests, in which they could, again, trust about verify, but take actions jointly. So, why in the world are the only nine nuclear weapons states today? That's one of the most incredible things of edge of, this is an 80-year-old technology. So, why only nine states have nuclear weapons? So,

the answer is pretty early on, the two nuclear weapons states decided that monopolies best,

but do monopolies better than polarity. So, they began, what became ultimately the non-proliferation regime, and that system has basically created a set of constraints that a technology that's so old that North Korea has mastered it, well, if North Korea can do this, anybody can do this, anybody can. So, why not the others? Well, it's hard to believe, but a system was built out, even among parties, or between parties that didn't trust each other, and ultimately,

it didn't go step by step. So, I would say that's at least a hopeful sign that human beings are capable of this. Now, no, that the AI piece is moving so fast that, again, every day we fall further behind. And I'm hopeful that, actually, the Trump, she talks that will happen in Washington on September 24th, where Basit and Ha, his counterpart, have been trying to develop at least some conversation about this, because after mythos, this, you know, really edgy model was applied

“to financial systems. People looked and said, "Yikes." What would be the consequences of this?”

That got the sense of attention, because he does about financial markets and their vulnerabilities,

bit of constraints, but I would say that's a start. Now, how you go beyond that to the bigger picture

that you mentioned, I think. I'm afraid probably evo's right, that in the American story,

“usually takes a disaster to finally wake up, and that's why something like AI Chernobyl,”

or AI 9/11, that then leads to a backlash, is one of the, my list of, how could this go wrong? I would say that's pretty high in the list, you know? Well, you know, I like to finish by asking you, you know, the same question. One of the things that I bring up sometimes in foreign policy discussions, because I think it's a useful point here, is I was struck by Martin and Dick's book

about Henry Kissinger flying around the Middle East, and how Henry, you know, was not flying

around the Middle East trying to achieve Middle East peace, which was big and hugely complicated. He would fly around trying to solve the one, whatever the one thing he could solve for us. And so, you know, he wasn't trying to do everything. He was trying to do one thing. And what we look at this, and we boil it down, and we say what's really, really important. It's probably issues of security. It's probably setting up guardrails on issues of security, and his Graham says it probably involves

before we get to big multilateral things, getting the U.S. in the Chinese to agree. And so

all of a sudden, well, oh, so let's, let's, I mean, all these things exist. But how about if we

get the U.S. in the Chinese to sit down and focus on security and use that then as the basis for what we do multilaterally and so forth? Is there a way to to boil down to get to something that's actually doable here, Eva? Well, in theory, yes. And I think Graham's analogy and the one that I use in my piece on the nuclear age is pretty darn good, which is to say it turns out that the incentive to the Soviet Union and the United States had to prevent others from acquiring the

technology that they had was sufficient for them to start thinking about cooperation. The cooperation actually doesn't really start on the bilateral side between their systems until after the Cuban missile crisis, but the collaboration on how do you regulate and minimize the acquisition of nuclear weapons while at the same time allowing the dispersal of the civilian nuclear technology. And

“and that's why the analogy is is at is what what concerned them throughout the 1950s. He had Eisenhower's”

Adams for peace. He had the atomic energy agency that was set up the IAA. We then got it all set of safeguards, and we set up the nucleus of a non-polliferation system because the two major powers decided that the control of this technology while they owned it, the control of that technology should be such that the benefits be shared widely, but the dangers be controlled. And seems to me, AI is a pretty perfect example of having extraordinary benefits for all of humanity and

those benefits should be shared widely. And we can then talk about which different model you want to use, whether it's the Chinese model or whatever. But at the same time, we can agree that the dangers are such that you need to control the technology and safeguard it in a way that allows for the proliferation of the civilian but not of the military or the security side. So the non-security versus security side.

“So I think we have the model. I don't think, frankly, we have the leadership on either side,”

unfortunately, that has both the in our case, the frankly, the historical and practical knowledge, nor is there anyone in high levels of government to think about it in those terms. These are not the eyes and hours in the action sense and even the John Foster dollesses of the world. The same is true when she who seems to be so absolutely concerned with maintaining his exclusive and dominant power base that he doesn't trust anyone in his own system and puts them

in jail or worse. It's hard to see that these two leaders, Trump and she are going to be able to do what frankly, cruise-chuff and Eisenhower and then cruise-chuff and Kissinger and Kennedy, we're able to do and give them the people who work for them. But I think there are plenty of smart people who could help them and there certainly is an opportunity for other parties around the

and others who can start thinking about helping to form a framework that, you know, the

“non-proliferation regime that pushed, well, does it, Graham, you probably remember it by the Irish.”

The Irish were a huge player on the non-proliferation disease, the Nordics

where huge players in order to push the intellectual firm it. So, yeah, we should encourage

this kind of discussion while being aware that, you know, there are limitations. I mean,

“think about climate change, we know, we know all that and we have even an international treaty”

and somehow in part because of this new technology, we are, you know, building gas plants which, as the New York Times just reported, in Texas, a gas energy plant that will have larger emissions than any other single power plant in the United States. That's pretty astonishing. Thank

“you, Amazon. Yeah, yeah, thank you, Amazon. It's hard to do, it's hard to do all this when”

the government doesn't believe in science, but, you know, we'll get to it during the first couple

years of the Osoff administration. In any event, I am very grateful that we could start having a kind of discussion like this with the two of you who are both brilliant and thoughtful and I encourage people to go and read your article, Graham's at Foreign Policy and Evo's at his sub-stack, which is an excellent sub-stack to which to subscribe and we will continue having this discussion and with where lucky we'll be able to continue it by having Graham and Evo back again. But for now, thank you

very much and please join us on all the other shows we're doing all the time and I'd still a consciousness each and every week. Thanks everybody, bye-bye. This was Siliconjustness, a production of the DSR Network.