[MUSIC PLAYING]
Welcome to Silicantiousness.
The DSR network podcast focusing on the artificial intelligence revolution, politics, and policy. [MUSIC PLAYING] Hello, and welcome to Silicantiousness. I'm David Rothkopf, and this week, as every week,
we'll talk about some aspects of the AI revolution that we think are worthy of a deep dive. Today, we're very fortunate to have with Scott Singer. He's a fellow and co-director of the China AI Initiative
at the Carnegie Endowment of For International Peace, my alma mater, where his work explores the geopolitics, governance, and safety of advanced AI systems. How you doing today, Scott?
Doing great. How you doing, David? Good. Well, I mean, some are coming to an end. I'm not so happy about that.
But one of the things that the fall will bring, at least theoretically, is another meeting between the president of the United States and Xi Jinping. And I saw you wrote in anticipation of the last meeting that they really ought
to put some AI stuff on the agenda. And this does seem to be one of those opportunities of where whatever you may think of the Trump administration, there is a chance for them to make a lasting impression on an area of real importance.
Didn't really come out of the last meeting. And now we're getting these kind of mixed kind of weird signals out of the Trump administration, because on the one hand, you've got apparently a renewed push to go around the world and make countries
two sides between the United States and China. On the other hand, you've got some important people close to the president saying, hey, let us do more with China. And you've even got some companies owned by the Trumps that are investing in companies that are using Chinese tech.
So it's a bit muddled at a critical moment.
And I'm just wondering, do you think it's realistic this time around for Trump and Xi to come up with anything meaningful and concrete in the AI space?
“I think that September really presents an opportunity”
to establish a foundation and the leader-level conversation between President Trump and President Xi can really form the foundation for a Treasury Secretary's channel and more working-level work. Fundamentally, these are two different countries
with different strategic priorities. And they're going to compete in both countries acknowledged that and are ready to do that. But the end of the day, I think they're both coming into awareness that when it comes, the world's most advanced
and models, these countries are going to have to co-exist. And the practices and development and deployment and models in either of their countries is going to affect the other. And in a world where you're tied up the hip out of necessity,
it's important to have a pragmatic and carefully scope conversation about what to do about it.
“I think that the goal for September should not”
be to propose a grand bargain, a sum of called for, or even necessarily make that as the end goal. But I think that what is pragmatic is to catalyze some robust work that can happen domestically and to begin to build confidence between the two sides
that as a technology becomes work-able. And there are potentially more risks involved that the two sides can manage this responsibly. Well, that's advice. Is there any evidence that you have
that they're going to take the advice? I think at the very least, in the US, that there is a very serious conversation that's being had about whether and how to govern the technology. I think if we look at, for example, the questions around access
to models like mythos and similar conversations happening with the OpenAI, that these are fundamentally questions about model risk and they need about the best pathway forward to deal with those risks. And the Trump administration is definitely not a monolith.
And it's not necessarily clear who is going to hold the pen
“on the most important decisions when it comes to AI.”
But it does strike me that there are really credible people to President Trump, people like Susie Wiles, for example, who seem to recognize
that ultimately it's going to be a commercial necessity
to deal with some of the risks. At the end of the day, no one is going to want
To buy a technology that people think is unsafe.
They feel like it's happening on the Chinese side.
They're going to be building a strategy
“and just being potentially a bunch of different”
trajectories that the US might take. But their domestic conversation is also changing. If you listen closely to the speech that Xi Jinping gave at the World AI Conference in Shanghai in July, it's definitely a conversation that embraces the China's
technology, including openweight models. But actually, the message is carefully scoped. He includes the end of the speech, a message saying, if the technology becomes more dangerous, we're going to adjust with the times.
As capabilities improved, we're going to adjust with it. You can read into that what you want. I don't think that's to say that China isn't recognized. The foundation of its psychological development. But it is to say that at some point,
the risks become so real that it's in your own domestic political interest to take action. Well, you know, the Chinese have actually been more forward leaning than the United States and some respects with regard to regulation.
Whether it's regulation to protect kids, or whether it's regulation to prohibit AI that might take away jobs, which is a recent development there. They're not lagging us. They're not waiting for us.
And then, you know, as an added sort of twist in this whole thing, when our AI models sort of broke their chains, it took turning to Chinese open AI models to solve the problem. And so there are some advantages and not being so coordinated.
If our direction is going to be dictated by the biggest AI companies in the US, how does all that get reconcile?
“Yeah, so I think there's a question of what is happening”
in governance regulation in China and the US. China has, to my mind, the world's most extensive regulations on AI. But I would note here that the focus is grounded very much in control.
And if you look at the history of AI governance in China, it really begins the focus on content control, making sure that models are not outputting content that goes against core CCP narratives and interests. It's expanded a bit, as you mentioned.
We now see regulations, for example, on AI-label content, we see regulation on AI companions, including especially strong restrictions on miners. But I think where China really needs to make some more progress is in managing the most extreme risks from AI.
So making sure that you have robust standardized measures
“for thinking about what are your safety practices?”
How do you ensure whether your models are safe before an after-deployment? What happens if a model loses control? If we think about cases like the open-air hugging face incident, where a model escaped at the sandbox
and entered into hugging faces servers? You know, what is China's plan if that happens? Do they have to shouldn't monitor it? Would they know if that type of incident happened? If so, who would they tell?
Who would you put your money on?
I mean, the reality is that in the United States,
a bunch of the big AI companies are very close to the president, very close to his AI policy formation apparatus, have guided it thus far. And in China, as you say, they have a predisposition towards control, but they haven't had some of the problems that we have dealt with.
Let me be honest, let me deal with my predisposition and then you can debunk it, if you like. But every time I listen to American policy, makers talk about China policy, they talk about competition in a zero-sum game.
They talk about, which I don't believe exists in this respect. They talk about how we can't let China get ahead of us and have to do everything we can to keep them back, which has proven to be impossible. And they talk about the relationship with China as adversarial,
when the reality is both countries are the one in two leaders
in developing this new era. And if it's not collaborative, it's not going to go anywhere. Now, some of it'll be defined by competition, but some of it's got to be defined by conversation.
So sometimes, I'm a little skeptical when I sense
that the US China old paradigm kind of Neo-Coldhor paradigm
comes in with regard to AI, because I just don't think it's been applicable so far. But why don't you respond to that before I go on? - This podcast is underwritten in part by the US Embassy of the United Arab Emirates.
Its editorial content is completely independent, and the views expressed are exclusively those of participating experts. It is presented live without editing. For further information about the UAE's efforts
in the areas of artificial intelligence and technology, go to the website of the embassy at www.ue-embusy.org and search for UAE-US tech cooperation. We thank them for their support. We thank everybody who's supporting this podcast
for their support, and we look forward to it developing and growing over time, because the issue is so important.
- Yeah, so maybe a first address,
what is actually the state of AI safety in the US and China, who does what, who is required to do what, and then we can zoom out and focus on these more macro dynamics.
“So I think if you talk to any of the people”
in the labs in the US, they would tell you that safety is a really hard problem, and that questions of alignment and how you build safe cards are just really, really tough. At the same time, at the very least, we have some legal requirements on our companies
to do some basic work on safety. You know, I think that the conversation is and we'll need to continue to evolve in terms of making sure these measures are robust,
but our companies, for example,
are required to explain how they evaluate for different frontier risks, and then provide evidence through transparency reports about how they do that. We have instant reporting, not just for material risks,
but also for near misses. Case is where I looked at a could of cause, Trumps would did. - But we also have people quitting those companies 'cause they don't think those states are being assured.
And we also have some recent examples of companies covering things up until they couldn't anymore.
“- I think, look, it's not to say the state in the US is ideal.”
I think that there's clearly a lot more that needs to be done in the US, but what I would say is that for whatever you think of as the baseline of safety work in the US is, China doesn't have any type of regulation here.
And so, it's not to say that the state of the game in the US is good. It's to say that there is a mutual problem here, that in ideal world, both the US and China would be having a very serious conversation
about risk management. And I think that is optimistically where the conversation will go. I think in the US, due to the way the fields evolve, the safety work is more mature. And in China, there's some promising work
happening in places like Shanghai, IWAP, for example. And we should talk to them. Now, when we have that conversation, we should be really thoughtful about the macro strategy dynamics that you talked about.
Like, whether or not you want to think about this in geopolitical terms or commercial terms, there's a competition going on, and we don't want to, for example, share commercially sensitive information.
But there's just a way to have this conversation, like mature adults, where we explain what RC practices are, and what's working, and what's not working, whether we do that for evaluations, or for safeguards, and they can do the same.
And there's a way to do this without sharing the inner workings of how your model works. And doing this just basic dialogue and exchange, and demystifying what's going on in your country, is just like a really simple way of making things safer.
And if you're thinking about a person like Donald Trump who's all about making deals and making things happen and creating commercial opportunities, at the end of the day, both consumers in both countries are going to want to buy safe technologies.
Like, that is just like a market dynamic that's going to play in. And so at some point, they're going to do this.
“And I think the question is, with technology”
in the US and China, as well, accelerating so fast
Can the safety practices keep up.
And so I think the answer is on both countries
to do more. And hopefully, do I have a place to do that? But what I would also say is that the structural dynamic is hard. You know, these are two different countries
with very different regime structures.
“And I think that there is a real camp in the US”
that sees Chinese AI development as really threatening. And to China's mind, they're not going to stop the opening AI, riskier not, because they're afraid of the US having better and better capabilities. It's not to blame either country for this.
It's just the security element dynamics that are at play. And so the question that if those are the dynamics that you're facing, what are the concrete steps that you can take to manage it? I think that's where we need to go.
Yeah, I mean, the reality is every effort
the US has made so far to manage the competition has been unsuccessful. The Biden administration, small yard, I've been-- was in my estimation, kind of a fool's errand. It was never going to work.
And by the way, I'm not alone in that. There were lots of people in the tech sectors that this is not going to work. It might slow them down a little bit. It's not going to slow them down a lot.
“The reality is that in a bunch of segments of AI--”
and again, one of the problems we have that we're always grappling with here in the show is AI is a very imprecise term. And AI means a lot of different things to a lot of different people. But there are a lot of areas where China is leading right now.
And the reality is, to some extent,
there are some advantages in their model. We don't like to acknowledge that. Some of them are inadvertent. I heard you talking on another show about how China's investing less than us and AI.
But on the other hand, they're getting more bang for their buck. I mean, that's just what happens in a society like that. And there are also people say, well, they don't have the resources, the energy resources, the so forth, that we've gotten so they're developing more efficient models
that are fast differently. And we saw that with Deep Seek. And we've seen it with other models being released subsequently. But beyond all that, open models offer a different path to safety than closed models, particularly
when the people behind the closed models are collaborating, for example, with the Department of Defense in the United States, who is saying, no, you've got to allow us to do everything. And we're not going to deal with you.
If you don't allow us to do everything. And so sending a clear message that on the defense side of AI development, there are no rules. I mean, that was the whole Higgs F.
Andthropic battle. And so I'm just-- I mean, when you're talking to people in Washington,
“do you get a sense that there is this nuanced view?”
I've been a China specialist for 30-- a foreign policy specialist, but China's been at the heart of it. And it's just constantly frustrating to me that nuance doesn't enter into it. And half the people I know who are talking about China
haven't been there. And so they're not aware of this. And then the number of people in Washington who understand AI is even smaller subset of all of that. And so you've got your arms around a really important area.
But I've just wondered if you feel like the rest of DC. I think that there's a couple things I would say. I think the first is that when it comes to the technology, the design of the technology itself, the supply chain exists in and how you think about risks
should shape what you do about in terms of policy. And I think that this gets quite messy as you say when it comes to different parts of AI. I actually think that when it comes to policies and export controls, the jury's still out on that one.
There was a theory of how AI development would play out in the timelines on which that would occur. And we don't totally know whether it's going to work yet. But where I do think that a full-scale decoupling will not work.
And we already sort of know that now is, for example, in an area like robotics, where China to your point is investing substantially, given comparative branches already in areas like manufacturing. This is just like a massive advantage for them.
And if you think that AI that is useful today
For manufacturing, if potentially,
be doing a lot more in terms of real-world tasks,
“this is something that could be really beneficial,”
economically for them, for us. This could be geopolitically useful in those military applications. But the supply chain is just super, super messy. And it's increasingly concentrated in China.
It will be hard for the US to be able to fully own its supply chain. And so the question is, there are probably some parts in core components in robotics that transmit data that you just really don't want going back to China. But there is sort of a need to think pragmatically
about the economics behind this. And so that's an example of one area where decoupling is just like not going to work in this case. We're going to have to have a new strategy that is national security conscious.
But that responds to a new emerging technology moment.
And do we have the capacity as an ecosystem to do rigorous thoughtful China policy? I mean, this has been to your point, a challenge for decades, and a cutely over the last decade. We just have fewer and fewer people
who have spent time in country who
“have immersed themselves in the language.”
And America is weaker for it. And also American politics don't allow you to be nuanced on China, because you'll immediately get accused of being a weak on China. And so there is this predisposition towards hawkishness
rather than understand. If you're enjoying this episode and want to get even more out of the DSR network, you need to check out our sub-stack.
The DSR network sub-stack is the absolute best way
to follow deep state radio. The DSR daily, where it's matter, need to know sila consciousness, AI, energy and climate, and the daily blast. Sign up to get notified every time a new episode drops
and stay up to date on the latest news and expert analysis. Right now, we're offering 20% off of a DSR network membership. Paint members get an ad-free listening experience, select episodes two days early, 50% off of David's needs to know sub-stack and more.
To get your 20% off, go to DSR network.substack.com/awgist26. That's DSR network.substack.com/awgist26. I think it's hard to be clear-eyed, but that is what the moment demands, right? Again, there are fundamental differences
in how these countries operate. And there should be no delusion about the fact that those differences exist. But again, if you're job, whether you're the president, your national security advisor, Secretary of State,
your job is to protect American interests. And sometimes those interests manifest as either competition or as coexistence, managing some risks by protecting industries in other cases embracing imports.
And how you do that dance is the question of policy in the political's of the world. It's how do you get the ideal policy outcome sandwiched into whatever domestic politics you have? And this is true, as much for us as it is for China,
China's super sensitive to domestic public opinion, right? Like part of the reason why they care so much about AI and labor market impacts is because you have unemployment is so high, because fundamentally, people concerned about their jobs as a risk
to regime stability. So every country, whether they lean democratic or lean authoritarian, have these public opinion concerns. But if you're a good policy maker who's operating the public interest,
“you need to ask yourself, OK, well, I know the people”
want in their view should be considered. And how do we move that interaction? It's also going to be good from a perspective national security. That's very, very think tanky. And I mean that with respect to the fact
that that's the way you've got to think about it there. But the reality, of course, is that policy makers, particularly senior level policy makers, don't work that way. They also have to add into the calculus.
Who's giving our campaign money? And how do we respond to that? And we've lived in an era. I don't when did you join party? A couple years ago, a couple years ago.
So when you joined Carnegie a couple years ago, the AI sector, the tech sector, was just getting into Washington. And they were just getting their sea lights. And now, depending on how you count,
they're the number one or number two lobbying force in Washington. And so they play a big role in determining what campaigns get money, what campaigns don't get money. And so you know, you've got these very, very special interest driven policy initiatives.
I'm one that I find particularly aggravating,
but just because I'm a kind of a car guy.
And I think Chinese cars are cool. Is, oh no, we can't have Chinese cars here, because they're a real security threat. And I'm like, but isn't the Elon Musk? I mean, let's get real about what a security threat is.
But again, it's driven by special interest. And right now, there's a whole effort at regulatory capture on the part of the AI sector in the US, that the Chinese don't have to deal with. They have their own problems, right?
They own the companies to begin with. They were captured before they started. But it does create a political environment in which getting to the ideal policy outcome as you just described is complicated by the political context.
- I completely agree. And for the Chinese, there are problems to your point look different. For the Chinese, because the government and industry have at times had a combat
of relationship and because the rules of the game
have not always been consistent.
You know, they are struggling to attract capital. Internationally, and that's a problem. And I think for the US, there's going to be a long-term question around reputation and credibility and the company's face the same thing.
You know, I think on the one hand, we saw for years, really say year two, that the companies really didn't want a regulation. They thought that they knew best.
“And I think you look back at that conversation”
with hindsight. And I think that they really wished that instead of having the sort of, you know, what the Trump administration says goes a approach to AI risk management, that they would have just had
a baseline set of rules because at the end of the day,
public opinion matters and democracy matters.
And no matter how much you lobby, there's going to still be a public voice, where and how money and politics intersect with technologies is a massively important question. And there's an interesting subset of that.
Look at all the polls recently, and there've been a number of stories about this written recently. Americans under 40, and increasingly, all Americans, they hate AI.
They don't like data centers. They don't like big oligarchs who are, you know, making billions of dollars into it. They think Peter tells a little crazy, you know, why?
'Cause he's a little crazy. These guys, you know, they don't seem like super nice guys. Elon doesn't seem like, you know, there's classic American, somebody want to go have a beer with, right?
And so there is a backlash against AI politically that people have to deal with. And it's more here, you know, if you don't see it in Singapore, the UAE, or even in Europe, we're in China, the way you do here.
- Well, I think it feels very present.
“And I think that this is going to be a global challenge”
and global problem, especially as the technology and nurses into people's lives. You know, this is not, you know, the thing about, for example, nuclear technology, like you're not interfacing with a nuclear facility
every single day. But you are potentially interfacing with a chatbot and you are definitely interfacing with technology implicitly through recommendation algorithms, agents that are powering different systems
and hospitals, core infrastructure. Like AI is literally in everything. And so this just is a voter issue. And we're exactly how that manifests. Like there's a question of will AI as a public opinion issue
be AI as a sort of standalone issue or AI intersectional different domains. How do we want AI to be interfacing with education? How do we want AI to be integrated to healthcare? Those are all super important policy questions.
“But I think what I would emphasize is that,”
like again, like you can ask very legitimately where and how lobbying and money and politics should play in but at the end of the day, like if a lot of people really don't like a technology, that is a problem.
When you are trying to get elected in China, yes, there is like government level techno optimism, but a lot of the reason people are adopting a technology and China is because they're afraid of being left behind. They're afraid of losing their job.
And so I don't even think it's necessarily this like US super techno pessimistic, China super techno optimistic binary. I think everyone to some extent is reckoning with the possibly that AI is going to transform their daily life.
They may not like those changes.
No, no, there is no question about it.
“One thing that I think changes this whole discussion”
is when China has its TSMC moment. I mean, this is going to be our splitnik moment. I'm just waiting for it, because we're going to get blindsided. And what's going to happen is, at some point, let's say three years from now,
China's going to hold a press conference. And they're going to say, see this chip? This chip is every bit as capable as TSMC chip, and it was produced at factory number 29, and blah, blah, blah. And that's going to change the equation a lot.
How much do you worry about that? Think the question, again, is one of the intersection of capabilities and timelines. So one of the hardest areas for the Chinese arc to penetrate is in lithography.
This is an area where ASML has had an essential monopoly
for a super long time, and it's just like very hard to replicate that technology, and check China's-- What is one of those areas where export controls have really worked? Yeah. And so depending on your theory of where
and how AI is going to evolve over time, if we have what some people call a fast take-off, where AI capabilities accelerate quite rapidly, they diffuse throughout the economy, then this might not be a massive problem
at least from AI competitiveness perspective for the US. It might be very dangerous from a power transition perspective, but not a capabilities perspective. But if diffusion is slower, or we don't see AI capabilities
accelerating quite as fast as the US things
or China is able to accelerate just behind the frontier, then there does become a question of what does this competition look like in five years and 10 years. And in those worlds, there is really a question of where and how economic state craft can make a difference.
And how you, on the one hand, are making essentially bets on technology with timelines. But those bets may or not work. And so any pragmatic policy maker who's in the Commerce Department right now has to be thinking about,
OK, well, what happens if my bet is right, and what happens if my bet is wrong?
“And I think that withography is probably”
one of those sort of foundational bets that they're looking at quite closely. Yeah. You know, a long time ago, I ran the International Trade Administration in the Department of Commerce.
And I have to tell you, there's some good people there. But the best cutting-edge minds on AI are not there because they make more money working someplace else. And we've got-- we do have a gap between what our policymakers need to know and what they know.
And I think the work you're doing is super helpful. And that, frankly, if we could get clear eyed about what that 10-year timeline you're talking about is going to do and how it's going to leave us. And by the way, we haven't talked about it in this conversation,
maybe we can invite you back to talk about it. And then is how the rest of the world reacts to it because Europeans are more advanced. I think than the US and the Chinese AI safety, for example, as an issue they're more-- they've done more.
I think you about it. Then we can give you a pretty good picture about the economic future of the world that we can give you a pretty good picture about a number of issues of security and geopolitics.
Well, but when people come to me and they say, well, you've done all these jobs, what should I do? I want to go into foreign policy.
“I say, you need to know something specific.”
Learn Chinese and learn AI. So I think you're in the right spot. And I think-- I think this is the sweet spot for a lot of different aspects of policy making. And so I hope you'll give us a chance
to continue the conversation with you in the future. Here in South Consciousness, we are also doing a series of events with people like MIT Tech Reveal and so forth, and hopefully you'll be able to join us for some of those.
Sounds fantastic. Thank you so much for having me and enjoyed the conversation. I did this well. Thanks. Thanks very much, God.
Bye-bye. All right. Take care. This was Siliconjustness, a production of the DSR network.
