This Is How OpenAI Goes Broke — ft. Sebastian Mallaby

Or maybe you're already in the thick of it and have no idea where to start? This week on Nettworth and Chill, I'm sitting down with Michelle Smith, one of the nation's most sought after divorce financial specialists, who helps high net worth women navigate the emotional and financial realities of splitting up.

“Michelle breaks down everything you need to know what assets are on the table, what mistakes”

to avoid, and what you should be doing right now to come out on top. Listen wherever you get your podcasts or watch on youtube.com/yourrichbf. Welcome to Prof. Markets. Cracks are forming in the Open AI story. Last week, the company reportedly proposed giving the US government a 5% stake worth roughly

43 billion dollars as a way to share the upside of AI with the public.

Critics, however, argue it would amount to a government bailout and see it as a troubling signal for both Open AI and the broader AI boom. That news came after reports that Open AI had pushed back its IPO plans until 27, adding to concerns about the company's financial position. So we wanted to speak with someone who has spent years studying the history of AI, and

“who also believes that Open AI could run out of money in the near future.”

Sebastian Maliby is a prominent journalist, author, Pulitzer Prize finalist, and Senior Fellow at the Council on Foreign Relations. And today, he's joining us to discuss what is next for Open AI, what is next for the AI industry and what investors should be watching. Sebastian, thank you so much for joining me on the show.

I'd love to start with an article you wrote back in January that was titled, "This is what convinced me Open AI will run out of money." And you said back then, quote, "My bet is that over the next 18 months Open AI runs out of money." We've been seeing a lot of red flags since then, the delaying of the IPO later.

We saw this proposal for the US government to take a stake in the company. I guess I'll just start with, "What do you make of the recent news and do you hold to your prediction?" Yeah, I do hold to my prediction. Back in January, the burn rate was just crazy, so that although Open AI had good products

and quite a lot of traction 900 million consumers, they weren't able to charge money for

the product, like 5% of the retail consumers were actually paying if you look at a chart of where these users are. The US is the number two market India is first and the next three are Brazil, Indonesia, and so forth. So these are not rich consumers, you can't charge them very much money.

So they had a business model that imagined that they could throw money in all directions. Collaboration with Johnny I, to have a new form factor which would supplant the iPhone, serving Sora, video generation models and all this stuff, all of which is very expensive, and yet the revenue side simply wasn't there, so the burn seemed to me to be totally unsustainable and even though Sam Altman is a magician when it comes to raising money, he wasn't going

to be raising $660 billion which is what the internally projected burn rate was for the next

five years. When you look to the documents back in January, now since then, what's happened is some

“good news, because OpenAI I think has recognized that it had to get the burn rate down,”

it's pulled out of a bunch of data-centered building products, Stargate, all that stuff. It's cancelled Sora, the video generation model, which was a total money loser, and it's tried to impose some sort of order on the chaotic management, but it's only being kind of half successful, and in the meantime, OpenAI is squeezed between anthropic, which is much better at the frontier enterprise applications, like coding assistance, and cyber

security stuff, and agentic stuff, and then on the other hand, it's squeezed by the Gemini model from Google DeepMind, which is now reached more retail consumers and is way better monetizing from that, because Google has plugged AI into its search advertising business, and that business is now doing more revenue than ever. So I just think that OpenAI is technically a good lab, but it's very hard to monetize,

when you have a product where there's a lot of competition, and it's kind of a commodity, and they're not terribly well-managed, and they've relied too much on the fake it to you make it so that a convalley tactic of kind of weird, smoke and mirrors fundraising gametes. If you look at the fundraising they did, and announced earlier this year, the headline number they raised was $122 billion, which is an astronomical amount, but when you dig in,

was kind of promises in the future, conditional upon having a self-lippier, or payment in kind, like access to compute.

“The actual real money was a small share of the total fund raise, which raised the question”

"Why announce this massive $122 billion headline number when anyone who digs into it can see it's rubbish?"

Well, the answer is they're trying to head fake investors into putting more money in,

trying to persuade people they have momentum, they don't, and this news that you pointed out just recently that they have delayed seems their IPO in the next year is just the latest icing on the cake, the latest evidence that they took a big game, but they are behind where they say they are. Do you think that the delay of the IPO was in large part because of all of this, because

perhaps I'm Altman and the company know that as soon as Wall Street actually gets an audited review of their financial statements, and suddenly the tide will turn on this company, and suddenly people will say, "Sure, you might have a great product, but this is not a sustainable business model." Do you think that that was the concern that people might actually see how the company actually

works? 100%, I mean everybody remembers the we work story when we work was this rocket ship back in like 2019 and it went out where they prospect us to do the IPO and people looked

at it and said, "This is a joke, and nobody wanted to buy the shares, and the IPO never

happened." So, you know, you can fail in going for the IPO, and OpenAI is in this very tough position where on the one hand it needs the IPO, because it can't hope to raise enough money if it stays private. On the other hand, if it tries to do the IPO, it may not succeed, and then it's really

cooked. Just looking at how much the spending of the moment could we just sort of financials that were released by Ed Zittrin, who's this independent journalist, he got his hands on the numbers. They generated $13 billion last year in revenue. They spent $34 billion, which means that their operating loss was $21 billion.

I mean, we could talk about the net loss, which was even higher than that number, but that seems to be like a good, roughly, estimate of how this business is actually doing. And you mentioned that they're stuck between on one hand Gemini, and then also on Thropic. And Thropic is an interesting one, because we also don't know much about that business, and we know that they're unprofitable, and we know that they're in a similar business

to OpenAI, and as you say, this technology is becoming increasingly commoditized. They said there were reports that maybe they were coming up on a quarter of operating

“profitability, but I think we probably have to take that with a grain of salt, because”

we don't know how they're doing their accounting. My question, how does Anthropic compare to OpenAI from a business model perspective? I think you're making a good point, and I agree with you, that we don't know as much as we would do if Anthropic were a public company or the prospectus was public. What we do know is that Anthropic has a risk targeted enterprise customers, which means

that type of customer that are actually pay for the product, and we know that it's being ahead on stuff like coding assistance and cybersecurity AI. Not that OpenAI is bad, by the way, I mean, it's not far behind, but I think Anthropic is the cutting edge on those particular applications that enterprises are really willing to pay for.

In meanwhile, Anthropic has not been sort of distracted into announcing a whole suite of retail oriented business initiatives, which came to nothing. I mean, OpenAI announced that it was going to do shopping at one point, and that doesn't seem to have happened. It said it was going to do ads, I'm not sure they've got terribly far with the ads.

We did generate the sort of video model, which just was a huge money loser, whereas Anthropic

never went down that path.

“So I think Anthropic has been way more laser-focused on the part of the market that”

makes sense, which is the enterprise part, and just better manage. The other point I'd make is that Anthropic, amongst all the frontier labs, is known as the one where the churn, in terms of the scientists, is the lowest. People go there, they believe in the mission, they believe in Dario Amadei as the leader, and they tend to stay there, they don't hop around.

Whereas all the other labs are subject to job-hopping, and that's obviously disruptive.

about the potential of an AI bubble, and the potential that an AI bubble might pop, is

is it an open AI problem, or is it an AI problem? Is it that OpenAI is just bad at managing their finances, and they pursue all these side projects, and they don't really know how to get their spending under control, or it is AI as a business model, just too expensive, relative to the amount of revenue that could be generated by charging customers for using Chatch UPD, or charging enterprises for

these larger enterprise-wide AI contracts. What is your view on that debate, and just for context for our listeners, you wrote the power law, which is one of the most famous books ever on venture capital, and it's kind of about how venture works as a business model, where you do lose money for a number of

“years, and then you figure it out eventually, is AI going to be that story, or is this different?”

My view is that we have an OpenAI bubble, but not a general AI bubble. So I think OpenAI, for the reasons we discussed this, is a 50/50, you know, it might work. I'm not saying I don't know that they're going to fail. I'm just saying there's a 50% chance that by next summer we'll find they couldn't really go public in the private markets, they can't raise enough money, and they have to sort of sell

themselves at some sort of discount to another company could be, you know, Amazon, or Microsoft, or some other big company that wants an AI team, because technically OpenAI is a good team, right? Now, on the more general issue, yeah, there's debate at the moment about whether enterprise customers are having a, oh my god, moment where they think that these tokens are just so

expensive now, I've spent the last 18 months telling my teams that they should just go out and go wild with AI and experiment, and do whatever they feel like and token max, and the more tokens are used, the better than employee you are, because you're showing that you're AI forward. And now, wow, this is expensive, and I've got haven't seen a productivity gain yet, and so

what am I doing here? I have to rationalise this, and there are lots of stories out there about how companies are imposing a sort of middle layer between the user in the enterprise and the models, and the middle layer is there to switch a query, so that, you know, if it's a simple query that I'm asking, it gets rooted to a cheap, low token consumption model, and then only

“if it's a seriously difficult one, will it go to a fable or something expensive?”

So, I think there's some sort of sensible rationalisation about how the AI customers are spending money on this technology, but fundamentally, if you look back at what's happened since the release of chatGPT, the clear story is, this is unbelievably exciting fast progress in the tech. I mean, when chatGPT came out, the thing hallucinated non-stop, then when GKT4 was plugged

in six months later, it basically stopped like 80% of the hallucination.

When you got very long context window, so you could put a whole toll store novel into the model, and then query it, then you got, you know, these reasoning systems that could do math and logic, which would be impossible before, then you get a genetic system, then you get coding assistants, then you get cybersecurity systems, now you've got, like, bespoke AI autonomous scientists emerging.

This is unbelievably fast progress, so I fundamentally think that AI as a sector, and therefore the demands for the semiconductors, the data center businesses, all these things that people

“worry about, I don't think that's a bubble, I think that's for real, and it's going to”

take a little bit of time for companies to figure out how to ration their people's use

of tokens, so it's sort of sensible, but basically they're going to consume a lot of tokens.

Another data point that the bears might present, that we saw last week, is meta launching their cloud business, and this would be the argument against what you're saying, which by the way, I agree with, but I want to play tables advocate. You know, the very thing that Meta said they wouldn't do, they're now doing, they said that they would only launch a cloud business if they had, quote, "overbuilt." These

was Mark Zuckerberg's words just a few months ago, the plan was, let's build out all these data centers, build out all of this compute capacity, because we within the meta organization need it so desperately, because we're going to build all of these internal AI products, and we're going to, you know, AI turbo charge our business, and then they turn around and say, "Actually, we don't have the demand internally that we thought we did," and so

we're going to sell it to someone else, and we're going to let someone else figure out how to sell an AI product, and how to make that a profitable business, which seems

but Meta would be the one to build out their own suite of AI products if Meta can't crack

“it, if open AI's struggling to crack it, TBD on unthropic, then who's going to crack this?”

Who's going to make this not just an interesting technology, but an interesting technology that makes money?" And we've seen the same with SpaceX, of course, that they also decided to sell their compute capacity to anthropic and others because XAI, their own model, hasn't really got much of a uptake, and so they don't need all the compute they've built for their own model, therefore they're sending it to others.

So you could view this as a best signal as you've just described, or you could view it as a build signal, because it means that you've got some consolidation going on in the frontier model space, and less competition means better margins for the remaining participants. It means that maybe there will be more pricing power for the ones that are less standing. So I don't agree that that's the proper reading, the proper reading is we have a rationalization

of the market. If you looked at the whole U.S. ecosystem three or four months ago, you had XAI trying to compete, and met a trying to compete, and then on top of that, you had the big three Google DeepMind, OpenAI, and Anthropics, so that's five, and that's before you can't me stradden for us, go here in Canada and all the Chinese models, right? As a lot of competition, and I don't think that this thing is going to consolidate down to a winner takes hold, sort of,

“you know, 19, so 2010's social media platform was something, but I think some consolidation is in order”

such that it looks like cloud computing, where there's kind of three or four big providers. So now we've got three leaders who are still standing within the U.S. plus the foreign ones. That feels good to me in terms of the future business stability of the sector. If OpenAI runs out of money as you say, per your prediction, what do you think the outcome would be? I mean, one of the things that you wrote is that maybe it would be absorbed by another company.

I mean, how does that play out if, indeed, what you're saying might happen does happen? So look, I think, you know, we've seen lots of examples of either acquisitions or more recently acquires, where, you know, you have a small AI company, like Infection, which was Dr. Suniman was running and then it got sort of sucked into Microsoft or like character AI, which got sucked back into Google. So there's a playbook here. Now, OpenAI is a lot bigger than

either of those two. So it would be a more complex playbook. But basically, it seems to me that,

you know, the demand for AI talent and for AI products and therefore the compute infrastructure that's served at demand. I don't think that's going away because fundamentally, I think this is useful stuff that people are going to figure out how to use productively. And so I don't know whether the whole of OpenAI gets bought by Amazon or Microsoft or some other require or alternatively, there's some kind of fancy acqua-high deal where part of OpenAI is sucked into a big company or

alternatively that like, you know, there's a bit of a splintering and the staff, the technical staff at OpenAI get individually hired into other labs. Who knows, right? One I'm saying is that there's a fundamental problem with the way they're going about their business model. I think they understand that, which is why, you know, they pulled out of data center building and various other things in the last six months. But they've got some way to go to fix things and patch it up. And,

you know, one of the lessons about how you do startups, coming out of my previous book, The Power Law, is that when you have a very high valuation, a down-round is super painful, right? You know, they're valued in the last round at $852,000,000,000 post money. And in the secondary market, they're trading for a lot less than that. And if they were to sort of just say,

okay, we accept, we're really worth 600 billion. You know, the hit to everybody's equity options

inside OpenAI would be horrible and they would lose people and the hit to investors who had believed in OpenAI would be bad and they would get pissed off. And the whole momentum machine that

“Sam Altman was built would really go through a convulsion. Now, it might be what you have to do”

to make this thing sustainable. Because what my point is, once you ratchet all the way up to this very high valuation, it's difficult to climb down. And that is why I think he says, why doesn't the

give 5% to the government and then the government will say, right, you know, OpenAI is too

important to fail now because we own 5% or 10% or something. And they'll do what they did within tell which they took a 10% stake in last year. And next thing you know, the Commerce Secretary Lutnic is like calling other tech companies and the valley is saying, you're going to do a deal with Intel. You're going to bring Intel in as a partner on your next project, blah, blah, blah. And so, you know, you've got the US government, a Trumpee US government, strong arming other companies into

“giving business once they're in your corner. So that I think that is what Sam Altman's”

strategy is here to kind of recruit the, you know, the investment banker to whom you can't say no

the US government. Which seems like he's basically just trying to take some sort of work around

short cut around capitalism. And it seems like we are increasingly seeing that. Like if you can't figure it out in the free market, then oh, let's just go over to Washington, walk in the White House, kiss the president's feet and then hopefully he'll save us. And we are increasingly seeing that that is what is actually happening. We're seeing the government taking up stakes in multiple companies. We're seeing the odds that the government will take stakes in even more companies. Those are going up.

They may indeed take a stake in opening. I lost I checked on the prediction markets. The odds of that happening were more than a third. It's impossible that they would do the same with Anthropics

“with Palantir, with Andro. It makes me very upset because I think of it as cheating. I think”

that you're kind of cheating the game of capitalism. I'd be curious to get your views there. And then following up on that, if that actually happens, say OpenAI is running out of money and then Trump just bails them out in whatever way. We use taxpayer dollars to just continue to subsidize the business. What comes after that? Does that mean that OpenAI is fine? Does that mean that the rest of the AI industry is on shaky ground? I'm not even sure how to even model out that potential scenario.

First of all, I think your formulation that they're cheating capitalism and they're going to the government and doing it and around around capitalism. I mean, I think that's a good perceptive and quite amusing insight. So, thank you for that. I also, though, would say that this is like just the way the world is going. I mean, or at least the US is going. So, if you look at the number of American companies in which the US government has announced either done a deal or has announced the deal

and it's yet to be consummated. You know, a colleague of mine called Jonathan Hillman at the Council of Relations did a formal count, which just went up on the Council of Relations website.

And the answer is there are 30 of them, 30 such companies since the Trump team came into power in

January 2025, where there's an equity stake from the US government and a private company.

“So, this is where the world is going. And I think this trend has been very much encouraged”

by the deceptive example of Intel, right? So, in the case of Intel, if you look at what the performance has been since the government took a state last August, it's been fantastic. I mean, it's been way better than the Philadelphia semiconductor index, which is the normal index you would look at as a kind of comparable for how Intel is done. Intel, I think, is up like almost 400 percent, the socks or the semiconductor index and sort of like that, that's up like 150 percent.

So, these Intel is done incredibly well since the government came in. And I think people just lose sight of the fact that, you know, yeah, it did well because you've got, you know, the commerce department, calling up other companies and ordering them to do business with Intel. So, Intel gets a whole bunch of contracts and it's like turning its game around because you've got the government behind, you know, picking a winner. Now, it's one thing to say, the government might

have a justification picking a winner when we have a problem with, you know, all of the cutting edge semiconductors being made in Taiwan. We don't want to be reliant on a island that could be invaded by China and so we want to mess it US semiconductor manufacturing. I get that argument, right? I don't believe in extending the same argument to open AI, which is just one of multiple American foundation model builders. We don't need open AI for any strategic reason, right? So, there

would be no justification for picking a winner around open AI. So, I think it's, I think that the capitalism is sometimes justifiably twisted because you have a natural security reason to do so. Backing open AI would not be a justifiable instance. Well, I could imagine that the justification

but it's systemic to the stock market because, you know, Microsoft's future revenues depends

so heavily on open AI. So, do, I mean, Google, Amazon, etc. I mean, all of, basically all of the

hype is against Oracle. A lot of these companies are very, very important to portfolio's. They are what make wealthy people wealthy in a lot of cases. And maybe the argument for Trump would be, oh, well, we need to keep this thing afloat. Otherwise, people stocks are going to go down.

“What would you make of that argument? I'd say welcome to China. I mean, that's what the kind of thing”

of the Chinese government would do is prop up the stock market with government and intervention of that sort. I mean, look, in the United States, when the federal reserve operates a policy that looks like it might be about stabilizing the stock market, people freak out and say, well, that's

a FedPURT. And, you know, that creates bubbles, more bubbles in the future. And, you know, capitalism

doesn't work unless there's real risk involved. And that's the Fed. If you have, like, a bunch of political types in Washington, you know, the commerce department, and so forth, picking winners and distorting outcomes in the market, you don't have a market anymore. It's not a free market. Your point about this is an end run around, you know, capitalism, or to say the same point differently. You know, this is an end run against the notion of a fair level playing field on which different

“companies compete fairly, and then the most efficient ones win. That's what we're supposed to”

believe in as the wellspring of efficiency in American capitalism. Well, if you start de-leveling the playing field by picking open AI as a winner, you've just trash that. We'll be right back after the break. And if you're enjoying the show so far, send it to a friend and please follow us on YouTube, Spotify, or wherever you get your podcasts. Support for the show comes from Delete Me. The internet makes it easy for scammers,

stalkers, and hackers to do their thing. That's just a fact. But it's important to remember that there are ways to protect yourself. That starts with protecting your data, and Delete Me can help you do just that. Delete Me does all the hard work to wipe your personal information from data broker websites. It was named the top pick for data removal services by Wirecutter, and I'll produce a Jennifer Sanchez, has tried Delete Me. She was surprised to find out

how much personal information was actually already out there, but Delete Me made it very easy for her to clean it up. Take control of your data and keep your private life private by signing up for Delete Me. Now at a special discount for our listeners, get 20% after your Delete Me plan when you go to joindeleteme.com/prophg and use promo code "prophg@checkup".

“The only way to get 20% off is to go to joindeleteme.com/prophg and enter code "prophg@checkup".”

We're back with "prophg markets". This is a good segue into China, which is a topic that you also read about recently. The title of your piece was "I went to China to see its progress on AI. We can't beat it." I was in the New York Times recently, you're off bed. What did you learn about Chinese progress on AI and why do you think that we can't beat it? We haven't mentioned this yet, but I'm going to mention it now because you've given the excuse.

I published a book this year called "The Infinity Machine" about Demis Versailles and... I was going to get through it. There you are. I'm glad you mentioned it. Go read the Infinity Machine folks. The thing about China is it does everything faster. They published, although they got my manuscript last and then they had a translator to the Chinese and then they wanted photographs and other embellishments. They produced in a way a more complex

product, but it actually published it before Penguin Press in the United States or any of the other deals I had in other countries. I got a China right at the beginning of my book tour. I spent eight days going to four different cities, Hanger, Shenzhen, Shanghai, Beijing, talking to both computer scientists at the private labs like Huawei and Group and so forth. They're also talking

to academic computer scientists from universities. What struck me about these guys is that first of

all, they talk about safety. They just bring it up, the notion which I've heard from friends in Washington that the Chinese don't give a down about AI safety is just not true. They do talk

majority view in China is that they want safety. China is like the US. China has some acceleration

and some people who want to go slower because they worried about the safety issue. That's the same as the US. So neither side is going to de-escalate and start going slower unless the other one does as well. But what I'm saying is it's to caricature China as like only acceleration is to 100% that's just wrong. And so there is scope to talk to them about safety and maybe it will come onto that. But the other thing which I observed is that China is very good and very focused on

applications. And so if you go to a company like you know, hike vision which is under US sanctions and it's kind of a out of body sort of double take experience when you go there because on the one

hand it feels like an American tech company. I love tech companies. They're kind of all about building

cool things and making the world better. I kind of buy that. I drink that cool aid. I kind of believe in it. I like technology, right? So I see these people trying to build cool technology and they show me stuff like, for example, there is an AI kind of scanning camera thing and you pointed at the some water and you get a reading on the pollution can in the water. And because they've created that, guess what? There is an internal market in water pollution reduction between different

Chinese cities. So if you're the downstream city, you will pay the upstream city to reduce the pollution in the water that's going to come downstream to you. And so you can do pollution reduction

“when you can measure the pollution. And this is what they're doing at this company. That's what”

they're building. But they're also under sanctions these guys by the US because the US says, you know historically this was actually true, that they're building other kinds of cameras which are good for surveillance and so forth in Xinjiang and whatever. So they're both bad guys and they're cool guys. It's a difficult thing to figure out. But whatever they think, whether they are bad or cool, they ain't going away. These guys are for real. They are building cool technology.

You go to Huawei. They've got application after application. Here is our special AI to service the bullet train between Shanghai and Beijing every evening. We used to have human technicians mechanics who would go under the train and make sure it's all fine. No, we just have AI cameras and a couple of robots that may fix the train for you. They are doing this. We're not stopping them. We have imposed chip export controls on China to try to hold them back.

It hasn't worked. These guys are moving ahead. And the latest thing was you probably saw this model from a group called Gipu in China which isn't quite as good as mythos from anthropic but it's pretty close. So we are kidding ourselves if we kind of assume away the reality of China being a technology superpower. And we need to on the contrary get our heads out of the ostrich position in the sand and start talking to China about what happens when they have a mythos

level model which could hack every single bank in the global financial system and wreak havoc. We need to persuade them not to release it on an open source open weight basis because then any

“criminal can do it and they won't be enough switch. What is your view there on AI policy with China?”

The big debate is should we have these export controls should we sell chips to China? Are we

selling weapons to our enemy or do we need to sell dumber chips? Basically dumber weapons to the enemy.

Or should we not have these export controls at all? Do you think that we should have a policy or the path forward more of a method of diplomacy? I believe in American power first of all. I work at the council and from relations in New York and we do geopolitics all day long and I believe that US power is generally a force for good. So I would rather that the Chinese were behind on AI. And so to the extent that a chip export ban helps us to be ahead, I supported and indeed

when it was first announced in 2022 I wrote a massive long essay in the Washington Post about why this was a good idea. But the reason I've had my doubts recently is that I look at the results and I'm not seeing that Chinese models are that far behind and in the meantime because they're

“not far behind. I think we have to reckon with the reality that they are building models which are”

going to destabilize the global cyber system and unless we persuade them not to release them on an open-weight basis which is what they're doing at the moment. We have serious trouble on our hands like everything inside a space will DB destabilize and we need a policy to deal with this proliferation risk and I would be willing I'm in favor of the chip export ban if we could have it for free and there'd be no downside. But if the effect of having chip export controls is that we can't talk

on the chip export ban. Just looking at some of the how these models have affected the ecosystem

“I'm something we were saying earlier in the US you've got anthropic you've got open AI you've got”

Gemini those are kind of the heavy weights in the US right now but it does seem that as pricing becomes more of an issue companies are more interested in cheaper models which usually means Chinese models

and indeed that is exactly what we're seeing when we look at open router which is basically a

tracks developer marketplace for AI models. Chinese models went from less than a third of developer traffic in late 2025 to 60% by mid-2026 there are some companies that American companies that have started using Chinese models, cursor, Airbnb, Shopify, Uber, Microsoft is currently testing deep seek. What do you make of this transition over to the Chinese models, specifically the cheap Chinese models and what role does that play in potentially the policy discussion? Well I mean it

shows you that they make good models that serious American companies are thinking of using and so that's another argument for why you can't just pretend that China can be beaten and that's the end of it and these guys are for real and we have to work with them not just against them.

“Now I think it's youthful to just for a moment think through the lens of the Cold War”

where when they were nuclear weapons in the Cold War there were two kinds of big risk right one was a nuclear configuration between the Soviet Union and the United States and the way we prevented

that was through mutually assured destruction basically close to parity in the power of the two

arsenals and therefore deterrence. On the other hand there was a different category of risk from nuclear weapons which was the proliferation of these systems to rogue states or terrorists and so forth and we dealt with that with a separate mechanism which was the non-proliferation regime. Now the point is we were both competing with Russia having an arms race with Russia having a Cuban missile crisis with Russia being told by the Russians at the United Nations we will

bury you as Christchurch said when he banged his shoe on the table so there was definitely serious competition between the two superpowers but at the same time there was cooperation on a non-proliferation agreement and the way I see the future with AI is that we'll do the same we will have inevitable competition between China and the US but we also I hope have collaboration because the proliferation risk is too awful to contemplate unless you have some collaboration. It seems that what they're doing

is basically stealing what we have and people are calling it distillation and you wrote about this

and your definition of distillation quote every time a US lab produces a cutting-edge model Chinese rivals quickly reverse engineer its capabilities and build a copycat version the follower has the advantage and when I look at how companies are switching to Chinese models because Chinese models are cheaper and as you say maybe they'll use the more advanced cutting-edge models in America that are more expensive for certain tasks the Chinese models for other tasks

which essentially means that we are kind of maybe we're collaborating but also you could say that was sort of seeding advantage to the enemy to the Chinese players in the AI ecosystem and it seems that the reason that those models are good cheap is because of distillation i.e. theft.

“I don't know if I'm being crude by calling it theft I don't think I am and I think”

the Chinese have shown a pretty strong track record of stealing intellectual property from the US and then going out monetizing it on their own terms. I mean what do we know about this process of distillation and what do we know about why and how the Chinese models have been gotten so cheap and therefore so successful on a global scale? So distillation is a process which involves asking a very strong model like a new American model comes out a Chinese

copycat would ask a ton of questions to that model and get the answers and the answers amount to training data such that you know you can train the Chinese model like if the question is like this the answer should be like that and when a frontier like a first mover an American lab has to train the model in some specific very complicated frontier expertise like let's say you know quantum ticket physics they're expensively higher a bunch of quantum physicists

generating that training data for the AI is a super expensive time consuming painful process

but if you've once you've created the AI that can replicate all those quantum physicists the Chinese can come along and not hire the human quantum physicists but just query the machine

“equivalent and and that's what distillation is. Now when these Chinese companies do this”

it is not illegal but it is against contract. In other words when you when you sign up to use an American model you sign you check some boxes and you sign an agreement saying you know I'm not going to like query you gazillion times and then train my own model by copying what you've

done and so they are violating contract but not sort of federal law that's my understanding of it.

Now whatever the legal niceties the question is can you stop it I mean I'm all in favor of stopping that if we can and it seems to me that anthropic and you know Google and and Ebeneir I have all of the commercial incentives in the world to put in anti distillation safeguards if they can come up with some so I think this is a like a self-solving problem in so far as it has a solution

“and by the way I should add you know Elon Musk the other day or a few months ago casually admitted”

that his company XAI had distilled from one of the US frontier competitors so it's not just the Chinese who do this but look this is this is the rough and tumble of of the marketplace it's not nice but the practical question is you know let's stop it if we can but in so far as we're current we have to live with a reality on the ground which is that the Chinese models are good something I can't figure out is I mean if these AI labs are as capable as they say they are

can they not figure out some cybersecurity method to stop the distillation from having if

mythos is the most powerful cybersecurity technology in software that the world has ever seen

but we can't figure out how to get these Chinese developers to stop querying and replicating the same software and sort of like surely you guys can configure it out I guess my thought it would be say they do figure it out say we do put an end to Chinese distillation of US AI would that not one solve America's problems in one fell swoop and two kind of put an end to Chinese AI or at least the progress that they have been making I mean is that not kind of a poison pill for

for China? I'm not sure is the answer whether if you could stop distillation and by the way I think the latest anthropic models do have some anti distillation technology built into them so we'll see how effective that turns out to be it's going to be obviously a you know catamass both sides trying to get smarter on this one but but to to ask your question let's posit that US labs figure out a way to stop distillation would the Chinese fall behind like a lot or just a bit I'm not sure

anybody really knows the answer I kind of suspect that you know if they needed to generate their own data they would and they would pay more money and it would be more expensive and it would take them a bit more time but they would get there because they've got plenty of extremely smart Chinese scientists that they couldn't engage in generating training data will be right back and for even more markets content sign up for our newsletter at profftymarchitz.com

we're back with profftymarchitz okay let's turn to the book for a moment your most recent book was the Infinity Machine Demis Hassada steep mind and the quest for super intelligence there was one quote from the book that really stood out you said quote if you couldn't negotiate safety mechanisms inside one company what chance would there be to negotiate common safeguards among multiple labs in multiple countries which really relates to kind of what we're discussing here in terms of

“ASFT and AI policy but also there's an important implication that which is that safety mechanisms”

were not able to be negotiated with in a company what did you learn about the inner workings of these AI labs and why can't they figure that stuff out embedded in the story of Google DeepMind and Demis Hassada is this sort of morality tale about somebody who really wanted to make AI safe and that

co-founder Shane Lake at ASFT lecture in which they discussed the potential for AI to attack humanity

“by the year 2030 which turns out to be perhaps a prescient projection at least the capability”

it's going to be there whether the AI attacks is a different question but I mean anyway the point is Demis Hassada's was thinking about safety since the beginning and so when he sold his company to Google in 2014 a condition of the sale was you got to give me safety and ethics over sightboard I can't let AI be rolled out into the world just on the safe side of the Google corporate board there has to be these independent people from outside they mentioned Barack Obama as an example

when Barack Obama was leaving the presidency you know could we have somebody that statue who would be on a board saying when it's safe to roll it out right that was Demis's vision and it was coupled with another hope which was that all of the major scientists would come together

“in one single effort to roll AI out into the world so there would be no competitive pressure to go”

unsafely and tooth quickly right and it turns out and this kind of transpire through the story that I tell that all of Demis's optimistic stories about how he was going to make AI safe they all crashed and burned the idea of just one lab building AI turned out to be a pipe dream it turns out that humanity is too tribal uncompetitive and disputatious they will be multiple labs when you are kind of confronted with a prospect of being able to build a god machine there'll be plenty of

different sex of worshippers trying to do that right and and the idea of oversight within Google

ultimately the Google board would not agree to giving some outside grandies a veto over how they use

this technology that they were spending billions and dollars on developing they weren't going to do that all of a fiduciary basis obligation to their shoulders they couldn't they felt right so the point being that you know the experiment that Demis ran at deep mind and I was I discovered all these internal documents which had the back and forth between the red lines from one team of lawyers to

“the other team about the exact safety mechanisms that they might use and all this of secret strategizing”

that Demis did to threaten to spin out of Google if he didn't get the safety oversight he wanted and then they're like the Google DeepMine General Council threatened me and said I wasn't allowed to publish any of this and I said the heck with you I'm publishing it anyway so there's all quite dramatic

but the bottom line of the story is you know it turns out to be impossible to impose safety

restraints within one AI lab when that lab is in a competitive posture with respect to others and we saw the same thing play out of course at open AI but more in public when the safety board temporarily fired some open for like five days so so you know what this shows us is that if you want to stop a race which has multiple players you need the government to enforce restraint on all of the players at once and if there are players in China you need the Chinese government to buy in

and also agree to put restraints on their guys when the US puts restraints on labs within the US

France Canada that's fine basically the US can compel compliance in those places because

coherent Canada or mistrad in France depend on American technology and the American market to function but with China you can't compel them so there needs to be two countries two governments that do a deal where everybody agrees to put some caution and like checking of models before they released it was the policy of the US government that they were going to do none of that and they said I mean the even issued an executive order banning states from trying to regulate

AI in their own way but then it seems like they've kind of turned on this last month Trump signed a new executive order where he basically asks companies to hand over their models to the government that the government checked them and then kind of greenlight them but I mean on the one hand it's progress in the direction that you think is the right direction but also it's not very harsh or strict or strong it's basically just like hey could you please send your model over

we'd appreciate that what do you make of of Trump's AI policy at this point in terms of government oversight over these AI models in their safety given my perspective that

when mythos first came on the scene and galvanized the US government into paying attention and restricting the release because although you could argue you know correctly that on paper the executive order it's kind of a voluntary collaboration system with Frontier Leads blah blah blah

the reality is it's not voluntary in the least right I mean commerce recently called up

Sam Oldman at open AI and ordered him to seek government permission before he gave his latest model to any customer right government has to sign off on each customer customer by customer

“this is extremely heavy-handed right so so I think they're in it for real and the government”

they have realized that they can't let this stuff disseminate around the world without being controlled by a government and and so we're going to get pretty tough controls I think the gap in the system is that they're not talking about doing this in coordination with China because the US policy world has two kinds of China expert that you've got the kind of people who

are always hawkish on China and then the people who used to be a bit hopeful about collaboration

with China but then Xi Jinping raised to power and seemed to kind of frustrate all those hopes of collaboration and so that group of former dads flipped and became Uber Hawks on China so you've basically got the the traditional hawks and the new hawks but they're both hawkish and nobody wants to say they want to talk to China this is the problem this is the huge gap in the posture because the US government has done a 180 on domestic regulation or domestic models and I welcome that

the next thing that's going to come just because it's necessary and they're not going to have a choice is they're going to have to get over that inhibition about talking to China so is that sort of the

solution is getting a room with Xi Jinping and become partners in tackling this together I mean

it sounds like kind of simplistic but maybe that actually is the way to do it the alternative would be force their hand in some way create some sort of policy ways to know you're not going to get any chips or you're not going to get this you're not going to get that your view is we just need

“to talk with them and have more of a relationship it's a bit more complicated in that I mean I think”

you can talk and also put pressure on them at the same time I mean going back to that Cold War analogy there was a vicious competition between the Soviet Union and the United States at the same time as there was collaboration of a proliferation and so I think there will be competition and by the way you know there are ideas around strengthening the chip export controls and I'm not against that like there is one theory the case we know you call them as sometimes talk about corner solutions

you can either have a fully pegged currency or a fully floating one but if you go for some mushy middle ground where it's kind of semi pegged then hedge funds speculators are going to see that you're not really determined to defend that and they're going to eat you for lunch breakfast and dinner so it's the same thing with this AI policy they're a corner solutions you could either like give up the export controls or be willing to give them up and go talk to them and say okay

we know you didn't like that as a show of our sincerity wanting to work with you you know we're going to offer to loosen those controls but in return we want you to collaborate on fixing this non proliferation risk right that would be one corner solution or the other corner solution is you don't say that to the country you tighten up the chip export controls this is massive loophole right now whereby if your a Chinese model builder get this you can train on Nvidia chips the most advanced

versions all day long because the cloud compute that you access is in Malaysia or some other offshore place which is fully allowed to import Nvidia chips the most recent sort right this is a crazy loophole you're telling the Chinese they can't use new Nvidia chips but then you're letting them just like use a data center kind of across the border it's nuts that that loophole exists right so the other corner solution is get serious about the policies that you've announced here to

it and cut off the loophole and cut off the destination and put China in a position where it's so weak it's kind of begging for collaboration now I'm agnostic I'm like we need to collaborate

“I'm flexible on how we get there I think there's different theories just go back to to”

Trump's changing of his positioning it used to be we're not going to regulate we're not going to have any oversight because we believe that if we do that then it stifles innovation and we want you know markets to do their thing in AI labs to sort of run free uninhibited etc then mythos

tune and issues this executive order which you believe in this I think is fairly so that that actually is like stringent they they do take it seriously why do you think that happened what was it about mythos was it maybe something to do with China like why did they do this thing that

ultimately did amount to a 180 on AI policy simply because mythos was so powerful that

was very threatening I mean the prospect that you could take this model and find code vulnerabilities in every single entity on the internet and then hack it like that's curtains for the financial

“system so that's why they took it seriously yeah I mean I think throughout this topic throughout”

this topic the logic of the technology is going to force governments to do things which six months earlier they said they would never ever do and that's happened with domestic regulation already in the US I believe it's going to happen with international collaboration I've already started to see pushback from I mean Silicon Valley spent a long time not being friends with Washington and then in the last couple years they became very close friends with people in Washington and I would assume that this

is going to be I don't know this is going to cause a rift again because a lot of the technologists said that what we want is government to have no involvement in this in AI in artificial intelligence capabilities and Trump said sounds good I'm with you and now he's not I'm not really sure what that means for the relationship between Silicon Valley and Washington but I assume I don't know if you have any insight into this I assume it's not going to be great well I mean you've got this sort of

Putin and the oligarchs sort of story you've got you know endless examples of authority and governments with you know big business titans and you know where does the power lie and how

how stable is that relationship and the answer is it tends not to be stable point one and point to the

“government wins because they have the monopoly on coercion and so I think Silicon Valley you know”

is figuring that out and they realize that the government is too powerful to ignore I mean you know Darryam a day tried to say to the government you shouldn't use these tools for certain things like mass domestic surveillance and the government said get lost you are going to call you a supply chain risk and we're not going to be dictated to you I mean who won that fight clearly the government for one it has been fascinating watching Trump gives the full power of that coercion even this week

when he decided to step into the proceedings of the World Cup and he got exactly what he wanted and the year absolutely got their player back just as we saw to wrap up here you have studied a lot of the characters in AI you wrote your book about Dennis a service founder of Google Deepman kind of the open AI before open AI you studied a lot of these characters from your research from writing that book what did you learn about the people in AI and what has that kind of told you about what

might ultimately happen next and who might ultimately win the AI risk you've got some

oldman who is essentially a commercial optimist who wants to win commercially or at least survive commercially and you know his drive is to to be a big shot and he thought of running for governor of California at one point and being a political big shot but then he decided that building AI was like a bigger big shot and he wants to just like put his imprint on it he's not obviously a PhD scientist he doesn't have even a first degree because he dropped out of Stanford to do other

stuff is not to say he isn't anything other than massively smart but he isn't a deep scientist then you've got people like Daryama Day and Emma Cisabis who are PhD scientists who come at this from that perspective who want to use AI to advance deep science that's their deepest motivation

“and I believe it's very deep with both of them and I believe that's the reason why they are”

number one and number two in this race. It's good for recruiting the best scientists it's also good for holding together and leading a fundamentally scientific enterprise like building artificial general intelligence and the point where this came home to me is you know when I was talking to Dennis one day about his motivation for building AI and he started to say listen when I'm reading scientific papers at two o'clock in the morning to bastion I see reality staring at me in the face

calling at me saying I'm here to be discovered and if I had artificial general intelligence I could discover the fundamental rules that explain the fabric of reality it would be like

and so in this sense my quest for AI is kind of like my way of getting closer to what I might

“call God. Sebastian Ali is the Paul A. Volker Senior Fellow for International Economics,”

the Council on Foreign Relations a two-time Pulitzer Prize finalist he is the author of six books

including more money than God and the power law which have become investment classics his latest

“book is the Infinity Machine Demesis Sala Steep Mine and the Quest for Super Intelligence he also”

co-host a weekly CFL podcast the spillover which examines the ripple effects of global events across

policy geopolistic economics technology and financial markets Sebastian thank you so much for your

“time thank you so much that's the talk to you this episode was produced by Claire Miller and”

Alison Weiss and engineered by Benjamin Spencer on video editors Jorge Carty on research team is Dutch law and Kristen Adona here and Mia Solverio Jake McPherson is on social producer Drew Barris is our technical director and Catherine Dylan is our executive producer thank you for listening to property markets from property media if you liked what you heard give us a follow and join us for a fresh take on markets on Monday

and the price of the market.