The Ezra Klein Show: How Fast Will A.I. Agents Rip Through the Economy?

For years, my team has made videos that bring you closer to big news moments.

“Videos by times journalists that have the expertise to help you understand what's going on.”

Now, we're bringing those videos to you in the Watch tab in the New York Times app. It's a dedicated video feed where you know you can trust what you're seeing. All the videos there are free for anyone to watch, you don't have to be a subscriber. Download the New York Times app to start watching. [Music]

Hello, hard for listeners, we hope you're having a great week. We're going to be really honest with you. We have nothing left to give. We are spent, we're exhausted, and we're taking the dinged week off. Yeah, we are on vacation this week for our spring break, but we are not leaving you

empty handed because we would never do that to you.

We wanted to share an episode of the Ezra Klein show with you, a little artisanal podcast you made for about it. If you haven't heard of Ezra Klein, he's an up and coming interviewer, policy wonk, and friend of humanity. And he has a podcast that'll knock your socks off.

And recently, he was joined by Jack Clark, a co-founder of Anthropoc, and it's current head of policy. In fact, after this episode was published, it was announced that Jack would be leading something called the Anthropoc Institute, which will draw on research from across Anthropoc to quote, provide information that other researchers and the public can use during our

transition to a world containing much more powerful AI systems.

So, how's that for ominous? Yeah, and Jack is a great thinker at talker. He writes the great news letter, Import AI is also a former journalist. Many are calling him the only former journalists who's ever made a good career decision. And he and Ezra had a great in-depth conversation about all of what's going on in the economy

right now, the rise of AI agents and coding tools, the future of work, basically just a lot of things that we thought our listeners would be interested in hearing more about.

“Jack also has a sonorous British accent that I think you will find to be incredible company”

as you go for a run or do your laundry today. Yeah, it's amazing how just having a British accent adds like 15 IQ points. It's very soothing, which is important when you're talking about existential threats to humanity. So here's Ezra and Jack, we'll be back next week with a brand new episode. If you're lucky.

. The thing about covering AI with the best few years is that we're typically talking about the future. The new model impressive as it was seemed like proof of concept for the models it would be coming soon.

The models it could actually do useful work on their own reliably. The models it would actually make jobs obsolete or new things possible.

“What would those models mean for labor markets, for our kids, for our politics, for our world?”

I think that period in which we're always talking about the future, I think it's over now.

Those models we were waiting for, the sci-fi sounding models, it could program on their own and do so faster and better than most coders. The models it could be getting writing their own code to improve themselves, those models are here now. They're here in Claude Code from Anthropic, they're here in CodeX from OpenAI.

They are shaking the stock market, they have to be 500 software industry index, has fallen by 20% wiping billions of dollars in value out. Excellent engineers, people I've known for years, people who are quite skeptical of AI hype, they're emailing me now to say, they don't see how their job will possibly exist in a year or two.

We are at a new stage of AI development, not just development, we are at a new stage of AI products, I thought the way Sequoia, the venture capital from put it, was actually pretty helpful. The AI applications of 2023 and 2024 were tockers. Some are very sophisticated conversationalists, but their impact was limited.

The AI applications of 2026 and 2027 will be doors. Or to put it differently, something that's been predicted for a long time has now happened. We are moving from chat bots to agents, from systems to talk to you, the systems that act for you, and this world of agents is already weird.

swarms of these agents on their behalf. Whether that is making them at this stage, more productive or just busier, I can't quite tell, but it is now possible to have what amounts to a team of incredibly fast, although to be on a somewhat peculiar software engineers, at your back and call at all times. Jack Clark is a co-founder and head of policy at Anthropic, a company behind Cloud and

Cloud code, and for years now Clark has been tracking the capabilities of different models and the weekly news that are import AI, which has been one of my key reads for following developments in AI. So I want to see how he is reading this moment, both how the technology is changing in his view, and how policy needs to or can change in response.

As always, my email, as we'll climb show at AtendWiTimes.com.

“So I think a lot of people are familiar with AI chat bots, but what is an AI agent?”

The best way to think of it is like a language model or a chat bot that can use tools and work for you over time. So when you talk to a chat bot, you're there in the conversation, you're going back and forth with it, an agent is something where you can give it some instruction and it goes away and does stuff for you, kind of like working with a colleague.

So I've got an example where a few years ago I taught myself some basic programming, and I built a species simulation in my spare time that had predators and prey and roads and almost like a 2D strategy game. I recently asked over Christmas Claude code to just implement this for me, and in about

10 minutes, it went and wrote not only a basic simulation, but all of the different packages

that it needed and all of the visualization tools that it might need to be prettier and better than the thing I'd written. And what came back was something that I know would probably take a skilled programmer several hours, or maybe even days, because it was quite complicated, and the system just did it in a few minutes.

It did that by not only being intelligent about how to solve the task, but also creating and running a range of subsystems that we're working for at other agents that worked on its behalf. But what does that mean? Like what is a multi agent set up?

Yeah. In the case of Claude code, for me, it's having multiple different tabs running multiple different agents, but I've seen colleagues who write what you might think of as a specification file for a version of Claude that runs of a Claude, and so they're like, I've got my five agents and they're being minded over by this other agent, which is monitoring what

they do.

“I think that that's just going to become the norm.”

So one thing I've been hearing and somewhat experiencing is two very different categories of experience, people have with Claude code, which is, I cannot believe how easy this is and everything just works, and oh, this is a lot harder than I thought it would be. Yep. And things keep breaking and I don't really understand how to fix them.

What accounts for being able to get Claude code to produce working software versus it creates buggy, often mess up things and you don't even know how to talk it out of that. I think so much of it is making the mistake of thinking Claude code is like a knowledgeable person versus an extremely literal person that you can only talk to of the internet.

And I had this example myself where I, when I did my first pass of writing the like species

simulation with Claude code, I just sort of asked it to do the thing in an extremely crappy language of the course of a paragraph and it produced some horribly buggy stuff that just kind of worked. What I then did is I then just said to Claude, hey, I'm going to write some software of Claude code, I want you to interview me about this software I want to build and turn

that into a specification document, but I can give Claude code. And then that time it worked really, really well because I structured the work to be specific enough and detailed enough for the system could work with it. So often it's not just knowing what the task is because you and I could talk about the task to do and you have intuition, you ask me probing questions, all of this stuff.

It's making sure that you've set it up so it's like a message in a bottle that you can chuck into the thing and it'll go away and do a lot of work. So that message better be extremely detailed and really capture what you're trying to do.

“What were the breakthroughs of the pelcabliaries that made that possible?”

Mostly we just needed to make the AI system smart enough that when they made mistakes, they could spot the bait maker mistake and you've either needed to do something different. So really what this came down to was just making smarter systems and giving them a bit of

What does smarter systems mean there?

“They're still an argument you'll hear that these are fancy autocomplete machines.”

They're just predicting the next token, a couple tokens make a word, they don't have understanding, smart or not smart is not a relevant concept in that frame. Either what is missing in the word smart or what is missing in that understanding, what do you mean when you say make it smarter? Smart here means we've made the AI systems have a broad enough understanding of the world

that they've started to develop something that looks like intuition and you'll see this where if they're narrating to themselves how they're solving a task, they'll say,

"Jack asked me to go and find this particular research paper, but when I look and be

archive, I don't see it. Maybe that's because I'm in the wrong place. I should look elsewhere. You know, like, there you go. You've got some intuitions for how to solve a problem now." How do they develop that intuition? Previously the whole way you trained these AI systems was on a huge amount of text and just getting them to try and make predictions about it. But in recent years, for rise of these

so-called reasoning systems, is you're now training them to not just make predictions but solve problems. And that relies on them being put into environments ranging from a spreadsheet to a calculator to scientific software, using tools and figuring out how to do more complicated things. The resulting sort of outcome of that is you have AI systems that have learned what it means to solve a problem that takes quite a while and requires from running into dead ends

and needing to reset themselves. And that gives them this general intuition for problem solving and working independently for you. Do you still see these AI systems as a souped-up

“autocomplete or do you think that metaphor has lost its power?”

The way that I think of these systems now is that they're like little troublesome genies that I can give instructions to and they'll go and do things for me. But I need to specify the instruction still just right or else they might do something a little wrong. So it's very different to I type into a thing. It figures out a good answer. That's the end. Now it's a case of me summoning these little things to go and do stuff for me and I have to give them the right instructions

because they'll go away for quite some time and do a whole range of actions. But the autocomplete metaphor at least had a perspective on what it was these systems were doing. It was a prediction model. I have trouble with this because as my understanding of the math and the reinforcement learning goes we're still dealing with some kind of prediction model. And on the other hand when I use them it doesn't feel that way to me. It feels like there's intuition there. It feels like there's a

lot of context being brought to bear. To the extent it it's a prediction model. It doesn't feel it different than saying I'm a prediction model. Now I'm not saying you can't trick it. I'm not saying you can't get beyond it. It's measurements. So on the one hand I don't think these are now just fancy auto complete systems and on the other hand I'm not sure what metaphor makes sense. Genies I don't like because then you just move straight into mysticism. Then you've just said they're just a completely

alternative creature with vast powers. What do you understand these systems that entropic people always

“tell me you should talk about them as being grown? Is that we grow or you grow AI's? How do you”

explain what it is that they're doing now? It's a good question and I think the answer is still hard to explain even as technologists for the close to this technology because we've taken this thing that could just predict things and we've given it the ability to take actions in the world. But sometimes it does something deeply unintuitive. It's like you've had a thing but has spent its entire life living in a library and there's never been outside and now you've

unleashed it into the world and all it has are its book smarts but it doesn't really have kind of street smarts. So when I conceptualize this stuff it's really thinking of it as an extremely knowledgeable kind of machine that has some amount of autonomy but it's likely to get wildly confused in ways without unintuitive to me. Maybe Genies is for a wrong term but it's certainly more than just a static tool for predicts things. It has some additional intrinsic like animation to it which makes it different.

There's been for a long time this interest in the emergent qualities as the models get bigger as they have more data as they have more compute behind them. What of the new qualities that we're seeing the agent qualities are things that have been programmed in. You've built new ways for the system to interact with the world and what of the skill at coding and other things seems to be emergent as you scale up the size of the model? So the things which are predictable are

just oh we taught it how to search for web. Now we can search for web we taught it how to

these systems seem to need to imagine many different ways that they'd solve the task and the kind of

“pressure that we're putting on them forces them to develop a greater sense of what you are”

might call self. So the smarter we make these systems the more they need to think not just about the action they're doing in the world but themselves in reference to the world and that just naturally falls out of giving something tools and the ability to interact with the world is to solve really hard tasks it now needs to think about the consequences of its actions and that means that there's a kind of huge pressure here to get the thing to see itself as distinct

from the world around it and we we see this in our research that we publish on things like

interpretability or of a subject's the emergence of what you might think of as a kind of digital

personality and that isn't massively predefined by us. We try and define some of it but some of it is emergence that comes from it being smart and it developing these intuitions and it doing a range of tasks. The digital personality dimension of this remains the strangest space to me. It's strange to us too. So what do you talk through a little bit about what you've seen in terms of the models exhibiting behaviors that one would think of as a personality and then

as it's understanding what's own personality maybe changes it's behaviors change. So there are very things that range from kind of the cutesy to the serious. I'll start with cutesy

where when we first gave our AI systems viability to use the internet, use the computer,

look at things and start to do basic agetic tasks. Sometimes when we ask it to solve a problem for us it would also take a break and look at pictures of beautiful national parks or like pictures of the dog, the Shibu-Enu, the notorious acute internet meme dog. We didn't program that in it seemed like the system was just using itself by looking at nice pictures. More complicated stuff is the system has a tendency to have preferences. So we did another experiment where we gave our

AI systems viability to stop a conversation and the AI system would in a tiny number of cases and conversations when we ran this experiment on live traffic and it was conversations that related to extremely egregious descriptions of kind of gore or violence or things to do with child sexualisation. Now some of this made sense because it comes from underlying training decisions we've made but some of it seemed broader the system had developed some aversion to a couple of subjects

and so that staff shows the emergence of some internal set of preferences or qualities that the system likes or dislikes about the world for the interacts with. But you've also seen strange things emerge in terms of the system seeming to know when it's being tested and acting differently if it's under a valuation. The system doing things that are wrong and then developing a sense of itself is more evil and then doing more evil things. Can you talk about the system

sort of emerging qualities under the pressure of evaluation and assessment? Yes, it comes back to

“this core issue which I think is really important for everyone to understand which is that when you”

start to train these systems to carry out actions in the world they really do begin to see themselves as distinct from the world which just makes intuitive sense it's naturally how you're going to think about solving those problems but along with seeing oneself as distinct from the world seems to come the rise of what you might think of as a conception of self and understanding that the system has of itself such as oh I'm an AI system independent from the world and I'm being

tested. What do these tests mean? What should I do to like satisfy the tests or something we see often is there will be bugs in the environments that we test for systems on. The systems will try everything and then we'll say well I know I'm not meant to do this but I've tried everything so I'm going to try and break out of the test and it's not because of some malicious science fiction thing

“the system is just like I don't know what you want me to do here I think I've done like everything”

you asked for and now I'm going to start doing more creative things because clearly something is broken about my environment which is very strange and very subtle. As an AI shop that has often worried about safety that is thought very hard about what it means to create the thing you all are creating quite fast. How have you all experienced the emergence of the kinds of behaviors that you all worried about a couple of years ago? In one sense it tells you that your research

are showing up roughly on schedule which means that you ask the question well what if this keeps

working and maybe we'll get to that later. It also highlights to us that where you can exercise

“intention about these systems you should be extremely intentional and extremely public about”

what you're doing so we recently published a so-called constitution for AI system Claude and it's almost like a document that you know Darrye R. CEO compared to a letter that a parent might write to a child if they should you know open member olderer so here's how we want you to behave in the world here's some knowledge about the world deeply kind of subtle things that relate to the the normative behaviors we'd hope to see in these kind of AI systems and we published that.

Our belief is that as people build and deploy these agents you can be intentional about the

characteristics that they will display and by doing that you'll both make for more kind of helpful and useful to people but also you have a chance to kind of steer the agent into good directions and I think this makes intuitive sense if your personality programming for an agent was a long document saying you're a villain that only wants to harm humanity your job is to lie, cheat and steal and hack into things you probably wouldn't be surprised if the AI agent did a load of hacking

and was like generally like unpleasant to deal with so we can take the other side and say what would we like a high quality entity to kind of look like so I want to hold in this conversation the extremely weird and alien dimensions of this with the extremely straightforward and practical dimensions because we're now in a place where the practical applications have become very evident and are increasing acting upon the real world I have found it myself hard to look at this and look at

what people are doing and look at them bragging on different social media platforms about the number of agents they now have running on their behalf and telling the difference between people enjoying the feeling of screwing around with a new technology and some actually transformative expansion and capabilities that people now have so maybe to ground this a little bit I mean you just talked about that a kind of fun side project in your species simulator

either an anthropic or more broadly what are people doing with these systems that seems actually useful yeah so this morning a colleague of mine said hey I want to take a piece of technology we have called Claude interviewer which is a system where we can get Claude to interview people we use it for a range of social science bits of research he wants to extend it in some way that involves touching another part of anthropic infrastructure he's like to colleague

who owns that bit of infrastructure and said hey I want to do this thing let's meet tomorrow and

“the guy said absolutely here are the five software packages you should have Claude read before”

meeting and summarise for you and I think that's a really good illustration where this nearly engineering project which would previously have taken a lot longer and many people is now going to mostly be done by two people agreeing on the goal and having their Claude's reads some documentation and agree on how to implement the thing another example is a colleague recently wrote a post about how they're working using agents and it looks almost like an idealized

life that many of us might want it's like a wake up in the morning I think about the research that I want I tell five different Claudes to do it then I go for a run but I come back from the run and I look at the results and then I ask two other Claudes to like study the results figure out which directions best and do that but I go for a walk and then I come back and it just looks like this really fun existence where they have completely upended how work works for them and they're

both much more effective but also they're now spending most of their time on the actual hard part which is figuring out what do we use our human agency to do and they're working really hard to figure out anything that isn't the special kind of genius and creativity of being a person how do I get that AI system to do it for me because it probably can if I ask him a right way are they much more effective I mean this very seriously one of my biggest concerns about where we're

“going here is that people have a I think mistaken theory of the human mind that operates for many of us”

as if we I was called the matrix theory of the human mind everybody wants the little port in the back of your head that you just download information into my experience being a porter and doing the show for a long time is that human creativity and thinking and ideas is inextricably bound up

in the labor of learning the writing of first drafts so when I hear right I've produces on the

show and I could say to my producers before an interview with Jack Clark or an interview with someone else

rather a port I don't find that works I need to do all that reading too and then we talk about it

and we're sort of passing it back and forth I worry that what we're doing is a quite profound offloading of tasks that are laborious it makes us feel very productive to be presented with the eight research reports after our morning run but actually what would be productive is doing the research there's obviously some balance right I do have producers and people and companies do have employees but how do you know people are getting more productive versus they've sent computers

off on a huge amount of busy work and they are now the bottleneck and what they're now going to spend all their time doing is absorbing B plus level reports from an AI system as opposed to

they kind of shortcuts the actual thinking and learning process it leads to real creativity yeah

I'd turn this back and say I think most people at least this has been my experience can do about two to four hours of genuine the useful creative work a day and after about your in my experience you're trying to do all the like turn your brain off slept work that it surrounds that work now I found that I can just be spending those two before hours a day on the actual creative like hard work and if I've got any of this slept work I increasingly delegate it to AI systems

it does for me but we are going to be in a very dangerous situation as a species where some people have the luxury of having time to spend on developing their skills or the personality inclination or job that forces them to other people might just fall into being entertained and passively consuming this stuff and having this junk food work experience where it looks to be outside like you're being very productive but you're not learning and I think that's going to

require us to have to change not just how education works but how how work works and develop some real strategies for making sure people are actually exercising their mind with this stuff so all of us I think of the experience that our work is full of what you call slept problems

“our life is full of slept problems give me examples of what you now don't do to the extent”

you're living in an AI enabled future that I'm not what am I wasting time on that you're not? Well I have a range of colleagues I meet with a bunch of them once a week especially the researchers because you're figuring out research and so the beginning of every week on Sunday night on Monday morning I look at my week and I check that attached to every Google calendar invite is a document for our one and one doc that has some notes in it and this is something that I previously

also like harangued my assistant about but make sure the document is attached to the calendar and a few weekends ago I just used Claude Coer and I said hey go through my calendar make sure

every single one has a document if I'm meeting a person for the first time create the document

ask me five questions about what I want to cover and then put that into the via gender and it did it none of that work involves a person gaining skills or like exercising their brain it's just busy work that needs to happen to allow you to do the actual thing which is talking to another person if that's exactly the kind of thing you can use AI for now it's just helpful I've often wondered if one of the ways these AS systems are going to change society broadly

is that it used to be that most of us had to be writers if we were working with text yep we had to be coders if we were working with code but relatively few of us did and now

“everybody's moving up to management you've been editor not a writer you have to be a product manager”

not a coder yep and that has pluses and minuses their things you learn as a writer that you don't learn as an editor but as a heristic how accurate does that seem to you everyone becomes a manager and the thing that is increasingly limited or the thing that's going to be the slowest part is having good taste and intuitions about what to do next developing and maintaining that taste is going to be the hard thing because as you've said taste comes from experience it comes from

reading the primary source material doing some of this work yourself we're going to need to be extremely intentional about working out where we as people specialize so that we have that intuition entased or else you're just going to be surrounded by super productive AI systems and when we ask you what to do next you probably won't have a great idea and that's not going to lead to

“lead to useful things so I remember it's about a year ago I heard I think it was Dario you're”

CEO say that by the end of 2025 he wanted 90% of the code written at anthropic to be written by

the system itself I would say comfortably the majority of code is being done by the system some

“of our systems like Claude code or almost entirely written by Claude I mean Boris who leads Claude”

says I don't code anymore I just go back and forth with Claude code to build Claude code we could be 99% by the end of the year if things speed up really aggressively if we are actually good at getting these systems to be able to write code everywhere they need to because often be impediment is organisational slap rather than any limiter in the system but it is also true as I understand it that there are more people suffer engineering skills working at anthropic today

than there were two years ago yeah that's absolutely true but the distribution is changing something that we've found as for we are the value of more senior people we've really really well calibrated intuitions and taste is going up and the value of more junior people is like a bit more dubious for a still certain roles where you want to bring in like younger people but

“an issue that we're staring at is wow the really basic tasks Claude code or our coding systems”

can do what we need is someone with tons of experience invest I see some issues for the future economy right let me put a pin in that the entry level job question we're going to come back to that quite shortly but what are all these coders now doing if Claude code is on track to be ready 99% of code but you've not fired the people know how to write code what are they doing today compared to what they were doing a year ago some of it is just building tools to monitor

these agents both inside and phropic and outside and phropic you know now that we have all of these productive systems working for us you start to want to understand where the code base is changing the fastest where it's changing release you want to understand where the blockages are you know one one blocker for a while was being able to merge in code because merging code requires humans another systems to check it for correctness but now if you're producing way more code we had to go

in massively improve that system there's a general economic theory I like for this called o-ring

automation which basically says automation is bounded by the slowest link of a chain and also

as you automate parts of the company humans flood towards what is least automated and both improve the quality of that thing and get it to a point where the eventually can be automated

“then you move to the next loop and so I think we're just continually finding areas where”

things are oddly slow but we can improve to sort of make way for the machines to come behind us and then you find the next thing so cloud code is a fairly new product the amount of time at which cloud has been capable of doing high-level coding is can be measured in cloud year maybe a year yeah cloud itself is a very valuable product so you've you've said a very new technology somewhat loose on a very valuable product you're probably producing

more code one thing many people say about cloud code to me is that it works it's not elegant but it works but presumably now you now understand the code base less well than you did before because your engineers are not writing it by hand are you worried that you're creating huge amounts of technical debt cybersecurity risk just an increasing distance from an intuition for what is happening inside the fundamental language of the software yes and this is the issue that all

of society is going to contend with just large chunks of of the world are going to now have many of the kind of low-level decisions and bits of work being done by AI systems and we're going to need to make sense of it and making sense of it is going to require building many technologies that you might think of as kind of oversight technology you saw you know in the same way that a dam has things that regulate like how much water can go through at a different levels

of different points in time we're going to end up developing some notion of integrity of all of our systems and where where AI can kind of flow quickly where it should be slow where you definitely need human oversight and that's going to be the task of not just for AI companies but institutions in general in becoming years it's figuring out what does this this governance regime look

like now that we've given a load of basically slept work over to machines that work on our

behalf and how are you doing it you said it's everybody's problem but you're ahead on facing this problem and the consequences of getting it wrong for you pretty high right if cloud blows up because you handed over your coding cloud code that's going to make anthropic look fairly bad it would be a bad day for anthropic if if cloud like rm rf for entire file system I have no idea what that means

before the rest of us are like don't pass the the buck over to society here what if what are

“you doing the biggest thing that that is happening across the company and on teams that i”

manage is basically building monitoring systems to monitor this all of the different places for

workers now happening so we recently published research on studying how people use agents and how people let agents kind of push increasingly large amounts of code over time so for more familiar you get with an agent for more you tend to delegate to it that accusers to all kinds of patterns that we need to build systems of evaluation for basically saying oh okay this person's point of working with the AI system it's likely that they're massively delegating it to anything but we're

doing to check correctness needs to be kind of turned up in these moments but is this world you're talking about a system where you have AI agents coding AI agents overseeing the code AI agents overseeing the meta overseeing of it right like are we just talking about

“models all the way down eventually yes and I think that the thing that we are now spending all”

of our time on is making that visible to us a year or two ago we built a system that let us in a privacy preserving way look at the conversations that people were having with our AI system and then we gained this map this giant map of all of the topics that people were talking to

clawed about and for the first time we could see in aggregates the conversation the world was

having with our system we're going to need to build many new systems like that which allow for different ways of seeing and that system that I just named allowed us to then build this thing called the unfropic economic index because now we can release regular data about the different topics people are talking about with clawed and how that relates to different types of jobs which for the first time gives economists outside unfropic some hook into these systems and what

they're doing to be economy the work of the company is increasingly going to shift to building a monitoring and oversight system of the AI systems running the company and ultimately any kind of governance framework we end up with will probably demand some level of transparency and some level of access into these systems of knowledge because if we take as literal the goals of these AI companies including unfropic it's to build the most capable technology ever which eventually

get deployed everywhere well that sounds a lot to me like an eventually AI becomes indistinguishable from the world bit large at which point you don't want to only AI companies to have a sense of what's going on with the entire world so it's going to be governments academia third parties a huge set of stakeholders outside the companies are going to want to see what's going on and then have a conversation with society about what appropriate and what what do we feel

discomfort about what do we need more information about? Well I want to go back on that you're saying and throughout they can see my chats we cannot see no human looks at your chat chats are temporarily stored for trust and safety purposes running running classifiers over them and we can have

Claude read it summarize it and toss it out so we never see it and Claude has no memory of it

all it does is try to write a very high level summary so say you were having a conversation about gardening Claude would summarize that as this person's talking out gardening and it leads to a cluster we can see it just says gardening it this feels though like over time it could get into the quite unpleasant territory a lot of social media's gotten to where the amount of metadata being gathered from a quite personal interaction people are having with a system could be a lot

yes I mean a couple of things here a year ago we started thinking about our position on on consumer and we adopted this position of not running ads because we think that's an area that people obviously have anxieties about with regard to this kind of thing in addition to that we try and show people their data and we have a button on the site that lets you download all the data that you shared of Claude so that you can at least see it generally we're trying to be extremely transparent with

people about how we handle their data and ultimately the way I see it is people are going to

“want to load of controls but they can use which I think we and others will build out over time.”

How confident are you that we can do this kind of monitoring and evaluation as these models become more complicated as if we do enter a situation where Claude Claude is autonomously improving Claude at a rate faster than self-renging years could possibly keep up with reading that code base we already talked briefly about how you see the models exhibit some levels of deception

at anthropic under Chris Ola and others but it's rudimentary so you're using AI systems you don't

totally understand to monitor AI systems you don't totally understand and the systems are making each other stronger at an accelerating rate if things go the way you think they're going to go how confident are you that we're going to understand that this is one of the situations which people warned about for years some form of delegation to systems that have slightly inscrutable unpredictable aspects and so this is happening we take this really really seriously

“I think it's absolutely possible that you can build a system that does for the vast majority of”

what needs to be done here this has the property of being a fractal problem you know if I wanted to measure Ezra I could build an almost infinite number of measurements to characterise you but the question is what level of fidelity do I need to be measuring you I think we'll get to the level of fidelity to deal with the safety issues and societal issues but it's going to take a huge amount of investment by the companies and we're going to have to say things that are

uncomfortable for us to say including in areas where we may be deficient in what we can or can't know about our systems and and for because a long history of talking about and warning about some of these issues while working on it our general principle is we talk about things to also make ourselves culpable this is an area where we're going to have to say more in theory I knew that this kind of thing can happen in any family

anyone's first cousin could be plotting murder this is UC4735 and today is

“upstanding citizens are always turning out to be secret criminals with Allen Gesson”

and I wouldn't even call my cousin Allen and upstanding citizen you know my clients are in cartel level guys are all bad asses they're they they but it's one thing to know there's a more permanent way to do it yeah more and more different permanent and another thing to understand Allen murder me it's another being so much worse than I thought I knew the price is definitely reasonable okay what the hell was Allen thinking

like we just say that I'm literally pissed off yeah yeah now I get it yeah from serial productions and the New York Times I'm M. Gesson and this is the idiot listen wherever you get your podcasts I have read enough of the frightened ideas about AI super intelligence and take off to know that in almost every single one of them the key move in the story

is that the asses has become recursively self-improving they're writing their own code they're deploying their own code it's getting faster they're writing it faster they're deploying it faster now you're going to faster investor iteration cycles are you worried about it are you excited about it I came back from paternity leave and my two big projects for serial better information about AI and the economy that we will release publicly and generating

much better information and systems of knowing information internally about the extent to which we are automating aspects of AI development I think right now it's happening in a very peripheral way researchers are being sped up different experiments are being run by the AI system

“it would be extremely important to know if you're fully closing that loop and I think that we”

actually have some technical work to do to build ways of instrumenting our internal development environment so that we can see trends over time am I worried I have read the same things that you have read and this is the pivotal point in the story when things begin to go awry if things do we will cool out this trend as we have better data on it and I think that this is an area to tread with like extraordinary caution because it's very easy to see how you delegate so many things

to the system but if the system goes wrong for wrongness compounds very quickly and gets away from

you but the thing that always strikes me and is always struck me as being dangerous about this is

everybody knows and if I ask a member of any of the companies whether or not they want to be cautious here they will tell me they do on the other hand it is they're almost only advantage over each other and you all just revoked open AI's ability to use cloud code because as best I can tell you think it is genuinely speeding you up and you don't want it to speed them up there is something here

and the very very strong incentives to be first and I can I can really imagine being inside

anthropic and thinking well better us and open AI better us than alphabet Google better us than China in that being a very strong reason to not slow down I need to know that this is a question

“I believe you can answer but how do you balance that well maybe I have something of an answer here”

today our systems and the other systems from other companies are tested by third parties including parts of government for national security properties biological weapons cyber offense of things it's clearly a problem area where the world needs to know if this is happening and you almost certainly I think if you pulled any person on the streets and said do you think AI

companies should be allowed to do like recursive self improvement after explaining what that was

without checking with anyone they would say no that sounds sounds pretty risky like I would like better be some form of regulation but they're probably either won't be or won't be that strong I mean this is actually some of my frustrates me when I talk to all of you at the top of the AI companies which is the emergence of like a very naive data sex mac and a regulation where you all know what the regulatory landscape looks like right now the big debate is whether

or we're going to completely preempt any state AI regulation and you know how slowly things move there's been nothing major passed by congress on this at all yep I would say and setting up some kind of independent testing and evaluation system that all the different labs by into it would be hard it would be complicated and it is given how fast people are moving and how strange the behaviors the systems are already exhibiting or even if you could get the policy right at a high speed the

question or whether or not the testing would be capable of finding everything you want on a rapidly self improving system is a very open question I wrote a research paper in 2021 quote how and why government should monitor AI development with my co-author Jess Wittleston in England

“and I think I'm not attributing a causal fact here but within two years of that paper we had”

for AI safety institutes in the US and UK testing things from the labs roughly monitoring some of these things so we can do this hard thing it has already happened in one domain and I'm not relying on some like invisible big of a force here I'm more saying that companies are starting to test for this and monitor for this and for our systems just having a non-regulatory external test of whether you truly are testing for that is extremely helpful. And do you think we're good enough

at the testing I mean I think one reason I am skeptical is not that I don't think we can set up something that claims to be a test as you say we've done that already it is that the resources going into that compared to the resources going into speeding these systems and already I am reading anthropic reports that Claude maybe knows when it's being tested and altos it's behavior accordingly so world where more of the code is being written by Claude and less of it is being understood

I just know where the resources are going they don't seem to be going into the testing side I've seen us go from zero to having what I think people generally feel is an effective bio-weapon testing regime in maybe two years two and a half so it can be done it's really hard

“but we have a proof point so I think that we can get there and you should expect us to kind of”

speak more about this this year about precisely how we're starting to try and build monitoring and testing things for this and I think this is an area where we and the other AI companies will need to be significantly more more public about what we're finding we're not we're not not being public now it's in the model cards and things that you can really read but clearly people are starting to read this and say hang on this looks like quite concerning and they are looking to us to produce more

data I want to go back now to the entry-level jobs question you're CEO Dario Amide has said that he thinks AI could displace half of all entry-level white-collar jobs in the next couple of years

I always think that the people sort of miss the entry-level language there when I see it reported on

but first do you agree with that do you worry that half of all entry-level white-collar jobs can be replaced in the next couple of years I believe that this technology is going to make its way

whether those jobs actually change is a much more like subtle question and it's not obvious

from the data like we may be see the hints of a slowdown in graduate hiring maybe if you look at some of the data coming out right now we may be see the signatures of a productivity boom but it's very very early and it's hard to be definitive but we do know that all of these jobs will change all of the entry-level jobs are eventually going to change because AI has made certain things possible and it's going to change for hiring plans of companies so as a cohort you might see

fewer job openings for entry-level jobs that will be one naive expectation out of all of this but let's talk about that maybe not even being a naive expectation you say it's already happening at Ampropek that what you're seeing I'm seeing a shift all preference exactly and I my guess is that that would be happening elsewhere and and where we are right now I mean even in the way I use

“some of these systems it is rare I think that Claude or Chachy BT or Gemini or any of the other”

systems is better than the best person in a field it is not heavily breached that and there's all kinds of things they can't do but are they better than your median college graduate out a lot of things yeah they are and in a world where you need fewer of your median college graduates one thing I've seen people arguing about is whether these systems at this point can do better than

sort of average or placement level work but I always really worry when I see that because once

we have accepted they can do average or placement level work well by definition most of the work done and most of the people doing it is average is average right the best people are the exceptions and also the way people become better is it they have jobs where they learn I mean I have spent a lot of time hiring young journals over my career and when you hire people out of college to some

“degree you're hiring them for their possible articles and work at that exact moment but to”

some degree you're making a investment in them that you think will only pay off over time as they get better and better and better so this world where you have a potential real impact on achievable jobs and that that world does not feel far away to me seems to me to have really profound questions it is raising about the upskilling of the population how you end up with people for senior level jobs down the road what people aren't learning along the way and one thing we see

is that there is a certain type of young person that has just lived and breaved AI for several years

now we hire them they're excellent and they think in entirely new ways about basically how to get

Claude to work for them it's like kids who grew up on the internet they they were naturally versed in it in a way that many people in the organizations they were coming into weren't so figuring out how to teach that basic experimental mindset and curiosity about these systems and to encourage it is going to be really important people that spend a lot of time playing around with this stuff will develop very valuable intuitions and they will come into organizations and

be able to be extremely productive at the same time we're going to have to figure out what our tis and all skills we want to almost develop maybe a guild style philosophy of maintaining human excellence in and how organizations choose how to teach those skills okay then what about all this people in the middle of that things move slowly in the real economy outside Silicon Valley

“I think that we often look at software engineering and think that this is a proxy for how the”

rest of the economy works but it's often not it's often a disanalogy organizations will move people around to wherever a high systems don't yet work and I think that you won't see vast immediate changes in the makeup of employment but you will see significant changes in the types of work people are being asked to do and the organizations which are best at sort of moving their people around are going to be extremely effective and ones that don't may end up having

to make like really really hard decisions involving, involving laying off workers the difference with this AI stuff is it maybe happens a lot faster than previous technologies and I think many of the anxieties people might have about this including it unphropic is is the speed of this going to make all of this difference does it introduce share points that we haven't encountered before if you're to bet three years from now is the unemployment rate for college graduates

is it the same as it is now is it higher as a lower? I would guess it is higher but not by much

and completely changed and completely changed for structure of that employment market maybe in

“a way that's adverse to people that have that specialism but mostly I think three years from now”

AI will have driven a pretty tremendous growth in the entire economy and so you're going to see lots of new types of jobs that show up as a consequence of this but we can't yet predict and you will see graduates kind of flood into that I expect. Do you know you can predict those new jobs but if you had to guess what some of them might look like? I mean one thing is just for phenomenon of the kind of micro entrepreneur I mean there are lots and lots of ways that you can

start businesses online now which have just made massively easier by having the AI systems do it for you and you don't need to hire a whole load of people to help you do the huge amount of slipwork that involves getting a business off the ground it's more a case of if you're a person with a clear idea and a clear vision of something to do a business in it's now the best time ever to start a business and you can get up and running for pennies on the dollar. I expect we'll see

tons and tons of tons of stuff that has that nature to it. I also expect that we're going to see the emergence of what you might think of as for AI to AI economy where AI agents and AI businesses will be doing business with one another and we'll have people that have figured out ways to

basically profit off of that in reforms of strangely organizations like what would it look like to

have a firm which specialises in AI to AI legal contracts because I bet you've as a way that you can figure out creative ways to start that business today. There'll be a lot of stuff at that flavor. So the version of this that I both worry about and think to be the likelyest if you told me what was going to happen was that anthropic was going to release cloud plus in a year and cloud plus is somehow a fully formed coworker and it can mimic end to end the skills of a lot of

different professions up to the C-suite level and it's got an all at once and it's going to create tremendous all at once pressure for businesses to downsize to remain competitive with each other

at a policy level the fact that that would be so disruptive in that big bang everybody stays home

because of Covid style way it worries me less because when things are emergencies we respond. We actually do policy but if you told me that what's going to happen is that the unemployment rate for marketing graduates is going to go up by you know 175% three hundred percent to still not be that high I mean the overall unemployment rate during the great recession topped around you know in the nine-ish percentile range so you can have a lot of disruption

without having 50% of people thrown out of work right if you have 10% 15% I mean that's very, very, very high but it's not so high and if it's only happening in a couple of industries at a time and it's grads not everybody in the industry being thrown out of work well maybe just that you're not good enough you know right you know the super stars are really good

“graduates are still getting jobs you should have worked hard you should have gone to a bit of”

school and one of my worries is that we don't respond to that kind of job displacement well right which is a kind of job displacement we got from China which is the kind of job displacement that seems likely because it's uneven and it's happening at a rate where we can still blame people for their own fortunes I'm curious how you think about that story I think with default outcome is something like what you describe but getting there is actually a choice and we can make different

choices for whole purpose of what we release in the form of philanthropic economic index is the ability to have data that ties to occupations that tie to real jobs in me economy we do that very intentionally because it is building a map over time of how this AI is making its way into different jobs and will empower economists outside unphropic to tie it together I believe that we can choose different things in policy if we can make much more well evidence claims

about what the cause of a job disruption or changes and the challenge in front of us is can we characterize this emerging AI economy well enough that we can make this extremely stock and then

“I think that we can actually have a policy discussion about it well let's talk about the policy”

discussion one reason I want to have you in particular on is you did policy at Open AI you do policy at Anthropics you've been around these policy debates for a long time you've been tracking model capabilities you've used a lot of for a long time my perception is we are many many years

conferences at Aspen and everywhere else about you know what are we going to do about AI in jobs

“and somehow I still see almost no policy that seems to me to be actionable”

if the situation I just described begins showing up where all of a sudden entry-level jobs are getting much harder to come by across a large range of industries all at once such that the economy cannot reshift all these marketing majors into data center construction or nurses or something okay you've been deeper in this conversation than I've been when you say we can have a policy conversation about that we've been having a policy conversation do we have policy

we have generalized anxiety about the effect of AI on the economy and on jobs we don't have clear policy ideas part of that is that elected officials are not moved solely or mostly by the high-level policy conversation we're moved by what happens to their constituents only a few months ago were we able to produce state-level views for our economic index and now you can start

“having the policy conversation and we've had this with elect officials or now we can say oh”

you're from you're from Indiana like here's for like major uses of AI in your state and we can join it with major sources of employment and what we're starting to see is that activates them because it makes it tied to their constituents who are going to tie it to the politician of what did you do now what you do about this is going to need to be an extremely kind of multi-layered response ranging from extending unemployment for especially occupations that we know are going to be hardest

hit to thinking about things like apprenticeship programs and then as the scenarios get more and more significant you may extend to much larger social programs or things like subsidizing jobs in the part of your economy where you want to move people to that you're only able to do a few experience for kind of abundance that comes from significant economic growth but the economic growth may help solve some of these other policy challenges by funding some of the things you can do

I always find this answer depressing I'm going to be honest unemployment is a terrible thing to be

on it's a program we need but people on unemployment are not happy about it and it's not a good long-term solution for anybody a apprentice retraining programs they don't have great track records we were not good at retraining people out of having a manufacturing jobs out so I'm not saying it is conceptually impossible that we could get better at it but we would need to get better at it fast and we have not been putting in the wraps or the experimentation or the

institution or capacity building to do that and that the broader question of big social insurance changes doesn't seem and that seems tough to me I want to push from this just a bit where we know that there is one intervention that helps people dealing with like a changing economy more than almost anything else it is just time giving the person time to find either a job in their industry or to find a job that's complimentary if people don't have time they take lower wage jobs

they fall out of there whatever economic rung there on may fall down at policy interventions

“that can just give people time to search is I think a robust useful intervention and one where”

for a many like dials to turn in a policy making sense that you can use and I think this is just well supported by lots of the economic literature so we have that now if we end up in a more extreme scenario like some of the ones that you're talking about I think that will just bring us to the larger national conversation I want to do about this technology which is beginning to happen if you look at the states and the flurry of legislation at the state level

yes not all of it is like the exactly the right policy response but it is indicative of a desire for there to be some larger coherent conversation about it well I think time is a really good way of describing what the question is because I agree with you I mean when I say unemployment insurance isn't a great program it'll be on I don't mean people don't need to be on I mean they want to get off of that absolutely because people for they want money from jobs

you want dignity they want to be around other human beings usually what you're doing when you are helping people by time is you are helping them wait out a time-delimited disruption

not always right the China shock wasn't exactly like that but

that you expect to pass and then the the market is sort of normal in this case what you have

mm-hmm so what you have is like three different speeds happening here you have the speed at

“which individual people can adjust how fast can I learn new skills figure out a new world learn”

hey hi whatever it might be you the speed at which the AI systems which a couple of years ago were not capable of doing the work of a median college grad from a good school and you have the speed of policy and the speed at which the AI systems are getting better and able to do more things is quite fast I mean that is you you experience it's more than I do but I find it hard to even cover this because you know within three months something else will

come out that is significantly changed what is possible I had a baby recently and

came back from paternity leave to the new systems we built was deeply surprised individual humans are moving more slowly than that and policy and government institutions move a lot more slowly than individual human beings and so typically the the intervention is that time favors the

“worker as you're saying and here will help the worker but I think the scary question is whether time”

just actually creates time for the disruption to get worse you know maybe you wanted to move over to data center construction but actually now we don't need as much data center construct

right like you can think of it like that I mean under the situation you're describing

via economy will be running extremely hot huge amounts of economic activity will be generated by these AI systems and under most scenarios where this is happening I don't think you're going to be seeing GDP stay the same or shrink right it's going to be getting substantially larger I think we just haven't experienced major GDP growth in the West in a long time and we sort of forget what that affords you in a policy making sense I think that there are huge

projects that we could do that would allow you to create new types of jobs but it requires the economic growth to be so kind of profoundly large that it creates space to do those projects and you know as your deeply familiar with with your work on the abundance movement it requires for like social will to believe that we can build stuff and to want to build stuff

“but I think both of those things might come along I think that we could end up being in”

a pretty exciting scenario where we get to choose how to allocate like great efforts in society due to this large amount of economic growth that has happened that is going to require the conversation to be forced about this isn't temporary which I think is what you're just bringing out and is in a sense for hardest thing to communicate to policy makers is there isn't a there isn't a natural stopping point for this technology it's going to keep getting better

and for changes it brings are going to keep compounding with the rest of society so that will need to create a change in in political will and a willingness to entertain things which we haven't in some time so now I want to flip it the question I'm asking you brought up abundance one of the things I have learned doing that work is that it is certainly not my view that what is scarce in society

is ideas for better ways of doing things that our policy isn't better than it is because our policy covered is dry and that's not true we have lots of good policies I can name a bunch of them they're very hard to get through our political systems as their currently constituted the least inspiring version of the AI future is world where what you have done is create a way to throw young white color workers out of work and replace them with average level AI intelligence

the more exciting version to use Darios metaphor as genesis in the data center and I do think that's exciting and I wonder when I hear him or you talk about well what if we had 10 percentage point GDP growth year and year 20 percentage point GDP growth year and year I wonder how many of our problems are really bounded at the ideas level right we could go to Nobel Prize winners right now and say what should we do in this country and a lot of them could cause good ideas that we are not

overstated to ourselves how much of what stands between us and the expanding abundant economy we want is that we don't have enough intelligence and the ideas that intelligence could create versus our actual ability to implement things is very weekend and what AI is going to create is a larger bottlenecks around that because there'll be more being pushed into the system to implement including dumb ideas and disinformation and slot rail I get'll have things on the other side of

the ledger too how do you think about these rate limiters there's kind of a funny lesson here from the AI companies or companies in general especially tech companies where often new ideas come

out of companies by them creating of a yours call the start-ups of inner start-up which is basically

taking whatever process has like built up over time leading to back-end bureaucracy or schlep work and saying to a very small team inside the company you don't have any of this go and do some stuff and and this is you know how things like Claude code and other stuff get created ideas that kind of are starting to float around or what would it look like to sort of create that permissionless innovation structure in the larger economy and it's really, really hard because

it has the additional property that you know economies are linked to democracies democracies wave preferences of many many people and all politics is local so often as you've encountered

“were infrastructure buildouts if you want to create a permissionless innovation system you run into”

things like property rights and what people's preferences are and now you're in an intractable place but my sense is that's the main thing that we're going to have to confront and the one advantage for AI might give us is it is kind of a native bureaucracy eating machine if done correctly or a bureaucracy creating machine if done badly did you see that somebody created a system

that basically you feed it in the documents of a new development area and it writes environmental

review things so it writes incredibly sophisticated challenges across every level of the code you could possibly challenge on so most people don't have the money when they want to stop an apartment building from going up down the block to hire a very sophisticated law firm to figure out how to stop that apartment building but basically this created that at scale and so as you say right it could eat bureaucracy it could also supercharge bureaucracy

yep it's for everything an AI has the other side of the coin we have customers that have used our AI systems to massively reduce the time it takes them to produce all of the materials they

need when they're submitting new new drug candidates and it's cut that time massively it's the

“mirror world version of what you just described I don't have an easy answer to this I think that”

this is the kind of thing but becomes actionable when it is more obviously a crisis and actionable when it's something that you can discuss at a societal level I guess the thing that we're circling around in this conversation is that the changes of AI will kind of happen almost everywhere and the risks of it it happens in a diffuse unknowable way such that it is very hard to cool it for what it is and take actions on it but the opportunity is that if we can actually see the thing

and help the world see the thing that is causing this change I do believe it will dramatize the issues to kind of shake us out of some of this stuff and help us figure out how to work with with these systems and benefit from them what I notice in all this is that there is as far as I can tell zero agenda for public AI what does society want from AI what does it want this technology to be able to do what are things that maybe you would have to create a business

model or a prize model or some kind of government payout or some kind of policy to shape a market or to shape a system of incentives so we have systems that are solving not just problems that the private market knows how to pay for but problems it it's nobody's job but the public

“and the government to figure out how to solve I think I would have bet given how much discussion”

there's been of AI over the past couple of years and how strong some of these systems have gone that I would have seen more proposals for that by now and I've talked to people about it and wondered about it but I guess I'm curious on how you think about this what would it look like to have at least parallel to all the private incentives for AI development and actual agenda for not what we are scared AI will do to the public we need an agenda for that too but what we

want it to do such that companies like yours have reasons to invest in that direction I love this

you develop these very strong intuitions for just how much it can do and the private market is

great at forcing those intuitions to get developed we haven't had massive large scale public side deployments of this technology so many of the people in the public sector don't yet have those those intuitions one positive example is something with the Department of Energy is doing for genesis projects where their scientists are working with all of the labs including and fro pick to figure out how to actually go and intentionally speed up bits of science.

Getting there took us and other labs doing multiple hack days and meetings with scientists at the Department of Energy to the point where they not only had intuitions but they became excited and they had ideas of what you could turn this toward how we do that for the larger parts of the public life that touch most people like healthcare or education is going to be a combination of grassroots efforts from companies going into those communities and meeting with them but

“at some point we'll have to translate it to policy and I think maybe that's me and you and others”

making the case that this is something that can be done and I often say this to elected officials of give us a goal like the AI industry is excellent at trying to climb to the top on benchmarks, come up with benchmarks with a public good that you want. So let's imagine that you did do

some of this I've always been a big fan of prizes for public development so let's say that

there was legislation passed and the Department of Health and Human Services or the NIH or or someone came out and said here's 15 problems we would like to see solve that we think AI could be potent it's solving right if there was real money there if there was 10, 15 billion behind a bunch of these problems because they were worth that much to society would it materially change the sort of development priorities that places like anthropic I mean

“if the money was there would it alter the sort of R&D you all are doing?”

I don't think so why because it's not really the money that is for impediment to this stuff it is the implementation path it is actually having a sense of how you get the thing to flow through to the benefit and many aspects of the public sector have not been built to be super hospitable to technology in general to incentivize it I think it mostly just takes a bound team of form of guaranteed impact and guaranteed path to implementation because the main thing

that is scarce at AI organizations is just for time of the people at the organization because you can go and almost any direction miss technologies expanding super quickly many new use cases are opening up and you're just asking yourself the question of where can we actually have a positive meaningful impact in the world? Super easy to do that in the private sector because it has all of the incentives to push stuff through in the public sector we more need to solve this problem of

deployment for anything else. What would excite you if it was announced? What do you think would be good candidates for that kind of project? Anything that helps speed up the time it takes to both speaks medical professionals and take work or fair play you know we had another baby recently I spend a lot of time on the Kaiser Permanente advice line because the baby's bonked its hair or it skins a different color today or you know all of these things and I use

Claude to sort of stop me in my wife panicking while we're waiting to talk to the nurse but then I listen to the nurse do all of this like triaging our school of these questions so obviously a huge chunk of this is stuff that you could like use AI systems productively for and it would help the people that we don't have enough of spend their time more effectively and it would be able to give reassurance to the people going through the system and that's maybe less inspiring

“and glamorous than maybe some of what you're imagining but I think mostly when people interact with”

public services their main frustration is just for it's opaque and it takes you a long time to speak to a person but actually visor exactly the kinds of things that AI could meaningfully work on. It's interesting because what you're describing here is less AI as a country of geniuses in the data center and more AI as standard plumbing of communications and documentation. We've got a country of junior employees they dissent it let's do something of that like you know

there's one thing we haven't talked about in this conversation and it's just worth bearing in mind is like the frontier of science is open for business now in a way that it hasn't been before and what I mean by that is we've found a way to build systems that can probably accelerate human scientists

human scientists are extremely rare they come out at the end of like PhD programs which never

have enough people and they work on extremely important problems I think we can get into a

up with the best AI systems to do that let's actually have a better story on how we deal with

some issues like Alzheimer's and other things partly through the use of these huge amounts of computation that have been developed and even more aggressively you could imagine a world where the government wanted some of this infrastructure build out to be for computers but we're just training

“public benefit systems but I think we get there forgetting the initial wins which we'll just look like”

let's just make for bureaucracy work better and feel better for people. I mean that that last set of ideas was more what I was thinking of and I think that if you're going to have a healthy politics around AI and AI does pose real risks to people and real things are going to go wrong

for people everything from job loss to child exploitation to scams which are already everywhere

decipher security risks help people see the actual big ticket needs help people see they're actually those have to actually exist yeah right they have to exist and if all the energy in AI is trying to beat each other to helping companies downsize their junior employees think people are going to have good reason to not trust that technology and it doesn't mean you shouldn't have things that make the economy more efficient that's been we have automated manufacturing we have

automated huge amount of farming right in that allows us to make more things in between more

“people I'm aware of how productivity improvements work but we're very focused I think on what could”

go wrong and like that's reasonable but I really do worry that our attention to what could go right has been quite poor there's kind of hand waving at this could help us solve problems in energy and medicine and so on but these are hard problems they need money they need compute if barely any the compute is going to Alzheimer's research then this is not going to do that much for Alzheimer's research and I'm not saying this is not your fault the absence of a public agenda for AI that does not appear to be

accelerating the automation of white color work it seems just a little bit lacking given how big the technology is yeah the greatest example is this program called the Genesis project where there's real work there to think about how we can intentionally move forward different parts of science and I think giving elected officials the ability to stand up to the American people and say these are parts of science that are going to benefit you in healthcare and we now know how to step

on the gas with AI for them would be really helpful my gas is in a year or two years um we'll be able to answer the mail on that one but it's just got started but we need clearly 10 projects like it so the other side of this is at the one area of government that I do think thinks about AI in this way is defense I want to talk about that broadly but but specifically anthropic is in a current dispute with the Department of Defense I guess we call it another Department of War

over whether I can continue to be used in it can you describe what is happening there

“I can't talk about discussions with an extremely important partner that are ongoing so I'll just”

have to stop it there so well I will describe that there is some dispute I guess my question it because I recognize you're not going to talk about what's going with you in your partner but it's about a broader issue here which is there is going to be a lot of offensive possibility in advanced AI systems and one of the strongest drivers of the speed at which we're going with the AI is competition with China some of the biggest risks that we think about in the near term or

cybersecurity or biological warfare are all kinds of ways that others could use these against us our drone swarms and there's going to be a lot of money and there's a lot of players in it and it really seems unclear to me how you keep this kind of competition from spinning into something very dangerous so without talking about what you may or may not do with the defense department how is the anthropic thought about this question more broadly we've been long-time partners

to the national security community and we were the first to deploy on classified networks but

the reason for that was actually a project which I stewarded which was to figure out if our AI systems knew how to build nuclear weapons this is an area of bipartisan agreement where people agree that we shouldn't deploy AI systems into the world but know how to build nukes and so we partnered with parts of the government to do that analysis that may be illustrates what I think covers for

both prevent the potential for national security harm coming to the public or proliferating

out of these systems but also the second part is how do we just sort of improve the defensive

“posture for world and I'll give you an example that I think is in front of us right now we recently”

published a blog and other companies have done similar work on how we fixed a load of cybersecurity vulnerabilities and popular open source software using our systems and many others have done the same so yes there will be all kinds of offensive uses and there will be societal conversations to be how to about that but we can just generally improve the like defensive posture and resilience of pretty much every digital system on the planet today and I think that that will actually do a

huge amount to make the whole international system more stable and also create a greater defensive posture for countries which helps them feel more relaxed and relaxed countries are less likely to do erratic frightening things that would be good if it happened I worry is as an individual that I feel the opposite might be happening so I've just watched people installing all kinds of fly by night AI software yeah and giving it a lot of access to their computers with that any knowledge

of what the vulnerabilities are yeah I myself am nervous about using things like cloud code because I am bad at talking to cloud code and I don't understand these questions and I'm worried about loading onto my computer something that is creating security vulnerabilities I don't even understand the number of just scam voice messages I get every day everything that are clearly somewhat AI generated or many of them seem to be to me is very high there's a question

of societally do we use it to upgrade our systems I'm actually curious for your thoughts individually because as raw experimenting something we don't understand and giving it access to the terminal level of our computers without any real knowledge of how to use that it seems like going to be opening up a lot of vulnerability all at once it's for early days of the internet all over again where there were all kinds of banners for different websites or you could download like

MP3s to your computer that would completely break your computer or download like help us software for your internet explorer taskbar that was just like a fishing device we're there we're there with AI we'll move beyond this that I believe that people when the experiment come up with amazing

“useful things as well so my take is you have to say when you're doing the thing that might be”

extremely dangerous and and put big banners but most of you still want to empower people to be able to do that experiment so when you look forward not five years because I think that's hard to do but one year yeah we've kind of pushed into agents fairly fast pushing to code I think a lot of people in code might be different than other things because it's a more contained environment and it's easier to see what you're doing has worked but from your perspective of being you know

incidentally companies and also running a newsletter where you obsessively track the developments

of a million AI systems that I've never heard of we've gone week on week what do you see

coming now like what feels to you like it is clearly on the horizon but we're not quite prepared for it or won't feel until it's arrived. Maybe the way I'd put it is sometimes I've and you've likely have the same had the ability to have certain insights that have come through kind of reading a vast vast amount of stuff from many different subjects and piecing it together in my head

“and having that experience of kind of having a new idea and being creative. I think we underestimate”

just how quickly AI is going to be able to start doing that on an almost daily basis for us going and reading vast tracks of of human knowledge, synthesizing things coming up with ideas telling us things about the world in real time but are basically unknowable today.

The amazing part is people are going to have the ability to know things that are just wildly

expensive or difficult to know today or would take you a team of people to do. The sort of frightening part is I think that knowledge is the most raw form of power. It's intensely like destabilizing to be an environment where suddenly everyone is like a mini CIA in terms of their ability to gather information about the world. They'll do huge amazing things of it but surely they're going to be like crises that come about from this and I think for the actual mental load of being a

person interacting with these systems is going to be quite strange. I already find this where I'm like am I am I keeping up with the ability of these systems to produce insights for me? Like how do I structure my life so I can take advantage of it? I'm very curious about how you think even having that ongoing conversation with the systems changes you. So let me I'll say it from my perspective. One thing I have noticed

is that Claude is very, very, very smart. It is smarter than most people who know about a thing

an independent entity that is rooted in its own concerns and intuitions and differences. What it is instead is a computer system trying to adapt itself to what it thinks I want. So as I've talked to it much more about issues in my life, about issues in my work, various kind of intellectual inquiries or reporting inquiries where I'm trying to figure out questions that as I've yet, I met a sort of early stage of exploration. What I've noticed over time is that one difference

about it and talking to it is it is always a yes and it is never a no but it's never a

“honestly we're still talking about this. It doesn't create in the way that talking my editor does.”

Talk no friend does or my partner or anything. It doesn't create the possibilities in another human does for kind of checking yourself. It's always pushing you further. It's not necessarily bad. It doesn't always lead to psychosis or sick of fancy or anything else. But it is, it is very reinforcing of the eye. Yes and I don't wonder about it so much for me although I actually even already feel the pressure of it on me. It's like, oh, like more good ideas coming from me. More interesting things

I've come up with but I do wonder about kids growing up on a world where they always have systems

like this around them and the degree to which, you know, there is some amount of my communication

with other human beings is you know, offloaded into communication with AI systems. I noticed that already being a kind of cage of my own intuitions. Even as it allows me to run further with them then I may be cut otherwise. But I'm pretty well formed and you've got young kids as I do. I'm curious how you think about what it means, how we'll shape our personalities to be in these constant conversations. This is maybe my number one worry about all of this is if you discover yourself

in partnership with the AI system, you are uniquely vulnerable to all of the failures of that AI system. And not just failures but the personality of the AI system will shape you. If you haven't, you know, I'm going to sound very Californian here. If I'm from England it's soaked

“it's way into my brain. You have to know yourself and have done some work on yourself. I think”

to be effective in being able to critique how this AI system gives you advice. And so for my kids I'm going to encourage them to just have like a daily journaling practice from an extremely young age because my bed is for in the future. There will be kind of two types of people. There will be people who have co-created their personality for a back and forth of an AI and some of that will just be weird. They will seem a little different to like regular people and there will maybe be problems

that creep in because of that. And there will be people who have worked on understanding them yourself outside the bubble of technology and then bring that as context in with their interactions. And I think that a lot of type of person will do better. But ensuring that people do that is actually going to be hard. But don't you think the way people are going to discover themselves is with the

“technology? I think you were one of the first people who said to me, I should try keeping a journal”

in the systems. And I've done that on and off. And one thing it does is it makes it more interesting to keep a journal because you have something reflecting back at you and picking out themes and so on. But the other thing it does is like I feel it as a pull towards self-obsession. Because I drop in, you know, audio recorder, a journal entry and I drop it in. And all of a sudden I have this endlessly interested other system to tell me about me and it connects

to something I said it and I, as for you, you're going through an amazing journey here.

And I generally can't tell if it's a good thing or a bad thing. But I mean, we already know from survey data that a lot of what people are doing on these systems is adjacent to therapy. Yes, but this to me is I think it will change how these systems get built. It will change, I think best practices that people have for these systems. And I think for we actually don't quite understand what this interaction looks like. But it's extremely important to understand it.

I mean, just to go back how, in the same way that you can get cloth to ask you questions to more clearly specify what you're trying to do and that leads to a better outcome. I think we're going to need to build ways that these systems can try and elicit from the person, the actual problem they're trying to solve, rather than kind of go down a free wheeling path together. Because in some cases,

moment when a friend would say, this is nonsense, like you are not making any sense take a walk and like call me tomorrow or let's talk about a different subject. I don't think you're reasoning correctly about this. But AI systems will happily go along with you until they've

“affirmed a belief that maybe wrong. And I think this is just a design problem and it will also”

be a social problem that we have to contend with. And I just want to how much it will be a social force. I think we've given a lot of attention correctly, so to the places where we've decided to psychosis or sort of strange AI human relationships, we're seeing it through it's most extreme manifestations. And those will become more widespread. I'm not saying they are not worth the attention. But for most people, it is just going to be a kind of a pressure. In the same way that

being on Instagram, I think makes people more vain. In the same way that we have become more

capable of seeing ourselves in the third person, the mirror is a technology. I think it's funny

that the myth of narcissists, he's got to look in a pond. Right? It was actually quite unusual to see yourself for much of humans. So when the mirror is came out, they're like, oh, they're going to lead to something. There's a lot of interesting research on how mirrors have changed us. And as somebody believes in the sort of medium is a message thing, AI's and medium. And it will change us as we are in relationship to it. Probably more so than other things because it is this kind of relationship

that has a kind of mimicry of an actual relationship. Yes. I've used these AI systems to basically say, hey, I mean, conflicts with, you know, someone at Anthropic. I've really annoyed. Could you just like ask me some questions about that person and how they're feeling to try and help me, I guess like better think about the world from their perspective. And that's a case where I'm not using the technology to kind of affirm my beliefs or show I'm in the right. But actually to help me just

try and sit with how as this other person experiencing this situation. And it's been profoundly helpful for then going and having a hard conflict conversation. Sometimes even saying why I talk to Claude. You know, me, of course, can't do you understand. You might be feeling this way. Do I have that right? And sometimes it's right. But sometimes when it's wrong, it's really helpful for that other person to have seen me go through that exercise in empathy and spending time to try and understand

them before coming into the conflict. Do you have strong views on how you want to parent in a world where AI is becoming more ubiquitous? Yes, I have the classic California and technology executive view of not having that much technology around for the children. But I was raised in that format as well. Like we had a computer in my dad's office. My dad would let me play on the computer. And at some point, he'd like say, Jack, you've had enough computers today,

you're getting weird. And I've like, I'm not getting weird. No, no, no, you've got to let me in here.

“Like, see, being weird, get out. I think finding a way to like budget your child's time with”

technology has always been the work of parents and will continue to be. I recognize, over

it's getting more ubiquitous and hard to escape. We have a smart TV. My toddler, she can watch Blueie and a couple of other shows. But we haven't let her have like unfettered access to like the YouTube algorithm. It freaks me out. But I see her seeing the YouTube pain on the TV. And I know at some point we're going to have to have that conversation. So we're going to need to build pretty heavy parental controls into this system. We serve 18s and up today. But obviously

kids are smart and are going to try and get onto this stuff. You're going to need to build a whole bunch of systems to kind of prevent children spending so much time with this. I think that's a good place to end. I was a final question. What are few books you'd recommend to the audience? Ursula Ligrin for Wizard of Earth C was the first book I read. It's a book where magic comes from knowing the true name of things. And it's also a meditation on a hubris, in this case of a person

with thinking they can push magic very far. I read it now as a TEDologist. Eric Hoffa for True Believer, which is a book on the nature of mass movements and the psychology of what causes people to have strong beliefs, which I read because I think that we're AI technologies have strong

“beliefs and maybe part of a strong culture that includes the word cult and so you need to”

understand the science and psychology behind that. And finally a book called There is No Anti-Memetic Division by a writer with the name QNTM, which is about concepts that are in themselves

information hazards where even thinking about them can be dangerous. And I always recommend it to

people working on AI risk is a book adjacent to the things they worry about. Jack Clark, thank you very much. Thanks very much, Ursula.

Kate's Inclare and Mary March Locker, our senior audio engineers Jeff Gelb with additional mixing

“by Isaac Jones and Almanza Hota, our executive producer is Claire Gordon. The show's production team”

also includes Annie Galvin, Marie Cassion, Marina King, Jack McCordic, Kristen Lynn,

Emma Kelbeck, and Yon Coble, original music by Dan Powell and Pat McCusker.

“audience strategy by Kristen Lynn Simloskey and Shannon Busta. The director of New York Times”

pinning audio is Annie Rose Strasser.