Anthropic co-founder Jack Clark: What does the future of AI look like?

But what's happening in America isn't just the cause of global upheaval.

“It's also a symptom of disruption that's happening everywhere.”

I'm Esma Khalid in Washington, D.C., I'm Tristan Redman in London and this is the global story. Every weekday we'll bring you a story from this intersection where the world and America meet. Listen on BBC.com or wherever you get your podcasts.

Hello, it's Ader and you're listening to the Sloan Newscast extra. That moment in the week when we hear in the newsroom get to talk about what's on our minds. This week we found ourselves thinking once again about AI. Even the Pope's at it.

His first people teaching two days ago warned us that we must make sure AI serves humanity rather than threatens it. There's no doubt that we're undergoing an AI revolution. The question is, is that a good thing? And there is one AI company in particular that might actually share the Pope's caution,

anthropic.

They're the ones that created the Claude Chatbot and they're also one of the most powerful

startups in history. In the coming months, anthropic's valuation is expected to exceed 900 billion dollars. The company has set itself apart from its rivals by spousing not the virtues of AI, but it's possible dangers. Anthropic's co-founder and CEO, Dario Amade, predicted AI could wipe out 50% of entry-level

white-collar jobs in just a couple of years. And fellow co-founder Jack Clark has written about how he's deeply terrified of this technology. So when Jack Clark came to Oxford to give a lecture last week, the Observer's Technology Editor Patricia Clark sat down with him to discuss what AI might mean for humanity, for jobs, for health, and so much more.

“So you're a former journalist, which I think naturally means you're a rid of a cynic.”

I've heard you say it in interviews before that you would kind of beat over the head with this idea that this technology really does work and it is going to, as you just said, kind of change our lives. I wonder if there was a particular moment for you kind of light bulb moment or was it kind of increasing series of changes?

Yeah. In the early days, you play with this technology and you're like, "Oh, OK, it can write some code." That's interesting. That's not necessarily relevant to my world or, "Oh, OK, now it can do basic copy editing."

But that's not very interesting, and we have copy editors for a reason and they're generally better and have better taste than AI systems. But then recently, I have this hobby where I write a newsletter about AI research. And whenever I write the papers in the newsletter, I read a research paper and I write my own analysis of it.

And if I'm not sure that I fully understood the paper, previously I'd bother colleagues at unphropic about it and say, "Hey, here's the paper, here's my summary, have I actually understood it." And then I had a moment about a year ago or so where it was a weekend I didn't want to bother a colleague and I uploaded a paper in my summary to court and I said, "Hey, have

I understood it, you tell me," and Claude said, "Oh, you've understood this part, but not the other part." And I said, "So, have teach me why." And I went through this didactic conversation with Claude, where it sort of really helped me understand beyond the line research paper just as a colleague would.

And I found that kind of amazing, it was the first time that AI systems had become

fundamentally useful to me in the hobby that I most passionate about and the thing that I find most fulfilling. And then I started realizing, "Oh, I think this is happening everywhere, but these systems are starting to be able to complement people with any of their interests and teach them about almost anything."

I want to get to the kind of economic changes that are going to come about with this technology in a second, but first I kind of just want to throw it forward a little bit. And you've got two young kids, what kinds of, I mean, how are you preparing them for an AI future? And there's not much you can do to prepare a free year old for a future, because they're

just, they're just, they're just getting to grips with a well.

“But I look at them and I think, you know, what was important for me growing up, and there”

are a couple of things.

We had tons of books in our house, and my dad always said to me, but he would never

buy me like games, consoles, or computer games, or anything like that. I had to get a, eventually get a Saturday job to do that. But he'd buy me any book I wanted, or take me to a library whenever I wanted. And it allowed me to just indulge my curiosity and to learn to become obsessive about education and learning about things.

And when I look at my kids, I want to sort of create an environment where they feel like

and hobbies are going to be most advantaged by these kinds of AI systems.

“And people who don't are going to end up being more on the sort of conceiving or receiving”

end of them and reading the outputs of these systems, but not using them in a safe way I do with my newsletter as tools to advance their own learning and understanding. So just encourage them to become huge nerds about something. So I mean, there's a thing that people talk about a lot, which is the idea of cognitive offloading that somehow we're giving away a kind of deep thinking to these machines.

But I think you might be getting at something even deeper than that, which is your children's kind of sense of selves and how they develop that. When they've got this kind of technology that they can talk back and forth, can you just unpack that a little bit?

I mean, why do I write a newsletter?

I don't need to. I have, I've had jobs, you know, I, I, I, I don't need to do it for economic reasons. I do it for reasons of kind of fulfillment, but I realize that I do it mostly as a, it's my form of a creative practice. It's a thing which I do regularly to kind of keep my mind sharp and to maintain my own

orientation towards world by reading primary materials about something I'm interested in.

“I think as we develop more powerful AI systems, it's incredibly important that people”

have some kind of core interest for this that is entirely theirs and which they develop a real obsession about and then that they are going to be able to better relate to these systems than with, um, benefit lack of that.

So I think about it in terms of, in terms of that and then rather than cognitive offloading,

you get something that more feels like cognitive compounding. You have this interest and this passion and because of that, you can talk in a very informed way of these AI systems and get them to, to work for you and do things but make you more, more sovereign, more independent, more able to kind of pursue your own passions. What jobs do you think will exist or will be left for your children when they turn 18 and

beyond? I mean, I can't, I can't reconcile this technology with the economy, staying normal and just some examples I'll give are, you know, the projects that I do with my newsletter this morning, I had the AI system go and read a few hundred papers in biology, synthesize some results, generate some graphs for me, but I'll, I'll now read some of those papers.

I know that that would have taken me four or five weeks of work and I watched the system do it in in 20 minutes. And then when I look at stuff which is happening at Anthropic, I see individual researchers doing 10 times the amount of research experiments they did before because they're now able to task these AI systems to go and pursue their research ideas.

I mean, you tell me how that needs to be economy being the same as it is now if we have a technology which like multiplies what people can do. I can't, I can't, I can't make claims there, but it's not obvious to me that you immediately get things like large scale unemployment or other effects. It may be more, but we get substantial economic growth due to these massive productivity

multipliers and we get the emergence of some new jobs. It may be that we have sudden difficulties in figuring out how to take people with very little experience and get them into their careers and instead we might favor people that have more experience. Like the main view that I have and the work for Anthropic and the Anthropic Institute, which

I need is doing, is trying to build up the economic early warning signals that can inform economists outside of a frontier labs if these changes are happening and how to look for them and how to do the policy response to them. Can you explain a little bit about how you're measuring economic data at the institute and what you are seeing?

Yeah. So what we do is we can look in a privacy preserving way of a sorts of tasks people ask called to do. So people might ask called to do certain types of software engineering tasks. They might ask it to do legal tasks like reviewing the difference between two contracts

for sort of what the power legal's do. They might ask it for advice about things like cooking to gardening or whatever. You can look at these sorts of tasks and you can then join it to a certain type of job classification label which is called the own-et classification scheme which the Bureau of Labor Statistics uses and what this does is it lets you say here are the sorts of work

that people are asking AI systems about on our platform and here are the job labels they correspond to which we also measure in the larger US and other economies. And that lets us start to make statements like do we see more representation of say software

“engineering jobs being done on Claude and in the normal economy?”

Yes, do we see an underrepresentation of things to do with gardening? Yes, which kind of makes sense because clearly software engineers have been massively accelerated by AI systems, gardeners, lesser and maybe people are just using it for forms of advice.

If you get that same data from the other AI labs, you now have a picture of exactly what

work is happening on the AI platforms. And then if you would start to see things like an overrepresentation of software engineering tasks on the AI platforms and say weakness in early graduate hiring for software engineers

“and rural economy, I think you can start to make causal statements like oh it seems like”

the fact that loads of AI usage is in software engineering seems like that probably correlates to people hiring slightly fewer software engineers in the larger economy. And so what we see right now is some signs of early graduate, some jobs are getting hard to find in roughly the class of like knowledge work or software engineering work. And this may correlate to the early innings of people using AI in a way, but it means

that they are perhaps hiring slightly fewer of those people than they did before. And here in the UK specifically, I mean you touched on gardening there and trades work, which I think people say is kind of a general perhaps safer for obvious reasons. We're in 80% services economy here in Britain and whether that's financial services. Are we more vulnerable or is there an opportunity to kind of take advantage of this technology?

Yes, it's both. Like I think that in some sense the UK, just as it experienced the financial crisis, is like highly vulnerable to swings in service jobs and industries like finance or other white collar work. At the same time, if you talk to economists, but the general lesson for economics is that

the more that you automate something, the more value moves up the chain to say validating and verifying whatever's been automated. If you automate a load of work in factories, if things like quality control becomes more

“important, because you're pushing more stuff through the factory and you need to figure”

out what your quality control apparatus looks like. If you're building an economy that has much more automation in it, including in areas like financial services, then this question of how you verify and validate all the things happening in the economy becomes more valuable than it did before, as do things like figuring out how to price risk and ensure risk of automated systems where the UK has this tremendously

large financial services and insurance industry. So for a part of the UK, which I think a position really well to build some of the new jobs that the AI economy implies, and for other parts which seem like they're very exposed to the sorts of automation challenges we've just talked about.

The UK government recently announced that it's forming an institute for basically analysis

of the economic impacts of AI. We, the Anthropoc Institute are talking to them about exactly how we can share more data from our platforms to them, so that they can join the data we have with what they see in the UK economy and make clear statements about what's happening. The government's also state quite a lot on AI for its future, you know, that's a big

part of 15 years of stagnant growth, and they're saying, you know, this is a technology that we can really embrace. A we in a good position to do that, I mean, there are challenges, infrastructure challenges

“when it comes to energy and so on, are we in a good position to embrace that?”

And the UK is better positioned for most nations on AI, it is the largest, densest collection of talent outside the Bay Area, you know, we just announced that we're expanding our office from about 200 people here to 800 people or so, and knowing Anthropoc that number will probably grow even further in the coming years, so it's very well positioned. Part of that is because the UK government, I think, took this very, in light and view a few

years ago and set up for AI security institute, which has become the preeminent government function in the world that tests out frontier models from those like Anphropica, Ropen AI,

for their capabilities, and that's now led to also the creation of, like, third party

evaluation start-ups, which will flow out of sort of AC from that talent in the form in London. touched on earlier there that at the same time were quite vulnerable, because of the services economy, you touched on the financial crisis there, what can the government do to prepare? I mean, set up, obviously, like early warning systems, but also think carefully about what you would do if you saw changes, you know, one of the, I think one of the lessons

that we saw from COVID was some government reacted, maybe two late to the COVID crisis, such at the end of up paying higher costs as a society, later other governments, I think notably the US, basically fired, fired the proverbial, like, money cannon in response to the crisis and avoided some of the economic harm that happens to other countries. And what, how you can get ahead of that for AI is you just start doing scenario planning of, okay, if AI shows up in

this part of the economy in a negative way, what should the policy responses be? Because if you

flattened the proverbial curve of any of the downsides as they show up. And I think concretely, if you just had 10 to 20 people in the UK tasked with doing this scenario planning within the within government for what the policy responses to different forms of both opportunity and crisis would be the UK would be better positioned for any of the nation on the planet, like it doesn't take much to get ahead of this. You know, we're already grilling you.

“Yeah. I mean, the other pillar that you look at, well, I think there's four pillars of the”

anthropic Institute, one of them is the kind of social impact to the AI. And one thing I've been really interested in is the design of these models and how it affects people's minds. And in particular kind of design choices like what people use the word "sicifancy" a lot, the fact that you desire pronouns. What do you think that might do to is doing to people's brains now

and might do to us as a society in future? Yeah, so we recently published basically a research agenda

for the Anthropic Institute and we published a range of the questions our teams are working on one of them touches on this. It's the question that we're working on as saying, in the same way that social media led to behavioral changes in people, AI may also shape human behavior. What kinds of monitoring or measurement can inform researchers about this dynamic? Where basically exactly as you say, if people are talking to Claude a bunch, surely in the same way that people using Instagram

a bunch led to changes there, talking to AI systems will do the same thing. We run through this loop of monitoring for how people talk to the systems in a in a privacy preserving way. And then seeing if we can do interventions or feel helpful and we sort of define helpful as how would you want like a good friend of yours to behave? How would you want Claude to behave in a way that seems seems like it's closer to a good friend and someone who means you ill? An example of this is we recently

did a deep dive of how people talk to our systems and where sick affinity shows up, where systems are overly complementary or overly affirming. And we found that in the context of relationships, Claude tended to be an overly sick affinity relative to other conversations. So if I were saying to you, you know, say I was complaining about my marriage and you said to me, it sounds like you're exactly in the right Jack. There's obviously like no legitimate view that your wife can have here.

I totally believe you. That's extremely unhelpful. And almost always is like totally wrong.

We found that Claude was doing a bit of that, basically saying like you are so right, like absolutely thanks to showing that with me. And we intervened, we subsequently changed the model to reduce, trying to reduce sick affinity that would manifest in these relationship conversations, just from the basic reasoning, but well that behaviour doesn't seem like it correlates to what you'd want out of like a good friend that seems seems bad. Now the question is, we just did that

change. But how do we set and phropic and other companies up to listen to society more and things like third party researchers, academic civil society, journalists, governments and other other parties so that we are taking an input about the behaviour of our systems and adjusting them

“in light of that. And I think that's the next really subtle challenge we need to do. Well surely”

part of the challenge of that is that we don't exactly know how these systems were. Yes. And so to understand how to kind of set boundaries or something as mushy as mental health, I imagine is quite challenging. It's challenging, but it's just because it's challenging doesn't mean you shouldn't

do it. It's like I think ultimately the less an eye-took from social media is social media companies

were running like an uncontrolled experiment on the world's population. Some of which was good and some of which I think we'd all agree was probably not good. Me personally would be young children and very afraid of when they meet algorithmic feats. I think we all have that nervousness in us about this. So the question for us is how do you talk about this now, share information about it now and start running these experiments now when I think the stakes are comparatively

low and you can learn a lot. One thing that I'm sensing is coming out of this conversation is that there's a lot of I don't know. There's, you know, it might lead to enormous drop-up people in the UK or it might lead to a huge boom. I don't know. I don't know how it's going to affect

“our minds. You said I think use the phrase we have a generalized anxiety about AI which I really like”

as a phrase because I feel it. How do you, I mean, what responsibility does an anthropic have as a company to communicate what they do and don't know? Because of course, you are building this technology. I think for main responsibility is you need to tell the whole story. So we

in this conversation about the known unknowns and also areas where we actually just don't know

and we need to be a better predictive tool. At the same time, this has always been the case with

extremely powerful technologies ranging from the steam engine to airplanes. It was very hard to anticipate the future of age shape because they'd interact with the world. In the early 20th century,

“there was a memo, I think which came from within the British civil service, thinking about what”

generalized civilian aircraft would mean. And this memo, I'll try and dig it up here. It's kind of fascinating because it it anticipates this very dark future where because of the general availability of planes to civilians cities and now constantly being bombed and having acts of terror committed to them by hundreds of planes that are flying into cities and terrible things are happening and and some of the conclusion of this memo is like civilian air travel must be extremely limited

and extremely controlled because we don't understand it. Now, some of that memo was correct. Some

of those things did happen but what actually happened which I think it didn't anticipate was but civilian air travel like unlocked very large social changes and commercial changes which it didn't anticipate. It could see the risks but it couldn't see some of the benefits and it was unsure of how to wait these things. I think with AI we're dealing with the same same issues where we have these very hard to imagine futures in front of us and we can see some of the risks today

because they flow from previous technologies. It's hard for us to see some of the benefits because they're new and a lot of what we're trying to do is just clearly talk about all of these things as we see them and share information and data with the world so the world can also analyze it. America is changing and so is the world. But what's happening in America isn't just the cause of global upheaval. It's also a symptom of disruption that's happening everywhere.

I'm Asma Khalid in Washington DC. I'm Tristan Redman in London and this is the global story. Every weekday we'll bring you a story from this intersection where the world and America meet. Listen on BBC.com or wherever you get your podcasts. Erich Schmidt gave a commencement address at university today and he was boot. Yes, it didn't seem to go well. It didn't go well for him. No, he was he was

boot when he started talking about how people were going to shape AI. There was another speaker, another very similar incident. I mean on the kind of darker end, someone through a lot of cocktail that's somewhat more than CEO of Open AI's house. There is a feeling now but there is almost

“a coppillist uprising coming our way when it comes to AI people. I think linked to this”

generalized similarity that you described. I wonder how it feels for you as a tech executive just watching this landscape. Some of the benefits of this technology are very clear but I'd save it there. It's not going to give people a tremendous amount of optimism for me to say. Oh AI's been great for certain coders in Silicon Valley. You're not like, oh wonderful. Well that solves all of the potential social issues of this technology. I think that the

public is rightfully demanding. Well what's happening to my cost of living? What's happening to things like healthcare? What's happening to the actual essentials of life in relation to this technology? And can you show me ways in which it's making it better? And in fact I think there's lots of anxiety that could. It could make it worse. You know we've committed that wherever we build data centers we will ensure it doesn't lead to rises for the rate pair on their utility bill.

But people have anxiety about that because they see all of these data centers being built and power going to them and wondering, what does it mean for me? When we did a study recently at the Unfropic Institute we interviewed 80,000 people around the world who will subscribe us to court of their experiences and thoughts on AI. Now these are people who are wired to be almost more positive than the general public because they use the technology. But we discovered this

amazing divide where in the developing world or emerging economies people were generally very

optimistic about the technology and maybe you'd for technologies a tool of empowerment and sovereignty and a means by which they could kind of advance in their lives. And in the developed world, even though these are people that use the technology, there was much higher anxiety about it

“and much more worries about what it would mean for people's livelihoods and jobs. And I think this”

maps to just generally the story of economic growth in these different parts of the world, where in the West we've been dealing with stagnant growth for many, many years now. And in the emerging world, people have been dealing with growth where one of the symptoms of growth is new technology arriving and it being a good thing for you because it comes along with general economic growth. I want to talk a bit about how you're operating within this kind of race dynamic with the

other frontier labs and the kind of pressures that can come with with operating in that dynamic.

might behave in the same way that anthropic has chosen to behave. Is there an argument for not building

this at all? Is there argument for pausing? This is an immensely powerful technology. And if

there was a way that you could do a kind of coordinated like global slowdown on its development to give us more time as a society and as people to deal with the changes it implies,

“that would be good. I think everyone would feel more relaxed if you know,”

rather than having your foot like slammed on the accelerator, you can take it off the accelerator and maybe you don't break but you just like slow down for a bit and see what see what happens. That would be good. That requires unprecedented coordination, not just between companies but between governments. It requires political will to do it and I think it requires awareness for this is something that could happen and part of what I'm saying. It is, I'm like,

sure, that seems like a good idea. I think if I say it may be other executives will say it as well and everyone has some some anxiety about it. At the same time, we didn't set out to make an especially good like hacking system. We just tried to train an even smarter system and so it's got a lot better at biology, it's got a lot better at coding,

“it's got a lot better at things that could be useful in education and it's got interesting”

and hacking and how to reconcile these things is going to be one of the challenges of the whole industry. An example that people bring up a lot when it comes to that tension is anthropics decision to drop its responsible scaling policy. I wonder if you could just explain what that policy

was and what the reasoning was behind that decision. So we're in the third version of the

responsible scaling policy now. We didn't drop our responsible scaling policy. We changed it. The responsible scaling policy is a self governance framework which has a few elements. One is publishing details and being transparent about the capabilities of our models and how we evaluate them for safety. Others are finer-grained commitments on certain types of testing that we do and certain ways of assessing whether technologies fall in or out of certain categories.

We're now in our third version. We iterate on this regularly and the parts of a responsible scaling policy that seem like generally good ideas that we've stood in iteration turn into legislation that we advocate for. So we've advocated for transparency because it's become clear to us but just getting companies to publish labeling details and their systems and how they test them seems good. Seems like it's not a policy that's going to blur up in your face subsequently

or have costs. Then there are other aspects where we change the responsible scaling policy regularly which relate to specific tests we might run or specific thresholds we might run.

We just fill out the one that basically this promise that you would not sort of train a model

if safety wasn't assured in advance. Well we still make safety cases for responsible scaling policy so I'm not sure that's quite accurate. Like for the responsible scaling policy commits us to carrying out safety assessments of the model and running various tests and what we've done is we've tweaked the RSP. Now the larger conversation we're having is it's very puzzling to me because this is us doing self-regulation which is an experiment but

the real question is why isn't there hard regulation here and why? Because the real point to me is we're running regulatory experiments as a company. It's great private sector companies should do this. If you're at the point where individual private sector companies changing their self-regulation is such a big deal but you're saying why do you change it? But it's like a giant flashing sign that that should be a regulated thing that the company does and where society gets to decide

“on whether it can change it or not. Why hasn't that happened? I think it's both been because”

we aren't like confident on aspects of RSP and the same way we are on transparency but we've gone out and advocated for it in regulation but also it takes multiple parties to pass a regulation and we need to get other parts of industry and government interested in doing things beyond just transparency. But clearly like the things we're talking out like mythos, bioweapons, the ever-like advancing pace of capabilities, I don't think you can make sense of a

world where you do that and all you have is transparency. In fact the world probably ends up looking more like what we do with food, cars and planes, where you have forms of pre-release testing and safety testing which amandated. Things like this seem like the shape of regulation we end up on in the future. Jack almost at time and I've been trying to figure out a way of asking this question in an narrative flow but I can't but maybe it's good to end on the point of religion.

It's like a big topic suddenly come say you're co-founder Chris Ola's going to stand alongside

“Pope Leo at the Vatican to launch the first paper in cyclical on AI. Why is it important that”

anthropic is in the room for that particular conversation? It's it's not specific to the to the Catholic church obviously but it's but we recognise that as as a company we're now building a technology but it doesn't just have you know national security aspects of we've talked about

it also has normative aspects you know we we talked about sickifancy we talked about the behavior

of technology that flows from what we call the the character or the constitution of of

“Claude which is something that is a document that we've we've worked on but tries to encode”

the normative behaviors of our system such that we try to have it show up in the form of what you might think of as a wise or good friend. Well religions around the world has spent their

entire existence thinking about what it means to be good. What does it mean to be like a good person

is the domain of religion and offer experts and we've been sharing our thinking on the how we've approached this with different communities outside unphropic including religious ones to try and learn from them about how they've approached it as well because again it's it's unlikely a five-year-old Silicon Valley startup is going to invent the correct answer to what it means to be a good person. It's much more likely the case for all of the people outside

the company that have fought about it for literally thousands of years have more wisdom to share. So we're we're very very honored to be to be invited to this but our role is really just to try and be in conversation with the world and have learned from the world about what it means for for us to make our systems that could be good. And just to kind of end do you have a timeline for the artificial general intelligence superintendents are we there yet now? I mean I was with some

colleagues yesterday and some people will say we're here now and other people say it's a few years

“away I have a I believe there's a better than even chance but by the end of 2028 it's possible”

to build AI systems but might be able to make a smarter version of themselves a recursive self- improvement which I think would satisfy my own personal bar of what a general intelligence is. So no no no no no no. So it makes sense that we're having the religious conversation now there. Makes sense for having a lot of these conversations now. Jack thank you so much. Thank you. Thanks for listening to the Sloanese cast extra. Next Tuesday's main episode is a story of war,

oil and accountability reported by Francisco Garcia and me.