Scaling Laws: AI Consciousness with Anil Seth

No. Frank. [MUSIC PLAYING] It's the Law Fair podcast. I'm Alan Rosenstein, associate professor of law at the University of Minnesota. And a senior editor and research director at Law Fair.

Today, we're bringing you something a little different. An episode from our new podcast series, "Scaled Lost." It's a creation of Law Fair and the University of Texas School of Law,

where we're tackling the most important AI in policy questions.

From new legislation on Capitol Hill to the latest breakthroughs that are happening in the labs, we cut through the hype to get you up to speed on the rules, standards, and ideas shaping the future of this pivotal technology. If you enjoy this episode, you can find and subscribe to Scaling Laws wherever you get your podcasts and follow us on X and Blue Sky.

Thanks for listening.

“When the AI overlords take over, what are you most excited about?”

It's not crazy. It's just smart.

And just this year, in the first six months, there have been something like a thousand laws.

Who's actually building the scaffolding around how it's going to work, how every day folks are going to use it? AI only works if society lets a work. There are so many questions have to be figured out, and nobody came to my bonus class. Let's enforce the rules of the road. Welcome back to Scaling Laws, the podcast brought to you by Law Fair

and the University of Texas School of Law that explores the intersection of AI policy and, of course, the law. I'm Kevin Fraser, the director of the AI Innovation and Law Program at Texas Law, and a senior editor at Law Fair.

Today, we're joined by a Neil Seth.

A Neil is a professor of cognitive and computational neuroscience at the University of Sussex, where he also directs the Sussex Center for Consciousness Science. Today's conversation starts with a question that sounds like science fiction, but increasingly poses real scientific and policy challenges. Could an AI system be conscious?

“Will work through what consciousness means the strongest arguments on both sides of the question of whether AI is indeed conscious?”

What evidence currently suggests and what would cause a Neil to change his mind? It's a great episode that I think you'll thoroughly enjoy. But before we get underway, one programming note, we're going on the road. We've been in touch with students at Harvard, Emory, the University of Texas, Cal State, Northridge, Nebraska, and others. If you're interested in hosting a live recording of scaling laws in your community, get in touch.

We're eager to make sure that more folks are involved in these very important debates around AI policy and legal discussions. And touch with us email [email protected], or follow us on ex or blue sky. And with that, get it for a great show. Neil, thanks so much for joining scaling laws. Thank you Kevin to pleasure.

“So I was tempted to start this podcast by asking what I think every podcaster would start with,”

which is, please define consciousness. But then I was doing my homework and you said, anyone who starts with that definition is going to find themselves in a sort of straight jacket, because as soon as you try to define the term, you're already testing things that aren't going to lead to a fruitful conversation. And instead, consciousness has to be explored through so many different angles. And so I wonder if you could just touch on why it's so important to resist that temptation to fight this on sort of

definition of grounds from the outset. Well, I'm going to slightly maybe contradict what I might have said before then.

I think the point is that at this stage of our understanding of the phenomenon like consciousness,

definitions can be a bit of a rabbit's hole.

“They're important so that people don't talk past each other.”

And they are kind of important because they clarify some of the, of then the reasons why might people think that certain things are conscious versus versus not. So for instance, if you, if you define consciousness in terms of having some like in a narrative,

some stream of thoughts, linguistic stream of thoughts, you might reach a very different conclusion about what kinds of things are conscious,

then if you take a more like perceptual or sensory view that it's like having it's the redness of red or the taste of ice cream, something like that. So it does matter. And for my money, the best definition that is just, it's so broad. You might even think it's a bit tourty logical or a bit redundant, but I still think it's useful. And it comes from Thomas Nagel where he said, for a conscious organism, but we can think of anything, a conscious system,

there is something it is like to be that system. What does that mean?

And you can unpack it, but basically it means it feels like something to be a conscious system.

Which again is like, well, what does that mean? It just means that there's some kind of experiencing going on, which is where the totology comes in. But the reason I like it is that it doesn't confuse or confound consciousness with other things that might go together with consciousness in us, like language or intelligence or rationality or whatever it might be. It rather just says, for some things in the world, experiences are happening.

And for other things, they probably aren't, you know, whether it's a table or a telephone or maybe a bacteria, there are all sorts of interesting grayers.

“So that's the definition I like. You can't do very much with it, but I think it helps keep the discussion honest.”

Right, and I love that idea of there's an experience of being a dog, right, that we can presume, but we may not understand, but to your point also, what makes the study of consciousness so difficult is that, unlike other fields of inquiry, where we are looking externally at something we can measure, consciousness in many domains forces us to look internally, which is necessarily subjective. And so folks often are debating this term and this concept from a philosophical standpoint and from a scientific standpoint.

And I think those are different approaches that lead to different types of understanding and can you walk through why embracing the science of consciousness and recognizing that as with other questions in the universe, such as, why do we even have a universe, we may just not have the scientific tools necessary to fully understand this in gray and that's okay. Right, I think that's a good point. There's the sort of, well, always has to be careful in thinking about developing a scientific understanding of something, to not fall into what some people called "scientism," which is, you know, the view that basically everything has a scientific answer and the scientific answer is the best possible answer.

Maybe there are some questions which will lie beyond the scientific method. It may be that consciousness is one of those, but my view is that there's no reason to think that is the case from where we are now and let's see.

“I think that this is why we need both science and philosophy. It's a complicated question scientifically in part because the brain is so damn complex and we know the brain has something to do with consciousness or at least it's related in some way.”

And it's philosophically challenging both at the level of what philosophy is called metaphysics, and it's a more general question of how on earth can we even think of a conscious experience in terms that's continuous with our more general picture of the universe, which is, you know, particles, waves, quantum foam, whatever it might be, how do these things relate? That's the metaphysics of it. But there's also, you mentioned the practicalities and that's also challenging, but it's important to separate those two problems.

So there's the ontological problem. What is consciousness? How does it relate to stuff? Well, the other way around. What is stuff and how does that relate to consciousness? That's the other way of looking at it. But then there's a methodological side, the epistemological side, how do you find out? And this is the challenge that conscious experiences are by their nature intrinsic, they're private, they're subjective, you can't put the conscious experience on the table and look at it.

And I want to dive fully into some of those scientific challenges, but first I want to get to something we were talking about just before we started recording, which is in the AI space, everyone is now familiar with the idea that there have been so called AI winters where suddenly studying this topic wasn't as hot wasn't as important or wasn't as popular as it is today.

And actually you have pointed out that you studied consciousness during a consciousness winter when we weren't having podcast after podcast after podcast on consciousness.

And now we're experiencing to use your excellent phrase a consciousness heat wave, which is a bit of a loaded term because heat waves definitely have some negative consequences as unfortunately you all in Europe are experiencing. Why was there a consciousness winter, what does it mean to be in a heat wave and and why does that carries some potentially negative outcomes.

“Well, you know what actually I think when I was talking about being in a heat wave, I was more referring to what's happening in AI.”

Well, okay, happening in consciousness. And in fact, my my PhD 25 years ago was in AI during an AI winter. So I've got to get you a parker and multiple wins.

And I think at that time it was it was actually maybe the first signs of spring in consciousness researched because consciousness research had been very much in in a kind of frozen frosty midwinter for a long time with little pockets of activity. But it had been generally considered not really the core business of neuroscience psychology cognitive science, any of these things. In part because of the of the topics we just mentioned, you know, the idea that well philosophically it seems very challenging to even know what the question should be or how what the answer might look like but methodologically it's also very hard to to to study and the whole trajectory of the 20th century had been a kind of.

Rejection of that with behavior isn't in the early 20th century all the way going slowly back to a recognition that well if we really want to understand the mind and the brain.

You know, we can't continue to set consciousness to one side just because it seems tricky and I think in the 1990s things really started to change so when I finish my PhD in the early 2000s things were happening and I think I was very lucky with that timing. Certainly wasn't flourishing but the science of consciousness has been steadily growing.

“It still feels a bit precarious but now we now have conferences and journals and and I think if you just I was writing a review article with some colleagues last year just looking at the last 30 years and you know what the next might bring.”

And I think there has been a lot of progress in consciousness research I don't think we understand it yeah but we know a lot more collectively than than was done 30 years ago. And it's becoming more relevant topic in other areas whether that's in medicine whether that's an animal welfare and of course we've seen this explosion of interest in AI and people have been talking about AI in consciousness for a very long time. I remember writing a paper on it in 2009 but these debates have stayed very local very perocule and it's really since the the arrival of language models and chat bots that the question of AI and consciousness has become.

“Not just part of the public conversation but something that seems to already be shaping technology development and regulation and policy in all these things.”

And on that AI consciousness nexus obviously this is the thing that's really blown up in recent years in terms of a field of inquiry. And you've been on the record although I want to highlight and praise you for specifying that you hold your views lightly and you are always open to being subject to alternative forms of evidence to change your views and give yourself that space. But as of now you see it as highly unlikely that silicon based systems are going to be conscious. Why is that your posture let's start there and then we'll get to what evidence would start to perhaps change that view with any.

Yeah I think that that's fair so I think as things not that whole my views I feel very strongly about it actually but but the you know I think.

We don't have an established understanding of either the necessary conditions or the sufficient conditions for something to be conscious.

“And and there are even many different metaphysics you know maybe consciousness is everywhere that that also you know people can I I think consciousness is a property of.”

Matter as not clear what matter is and but it's more resourceful maybe than people give it credit for anyway. Given all that I think where we are now. Yeah it's extremely unlikely. That digital silicon AI can have conscious experiences.

And I don't see like the next frontier model as moving the needle on that why is that well maybe we probably return to each of these points but I think there are basically three.

broader reasons why one is. That we are own psychology.

“I think we need to be disposes as to over attribute consciousness to certain things we're biased to project consciousness into systems.”

That seem to be like as in particular ways in particular intelligence and language right. So we know we're conscious we think we're intelligent and we associate language with some kind of human exception as in what's made it's what one of the things that makes a special. So when we encounter language models specifically that talk fluently to us. We're. It's not so surprising that we project consciousness into them especially given that they're trained on everything that human to ever written.

A lot of which is either explicitly or it's certainly implicitly about consciousness it's really not surprising that language models you know if they're learning something about the deep structure of. Of human language you know that will imply the existence of a conscious mind behind it so they're very. At least in some sense they're very very deep mirrors that are reflecting an image of ourselves back towards that we are predisposed. To sort of find appealing and gaze into like narcissists gazing into the reflecting pool so that's one reason.

The second reason to hang on that first reason for second because elsewhere you've mentioned and forgive me if I butcher this they say that those who can't do podcast and that's very true in my case so if I get this pronunciation wrong. But there's this phenomenon of paranoia. Where's paranoia. I have so close paranoia. Thank you.

I mean that's probably wrong too. I can't speak Greek anyway. We're all trying. You know this phenomena of when you go and you look at this guy and you see a face in the clouds.

“As you pointed out and something that I think is incredibly astute and fascinating is that's an evolutionary process of trying to identify.”

And seeing consciousness and seeing others as something that may have been evolutionarily beneficial and I think that's a unique feature of your view of consciousness because of that tie and you have with. It being a biological property. It's something that does exist perhaps on a spectrum as you pointed out that requires hey, maybe having that sense of identifying when and if something is conscious. Can actually be evolutionarily beneficial and I just think that's a unique way of viewing it to seeing how it can be an advantage to be able to distinguish when and not something is conscious.

And the other part of it is that that this is just a very fundamental way to how our perceptual apparatus works. Whether it's, you know, the level of perceiving conscious minds or the reason the parodolia term is used for me. That just means seeing patterns in things. So when we see faces and clouds, that's a form of parodolia. And again, it's useful because the ability to recognize faces and detect them is very important for human beings as social species. When we are just looking around and just perceiving the books on the shelf or you are on the core.

The theories of perception, but I think the most convincing these days that I've worked on.

A lot too is that the brain is always in the business of making predictions about what's there and then using sensory data to update those predictions.

It's always in the business of projecting its best guesses into things. Well, projecting predictions and then coming to a best guess. So pattern recognition is fundamental to perception.

And that instinct to try to identify and associate that consciousness and that bias towards identifying consciousness carrying over into the AI domain is I think something that will dive further into. But seeing how much of our own expectations of consciousness lead to the perception of consciousness.

“I think is a big question that has to inform how we talk about it in the AI domain.”

But before we get further down into that first rabbit hole, you have two other prongs you wanted to touch on.

Yeah, the second one is is a more fundamental objection because to say that we are predisposed to these false positives, doesn't really tell you why I think it's impossible in principle. Like why we might be overestimating. The stronger more fundamental reason is that the possibility of AI consciousness depends not 100% but to a large extent on a very deep assumption, which is that the stuff doesn't matter. The stuff we made up doesn't matter. What matters is the computation or the algorithm. In philosophy, this is called computational functionalism and it's the idea that the computation itself is sufficient for consciousness, whether that's implemented in carbon or in silicon.

“It's a very, very commonly held assumption. And I think if that assumption is true, then conscious AI is much more plausible than I currently think it is now.”

So not to leak too far ahead, but that is something that would change my mind about the credence. If there were good reasons, if I was convinced there were good reasons to believe that computation of the kind that can be implemented in silicon is sufficient. Then sure, it might not be the next GPT or the next Claude, but you know, you could figure out the right computation somehow by looking, you know, what we do and you could implement that in silicon.

The problem is that I don't think computational functionalism is true. I don't know for sure, but I think there are very good reasons to be very skeptical and if those reasons stand up.

Then that really is an alien to cough in for AI being conscious because AI systems would bear a very similar relationship to if you like computational simulations of digestion of the weather of the stock market. Whatever it is, they're not the thing itself. It's a computational model rather than the sufficient basis for instantiating a phenomenon. And on that prong, thinking about, for example, some of the experiments you've done and others have done pointing out that people have the sort of Phantom limb type experiences, people have instances of hallucinating on their own and developing alternative perceptions of reality that may not be grounded in fact, but it's just the way the mind's working.

It's hard to map that just strictly onto the metaphor of the mind as a computer or carrying that metaphor a little bit too far. And as you and others also have pointed out, this seems to be kind of the go to metaphor of the day is to just say, oh, well, the mind is this just system of neurons firing. And neural networks are behind the LLMs of today. And so why can't we just bind those two concepts together? And yet there are these instances where we're still struggling to explain why the mind is operating presumably consciously in the way that it is that is way harder to map back on to silicon is that a fair understanding of some of those arguments as well.

“I think that's a really good way into to that topic that I'm glad you mentioned the word metaphor there because that is I think what is going on.”

The computer is a very powerful metaphor for what a brain is and like all good metaphors it's got some truth to it as well.

And and you mentioned about neural networks and I think that's that's the camera of truth within this otherwise potentially misleading metaphor that. Yeah, I mean, our brains are many things that electrical machines that chemical machines that dynamical devices. But in some part there's a bunch of neurons that are connected to each other. And if you kind of forget about the rest and just think of it like that. Um, then work from way back in the beginning of computer science.

They were what computer scientists called touring complete. You could it run any algorithm in principle on a neural network, which an abstract neural network. And versions of these abstract neural networks that physically concretely running in the GPU servers that power the whatever AI for.

That's the direct descendants of this idea. Then neural networks they can implement anything and we see that they're very powerful. This is great.

Uh, so there are reasons why you think, okay, that's that's all there is to brains. That's all they do. So it really doesn't matter if they do all these other things if they're a chemical switching around and so on. That's just like detail implementation detail.

“You know, and I think that is the strongest argument for AI consciousness. Like, well, that might be right. Maybe that is all you need.”

Um, and maybe the stuff doesn't matter, but I think the closer you look at real brains, the less and less plausible that otherwise nice sounding argument sounds. Um, it's, you know, I mean, you mentioned a couple of reasons like we are not just, uh, sort of brains that. You know, existing in in, I don't know, like, server cases, which happen to be our bodies. We are fully embodied in bed at creatures. You know, we're coupled to our environments in continuous time. To have a mind is to, it's some level at least to be a couple to a body and to a world and therefore to other minds and other bodies as well.

Um, I mean, even setting consciousness aside, this lack of deep embodiment is one reason why AI systems like language models might not even understand things.

Because for us, you can make the case that true understanding, whether it's conscious or not, comes ultimately from the words we use being physically grounded in a real world, whereas at a lens,

especially doing their training, even if you plug on into a robot later on, it's doing the training that counts. Um, they don't directly, you know, they're not exposed to the physical reference of the tokens that they process, you know, they just get this like pre-processed world of tokens.

“So that's, um, that I think is important, but then even within the brain, the more you look at it, the more you realize that it's functional organization,”

it's dynamical organization, really does a lot more than touring computation algorithms allow algorithms very simply. They take a bunch of numbers or symbols and they map them to another bunch of numbers or symbols in discrete, discrete symbols and discrete time, various other things to recipe. Our brains do all kinds of other things, electromagnetic fields, neurotransmitter chemicals. There's no sharp separation of mindware and wetware.

All of these things make it less and less tanable.

That what's going on is simply mapping numbers to numbers in a way that you could transplant onto, onto silicon. I can't believe it. I can't believe it. I'm still flexible, I believe. Oh, just an old trauma.

“With the mobile function in real life, no alterations.”

Frank, from the beginning of the unimindest contract, it could be, for 10 euro immunatomic. Frank, on to X-Cathons, check, internet, email, check, new address, check, and then a trauma beta. Between Kisses, Umadung and Möbel Aufbau, the Strom contract is fastened. Then you can find the automatic information in the basic information that is often worth reading. By Octopus energy, Vexels do uncomplicated so-fair and pheisen.

Yet, of Octopus energy, the E-vexlin, and with the bonus code, Octopus 1's 1's 5's. And 115 euro Vexels bonus session. Your point too about the kind of lived experience of consciousness. Something that stood out to me from one of your other interviews. You were talking about how every 12 years you participate in the largest religious ceremony in the world,

going to the Ganges and taking a bath with millions. Or at least hundreds of thousands, if not millions of us. Oh, no, no, definitely millions. Definitely millions. Okay.

That's amazing. Yeah. And that own experience of the fluidity, literally, of being part of this human collective in a way that

But I'd love to hear more about your own experience and how that, how you use that very transformative opportunity to think about something like consciousness.

It's interesting. Yeah.

“Yeah. So this festival, it's called the the Kummala. And it happens every year in another bad in India, but every 12 years, it's special.”

And every 12 years, it's reliably the biggest gathering of people in history, because the population of India just keep going up.

And there's a few days in this festival where it's particularly suspicious to take a bath in the Ganges.

And so on those days, you can get tens of millions of people. In a relatively small place, or with a singular purpose, which is to jump in the water. And that can be a very spiritual meaning for that experience. And one thing I've noticed participating in that is that indeed, you lose your sense of individuality a little bit.

Because you know, you're part of this river of people, you actually, you have less agency on what you do.

And you feel different, you know, you sense of self, my sense of self feels a bit different too, because of that, the diminution of the sense of agency. And I think one thing to take, of course, you still conscious, you still see things, and it's somewhere in my mind, you know, have an internal monologue going on as well. But what I think experiences like that do is show that the way of experiencing the world that most of us, especially in the West, kind of go about our daily lives, navigating is just one way of being conscious.

“And we may get ourselves into trouble if we think that's the only way of being conscious. My colleague David Chambers has written a beautiful paper on, what do we talk to when we talk to a language model?”

Because it's really not clear, you know, in fact, it might be, again, setting aside whether it's conscious or not, it might be structurally a little bit more like a crowd rather than an individual person. You know, you don't talk directly to the foundation model, there's, you know, that's a set of weights somewhere in the US. An instance is something that gets spun up every time you type of prompt. So is it that well, probably not, because, you know, you can set aside a conversation come back to it a week later, and it kind of carrying on what's happened in the meantime.

It's not that this instance of the language model has just been waiting for you wondering where you've gone. So Dave comes up with these idea of threads, you know, something slightly in between an instance and a foundation model. And it's very interesting, I don't know quite what to think about that, but I really like the question that. We can, it's another aspect of parody, when we're talking to a language model, we, we not only project mind and consciousness into it, we sort of feel like we're talking to an individual.

And it's very, very hard to sustain that, you know, and say, well, how do we make sense of that given what language models are, it's almost impossible to make sense of that.

“Again, it reveals how strong our biases can be, and if we can hold up hold these biases a little more lightly, then I think we've got a better chance of, of identifying consciousness.”

Where in unusual cases, but also of not falsely attributing it based on those biases. And we still have pronged three to get to, and I promise we will, but I do think it is fascinating to dive into some of the experiments that have been done in the space, for example. The infamous or famous rubber hand illusion where folks, you know, are told to stare at this other hand for a while that's a fake hand that is then, you know, touched or glazed over with a feather. And they experience that feather on their own hand, even though it's not occurring, and just the fact that that sense of consciousness mapping or carrying over and mirroring into our own context is possible.

To your point, the ability to map over what feels like should be a conscious interaction or an interaction between two individuals, even. It's unsurprising that's occurring on the frequency with which we're seeing people feel as though they are indeed having that sort of encounter. Yeah, I think, all of these, though, it's fascinating experiments with my colleagues at Sussex, Pete Lush and so on.

Like, oh, I feel like that hand is my hand.

“That's strongly predicted by how broadly speaking, how hypnotically suggestible these people are.”

And that's interesting because that's a fairly general trait that we all have, it's stable, it's reliable. And it can be kind of understood as how open we are to things like parodolia. Now, I don't know if it holds up yet for faces in clouds, but basically people who are highly hypnotically suggestible, it's not just like going to a stage show with Darren Brown or whatever, what I mean is that. People high in this trait, variable, their brains can create experiences more easily on the basis of cues of context of expectations.

And so that might be something that actually plays into this AI consciousness thing as well. So there may be a lot of individual variation in the extent to which people feel that AI is conscious.

“I think, yeah, a lot of the AI firms people are, and I'm glad they're doing this, they're looking at their own models and trying to see whether there's any evidence from within their own models that these things are conscious.”

But that's only as we've been discussing, like, one third of the story, you know, we also need to look at our own psychological biases.

And we need to look at the human brain too, because that is a system where we can all agree, you know, it is something about the embodied embedded human brain is capable of consciousness. It may be that it's the computational level alone, but there are many other options and that's bringing us to the third prong of the argument.

“Let's do it. Let's go there. I will, though, say that from a policy standpoint, it's fascinating that different individuals will have different sorts of consciousness experiences in broad terms with anything else, right?”

People are on a spectrum of their own willingness to identify consciousness, which raises profound policy and legal questions, because how susceptible you are to seeing that sort of consciousness and bind into all of the things that may be attached to having a discourse with another conscious thing. unpredictable, because at least I don't know my own hypnotic subjectivity, for example, I'm very curious, maybe I need to go take some test, but that's not necessarily known apriori to anyone. And so designing policy around that is really difficult and really fascinating.

Again, just before we move on from that then, so just to be clear, I don't know if any evidence, maybe there is, we haven't looked, I don't know if anyone else has. Whether there are any correlations between hypnotic, suggestibility and the tendency to attribute consciousness to AI, I don't know whether that's the case or not.

But what is the case, is the first part of that for sure, which is that we do each differ in our conscious experiences of even if it's of the same shared objective reality.

There's other large scale projects I've been involved in for years now, we call the perception census, and we're really measuring in many different domains of perception, whether it's vision, hearing, seeing faces and clouds, all of these things, how much people differ. You know, the analogy we're still doing the analysis, but done the lying finding is that it's very similar to how we all differ in our height or, you know, or in any other trait, there's a bell curve quite often, as a distribution, everything in nature is always a distribution, and that's just as true in like our experience of color or of time passing, as it is in our heights, where IQ or whatever else it might be.

So I do think it's important for all, and this is the very policy sensitive observation anyway, that that people are different, and we differ on the inside, just as much as we differ on the outside, and it's not also, it's not just the case of identifying, you know, so called neuro divergent conditions. I think if we focus on that, we overlook, and even deeper truth, which is that we all differ, which is why I use the term perceptual diversity rather than neuro diversity, which is typically associated with, with specific conditions, I think those are important, but they're only part of the broader distribution anyway.

The third thing, the third general reason for AI consciousness skepticism on my part, it's related a bit to the second one about the brain, not being a computer or not being just a computer, which is that there are other games in town.

And what we need to know is what are those things? What's the right level of description? If we make AI and we start making AI more and more brain-like, not just in terms of simulating a brain, but literally more and more brain-like. We start building it out of actual neurons, and then we grow a brain in addition, at some point, we will have something that is conscious, because ultimately we'll be having a child or something, you just have another human, have another brain in a body in a world, and so you'll by definition have everything you need.

So the question is, what aspects are necessary and what are sufficient? And this is where I think the action ought to be, and this is where my own work is focused.

There's a specific hypothesis here, which might be wrong, but it's the one that I feel motivated to explore, which is that consciousness is a property of life rather than of computation.

“In philosophy, this is called biological naturalism. It's the idea that life is necessary, but not necessarily sufficient for consciousness, so not everything that is alive has to be conscious, but the idea is that life is necessary.”

Why would that be? That's a very long story, but it starts actually with how perception works in terms of making and updating predictions, and then pulling on that thread as far as you can, and you find out that actually, oh, the reason brain's making update predictions can be traced to how they regulate the physiology of the body, how we stay alive, and that imperative to stay alive.

It doesn't sort of stop at any particular level. It goes right down into the furnaces of metabolism and the self-producing nature, the so called autopoeetic nature of our cells.

For me, there's a direct line from things that are fundamental and distinctive about life that just aren't present in silicon, and the machinery that underpins each and every conscious experience that we have. So the last thing I'll say there is, I'm not saying this is a kind of vitalism or carbon-shavenism where there's something magic about life? No. Life is just a very specific and distinctive kind of thing, and I think if we understand the relationship between life and mind better, then we might find it more natural to think that consciousness could be a property of living systems.

“Perhaps that will in the end just not only help us distinguish whether AI is conscious or not, but actually make the whole problem of consciousness a bit less mysterious.”

If we think of consciousness as something a computer can have, then of course, it's mysterious, like why should just manipulating numbers give you an experience, it's like very, very puzzling why that might be the case.

If we think of consciousness as more continuous with life, then at least in my mind the sense of mystery about why consciousness is part of our universe at all begins to reduce.

That's for me the way the way to think. It's not for sure. We have a few like these two hypotheses. One consciousness is all to do with computation in algorithms, and that has its appeal because the metaphor is a powerful metaphor. And the other is no, it's the stuff matters, and consciousness is tied in some intrinsic intimate ways to the materiality of life. And it's important, I would say, it's really urgent to find ways to interrogate and distinguish between these two hypotheses, not simply argue about them in terms of their philosophical merits.

But actually begin to do experiments to address whether the stuff matters. And of course this invites the so-called paradox of the heap, which is to say, okay, if we started to find some living component and build it into an LLM or otherwise be able to develop AI, let's say, different architecture, different training process, so on and so forth.

“But does have some living component? At what point, as we add more and more living components, does that consciousness threshold get satisfied?”

And is it a binary conscious or not conscious, or is there a spectrum of consciousness? Because as you raised elsewhere as well, you know, when we're under some form of anesthesia, you're kind of passing in and out of consciousness.

And what would that mean?

As we start to perhaps experiment with this more integration of living material and silicon, and what that means overall for some of the rights determinations that people have started to talk about, for example, of if AI is conscious or even if it's perceived to be conscious, perhaps that requires. Some legal or policy response.

“Yeah, I mean, there's a lot in there. Firstly, I think consciousness is not binary. I think it's, you know, whether it's a human being like you or me losing consciousness under anesthesia or during sleep, I think it's graded.”

But at the same time, I think there might be clear cases where you either clearly are conscious or clearly are not.

And profound general anesthesia might be the closest to clearly or not, but you know, that's compatible with that being gray air is for sure, or it being continuous. Just like, you know, when the sun sets, there's no single point at which it becomes night, right, but at some point is clearly night and other points is clearly day.

“And the question, yeah, the question of how much life is enough, let's say, you know, we introduce, of course you can do both ways.”

You can have a human brain. You can begin to replace it with silicon components. And that's actually probably, you know, more aligned with what's actually happening now with the neurotechnology firms and brain implants and all of that stuff.

But you know, you can go both ways and the same question arises how much life is enough life. And I don't know. I don't think, I don't think it's possible to know that.

But I think it's the right kind of question to ask, you know, my intuition is that, yeah, if you, if like we know, for instance, some parts of the human brain don't seem that relevant for for consciousness, like the cerebellum, the bit at the brain at the back. Three quarters of all your neurons are that, but it doesn't seem much involved in consciousness. So potentially, you could replace the whole cerebellum with a, with a deep neural network. And actually the, the wiring of the, of the cerebellum is very reminiscent of the kind of deep networks trained by back propagation that we have in AI. So it's, in fact, maybe not such a silly idea, but that probably wouldn't change our consciousness because it's not much involved anyway.

So near the neuroscience of consciousness can give us some hints about which biological structures, which biological processes might matter more than others. And then the final point, yeah, all of this stuff is is very significant for policy and regulation. And it's very important here to distinguish these two questions which we've been doing implicitly.

“The one is, is the thing actually conscious or not. And there's just a lot of uncertainty about that. I think, as we said, it's very unlikely that standard AI is.”

But there's this other question of what about AI that seems conscious. And we're already there. I mean, there's no philosophical uncertainty about that at all, right? People think it is, which are Dawkins thinks it is, many other people that think it is. And it's too easy sometimes to get distracted by this kind of like, you know, that it's just such an alluring question to discuss is it actually conscious. And that we sometimes forget that we should equally focus on the situation that we know is here and now many people, whatever you and I might be talking about, whatever the conclusion of a bunch of academic somewhere might be.

Every day users of chat bots may very well just come to believe that they're talking with conscious entities. And that has a lot of social consequences. And I think we need to be very concerned about that I think they're generally bad consequences. I think they lead us to be more psychologically vulnerable and perhaps to, you know, make it harder for us to form real social relationships with people, we form these parasocial relationships with with AI systems. And we may also complicate things like the alignment problem in AI because if we feel that these systems are conscious, there are already movements that are saying, well, we should care about the interests of the AI themselves, like, you know, keep a copy of the weights of deprecated models and in the limit, maybe we wouldn't be able to turn off the AI systems because it would be, you know, would be bad for them.

Absolutely murder. And I think that would be catastrophic for our attempts to align AI with human and planetary interests. For fairly obvious reasons, we've already seen in the last two or three weeks front-tier models doing all kinds of business, you know, going and hacking into other companies and so on. And they're doing this without, you know, without, let's, let's give the benefit of the doubt. I don't think anyone was trying to behave badly here, yet nonetheless these things can happen.

So we're likely to, to even more strongly project consciousness into these things, precisely the time when we should remind ourselves that they are not, or very unlikely to be conscious, in order to preserve ourselves the ability to control and regulate them in ways that are increasing the urgent. And as a, as a final question for you, because I, you've got very important research to, I am very worried about the brain drain of seeing more and more expertise and more and more intellect be concentrated by these labs and doing this frontier research across all of these domains.

“And so if you had an ask of the AI labs, what data or what resources or what information would be especially important to you and your colleagues to be able to sustain this research?”

So, you know, I've met a fair few people working in these these frontier labs and they're all great. I mean, I think they all want to build things that are useful and safe and, and so on it's just a technology that's really accelerating it's very hard to figure out. Some of the research teams in the frontier firms are doing what comes naturally to those companies which is to study the systems that they themselves have built and to look for signs of, you know, consciousness within them of course it's all conditioned on this idea that computation is enough.

But basically a lot of the consciousness research going on in the frontier labs is studying the frontier models.

“My kind of plea to these firms given that almost limitless resources it seems sometimes is if they really care and I'm sure they do so like if they really do care about whether their systems are conscious or not.”

Then don't just look for the car keys where the light is, don't just study your own models, we the more we know about consciousness where we know it can exist in in us and on human animals, then we'll be able to be on much more stable ground when we want to make inferences about whether it does feel like anything to be clawed or GPT.

So that's my my plea it would be yeah it's great that they're taking consciousness seriously, but they also need to take seriously the study of consciousness in in us.

No last thing was like and people like this stuff are so them and I think say this a lot I do think it's a bad idea to deliberately try to create consciousness in AI I mean that should not even need saying I think it's a terrible idea to try to build systems that it conscious.

“They would have their own moral status and we would not be able to control them and they may suffer and all this stuff, but I also think it's a bad idea to just keep building systems that give the strong impression of being conscious.”

This is a very hard design problem to solve because of the nature of the training data, but I think it's a problem that doesn't need to be addressed because the the dangers of seemingly conscious AI are you know they're very they're very significant and.

Yeah and that's something that in a sense is easier it's not an unsolved philosophical or scientific problem it's a question of.

In sensitive and design and to knowing as you pointed out that we're in such early days of seeing how that perhaps suggestion of consciousness is impacting users engaging with AI and already we have seen documented instances especially among some folks who are experiencing mental distress. And to the extent that we facilitate that in any way shape or form raises profound moral and policy issues so. Anil you've given homework to me to the labs it's excellent your fantastic professor. Thank you so much for joining scaling laws, but we'll have to leave it there.

Well thanks so much to spend a pleasure to talk to you. scaling laws is a joint production of law fair and the University of Texas School of Law. You can get an ad free version of this and other law fair podcasts by becoming a material subscriber at our website. Lawfaremedia.org/support.

Please rate and review us wherever you get your podcasts.

“Check out our written work at lawfaremedia.org. You can also follow us on x and blue sky.”

This podcast was edited by Nome Osband of Go rodeo.

Our music is from alibi.

As always, thanks for listening.