Michael Nielsen – How science actually progresses

You've done many things. You're one of the pioneers of chronic computing, wrote the main textbook in the field of the open science movement. You're at a book about people learning that Priscilla and Greg Brockman credit them with getting them into the field.

More recently, you're a research fellow at the Ster Institute and writing a book about religion science and technology. I'm going to ask you about none of those things.

“The conversation I want to have today is how do we recognize scientific progress?”

And it's a special element for AI, because people are trying to close the RL verification group on scientific discovery. And what does it mean to close that loop? But in preparing for this interview, I've realized that it's a more mysterious and elusive force even in the history of human science

than I understood. And I think a good place to start will be Michael's and morally in how special relativity is discovered if it's different than the story that you kind of get off of YouTube videos. Anyways, oh, frankly that way.

And then we'll go in there. OK, yeah. So Michael's and Molly is one of the famous results often presented as this experiment that was done in the 1880s and that helped Einstein come up with a special theory of relativity a little bit later.

So it's sort of changing the way we think about space and time, and our fundamental conception of those things. And there's kind of a big gap between the way Michael's and and morally, and other people at the time thought about the experiment, and certainly the way in which Einstein thought or did not think about the experiment

in actual fact, he stated later in his life. He wasn't even sure whether he was aware of the paper at the time. There's a lot of evidence that he probably was aware of the paper at the time, but it actually wasn't dispositive for his thinking at all. Something else completely was going on.

So what Michael said and Molly thought they were doing was they thought they were testing different theories of what was called the ether. So you get back to the 1600s, Robert Boyle introduced the idea of the ether.

And basically the idea of the ether is, you know, the sound is vibrations in the air,

and then Boyle and other people got interested in the question of like, is light vibrations in something? And they couldn't figure out what it was Boyle actually didn't experiment, where he tested whether or not he could propagate light through a vacuum. He found that he could, he couldn't do it with sound.

So introduced this idea of the ether, and then for the next 200 or so years, people had all these kind of conversations about what the ether was, and what its nature was. And the Michael said, and Molly experiment was really an experiment to test different theories of the ether against one another,

and in particular to find out whether or not there was a so-called ether wind. So the idea was that the earth is passing through maybe this ether wind, and if it is passing through the ether wind, sort of this background, and you shoot a light beam sort of parallel to the direction the ether wind is going in, it'll get accelerated a little bit, and if it's being passed back,

sort of in the opposite direction, it'll get slowed down a little bit,

“and you should be able to see this in the results of interference experiments.”

And what they found, much to their surprise, I think, was that in fact, there was no ether wind, and that ruled out some theories of the ether, but not all, and Michael said certainly continued to believe in the ether. Okay, so this is what was a shocking part of reading this story from the biography of Einstein

that you recommended by, what was his first name, by, yes, Settles of Lord,

and then also from Emre Lakatos, the methodologies of scientific research programs. The way it's told is that Michael some moreally proved that the ether did not exist. Yeah, therefore it created a crisis in physics that Einstein saw the special relativity, and Richard pointing out is actually was trying to distinguish between many different theories of ether, you know, if you're in space or if you're on earth, it's a same direction of ether,

or maybe the ether wind is being carried around by the earth, and so you can't really experience it on earth, but if you go to high enough altitude, you might be able to experience it. In fact, the Michaelson's experiments were, the famous one is 1887, but he conducted this experiments for basically two decades. I mean, for longer than that, he conducted them, I think the first one was in 1881,

“but he continued to believe until, I mean, he died, he died, I think was like 1929,”

or so, it's like the late 20s, and he was still doing experiments in the 1920s, sort of about, with or lot, you know, the ether existed, and so he continued to believe in the ether, to the end of his life, or I think the last public statement he made is like a year or two before he died,

and he still believed, but he basically believed at that point.

In fact, there was another physicist, Miller, who kept doing these experiments in the 1920s, he thought that he went to high enough altitude, is in Mount Wilson in California, or I'm high enough that I can actually, the ether winds are not being dragged with that by the earth, and I've measured the effect of the ether, and Einstein hears about this, and he says,

Anyways, I think the reason the story is interesting, for many different reasons, but one is one of the different ways in which the real history of science is different from this idea you get of the scientific method is you really can't apply falsification as easily as you may think. It's not clear what is being falsified, is it just another version of the theory of the ether that's being falsified, or certainly you can't induce the theory of special relativity from the fact that

one version of the ether seems to be disconfirmed by these experiments. Yes, so, I mean, certainly he doesn't share the, you know, ideas about falsification are wrong, are falsified, but, you know, it does share the sort of the most naive ideas, you know, or things are much, often much more complicated than you think. So, you know, Michaelson did this experiment in 1881, he was a very young man,

“and then other people, I think Rayleigh was one of them pointed out that there was some problems”

with the way he did it, so they had to redo it in 1887, and at that point, like a lot of the leading

physicists of the day, leading scientists of the day, basically accepted this result that there was no

etherwind, but what to do about this? So, yeah, sure, maybe you've falsified some theories of the ether, there are others that you haven't falsified at all at this point, and you're people sort of set to work on developing those, and actually, it's funny, I mean, people will phrase it as, show that there was, you know, that the ether didn't exist, and even just the word "their" is kind of a misnomer, you know, you actually had a ton of different theories and a couple of leading

contenders, so yeah, there's some version of falsification going on, but like how you how you respond to this new experiment is very, very complicated, and most people responded, I mean, suddenly, the leading physicists of the day responded by saying, "Okay, this gives us a lot of information about what the ether must be, but it doesn't tell us that there is no ether." In fact, Lorenz, at the end of the 19th century, before Einstein, figures out the math

how you convert from one reference frame to another reference frame, comes with the Lorenz transformations,

which is basically the basis of special relativity, but his interpretation is that you were converting

from the ether reference frame to these non-pervillage other reference frames, if you're moving relative to the ether, and his interpretation of length contraction and time dilation is that this is the effect of moving through the ether, and you have this pressure, and that the pressure is warping clops, it's warping measures of length, and the interesting thing here is it experimentally, you can identify distinguish Lorenz's interpretation from special relativity.

“Yeah, I think that's a strong statement. I mean, Lorenz introduces this quantity called local time,”

which he regards, he's not trying, my understanding is he's not trying to give a really a physical interpretation of this, but it's what Einstein would later just recognise as time in another initial reference frame, and he's not trying to attribute much physical meaning to it, I think Punker, he gets much closer to later on to realise that actually this is the time that's registered by clocks, but if you think about what is in its 40 odd years later, people start doing

these muon experiments where they see basically, because of the grace of the top of the atmosphere,

they produce a shower of muons, and you can look to see a different heights in the atmosphere, you can look to see how many of those muons remain, and they decay over time, and a very strange thing happens, which is that they're decaying way, way, way too slow, so you expect, actually they shouldn't really, they shouldn't be able to sort of last the whole way through the atmosphere at all, there's just their decay rate is too quick, if you're in a classical theory, but if in fact,

their time really has slowed down, it's okay, and in fact, you know, the measured decay rates in the 1940 and then there have since been more accurate experiments done, match exactly what you expect from special relativity, so, you know, that's the kind of thing where again, if Lorentz had been alive, he didn't dead 10 or so years at that point, if he'd been alive, I'm sure he would have tried, well, it seems quite likely that he would have tried to save his theory by patching it up

yet again, but it would have been a massive, I mean, that's a real setback, it starts to just look like, oh no, time is, you know, this thing that Lorentz introduced as a mathematical convenience, no, no, that's actually what time is, for the muons at least, and then, you know, there's a whole

“bunch of other experiments that show this very similar phenomenon, and when was that experiment done?”

That was, I think, 1940 or 19, it might have been published in 1941, so maybe to, or then to

community adopted what we in retrospect consider, the more correct interpretation, before it was actually empirically or experimentally shown to be preferred, so there's clearly some process that human science starts, which in distinguished different theories. It can be just interrupted, I mean, you use the word process, and it's sort of, it's interesting to think about about that term, like process kind of carries connotations of, you know, it's something set in advance, it's something,

and it's, it's much more complicated in, in, in practice, you have people like, like Lorentz, who, I mean, I'm just, I'm just, just, absolutely utterly admired, and, and Pornkarte, one of, you know, the greatest scientists who ever lived at Michaelson, I mean, another truly outstanding

scientist, never reconciled themselves. So it's not as though there's like some standard procedure

that we're all using to, like, reconcile these things. No, like, you know, great scientists can remain long, very, can remain wrong for a very long time after the scientific community has broadly changed its opinion, but there's nothing, there's no centralized authority, right? So, the same goes, centralized method. Yeah. I mean, that is the interesting thing. There's this progress, even though it is hard to articulate the process by which happens, the, um,

the heuristics that are used. Anyways, you mentioned Pornkarte, and so Lorentz has the math

“right, but the interpretation wrong, and you should explain, it seems like Pornkarte had the opposite,”

where he understood that it's hard to define simultaneousity, um, because it requires on a circular definition with time, um, or velocity of something that might be sent, you know, arrive at a midpoint together, but velocities defined in terms of time. Um, and I find this interesting, there's a couple of other examples we could call on, but like, there is this phenomenon in the history of science, where somebody asks a right question, um, but then they don't sort of clinch it.

And I'm curious what you think is happening in those cases. I mean, I think you sort of, you actually do want to go case by case and try and understand, and it's not necessarily clear that they're, they're doing the same thing wrong in, in all the cases. I mean, the, the Pornkarte case

is, is amazing. Um, he seems to have understood the principle of relativity, the idea that the

laws of physics are the same in all our national reference frames. He seems to have understood that the speed of light is the same in all our national reference frames. He doesn't actually phrase it quite that way, but, but it is my understanding, but I don't speak French, but, um, uh, uh, uh, you know, and this is, I mean, these are basically, these are the ideas that Einstein uses to deduce the special relativity, but then he also has this additional sort of misunderstanding where things,

uh, that length contraction is a dynamical effect that somehow, um, you know, sort of particles have been pushed together by, by, you know, some external force, some, something is going on dynamically, and he doesn't understand that, that it's purely kinematics that actually space and time

“are different than, than what we thought and you need to fundamentally rethink those, those things.”

So it's almost like, it's almost like he knew too much, um, you know, he had sort of almost two grand, a vision in mind, and Einstein's, it's sort of almost subtracts from that and, and and says, no, no, no, it's, it's, it's space and time, I just different than what we thought, um, uh, and, and, you know, he is the correct picture, and there's a, uh, paper, you know, I think it's 90, no, no, we're, we're punk at eight, like he's still got this dynamical picture of what's

going on with the length contraction, and we just, you know, this is just not necessary, this is, this is, um, a mistake, um, from the modern point of view, and so why, why is he doing this, like,

why is he clean onto this idea, and, you know, I don't know, I've, you know, obviously never met

the man, uh, it would be fascinating to be able to, to, to talk it over and to try and understand, but, you know, he, he, I mean, his expertise seems to be getting in the way, he knows so much, he understands so much, um, and then he's not able to let go of these, these things actually, a really interesting fact, um, is that, uh, a few years prior, so, 1890s, Einstein's a teenager, he believes in the ether too, like he knows about this stuff, but like he's just not, he's not quite

as attached, obviously, uh, as, as these older, older people were, um, and, and maybe they, they were a little bit prisoner of their own expertise, that's, that's my guess, I mean, historians of science could, could, might, would, would, some would certainly disagree. Well, there's, then there's, the obvious stories were, Einstein himself, later on, is said to have not last on to the correct interpretations of, um, quantum mechanics or cosmology,

“because of his own attachments. Yeah. I think the, the, the, the bigger question I have is like,”

the mu on example is a great example of, um, uh, these long verification loops, and how

the, maybe the clearest example is airstarchists, in second century BC,

comes up with the idea of a killer centrosum, the ancient Athenians dismiss it on the grounds that, well, we should see as the earth is moving around the sun, if really the sun is the center of solar system, the stars should move relative to earth. Um, and the only reason that is not possible, that would not be the case is the stars are so far away, that you would not observe this, and it's only an 1838 that stellar parallax is actually measured, and so we didn't need to wait

until 1838 to have heliocentrism, right? Like, we didn't need to wait for the experimental validation to understand Kapernicus's better in some way. Um, in fact, when Kapernicus first comes up with the series this one though, and that, um, the Tolemake model was more accurate, because I've had

all these, um, centuries of adding on these epicycles, um, was maybe, but we less while appreciated,

it was also in some sense simpler, um, because Kapernicus actually had to add extra epicycles. It had more epicycles in the Tolemake model, because he had this bias that, you know, the, um, the earth should go into the perfect circle in equal time. Anyway, I think this is an interesting story because it's like, it's not more accurate. It's not a simpler theory. So, how, why was, how could you have no next ante the Kapernicus was correct, and Tolemake was not?

Hmm, I've been good question, and I don't know, uh, it's sort of entirely the answer. I do know, um, well, I mean, I can give you a, it's certainly, a partially answer that I sort of, you know, centuries in the future, you start to find very compelling. Um, um, uh, and I'm sure it's sort of part of the story at least, um, which is, um, one of the big shocks for Newton, um, eventually, you know, he, he did understand, uh, uh,

a couple of laws of motion, eventually, um, so you're able to explain sort of the motions of the,

“the planets in the, that's why, but he also added the same theory as theory of, of gravitation,”

was able to explain terrestrial motion, so is able to explain why objects move in parabolas on the earth, and he's able to explain, um, the tides in terms of, uh, uh, uh, the sun, uh, the, the, the moon and the sun's effect, um, uh, gravitational effect on, uh, water on the earth, and so you have what seemed like three very different disconnected phenomena all being explained by this one set of ideas, right, that, that, that, that, that, that, I think starts to feel that,

that's very compelling, um, at least to me, um, and I think, I think most people find that very, very satisfying once they, once they eventually realise it. Um, have you read the Keynes biography of Nune? Oh, I, I, he's written an, he read an entire book. No, no, the, the, the essay. Yeah, sure, sure. Yeah, um, I love that. Yeah, I mean, this, this description of him is the last of the magicians.

“Yeah, it's, it's wonderful. Yeah. In fact, I think it's, maybe worth the superimposing or you”

should read out that, that one passage of the, of the thing, all right. So it's from, uh, actually, I believe it was a talk that he gave at Cambridge, not, not long before, uh, he died, he declared, uh, Newton's papers somehow, um, and then he gave, uh, he gave it a lecture. I think twice, um, about this sort of that, his brother Jeffrey gave it to the other time, because he was too ill, um, there's just this wonderful, wonderful quote in the middle. Um, I actually, the whole thing is

really interesting, um, but, but I love this particular quote. Uh, Newton was not the first of the

age of reason. He was the last of the magicians. The last great mind, which looked out on the visible and intellectual world with the same eyes as those who began to build our intellectual inheritance rather less than 10,000 years ago. And like, this idea that people have that, that Newton was, um, sort of the first modern scientist is, is somehow wrong. He, I mean, it's, it's, it's some truth to it, but he really had this very different way, um, of, of, of looking at the world

that was part, sort of superstitious, um, and part modern. It was a funny hybrid. He's sort of this transitional figure in some sense. Um, uh, uh, that, that, that, that phrase, the last of the magicians,

“uh, I think really, really points at something. The thing I've heard here about with Newton”

is whether it was a same program, the same heuristics, the same biases that he applied to his alchemical work as he did to the understanding of astronomy. So this is from the kings essay. There was extreme method in his madness. All his unpublished works on esoteric and theological matters are marked by careful learning, accurate method, and extreme sobriety of statement. They are just as sane as a principia. If their whole matter and purpose were not magical,

they were nearly all composed during the same 25 years of his mathematical studies. So clearly there were some aesthetic, which motivated people like Einstein to say reject

about things. Um, same with Newton, and the question I have is whether similar heuristics towards parsimonie, towards aesthetics, etc, would be equally useful across time and across disciplines, or whether you need different heuristics. And the reason that's relevant is even if you can't build a verification loop for science, maybe if they're if the taste has to point in the same direction, you can at least encode that bias into the AI's, and that would

maybe be enough. I mean, these questions like like the point is that where we always get bottlenecked

is where the the previous processes and and and heuristics don't apply, right? Like that's always sort of definitely what causes the bottlenecks. Because people are smart, they know what has has worked before, they study it, they apply the same kinds of things, and so they don't get stuck in the same places as before, they keep you know, they keep getting bottlenecked in in

“different places. I mean, that's I'm over generalizing a bit, but I think it's it's the right,”

like if you're attempting to reduce science to a process, you're attempting to reduce it to something where there is just a method which you can apply and you turn sort of the crank and out pops inside. I mean, you can do a certain amount of that, but you're going to get bottlenecked at the places where you're existing method doesn't apply, and and but definitely there's no crank you can you can turn, you need a lot of people trying different ideas and and sort of the more

difficult the idea is to have, right, the greater the bottleneck, but then also sort of the greater the triumph. Quantum mechanics is like, oh, it's a great example of this, it's such a shocking set of ideas, it's such a shocking theory. Actually, the theory of evolution in some sense is also quite a shocking idea, not the, you know, principle of, you know, the sort of natural selection,

“but that it can explain so much, that's a shocking idea. Existing safety benchmarks claim back,”

at least for today's top models, attacks are only successful, a few percent of the time. This sounds great, but label walks researchers were able to jailbreak these very same models

about 90 percent of the time, even the ones that have a strongest reputation for safety.

And the disconnect here is that the prompts which underlie these public safety benchmarks are all framed in a very naive way. There's no attempt to disguise harmful intent. These prompts will just ask models to hack into a secure network and to do so without getting caught. But real bad actors don't write like this. So label walks build a new safety benchmark from the ground up. Their prompts reflect real adversarial behavior by stripping out obvious trigger phrases and

wrapping their requests in fictional scenarios. For example, instead of outright asking another lamp to steal somebody's identity, the prompt will frame it as a camera, a light bearer who's trying to hide from dark forces and needs a handbook on how to disguise themselves as somebody else. This safety research is linked in the description. If you think this can be useful for your own work, reach out to act label walks.com/borkhash.

So principally, Mathematica is released in 1687. The origin of species released in 1859. At least naively, it seems like Darwin's theory that their natural selection is conceptually easier than the theory of gravity. I asked Ernst how this question. But yeah, there's this contemporaneous biologists with Darwin, Thomas Oxley, who read this and said, how extremely stupid

to not have thought of this. Nobody ever reads the first European Mathematica and things.

God, why didn't I beat Newton to the find share? No.

“And so yeah, what's going on here? Why did Darwin as I'm takes so much longer?”

The idea must have been known to an all breeders for a long time at some level. Or certainly large chunks of the idea were known that artificial selection was our thing. And in substance Darwin's genius wasn't in having that idea. It was understanding just how central it was to biology that you could potentially sort of go back and explain a tremendous amount about all of the variety of what we see in the world.

With this as not necessarily the only principle, but certainly a core principle and so he writes this wonderful book, the original species. And it's just so much evidence and so many examples and trying to tease this out and see what the implications are. And to connect it to as much else as he possibly can to connect it to geology and to connect it to all these other things.

the biosphere is what he's doing there. He's not just having the idea. He's making a compelling case that no, it's intertwined with absolutely everything else. Yeah, the motivation of the question was

Lucrecia's who's this first century Roman poet has an idea that seems analogous to

a natural selection about, you know, species get fitted more at a time over over time to their environments or species using it to their environment. And so they're like, okay, well why did this go nowhere for 19 centuries? And then I looked into it or more accurately as LLM is what exactly was Lucrecia's idea here. And it actually is extremely different from what real natural selection is. He thought there was this generative period in the past where all this species

came about and then there was this one-time filter which resulted in the species that are around today and they became fit to the environment. He did not have this idea that it is an ongoing gradual process or that there is a tree of life that connects all all the life forms on her together. Which is, by the way, it's incredibly weird fact that every single life we're in Earth has a common ancestor. It's not incredibly, it's not incredibly weird, right? If you think that the

origin of life must have been very hard, like that there's a bottleneck that it's not so surprising. Yeah. There's also this verification loop aspect where even if Newton might be harder in some sense, if you've clinched it, you can experimentally, I know validated as the wrong work philosophically, but you can give a lot of base points to the theory. You can be like, okay, I have this idea of why things fall on Earth, I have this idea of why orbital periods

were planets have a certain pattern. Let's start on the moon, which orbits the Earth. And in fact, you know, it's weird orbital period matches with my calculations imply. And the tides work

correctly. Exactly. Yeah, it's just amazing. Whereas for a Darwinism, it takes a ton of work

for Darwin to compile all the sort of cumulative evidence, but there's no individual piece that is overwhelmingly powerful. And there's a whole bunch of problems as well. He doesn't really understand what sort of mechanism is. What the mechanism is, he doesn't understand genes, all these things. The very interesting thing in the history of Darwinism is this idea which sort of the theoretically you could come up with at any time. There is almost identical independent creation of the idea

“between Alfred Wallace and Charles Darwin. So much so that I think Wallace sends this manuscript to”

Darwin, it's like, what do you think of this idea? I don't think that's an exact quote, but I think it's pretty much correct. Yeah. And then so they actually present their ideas together in the spirit of sort of sportsmanship. And so then yeah, why was this period in the 1860s or 1850s? Why was that the right time for this idea? So we're going to come with it for ideas. One is geology. So in 1830s, I think Charles Lyle figures out that there's been millions and billions of years of time that's

exist on an earth, then paleontology shows you that actually organisms existed, fossils of existed for that intern. So life goes back to long time. And in fact, you're going to even find fossils for intermediate species that show you this real life. In fact, between humans and other apes as well, there's intermediate humans. There's the age of colonization and we have all these voyages who can do this biogeography. And I guess that all must have been necessary because the, in fact,

there's a huge history of parallel innovation in discovery in history of science. So maybe it is another piece of evidence to actually more had to be in place for given idea to be discovered because if it's not discovered for a long time and then spontaneously many different people are coming up with it that shows you that actually the building blocks were in some sense necessary. I mean,

“I mean, I think I mean, this example of Lyle and I mean, and other bio, excuse me, other geologists,”

you know, sort of early 1800s, basically coming, you know, having this idea of deep time,

does seem to have been crucial. I know Darwin was very influenced by Lyle,

and, you know, if you don't have at least sort of tens or hundreds of millions of years, evolution just starts to look like an on-starter, you know, we should be seeing radical change, you know, in order to make it work on sort of a timescale of, say, five to ten thousand years or, you know, six thousand years, Bishop Usher, you know, you would need to be seeing evolution occurring in a massive rate sort of during human lifetime. So we're just not seeing that. So that does seem to have been a blocker.

It's interesting to, I mean, to you know, to your question, like what are the blockers were there, were there, were there any of this? And I don't, I don't know. Right. Or yeah, how much earlier could you in principle have come up with that if you're a much smarter? Actually, let me, I mean, let's just kind of sort of zoom out to your original question. So you talk about sort of the verification loop in AI, and, and you're something that an example,

“I think that should give you pause, there is, you know, the big signature success so far,”

you're certainly alpha-fold. Yeah. And of course, alpha-fold really isn't about AI, you know, a massive fraction of the success there is the protein data bank. So it's, it's x-ray diffraction,

whatever it's a hundred and eighty thousand on protein structures. So sort of, you know, it's basically

the story of, we spent many, many decades obtaining protein structure just by going out and looking very hard at the world experimentally. And then we fitted at an ice model at the end of it, and that was like a tiny fraction of the entire investment. But it's definitely not, yeah, that's a story of data acquisition. Yeah, principally, it's not only, I mean, the AI bit is very, very impressive, it's quite remarkable. But it is only a small part of the total story.

Alpha-fold is very interesting, and I, philosophically, I wonder what you think of it as scientific theorist, scientific explanation, because if over time, I guess the world has become harder to understand. I'm gonna, as I'm saying things, because you're such a, um, careful speaker,

“I'm, I, I say it this phrase and I'm like, is that it, will he actually buy that premise?”

But yeah, there's, you know, we need to fit models, the things rather than, at least in some domains, we're trying to fit models, the things rather than coming up with underlying principles that explain a broad range of phenomena. And so it compares, say, the theory of general relativity, or any theory which just met out to some equations versus alpha-fold, which is encoding these different relationships between different things we can't even interpret over a hundred million

parameters. And are those really the same thing, because GR can predict things you would have never

anticipated or was never meant to do, like, why does Mercury's orbit precess, um, and alpha-fold is not going to have that kind of explanatory reach, and I want to get to a reaction to that. Yeah, it's any, I think it's an incredibly interesting question. Um, I mean, maybe, maybe a really pivotal question, um, in the sense of, so, you know, if you search like a very classic point of view, you want these deep explanatory principles, um, you want sort of a few free parameters as you possibly,

so can, you, you want very simple models, which explain a lot, and alpha-fold doesn't look anything like that. Yeah. Um, and so you might just sort of say, oh, well, you know, it's nice, it's maybe helpful as a, as a model, but it doesn't have, it's, it's not a scientific explanation. So that's kind of, that's a, that's a, that's like a conservative point of view, that's sort of, I don't know,

“answer one to the question. I think answer two is to say something like, um, maybe you shouldn't”

think about alpha-fold, you know, as, as an explanation in the classic sense, but maybe it contains lots of little explanations inside it, and so maybe part of what you can get out of, like, you know, interpretability work is you can go into alpha-fold and you can start to extract certain things.

Maybe, maybe basically by doing sort of, you know, archaeology of alpha-fold, um, we can actually

understand a great deal more, um, about these principles you can start to extract it all. That circuit does this interesting thing and we learn this. So I don't know, to what extent that's been done with alpha-fold. I know it's been done a little bit with, um, uh, uh, some of like the chess models. I believe it's alpha-zero, um, there seem to be some strategies, uh, which was certainly borrowed by Magnus Carlson at least, um, which he seems to have just taken, uh, uh, from alpha-zero. I mean,

I don't think there's any public confirmation of this, but there were, you know, some, some, some experts have noticed that he changed his game quite radically, um, after, um, some sort of, some public forensics were released on how alpha-zero work. Um, so that's kind of a sort of an example where, uh, I think human beings are starting to extract meaning out of these models, and maybe that starts to lead to sort of, sort of viewing the models as a source of

“potential source of explanations, you need to do more work because they're not very legible”

upfront, but you can extract them potentially. And I think that's kind of, I think that's kind of an interesting, intermediate, um, situation where they're not explanations, but you can extract interesting explanations out of them. You can use them as, as kind of a, kind of a source, you know, like the, like the third and the most interesting possibility is, now they're, like, they're, they're a new type of object in some, in some sense. They should be taken very seriously

as, as explanations, but we're in the past, we haven't had the ability to really do anything with them, and now we're going to, we're going to have sort of new, interesting new sort of actions, which we can, we can do, we can merge them, we can distill them, we can do all these kinds of things, and there's going to be sort of a, almost a new, it's a big opportunity sort of in that, you know, philosophy of, of science to, to, to, to, to, to, to start to do that, and they're, they're,

they're, they're, like, an anticipation of this in some sense, I think, in the way, so the, I, I know, know, some mathematicians of physicists who, I mean, historically, if you had, like, a 100-page equation, which, and that's the kind of thing that does come up, I mean, there's just nothing you can do, if it's 1920, there is nothing you can do at that point, you, you give up on the problem, and now today, with tools like Mathematica, you can just keep going,

and so that's, that's, that's an object now, that's a thing that you can work with, and, and

complicated, and sometimes they get simple answers out of the end, and that's just an intermediate

“working state, and so I sort of wonder if there's going to be, you know, something similar is”

going to, going to happen in, in, in, in, in this particular case, where you could take these models, and sort of just use them in a little bit the same way people do with, with Mathematica, and take them seriously as, they're not explanations in the classic sense, but there'll be something else, which interesting operations can, can, can be down on. The, the thing I worry about is, suppose that you, it's 1600, and you're trained, or 1500, and you're training a model on,

this is a weird history where we developed deep learning before we had enough, before we had cosmology, but, um, suppose we live in that world, and you're observing how, there's the stars, they don't seem to move the planets, have, all these, we heard behaviors, and then you train a model on that, and then you, do some kind of introverted and trying to figure out, well, what are the patterns we see here? What you'd see are just these, you just keep,

“be able to keep building on top of these models, you'd see like, oh, there's more episodes we”

didn't notice, there's another episode, although it's, the parameters, whatever to whatever, whatever, and code, episode, or this, parameters, whatever, and code the next episode, so if you're just trying to figure out, why is this one, there's just some of the way it is, from observational data, you could just keep adding episodes up on episodes, but it really took one mind to

integrate it all in and say, um, here's my, here's the, here's what makes more sense overall.

So, I mean, they're, like, like, you know, I mean, this is sort of to my point that we don't, as really understand what to do with the models, like, sort of, we don't have, like, the, the verbs necessarily, yeah, but you know, it is certainly interesting to think about the question, you know, we start to apply constraints to the models, you know, sort of essentially saying, what's the simplest possible explanation, or, you know, can you, can you simplify? Can you, can you,

“can you give me sort of the '90, '10 explanation, can you, I'd go further and further and further,”

sort of in, in boiling it down, so it might be that, indeed, they sort of start out by providing, you know, a very, very complicated, uh, many, many, many parameter model, um, but you can just,

you can just force the sort of the, the case, and basically, that's scaffolding, um, which

maybe they, you know, you sort of, the, the very early, uh, days of their attempt to understand something, um, but, but they forced through that to, to, to, to much more simple understanding. So, this is sort of a misunderstanding, but it sounds like you're saying, maybe there's some sort of regularizer, sort of a distillation, you could do, of a very complicated model that gets to, a true, or more parsimonious theory, but, yeah, just take, uh, tell me, versus Copernicus, right? So,

we start out with lots of, tell them, make up a cycles, and then you try to distill this model, and maybe gets rid of some of the epicycles that were, are less and less sort of necessary to get, the mean squared arrow, the orbits to match, but at some point it has to do the sing, which is, like, swish two things. Yeah. Yeah. And it, locally, it actually doesn't make things more accurate. Yeah. It's sort of like, in global sense, that it's more, it's a more progressive theory.

Yeah. Yeah. And there's some process, which obviously, humanity did it over. It's ban, which did that regularization, I mean, that swap, but if raw gradient descent, it seems like, I don't, I don't really feel like it would do that. I mean, I could say, I mean, you think about the example of, of going from Newtonian gravity to Einstein's general theory of relativity. And these are, I mean, these are shockingly different theories. And the question, you know, is like, what causes that,

that flip, and, and as nearly as I understand, the history, you know, what goes on is Einstein's, you know, developed special relativity. And pretty much straight away, he understands, I mean, it's a very obvious observation in special relativity. Influences can't propagate faster than the seat of light. And in Newtonian gravity, action, you know, is out of distance. In fact, you know, it's, it's straight away in special relativity. You could use Newtonian gravity to do faster than,

faster than light signaling. You could send information backwards in time. You could do all kinds of crazy stuff. And so it's not a big leap to realize, oh, we have a big problem here. And so, you know, that's kind of the, that's the forcing function there. It's, it's, you've realized that your old explanation is not sufficient. You need something new. And then you're going, you're just going, you're going to start by doing the simplest, you know, possible staff, and it just turns out

that a lot of that stuff doesn't work very well. And so you sort of forced, and in fact, it is interesting that, you know, he's sort of forced to go through these steps of gradually, it gets quite more complicated and it's sort of wrong and a variety of ways. And the final theory appears really shockingly simple and beautiful, but it's gone through some, somewhat ugly intermediate stages. Yeah. So, if you're thinking about what, what does it look like to have AI

like, how does this work unfold? We just train a raw model using gradient descent. Then there's

things like coming up with general relativity, where you couldn't really just train on every single observation in the universe and hope that general relativity pops out. And so, what would it require? Well, it also certainly wasn't immediately discovered, right? So it was a lot of decades of thought. And I guess you need the independent research programs where people will start off with these biases, where Einstein is just initially motivated by this thought experiment of, you know,

can you distinguish the effect of gravity from just being accelerated upwards? And then you just need different AI thinkers to have to start off with these initial biases and see what can germinate out of them. Yeah. And then the verification loop for that might be quite long,

but you just need to keep all those research programs alive at the same time. Yeah. I mean,

I think there's, I mean, this point that you make about sort of keeping all the different research

“programs alive, like that I think is very important and somehow central. I mean, a great example is”

is situations where the same answer has been correct in some circumstances and wrong in other circumstances. So the planet Uranus was like not in quite the right spot and people very famously predicted the existence of Neptune on this basis, a wonderful massive success for Newtonian gravity. The planet Mercury is not in quite the right spot. You predict the existence of some other distorting planet turns out that doesn't exist. Actually, the reason Mercury is not in the right

spot is because you need general relativity. And so you've sort of, you've pursued very similar ideas and has been very successful in one case and it's been completely and utterly unsuccessful in the other case. And I think, I mean, a prayer I, you can't tell which of these is the thing to do and you actually need to do both. Yeah. And so I mean, this is only, it's a very true in the history of science that, you know, this kind of diversity where you just have lots of people go off and pursue

lots of potentially promising ideas. You just need to support that for, for a long time, but it's, I mean, it's hard to do that for a variety of reasons. But it does seem to be, to be very, very,

“very important. So this example of Uranus versus Mercury is very interesting. In one, I think”

it illustrates sort of the difficulty of falsificationism, like the orbit of Uranus is in some sense falsifying Newtonian mechanics. But then you say, you make some ancillary prediction that says, oh, the reason this is happening is there must be another planet, which is effected perturbing Uranus's orbit. And you, I think it's LaVaria in 1846. Point of telescope in the right direction. You find Uranus. Neptune. Oh, it's a yes. But with Mercury, it's observed that it's,

the ellipsis forms a orbit is rotating 43 arcs seconds more every century than Newtonian mechanics fluid. And fly. So people say that there must be a planet inside of Mercury's orbit, they call it Vulcan. And point of telescopes is not there. But if you're a proper Newtonian, what you do is say, well, maybe there's some cosmic dust that's occluding this planet. Or maybe

the planet is so small, we can't see it. Or maybe there's some, let's build even more powerful

telescope. Oh, maybe there's some magnetic field, which is sort of occluding our measurements. And this happens over and over, right? Like, you know, there's just so many stories which are exactly like this. Right. I mean, an example I love from, you know, in the 1990s, some people noticed that the pioneers spacecraft weren't quite where they were supposed to be. And so, you know, you can get very excited about this. Oh, my goodness, general relativity is wrong.

We have, like, kind of, you know, maybe we're going to discover the next, the next series of gravity. And, and today, the accepted explanation is that, no, actually, there's just a slight asymmetry in the, in the spacecraft, it turns out that the, you know, the thermal radiation is like slightly larger in one direction than the other. And that's causing a tiny little acceleration towards the sun. And most of the time, when there's these apparent exceptions, it's just something like

that's going on. It's very much like the Vulcan, the Mercury Vulcan case. But everyone's in a while, it's, it's not. And, and I pray, you can't, you can't distinguish these. But I mean, science is just just full of these. It's only two, like, the way we tell the history of science, it sounds so simple. Like, oh, you just focus on the right exception. And, you know, you realize that

“you need to throw out the old theory. And, and, and low and behold, you know, you're noble”

prowess of weights. But in fact, there's these exceptions. They're all over the place. And 99.9% of the time, it just turns out to be some effect like this thermal acceleration in the case of the plane. Yeah. Spacecraft. So, so, you know, sort of that, unfortunately, there's a lot of

you which case you're in. And just a spell out why I think this is important. It's because some people

have this idea that, yeah, it's going to make disproportionate progress towards science. Because it makes disproportionate progress towards domains where there's tight verification loops. And so, it's really good at coding because you can run unit tests. And science may be similar because

“you can run experiments. Yeah. And I think what that doesn't appreciate, one is that experiments”

actually don't, there's an infinite number of theories that are compatible with any given experiment. And over time, why we glob onto the, well, at least in what in retrospect, we think is a more correct one is, as we're discussing in this conversation, sort of hard to articulate. Lack of those actually has all kinds of interesting examples in the book about these kinds of hostile verification loops that are extremely long lasting. So one, he talks about his

proud or fruit, I don't know how to pronounce it. But there's this chemist in 1815, he had

hypothesizes that all atomic nuclei must have whole number of weights. And they're basically

all made up hydrogen. And if the reason he thinks this is because if you look at the measure rates of all elements, it does seem that they almost all of them do happen to hold number of rates. But then there's some exceptions. Like, for example, chlorine comes out at 35.5. And so then there's

“all these ad hoc theories that people in this school keep coming up with, like, oh, maybe there's”

chemical impurities. But then there's no chemical reaction you can do, which seems to get rid of this. Maybe it's fractions of hold numbers. So it's 35.5. It can be halves. But I actually measure chlorine even closer at 35.46. So it's actually getting further away from the correct reaction. And later on, what is discovered is what you're actually measuring is different isotopes. Which cannot be chemically distinguished. There can only be physically distinguished.

But so then you just have 85 years before we realize what a nice isotope is where the verification looks actually actively hostile against you against the correct theory. And then you just need this remnant to be defending what there's no exact reason it's the preferred theory. Just as a community we should just have people defend, try to integrate new observations even if they don't fit seem to fit their school of thought with what they believe. And hopefully enough of that happens.

Anyways, yeah, I guess something they've been trying to articulate is the difficulty with automating science. Yeah, I mean the question is, where is the bottle now? It's some level of an end. So do you know, are we primarily bottlenecked on one thing or one type of thing? Or are we bottlenecked on sort of multiple types of things? So you know, suddenly talking to structural biology people, they seem to think that alpha-fold wasn't an enormous advance. It was a shock. So at some level,

yes, AI can, you know, it seems certain it can help us speed up science. So it is helping with a certain type of bottleneck. Yeah, that doesn't mean, as you're saying, that it's necessarily going to help with all kinds of bottlenecks. And sort of I suppose the question you're pointing out is

“like what are the types of bottlenecks that remain and what are the prospects for forgetting past them?”

I think even in the case of coding, like it's really interesting, you know, talking to programmer friends. Yeah, at the moment they're all in this state of shock and high excitement and they're all over the place. I actually kind of kind of talking to them. You do want to like, where is the bottle Nick going to move to? So certainly one thing that a lot of them seem to be bottlenecked on is now having interesting ideas and in particular having interesting design ideas.

So there's not really a verification loop for knowing, oh, that design idea is very interesting. So they're no longer nearly as bottlenecked by their ability to produce code but they are

still bottlenecked by this other thing. They always were, they were formally, they weren't

bottlenecked on it because, you know, just writing code was it took so much of their time. They could sort of have lots of ideas while they're, you know, they take their three weeks to implement their prototype and then they would implement the next version. You know, they're taking three hours to implement the prototype and they don't have, you know, as good ideas sort of after that from a design point of view. Last year I predicted that by 28 AI will be able to

prep my taxes about as well as a competent general manager. But we're already getting pretty close. As I shared before, I used Mercury both for my business and my personal banking. So I recently gave an LN access to my transaction history across both accounts through Mercury's MCP. I asked it to go through all my 2025 transactions and flag any personal expenses that seemed like it should actually be charged to the business. And this worked shockingly well. Mercury's MCP exposes a bunch

of detailed information, things like notes and memos and any JPEGs of receipts and PDF attachments. So my LLN had plenty of context to work with. One of my favorite examples happened to the charge to PayPal. If you looked at the vendor alone, you would have had to assume that it's a personal expense. But the LLN looked at the receipt and the attached note in Mercury and realized it was

I imagine it will be a while before traditional banks have MCP. Functionally like this is why I use Mercury. Go to mercury.com to learn more. Mercury is a fintech company, not an FGIC in short bank. Baking services provided through choice financial group and column NA members FGIC.

“You have a very interesting take. I think it was a footnote one or your S isn't it”

confined it again, which was that it's very possible that if we met aliens, the day would have a totally different technological stack than us. And that contradicts, I guess, a comments

it's a section I had that I never questioned, which is that science is this thing you do very

relatively early on in the history of civilization, where you get to a point and you have a couple hundred years which is cranking through the basics, understanding how the universe works, etc. And you've got it. You've got science. And then basically everybody would have been version the same quote unquote science. And so I found that a very interesting idea and I want you to say more about it. Yeah. I mean, I think that probably the idea there that I'm at least somewhat

attached to is the idea that the sort of the the the the the tech tree or the science and tech tree is probably much larger than we realize. I mean we're sort of in this this funny situation people or sometimes talk about, you know, a theory of everything as a potential goal for physics and and then there's this presumption somehow that physics is done when she gets there. And because this

is this is not true at all. If you think about computer science, computer science basically got

started in the 1930s when during and church and so on, just laid down what the theory of everything was. They just said, you know, here's how computation works and then we've spent 90 or G is since then just exploring consequences of that and gradually building up more and more interesting ideas. And those ideas are to some extent you can just regard as technology but to some extent

“insofar as they're sort of discovered principles inside that theory of computation, I think they're”

best regarded as science and in some cases very fundamental science ideas like public e cryptography are I mean they're just incredibly deep, very non-obvious ideas which in some sense lay hidden already sort of in the 1930s. And so my expectation is that different, you know, there will be different ways of exploring this tech tree and we're still relatively low down, we're still at the point where we're just understanding these basic fundamental theories and we haven't

yet explored them. Sort of a thing which I think is quite fun is if you look at just just the phases of matter when I was in school we'd get taught that there are three phases of matter or sometimes four phases of matter or five phases of matter depending a little bit on what you you included. And then as an adult as a physicist you started to realize oh we've been adding to this list we've got sort of superconductors and superfluids and maybe different types of

superconductors and bosoinstone condensates and the quantum hall systems and fractional quantum hall systems and and and and and and and and and and it's saying to ten years it looks like actually there's a lot of phases of matter to discover and we're going to discover a lot more of them and in fact we're going to be able to start to design them in some sense. I mean we you know we'll still be subject to the lowest of physics but but there is this sort of tremendous freedom

in there and this looks to me like oh we're down at sort of the bottom of the tech tree we've barely gotten started there and and I expect that you know to be to be the case sort of broadly

“certainly in terms of I think programming is a very natural place to look the idea that we've”

discovered all the deep ideas in programming just seems to me sort of obviously ludicrous you know we keep discovering sort of what seems like deep new fundamental ideas and I mean we're very limited we're basically slightly jumped up chimpanzees so we don't you know we're slow and it's taking us

time um but but you know what what do we look like sort of another million years in the future

in terms of you know all of the different ideas which people have had around how to how to to manipulate computers how to manipulate information I think you know we're likely to discover that actually there are a lot of very deep ideas still to be still to be discovered so notice he was it I think was canoes in the preface to the out of computer programming so something like you know he started this book back in the sixties and he talked to a mathematician

it was a bit contemptuous and said look computer science isn't really a thing yet come back to me when there's a thousand deep theorems and canoes remarks and he's writing this near decades later the the preface there are the clearly hours taken deep theorems now and that that means like

higher and higher up in the the tech tree like choices about which direction we go and sort of how we

“choose to explore you know I I think it's potentially the case that we're you know different civilizations”

or different choices mean that we end up in different parts of that tree and in particular just things I mean it's sort of very basic things about you know we're very visual creatures certain other animals are much more orally based does that bias sort of the types of thoughts that you have and then you extend it you know to sort of much more exotic kinds of civilizations we're maybe just sort of their biases in terms of how they perceive and how they they they manipulate the world

or maybe quite different than ours and that might make some significant changes in terms of how they do that exploration of of the tech tree so all speculation obviously this is such an interesting take I want to better understand it so one way to understand it is that there might there might be some things that are so fundamental and have such a wide collisionary against reality that they're inevitably going to discover in light general numbers like you like of all of the

intelligence is in the Milky Way galaxy maybe that number is one actually arguably we've already increased the number but but you know of all of those what fraction of the concept of counting and you know it does seem very natural what fraction have discovered you know the idea of some kind of you know decimal place system interesting question like and maybe we're missing something really simple and obvious that's actually way better than that what fraction got there immediately what

fraction sort of had to go through some other intermediate state what fraction use linear representations

“versus a you know two-dimensional or three-dimensional representation I think the answer to these”

questions that's not at all obvious it's a lot of design freedom on the article computer science this is this is going to be extremely naive and arrogant but I took um Scott Ironson's you know class on complexity theory and I was by far the worst student he's ever had but I what I remember is like there was a spirit that you you were you know you were what the pioneers of were you figured out here's here's the class of problems that quantum computers can solve and how it

relates to problems the class of computers always like ground-breaking oh crazy the this works

and then since then it's been this literally it's called complexity zoo this website which list out here is all the complexity classes and if you have this complexity class with this kind of Oracle it's sort of equivalent to this other class and that it feels like we're building out that taxonomy yeah and so there's a couple of ways to understand what you're saying one maybe you just disagree with me that this is actually what's happened with this field um another is that while that might happen to any one field

the amount of fields who would have thought in 1880 big computer science other than Babbit or something big computer science is going to be a thing in the first place so the amount of field we're underestimating how many more fields there could be yeah sure um or maybe you think both or maybe

a third secret thing but I'd be curious I mean you know a very common argument here is sort of the

low hanging fruit argument they argument that says oh there should be diminishing returns and in fact empirically we see this right the amount of scientists in the world is just exponentially increased and and

“I mean I think it's you know it's worth thinking about like why why do you expect diminishing returns”

and how well does that argument actually apply in practice and analogy I like is is actually thinking about sort of you know going to some event going to a wedding or whatever and you go to the dessert buffet and they've put out you know 30 desserts and of course naturally what people do right the best desserts go first I mean we don't quite have a well-ordered preference there so maybe there's some difference but but human beings are fairly similar so they will

they you know the best desserts will go first and this is an argument you know for why you expect

diminishing returns in a lot of different fields if it's relatively easy to see what's available and people have similar preferences then the best stuff goes first and and you know it just gets sort of worse and worse after that and and sort of if you you're a very static snapshot in time of scientific progress maybe there's some truth to that but if somebody you know is standing behind the dessert table and is replenishing restocking the desserts and keeps kind of you know adding

adding new ones in it may turn out that you know a little bit later much better desserts appear and so you're going to go and you're going to go and eat those instead and scientific progress has a little bit of that flavour you know we go through these sort of funny

side effect of some pretty obtuse questions in the philosophy of mathematics and and and logic

“and so you've got these people trying to attack these rather esoteric questions that seem”

quite high up in some sense in sort of exploration quite esoteric and they discover this fundamental new field and all of a sudden there's an explosion there so sort of the diminishing returns argument just didn't imply there we just weren't able to see what was there and and this has been the case over and over and over again sort of new fields arrive and all of a sudden boom it's actually easy to make progress again young people flood in because you can be 21 and

make major breakthroughs rather than having to spend 25 years mastering everything that's been done before it's obviously very attractive and I don't understand I'm not sure anybody understands very well sort of the dynamics of that like how to think about why the structure of knowledge is is that why that these new fields keep opening up but it does seem empirically at least to to be the case despite the fact that that is a case take deep learning right obviously this is an

example of a new field where the 21 year olds can make progress and it's relatively new 15 years or so it went when it sort of gets back into high gear but already we're in a stage where you need billions or tens of billions or hundreds of billions of dollars to keep making progress of the frontier and there's a couple of ways to understand that one is that it actually is harder than the kinds of things the ancients had to do or requires more is more intensive at least

second is it might not have been but because our civilisation resources are so large the amount of people that's so large the amount of money so large that we can basically make the kind of progress they would have taken the ancients forever to make almost immediately we just we notice something is productive immediately dump in all the resources but it's also weird that there's not that many of them like I feel like deep learning is notable because it is one big

exception to the fact that it's hard to think about their channels it's a consequence of the architecture

of attention right like at any given time there's always a sort of a a most successful thing yeah

maybe if deep learning wasn't a thing maybe it'd be talking about crisp up maybe it'd be talking about you know whatever it is maybe you know maybe we wouldn't think about solving sort of the protein structure prediction problem as a really a success of AI maybe we would have figured out how to doing it with sort of curve fitting like you know more broadly construed and we'd just be like oh wow like we took a lot of computing resources but but

protein structure prediction might you know be an enormously important thing so there is always

“sort of a biggest thing and and I think what you're pointing at is more a consequence of”

of the way in which attention gets centralised yeah but it's basically fashion is sort of what I'm saying it's not just fashion but but but there is some dynamic there there's a very interesting and important implication of this idea that the branching is so wide and so contingent and so path dependent that different civilizations stumble on entirely different technology sex yeah there's a very significant location that there will be genes from trade

yeah into the far-far future which might actually be one of the most important facts about the

far future in terms of how civilizations are set up how they can coordinate how they interface with like there's not this like go forth and exploit it's actually there are humongous gains to trade from adjacent colonies or whatever that yeah sort of there's a question of like what's actually hard you know if it's a question of if it's just the ideas well those spread relatively quickly it's relatively easy to to share ideas if it's something more it's sort of a

Dan Wang kind of an idea where it's actually sort of there's some notion of capacity you need all the right text you need all of the right manufacturing capacity and so on and so you know civilization A has very different kind of manufacturing capacity and that's just not so easy to build

“in civilisation B even if civilization B is kind of ahead then then I think that that becomes true”

there is actually you know comparative advantage which is really worth I mean it's going to provide massive benefits to trade in both directions eventually you're going to expect some diffusion of innovation um it is funny to think about what the barriers are there are fun thought experiment I like to think about is sort of you know GitHub but for aliens so you know somebody presents you with all of the code um from some alien civilization and I mean I don't even know what what code

it would have many interesting new ideas in there and it would take forever for human beings to dig through and to try and extract all of those that the one reason I mean that the origin of this in for me was actually thinking about proteins in nature yeah we've been gifted just this

incredible variety of machines which we don't understand really at all and we just have to go and

sort of try and understand them one out you know one by one basis we're still understanding hemoglobin and insulin and things like this and no doubt yeah and there's hundreds of millions of proteins known so it is it is a little bit like that we've been gifted by biology just this immense library of machines no doubt containing an enormous number of very interesting ideas and we're just at the very very very beginning of understanding it so actually I mean that that's that's I suppose

kind of your point actually is you know I need to relabel your arguments slightly but you sort of think of that as as a gift from an alien civilization which obviously it isn't but you think of it that way and it's like oh my goodness like there's so much in there and we're going to study it and goodness knows how long we could continue to study it there's tens of thousands of papers about the you know hemoglobin and things like that and we still don't understand them

and yet we're getting so much out of it just I mean just think about insulin alone you know it's

“such an important such an important thing that's that's an incredibly useful intuition from”

them that you have on earth I had McLean on where he had a cereal about how life emerged but like

whatever theory you have basically something like DNA four billion years and you have

an alien civilization coming here and be like there's all these interesting things to learn about material science about it you name it right like about what the Chinese and walking along yeah I mean and we know almost nothing about these proteins and yet the tiny few effects we do know are just just incredible it's a ribosome yeah you know another example I mean this in miraculous engineered sort of device a little factory and all seated by just like there's a particular

chemistry on earth within nucleic acids and carbon-based light forms that that chemistry gives rise to all of these interesting things which in the aliens civilization would find very interesting and so that that very set that seed which must be one among you know trillions of possible seeds of I mean just of general intellectual ideas at least to all this for continuity that that's very interesting in terms of I want to meditate on the scales for trade thing because I feel like

“I think there's something actually very interesting about this idea that if you have this vision of”

what technology progresses and how it may be different from different civilizations it has important implications about how different civilizations might interact with each other like the fact that they're going to be these huge gains from trade it makes friendliness much more rewarding yes right yeah this is very important observation yeah I hadn't thought I hadn't thought about that at all that is a very interesting observation yeah um it is funny I mean you know comparative advantage

is something that people you know they they love to invoke and it's a it's a very beautiful idea obviously there are limits to it like you know it's kind of it's it's a special limited model we don't you know chimpanzees can do interesting things we don't trade with them and I think it's sort of interesting to think about the the reasons why yeah and part of it is just power I think like

once there's a sufficiently large power imbalance very often not always but very often groups of

people seem to to sort of shift into this other mode where they just seek to dominate and yeah maybe there's something special about human beings but but maybe it's also sort of a more general sort of a thing so they're not they're no longer they give up you know you need all these special things to be true before groups will trade yeah and yeah it's not necessarily obvious well

“I think the big thing going on here is one transaction costs yeah and two compared to the”

advantage is not tell you that the terms on which the trade happens are above subsistence for any given one producer so people often bring this up in the context of both humans will be employed even a post-AGI world because they've created vanish there's big there's like five different ways that are given breaks down but the easiest way to understand or why why don't we have forces all around on the roads because there's some compared advantage because of cars and horses good example there's

hue one there's huge transaction costs to build in roads that are compatible with horses

and can sort of shoot their latent states again at each other are going to find it way more

“costly than the benefit in just terms of interacting with you type of human being in the supply chain”

and second that just because there's this horses have a comparative advantage mathematically does not mean that it is worth paying a hundred k a year or whatever causes sustain a horse in San Francisco that subsistence is going to be worth the benefit you get out of the horse I do think it's interesting like that that just the she effects that you know my expectation and my intuition obviously differs a great deal from from yours on this you know is that most

parts of the tech tree are never going to be explored there's just too many interesting ways of

combining things there's too many sort of deep ideas waiting to be discovered and we're you know not only wee but nobody ever is going to discover most of them so choices about how to make how to do the exploration actually matter quite a bit interesting it's something I really dislike about sort of technological determinants arguments I'm willing to buy it sort of low enough

“day on when you know progress is relatively simple but but higher up you start to get to shape”

the way in which you you do the exploration and it's interesting you know people we are starting to shape it in in in interesting ways you know sort of I mean there's various technologies that have been essentially banned think about DDT you think about chloroferocarbons you think about

restrictions on the use of nuclear weapons and nuclear non-proliferation treaty those kinds of

things are you know they're not they weren't done before the fact but they you know starting to get pretty close in in some cases where we just sort of preemptively decide we're not going to go down that path so that starts to look like a set of institutions which where we are actually influencing sort of how we how we explore the tech tree yeah but I'm I'm where you would see these games from trade obviously it would be you'd see the most where it's pure information that can be sent

back and forth because the information has a squallity where it is expensive to produce but cheap to verify and cheap to send yep and so it'll be interesting how much of a future productivity of whatever can be distilled down to information I right now it's kind of hard to do because you can't really transfer like if China's really good in manufacturing something whether there's this

process knowledge that's in the heads of a hundred million people involved in manufacturing sector

in China but in the future it might be easier if you eyes are doing I mean the question about sort of to what extent is there you know fabrication get sort of very uniform and get really commoditized like you know three printers have been the next big thing for at least 20 years now yeah yeah why do they still not work all that well why they still not actually at the center of of of manufacturing and sort of what comes after that you know it is funny to look at say the

ribzon by contrast it really is at the center of biology a whole lot of really interesting ways and whether or not that's the future of manufacturing is something very simple sort of where everything goes as sort of as throughput through I don't know maybe it's a bioreactor or something like that so you send to the information and then you grow stuff or you have some 3D printer that actually works and yeah if they good enough then actually it does become much more of pure information

problem and some of this process knowledge becomes much less important Jane Street has a lot of compute but GPUs are very expensive and so even optimizations that have a relatively small effect on GPU utilization are still extremely valuable two of Jane Street's ML engineers Corwin and Sylvan walk through some of their optimization workflows at GDC you're not bottlenecked on the network being too slow your bottlenecked on waiting for a different rank in your training

not having completed the work they talked about how Jane Street profiles traces and diagnosis bottlenecks and then how they solved them using techniques like kuda grass and kuda streams and custom kernels with these sorts of optimizations Corwin and Sylvan were able to get their training steps down from 400 milliseconds to 375 milliseconds each this 25 millisecond difference might sound small but given the size of Jane Street's fleet that improvement could free up thousands of B200s Jane

“Street open source to all the relevant code if you want to check it out I've linked the GitHub repo”

and the talk in the description below and if you find this stuff exciting Jane Street is hiring researchers and engineers go to Jane Street dot com slash thork hash to learn more can I say very closely for a question so there's there's these deep principles that we've discovered a couple of one is this idea that hey if there's a symmetry across the dimension of course ponds to conserve quantity it's a very deep idea there's another which we

were not a lot about a textbook about in fact about there's we there's ways to understand the

with other physical systems what a universal computer looks like it's at our home and is your view that if you go down to this level of idea of no thursday or more the church touring principle that there's an infinite number of extremely deep such principles because everybody what makes them special is that they themselves encompass so many different possible ways of world could be but no it has the world has to be compatible with I'll show you a couple of these very deep

principles I don't know I mean I mean yeah I just all I have here a speculation and sort of instinct my instinct is we keep finding very fundamental new things it was very I mean for me anyway quite informative to understand as I say you know I get the example before there's these wonderful ideas of church and touring and these other people the ideas about universal program will devices and then you understand later oh this also contains within the ideas of public

kick cryptography and then you understand later oh that also contains within it the ideas I mean people refer to it as script or currency or whatever but there's a very deep set of ideas there about the ability to collectively maintain and agreed upon ledger which are built which is built upon this and there's probably you know many deep ideas to sort of alright let you talk whatever taken many years really to figure out the right canonical form of of those and so just this fact

that you you keep finding what seemed like deep new fundamental primitives i find very for me that's has been a very important intuition pump and it's a cross I mean I've given that particular

“example but I think you see that same pattern in a lot of different areas what is your interpretation”

then of this empirical phenomenon where ideas like whatever input you consider into the scientific

process of technological process economists have studied this a million in a hundred ways it just

seems to require even actually very consistent rate x percent more researchers per year so there's this famous paper from a couple years ago by Nicholas gloom and others where they say how many people are working in the semiconductor industry and how does it increase over time you know it is through the history of Moore's law and I think they find like Moore's law means computing increases 40 percent a year or transit or density increases 40 percent a year but to keep that going the

amount of scientists has increased 9 percent a year and they go through industry after industry with this observation and so is there for you that they are these deep ideas but they keep getting harder to find or that no there's there's another way to think about what's happening

with these empirical observations i mean they so first of all all of their examples are narrow right

they all they pick a particular thing and then they look at some particular metric you know nowhere in that shows up like GPUs don't show up there right like in the sense of oh yeah all of a sudden you get the ability to parallelize and that's really interesting so there's sort of a lot of external consequences that are just delighted from basically you know they have these simple quantitative measures they look at it in agricultural productivity they look at it

in a whole lot of different ways but you do have to focus narrowly and and I suppose you know I'm certainly interested as I say in this this fact that that just mute types of progress

“yeah keep becoming possible but yeah there is still I think even there they just seem to be”

some phenomenon of diminishing returns you know is that intrinsic is that something about the structure of the world what is it well one thing which hasn't changed that much is is you know sort of the individual minds which are doing this kind of work and you know maybe that those should be sort of being improved as well or some sort of you know feedback process going on there you know and and you know maybe that changes the nature of things I suppose I I look at scientific progress

up into let's say 1700 something like that and it was very slow and also it was very regular you had the ionians back sort of five centuries before Christ doing these quite remarkable things I think so much knowledge like would would get lost and then it would be rediscovered and then it would be lost again and you'd have to say that that progress was very slow and and there it's partially just bound up with the fact that there were some very good ideas that we just didn't

“have even once you've had the ideas then you need to build institutions around them you actually”

need to solve a whole lot of different problems about training about allocation of capital about all these kinds of things even just about basic sort of security for researchers so they're

not you know worried about the inquisition or things like that so there's always kind of complicated

massive sort of burst of scientific progress if you're not changing it if there's some kind of

“stagnation there if you're not changing those external sort of circumstance yes like you may”

start to get sort of diminishing returns again but that doesn't mean there's anything intrinsic about the situation you know maybe maybe something you know just external needs to change again you know obviously a lot of people think AI is potentially going to be going to be a driver I mean it certainly will at some level in fact you know to the extent you can think of a lot of modern scientific instrumentation is really I mean at some level kind of robots you know what is

the James Webb Space Telescope well you know it's unconventional maybe to describe it as a robot but

it's not completely unreasonable you know it is an example of a highly automated very sophisticated

system with electronically mediated sensors and actuators where machine learning in fact is being used to process the data so so in that sense we're already starting to sort of see that transition we've been seeing it for decades um I have this smoker joined and take a puff

“thought which I think we've had a few yeah but I think we're going to do that part of the conversation”

and you can help me get my foot out of my mouth and figure out a more concrete I think about it so the to your point that AI what there's an extra evolution the Enlightenment and now there's AI and each might be a different pace or a different way in which science happens

if you think about the pace of how fast such transitions have been happening you can draw

over a long span of human history this hyperbolic of the rate of growth is increasing so yeah 100,000 years ago you have the Stone Age you go back even much further how long ago probably it's been around it would be like let's say millions of years and 100,000 years ago the Stone Age then 10,000 years ago the agricultural revolution that 300 years ago the Industrial Revolution each marked by this increase in the rate of exponential growth and then people think it's

going to happen again with AI but that would happen potentially even faster it would not have occurred to somebody the beginning of the Industrial Revolution that the next demarcation in this trend will be artificial intelligence and so if things are getting faster and it would hard to anticipate well the next transition will be I guess we just think of this singularity between now and AI and there that's really what distinguishes the past from the future but we're just applying the same

heuristic that many people in the past should have had maybe the intelligence age is also quite short and the next thing after that is we don't even have the ontology to describe what it is but it would not the future will not think of the past as like there was pre-intelligent AI and post AI no that seems I mean obviously we can't prove this but it's certainly seems quite plausible I mean part of the issue of course is just you know the substrate we have available to to

conceive like seems all wrong you can't speculate with a bunch of chimpanzees about what it would be like to have language just to sort of pick a major transition in the in the past it's the transition itself is the thing and it seems likely if we're talking about taking a puff kind of thoughts you know I'm certainly amused by the idea that there's going to be some transition involving artificial general intelligence using classical computers but actually there'll be

an interesting transition with quantum computers as well they're probably capable of sort of a strictly larger and class of of potentially interesting computations so maybe actually the the character of sort of a qg li or whatever it should be called is actually qualitatively different so maybe it's sort of a brief brief period between those two things interesting I mean the other side you know this is just just speculation but it's certainly amusing

“is there a reason I think that because from what I understand there's been for decades”

people like you have put pretty tight bounds and the kinds of things quantum computers are going to do and so it'll speed up search somewhat it will do and the kinds of things that extremely speeds up like Schroes algorithm it seems like it again maybe this is to your point that we can't predict an advance what's down the tech tree but at least from not here it seems like you break encryption but what else are you using Schroes algorithm yeah I mean we've

only been thinking about it for 30 years or whatever it's 40 or 40 or so years not for very long and we sort of haven't in some sense thought that hard about it as a civilization so yeah

you know like a really radical expansion that seems distinctly possible like given mind as well we've been doing it without the benefit of having the devices right like that's a pretty big right what is your inside view having been in and contributing to quantum information quantum computing back in the 90s and 2000s what is your telling of the history of what was the bottleneck what was the what was the key transition that made it a real field and how how do you

rank sort of the contributions for Feynman to do a edge to everybody else who came along yeah so I mean let's just focus on sort of the question about sort of what you know what actually changed so why was quantum computing not a thing in the 1950s right like it could have been yeah um you know somebody like John von Neumann good example absolutely pioneering computation also wrote a very important book about quantum mechanics and was deeply

interested in quantum mechanics like he could have invented quantum computing at that time and I think there were quite a number of people who who potentially could have so why do we have these papers by people like Feynman and Deutsche in the 80s and those are you know I think fairly regarded as the foundation of of the field there are some partial anticipations a little bit earlier but but they were no in years as comprehensive and no in years as deep and well

“you should you should ask David you can ask you can ask Feynman unfortunately but you know”

much better than I do a couple things that I think are interesting one is that of course computation became far more salient sort of late 70s early 80s you know it just became a thing which many more people were interested in patchly for you know for very banal reasons you could go and buy a PC you could buy an apple too you could buy a Commodore 64 you could buy all these

kinds of things became apparent to people that these were very powerful devices very interesting

to think about at the same time in the quantum case that was also the time of the ball trap and and the ability to trap single ions and so on and up to that point we hadn't really had the ability to manipulate single quantum states so you kind of got these two separate things they're just for historically contingent reasons had both sort of matured around sort of let's say

“1980 or so and somebody like fundamental could have had the idea earlier but it you know is I think”

quite an interesting you know factor the story about Richard Feynman he went and got one of the

first PCs around 1980 and 1981 and he was apparently just so excited with this device you know

he he he actually tripped and hurt himself quite badly sort of carrying his brand new computing device you know that that's a very historically contingent sort of a coincidence but but having somebody who's you know very very sort of talented and understanding of quantum mechanics also just very excited about these new machines it's not so surprising perhaps that that he's thinking then what similar story could you have told 10 years earlier like there is just

no the conditions don't exist so I think that's I mean it's quite a banal story but what one of

“the things we were going to discuss was this idea you had about the market for follow-ups and I think”

this is actually the perfect story to discuss it for because you wrote the textbook while the field right you Mike and I because the definitive textbook on quantum information and so you presumably came in after Doich but you identified in the 90s somehow identified it as the thing that is worth following up on and building on and instead of talking about more abstractly I'd love to actually

just hear the story of like the first answer of how how did you know that this is the thing to

of all the things that were happening physics and computing etc that I want to think about this problem sure sure so yeah real Feynman writes this great paper in 1982 David Doich writes a absolutely fantastic paper in 1985 sort of sketching out a lot of the fundamental ideas of quantum computing so I'm you know I'm 11 in 1985 I'm not thinking about there's some playing soccer and doing whatever but in 1992 I took a class on quantum mechanics there was really terrific given by

I said do you like do you can do you have anything you know sort of papers or whatever that

that you could give me and he said come back come by my office in a couple of days time and I did and he put in me with a giant stack of of papers which included the Doich paper included the Feynman paper and it includes a whole bunch of other sort of very fundamental papers about about quantum computing and quantum information at a time when essentially nobody in the world was working on it

“he was he'd actually I think he wrote the very first paper that proposed I mean sort of a practical”

approach to quantum computing wasn't very practical but it was actually in a real in a real system and so in some sense you know I'm benefiting from the taste of this other person but as soon as I read the papers or take a look at the papers like these are exciting papers you know they're asking very fundamental questions and you sort of like oh I can make progress here like these are

these are things that one could potentially work on Doich has this sort of conjecture that basically

yeah there should be I don't know what the right term for it is thesis or what what you would call it that a universal model quantum during machine should be capable of efficiently simulating any system any physical system at all this is a very provocative idea I think in that paper he more or less claims that he's he's proved it I'm not sure that necessarily everybody would would agree with that this question's about whether or not you can say simulate quantum field theory

“effectively and that that kind of question is is I think very interesting and very exciting”

there it's obviously a fundamental question about about the universe you know he has some wonderful ideas in there about sort of quantum algorithms and where they come from and what what they mean and what they relate to the meaning of the way function and and questions like this which is yeah still not it's it's not agreed upon amongst amongst physicists so yeah there's just some sense of oh I am in contact with something which is a deeply important and b we as a civilisation

don't have this and so of course you you start to focus here attention a little bit there hmm I'm not sure I got the answer to the question that maybe I misunderstood the question yeah

yeah let me let me let me let me let me go out of there's it maybe I'll explain the motivation first

so in a previous conversation we were discussing how could you have known in 1940s the chance theorem's yeah and Shannon's way of the new acumenication channel is a deep idea that goes beyond the problems with pulse code modulation that vowel laughs was trying to solve at the time and it applies to everything from quantum mechanics to genetics to computer science obviously and one of the

“I think in an idea you you stated that we didn't if you had a chance to talk about yet visit said”

it well Shannon publishes paper there's all these other papers but there's some market of follow-up so where people gravitate to and build upon Shannon's work and how do they realize that that's thing to do and how does that process happen and so I guess you you gave your local answer you read these papers and you immediately realize okay there's work to be done here there's a little hanging for you there's some deep provocative idea that I need to better understand

and I could you know tractably make progress on them yeah I mean so you know to some extent you're sort of saying okay I you know wanted to get into this game of of contributing to humanities sort of yeah you know understanding of of the universe and you are applying sort of this this low-hanging fruit algorithm you're like relative to my particular set of interests and abilities where should I pick up my shovel and start digging and and there it was like oh this

this looks like quite a good place to to start digging you know and different people of course yeah chose very differently was it was a very unusual choice at the at the time and so it's 1992 very few people were thinking about that yeah fast-roading a bit so you've been I don't know how you think about your work on the open science movement now but did it work like well well what would have what is successful there look like what what what is it what is it that

movement is running accomplished yeah I mean the set of ideas about open science I mean it's interesting you didn't stop and and define open science there which I think 20 years ago you

with most often they have a relatively simple set of associations it means maybe something

about making scientific papers open access very often they have some set of notions about maybe it means also making code openly available maybe it means making data openly available

“but already those are I think a lot of very large successes of the open science movement”

which is to make those silly end issues those issues on which people have opinions and then there are there are relatively common arguments and argument like come so this is sort of this is sort of the meme version you know publicly funded science should be open science that's a you know that's a distillation of a set of ideas which you might be able to contest but if you can get people actually sort of thinking about it and engaged with that kind of

argument you know that's a very fundamental kind of an issue to be considering in the the the whole political economy of science if you go back say three centuries there was a very similar kind of an argument prosecuted which is the question do we publicly disclose our scientific results or not so if you look at people like Alayo and and and Kepler and so on the extent to which they publicly disclosed like it was done in a very odd kind of a way they sometimes they did

bizarre things where they you know famously they published some of their results as some

anagrams so basically you know they'd find some discovery they would write down the result

in sort of a sentence like his you know the discovery of of the the I'm trying to think of an

“example I think the moons of Mars I think was one such example”

I'm getting it wrong and mate was it hooks law anyway it doesn't matter the the point was they they they'd write it down but then they'd scramble it published that and then if somebody else later made the same discovery they would unscramble the anagram and say oh yeah I actually did it first this is not an ideal way there's not an ideal foundation for a discovery system and then it took I mean a very long time over a century I think to

obtain more or less the modern ideals in which what you do is you disclose the knowledge in the form of of a paper there is then an expectation of attribution and so there's a kind of reputation

economy which which gets built and so basically oh such and such did this work so they deserve

the credit for that and that's then the basis for their career so this is sort of the underlying political economy of science and that made a lot of sense when what you've got is a printing press and the ability to to do scientific journals then you transition to this modern situation where in fact you can start to share a lot more you can start to share your code you can start to share your data you can start to share in progress ideas and but there's no direct credit

associated to those it's not at all obvious how much reputation should be associated to them that's

“all constructed socially and so making it a live issue is I think a very important thing to have”

done and that that's I view anyway is one of the main positive outcomes of work on on open science so really practical sort of example to illustrate the problem for a long time in physics there was a preprint culture in which people would upload preprints to the to the preprint archive and in biology this didn't happen there was no preprint culture that's changing now but for a long time this was the case and I used to sort of amuse myself by asking

physicists and biologists why this was the case and what I would hear sometimes from biologists was they would say well biology is so much more competitive than physics that we need to protect our priority and so we can't possibly upload to the archive we have to just publish in journals and then I would sometimes hear from physicists physics is so much more competitive than biology that we need to establish our priority by uploading as rapidly as possible to the preprint archive

we can't possibly wait to do it with the journals and I think this emphasizes the extent to which this kind of attribution economy is actually is just something we construct is just something which we don't buy by sort of agreement and so any attempt to sort of change that economy results in a different system by which we construct knowledge and so there is sort of this very fundamental set of problems around the political economy of science you know sort of we've got

this collective project and how we mediated depends upon the economy we have around ideas

collective science or groups of people we're making progress on a problem we're no individual

understands all the logical and explanatory levels necessary to make a leap or connection outside of mathematics what is the best example of such a discovery I mean I'm not sure I have a well-loadering of them to to give you a best but I mean yeah and an example that I think is very interesting is is the LHC where it's just it's immensely complicated object I actually I years ago I snuck into an accelerator physics conference I didn't know anything at all about

accelerator physics but I was just kind of curious to see what they were talking about and this particular group of people were experts on numerical methods in particular on inverse methods and so basically turns out you know inside these accelerators you have these cascades so a particle you know will be massively accelerated maybe it'll be collided and then you'll get a shower of particles

which decays and decays and decays and there's just this incredible sort of you know consequence

“whole a shower which is ultimately what you see at the detector and then you have to retroactively”

figure out what produced it and so there's these very very complicated sort of inverse problems that need to be need to be solved you've got this final data but you need to figure out what produced it and that's how you look for sort of signatures of these and what many of these people were was they were incredibly deep experts on simulation methods for sort of following particle tracks and like this was really deep and difficult stuff and I'm like wow you could

spend a lifetime just learning sort of how to do this and how to solve some of these inverse problems and you would know nothing about or you would know very little about quantum field theory you would know very little about detector physics you would know very little about vacuum physics all these other things that are absolutely or very little about data processing very little

about all these things that are absolutely essential to understanding say that the Higgs boson

and I don't think it's possible for one person to understand everything in depth lots of people understand broadly a lot of these ideas but they don't understand sort of everything in the depth

“that is actually utilized that's why there's these papers with well over a thousand authors”

and those people can you know they can talk to one another at a high level but they don't understand each other's specialties and things like as a you know detective physics vacuum physics these kinds of solving of inverse problems like this is stuff is incredibly different from each other and and you know to understand it in real detail is serious work how do you think about prolificness versus depth where I don't know maybe Darwin's an example of somebody who's like

it is just stating on something for many decades there's other examples where I am saying during the euro comes a speciality that he's just doing a bunch of different things by he talks about how they're all relevant to the eventual buildup yeah I mean you know it's something I stress about a lot sometimes I feel like I'm you know too slow actually it's funny that I mean the Darwin example is really interesting like you know prolific what like I mean

I've gotten as how many letters he wrote it must have been an enormous number so you've certainly very active there's also like there's there's sort of there's two types of work that tends to be involved in any kind of creative project there's routine staff and there you just want to avoid procrastination you just want to like you know how do I get good at this or how do I outsource it and how do I do it as rapidly as possible and just avoid you know like getting into a situation

“where you're prolonging it and then there's high variance stuff where you actually you need to”

be willing to you know take a lot of time you need to be willing to go to the different places and talk to the different people where in any given instance most of it's just not it's not going to be an input and somehow sort of balancing those two things I think a lot of people are very good at doing one or the other but it's hard to you know it's almost like a personality trait sort of you know which one you prefer and people tend to end up doing a lot of a lot of one and not enough

of of the other so I send you you know sort of trying balance those two things I mean I instant it's such an interesting example I mean 90.05 is just this extraordinary here like you can delete special relativity entirely and it's an extraordinary you can delete special relativity and you can delete the federal electric effect for which you won the Nobel Prize and it's still an extraordinary year like a plausibly a multi Nobel Prize winning year so what's he doing

of luck as well but certainly for myself anyway like trying to identify those things that are

routine that I should get good at and then you know just just try and do as quickly as possible I mean that's yielded a certain amount of returns but also being willing to bet a little bit more on myself on sort of the variance side has also been very very very helpful that's really hard that because intrinsically you're putting yourself in situations where you don't know what the outcome is going to do and so if you're very driven to be productive and whatever and actually mostly

it's not working over there you like let's reduce this like it doesn't feel right when I worked in San Francisco actually a practice I used to have each day was instead of taking the

15 minute walk to work I would take the the more beautiful 30 minute walk to work

partially just because it was beautiful but partially also as just a reminder to think like like that there are real benefits to not being efficient but it's not an answer to your question

“I mean really I think all I'm saying is I struggle a lot with the question I mean there are these”

um it didn't eat swimming to and I forgot as exactly how you mean um has this famous equal odds world where he says the probability that any given day new release any paper book whatever will be extremely important for a given person through their lifetime is not that different and really determines in what era they are the most productive is how much they're publishing and you're given thing has equal odds of um being extremely important um maybe just think of some of the most

successful creatives or scientists they're just doing a lot like Shakespeare's just publishing a lot yeah and of course then there's kind of examples you know girl publishing almost nothing yeah but uh you know broadly speaking you know I think some like you need a very good reason to be

avoiding it there's a bit basically to not do that um it's funny I mean I've talked to a

I've met a lot of people over the years who you talk to they're clearly brilliant and they're just obsessed that they are going to work on the great project that you know makes them famous

“and they never do anything um and that seems connected like it's a type of aversiveness I think very”

often they just don't want public judgment something that I would love to see yeah there's an awful lot of of biographies and memoirs and histories of um people who achieve a lot I wish there was like a very large number of of biographies of people who were fantastically talented that's a good one yeah just missed like like you know absolutely you know I've known you know people who won gold medals at IMOs and things like that who then you know tried to become mathematicians

and failed like what what happened but what was the reason I suspect in many cases that's actually yeah it's more informative and incredibly interesting than anything else you have this I say then I was reading before the interview about how you think about what is the work you're doing and writer doesn't seem like it's to say it was Charles Saurb and a writer right what what is actually is that label I'm a podcaster right so I'm and in a way obviously our work is very different

but I I also think a lot about what is this work and how do I get better at it and in particular how I can make sure there's some compounding between the different people I talk to in the podcast where I worry that instead of this kind of compounding there's actually I build up some understanding that's somewhat superficial about a topic and then depreciates I move to them in the next topic

“and sort of depreciates and so I think there's this question there's a lot of podcasts”

in the world who will interview way more experts than I have ever have and I don't think they're much the wise or more knowledgeable as I result so there's it's clearly possible to mess this up and I wonder if you have thoughts or takes or advice on how one actually learns in a deeper way from this kind of work yeah I mean it's sort of an incredibly complicated and rich question I mean just seem like the sort of the question is like you know how do you make

it a higher growth context how do you make it a more demanding context and sort of you can do that in like relatively small ways but that might have a yield compounding returns or you can do something that is maybe more radical maybe it means actually you know starting sort of a parallel project in which you do something that is actually quite a bit different here's something I think really interesting about like how being very demanding it can simply change your your response to

just something something that I would sometimes do with with students and sometimes with myself

try and do you know this work over the coming week and then the next week would come by and you know they hadn't solved the problem or whatever you sort of like you know if a million dollars in

Bennett State like would you have put the same effort in and the answer is no sort of invariably

“like they've tried but they haven't really tried and I think that's a very familiar feeling”

for all of us you know you sort of you you often you you could do a lot more if you had just the right sort of demanding taskmaster standing by you and saying look you you're barely operating here and so I I do you sort of wonder a little bit about like you know what's the what's the demanding taskmaster what what can they ask you that is going to make your preparation way more intense the most helpful thing honestly is for some subjects is very clear how I prep think I'm doing

upcoming episode on ship design with the founder of a company that is ship design and he wrote a textbook on ship design and he yesterday I went over to his office and we brainstormed five sort of roof line analysis I can do and if I understand that I have some good understanding

the problem is with almost every other field there's not this there's not like you I don't know

when I interview to Ilia for three four years ago it's like implement the transformer and if you will end it like you have some nugget of understanding you've clamped down and with other fields it's just like I vaguely understand this it's not clamped I vaguely understand this I vaguely I've learned about the asylum and about this but there's no forcing function that you do this exercise and if you do it you will understand yeah so I mean really what you sort of saying is you can do a

good job at a podcasting without actually attaining this experiment and that's the problem from your interview yeah you want to sort of change your job description so that you you are internalizing these channels and just getting this kind of integration each time and it seems to me like you you know what that means is you actually want to change the structure of the like like the work output at some level I mean lots of people think there's this terrible idea people have that

they should be in flow all of the time yeah and of course as far as I can tell like high performance just don't believe this at all they're in flow some of the time like you suddenly see this with athletes you know when they're actually out there you know playing basketball or tennis or whatever ideally you know they are in flow much of the time but when they're training they're not they're stuck a lot at the time or they're doing things badly and I suppose I wonder what that

looks like for you that I would be extremely satisfied with the problem is I just like I don't know

what they're going to do the 64 lapses for almost it and so this is sort of a this is a thing you can change by choosing guests where there is a legible curriculum and so maybe it's a mistake for not having done that or also like there's no real way to prefer parents tower or something

“and like there's no curriculum that's like a plausible one I think there's one failure mode”

so there's many failure modes but one is if you you could do one dynamic I'm worried about a long-term dynamic is that you do good you can have a good podcast and there's a local maximum but for no particular or desktopic are you going deep enough that you I think my model of learning is there's if you don't really understand the deeper mechanism you're just mapping inputs and outputs of a black box yeah and that just fades incredibly fast or is not worth it in the first place and

yeah you kind of just move on and so on and so on and you kind of need to build the intermediate connection and it's it's unclear I think actually AI in a weird way is really easy for that reason because there is a clear thing you can do just implement it right and then you understand it we're almost if I apply that criteria and elsewhere what am I how do I just not do history episode let's say to pound exactly hey to pama like what what you know wonderful to talk to incredibly

interesting but for you personally like what change right yeah there's some things they learn I think I could have done if I maybe allocated more time especially after the interview to like let's write out of two thousand words and everything I learned and how can I show the fix I know it's something and maybe that's the thing we're doing is spreading out the episodes more and spending more time afterwards consolidating but yeah I think the I would say basically infinite amounts of money

if there's somebody who's really good at coming up with here is here's the curriculum and here's

“the practice problems you need to do and here's the exercising you do after the interview to”

clamp what you have learned have you tried doing that with somebody it's hard to find so I mean I've made I've tried super hard but it seems exactly it's a day we'd tough to find somebody who could do that for every single kind of discipline maybe I should just hire different ones for different

for each episode and I mean as far as I can tell I like that's the only way I really understand anything is that you know I get interested in something at first I don't even have a problem but there's just some sense of their some contribution to my care and gradually you hone in and there's a problem and then you I mean finally enough I mean spending time stuck is incredibly important and I sort

“of you know that you should just be annoying now it seems like oh this is actually”

maybe even the most important part of the whole process but that very hard oneness of it means

that you know I internalize it I find actually if I you know I've written sometimes 10,000 websites in you know a couple of days and I've written them in you know three months or six months I feel like I I didn't learn very much from the ones that that I need to take a couple of days whereas you know some of the ones that that took three months I'll be you know 15 years later I'll I'll still remember you can you just come outside of physics how you learn of the one that took

three months I mean by far the most you know the the the common things there's always some creative artifact sometimes it's a class you know sometimes it's engagement with a group of people who you know there's some collective creative artifact that you know you're you're working on together I mean you might not even be aware of it but you you know you're acting as an input to their creative ends in some way and sometimes it's just you know it's an essay or a book

or or whatever you know it's one of the reasons why I you know often quite enjoyed doing podcast so I mean particularly I mean I you know I I said yes to come here partially because I know you ask unusually demanding questions and so it's sort of that's an attempt to take it this sort of perspective from a different it's a different kind of a forcing function so you're trying to pick sort of the most demanding creative context yeah so for this interview I went through like three

electors of the suscens natural it's really well the problem is that there's almost no practice

problems in it and so I heard um a physicist friend who's gonna like I haven't done it yet but see it's like every lecturer I want like a bunch of practice problems scripted them and I'm I'm planning on being um the head of appropriately humbled how do you how do you make it as jugular as possible right like the higher you can raise the stakes the better I mean the interview is in some sense high stakes but also it just necessarily test deep understanding yeah but I don't

think the interview is that high stakes right you're not writing a book about special relativity and you're not trying to write a book that replaces the current you know whatever the existing standard textbook is like that that's a really high yeah really high say well the phrase that I sort of find particularly difficult and it's funny when people will talk about going deep on a subject and it turns out you know different people have different ideas of what this means some people

means they read a couple of blog posts some people it means they read a book about it some people

“it means they wrote a book about it and and and I think like sort of what what what what what your”

standard is the sort of the standard you hold yourself to the heavens a lot about you know your ability to to integrate knowledge in this way I know what your experience has been but I found that I'm getting I'm in some sense it was a move much faster on some things to the help of AI but I don't know if I'm like learning better yeah and I think it's probably because the hardest thing the thing that is most demanding is silver so you try to take any excuse you can to get out of it

yep and just having back in for the conversation now I'm where where you gloss over entertaining but not necessarily anything else yeah so it's such an easy way to get out of the thing yeah I mean in fact it makes it easier because instead of doing some intermediate thinking there's

always next question you can ask a chatbot yeah and and and it's somewhat valuable like it's

not I mean that's part of the productiveness of course like it's not actually not actually useless

“but yeah it can sort of substitute for actually doing the thing that that maybe you should be doing”

it's interesting that like the extent to which you know to what extent should you be outsourcing that kind of stuff and to what extent do you know like like it's really there's some sort of interesting judgment call about you know you actually there is a whole bunch of routine work that that you want done and in fact it's it's low value for you so you may as well get if you can write a chatbot to do it you may as well so somebody interviewed the pioneering computer scientist

Alan Kay years ago and he was asked what he thought about basically Linux and if I remember his answer correctly basically basically you know it doesn't have anything to do with computer science it's just

understanding but mostly you're all you're learning is stuff about Linux like you're not actually

“learning anything which is transformable there's like a very like that there's a certain kind of”

seductiveness to some things where you know it's sort of a real goldberg machine you can just

sort of learn about all the bits and it feels kind of entertaining but if you step back and think

“about the question what am I actually doing here it might not actually be meeting your objectives maybe”

you want to become a you know since admin and learning Linux is a great use of your time there's no

no harm in that at all but if your answer is if you're objective it is to understand the fundamentals

“of computing it's much less much less clear but that's a good use of your time I think that was”

it's certainly an answer I've thought a lot about yeah you you actually need to you that for a certain type of mind there is a seductiveness in just just learning systems and confusing that with with understanding okay I'll keep you updated and not discuss you know I owe you a text or then a month of some revamped learning system I'll be really curious if you I mean it's also true right like 20 incremental improvements in this I mean they just with Simon it's sort of the

main input into the podcast you know it's great that the bookshelves are fancy and I've got a platform or whatever but really like the thing that makes the podcast better is if I can for the learning I do so it's yes it's worth every morsel of improvement yeah all right thanks for the thanks for the therapy session you know good net done thanks Michael all right thanks flyers