What Yann LeCun is Missing: Karl Friston and Gary Marcus on Uncertainty, Agency, and the Alternative to Deep Learning

Speaker 1 – 00:00

Foreign.

Speaker 2 – 00:12

Hi everyone, and welcome to the Spatial Web AI Podcast. This video is something I feel incredibly

grateful to share

with you today. While attending the International Workshop on Active Inference this past October, I had

the great

privilege of sitting down with world renowned neuroscientist Dr. Karl Friston for an extended

conversation.

Interestingly enough, one of my questions for him was in regard to Yann LeCun, exactly one month

before LeCun’s

departure from Meta. Jan is very familiar with Karl’s work on active inference and the free energy

principle. Yet in a

conversation at the World Economic Forum in Davos more than a year prior, Jan was adamant that he

agreed with

Karl’s direction. Yet he insisted that we don’t know any other way to train systems than deep learning.

So I asked

Karl why? What is it that Jan Lecun is missing?

Speaker 2 – 01:08

I also ran into Gary Marcus, who was on such a tight schedule that day, but was gracious enough to

give me a few

minutes for a short interview right before he went on stage with Karl for an interview discussion that

was

moderated by Tim Verballen of Versus, who is one of the original organizers of the IUI annual event.

There’s no

formal intro to any of this footage. It was all shot on the fly as opportunities were arising. We simply

drop into the

conversations as they happened and I’ve included all three of these segments, including the interview

with Karl

and Gary moderated by Tim, and it’s all in this one video that you’re about to watch today.

Speaker 2 – 01:48

I’d also like to extend my gratitude to the International Workshop for Active Inference for welcoming me

as a media

partner this year and for the extraordinary work that they do in bringing together the most brilliant minds

in active

inference from across the globe to share their cutting edge research and exchange ideas. This is an

event that has

grown year over year and I think it will continue to gain much attention as active inference gains more

and more

traction for its capabilities in fields like robotics and autonomous adaptive systems. I hope you enjoy

these

conversations as much as I did. Karl and Gary, thank you so much for your time and for so graciously

entertaining

my curiosities. So thank you so much for being here and enjoy the video.

Speaker 3 – 02:36

It is my great pleasure to have a few moments with Dr. Karl Friston today to talk about what work he is

doing and

really leading in this entire active inference field. Karl, it’s been a dream of mine to have a conversation

with you for Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

1 / 43

quite some time and so I I have some burning questions. You are at University College London and

Geoffrey Hinton

also came from University College London, and while you were there, he was working on DeepMind.

You were

working on something called Nested Minds Network. And I know that some of my viewers are probably

very

curious. Why did you not join with DeepMind and what they were doing? What is it that made you want

to pursue

this active inference direction for AI for autonomous intelligence?

Speaker 4 – 03:36

That’s a very challenging question to ask. First of all, it’s lovely to meet you in person. I read a lot of

your material.

It’s nice to actually be able to talk to you.

Speaker 3 – 03:44

Thank you so much.

Speaker 4 – 03:46

Yeah. So the history of this was were located in Queen Square in the Bloomsbury area, the sort of

intellectual heart

of London, publishing and academic and actually clinical. We were briefed and were developing at that

time, state

of the art methods and analysis tools to understand the living brain using neuroimaging. And then

fortuitously,

Geoffrey Hinton arrived with Peter Dayan to direct the Gatsby Computational Neuroscience Unit, which

was just

next door. So were academics. In fact, interestingly, Demis Hassebis, who subsequently became Sir

Demis and got

his Nobel Prize, was one of our PhD students. So he sort of learned about the hippocampus, working

with Eleanor

McGuire with us doing theoretical neurobiology and again, state of the art image analysis procedures.

And then he

went off and founded Google DeepMind and then asked Jeff actually to be an advisor.

Speaker 4 – 05:00

So that was, if you like, the divergence between academia and industrial applications such as Google

DeepMind.

So it wasn’t really a question of why did we not join Google DeepMind. We were just admiring

spectators. And

when I say we to a certain extent, Geoffrey Hinton as well, he had primarily an academic appointment.

So our brief,

both ourselves or myself and Jeff, was really to do theoretical work, advance the field, but also teach

students and

raise the next generation of theoreticians.

Speaker 3 – 05:48

And.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

2 / 43

Speaker 4 – 05:52

I never see myself as diverging from Geoff, although I do see over the subsequent decades, Jeff

becoming

somewhat seduced by the engineering power of Deep rl, as we see evinced in generative AI in large

language

models at the moment. To my mind, and I hope he never sees this, I think that was something of a

distraction from

the high road of doing it properly. And he certainly was inspirational and instantiating, certainly in me

and a

number of my colleagues, the basic principles of artificial intelligence, which of course are those of

natural

intelligence, which as brain imaging scientists were deeply invested in trying to unearth.

Speaker 4 – 06:41

And those notions at that time, when were both in London, were focused Very much on the notion of

the brain as a

generative model, as a, well, any intelligence basically working under a generative model that would

explain the

causes of the sensory inputs. So this is a very different use of the notion of generative models from

generative AI.

So you could argue that generative AI as understood in machine learning is exactly the opposite of

what Geoff was

working on and what I have been working.

Speaker 3 – 07:20

On since that time, which is really generating understanding, right? Models of understanding. Yeah,

very different. I,

you know, I tend to describe to people that right now what we’re seeing with current AI, that it is the AI

that is built

for our current Internet. You know, our current Internet is websites, web domains and applications. And

you can

generate content and use content, generate data. And that’s what these AI are doing. They’re

generating content

and doing things with data. But I feel like what you’re working on with active inference, that’s for what’s

coming

with this new layer on the Internet that takes us into spatial web and takes us into this grounding layer

of context

for these active inference AI models and agents that can then be distributed through networks.

Speaker 3 – 08:16

And I know that is deeply attributed to the white paper that you and Versa’s partner put out, I think it

was December

of 22 on, you know, distributed ecosystems of intelligence. So do you think. Do I have that right? Is that

how you

would describe it? Is there anything you would add to that?

Speaker 4 – 08:36

Nothing I would add, but I’ll certainly pick up. You used all my favorite words there, or some of my

favorite words.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

3 / 43

Speaker 3 – 08:42

I’m sure I got them from you.

Speaker 4 – 08:45

Some of them, others not. But so just to pick up on a couple of things that you said though, which I think

are

absolutely clear, you’re talking about the sort of the next level, and I guess most people understand the

next level

as moving from AI to some kind of generalized or super intelligence or as versus likes to say, a

universal

intelligence. And what is that? Well, I think you said grounding. I think that’s really important. So to have

an

understanding of how your content was generated, that has to be grounded in meaning.

Speaker 4 – 09:18

So I think understanding and solving the grounding problem is basically the bright line between current

generative

AI that just generates content and the understanding underwrites that and enables the right kind of

message

passing on the next generation of the World Wide Web or certainly as that envisaged by the Spatial

Web

foundation and as installed in the IEEE standards, for example. So what is that predicated on? I think

you’re

absolutely right. It’s predicated on understanding. But there’s another key word you mentioned, which is

of course,

to pass messages, you have to instantiate that as an agent. You have to actually broadcast. You have

to decide

what to send and what to listen to. So your use of the word agent, I think, is really important here.

Speaker 4 – 10:11

So to move towards a universal intelligence that could be unrolled in terms of our transactions on the

World Wide

Web, I think, is to basically understand the physics of agency. And to be an agent under this premise

that

everything has to be predicated on a generative model means that you have to have generative models

of the

consequences of your own action. I think that’s really what active inference speaks to, that you have the

ability to

model what will happen if I do that.

Speaker 4 – 10:47

And once you’ve got that in mind, then not only do you have an understanding of the way the cause,

effect,

structure of your world and the way the content is generated, but you can now select the right moves in

accord

with the right objective functions, which for me, of course, are the expected free energy, the variation of

energies,

or more simply, garnering evidence for your generative model of the world and in so doing, understand

how your

content was generated. So I think agency is an important aspect of that. And I guess you could sort of

induce that

from a very, you know, from what’s on the tin in active infants. It’s about action. And then you ask, well,

are not. Is Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

4 / 43

not current generative AI agentic? Well, no. I mean, to be agentic, the large language model would

have to start

prompting you.

Speaker 4 – 11:40

It would have to have the kind of actions and the imperatives that underwrite those actions that you and

I possess,

which is effectively information seeking. It’s curiosity. I mean, why do we use the web? We don’t use the

web to

feed ourselves or to look after our children. We just use it to resolve uncertainty and get information. So

it’s a great

tool. But if you really want that to work in the spirit of that ecosystems of intelligence thesis that you

mentioned

earlier, then the users on the web, each node of the web really has to be seeking information about

what? Well,

about everything else that’s on the web. So we’re just trying to build models of each other or artificial

and natural

intelligences that. That are actually exchanging.

Speaker 3 – 12:30

So like a reciprocation of curiosity from both parties or all parties?

Speaker 4 – 12:35

Yes.

Speaker 3 – 12:36

Yeah. Okay, So I have another question for you, and feel free to answer whatever part of it that you feel

comfortable with. Two years ago in Davos, you and Yann Lecun were on the stage together. I’m sure

you saw. I

wrote an article kind of summarizing what I saw and witnessed in that conversation. But one thing that’s

kind of

stuck with me, and I know it’s a question people have asked me. You guys are obviously friends. You

are well

aware of each other’s work. You have a lot of high respect for each other’s work. I mean, it was really

clear that he

has so much respect for your work with active inference. And in that conversation, he was saying, yes,

Karl, I agree

with you, Karl. You know, with active inference, everything. But we just don’t know how to do it any

other way.

Speaker 3 – 13:25

And he was referring to reinforcement learning. Why do you think that somebody who is well aware of

everything

happening in these ecosystems, or maybe there’s something he’s missing that he’s not aware of. Right,

but why?

Why do you think he has that perspective? And then the larger question in that is there are a lot of

really big tech

companies who are, you know, building a lot of tools in AI, but they don’t seem to be looking in the

direction of

active inference very seriously. I know since that conversation in the last two years, there’s been a lot of

progress

with the research around scaling and things like that. Was it just a scaling issue? Is it because it doesn’t

fit the

centralized business model of a lot of these? Like, what do you think is the. The issue there?

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

5 / 43

Speaker 4 – 14:16

What, amongst the five issues that you.

Speaker 3 – 14:18

Brought to the table, they’re all kind of interrelated, right?

Speaker 4 – 14:21

No, they certainly are. They certainly are. And different perspectives and the genesis of, you know, so

let’s take Jan.

And what’s the difference between somebody like me and something like Jan? I think it’s a very simple

difference. I

am committed to first principles and was trained as a physicist and think as a physicist, he is a really

skillful

engineer. And he’s more like my father, who’s a civil engineer. So engineers, if it works, that’s the way

to do it. From

my point of view, no, you need to identify the first principles. But I would say that despite the fact that

Jan uses the

same words in a completely different way than I do. So for him, he says, don’t use generative models.

Well, what he

means is don’t use generative AI.

Speaker 4 – 15:06

Whereas, as opposed to the kind of generative models that Geoffrey Hinton introduced, in the context

of things like

the Helmholtz machine and all that wonderful work, you could, I think, trace through to current

technology via

things like variational autoencoders. So I think there’s a much greater convergence between Yann

Lecun and

myself and certainly everybody in the active director interest fields and appear so on the surface.

Certainly his

work with the jet, you know, his more recent work trying to move the field to a more sort of biomimetic

kind of

approach. Technically there is one thing that the engineer in him makes, denies him. So you ask, what

is he

missing? He’s not missing anything. It’s just that he’s self denying, self sabotaging, I would say, in a

particular way.

Speaker 4 – 16:02

And that’s that he does not think it is in an engineering sense easy or possible to include uncertainty

into his neural

networks. So the way that translates is he would say to me, Karl, I think you’re absolutely right. We just

need to do

computation, we need to do sense making, decision making under the right kind of generative models,

in my sense

of the word, using a free energy functional, exactly in the same way that active inference appeals to

variational free

energy. But he says, well, why don’t we just put the temperature to zero? Now what that means

technically is you

remove uncertainty from the objective function.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

6 / 43

Speaker 4 – 16:45

So now he can work with really simple reinforcement learning like neural networks that don’t have

uncertainty, they

don’t have a calculus or the capacity to deal with belief structures where a belief is read as a probability

dispute. To

have a probability distribution, a belief, you have to equip it with precision or uncertainty. And that has

to be

instantiated at your. From an engineering perspective, he doesn’t do that. So he does everything

exactly like I would

like it done. But he does it, he takes the, he takes a sort of engineering shortcut and just ignores the

uncertainty.

But that you pay a very heavy price for that.

Speaker 3 – 17:23

And what is that price?

Speaker 4 – 17:25

That price is you’re not now able to deal with uncertainty in a principled way. One really important

consequence of

not dealing or encoding uncertainty, sometimes known in engineering as uncertainty quantification, is

you can’t

evaluate the quality of your models because in order to evaluate the quality of your models, you have to

average

over your uncertainty. So literally the model evidence is something that can only be evaluated if you

know the

problem, if you know your uncertainty. So you have to be able to quantify that and evaluate it. But more

importantly, if you don’t know what you don’t know in terms of evaluating uncertainty, you don’t know

what

questions to ask. So you can never be A large language model that prompts, that asks questions. So

again, we

come back to this. You know, what is agency?

Speaker 4 – 18:24

Well, agency is just querying the world in a way, in the right way, that resolves your uncertainty. But in

order to know

what the right way is, you have to know what you don’t know before asking the right questions. So that,

I mean in

one sense that is the heart of active inference. It’s that sort of epistemic affordance or imperative, that

information

seeking aspect that makes active inference, if you like, much more like you and me, much more like

natural

intelligence.

Speaker 3 – 18:52

Right. And that’s where the minimizing complexity comes in. Right. Whereas with, you know, deep

learning and

these LLMs, it’s all the information that it has to go through, whether it means anything or not, to the

prompt at

hand, but it still has to waste all of this energy.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

7 / 43

Speaker 4 – 19:09

Absolutely.

Speaker 3 – 19:10

Whereas if you have an agent that understands what it doesn’t know, and it can seek exactly what it

needs to know,

rather than shifting through a bunch of needless information, time, energy, all of that.

Speaker 4 – 19:24

Yeah, we’ve missed the other four points you want to talk about, but I think that’s such an important

point. I think

we should just unpack that a bit. Yeah, so. Absolutely. So technically your objective function, if you use

a full free

energy, which is just basically an expected energy, which is what people in reinforcement learning

might use, and

it’s got the entropy term, which is the uncertainty part. So usually you multiply the entropy by

temperature, which is

why Yan gets rid of it by setting temperature to zero. But if you’re trying to minimize your free energy,

you’re going

to maximize your entropy. That may sound strange, but it’s not because it keeps your options open. It’s

exactly

Occam’s principle. And if you now put the.

Speaker 4 – 20:07

If you just mathematically express complexity, then the complexity is effectively the difference between

your

posterior and your prior, which is basically the negative entropy that we’re talking about. But now

thinking about

the sort of uncertainty associated, or your priors. So there’s a deep connection between this epistemic

affordances, information seeking aspect of true intelligence or universal intelligence, Occam’s principle

and

complexity minimization. And furthermore, if you’re talking to people like Joachim Schmidt here about

complexity

minimization, and if you go in the other direction, it’s exactly the same as James’s maximum entropy

principle. All

these are mathematically the same. But just to focus on what you focused on, which was a complexity,

I think

that’s a really key thing. So, because you can’t. The machine learning as it currently stands, of the kind

that does

large, you know, say transformer architectures cannot encode uncertainty.

Speaker 4 – 21:13

It can’t measure its complexity. So what does that mean? It basically means they’ve got to find all sorts

of

engineering widgets and gadgets and heuristics to elude the problems of overfitting and not complying

with the

right complexity constraints because it cannot be measured. So you get sort of mini batching, you get

dropout, you

get all the little things that engineers have found that have worked over the past decades, if not

centuries. They’re Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

8 / 43

all sort of little devices that are just symptoms of the fact that they’re using the wrong objective function.

Because

they’re overfitting. Exactly. Because they’re not paying attention to the complexity term, which is one

way of carving

up the free energy of the. Or the evidence.

Speaker 3 – 22:00

Interesting. So I’ve had a question in the back of my mind and, you know, we may or may not use this

because this

is purely a personal question that I have, but if it makes sense, I, you know, we can include it. This is

literally

something that I have wondered because, you know, I understand that active inference is looked at and

the free

energy principle is looked at as the physics of intelligence, you know, for how it works. And when I look

at the free

energy, how, you know, and active inference, and you have this action, perception, feedback loop in it,

the whole

objective is to close that gap on uncertainty. Right. But to me, when I look at physics and what I

understand of

physics, you know, superposition lies within uncertainty. Right. And we kind of want to leave that open

because.

Speaker 3 – 22:51

Because that’s where possibility lies. Right. So how do you, you know, how do you, how should I

resolve that in my

mind, you know, if, if the way we learn is this constant, you know, loop of trying to get to certainty and

close that

gap, yet possibility is in there. We kind of want to play in that space. So where’s the balance?

Speaker 4 – 23:15

Right. Well, I think the. But you can read this in many ways. You could read it in terms of the bounce

between

exploration, exploitation. You could read it in terms of the bounce between accuracy and complexity.

You could

even read it in terms of what were just talking about, which is the balance between the energy and the

entropy. So

any one of those, I think, would make you happy. But let’s just take, let’s go back to free energy, carved

or split,

decomposed into energy and entropy. You want to minimize your free Energy, therefore, you want to

maximize your

entropy. This just is James’s maximum entropy principle. It just is finding those solutions, those

explanations for

your lived world or your content, that have the greatest uncertainty, that have the greatest span within

which you

can play around.

Speaker 4 – 24:04

So just to put that sort of in narrative form, you want to avoid committing to an overly precise

explanation. You

want to have that latitude in your head. Which of course brings us back to Occam’s razor or Einstein’s,

you know,

keep everything as simple as possible, but no simpler. So there is a Goldilocks regime that plays

exactly to your

notion of, you know, how do you get that balance right? The other way, I think, which sort of goes well

mathematically is a special case of this. Of this balance would be when you apply it to what we do. I

mean, at the

moment, I’ve just been talking about finding the right explanation for some content.

Speaker 4 – 24:51

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

9 / 43

You know, as a scientist who’s trying to comply with Occam’s principle or maximum entropy principle or

the free

energy principle, when it comes to what am I going to do next? I think that’s quite interesting in terms of

doing stuff

that is exploratory, that renders now surprise alluring. So renders the opportunity to resolve uncertainty

means that

if something is surprising, there’s an opportunity to act in a way to find out what caused that, what

would happen if

I did that. So that, I think, makes us into sort of curious creatures simply because we’re trying to

minimize

uncertainty in the long term. But now we find cues that enable us to or portend an opportunity to resolve

uncertainty. Very alluring. You know, you could discuss in terms.

Speaker 3 – 25:51

Of epistemic affordance, expansion of opening up your. What are the options before you click close

and, you know,

absolutely have the observation.

Speaker 4 – 25:59

Yeah, yeah. And if you take that notion not just to sort of beliefs about hypotheses or explanations for

the data

from the current state of affairs that you’re trying to infer, but also to the very structures and. And ideas

and

constructs that you bring to the table. You’re now in the world of structure learning. And how do you

grow your

models? How do you reconfigure your generative models in a way that’s apt for this situation or indeed

can explain

this novel situation that you’ve never encountered before that I think is a hallmark of. Certainly it is

active inference,

but it’s in the domain of the various structures that we bring to the table to make sense of things. You

may ask,

well, how does machine learning and its Current guys cope with that.

Speaker 4 – 26:53

It just starts off with something that’s too big, overly expressive, overly parameterized and shrinks it.

What you’re

talking about is how do we organically and in the right way grow our little models, you know, to

accommodate and

explain things that we’ve never seen before. So there’s this sort of top down approach that we see in

current ML,

which is vastly inefficient of course, because you’re completely over parameterized versus you as an

infant, a

neonate, growing your little models, growing your brain through it, through epistemic foraging. So that

will be a

natural way to get to the right level of complexity in the models.

Speaker 3 – 27:31

So almost like optimizing the learning opportunity in every moment, right? Yes, Interesting. So the other

question

that I have for you then is the work that you’re doing with Versus, what is the most exciting thing about

the work

that you’re currently doing in the research and with what you see evolving with their genius platform and

you know

where you see that heading, what excites you about that?

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

10 / 43

Speaker 4 – 28:00

I get excited by different things every week. So my general rule of thumb, me too. So effectively what is

exciting, it

changes week by week. So I’ll just answer in a very sort of simple minded way. What is currently

exercising me and

my colleagues at Versus in one part of the endeavors? I can think of many other exciting things, but

one particular

one is actually closely what were just talking about, which is this. How do we, how do you optimize the

learnability

and how do we respond and act in a way that will resolve uncertainty about the very structures that we

bring to

solve problems.

Speaker 4 – 28:55

So I think this is nicely exemplified not just by the current preoccupations or versus in terms of

aspirations of

equipping the web or the community or our lighthouse customers, you know, with the most intelligent

solutions,

but also something which I see emerging in the machine learning community, which is this notion that

we need to

move to the next stage. Sometimes cast in terms of moving away from Kahneman System 1 to System

too. So

we’re talking about understanding as revealed by the ability to reason. And in my world, reasoning

basically is a

quintessentially agentic thing. To reason is a verb. So it involves some kind of action. I think it’s exactly

the kind of

action that were just talking about.

Speaker 4 – 29:48

It’s basically acting in a way very much like a scientist would to design experiments that disclose

evidence or data

from the world that allows you to disambiguate between all sorts of different hypotheses that you might

have base

the structures, the explanations about which you’re reasoning. So if you’re a scientist, you keep your

world quite

simple. You’d have your favorite hypothesis, your alternate hypothesis and your null hypothesis. And

then you

design an experiment. Experiment that gives you exactly the right kind of data that will disambiguate

between the

two. I think we do that all the time online, all the time throughout our life. We do it a lot more when we’re

young, but

we still do it. So people like you do it in terms of your epistemic foraging.

Speaker 4 – 30:38

We are constantly building hypotheses and models and then getting the right kind of data that enables

to say, no,

it’s that hypothesis or that set of hypothesis and not that, and then continue refining and growing and

refining and

growing, refining. So that’s what’s exercising me this week. How do you do that automatically? I think it

can be

done. It’s just an extension of active inference, not to the problem of inferring states of affairs in the

moment. And

it’s even beyond what David McKay would call active learning, which is basically getting the right data

that will

enable you to infer the parameters of a given model with a certain structure without going even beyond

that. Now

we’re asking getting the right data that enables you to disambiguate the structures in and of

themselves. So that’s

a really fascinating problem which we’re currently. I should say.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

11 / 43

Speaker 4 – 31:33

Yeah. Another, I think, point, another pointer to this sort of people’s. I think we’re getting bored with

large language

models as we see them, and people keep going about the hype cycle and the bubble, which may or

may not be. It’s

slightly beyond my comfort zone, but certainly I see an enthusiasm for what’s next.

Speaker 3 – 31:57

Yeah.

Speaker 4 – 31:57

And I think what’s next is exactly this. It’s. It’s this truly grounded ability to reason as. And, you know,

what brought

that to the fore for me was the focus of the NeuroAI workshop at NeurIPS last year. But also then I

attended

System 2 sessions. So Francois Chollet introduced ARC3. So I think that style of thinking, that move, I

think is

symptomatic of the fact people are hungry for the next big thing. Yeah.

Speaker 3 – 32:30

And I think that people are starting to realize the potential power with having these intelligent agents

that we can,

you know, kind of evolve with in whatever way that looks. Right. But we need to be able to trust them.

We need to

be able to have them explainable. And, you know, and the thing that I like about active inferences is

you’re not just

talking about, you know, building these systems that can cooperate or collaborate with you, but these

are co

regulating there. And I think it has to do with that symbiosis you’re talking about how it even relates to

us. You

know, there’s a give and take, there’s a, a full circle of a relationship happening there. So to me that’s

really exciting

because I see that that’s the potential. And I think that leads me to my last question for you.

Speaker 3 – 33:32

I’m really curious because you probably have a much further lens on all of this than any of us do. So if

you’re

looking 10 years, 20 years down the road, what do you envision life is going to be like for people?

Given that all of

this technology keeps going down the trail trajectory that you see it on its path, what do you think it’s

going to be

like for humans to live 10, 20 years from now?

Speaker 4 – 34:03

It’s not going to be a very clever answer. I think you’ll be surprised how similar it is to live now, but

without all the Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

12 / 43

angst that we currently experience. I often, you know, so for example, I can remember watching 2001

Space

Odyssey and thinking, wow, I hope I’m alive to see 2001. And then you look at 2001, then you look at

2021 and

actually from. In many respects nothing has really changed all the good bits of life, nothing has really

changed that

much with the exceptions of iPhone. But you know, we had telephones, you know, when I was, you

know, when I

was growing up. So I don’t think much is actually going to change very much. And of course from the

point of view

of theology principle, the whole point is you’ve got this pullback attractor.

Speaker 4 – 34:55

The whole point is it’s a physics of sustainability. You know, if we are here, and we are here in a

characteristic way,

then mathematically speaking then we are part of the, this attracting set and that sort of unpacks in a

sort of scale

free fashion which means that, you know, we should celebrate retaining the things that characterize us.

So I think

to answer your question, in 10 years we’re doing exactly what we’re doing now. But hopefully there will

be less

worries and uncertainty.

Speaker 3 – 35:30

We’ll solve some big problems maybe with us.

Speaker 4 – 35:32

Well, for me of course problems just are uncertainty and of the kind that you’re induced by geopolitical

conflicts, of

the kind that are induced by your personal life trajectories of the kind that are induced by am I a

member of this

group or that group? Does that person like me? What would happen if I did that? All the problems that

we contend

with are just resolving uncertainty. So I think if the free energy principle is going to be realized or we

conform to the

free energy principle, what that means is that all the tech that is developed will be in the service of

jointly

minimizing our uncertainty and our free energy, which just basically means there will be a better joint

mutual

understanding. You mentioned this in terms of symbiosis or reciprocity, generalized synchronization.

Speaker 4 – 36:35

These are the dynamical hallmarks of self organization to this attracting set that characterizes us as a

particular

species. So the end point of this is basically communication, because via communication we resolve

uncertainty

about ourselves. That’s not to say we’re all going to be homogenous. There will be in groups and out

groups and

there will still be an epistemic affordance of being curious about what this out group says or this

community or

this theology, or this alternative. That’s not part of my in group. But that will be, if you like, in the context

of a much

more synchronized, harmonious exchange and information exchange within well formed groups.

Speaker 3 – 37:29Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

13 / 43

You know, one of the things that really stood out to me in that white paper from back in 22 was when

you were

describing what’s necessary for this evolving intelligence is that it depends on the diversity of all the

intelligences.

And so to me, when I look at the potential with active inference and then what the spatial web enables

with this

distribution of these agents, that kind of become this organism of intelligence within networks, it

preserves what

you’re talking about, all the different diversity, you know, different belief systems, different governing

styles, all of

that. To me, that alone tells me this is the direction that it has to go in. Because, because we’ll never

ever agree on

anything. Globally there’s too many, you know, regional differences, too many.

Speaker 4 – 38:20

So which should be celebrated. I mean, you know, I mean, just to really reinforce that point, if you look

at this

through the lens of theoretical biology and evolutionary thinking, the co dependence we have on all

sorts of things,

from you know, atoms of molecules of oxygen through to bees through the trees. Yeah, we only are

here because

of that diversity. We could not exist without everything in that ecosystem. And that has enormous

diversity over

many different scales. And the same applies to sort of, you know, human and indeed political culture.

But to have

that diversity that you have those different kinds of things, they have to be demarcated. So they have to

be able to

preserve the Markov blankets.

Speaker 4 – 39:02

So you are talking about a physics that enables the preservation and the segregation, that identifies the

individuates, demarcates this kind of thing that enables the diversity to be self maintaining. So I think it’s

a really

crucial thing. Of course, you could start to argue now, you know, what would that look like at a

geopolitical level?

Well, Markov blankets can sometimes be literally thought about as borders. And that’s what I had in

mind when I

was talking about problems and our uncertainty. And at a geopolitical level, it’s uncertainty about

borders as we

are seeing in Gaza, as we’re seeing in Ukraine and elsewhere in the world.

Speaker 4 – 39:41

So one would hope that all of the, all these Markov blankets will be in a much more certain

configuration that do

celebrate the diversity simply because we’ve now demarcated this kind of thing and that kind of thing,

this kind of

culture, this, you know, in a stable but synchronized, in mutually respectful way. And that will be the

free energy

minimizing solution that will be facilitated with the right kind of communication.

Speaker 3 – 40:08

Right. I, I, I see that. I, that makes a whole lot of sense. Karl, thank you so much for taking this time with

me today.

It has been such a, a deep pleasure to be able to have a conversation with you like this. And thank you

for

humoring me with my own questions that were just personal to me. And if you were to leave our

audience with any

parting words, what would they be?

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

14 / 43

Speaker 4 – 40:34

Keep your options open. That’s.

Speaker 3 – 40:37

I love that’s solid advice. Karl, thank you so much. It is such a pleasure to meet you in person. I love

your work, I

love to hear your opinions because I feel like you are one of the truth speakers about the current state

of AI. I

would love for you to share with our audience kind of who you are, why you do what you do.

Speaker 1 – 41:02

So I’m Gary Marcus. I’m a cognitive scientist by training, but with a lifelong interest in AI. I study, I would

say, both

natural and artificial intelligence and I speak truth to bullshit. And I actually said on 60 Minutes that what

large

language models produce is authoritative. And that was always true of the models. It’s become

increasingly true of

the industry that there’s a lot of authoritative meaning. It sounds like it’s true, but it’s actually, and I’m

just allergic to

that as a scientist. Right. First and foremost, I am a scientist, a cognitive scientist in particular, and I’m

interested

in what the truth is, what the arguments are, how did we get there. Are we in the right place? And I don’t

think we’re

in the right place now. I often get called an AI hater. That’s not me at all.

Speaker 1 – 41:51

I started working on AI when I was 10 years old. I’m very interested in AI. I want it to work. I want it to be

good for

society. I just don’t think that what we’re doing with large language models and stealing copyright and

having these

unreliable systems and giving them lots of authority is the right thing to do. That doesn’t mean I hate AI.

It means I

hate the way we’re doing it now. Sometimes I feel like a parent with a recalcitrant teenager. I actually

have two

great kids. It’s not my personal experience but you know, imagine that you have a teenager who’s kind

of like up,

but you still love them.

Speaker 3 – 42:21

Yeah, right.

Speaker 1 – 42:22

That’s how I see AI right now. It’s up. I still love it. It’s not doing the right thing. I want the best. It needs

some

advice right now. How?

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

15 / 43

Speaker 3 – 42:31

Wow. Yeah, no, I, I, I hear that. And, and I think there’s a lot of people who really do recognize that. So

then what do

you think is the solution? What do we need in AI? What, what would be the right path for AI?

Speaker 1 – 42:47

I mean there are moral, political and technical solutions because there’s actually problems all over the

map. So you

know, on the governance side, we do actually need government side. Every technology needs

governance.

Somehow there’s this insane idea that AI doesn’t. But we all know AI could actually kill a lot of people.

Like it’s

crazy to have no governance of it. We could argue, you know, this regulation is, you know, too

burdensome. This

one’s just right or whatever. Like you know, we can have an intelligent argument but to say that we

don’t need any

governance at all is absurd. So we need governance. Most important thing there is, we need kind of pre

flight

check. If you’re going to release something to 100 million people, you should figure out if it’s safe, like

what are the

consequences? And not just like roll it out.

Speaker 1 – 43:25

Like if some certain large companies I could name are about to do with say porn products, like how’s

that going to

affect society? Like do we have any data at all about this? Yeah, so that’s one side of it. On the

technical side,

LLMs, I don’t think they’re ever going to be reliable. I warned about their predecessors hallucinating in

2001. It’s 25

years of later the core problem that leads to that, a kind of over generalization when you don’t have

good records

of individuals versus kinds, you know, hasn’t changed since I wrote the book the Algebraic Mind. And

people just

keep hoping, like magic, like kind of religion here. And it’s not actually getting better. I think to do better,

what we

need are systems where the core is actually a model of the world that you can explicitly interrogate.

Speaker 1 – 44:14

You know, what are the things there? So we’re sitting in a room, there are chairs, there are exit signs,

there’s a

phone with a kind of lighting source on it, etc. There’s two people. You’re nodding your head, you’re

paying

attention to me. You seem to like what I’m saying. Doesn’t look like we’re out of time or that I’ve lost

you. And so

I’m building all of this and, you know, I can probably get it, get a laugh out of you. So I’m building this

model in my

head of where we are and so forth. I have different models for other things. I just watched, you know,

the Harry

Potter movies with my kids and like, there’s magic. It doesn’t apply here.

Speaker 1 – 44:46

I’m not expecting any wizards to walk in here, but in the context of that movie, if a wizard didn’t show

up, I’d be like,

what’s going on here? Right. So we have different models of different parts of the world, including

fictional worlds.

That is core to our understanding how the world works. And LLMs don’t have that. So we need

approaches where

that’s core. And then you. And second to that, you might know Kahneman’s system one and system

two, fast and

automatic versus slow and deliberative. That’s roughly neural networks versus classic symbolic AI. We

need both.

There’s been so much tension in the field. People trying to say, my flavor is the only flavor. My flavor is

the only. We

actually need some of both.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

16 / 43

Speaker 1 – 45:21

And we need some new ideas to integrate those so that, for example, if you have a world model, you

can consult it.

Like, LLMs will make up stuff. Stuff like, you know, where somebody’s born. Even when it’s amply

documented in

the Internet, they don’t have a good connection between world knowledge and the way that they’re

doing the

processing. So we need to solve that as well.

Speaker 3 – 45:41

Yeah, no, 100%. So when you look at the future, because one of the big issues with these LLMs too, is,

you know,

they are still. Still useful tools, but it’s not a sustainable path for AI, when you look at the energy

requirement and

you look at, to me there’s just so much.

Speaker 1 – 46:05

Teenagers only need 20 watts. Right? Like imagine a teenager but with a gigawatt or 50 gigawatt.

Speaker 3 – 46:11

I mean, quite right about that. What do you think is going to happen with that? Because to me what I

see

happening here is you’ve got all of these companies that are holding really tight to this centralized

business model

and this whole, you know, giant databases, giant data centers to run these AIs. Instead of this

distributed AI

scenario where you can have agents distributed through networks, through operating at the edge, that

is an energy

efficient way. Especially when you consider Karl’s free energy principle and the way that works. Do you

see this

being a transition? Do you’re about to get on stage with Karl and have a conversation, where.

Speaker 1 – 46:54

Do you see us going on the central versus distributed? I think it remains to be seen. I think there’s a lot

of factors

there. So one factor is like what is the computation you’re doing? And to make an LLM work you need

these

massive databases to train them on. You don’t need that for humans child, they can learn on a much

smaller

fraction of the Internet. So there’s that side of it.

Speaker 3 – 47:16

There’s a kind of babies aren’t pre trained, they’re born.

Speaker 1 – 47:19

Well, babies are pre trained in a certain weird sense of that word, by evolution. And part of the problem

with these Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

17 / 43

neural networks is they don’t really have evolved minds. They just do everything from actual training.

Whereas we

have evolved minds that allow us to for example, understand the world in terms of sets of objects and

so forth.

One of the things I, I do like about versus is they’re trying to get what I would call innate priors. And that

connects

to some of what Karl’s thinking about in their systems rather than just learning everything from data. So

a neat

prior is for example about objects traveling on spatially connected paths. That’s something that we

need more

broadly. Karl is, you know, fantastic scientist. We’ve spoken before and I’m looking forward to going out

and talking

with them again.

Speaker 3 – 48:06

Yeah, well, I am so glad that you’re here and thank you so much for giving us a few minutes of your

time today. And

Gary, it’s just such a pleasure to meet you.

Speaker 1 – 48:15

Thanks a lot.

Speaker 5 – 48:15

Super exciting. We are honored to have those two people, those two giants in the field sitting next to

me. So on the

one hand we have Gary Marcus in the primitive side. Scientist and author, professor emeritus of

psychology and

neuroscience at NYU entrepreneur and a leading public voice on AI and AI policy. And then we have

Karl Friston,

the professor at ucl, chief scientist at Versus. And if you don’t know who he is, you’re in the wrong

room. So that’s

get started. So we intended to talk about the future of AI, but let’s get started by reflect a bit on the

current state AI.

And when people outside of this room think about AI, they typically think about dread of AI or LLMs.

Speaker 5 – 49:12

And rumor has it, Gary, that you’re not that f. So very quickly give a summary of what your critiques are

on the

current state of algorithms and where they’re going.

Speaker 1 – 49:26

Sure. But first I’ll mention that yesterday Noah Smith, who goes by no Opinion on Twitter, had a tweet

about how AI

haters were medieval peasants. And I couldn’t tell whether that was supposed to refer to me because a

lot of

people describe me as an AI hater. But you got the scope right. I don’t hate AI. I actually love AI. I

wouldn’t spend all

of my time reading and writing and built a company which I sold to Uber. I wouldn’t spend all this time

on AI if I

hated it. But I do hate LLMs. I’ll be honest about that. They have their uses. Another straw man of me

yesterday

was to imply that people of my kind anyway think that LLMs are useless. And I’ve never said that. I

think that they

do have some uses.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

18 / 43

Speaker 1 – 50:17

I just think that the negatives are probably outweighing the positives. There’s multiple kinds of

negatives. So the

positives are. For example, they help sophisticated coders do some of the drudge work of coding. They

may help

poor coders make really insecure code. We could talk about that. But they certainly help coders. They

help with

brainstorming, they help with translation, they help non native speakers write. So they do have some

value. But I

think that there’s a wrong way to approach AI. I think you need world models at the core of how you

build AI, or

else you hallucinate. And I’ve been warning about that since 2001. Before there were LLMs. In my book

Algebraic, I

said, if you use these kinds of neural networks and the LLMs are descendants of the kinds that I was

talking about,

then you’re going to have problems of overgeneralization.

Speaker 1 – 51:07

You’re not going to get facts right. That’s been a problem for a quarter century. So there are reliability

problems.

There’s also a weird kind of cultural problem, which is we have an intellectual monoculture right now

where

everybody is basically doing the same thing at really enormous scale. There’s this old line about the

definition of

insanity is doing the same thing over and over again and expecting different results. Well, may I present

to you the

field of AI right now, where people keep building larger and larger LLMs, hoping it’ll magically solve the

hallucination problem and the, what I call the boneheaded reasoning or boneheaded error problem. We

still get

that, like, none of this has really changed. Maybe it’s reduced a little, but clearly these are not

fundamentally safe

marketing. And then you have the kind of moral and political problem.

Speaker 1 – 51:55

So how do you make LLMs work? Well, first thing you do, apparently, is you steal all of the data from

every artist

and writer ever. And then you try to persuade the government that’s okay. The copyright laws don’t

apply to you. I

don’t think that’s okay. And it’s led to, I think, some very poor choices on the part of a lot of people

running the AI

companies. And clearly what they want to do is take your job would make you watch slopping porn all

day. Like, I

mean, it’s almost not even a joke. Like, Altman two weeks ago was like, we have really hard choices

because we

have a shortage of chips. Of course he needs a lot of chips because his technology is not very efficient.

Speaker 1 – 52:34

But putting that aside, he says, you know, we have a shortage of chips and so we have to make hard

decisions like

solving cancer in the next year or two or fixing education. The reality is that LLMs are not going to solve

cancer in

the next year or two. Even if they were like these super genius things that they pretend they are, you’d

still need to

do the clinical work. And it just tells you that Sam has no clue about clinical work if he thinks it’s

remotely possible

to solve cancer in a year or two. So it’s all bullshit. Can I say that here, aside from the fact that it’s all

bullshit, that

was late September. Here we are in mid October. What is the actually doing with his chips? He’s

revealed that he’s

going to. In. In this inside a mental health announcement.

Speaker 1 – 53:16

He reveals in the last paragraph that they’re going to start asking for ID because they’re going to have,

quote,

erotica, which I think rather dignifies the garbage that’s going to come out. And he said that he’s going

to make a

competitor to TikTok that’s going to be all AI slo generated by Sora. Unlicensed all the time. So like

there’s the Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

19 / 43

political side. That also makes me hate not AI, but LLMs used in this way. Like there got to better

things. I had an

op ed and then I’ll end with the microphone because I’m obviously going on for a long time.

Speaker 1 – 53:49

I had op ed yesterday in the New York Times and the way I ended it is by saying maybe we need to

focus on more

special purpose models that don’t make all of these mistakes that do particular things in medicine,

education and

so forth. I didn’t say it rather than making foreign and slot, but I think it was implied. And so like I think

we can do

great things with AI, but maybe not in the way that LLMs are doing.

Speaker 5 – 54:12

Yeah. I think this gives us lots of food for thought for the next 30 minutes. Karl, on your end, about a

year ago you

said, I quote deep learning is rubbish.

Speaker 4 – 54:25

Do you.

Speaker 5 – 54:25

You still stand with that quote?

Speaker 6 – 54:28

Yes, I. Yes, I do. Let’s just contextualize. I had to say something entertaining and controversial, very

much in the

spirit that the. Marcus has been invited here to.

Speaker 4 – 54:40

To. Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

20 / 43

Speaker 6 – 54:41

Sorry, Gary’s been invited here to entertainers. So you should say the things because people love to

hear that but

none of us dare actually say themselves. So explain that you’re actually.

Speaker 1 – 54:52

Can I interrupt just briefly? What’s really interesting to me is how much the world has changed in the

last three

months. I got so much. As long as I’m being found out so much in the last seven years as quote, AI

hater and

people question my intellectual integrity. It’s really kind of hard being me for the last seven years. And

then people

noticed when GPT5 was not what it was supposed to be. So one of the things I kept saying is that

GPT5 is going to

be late and it’s not going to be AGI. It’s not going to be magic. And it wasn’t. And people really noticed.

And I have

been treated so much better in the last couple months now people want to hear me dis LLM. So it’s a

fun moment.

Speaker 6 – 55:33

Your time has come. Why did I say that LLMs of deep learning was rubbish? I think you could play that

angle or

sort of justify that perspective from a number of different angles. You mentioned efficiency. We certainly

want your

one angle. What I was specifically getting out of the conversation with Yarmulke at that point was the

fate of. Of

large language models to encode uncertainty basically in his language. Not that came out in the

conversation, but

in his language, he would certainly be perfectly okay with formulating AI as a free energy minimizing

process. But

the free energy would have to be equipped with a zero temperature. And what that basically does is it

nullify the

entropy. So you’re basically talking about a tech that has no capacity to encode uncertainty. And what

does that

mean?

Speaker 6 – 56:42

Well, it means you can’t encode what you don’t know. So you have to look everywhere, you have to

ingest, steal all

the data, because you’ve got no way of work knowing how to be curious, you know. So we just see

Jantana’s

beautiful presentation, the fundamental importance of being curious, resolving your uncertainty to get to

that kind

of sound efficiency which precludes or obviates the need to then go and steal everybody’s data in a

really efficient

way. I can’t remember how the conversation developed, but certainly from the point of view of

sustainability, I think

that’s a really important.

Speaker 4 – 57:29

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

21 / 43

Argument.

Speaker 6 – 57:30

And you could play that in many different ways. One thing which, having introduced Jan Lecant, I’d like,

because

you’ve had lots of lovely arguments with him.

Speaker 4 – 57:41

I sometimes see.

Speaker 6 – 57:42

Him and you on the same side because you’re both rebels without a cause in some respects. But

there’s one thing

which I don’t understand. It’s the use of the word world models and generative models. Now, I’ve

always

understood generative models in the spirit of Geoffrey Hinton and Helmhurst Machine, as a sort of

proposal

specification of the cause, effect, structure of content. And yet Jan seems to say we shouldn’t be using

generative

models. And notice you carefully use the word world models. So I’d be interested to hear, what do

people mean by

generative AI and what is the implication for generative models in the context of generative AI?

Speaker 1 – 58:26

Those are two really different questions. So before I get to either of those questions, I’ll just say I think

you’re

exactly right about uncertainty and these systems not representing them. And that’s a very serious

problem. So all

the hallucinations, for example, they fail to give you a confidence around it. I wrote a whole essay about

my friend

Harry Shearer, who’s all of you know his voice because he does voices for some of the characters in

the Simpsons,

probably all of you. And he was a bass player in Spinal Tap. So he’s a pretty well known actor. And it’s

very easy to

figure out where he was born. In fact, you can just go to Wikipedia or you can go to IMDb or rotten

tomatoes or

anything like that.

Speaker 1 – 59:02

And somebody sent him a biography, which he gleefully sent to me, where it said that he was a British

actor,

comedian, etc. So the system hallucinated where he’s from. Even though it’s trivially easy to find on the

Internet, a

proper system, the very least it should do is to weigh the evidence and say, I see thousands of things

on the

Internet saying he was born in Los Angeles, and yet here I am compelled, because I have a language

model that

sticks things in embedded spaces and loses track of them. To say that he’s British must, you know, you

should

have some probability assigned to that, some confidence interval. It’s one of zillion examples of these

systems.

That’s why I say it’s authoritative. The authoritative part is it presents everything with 100% confidence,

even when Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

22 / 43

it should not, with 100% certainty when it should not.

Speaker 1 – 59:50

Okay, so I completely agree on your first point that is central. This audience fully understands that. I

don’t always

use that terminology in the lay audience, but your point about uncertainty is 100% right. Second,

LeCun, we’re

famous for having arguments, but the arguments all happened in 2019. In 2018, when I said that deep

learning, I

didn’t say it was rubbish, but I said that it had problems. And he said that I was mostly wrong and never

spelled it

out. And then in 2019, I said LLMs didn’t work, and he said I was fighting a rear guard battle. So what

people don’t

remember, they all remember that we had a fight, but they don’t remember what the positions were.

And his

position was that LLMs were hunky dory and that this was great progress.

Speaker 1 – 01:00:30

This is in fact his position up until November of 2022, when he released the model called Galactica,

which he

thought was fantastic, and he bragged on it. He said, look how great it is. And a bunch of us, including

me, but not

limited to me, showed that it had problems too. Like there was a journalist who showed that it would

happen, write

essays like what are the benefits of eating crushed glass? Or anti Semitism and so forth. And so

Facebook, not

me, but really, Yan decided to remove it. And Jan’s very bitter about that. Then what happened is, it’s

true. And then

what happened is two weeks later, ChatGPT came out. It was actually a very similar model. And he

was.

Speaker 1 – 01:01:14

And I sort of understand because that model became really popular and his, which was technically

really not that

different, he didn’t have the guardrails that should have you know, he felt very bit about it and that’s

when he turned

against LLMs. So ever since then he sounds just like me, except that he won’t cite anything by Marcus

and I often

do. But that’s the only difference between us now. So I mean, it’s another way of saying I won the

argument.

Nobody remembers what the argument was. They just remember that people were upset. If you go

back and look

at the actual arguments, there’s nothing that he said then that he still believes and everything that I said

then, he

now does. So it’s a little bit strange. Okay, so now world models, there are actually some genuine

differences

between his view and my view.

Speaker 1 – 01:02:03

Although we never really argue about it because we don’t know, really talk anymore because he won’t

psycho that

anymore. So world model for me is partly a symbolic representation. I can conceive that there might be

a way of

doing it without things like databases and ontologies and so forth, but I’ve never seen anything remotely

successful doing that. All the AI is probably too strong. Most of the AI that I know actually does have

access,

explicit world models. I’ll give you an example. The navigation system that got me here today. I kind of

know

Montreal a little. I don’t, I didn’t know exactly from the hotel, whatever, so I just followed the GPS

system. Well, the

GPS system has a world model. Literally, it has a model of what the streets are, what the connections

are, how

long it takes to go.

Speaker 1 – 01:02:49

So I chose to walk. It told me 10 minutes, which was a bad accurate, except I walked faster, but you

know, it was Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

23 / 43

close enough. And so it does its computation by a simple classical AI algorithm of, you know, finding

shortest path

given a list of routes. Similarly, Wikipedia gives you a world model that tells you a bunch of things about

Harry

Shearer, like what are the movies that he’s been in. The thing also hallucinated that he was in Jaws and

he had

nothing to do with Jaws. Although there, it turns out that there was one web page that got it wrong and

the system

doesn’t know it. You know, again, lacking certainty, it doesn’t know that like one place reported this and

it’s not

verified. Can’t actually reason through those kinds of things. So, you know, there is the Wikipedia stuff.

Speaker 1 – 01:03:34

I mean, this is even in the boxes, right? The boxes in Wikipedia are the easy stuff that we actually do

know how to

deal with structured text. All of that gives you a world model, you know, where was he born, when was

he born, et

cetera. Just look it up Any, you know, credible AI should at least be able to use that stuff. So for me, a

world model

is that. And I think we have lots of world models. For example, my kids just watched the Harry Potter

movies. I

watched a little bit with them. And so you have a fictional world model where there are wizards and

things happen

with those wizards. And like, you understand that’s not going to happen here. Nobody’s going to ride in

here on a

broomstick.

Speaker 1 – 01:04:10

But it would be weird to watch a whole Harry Potter movie with nobody flying on the roadstick.

Speaker 3 – 01:04:14

So.

Speaker 1 – 01:04:14

So, you know, we have multiple world models, in fact, and they can be incomplete. Like, I have a model

of this

room. I’d say there’s 100, 200 people in here or something like that, but I can’t tell you the exact

number. I can get

some sense of like, you know, distribution, demographics, age and stuff like that. I can’t give you all the

details. And

that’s fine. You don’t need to have all the details. I know enough for present person present. If I need to

know more,

I can zero in on this table and try to figure out, okay, you know, there’s this many people there, and so

you can make

details. So world models don’t have to be infinitely detailed, but they need to be structured. And I don’t

even know

how to say this word, interrogable.

Speaker 1 – 01:04:51

You need to be able to ask them questions and reliably get answers out of them. You should have a

world model,

and I should be like, who’s this guy sitting next to me? And I should be able to figure out something

about it or

remember something. Remember some of the places we’ve met before and some of the places that

we’ve spoken

might not be perfect. I might, you know, we might have met enough times that I don’t remember all of

them, but I

remember, you know, sitting in your office, whatever it was, six months ago. And I remember we did an

event in

Davos two years ago. And so, like, I have a lot of detail there, not infinite detail. I don’t remember every

sentence

that we said, but I use that centrally for almost everything that I do.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

24 / 43

Speaker 1 – 01:05:27

Now there’s a different sense of world model, which is. Which I’ve seen DeepMind use and which I

suspect that

Yang might use, which is something like, and I think you alluded to it is something like a function that

predicts

things, and that’s also a kind of world model. But I think it’s not what I’m looking for, not all of what I’m

looking for.

So there is a sense in which a world model is, you know, a function that predicts things. But I’m also

looking for

something that has the structure and the facts and you should probably have probabilities of attached

to it. Right.

You know, the classic symbolic AI version of it was basically not quantitative and not probabilistic. And I

think, you

know, a fair discovery of the last 30 years is you really want that probabilistic information.

Speaker 1 – 01:06:07

In fact, the thing that LLMs do well is they have some probabilistic distributional information, but they

don’t have

the uncertainty represented. All you actually want all of this. So, you know, getting to the right AI is

going to require

doing some hard work around it. I’ll say one more thing which is people might have seen, I think it’s

called cts on

the day. There’s this new model just came out a day or two that’s a biological model predicting things

about cells

and so forth. Ctus scale I think is the last word. This is a system that actually has many different

carefully

engineered bits of knowledge structures. Like there’s this imagination. You just throw everything into a

black box.

But that’s the most successful black box ish model that I’ve seen. And it still like has specific

representations for

how it represents cells.

Speaker 1 – 01:06:59

And there’s this incredibly intricate curriculum for how you train it. What pieces of knowledge and so

forth. The

right answer to AI is really going to be different in different domains, is going to have a lot of structure

inside.

Some of that’s probably going to be a world model.

Speaker 5 – 01:07:14

So if I understand you well, your notion of a world model is more or mainly about structuring your

knowledge and

being able to enter into your gate structure. But what I’m missing from your interpretation is the concept

of action.

Like how can you a structure, how can you build a structure of this room if you’re not able to walk

through this

room and experience walking through the room and meet the people in order to have like a concept of

how many

people are there? Like, don’t you need the action fundamentally there to build the model?

Speaker 1 – 01:07:55

So there’s two pieces to that. The first I will call a friendly amendment to what I’m saying. The second

I’m not sure I

agree with world models. Also I didn’t go into all this. World models also have to have a notion of

causality of

action, events and so forth. So you’re quite right that I should have stressed that. The second part is do

you

actually need to be embodied to have world models? I don’t think so, or at least not as much as people

think, but

it’s probably helpful. So I look to psychology to understand that it’s a guide. It might not be a perfect

guy, but what I

notice is that there are people in the world who are paralyzed.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

25 / 43

Speaker 1 – 01:08:33

For example, there are people with things like SMA who basically never moved around the world and

still build

pretty good models of the world in the sense that you can talk to them about it. They seem to

understand things

even that they can’t physically do and so forth. So my view about embodiment is awfully helpful if you

look at

developmental psychology literature. So once kids crawl around, they very quickly learn a lot of things

about the

world, but it may not be strictly necessary. And when we get to AI, like, it’s kind of an open question.

There was a

critique by Emily Bender that I thought was too strong of LLMs that said that because they only have

language,

they can’t really learn that much. They don’t really have much understanding.

Speaker 1 – 01:09:12

I think the right view is if you only have language, you’re limited, but you could probably actually do a

lot. Or you can

think about Helen Keller. She had a little bit of experience before she went blind and deaf at 18 months.

She had a

little bit of physical experience and she’s continued to haptic experience, but she constructed very

intricate models

of the world and was able to write and be successful and influential and so forth. So I don’t know how

much and

what kinds of experience are absolutely necessary. I do think that it’s really great if you get that

experience. There’s

a field called developmental robotics which I think is yielded nothing really important in 30 years of

trying, but

might, it might eventually.

Speaker 1 – 01:09:51

So the problem is, in my view, so developmental robotics is like, you let those robots roam around the

world and it’s

supposed to learn something that it couldn’t otherwise learn. And it stands to reason that you could do

that. But

what I often see are like, experiments where somebody puts an apple back and forth in front of the

robot and says,

apple, apple, and like, it doesn’t get much further than that. I think that the problem is prejudice in the

field of

machine learning. So you know the old thing about give a man a hammer, he thinks everything is a nail.

So in

machine learning, they tend to think that you need to learn everything. But I think if you look at biology,

lots of stuff

is innate. And in fact, the innate stuff is what allows you to learn.

Speaker 1 – 01:10:32

And I Think the sophisticated people in animal psychology talk about innately guided learning. People

like Peter

Marlowe, Randy Gala Stow and so forth. So what you can learn is a function of what’s innate. What

kind of

structures do you have for world models? What do you have? Observational learning, imitation

learning? Can you

do more sophisticated things like that allow us to learn calculus or learn about cognitive science? So I

think that

the problem with developmental robotics is that they always start from completely blank slate. And I

think that

there’s one thing LLMs have taught us, it’s that starting with a completely blank slate only gets you so

far. Like

LLMs, you know, get all the sense data in the world, in a certain sense, entire Internet, we get lots of

video data and

they still don’t understand that objects persistent space and time.

Speaker 1 – 01:11:24

So you want to start with that, which is something your company does some work, but you want to start

with stuff

like that. And then maybe the developmental robotics project might actually be really super interesting.

It hasn’t

worked yet, but I wouldn’t count it out for exactly that reason. Meeting Title: EPISODE 19 – Karl Friston

Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

26 / 43

Speaker 5 – 01:11:38

So, Karl, what’s your take on the requirement of action to build your work model?

Speaker 6 – 01:11:44

I think with the stamp I can think about, can you be intelligent if you’re not an agent? And if you’re not

an agent or

to be an agent, does that necessarily imply some kind of action? So I agree entirely that physical

embodiment is a

really important aspect of being an agent because you execute your agency through the body. But you

often

imagine having agency without necessarily a sort of physically instantiated body. I think agency is the

key thing

here and sort of begs the question, what is it to be an agent as a thermostat an agent. And I would

argue that from

the point of view of artificial intelligence, agency is to be an agent is to have intentions, and to have

intentions is to

have a will model of the consequences of your actions, which necessarily implies a world model.

Speaker 6 – 01:12:49

That is future pointing is counterfactual, that can deal with things that have not yet happened that you

intended to

happen. That gives you a sense of selection. So I think agency is a key thing that would draw a broad

right line

between intelligence or artificial intelligence as represented by large language models and the kind of

intelligence

that we’re talking about. As far to talking about here, like just to why would large language models not

meet that

mark? I think it was implicit in one phrase that you were said earlier on in relation to large language

models,

implying that it’s basically just a mapping between content and content. Another way of Putting that

your

engineers have found some way to utilize universal function approximators to map from content to

content that

there’s nothing, if you like, in the middle. There’s nothing. There’s no understanding.

Speaker 6 – 01:14:02

Very much in the spirit of sort of us statistician being comfortable with an autoregressive model as a

description

of sequences of data without understanding how those data were generated and how they might be

generated

under different interventions or different actions. So I’m coming back to defending my notion that large

language

models are rubbish because that’s another the way in which they are rubbish, that they don’t have

agency. So when

large language models now demonstrate evince agency in the sense of being curious about their world

when they

start asking questions, then I think large language models would be shown some kind of authentic

intelligence

simply because they now become agents. So, so I, I should say defense. I was looking at

developmental neuro

robotics. Yes, there’s a, there is a lot of rubbish there. I think there’s also some wonderful stuff.

Speaker 6 – 01:15:06Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

27 / 43

Yeah, I have to say some of the deepest questions which you actually spoke to certainly from a sort of

cognitive

science and psychological perspective, some of the deepest questions about agency and how we

acquire agency,

you know, beyond learning and how it’s contextualized by name things. And I should also slip in here,

you know,

inference is not learning. So inferring what’s going on is, you know, you can learn to infer certainly. But

inference is I

think another level of sense making that we need. Most of those. The deepest questions to me actually

come from

either developmental psychology or developmental robotics, if you’re committed to an embodied

context in the

context of robotics.

Speaker 4 – 01:15:54

So.

Speaker 6 – 01:15:57

Just to sort of make sure that not all robotics has failed to live. And I think something really interesting

developments.

Speaker 1 – 01:16:09

At the very least I think the questions, and I mean I should have made this clear. I think the questions in

developmental robotics are really interesting. It’s just most of the work that I’ve seen I think has a, a

bias towards

let’s learn it all from scratch. It’s probably not true of all, all of the work. And I’m sorry that I didn’t see the

talk

earlier which may have been very interesting, but in general there’s been this. I’m going to do things

with a blank

slate, which is the same thing we see with llc, right. They’re completely blank slate and people keep

getting the

same results. The questions that developmental robotics tries to answer I think are really interesting.

And for that

matter you could Say the same about LLMs.

Speaker 1 – 01:16:45

The question they’re trying to answer is how do you get a mind from data? It’s just that the real answer

to that is

you also have some stuff built in to have scaffolding around that data in terms of the entrance

mechanisms, in

terms of learning mechanisms, in terms of some set of priors. And people tend to exclude those for

whatever

reason. But it’s interesting and just to close the circle on agency, people do use LLMs as agents, they

just don’t

work that well. There was a system a few years ago that nobody even remembers called Auto GPT.

And there was

a flurry on Twitter of people saying how it was amazing and how it’s going to changed the world. And

after about a

month, people realized it didn’t work very well because of the reliability problem.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

28 / 43

Speaker 1 – 01:17:29

So we do things like you’d say, you know, I want you to place this stock trade, you know, such and

such, and it

would say I’m on it. Because that’s something that, you know, content to content, you can approximate

that people

will say I’m on it when you give them a request. But the system didn’t have the passwords to actually

place the

trade. It didn’t have the wherewithal to say I don’t have the passwords. And so, you know, it was say

that it had

done this thing for you, but it hadn’t. So it would feign agency. It couldn’t really do agency because it

didn’t really

have the world model, didn’t really have the causal understanding, didn’t really know what it actually

was. And so

you can make it pretend to be an agent, but it’s not a very good one.

Speaker 5 – 01:18:05

But then again, you could say let’s put a structure in there from the get go. But also that we tried for an

awful lot of

time with expert systems before the rise of deep learning, everybody was trying to do that.

Speaker 1 – 01:18:20

Expert systems actually worked better than people remember them too. Their problem was they were

brittle. And

the problem with LLMs is they’re brittle. So in the expert systems day, the really lousy thing about it was

you had to

pay a lot of like graduate students a bunch of money to fix all of the edge cases that they didn’t do well,

but they

kind of worked. LLMs, they kind of worked. But now you pay Kenyan contractors to fix all the errors, or

you pay

mathematicians or whatever, but they’re still brittle. It’s like very much a similar reality to before.

Speaker 5 – 01:18:52

Yeah, that’s all I can have instead of having two things that kind of work rather than the road to have

something

that works well.

Speaker 1 – 01:18:58

So I mean, the question is, how do we get something to work? So my view is if you look at classic

symbolic AI and

neural network, that they are very complementary. It is obvious that we should be spending all of our

effort not

gathering bigger databases, but figuring out how we can get the best of both worlds. So classic

symbolic AI is

great at representing world models. That’s a lot of what the work was on. It is great at generalizing from

small

amounts of data in narrow contexts. Representing something as a function is an incredibly powerful

thing, but a lot

allows you to generalize beyond the cases that you’ve seen before, which is a weakness in the neural

networks.

But the problem is it’s always had to be handcrafted.

Speaker 1 – 01:19:39Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

29 / 43

And of course the really early stuff didn’t represent probability and uncertainty very well at all. Then you

have neural

networks that are great at learning from massive amounts of data, but everything that they learn is too

superficial

and it doesn’t generalize very well. You wind up with Teslas being summoned across an airplane part

parking lot

and running into an airplane because the knowledge that they have is so superficial. So it seems

obvious to me

that we should want to bring them together. And it’s just a political history. You have people like Jeff

Hinton that

are bitter because they didn’t get funding and so they dump on the symbolic stuff and tell people, don’t

try to bridge

this world.

Speaker 1 – 01:20:16

The exciting reality is in the last few years, people have stopped listening to the Hintons and the

queens of the

world who said, you know, don’t do that symbolic stuff. It’s bad, it’s a waste of time. It’s like putting gas

engines

when you already invented electric. Really said this. And what’s happened the last few years is people

have worked

on neural symbolic AI and got good results. Alphafold is a neurosymbolic system that does try to

combine a

generative system with classic search, for example.

Speaker 5 – 01:20:45

So Karl, what’s your take on this? Is combining those two systems together, Frankenstein, the system

together, is

that the way forward or is something else or more needed?

Speaker 1 – 01:20:57

That’s your word, not mine. Frankenstein.

Speaker 6 – 01:21:01

That’s a good word. What did you imply though? What did you imply by that?

Speaker 5 – 01:21:08

Well, basically it’s as if you have system A that kind of works and has certain limitation. You have

system B that

kind of works with different limitations. So the obvious solution then is to bring them together, but I don’t

think it’s

so straightforward to just glue these vastly different things together and have the best of both worlds.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

30 / 43

Speaker 1 – 01:21:28

How to do it is not obvious.

Speaker 5 – 01:21:30

You might also end up with the brittleness of both.

Speaker 6 – 01:21:36

Yeah, so, you know.

Speaker 4 – 01:21:40

I can see.

Speaker 6 – 01:21:40

From an engineering perspective you might want to try and do that and of course the dangers that are

entailed. But

from the physicist perspective, I think you’ve been looking for what you want your intelligence to do and

what

principles should it comply with, will it leverage or rest upon?

Speaker 4 – 01:22:02

Technology.

Speaker 6 – 01:22:05

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

31 / 43

Underlies generative AI as we currently know it. Possibly to a certain extent, but you know, I think that

would be an

add on. Of course, you know, we spent many months, years talking about amortization as one way of

sort of using

deep neural networks to finesse the problem of evaluating your beliefs about the causes of your content

and what

you are going to do next. Mapping from content to belief structures, probability distributions, including

that

uncertainty, just to bring in the sort of symbolic aspect. I am reading symbolic here basically as having

a particular

functional form for your world models or your generative models in which you have a discrete amount of

discret

small number of states in the world that can be associated or equipped with the semantics or the

symbolic

aspect.

Speaker 6 – 01:23:03

So I think you’ve now got a calculus or an aspect of these generative models that enable you to sort of

make

logical adaptive kinds of if this, then that, and therefore what one will be looking at from a first principles

perspective is basically how would you use technically how you did, how would you invert a generative

model that

generated continuous consequences from discrete symbolized causes? And of course that then speaks

to a

particular class of generative models. And an interestingly sort of interesting technical point here is the

message

passing on the inversion or the realization or.

Speaker 4 – 01:23:53

The solution.

Speaker 6 – 01:23:56

That would be required in terms of sense making, decision making action will be very different under a

symbolic

architecture. So you’d be using propagation or variational message passing. You would not be using

the kind of

continuous dynamics associated with transformer architectures or continuous state space models. So I

think

there are implications about how usable current deep RL like technology would be if you’re committing

to a purely

symbolic kind of architecture.

Speaker 1 – 01:24:38

That I love most of that and I wind up being slightly more sympathetic to LLMs than you, but only

slightly. So I think

first of all that I would like to strongly endorse the first principles thinking that you’re talking about and I

agree with

most of the principles that you laid out. I think that you need to have systems that do the kinds of things.

Speaker 6 – 01:25:01Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

32 / 43

That you talked about.

Speaker 1 – 01:25:02

And I would agree that LLMs do not do those things. The probably the only difference is I think that

LMS can be

subcomponents in systems that do those things. There are certain places where they’re good at

calculating kind of

distributions in a very broad sense, and I think that they can help. I think that you want the sort of

master of the

system to be a symbolic system for a lot of the reasons that you just laid out, if you can, to do the

inferences

properly, you really need to be able to look at the data, you need to go look at the priors, you need to be

able to

adjust them and so forth. And it just doesn’t look like LLMs are good at that.

Speaker 1 – 01:25:39

What we have right now actually is a bit Frankenstein, which is in recent months, maybe the last couple

years,

people discovered in the industry that scaling was not really working. Scaling a pure pre training. So

most of the

money was back. Depends on the notion that if we just put in more data we would kind of. The word

was

emergence, but I, I think you can call it magic. The notion was when we magically emerged, that these

systems

would, you know, solve all the problems that critics like me had been pointing out for a long time. And

what they

realized last November, roughly speaking, so about a year ago, was that this was not working. And they

had some

intimation of that before. There’s very interesting data from Grok4 showing that what’s really helping our

tools. But

what are the tools?

Speaker 1 – 01:26:23

The tools, they don’t like to tell you this, but they’re actually symbolic systems. The tools are things like

Python,

which is a code interpreter, which is as symbolic as you can get. It’s full of functions operating over

variables. Go

back to Marcus2001 who tries to defend what symbol manipulation even means. It’s functions operating

over

variables. And so what they have done is they have stuck those unsettling inside of LLMs. And that’s

actually most

of what’s helping now. But it is Frankensteined, if I understand the connotation of that word, in the

sense that it’s

kind of like bolted on, which really fits the metaphor. It’s kind of bolted on and it sort of works. It works

better than

not doing that, but they actually get some mileage out of it.

Speaker 1 – 01:27:05

You look at this graph from graph 4, I have a paper called How O3 and Grok for accidentally vindicated

nerves like

ki, which you can find on substack that has this graph and tools are doing most of, you know, explaining

most of

the lift of the improvements that systems have constantly gotten. So there is actually empirical leverage

in

Frankenstein, but it’s Frankenstein in the sense that it’s added on as an afterthought rather than doing

what Karl is

rightly recommending, which is the same from first principle. What do we want cognition to even look

like? We

want to have agency, we want to have representations of uncertainty. I would say we want to have

what I call free

generalization, which I think is the world in the sphere of things that Karl said. How are we going to do

that?

Speaker 1 – 01:27:52Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

33 / 43

We’re obviously not going to do that with these opaque black boxes that don’t allow us to make those

kind of

inferences in an informed way representing uncertainty and so forth. So yes, you get something out of

Frankenstein and no, that’s not what we really want here. What we really want is a principled way of

bringing these

things together. And that’s where I think we should be putting our effort. And putting it together doesn’t

necessarily

mean equal weight. It doesn’t, you know, there’s, I think, a whole universe of ways in which you can

imagine, let’s

say broadly speaking, putting together things like neural networks that track distributions in some way

with things

like symbolic AI that can do a bunch of inferences and operations of the variables and so forth.

Speaker 5 – 01:28:34

Yeah, I, I mainly heard Karl was right and that’s how it always ends. We have a couple of minutes left.

Maybe if

there’s a. An urgent question from the audience.

Speaker 7 – 01:28:53

Thanks. Thanks so much for the discussion. I think it’s very interesting. First of all, I have to say I don’t

use

ChatGPT. I don’t use Cloud, I don’t use any generative AI for programming. I use. I hire people for that.

No, but I

wanted to say one important thing.

Speaker 1 – 01:29:09

If a man who cares about the quality of his results.

Speaker 7 – 01:29:14

So no, I wanted to say that if you go back to Moraveco paradox and all these things that he said are

way

impossible to solve and we are solving it. I mean, maybe not the right way that you want it, but

language has been

cracked somehow by statistical analysis. So. And this is kind of a result, right? So we cannot. So we are

seeing

that our machines talking, solving the TRIM test and this is very important, even if you can pretty turn.

Speaker 1 – 01:29:43

This is not at all important now that you’ve been your question. But I was important in the 60s at least.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

34 / 43

Speaker 7 – 01:29:49

Right? It was important in the 60s at least.

Speaker 2 – 01:29:51

Right.

Speaker 1 – 01:29:52

It was important in 1950. I just went to. I mean, if we’re going to pause there. I just was part of a 75th

anniversary

of Turing Test symposium at the Royal Society. Alan Kay spoke, I spoke. Some people dissected the

test in great

detail. The whole thing is available on YouTube. We spoke for seven hours. Not one person in those

seven. Oh, and

Dermot Turing was the last speaker. He’s Turing’s nephew. Not one person defended the test in

modern form. We

all agreed that it was nice to open the discussion about what thinking is and could a machine do it. But

we also all

agreed that test is really a test of human gullibility and how you could fool people.

Speaker 1 – 01:30:30

And we all agreed that we’ve known that since 1965 with the advent of Eliza, which fooled a lot of

people and

contributed nothing to the understanding of intelligence. Literally zero. The same thing with a system

called

Eugene Guzman that was arguably the first to win the Turing Test. Nobody would ever look to Eugene

Guzman’s

techniques for anything except how to fool people.

Speaker 7 – 01:30:52

Yeah, but so just to conclude, if you can criticize it as you don’t want that the way to do to solve AI as

you want so

that we do to. To understand cognition, but it is right that is working. So and this is something that.

Speaker 1 – 01:31:08

All right, I’m going to go after your second premise too.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

35 / 43

Speaker 7 – 01:31:11

So let me rephrase it. People are using it, so this is happening. So people are using as another

computational

element in their lives. So it’s a compliment to our intelligence. So this is happening.

Speaker 1 – 01:31:24

So we cannot say that their question in their.

Speaker 7 – 01:31:27

Well, I’m just saying that we can criticize it.

Speaker 1 – 01:31:30

That’s not a question. So I’m going to respond now since it’s not a question. Yes, I’m going to derive a

question

from your knock question which is what do you make of the fact that people are using it? How does that

relate to

the rest of the conversation? So yes, people are using it, sometimes successfully, sometimes no. There

was an

MIT study that showed 95% of companies using it or not getting return on investment. There was a

study by meter

showing that productivity was actually cut encoders who were sure that it was going down. So results

are mixed.

But some people are genuinely getting utility out of it. Not however universally. And there’s a set of

problems that

works really well on and a set that doesn’t like people are sticking them into robots, but nobody knows

how to

make a robotic plumber.

Speaker 1 – 01:32:15

Like there are lots of things that we have no idea right now how to use these things for. These things

are not that

great at science unless a lot of scientists spend a lot of time carefully constraining them to do a

particular

problem. You just throw into a chatbot and say, you know, solve cancer for me. You’re not going to get

any throw

into lm. You’re not going to get anything out of it that is useful. So there are things that are useful for

and there are

things that are not. One of my favorites quotes in Cognitive Sciences from Chaz Firestone and Brian

Schultz, two

cognitive psychologists. I could almost do it from Mary. They say cognition is not one thing, but many.

And they go

on to give a bunch of different examples.

Speaker 1 – 01:32:56

I don’t remember off all the examples, but the point is that there are many different facets of

intelligence. In fact,

I’m co author in a paper with yasha Bengio and 30 other people. Dan Hendricks is the lead author that

came out

yesterday. I think, I think the website is AGI definition AI. And the point of the paper, which is not

entirely Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

36 / 43

successful, I criticized my own paper and my own substack yesterday. It’s not entirely successful, but

what it tries

to do is to fractionate cognition into a bunch of different things. And there’s a nice little graph in there

where they

have 10 aspects of cognition that they look at based on a classic framing of cognition from Cattell. You

can find

lots of others. That’s part of my critique.

Speaker 1 – 01:33:34

But on that you could say very reasonably, LLMs are good at, you know, three or four of these and not

so good at,

you know, seven of them or something like that. Six or seven of them. So they are useful because they

do certain

parts of cognition well. They are much less useful than you could imagine AI to because there are many

aspects of

cognition that they don’t do well. So you know, thermostats, which are the most world’s most primitive

agents, as

we.

Speaker 5 – 01:33:59

Just learned.

Speaker 1 – 01:34:01

I mean, you can actually make that argument. Thermostats are useful, right? And they do an intelligent

thing, which

is they determine the temperature and they control the temperature in a room. They are a bit of

intelligence by

some definition. You could argue they have a semantics and argue that LLMs don’t have a semantics if

you really

want to have some fun. But they’re limited. They do some things and not others. We should take

another question.

Speaker 5 – 01:34:26

We’re actually at time, but let’s take an actual question.

Speaker 1 – 01:34:30

So my question is about agency.

Speaker 3 – 01:34:34

Sorry.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

37 / 43

Speaker 1 – 01:34:35

So I think the lack of agency also creates some of the problems in the LM we see today. Like GPT5

made a lot of

mistakes. But you cannot threaten and say, if you keep making mistake, I’m going to.

Speaker 5 – 01:34:48

Replace you with GPT6.

Speaker 1 – 01:34:49

Because, well, you can tell it that will make a difference. Exactly. Because it doesn’t have agency. Don’t

take agency

away from it. I’ll let Karl answer too, but my view is that’s not the core problem for why LLMs don’t

reason. Well, the

core problem is they don’t have world models. They don’t have proper semantics, they don’t have

abstract

representations. It’s not a motivational question. I mean, they kind of do what you’re telling them or they

try to do it.

They don’t really understand any of it. They’re all just doing pattern matching over language. Something

that I find

hilarious and then I’ll turn it over to the mic, is that the world that we have wound up in is one where

everybody in

machine learning. I’m exaggerating.

Speaker 1 – 01:35:33

Many people in machine learning have a violent aversion to putting any innate structure in their models

versus AI

to its credit, does not, but most of them do. But then they put out all this innate stuff in their system

prompts that’s

actually innate. The system didn’t learn what’s in the system problem. Some engineer tried a bunch of

stuff out and

did a bunch of experiments, said let’s build that in. That’s more like evolution than learning. They’re

building that

thing in after saying that they hate innateness, after dumping on me for saying we need innateness,

they stick it in

the system problem. But what’s even more embarrassing, or maybe equally embarrassing, is that you

say stuff like

don’t hallucinate in your system prompt. Like Apple literally had that. Does anybody remember the

Apple DTC news

summary debacle?

Speaker 1 – 01:36:16

Like it literally said don’t hallucinate and it hallucinated anyway. So like talking to a wall and telling the

wall don’t

hallucinate doesn’t really work.

Speaker 6 – 01:36:28

Very good entertainment. Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

38 / 43

Speaker 1 – 01:36:32

I try.

Speaker 6 – 01:36:35

So briefly responding to this, I think one has to acknowledge that sort of from an engineering

perspective, large

language models are beautiful. I mean, they are a really impressive bit of engineering and it’s

interesting to think,

why are they so beautiful? Why do we find them so alluring? Because it’s not the RL part of it that

makes them so

compelling.

Speaker 4 – 01:36:59

Because.

Speaker 6 – 01:37:00

The objective function is just prediction, it’s just predicting the statistics. Very much like a sort of

watering code of

in time. So what are the. So I think that’s interesting to note. It’s not reinforcement learning. That’s

realized the

beauty that we witnessed with large language models. It’s the use of universal function approximators

to do

prediction, basically. But what. And I think that’s the key thing. It’s just predicting stuff that we do. So I

think it’s the

fact that generative AI just generates a kind of stuff that we generate that renders them beautiful and

interesting.

But beyond that there is no utility.

Speaker 1 – 01:37:42

In other words, Karl has told you today, and I’m not fully endorsing, LLMs are rubbish and they are

parasites.

Speaker 5 – 01:37:51

You have one final question and then we can.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

39 / 43

Speaker 7 – 01:37:56

It’s a two parter. The first part is I’m curious whether you have agreement on the kinds of systems that

like Tim,

you’ve been working on, could be and have been developing could be considered to be a kind of

neurosymbolic AI.

And the other is. So the question of do language models have world model or not? So currently

organizing a

special issue of Royal Society, Yuri’s contributed really excellent article.

Speaker 4 – 01:38:22

On the topic of world models.

Speaker 7 – 01:38:24

But the question is like, is it something of a suitcase where we need to unpack? And maybe there’s

multiple kinds

of world models that are differently meaningful, I guess, rather than is there one kind of role model or

not? How

would people feel about there being like a typology of world models?

Speaker 6 – 01:38:38

And do you think it would be.

Speaker 7 – 01:38:39

Possible to come to agreement on this?

Speaker 1 – 01:38:43

Taking the last part first? No, unlikely that we will come to agreement. I think technology is actually a

good thing

and a worthwhile project. I just wrote a piece for my substack about, forget the title, but basically how

the

complete lack of world models was undermining LLMs. And the example that I gave, and I’ll come to

your point

about multiple definitions, was that you can’t take an LLM and have it reliably played a game of chess.

And there’s Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

40 / 43

a really interesting pattern of behavior which is an LLM can repeat the rules of chess. It can even

answer questions

like can a queen jump overnight? But then it will actually have a queen jump overnight illegally when it

plays.

Speaker 1 – 01:39:28

And so my argument there, and I would admit that there are different ways you can construct the term,

but my

argument there is it has not made a model of chess. This is despite being trained on millions of games

that are

there on the Internet. Being trained on the rules, which are Wikipedia and Chess.com, being trained on

books like

Bobby Fischer Teaches Chess, which is where I learned how to play a decent game of chess. And so

forth. That’s

all there. And it doesn’t actually absorb it. So for me, I would want to say if you can’t do that, you don’t

really have a

world model human of chess. And in fact, LLMs have been beaten by an Atari 2600 from 1975 with its

cartridge

from 1975 in chess.

Speaker 1 – 01:40:09

And so the Atari 2600 does have a world model of chess in the sense that I’m defining. I’m open to the

fact that

other people talk about it in different ways. And we talked about that a moment ago. There’s this

function

prediction sense. And I think that carefully articulated articulating all of that might have some value. Like

there’s

the sense that I’m describing where you have an interrogable system. You can ask questions like can

queens jump

overnight? You can expect that it will behave in a coherent and consistent way with those things that it

repeats. So

do LLMs have coherent world models as. As I’m describing? Clearly not. But are they predictive

functions? Well,

you could make that argument. I mean, what is it mean if you get the predictions wrong? But, you know,

you could

get into that.

Speaker 1 – 01:40:52

I think it would be a service to the field to try to spell that out more carefully.

Speaker 5 – 01:41:00

Closing ver.

Speaker 6 – 01:41:02

Well, actually, I don’t think I need to. I think I agree entirely.

Speaker 1 – 01:41:06

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

41 / 43

He agrees with means.

Speaker 6 – 01:41:07

Yeah, absolutely. Oh, I was due covertly.

Speaker 4 – 01:41:15

Too exquisite.

Speaker 6 – 01:41:15

About it because I’ll get into trouble. But I do think the question was, do we have a need for an ontology

of world

models?

Speaker 4 – 01:41:25

I certainly think that’d be really useful.

Speaker 6 – 01:41:27

I just made the obvious point that of course each artifact and each person, any instance of intelligence

will have its

own world model fit for purpose for the world and the niche and the content with which it has to

exchange. But

there seems to be in a sort of deeper thing here, which is a distinction between the implicit world model

in a

current generative AI, which is, I repeat, much more so in the universal function approximation sort of

analogy,

Alice, you’re implying a mapping from content to content or consequence to consequence. And to me,

that is

about very different kind of model typology, speaking from a mapping from cause to content, a

consequence. So I

think these are fundamentally different approaches to dynamics intelligence agency, however you want

to frame it.

Speaker 6 – 01:42:25

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

42 / 43

So I think it would be useful to have that fundamental distinction in play. I see this distinctly many other

fields and,

you know, it may be useful just to sort of reinforce that. So you’ve got a commitment to either this kind

of

descriptive model is consequence to consequence of content to content mapping. Or you’ve got a

commitment to

the true kind of understanding. World models that implicitly symbolic.

Speaker 1 – 01:42:50

And I will end on a note of agreement and with a reference to because I think we have to wrap up. I just

everything

that Karl just said, and we didn’t mention Judah Perl’s stuff on causality, his ladder of causality, but it’s

very

consistent with this, the three runs of causality. It’s not framed in quite the same language, but really,

world models

are core to what he’s saying as well. And it’s exactly consistent with what Karl is saying. You don’t want

to just

learn correlations between content. You want to have a world model that intervenes that you’re doing

causal

analysis over. So Karl agrees with me. I agree with Karl. And we thank you very much.

Meeting Title: EPISODE 19 – Karl Friston Gary

Marcus IWAI 2025.mp…Meeting created at: 9th Jan, 2026 – 12:47 PM

43 / 43