What’s Next for AI? OpenAI’s Łukasz Kaiser (Transformer Co-Author)

There is this thing that's happening in AI and in AI every week now a lot is happening. Fundamentally if you look at

happening. Fundamentally if you look at AI progress it's been a very smooth exponential increase in capabilities.

This is the overarching trend. It's not

like pre-training fizzled out. It's just

we found out a new paradigm that at the same price gives us much more amazing development. And this paradigm is still

development. And this paradigm is still very new. I think one of the biggest

very new. I think one of the biggest things that I would say people kind of know on the inside and others don't is that already right now it's not about the progress. There are so many things

the progress. There are so many things Chad or Gemini any LLM can do for you that people just don't realize. You can

take a photo of something broken ask how to repair it. It may tell you you can give it a college level homework and it will do it for you.

>> Hi, I'm Matt. Welcome to the Matt podcast. My guest today is Lucas Kaiser,

podcast. My guest today is Lucas Kaiser, one of the key architects of modern AI who has quite literally shaped the history of the field. Luc was one of the co-authors of the attention is all you need paper, meaning he's one of the

inventors of the transformer architecture that powers almost all the AI that we use today. He's now a leading research scientist at OpenAI, helping drive the second major paradigm shift

towards reasoning models like the ones behind GPD 5.1. This episode is a deep exploration of the AI frontier. Why the

AI slowdown narrative is wrong, the logic puzzles that still stump the world's smartest models, how scaling is being redefined, and what all of that tells us about where AI is heading next.

Please enjoy this fantastic conversation with Luc.

>> Lush, welcome.

>> Thank you very much. there was a uh narrative at least in some circles maybe outside of San Francisco throughout the year that um AI progress uh was slowing

down that we had maxed out pre-training that scaling laws were hitting a wall yet we are recording this at the end of a huge week or couple of weeks with

release of GBD 5.1 GPD 5.1 Codex max GVD51 Pro as well as Gemini Banana Pro um for Gro 4.1 Almo 3. So this feels

like a a major violation of of that narrative. What is it that people in

narrative. What is it that people in Frontier AI labs know about AI progress that at least parts of the rest of the world seem to not understand?

>> I think there is there is a lot to unpack there. So so so I I want to go a

unpack there. So so so I I want to go a little slower. There is this thing

little slower. There is this thing that's happening in AI and in AI every week now a lot is happening. you know,

new model, coding, doing slides, self-driving cars, images, videos, you know, there there's it's it's a it's a nice field that that doesn't make you be

bored for a long time. Um, but through all of this, it's sometimes hard to see the fundamental things that are happening. And fundamentally, if you

happening. And fundamentally, if you look at AI progress, it's been a very smooth exponential increase in capabilities. This this is the

capabilities. This this is the overarching trend and there has never been much to make me at least and I think my

my colleagues in the labs believe that this trend is not happening. It's a

little bit like Mors law, right? We we

Morse law happened through decades and decades and arguably you would say it's still very much going on if not speeding up with the GPUs. But of course it did not happen as like one technology was

bringing you there for 40 years. there

were was one technology and then another and another and another and another and and and this went on for decades, right?

So, so from the outside you see a smooth trend but from the inside of course you know progress is made through new developments in addition to the increase

of computer power and better engineering and so so all of these things come together and in terms of language models I think there was a big pivotal point I

mean one point was of course the transformers when it started but the other point was reasoning models and that happened I think one preview was a

bit a year and a month ago or something like that. So we started working on it

like that. So we started working on it maybe 3 years ago but but you you you know it's very recent if if you think of it as a as a paradigm that that's a very

recent thing. So, so it's always like

recent thing. So, so it's always like these S-curves, right? It it starts then it gives you amazing growth and then it flatlines a little bit though. Yeah, we

we'll get to the pre-training, right? I

feel pre-training in some sense it's on the upper part of the S, but it's not like scaling for pre-training don't work. They they totally work like if you

work. They they totally work like if you what scaling clause says is that your loss will log linearly decrease with with your compute. We totally see that and clearly Google sees that and all

other labs. The problem is, you know,

other labs. The problem is, you know, how much money do you need to put into that versus the gains you get and it's just a lot of money and people are

putting it. But with the new paradigm of

putting it. But with the new paradigm of reasoning, you can get much more gains for the same amount of money because it's on this like lower and like they're just discoveries to be made and these

discoveries unlock insane capabilities.

So, so it's not like pre-training fizzled out. It's just we found out a

fizzled out. It's just we found out a new paradigm that at the same price gives us much more amazing development.

And this paradigm is still very new. It

happens so fast. I think you if you blink you may miss it cuz it was basically you had chat 3.5 right GPD 3.5 in chat and it would give you answers

and it used no tools, no reasoning. It

would answer you something. And now you have chat and you know if you were not into it you may have blinked and it also gives you answers and you may say okay it's more or less the same except the

chat now will you know go look on some websites reason about it and give you the right answer instead of something it memorized in its weights. I very much

used to like this example of you know when what time does the SF zoo open tomorrow like the old chat would tell you right totally hallucinate from its

memory an hour that it read zoo opens on probably the zoo's website from 5 years ago and it didn't know what's today or tomorrow so it would just assume it's a weekday chat now knows what's today

because it's in the system prompt it goes to the zoo website reads it extracts the information if it's ambiguous probably checks three other websites just to confirm and then gives you the answer. But if you blink, you

may think it's the same, but no, it's it's dramatically better. And you know, as a consequence, since it can read all the websites in the world, it can give

you answers in in stuff that it wouldn't be able to even touch before. So, so

there is tremendous progress, right? And

it happened so fast it it may even be missed. I I think I think one of the

missed. I I think I think one of the biggest things that I would say people kind of know on the inside and others don't is that already right now it's not

about the progress like there are so many things Chad or Gemini any any LLM can do for you that people just don't realize like you can take a photo of

something broken ask how to repair it may tell you you can give it a college level homework and it will do it for you probably so so That that's absolutely amazing.

>> So there is an education gap to some extent.

>> Well, it just happened like I mean if you think you you said codeex, right?

You know programmers are conservative a little bit. I I I still use Emacs from

little bit. I I I still use Emacs from time to time. All the coding tools like okay it will complete one line for me but but people are very like this is my editor. I write code here now people are

editor. I write code here now people are like no this is codeex. I ask it to do stuff. I will fix it later. Right? But I

stuff. I will fix it later. Right? But I

think it's the recent few months when the transition happened from you know people using it sometimes but rarely to now basically this being how a lot of people work in coding that that's quite

big. I'm not sure everyone's aware of it

big. I'm not sure everyone's aware of it but but it's also like if you don't do programming why would you be aware of it? I I do believe though that this will

it? I I do believe though that this will come to more and more domains >> to the point of all of this being very new and somewhat sudden. Uh something

that you or I hear from time to time when talking to people um is that part of the reason why people are so optimistic is that there is a lot of

lowhanging fruit very obvious things to improve uh for those models in the next few months. First of all, do you agree?

few months. First of all, do you agree?

And second, can you give us some examples like obvious thing that you need to fix next and that the industry will fix?

>> Yes, there is a ton of extremely obvious things to fix. A larger part of this ton is is just hard to talk about on a podcast because it is in the engineering

part. You know, every lab has their own

part. You know, every lab has their own infra uh and their own bugs in the code.

Machine learning is beautifully forgiving in some sense in contrast to old software engineering which would just yell at you when you made a mistake. You know, our Python code

mistake. You know, our Python code generally probably run except much slower and give you worse results if you run it wrong. So you realize, oh no, it was wrong. And you improve it and the

was wrong. And you improve it and the results get better. These are huge distributed computing systems. They're very complex to run. So So there is a huge amount to improve and fix and

understand in in the process about just how to train your model and how to do RL because RL is more finicky than pre-training. It's harder to do really,

pre-training. It's harder to do really, right? So every day this is our

right? So every day this is our day-to-day work. On top of that, there

day-to-day work. On top of that, there is data. You know, we used to train on

is data. You know, we used to train on just like common crawl basically. It's a

it's a big repository of the internet that people just scraped without regard of what and and some things came in, some didn't. It it was a mess. So now,

some didn't. It it was a mess. So now,

of course, every larger company has a team that that tries to filter this and and improve the quality. Um it's a but it's a lot of work to to really extract

better data. Now synthetic data is

better data. Now synthetic data is becoming a thing but when you generate synthetic data it really matters how you do it with what model what you the whole how the in the engineering aspects of

everything it's such a new domain that the the the you know it was done somehow it works it's beautiful but there is just so much to do better that that people I don't

think have any doubts that there is a lot there and on top of that there are the big things like multimodal I mean language models they they are now as as

I as I'm sure you know and and most people realize actually vision language models because and they can also do audio. So they're multimodal models to

audio. So they're multimodal models to some extent but the multimodal part still lags behind the text part to a large extent. So that's one big area

large extent. So that's one big area where obviously you need to do better and it's not a huge secret how you can do better. You know there there are some

do better. You know there there are some methods that maybe will make it even amazingly better but there are some very simple methods how you can do just better but you know this maybe requires retraining your whole base model from

scratch and that takes few months and it's a huge investment so we need to organize it so so so there is a lot

of just work that that will undoubtedly make things better. I think the big question that

better. I think the big question that that people have in their mind is how much better will it make them?

>> So I'd love to do uh a little bit of a deep dive slash educational part on um the whole reasoning model aspect uh because uh as as you just mentioned

since it's so new uh some people truly understand how those work many people don't. At a very simplistic level, what

don't. At a very simplistic level, what is a reasoning model and how is that different from um your sort of base LLM?

>> So, a reasoning model is like your base LLM, but before giving you the answer, it it thinks what people call in the chain of thought, meaning it generates

some tokens, some text that's meant not for you to read, but for the model to give you the better answer. And while it does this these days, it is also allowed

to use tools. So it can for example in its thinking pro so-called thinking process go and browse the web and to give you a better answer. So so that's

the superficial part of the thinking models. Now the deep part is that you

models. Now the deep part is that you start treating this thinking process as part of the model basically. So it's not something the model generates and and

it's an output for you. It's something

you want to train right. you want to tell the model you you should think well you should think so that the answer after this is good in in whatever way and this leads you to a very different

way of training the model because models were in usually trained with just gradient descent the way deep neural networks are trained meaning you say

predict the next word and you do a gradient you differentiate your function from the model they're not fully differentiable but you approximate it

and you train your weights to to do that and that it was quite amazing that doing just that you could make a chat but with a reasoning model you you can do that because there is this reasoning part you

can differentiate through that so we train this with reinforcement learning and reinforcement learning basically you okay there's just this reward and you need to do a bit of tries and reinforce

meaning push the model towards doing more of the things that lead to better answers and this kind of training is a bit more like it has more restrictions than the training we used before. The

training we used before, you took all of the internet, put it in, even if you didn't filter it very well, it would mostly work. Reinforcement learning, you

mostly work. Reinforcement learning, you need to be careful like you need to tune a lot of things, but you also need to prepare your data very carefully. So

currently and current for at least the most basic ways we use it currently, it needs to be fairly verifiable. So there

isn't is your answer correct or not? You

prepare data for that. You can do that in mathematics, coding very well. Uh you

can do this in science to some extent right you can have test questions they're correct or not but you know if it comes to like writing poems is this poem good or not it's for now the

reasoning models are really shine in in domains like science and they've brought some improvements to to non-science domains but but it's not quite as huge

maybe yet as it as it could be I mean at least compared to mathematics and coding then there is the multimodal question.

How do you do reasoning in multimodel? I

think this is starting like uh I saw some Gemini creating images in the reasoning part that quite exciting but it's very very yeah >> retraining and reinforcement learning

part is is um particularly interesting from an educational point of view again because um it seems that people have have come to the conclusion that there is the pre-training world and then the

post-training world and the post-training world is mostly reinforcement learning but this idea that there is reinforcement learning in the pre-training uh I don't think is as understood by everyone

>> at the beginning of chat let's say there was pre-training uh people did not do RL right then but then you couldn't really chat with it so chat was RLHF applied to

a pre-trained model but the RLHF was was different kind of RL of sorts right it was like very small and it was human preference that was telling you what is

better that's what the HF is human feedback right you You showed people like pairs of stuff, you learned a model that says, "Well, people seem to prefer this as an answer." You trained with

that, it would very quickly what we today say hack this model. Like if you trained the RLH of too long, it would start giving things that satisfy this model that seems supposed to model human

preferences. So it was a bit of a

preferences. So it was a bit of a brittle technique, but it was a bit of RL that was extremely crucial to to making the models chat. These days, I think most people move towards this big

RL. It's still not as big as

RL. It's still not as big as pre-training in the scope, but but it it says you have a model that says whether this is correct or not, or if it's a

preference, it it it's a very strong model that analyzes things and says you should prefer that and you have data that's restricted to a domain where you can do this well. And then you can also

put some human preferences on top, but but you make sure that that you can run a little bit longer without making this whole grading fall apart. But again,

this is this is around today. I I do believe the role of tomorrow will will be broader. It will work on general data

be broader. It will work on general data and maybe then it will expand to like domains that that go beyond where where it shines today. Now will it shine there? is a is is a different question.

there? is a is is a different question.

Do you really need to think very much doing some of the things? Maybe not. But

maybe yes, we maybe we think maybe we do more thinking and reasoning than we than we kind of consciously call thinking.

>> What would it take for RL to generalize?

Is that better evaluations? like you you guys released GDP val a few weeks or months ago uh to sort of measure performance against sort of broad

economic sectors. Is that is that part

economic sectors. Is that is that part of what the system needs?

>> I think this is a small part of it. I I

I think that's one part. But but if you think of economic tasks, you know, making slides is important there, following instructions, doing

calculations. It's not math, but it's

calculations. It's not math, but it's still very verifiable, right? What what

I'm thinking about is when you do pre-training, you take the internet and you just say ask what's the next word.

You know, you could think before you ask what's the next word. Obviously, now you don't want to think before every word probably, but I don't know if you've ever looked at the training data for

like a real pre-training run cuz I think people mostly don't realize how bad this is. Like hotels.com is a great website

is. Like hotels.com is a great website compared with the average chunk of 2,000 words from the internet. It's it's a mess, right? And also a miracle that

mess, right? And also a miracle that from this the pre-training process gets you something reasonable. So you

probably don't want you know imag imagine you have a hotel website telling you know it's a beautiful vacation you don't necessarily want to have a very long chain of thought before that right if it was written by a person there was

probably some kind of thinking that went into it maybe maybe not as elaborate as the math and coding thinking but maybe there was something going on so maybe you want a little bit of thought before

at least some of the text and that our models can't do very well yet I I think they're starting I think it it there is a lot of generalization in in this reasoning. If you learn to think for

reasoning. If you learn to think for math, you can you will sometimes do some you know some strategies are they transfer very much like look up on the web and see what they say and and use

that information. So some of these

that information. So some of these things are very generic and they start to transfer. I feel like summer maybe

to transfer. I feel like summer maybe not yet especially thinking in in the visual domains is very undertrained I believe

but you know we work so so so we will try to try to push for for more of that.

>> Going back to chain of thought how does that actually work? How does the model decide to create that chain of thought and is what we see so the little um you

know intermediary steps that we see on the screen as users uh the the chain of thought that's exposed to us is that what's actually being processed by the

model or is there a deeper longer broader chain of thought that happens behind the scenes >> so in the current chat GPT you will see a summary of the chain of thought on the site so there is another model that

takes the full chain of thought shows you a summary because the full ones are usually not very nice to read. They're

more or less say the same thing just just in more messy words. So So it's better to have a more readable summary.

When you start with a chain of thought, the the first paper about chains of thought, you basically just ask the model please think step by step and it would it would think. So if you just pre-train a model on the internet and

ask it to think step by step, it will give you some chain of thought.

Interesting and most important point is you don't stop there. You say okay so you have you start with some way of thinking and then you say sometimes this leads to a correct answer and sometimes

it leads to a wrong answer. So now I'm telling you I have some training examples you will think a 100 times and say 30 lead you to the correct answer then I'll train you on these 30 examples say this

is the way you should be thinking that's the reinforcement learning part of training changes dramatically how the models think. We see this for math and

models think. We see this for math and coding. But but the big hope is it could

coding. But but the big hope is it could also change how the models think for many other domains. Even for math and coding, you you start seeing that the models start correcting their own

mistakes. Right? Earlier if the model

mistakes. Right? Earlier if the model made a mistake, it generally just tell you what it did and insist that the mistake was right or something like that. With with with the thinking, it's

that. With with with the thinking, it's like, oh, I often make mistakes, but I need to verify and correct myself to to give the correct answer. So, so this just emerges from from this

reinforcement learning which is beautiful, right? It it's clearly a good

beautiful, right? It it's clearly a good thinking strategy to to verify what you want to say. And if you think it may be an error, then think again.

That that's what that's what the model learns on the most abstract level.

>> Great. Thank you for this. All right. As

a quick uh detour and we'll go back to more Frontier AI topics. I'd love to talk a little bit about um your story. I

mean you have the incredible distinction of uh having been at the forefront of this industry both the transformers paper which was the the birth of one paradigm and now you're very much

leading the charge on the reasoning model part which is another paradigm so this just incredible story how did you become an AI researcher >> I was a mathematician and a computer scientist but but in theoretical

computer science >> and that started in high school as a kid >> yes I was definitely very into math math in high school and into computers also

later in high school. Yes, I did my studies in Poland. I went for a PhD in Germany. It was the theoretical computer

Germany. It was the theoretical computer science and mathematics PhD. So I very much I'm a mathematician. Yeah. I I I was always fascinated by you know how

how is this thinking going? What is

intelligence? As as a child I always wanted to like emulate the brain. They

thought well okay maybe higher level explanations are more interesting. I did

research in logic but a little programming but then there was this opportunity to join Google just as the deep learning was starting off. I

already had my tenure position in France and the French system has this beautiful thing that you can take a leave of 10 years.

>> Yes.

>> And and you can still return anytime you want from the so so it's a no risk.

>> So at some point when you when you've solved AGI you may return back to France and be a professor. Well, if you solve AGIA, they may take you anyway.

The the nice part about the leave is that they will take you back even if you don't. So,

don't. So, but it's actually very important. I a

number of I I think there's a number of Nobel Prize winners who who took this leave to just try something more risky and you know, sometimes it works, sometimes it doesn't. there there's a

lot of luck in in in science and research but it's very good to have this opportunity to take it so I came to Google that was Google brain at the time you said right >> I came to Ray Curtzswwell's group he was

my first manager he interviewed me and was very inspiring I I I was my first interview was to join like the YouTube UI team and I was like okay I'm not going and and then I had an interview

with Ry and I knew him of course from his books and and he's a very inspiring person so I was like Okay, let's go. The

team was separate from Google Brain at that time. Then I moved to Google Brain,

that time. Then I moved to Google Brain, worked with Ilia Discover, another very inspiring person. There's an amazing

inspiring person. There's an amazing number of great people in in AI in the Bay in general.

>> I have to ask you at this point about the Transformer paper story, how it all came about. The eight of you, right?

came about. The eight of you, right?

Seven or eight of you. How did you all get together?

>> Well, we never got together.

>> You never got together. Okay. I I

recently got a photo on Twitter of a photo session of all eight of us and it it it's it was saying it was fake but I knew it was fake because I don't think all eight of us were ever in the same

physical room. These ideas developed

physical room. These ideas developed from many sites before and after like Jakob were work and Tilia Pushukin worked on attention like self attention

of course attention was there from the encoder decoder side >> and maybe maybe one minute for the broad public on what attention actually means since it's such a fundamental concept.

So, so, so attention is the mechanism that tells the model as as you're doing the next thing, look into your past and find the most similar things that you

see in the past to to what you are seeing right now. It came from the machine translation times where people wanted to align words in one language

with words in another. They were like, okay, so this word where in this previous sentence would it be? It it

it's it's an analog of alignment for deep learning. It's now called attention

deep learning. It's now called attention and in in AI just says you know think of like what comes to your mind as you are here now in in this environment what

things from the past are are similar to it and this mechanism was already used in in in deep learning translation before but it was used there was like one encoder model and the decoder would

be looking at the encoder but never at its own states. main novelty of Transformer was self attention. But

Transformer is more more than just this idea. I I think that's important. I

idea. I I think that's important. I

think that's the the beauty of these eight weird people somehow came together even though not physically to to do it is that we all approached it from

different sides. So so so there was

different sides. So so so there was people working on the attention idea.

There is the you know you need to put this in the network that needs to have a lot of knowledge. So, so there is the feed forward layer that expands and then contracts. Zunam was working on this and

contracts. Zunam was working on this and nowadays used mixtures of experts which actually came before the transformer.

So, so how do you store knowledge in neural networks is is another important question and it's part of this model too. And then you know in deep learning

too. And then you know in deep learning people laugh that ideas are cheap making them work is is the hard part. So

>> how do you write the systems and the code and the baselines to actually make this train? And and this is funny to say

this train? And and this is funny to say now because nowadays you can take any deep learning framework and say you know X equals transformer X train and it will

basically work but back then it totally did not. So you need things like

did not. So you need things like learning rate warm up or or tweaks to the optimizer that that were just work.

And I did a lot of coding and at that time was working on TensorFlow and parts of the framework. And I I remember distinctly that people were like, "So you want to use the same model for a few

different tasks?" Like why do you even

different tasks?" Like why do you even do that? Like if you have a different

do that? Like if you have a different task, like if you do translation, you train one model, if you do parsing, you train another. If you do image

train another. If you do image recognition, you train a third. Like you

never train the same model for three different tasks. Why do you even write

different tasks. Why do you even write like APIs to to do multiple tasks on one model? And I was like, "No, no, we're

model? And I was like, "No, no, we're going to do all tasks in one model." And

people were like, "No, no."

>> So there was a lot of push back against the idea.

>> Not against the idea. Google was also an amazing place at that time that they would very happily let you work on whatever you wanted. But but I don't think there was widespread belief in

doing multiple tasks with the same model. Not to mention,

model. Not to mention, you know, this idea that you I I still find this idea that you take basically the same model as transformer. Like now

there is a bunch of changes to it, but you could in principle take the same architecture as the decoder from the paper, train it on all of the internet and it will basically start chatting with you. It would have back then

with you. It would have back then definitely sounded as a worthy dream. We

maybe had as a dream but not reality that you'd expect 5 years later. It's

very lucky that it actually works so well right?

>> Talk about the transition from uh from Google to open AI and perhaps how those two cultures are different.

>> So, I cover was my manager at Brain.

Then he he went on to to found OpenAI.

He asked me a number of times whether I would like to join in in the years. I

found it a little bit too edgy at the time. Then Transformers came. So, we had

time. Then Transformers came. So, we had a lot of work with that. And then COVID came and and COVID was a tough time for for for the whole world, right? But

Google totally closed. Google was

reopening extremely slowly. So So one part of me was I I find it very hard to do remote work. I much prefer to to work with people directly. That was one

reason. But the other was also Google

reason. But the other was also Google Brain when I joined it was a few dozen people, maybe 40, something like that.

When when I left it was 4,000 3,000 people. spread across multiple offices.

people. spread across multiple offices.

It it's very different to work in a small group and to work in a in a huge company. So with all this, IA was like,

company. So with all this, IA was like, you know, OpenAI though is in a much stabler state. We're doing language

stabler state. We're doing language models. You know, something about this

models. You know, something about this that may look like a good match. And I

was like, okay, let me try. I've never

worked in any company other than Google before, other than the university. So it

was quite a change to the small startup group, but I I I I like working in smaller groups. It's it it it has its

smaller groups. It's it it it has its pleasure, right? It has a little bit of

pleasure, right? It has a little bit of a different in intensity sometimes. In

general, I I I I found it very nice. On

the other hand, Google, you know, has merged made the Gemini and I hear it's also a very nice place. I I think in in general the the tech

labs are more similar to each other than people think. There are some differences

people think. There are some differences but but I think if if I look from at it from the world you know from the university in France the difference between this university and any of the

tech labs is much larger than than between one lab or the other.

>> How are the research teams organized within OpenAI?

>> Um they're organized they're not very organized. I mean we do organize them but some you know people have managers and we sometimes talk to

them. Um no but mostly people find like

them. Um no but mostly people find like projects there there are things to do right like improve your multimodel models improve your reasoning improve

your pre-training improve whatever this part of the infrastructure people work on it you know as we go through these parts right there is infrastructure pre-training reasoning I think the parts

are the same for for most of the labs so so there will be teams doing these things and and then sometimes people um change teams sometimes new things emerge

There is always some smaller teams doing like more adventurous stuff like diffusion models at times. Then you know some of the more adventurous stuff like

video models gets big and and and then maybe they need to grow.

>> Do people compete for GPU access?

>> I don't think it's so much people that compete. I think it's more projects that

compete. I think it's more projects that compete for GPU access. There's

definitely some of that. On the other hand, like on the big picture of GPU access, a lot of this is just determined by how the technology works, right?

Currently, pre-training just uses the most GPUs of all the parts. So, it needs the most GPUs, right? RL is growing in the use. Now, video models of course use

the use. Now, video models of course use a lot of GPUs too. So, you need to split them like this. Um, then of course, you know, people will be, "Oh, but my thing would be so much better if I had more

GPUs." And I've certainly said that a

GPUs." And I've certainly said that a number of times too. Um so then you kind of push like you know I really need more and then some people may say well but you know there's only so much there is

never enough GPUs for for everyone. So

there is some part of the competition but the big part is just decided by the fact how the technology works currently.

>> Great. What is um next for pre-training?

We talked about uh data, we talked about engineering a big uh GPU compute aspect to this. Uh what happens to pre-training

to this. Uh what happens to pre-training in the next year or two? pre-training as

I said I I think it has reached this upper level of the scurve in terms of science but it can scale smoothly meaning if you put more compute you will

get better losses if you do things right which is extremely hard and that's valuable um you don't get the same payoff as as

as pushing carell but it generally just makes them all more capable and that's certainly something you want to do I think what people underestimate a little

bit in in in the big narrative is you know open AAI 3 four years ago I I joined even before that was a small

research lab with a product called API but you know it it was not such a big there was no GPU constraint on the product side for example all GPUs were

just used for training so it was very easy as a decision for the people to say you know we're going to train GPT4 this will be the smartest and largest model ever. And what do we care about small

ever. And what do we care about small models? I mean, we care of them as to

models? I mean, we care of them as to make like to debug the training of the big model, but that's it. So, GPT4 was the smartest model and it it was great,

right? But then it turned out, oh, there

right? But then it turned out, oh, there is this chat and now we have a billion users and you know, people want to chat with it a lot every day and you need GPUs.

So you train the next like huge model and it turns out you cannot satisfy this like people will not want to pay you enough to chat with the bigger model. So

you just economically need the smaller model. So and this happened of course to

model. So and this happened of course to to all the labs because like the moment the economy arrived and it became a product you had to start thinking about

price much more carefully than before.

So I think this caused the fact that instead of just training the you know largest and largest thing you can for for the money you have we said well no we we're going to train like the same

thing but same quality but smaller cheaper the pressure to to to give the same quality for less money is very large in in some senses as a researcher almost

makes me a little sad I have you know a big love for these huge models that um you know people say human brain has 100 trillion synapses or orders of magnitude of course are not that exactly

calculated but our models don't have 100 trillion parameters yet so maybe we should reach it I I would certainly love it but then you need to pay for it so I think this is this may be why people

kind of think that pre-training has paused because a lot of effort went into training smaller models now on the side people kind of rediscovered how amazing distillation is distillation means you

can train a big model and then put the same knowledge from the big the big is a teacher to the little one. People knew

about distillation. It's a paper or a long time ago but somehow at least for open AI I think maybe it was more in Google's DNA when Oriel was

there. Uh but people kind of

there. Uh but people kind of rediscovered how important that is for the economics but now it also means that oh training this huge model is actually good because you distill all the little

ones from it. So now maybe there is a bit more of a return to like it's also matter you know once you realize you have the billion users and you need the GPUs you need to invest into them and of

course we're everyone sees this there's a huge investment but the GPUs are not online yet so when they come back online and I think this may play into this what people call resurgence of pre-training

we both understand that you can distill this amazing big model and there is now enough GPUs to actually train it so so so it's resurging But all of this fundamentally happens on the same

scaling curve, right? It's it's not like we didn't know that you could do this.

It's more like the different, you know, different requirements of different months have have changed sometimes changed the priorities. But I think it's good to step back from it and and and

think of the big picture, which is that pre-training has always worked. And and

the beautiful thing is it even stacks with RL. So if you run this thinking

with RL. So if you run this thinking parallel process on top of a better model, it works even better than if you run it on top of a of a smaller model.

So So >> one question that I find fascinating as I hear you speak and the evolution of the modern AI systems has been this combination of LLM plus RL plus a lot of

lot of things going on. It used to be at some point and maybe that was back in the deep learning days that uh people would routinely say that uh they they

understood how AI worked at a at a micro level like the matrix multiplication aspect but didn't fully understand once you had everything together what really

really happened at the end of the day in the model and I know there's been tons of work done on uh interpretability over the last couple of years in in particular uh But particularly for those

very complex systems, is it increasingly clear what the models do or is there some element of blackbox that persists?

>> I would say both. There is a huge progress in understanding models.

Fundamentally, I mean think of the model that is chat. It it talks to a billion people about all kinds of topics. It

gets this knowledge from reading all of the internet. Obviously, you cannot

the internet. Obviously, you cannot identify like I cannot understand what's going on in there. I don't know the whole internet. What we can identify is

whole internet. What we can identify is there was a beautiful paper just I think last week with from open AI about if you tell the model that lots of its weight should be zeros it should be very sparse

then you can really trace when it's thinking about one particular thing then you can trace what's actually what it's actually doing. So if you say limit

actually doing. So if you say limit ourselves to this and you really study this inside a model then then you can get a lot of understanding and and there there's circuits in the models entropic

had great papers on that.

The understanding of what the models are doing on a higher level has progressed a lot. But then it's still an

lot. But then it's still an understanding of what smaller models do, not the biggest ones. But it's not so much that these patterns don't apply to bigger models. They do. It's just the

bigger models. They do. It's just the bigger models just do so many things at the same time that there is some limit to what you can understand. But I think this limit is is a bit more fundamental

than people think. It's like every very complex system. You can only understand

complex system. You can only understand so many things and then you don't.

Right.

>> Thank you for all of this. I'd love now to talk about 5.1 and do a little bit of a deep dive on all the latest stuff that you guys have released in the last couple of weeks which has been very uh

impressive. In particular as a user

impressive. In particular as a user think that the 5.1 moniker doesn't do justice uh to the evolution between 5.1

and five. It it feels like a much larger

and five. It it feels like a much larger improvement than the number would would indicate from uh again my perspective as a as a user. Walk us maybe through the

evolution of uh from GD4 to 5 to 5.1.

What has actually changed?

>> That that's a very tough question. I

think less than you think it's um no I mean the from GPD4 to 5 I think the biggest thing that changed is

reasoning meaning RL and synthetic data as I told you the pre-training part in that time frame was mostly about making things cheaper not making things better so of course the price has changed

dramatically too right thousand times I think or some of these orders of magnitude the the main improvements from four to five is adding reasoning with

reinforcement learning and this allowed to generate synthetic data which also improves the model. So that's the that's the big picture. In addition to that, CH

GPT is is now a product used by a lot of people. So the the post- training team

people. So the the post- training team has learned a tremendous number of lessons and it's added you know things clearly experimented wanted the model to be very nice to you then it turned out

to be too nice. Then now when a lot of people use it, you need to be really careful about safety, right? There may

be people that are in distress using the model. The model needs to do something

model. The model needs to do something reasonable in these cases. It was not trained for it before. Now it is and and it makes the model much better. But you

know at the same time you don't want to refuse to answer any question that has any sign of anything. So, so as you work on these things, you make the model much better in use, not not just for the

people in distress, but but for everyone who wants questions answered, but the answers to be reasonable. And you know, there was these things called hallucinations. It's still with us to

hallucinations. It's still with us to some extent, but dramatically less than than two years ago. Some part of that is because reinforcement learning can now

use tools and gather data and it also encourages the model to to like know you know verify what it's doing. Uh so

that's an emergent thing from from this reinforcement learning of reasoning but also you just add data because you realize sometimes the model should say I

don't know. So you add this to the post-

don't know. So you add this to the post- training data. You say like well we

training data. You say like well we really need to give it a thought how the model should answer people in in various situations. The 5 to 5.1 is is mostly

situations. The 5 to 5.1 is is mostly this kind like it's mostly a post-training improvement.

>> Yeah. So to double click on this because this super interesting. Indeed. Um, as

part of 5.1, there's, uh, the ability to choose different kind of styles from nerdy to professional. And, uh, that's, uh, I guess in reaction to the fact

that, um, some people were missing the sequantic aspect of earlier models when chat GBD when when GBD 5 came out. And

so, so adding more tones, that's all post trending stuff. So you tell the model those are examples of like how you should be responding which is more like

a like supervising kind of a paradigm or is that uh is that RL like right or wrong with rewards? How does that work?

>> I don't work on post training and it certainly has a lot of quirks but I think the main part is is indeed RL where you say okay is this response cynical is this response like that and

and and you say okay if you were told to be cynical this is how you should response. If you were told to be funny,

response. If you were told to be funny, try this one. So I I do think there is RL is is is a big part of >> in between models uh or or or different

versions of the models. Are the are the releases aligned with pre-training efforts or sometimes you have like one big pre-training effort and like several models that that come out based on that.

There used to be a time not that long ago be half a year distant past where where where the models were did have an

alignment with with technical stuff right so they would align whether with either with RL runs or pre-training runs that's why you had um a beautiful model

called 40 which was aligned with a pre-training run which uh was obviously worse than the 03 aligned with an RL run that was the follow-up to 01 naturally

because you couldn't use the name O2 but it was slightly better than the O4 mini because that one was mini and you know we had this beautiful model picker and

people kind of thought this was not the best naming for some whatever reason so um no I mean it was fairly obvious that

this was very confusing right so so so now the naming is by capability right GPD5 is a capable model 5.1 is a more capable model. Mini is the smaller model

capable model. Mini is the smaller model that's slightly less capable but faster and cheaper and the thinking models are the one that do more research. Right?

And that sense the naming is detached from any technical in particular you know 5.1 may be just a pre-training uh sorry post-training thing but maybe 5.2

towards the newly pre-trained model or maybe not but but but the naming has detached from the technology which also gives some you know as open has grown there is a number of projects right

there is RL and pre-training and there maybe you know something just to make slides better or whatnot um and with distillation you have the ability to put a number of projects into one model it's

kind of nice that you don't need to wait on all of them to complete at the same time and you can try to periodically put together actually make sure that as a product it's nice to the users and good

and and do this separately from you know waiting on the new full pre-training run that takes months and so on. So I feel like even though you know a little tear in my eye goes for the times where it

was that pre-trained model number that that was the number as it's a product serving a billion users it's maybe inevitable that you should name it by what the user should expect from it

rather than >> in 5.1 uh you have additional granularity uh in terms of uh telling the model how

long it should think by default. How

does the model decide how long it should think?

>> So the model sees the task. It will

decide on its own a little bit how long it should think. But you can put give it an additional it's trained with an additional information that can tell it think even harder and and then it will think longer. So you have now the

think longer. So you have now the ability to steer that I I still think it is important to realize.

So this is the fundamental change that came with reasoning models that using more tokens to think increases your capability and it increases it given the

computation way faster than pre-training right so so if you give GP5 the ability to think for long it can solve tasks

that are you know we had these gold medal at mathematical olympiad and computer science olympiad so so amazing abilities.

At the same time, the the fundamental training method of of reasoning is very limited to science data. So, so it's not as broad as the

data. So, so it's not as broad as the pre-training, which I I think like pre-training models felt kind of almost uniformly good or bad at things. I mean,

this was still not uniform because it's not like teaching humans, right? But the

reasoning models are even more people call it jagged, right? They have amazing abilities somewhere and then close by not so much. And that can be very

confusing. It It's something I I always

confusing. It It's something I I always love that it's weird because you can say the model is amazing at mathematical olympiad. At the same time, I have a

olympiad. At the same time, I have a math book for um I have a first grader daughter in the first grade. She's 5

years old. I took one exercise from this math book and none of the frontier models is able to solve it. and you

would be able to solve it in 10 seconds.

So that's something to keep in mind.

Models are both amazing and there are tasks that they cannot do very well. I I I can show you this as an

very well. I I I can show you this as an example. I think it's quite interesting

example. I think it's quite interesting to keep in mind. Let me start with Gemini 3 just to blame the competitors.

>> Yes, please.

>> So it has you see two two groups of dots on both sides and the question is is the number of dots even or if you look at it you see oh they're like two identical things. So that would be even. That's

things. So that would be even. That's

what the 5-year-old is supposed to learn. But there is one dot that's

learn. But there is one dot that's shared. So So now that that must be odd

shared. So So now that that must be odd for this simple one which has like you know I don't know 20 dots or so. Gemini

3 actually does it right? That it it finds out that that that it's an even number of dots and and it says that and that's great. And then you have another

that's great. And then you have another puzzle which is very similar except now there are two mountains of dots and there's also one dot shared at the bottom now and right in context right

after that you ask okay how about this one and then it it it it does some thinking and it just totally misses that there is a shared dot and it says the number is even and it's like in context

where you've seen this first example how would you ever miss that you know and here is the same the exact same prompt for GPT 5.1 point when thinking and it also solves the first it it it sees the

dot. It says it's odd and then it sees

dot. It says it's odd and then it sees the mountains and somehow it doesn't see the dot and it says it's even. The the

the nice thing is if you if you let it think longer this is like or if you just let it think again it will see it. So so

if you use GPT5 Pro it takes 15 minutes.

So you know this is the the human 5-year-old takes 15 seconds. The GPD51

Pro will run Python code to extract these dots from an image and then it will count them in a loop. So that's not quite >> and and why why is that? What trips up the model?

>> I I think this is mostly multimodal part. The models are just they're

part. The models are just they're starting like you see the first example they managed. So, so they've clearly

they managed. So, so they've clearly made some progress, but they have not yet learned to do good reasoning in multimodal domains and they have not yet

learned to use one reasoning in context to do the next reasoning. What is

written in context is is you know learning in context happens but learning from reasoning in context is still not very strong. All of these though are

very strong. All of these though are things that are very wellnown and like the models are just not trained enough to do this. it's it's just something we know we need to add into training. So I

think these are things that will generally improve. I do think there is a

generally improve. I do think there is a deeper question whether so you know like multimodel will improve this will improve like we keep finding these examples. though as the frontier will

examples. though as the frontier will move it will certainly move forward some things will smoon but the question is will it still be just other things that you don't need to you know teach the

human like every you know okay now you know how to use a spoon um and a fork but now if the fork has four instead of

three ends then you need to learn a new that would be a failure of machine learning you know I am fascinated by generalization I think that's the most important topic I always thought this was the key topic

in machine learning in general and in understanding intelligence. Pre-training

understanding intelligence. Pre-training is a little different, right? Because it

increases the data together with your increase in model size. So it doesn't necessarily increase generalization. It

just uses more knowledge. I do believe that reasoning actually increases generalization, but now we train it on such narrow domains that that it's may still be to see. But I think the big

question in all of AI is is reasoning enough to increase generalization or do you need like more general methods? I

think the first step is to make reasoning more general. As we talked before, I that that's my passion. That's

also what I work on. There is still something there, right? We we push the models. They they learn they learn

models. They they learn they learn things that are around what we teach them. they they still have limitations

them. they they still have limitations because they don't live in the physical world because they're not very good at multimodal because reasoning is very young and there's a lot of bugs in how

we do it yet. But once we fix that there there will be this big question is that enough or is there like something other big to to make models generalize better so we don't need to you know teach it

every particular thing in the training data that it just learns and generalizes. I think that's the most

generalizes. I think that's the most fascinating question. But I also think a

fascinating question. But I also think a good way to approach a question like that is to first solve everything that leads up to it. You know, you cannot know whether there is a wall or not

until you come close to it because otherwise there you know we AI is moving very fast. Someone said it's like

very fast. Someone said it's like driving fast in a fog. You never know how how far or close you are. So so we we're moving we are learning a lot. And

does that does that mean so so that that central uh question of basically learning with with very little data the way a child would and the fact that the child is able to do things that even the

most powerful model cannot do. So this

as you said to unpack this uh making progress on reasoning and showing how far we can get into generalization with reasoning and then the separate question is as you as you said whether we need an

entire different architecture and that's where we get into for example Yan Khan's work do you see uh promising fundamental

architectural changes outside of transformers that have caught your attention and and feel like they could um like a serious path to explore in the future. I

future. I >> I think there is a lot of beautiful work that that people are trying out. You

know, the arc challenges inspired one set of people. The their models now that are very small and solve them very well, but with methods that I'm not sure are actually general. We'll need to see. Yan

actually general. We'll need to see. Yan

Leon has been pushing for for other methods. So, I feel like his approach is

methods. So, I feel like his approach is is more towards the multimodal part. But

maybe the maybe if you solve multimodal, right? Maybe if you do jetpa it also

right? Maybe if you do jetpa it also helps your other understanding. There is

a lot of people pushing fundamental science still. It's maybe not so much in

science still. It's maybe not so much in the news as as as the you know the things that that push but whatever you do you know it will probably run on some

GPU. If you get a trillion dollar of new

GPU. If you get a trillion dollar of new GPUs, the the old GPUs will be much easier to get also for for so I think this growth in LLMAI on on the more

traditional side is also helping people to to have an easier time to run more experimental research projects on on various things.

So, so, so I think there is a lot of exploration, a lot of ideas. It's still

a little hard to implement them at a higher scale. the engineering part is

higher scale. the engineering part is the biggest bottleneck. I mean GPUs are a bottleneck too when you scale really up but implementing something that's larger than one machine it's an

experimental research project so you don't have a team to do that I think that's still harder than it should be but you know codecs may may get there or

coder this is the thing where AI researchers have great hope to help themselves and and also other researchers is that if you could just say hey codeex this is the idea and it's

fairly clear what I'm saying please just implement it so it runs fast on on this eight machine setup or or 100 machine setup. That would be amazing. It's not

setup. That would be amazing. It's not

quite capable to do of doing that yet.

But, you know, it's capable of doing this more and more. I think that's what what OpenAI says is they say, you know, we we say we'd like an AI intern by the end of next year. That's how I understand this. You know, can can

understand this. You know, can can someone help us? And is is is part of Codeex's um the path for Codeex to be able to do some of this uh does that

revolve around how long it can run?

Context behind the question being that uh again like two days ago as we record this you guys released GPT 5.1 C codeex max described as a frontier agent coding

model trained on real world software engineering tasks designed for longunning workflows and using compaction to operate across multiple context windows in millions of of of

tokens. So I'd be interested in

tokens. So I'd be interested in unpacking some of this. What does that mean to run for a very long time? Is

that an engineering problem or a model problem? And then maybe a word on

problem? And then maybe a word on compaction.

>> So it is an both an engineering and model problem. Um you know you want to

model problem. Um you know you want to do some engineering task like write a you have some machine learning idea. You

want Codex to implement it for you. Test

it on some simple thing. Find the bugs.

So it needs to run this thing. This is

not something you would do in an hour, right? That's it's something you'd spend

right? That's it's something you'd spend a week on. So the model needs to spend a considerable amount of time because it needs to run things, wait for the results, then fix them. The model is not like it's going to come up with the

correct code out from thin air, right?

It's just like us. It needs to go through the process and often times in the process since it was not trained on anything very long in its in its training or maybe very few but but

nothing certainly nothing that went on for a week, it can get lost. it it can start doing loops or doing something weird. That's of course not something

weird. That's of course not something you want. So, so, so we try to train in

you want. So, so, so we try to train in a way that makes it not happen, but but it does. So, so you know, how can you

it does. So, so you know, how can you make the model actually run a process that requires this larger feedback loop without tripping up? And

the other thing is transformers have this thing called context. So they they remember all the things that they have seen in in the current run and that can

just exceed the memory available for your run and and the attention matrices are are n byn where n is this length so so they can get huge. So instead of

keeping everything, you say, well, I'm going to just ask the model on the side to summarize the most important thing from the past, put it in context a new and forget some part of it. Right? So

it's a very basic form of forgetting the compaction. Right? And that allows you

compaction. Right? And that allows you to run for much longer if you do this repeatedly. But but again, you need to

repeatedly. But but again, you need to train the model to do that. And and when you do it, it works to some extent. I

don't think it works well enough to replace an AI researcher yet. It made a fair bit of progress. I I think another part of progress that that's a little understated on the research side but is

very important is allowing the model to connect to all of these things. So

models now use tools like web search and Python routinely but to run on a GPU to have access to a cluster. It's hard to train models with that because then you need to dedicate

for the model to use and and and that has security problems and this thing like how do models connect with the external world? It's a fundamentally

external world? It's a fundamentally very hard problem because you know when you connect in an unlimited way you can break things in the real world and we don't like models to break things for us. So that's a part where people work a

us. So that's a part where people work a lot. This it overlaps with security,

lot. This it overlaps with security, right? You need to like have very good

right? You need to like have very good security to allow models to go on and train on the things they need to train.

One theme that people like me, VCs and uh you know founders and startup think about a lot as we see all the progress at OpenAI is as the models keep getting

more general with more agent capabilities the ability to run for a very long time you know going to areas like science and uh math and you know

recently it was reported that um there were some investment bankers hired to help improve the model's capability to do grunt investment banking uh work all

of that taken into account uh is is there a world where basically models or maybe just one model does uh everything and I don't know if that's a GI let's not necessarily go into that debate uh

but what's left for uh people that build products that sit on top of models >> I just showed you a 5-year-old exercise that the model doesn't do I I I I think

we need to keep that in mind >> so you're saying there's hope There is hope the next model will do it right >> that hope okay >> well for for me yes I I still think we

have some way on the models progress has been rapid so so there is good hope there will be less and less of this but but on the other hand for now it's you

don't need you know to do a deep search to find things where you'd really want a human to to do that task because the model's not super good on the other hand I you know transformer paper started

with translation I recently went to a translation industry conference. Uh the

translation industry has grown considerably since then. It has not shrunk. There's more translations to be

shrunk. There's more translations to be done. Translators are paid more. The

done. Translators are paid more. The

question is why would you even want a translator if the model's so good in most of the cases. The answer is sometimes imagine you do a listing for a newspaper but in a language you don't

know and GPD5 will almost certainly translate it correctly for you. If it's

into Spanish or French or any high resource language, would you still publish it without having a human who speaks that language look at it? Would

you publish it if it's a, you know, UI of Chat GPT that a billion people are going to see? It's a question of trust probably, right? But if you have a

probably, right? But if you have a million users, a billion users, maybe you will pay the $50 for someone to just take a look over it before you you

translate it. So this is in an industry

translate it. So this is in an industry that fundamentally is totally automated, right? There is still the question of

right? There is still the question of trust and I think that's a question we will grapple with for a long time. There

are also just things you want a person to do like I I I don't think we will have no things to do but but that

doesn't mean that some things we do may not dramatically change >> uh and that you know that can be very painful for people who do these things and and so this is a serious topic that

happy people are engaging with. But I

don't think like there will be this global lack of anything for people to do.

>> And maybe as a last question to help us uh get a sense for what people at the frontier of AI are currently thinking about or working on, you know, some of

the topics that uh one may see are things like continual learning, world models, uh robotics, uh embedded intelligence. What do you

personally find very interesting in addition to what you mentioned a prompt multimodel uh but what do you personally find really interesting as a research area? Well, you know, I I always I find

area? Well, you know, I I always I find this general data reinforcement learning is is my my pet peeve and what I work on

luckily that for example, robotics is probably just an illustration that we are not doing that well in multimodel and that we're not doing that well in

general RL in general reasoning yet. The

moment we do really well in multimodel and we manage to generalize reasoning to to the physical domains that the robot needs, I think it will see amazing progress. When it does, I have a feeling

progress. When it does, I have a feeling given that you know a lot of companies are launching hardware that's kind of teleoperated or glove operated or or or something. So my suspicion is by the

something. So my suspicion is by the moment we make this progress which you know maybe maybe it will be next year maybe it will be in a few more years but

the hardware will may be there by then and having a robot in a home may be like a big visible change

more visible than you know chat I mean given how quickly we got used to the self-driving cars in San Francisco maybe it will be only visible for like the first two days and then be Yeah,

sure. The robots's there. It's always

sure. The robots's there. It's always

been cleaning since I can remember like the last three months.

It's It's stunning to me how quickly we get used to these things, right? The the

self-driving cars in San Francisco are something people got used to so quickly.

So, maybe this will happen for robots, too. Nevertheless, I do think it will be

too. Nevertheless, I do think it will be quite dramatic in in our perception of of the world when it happens. It

hardware is hard though, right? Robots

may have accidents in the house. you

need to be very careful. So, so maybe it will take longer to to deploy them and actually make it a scalable business.

We'll see. It is amazing that you know we are at this point where we can start thinking like yes maybe that will come soon.

>> Lucas, it's been absolutely wonderful.

Uh thank you so much for spending time with us today.

>> Thank you very much Matt. Thank you for the invitation. Great to talk to you.

the invitation. Great to talk to you.

>> Hi, it's Matt Turk again. Thanks for

listening to this episode of the Mad Podcast. If you enjoyed it, we'd be very

Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already, or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This

really helps us build a podcast and get great guests. Thanks, and see you at the

great guests. Thanks, and see you at the next episode.