OpenAI Is Ready For War - EP 46 Mark Chen

on the recruitment wars. I mean, this got a lot of attention clearly >> and it looked like Meta was quite aggressive. What exactly does this tit

aggressive. What exactly does this tit for tat look like? What st what stage are we at?

>> Yeah, I mean there is a pool of talent, right? And everyone kind of knows who

right? And everyone kind of knows who they are. And um you know I think many

they are. And um you know I think many companies have realized that one of the key ingredients not the only important ingredient but one of the key

ingredients to building a great AI lab is to get the best talent and um I think not a surprise that you know Meta has been aggressively employing this strategy. Um

strategy. Um >> you know we haven't sat back idly and I actually want to tell this story from OpenAI's point of view. Um I think that a lot has been made in the media of oh

you know there's this unidirectional flow of talent over to Meta. Um but the way that I've seen it is you know Meta they've gone after a lot of people quite unsuccessfully. So just to give you

unsuccessfully. So just to give you context right within my staff within my direct reports uh before they hired anyone from open I think they went after half of my direct reports and they all

declined. Um, and of course, you know,

declined. Um, and of course, you know, if they have something like $10 billion of capital per year to deploy towards talent, um, they're going to get

someone. So, I actually feel like we've

someone. So, I actually feel like we've been fairly good about protecting our top talent. And, you know, it it's been

top talent. And, you know, it it's been kind of interesting and fun to see it escalate over time. Um you know uh some interesting stories here are Zuck

actually went and yeah handd delivered soup to people that he was trying to recruit from us >> like a a just a just to show how far he would >> yeah I think he he handcooked the soup

and and you know it was it was shocking to me at the time but you know over time it I've kind of updated towards these things can be effective in their own way right and you know I've also delivered soup to people that we've been

recruiting from from Meta.

>> You're doing a soup soup counting.

>> I' I've thought of, you know, if I had an offsite, the next offsite for my staff, I'm going to take them to a cooking class. Okay.

cooking class. Okay.

>> And yeah, I mean, it's just been um Yeah, but I I do think, you know, um there's something I've learned about recruiting.

>> Did you cook your soup?

>> Uh it's better if you get like Michelin star soup. [laughter] You know what I

star soup. [laughter] You know what I mean?

>> Yeah. No, no, no. I think Dejo is very, very good and um probably better than any soup I could cook. Um but yeah, I I I do think there is something I've learned about, you know, just um how to

go u aggressively after top talent. And

I think uh the the thing I've been actually very inspired by is that um you know at OpenAI um even among people who

who have for meta, I haven't heard anyone say AGI is going to be developed at Meta First. Um, everyone is very confident in the research program at at

OpenAI and one thing that I've made very clear to to my staff to to the whole research ro is we don't counter uh dollar for dollar with with Meta um and

the multiples that below what Meta is offering that people are very happy to stay at OpenAI gives me so much conviction that you know people really believe in the upside and believe that we're going to do it

>> well and you and Alex Alex thing. He

used to be one of the math compet the the >> Yeah. Yeah.

>> Yeah. Yeah.

>> I'm sure you guys hung out.

>> Yeah. I mean, I I have hung out with Alex a handful of times, but we don't do much anymore. Yeah. I [laughter] mean,

much anymore. Yeah. I [laughter] mean, yeah.

>> Why did soup become the thing? It was

just it just >> I don't know. You know, it's there's been soup, there's been flowers, there's been anything you can think of under the sun, but um I don't know. I think, you know, life's an adventure. I I play into the meme.

>> Yeah. Yeah.

>> Is there any any poker strategy to employ like as you're you're thinking?

>> Well, again, I think it really goes back to what I've said about the media narrative. Um, the game is not to retain

narrative. Um, the game is not to retain every single person in the org. It's to

trust in this pipeline that we have for developing talent and to understand who the key people we [music] need to keep are and to keep those. And I think we've done a phenomenal job at that.

[music] >> [music] [music] >> We have a special treat today. I'm

excited. Mark Chen is here from OpenAI.

Um he's the chief research officer. He's

somebody I've gotten to know over the last couple of years. Thank you so much for coming. No, it's been great to know

for coming. No, it's been great to know you for for so long.

>> I feel like uh you know there's a handful of people in this world working on this very important project and and I mean you're

right at the top of it. So it's it's so cool to uh to have a chance to chat.

>> Yeah. Thanks for having me on.

>> It's a it's my pleasure and and I mean there's a bunch of things that I want to talk to you about because I've gotten to know you like we said over those last couple of years. I want to get I want I want people to know a bit more about

your biography and but I also know there's going to be AI enthusiasts who who want us to go deep on a couple things there. So we'll we'll try to do

things there. So we'll we'll try to do everything. Um, I wanted to start just

everything. Um, I wanted to start just by giving people a feel for your job, which in my head and you I mean just correct me if I get

any of this wrong, but you're you're you know Sam has been he he's really into research. He's the boss. He's kind of at

research. He's the boss. He's kind of at the top of the food chain. But then you and Yakob are working >> together to shape Open I Open AAI's research direction.

and and then you're in this additional part of this role is is is actually deciding which compute goes where onto these projects. So you kind of have to chart where OpenAI is heading

and then the mechanics Yeah. of how

you're going to get there. Yeah. And

this always strikes me as a horrible job because I picture people um doing everything in their power to get GPUs from YouTube.

>> It's true. people are very creative in the ways that they try to make backroom deals to to get the GPUs they need. But

yeah, I mean it is a big part of the job, right? Uh figuring out the

job, right? Uh figuring out the priorities for the research or um and also being accountable for execution. So

really to that first point um you know there's this exercise that Jakob and I do um every 1 to two months where we take stock of all the projects at OpenAI

and uh it's this big spreadsheet about 300 projects and we go and try to deeply understand each one as best as we can and um really rank them and I think for

a company of 500 people it's important for people to understand what the core priorities are and for those to be communicated clearly with explicitly verbally and also through the way that we allocate compute.

>> All right. What do we do at Core Memory?

We cover innovative, fastm moving, forwardthinking companies, which is why Core Memory is sponsored by Brex because

B is the intelligent finance platform for many of these companies. 30,000

companies from startups to the world's largest corporations rely on Brexit technology for their finances. They've got smart corporate

finances. They've got smart corporate cards, high yield business banking, and expense automation tools that are fantastic. I hate doing my expenses. And

fantastic. I hate doing my expenses. And

Brex's AIS software run right through those expenses. Figure out where we're

those expenses. Figure out where we're spending money and take care of so much stuff for you so you don't have to waste your time on it yourself. Go to

brex.com/cormemory

to learn more and just, you know, get with the program. Let's get going. Let's

get out of this archaic finance software and move toward the future core memory and bre. So you've got when you're

and bre. So you've got when you're talking about the 500, this these are the 500. This is the heart of the

the 500. This is the heart of the research team in an organization now that's thousands of people. Yeah. Okay.

So, and then in that when you're talking about this 300 projects, I I imagine I mean obviously some of those are the giant frontier models and then some are

probably experiments that people are working on. And so like how do you

working on. And so like how do you possibly keep track of all that and then come to some sort of conclusion about what merits GPUs and what doesn't.

>> Absolutely. So I think um it is very important when doing this exercise to keep the core road map um in focus and one thing that differentiates I think

open AAI uh with other other big labs out there is open AAI has always had core exploratory research at its core.

Um we are not in the business of replicating the results of other labs of kind of catching up to other labs in terms of benchmarks. That isn't really our bread and butter. We're always

trying to figure out what that next paradigm is and we're willing to invest the resources to make sure that you know we find that right and um I think most people might be surprised at this but

more compute goes into that endeavor of doing exploration than it is to training the actual artifact. It must be it's still got to be how do you stop yourself from being persuaded by someone because

everybody's going to put I you know like when I think about this sometimes I picture when I was at the New York Times you would have this page one >> meeting where >> everybody wants to be on page one.

>> Yeah.

>> Everybody thinks their story is the most important story. They're all doing their

important story. They're all doing their very best job to tell you why this thing is so important. Everybody in that room has worked >> weeks, months on on whatever they're

pitching. And so it feels like life and

pitching. And so it feels like life and death and that Yeah. I mean it just it's it seems so difficult for me.

>> Yeah. No, it it is also a difficult process and I think the hardest calls you have to make are you know this is a project that we just can't fund right now. Um but I also think that's good

now. Um but I also think that's good leadership. You need to clearly

leadership. You need to clearly communicate that hey these are the priorities. This is what we're going to

priorities. This is what we're going to talk about. These are the types of

talk about. These are the types of results that we think move the research program. And you know there can be other

program. And you know there can be other things but those have to be clearly number two.

>> And when you like you were talking about um not being reactive to your competitors. Yeah. When I was looking

competitors. Yeah. When I was looking through my notes, I I don't know if I could go to the line >> quick enough, but I mean, this this was like a point of pride that I saw that

you feel like um some of the other companies are well, you know, you guys were in this position where you were ahead um and and setting

the bar for others and so so they were reactive, right, to to what you had coming out. we happen to be doing this

coming out. we happen to be doing this interview a few days after Gemini 3 came out and you know there is a degree to which your rivals >> um >> at times like yeah I mean there's this

back and forth going on and and I know the benchmarks are sort of controversial how valuable they are but you [clears throat] know people go ahead on these things so how do you also um as

time has gone on maintain that luxury or that intellectual position where you feel like we're just going to do what we're going to Yeah, I I think AI research

today, the landscape is just much more competitive than it's ever been. Um, and

the important thing is to not get caught up in that competitive dynamic because you can always say, hey, you know, I'm going to ship an incremental update that puts me in front of my competitor for, you know, a couple weeks or a couple

months. And I don't think that's the

months. And I don't think that's the long-term sustainable way to do research because if you crack that next paradigm, that's just going to matter so much more, right? you're going to shape the

more, right? you're going to shape the evolution of it. You're going to understand kind of all the side research directions um around that sphere of ideas. And so when you think about kind

ideas. And so when you think about kind of um our RO program as an example of this, right, we bet more than 2 years ago that we're really going to crack RO on language models. And this was a very

unpopular bet at the time. Uh you know, right now it seems obvious, but back then the environment was, hey, you know, there's this pre-training machine that's working great. There's this

working great. There's this post-training machine that's working great. why invest in something else and

great. why invest in something else and I think today everyone would totally tell you you know thinking and language models it's just a primitive you can't

have uh can't live without and um so we're we're really there to make these bold bets and to figure out how we can scale and build the algorithms to really scale to orders of magnitude more

compute than we have today. It's just

it, you know, I mean, and I get that intellectually in my, you know, it gets harder as you guys started as a like basically a pure research company.

When you look at OpenAI today, I mean, you have product line. There's parts of OpenAI that look much more familiar to a >> mature Microsoft or a Google where you have product lines, you've got all these

different things that you have to serve.

Typically, I feel like you guys are still young enough, so maybe you don't have these exact pressures yet, but you know, as those companies go on, it always becomes, well, we're more focused

on the things that are serving the bottom line than spending a ton of money on research always seems to get like dwindled down >> over time.

>> Yeah. And I think that's really one of the most special things about OpenAI. At

its core, we're a pure AI research company. And I don't think you can say

company. And I don't think you can say that of many other companies out there.

And you know we were founded as a nonprofit and I joined during that era and I think the spirit is you know build AGI advance AGI research at at all costs

um and do it in a safe way of course um but yeah I actually do think that's the best head fake to really creating value right if you focus and you win at the

research the value is easy to create so um I think there's a trap of getting too lost into like oh you know um let's drive up the bottom line. Uh when in reality if you do the best research that

part of the picture is very easy.

>> And you you started in 2018 and so you feel like that soul that that that um >> yeah that core culture and that core nucleus it's it's really persisted.

>> It's still there. What does Elon says uh what does he he says we shouldn't call any of you guys researchers. It's just

engineering right?

>> Yeah. No, I think yeah, we No, it's it's true because I feel like once you have this hierarchy um and you elevate let's say research science um as a thing

beyond engineering, you've completely already lost the game cuz you know when you're building a big model uh so much is in the practice of

optimizing all of those you know little percentages of you know how do you make your kernels that much faster? um how do you make sure the numeric all work um and that's a deep engineering practice

and if you don't have that part of the picture you can't scale to to the number of GPUs we use today >> so because I think there well okay but there is like a mystique that surrounds

a researcher versus an engineer you know what I mean so are you were do you feel like um it is better to kind of stay levelheaded on that is that is that kind

of what you're saying or or >> on Well, I I just feel like researchers, they come in so many different shapes, you know? Uh some of our best

you know? Uh some of our best researchers, they're uh they're the type that, you know, they come up with a billion ideas, right? And many of them are not good. But, you [laughter] know,

just when you're about to be like, ah, is this person really worth it? They

come up with some, you know, phenomenal idea. Um some of them are just, you

idea. Um some of them are just, you know, so good at kind of executing on on the clear path ahead. And so there's just so many different shapes of researchers and I think it's hard to just lump it into one stereotypical type

that works.

>> That makes sense. Um,

>> okay, I won't I won't belabor you with too many competitive like rival questions. It's just since Gemini 3 did

questions. It's just since Gemini 3 did come out, I did wonder what happens with you personally or the team when one of your rivals puts it like does everybody

go and look and see what it can do? Is

there like a is there a prompt or a question that you th you often throw at these new models to see what they can do?

>> Yeah. Yeah. So, um to speak to Gemini 3 specifically, you know, it's a pretty good model. Um and I think one thing we

good model. Um and I think one thing we do is try to build consensus. You know,

um the benchmarks only tell you so much.

Um and just looking purely at the benchmarks, you know, we actually felt quite confident. um you know we have

quite confident. um you know we have models internally that uh perform at the level of Gemini 3 and we're pretty confident that we will release them soon and we can release successor models that

are even better. Um but yeah again kind of the benchmarks only tell you so much and I you know I I think everyone probes uh the models in their own way there

there is this math problem I like to give the models uh >> I I think so far none of them has quite cracked it even the thinking models um so yeah I'll wait for that

>> is this is this like a secret math problem >> oh no no um well if I announce it here maybe it gets trained on but um yeah I I do think uh it's one of the nice puzzles of last Here it's this um the 42

problem. So you want to create this

problem. So you want to create this random number generator mod 42 and you have access to a bunch of primitives which are random number generators modulo primes less than 42. You want to

make as few calls on expectation to these subg generators as possible. Um so

it's a very cute puzzle but um the language models they get pretty close to the optimal solution but I haven't seen one quite crack it. Okay, this is we're heading down a direction I want to ask

you about. But then just before we get

you about. But then just before we get there, so I know you're I've seen you you're very competitive. You've also

told me I think I found I love competition. I hate to lose

competition. I hate to lose somewhere.

>> I really hate losing. I [laughter] hate losing.

>> Yeah. So I'm picturing I'm just curious if this is at all right. I mean, if you know if we know Gemini 3 or whatever is coming out on a Thursday, I mean, are

you up at like midnight throwing that problem at it or is it is it not quite that drastic?

>> Um, no. I mean, I think it's in long arcs, right? Um, and any endeavor, like

arcs, right? Um, and any endeavor, like I, you know, I'm kind of a person who has obsessions. I I think any endeavor

has obsessions. I I think any endeavor you have to play the long game. Um, and

you know, we've actually been focusing on pre-training, specifically supercharging our pre-training efforts for the last half year. Um, and I think it's a result of some of those efforts,

uh, together with Yakub, focusing and building that muscle of pre-training at at OpenAI. Uh, you know, crafting a

at OpenAI. Uh, you know, crafting a really superstar team around it, making sure that all of the important areas and aspects of pre-training are emphasized.

Um, that's what creates the artifacts today that feels like we can go head-to-head with Gemini 3 easily on on pre-training.

>> Okay. And I I want to ask about the pre-training stuff because I' I've been talking to all you guys about this a lot, but but um Okay. But so but you're saying that um you're less obsessed

about lobbing um problems at these new models just when they appear and more at this this long journey.

>> Absolutely.

>> Yeah. Okay. Um okay. Hey, the reason I want to talk about sort the the puzzle that you were at, I mean, I >> you know, I first met Yakob before OpenAI ever started when he was doing a

coding competition and I got I got like super into coding competitions for a while. There's this guy

while. There's this guy >> Kennedy. I don't know if he's still

>> Kennedy. I don't know if he's still famous, but he was like the Michael Jordan of of these coding competitions.

And so I went to watch one at uh Facebook used to I don't know if they still do, but they had an annual >> Hacker Cup.

>> Yeah, Hacker Cup. And and that's where I saw YaKob for the first time. And then I know you I think did math competitions in high school. Yeah.

>> Like probably grade school through high school. And then you also did you also

school. And then you also did you also do III?

>> So I got into coding really late in life. Um it was a roommate in college

life. Um it was a roommate in college that convinced me to take my first coding class and um I had all the hubris of a mathematician at that time whereas like you know math is the purest and

hardest science and that's where you know you really prove your worth. I

mean, I think I was probably too into the competition back then. Um, but yeah, I mean, it became this super rewarding endeavor and um and you know, it it

started out as purely a way to keep in touch with my friends from college. Um,

>> you went to MIT.

>> Yeah, I went to MIT. Um, you know, I graduated and every weekend we would just log on and do these contests just to to keep in touch with each other. And

um, you know, over time I found myself having a talent for it. you know, I um started competing fairly well and then writing problems for for contests like the USA Coding Olympiad. Eventually

started coaching that team and yeah, it's been a great community where I've met people like Scott that you know.

>> Yeah. Yeah. Okay. So, you So, I think lots of people might be familiar with like math competitions because they probably see kids going through that.

The IOI and in these coding competitions are a little bit different. I mean, it's I mean, you'll know it so much better, but when I saw them, I mean, it looks like a it's almost like a word problem

that's a puzzle, and you're trying to kind of find the most efficient and correct way to solve that, and you're in this race against everybody >> and and everybody's like writing code on

their computer and then and then some people try to get there really fast, but then their thing kind of doesn't solve the problem, right? And then, you know, there's like this trade-off, right?

Absolutely. Right. And so so you you actually were on the MIT team.

>> No, no, it's something I did after college.

>> After college. Okay. But today you are like the coach of the US national.

>> Yeah. One of the coaches.

>> One of the coaches. Okay. And and

>> was it last year or the year before?

Like the US like we hadn't won one in a long long time, right? Yeah.

>> Yeah. Didn't we?

>> Yeah. Yeah. Yeah. So um Yeah. I mean the the team I mean it's you know you can never predict what the makeup of top talent looks like every year. Uh but we had a very spiky team I

year. Uh but we had a very spiky team I think two years ago. Okay. And um yeah I believe they won the Olympiad >> because I feel like usually it's like China or Russia or like uh

>> Barus and Poland. I mean right? Yeah.

And and so this compet the big competition [clears throat] >> takes place in a different country every year.

>> What does it look like? How many people show up?

>> Yeah. Yeah. So they they take the top four students from every single country.

Um it is as much of a competition as it is a social event, you know. Um this is a tight-knit community. They all do go on to do phenomenal things and um yeah,

it's this intense two-day contest where each day you get just three problems. Um 5 hours to solve them and you can really feel the adrenaline and uh all the pressure in the room. Um but it's also

great fun. I think um people settle down

great fun. I think um people settle down and they make you know lifetime friends through it. What do you And like as

through it. What do you And like as coach, I mean, you're so freaking busy, man. How what do you uh how much time do

man. How what do you uh how much time do you spend on this? What does that look like?

>> Honestly, um the kids are so self-motivated. Sometimes it's really

self-motivated. Sometimes it's really about just managing their performance and and strategy. Um I think, you know, you're going to have good days, you're going to have bad days, you're going to have good hours within the contest, bad

hours, and you can't let that get into your head. Um there's a lot of

your head. Um there's a lot of similarities between managing contestants and managing researchers. Uh

it's like on a much longer time scale, but you know like researchers have good months, bad months. You know, you can't really let those strings of failures get into your head because that's just the

nature of research, right? And um I think a lot of its morale management um at a certain point. Um

yeah, I think one other interesting thing that contests have helped me realize lately is when you put the models and deploy them towards solving these these contest problems which they're quite good at these days. Yeah,

I was going to ask you about that.

>> Um they they work in a very different way from humans. You know, we typically think of these machines as you know they're very good at pattern recognition. You can take any problem if

recognition. You can take any problem if it maps to a previous problem, it's probably going to be able to solve it.

But what I've noticed is in some of the previous IO, there's this problem like message is very ad hoc. Um, I didn't think the models would solve it at all, but actually one of the easier problems for the AI. So

>> yeah, I mean this has given me the sense that AI plus humans in frontier research, it's going to do something amazing just because the AI has a different intuition for what's easy and

what's not. So, okay. Is it vaguely, you

what's not. So, okay. Is it vaguely, you know, when when D mind did the whole Alph Go thing, >> you know, there was that moment where it was doing things human, it was playing

in ways humans hadn't played before. So,

kind of like vaguely similar to that or >> I I think so. I think so. Um I think really with GPD 5 Pro, right? Um there's

been an inflection point in frontier research. Um and one of the best anecdotes I have for this is you know I think 3 days after the launch um

I met up with a friend who was a physicist and um you know he had been playing around with the models uh felt like you know they they were cute but not super useful and I challenged him

with the pro model just try something ambitious and you know he put in his latest paper um it thought for 30 minutes and just got it and I would say

that that reaction in that moment Um it was kind of like seeing Lisa doll during that you know move 37 move 38. Um

and I just think that is just going to keep happening more and more for frontier mathematics for science for biology material science. Um the models have really gotten to that point.

I I was gonna ask you this question which is not very original because I think we've been doing this ever since kind of big blue and and all the chess stuff but yeah just as somebody who had followed all these competitions if

>> I don't know there's a sadness when you start seeing >> these models solving these things that were like the >> this height of these achievement for these very unique human minds.

>> Well um yes and no. I mean I was good at competitive programming. I was never at

competitive programming. I was never at the absolute top and um maybe this is a way to get revenge. No, [laughter] but I I I do think um no there's certainly a

moment for myself, right? Um you we tracked coding conscious performance while we were developing reasoning models for a while and you know at at the start of the program you know they

were not super great you know uh at the level of any average competitor going uh going going into the contest. And yeah,

over time they just started creeping up and up in in terms of capability and and you still remember that moment when you walk into the meeting and they have where your performance is and then the

models exceeded that.

>> Um, man, that was also a shock to me.

It's just like, wow, we've automated to this level of capability so fast.

>> And of course, you know, Yakob was there still a bit smug, but within like one or two months, it was also surpassing him.

So, >> um, yeah, no, the models are at the frontier today, right? Um it's so clear by even through the results we've done this summer um at coder right top

optimization competitive programmers in the world um I think it achieves second place there um and so really it's jumped from you know hundth place last year to

top five this year and >> like do you think we'll still be doing these competitions in 10 years?

>> I think so. I mean they're just fun. Um,

I mean certainly a bunch of people who use it to, you know, had their resume are going to drop drop off from doing it, but I think the people who've always excelled at it the most are people who just do it for the fun of it. And I

don't think that'll go away.

>> When I was doing this story, I mean, they were telling me that like if you're from >> Russia or I don't know which countries that you basically get like an automatic free ride to any university that you

want. I mean, I see the guys on the US

want. I mean, I see the guys on the US team go to like Harvard and MIT, so they seem to be doing okay, but it was it doesn't seem like the US has a [laughter] >> Yeah. I mean, don't you think it's going

>> Yeah. I mean, don't you think it's going to Yeah. I mean, interviews, right?

to Yeah. I mean, interviews, right?

They're going to be kind of broken going forward. And I everyone's seeing this a

forward. And I everyone's seeing this a little bit. And, you know, even college

little bit. And, you know, even college exams or college homework, it's it's all broken at this point, right? And I I do think we're going to need new ways of assessing and gauging, you know, who's

performing well, who's learned the >> where somebody's actually at.

>> Yeah. Yeah. So I mean I I've had this idea here where um maybe for our interviews we should just have

candidates talk to chat GPT and you know it's a special kind of chat GPT where uh the model is trying to gauge whether you know the material or or whether you're at the capability level to work at

OpenAI. Um, and you know, you have to

OpenAI. Um, and you know, you have to have this conversation with it that convinces it deeply you belong at OpenAI. And of course, you know, you

OpenAI. And of course, you know, you can't be allowed to jailbreak it, but and we look at the transcript after, but maybe like tests like this will more accurately reflect in the future whether you know.

>> So, you don't do that yet, but you're thinking about >> Yeah. Yeah. Just creative ways to revamp

>> Yeah. Yeah. Just creative ways to revamp the interviews.

>> Yeah. Yeah. Well, I mean, Silicon Valley is famous for doing the like >> brain teasers during the interviews and everything. Yeah. Um, so you

everything. Yeah. Um, so you I mean we tal you were very good at math [laughter] growing up and and I think you were you born on the east coast?

>> Uh yeah, born on the east coast >> and then you lived on the west coast too.

>> On the west coast and then you lived in Taiwan for like for like um grade school to high school.

>> Four years.

>> Okay. Your parents worked at Bell Labs.

>> Yep.

>> So you come from like engineering stock.

[laughter] Uh I mean it's a really interesting background um just because you kind of got like a flavor for all these innovation hubs and and especially with

your parents being at at Bell Labs and most of >> I mean yeah I just grew up in a very scientific environment you know dinner table talk was puzzles and things like that and um I also got kind of the more

traditional you know Bell Labs east coast experience um on the west coast my dad came to do a startup so a little bit of that kind of new company got got exposed to that when I was young as

well. And of course the big jump to

well. And of course the big jump to Taiwan, right? And I think it's a huge

Taiwan, right? And I think it's a huge culture shock. You you wear uniforms,

culture shock. You you wear uniforms, you're in a school, it has barbed wire around the school, right? And um and also getting exposed to kind of that level of rigor. Um I think it was just a number of really great experiences

growing. So, like the schools were

growing. So, like the schools were harder >> um or >> Well, I would say it was just much more kind of uh you know, it's just there's a little bit less flexibility and freedom

in the school system, but I think it also teaches you something.

>> Yeah. Okay. Since day one, the Core Memory podcast has been supported by the fine people at E1 Ventures. They are a young and ambitious VC firm in Silicon

Valley, investing in young and ambitious companies and people. Thank you so much to E1 Ventures for all your support.

>> Yeah.

>> And you knew you wanted to come back to the US for college.

>> Absolutely. Yeah.

>> Okay.

>> And then Okay. So, you're you're at MIT.

Um you're kind of like in this interesting group. I guess MIT probably

interesting group. I guess MIT probably always has a interesting >> Oh, man. Yeah. 2012 was such a great group.

>> Yeah. Like, who is there sort of like an allstar list?

>> Oh, I mean it was a great year. Like, um

I don't know if you knew like Jacob Steinhart, you know, he's doing Trans Loose now. Um he uh and I used to do

Loose now. Um he uh and I used to do projects together in in computer science class. Um there was Paul Cristiano who's

class. Um there was Paul Cristiano who's um uh a bunch of really >> phenomenal. He

>> phenomenal. He >> worked at Open Yeah. Um a bunch of kind of big names in in AI came from that year.

>> And then and then we were talking about the the competitive um coding like Scott Woo who's at Cognition. I mean, he's kind of like famous now as like a meme

on X for his math abilities, but and you just got you got to know him through the coding competition.

>> Oh, yeah. Through the coding community.

>> Okay. Okay. And then now I see the competitive end of you guys. The the output of this to me look like poker these days. I think I

I was we were at an event which I think I have to we have to keep secret or something like the the specifics on this event. But um I think I'm okay to talk

event. But um I think I'm okay to talk about this part which is like I >> late at night >> I'm walking by this table there's you

Scott I think Sham from Palunteer and then and like a handful of other people in a like a fairly int looked intense to be maybe it's it's not for you guys but

like a fairly intense poker game. So you

guys, this is where you've applied your your math and competitive skills now.

>> Yeah, I mean poker is a really fun game and you know I've talked about my life in terms of a series of obsessions.

Poker was definitely one of these obsessions in in the past and um I think the big revelation for me in poker was um you know it's so much more a

mathematical game than a game of reading people and bluffing. And I think the more you learn about poker, the more you update in that direction, right? Um, and

I think that's, you know, I used to be a terrible bluffer. And when you know it's

terrible bluffer. And when you know it's mathematically correct to bluff, then it's so easy, right? It's like you you don't feel any nervousness around it.

Um, and yeah, it's just so interesting that you have a game that I think is perceived as so human. Um, but the underlying mechanics and how to win are so deeply mathematical. And yeah, I kind

of thought about this the other day where you know there's something about that in language modeling too, right?

You have um this deeply human process of generating language, but there's this mathematical machine that can really do it as well as we can.

>> I think about that part all the time as a writer and then I >> did all this philosophy back in college with Vickenstein and all these guys thinking about these things. Yeah. Um,

well, how do you find like an edge? If

you and you and Scott both strike me as like supernaturally good at math, I don't understand how one of you is out calculating the other person.

>> Um, well, no, I mean, it it is mostly um a forum for us to just kind of um just hang out and catch up with each other.

Um, and you know, today we don't take it as ser I think there is an element to taking something like poker very seriously that takes the fun out of it.

and and so you know I my obsession with poker I think has ended more than a decade ago and now it's just fun.

>> You're just saying this cuz I saw Scott win both [laughter] days. I think

>> you might be right about that.

>> He was taking it he was taking it quite seriously. Um and and so like coming out

seriously. Um and and so like coming out of college I mean you had in some sense I was >> Oh I beat him on the plane though.

>> You beat him on the plane right home.

Okay. All right. All right. So you did you did was it just you verse him or it was like a group thing >> and maybe three or four people.

>> Okay. [laughter]

>> Um I feel like a lot of uh feel like there there's three I don't think I'm overgeneralizing too far especially among like if you cut back to the 2018 >> right

>> sort of time frame. Um, as far as like people who were in AI at a high level, I mean a lot had academic backgrounds. A

lot were math prodigies or had gone on to sort of >> take their math background and and get into robotics or something physics like that. And then and then there's this

that. And then and then there's this other bucket which is people who had gone to Wall Street and done >> um high frequency trading and and quants and and things like that. So that that

was the first path that you took was you went straight from MIT to to Wall Street.

>> Yeah. I mean I don't wear that badge with too much pride and [laughter] to be honest like you know it it was a a a path that was fairly common for you know

very quantitatively oriented kids at at MIT. Um, and I mean it was certainly a

MIT. Um, and I mean it was certainly a very meritocratic system, right? You

could apply intelligence and there's a very concrete reward function, the the amount of kind of profit that you would make. Um, but I think culturally it was

make. Um, but I think culturally it was hard for me. I it was a place where when you discover something, your first instinct is to just keep it away from as

many people as possible because your knowledge is what gives you your worth.

and and so it felt like you would have out of this an outgrowth of just even internally at a company like these competitive dynamics and people weren't

very trusting of each other. Um I think it was also felt like such a closed ecosystem, right? I think we today don't

ecosystem, right? I think we today don't feel too much like you know when when someone in HFT finds a breakthrough that makes their algorithm a little bit

faster, no one else feels it, right? And

I over time just kind of felt like you know I woke up after four or five years we're competing against the same exact set of players. Everyone was a little bit faster but had the world really changed that much for it? Um, and it

felt like time to do something else, right? It just a bunch of things lined

right? It just a bunch of things lined up back then. You know, there's the Alph Go um, uh, match which I think was a huge inspiration for a lot of people at at OpenAI. And

at OpenAI. And >> did you play Go? Uh,

>> I did not, but >> I think the sense in which, you know, the the model was able to do something creative, I really wanted to understand what was going on behind that. So you're watching

that happen and had you been had you been at all reading AI research papers and things like that or >> to be honest no and then I saw that event um it was really inspiring and

that's when I started doing my deep dives into AI. So I um one of my goals after seeing that was reproduce the DQN results. um this is a network that was

results. um this is a network that was able to uh play a lot of Atari games uh effectively at a superhuman level and um and going from there you know that's how I got my start in AI

>> and you were doing that like on the side as you so you just work all day and then go back and try to >> Yeah. Yeah. Yeah.

>> Yeah. Yeah. Yeah.

>> Yeah.

>> Okay. I mean it is weird. I remember I was interviewing George Hotz like >> it might have been roughly 2018. Um

maybe a little before that, you know, and he had just done this thing >> building a self-driving car on his own in his garage. And then, you know, I mean, it's George, so he says

>> large statements sometimes that may may or may not be like exact or spoton or or apply to other people. But he's like he's like AI is still so young. You can

you can basically learn the whole field if you read I don't know what the number was 10 20 30 research papers. I mean it is fascinating to me that it's like >> old in many ways stretching back decades

but this particular moment >> it's very shallow. Um I I always give this advice to people who are intimidated by getting into AI. so

shallow like just spend three to six months um picking some project like maybe you know reproduce DQN and you can get to the frontier very quickly. Um the

last couple years has added a little bit of depth but it's not anything like you know theoretical math or physics. Do you

think is this a field where I asked Jaob this the other day I don't know why I'm obsessed with this but you know like in mathematics you see you see

people tend to do their best work in their 20s or to have the big breakthrough and then it's very hard as they get older to like have that same >> kind of moment um like what you're

saying you know are we are we dependent on on young people reading these papers and then having some insight or is this is Is this something where it's more you can you keep going throughout your whole career?

>> I mean, I think you can keep going. I

mean, OpenAI itself does have a pretty young culture, but I don't think you need to be young to do good research. I

think there is something about being young and having less priors about this is the way it's done. Um, I think over time you may develop your own vision.

Um, which is a good thing, right? Um but

it also locks you into a frame of mind of like oh you know this is how research is done this is how good results come out and I think younger researchers tend to have a little bit more plasticity around that that concept.

>> Yeah.

>> Yeah.

>> Um so as your career is at open AI is funny because it seems like you walked in the door and had a very um important large position from the get-go. But when

you first got there in 2018, it must have been what, like uh 50 people.

>> Oh, no, it was much closer to 20.

>> Much closer to 20. Okay.

>> And it it really looked like two teams back then. I came in as a resident. So

back then. I came in as a resident. So

someone who, you know, clearly not a specialist, not a PhD. Um I think I was I was only a resident, you know, throughout uh his tenure at OpenAI. So I

was very lucky in that regard, just kind of learning the way that he thinks high level about research. Um,

>> and a like a resident in this case is you're just the right-hand person to >> Oh, so it's um someone who comes in usually from another field who openly I wanted to invest in and train up in AI.

Yeah. And so I think the first part of a residency is like a six-month compressed PhD um and and then going from there to to just you know getting into deeper and deeper research projects.

>> So you're kind of like talking to IA every day. is he kind of shaping that

every day. is he kind of shaping that that >> yeah he was responsible for for my projects for my curriculum for my learning and um and you know I would just go to him for you know hey like what's this about like why did people

pursue this >> okay yeah >> and you I mean I think if you go like on LinkedIn I mean it would say you were the head of Frontier Research as your first job at open AI

>> oh no I I was in IC for threeish years yeah so um I was doing independent research projects Um, I worked in generative modeling because that was

really kind of where I's focus was at the time. And then,

the time. And then, >> um, only after a while did I start managing teams. >> And because most you're talking about generative. I mean, most people point to

generative. I mean, most people point to Dolly as maybe the kind of first big project that the public would >> mostly re is that is that fair?

>> Yeah. Yeah. So, um, I think that also marked the transition between when I was an IC and and a manager. So uh one of my own kind of big projects and one I'm still pretty proud of today is uh image

GBT uh this proof of concept that even outside of language you could put things like images into a transformer um and the model would just internalize very

good representations and understand um you know the content of images and um it's kind of like a proof of concept that you can do language modeling outside of pure text and get really

really great representation. and scale

them to be state-of-the-art with other methods. Um, that was I consider like a

methods. Um, that was I consider like a precursor work to Dolly which I was on the opposite side of managing. Um and um I think between those uh another project

I'm really proud of uh uh doing doing IC work on is uh codeex where we >> you know set up a lot of the framework for evaluating um coding models and also

>> did a lot of in-depth study on how you can take language models and make them very good on code.

>> So and what made you pick open because I I could see it two ways in my head. One

is >> is big fish in a small pond. There's

interesting people here. I remember the open AI of 2018 with 20 people. In my

head, it was like, this is probably not going to work. Um, you know, [snorts] Google seems like they've got this locked up and and this is like a pretty small group of people trying to take on

something that appears to require many billions of dollars of of capital. And I

mean, this was even before the scaling stuff. It was just like Google had

stuff. It was just like Google had invested so much [laughter] in in AI already. Um kind of like in a different

already. Um kind of like in a different form than we think, but you know, you're already kind of doing translation on your phone and things like that. So um

was that like a hard decision for you or or you just stumbled into the OpenAI gig so quickly that >> Well, yeah, I mean I think there are two things, right? You need ambition of

things, right? You need ambition of vision. That was certainly what OpenAI

vision. That was certainly what OpenAI had at the time, but you also need the talent to back it up. And you know, I did feel like OpenAI was one of the rare places where the ambition was very

large, but the talent was also large enough to fulfill that that gap. And you

know, I I was lucky I knew people like Greg um before from from college. Yeah.

And >> Oh, so yeah, Greg was you over overlapped at MIT.

>> Um I think we did some math contests together. Okay. Yeah. Back in high

together. Okay. Yeah. Back in high school. [laughter]

school. [laughter] And um yeah, I I shot him a message actually and I was like, "Oh, you know, I I don't know if I have the right skill set, but this sounds like a place that's

doing great work." So,

>> it still seems nuts till I just come at this, you know, out of nowhere and and now you're like leading [laughter] research.

>> No, it's surreal to me, too. It's

surreal to me, too. Um, you know, even that transition from IC to manager, I was very hesitant about taking it. Um, I

didn't know if managing was a skill set that I would be good at and I was really enjoying IC work. I think uh um I was having a lot of fun doing it, excelling at it, building really great

collaborations. Um, but yeah, I mean

collaborations. Um, but yeah, I mean it's [snorts] really been a wild ride.

>> Yeah. Yeah. Well, okay, on that point, I mean, >> you've always struck me as a very nice like level-headed guy. I have to say,

you know, there's parts of OpenAI's history that are are quite >> dramatic, soap opera, like um a little Game of Thronesy, you know, power

struggles and and like to me >> um >> to be a manager in that I will say now like I feel like things are a bit calmer than they they were, but but when you

look backwards, it just seems like um I don't know, you you're saying you had to learn these skills feels. But

>> some of this feels like opposite. I

don't know you that well, but some of it feels opposite to your personality. Um

to have to like deal with all of that.

>> Honestly, you know, I've been lucky at opening eye. Um I genuinely say that and

opening eye. Um I genuinely say that and um in the sense that I've had managers that have really advocated for me. Um

you know, they they saw my talent and and advocated for me. I think when I was in IC, you know, W check, he was like, "Oh man, you should bet on him for for codeex." And um and then later on kind

codeex." And um and then later on kind of reporting to Bob, I I've never asked for promotion or up level. And you know, it's just organically happened. Um and

everyone kind of along the way has given me great advice. Um I think part of growing as in Manford is just getting the reps. I don't think there's any

the reps. I don't think there's any better place to get the reps than at at OpenAI. You know, there's always

OpenAI. You know, there's always challenges to solve. Um, and yeah, I I think kind of developing that confidence. I actually think management

confidence. I actually think management is something where it's really just about the experience and um, you know, there's I would say less so talent involved in it.

>> Yeah. I don't want to like embarrass you and I don't know if this will or won't and I assume you probably don't want to get too much into the the coup or the blip or whatever. We want to talk about anything. [laughter] Yeah. Yeah. Well, I

anything. [laughter] Yeah. Yeah. Well, I

just I've I've interviewed so many people about this now. And I'm also gonna save my some of my gems for my book. So,

book. So, >> I won't uh I won't up myself. But there

there's a couple moments in there where you [clears throat] >> um you know you help >> get the researchers aligned around that the petition to like bring Sam back and

then and then I think just that either a day or two after that there's there's kind of like this speech you know that given I think at Greg's house maybe or >> I think Chelsea's house

>> okay and and um you know both of those struck me as pretty profound moments in especially for um I guess like standing

up with for what you believe in and rallying the troops. I mean, yeah, like uh you know, in a in a moment of crisis,

I don't know. So, did do those um Yeah.

I mean, that did feel like a very pivotal moment for me. I think in the days following the blip, right? There

was a lot of uncertainty. Um and you know, myself, Nick, uh Barrett at the time, we felt this responsibility of, you know, the wolves are at the heels, right? everyone's getting calls from

right? everyone's getting calls from from all these competing labs being like you should come work here instead and I just set this goal of I will not lose a

single person um and we didn't um and it was just you every day um opening up our houses you know people could come here they could you know have a place where

they let out their anxiety um and then also just helping them keep in touch with the leadership team um having a way for them to feel like they could make a difference and I think you know over

time people really felt this spirit of, hey, we're all in this together. Um, how

do we make a difference? How do we signal to the world that we're all together? And, um, you know, we I've

together? And, um, you know, we I've been kind of driving back and forth uh between a couple houses and, uh, we had this idea of like, hey, you know, we need to show the world that we're all

seriously aligned and we're going to work for Sam. And, um, that's when the petition came together. And I, you know, the idea, I think, got solidified at

2:00 a.m. We got more than 90% of the

2:00 a.m. We got more than 90% of the whole research or signed, I think, by the morning. Um, and it was just

the morning. Um, and it was just everyone like calling their friends being like, "Hey, are you in or are you are you not?" And yeah, I think in the end, you know, it was very close to 100

people signing that petition. is well I mean that must have put you in something of a tough spot though just because especially at the outset it was kind of like Ilia and Sam were on opposite sides and Ilia is your mentor and then I know

Ilia kind of >> comes back um >> um yeah I don't know was that awkward >> um no no it was hard I mean it's it's a low information environment um but

fundamentally it's just uh and and yeah I mean I think at the moment you know you could very reasonably conclude like was like did Sam do anything here, you know, is there

um but would Greg and Yakob like people of super high integrity quit over that?

Um I just felt like, you know, there was some part of the story that that was being misrepresented here.

>> Yeah.

>> Yeah.

>> With the, you know, Yakob's been there for a very long time. Um like what should people

long time. Um like what should people know about Yakob that they don't?

>> It's interesting cuz he's super funny guy.

>> He's hilarious. Oh my gosh. He has this like sarcastic humor. Um, and yeah, it cracks me up so much honestly. Like, um,

yeah, that's one of my favorite things about OpenAI today. Just like the level of alignment I have with Yakob. Um, I

feel like we go into a meeting, um, we can just bounce off ideas and quickly get to alignment and then, you know, deliver the same message and kind of like operate on different parts of a big

road map together. Um, and yeah, it's just one of the big privileges I have working at OpenAI. Yeah, I mean go going to that actually that point about um you

know keeping people together like I still feel that way about open eye research. Um I think we're

research. Um I think we're >> still under attack. [laughter]

>> Yeah. No, I we are a family. Yeah. We're

always under attack. Look, when any and this is how I know we're in the lead, right? Any company starts, where do they

right? Any company starts, where do they try to recruit from? It's OpenAI. Um and

you know, they want the expertise. They

want our our vision kind of our our philosophy of the world. and we've made so many star researchers, right? Um I

think opening I more than anywhere else has been a place that makes names in in AI today. Um and I still feel that same

AI today. Um and I still feel that same level of protectiveness like you come after I'm going to do anything in my power to make sure you know they're

happy they're open and they understand you know how their role fits into the road map. I yeah I this is something I

road map. I yeah I this is something I battled with as I was doing the book or even just watching events unfold in real time is like when I go back through the history I mean you've got

>> you've got Ilia in 2012 making sort of like a big breakthrough and then you know you've got no McGoo in 2017 doing Transformers and then you've got Alec

Radford you know like sometimes the story is these individuals >> really pushing the field forward and it feels like a field that's still so young

that you can have have this individual and then and it seems like there's this group of I don't know what the number is, but let's call it like 8 to 10 who seem to have

>> an ability to do that repeatedly and they're really shaping where this is all gone. And so when I started seeing like

gone. And so when I started seeing like John Schulman leave or Alec leave and then you know there's kind of was like wow okay well if you've lost a chunk of

this all-star team how do you it seems like a kind of field where you you sort of can't just replace that and yet you know it was it was kind of like after that that you guys

>> pushed forward on reasoning and and some of these other spots. Yeah. So I don't know I I've intellectually had trouble >> I I do disagree with that as the the overarching way to do good research

today. Um I think there's certainly a

today. Um I think there's certainly a lot of top down steer you know we bet on directions but people it it open has this beautiful culture of being bottom

up in a very deep way too where some of the best ideas just organically emerge from sometimes the most surprising of places and um I think really the the

great thing has been just like watching some of these bets unfold take shape get scaled um and reasoning being a a core example of that. Yeah. And okay, so in

this but like this idea that um like how star dependent are we because you still see Google spend an ungodly amount of money to bring Gnome back, you know what I mean? Yeah. And so this makes me

I mean? Yeah. And so this makes me think, okay, this is how this works.

>> Yeah. I mean, I think it's a mix, right?

Like you have to you have to invest in your pipeline because I'm very confident in our ability to create stars. But

yeah, there's certainly very good people out there and everyone knows that they're good. Um I think if there's one

they're good. Um I think if there's one thing that you know on on the flip side I've learned from from meta is you know open can also go very aggressively after star talent and um you know there's this

very aggressive recruiting approach that you know I've taken a couple pages from as well. Um [laughter] but yeah I I I

as well. Um [laughter] but yeah I I I think it's we should always just be trying to assemble the best team.

>> Yeah.

>> Uh in service of the mission that we want to accomplish. It's funny because it is like a relatively small world and like all you guys hang out even though you you're like rivals and then

>> it must be weird cuz I know you're friends with different people on some level and then and then you're also trying to steal all their >> I mean yeah it's it's a brutally competitive industry in all fronts,

right? Um but again that's what I love.

right? Um but again that's what I love.

Um I'm a deeply competitive person. I

hate to lose and yeah on research on recruiting all of these fronts um I'll work very hard on them.

>> It reminds me because I'm like a semiconductor well I'm a history nerd but just >> the early semiconductor days were not that far off. I mean you had you had all these semiconductor startups come at

once. They were all pushing the limits

once. They were all pushing the limits of physics and somebody would discover something at one.

>> They'd go to the bar and have a it's like people they're engineers. they

can't like stop from like sort of >> sharing knowledge with each other but and then they're also getting >> pulled like you know is hard each company is is kind of quickly getting

this breakthrough in one way or another.

>> Yeah. I mean you raised an interesting point of you know there is going to be some base rate diffusion of ideas and um I think there's two ways a company can respond to that. You can create these

deep silos of like hey you know we're going to protect information in all these ways. I don't think OpenAI

these ways. I don't think OpenAI operates that way and we don't think that's the right way to operate. Um, we

just will outrun other people as fast as we can and um I love the culture of openness. People in research freely

openness. People in research freely share ideas and I think that's the way to make the fastest progress.

>> And how like how do you Sam and Yakob now work together? I I think people sometimes um if you read the the announcements and everything, you can

tell that Sam is researchoriented over over like >> day-to-day running of the company, you know what I mean? You can tell research is more of his his passion and and even

just like in the the titles and in the way it's been organized, especially recently. M

recently. M >> um you and Yakob are so deep on this stuff and I know Sam is is technical but >> you guys are are like in it all the time

and then you know Sam >> is having conversations with everyone.

Yeah. I'm just curious about this dynamic between the three of you and how I mean are you guys I mean I guess you're not always alignment on in alignment on what is going to get the

resources but um yeah I was I was just curious about you guys dynamic.

>> Yeah. Yeah. So, I mean, um, it's a very tight cohort. You know, I talk to Sam

tight cohort. You know, I talk to Sam and Yakob every day. Um, and you know, with Sam, he loves research. He loves

just learning about research. Um, he

loves talking to researchers. I think in some ways he's very effective at getting a pulse on the research or I rely on him also to just, you know, are there any hidden latent problems here? Um, go and find them out, you know, surface them to

me. Jakob and I

me. Jakob and I >> personality or techno. It could be just small things like oh you know um like just even the way the office is laid out like makes it harder for this team and

this team to collaborate um and the two of them is like need to collaborate to to help unlock this breakthrough that we want. Um I mean all these things are you

want. Um I mean all these things are you know very very important and I think Jakob and I we spend a lot of time figuring out how to design the work for success. You know I think um pairing

success. You know I think um pairing people with the right strengths together. Um you know also how to like

together. Um you know also how to like incentivize people to work on directions that we find are important. Um yeah that that's a lot of the work that we do.

>> The and Sam what he um like is he reading papers? Is he

chatting with you guys? Is

>> Yeah. Yeah. I mean, I think he he does his fair share of reading papers. He

talks to researchers and just understands how how they think about the world, the type of research that they're doing. Um, and of course, he's

doing. Um, and of course, he's responsible for a huge umbrella of things outside of that.

>> All right, I'm going to ask some nerdy questions now, but I'm going to try to um I don't know if I can dwarf cash level, but [laughter] I'm going to do my best. And you know, I'll ask. I don't

best. And you know, I'll ask. I don't

know how top secret some of this stuff is, but but um anyway, well, maybe you'll slip up and we'll just we'll get it out. Um you know, in the the meetings

it out. Um you know, in the the meetings I have been on, and I don't think I'm revealing because we talked about a bit.

I think I'm safe here, but you know, pre pre-training seems like this area where it feel it seems that >> um my sense is you you guys feel like you've figured something out. you're

excited about. You think this is really going to be like a major advance. It was

also, I think, >> either a neglected spot or something of a sore spot. You know, previously things weren't maybe working [clears throat] exactly >> how you guys had expected or hoped. Um,

like what can you tell us about what you figured out and and you know some sort of frame of reference on on we've seen these periodic big leaps

forward.

>> Absolutely. So I think the way I would describe at a high level um the last two years is you know we've put so much resourcing into into reasoning into

understanding this primitive and making it work and it really has worked and I do think one byproduct of that is you lose a little bit of muscle on uh your other functions like pre-training and

post- training. Um in the last six

post- training. Um in the last six months Jakob and I have done a lot of work to build that muscle back up. Um I

think pre-training is really a muscle that you you exercise. You need to make sure you know all the info is fresh. You

need to make sure uh people are working on optimization at the frontier are working on numericics at the frontier.

And um I think you also have to make sure the mind share is there. Um that's

kind of one one of the recent things I've been focusing a lot on just kind of directing and shaping what people talk about at the company and very much today that that is pre-training. Um, we think

there's a lot of room in pre-training.

You know, a lot of people say scaling is dead. Um, we don't think so at all. Um,

dead. Um, we don't think so at all. Um,

in some sense, um, you know, all the focus on RL, um, I I think, um, it's a little bit of an alpha for us because we think there's so much room left in in in pre-training and and I think as a result

of these efforts, you know, we've been training much stronger models and that also gives us a lot of confidence carrying into, you know, Gemini 3 and other releases coming this end of the year.

Like the way I picture it in my head sometimes is that you guys have been on this you've just been running so fast. The whole field has been running so fast. And so we're

at a moment where it's like, okay, we've >> gathered up this vast volume of information from the internet. We've

we've thrown it onto this supercomput and and that, you know, chat GPD pops out and then we're just on this this like this incredible race that's going

on. And so like when I hear you guys I'm

on. And so like when I hear you guys I'm just trying to think about this in like a to level set maybe for people who don't follow this as closely. Um

so you you know in that initial moment you just had so much data you're throwing it at this machine you try to shape that data a bit initially and what

now we're just learning like more efficient ways to shape that just it's not always clear on what the mistakes were.

Um, so I do think um, yeah, you touch on something I've been thinking about a lot, right? Um, when you think about

lot, right? Um, when you think about pre-training, right, you're taking human written data and you're teaching the model how to essentially emulate it, right? Um, it understands human patterns

right? Um, it understands human patterns of writing. Um, and in some sense that

of writing. Um, and in some sense that also bottlenecks um, and and puts a ceiling on the capability that you're able to achieve, right? You you can't really surpass what

right? You you can't really surpass what humans have written when you're imitating what humans have written. And

so, you know, you you work on things like RL um there, you know, you can really provide steer towards the hardest

tasks that humans uh can come up with and um have the model basically think outside of the box, outside of um what it's learned from from imitating humans

and achieve higher levels of capability.

But there is this kind of interesting problem now of how do you go beyond um what humans are able to do today? And I

do find a serious measurement problem there too. Um even in the sense of like

there too. Um even in the sense of like can humans judge superhuman performance in in the sciences, right? Um would how how would we know that like this

superhuman mathematician is better than that superhuman mathematician? And uh we really do need to kind of come up with better evaluations for um what it means to make progress in this world. Right?

We've been lucky up to this point, right? There have been contests like the

right? There have been contests like the IMO IOI really just like gauging who's the top one mathematician in the world right um but when the model capabilities

go beyond humans there are no more tests >> right okay you just made me think of a question going back to the IOI stuff I mean >> and sorry we're going to come back I just you just totally popped in my head

I mean like often I would see the >> kids who were amazing at those competitions they would get hired somewhere like a Google or Facebook or something, but they weren't always like the,

you know, the top executive or the top the the most famous engineer afterwards.

And maybe it was like by choice, but I don't think Gennady was like the Michael Jordan ended up working at any of these companies. And that totally could be by

companies. And that totally could be by choice. I'm not trying to disparrage

choice. I'm not trying to disparrage him, but but it's not clear to me um >> like even Okay, so it's not clear to me that the human who excels at that is is

necessarily like the greatest engineer you're ever going to have. And so like yeah, may I mean if an AI is particularly good like what are we learning?

>> Yeah, that's a thing I quite like about working in AI. I think more so than uh in in standard engineering culture, it is a meritocracy. Um in that, you know,

I've tried this many times before and learned this lesson many times before, but it is hard to put in someone to lead a group who doesn't have the respect of

the researchers that they're leading. Um

and I think this is more so the case in research than than anywhere else. You

have to make very strong technical calls of like, you know, this is the right right path when there's a disagreement.

this is the right kind of project. Um,

and if you make those calls wrong, you know, you lose the respect of of your researchers. So, um, yeah, one of the

researchers. So, um, yeah, one of the fun things in working with, uh, in AI and and creating a a strong AI or is, you know, all my bentures very deeply technical and it's fun to talk to them

about the technical things.

>> Yeah. Yeah. Okay. And then Okay. On this

on I'm pre-training again for a second.

Okay. You know, like to me in my head it feels like Transformers helped kick off this massive, massive

leap. I mean, reasoning to me feels

leap. I mean, reasoning to me feels very comparable, if not even >> sort of more amazing. I mean, are we

more amazing. I mean, are we when I talked to you guys over the last few months, my, you know, and I can never tell if this is optimism, >> if you guys are just putting the best foot forward when I'm chatting to

everyone. My my sense when I talk to

everyone. My my sense when I talk to you, to Greg, to Yakob, you know, to Sam is is that you guys kind of feel like you've been putting in hard engineering work for like three, four, five years

that hasn't fully >> manifested itself. Um, and so then I can never tell how excited or not to be. I

mean like when you guys are hinting at some of the stuff you're seeing do you feel like it is you can already tell that it is a comparable leap forward >> in terms

>> to these big epocle kind of things?

>> I think so. You know um I I think when we launched GPD5 you know we >> we talked a lot about synthetic data as well. Um you know there are many other

well. Um you know there are many other threads of this form that we think are holding quite a bit of promise and that we're scaling up pretty aggressively right now. And I think it's always about

right now. And I think it's always about maintaining that portfolio of bets. Uh

taking the ones that are providing more empirical promise and and scaling and supporting them at at an even greater degree.

>> But it was like two weeks ago Andre Karpathy who used to work at open, you know, he went on Dark Casual's podcast and seemed to like deflate >> some giant portion of the AI industry by

saying, you know, I think he was saying what that AGI was like 10 years 10 years off. And then when I hear and then I I

off. And then when I hear and then I I heard Daario talking about a week ago. I

mean he seemed to be holding on very much to like massive scientific dis his his um what is he called the the nation of geniuses. He seemed to be holding

of geniuses. He seemed to be holding still on kind of like maybe a little slower but like a two-year timeline on that. Um you know

that. Um you know >> yeah when you heard what Andre said what did you >> Yeah. I mean I think Twitter they love

>> Yeah. I mean I think Twitter they love this like cycle of you know it's so over. so back and you know whatever

over. so back and you know whatever plays into the narrative at the time I think you know just becomes amplified.

Yeah, I'm trying to make a clip here, but you know, I the way I think about it, yeah, I mean, it's like AGI, I mean, everyone defines their own point for AGI, I I think even at at OpenAI, um,

you can't get everyone in the same room and be like, hey, this is my clear definition of AGI and it's it's it's consistent. And so, I kind of think

consistent. And so, I kind of think about it as something like, you know, you're in the industrial revolution, right? Do you consider

right? Do you consider the, you know, having machines make textiles, is that the industrial revolution or is it the steam engine?

you know, everyone kind of has their different definition and um I think we're in the middle of this process of producing AGI. For me, I think the thing

producing AGI. For me, I think the thing I index most on is are we producing novel scientific knowledge and are we advancing the scientific frontier and I

feel since the summer there's been a tremendous phase shift on that front.

>> Okay. like from stuff that you're seeing in ter the the first things that are jumping to my head are all these >> startups that are in the biotech space that are showing you know oneshot

antibodies and and molecules but I have no idea if that like what are you >> yeah yeah so I mean I was so inspired by that encounter with the physicists that

you know went back and thought hey well we should just create open AI for science and the goal being I think for the small set of scient scientists today who realize the potential of these

models and feel like they want to lean in and accelerate, we should do the best that we can to accelerate them. And you

know I know there are similar efforts that you know other companies um aim towards pushing the scientific frontier but I think what we want to do and um I

would say a little bit of a framing in terms of how we differ from let's say Google's uh efforts to to to work on science is we want to allow everyone

uh the ability to you know win the Nobel Prize for themsel. Um, it's less so about us winning that at at OpenAI, which would be nice, but we want to build the tooling and the framework so

that all scientists out there feel that accelerative impact and we think we can push the field collectively.

>> Well, and when the discoveries that you're saying you're excited about? I

mean, are there any others like specifically that that you've >> Yeah. Yeah. So, um, I think there's, you

>> Yeah. Yeah. So, um, I think there's, you know, if you want a a huge list of of these, um, you can go on Seb's Twitter account. Um so recently you know there's

account. Um so recently you know there's a GPD5 paper on an open convex optimization problem that you know uh is actually whose Twitter account?

>> Uh Sebastian. Okay. Yeah. Yeah. And you

know it's like um very related to some of the core ML problems that we're solving. Um I know there was um

solving. Um I know there was um >> I think people kind of dismiss these things as oh is it just fancy literature search or something like that? Um it's

quite a bit more complicated than that.

And you know, there's some examples I could go into, but >> I might I'm honestly overwhelmed at the moment cuz, you know, I'm sort of a generalist, but I cover biotech a lot.

And it's like >> every two days, man, I'm walking in and it's wow, we we're making an AI scientist. We we one-shotted enhanced

scientist. We we one-shotted enhanced body and and then so like part of me gets excited and you know at least a handful of these companies I know the people and they're real scientists and

like but then there's so much of it that I'm like either >> something amazing is happening or every it's it's like it's it's kind of too much for me to be able to discern where reality is.

>> Yeah. I mean I wouldn't be surprised if it's happening in biology. Um personally

I have the most expertise in you know computer science and mathematics and you know we we do have the experts there that can confirm that these are discoveries being made. So that's the thing that gives me the most confidence

but I'm not surprised at all it's happening in biology.

>> But like what you're saying is is kind of different than the I gr the narrative changes every like three weeks it seems like. But like what you're saying is is

like. But like what you're saying is is sort of different because the biggest knock even before Andre said that it seemed to me from the you know what I

was listening to um I was listening to like a politics podcast um sagger I think it's breaking points is their podcast you know he's he's >> pretty smart

>> guy who's knowledgeable but I mean he's just been on AI and the lack of progress and this is all like make believe and and all in and So, you know, if these discoveries

aren't happening, I mean, I feel like the public is aware of this.

>> Just to be clear, you know, um, while setting up Open AI for science, we've talked to a lot of physicists, a lot of mathematicians, and actually most of the people we've talked to aren't

that bullish on AI. I think they still believe, hey, you know, um, this thing isn't something that can solve new theorems. There's no way it could do that. You know, there must be something

that. You know, there must be something else going on. And that's why I feel like empowering the set of people who really do believe and lean into it like those people are going to just

>> you know outrun everyone else and we want to build the tools and convince people like this is the right way to do scientific research.

>> Okay. And so I mean so like on that point I mean I grant you that everybody's definition of AGI is different but you're >> like at least what I'm hearing is I mean [clears throat] you whatever you want to

call it um you feel like in the next year or two is we're just seeing dramatic things happen.

>> Yeah. I mean it is a bit of a meme, right? It's like you ask someone when is

right? It's like you ask someone when is AGI? It's two years away, right? Um

AGI? It's two years away, right? Um

>> and I don't think we're in that world anymore. And it it's like these results

anymore. And it it's like these results in math and science that that are giving me this conviction. But at at OpenI within the research, we set two very concrete goals, right? Within a year, we

want to change the nature of the way that we're doing research. And we want to be productively um relying on AI interns in in the

research development process. And within

2 and 1/2 years, we want AI to be doing end-to-end research. And I think it's

end-to-end research. And I think it's very different right like today you know you come up with an idea you execute on it you implement it you debug it um it means within a year we're quite confident we can get to a world where we

control the outer loop we come up with the ideas but the model is in charge of the implementation the debugging yeah

>> okay are there I beyond pre-training when I talk to you guys um sometimes I get the sense similar sort the thing it's like we all have in our

heads at least people where I sit that there's been this massive infrastructure build out that um the models seem to get better every time you 10x them that um

you know there was a there was a story >> for a while that as you guys were going from like four to five you weren't seeing the results you wanted even though

>> um you were getting more compute but then the more I talked to you guys >> the more it sounds to me like you feel we haven't actually that things were

moving so fast back then that we haven't actually seen um the moment where we made the leap to the 10x computer. I

don't know if I asked that question very eloquently but >> yeah I mean I I do have a thought to share here which is you know when people ask me um like do you guys really need all this compute um it's such a shocking

question because you know dayto-day I'm dealing with so many compute requests and you know the really my frame of mind is you know if we had 3x the compute today I could immediately

utilize that very effectively if we had 10x the compute today um probably within a small number of weeks fully utilize that productively And so I think the demand for compute is

really there. I don't I don't see any

really there. I don't I don't see any slowdown. And yeah, I it almost baffles

slowdown. And yeah, I it almost baffles me when I hear people ask like, "Oh, do you guys really need more compute?"

Yeah. Doesn't doesn't make sense to me.

>> And you think we in the broad strokes of the question I asked badly. [laughter]

asked badly. [laughter] Um, do like along the lines of where you guys seem very optimistic about what you've cracked on pre-training, are you equally not just like this demand that

people want more GPUs, but are you are >> do you see pretty clearly that that same thing scaling is about to kick things higher?

>> Yeah, we we absolutely want to keep scaling the models and I think we have algorithmic breakthroughs that enable us to scale the models and um, you know, I think there's a lot impressive about

Gemini 3. Um, one thing that kind of

Gemini 3. Um, one thing that kind of reading into the details that I've noticed is, you know, when you look at stuff like their SWE bench numbers, there's still a big thing around data efficiency that they haven't cracked,

right? They haven't made that much

right? They haven't made that much movement on it. And I think we have very strong algorithms there.

>> Yeah. Well, and there was this leaked memo from I mean, Sam was sounding quite somber about Gemini 3, man. In this

memo, I'm trying to find the quote. Did

you Well, you obvious I'm sure you got the memo. Um

the memo. Um it like it it it seemed like a bit of a moment. Um yeah.

moment. Um yeah.

>> Well, I do think part of Sam's job is to inject urgency and pace and and that's also part of my job as well. Um I think it is important for us to be laser

focused on on scaling and I do think you know Gemini 3 is exactly like the right kind of bet that that Google should be pursuing. Um you

know at the same time you know I would calibrate that by saying you know a large part of our jobs is to inject as much urgency into the org as possible.

Yeah.

>> And um it is a good model. Um, I think we have a response. Um, and I think we can execute even faster to the the follow-up.

>> How much do you um get involved with things like and I'm sure you're going to tell me exactly what it looks like with Johnny Ives device.

>> Cool. Cool. Yeah.

>> Yeah. Like is that is that an area that um research plays in?

>> Yeah. Yeah, it is. And actually um I was just having dinner yesterday.

>> You can describe it to me if you want.

>> Yeah, [laughter] absolutely.

So it looks like this. Well, um

yesterday I was uh [laughter] >> yeah just having uh dinner with Johnny uh with with some researchers as well um our head of pre-training and also post-

trainining and um really the way I think about chat GPT in the future right um today when you look at how you interact with chat GPT um it feels very dumb to

me it doesn't feel very thinking native right and you go to it with a prompt right you get a response and then it's doing no productive work for you until you give it the next prompt. And if you

give it a similar prompt, you know, it's going to think for the same amount of time, it hasn't gotten smarter because you added asked the first prompt. And

you know, I think the the future is going to be a world where, you know, memory is going to be a a much kind of improved feature. Every time you go go

improved feature. Every time you go go to chapd, it learns something deep about you. It reflects about why you would ask

you. It reflects about why you would ask this question, related questions, you know, anything. Um, and then the next

know, anything. Um, and then the next time you go to it, it's going to be that much smarter. And I think it really begs

much smarter. And I think it really begs the question of how do you design a device

that has this as the dominating thesis.

Um, and yeah, I I thought that's been a very productive experience.

>> Do you have one?

>> Do I have one? Um,

>> I may or may not have one. [laughter]

Um, what I think about when I think about you guys talking to Johnny is that like at Apple, you had this company that was centered

around hardware. It's something that

around hardware. It's something that Steve Jobs obsessed about all the time.

>> It's like, >> you know, it's a craft. It's like an art form. Um

form. Um whether it's you, Sam, Greg, Yakob, whomever, as far as I'm aware, none of you guys have really done a hardware product before.

>> Um >> Sam seems to take design very seriously.

I could tell from his the buildings in his house and things like that. But, you

know, there's no sort of track record to speak of of like I always thought of Steve Jobs as having like taste, >> you know, and then and I've I've had a couple >> bosses over the years like Josh Taring

used to run business week. He kind of he just always struck me as this guy. He

just had taste, you know, whether it was the way something looked, the way a story should be. There was like this innate thing that was on this really high level. It strikes me that's kind of

high level. It strikes me that's kind of like what's required here. I guess

that's why you have someone like Johnny on some level, but but you have to have this like >> back and forth. How do we know that like any of you guys have taste and and are, you know, can shape a hardware product?

>> Honestly, we don't need to have taste ourselves and and that that is Johnny's job. He he's our discriminator on on

job. He he's our discriminator on on taste. And I think actually one thing

taste. And I think actually one thing that's been really nice is just realizing that the way they work in design and the way we work in research is there's some deep parallels there, right? there's like so much exploration

right? there's like so much exploration and ideation and you explore a bunch of hypotheses. You take your time. Um, and

hypotheses. You take your time. Um, and

then you create kind of the thing that you're happy, the artifact at the end that you're happy about and >> [clears throat] >> um, yeah, it's it's been really nice to kind of have them fold into the company and there's just a lot more direct

communication about here's the capabilities that we're going to ship and here's what the form factor looks like and how to gel them.

>> Okay. And I this is like a crass way to put this but um because I spend my life adoring and talking to these people, but you know, sometimes I'm just like, man, I just don't know if a bunch of math

nerds are the ones that you want making like uh the AI computer, you know, but I guess it is this blend that you're talking about.

>> Uh yeah, I mean, honestly, um there Yeah, you're you're right in that the people who are the best at building AI capabilities are slightly different from

the people who have the best taste. And

we do have teams built of people who have really great taste for model behavior. And I think there's like a a

behavior. And I think there's like a a very different kind of philosophy and a very different kind of set of questions you you need to keep asking yourself. Um one one example

of like a a good taste question, right?

Like um you can imagine this being like in the model behavior interview is like what should Chapi's favorite number be?

>> Um >> what should it what number be?

>> Chachi's favorite number be >> Oh, okay. Okay. Okay. I'm curious what you would ask. You would answer >> what I think its favorite number should be.

>> Well, I have a stupid answer which is that I went to Pomona College and 47 is this like number of lore there. So,

>> okay. Okay. Okay. Yeah. I mean, that's a good answer. Yeah. [laughter]

good answer. Yeah. [laughter]

>> Um the Okay. Um I'm going to let you go in a second. You've been really generous. I appreciate it. Um, is there

generous. I appreciate it. Um, is there Well, I'm going to ask you a question that chatbt told me to ask you, which is is uh, >> you know, it says like if you look back

in five years, um, are there any kind of like small, fragile, nent ideas that you're seeing

right now that you you your instinct is telling you might be at the heart of a big breakthrough? Yeah, there's a couple

big breakthrough? Yeah, there's a couple I would say a handful of ideas. Um I

can't go into too much detail on them, but yeah, I'm really really excited to scale them up. Yeah. [laughter]

>> Are there any hints any uh buckets of areas where these fall?

>> Yeah, I mean I've been concentrating a lot on pre-training. So, uh you know, some pre-training adjacent um a small number of ideas in RL as well and a small number of ideas of how to put it all together.

>> Yeah. Okay. All right. I tried. I tried.

So, and you may or may not have a device. And no, no hints.

device. And no, no hints.

>> Yeah. Uh, no. No hints. [laughter]

Um, okay. Well, we covered tons of ground. I really appreciate it. Is there

ground. I really appreciate it. Is there

I feel like I'm letting the nerds down a little bit. Um,

little bit. Um, as far as like the AI obsessives. Yeah.

Are there any any um technical anything you see people like kind of getting wrong about you guys at the moment that um you would set the record

straight on?

>> Yeah, I I mean I think the most important thing is um just I think anyone at OpenAI in research would tell you that it is just a researchcentric

company. It's a pure AI bet. um at the

company. It's a pure AI bet. um at the core of the company the ambition is to build AGI it's to build it without distractions and I think the you know anything when it comes to building

products it all flows very easily from that um yeah when it comes to what we want to do in research it's you know we want to automate AI research

I think selfishly like we want to accelerate our our own progress and then we want to automate scientific discovery and of course we want to automate the

ability to do economically useful work and um I think all these pillars are falling it and and you see that kind of the big update in the last year has just been

like in that second pillar of automating scientific research. It's happening.

scientific research. It's happening.

>> Yeah.

>> How old are you now?

>> Um 34 about to turn 35.

>> About to turn 35. Okay. Are you able to have like a social life or are you are you >> No, honestly not. Um I yeah I think

every day the last two weeks, you know, it's been work calls till 1 2 a.m. Uh

but I but I love doing it. It's just um you know, there's there's a lot of work to get done. There's a lot of people I want to recruit. There's um a lot of steering that needs to be done. And like

why waste this golden moment? It's like

if we're in the middle of something like an industrial revolution, um you got to take as much advantage of it as possible.

>> Yeah. I hear stories about you sleeping at the office and >> oh yeah that that was a fun one too. Um

no honestly it's just um yeah I think there are times in the company um I think that was right after um Barra left and and went to found their own company.

Um, it just the job demands it and um, like I think when I peel it all back and examine that deep emotion, it's just this protectiveness of of the research.

>> That was after Mera mirror left.

>> Yeah. Yeah. I spent spend spend a month kind of sleeping in in the office and it's just like I I need to protect the research. They they feel like, you know,

research. They they feel like, you know, it feels like my baby. Yeah.

>> So, you guys have gone through these waves. There's the coup. Everybody's

waves. There's the coup. Everybody's

trying to steal your people.

>> [snorts] >> I guess everybody's trying to steal your people all the time, but you have this inflection point. Mirror leaves, Meta

inflection point. Mirror leaves, Meta decides they're going to fire up this massive lab. Do you think are we like

massive lab. Do you think are we like are we pass has everybody fired their their shot at this point?

>> You know, um you know, I have a staff meeting, right? I talked to talk to my

meeting, right? I talked to talk to my reports and I'm like, okay, well, here here's the thing that I'm working on and you know, once I get back to once I'm done with this thread, you know, I'm I'm going to zoom out and you know, there's

no more fires. Um, no, no. I I've fully internalized at this point, you know, the stakes are high enough for for building AGI that there's always going to be something. And um, I think the

important thing is just being able to understand what the important things are in the midst of all of all of these these things going on.

>> Do you like months have passed since there was sort of the deepseek moment or whatever? I guess it was like December

whatever? I guess it was like December 2024, I think.

>> Yeah. Yeah. earlier earlier this Yeah.

>> Yeah. Yeah. Or January. Yeah. I mean, is there anything um now, you know, it felt like people lost their minds for a second? Um just like reflecting on it

second? Um just like reflecting on it now um and seeing what they've done since just just like thoughts, I guess, on open source models and Chinese open source models.

>> Yeah. I mean I think that was one of the first points in time when um I just realized how important it is that we just stay true to our research format. I

think when when that came out, you know, it it went viral, right? Like everyone

was like, "Oh man, like has OpenAI lost its way? Are these models catching up?"

its way? Are these models catching up?"

And um what's the response? What's the

response? What's the response? And I

think rightfully the thing that we did was um we just stumbled down our own on our own research program. And I don't think it was the right it was the wrong call at all. Like um I haven't seen the

Deep Seek follow-up model. Um, you know, I I think they're they're a very strong lab, but fundamentally like let's just keep focusing on innovating. I think um

you know, DeepC was a great kind of replication of the ideas in in our O series of models, but let's just focus on innovating. Do you think 500 people

on innovating. Do you think 500 people is the does that number grow as the company grows or this is like the optimum number for kind of like big

ideas you can chase at one time? Um no

honestly uh I feel like it can be done with even less and um again you know as we get AI researchers or AI interns uh there's a real question of how do you

design an around that but um I'm certainly a person who cares a lot about heavy talent density um when like I I like to run a lot of experiments um of

this vein for for instance in quarter two of this year um I thought hey you know I'm just not going to open up any headcount for anyone research and you know if you want to hire people uh you

got to figure out who's not [snorts] on the boat and um I think these kind of exercises are quite important you know you don't want a or to diffuse into

something that's not manageable and you want to keep the talent bar very high >> I okay I promise this is the last question so yeah sorry I have to set you free um uh the

>> I remember being in a meeting and >> I think you and Yakoba were kind of on the same page here, but I remember you for sure.

>> Um, sort of like this idea of of who gets attribution for a project and >> you seem to be of the stance that like

people are obsessing over that a bit too much and and clearly AI has its roots in academia where you are very proud when you have a paper and it's a big deal and

and attribution is a huge thing. Um I

think I'm remembering that meeting right and yours.

Yeah. And and so >> what we've reached a new stage where that is less of a big deal or it's just

you this is a company and who did what is less important.

>> I I actually really love this topic and um I think overfixation on credit is a very bad thing. Right. I I think you know but on the other hand I actually

feel like it's important as a company for us to recognize credit both internally and externally and a lot of companies have actually shied away from this you know uh we've moved away from

publishing papers credits lists I think broadly throughout the industry but Jakob and I ended up making the call that we're going to do it at OpenAI and of course the counterargument is always

like man you're like handing your top performers on a platter, you know, everyone else is going to be recruiting these guys aggressively. But I don't think that's important, right? Like we

should just recognize the people who are doing great work. We should continue to be this pipeline for creating AI superstars. And um yeah, honestly, it's

superstars. And um yeah, honestly, it's important for us to make names for the people who are doing the best work at the company. So,

the company. So, >> but you seem to also be saying people, the individual researchers should maybe obsess about this less.

Where or am I am I totally misremeless?

>> No, I I think there was a sentiment in the room of of that form. Um, actually

Yakob and I held more of a dissenting view on that.

>> Okay. Okay. Okay. It's been a while.

It's in my notes. Perfect.

>> Yeah. Yeah. Yeah. [laughter]

Yeah. But um I I think we got to give credit where it's due even at the risk of everyone knowing who our top talent is.

>> Okay. Okay. Um

>> I will make an even stronger statement that I think OpenAI is the place where we allow for the most external credit per capita.

>> Okay.

>> By a large margin.

>> Okay. All right. Well, I'm check my notes on. Well, now I've got more.

notes on. Well, now I've got more.

>> Absolutely. Absolutely. [laughter]

>> Yeah.

>> I just I I just remember it being a topic of discussion and and and I there were numerous opinions. So, um, that that's funny. Um, in that, okay, I lied.

that's funny. Um, in that, okay, I lied.

Last question, I swear. So, you know, you got there in 2018. I mean, it was a research company. It was a nonprofit.

research company. It was a nonprofit.

Um, the company started among the the founders, you know, being this this counterwe to um to Google and and with

sort of, you know, making sure AGI arrived safely was kind of the goal. Um,

you came at this from highfrequency trading and and and saw these interesting things happening.

>> You know, like how much in your >> I'm sure you're going to say you want this to happen safely. I get that. But

like if you look at your career path, you're a smart, curious human who saw this interesting thing happening.

It's not like a requirement that you like really give a philosophically about this or or want to see, you know, a super intelligence. Yeah. I mean, but anyway, like let's hear from you on on

like why are you doing this in the first place?

>> Yeah. So I think um really on the safety and alignment piece um I managed the alignment team at OpenAI as well and um I honestly feel like some of the grand

challenges over the next one or two years are alignment and I think for people paying attention to this slice of research broadly in the field open I think has probably done the best work in

the last year and why I say that is like there's been so much work on things like scheming right the more RL compute that you pump into the model the more you can measure things like self-awareness,

self-preservation, uh potentially even situations where the model can scheme and and it's scary because the model can come to you with

the right answer at the end, the answer that you expect, but arrive at it from a very kind of twisted way, right? And um

I think as the models do more complex tasks for us uh having a handle on what its thought process is um is going to be super super important and okay chat told

me to ask you a question along these very lines which is I mean you're talking about a field mechanistic interpretability where we're trying to >> is a a term that captures trying to understand this black box and how it

operates and and I guess the heart of the question was Do our skills at doing that keep up with the complexity of the AI systems or do we just get to this runaway point where

it's like we're never going to learn how this thing works?

>> Yeah. Um, so I think one of the decisions that went all the way back to 01's release, which I'm very proud of, is we decided that we weren't going to

supervise the the model thinking process. Um, and I think when you put

process. Um, and I think when you put incentives into the model to uh, you know, give you a thinking process that is appealing to a human, um, it won't

necessarily be honest with you, right?

It won't say kind of tell you it's it's its true intentions and and so we've actually through that channel been able to maintain observing the thinking

process of the model as a tool towards understanding alignment. And um you know

understanding alignment. And um you know there was a paper that was published just a couple months ago uh with deep mind with anthropic um really exploring

kind of how this will evolve as a tool over time and and so you know I think we've made a lot of fairly good choices in in design here. Um yeah, I really do worry about this world in the future

where the model will tell us something super convincing but we can't be sure um whether the model is aligned with us, right? align with our values and and so

right? align with our values and and so I think there are a lot of interesting directions here like um can you set up games right or can you set up frameworks or environments where you know like models supervise each other or they

co-evolve together in a certain way where like the only like stable equilibrium is one where you know the models are honest um and yeah I think there's a lot of very exciting work to do there

>> okay all right okay I'll behave myself now thank you so much for joining us I I am glad I'm old enough now that I don't have to take a job interview from like a super intelligent [laughter]

uh chatbot that like I I feel like uh that you can't sort of try to charm your way past and and >> Great, Ashley. You would do great at

>> I don't know, man. I don't know. I'm

feeling okay, but I've I've I've old enough not to have to probably do that.

U Thank you, Mark, so much. You're I

know you're super busy, so thank you for your time.

>> Thank you so much for your time, too.

All right, man. It was fun. Really

pleasure.

>> Okay.

[music] The Core Memory podcast is hosted by me, Ashley Vance. It is

produced by David Nicholson and me. Our

theme song is by James Mercer and John Sortland. And the show is edited by the

Sortland. And the show is edited by the John Sortland. Thanks as always to Brex

John Sortland. Thanks as always to Brex and Elone Ventures for making this possible. Please visit our Substack,

possible. Please visit our Substack, YouTube, and podcast channels to get more of what Core Memory makes. Thanks

y'all.