Amjad Masad & Adam D’Angelo: How Far Are We From AGI?

Nothing seems fundamentally so hard that

it couldn't be solved by the smartest

people in the world working incredibly

hard for the next five years.

>> Humanity went through the agricultural

revolution and the industrial

revolution. We're going through another

revolution. We will not be able to call

it something. It's like future people

will call it something. But we are going

through something. The number of solo

entrepreneurs that this technology is

going to enable. It's vastly increased

what a single person can do.

>> For the first time, opportunity is

massively available for everyone. Just

the ability for more people to be able

to become entrepreneurs is Yeah,

>> it's massive.

>> Adam, welcome to the podcast.

>> Thank you. Yeah, thanks for having us.

>> So, a lot of people have been throwing

cold water over LLMs lately. It's been

some general bearishness. People talking

about the limitations of of LLMs, why

they won't get us to AGI. Well, maybe uh

what we thought was just a couple years

away is now maybe 10 years away. Adam,

you seem a bit more optimistic. Why

don't you share your broad general

overview?

>> Yeah, I mean I I actually honestly I

don't know what people are talking

about. I think I think if you look a

year ago, the world was very different.

And so just judging on how much progress

we've made in the last year with things

like reasoning models, um things like

the improvement in code generation

ability, um the improvements in video

gen, it seems like things are going

faster than ever. And so I I don't

really understand where the the kind of

bearishness is coming from. Well, I

think there's some sense that we hoped

that they would be able to um replace

all of tasks or all all jobs. And maybe

there's some sense that it's like middle

to middle but not end to end. And maybe,

you know, labor won't be automated in

the same way that we we thought it would

on the same timeline.

>> Yeah. I mean, I I don't know what the

previous timelines people were were

thinking were, but you know, I think I

think if you if you go 5 years out from

now, we're in a very different world. I

think I think a lot of what's holding

back the models these days is not

actually intelligence. It's getting the

right context into the model so that it

can be able to to use its intelligence.

Um, and then there's some things like

computer use that are still not quite

there, but I I think we'll almost

definitely get there in the next year or

two. And when you have that, I I think

we're going to be able to automate a

large portion of what people do. I don't

think I don't know if I would call that

AGI, but I I think it's going to satisfy

a a lot of the critiques that people are

making right now. I I think they won't

be valid in in a year or two.

>> What is your definition of AGI?

>> I don't know. Everyone everyone thinks

it's something different. I think I mean

you know one one definition I I I I kind

of like is um if you say that you have a

remote worker a human any job that could

be done by someone whose job can be done

remotely

um that that's AGI you know you can you

can then say does have to be better than

the best person in the world at every

single job some people call that ASI um

does have to be better than teams of

people

you can you can argue with those

different definitions. But I I think

once we get to be better than a typical

remote worker at the job they're doing,

we're living in a a very different

world. And I think that's that's

effectively what people that that's a

very useful anchor point for for these

definitions.

>> So in summary, you're not sensing the

same limitations of LM that other people

are. You think there's a lot more room

that LMS can can go from here? We don't

need like a brand new architecture or

other breakthrough.

>> I don't think so. I mean I I think there

are certain things like memory and

learning like continuous learning that

are not very easy with the current

architectures. I think even those you

can sort of fake and maybe we're going

to be able to to get them to work well

enough. Um but we we just don't seem to

be hitting any kind of of limits. The

the progress in reasoning models is

incredible. And I think the progress in

in pre-training is is also going pretty

quickly. Maybe not as quickly as people

had expected, but certainly fast enough

that you can expect a lot of progress

over over the next few years.

>> Amad, what's your what's your reaction

hearing all this? Yeah, I I I think I've

been pretty consistent and consistently

right perhaps

>> dare I say

>> consistent with yourself or consistent

with what I'm saying

>> with with um with myself and with I

think how things are unfolding that uh

you know I started being a bit of a more

public doubter of of things around uh

the time when the AI safety discussion

was uh was reaching its height back in

maybe 22 23. Um, and I I thought it was

important for us to be realistic about

the progress. Um, because, you know,

otherwise we're going to scare

politicians. We're going to scare

everyone. You know, uh, DC will descend

on Silicon Valley. We they'll shut

everything down. So my criticism of the

idea of like AGI 2027, you know, that

paper that I think it's called

Alexander, someone else wrote

>> uh and then um and the situational

awareness and all this uh hype papers

that are not really science, they're

just vibe. Here's what I think will

happen. Uh you know, the whole economy

will get automated. You know, jobs are

uh are going to disappear. all of that

stuff is that again is just I think um

it's unrealistic. It is not following

the kind of progress that we're seeing

and it is uh going to lead to just bad

policy. So my view is LMS are amazing

amazing machines. Uh I don't think they

are exactly human uh intelligence

equivalent. You can still trick LMS with

things like they might have solved the

strawberry one, but you can still, you

know, uh trick it with like single

sentence questions like how many Rs are

in this sentence. I think I think I

tweeted about it the other day, which

was like three out of the four four

models couldn't didn't get it even. Um

and then GP5 with high thinking had to

think for like 15 seconds in order to

get a question like that. So uh LMS are

I think a different kind of intelligence

than uh what humans are uh and also

uh they have they have clear limitations

and we're papering over the limitations

and we're kind of working around them in

all sorts of ways whether it's in the

LLM itself and the training data or uh

and the infrastructure around and

everything that we're we're doing to

make them work. Um but that that makes

me less optimistic that we're we've

we've cracked intelligence. And I think

once we truly crack intelligence

um it'll feel a lot more scalable and

that you can uh and that the the idea

behind the lesson will actually be true

and that you can just pour more um more

power, more resources, more compute into

them and they'll they'll just scale more

naturally. I think right now uh there's

a lot of manual work going into making

these models better. In the pre in the

true pre-training scaling era, you know,

GPT2, 3, 3.5, maybe up to four, um it it

felt like you you can just uh put more

internet data in there and just it just

got better. uh whereas now it feels like

there's a lot of labeling work

happening. There's a lot of contracting

work happening. A lot of these uh

contrived RL environments are getting

created in order to make uh LLMs good at

coding and becoming coding agents and

they're going to go do that. I think the

news from OpenAI that they're going to

do that for for investment banking. And

so I uh try to coin this term I call

functional AGI which is the idea that

you can automate a lot of aspects of a

lot of jobs by just going in and like

collecting as much data and creating

these RL environments. It's going to

take enormous effort and money and data

and all of that in order to do and I

think we're yeah I I I agree with Adam

that you know things are going to get

better uh 100% over the next 3 months 6

months cloud 4.5 was a huge jump uh I

don't think it's appreciated how much of

a jump it was over over four there's

really really amazing things about cloud

4.5 so there is progress we're going to

continue to see progress I don't think

LM as they can understand are on on the

way to AGI. And my definition for AGI is

I think the old school RL definition,

which is um a machine that can go into

any environment and learn efficiently in

the same way that a human could go into

uh you can put a put a human into a a a

pool game and you know within 2 hours

they can like shoot pool and be able to

do it. Uh right now there's no way for

us to have machines learn skills like

that on the fly. You know everything

requires enormous amount of data and

compute and time and effort and and and

uh and more importantly it requires

human expertise which is the non bitter

lesson uh idea which is you know uh

human expertise is not scalable and we

are relying today we are in a human

expertise regime. Yeah, I mean I I think

that

humans are certainly better at learning

a new skill from a limited amount of

data in a new environment than the

current models are. I think that on the

other hand, human intelligence is the

product of evolution which used a

massive amount of effective computation.

And so this is a different this is a

different kind of intelligence. And so

because it didn't have this this massive

equivalent of evolution, it just has

pre-training for for that which is not

as good. You then need more data to

learn everything, every new skill. But I

guess I think in terms of like the

functional consequence. So like if if

you're like when when will the world

when will the job landscape change? When

will the e economic growth hit? I think

that's going to be more a function of

when we can produce something that is as

good as human intelligence. Even if it

takes a lot more compute, a lot more

energy, a lot more training data, we

could just put in all that energy and

still get to software that's as good as

the average person at doing a typical

job.

>> So, I don't disagree with that and and

that's it is it feels like we're in a

brute force type of regime, but but

maybe that's fine. And

>> yeah.

>> Yeah.

>> So, where's the disagreement then, I

guess? So, there's agreement on that.

Where is the deer?

>> I I don't think that we'll get to the

singularity or I don't think that I

don't think we're going to get to the

next level of human civilization

uh until we um we we we crack the true

nature of intelligence

like until we understand and have

algorithms that are actually uh not

brute force and and you think those will

take a long time to come? Uh I I'm sort

of agnostic on on that. It just does it

does feel like the LMS

uh in a way are distracting from that

because um all the talent is going there

um and therefore there's less talent

that are trying to do basic research on

on intelligence.

>> Mhm. Yeah. At the same time a huge

portion of talent is going into AI

research that used to previously

wouldn't have gone into AI at all.

>> Mhm. And so you have this this massive

industry, massive funding, you know,

funding compute but also funding human

employees. And that is

I guess I nothing seems fundamentally

so hard that it couldn't be solved by

the smartest people in the world working

incredibly hard for the next 5 years on

it. But but basic research is is

different, right? like trying to

um like trying to get into the

fundamentals and as opposed to like

there's a lot of industry research like

how do we make these things more useful

uh in order to generate profit and um so

I I think that's that's different and

often I mean Thomas [ __ ] this

philosopher of science talks a lot about

how these research programs end up you

know becoming like a bubble and like

sucking all the attention and ideas and

like think think about physics and how

there are like these industry of a

string theory and like it pulls

everything in and there's sort of a plug

black hole of progress and you know

>> Yeah. Yeah. No, and I think I think one

of his things was like you got to wait

until the current people retire even

have a chance at changing the paradigm.

>> He's very pessimistic about paradigms.

But I I guess I feel like the current

paradigm, this is maybe our disagree, I

think the current paradigm is pretty

good and I think we're nowhere near the

sort of like diminishing returns of

continuing to push on it.

>> Mhm.

>> And I bet Yeah, I guess I would just bet

that you can keep doing different

innovations within the paradigm to to

get there.

>> So let let's say we continue to brute

force it. um we're able to automate a

bunch of labor. Do you estimate that GDP

is is something you know four or five

percent a year or are we going up to 10%

plus or what does it do to the economy?

>> I think it it depends a lot on exactly

where we get to and what what AGI means.

But so so let's say you have let's say

you have

LLM that with with an amount of energy

that costs

$1 an hour,

they could do a job of any

human. Let's just just just take that as

a as a theoretical point you could get

to. I think you're going to get to much

more than four to 5% GDP growth in that

world. I think the issue is you may not

get there. So it may be that the LMS

that can do everything a human can do

actually cost more than humans do

currently or they can do kind of like

80% of what humans can do and then

there's this other 20%. Um and and I I

think I do think at some point you get

to also

like I I don't see a reason why we don't

eventually get there. That may take

five, 10, 15 years. But I think until

you get there, we're going to get

bottlenecked on the things that the LM

still can't do or the, you know,

building enough power plants to to

supply the energy or

other bottlenecks in in the supply

chain.

One thing I worry about uh is uh the

delotterious effect of LMS in the

economy in that say LM's uh you know

effectively automate uh the entry level

job but not but but but the but not the

expert's job right so um let's take uh

you know QA Q quality assurance um And

uh it it's it's so good, but uh there's

still all these longtail event uh you

know events that it doesn't handle. And

so you have a lot of uh really good QA

people now like managing like hundreds

of agents and you effectively increase

productivity a lot. Uh but they're not

hiring new people because the agents are

better than new people. Uh and and and

that that feels like a weird equilibrium

to be in, right? And I don't think that

many people are thinking about it.

>> Yeah. Yeah. For sure. Yeah. No, I I I

think that's, you know, I think it's

happening with um CS majors graduating

from college, there's just not as many

jobs as there used to be. And

>> and um LLMs are a little more

substitutable for what they previously

would have done. And I'm I'm sure that's

contributing to it. And then it means

that you're going to have fewer people

going up that ramp that, you know,

companies paid a lot of money to to

employ them and and and train them. Um

and so I I think it's a real problem. I

think it's going to I'm guessing you'll

probably see some kind of like that

problem also creates a economic

incentive to solve the problem. So

>> it may be that there's like more

opportunities for companies that can

train people or maybe use of AI to to

teach people these things. Um but for

sure that's that's an issue right now.

Another related problem is that uh since

we're dependent on uh expert data in

order to train the LLMs and the LM start

to substitute

um those workers but but but you know at

some point there's no more experts

because they're all out of jobs and and

and and they're equivalent to the LLMs.

If the LMS is truly dependent on on

labeling data, expert RL environments,

then how would they improve beyond that?

I think that's something question for an

economist to really sit down and think

about is like once you get the first

tick of automation, I mean there there

are some challenges there. And so how do

you go how do you go how do you go to

the next part? Yeah, I mean I I think it

a lot of it is going to depend on how

good of RL environments can be

>> created. So, you know, in the one

extreme you have something like Alph Go

where just a perfect environment and you

can just blast past expert level. Um,

but I think a lot of jobs have limited

data that anyone can can train from. And

so I think it'll be interesting to see

how how easy is it for

research efforts to to overcome that

that bottleneck. If you had to make a

guess on what job category is going to

be introduced or explode in in the

future um you know some people say it's

like the you know everyone's an

influencer you know or in some sort of

caring um field or um you know

everyone's employed by the government

and some sort of bureaucrat thing or um

you know maybe training the AI in in

some way uh you know as as more and more

things start to get automated you know

what is your your guess as to what

more and more people start to

you know, doing art and poetry is

>> at some point you have everything

automated and then I think people will

do art and poetry and you know there's a

data point that the people playing chess

is up since computers got better at

human than than humans at chess. So I

don't think that's

a bad world if people are all just kind

of free to to pursue their their

hobbies. uh as long as you have some

kind of you know way to distribute

wealth so that so people can afford to

to live. Um but I you know in the near

that that's a while away and in the near

term

>> well like 10 15 years out

>> I I don't know how much but yeah in the

in the I'll put it in the at least 10

years range. Um, I I think in the near

term the job categories that are going

to explode, the jobs that can really

leverage AI and so so people who are

good at using AI to to accomplish their

jobs, especially to accomplish things

that the AI couldn't have done by

itself, there's just there's just

massive demand for for that.

>> I don't think we're going to get to a

point where you automate every every

job. Uh, definitely not in the current

paradigm. I would uh I would doubt it

happening.

I I I'm not certain it would ever

happen, but definitely not in the

current paradigm. Now, here's why I

think because a lot of jobs is about

servicing other humans. You need to be

fundamentally human in order to you need

to be actually human in order to

understand what other people want, you

know, and so you need to have the human

experience. So unless we're going to uh

create uh human humans, unless the m

unless AI is actually embodied in the

human experience, then humans will

always be the generators of ideas in the

economy. Adam, respond to Andre's point

around the human part because you

created one of the most, you know, the

best wisdom of the crowds, you know, uh

platforms in in the universe. Um and now

you've gone, you know, all all in with

Po. Um what are your thoughts on you

know to what extent will we be relying

on um humans versus will we be trusting

AIs to you know be our therapists be our

you know caretakers in other ways.

Humans have a lot of knowledge

collectively and you know even like one

individual person who's an expert and

has lived a whole life and had a whole

career and seen a lot of things they

they often know a lot of things that are

not written down anywhere

>> tacet knowledge

>> and um you call it tested knowledge but

also also what they're capable of

writing down if you did ask them a

question I think there's still an

important role for for people to play in

the

by sharing their knowledge, especially

when they have knowledge that that just

wasn't otherwise in an LLM's training

set. Um, you know, whether they will be

able to make a full-time living doing

that, I I don't know. But if that

becomes a bottleneck, then then for sure

that's going to mean that all the sort

of like economic pressure goes goes to

that. I don't in terms of the like you

know you have to be human to know what

humans want. I don't know about that. So

like as an example I think I think

recommener systems

the system that ranks your Facebook or

Instagram or Kora feed those recommener

systems are already superhuman at

predicting what you're going to be

interested in in reading. Like if if if

I gave you a task that was like make me

a feed that I'm going to read, like

there there's just no way. No matter how

much you knew about me, there's no way

you could compete with these algorithms

that just have so much data about

everything I've ever clicked on,

everything everyone else has ever

clicked on, what all the similarities

are between all those those different

data sets. And so I don't know, you

know, it's true that as a human you can

kind of like simulate being a human and

that makes it easier for you to like

test out ideas. And I'm sure that

composers and artists are this is an

important part of their their process

for doing work is they

>> or chefs or Yeah.

>> Yeah. They they produce something and

you know a chef will cook something and

they taste it and it's important that

they can taste it

>> but I don't know you know they they just

they have very little data compared to

what AI can be trained on. So So I I

don't know how that's going to shake

out.

>> That's a that's a good point. I mean

ultimately what recommended systems uh

are they're like aggregating all the

different tastes and then sort of

finding where you sit in the sort of

multi-dimensional taste vector space and

like getting you the best content there.

So I guess there's some of that. I think

that's more narrow than we think like

like yes it it's true in recommener

systems but I'm not entirely sure it's

true of of of everything. Um but so I I

think the best prediction for

where the world is headed and this is

not a

endorsement or necessarily like this is

where I think the world's headed because

I think part of it is

uh will be slightly in uh instable

unstable system but I think the

sovereign individual continues to be I

think a really good set of predictions

for the future although it's not a

scientific book or not. It's a very pyic

book and um but but the idea is uh you

know in the late 80s early 90s

um are they economists? I'm not sure. I

think they're economists or political

science majors uh two people out of the

UK um wrote this book about trying to

predict what happens uh when computer

technology matures, right? They're like,

you know, humanity went through the

agricultural revolution and the

industrial revolution. We're going

through another revolution. Uh clearly,

uh information revolution, now we call

it intelligence revolution, whatever. I

think we will not be able to call it

something. It's a future people will

call it something, but we are going

through something. And so they're trying

to predict, okay, what happens from

here? And what they arrive at is that

the um ultimately you're going to have

large swaths of people that are

potentially unemployed or economically

not um contributing, but you're going to

have the entrepreneur the entrepreneur

capitalists going to be

so highly leveraged because they can

spin up these companies with AI agents

very quickly. Oh, because they have this

because they're very generative. They

have interesting ideas. They're human.

They've uh they have interesting ideas

about what other people want. They can

create these companies very quickly in

these products and services and they can

organize the economy in certain ways.

And the politics will change because uh

to you know today's politics is based on

um

every human being uh economically

productive. Uh but when you have only uh

when you have massive automation and

then you have a few entrepreneurs and

very intelligent generative people are

actually uh able to be productive then

the political structures also change. Um

uh and so they talk about how the you

know nation state sort of subsides and

instead you go back to uh to an era

where um states are like competing over

people over wealthy people and like they

you know uh as a sovereign individual

you can like uh negotiate your tax rate

with your favorite state and so it

starts to sound like biology a little

bit and I don't think it is far from

where I where it might be headed. Now

again, it's it's not a sort of a value

judgment or or desire. Uh but but I do

think it's worth thinking about when

when people are not the the

you know unit of economic productivity,

things have to change, including culture

and and politics.

>> Yeah. I I think there's a question with

that book and in some of this

conversation more broadly of like when

does a technology reward the uh you know

the defender versus this the sort of

aggregator or something or like the um

when does it incentivize more

decentralization versus centraliz like

uh remember Peter Tiel had this quip a

decade ago of like you know crypto is

libertarian is more decentralizing AI is

you know communist or more centralizing

and it it um it's not obvious to me that

that that that's entirely accurate. um

on on either side AI does seem to

empower a bunch of individuals as you

were saying and then also you know

crypto turns out is like fintech or it's

like stable you know uh it does empower

sort of uh you know in nation states

we're talking about doing the sort of

like you know the the China thing that

they were going to do so yeah I think

there's an open question as to

you know which technology leads to who

does it empower more the edges or the

the center and I think if it empowers

the edges it seems like the sovereign

individual is is and and maybe there's a

barbell uh where it's like both

basically the big the incumbents just

get much much much much bigger and

there's like these edges but anyways

that's

>> I'm I'm very excited for the um the

number of solo entrepreneurs that

>> this technology is going to enable. I

think it's it's just greatly it's vastly

increased what what a single person can

do and there's so many ideas that just

never got explored because it's a lot of

work to get a team of people together

and maybe raise the funding for it and

get the right kind of people with all

the different skills you need. Um and

now that one person can can bring these

things into existence, I I think I think

we're going to see a lot of really

amazing stuff. Yeah, I get these tweets

all the time about people who like quit

their jobs because they started making

so much money. You're using tools like

like rapid and um it's it's really

exciting. I think uh if for the first

time opportunity is massively available

for for everyone

>> uh and I think that that is to me the

most exciting thing about this

technology other than all the other

stuff that we're talking about just the

ability for more people to be able to

become entrepreneurs is yeah it's

massive

>> that that trend is obviously going to

happen as we look out of the next decade

or two do you think that AI is more

likely to be sustaining or disruptive in

the Christian sense to ask it another

Okay. Do you think that most of the

value capture is going to come from

companies that were scaled pre OpenAI

starting? Um uh so is so replet still

counts as the the latter and so does

court to some degree or or do um do you

think most of the value is going to be

captured by companies that started you

know after let's say 2015 2016? So

there's a related question which is how

much of the value is going to go to the

hyperscalers versus everyone else and I

think on that one we are I actually

think we're in a pretty good balance

where there's enough competition among

the hyperscalers that the

um there's enough competition that as an

application level company you have

choice and you have alternatives and the

the prices are coming down incredibly

quickly. Um, but there's also not so

much competition that the hyperscalers

and the you know labs like Anthropic and

OpenAI, there's not so much competition

that they are unable to raise money and

make these long-term investments. And so

I actually think we're in a in a pretty

good balance and and we're going to have

a lot of a lot of new companies and a

lot of growth among the the

hyperscalers.

I think that's that's about right. So

the terminology of sustaining versus

disruptive comes from uh uh the

innovator's dilemma. Uh and uh it's it's

this idea that uh whenever there's a new

technology trend, it sort of there's

this idea of a power curve. It starts as

a toy almost or something that doesn't

really work or captures the lower end of

the market. But as it sort of evolves,

uh it goes up the power curve and

eventually disrupts even the incumbents.

So originally the encompass don't pay

attention to it uh because it looks like

a toy and then eventually disrupts

everything and eats the entire uh sort

of market. Uh and so that that was true

of PCs. You know, when PCs came along,

the big main mainframe manufacturers did

not uh uh pay attention to it and and

initially it was like yeah, it's for

it's, you know, for kids or whatever. Uh

but we we have to run these large

computers or data centers or whatever,

but now even data centers are running on

PCs and so on. Um and and so PCs were

just a hugely disruptive uh force. Uh

but there are technologies that come

along and really benefit the incumbents

and really don't really benefit the uh

the uh new players, the startups. Uh I

think Adam's right. It's uh it's both.

Um and maybe for the first time it's

kind of both like a a huge technology

trend cuz the internet was hugely

disruptive. Um but but this time uh it

feels like it is an obvious supercharge

for the incumbents for the hyperscalers

for the large uh internet companies but

it also enables uh new business models

that uh that is perhaps counterposition

against the uh the existing existing

ones. Al although the the you know I

think what happened is everyone read

that book and everyone learned how to

not be disrupted. Uh for example Chad

GPT was fundamentally counterposition

against Google because uh Google had a

business that that was actually working.

Uh Chad GPT was seen as this uh

technology that hallucinates a lot and

creates a lot of bad information and

Google wanted to be trusted and so

Google had chatb internally. they didn't

release Gemini until like two years

after Chachup and Chachup had sort of

already won the like at least brand

recognition. Um and and so there there

was in a way open AI came out as a

disruptive technology uh but but now

Google realizes it's a disruptive

technology and kind of responds to it.

At the same time it was always obvious

that AI is going to benefit Google at

minimum. It's uh you know overview uh

search overview has gotten a lot better.

um all its uh you know workspace suite

is is getting a lot better with Gemini.

Uh their mobile phones, everything gets

better. So it's it seems like it's it's

both. Yeah, I I really agree. Like

everyone read the book and and that

changes what the theory even means

because you have

>> you've like all the all the public

market investors have read that book and

they

>> now are going to punish companies for

not adapting and reward them for

adapting even if it means they have to

make long-term investments. I think, you

know, all the the management leadership

of the companies have have read the book

and they're on top of their game. I

think also just like the people running

these companies are

in I I guess I would say smarter I think

than like the the companies from the

generation that that book was sort of

built on. and they're they're on at the

top of their game and they are a lot of

them are founder controlled and so they

can make it's it's easier for them to

sort of take a hit and and make these

these investments. So that's I actually,

you know, I think if if you had an

environment more like we had in say like

the '90s,

I think this would actually be more

disruptive than than the the current

hyper hyper competitive uh

>> world that we're in now.

One mistake that we as a firm have

reflected on over the past few years,

though of course I haven't been here for

more than just a few months, is this

idea of we've that we've passed on

companies because we they weren't going

to be the market leader or the or the

category winner. And thus we thought,

oh, you know, learning the lessons from

from web 2, you have to invest in the in

the category winner. That's where things

are going to consolidate. Value is going

to acrew over time. And um it seems so

you why do the the next foundation model

company if the first one already has a

has a head start. Um but it seems like

the market has gotten so much bigger

that in foundation models but also in

applications there's just multiple

winners and they're kind of you know

fragmenting you know and taking parts of

the market that are all venture scale.

I'm curious if this is a durable

phenomenon or but um it that seems just

one difference than than the web two era

is just more winners um across more

categories.

>> I think network effects are playing much

less of a role now than they did in the

web 2 era also and that that makes it

easier for competitors to get started.

There's still a scale advantage

>> because you know if you have more users

you can get more data. If you have more

users, you can raise more capital. But

that advantage is not it doesn't make it

absolutely impossible for a competitor

of smaller scale. It makes it hard, but

it's there there's definitely like room

for more winners than than there was

before. I I think another difference is

that people are seeing the value um so

strongly that they're willing to pay um

early on in maybe a way that they the

question with web two companies was how

are they going to make money you know

you were Facebook super early obviously

you know Google etc was like oh how are

they going to monetize and you know the

companies here are monetizing from from

the get-go you know your guys' companies

included

>> yeah yeah and the I I think with the

earlier generation of companies

the monetization kind of depended on

scale.

>> Like you couldn't build a good ad

business until you got to millions, tens

of millions of users. And now with

subscriptions, you can just charge right

away, I think, especially thanks to

things like Stripe that are making it

easier. Um, and so that that that's also

made it a lot more friendly to to new

entrance. There's there's also uh

questions of geopolitics like you know

it seems clear that we're not uh in this

um globalized era and perhaps it's going

to get much worse and so investing in

the foundation in the open AI of of

Europe might be a good idea and like

similarly China being an entirely

different different world and so there's

um sort of a geo aspect of it that

interesting

>> all of a sudden our geopolitics you know

nerdiness is helpful is is useful. Um,

Adam, you were talking about sort of

human knowledge. Did you see yourself

with Po kind of disrupting yourself in a

sense or or talk about the the the bet

that you you made with with PO and the

sort of evolution there?

>> You know, I I think we saw Po more as

just an additional opportunity than than

as disruption to to Kora. Um the the way

we got to it was we in early 2022 we

started experimenting with using GBD3 to

generate answers for Kora and and we

compared them to the the human answers

and sort of realized that they weren't

as good but what was really unique was

that you could instantly get an answer

to anything you wanted to ask about and

we realized it didn't need to be in

public. It actually was your preference

would be to to have it be in private and

so we felt like there was just a new

opportunity here to to let people chat

with with AI and in private.

>> Yeah. And it seemed like you were also

making a bet on how the different

players were going to that there was

going to be

>> Yeah. Yeah. So it was also a bet on

diversity of of model companies which

took a while to play out. But I think

now we're we're getting to the point

where there's there's a lot of models.

There's a lot of companies especially

when you go across modalities. You think

about image models, video models, audio

models. Um especially like the reasoning

research models are are sort of

diverging. Agents are starting to be

their own source of diversity. Um, so,

so we're lucky to to now be getting into

this world where there's there's sort of

enough diversity for a a general

interface aggregator to to make sense.

Um, but yeah, it was it was a bet early

on. We kind of

>> it's surprising actually that um even uh

not particularly technical consumers

actually do use multiple AIS. Uh like I

didn't expect that like you know people

only used Google. they never like looked

at Google and then Yahoo or like very

few people do it. But now you talk to

just average people and they'll say,

"Yeah, I use CHP most of the time, but

Gemini is much better at like these

types of questions." And it's like, "Oh,

interesting. The sophistication of

consumers have gone."

>> And even people saying that they have

different personalities and they, you

know, you know, sort of resonate with

Claude more, you know, or whatever. the

um I want to return back to this point

you said earlier Adam about you're kind

of talking about like dark matter about

how we're going to you know brute force

there's a lot of knowledge that people

have that's you know sort of not um sort

of categorized yet and it's not just

task of knowledge it's actually

knowledge that you could you know ask

them about and they could describe it

how you know because one question people

have with LMS is like how much we've

already trained the whole internet how

much more knowledge is there um and so

is it like 10x is it like a thousand

like what is sort of the what is kind of

intuitive sense of if we do brute force

it and build this whole you know machine

that gets all the knowledge out of

humans onto sort of you know a data set

that we can then you know implement

how do we think about the upside from

there

>> you know I think it's very hard to

quantify but

there's a massive industry developing

around getting human knowledge into for

the form where AI can use it so this is

things like scale AI I Surge

Merkore, but there there's a massive

>> long tale of other companies just

getting started. And

as you have, you know, as intelligence

gets cheaper and cheaper and more and

more powerful, the bottleneck, I think,

is increasingly going to be on the data

and what do you need to create that

intelligence? And so that's going to

cause this that's going to cause more

and more of this to happen. It might be

that people can make more and more money

by training AI. It might be that more

and more of these companies get started.

Um or it might be it might be that

there's there's other forms of it. But I

I think I think it's going to be sort of

like the economy is going to naturally

value whatever the AI can't do.

And

>> what is the framework for what it can't

like? what has meant a model for what it

can't do?

I don't, you know, you could you could

ask a an AI researcher, they they might

have a a better answer, but to me,

there's just information that's not in

the training set. And that is something

that's inherently going to be, you know,

going to be something AI can't do. There

will be, you know, the AI will get very

smart. It can do a lot of reasoning. It

could prove every math theorem at some

point. If it starts from, you know, some

axioms that you that you give it, but if

it doesn't know

how did this particular company solve

this problem 20 years ago, if that

wasn't in the training set, then only a

human who who knows that is going to be

able to answer that question. And so

over time, how do you see Kora um

interfacing with or like how are you

running these in parallel? How how do

you think about this?

>> Yeah, so I mean Kora, our focus is on

human knowledge and and letting people

share their knowledge and um that

knowledge may be helpful for you know

it's it's it's helpful for other humans

and it's it's also helpful for AI to to

learn from. um we have relationships

with some of the AI labs um and we're

going to sort of play the role core will

play the role that it is meant to play

in this ecosystem which is a as a a

source of of human knowledge. Um at the

same time AI is making core a lot

better. we've been able to make

uh major improvements in moderation

quality and in uh in ranking answers and

in uh just just improving the product

experience. So uh so it's gotten a lot

better by applying AI to it.

>> Yeah. And and talk talk about your

future as well. Obviously you know you

had this business for for a long time

you know focused on developers. Because

at one point you're targeting you know u

nonprofit. No

>> exactly the edtech market I believe you

did two or three million in revenue

reported and then you know recently

techrunch I know it's outdated but I

think it reported something like 150

million. I know it's since you've had

this incredible growth as as you've

shifted the the business model um and

and the customer segment. How do you

think about the the future of replet?

>> Um I think Kpathy uh recently said that

it's going to be the decade of agents.

Uh and I think that's absolutely right.

It's um uh as opposed to like prior

modalities of AI like when uh AI first

came to coding it was autocomplete with

co-pilot then it became sort of chat

with chat

then I think cursor innovated on this

composer modality which is like editing

like large chunks of uh files but that's

it. I think replet what Replet innovated

is is is is the agent um and the idea of

like not only editing code, provisioning

infrastructure like databases, doing

migrations,

um you know connecting to the cloud,

deploying uh having the entire debug

loop like executing the code, running

tests, um and so just like the entire

development life cycle loop happening

inside an agent and that's going to take

a long time to mature. So we're agent in

beta came September 2024 and it was the

first of its kind that did this both

code and infrastructure but it was you

know fairly janky didn't work very well

and then agent v1 around December

um it took another

um uh generation of models so you go

from claw 3.5 to 3.7 3.7 was the first

model uh that uh really knew how to use

a computer, a virtual machine. So,

unsurprisingly, it was the first also

computer use model. Um, and these things

have been moving together. Uh, and so

with every generation of models, we see

we find new capabilities. And, um, you

know, um, Agent V2 improved on autonomy

a lot. Agent V1 could run for like 2

minutes. Agent V2, uh, uh, ran for 20

minutes. Agent 3, we advertised it as

running for 200 minutes. just felt like

it should be symmetrical, but like it's

actually runs kind of indefinitely. Like

we've had users running it for 28 plus

hours. Wow.

>> Um, and the main idea there was that if

we put a verify on the loop. I remember

reading Deepseek uh a paper from Nvidia

about how they um used DeepSeek to write

CUDA kernels and they were able to run

Deepseek for like 20 minutes if they put

a verifier in the loop like being able

to run tests or something like that. And

I thought oh okay so what kind of

verifier can we put in the loop?

Obviously, you can put unit tests, but

unit test doesn't really capture whether

the app is working or not. So, we

started kind of digging into computer

use and whether computer use was going

to be able to test apps. Computer use is

very expensive and um it's actually kind

of still very buggy and like Adam talked

about that's going to be uh a big area

of improvement that'll unlock a lot of

applications. But we ended up building

our own framework with like bunch of

hacks and some some AI research and

repless computer use I think testing

models. I think one of the best. Um and

uh and once we put that into the loop

then you can put replet in high

autonomy. So we have an autonomy scale.

Uh uh you can you can you can choose

your autonomy level and then it just

writes the code goes and tests the

applications. If there's a bug it reads

the error log and like writes the code

again and and can go for for for hours.

And we've seen people build amazing

things by letting it run for for a long

time.

Now, that needs to continue to get

better. That needs to um to get cheaper

and faster. Uh so, it's not necessarily

a point of pride to run for a lot

longer. Like, it should be as fast as

possible. So, we're working on that. Um

a agent for there's a bunch of ideas

that are going to be uh coming out.

Agent 4, but one of the big things is

you shouldn't be just like waiting for

that one feature that you requested. you

should be able to work uh on a lot of

different features. So the idea of like

parallel agents is very interesting to

us. So you know you ask for a login page

but you could also ask for a stripe uh

checkout and and then you ask for an

admin dashboard. The AI should be able

to figure out how to paralyze all these

different tasks or some tasks are not

paralyzable but should also be able to

do merge across the code. So being able

to do collaboration across AI agents um

is very important and that way the

productivity of a single developer goes

up by a lot. right now even when you're

using clot code or cursor and others

that there isn't a lot of parallelism

going on but I think the next uh boost

in productivity is going to come from

sitting in front of programming

environment like replet and being able

to manage uh tens of agents maybe at

some point hundreds but you know at

least you know five 6 7 8 9 10 agents uh

all different all you know working in

different parts of your your product. I

also think that um UI and UX uh could

could use a lot of work in terms of um

right now um you're trying to translate

your ideas uh into this like textual

representation. I'm just like like a

PRD, right? The what product managers

do, right? Just product descriptions.

But product descriptions don't it's

really hard and you see it in a lot of

tech companies. it's really hard to

align on the exact features because it's

l language is fuzzy. And so I think

there's a there's a world in which

you're interacting with AI in a more

multimodal fashion. So open up uh like a

whiteboard and being able to draw and

like diagram with AI and and and really

work with it like you work with a human.

Uh and then um then the next stage of

that uh having uh like better memory

better memory inside the project but

also across project and perhaps having

different instantiations of replet agent

that uh you know that this this agent is

really good at like um Python data

science because um you know it has all

the information and skills and memories

of about my company what it's done in

the past. So I'll have a data analysis

like sort of rapid agent and I'll have

like a front-end replet agent and they

have memory over multiple projects and

over time and over interactions and

maybe they sit in your Slack like a like

a worker and you can like talk to them.

So again like I can I can keep going for

another 15 minutes about a road map that

could span like 3 to four to 5 years

perhaps. and but but this this agent

this agent phase that we're in is just

there's so much work to do and it's it's

it's going to be a lot of fun.

>> Yeah, it's a I was talking to one of our

mutual friends, one of the co-founders

of one of these uh you know big

productivity companies and he leads a

lot of their R&D and he's like man uh

during the week these days I'm not even

talking to humans anymore as much. I'm

just like it's just you know using all

all these agents to to build. So it's

living in the future to some degree is

already in the present.

>> There's something interesting about that

and that are people talking to each

other less at at companies

>> and is that a bad thing? Um

>> so it's a you know I think uh I I I'm

starting to think more about these

second order effects of of things like

that. um uh you know will it make it

awkward for like again the new grads I

feel so bad for them like uh you know if

if people are not sharing as much

knowledge between each other or it's

like

>> it's not culturally easy to go ask for

help because like you should be able to

use AI agents

>> uh there's something there's some

cultural forces that I think need to be

reckoned with.

>> Yeah, I think a lot of tough cultural

forces for zoomers these days. Yes. Um

let's gearing towards closing here. Um

obviously you guys are you know focused

on running your companies but to stay

current on the AI ecosystem. You you

guys also make angel investments as

well. Um where are you guys most uh most

excited? Um you we haven't talked about

robotics. Are you guys bullish on on

robotics in the in the near term or any

emerging categories or use cases or

spaces that you're looking to make more

investments in or you have made some? I

just think vibe coding generally is just

unbelievably

>> like high potential. Um just the idea

that all the you know this

>> you think underhyped even still

>> I think so I I I think

>> you know just opening up the potential

of software to the mainstream of you

know every everyone. I think that and

yeah and actually I think one reason I

think it's underhyped is that the tools

are still very far from what you can do

as a professional software engineer and

if you imagine that they're going to get

there and I think there's no reason why

they wouldn't might it'll take a few

years but um then it's like everyone in

the world is going to be able to

create any things that would have taken

a team of 100 professional software

engineers that's just going to massive

open up opportunities for for everyone.

So I think Replet is like a great

example of this, but I think it's also

going to that there will be cases other

than just like building applications

that that this also creates. By the way,

just on that note, if you were going to

Stanford or Harvard, you know, today

2025, just entering, would you major

again in computer science or just focus

on building something or

>> I think I would. I mean I I I

went to college starting in 2002 and it

was right after the dotcom bubble had

burst and there was a lot of pessimism

and I remember my um my roommate his

parents had told him like don't study

computer science even though that was

that was something he really liked. Um

and I just kind of did it because I I

liked it. And

I think that

I think that it's definitely like the

job market is worse than it was a few

years ago.

At the same time, I think having these

skills to understand the sort of

fundamentals of what's possible with

algorithms and data structures, I think

that actually really helps you in in

managing agents when when you're using

them. Um, and I I I'm guessing that it

will continue to be a valuable skill in

the future. I also think the other

question is like what else are you going

to study? And and every single thing you

could imagine, there's an argument for

why it's going to be automated.

>> So, I think you might as well study what

you enjoy and and and I think this is as

good as as anything.

>> Yeah. I um I think there's a lot to to

get excited by. One thing is maybe kind

of random, but like I get really fired

up to see like mad science experiments

like the uh Deepseek OCR that came out

the other day. Did you Did you see it?

It's It's wild where um correct me if

I'm wrong cuz I only looked at it

briefly, but basically you can um get a

lot more economical with a context

window if you like have a screenshot of

the text instead of the [ __ ] text.

>> Yeah, I'm not I'm not the right person

to be

>> correcting you on. than that. But like

it's there's there's definitely some

some really interesting things. Yeah, I

saw another thing on hacker news the

other day where um you know uh text

diffusion uh where someone made a text

diffusion model by instead of doing go

saying dnoising he would take like a

single BERT instance and like try to you

know mask different words and uh and

just predict like these different tokens

and um and so we have a lot of

components like I don't think people

think a lot about that you know we have

now the you know base pre-trained

models. We have the all these RL

reasoning models. We have the uh you

know encoder decoder models. We have

diffusion models. We have there's all

these different things like just like

you know you mix them in different ways.

>> Yeah.

>> Uh I feel like there isn't a lot of

that. I mean it' be great. It'd be great

if like a new research company just like

comes out and is like not trying to like

compete with OpenI and things like that

but instead uh is just trying to like

discover how to put these different

components together in order to create a

new flavor of these models.

>> Yeah. In crypto they talk about

composability and like mixing primitives

together and and AI maybe there needs to

be more exploation.

>> There's less playing around I found like

there is like I remember in the like

>> web 2.0

era when we were like playing around

with JavaScript what browsers could do

and what web workers could do whatever

there was a lot of like really

interesting weird experiments I mean

replet was born out of that the original

version of replet in open source pre pre

the company which my interest was like

can you compile C to JavaScript right

that was like one of the interesting

things that became WM by the time it was

uh mcriptton and it was like such a such

a nasty hack and um but I think there's

so much I think We're in an era of

Silicon Valley where it's like very uh

very getrich driven and that makes me a

little sad and that's partly why I moved

the company out of SF. I feel like the

culture in SF has has gotten maybe to

maybe I I I wasn't there but like during

the com era a lot of people talked about

how it's sort of like get rich fast or

the crypto thing. So I feel like there

needs to be a lot more tinkering and I

would love to see more of that and more

companies getting funded that are trying

to just do something a little more novel

even if it doesn't mean like it

fundamentally new new model.

>> Last question. Um Amad you've uh been

into consciousness for a long time. Are

are you bullish that we will um via some

of this AI work or just some you

scientific progress elsewhere make some

progress in understand in in uh you know

getting across this this hard problem or

you know something happened recently uh

which is interesting um uh cloud 4.5

uh seemed to have to become more aware

of its context length. So as it gets

closer to the end of the context, it

starts be becoming more economical with

tokens, it also it looks like its

awareness when it's being redteamed or

in a test environment like jumped

significantly. And so there's something

happening there that's quite

interesting. Now I think uh in terms of

you know the the question of of

consciousness it is still fundamentally

not a scientific question and there is a

sort of uh we've given up on trying to

make it scientific but I think it I

think this is also

uh the problem that I talked about with

all the energy going into LMS um uh no

one is trying to really think about the

true nature of intelligence, true nature

of uh consciousness. Um, and there's a

lot of really core core questions. Like

one of my favorite one is uh the uh

Roger Penrose um Emperor's New Mind

where he wrote a book about how everyone

in the sort of philosophy of mind space

uh and perhaps the larger scientific

ecosystem start thinking about the brain

in terms of a computer. And in that book

he tried to show that it fundamentally

is impossible for the brain to be uh a

computer because uh humans uh are able

to do things that touring machines

cannot do or Turing machines like

fundamentally get get stuck on such as

um uh you know just uh basic logic um

puzzles

uh that we're able to kind of detect,

but like there's no way to encode that

in a in a in a cheering machine. For

example, like this statement is false.

You know, those like old logic puzzles.

Um and uh anyways it's like a

complicated argument but uh if you read

that book or or many others uh there's

like a core strain of arguments in the

theory of mind about how uh computers uh

are fundamentally different from from

human intelligence and uh and so yeah I

I haven't really I've been very busy so

I haven't really updated my thinking too

much about that But

but I think there's there's a there's a

there's a huge field of study there that

is not being studied.

>> If you were a freshman uh entering

college today, would you study

philosophy?

>> I would do that. I would definitely

study philosophy of mind. I would

probably go into neuroscience. Uh cuz I

think those are the core questions that

are kind of become very very important

as AI kind of continues to see more of

jobs and economy and things like that.

>> That's a great place to wrap. I'm John.

Adam, thanks for coming on the podcast.

>> Thank you.

>> Thank you.

[Music]