Andrew Ng: Building Faster with AI

It's really great to see all of you.

What I want to do today since this is

build as startup school is share with

you some lessons I've learned about

building startups at AI fund. AI funds a

venture studio and we build an average

of about one startup per month. And

because we co-founded startups, we're in

there writing code, talking about

customers, design on features, detering

pricing. And so we've done a lot of reps

of not just watching others build

startups, but actually being in the

weeds, building startups with

entrepreneurs. And what I want to do

today is um share with you some of the

lessons I've learned building startups,

especially around this changing AI

technology and what it enables. And

it'll be focused on the theme of speed.

So it turns out that for those of you

that want to build a startup, I think a

strong predictor for startup's odds of

success is execution speed. And I

actually have a lot of respect for the

entrepreneurs and executives that can

just do things really quickly and new AI

technology is enabling startups to go

much faster. So what I hope to do is

share with you some of those best

practices which are frankly changing

every two to three months still to let

you get that speed that hopefully lets

you have a higher odds of success.

Before diving to speed, you know, a lot

of people ask me, hey Andrew, where are

the opportunities for startup? So this

is what I think of as the AI stack where

at the lowest level are the

semiconductor companies then the clouds

are hyperscalers built on top of that. A

lot of the AI foundation model companies

built on top of that. And even though a

lot of the PR excitement and hype has

been on these uh technology layers, it

turns out that almost by definition, the

biggest opportunities have to be at the

application layer because we actually

need the applications to generate even

more revenue so that they can afford to

pay the foundation cloud and

semiconductor technology layers. So for

whatever reason media and social media

tends not to talk about the application

layer as much but for those of you think

you're building startups almost by

definition the biggest opportunities

have to be there although of course the

opportunities at all layers of the

stack. One of the things that's changed

a lot over the last year um and in terms

of AI tech trends I if you ask me what's

the most important tech trend in AI I

would say is the rise of agentic AI and

about a year and a half ago when I

started to go around and give talks to

try to convince people that AI agents

might be a thing I did not realize that

around last summer a bunch of marketers

would get a hold of this term and use it

as a sticker and slap it on everything

in site which made it almost lose some

of meaning but I want to share with you

from a technical perspective why I think

agentic AI is exciting and important and

also opens up a lot more startup

opportunities. So, it turns out that the

way a lot of us use LMS is to prompt it

to have it gener output. And the way we

have an LM output something is as if

you're going to a human or in this case

an AI and asking it to please type out

an essay for you by writing from the

first word to the last word all in one

go without ever using backspace. And

humans, we don't do our best writing,

being forced to type in this linear

order. And it turns out neither does AI.

But despite the difficulty of being

forced to write in this linear way, um

our LMS do surprisingly well. With

agentic workflows, we can go to AI

system and ask it to please first write

an essay outline, then do some webs

research if it needs to and fetch uh

some web pages to put in their own

context, then write a first draft, then

read the first draft and critique it and

revise it and so on. And so we end up

with this iterative workflow where your

model does some thinking and some

research does some revision goes back to

do more thinking and by going around

this loop many times. Uh it is slower

but it delivers a much better work

product. So for a lot of t lot of

projects AI fund has worked on

everything from um pulling out complex

compliance documents to uh medical

diagnosis to reasoning about complex

legal documents. We found that these

agentic workflows are really a huge

difference between it working versus not

working. But a lot of the work that

needs to be done, a lot of the valuable

businesses to be built still will be

taking workflows existing or new

workflows and figuring out how to

implement them into these size of

agentic workflows. So just to update the

picture for the AI stack, um what has

emerged over the last year is a new

agentic orchestration layer that helps

application builders orchestrate or

coordinate a lot of calls to the

technology layers underneath. And the

good news is uh the orchestration layer

has made it even easier to build

applications. But I think the basic

conclusion that the application layer

has to be the most valuable layer of the

stack still holds true with a bias or

focus on the application layer. Let me

now dive into some of the best practices

I've learned for how startups can move

faster.

It turns out that um at AI fun we only

focus on working on concrete ideas. So

to me a concrete idea a concrete product

idea is one that's specified in enough

detail that an engineer can go and build

it. So for example if you say let's use

AI to optimize healthcare assets you

know that's actually not a concrete

idea. It's too vague. If you tell me

it's a very software to use AI to

optimize healthcare assets different

engineers would do totally different

things and because it's not concrete you

can't build it quickly and you don't

have speed. In contrast, if you had a

concrete idea like let's write software

to let hospitals, let patients book MR

machine slots online to optimize usage.

I don't know if this is a good or a bad

concrete idea. Actually business is

already, you know, doing this. But it is

concrete and that means engineers can

build it quickly. If it's a good idea,

you find out it's not a good idea, you

will find out. But having concrete ideas

buys you speed. Or someone to say, let's

use AI for email personal productivity.

Too many interpretations of that. That's

not concrete. But if someone says could

you build an app Gmail integrate the

automation that use let's use the right

prompts right filter entire emails that

is concrete I could you know I could go

build that this afternoon. So

concretness buys you speed and the

deceptive thing for a lot of

entrepreneurs is the vague ideas tend to

get a lot of kudos. If you go and tell

all your friends we should use AI to

optimize the use of healthcare assets.

Everyone will say that's a great idea.

But it's actually not a great idea at

least in the sense of being something

you can build. Uh it turns out when

you're vague, you're almost always

right. Uh but when you're concrete, you

may be right or wrong. Either way is

fine. We can discover that much more

fast, which is what's important for SA.

In terms of executing on concrete ideas,

I I I find that at AI fun, I ask my team

to focus on concrete ideas because um a

concrete idea gives clear direction and

the team can run really fast to build it

and either validate it, prove it out or

falsify and conclude it doesn't work.

Either way is fine. So we can do that

quickly and it turns out that finding

good concrete ideas usually requires

someone could be you could be a subject

matter expert thinking about a problem

for a long time. Uh so for example

actually before before you know starting

Corsera um I spent years right thinking

about online education talking to users

holding my own intuitions about what

would make a good edtech platform and

then after that long process I think YC

sometimes calls it wondering the idea

maze but after thinking about it for a

long time you find that the guts of

people that have thought about this for

a long time can be very good about

rapidly making decisions as in after

you've thought about this talked to

customers and so on for a long time if

you ask this expert, should I build this

feature or that feature? You know, the

gut, which is an instantaneous decision,

uh, can be actually a surprisingly good

proxy. It can be surprisingly good

mechanism for making decisions. And I

know I work on AI, you might think I'll

say, oh, we need data. And of course, I

love data, but it turns out getting data

for a lot of startups is actually slow

mechanism for making decisions. And a

subject matter expert with a good gut is

often a much better mechanism for making

a speedy decision. And then one other

thing for many successful startups at

any moment in time you're pursuing one

very clear hypothesis they building out

and trying to sell China Valley of

Hospice. Um and a startup doesn't have

resources to hedge and try 10 things at

the same time. So pick one go for it and

if data tells you to lose faith in that

idea that's actually totally fine. Just

pivot on a dime to pursue a totally

different concrete idea. So that's what

often feels like an AI fund. We're

pursuing one thing doggedly with

determination until the world tells us

we were wrong then change and pursue a

totally different thing with equal

determination and equal doggedness. And

one other pattern I've seen, if every

piece of new data causes you to pivot,

it probably means you're starting off

from two weaker a base of knowledge,

right? If every time you talk to a

customer, you totally change your mind,

probably means you don't know enough

about that sector yet to have a really

high quality concrete idea and uh

finding someone that's thought about a

subject for longer may get you on to

better path in order to go faster. The

other thing I often think about is the

built feedback loop which is rapidly

changing when it comes to how we build

with AI coding assistance. So when

you're building a lot of applications,

one of the biggest risks is custom

acceptance, right? A lot of startups

struggle not because we can't build

whatever we want to build, but because

we build something and it turns out

nobody cares. And so for a lot of the

way uh you know I build startups

especially applications less so deep

tech less so technology startups but

definitely application startups is we

often build software so this is an

engineuring toss and then we will get

feedback from users and this is a

product management toss and then we'll

go back you know then based on the user

feedback we'll tweak our views on what

to build go back to write more software

and we go around this loop many many

times iterate toward product market fit

and it turns out that with AI coding

assistance which Andre talked about as

well um rapid engineering is becoming

possible in a way that just was not

posing much more feasible. So the speed

of engineing is going up rapidly and the

cost of engineing is also going down

rapidly. This changes the mechanisms by

which we drive startups around this

loop. When I think about the software

that I do, I maybe put into two major

buckets. Sometimes I'll build quick and

dirty prototypes to test an idea. You

say build a new customer service

chatbot. Let's build AI to process legal

documents whatever build a quick and

dirty prototype to see if we think it

works. The other type of software where

I do is write maintain production

software maintain legacy software but

these massive production ready code

bases depending on which analysts report

you trust. It's been hard to find very

rigorous data on this. You know when

writing production quality code maybe

we're 30 to 50% faster with AI systems.

Hard to find a rigorous number. maybe

he's plausible to be but in terms of

building quick and dirty prototypes

we're not 50% faster I think we're

easily 10 times faster maybe much more

than 10 times faster and there are a few

reasons for this uh when you're building

standalone prototypes there's less

integration with legacy software

infrastructure legacy data needed um

also the requirements reliability even

scalability even security are much lower

and I know I'm not supposed to tell

people to write insecure code right

feels like the wrong thing to say, but I

routinely go to my team and say, "Go

ahead, write insecure code." Because if

this software is only going to run on

your laptop and you don't plan to

maliciously hack your own laptop, it's

fine to have insecure code, right? But

of course, after it seems to be working,

please do make it secure before you ship

it to someone else. And you know, like a

leaking PI, leaking sense data that that

is, you know, very damaging. So before

you ship it, make it secure and

scalable, but they're just testing it.

It's fine. And so I find increasingly

startups will systematically pursue

innovations by building 20 prototypes to

see what works, right? Uh because I I I

know that there's some angst in AI. A

lot of proof of concepts don't make to

production. But I think by driving the

cost of a proof of concept low enough,

it's actually fine if lots of proof of

concepts don't see the light of day. And

I know that um the mantra move fast and

break things got a bad rep because you

know it broke things. And some teams

took away from this that you should not

move fast, but I think that's a mistake.

I tend to tell my teams to move fast and

be responsible. And I think they

actually lots of ways to move really

quickly while still being responsible.

And in terms of the AI assistance uh

coding landscape, I think was it three

four years ago code autocomplete right

popularized by GitHub copilot and then

there was a cursor windserve generation

of AI enabled ids which great use winds

and cursor quite a lot um and then

starting I don't know six seven months

ago uh there started to be this new

generation of highly agentic coding

assistants uh including that she's using

o3 a lot for coding um cloud code is

fant Fantastic. Since quad 4 release,

it's become and and ask me again in a

few months, I may use something

different. But the tools are evolving

really rapidly, but I think uh cloud

codeex this is a new generation of

highly agentic coding assistance that is

making developer productivity keep on

growing. And the interesting thing is if

you're even half a generation or one

generation behind actually makes a big

difference compared to if you're on top

of the latest tools and I find my team

is taking really different approaches to

software engineing now compared to even

three or six months ago. One surprising

thing is we we're used to thinking of

code as this really valuable artifact

because it's so hard to create but

because the cost of software engine is

going down code is much less of a

valuable artifact as it used to. So I'm

on teams where you know we've completely

rebuilt a codebase three times the last

month right because it's not that hard

anymore to just completely rebuild a

codebase pick a new data schema is fine

because the cost of doing that has

plummeted some of you may have heard of

Jeff Bezos's terminology of a two-way

door versus a one-way door. A two-way

door is decision that you can make. If

you change your mind, come back out, you

know, reverse it relatively cheaply.

Whereas a one-way door is you make a

decision and you change your mind is

very costly or very difficult to

reverse. So choosing the software

architecture of your tech stack used to

be a one-way door. Once you built on top

of a certain tech stack, you set a

database schema, really hard to change

it. So that used to be a one-way door. I

don't want to say it's totally a two-way

door, but I find that um my team will

more often build on a certain tech stack

a week later, change your mind, let's

throw the code base away and redo it

from scratch on a new tech stack. I I

don't want to overhype it. We don't do

that all the time. There are still costs

to redoing that. But I find my team is

often rethinking what is a one-way door

and what's now a two-way door because

the cost of software engineering is so

much lower now. And maybe going a little

bit beyond software engineering, I I I

feel like this actually a good time to

empower everyone to build of AI. Uh over

the last year, a bunch of people advised

others not to learn to code on the

grounds of AI were automated. I think

we'll look back on this as some of the

worst career advice ever given because

as better tools make software engineing

easier, more people should do it, not

fewer. So when many decades ago the

world moved from punch calls to keyboard

and terminal that made coding easier.

When we moved from assemblies high level

languages like cobalt um there actually

people arguing back then that now we

have cobalt we don't need programmers

anymore like people actually wrote

papers to that effect but of course that

was wrong and programming languages made

it easier to code and more people learn

to code text IDs ID is the AI coding

assistant um and as coding becomes

easier more people should learn to code.

I have a controversial opinion which is

uh I think actually it's time for

everyone of every job role to learn to

code and in fact on my team you know my

CFO my head of talent my recruiters my

front desk uh uh person all of them know

how to code and I actually see all of

them performing better at all of their

job functions because they can code and

I think um I'm probably a little bit

ahead of the curve probably most

businesses are not there yet but in the

future I think we empower everyone to

code a lot of people can be more

productive I want to share with you One

lesson I learned as well on on why we

should have people learn to do this

which is um when I was teaching

generative VI for everyone on Corsera,

we needed to generate background art

like this uh using midjourney and you

know one of my team members uh new art

history and so he could prompt

midjourney with the genre the palette

the artistic inspiration had a very good

control over the images he generated. So

we end up using all of Tommy's generated

images. Whereas in contrast, I don't

know art history. And so when I prompt,

you know, image generation, I could

write, please make pretty pictures of

robots for me, right? And and I could

never have the control that my

collaborators could. And so I couldn't

generate as good images as he could. And

I think with computers, one of the most

important skills of the future is the

ability to tell a computer exactly what

you want. So they'll do it for you. And

will be people that have that deeper

understanding of computers that will be

able to command a computer to get the

outcome you want. And learning to code,

not not that you need to write the code

yourself. Steer AI to code for you seems

like it will remain the best way to do

that for a long time. with software

engineering becoming much faster. The

other interesting dynamic I'm seeing is

that the product management work getting

user feedback deciding what features to

build that is increasingly the

bottleneck and so I'm seeing very

interesting dynamics in multiple teams

over the last year a lot more of my

teams have started to complain that

their bottlenecks on product engineering

and design because the engineers have

gotten so much faster some interesting

trends I'm seeing three four five years

ago Silicon Valley used to have these

slightly suspicious rules of thumb but

nonetheless rules of thumb will have 100

p.m. to four engineers or 1 PM to seven

engineers was this like PM product

manager to engineering ratio right which

should take with a grain of salt but it

was typical of a 1 PM to six seven

engineers and with engineers becoming

much faster I don't see product

management work designing what to build

becoming faster at the same speed

engineers I'm seeing this ratio shift so

literally yesterday one of my teams came

to me and for the first time when we're

planning headcom for a project this team

proposed to me not at 1:00 PM to four

engineers but to have 1 PM to 0.5

engineers. So the team actually proposed

to me I still know no this is a good

idea for the first time in my life that

I saw you know managers proposed to me

having twice as many PMs as engineers

was a very interesting dynamic. I I

still don't know if this proposal I

heard yes is a good idea but I think

it's a sign of where the world is going

and I find as PMs that can code or

engineers with some product instincts

often end up doing better. The other

thing that I found important for startup

found for startup leaders is because

engineing is going so fast. If you have

good tactics for getting rapid feedback

to shape your perspective what to build

faster that helps you get faster as

well. So um I'm going to go through a

portfolio of tactics for you know

getting product feedback to keep shaping

what you will decide to build. And we're

going to go through a list of the faster

maybe less accurate the slower more

accurate tactics. So the fastest tactic

for getting feedback is look at the

product yourself and just go by your

gut. And if you're a subject matter

expert, this is actually surprisingly

good you if you know what you're doing.

A little bit slower is go ask three

friends or teammates to get feedback to

play with your product and get feedback.

Um little bit slower is ask three to 10

strangers you know for feedback. Um, it

turns out for when I built products, one

of the most important skills I think I

learned was how to sit in the coffee

shop, how to sit in a when it's

traveling, when I travel, I often sit in

the hotel lobby. It turns out learn to

spot places of high foot traffic and

very respectfully, you know, grab

strangers and ask them for feedback on

whatever I'm building. This used to be

easier when I was less known. When when

people recognize you, it's a little bit

more awkward. I found that um I've

actually sat with teams the hotel lobby

very high foot for traffic and you know

very respectfully ask strangers hey

we're building this thing do you mind

taking a look oh and I actually learned

in a coffee shop there a lot of people

working a lot of people don't want to be

working so we give them excuse to be

distracted they're very happy to do that

too but I've actually kind of made tons

of product decisions in a hotel lobby or

a coffee shop with collaborators just

just just like that send prototypes to

100 testers you if you have access to

logic group of users and prototype to

more users and these are these get to be

slow and slower tactics and I know

Silicon Valley you know we like to talk

about AB testing of course I do a ton of

AB testing but contrary to what many

people think AB testing is now one of

the slowest tactics in my menu because

it's just slow to ship it yeah it depend

on how many users you have right so and

then uh the other thing is um as you use

anything but the first tactic some teams

will look at the data they make a

decision but the missing piece is When I

AB test something, um, I don't just use

result of AB test to pick product A or

product B. My team will often sit down

and look carefully at the data to hone

our instincts to speed up to improve the

rate. I wish we're able to use the first

tactic to make high quality decision.

Often sit down and think, gee, I

thought, you know, this product name

will work better than her product name.

Clearly, my mental model the users

wrong. to really sit down and think to

update our mental model using all of

that data to improve the quality of our

guts on how to make product decisions

faster. That turns out to be really

important. All right, so talked about um

concrete ideas, speed up engineering,

speed up product feedback. This is one

last thing I want to touch on which is

I've seen that understanding AI actually

makes you go faster. Um and and here's

why. As a AI person, maybe I'm biased to

be pro AI, but I want to share you why.

So it turns out that when it comes to

mature technology like mobile, you know,

many people have had smartphones for a

long time. We kind of know what a mobile

app can do, right? So many people

including nontechnical people have good

instincts about what a mobile app can

do. If you look at mature job roles like

sales, marketing, HR, legal, they're all

really important and all really

difficult. But you know, there are

enough marketers that that have done

marketing for long enough and the

marketing tactics haven't changed that

much in the last year. So there are a

lot of people that are really good in

marketing and it's really important

really hard but that knowledge is

relatively diffused because you know the

knowledge of how to do HR like it hasn't

changed dramatically you know in the

last six months but AI is emerging

technology and so the knowledge of how

to do AI really well is not widespread

and so teams that actually get it that

understand AI do have a advantage over

teams that don't whereas if you need if

you have an HR problem you can find

someone you know that knows how to do it

well probably but If an AI problem,

knowing how to actually do that could

put you ahead of other companies. So

things like what accuracy can you get

for a customer service chatbot? You

know, should you prom fine tune a

workflow? Um how do you get a voice out

to low latency? There a lot of these

decisions that if you make the right

technical decision, you can like solve

the problem in a couple days. They make

the wrong technical decision, you could

chase a blind alley for three months,

right? And and one one thing I've been

surprised by, it turns out if you have,

you know, two possible architecture

decisions, it's one bit of information.

It feels like if you don't know the

right answer, at most you're twice as

slow, right? One bit, you know, try

both. It feels like one bit of

information can at most buy you a 2x

speed up. And I think in some

theoretical sense that is true. But what

I see in practice, if you flip the wrong

bit, you're not twice as slow. You spend

like 10 times longer chasing a blind

alley. which is why I think going in to

have this right technical judgment, it

really makes startups go so much faster.

The other reason why I find staying on

top of AI really helpful for startups is

um over the last two years we have just

had a ton of wonderful genai tools or

genai building blocks right partial list

but prompting workflows evals guardrails

rack voice act async programming lots of

ETL embeddings fine-tuning graph DB how

to integ

models there's a long and wonderful list

of building blocks that can quickly

combine to build software that no one on

the planet could have built, you know,

even a year ago. And this creates a lot

of new opportunities for starters to

build new things. So when I learned

about these building blocks, this is

actually a picture that I have in mind.

If you own one building block, like you

have a basic white building block, yeah,

you can build some cool stuff. Maybe you

know how to prompt. So you have one

building block, you build some amazing

stuff. But if you get a second building

block like you also know how to build

chat bots. So you have a white Lego

brick and a black Lego brick, you can

build something more interesting. Um if

you acquire a blue building brick as

well, you can build something even more

interesting. Get few red building bras,

maybe a little yellow one, more

interesting, get more building bras, get

more building bras, and very rapidly the

number of things you comb combine them

to into grows kind of combinatorily or

grows exponentially. And so knowing all

these wonderful building blocks lets you

combine them in much richer combination.

One of the things that deep learn does

so I actually take a lot of deep learn

courses myself you know to because work

with great we work with I think like

pretty much all the leading AI companies

in the world and cert and and um and uh

try to hand out building blocks. Um but

when I look at the deep learning course

catalog this is actually what I see. And

whenever I take these courses to learn

these building blocks, I feel like I'm

getting new things that can combine to

form kind of combinatorally or

exponentially more software applications

that were not possible just one or two

years ago. So just to wrap up, this is

my last slide. I then want to take

questions if if y'all have any. I find

that there are many things that matter

for startup, not just speed. But when I

look at the startups that AI fund is

building, I find that the management

team's ability to execute at speed is

highly correlated with its odds of

success. And some things we've learned

to get you speed is, you know, work on

concrete ideas. Um uh it's got to be

good concrete ideas. I find that as a as

executive, I'm judged on the speed and

quality of my decisions. Both do matter,

but speed absolutely matters. rapid

entrying with AI coding assistance makes

you go much faster but that shifts the

bottleneck to getting user feedback on

the product decisions and so having a

portfolio of tactics to go get rapid

feedback and if you haven't learned to

go to coffee shop and talk to strangers

it it's not easy but then just just be

respectful right just be respectful of

people that's actually very valuable

skill for entrepreneurs to have I think

and I think also um staying on top of

the technology buys you speed all right

with that let me thank Thank you very

much.

[Applause]

Happy questions.

As AI advances, do you think it's more

important for humans to develop the

tools or learn how to use the tools

better? Like how can we p position

ourselves to remain essential in a world

where you know intelligence is becoming

democratized? I feel like AGI has been

overhyped and so for a long time

there'll be a lot of things that humans

can do that AI cannot and I think in the

future the people that are most powerful

are the people that can make computers

do exactly what you want it to do and so

I think staying on top of the tools some

of us will build tools sometimes but

there were a lot of other tools others

will build that we can just use but so

people that know how to use AI to get

computers to do what you want it to do

will be much more powerful, not worry

about people running out of things to

do, but um people that can use AI will

be much more powerful than people that

don't.

Hey, so well first of all uh thank you

so much. I have a huge respect for you

and I think that you are true

inspiration for a lot of us. My question

is about uh the future of compute. So as

we move towards uh more powerful more

powerful AI, where do you think that

comput is heading? I mean we see people

saying let's ship GPUs to space. Some

people talking about nuclear power data

centers. What do you think about it?

There's something I'm debating what I

wanted to say in response to the last

question about kind of AGI about maybe

I'll answer this and a little bit the

last question. So it turns out there's

one framework you can use for deciding

what's hype and what's not hype. I think

over the last two years there's been a

handful of companies that um hyped up

certain things for promotional PR

fundraising influence purposes. And

because AI was so new, um, handful of

companies got away with saying almost

anything without anyone fact-checking

them because the technology was not

understood. So, one of my mental filters

is there's certain hype narratives that

make these businesses look more powerful

that's been amplified. Um, and so, for

example, this idea that um, AI is so

powerful, we might accidentally lead to

human extinction. That's just

ridiculous. But it is a hype narrative

that made certain businesses look more

powerful and it got you know ramped up

and actually helped certain businesses

fundraising goals. AI is so powerful

soon no one will even have a job

anymore. Just not true, right? But again

that made these businesses look more

powerful got hyped up or we are so

powerful so when the hype narrative

we're so powerful that by training a new

model we will casually wipe out

thousands of startups. That's just not

true. Yes, Jasper ran into trouble.

Small number of companies got wiped out.

But it's not that easy to casually wipe

out thousands of startups. AI needs so

much electricity. Only nuclear power is

good enough for that. You know, that

wind solar stuff not that's just not

true. So, I think a lot of this um GPUs

in space, you know, I don't know. It's

like um go for it. I think we have a lot

of room to run still for terrestrial

GPUs. Uh yeah, but but I think uh uh

some of these hype narratives are have

been amplified that that I think uh are

a distortion of what what actually will

be done.

There's a lot of hype in um AI and how

and nobody's really certain about how

we're going to be building the future

with it. But what are some of the most

dangerous biases or overhyped narratives

that you've seen people talk about or

get uh poisoned by that they end up

running with that we should try to avoid

or be more aware of and allow us to have

a more realistic view as we are building

this future.

So I think the dangerous AI narrative

has been overhyped. Uh AI is a fantastic

tool, but like any other powerful tool

like electricity, lots of ways to use it

for beneficial purposes. Also some ways

to use it in harmful ways. I find myself

not using the term AI safety that much.

Um not because I think we we should

build dangerous things, but because I

think safety is not a function of

technology, it's a function of how we

apply it. So like electric motor you

know you can't the maker of electric

motor can't guarantee that no one will

ever use it from unsafe downstream toss

like use electric motor can be used to

build a Dallas machine electric vehicle

can be used to build a smart bomb but

the electric motor manufacturer can't

control how be used downstream. So

safety is not a function of the electric

motor as a function of how you apply it

and I think the same thing for AI. AI is

neither safe nor unsafe. It is how you

apply it that makes it safe or unsafe.

So instead of thinking about AI safety,

I often think about responsible AI

because it is how we use it responsibly

hopefully or irresponsibly that

determines whether or not what we build

with AI technology ends up being harmful

or beneficial. And I feel like sometimes

that the really weird corner cases that

get hyped up in the news. I think just

one or two days ago there was a Wall

Street Journal article about AI losing

control of AI or something. And I feel

like that article took uh corner case

experiments run in a lab and you know

sensationalized it in a way that I think

was really disproportionate relative to

the lab experiment that was being run

and unfortunately technology is hard

enough to understand that many people

don't know better and so these hype

narratives do keep on getting amplified

um and I feel like this has been used as

a weapon against open source software as

well right which is really unfortunate.

Thank you for your work. I think your

impact is remarkable. Uh my question is

um as aspiring founders, how should we

be thinking about business in the world

where anything can be disrupted in a

day? Whatever great mode, product or

feature you have can be replicated with

VIP code and competitors in basically

hours.

It turns out when you start a business,

there are a lot of things to worry

about. The number thing I worry about is

uh are you building a product that users

love? Um it turns out that when you

build a business there are lots of

things to think about the go to market

channel competitors technology mode all

that is important but if I were to have

a singular focus on one thing it is are

you building a product that users really

want until you solve that you know is

very difficult to build a valuable

business. After you solve that the other

questions do come to play. Uh, do you

have a channel to get to customers? What

is pricing long-term? What is your moat?

I find that moes tend to be overhyped.

Actually, I find that more businesses

tend to start off with a product and

then evolve eventually into a moat. But

consumer products rand is somewhat more

defensible. Um, and if you have a lot of

momentum, it becomes harder to catch

you. But enterprise products sometimes

if you have a uh maybe mo is more of a

consideration if they're channels that

are hard to get into enterprises. So I

think um sorry when when AI fund looks

at businesses we actually wind up doing

a fairly complex analysis of these

factors and writing a you know two to

six page narrative memo to analyze it

before we decide whether or not to

proceed it or not. And and I think um uh

all of these things are important, but I

feel like at this moment in time, the

number of opportunities, meaning the

amount of stuff that is possible that no

one's built yet in the world, seems much

greater than the number of people with

the skill to build them. So definitely

at the application layer, it feels like

there's a lot of white space for new

things you can build that no one else

seems to be working on. And I would say,

you know, focus on building a product

that people want, that people love. Um,

and then figure out the rest of it along

the way. Although this important figure

along the way.

Uh, hi professor. Uh, thanks for your

wonderful speech. Uh, I am a

Andagramress researcher from Stanford

and I think your uh, metaphor in your

speech is very interesting. You said the

uh, current AI tools are like bricks and

are be uh, and can be built upon

accumulation. However, so far it is

difficult to see the accumulative

functional expansion of the uh

integration of AI tools because they

often align on the stacking of functions

based on uh intent distribution and are

accompanied by dynamic problems of

tokens and time overhead. So um which is

uh which is different from static

engineering. So what do you think will

be the perspective of a possible agent 2

accumulation effect in the future? But

hey just just some quick remarks to that

right you mentioned agent uh OM token

cost my most common advice to developers

is to first approximation just don't

worry about how much tokens cost only a

small number of startups are lucky

enough to have users use so much of your

product that the cost of tokens becomes

a problem it can become a problem I've

definitely been on a bunch of teams

where the cost you know users like our

product and we started to look at our

right geni uh uh bills and it was

definitely climbing in a way that really

became a problem. But it's actually

really difficult to get to a point where

your token usage costs are a problem.

And for the teams I'm on where we were

lucky enough that users made our token

cause a problem, we often had engine

solutions to then bend the cursor and

bring it back down through prompting

fine-tuning USDs by optimize or

whatever. And then what I'm seeing is

that I'm seeing a lot of agentic

workflows that actually integrate a lot

of different steps. So for example, if

you build a customer service chatbot,

we'll often have to use prompting, maybe

optimize some of the results in DSPI,

build evals, build guard rails, maybe

the customer service chatbot needs rag

up part of the way to get information to

feedback to the user. So I actually do

see these things grow. But one tip for

many of you as well is I will often

architect my software to make switching

between different building block

providers relatively easy. So for

example, um have a lot of products that

build on top of OM, but sometimes you

point to a specific product and ask me

which OM are we using? I honestly don't

know because we built up evals and when

there's a new model that's released,

we'll quickly run evals to see if the

new model is better than the old one.

And then you'll just switch to the new

model if the new model does better on

evals. And so the model we use week by

week, you know, sometimes our engines

will change it without even bothering to

tell me because the eval show the new

model works better. So it turns out that

switching cost for foundation models is

relatively low and we often architect

our software. Oh, AI suite is open

sourcing that my friends and I worked on

to make switching easier. Um, switching

cost for the orchestration platforms is

a little bit harder. Uh but I find that

preserving that flexibility in your

choice of building blocks often let you

go faster even as you're building more

and more things on top of each other. Um

so hope that helps.

Thank you so much.

In the world of education in AI, there

are two paradigms mostly. So one is AI

can make teachers more productive. Uh

automating grading and automating

homeworks. But another school of thought

is that there'll be personal tutors for

every student. So every student can have

one tutor that gets feedback from an AI

and gets personal questions from them.

So how do you see these two paradigms

converge and how would education look

like in the next five years?

I think everyone feels like a change is

coming in edtech but I don't think the

disruption is here yet. I think a lot of

people are experimenting with different

things. So you know Corsera has Corsera

coach which actually works really well.

Um deep learn is more focused on

teaching AI also has some built-in chat

bots. Um a lot of teams experiment of

autograding. Oh there's an avatar with

me on the deep learn website you can

talk to if you want. Uh deep learn.ai.

And then I think for some things like

language learning with you know speak

Dolingo that has become clearer some of

the ways AI would transform it for the

broader educational landscape the exact

ways that AI would transform it I see a

lot of experimentation I think what key

learning which I've been doing some work

with is doing is is very promising for

K12 education but I think uh what I'm

seeing is frankly tons of

experimentation but the final end state

is still not clear. I do think education

will be hyperpersonalized. Uh but that

workflow is an avatar, is a text

chatbot, what's the workflow? I think um

I feel like the hype from a couple years

ago that with AGI soon and it will be

all so easy. That was hype. The reality

is work is complex, right? teachers,

students, people do really complex

workflows and for the next decade we'll

be looking at the work that needs to be

done and figuring out how to map it to

agentic workflows and education is one

of the sectors where this mapping is

still underway but it's not yet mature

enough to the point where the end state

is clear. So I think I think we should

all yeah just keep working on it.

All right. All right. Thank you so much

Andrew.

Thank you. Uh hey my question is I think

AI uh it has a lot of great potential

for good but there's also a lot of

potential for bad consequences as well

such as exacerbating economic inequality

and things like that and I think a lot

of our startups here while they'll be

doing a lot of great things will also be

you know just by virtue of their product

be contributing to some of those

negative consequences. So I was curious

how do you think you know us as AI

builders should kind of balance our uh

product building with also the potential

societal downsides of some AI products

and essentially how can we uh both move

fast and be responsible as you mentioned

in your talk

look in your heart and if fundamentally

what you're building if you don't think

it'll make people at large better off

don't do it right I I know it sounds

simple but actually really hard to do in

the moment but AI fund we've killed

multiple projects projects not on

financial grounds but on ethical grounds

where there are multiple projects we

looked at the economic case is very

solid but we said you know what we don't

want this exist in the world and we just

killed it on that basis so I hope more

people will do that and then I worry

about uh bring everyone with us one

thing I'm seeing is um people in all

sorts of job roles that are not

engineering are much more productive if

they know AI than if they don't and so

for example on my marketing team my

marketers they know how the code.

Frankly, they they were running circles

around the ones that don't. So, then

everyone learned to code and then they

got better. But I feel like um trying to

bring everyone with us to make sure

everyone is empowered to build with AI.

That'll be an important part of what all

of us do, I think.

Um I'm one of your big fans and thank

you for your online courses. Your

courses make the deep learning like uh

much more accessible to the world. And

my question is also about education. uh

as AI becomes more powerful and

widespread, there seems to be a growing

gap between what can actually do and

what people perceive it. So what do you

think about like is it important to

educate the general public about deep

learning stuff and not only like uh

educate those technical people and make

people understand more what really uh

what AI really do and how it works.

I think that knowledge will diffuse deep

learn AI we want to empower everyone to

build with AI. So we're working on it.

Many of us work on it. I'll just tell

you what I think is the main d I think

there are maybe two dangers. One is if

you don't bring people with us fast

enough, I hope we'll solve that. There's

one other danger which is um it turns

out that if you look at the mobile

ecosystem, mobile phones, it's actually

not that interesting. And one of the

reasons is there are two gatekeepers,

Android and iOS. And unless they let you

do certain things, you're not allowed to

try certain things on mobile. And I

think this, you know, hampers

innovators. These dangers of AI have

been used by certain businesses. They're

trying to shut down open source because

a number of businesses that love to be a

gatekeeper to large scale foundation

models. So I think hyping up dangers,

supposed false dangers of AI in order to

get regulators to pass laws like the

proposed SP 1047 in California, which

thank goodness we shut down, would have

put in place really burdensome regry

requirements that don't make anyone

safer, but would make it really

difficult for TS to release open source

and open weight software. So one of the

dangers to inequality as well is if

these regulatory you know awful regry

approaches and I've been in the room

where some of these businesses said

stuff to regulators that was just not

true. So I think that um some of these

arguments the danger is if these

regulatory proposals succeed and end up

siphoning regulations leaving us with a

small number of gatekeepers where

everyone needs the permission of a small

number of companies to fine-tune the

model prompt in a certain way that's

what will cipher innovation and prevent

the diffusion of this information to let

lots of startups you know build whatever

they want responsibly but the freedom to

innovate so I think so long as we um

prevent this line of attack on open

source open weight models from

succeeding and we we've made good

progress but the threat is still there

then I think eventually we'll get to the

diffusion of knowledge and we can

hopefully then bring everyone with us

but this fight to protect open source

we've been winning but the fight is

still on and we still have to keep up

that work to to protect open source

thank you all very much it's wonderful

to see my thank you