Anthropic CEO Dario Amodei: AI's Potential, OpenAI Rivalry, GenAI Business, Doomerism

I get very angry when people call me a

doomer. When, you know, when when

someone's like, "This guy's a doomer. He

wants to slow things down." You you

heard what I just said, like, you know,

my my father died because of, you know,

cures that, you know, could have could

have happened a few years later. I

understand the benefit of this

technology. I'm sure you've heard the

criticism from people like Jensen who

say, "Well, Daario thinks he's the only

one who can build this safely and

therefore wants to control the entire

industry."

I've never said I've never said anything

like that. That's an outrageous lie.

That's the most outrageous lie I've ever

heard.

Anthropic CEO Dario Ammoday joins us to

talk about the path forward for

artificial intelligence. Whether

generative AI is a good business and to

fire back at those who call him a

doomer. And he's here with us in studio

at Anthropic headquarters in San

Francisco. Dario, it's great to see you

again. Welcome to the show.

Thank you for having me.

So, let's recap the past couple months

for you. Uh, you said AI could wipe out

half of entry-level white collar jobs.

You cut off Windsurf's access to

Anthropic's top tier models when you

learned that OpenAI was going to acquire

them. You asked the government for

export controls and annoyed Nvidia CEO

Jensen Wong. Uh, what's gotten into you?

Um, you know, I think I think Anthropic,

myself and Anthropic are always focused

on kind of trying to do and say the

things that we believe. Um uh and I

think as we've gotten uh more close to

AI systems that are more powerful um you

know I think I've wanted to say those

things um you know more uh more

forcefully more publicly to make the

point clearer. Um you know I've been

saying for many years that you know we

have these we can talk in detail about

them but you know we have these scaling

laws. AI systems are getting more

powerful. They're going from the level

of, you know, a a few years ago they

were barely coherent now, you know, a

couple years ago they were at the level

of a smart high school student. Now

we're getting to smart college student,

PhD, and they're starting to they're

starting to apply across the economy.

And so I think all the issues related to

AI ranging from kind of the national

security issues to the economic issues

um you know are are are starting to

become quite near to where to where we

um you know to where we're actually

going to face them. And so and so I

think as these problems have come closer

I've you know even though you know in

some form anthropic has been saying

these things for a while I think the

urgency of these things has gone up and

and and and and you know I I I want to

make sure that we uh you know I want to

make sure that we say what we believe

and that we warn the world about the

possible downsides even though you know

no one can say what's going to happen.

we're, you know, we're saying what we,

you know, what we think might happen,

what we think is likely to happen. You

know, we we back it up as as best we

can, although it's often, you know,

extrapolations about the future where

where no one where no one can be sure.

Um, but, you know, I think we we see

ourselves as kind of as kind of having a

duty to, you know, to kind of warn the

world about what's going to happen. And

that's not to say, you know, I think

there's an incredible number of like

positive applications of AI, right? I've

I've kind of continued to talk about

that. I read this I wrote this essay,

Machines of Loving Grace. Um I I I feel

in fact that I and anthropic have often

been able to do a better job of

articulating the benefits of AI than

some of the people who call themselves

optimists or accelerationists. Um so I

think we probably appreciate the

benefits more than more than anyone. Um,

but for exactly the same reason, because

we can have such a good world if we get

everything right, I feel obligated to

warn about the risks.

So all of this is coming from your

timeline. Basically, you it seems like

you have a shorter timeline than most

and so you were feeling a sense of

urgency to get out there because you

think that this is imminent.

Yes, I'm not sure. Um, you know, I think

it's very hard to predict particularly

on the societal side. So if you say you

know when are people going to deploy AI

or when are companies going to use you

know XX X dollars of spend of AI or you

know when will when will AI um you know

be be be used in these applications or

when will it drive these medical cures

that's kind of harder harder to say I

think the underlying technology is more

predictable but still uncertain still no

one knows but I think on the underlying

technology I've started to become more

confident. There isn't no uncertainty

about it. You know, I think the the

exponential that we're on could could

kind of still um you know, could totally

peter out. You know, I think there's

maybe uh I don't know 20 or 25% chance

that sometime in the next 2 years the

models just start getting stop getting

better for reasons we don't understand

or maybe reasons we do understand like

you know data or compute availability

and then everything I'm saying just just

seems seems totally silly and everyone

makes fun of me for all the warnings

I've made and and you know I'm just I'm

just totally fine with that given given

the distribution that that that given

the distribution that I see

and so I should say that this is part

our conversation as part of a profile

I'm writing about you. I've spoken with

more than two dozen people who've worked

with you, who know you, who've competed

with you, and I'm going to link that in

the show notes if anybody wants to read

it. It's free to read. Uh, but one of

the themes that has uh come through

across everybody I've spoken with is

that you have about the shortest

timeline of any of the major lab leaders

and you just referenced it uh just just

now. So, why do you have such a short

timeline and why should we believe in

yours? Yeah, it it really depends what

you mean by timeline. Um, so one thing

and you know I've I've I've been

consistent on this over the years is you

know there are these terms in the AI

world like AGI and super intelligence.

Like you'll hear leaders of companies

say we've achieved AGI, we're moving on

to super intelligence or like it's

really exciting that someone stopped

working on AGI and started working on

super intelligence. So I think these

terms are totally meaningless. I don't

know what AGI is. I don't know what

super intelligence is. It it sounds like

a it sounds like a marketing term. Yeah,

it sounds like, you know, something

something designed to activate people's

people's dopamine. So, you'll see in

public I never use those terms and uh I

um you know, I'm I'm actually, you know,

careful to criticize the use of those

terms. Um but I think I think despite

that, I am I am indeed one of the most

bullish about about AI capabilities

improving very fast. The thing I think

is real that I've said over and over

again is the exponential. The idea that

every few months we get an AI model that

is better than the AI model we got

before. And that we we get that by

investing more compute in AI models,

more data, more new types of training

models. Initially, this was done by

what's called pre-training, which is

when you just feed a bunch of data from

the internet into the model. Now we have

a second stage that's reinforcement

learning or test time compute or

reasoning or whatever you want to call

it. I I think of it as a second stage

that involves reinforcement learning.

Now both of those things are scaling up

together as we've seen with our models

and as we've seen with models from other

from other companies and I don't see

anything blocking that the further

scaling of that. There's some stuff

about you know how do we broaden the

tasks on the RL side of it. we've seen

uh more progress on say math and code

where where the models are you know

getting pretty close to like a high

professional level and less on more

subjective tasks but I think that is

very much a temporary obstacle um uh so

when I look at it I see this exponential

and I say look people aren't very good

at making sense of exponentials right

like you know if something is doubling

every six months then uh you know two

years before it happens it looks like

it's only 1/16th of the way there and

and and so we are sitting here in the

middle of of 2025

um and the models are really starting to

explode in in in terms of the economy

right if you look at the capabilities of

the model they're starting to saturate

all the benchmarks if you look at

revenue and you know anthropics revenue

every year has grown 10x um uh every

year we're we're kind of conservative

and we say you know it can't grow 10x

this time you know I I I you know I

never I never assume anything and and

actually always am very conservative in

saying ah I think it's going to slow

down on the business side but we went

from zero to 100 million in 2023 we went

from 100 million to a billion in 2024

and you know this year in this first

half of the year we've gone from 1

billion to you know I think as of as of

speaking today it's it's well above four

it might be 4.5 um uh and so if you

think about it you know suppose that

exponential continued for two years I'm

not saying it will but suppose it

continued for two years you know you're

you're like well into the hundred

billions. I'm not saying that'll happen.

I'm saying the situation is that when

you're on an exponential, you can really

get fooled by it. Two years away from

when the exponential goes totally crazy.

It it, you know, it looks like it's just

starting to be a thing. Um, and so

that's the fundamental dynamic. You

know, we saw that with the internet in

we saw that with the internet in the

'9s, right? where it was like you know

networking speeds and the underlying

speed of the computers were getting fast

and over a few years it became possible

to have to basically build a digital

global communications network on top of

all this when it wasn't possible just a

few years ago and and almost no one

except for a few people really saw the

implications of that and how fast it

would happen and so that's that's where

I'm coming from that that's what I think

now I don't know like if a bunch of

satellites crashed maybe the internet

would have taken longer. If there was an

economic crash, maybe it would have

taken a little longer. So, we can't be

sure of the exact timelines, but I think

people are getting fooled by the

exponential and and not realizing how

fast it it might be. How fast I think it

probably will, although I'm not sure.

But so many folks in the AI industry are

talking about diminishing returns from

scaling. Now, that really doesn't fit

with the vision you just laid out. Are

they wrong?

Yeah. uh I I from what we've seen I can

only speak in terms of the models at

enthropic um but what I think seen in

terms of the models at enthropic if we

look at you know let's take coding

coding is one area where you know I

think anthropic models have advanced

very quickly adoption has been very

quick we're not just a coding company

we're planning to expand to many areas

but if you if you look at if you look at

coding um you know every you know we

release 3.5 sonnet a model we call 3.5

sonnet V2 um uh which I you know let's

call it 3.6 Sonnet now um 3.7 sonnet uh

and then 4.0 sonnet and 4.0 obus and you

know that series of four or five models

each one got substantially better at

coding than than the last. If you want

to look at benchmarks, you can look at,

you know, SweetBench growing from uh,

you know, I think 18 months ago was at

like 3% or something um, growing all the

way to, you know, 72 to 80% depending on

how you how you measure it and and the

real usage has grown grown exponentially

as well where we're heading more and

more towards autonomously you can just

use these models. I think the actual

majority of code um uh uh that's written

written at Enthropic uh is you know at

this at this point uh written by or at

least with the involvement of one you

know one of the clawed models um and

various uh various other companies have

have said you know have said you know

similar similar statements to uh similar

statements to that. So we see the

progress as being very fast and the

exponential is continuing and and we

don't see any diminishing returns.

But there are some liabilities it seems

like with large language models. For

instance, continual learning. We had

Dark Kesh on a couple weeks ago. Here's

how he put it and he wrote about it in

his Substack. The lack of continual

learning is a huge huge problems. The LM

baseline at many tasks might be higher

than an average human, but you're stuck

with the abilities you get out of the

box. So you just make the model and

that's it. It doesn't learn. that seems

like a glaring liability. What do you

think about that?

So, first of all, I would say even if we

never solved continual learning, um even

if we never solve continual learning and

memory, um I think that the potential

for the LLMs to do, you know, incredibly

uh incredibly well to, you know, affect

things at the scale of the economy will

be very high. Right? If I think of the

field I used to be in biology and

medicine like you know let's say I had a

very smart Nobel prize winner and you

know I I I said okay you know you're you

know you've you you've discovered all

these things you have this incredibly

smart mind but you know you can't you

can't you know you can't like read new

textbooks or absorb any new information.

I mean that would be difficult but like

still if you had like 10 million of

those like they're they're still going

to make a lot of biology breakthroughs.

Like they're going to be limited.

they're going to be able to do some

things humans can't and there are some

things humans can do that they can't.

But but but even that even if we impose

that as a ceiling like man that's pretty

damned impressive and transformative and

even if I said you never solve that like

I think you know I think people are

underestimating the impact but but look

context windows are getting longer and

models actually do learn during the

context window right so so as as I you

know talk to the model during the

context window I have a conversation it

absorbs information the underlying

weights of the model may not change but

but you know just just like I'm talking

to you here and we're having a

conversation and I listen to the things

you say and I I you know I think and and

I like respond to them. The models are

able to do that and and from a from an

from a machine learning perspective,

from an AI perspective, there's no

reason we can't make the context length

a 100red million words today, right?

Which is roughly what a human hears what

what a human hears in their lifetime. Um

uh there's no reason that we can't uh do

that. It's it's really inference

support. Um uh and so again even that

fills in many of the gaps not all the

gaps but it fills in many of the gaps

and then there are a number of things

like learning and memory that do allow

us to update the weights. So um you know

there there there are a number of things

around you know types of reinforcement

learning learning training you know you

know we used to many years ago talk

about inner loops and outer loops right

the inner loop is like I have some

episode and I learn some things in that

episode and I'm trying to optimize for

the lifetime of that episode and and

kind of the outer loop is is is the

agents learning over episodes. Um and so

I think maybe that inner loop outer loop

structure is a way to learn the

continual learning. One thing we learned

in AI is whenever it feels like there's

some fundamental obstacle like two years

ago we thought there was this

fundamental obstacle around uh around

reasoning turned out just to be just to

be RL you just train with RL and you let

the model write some stuff down you know

you you let the model write things down

to try and figure out objective math

problems um without being too specific

um you know I think and we already have

maybe some some you know some some

evidence to suggest that this is another

of those problems that is is not as

difficult as it seems that will fall to

scale plus a slightly different way of

thinking about things. Do you think your

obsession with scale might blind you to

some of the new techniques like Deis

Sabis says, you know, to get to AGI or

you might call it super powerful AGI,

whatever human level intelligence is

what we're all talking about. We might

need a couple new techniques for that to

happen. If you're

developing new techniques, we're

developing new techniques every day.

Okay.

Um uh you know, Claude is very good at

code and we you know, we don't really

talk externally that much about why why

Claude is so good at code.

Why is it so good at code? Um it

like I said we don't talk externally

about it.

Um uh uh uh so you know every new

version of claude that we make has you

know improvements to the architecture,

improvements to the data that we put

into it, improvements to the methods

that we use to train it. So we're

developing new techniques all the time.

Um new techniques are a part of every

you know model that we build. And you

know that's why we you know I've said

these things about like you know we're

trying to optimize for like talent

density as much as possible like you

need that talent density in order in

order to invent the new techniques. You

know, there's one thing that's been

hanging over this conversation, uh,

which is that maybe Anthropic is the

company with the right idea, uh, but the

wrong resources. Because you look at

what's happening with XAI and, um, and

inside Meta where, uh, Elon's built his

massive cluster, Mark Zuckerberg is

building this 5 gawatt data center and

they are putting so much resources

towards scaling up. Um, is it possible?

I mean anthropic obviously you have

raised billions of dollars but these are

trillion dollar companies.

Yeah. So we've raised I think at this

point a little short of $20 billion.

It's not bad.

So that's that's not that's not nothing.

And I would also say if you look at the

size of the data centers that we're

building with for example Amazon. Um I I

don't think our data center scaling is

substantially smaller than that of any

of the other companies in the space. Um

you know in many cases these things are

limited by energy. they're limited by

capitalization. Um, you know, when when

when when when people talk about, you

know, these large amounts of money,

they're they're talking about it over

over over several years, right? And when

you hear some of some of these

announcements, sometimes they're not

funded yet. They're, you know, we've

we've seen the size of the of the of the

data centers that folks are building and

we're we're actually we're actually

pretty confident that, you know, we're

we will be within we will be within a

rough range of the size of data centers

they build. You talked about talent

density. What do you think about what

Mark Zuckerberg is doing on the talent

density front? I mean, combining that

with these massive data centers, it

seems like he's going to be able to

compete.

Yeah. So, uh this is this is actually

very interesting because um you know,

one thing we noticed is that relative to

other uh companies um you know, I think

I think I think uh very you know, a lot

fewer people from Enthropic uh have been

have been caught by these. And it's not

for lack of trying. I've talked to

plenty of people uh you know who who got

these offers at Enthropic and and who

just turned them down. um who wouldn't

even talk to Mark Zuckerberg who said um

you know uh uh no I'm I'm I'm staying at

Anthropic and and our our general

response to this was you know I posted

something to the to the whole company

Slack where where where I said look um

you know we are not willing to

compromise our you know our compensation

principles our principles of fairness to

respond individually to these offers.

The way things work at Enthropic is

there's a series of levels. One

candidate comes in, they get assigned a

level, and we don't negotiate that

level. Um, uh, uh, because because we

think it's unfair. We want to have a

systematic way. If, you know, if Mark

Zuckerberg, you know, throws a dart at a

dart board and hits your name, that

doesn't mean that you should be paid 10

times more than the guy next to you

who's who's, you know, who's who's just

as skilled, who's just as talented. Um

uh and and and and my view of the

situation is that you know the only way

you can really be hurt by this is if you

allow it to destroy the culture of your

company by panicking by treating people

unfairly in an attempt to to defend the

company. Um and I think actually this

was a unifying moment for the company

where um you know we we we didn't give

in. we refused to compromise our

principles because we had the confidence

that people are enthropic because they

truly believe in the mission. Um uh and

and you know I think um I I that that

gets to kind of how I see this. I think

that what they are doing is trying to

buy something that cannot be bought. Um

and and that is alignment with the

mission. Um uh and you know I I you know

I you know I think there are selection

effects here like I you know I you know

are they getting the people who are most

enthusiastic who are most missional

aligned who are most excited to

but they have talent and GPUs you're not

underestimating them.

I I

we'll see how it we'll see how it plays

out. Um I am pretty bearish on what

they're trying to do.

So let's talk a little bit about your

business because a lot of people have

been wondering is the business of

generative AI a real thing? And I'm also

curious. I have questions all the time.

You talked about how much money you've

raised, close to 20 billion. Um, you've

raised three billion, three billion from

Google, 8 billion from Amazon, three and

a half billion uh from a new round led

by Lightseed who I've spoken with. Um,

what what is your pitch? Because you are

not part of like a big tech company.

You're out there on your own. Do you

just bring the scaling laws and say,

"Can I have some money?" So my my view

of this has my view of this has always

been um that uh talent is the most

important thing. Um so you know if you

go back three years ago um you know we

were in a position where we had raised

mere hundreds of hundreds of millions.

Open AAI had already raised, you know,

13 billion from from Microsoft. And of

course, you know, the the large hyper

cap tech companies were sitting on 100

billion, $200 billion. And and basically

the pitch we made then is we know how to

make these models better than others do,

right? There may be a curve. There may

be a curve of scaling laws. But but

look, if we are in a position where we

can do for a hundred million what others

can do for a billion and we could do for

10 billion what they can do for a

hundred billion then it's 10 times more

capital efficient to invest in entropic

than it is to invest in in these other

companies. Would you rather be in a

position where you can do anything for

10 times cheaper or where you start with

a large pile of money? If you can do

things 10 times cheaper, the the you

know the the the the the money is a

temporary defect that that that that you

can remedy. If you have this intrinsic

ability to build things for the same

price, much better than anyone else or

as good as anyone else for much lower

price. You know, investors aren't aren't

idiots or at least they aren't always

idiots. Um

depends which one you go to.

I'm not going to name any names. um uh

uh uh but um uh uh you know they they

basically understand the concept of

capital efficiency. Um and so we've been

in a position you know 3 years ago where

you know these differences were like a

thousandx and now you're saying you know

with $20 billion can you compete can you

compete with hundred billion um and and

and my answer is basically yes because

of the talent density. You know, I've

said this before, but uh you know, the

the the Anthropic is actually the

fastest growing software company in

history at the scale that it's at. Um so

we grew from zero to 100 million in

2023, 100 million to billion in 2024.

And this year we've we've grown from 1

billion to I think I said this before

4.5. So that that 10x a year I mean you

know every every you know every year I

like suspect that we'll grow at that

scale and and every year I'm almost

afraid to say it publicly because I'm

like no it couldn't it couldn't possibly

happen again. So you know I think I

think the growth at that scale like kind

of speaks for itself in terms of our

ability to compete with the big players.

Okay. So CNBC says 60 to 75% of

anthropic sales are through the API.

That was according to internal

documents. Is that still accurate? Um I

won't give exact numbers but the

majority does come through the API

although we also do have a flourishing

apps business and you know I think more

recently um the you know max tier which

power users use as well as claude code

which which coders use. So you know I

think we have a we have a thriving and

fast growing apps business but yes the

majority comes through the API. So

you're making the most pure bet on this

technology like you know OpenAI might be

betting on chat GPT and Google might be

betting on the fact that no matter where

the technology goes it can you know

integrate into Gmail and calendar. So

why have you made this bet on this the

pure bet on the tech itself?

Yeah I mean I would say I wouldn't quite

put it that way. I think we've I would

describe it more as we've bet on

business use cases of the model um more

so than we've bet on the API per se. And

it's just that the first business use

cases of the model come through the API.

So you know as you mentioned OpenAI is

very focused on the consumer side.

Google is very focused on kind of the

existing products that Google has. Our

view is that if anything the enterprise

use of AI is going to be greater even

than the consumer use of AI or I should

say the business use because it's

enterprise, it's startups, it's

developers and it's kind of you know

power users using the model model for

productivity. Um I I also think that uh

being a company that's focused on the

business use cases actually gives us

better incentives to make the models

better. Um a a thought a thought

experiment that I think is worth running

is you know suppose I have this model

and it's uh it's um you know it's it's

as good as an undergrad at biochemistry.

Um and then I improve it and it's as

good as a PhD student at at

biochemistry. If I go to a consumer,

right, if I give them the chatbot and I

say, "Great news. I've improved the

model from, you know, undergrad to

graduate level in in biochemistry, um,

you know, maybe I don't know 1% of

consumers care about that at all, right?

99% are just going to be like, I don't

understand it either way." Um, but now

suppose I go to Fizer and and I say, you

know, I've improved this from

undergraduate biochemistry to to

graduate biochemistry. Like, this is

going to be the biggest deal in the

world, right? They might pay 10 times

more for something like that. it might

have 10 times more value to them. And so

the general aim of making the models

solve the problems of the world to make

them smarter and smarter but also able

to to bring many of the positive

applications right the things I wrote

about in like machines of loving grace

of like solving the problems of biio

medicine solving the problems of

geopolitics solving the problems of you

know of economic development um you know

as well as more prosaic things like

finance or legal or productivity or

insurance. Um I think it gives a better

incentive to kind of develop the models

um as as far as uh as as as as far as

possible and I think in many ways it's

like it may even be a more a more a more

positive business. So I would say we're

making a bet on the business use of AI

because it's most aligned with kind of

the exponential.

Okay. Then briefly, how did you decide

to go with the coding use case?

Yes. So um you know uh originally as

happens with most things we're trying to

optimize for making the model better at

a bunch of stuff and you know coding

particularly stood out in in terms of

how how valuable it was. You know I've

worked with thousands of engineers and

there was a point about a year year and

a half ago where one of the best I'd

ever worked with said um uh you know

every previous coding model has been

useless to me and and this one finally

was able to do something I wasn't able

to do. And then after we released it, it

started getting quick adoption. This was

around the time that you know a lot of

the coding companies like Cursor,

Windsurf, GitHub, Augment Code started

started exploding in in popularity and

then when we saw how popular it was, we

kind of doubled doubled down on it. Um

my view is that coding is particularly

interesting because a the adoption is

fast um and and b getting better at

coding with the models actually helps

you to develop the next model. Um, so it

has it has a number of uh you know you

know I would say advantages

and now you're selling uh your AI coding

through claude code. Um but it's very

interesting the pricing model um has

been confounding to some. You can spend

$200 a month and get the equivalent I

spoke to one developer they got the

equivalent of $6,000 a month uh from

your API. Um, Ed Zitetron has pointed

out the more popular that your models

get, the more money you're going to lose

if people are super users of this

technology. So, how does that make

sense? So, um, uh, uh, so actually

pricing schemes and rate limits are

surprisingly complicated. Um so so some

of this is basically the result of when

we released our um uh when we when we

released claude code and the max tier

which we eventually tied together

actually not fully understanding the

implications of you know the ways in

which people could use the models and

how much they were actually able to get.

So over the last few days, as of the

time of this uh as of the time of this

interview, we've adjusted that

particularly on the larger models like

Opus, I think it's no longer possible to

spend that much um uh uh with a with a

with a $200 uh uh subscription. And you

know, it's possible more changes will

more changes will come in the future,

but we're always going to have a

distribution of users who use a lot and

and and users who lose who use some

amount. And it it doesn't necessarily

mean we're losing money that there are

some some users who get more um uh you

know who if you were to measure via API

credits spend you know get get a better

deal on the consumer on on the consumer

subscription than they would on the API

products. Right? There's there's a lot

of assumptions there. Um and I can tell

you that that some that some of them

that some of them are wrong. Um uh we

are not in fact uh losing money. But I

guess there's another question about

whether you can continue to serve these

use cases um and not raise prices. So uh

just to give you a couple stats, there

are some developers that are upset

because uh using Anthropic's newer

models in cursor is costing them more

than it ever has. U startups that I've

spoken with say Anthropic is down a much

down a bunch because they can't get

access to the GPUs. At least that's what

they um what they imagine is happening.

And I was just with Amjad Masad at

Replet um in an interview that we're

going to air next week who said there

was a period of time where the price per

token price per to use these models was

coming down and it stopped coming down.

So is it the is what's happening that

these models are just so expensive for

anthropic to run that it's hitting it's

a wall of its own

again I think you're I think you're

making assumptions here.

That's why I'm asking the CEO. Uh yeah

um uh you know uh you know the the the

way I think about it is we think about

the models in terms of how much value

are they creating right um so as the

models get better and better I think

about how much value they create and

there's a separate question about how

the value is distributed between those

who make the model those who make the

chips and and you know those who make

those who make the the the um the the

the the underlying applications. So

again, without being too specific, like

I think there are some assumptions in

your question that are not necessarily

correct. Um uh you know, I

tell me which ones

I I so I'll say this. I do expect the um

I I I expect the price of providing a

given level of intelligence to go down.

Um I expect the price of providing the

frontier of intelligence which will

which will provide kind of increasing

economic value that might go up or it

might go down. My guess is it probably

stays about where it is. But again the

value that's created you know goes way

up. So two two years from now my guess

is that we'll have models that cost of

the same order of magnitude that they

cost today except they'll be much more

capable of doing work much more

autonomously much more broadly than they

are capable of today. One of the things

that Amjad mentioned was he thinks that

the bigger models are not as intensive

to run or more intensive to run given

their size because of the architecture

and some of these techniques that we

talked about that they're lighting up

only certain sections of the model. So

his idea I'm hopefully conveying this

truthfully is that um Anthropic can run

these models without too much bulk on

the back end but is still keeping those

prices uh where they are. And I think

the the line that I'm going to draw

there is maybe that um to get to

software margins, there were some

reports that Anthropic is slightly below

software gross margins. You're going to

have to charge a little bit more for

these models.

So, um yeah, again, um I think larger

models cost more to run than smaller

models. Um uh uh you know, I think the

technique you're referring to is maybe

mixture of experts or something like

that. So whether your models are mixture

of experts or not, like mixture of

experts is is like a way to run models

more cheaply that have a given number of

parameters. It's a way to train models.

But if you're not using that technique,

then larger models that don't use that

technique cost more to run than smaller

models that don't use that technique.

And if you're using that technique,

larger models that use that technique

cost, you know, more to run than than

smaller models that that that are using

that technique. So I think I think

that's sort of a distortion of the I

think that's sort of a distortion of the

situation. Um

basically I'm I'm just guessing and I'm

trying to find out what the truth is

from you. So

yeah look so I um you know in in in in

terms of like the cost of the models

like one thing you'd be surprised by

people you know people kind of impute

this thing to like oh man it's going to

be really hard to get the margins from

like x% to y%. We make improvements all

the time that make the models like 50%

more efficient than they are before. We

are just the beginning of optimizing

inference. Um uh inference has improved

a huge amount where from from where it

was a couple a couple years ago to where

it is now. That's why the prices are

coming down.

And then how long is it going to take to

be profitable? Because I think the loss

is going to be like three billion this

year. That's what they would distinguish

different things. Okay.

Um there's the cost of running there's

the cost of running the model, right? So

so for every dollar the model makes it

costs a certain amount. Um that is

actually already fairly profitable. Um

there are separate things. There's you

know the cost of paying people um and

like buildings that is actually not that

that large in the scheme of things. The

big cost is the cost of training the

next model. Um, and I think this idea of

like the companies losing money and not

being profitable is it's a little bit

misleading. Um, and you start to

understand it better when you when you

look at the scaling laws. So, as a

thought exercise, these numbers are not

exact or even close for entropic. Let's

imagine that in 2023 you train a model

that costs $und00 million. Um, and then

in 2024, you deploy the 2023 model and

it makes $200 million in revenue, but

you spend a billion dollars to train um,

you know, to train a new model in 2024.

So, and then, you know, and then in

2025, the billion dollar model makes two

billion in revenue and you spend 10

billion to train the next model. Um, so

the company every year is unprofitable.

It lost 800 million in 2024 and then 20

2025 it lost$8 billion. Um so you know

this looks like a hugely unprofitable

enterprise but if instead I think in

terms of is each model profitable right

think of each model as a venture. Um I

invested 100 million in the model and

then I got then I got then I got 200

million out of the model the next year.

So that model had 50% margins and and

you know and like and like made me

hundred $100 million the next year. um

you know the the the company invested a

billion dollars and made and made $2

billion in in or sorry the next model

the company invested a billion dollar so

every model is profitable but the

company is unprofitable every year I'm

not I'm not this is this is this is a a

styliz I'm not like claiming these

numbers for anthropic or claiming these

facts for but this general philanthropic

this general dynamic is is this this

general dynamic is in general terms the

explanation for what is going on. And so

at you know you know at at any time if

the models stopped getting better or if

a company stopped investing in the next

model um you know you would you know you

would have probably a viable business

with the existing models but everyone is

investing in the next model and so

eventually it'll get it'll get to some

scale. the but the fact that we're

spending more to to to to this fact that

we're spending more to invest in in the

next model suggests that the the scale

of the business is going to be larger

the next year than it was than it was

the year before. Now, of course, what

could happen is like the models stop

getting better and there's this kind of

one-time cost that's like a boondoggle

and we spend we spend a bunch of money,

but then the the you know the the

companies the industry will kind of

return to this you know to this plateau

to this level of profitability or the

exponential can keep going. Um, so I

think I think that's a long-winded way

to say I don't think it's really the

right way to think about things,

right? But what about open source?

Because if you stopped, let's say you

stopped investing in the models and open

source caught up, then people could swap

in open source. Now, I'd love to hear

your perspective on this because one of

the things people have talked to me

about when it comes to the anthropic

business is there is that risk

eventually that open source gets good

enough that you can take anthropic out

and put open source in.

Yeah. So, you know, people have I you

know, I think one of the things that's

been true of this industry is that and

you know, I saw it early in I saw it

early in the history of AI. Every

community that that AI has gone through,

it has this set of heruristics about how

things work. like back when I was in you

know AI back in 2014 there was an

existing kind of AI and machine learning

research community that like thought

about things in a certain way and we're

like this is just a fad this is a new

thing this can't work this can't scale

and then because of the exponential all

those things turned out to be false then

a similar thing happened with kind of

like people deploying AI within

companies to various applications then

there was the same thought in the

startup ecosystem and I think now we're

at the phase days where kind of the

world's business leaders like the

investors and the business they have

this whole lexicon of commoditization

um you know modes which layer is the

value going to going to which layer is

the value going to acrew to and open

source is this idea that you can kind of

see everything that's going on you know

that that it has a significance that it

kind of undermines

um uh uh the the fact that it you know

the idea that it undermines business and

I actually find as someone who didn't

come from that world at all, who never

thought in terms of that lexicon. This

is one of these situations where not

knowing anything often leads you to make

better predictions than kind of the

people who have their way of thinking

about things from the last generation of

tech. Um, and I, you know, this is all,

I think, a long-winded way of saying I I

don't think open source works the same

way in in AI that it has worked in other

areas. primarily because with open

source you can you can see the you know

you can see the source code of the model

here we can't see inside the model um

you know it's often called open weights

instead of open source to kind of

distinguish that but a lot of the

benefits which is that many people can

work on it that it's kind of additive it

doesn't quite work in the same way um so

you know I've I've actually always seen

it as a red herring when I see it when I

see a new model come out I don't care

whether it's open source or not like if

we talk about deepseeek I don't think it

mattered that Deep Seek is open source.

I think I ask is it a good model? Is it

better than us at at you know the things

that that's the only thing that I care

about it. It actually it actually

doesn't doesn't matter either way. Um

because ultimately you have to you have

to host it on the cloud. The people who

host it on the cloud do inference. These

are big models. They're hard to do

inference on. And conversely, many of

the things that you can do when you see

the weights um uh uh you know, we're

increasingly offering on clouds where

you can fine-tune the model. You can you

know um you know we're even looking at

at ways to you know to to kind of you

know investigate the activations of the

model as part of like an

interpretability interface. We did some

little things around steering last time.

Um so I think it's the wrong axis to

think in terms of when I think about

competition I think about like which

models are good at the task that we do.

Um I think open source is actually a red

herring.

But if it's free and cheap to run

it's not free. You have to you have to

you have to you have to run it on

inference and someone someone has to

make it fast on inference.

All right. So I want to learn a little

bit more about Daario the person. Yes.

So we have a little bit of time left. Um

so I have some questions for you about

early life and then how you became who

you are. Yes.

So, what was it like growing up in San

Francisco?

Yeah. Um I, you know, the city when I

first grew up here had not really had

not really gentrified uh uh that much.

You know, when I grew up, the tech boom

hadn't hadn't happened, uh hadn't

happened yet. Um you know, it happened

as as I was going through high school.

And actually, I had no interest in it.

Um it was totally it was totally boring

to me. Um you know, I was interested in

being like a scientist. I was interested

in physics and math and you know the

idea of like you know you know like

writing some website actually had no

interest to me to me whatsoever or like

founding a company like those weren't

things that I was uh that I was

interested in at all. Um you know I was

interested in discovering fundamental

scientific truth and I was interested in

like you know how can I how can I do

something that like makes the world

better. Um uh so so you know that was

that was kind of more and you know I

watched the tech boom happen around me

but I I feel like you know there was all

kinds of things I probably could have

learned from it that would have been

helpful now but I just actually wasn't

paying attention and had no interest in

it even though I was like right at the

center of it.

So you were the son of a Jewish mother

Italian father that is true

from where I'm from in Long Island. We

call that a pizza bagel.

A pizza bagel. I've never I've never

heard that term before.

So what was your relationship with your

parents like? Yeah, I mean, you know, I

was I was always uh I was always I was

always pretty close with them. You know,

I feel like they gave me a sense of, you

know, of of kind of right and wrong and

what was important in the world. I feel

like, you know, kind of imbuing a strong

sense of responsibility is is maybe the

thing that I remember most. you know,

they were always people who felt that

sense of responsibility and, you know,

wanted to wanted to make the world um uh

wanted to make the world better. And and

I feel like, you know, that's one of the

one of the main things that I that I

learned from them. You know, it was

always a very a very um a very loving

family, a very caring family. I was very

close with my sister Daniela, who of

course became my became my became my

co-founder. And you know, I think we

decided very early that we wanted to

work together in some in some capacity.

I don't know if we imagined that it

would happen, you know, at quite the

scale that that that that it has

happened. Um uh but it um you know, I

think I think it really um you know,

that was that was something we kind of

decided early that we wanted to do.

The people that I've spoken with that

have known you through the years have

told me that your father's illness had a

big impact on you. Can you share a

little bit about that?

Um yes. Yes, he was. Um yeah, you know,

he was ill for a long time. Um and uh

eventually uh uh died in uh eventually

died in uh in in 2006. Um uh so that you

know that was actually one of the things

that drove me to you know I I don't

think we mentioned it yet in this

interview but before um you know before

I went into AI um you know I went into

biology. So, you know, I'd gone to uh I

I you know, I'd shown up at I'd shown up

at Princeton um uh wanting to be a

theoretical physicist and you know, I

did some did some work in in cosmology

for the first few few months of my time

there. um you know and and you know that

was that was around the time that that

my father died and you know that did

have an influence on me and kind of was

one of the things that convinced me um

you know to to go into biology you know

to try and address um uh uh uh you know

human illnesses and biological problems

and so I started talking to some of the

folks who worked on biohysics and

computational neuroscience in the

department that I was at at Princeton

and that was what led to the switch to

biology and computation neuroscience and

then you know of course after that I

eventually I eventually went into AI and

the reason I went into AI was actually a

continuation of that motivation which is

that um you know as I spent many years

in biology I realized that the

complexity of the

underlying problems in biology felt like

it was beyond human scale. you know, in

order to understand it all, you needed

hundreds, thousands of, you know, human

researchers and, you know, they often

had a hard time collaborating or sharing

their, you know, combining their their

knowledge. And AI, which was, I was just

starting to see the discoveries in it,

felt to me like the only technology that

could kind of bridge that gap, could

bring us beyond human scale to, you

know, to to fully understand and solve

the problems of biology. So, yeah, there

is a through line there,

right? And I could have this wrong. Uh

but one thing I heard was that his

illness was um largely unccurable when

he had it. And there have been advances

that have been Can you share a little

bit more? Yes. There are advances that

have made it much more manageable today.

Yes. Yes. That is uh that is uh that is

that that that is true. actually

actually only um uh uh only in the um uh

maybe 3 or four years after he died the

the cure rate for the disease that he

had went from uh went from 50% to uh uh

uh to roughly 95%.

Yeah. I mean it has to have felt so

unjust to have your father taken away by

something that could have been cured.

It it of course of course um but it also

tells you of the the urgency of solving

the relevant problems, right? that um

you know that that that you know there

there was someone who worked on the cure

to to this disease that you know managed

to cure it and save a bunch of people's

lives but you know could have could have

um saved even more people's lives if if

you know they they had managed to to

find that that to find that cure you

know a few years earlier than they did.

Um, and I think that's that's one of the

tensions here, right? That, you know,

um, I think AI has all of these

benefits. Um, and, you know, I want

everyone to get those benefits as soon

as possible. You know, I probably

understand, you know, better than almost

anyone how urgent those benefits are.

Um, and so I really understand the

stakes. You know, when I speak out about

AI has these risks and I'm worried about

these risks, I get very angry when

people call me a doomer. I got really

angry when, you know, when when

someone's like, "This guy's a doomer. He

wants to slow things down." You you

heard what I just said, like, you know,

my my father died because of, you know,

cures that, you know, could have could

have happened a few years later. I

understand the benefit of this

technology. When I sat down to to to

write machines of loving grace, you

know, I wrote out all the ways that

billions of people's lives could be

better with this technology. Some of

these people, some of these people who

on Twitter, you know, cheer for

acceleration. I don't think they have a

humanistic sense of the benefit of the

technology. Their their brain's just

full of adrenaline and and they're like

they want to cheer for something. They

want to accelerate. I don't get the

sense they care. Um and so when these

people call me a doomer, I think I think

they just completely completely lack any

moral credibility in doing that. Um uh

you know, it really makes me lose

respect for them. And I've been

wondering what this this word impact has

uh been because it's come up so often

that those who have been around you have

said you've been singularly obsessed

with having impact. In fact, I spoke

with someone who knew you well who said

you wouldn't watch Game of Thrones

because it wasn't tied to impact that it

was a waste of time and you wanted to be

focused on impact.

Actually, that's not quite right. I

wouldn't watch it because it was so

negative sum people were playing playing

such negative. It was like these people

start off and they're partly the

situation and partly because they're

they're just horrible people. They like

create this situation where at the end

of it everyone is like worse off than

everyone was before. Um I'm I'm really

I'm really excited about like creating

positive some situations. Um

I recommend you watch it. It's a great

it's a great show. But I hear I hear

some parts of it I was just very

reluctant and didn't watch it for a long

time.

Let's get back to the impact.

Let's get back to the impact. So that's

what impact is is effectively your

career has been this I this quest to

have that impact to be able tell me if

I'm going too far to prevent other

people from being in similar situations.

I you know I I I think I you know I

think I think that's a piece of it. I

mean you know I I have looked at you

know many uh you know many attempts to

help people and you know some of them

are more effective than others. Um, and

you know, I think I think I've always

tried to, you know, there should be

strategy behind it. There should be

brains behind um, you know, trying to

trying to help people. Um, you know,

which often means that there's a long

path to it, right? It can run through a

company and, you know, many activities

that are technical and not immediately

tied to the the kind of impact impact

that you're trying to have. But, you

know, the the the the arc is I'm always

trying to bend the arc towards that. I

think I think that's my that's my

picture of it. That's that's really why

I that's really why I got into this,

right? You know, um I think you know,

similar to the reason to get into AI was

that um you know, I I saw the problems

of biology as as almost intractable

without it or at least too slow moving.

Um, you know, I think my reason to start

a company was that I had worked at other

companies and I I I I just didn't feel

like the way those companies were were

run was was really oriented towards, you

know, trying trying trying to have that

impact. There was a story around it that

was often used for recruiting, but it

became clear to me over the years that

story was not sincere.

I'm going to circle around a little bit

because it's clear that you're referring

to OpenAI here. Um,

from what I understand, you had 50% of

OpenAI's compute. I mean, you ran the

GPT3 project. So, if anyone was going to

be focused on impact and safety,

wouldn't have been you.

Uh, yes. I I was, you know, there was a

period during which uh during which that

was uh that was true. That wasn't true

the entire time. That was, for example,

when we were scaling up GPT3. Um, yeah.

So, you know, I I when I was at OpenAI,

I and a lot of my colleagues, including

the people who, you know, eventually um

eventually founded Anthropic,

the pandas,

um the pandas. Um

that's the name you gave them.

I that that isn't a name I gave them.

The name they took.

Uh that isn't a name they took. Um

that's the name other people called

them.

Uh I I maybe it's a name other people

called them. Uh that's not a name I ever

used for my team.

Okay. Sorry. Go ahead.

Uh

that's good clarification. Thank you. Um

uh so uh yeah um you know we were

involved in scaling up these models

actually the original reason for

building GPT2 and GPT3 it was an

outgrowth of the kind of AI alignment

work that we were doing right where

myself and Paul Cristiano and some of

the anthropic co-founders had invented

this technique called RL from human

feedback um uh and that was designed to

help steer models in um uh you know in a

direction to follow human intent. It was

actually a precursor to you know we were

trying to scale up another method called

scalable supervision which I think is

just starting to to to work many years

later to help models follow more kind of

scalable uh uh uh human intent. But what

we found is even with the more primitive

technique RL from human feedback it

wasn't working with the small language

models with you know GPT1 that we

applied it to and that had had been

built by other people at OpenAI. So the

scaling up of GPT2 and GPT3 was done in

order to kind of study these techniques

in order to apply RL from human feedback

at scale. Um uh you know this goes to

one thing which is that I think in this

field the alignment of AI systems and

the capability of AI systems is

intertwined in this way that always ends

up being kind of more tied and more

intertwined than we think. Um actually

what this made me realize is that it's

very hard to work on the safety of AI

systems and the um capability of AI

systems separately. It's very hard to

work on one and not the other. I

actually think the value and the um way

to inflect the field in a more positive

way comes from organizational level

decisions. when to release things, when

to study things internally, what kind of

work to do on systems. Um, and that was

one of the things that kind of

motivated, you know, me and some of the

other, you know, to be entropic founders

to kind of go off and do it our own way.

But again like if you were driving if

you think language uh if you think

capabilities and safety are interlin and

you were the guy driving the cutting

edge models within open AI you know you

if you left you knew they were going to

be a company that was still doing this

stuff. That's right.

It seems like if you're driving the

capabilities you'd be the one in the

driver's seat to help it be safe the way

that you wanted to. Again, I will say um

you know if there's a decision on

releasing a model, if there's a decision

on the governance of the company, if

there's a decision on you know how the

personnel of the company works, um you

know, how the company represents itself

externally, the decisions that the

company makes with respect to

deployment, the claims it makes about

how it operates with respect to society.

um you know many of those things are not

things that you control just by just by

uh training the model and you know I

think I think trust is really important

I think the leaders of the company of a

company they have to be trustworthy

people they have to be people whose

motivations are sincere no matter how

much you're driving the the forward the

company technically if you're working

for someone whose motivations are not

sincere who's not an honest person who

does not truly want to make the world

better it's not going to work you're

just contributing to something bad.

So, and I'm I'm sure you've heard the

criticism from people like Jensen who

say, "Well, Daario thinks he's the only

one who can build this safely and

therefore, speaking of that word

control, wants to control the entire

industry."

I've never said I've never said anything

like that. That's an outrageous lie.

That's the most outrageous lie I've ever

heard. Um,

by the way, I'm sorry if I got Jensen's

words wrong, but

No, no, no. The words were correct.

Okay. the but but but but the words are

the words are outrageous. In fact,

I've said multiple times and I think

Anthropic's actions have shown it

that um

you know, we're aiming for something we

call a race to the top. Um you know,

I've said this on podcasts over the

years and I think anthropics actions

have shown it where you know with a race

to the bottom, right, everyone is

competing to like, you know, get things

out as fast as possible. And so I say

when you have a race to the bottom, it

doesn't matter who wins, everyone loses,

right? Because you make the unsafe

system that you know helps your

adversary or causes economic problems or

uh you know is is unsafe from an

alignment perspective. The way I think

about the race to the top is that um it

doesn't matter it doesn't matter who it

doesn't matter who wins, everyone wins,

right? So the way the race to the top

works is you set an example for how the

field works. to say um uh uh you know

we're going to engage in this practice.

So a key example of this is responsible

scaling policies. We were the first to

put out a responsible scaling policy and

you know we didn't say everyone else

should do this or you're bad guys. We

didn't you know we didn't you know kind

of try to use it at his advantage. We

put it out and then we encouraged

everyone else to do it. Um and many and

and then we discovered in the months

after that that you know there were

people within the other companies who

were trying to put out responsible

scaling policies but the fact that we

had done it allowed you know gave those

people permission right kind of kind of

enabled those people to um you know to

make the argument to leadership hey

anthropic is doing this so we should do

it as well. The same has been true of

investing in interpretability. we

release our interpretability research to

everyone um uh and allow other companies

to copy it even though we've seen that

it sometimes has commercial advantages

same with things like constitutional AI

same with the the the measure the the

the you know the measurement of the

dangers of our system dangerous

capabilities evals so we're trying to

set an example for the field but there's

an interplay where it helps to be a

powerful commercial competitor I've said

nothing that that any that anywhere ware

near resembles the idea that this

company should be the only one to build

the technology. I don't know how anyone

could ever derive that from anything

that I've said. Um it's it's it's it's

just Yeah. Yeah. It's just it's just

it's just a it's just an incredible and

bad faith distortion.

All right, let's see if we can

lightening around like one or two before

I ask you the last one, which we'll have

five minutes for. Um

what happened with SPF? Like

what happened with SPF?

I mean, he was one of Go ahead.

I couldn't tell you. I couldn't was what

was the what didn't you answer?

I I probably met the guy four or five

times.

Um uh so I have no great insight um into

the you know what in you know into the

psychology of SPF or or you know why why

he did things as stupid or immoral as as

as as as

he did. I think the only uh the only you

know the the only thing I had ever seen

ahead of time with uh SPF was uh you

know a couple people mentioned to me

that he was like hard to work with that

you know he was he was like a bit of a

move fast and break things guy.

Um and I was like okay you know there's

like plenty of people

Welcome to Silicon Valley.

Yeah. Like welcome welcome to Silicon

Valley. Um and so I remember saying okay

I'm going to give this guy non- voting

chairs. I'm not going to put him on the

board. He sounds like a bit, you know,

he sounds like a bad person to deal with

every day. Um, but, you know, he's

excited about AI. He's excited about AI

safety. He's, he's, you know, he's a

bull on on on AI and he's interested in

AI safety. So, you know, seems like a

seems like a sensible seems like a

sensible thing to do. you know in in uh

in in you know in like in in in in

retrospect um you know that that you

know move fast and and and break things

you know was turned out to be much much

much more extreme and bad than than you

know than than than I ever imagined.

Okay. So let's end here. So you found

your impact I mean you're you're working

the dream pretty much right now. I mean

think about all the ways that uh AI can

be used for biology uh just a start. You

also say that this is a dangerous

technology and I'm curious if your

desire for impact um could be pushing

you to accelerate this technology um

while you know potentially devaluing the

possibility that it could that

controlling it might not be feasible. So

you know I think I have more than anyone

else in the industry warned about the

dangers of the technology. Right? We

just spent 10 20 minutes talking about,

you know, the the frightening the, you

know, the large array of, you know,

people who run, you know, trillion

dollar companies criticizing me for, you

know, for talking about the the the

dangers of these technologies, right?

You know, I have US government

officials. I have people who run $4

trillion companies criticizing me for

talking about the dangers of the

technology, right? imputing all these

bizarre motives that bear no

relationship to, you know, to anything

I've ever said, not supported in

anything I've ever done. And yet, I'm

going to continue to do it. Um, I

actually think that, you know, as the

revenues, as the economic business of AI

ramps up, and it's ramping up

exponentially, you know, if if I'm

right, in a couple years, it'll be the

biggest source of revenue in the world,

right? It'll be the biggest industry in

the world. And people who run companies

already think it. So we actually have

this terrifying situation where uh you

know hundreds of billions to trillions

to I would say maybe 20 trillion of

capitals on the side of accelerate AI as

fast as possible. we have this, you

know, company that's very valuable in

absolute terms, but you know, looks very

small compared to that, right? 60 60 $60

60 billion. And I keep speaking up even

if, you know, it makes folks in, you

know, there have been these articles f

you know, some folks in the US

government are upset at us, for example,

for opposing the moratorium on AI

regulation, for being in favor of export

controls for chips on China, for talking

about the economic impacts of AI. Every

time I do that, I get attacked by many

of my peers.

Right. But you're still assuming that we

can control it. That's what I'm pointing

out.

But I'm I'm just I'm just telling you

how much how much effort, how much

persistence, how much despite everything

that stacked up, despite all the

dangers, despite the risk that it has to

the company of being willing to speak

up, I'm willing to do it. And and and

that's that's what that's that's why I'm

saying that look if if I thought that

there was no way to control the

technology, right? If I thought even

even if I thought this is just a gamble,

right? Some people are like, "Oh, you

think there's a five or 10% chance that

AI could go wrong, you're just rolling

the dice." That's not the way I think

about it. This is a multi-step game,

right? You take one step, you build the

next step of most powerful models, you

have a more intensive testing regime. As

we get closer and closer to the more

powerful models, I'm speaking up more

and more and I'm taking more and more

drastic actions because I'm concerned

that the risks of AI are getting closer

and closer. We're working to address

them. we've made a certain amount of

progress, but when I worry that the

progress that we've made on the risks

does not you know is not fully aligned

with the um uh you know is not going as

fast as we need to go for the speed of

the technology then I speed up then I

then I speak up louder. Um, and so you

know, you're asking why am I why am I

talk, you know, what, you know, you

started this interview by saying what's

gotten into you? Why are you talking

about this? It's because the exponential

is getting to the point that that I

worry that we may have a situation that

our ability to handle the risk is not

keeping up with the speed of the

technology. And that's how I'm

responding to it. If I believe that

there was no way to control a

technology, which I I I I I see

absolutely no evidence for that

proposition, we've gotten better at

controlling models with every model with

every model that we release, right? All

these things go wrong, but like you

really you really have to stress test

the models pretty hard. That doesn't

mean you can't have emergent bad

behavior. And I think, you know, if we

got to much more powerful models with

only the alignment techniques we have

now, then I'd be very concerned. then

I'd be out there saying everyone should

stop building these things. Even China

should stop building these. I don't

think they'd listen to me, which is one

reason I think export controls is a

better is is is is a better measure. But

if if we got a few years ahead in models

and had only the alignment and steering

techniques we had today, then you know,

I would definitely be advocating for us

to to, you know, to to to to slow down a

lot. The reason I'm warning about the

risk is so that we don't have to slow

down. So that we can invest in safety

techniques and can continue the progress

continue the progress of the field. It

would be a huge economic effort even if

one company was willing to slow down the

technology. You know that doesn't stop

all the other companies that doesn't

stop our geopolitical adversaries to

whom this is a existential fight fight

for survival. So, you know, there

there's there's there's very little, you

know, there's very little latitude here,

right? We're stuck between all the

benefits of the technology, the race to

the race to accelerate it and the fact

that that is a multi-party race. And so,

I am doing the best thing I can do,

which is to invest in safety technology

to speed to to speed up the progress of

safety. I've written essays on the

importance of interpretability on how

important various directions in uh in in

in safety are. We release all of our

safety work openly because we think

that's the thing that's a public good.

That's the thing that everyone that that

that that that everyone needs to share.

So if you have if you have a better

strategy for balancing the benefits, the

inevitability of the technology and the

risks that it face, I am very open to

hear it because I go to sleep every

night thinking about it because I have

such an incredible understanding of the

stakes in terms of in terms of the

benefits in terms of you know what it

can do, the lives that it can save. I've

seen that personally. I also have seen

the risks personally. We've already seen

things go wrong with the models. You

know, we have an example of that with

Grock. And you know, people dismiss

this, but they're not going to laugh

anymore when the models are taking

actions, when they're manufacturing, and

when they're in charge of, you know,

medical medical interventions, right?

People can laugh at the at the risks

when the models are just talking. But I

think it's very serious. And so I think

what this situation demands is a very

serious understanding of both the risks

and the benefits. These are highstakes

decisions. They need to be made with

they they they they need to be made with

a seriousness. And and I think something

that makes me very concerned is that on

one hand we have a cadre of people who

are just doomers. People call me a

doomer. I'm not. But there are doomers

out there. People who say they know

there's no way to build this safely. You

know, I I've I've looked at their

arguments. They're a bunch of

gobbledegook. The idea that these models

have dangers associated with them,

including dangers to humanity as a

whole, that makes sense to me. The idea

that we can kind of logically prove that

there's no way to make them safe, that

seems like nonsense to me. So, so I

think that is an intellectually and

morally unserious way to respond to the

situation we're in. I also think it is

intellectually and morally unserious for

people who are sitting on 20 trillion

dollars of capital who all work together

because their incentives are all in the

same way. There are dollar signs in all

of their eyes um to sit there and say we

shouldn't regulate this technology for

10 years. Anyone who says that we should

worry about the safety of these models

is someone who just wants to control the

technology themselves. That's an

outrageous claim and it's a morally

unserious claim. We've sat here and

we've done every possible piece of

research. We speak up when we believe

it's appropriate to do so. We've tried

to back up, you know, when we make

claims about the economic impact of AI.

We have an economic research council. We

have an we have a we have a economic

index that we use to track the model in

real time. And we're giving grants for

people to understand the economic the

economic impact the the economic impact

of the technology. I think for people

who are far more financially invested in

the success of the technology than than

than than I am to just you know breezily

lob add ad homonym attacks you know I

think that is just as intellectually and

morally unserious as the doomer's

position um I think what we need here is

we need more thoughtfulness we need more

honesty we need more people willing to

willing to go against their interest

willing to not have you know breezy

Twitter fights, uh, hot takes. We need

people to actually invest in

understanding the situation, actually do

the work, actually put out the research,

and and actually add some light and some

insight to to the situation that we're

in. I am trying to do that. I don't

think I'm doing that perfectly as no

human can. I'm trying to do it as well

as I can. It would be very helpful if

there were others who would try to do

the same thing. Well, Dario, I said this

off camera, but I want to make sure to

say it on as we're wrapping up. Um, I

appreciate how much Anthropic publishes.

We have learned a ton from the

experiments, everything from red teaming

the models to vending machine Claude,

which we didn't have a chance to speak

about today. Um, but I think the world

is better off just to hear everything

going on here. And, and to that note,

thank you for sitting down with me and

spending so much time together.

Thanks for having me.

Thanks everybody for listening and

watching. and we'll see you next time on

Big Technology Podcast.