ISPE 2025: Burdens of Proof in the Pharmaceutical Century

Welcome to day two. I hope everyone enjoyed the first day of the meeting.

We know that some of the rooms are very cold. Nothing makes me less happy at a

cold. Nothing makes me less happy at a meeting and I'm sorry, we're trying. Um,

bundle up. Uh, I want to just encourage to everyone to visit the posters which are hanging all day including during lunch. So senior members of ISPY, make

lunch. So senior members of ISPY, make sure you go by the posters and talk to the newcomers and provide feedback.

That's such an important part of our meeting.

So yesterday's plenary I thought was very thoughtprovoking and today's will be as well. So I'm delighted to welcome Dr. Jeremy Green. Dr. Green is the

William H. Welch, professor of medicine

William H. Welch, professor of medicine and the history of medicine and director of the department of history of medicine and the center for medical humanities

and social medicine at Johns Hopkins University. Dr. Green received an MA in

University. Dr. Green received an MA in medical anthropology from Harvard and MD and PhD degrees in the history of science from Harvard and completed his

residency in internal medicine at the Bighamin Women's Hospital. He works as a primary care doctor in Baltimore City and as a historian.

Dr. Green is a prolific author. His

first two books, Prescribing by Numbers: Drugs and the Definition of Disease, and Generic: The Unbranding of Modern Medicine, describe the complex history

of medical technologies and the series of legislative, regulatory, clinical, and consumer decisions that guide their production and consumption. His most

recent book, The Electronic Patient: Medicine and the Challenge of New Media, examines how changing expectations of instantaneous communications through

electronic and digital media transform the nature of medical practice. His

newest work focuses on the scientific, social, and economic basis of the shift towards disposable technologies in hospitals and clinics. important to us

as we're concerned with the sustainability of this meeting and have made healthc care one of the largest carbon emitting and plastic wasting producing in sectors of the global

economy.

His work has been widely recognized with awards including from the American Association for the History of Medicine and from the American Institute for the History of Pharmacy. I'm sure you're

going to enjoy and learn from Dr. Green.

Welcome.

>> Thank you. Um, thanks so much Jody.

Thanks to all of you um for the great privilege of addressing you today at the 2025 ISP meetings. Um, you know, I I appreciate as a as a historian being um

being being welcomed into this group. I

I want to make the argument today that you're all historians as well whether or not you have these cards in your wallet saying so. Uh, because pharmaccoiology

saying so. Uh, because pharmaccoiology is fundamentally a historical science.

Now, we can argue over this and there are microphones in the aisles and so um if if you if you want to uh interrupt me at some point and uh and and offer some

more personal reflections or um some personal experience on the history that I'm going to be talking about today, please do. I know there's a lot of

please do. I know there's a lot of collected experience in this room that can testify to the history of pharmaccoepidemiology.

Um I've been engaged in the field of phicopidemiology since my first faculty position which was offered to me by this gentleman. Uh Jerry Aornne I want to

gentleman. Uh Jerry Aornne I want to offer some special thanks to Jerry uh who was my research mentor during during my medical residency and um enabled my first faculty position as a historian

and internist in the division of phiccoepidemiology and phiceconomics at the Brigham Women's Hospital. And um

really I I'm just um I'm incredibly indebted to Jerry um for both um believing in the value of history in pharmacco epidemiology and health policy, pharmaceutical policy in

particular and my own ability to develop a career at this intersection which became the dominant theme of my first several books and um also really a continuing site of engagement of how

history can inform policy in the pharmaceutical world. um is all because

pharmaceutical world. um is all because he was willing to take a risk on me as an unorthodox pharmacco epidemiology faculty member. So this is just to say

faculty member. So this is just to say I'm a biased party when the subject of Jerry Aborne is concerned and that'll become relevant later on in this talk.

Um I'm also indebted to Jerry for introducing me first of all to the word phco epidemiology in its unhyenated form. I know that there's still some

form. I know that there's still some controversy about the hyphen um but also with a concept of his own coining which is pharmarmacco epistemology. Show of

hands, how many of you have used this term? That's pretty much what I

term? That's pretty much what I expected. Oh, there fantastic. So there

expected. Oh, there fantastic. So there

there there is there is a there is a small group here. Uh Jerry used to give uh a lecture on pharmicico epistemology as part of the introduction to social medicine course required of first year

medical students at Harvard. Um and

really uh you know epidemiology is a form of epistemology. There is an epistemology of epidemiology.

Epistemology being the philosophical branch that deals with how we know what we know. And this is a crucial question

we know. And this is a crucial question for the talk today. What is the burden of proof? Who bears the burden of proof?

of proof? Who bears the burden of proof?

How do we mobilize evidence in the field of phicco epidemiology? Where is that evidence resisted and contested, met with skeptical audiences or outright

denied? And this image of Jerry comes

denied? And this image of Jerry comes from a series of oral histories of Isp um which I highly recommend to you.

They're up on the website. They're free

available um as a course you can you can sign up for. Um and these oral histories are one way of telling the story about burdens of proof in what I'm calling the pharmaceutical century. really this

pharmaceutical century. really this period of roughly a hundred years since the birth of the researchbased pharmaceutical industry as we know it especially in North America and the moment we find ourselves in right now.

Now Jerry refers to the early days of phiccoepidemiology as quote a small band of deviants that were not quite at home in clinical pharmarmacology or clinical

epidemiology people with without any natural constituencies and Ispi is that affinity group that emerged from that intersection. Um now these histories

intersection. Um now these histories which are oral histories um they start in the story tends to start in 1984. Uh

we have uh figures some of whom I think are in this room today. Stan Edlavich

Hugh Tilson, David Lillianfeld, Brian Stro, many others um talk about this first conference in 1984. uh a group of people impressed with what one could do with newly computerized Medicaid data,

approaches to studying drugs in the real world, especially using the computer as a tool at the need to deol to devise um new methods that could actually help build credibility for the field as a

science. Um and we could talk about a

science. Um and we could talk about a history of conferences. That's one way of talking about a history of a field.

We could talk about a history of textbooks. Here's the phical

textbooks. Here's the phical epidemiology textbook edited by Brian Strom. uh going through its second,

Strom. uh going through its second, third, fourth editions. This can be tracked over time, both an expanding page volume and circulation um and you know to the present day and I'm working

on my draft for the uh ethics of pharmacco epidemiology for the seventh edition. I want to promise Brian if he's

edition. I want to promise Brian if he's in the room here that I I am working on it. Um but when we think about the way a

it. Um but when we think about the way a field tells its own history, there are things that get missed. Um and I want to suggest that this story of how Pharmacco

epidemiology emerged as a field because of ispy is a short-term history right it's an explicit history using actors categories and historians can also use what we call analyst categories which is

to say what is it that people were doing at different points in time that might not have been called pharmacco epidemiology explicitly with or without a hyphen but was definitely the process

of doing of building new ways of knowing about drugs in real time that we recognize as a genealogy ology of the field. And when we look in this longer

field. And when we look in this longer perspective, right, when you take a deeper history of pharmaccoy as something that existed in the world and also struggled with burdens of proof

well before the 1980s, we actually gain additional insights that can help us with some of the challenges of the present day. Um, so I want to rate your

present day. Um, so I want to rate your time and I'm going to kind of give you a road map for for what we're doing for the next little bit. I I want to do this in four parts. First, I want to talk

about a persistent problem of the relationship of pharmaccoepidemiology as an observational field, which I would argue is also a historical field to other forms of pharmaceutical knowledge,

especially randomized control trials, um, which oftentimes imply a hierarchical relationship of superiority and inferiority. I'm sure this is a

and inferiority. I'm sure this is a surprise to everybody in this room, but but this contestation of phiccoi is a long-term problem and the long history of it actually helps us understand more

about it. Then I want to talk about the

about it. Then I want to talk about the trouble of mobilizing disinterested knowledge um in a field that is often called in to solve crises in which there

are many interested parties and urgent problems to be solved in the present day. Um, third, I want to think about

day. Um, third, I want to think about the history of this field as an infrastructure of prescription data that gets mobilized and put to new purposes, right? So, how do we build systems of

right? So, how do we build systems of prescription surveillance? What are the

prescription surveillance? What are the benefits, but what are some of the risks that emerge in that space as well? And

then finally, um, it may be a bit more provocative to talk about pharmaceuticals themselves, not just knowledge about them as something that changes over time. I want to think about pharmaceutical properties like efficacy

and safety as things that don't exist in an absolute sense out in the world but are what we would call interactive kinds highly dynamic and they change as our knowledge about them changes as well. So

that's the road map. The first story is probably the one you know best already.

Really all regulatory sciences and phiccoy has one foot as a regulatory science are born over controversies over burdens of proof. So perhaps the best known example of this is the EPA, the

Environmental Protected Agency. Um,

which is a very different entity than the FDA. Um, and uh, or we could talk

the FDA. Um, and uh, or we could talk about the EMA, other other regulatory agencies in different parts of the country, different parts of the world.

Um, but there's a constant pendulum over burdens of proof like whose proof is demanded, right? Is the burden of proof

demanded, right? Is the burden of proof on those claiming evidence of harm after the fact or is the burden of proof on those who are introducing a pent potential harm before the fact and this

is a notion popularized in environmental science and environmental epidemiology and we can look at Rachel Carson's silent spring is introducing this term as the precautionary principle. So in

the 55 years of the EPA this pendulum has moved back and forth several times.

We're in the middle of a dramatic pendulum swing right now. But the EPA has made its long-term claims at being a bureaucratic structure for mobilizing scientific bases for adjudicating

evidence of risk. And the field of pharmi does very similar things. Um at

the same time, epidemiology and especially pharmaccoepidemiology has been a recurrent subject of popular skepticism especially because the claims of risk and benefit around

pharmaceuticals and practice will often change dramatically from study to study, decade to decade, even with the same drug. So these concerns which are

drug. So these concerns which are fundamentally about pharmarmacco epistemology um have recently been updated and turbocharged in the current

informationational landscape um and substantial shifts in knowledge production in the present day right now.

But I want to take us back a little bit to this article from 2007 which is the first time I encountered this critique myself. Some of you may have read this

myself. Some of you may have read this article in the um magazine of the New York Times when it came out by Gary Tabs in 2007 when he published a very influential article on the women's in

the aftermath of the women's health initiative trial and many of you will remember the WHI randomized control trial was seen as a seismic moment in what be what has been called in many

spaces medical reversal really overturning the kind of clinical practice that I had been taught in my clinical rotations in medical school as where um observational studies,

pharmaccoepidemiological knowledge have been crucial until that point in guiding practice um recommendations on the use of hormone replacement therapy in post-menopausal

women. And so because the WHI trial

women. And so because the WHI trial undid this and suggested that the benefit harm calculus was radically different than we had thought on the basis of phiccoepidemiological

knowledge, TABS offered this essay to suggest that really phic in general and epidemiology in general was was

overrated and dangerous. Um and um HRT had been the textbook example for teaching the value of phicopidemiology and how it was used as a means for undermining the whole field. So, a

defense of phicco epidemiology was mobilized um by by by by Jerry Aor and by many others in this room as well, but I'm going to highlight Jerry's article from the New England Journal of Medicine

in defense of phicopidemiology embracing the yin and yang of drug research. Um,

and this, as it turns out, is part of a even longer battle for the stakes of epidemiology as a field at all, bound in real world experience and not the sterile, more controlled space of either

a laboratory or a randomized control trial. And I'm going to return to

trial. And I'm going to return to Jerry's defense in a bit, but first I want to flesh out a bit of the stakes of the controversy. Just a show of hands,

the controversy. Just a show of hands, how many of you had seen that 2007 article? All right, that's a sizable

article? All right, that's a sizable chunk, but then again, this is a room of epidemiologists. Um, how about this

epidemiologists. Um, how about this earlier article? Because that actually

earlier article? Because that actually turns out was a 10-year anniversary of Gary Tab's original article in science called epid epidemiology faces its limit. Show of hand on this one. Okay, a

limit. Show of hand on this one. Okay, a

much smaller fraction. Um, now this was an incredibly influential article. Um,

and uh, it was a searing critique of epidemiology as a field that often in tabtorms did more harm than good, breeding widescale anxieties that undermine trust in medical knowledge

altogether. and Tabs reported on a

altogether. and Tabs reported on a series of false positives in which evidence had been reversed in similar to uh the the what the later large-scale

crisis of the WHI trial and HRT later on in which there are weak findings whose evidence might be likely to be reversed and for him this included things like the epidemiological linkage between

cigarette smoking and breast cancer or pancreatic cancer, red meat consumption and colon cancer. Um so this initial critique was not about drug data. It was

not phiccoi but it was a broad critique that epidemiology was doing harm in the world and part of his reporting involved the careful comments of people involved in the creation of ISP. So I have

highlighted you know a quote from Ken Rothman here who wanted to develop careful attention to a nature's the nature of a field's methods and inherent constant suspicion of the possibility of

confounding and for confusing correlation with causality because good epi is critical of hasty conclusions.

Right? I think those of us in this room who have been doing epidemiological work understand this. Right? Um as a

understand this. Right? Um as a historian in a pharmaccoi division um during the times of this earlier tabs article I recognized this practice because the division of phiccoia at

Harvard had regular epi rounds right every few weeks in which we'd present new data look at signals that were emerging from the data second guess each other try to think of other reasons

beyond that which was claimed that might explain the difference right so if one is making a claim for a for a for a correlation that might have some causal implications It was the responsibility

of everybody else in the world to play this game of spot the confounder. What

are the possible confounders? What else

is happening either in this data? What

else was happening at that time that might explain that shift? So I want to point this out that as a historian who was learning to be a a quai pharmaccoepidemiologist in these settings. I realize that the

settings. I realize that the similarities between good epidemiology and good history were actually quite substantial. A claim you're making about

substantial. A claim you're making about the past could be true or it could be an artifact of the way you choose your data or it could be a token of something else. Good pharmacco like good history

else. Good pharmacco like good history required a continuous skeptical mindset and the capacity to receive continuous constructive critique. Um now skepticism

constructive critique. Um now skepticism is a key part of the spirit of pharmic.

We think something works. We think it's safe. We even think it might be

safe. We even think it might be economically valuable in the long run.

But how do we really know? How do we continue to know? How does it hold up to the test of time? So, Talbs's questions posed properly are not really outside of the field of epi, right? They're part of

it. They're part of good epic practice.

it. They're part of good epic practice.

But skepticism unhinged can lead to paranoia and nihilism. So, how do we encourage a form of skepticism that doesn't bring the whole house down around us? And I want to point out here

around us? And I want to point out here that just a few years ago in 2022, um a a retrospective study of Ta's claims from 1997 was published in science,

right? a group of epidemiologists

right? a group of epidemiologists re-evaluated the 41 false positives that he reported in 1997 and found that a quarter of them turned out to be false false positives or actually just

positives right so that they actually held up the test of time now for some for Tabs and if Gary Tabs is in this room you know I I um this is really not a critique of what his work is doing as

an investigative journalist right but for someone like Tabs a double reversal is just as dangerous as a single reversal because it signifies a kind a topsyturvy approach to medical knowledge that can throw public opinion and

greatly undermine trust in medical knowledge. But for those of us involved

knowledge. But for those of us involved in the careful production of medical knowledge or its use in policym, we know this is a necessary part of the game. So

how do we signal that the form of knowledge making we do must be alert to inherent dynamism of the way that we understand the world as we move forward and that these changes don't signal that

the whole game is rigged or that it's off but that we actually need to be supple in the way that we approach novel. So I'm going to quote from the

novel. So I'm going to quote from the conclusions of this study. Many of the associations selected by TABS as examples to denigrate epidemiological research have proven to be important public health implications as evidenced

by policy recommendations from reputable national international agencies to reduce risks arising from the associations. The utility of

associations. The utility of epidemiologic research in this regard is all the more impressive when one remembers the associations were selected because tabs thought they would prove to be false positives. 25 years later,

epidemiology has reached beyond its limits. This history should inform

limits. This history should inform current debates about the rigor and reproducibilability of epidemiologic research results. But as they pointed

research results. But as they pointed out, if you look at citations, Tabs' article was a citation classic. Several

orders of magnitude more citations going to the reputation of epidemiology than to those studies that actually demonstrated that these were actually reasonable claims. So why is this right now? You may have your explanations and

now? You may have your explanations and again I encourage you there's microphones. Um, you know, I'd like to

microphones. Um, you know, I'd like to suggest that Tabs is doing his job as an investigative reporter, but the structures of narrative that we live within, right? So, we don't simply live

within, right? So, we don't simply live in a world in which data changes policy or data changes hearts and minds. Data

suspended within narrative structures does. And there's a very strong

does. And there's a very strong narrative claim of critiquing expert knowledge and um and especially being skeptical of knowledge when it changes in the field of health in medicine. This

has happened many times throughout American history, right? It is clearly happening right now as well. And we need to be very alert to how we communicate that as a part of normal science. Now,

this is true not only with American popular culture, but also American medicine itself. It's easy to cast

medicine itself. It's easy to cast dispersions on observational research.

And I want to come for a moment to this this subject here, which you should all be familiar with as the evidencebased medicine pyramid, the hierarchy of

medical knowledge. Um, and we all exist

medical knowledge. Um, and we all exist within an evidence-based medicine framework. As a clinician, I can't claim

framework. As a clinician, I can't claim to be practicing evidence-free medicine, or at least I wouldn't be able to claim that for long. Um, but there's this

problem that we have in which a um a straightup pyramid, which becomes the OB omnipresent symbol of evidence-based medicine with RCT's meta analyses in the

top um has many variations. Some seek to differentiate cohort and case controlled research along with RCTs as quote unquote unfiltered information as opposed to background information and

expert opinion on the very bottom but then with metaanalyses and systematic reviews on the top.

Others do not. Um, but when we linger on this, it it's a convenient symbol and there's meaning there's important meanings in the idea of quality, higher

quality and lower quality that we don't want to reject outright. But I I'd like to ask you, you know, what are the problems for this room with this symbol

of medical evidence?

Now, you might say none that we're all happy.

So, we're not at the top. Thanks, Jody.

I think that's so that that would imply that this would be fine as long as we were at the top.

Yeah. So there's no appraisal built in.

Something is inherently better because of what it is rather than how it's done.

So one can simply cast more attention to an RCT even if it's poorly done. You

know, we know we can be very critical of we we train ourselves to actually read critically any study, but a very well done case control or, you know, or court

study could be much more useful than a very poorly done RCT. There's no space in this kind of chart for that. What

else?

Sure. So, it ignores target populations.

There's a universality here, right? That

this should work well for the mainstream of all people. But when we are getting into areas where we need to mobilize knowledge for smaller subgroups, it ignores the the nimble ability of

observational research to do this work in a way that RCTs can't always do. What

else?

You can't always do an RCT. So it

basically takes away the possibility of highquality knowledge making in fields in which RCTs are not possible. And I

know if we continue to go around, we could field more questions like this.

And these are what you're doing is really filling in many parts of this table that that Jerry had offered in this defense of phiccoide back in 2007, right? That in the aggregate, yes, the

right? That in the aggregate, yes, the RCT solves a number of the epistemological problems of confounding of observational research. But simply

insisting that RCTs are always better than phiccoi commits another fallacy.

And that's the fallacy that information is some vast uninterested pool in which each form of data for every possible person can simply be evaluated based on abstract principles of quality rather

than the actual local timebound social context in which it's generated. And I'm

going to argue as a historian that these things really matter a lot. Um, and it's like the old joke about the policeman who finds a man looking for his keys under a street light. Some of you may know this one joins them thinking that

he had dropped his keys there only to find that the man had dropped them elsewhere but that the light was better under the street light. And so RCTs might shine a brighter light on certain questions but they can't be deployed

everywhere. To insist an RCT knowledge

everywhere. To insist an RCT knowledge is better in areas where it does not yet exist or cannot exist is to risk an informationational landscape that only favors those things that are easily RCT.

So the broader point here is fairly simple. not that not all relationships

simple. not that not all relationships between clinical trials and observation can be summed up with a symbol that just places the RCT above the observational study. Yes, perhaps that works in terms

study. Yes, perhaps that works in terms of claims of causality, but it's often glossed to mean better overall in terms of value of research overall. And

depending on how we build that value system, what data we need, when, where, and why, there are ways in which phicco epidemiological studies at any given moment are better than RCTs. So there's

an alternate pyramid that's been suggested. Have any of you seen the wavy

suggested. Have any of you seen the wavy pyramid?

Okay, a couple of hands here. It's a I think it's a new and underappreciated way of of reevaluing this and maybe it just looks like you know it's just it's just too late at night. um or uh but

this has been proposed by Murad Ai Alawad and Aladab in uh in the British medical journal evidence-based medicine um in which in any given vertical strata of evidence quality if you move

horizontally there's some wiggle room right so sometimes a given cohort study may have better quality evidence for the problem at hand than a given RCT and I think even this sort of smudging of the

borders can help in a way get away from what is an otherwise knee-jerk hierarchical response that to the relationship between observational research and u and experimental research that doesn't actually help us. Um now

this model also suggests we should break off these very different active systemic review and metaanalysis and use it and here the metaphor gets a bit maybe overdone as a micro as a form of a magnifying lens to then look more

critically at the individual studies themselves. But the value here is

themselves. But the value here is suggesting that um it's a critical it's in this model context matters critical skills matter and pharmacco epidemiology

matters. So to come to this second issue

matters. So to come to this second issue of the fundamental problem of phicco epidemiology as a science that's born of controversies and here I've promised an alternate history from the official ISP

history um and I want to take us back further because phiccoi has its roots in real world problems in real time. This

is one of its greatest strength but also part of the reason why it's always subject to aspersions that are cast from others who will say you see this as a political issue now or you see these are economically interested parties now or

these are parties that are interested because they are advocates for a particular patient population and there's a call for a space of purity a view from nowhere in which a more

appropriate scientific knowledge can be created and yet for the FDA for the EPA right there's no space of nowhere Right?

We're always working from fraught moments in which harm is being done or our risk of harm is being done and we have to evaluate multiple stakeholders in that space. So some of this um comes

from the work of the late Harry Marx, a historian of medicine who preceded me at the Institute of History of Medicine. A

remarkable book I recommend to you called the progress of experiment science and therapeutic reform in the United States which traces very different attempts of scientists, clinicians and regulators to produce an

objective science of therapeutic evaluation from before the founding of the FDA to the um late 20th century. Um,

and as many of you will know, for the first four decades of the 20th century, even after the founding of the FDA, it's really the American Medical Association and not the US Food and Drug Administration that has the chief role

in the scientific value of therapeutic efficacy and safety. And this is done through an office of the AMA called the Council on Pharmacy and Chemistry. and

they developed their own laboratories to do so. Partly because there's a growing

do so. Partly because there's a growing skepticism of the commercial markets for pharmaceuticals growing at such a pace that it was exceeding the claims of scientific knowledge being attached to

therapeutics at the time. And I'll have a quote here from the first head of the council on pharmacy and chemistry, Dr. FE Stewart, who noted, "There's a good deal of difference between the introduction of new products to science

and new brands of manufactured to commerce. The former belongs to the

commerce. The former belongs to the sphere of science and the latter to the sphere of commerce. The exploitation of new products by exaggerating their merits and repressing knowledge of failures is one of the most dangerous

forms of quackery. So the primary tool of the council on pharmacy drugs was the um was a laboratory an analytic laboratory. Was this did this drug

laboratory. Was this did this drug contain what it claimed to contain?

Could there be evidence of toxicological studies or a mechanism of action? But as

you know this is only going to capture some of the problems of safety of efficacy and where this really led to a crisis um which enabled the the the

strong hand of the FDA to emerge as a science-based regulator is the sulfanylamite crisis of 1937. I'm going

to ask for another show of hands. This

is not to shame you. I just kind of want to know where the room is in terms of how many have heard of sulfanylamide elixir as a problem. Okay. So that's

actually about a 50% rate which is fantastic. Um, so this is a moment in

fantastic. Um, so this is a moment in which the FDA emerges as a gatekeeper, right? Where the ability to truly

right? Where the ability to truly regulate access to the US market is mediated through the FDA's role in shaping what counts as valid medical knowledge and what kinds of proof need

to be offered by a manufacturer to assure the public of its safety. So why

did this happen? This happened because of a disaster. It happened because more than a hundred children died taking what was one of the most important wonder

drugs, anti-infective drugs of the late 1930s, sulfanylamide, a early precursor to antibiotics. Um that was suspended in

to antibiotics. Um that was suspended in a solution of ethylene glycol which is you know often understood as antifreeze and which um which which uh which was

nephro and hpatoxic and led to a wave of deaths. Um now the history of how this

deaths. Um now the history of how this happened, how it became a scandal, how it seared through the newspapers of the country and then the world and then led to the passage of the food drug and

cosmetics act of 1938 which strengthened the FDA to require proof of safety of drugs before anything could be marketed pre-market approval and then that shaped the basis of knowledge that had to be

produced around any pharmaceutical product in the world eventually. Um is

seen as a history of toxicology rather than pharmaccoy. I want to suggest it's

than pharmaccoy. I want to suggest it's a history of phiccoi and I want to offer you a letter written by a physician um in 1937.

His name is uh Dr. As Calhoun and he says, "Nobody but almighty God and I can know what I have been through these past few days. I've been familiar with death

few days. I've been familiar with death in the years since I received my MD from Tulain University School of Medicine with the rest of my class of 1911.

Covington County has been my home. I

have practiced here for years. Any

doctor who has practiced more than a quarter of a century has seen his share of death. But to realize that six human

of death. But to realize that six human beings, all of them my patients, one of them my best friend, are dead because they took medicine that I prescribed for them innocently. And to realize that

them innocently. And to realize that that medicine which I had used for years in such cases suddenly had become a deadly poison in its newest and most modern form as recommended by a great and reputable pharmaceutical firm in

Tennessee. Well, that realization has

Tennessee. Well, that realization has given me such days and nights of mental and spiritual agony as I did not believe a human being could undergo and survive.

I have known hours when death for me would be a welcome relief from this agony.

And I want to suggest that it is the observation of these forms of unexplained death that then gradually become linked through a set of prescriptions that leads to the

understanding of the toxicity of this drug that gives it the force that it has. That this is a mode of pharmacco

has. That this is a mode of pharmacco epidemiology. There was a signal. It was

epidemiology. There was a signal. It was

a clinical signal. It was followed back through prescribing networks and that led to a generation of a knowledge basis that drove substantial policy, practice

and change. Um,

and change. Um, we could say the same thing about this next moment 1962 which pharmaceutical regulatory history tends to go to. This

is the Kever Harris amendments. I'm not

going to make you show your hands, but this is the act of Congress, Kev Abber Harris amendments to the Food and Drug Cosmetics Act of 1938 that instituted the requirement for proof of efficacy as

well as safety and also started the process by which we now generate it. Not

only did it enshrine the randomized control trial as a key element of pharmaceutical approval, but also the sequence of phase one, two, three, and later four trials. Um and it's linked to

the heroic figure of Francis Kelsey um as a iconic drug detective. And again,

Kel Kelsey is not seen typically as a phiccoideologist, but as a rigorous bureaucrat whose belief in evidentiary standards um led her to halt the approval of theomide in the US after it

had already been approved in many countries around the world. Um because

her understanding of the evidentiary burden of proof had not yet been met.

Now John F. Kennedy recognizes Francis Kelsey as a hero. Her story helped speed the passage of the Keith Alver Harris amendments and ultimately the formation of one through four clinical trial system as we know it. Her nar her

narrative is mythologized swiftly. Here

you see an FDA recruitment poster from 1963 showing Francis Kelsey as a drug detective, right? Searching for signals

detective, right? Searching for signals and data of drugs. I'm not going to linger on the phytoomide story too long today. I think you all probably know it

today. I think you all probably know it well and know it also as a global history, right? So here for example is

history, right? So here for example is this is a memorial to phytoomide in Australia. It's remembered differently

Australia. It's remembered differently in different parts of the world with differential impacts. Um, but uh,

differential impacts. Um, but uh, theomide was especially marketed to pregnant women with morning sickness precisely because of its demonstrated safety and toxicological studies to

date. And I want to suggest the story of

date. And I want to suggest the story of Lyomide emerged on the global scene largely through the work um, of Whitiken Lens, also called a Pholyomide

detective. You may not know Lens's name.

detective. You may not know Lens's name.

You may even if you know Kelsey's. Here

you see an article in El Pais calling him eld detective de la talida. Um

I would suggest lens is part of the evolving image of a phicco epidemiologist even if that word didn't exist. He was a medical geneticist a

exist. He was a medical geneticist a specialist who noted in 1961 an increasing number of referrals to him for a very rare form of congenal birth anomaly known as fomeia. And I'm not

going to display images of children of theomide here partly because the images in Lens's own territological publications were so graphic and objectifying and many people living with the effect of theolyomide today or

theoliters um as they're often called have worked hard to counter them and produce their own images of healthy theor bodies um and some images like this one which depicts theers as adults

with voices of their own and an ability to have a say in their own visual representation. But Lens, I would argue,

representation. But Lens, I would argue, was a pharmac epidemiologist. He had

traced a signal that he saw in the clinic back to a drug, produced data to actually produce convincing arguments that that drug could be linked to this new clinical signal, implied causality,

checked for confounders, as he began to make the case that the safest of all anti-nausea medicines was not as safe as it seemed to be in the midst of an emerging scandal. Now the irony of

emerging scandal. Now the irony of course is that pholitoolide helps to enshrine the role of the RCT on that pyramid as we know it today. The keer

Harris act doesn't call for more active phco vigilance. It doesn't call for a

phco vigilance. It doesn't call for a cadri pharmarmaccoepidemiologist again a word that didn't exist. Um even though it was that kind of knowledge production and not a clinical trial that actually

led to the signal that drove the phytolomide scandal. Um so from the

phytolomide scandal. Um so from the sulfanylomide tragedy to the phomiite tragedy from the vio scandal to the aandia controversy the evidentiary claims that come to form the methods based pharmacco epidemiology often

receive their public airings in times of intense public controversy. On the one hand these controversies are politically useful as they focus attention that can help overcome inertia and inculcate new

and necessary regulatory science to help provide more public trust in the safety and efficacy of our medicines. On the

other hand, because this knowledge emerges during controversial moments, it can be debased and it can be used to undermine public trust in phicopidemiology or medical knowledge as

a field in general. Now, many people in this room who are familiar with the Aandandy and Vio controversies might point out that actually phase four clinical trials had had more to do with driving these concerns than pharmarmacco

vigilance. But I do think that these

vigilance. But I do think that these networks of pharmarmaccovigilance, things like the FDA sentinel initiative, which I know is well represented in the meetings today, were simply not possible um with randomized clinical trial

design, but were enabled by the emergence of postmarket surveillance data. So these sections are getting

data. So these sections are getting shorter as I move along. I want to give you a brief glimpse into this architecture of surveillance data that emerges at the same time. So, Sentinel

is a good example of the kind of things one can learn from what I call the afterles of the prescription or the telltale RX. The dreams, some realized,

telltale RX. The dreams, some realized, some still fantastic, some benign, and some perhaps more sinister that have emerged from the hope of a seamless world of prescription data tied to clinical social economic outcomes data

as well. And as a historian, I want to

as well. And as a historian, I want to suggest to you that 19th century doctors never meant for 21st century phacidemiologists to be using prescription data the way that we do

right now. Some would be amazed. Most

right now. Some would be amazed. Most

would probably be horrified.

And I I'll I'll take you to um the sense of what the prescription was. Here's an

a textbook on prescription writing from the 1880s. Correct prescription writing

the 1880s. Correct prescription writing is an accomplishment which is to the physician what elegant clothes are to a gentleman or a handsome frame to a fine painting. If it is not an essential part

painting. If it is not an essential part of his education, it at least displays his other acquirements to best advantage. And in the 1880s, many older

advantage. And in the 1880s, many older doctors were really quite concerned that the younger doctors didn't know Latin anymore and were writing their prescriptions in English. Um, and the

prescription was originally used to literally tell a ph a pharmacist or an apothecary how to make a drug. And in

the pharmacy, pharmacists would actually assemble drugs from raw or mixed ingredients. Um, by the mid- 20th

ingredients. Um, by the mid- 20th century, of course, this had changed dramatically with the rise of major pharmaceutical houses. So the work of

pharmaceutical houses. So the work of the pharmacist is changing at this time from literally compounding most medicines to taking a larger bottle, taking pills out of it and then putting them in a smaller bottle with a label on

it, handing to the patient. Now I don't mean to say this in a way that is insulting to pharmacists, right? In many

ways that transformation gives rise to the whole field of clinical pharmacy, which is fascinating things. But in this time, it's realized that um the prescription becomes more

standardizable. the Latin starts

standardizable. the Latin starts shifting to a standardized form that is more amenable to standardized databases.

Um, and so there's a newformational structure of what you can do with a prescription that a prescription once filled is still valuable as a form of data. Um, and this is valuable to a

data. Um, and this is valuable to a number of different parties, not just pharmacists for whom it's mostly trash or at least something that needs to be kept in a book to fulfill regulatory requirements. And as this room knows

requirements. And as this room knows better than any other collection of human beings, perhaps the prescription once fills is neither inert nor forgotten. Each script leaves a residue

forgotten. Each script leaves a residue of doctor, patient, diagnosis, and therapeutics for those interested in studying pharmaceuticals and their consumption. And one very interested

consumption. And one very interested group was market researchers for pharmaceutical firms. When Paul Dehane, a marketing consultant to the drug industry and a prominent contributor to the new journal medical marketing,

published one of the first textbooks of prescription drug marketing in 1947. He

warned the field of pharmaceutical market research needed more sophisticated tools for visualizing prescriptions in practice. Quote,

"Market research in the pharmaceutical industry, Deain said, does not merely require the collection of certain statistical and commercial data. It

covers a much broader field requires constant observation of the entire medical and pharmaceutical horizon so that the physicians prescription habits can be thoroughly studied." And Owen

Wade would note a few decades later in 1969 with considerable enthusiasm that the computer had offered a new tool to this process. That the computerization

this process. That the computerization of claims data had transformed quote such a mundane procedure as the recording of the sale of prescription drug into a technology that now provided

penetrating insight into the nature of disease drugs and clinical care. This is

the satisfaction of the cook, Wade observed, who finds old leftovers in the kitchen and creates a splendid dish. So,

I can talk for way too long about the process by which the tools that we now use to survey the landscape of pharmaceutical claims data were created.

I'm not going to do that. I just want to give you a couple of images. This is an image from a textbook for pharmaceutical sales representatives by Arthur Peterson um who encouraged sales representatives

to collect data on the prescribers that they visited on their route so that they could more accurately market to them. Um

and then also get a sense of the drug categories that they prescribe. that at

first sales representatives or detail men as they were known at the time in the 40s and 50s would be the organ for actually stopping by localarmacies and

asking to look through the the log books to understand what local physicians were prescribing. Um and uh as the

prescribing. Um and uh as the entrepreneurial detail men of the 40s became the pharmaceutical marketing executives of the 50s, they sought to expand these data practices into more

reliable systems of prescription surveillance. And um I I'll give you one

surveillance. And um I I'll give you one example which is uh Raymond Gosselin who was pursuing a masters in pharmarmacology at the Mass College of Pharmacy and an MBA at Boston University. and Gosselin's thesis, which

University. and Gosselin's thesis, which is a fascinating read. It's submitted in early 1950, and it's the first systematic and reproducible prescription drug audit of a panel of pharmacies in

the Boston area. Um, and he developed a technique of coming up with a representative sample of pharmacies and then getting a sense of what drugs were being prescribed, what their costs were, how how the physician prescribers

correlated. Um and shortly after this

correlated. Um and shortly after this thesis he uh created RA Goslin and Associates which began to market the first prescription data product in the early 1950s. This is a national

early 1950s. This is a national prescription audit um a subscription database that could segment the drug market by local region give quartertoquarter information and he used IBM punch cards right the use of

computerization to build a database which at first was updated quarterly and then monthly and then even more regularly than that. Um so by by 1970

when um RA Goslin company is sold to IMS health um to become part of a larger uh you know claims data data empire um it remains a key plank in pharmaceutical market research today. Now there's other

tools that are developed as well the national disease and therapeutic index for example but I want to highlight here that the IBM punch card was also used um by the American medical association. The

role of the AMA in compiling and distributing prescriber databases is a unique case to understand how the profession participated in its own surveillance. And this is a a product

surveillance. And this is a a product that the AMA would sell to pharmaceutical marketers called the AMA physician master file. And acted as a key linking device between prescribers

and pharmacy claims data. Um and I yeah I I I I want to suggest also that in health systems the interest in building a universal digital set data set for drug experience it starts in the late

50s when early enthusiasm about computers and also the possibility of artificial intelligence before what's known as the first AI winter right in the 60s there's a lot of thought that

computed data processes could actually lead to self-mated machine learning processes that would tell us things about prescribing and other medical forms of information that we could not glean on our own. Um, and this takes

place in a lot of different localities, especially for safety data, um, in Copenhagen and Aberdine and Geneva. I

know best the experience in Boston and what becomes the known as the Boston collaborative drug surveillance program.

So wouldbe epidemiologists of the safety profiles of adverse drug events needed a way to circumvent the physician as an unreliable reporting device. That there

are many reasons why you couldn't really trust a network that was based on physician self-report of harms. And what the Boston Collective did is began to find ways of using data sets to

self-generate correlations that would find signals that would otherwise be missed. Um and by 1968 over 25 patients

missed. Um and by 1968 over 25 patients have been entered and discharged from this automated surveillance system in Boston with over 26,000 monitored drug exposures representing more than 700

init individual drugs and commonly prescribed drugs like Doxin. Dejoxin was

one of the first major publications of the collaborative. New forms of toxicity

the collaborative. New forms of toxicity that were only present at low frequencies now emerge from these data sets. So I want to suggest that um you

sets. So I want to suggest that um you know the panel before me was fascinating you know looking at how phicopy can be adapted for pharmaceutical policy. Lots

of interest in how uh large language models new forms of AI and the present day can be used to mobilize and automate data connections but this was present in

origins of phicco epidemiology back in the 1960s and 1970s as well. So this uh this this expands and um you know we can talk about a number of scandalous forms

of information that then emerge here right so as the Boston collaborative drug surveillance program escalates these activities uh cleoquinol and anti-infective in use since the 30s was found to be associated with subacute

myoic neuropathy in a way that had never been seen before 1970 when it's reported um an association between the synthetic estrogen diylol or dees um and a rare

form of cervical clear cell adinoma was reported in 1970 71 evidence of a 20-year latency period between the use of the drug and the detection of the cancer. The beta blocker practol became

cancer. The beta blocker practol became the focus of a broad scandal after it was associated with potentially fatal inflammation of the skin and soft tissues known as oculucan oculomou

mucoututaneous syndrome five years after its broad release in the British market.

And these examples simultaneously underlined the necessity for drug surveillance units but also underscored the impossibility of inpatient or clinicianbased surveillance systems to

capture drug disease associations. They

made the case for automated phicco vigilance. Um digitization of Medicaid

vigilance. Um digitization of Medicaid data by the 70s was also a fundamental moment in in ratcheting up the automation of phicco vigilance and the emergence of phicopidemiological

systems. And here I think we begin to approach the history that ISP offers for itself where automation computerization of medical claims data is leading to a number of researchers finding their way

into this field needing a space to work together and that space becomes. Now I'm

going to leave you with just a few final reflections about why I think you're all historians. um whether or not you

historians. um whether or not you recognize that fact and that I my hope is that by recognizing that fact you might find bits of the toolkit of the

historian that actually help you in your own work and to do this I'm going to first introduce one more really long word so we've talked about pharmarmacco epidemiology and pharmacco epistemology

right epistemology how we know what we know and in a move that may make you regret ever asking a humanities professor to deliver a plenary at ISP I want to take us one step further in the

philosophy of knowledge and think about ontology. Um now this is a relationship

ontology. Um now this is a relationship between epistemology and ontology. These

are two fundamental branches of philosophy of philosophy of knowledge and um this also follows the work of the philosopher of science Ian hacking who has a book on what he calls historical

ontology historical epistemology. We've

been doing some of that already. We

think about how is it not not only how do we know what we know but how did we know what we knew then or how is it that the way that we prove things changes over time. requiring a randomized

over time. requiring a randomized clinical trial for an intervention that was introduced before the randomized clinical trial was invented doesn't make sense historically speaking, right? One

can only think about RCTs after a point in which there is such a thing as an RCT, which is to say that the way that we produce knowledge changes over time.

But hacking um suggests there's something even more complicated. It's

not just that we change the way we think about the world, but that the actual structure of things and things I would argue like efficacy and safety

themselves change over time in relation to the way that we produce knowledge. So

I'll give you an example of chloricquin um chloricquin resistant uh malaria, right? Malaria is different after

right? Malaria is different after chloricquin. We produce a way of knowing

chloricquin. We produce a way of knowing about malaria which is which leads us to then produce a drug chloricquin that we then actually deploy in large amounts and that that knowledge changes the

world of malaria such that now we have a wide problem of chloricquin resistant malaria in the world that we didn't have before. Antibiotic resistance in general

before. Antibiotic resistance in general is an illustration of what hacking would call a looping effect. Whereas concepts

concepts and knowledge leads to changing practices that changes the material things of the world. And I want to suggest that this is also true for even abstract principles like efficacy and safety. And I want to give you just a

safety. And I want to give you just a very brief example. And this is an example of emodium ad. Now emodium is a drug that many of you will know. Uh it

is a it is the most popular over-the-counter antid-diarral agent. Um

it is also um an opioid. Now I'd like to ask you how many of you already knew that emodium was an opioid. Okay. So we

got maybe a 20% which is good for a room of pharmacco epidemiologists. Now op

emodium is an opioid and not an opioid right it's developed in the 50s out of the labs of Paul Jansen um of Jansen Pharmaceuticals. Jansen also develops

Pharmaceuticals. Jansen also develops fentinyl at the same time. Um, and what is happening in in Jansen's labs is tweaking the structures of morphine to

develop more effective more more analesic properties. But also in the

analesic properties. But also in the case of emodium recognizing that something that we think of as a side effect of opioids today, right, which is the constipating quality could be isolated as long as it could be produced

in a drug that then doesn't have analesic or or hedonic effects to become an effective agent against diarrhea. And

this is how lowamite emerges into the world. Um so what happens in 1977 when

world. Um so what happens in 1977 when it's first market in the United States?

Emodium is an opioid, right? It is

scheduled as a narcotic, schedule five.

There's no evidence of addiction, right?

There's no claim that any person has ever been addicted to lowamide or sought pleasure by using loperamide at all, but it's still a narcotic just because of

caution. By by 1980, um, uh, Jerome

caution. By by 1980, um, uh, Jerome Jaffy, who is the drugsar under President Richard Nixon, writes a really decisive review of all the evidence of clinical experience that has emerged

under low paramide. um a form I would argue of phicco epidemiology in which there's no evidence of addiction or hydonic effect and the conclusion is that leramide based on a world of

clinical experience is in fact safe should not be considered a narcotic in 1982 leramide becomes the first narcotic to

become descheduled. It is an opioid that

become descheduled. It is an opioid that is no longer treated as an opioid. And

then by 1986, it is put forward as safe and effective enough to be used by individual consumers who could moderate on a symptomatic basis the appropriate

use of this medicine. And it becomes an index over-the-counter drug, an index for RX to OTC switch. Um, it leads to tremendous advertising, right? And so

I'm I I I could have filled an hour with just the different 1980s advertisements for Emodium. Um but the point I want to

for Emodium. Um but the point I want to make here is that while this is going on, Emodium still shares a lot of properties with opioids, including for example when there is lowamide, you

know, that's the active agent, low paramide overdose, um that nlloxxone can be used to reverse it, right? Um

eventually though one starts seeing reports of what's being called poor man's methadone and new forms of toxicity emerge in correlation with um

uh with low paramide very dangerous cardiac arhythmias. There are deaths

cardiac arhythmias. There are deaths associated from ventricular tachicardia for a drug that was deemed safe enough to be made over the counter. How did

this happen?

Someone in this room knows the answer.

How does a drug go from being an index descheduled narcotic that becomes an over-the-counter market leader to then having newly discovered capacity for inducing ventricular

tachicardia?

You guys must have hypothesis. This room

in particular, you must have hypothesis.

H tampering is a good question. Where

where what are you where are you going with that?

>> Yeah. So so here we might think of uh you know pseudafed right. So we know how pseudafed goes from being a front of store drug to a behind-the-counter overthe-counter drug in which it becomes heavily scheduled because we know that

in a bathtub you can take pseudafed and make it into meth if you have enough pseudafed. Right? But it turns out the

pseudafed. Right? But it turns out the the the leramide story is slightly different.

It's really one of doymmetry.

So at a certain point um folks who are pharmacco epidemiologists I would argue who begin doing um webbased research on chat rooms of how people are using

overthe-c counter drugs find a subpopulamide users that are talking about how

leramide if you are if you're if you're a a long-term opioid user and you uh you are having going into withdrawal and you

can't get access to methadone that a ultra high dose of leramide um really hundreds of tablets sometimes thousands right but hundreds of tablets consumed

all at once can actually have a functional effect of staving off the worst symptoms of opioid withdrawal now why would anyone need to find themselves

in a situation where they would be consuming several hundred tablets of an antid-diarral drug instead of having access to a safe and effective drug for

preventing opioid withdrawal. I'll let

that question linger in the room. I want

to suggest that the answer to that question doesn't result in reside in any absolute physico, chemical, biological property of lowamide, but it has to do with how usage patterns are dependent on

social, political and economic contexts that drive the availability of other options. So the attractiveness of

options. So the attractiveness of leramide as a solution for withdrawal is drawn by other changes happening in the world at the time other restrictions that are moved into place. Now as this

becomes news the FDA kicks into action because it's also becoming clear that many people are using low paramide now as as it goes through several media cycles. It's not not only a source of

cycles. It's not not only a source of staving off opioid withdrawal but a source for hydonic effect in its own right. And um here we see one campaign

right. And um here we see one campaign be alert to the potential for leperide abuse. And we see a uh an EKG not

abuse. And we see a uh an EKG not depicting ventricular tacic cardia. But

all the same nobody ever considered the possibility of people taking hundreds or thousands of these pills at the same time when evaluating the safety or efficacy of a drug as an OTC. This is

what I would call a looping effect.

Right? our knowledge, our concepts, our other practices come back and then actually change the basis of how something like safety or efficacy actually exists in the real world and

this is looping effect is a context that hacking develops to suggest that actually our frameworks of knowledge have huge effects in the world around us as well. So I want to leave time for

as well. So I want to leave time for questions. I'm going to get and and so

questions. I'm going to get and and so putting it all together though I want to suggest history is something you live and breathe every day as people working in historical science that we have to

push back on simplified notions of hierarchies of evidence and assert the relevance of observational research in the world as something that is increasingly important. What you do is

increasingly important. What you do is harder now than it was a year ago but it's all the more important. The role of history can give you tools to understand that the place of pharmacco epidemiology is not just an inferior one in the

hierarchy of medical knowledge but a flexible and practical and crucial one.

It's inevitable that phicopidemiological knowledge will emerge as crucial in moments of crucial social controversies and then be heavily critiqued for that interest. But it is all the more

interest. But it is all the more urgently needed for it as well. that

this field thousand strongs today has been strengthened over the past 75 years by the emergence of a data infrastructure that prescribing physicians in the mid- 20th century could not have dreamed of. But this

surveillance can be used for many different purposes and they don't all move in the same direction. The ethics

of pharmacco epidemiology is going to require a lot more attention to how this data is used by human or by AI agents in the future. And finally, I hope to offer

the future. And finally, I hope to offer you a sense that the knowledge that you build actively reshapes the world that we inhabit in ways that make the objects of our study sometimes more elusive.

Right? We can't pretend to exist in a world in which we are simply discovering pure and unchanging knowledge about these things called drugs. That actually

these things are in interaction constantly with the knowledge we produce and the regulatory and economic frameworks that they create. And that is my address to you today. It's such a pleasure and a privilege to be able to

address ISP as a body. Thank you and I welcome your questions.

>> Um thank you for a wonderful talk. I I

really enjoyed it. But I think uh you touched upon something in the hierarchy of evidence which I think is a a blind spot and particularly that uh detaching

of systematic reviews and metaanalysis from the rest of the evidence because first of all most importantly systematic reviews and meta analysis are two very different things. And while I think it

different things. And while I think it it makes perfect sense, particularly when looking at observational studies, to go over all the known studies, talk about their different strengths and

weaknesses and possible biases, and then come together and try to assemble a jigsaw puzzle that gives you the the best possible picture of of of

knowledge. Um, that that works pretty

knowledge. Um, that that works pretty well. However, uh including metaanalysis

well. However, uh including metaanalysis in that um plays upon I think the inferiority complex that we may have in

epidem phicco epidemiology that we kind of use it as a trump card. We can go to the top of the pyramid there and in fact metanalysis of observational studies is

a categorical error. When you do meta analysis, basically you're trying to collect studies at least randomized controls trials to try to put them all

together and say we think that these studies are unbiased and that their differences have to do with statistical noise and statistical and we can pull statistical power and come up to what

they're what they're all really saying.

Observational studies don't work that way. Statistical power is seldom the

way. Statistical power is seldom the problem.

the problem of differences in methodologies that lead to different biases. And so when you're doing a meta

biases. And so when you're doing a meta analysis of an observational study, all you're doing is averaging out the biases. And in fact, you're kind of

biases. And in fact, you're kind of favoring the the less careful studies because the more confounding control you do, the larger the standard errors you get. And so you're you're you're biasing

get. And so you're you're you're biasing that in purpose. And it's kind of interesting how unrecognized that is and how much so then not just in pharmic

epidemiology but in almost anywhere in the social sciences.

>> No, thank you for that point and it's fascinating. I had not considered that.

fascinating. I had not considered that.

It it strikes me um my initial response is to think that you're illustrating again a part of that tacit like building the pyramid around the RCT

and placing the metaanalysis at the very you know at the very top is actually a further way of privileging RCT based knowledge and actually casting aside observational. Now one of the

observational. Now one of the interesting things is that you know you might say well each observational study the biases are so very different right and yet we don't think that way when we

think about poolled analyses of RCTs but if one thinks of how a set of different RCTs done at different moments in time actually contain different bases of what

is the standard of practice right that even different RCTs can't be pulled in the way that we like so we're privileging meta analysis in that way still also part you know participates in that fiction I I think if you do a

careless meta analysis and don't take those factors into consideration, you have that problem with RCTs. And the

whole point >> is that you know you try to do a quality screen and all these sorts of things in as as good methodology.

>> But it you know it still has a leg up on observational studies and and that at least in theory a meta analysis will work.

>> Yeah.

>> Well, it it I can't see what it would it what it would be appropriate for observational studies. You don't have no

observational studies. You don't have no reason to believe that those errors will average out.

>> Yeah. Yeah, and I think that you're right in this notion of a category error, right? So that the way that that

error, right? So that the way that that magnifying glass is drawn, you know, perpetuates that error by suggesting that this lens can be used in the same

way of looking at every study. Um, oh,

sorry.

>> Wonderful talk was was very fun. Um, I

wanted to I have a comment. I have a suggestion actually. Okay. So, so

suggestion actually. Okay. So, so

specific to our to our discipline, um there's also this evolution of different types of designs, different sort of things that we do now that we didn't do 30 years ago. The new user design didn't

always exist. The active comparator

always exist. The active comparator didn't always exist. What you did that there were propensity scores, what you did with propensity scores where you're matching or you're waiting. Now, we have

target trial emulations. And some of these concepts actually come and go, right? So there there in some instances

right? So there there in some instances there's this very specific thing that one should do and 10 years later it actually got dropped and you know we moved on into another direction. I'm

wondering whether you know this wouldn't be something you know where you could expand on I don't want to propose your work but I mean is has um you know some

committee that also focuses on history and things like that. I I think it would be really interesting to track that you know how we actually have evolved as a science and developed our ontology and

changed our ontology to use your your terminology.

>> Well, that's a fascinating comment. I

appreciate it because you know one of the problems in histories of science or histories of medicine is uh a a sort of a we oftentimes revert to linear

progressive narratives. Right. Right. If

progressive narratives. Right. Right. If

I'm drawing it for you, it would go like this. Right. which is that we started

this. Right. which is that we started here, we developed this method, we developed that, now we know this, and those people before us weren't stupid, but you know, maybe they're we still think of them as somehow like children

compared to us and that we have these techniques now. And what your comment

techniques now. And what your comment highlights is that actually the history of a discipline, even in terms of its methods, doesn't simply work in that way. It's not simply a cumulative, you

way. It's not simply a cumulative, you know, building of better and better. Um,

but that's actually there's there's there's methodological vogues that come and go. there are things that get

and go. there are things that get discarded that maybe we should pay attention to once more. I I found this myself um again in my in my first job in in in division of phicop and phmic

economics um in the moment of the ascendancy of comparative effectiveness research in the early um 2000s in which if you went back to the late 50s there

was a proposal for comparative effectiveness research before the keer Harris amendments were even passed.

Right? And so rather than simply building a system of RCTs, we could have seen a drug knowledge landscape that was built on comparative effectiveness

research, but that's not the course that was pursued at that time. So

historically, you go back in the notebooks of people in that time and you actually find methods that are worth redeveloping in the present day. Um, so

I really like that as another invocation of a usefulness of what history can do.

a set of solution spaces that we've abandoned in the past but that are actually still vibrant and possible.

>> So great talk. I'm I'm a pharmacist in the room and I did not take offense to your uh discussion about >> uh the prescription but along the same lines around the history of the evolution of the methods you started to

hint at some of the history of of the data evolutions and I remember just being my own pharmacide epidemiology historian filling prescriptions in the '9s and getting real time adjudicated

claims and here we are and we still don't have real time medical claims because those take six months to to settle down and then we also have the the wearables and all of the linkage that can happen but now because of the

environment that we live in with regards to privacy and HIPPA and deidentification and tokenization sometimes linking this data uh can be very difficult I also remember a time

where I sat in a clinician's office with a laptop fund I was working for I was interning at a pharma company and collecting data out of charts something I could not do today right and so I

wonder if there's also some thread of that um continuing that discussion I mean you started with the AMA and some of other prescription type things, but you could really continue that story of the data and where we really need to go

to advance the field given that we have the methods, but we often don't have the data.

>> I I I love that. I love the point you're making here. And I I'm not trying to

making here. And I I'm not trying to sell you books, but the um the the last book that I wrote, which was uh the doctor who Wasn't there, it's really a history of what we might think of as the

iterative promise of electronic media in medicine, right? And so uh I started

medicine, right? And so uh I started writing it in in a moment when I had just taken this job at Hopkins was you know about 12 or 13 years ago and I met the director of the um the medical

residency program and it was July and I asked him how he was doing. He said,

"Well, you know, it's it's July, so at this point, I can recognize most of the interns from the back of their heads."

And this was this real realization that, you know, the computer was part of a dramatic shift, the shift into digital medicine, which was both feared and seen as something that might take away certain elements of doctor patient

relationship, but also had a lot of hopes associated with it as well. And I

realized as a historian that there's many other moments earlier in the 20th century, the the advent of the telephone, right? the advent of the

telephone, right? the advent of the radio, the advent of early mainframe computer systems right in the 50s and 60s in which similar promises of both this is either going to ruin medicine as

we know it or this is going to completely create a seamless environment for information to flow um are presented and one can come up with a really an undulating history of the outsized hopes

and fears that are made for a new technological platform and then what actually happens and and the takeaway I want to give here is really that What's

crucial is that we don't follow up. So

the if we if we go back to um the the early uh the transformations you know in in you know that led to the overall shift into electronic medical records in the early 20th century you know early

21st century so many of them were based on this promise of seamless linkage so that any patient could have all of their data compared and I have to say I'm looking at Jod here because I know that as a clinician in Baltimore she faced

the same thing but you know every week I spend a tremendous time in my clinic trying to parse out the medical records of a patient who has not been receiving all of their care in the Hopkins system

assumes because it's a very reasonable assumption that all this data should be available to me electronically and and it's not right. So how is it that we continue to fail on what are the fundamental capacities of these

technological systems is it requires a mode of governance of following up on those promises that we simply do not insist on in this country or >> Yes.

>> Thank you so much. It was a very interesting talk.

>> Thank you.

>> I have a quick question to provide more comprehensive overview of the historical reflections. What are your predictions

reflections. What are your predictions for the future in in your study? and and

you already hinted that we're in a midst of a pendulum uh swing, right? So,

please elaborate on that. Thank you.

>> So, thank you for that. You know,

historians are we're terrible as futurologists. We like to claim that

futurologists. We like to claim that we're we're much better at the rear view mirror, right, than in looking ahead of us. Um I do think that the kind of the

us. Um I do think that the kind of the kind of predictions that I can give are kind of analogous to building off of the response to the last question, which is that what are the things that we fail to

learn from past occasions that we can try to insist on in the present. Right?

So when we look at the radical ways in which we understand AI to be transforming our landscape in both very hopeful ways of new muscular approaches to research that we can do that we never

could before but also in really you know deeply concerning um sinister ways in which we were concerned about health privacy. We're concerned about the the

privacy. We're concerned about the the accumulation of data and the building of more firewalls as data is understood as a form of intellectual property. Right?

So the real question for me is um my pred rather than predict what would happen. I would love to say what I would

happen. I would love to say what I would insist on is that history teaches us that we should not expect the promises of a newly emergent technology to simply be delivered unless we actually follow

up on them and insist on them. And so we need robust and tighter loops so that we understand those looping effects. We can

monitor what these technologies are doing and we can alter course before it becomes too entrenched and sentiment.

Yes, thank you.

>> Thank you, Jeremy, for this is spectacular talk and you gave me a a reading list that's going to keep me busy for the rest of the year. So, thank

you for that. Um, I wanted to ask for your your last comment there um made me want to make sure I clarified when we talk about a looping effect. It it

struck me that as a discipline, our task as a community is to identify risks, develop strategies to mitigate them so that those risks are not observable. the

consequence is future time. We don't

observe the risks that counterfactually would be there if we're successful and and as you're highlighting like future premise like one of the challenges there is if we're not observing the risks that

we've appropriately mitigated that means that they're not observable which means future of AI recommendations based on data that doesn't observe things is not going to have that state of knowledge.

So I just want to make sure I'm understanding the phenomenon, the looping effects, and how is we as a community can think about the fact that we're positively constructing these

situations, but the consequence is that the counterfactual world is different than had we not actually been having the impact that we aim to have.

>> No, thanks for that. And and I I I uh I love this question. I think that the looping effect is not necessarily a bad thing, right? It's just understanding in

thing, right? It's just understanding in a way that our concepts influence the world that that we live within, right?

On some level, it's deeply intuitive, right? That we would hope that that the

right? That we would hope that that the knowledge we produce has an impact in the world. Um, and in a way kind of, as

the world. Um, and in a way kind of, as you're pointing out, tying these three answers together. I would argue that a

answers together. I would argue that a mode of regulatory knowledge making that only thinks about how an RCT can produce some kind of knowledge that is timeless

about a drug once it's set into action.

um is brittle, right? And is going to get us into trouble. That a field that is capable of continuous regular loops that monitor what's happening in real world data and can actually bring them

back and show how they're correlating with the present is actually supple and flexible and able to help us understand feedback loops in real time in beneficial ways. And so what I'm

beneficial ways. And so what I'm basically doing is presenting another stump case for pharmacidemiology as a field overall. Right? So what is

field overall. Right? So what is pharmacco vigilance but the ability of insisting that there's kinds of knowledge that just require tight loops.

Yes.

>> Thank you very much. Fascinating talk

and I learned a lot. Um my question to you is a combination of a clinician and a medical historian because you would see the big picture and the long-term trends. When I came to

practice medicine um death from myioardial inffection was about 30%.

Now you go to hospital it's about 5 to 7% with all the invasive procedures and so on. Now in the same period say 40

so on. Now in the same period say 40 years in pharmacco epidemiology and pharmacco vigilance how much did we improve the outcomes in

terms of drugs staying on the market before decisions about uh the restrictions or withdrawals are ma are

made and so on. We have done some work which shows some improvements. So the

the the the evolution has has delivered but not as spectacular as cardiology or or other disciplines. I am interested because you are in a very unusual

situation of being both a clinician and a historian. What is your perspective on

a historian. What is your perspective on on the point which I'm making?

>> So that's a great point. I'm I'm

recognizing partly because of Jod's presence at the podium that I'm not gonna have time to give you a full answer, which is good because I I actually don't have a full answer, but what I what I have is a sympathy to the

problem, right? Which is that yes, other

problem, right? Which is that yes, other fields of medical knowledge production, other fields of therapeutic knowledge production can document their public impact that I think people in this room understand that pharmacy has had a huge

impact on the public health. Our need to document it right now is substantial. So

thinking methodologically how to do that historically in a long array and shows in a pre-farmarmaccovigillance and a post-farmarmacco vigilance world here are the improvements in longevity um and health that can actually be attributed

to the knowledge produced in this field.

Um I think that's a that's an an important project and I do have some ideas methodologically how to do it which I'll share with you on the side afterwards but for now I want to I I need to bring us to a wrap and so I want to thank all of you. It's just been such

a pleasure, especially since my own academic career started in the field of phiccoi to have the the ability to be here today addressing this. So, thank

you again.

I want to thank Dr. Jeremy Green for a wonderful talk. Um, please everyone

wonderful talk. Um, please everyone enjoy lunch and come back here for the annual meeting and award ceremony at two. Thank you again.

two. Thank you again.