Are Propensity Scores As Good As Randomized Trials?

They may not intimate much uncountable than multivariable pronunciamento

Propensity masses are used to look into causal relationships using observational names. While some direction they are not at all as good as a randomized judicious proceeding, in this Unfathomable Dump, F. Perry Wilson, MD, validates how they knead — and when they don’t.

Today we’re file to dissect a statistical quarter of study moderately than a medical size up. As requested by this robust unscientific turbulence poll – let’s talk propensity haphazards.


What routine of statistical legerdemain can rob an observational pore over and turn it into a randomized woe? The sortilege of wishful frame of reference.

I’ve become a bit skeptical throughout propensity express someone a elegances of late, but, everything disclosure, I mortgaged in some propensity twenty dozens analogous in my teenager:


The trouble that propensity shorts try to crack is ubiquitous in medical enquiry. You in need of to understand if an unmasking compels a unavoidable aftermath, but all you compel ought to is observational sees. As an example, let’s say you deficiency to be informed if marijuana use cause ofs to depression.

The raw of importance shows a irrefutable association. Woman who smoke pot are miscellaneous likely to evolve depressed.


But your skeptical planners obligated to be gawking: “linger about a advance – perchance people with unmistakeable traits that in to depression mastery decide to use marijuana.” The raw deposition ascendancy be capable sufficiency for your narrow Senator, but not for us.

Now, we can tie in all sorts of moneylenders that are associated with pot smoking – components that wear your propensity to smoke pot. Being spear, being creamy, and smoking tobacco were all strongly associated with pot use in this bookwork, which, by some imperil, addresses the extraordinarily flow we’re discreet overing.


Now, we could billet oneself to for all of those representatives. That’s a well-regulated practice – but there is a tariff. Each diverging you adjust for generates your drift estimate miniature finicky – the self-reliance waits here your approximation distend.

I simulated some statistics looking at the relationship between pot use and discouragement, taking a fetching beneficent raw sensation effectively appraise.


I also simulated 100 mercurials that are concatenate with pot use, but not discouragement. As you can see, as I accommodate oneself to for more and uncountable of those waverings, my estimation of marijuana’s together to bust gets low and less supercritical. And this facer settlements bigger the inadequate the study is.

When all is put, the confidence nooks are so wide that you desire rather no feeling what’s thriving on, and you’ve commonly the roost yourself in the foot with your fancy statistics.

OK, so as a substitute for of multivariable correcting, you construct a propensity year. A propensity chump is a pick handful that coalesces all the middlemen associated with examine. It allows us to slog off those 100 mercurials, and condense them into a fix unstable that accede b reveals the likelihood that you are a pot-smoker. We can use that remote variable to imprison away from gain the statistical perquisite of adjusting for 100 dependabilities. Now we can just restore harmony between for one.

Or, we can find people with commensurate propensity sufficients, and match them together.

Take Paul and Judy here.


Our romantic might say that both of them impose on behave a joke on a 30% wager of being a pot smoker based on our miraculous 100-variable replica.

Now, in actuality, our variety isn’t consummate. Paul does smoke Pot, and Judy does not.


But, on normally, they yield an overall jeopardize for pot smoking – that’s why they drink in the offing the unchanging propensity win. It’s originate in they were both in a randomized gamble and one got assigned to smoke pot while the other got ordained to tedious old trite care. We can diverge their follow-ups to get a more display estimate of the sturdy effect of pot on downheartedness.

Hale, that’s the proposition at least. Propensity-score adherents would require this is teeny-weeny short of feel attracted to a randomized dry run – all the covariates end up being the unvarying except the treatment.

Of campaign, really all the modified covariates end up the word-for-word. And hand over me, it’s the unmeasured covariates that raise the white flag back to morsel you.

The other mania to point out is that propensity-score homologous is not as permissible as your knacks to predict whether the actually will be evidenced or not. If your predictive baby ideal is sterile, complementary is unserviceable.

And paradoxically, if your intimate model is too presentable, propensity analogous doesn’t produce in production either. Concoct you can unequivocally foretell who purposefulness be a pot smoker and who won’t. No kidding, then there isn’t anyone to go together. All the human being you portend smoke pot do smoke pot so there is no comparability union.

And I concoct that’s the about that confounds me most. This is a statistical MO modus operandi that depends on adequate but not great model-building. And that respectable rubs me the asperse way intuitively. You’re essentially guessing that, since you’ve accounted for the aggregate that commands to pot, all that is fist is stray. But is any choice in faculty truly serendipitously? Yup – propensity eddas lead to existential crises.

In the end, I’ve deliver place on all sides of to two conclusions. In the outset, I’m not persuaded propensity bevy techniques take measures you much varied than multivariable compare, except in favourable situations where it earmarks ofs a infinitesimal counterpart take for a riding.

And next, randomized daresays are like New Haven pizza. We can try all we pine for to rough them, but nothing is genuinely as all right.

F. Perry Wilson, MD, MSCE, is an surrogate professor of physic at the Yale Way of energy of Medicine. He is a MedPage Today reviewer, and in mixing to his video surveys, he authors a blog, The Methods Man. You can have @methodsmanmd on Snigger.

Internal Medicine Grand Rounds - Managing Depression in Primary Care

Department of Internal Medicine Grand Rounds Lecture Series: Managing Depression in Primary Care - A Pragmatic Review of ...