How to Understand Science Better Than 80% of People
Or why not thinking in terms of distributions causes errors
I’m about to share some ideas that will almost certainly inflame a percentage of my readers. In anticipation of their wrath, I want to clarify something about psychological information.
Nearly all psychological information is information about distributions. Yet most of us are not accustomed to thinking about the world in terms of distributions.
I hope that today’s post will be a simple and helpful primer on how to think clearly about distributions. Maybe it will even make what I say in my next set of posts a little easier to stomach.
A few weeks ago, at the prison, I was explaining the research on parenting styles to a group of inmates. I drew this nifty little matrix and explained that the “Authoritative” parenting style—one that is high in responsiveness and demandingness—delivers the best outcomes.
Immediately, an inmate said, “How come that’s the style I got growing up, and I ended up here in prison?”
It was an “I could have had a V8” moment.
Oh, yeah, I thought, “Why do I keep forgetting that at least one person claps back at me nearly every time I share a bit of psychological research?”
I’ve seen this little fact-to-outrage pipeline so many times that I have no right to be surprised when I trigger it.
The main reason people push back on psychological factoids is that their intuition contradicts the factoid. But that’s not the only reason for the pushback. People don’t know how to think in terms of distributions.
Take this simple and true statement: Men are taller than women.
When people disagree with that statement, their mental model is probably something like this:
I assume their mental model looks like this because they often asay something like, “Uh, uh. My niece is 6 feet tall,” or " My neighbor’s wife is taller than he is.”
I have to believe that they wouldn’t argue with the statement if their mental model was of two distributions of height data—something more like this:
When the above is your mental model, statements like “Men are taller than women” don’t seem so ripe for a clap back.
When we see the distribution of the data, a few things become clear, and we are less likely to dismiss the factoid out of hand.
First, height is on a continuum. So, there is a lot of variability in height, with most people being average and only a few being really tall or really short.
Second, men's and women’s heights overlap. So, the statement, “Men are taller than women,” doesn’t mean that every man alive is taller than every woman. Instead, it means that the average man is taller than the average woman. Based on large surveys, that fact seems incontrovertible.
Third, it shows that the tails are very different. [The tails are the right and left ends of the distribution where the line slopes down to the baseline.] Five percent of men are 6’4” or taller, while almost no women are. The reverse is also true. Five percent of women are 4’9” or shorter. But, there are virtually no men that short.
Psychological facts are distributive facts
Nearly every factoid a psychologist or social scientist shares is information about a distribution, not an individual. A careful scientist understands this. But, most often, the general public does not. Take this example.
The average psychotherapy client is better off than 80% of those who did not receive treatment.
More than one person—WITH A MASTER’s degree—has challenged me on this robust research finding. They are certain that I am wildly exaggerating the effectiveness of therapy and have told me with grave and dour faces that I had better have a citation for such an audacious claim because they had serious reservations about my ethics. My best guess as to why they would panic about my statement is that they think I have said, “Therapy cures 80% of patients.” Their mental model probably looks something like this:
With an understanding that nearly all psychological facts are about distributions, the mental model should look more like this:
Three things are important here.
First, people are not cured or uncured. Their symptoms of distress improve along a continuum. For example, a very successful drug treatment program won’t cause 80% of participants to become abstinent. It will reduce drug use. Some will become abstinent, but others will simply do fewer drugs, have fewer arrests, or have fewer sick days. To state it technically, “The average treated person has better outcomes than 80% of the untreated group.”
Second, there is a tremendous amount of overlap. Some untreated people will have fewer symptoms than some treated people, and vice versa.
Third, the tails matter. There are almost no treated persons in the left tail and almost no untreated persons in the right tail. Tails matter in distributions.
Prisons are for tails
To see how important tails are, consider this basic psychological research finding: “Men are more disagreeable than women.” In most cases, men’s disagreeableness is either not noticed or not a problem. But extremely disagreeable people end up in jail because they don’t like to do what other people—or society, or cops, or anybody, really—tell them to do. They care so little about being agreeable that they cause problems for themselves. “Yeah. I saw the sign that said 55 MPH. So what, officer? You wanna make something of it?”
Men’s disagreeableness explains this factoid: “As of December 2024, the Federal Bureau of Prisons (BOP) reported that 93.5% of inmates were male and 6.5% were female.” Citation
This doesn’t mean that most men are in prison, or that there aren’t a ton of women in prison. But it does mean the ratio of men to women is such that nearly all prisoners are men.
In my next few posts, I am going to say some facts about liberals and conservatives and men and women as they relate to addiction. When these posts inevitably tweak your nose, please remember what I shared here:
Psychological information is information about distributions and:
Distributions display information along a continuum with considerable variation.
The distributions often have significant overlap
The tails of distributions really matter