Misleading with Numbers
Reading: Misinformation in and about science (West and Bergstrom, 2021, PNAS).
Mentions:
Matthew effect (Wikipedia):
…the tendency of individuals to accrue social or economic success in proportion to their initial level of popularity, friends, and wealth. It is sometimes summarized by the adage or platitude "the rich get richer and the poor get poorer".[1][2] The term was coined by sociologists Robert K. Merton and Harriet Zuckerman[3] in 1968[4] and takes its name from a loose interpretation of the Parable of the Talents in the biblical Gospel of Matthew.
Simpson's Paradox is a statistical phenomenon where an association between two variables in a population emerges, disappears or reverses when the population is divided into subpopulations.
Examples of Simpsons paradox -
University of California Admission Rates 1. A study showed that, overall, men were accepted more than women (44% vs 35%). However, looking at each department, women were usually accepted at a rate equal to or higher than the rate at which men were accepted. What was happening? In fact, women tended to apply to departments which were harder to be admitted into. 2. Kidney Stone Treatment / Ambulances vs. Helicopters Advanced surgical procedures should perform better than traditional treatment on kidney stones. When the data is grouped into treating small kidney stones and large kidney stones, the advanced surgical procedures outperform traditional treatment in each group. However, when all of the cases are combined, the traditional treatment outperforms!
How could this be? Well, the advanced surgical procedures were used more frequently when the kidney stones were large. Accordingly, these cases had high failure rates relative to smaller stones. Thus, because the advanced surgical procedure was used most in "tough" surgeries, it performed "worse" overall than traditional treatment.
In fact, there is an analogous result with medical evacuation helicopters and traditional ambulances. In the overall data, the helicopters actually do worse at saving lives than ambulances, but this is because they are sent to the higher-risk situations.
The Will Rogers phenomenon, also called the Okie Paradox, is when moving an observation from one group to another increases the average of both groups. It is named after a joke by the comedian Will Rogers in the 1930s about migration during the Great Depression:[1]
When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states.
All Rogers was attempting to say was that, in his view, the least intelligent Okie is always smarter than the most intelligent Californian…
One real-world example of the phenomenon is seen in the medical concept of cancer stage migration,[2] which led to clinician Alvan Feinstein coining the name Will Rogers phenomenon in 1985, based on a remark by a friend who attributed the quote to Rogers.[3]
In medical stage migration, improved detection of illness leads to the movement of people from the set of healthy people to the set of unhealthy people. Because these people are actually not healthy--merely mis-classified as healthy due to an imperfect earlier diagnosis--removing them from the set of healthy people increases the average lifespan of the healthy group. Likewise, the migrated people are healthier than the people already in the unhealthy set--their illness was so minor that only the newer more sensitive test could detect it. Adding them to the unhealthy raises the average lifespan of that group as well. Both lifespans are statistically lengthened, even if early detection of a cancer does not lead to better treatment: because it is detected earlier, more time is lived in the "unhealthy" set of people. In this form, the paradox can be viewed an instance of the equivocation fallacy.[citation needed] Equivocation occurs when one term is used with multiple meanings in order to mislead the listener into unwarranted comparisons, and life span statistics before and after a stage migration use different meanings of "unhealthy", as the cutoff for detection is different.