Probability is not intuitive – even for very numerate people. I have heard accountants and engineers say their stats course was the one they struggled with most of all.
For someone in the social sciences it is the bane of their lives, because social sciences rely on statistics a lot and they tend not to be very numerate.
Doctors, who have to assess life and death risks, struggle with it too. Medicine will be my first example.
Suppose something feels wrong and you go to your doctor for help. He thinks it might be a form of cancer that afflicts one percent of people your age. He suggests a test which is 95% accurate, both if you have it and if you don’t.
The test comes back positive. You ask your doctor what your chances are of having the cancer. He replies he is afraid it is 95% and recommends chemotherapy.
Well, that is what over 90% of doctors tend to do, but what is the real chance that you have cancer given that you test positive for it? It is actually only about 16%. Say whaaat!!?
Suppose you are one of 2,000 people who had the test. Only one percent, or 20, are likely to have the cancer, and 19 (95%) of them will test positive.
“See?” says your doctor.
Not so fast. Of those 2,000 who had the test, 1,980 are likely to not have the cancer, and 99 (5%) of them will falsely test positive. So, of the 19+99 = 118 potentially positive test results, only 16% (19/188) will have the cancer. The other 84% won’t.
You could end up having a form of therapy that involves poisoning you for nothing. It really pays to have other, perhaps a few other, tests that rely on a different mechanism before you submit to harsh treatments. The medical fraternity has learned and now recommends not having a routine test when the condition is rare and the treatment harsh.
The same principle applies to random drug testing. A lot of innocent kids could suddenly be in real trouble. The general principle is, if the incidence of the target – such as cancer rates – is low and the test is less than perfect (they always are), then there will be a lot of false positives which may swamp the true positives.
On a different tack, it turns out people have a radically different idea of what probability is high, modest or low. Intelligence agencies often ask agents to give their view as to whether a particular outcome was highly likely.
For years they happily collated the opinions and acted on the collective judgements. Then one day someone thought to ask the agents what probability of something happening counted as ‘highly likely’. He discovered that the same verbal judgement was associated with a very wide range of probabilities.
Operationally useful
Some think 5% is unacceptably anxiety-provoking, while others think we should call anything with a probability of less than 60% ‘likely’. There will always be people who are anxious or calm about any number. However, because the actual number is more operationally useful, they now ask the agents to give an actual number instead of a verbal label.
Superforecasters know to use a base rate. If we know that projects tend to take twice as long, and cost twice as much, as planned, then that is the base rate for project plans? To predict the finishing time and cost of a project, one should start with the assumption it will be close to twice as long and costly as planned.
One can use any unusual circumstances to adjust slightly up or down from there. The outcome of any event is more likely to fall close to the average than far away from it. So, those who pick the average tend to be right more often that those who pick something more interesting. Superforecasters know the value of the typical rate of the event they predict. That is why they outperform other forecasters.
The average values of say mathematics ability, to a field that depends heavily on mathematics, may differ by an insignificant amount between two groups. Yet at the extremes one of the groups may be overrepresented at remedial arithmetic and underrepresented among Harvard science and mathematics professors.
That seems like evidence of systematic discrimination. It isn’t. For example, the variation in IQ scores is about 15 points. If two groups differ by a mere 1.5 IQ points on average, the lower scoring group will have only 8.7% fewer people exceeding the average IQ of the other group – 46% vs 50%. A mere 1.5-point difference is well within the error range.
No one will be able to notice any difference between people with scores that differ that little.
But things change at IQs above 130 (elite), or below 70 (borderline retarded), even though an IQ of 128.5 is also not noticeably different from one of 130. The problem is that the lower scoring group must have 27.4% less representation (1.786% vs 2.275%) at IQs above130. The same, but in reverse, applies to IQs below 70.
Accusations of bias
Those sorts of differences in an elite profession or remedial classes, are noticeable and frequently lead to accusations of bias and discrimination.
That happened at the Harvard science and mathematics faculties, where women accounted for well under half the faculty. These faculties select largely on the basis of mathematical ability. The mathematics part of the Scholastic Aptitude Test (SAT) is a good and fair test of mathematical ability.
The difference between the average scores of male and female is very modest. The scores of women do however vary less than those of men.
As a consequence, the representational differences at the extreme ability levels (1 in 1,000 or better) required by Harvard are much more noticeable. There was a bitter fallout resulting in Larry Summers resigning as president of the university. One reason for that phenomenon in the US could be that US women are just less interested in mathematics and physics.
In Iran, the picture is different and it is women who are overrepresented in elite mathematical professions.
The same applies to some crime stats, where the overall crime rate difference between groups can be minor, but the differences in prison rates quite marked. A related ‘fun’ fact is that more lenient punishment exaggerates those differences.
The above examples show that a misunderstanding of probability, or statistical issues, can lead to a lot of social strife or actual harm. These are some of the statistical principles that people should know. Every field, such as economics, for example, has a few.
[Image: by GoldenDayz]
The views of the writer are not necessarily the views of the Daily Friend or the IRR.
If you like what you have just read, support the Daily Friend