Posted by: zyxo | July 13, 2009

Men Are More Accurate than Women… or lousy statistics ?


Target Corporation
Image via Wikipedia

A study of Duncan Irschick at the university of Massachusetts drew my attention. It says :

Men Are More Accurate than Women When Hitting a Target with Force in the Dark

The story in itself is interesting, but what particularly struck me was that It was quote ” … a small study …” end quote.
I totally agree with that since they “…tested four male and three female adults”.

Yes, right : 4 men and 3 women.

The first reflection of somebody with a statistical/data mining background is : “How on earth can a self-respecting scientist publish results on differences between men and women with such a small sample ?”. I not only mean self-respecting, since he is also respected by oythers and a first-class scientist.

So there must be something else. Could it be that with such a small sample you can indeed do some thorough statistics ?

Let us try it out.

The case at hand is men and women hitting at something with a hammer. I do not know the details, but for the present purpose it is simple to use some fake data.
Let us take an extreme case :
If they must hit some target, suppose that the four men missed the target by 20, 18,22,and 21 centimeters respectively. The women, being much accurate missed only by 3,5 and 6 centimeters.
With a simple t-test we find out that the two means of 20.25 for the men and 4.67 for the women are significantly different (p=030003; two-tailed).
So if all 3 women are far better than all 4 men we have a proven case !

With one woman being a bit less accurate than one men we get the following : let us assume that the best man in stead of missing by 18 centimeters misses by 10 centimeters and the worst woman misses by 11 centimeters in stead of 6.
The difference is still significant (p=0.0227; two-tailed).

Let us try a third one : we take case 2 but make the two of the three worst men somewhat better and the second best women somewhat less accurate (men: 10,14,15,22; women 3,8,11) : it is not significant any more (p=0.0664; two-tailed)

So, even with such small samples it is perfectly acceptable to draw conclusions.

But for a data miner, who is used to work with millions of observations it still feels a bit weird !

Did you liked this post ? Then you might be interested in the following :
Howmany inputs do data miners need ?
Oversampling or undersampling ?
are men and women different ?

Reblog this post [with Zemanta]

Responses

  1. How would we know that it was their sex, specifically, which was the determining difference in performance between these two very small (!) groups?

    For any 7 people chosen at random, surely there are many differences between a subset of 4 of them and the remaining 3, such as: their occupations, experience wielding hammers, education, age, race, health, visual acuity, general physical condition and so forth.

  2. Will,
    you are totally right. With statistics you never know for sure. The tests only say that the probability that the observed difference between the two groups (genders) are due to chance alone is less than 5%. Meaning that in 5% or 4% … of the cases you will see another outcome.
    Compare it to what we know for sure : men are on the average taller than women. So take at random 4 men and three women and do the same exercise … I know a lot of women who are taller than a lot of men. And yet, statistics will tell you that men are taller than women.
    About the other differences (education, age …) you could test these characteristics as well.
    But allways remember that if you do 100 tests, you will find 5 or 4 or 3 or… of them that will be statistically significant, just by chance and hence give you false information.
    Zyxo


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: