Wednesday, August 03, 2005

Statistics as anecdotal thinking...

I am reviewing a paper where the authors prop up their not-on-strong-theoretical-ground claims with loads of statistics. I couldn't but help recall
Statistics is to engineers, like lamp-posts are to drunks - they are used more for support than for illumination.
This got me thinking - where is statistics used for illumination? Before we proceed it would be worthwhile checking the definition of statistics given here.

Census, I feel, is a practical branch of statistics mainly because its sample is the entire population. Analyzing the census data could illuminate what already exists, but when one tries to use this data for prediction, it is no longer practical. Similar is the case with polls. How scientific are scientifically conducted polls?

Michael Schemer, differentiating between anecdotal thinking (that leads to superstition) and science in his column Skeptic in The Scientific American May 05 issue says:

What we havehere is a signal-to-noise problem. Humans evolved brains that are pattern-recognition machines, adept at detecting signals that enhance or threaten survival amid a very noisy world. This capability is association learning - associating the causal connections between A and B - as when our ancestors associated the seasons with the migration of game animals.

Unfortunately, the system has flaws. Superstitions are false associations - A appears connected to B, but it is not (the baseball player who doesn't shave and hits a home run). Las Vegas was built on associated learning.

..... - if you scan enough noise, you will eventually find a signal, whether it is there or not.

.....

We evolved as a social primate species whose language ability facilitated the exchange of such association anecdotes. The problem is that although true pattern recognition helps us survive, false pattern recognition does not get us killed, so the overall phenomenon has endured the winnowing process of natural selection. .... Anecdotal thinking comes naturally; science requires training.

The key from the piece above is - if you scan enough noise, you will eventually find a signal, whether it is there or not. This is similar to the fact that any set of data can be fitted to a polynomial equation - I vaguely remember reading that the figure of an elephant can be fitted with a polynomial equation. But what value does this equation have in predicting the next data point?

Statistics is a form of pattern recognition, or rather a method for pattern recognition. If used without the right set of data, it becomes nothing more than anecdotal thinking. Thus statistics are a rigorous method in science is questionable.

Statistics has played, and certainly will play, a huge part in science and engineering - in my area of expertise, thermodynamics, Statistical Thermodynamics is important to answer question Why? rather than How? answered by Classical Thermodynamics. I understand that regression analysis, parameter estimation etc are widely used in science and engineering. But the most important footnote when using any of these methods is their range of applicability - which again depends on the data used.

What I am trying to convey through this muddled post is that unless used within a clear predefined set of constraints or bounds, the use of statistics is equivalent to anecdotal thinking and is divorced from science.

5 comments:

Anand said...

Interesting post Rahul. In a lighter vein, you really can't fit an elephant with a polynomial equation (unless you forget the legs, tail, tusk etc)!

You may find an earlier post of mine interesting: In defense of anecdotes.

Iyer the Great said...

Thanks Anand. Read your post - I certainly agree with you. Anecdotes and statistics work together. Unassociated numbers (statistics) mean nothing.

Charu said...

I was going to point out Anand's post to you... I have seen that people are always cmfortable when numbers are presented - as if they are irrefutable proof - as opposed to stories or anecdotes... I am a qualitative researcher and I face this barrier all the time!

Iyer the Great said...

Charu, I had a long discussion on qualitative research with a couple of friends who are doing research in psychology last week (after reading Anand's post). They were also of the same opinion as you.

Unfortunately, most of the world believes in numbers only. When I was a consulting engineer earlier, I was guilty of using numbers to present the situation to our benefit. I probably should not compain!!!

Anonymous said...

When the Wow Gold wolf finally found the Buy Wow Goldhole in the chimney he crawled wow gold cheap down and KERSPLASH right into that kettle of water and that was cheapest wow gold the end of his troubles with the big bad wolf.

game4power,buy cheap wow gold

The next day the cheap wow gold buy gold wow little pig invited his mother over . She said "You see it is just as I told you. The way to usfineget along in the world is to do world of warcraft gold things as well as you can." Fortunately for that little pig, he belrion learned that lesson. And he just lived happily ever after!