Monday, July 2, 2007

There is something about statistics

Statistics is used everywhere to justify company policies. The challenge on using statistics is to learn not to torture the data. Amazingly, when one has some data, statistics is always use to make the data confess a particular response that the user would like to see. Once the user find something extraordinary (that he/she doesn't like), methods of regrouping, taking out the so-called outliers, etc will be employed to "smooth" the data and ta-daaa... the data analysis will produce something that the user would love to see...

Amazingly, although we know it's not so wise to torture the data, we still do it. Countless of times, I saw it happening, not only in academic scene, but more so in corporate scene. It's just very hard to see that the initial data analysis does not produce something we want.

But, do we really ever do a 5-why analysis on why the data looks wrong or why A is higher although it should be lower? Instead of fiddling with statistics to produce the "wanted" result, perhaps additional analysis (that MAKES SENSE) or simply letting the data as it is is wiser. As my friend said, "Data does not lie". Perhaps there is genuine explanation to the seemingly bewildering scene. It just that when we torture the data, the explanation will almost be definitely overlooked.

No comments: