Interesting study

Alex Simmons said:

I hear Powertap are introducing a new CPU with an inbuilt "Infinite Improbability Drive"

Anyone with better splits will have either a whale or a bowl of petunias land on their head.

<clapping> Wow....you went deep for that one. Very nice. 🙂

Alex Simmons said:

I hear Powertap are introducing a new CPU with an inbuilt "Infinite Improbability Drive"

Anyone with better splits will have either a whale or a bowl of petunias land on their head.

..... apologies to the late Douglas Adams

Resturant Mathematics, truely mistifying :eek:

acoggan said:

The end result is that although many scientific studies read as if they merely confirm the investigators' preconceived notions, this should not be viewed as "confirmation bias".

I'd agree that having the findings match the hypothsis doesn't neccessarily mean it's always a care of confirmation bias. In this case however?? It's pretty clear cut. I know it's not PC to crack on others in your profession (which is why I did it!).

Let's face it: all this study shows it that the specfic training was more effective that what the control group did--it certainly does nothing to speak to the efficacy of 'SE training'. If her 'likely conclusions' aren't a case of confirmation bias, I don't know what is....

daveryanwyoming said:

My thoughts exactly. The scientific process requires formulating a hypothesis prior to testing. By definition you have some preconceived result you're testing. That doesn't undermine the results if you carefully structure the test including the control and are rigorous and honest in your interpretation of the results. If you aren't you can be sure an unbiased peer review will set you straight. If you hand pick your "peers" well then you can get away with a lot but I'd argue that stretches the term "science".

BTW there are thousands of published studies that produced negative results. Sometimes they were structured to challenge an untested belief(in which case they're again proving their hypothesis) but in my field (RF engineering, signal processing and wireless systems) I've made use of many research studies that failed to prove the original hypothesis but led to other important findings and spawned future studies. Lot's of scientists publish results of negative findings. Heck classic science is full of them, take Michelson and Morly who set out to find the "ether" believed to be necessary for light waves to exist. They failed miserably because it doesn't exist and set the stage for Max Planck to create Quantum Theory and eventually for Einstein to develop the Special Theory Of Relativity.

-Dave

I think the distinction between preconceived bias and hypothesis is that a bias is the conclusion that the researcher hopes and wants the study to show where a hypothesis is less dependent upon the personal desires of the researcher. I think the Daubert standards used by the federal courts are actually a pretty good test of whether a particular study or theory is really scientific: (1) whether the theory can be and has been tested; (2) whether the theory has been subjected to peer review and publication; (3) the known or potential rate of error; (4) the existence and maintenance of standards controlling its operation; and (5) whether it has achieved general acceptance in the relevant community. In other words, by this standard, this researcher would not be able even to testify about her theory. It really satisfies only the first prong. And, in my view, there is a lot of junk out there passing for science that fails several prongs of this test. IMHO, constructing a test that essentially is designed to prove your own pet theory is not any better than gut instinct. Don't get me wrong, a lot of great coaches and riders have gotten pretty far with gut instinct. If we are at the point where it is difficult to recognize the distinction between confirmation bias and an intelligent hypothesis, then I think it is fairly self evident