Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> These people obviously know that their studies can't be replicated

I don't think that's fair to say at all. I can come up with several ways a study can non-maliciously arrive at a false conclusion:

1. The researchers may accidentally leak information to the participants regarding the hypothesis being studied. When using human subjects, it's very difficult to avoid biasing them toward results they may think you're looking for.

2. Researchers may not sufficiently blind themselves during the experiment, causing them to have undue influence on the outcome, even if they're not consciously trying to exert it.

3. Some unknown confounding effect may be at play during the experiment that wasn't properly accounted and controlled for.

Those are just three I could think of. When dealing with behavioral sciences, I can't imagine how difficult it must be to design a test protocol that eliminates all the messiness of the meatbags being studied.



I helped run a clinical trial for an antidepressant one time. It was a double-blind randomised within-subjects-replicated crossover design. All that control was complicated, time consuming and very expensive. Only the hospital pharmacy knew whether participants were in treatment or placebo for a given session, and we never met the pharmacist, just received an orange vial. But we're pretty sure information still leaked, because drugs have side-effects. There's nothing to be done about this. Use an active placebo, you might say, but now you've added "get ethics approval to administer a nauseating placebo" to your 9-month-long ethics todo list. And then your placebo is worse than your active and the whole study is screwed because everyone spewed up in the MRI. Meatbags are tricky indeed.


4. Bad luck

A proportion of studies will produce false positives just by chance even if they're conducted perfectly.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: