"I vividly remember the first experiment I conducted for my PhD in economics, investigating the conditions under which trust forms between strangers. I had built a solid theoretical framework, designed the experiment - in which students played a trust game - carefully, and optimistically named my database 'AwesomeData'. But when I ran my first sessions, the results made no sense. My participants weren't behaving as theory - or even common sense - would suggest, because my set-up made the task too confusing."
"The open-science movement has promoted the sharing of scientific protocols, statistical-analysis programs and data files. These efforts have undoubtedly made it easier to see how researchers come to their conclusions. But methods and findings are just the tip of the iceberg. Research often involves a long journey, during which scientists reassess their expectations, pivot and adapt to unexpected challenges and constraints. All too often, this is hidden from view."
Open-science movement has promoted the sharing of scientific protocols, statistical-analysis programs and data files. Methods and findings represent only the visible portion of research work. Research commonly involves long journeys in which scientists reassess expectations, pivot and adapt to unexpected challenges and constraints. Experimental work can produce confusing or nonsensical results when designs inadvertently make tasks unclear. Many projects undergo multiple iterations, redesigns and rejections before eventual publication. Small pilot tests frequently inform important design choices, such as incentives or wording, which determine whether interventions succeed. Failure to disclose iterations and pilot details undermines assessment of robustness, generalizability and appropriate policy scaling.
Read at Nature
Unable to calculate read time
Collection
[
|
...
]