This resource was featured in the NeuroJobs Career Center. Visit today to search the world’s largest source of neuroscience opportunities.
Researchers desire to make the most eye-catching discoveries to have a sensational impact on the academic community and general public. For this reason, researchers strive to obtain results that support their claims to the greatest extent possible. This "sexy science" has led to a spate of vigorous publications by researchers, and has shaped the scientific glory of today.
However, the results of such studies are sometimes unreproducible or derived from falsification. No matter how eye-catching a study is, if it is not credible, it is completely meaningless. Follow the advice in this article to introduce rigorous practices to your research.
Pre-register the study
Pre-registration of research protocols is considered an effective means of addressing issues of credibility and reproducibility of empirical studies. Pre-registration allows methods and analyses to be fixed and open before data collection or processing can begin. It is preferred that reports are peer-reviewed prior to registration so the accuracy of the registered content can be proven.
In conventional practices, researchers have a large degree of freedom. There have been several reports of false-positive results, because researchers can create or rebuild the hypothesis after seeing the results. Or researchers try various statistical methods, and selectively report results which support their hypothesis.
Pre-registering experiment protocols restricts the researcher’s freedom to change their hypothesis or seek multiple outcomes before reporting their work. It is even better if the protocol is peer-reviewed before registration. Various journals such as Cortex have already accepted submissions in this format, and hopefully more will in the future.
Open the data and code, and preprint
The researcher's degree of freedom also lies in the ambiguity of methods and results. The best way to reduce such ambiguity is to make the materials, dataset, and analytical codes directly visible to others. There are already many repositories that can be used for this purpose, for example, GitHub, OSF, and OpenNeuro. Researchers should take advantage of these existing platforms to make their research as open as possible. In addition, the manuscript itself should be made available as a preprint to increases access to the peer review process.
Seek many peer reviews and open their contents
Do peer-reviewed and non-peer-reviewed papers have the same credibility?
Most people would probably trust a peer reviewed paper over one that has not been peer reviewed. Therefore, we can increase the credibility of a manuscript by showing it has been peer-reviewed as often as possible. In recent years access to peer review has become available during pre-submission and post-publication stages.
Many cases of research misconduct are discovered after publication, so if a multi-layered review process is open, readers can see the validity of the research themselves.
I discuss the benefits of multi-layer reviews in more detail in a recent paper.
Do not dress up the study
It is easy to exaggerate when discussing or promoting research results.
Scientists can fall into the trap implying a definitive conclusion, even though the evidence is not as strong as it could be. Such discourse can affect the lives of the public via public dissemination of conclusions and policy decisions.
It is important for scientists to state honestly what and how their research contributes to science.
Changing values changes science
In recent years, it has become clear that it is not possible for an individual or single team to correctly arrive at any major discovery, and each player in science continues to take detours before arriving at the truth. What is needed to make major strides is direct and indirect evidence to test claims, models, and theories, along with replication/reproduction of claims, models, and theories to further ensure their accuracy.
Rather than trying to prove we have done something great with our own research alone, we should construct our research as a recipe for building a system to do great things with.
To provide others with a recipe to build from and reproduce, we must practice rigorous science, we must build a coherent research plan, share stimuli and analytical methods, and seek advice from a variety of people. These steps to move away from “sexy science” and towards rigorous, open research will help set the tone for the value and culture of the scientific enterprise and improve our own credibility as researchers.
About the Contributor
