Picho, K., & Artino, A. R. 7 Deadly Sins in Educational Research. J Grad Med Educ. 2016 Oct; 8(4): 483–487.
A very interesting article as the title says! In conducting and reporting of medical research, there are some common pitfalls which may raise the question of validity of research. With the purpose of creating awareness about these mistakes and to encourage better research outcome, this article focuses on what the authors term as “7 deadly sins” in educational research.
Sin 1: The Curse of the Handicapped Literature Review
In scientific research, a comprehensive literature review is very important because it provides understanding of strength, weaknesses and gaps in research and establishes the framework for current research. However, this important aspect is often affected by bias in the review to support the hypothesis, and still worse is when the review is sometimes done after completion of the study. (Yes, this does happen!)
We would add here that a poor literature search without a proper search strategy results in an inadequate literature review. Only when a search is executed with a systematic methodology, can the literature review be good and can the rest of the research be properly executed.
Sin 2: Inadequate Power
In quantitative research, power is affected by small sample size, size differences between different groups or unintended consequences affecting on true positive findings. Hence it is important that a power analysis should be conducted prior to data collection to avoid these negative consequences. And besides increasing sample size, power can be increased by improving experimental design efficiency, such as through the use of equal cell sample sizes; matching participants; measuring covariates a priori; and correcting for covariates in subsequent analyses.
Sin 3: Ignoring the Importance of Measurement
Measurement of research outcome can increase or decrease the validity of the whole research study. Measurement errors are usually associated with poorly designed measurement tool, overemphasis on specific aspects, exclusion of a particular aspect and also when the outcome is difficult or too easily anticipated.
Sin 4: Using the Wrong Statistical Tool
Prior to data analysis researchers must confirm that their data format meets the assumption of the statistical tool which they plan to use for analysis. Failure to do so may require a change of the tool or the use of an alternate tool. Hence in order to avoid this error it is better to consult the statistician for the most appropriate tool for the study, at an early stage.
Sin 5: Merciless Torture of Data and Other Questionable Analysis Practices
Data torture is mainly associated with one sided research focus, influencing statistical data and at the extreme, data fabrication and data mining to suit prespecified research hypotheses. Data torture obviously suits the researcher, but is really detrimental for the progress of research.
Sin 6: Slavery to the P Value
P value is generally used in connection with null hypothesis, to represent existence or non existence of differences in population from which the data were sampled. Mostly outcomes of p test (rejecting null hypotheses) are falsely interpreted with confirmation of research hypotheses. Even in the case of a big population, tiny differences make the result practically unimportant. So it is suggested that researchers should supplement the p test with other techniques of analysis.
Sin 7: Lack of Transparency in Reporting Results and Maintaining Raw Data
Wrongly reported data not only affect current studies but also future studies based on it (ie, meta analysis). Hence primary level research must be properly reported with full descriptive statistics with transparency and integrity of raw data.
This article also includes a well structured checklist of recommendations for better research. Referring to this checklist can make a great difference in the conduct of research.
