Cohen's (1994) criticism of null hypothesis significance testing (NHST) highlighted several key points that many researchers and colleagues have emphasized, as well as problems in statistics and research that have been discussed for decades. The practicability of null hypothesis significance testing (NHST) remains an issue and is often insisted upon as a panacea, especially in psychology. A result can be statistically significant but still not meaningful in a real-world context.
Cohen (1994) reasoned using a syllogism to illustrate a positive test for schizophrenia, demonstrating an ironic result through statistical significance testing. An error in elementary logic made frequently by NHST proponents and pointed out by its critics is the thoughtless, usually implicit, conclusion that if Ho is rejected, then the theory is established: If A then B; B therefore A. But even the valid form of the syllogism (if A then B; not B, thus not A) can be misinterpreted (Cohen ,1994).For my understanding, taking random people with PTSD, such as veterans who lost legs in war, women who were raped, adults who were molested at home as children, and other typical stressors causing PTSD, as a sample to test. The null hypothesis significance testing (NHST) may be positive and practical for the general population with PTSD, but it is difficult to interpret.
Cohen advocated for a greater emphasis on effect sizes and confidence intervals, which provide more information about the magnitude and precision of an effect. Meta-analysis, with its focus on effect sizes, is a notable feature in the contemporary research landscape (Cohen, 1994). Effect sizes help researchers understand the practical significance of findings, which is essential for applying research results in real-world settings. Cohen (1994) pointed out what is alarming in the psychology field, in which null hypothesis significance testing (NHST) can be fallacious or even erroneous in the real world. As researchers, we have a considerable array of statistical techniques that can help us develop theories of depth, but they must be used sensibly and heavily informed by informed judgment (Cohen, 1994). Statistical analysis needs nuanced approaches that work effectively.
Reference
Cohen, J. (1994). The Earth is round (p < .05). American Psychologist, 49(12), 997–1003. https://doi.org/10.1037//0003-066x.49.12.997