MPTUO1a: Unit 1 Videos / Contemporary Issues in Methodology and Practice
Week 1 / Unit 1
Video: Contemporary Issues in Methodology and Practice
(insert video link)
Video Transcript – Contemporary Issues in Methodology and Practice
The field of metascience has revealed significant problems with the methodology of psychological research. Psychological research suffers from high bias, low reproducibility, and widespread misuse of statistics. These finding have led to calls for reform from within and from outside the scientific community.
In 1959, statistician Theodore Sterling examined the results of psychological studies and discovered that 97% of them supported their initial hypotheses, implying a possible publication bias. Similarly, Fanelli (2010) found that 91.5% of psychiatry/psychology studies confirmed the effects they were looking for, and concluded that the odds of this happening (a positive result) was around five times higher than in fields such as space- or geosciences. Fanelli argues that this is because researchers in “softer” sciences have fewer constraints to their conscious and unconscious biases.
Further information: Replication crisis § In psychology
Over the subsequent few years, a replication crisis in psychology was identified, where it was publicly noted that many notable findings in the field had not been replicated and with some researchers being accused of outright fraud in their results. More systematic efforts to assess the extent of the problem, such as the Reproducibility Project of the Center for Open Science, found that as many as two-thirds of highly publicized findings in psychology had failed to be replicated, with reproducibility being generally stronger in studies and journals representing cognitive psychology than social psychology topics, and the subfields of differential psychology (including general intelligence and Big Five personality traits research), behavioral genetics (except for candidate gene and candidate gene-by-environment interaction research on behavior and mental illness), and the related field of behavioral economics being largely unaffected by the replication crisis. Other subfields of psychology that have been implicated by the replication crisis are clinical psychology, developmental psychology (particularly cognitive and personality development), and a field closely related to psychology that has also been implicated is educational research.
Focus on the replication crisis has led to other renewed efforts in the discipline to re-test important findings, and in response to concerns about publication bias and p-hacking, more than 140 psychology journals have adopted result-blind peer review where studies are accepted not on the basis of their findings and after the studies are completed, but before the studies are conducted and upon the basis of the methodological rigor of their experimental designs and the theoretical justifications for their statistical analysis techniques before data collection or analysis is done. In addition, large-scale collaborations between researchers working in multiple labs in different countries and that regularly make their data openly available for different researchers to assess have become much more common in the field. Early analysis of such reforms has estimated that 61 percent of result-blind studies have led to null results, in contrast to an estimated 5 to 20 percent in earlier research.
Misuse of statistics
Further information: Misuse of statistics and Misuse of p-values
Some critics view statistical hypothesis testing as misplaced. Psychologist and statistician Jacob Cohen wrote in 1994 that psychologists routinely confuse statistical significance with practical importance, enthusiastically reporting great certainty in unimportant facts. Some psychologists have responded with an increased use of effect size statistics, rather than sole reliance on p-values.
“WEIRD” redirects here. For other uses, see Weird.
In 2008, Arnett pointed out that most articles in American Psychological Association journals were about US populations when U.S. citizens are only 5% of the world’s population. He complained that psychologists had no basis for assuming psychological processes to be universal and generalizing research findings to the rest of the global population. In 2010, Henrich, Heine, and Norenzayan reported a systemic bias in conducting psychology studies with participants from “WEIRD” (western, educated, industrialized, rich and democratic) societies. Although only 1/8 people worldwide live in regions that fall into the WEIRD classification, the researchers claimed that 60–90% of psychology studies are performed on participants from these areas. The article gave examples of results that differ significantly between people from WEIRD and tribal cultures, including the Müller-Lyer illusion. Arnett (2008), Altmaier and Hall (2008), and Morgan-Consoli et al. (2018) saw the Western bias in research and theory as a serious problem considering psychologists are increasingly applying psychological principles developed in WEIRD regions in their research, clinical work, and consultation with populations around the world. In 2018, Rad, Martingano & Ginges showed that nearly a decade after Henrich et al.’s paper, over 80% of the samples used in studies published in the journal, Psychological Science, were from the WEIRD population. Moreover, their analysis showed that several studies did not fully disclose the origin of their samples, and the authors offer a set of recommendations to editors and reviewers to reduce the WEIRD bias.
From an anthropological perspective, scholars applied the WEIRD model to European history, arguing that a powerful Christian Church forced a radical change away from incest and cousin marriages that undermined the role of clans and created individualism in Europe by 1500 CE. They argue that a distinctive Western psychology thus emerged that valued agency, autonomy, and kindness toward stranger. Historians were not involved in that project, and have since pointed out its historical fallacies regarding an all-powerful Church at too early a point in time, and a rejection of cousin marriage that did not happen.
Unscientific Mental Health Training
Some observers perceive a gap between scientific theory and its application—in particular, the application of unsupported or unsound clinical practices. Critics say there has been an increase in the number of mental health training programs that do not instill scientific competence. Practices such as “facilitated communication for infantile autism”; memory-recovery techniques including body work; and other therapies, such as rebirthing and reparenting, may be dubious or even dangerous, despite their popularity. In 1984, Allen Neuringer made a similar point[vague] regarding the experimental analysis of behavior. Psychologists, sometimes divided along the lines of laboratory vs. clinic, continue to debate these issues.
The above video and transcript use material from the Wikipedia articles “Introduction to Psychology” and “Psychology“, which are released under the Creative Commons Attribution-Share-Alike License 3.0.
- Introduction to Psychology. (n.d.). In Wikibooks. Retrieved August 13, 2020, from https://en.wikibooks.org/wiki/Introduction_to_Psychology
- Psychology. (n.d.). In Wikipedia. Retrieved August 13, 2020, from https://en.wikipedia.org/wiki/Psychology
This work is licensed under a Creative Commons Attribution NonCommercial-NoDerivatives 4.0 International Licence. Contact My Psych Teacher for more information.