Critical Thinking

Critical Appraisal of Research Communication: Exploring the Science behind the News
Since 2014, with Marta Gwinn, MD, MPH (CDC)

Critical Thinking - an animation

Video by QualiaSoup

This course focuses on the coverage of scientific research in the media and other nonscientific publications. The goal of the course is to help students develop a critical attitude toward reported scientific findings (‘the News’) by learning how scientific research is covered and what relevant details are typically overlooked or misinterpreted. We aim to encourage curiosity, increase scientific perceptiveness, and develop healthy skepticism. The course includes lectures about science communication, the scientific method, logic and reasoning, heuristics and framing, knowledge synthesis, and common misunderstandings in statistical inference (including p-values and sample size).

The focus of the course is on the coverage of epidemiological and public health research, which includes genetic epidemiology, psychiatric epidemiology, nutrition research, behavioral research, and others. Examples will come from newspapers, magazines, gray literature (informally published written materials, such as reports from government agencies and working papers), health promotion campaigns and other sources. The course consists of lectures about key theoretical and methodological concepts, and discussions of timely examples from the news and other non-scientific sources.

The lectures cover the following topics:

  1. The science communication process: how does scientific research get in the news, who is involved and what are their interests and incentives?
  2. Scientific method and knowledge synthesis: process of scientific research; nature of evidence; hierarchy of evidence; building evidence; knowledge synthesis; nature of argument: premises and conclusions.
  3. Critical thinking and scientific skepticism: critical reflection versus criticism; related concepts; skepticism versus pseudo-skepticism; analyzing arguments: identifying (hidden) premises, conclusions and distracting irrelevancies
  4. The validity of scientific facts: interpretation of p-values; borderline significant p-values; statistical significance versus clinical relevance; deductive and inductive arguments; small sample size; multiple testing; subgroup analyses, small effect size.
  5. The strength of evidence and validity of scientific claims: critical reasoning; inductive strength; strong and weak claims.
  6. Heuristics and framing: how do people make judgments and decisions when they are not fully informed? What are common flaws (fallacies) in our judgments? And how do journalists use that to give importance to a news

We also teach this course for undergraduates at Emory College, as part of the course “Critiquing Health Evidence in the News” (Course director: Amanda Freeman). The course is also in a lecture format (see example here).