Understanding Criticisms of John Hattie’s Visible Learning

If you’ve ever been curious about what makes a good education for your child, you might have come across John Hattie’s book Visible Learning. It’s widely known for claiming to rank what works best in education using a massive amount of research data. But what if we told you that some experts argue Hattie’s conclusions are misleading? Let’s break down the criticism in simple terms so parents can better understand what’s at stake.

What Is Visible Learning?

John Hattie’s Visible Learning presents a list of factors that supposedly influence student achievement, from class size to teaching methods. Hattie claims to summarize over 800 meta-analyses (studies that combine results from multiple research projects) into a single guide for improving education. The book’s big appeal lies in its simplicity—giving clear rankings of what “works” based on effect sizes, a statistical measure of how much something changes outcomes.

But statistics are complicated, and some experts believe Hattie’s approach to interpreting the data is flawed.

What’s the Problem?

A recent article by Pierre-Jérôme Bergeron, a statistician, highlights major issues with Hattie’s methods (McGill Journal of Education, Bergeron, 2021). Here are the key concerns:

1. Effect Sizes Aren’t Everything

Hattie uses a cut-off point to decide what’s “effective” in education. For example, he suggests that any factor with an effect size above 0.4 is worth paying attention to. But experts argue that this threshold is arbitrary and doesn’t account for differences in how studies are conducted. Comparing a small, focused classroom experiment to a nationwide study isn’t fair—they measure different things.

2. Mixing Apples and Oranges

Meta-analyses pull data from various studies, but not all studies are equal. For instance, studies about preschoolers learning to read shouldn’t be lumped together with research on high school math. Bergeron points out that Hattie’s approach often combines unrelated findings, leading to conclusions that oversimplify complex issues.

3. Correlation vs. Causation

Just because two things are related doesn’t mean one causes the other. For example, Hattie’s rankings might show a strong link between student self-esteem and academic performance. But does improving self-esteem cause better grades, or do good grades make students feel better about themselves? Hattie’s analysis doesn’t always clarify these questions.

4. Ignoring Context

Education doesn’t happen in a vacuum. What works in a wealthy, well-funded school might not work in an under-resourced one. Bergeron criticizes Hattie for ignoring these nuances and presenting a one-size-fits-all solution.

Why Should Parents Care?

When educational policies and practices are based on flawed interpretations of data, it can lead to ineffective or even harmful decisions. For example, schools might focus on trendy but unproven strategies instead of addressing the real needs of students and teachers. Parents who rely on Visible Learning as a guide might unknowingly push for changes that don’t actually benefit their children.

What Can You Do?

Here are some steps parents can take:

1. Ask Questions: If your child’s school adopts a new policy or method based on Hattie’s rankings, don’t hesitate to ask how it aligns with the school’s specific needs and context.
2. Look Beyond Rankings: Remember that education is complex, and what works best often depends on individual circumstances, not a universal checklist.
3. Stay Informed: Read critiques like Bergeron’s to understand the limitations of popular educational theories. Being an informed parent can help you advocate for better decisions in your child’s education.

Final Thoughts

Data is powerful, but it’s only as good as the way it’s used. While Visible Learning has brought attention to evidence-based education, it’s important to approach its conclusions with a critical eye. By understanding the criticisms, parents can play an active role in shaping a more thoughtful and effective educational experience for their children.

For a deeper dive into this topic, see Pierre-Jérôme Bergeron’s article: “How to Engage in Pseudoscience With Real Data: A Criticism of John Hattie’s Arguments in Visible Learning From the Perspective of a Statistician,” McGill Journal of Education 56, no. 1 (2021): 151-169, https://mje.mcgill.ca/article/view/9475/7229.