The Uncomfortable Truth: When Data Challenges Our Educational Beliefs

The Uncomfortable Truth: When Data Challenges Our Educational Beliefs - Learning-Focused

And What to Do About It

We live in an era that worships data. We track our steps, optimize our sleep, and quantify our productivity. Yet there is a particular kind of discomfort that arises when data challenges beliefs we have built systems around.

In education, beliefs rarely stay abstract. Over time they shape policies, expectations, and classroom behaviors that influence student learning.

In education and professional development, we have embraced a set of widely shared beliefs: that critical thinking is more important than memorization, that knowing how to learn matters more than knowing facts, and that technology frees learners from the burden of retention. These beliefs were not careless or misguided. They emerged in response to real needs and real limitations.

Over time, however, beliefs do not remain abstract. They become schedules, curriculum priorities, instructional expectations, assessment practices, and technology use norms. In other words, beliefs turn into systems.

A systems mindset does not ask whether a belief is right or wrong. It asks a more practical question: What patterns of behavior and learning does the system consistently produce?

What if the data suggests we have overcorrected?

A growing body of neuroscience and cognitive research, including Jason Cooney Horvath’s The Digital Delusion (2019) and the paper The Memory Paradox (Oakley et al., 2025), suggests that our heavy reliance on digital offloading is not just changing how we learn. It may be actively eroding our capacity to think. The data points toward an uncomfortable conclusion: to build a truly digital-ready mind, we must start with a strictly analog foundation.

For school and district leaders, this is not an abstract debate. Beliefs about memorization, knowledge, and technology quietly shape schedules, device policies, curriculum pacing, instructional expectations, and walkthrough feedback. Over time, those beliefs harden into systems. When data challenges them, the discomfort is not personal. It is structural.

A systems mindset asks a different question: If the data is right, what in our system might be unintentionally working against learning?

The Data We Don’t Want to See

For much of the twentieth century, humanity experienced the Flynn Effect, a steady, global rise in IQ scores. Educators and policymakers attributed this to better nutrition, richer environments, and more advanced schooling.

Recent data, however, reveals a troubling shift. In several high-income countries, particularly in Scandinavia, where longitudinal data is strongest, IQ scores have leveled off and, in some cases, declined (Bratsberg & Rogeberg, 2018; Dworak et al., 2023).

In addition, in this video, Dr. Horvath explains that Generation Z is the first generation to perform at a lower rate on all cognitive tests than their parents.

From a leadership perspective, this is a moment for caution and curiosity. Data does not accuse teachers, students, or tools. It reveals patterns created by systems. The decline does not point to a single cause, but it does demand examination of how instructional time, cognitive effort, and tool use are sequenced across Tier 1.

The risk is not technology itself. The risk is systematic over-reliance before cognitive foundations are built.


The 1:1 Technology Correlation

The decline in cognitive performance coincides with several cultural and instructional shifts, including widespread adoption of 1:1 student devices, the rise of “look it up” culture, and changes in how schools allocate instructional time. Correlation is not causation, but patterns this consistent warrant scrutiny.

  • The NAEP Warning: Since the widespread adoption of 1:1 technology (accelerating post-2012), national benchmarks like the NAEP (National Assessment of Educational Progress) have shown historic drops in math and reading scores.

  • The Mode Effect: A meta-analysis by Delgado et al. (2018) found that students comprehend less when reading on screens compared to paper. The medium itself taxes cognitive load, draining the mental energy needed for deep analysis.

  • The Paradox: We believed that offloading memory to Google and calculators would free up "processing power" for higher-order thinking. The data suggest the opposite: without the "lower-order" facts stored in our biological memory, the "higher-order" processing stalls (Risko & Gilbert, 2016; Barr et al., 2015).

A systems-minded leader treats this moment not as a verdict, but as an invitation to disciplined inquiry.

Instead of asking, Should we have 1:1 devices?

A better question is: When, for what purpose, and after what cognitive work should devices be used?

That shift moves leadership away from policy debates and toward instructional design.

Why “Looking It Up” Short-Circuits the Brain’s Learning Cycle

Resistance to memorization is often framed as a rejection of “drill and kill” in favor of creativity. But research on how the brain learns, specifically through prediction error, which signals when expectations are wrong and prompts the brain to update its understanding, challenges this narrative.

The brain learns best when it is surprised. When an expectation is violated, prediction error triggers dopamine release, prompting the brain to update its internal models (Becker & Cabeza, 2024).

The catch is simple: to have a prediction, you must have internal knowledge.


If a student calculates 3 × 3 and gets 12, the error will be registered only if the student has internalized enough number sense to expect 9. Without that internal benchmark, the mistake passes unnoticed and unlearned.

If a student relies entirely on a calculator or AI, they have no internal expectation. They accept the output without surprise. Without internal knowledge, the brain’s primary learning mechanism remains dormant.

This is where leadership decisions matter. When systems normalize immediate access to answers, they unintentionally eliminate prediction, error recognition, and correction. The leadership response is not to ban tools, but to protect moments of unaided thinking inside Tier 1. One clear example of this principle in action can be seen in retrieval-based approaches to building number sense, such as those described in Mastering Multiplication: The Science and Practice of Retrieval-Based Learning

That protection can start small: mental estimation before calculators, draft thinking before search, explanation before AI assistance. These are not restrictions. They are intentional pauses designed to activate learning.

The Argument for “Analog First”

The most important insight from the data is not that technology is harmful. It is that our timing is wrong.

Learning depends on the interaction between two memory systems.

  1. Declarative memory involves conscious recall of facts. It is flexible but slow.

  2. Procedural memory governs automatic, intuitive skills. It is fast and effortless.

Expertise develops when knowledge migrates from declarative to procedural memory. This transfer requires repetition, effort, and struggle, processes often bypassed by educational technology.

When calculators or AI are introduced too early, they short-circuit this transfer. Students may produce correct answers, but they never develop the neural patterns that support intuition and transfer.

Analog first” is not nostalgia. It is a sequencing principle. Leaders who understand this treat technology as a finishing tool rather than a foundation.

The Transfer Gap: Why “Google It” Can Short-Circuit Learning

True expertise emerges when information moves from conscious recall to automatic intuition. Your brain has two distinct memory systems. True expertise only arises when you successfully transfer information from System A to System B.

The 1:1 Tech Trap:

To move knowledge from A to B, you need repetition and struggle.

  • The Analog Path: You practice multiplication tables until they become automatic. The struggle moves the data to the Basal Ganglia. You now have "Number Sense."

  • The Digital Path: You use a calculator or AI. You get the right answer immediately. The struggle is removed. The data never leaves the Declarative system.

  • The Result: You have the answer, but you never develop the intuition. You are "expert dependent"—helpless without the tool.

The Illusion of Competence

Perhaps the most dangerous effect of digital tools is the illusion of competence. Research on metacognitive laziness shows that students using AI often produce higher-quality outputs while retaining less knowledge (Fan et al., 2024).

A 2025 MIT Media Lab study (Kosmyna et al.) tracked students writing essays using ChatGPT, search engines, or no tools. Eighty-three percent of ChatGPT users could not recall key points from their own writing minutes later. EEG (electroencephalogram) data, which consists of recordings of the brain's spontaneous electrical activity, showed significantly lower cognitive engagement among AI users.

For leaders, this finding is especially important. Systems can mistake polished output for learning. But systems-minded leadership asks a harder question: What remains when support is removed?

The study showed that students who began without tools and later used AI demonstrated stronger recall and broader brain activation. Sequence, not access, made the difference.

This is a Tier 1 issue, not an intervention issue.

The “Analog First” Protocol: Rebuilding Cognitive Capacity

If the goal is deep thinking, the data suggests we must treat technology as a finishing tool, not a foundation. Here is a protocol for reversing the slide:

  1. The "Pen-to-Paper" Rule: First drafts, brainstorming, and initial problem-solving should be done by hand. Research by Mueller and Oppenheimer (2014) found that handwriting engages different neural circuits than typing and promotes "desirable difficulty," aiding encoding.

  2. Physical Texts for Deep Reading: Digital scrolling disrupts the creation of "spatial schemata." Physical navigation (knowing a fact was "on the bottom left of the page") helps the brain create a mental map of the information (Delgado et al., 2018).

  3. Ban the "Just in Time" Search: During a learning session, ban external lookups. If a student doesn't know a fact, they must mark it and keep going, or reason it out. Instant lookup kills the "Prediction Error" cycle. By forcing the brain to guess (even wrongly), you prime it to learn the correct answer later.

Leader Moves: Starting Small Without Mandates

Data this complex does not demand sweeping reform at the school level. It demands disciplined experimentation.

Systems-minded leaders start small.

Examples include:

  • Piloting No-Tech Tuesday or Thursday in a single classroom
  • Requiring handwritten drafts before digital revision
  • Delaying calculators or AI until students can explain expected outcomes

The leadership role is not enforcement. It is an observation. Leaders watch for changes in engagement, struggle, discussion, and transfer. Small pilots create local evidence and reduce ideological resistance.

Some institutions have already begun responding this way. One contemporary example is Sweden. After years of embracing 1:1 technology and screen-centric instruction, the Swedish government has reintroduced printed books, handwriting practice, and limits on digital use in early grades. The shift was not framed as a rejection of technology, but as a recalibration of learning conditions in response to outcomes that suggested foundational skills were being undermined.

This kind of adjustment reflects a broader design principle leaders would do well to revisit.

Designing for Learning, Not Smoothness

Many modern systems are designed to remove friction. That makes sense in manufacturing, logistics, and customer service. It does not translate cleanly to learning.

Learning requires effort, uncertainty, and occasional error. When systems are optimized to feel smooth, efficient, or immediately successful, they often hide stalled learning beneath polished performance.

This is the risk of over-scaffolded instruction and premature tool use. When answers arrive too quickly, students lose the opportunity to predict, struggle, and revise. The system appears effective, but the cognitive work has been outsourced.

For leaders, this distinction matters. Smooth classrooms are not necessarily learning classrooms. True learning systems preserve moments where thinking is visible, effort is required, and mistakes are treated as information rather than failure.

Conclusion

It is uncomfortable to tell students or teachers that memorization matters in an age of infinite storage. It is unpopular to suggest that knowing is cognitively superior to looking something up. But leadership is not about protecting comfort. It is about designing systems that work.

The data does not argue for rejecting technology. It argues for restoring sequence. Analog first. Digital to extend.

For systems-minded leaders, the task is not to remove tools, but to rebuild the conditions under which tools strengthen thinking. That work does not begin with mandates or programs, but with small, intentional design choices that preserve cognitive effort where it matters most.


If schools are going to improve, the system has to improve first.

The ideas in this article point to a larger leadership challenge: meaningful change does not happen through isolated strategies, new programs, or good intentions alone.

Making Progress helps school and district leaders build coherent systems that translate beliefs into daily practice, strengthen instruction, and support long-term improvement.

  • Move from disconnected initiatives to aligned action
  • Strengthen instructional systems across classrooms
  • Lead improvement in ways that are practical, visible, and sustainable

If you are ready to build progress through design rather than hoping improvement happens on its own, this book is a strong place to start.

Learn More About Making Progress


References

  • Cooney Horvath, J. (2019). The Digital Delusion: How Your Brain Really Learns and Why Education Tech Fails. Corwin.

  • Barr, N., Pennycook, G., Stolz, J. A., & Fugelsang, J. A. (2015). The brain in your pocket: Evidence that smartphones are used to supplant thinking. Computers in Human Behavior, 48, 473–480.

  • Becker, M. & Cabeza, R. (2024). Prediction error minimization as a common computational principle for curiosity and creativity. Behavioral and Brain Sciences, 47, e93.

  • Bratsberg, B., & Rogeberg, O. (2018). Flynn effect and its reversal are both environmentally caused. Proceedings of the National Academy of Sciences, 115(26), 6674.

  • Delgado, P., Vargas, C., Ackerman, R., & Salmerón, L. (2018). Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review, 25, 23–38.

  • Dworak, E. M., Revelle, W., & Condon, D. M. (2023). Looking for Flynn effects in a recent online U.S. adult sample: Examining shifts within the SAPA Project. Intelligence, 98.

  • Fan, Y., et al. (2024). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology.

  • Hirsch, E. D. (2000). "You Can Always Look It Up"... Or Can You? American Educator, 24(1), 4–9.

  • Kosmyna, N., et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. arXiv preprint.

  • Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168.

  • Oakley, B., Johnston, M., Chen, K.-Z., Jung, E., & Sejnowski, T. (2025). "The Memory Paradox: Why Our Brains Need Knowledge in an Age of AI." The Artificial Intelligence Revolution: Challenges and Opportunities. Springer Nature.

  • Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading. Trends in Cognitive Sciences, 20(9), 676–688.

  • Tyner, A., & Kabourek, S. (2020). Social Studies Instruction and Reading Comprehension: Evidence from the Early Childhood Longitudinal Study. Thomas B. Fordham Institute.

  • Wexler, N. (2019). The Knowledge Gap: The Hidden Cause of America's Broken Education System—and How to Fix It. Avery.