Continuing the Cognitive Science sub-series with the paper that made “information” a measurable quantity—and gave the cognitive science its central metaphor.
A Simple Introduction
In 1948, Claude Shannon, a mathematician at Bell Telephone Laboratories, published a paper in the Bell System Technical Journal that was not about the mind. It was about telephone wires. Bell Labs needed to know how much information a wire could carry, how to encode messages efficiently, and how to protect against noise. Shannon’s paper answered those questions with a theory that remains one of the most influential in the history of science.
The paper does not mention psychology, learning, or cognition. It treats information as a property of a message source—a measure of how much choice is involved in selecting a message from a set of possible messages. The meaning of the message is irrelevant to the engineering problem. What matters is the statistical structure of the message and the capacity of the channel.
But this austere, engineering-focused theory had a profound effect on how psychologists came to think about the mind. If information could be measured, encoded, transmitted, and decoded by a machine, perhaps the mind could be studied as an information-processing system. The vocabulary of Shannon’s theory—channel capacity, bits, redundancy, noise, coding—became the vocabulary of cognitive psychology. The paper is not cognitive science, but it made cognitive science possible.
10 Key Takeaways for Cognitive Science and Learning
- Information can be measured. Shannon showed that the amount of information produced by a source can be quantified. This made it possible to treat thinking and learning as processes that handle measurable quantities of information.
- The mind has limited channel capacity. The concept of channel capacity—the maximum rate at which information can be transmitted—was applied by George Miller and others to human attention and memory, launching the study of cognitive bottlenecks.
- The computer analogy became plausible. If a machine can process information, encode it, and decode it, then the mind can be modeled as an information-processing system. This is the foundational premise of cognitive psychology.
- Noise and redundancy matter. Shannon showed that redundancy in a message helps combat noise. This insight applies to learning: well-structured material with redundancy is easier to remember and more robust to interference.
- Uncertainty is a measurable quantity. Shannon’s concept of entropy quantifies how uncertain we are about an outcome. Learning reduces uncertainty; it is the process of reducing uncertainty about the world.
- Coding is central. Shannon showed that the way information is encoded determines how efficiently it can be transmitted. This maps directly onto learning: how material is represented and organized determines how well it is learned and remembered.
- The source matters as much as the channel. Shannon’s theory treats the statistical structure of the message source as crucial. In learning, the structure of the material—its organization, its predictability—determines how easily it is acquired.
- Prediction is possible. Shannon’s theory allows prediction of the rate at which information can be transmitted. Cognitive science adopted this predictive ambition: to predict learning outcomes from the structure of the material and the capacity of the learner.
- Information is relational. Information is not an intrinsic property of a message; it depends on the set of possible messages and their probabilities. Similarly, what is learned depends on the learner’s existing knowledge and expectations.
- The theory is a framework, not a psychology. Shannon explicitly excluded meaning from his theory. Cognitive science had to add meaning back—but it did so using Shannon’s framework. The tension between information and meaning remains a central issue in learning science.To him, “Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.“
- The Engineering Problem: Shannon’s goal was to measure how efficiently and accurately a message could be sent from point A to point B through a noisy channel. To him, a string of gibberish required the exact same engineering resources to transmit as a line of profound poetry.
- Information as “Surprise”: In Shannon’s theory, “information” does not mean “knowledge” or “truth”. Instead, it is a purely statistical measure of the reduction of uncertainty. The more unpredictable a message is, the more data (bits) it contains.
- Weaver’s Addition: When Warren Weaver co-authored the book layout of the theory in 1949, he acknowledged Shannon’s focus on the technical level (Level A), while identifying that the semantic level (Level B, dealing with meaning) and the effectiveness level (Level C) were separate problems to be tackled later
The Paper’s Core Ideas
Shannon’s paper is long and technical, but its central ideas can be stated simply.
The communication system. Shannon models communication as a system with five parts: an information source, a transmitter, a channel, a receiver, and a destination. The source produces a message; the transmitter encodes it into a signal; the channel carries the signal (and may add noise); the receiver decodes it; the destination receives the message. This schematic diagram became the canonical model of communication—and, by extension, of information processing in the mind.
Information and uncertainty. The amount of information produced by a source is measured by how uncertain we are about which message will be selected. If one message is certain, there is no information; if all messages are equally likely, there is maximum uncertainty, and therefore maximum information when one is chosen. Information, in Shannon’s sense, is the reduction of uncertainty. He measured it in “bits,” a unit that became the standard currency of the information age.
Channel capacity. Every channel has a maximum rate at which it can carry information—its capacity. Shannon’s fundamental result is that if the source produces information at a rate below the channel’s capacity, there exists a way of encoding that carries the information with arbitrarily small error. If the rate exceeds the capacity, errors are inevitable. This is a profound result: it says that reliable communication is possible up to a limit, and impossible beyond it.
Coding and redundancy. Shannon showed that efficient coding—matching the statistical structure of the source to the channel—can achieve the channel capacity. He also showed that redundancy (the difference between the actual information rate and the maximum possible) can be used to combat noise. English is about half redundant, which is why we can often understand a sentence even when letters are missing. Redundancy is a form of protection against error.
The noisy channel. When noise is present, the receiver cannot be certain what was sent. Shannon showed that reliable communication is still possible if the rate is below the capacity, with the noise corrected by proper coding. This result is one of the most important in the paper.
From Engineering to Mind: How Shannon Shaped Cognitive Science
Shannon’s paper was not about psychology, but it became a cornerstone of cognitive science for several reasons.
The computer analogy. The paper provided the foundation for digital computers. Once computers existed, cognitive scientists could ask: if a machine can process information, store it, and retrieve it, can the mind be understood in the same way? The answer, from the 1950s onward, was yes. The mind became software; the brain became hardware.
Channel capacity and human limits. In 1956, George Miller attended the Symposium on Information Theory at MIT—a conference born of Shannon’s work—and applied the concept of channel capacity to human short-term memory. His paper, “The Magical Number Seven, Plus or Minus Two,” argued that human short-term memory has a strict, measurable capacity of about seven items. This was the first demonstration that Shannon’s concepts could be applied directly to the mind.
Linguistics and generative grammar. At the same symposium, Noam Chomsky responded to Shannon’s use of statistical probabilities to model language. Chomsky argued that human language is too complex to be captured by simple statistics. But he used Shannon’s formal, algorithmic approach to develop generative grammar—a system of rules that generates all and only the grammatical sentences of a language. This shifted linguistics from cataloguing speech to studying the internal structures that generate it.
Connectionism and modern AI. Shannon’s paper laid the groundwork for understanding how networks process signals. In the late twentieth century, this evolved into connectionism—the modeling of the mind as a network of interconnected artificial neurons. Modern deep learning networks are, in a direct sense, descendants of Shannon’s information theory and the neural network models that followed.
Predictive processing. Contemporary cognitive science uses the idea of reducing uncertainty to explain how the brain predicts and interprets sensory input. The brain, on this view, is a prediction machine that minimizes surprise. Learning is the process of updating predictions to reduce uncertainty. This is a direct application of Shannon’s framework to the mind.
Information Theory and Learning Science
Shannon’s theory has several implications for the science of learning, some direct and some indirect.
Learning as uncertainty reduction. If information is the reduction of uncertainty, then learning is the process by which uncertainty about the world is reduced. A learner who knows nothing about a topic has high uncertainty; a learner who has mastered it has low uncertainty. Instruction, on this view, is the provision of information that reduces the learner’s uncertainty.
The structure of the material matters. Shannon’s theory shows that the statistical structure of the source determines how efficiently it can be transmitted. In learning, the structure of the material—its organization, its predictability, its redundancy—determines how easily it can be acquired. Well-organized material is easier to learn because it requires less effort to specify.
Channel capacity and cognitive load. The concept of channel capacity suggests that learners have limited capacity for processing information. Cognitive load theory, developed by John Sweller in the 1980s, is built on this idea: instruction should be designed to minimize extraneous load and maximize the learner’s capacity for meaningful processing. This is a direct application of Shannon’s concept to education.
Redundancy and robustness. Shannon showed that redundancy protects against noise. In learning, redundancy—repetition, elaboration, multiple representations—can protect against forgetting and interference. But too much redundancy is inefficient. The optimal level of redundancy is a design problem for instruction.
Coding and representation. Shannon’s theory shows that the way information is encoded determines how efficiently it can be transmitted. In learning, the way material is represented—verbally, visually, schematically—determines how well it is learned and remembered. The encoding specificity principle in memory research is a direct descendant of this insight.
The limits of the analogy. Shannon’s theory is about the transmission of information, not about its meaning. Learning is not just the transmission of information; it is the construction of meaning. Cognitive science had to add meaning back to the information-theoretic framework. The tension between information and meaning remains one of the central problems in learning science.
Conclusion
Shannon’s 1948 paper is not a cognitive science paper. It is an engineering paper about telephone wires. But it is one of the founding documents of cognitive science because it made information a measurable, trackable, modelable quantity—and in doing so, it made it possible to study the mind as an information-processing system.
The paper’s legacy is everywhere in modern learning science. The concept of channel capacity underlies cognitive load theory. The idea of reducing uncertainty underlies predictive processing and the Bayesian brain. The concept of coding underlies research on representation and memory. The computer analogy, born of Shannon’s ideas, is the foundational metaphor of cognitive psychology.
Shannon himself was not interested in the mind. He was interested in communication. But his theory of communication became a theory of mind—or at least, the framework within which a theory of mind became possible. The paper is a reminder that the most important advances in one field often come from another. And it is a reminder that information, once made measurable, becomes a tool for understanding everything—including learning.
About the Author
