For centuries, humanity has dreamed of creating machines that can think. In the last few years, that dream has seemingly inched closer to reality with the rise of Large Language Models (LLMs) like ChatGPT, Gemini, and Claude. These sophisticated programs can write essays, compose poetry, summarize complex documents, and hold conversations that feel almost human. It is easy to look at these achievements and conclude that we are on the verge of creating true artificial intelligence.
However, a growing chorus of top researchers and philosophers argues that we are mistaken. They claim that while LLMs are incredibly knowledgeable—able to recall and recombine vast swathes of human information—they are not intelligent in any meaningful sense of the word. They are, to put it simply, vast repositories of knowledge without a shred of understanding .
This debate goes to the very heart of what we mean by "intelligence." To understand it, we must first define our terms, then examine how today's most advanced AI models work and where they fall short. Finally, we need to return to the basics of computer science and neuroscience to understand the original vision of neural networks—systems that were not just knowledgeable, but truly capable of learning.
Part 1: Defining the Terms - What Do We Mean by Knowledge and Intelligence?
Before we can argue whether a machine is intelligent, we need to agree on what intelligence is. This is a surprisingly slippery concept, one that philosophers and psychologists have debated for millennia .
The Nature of Knowledge
For our purposes, knowledge can be thought of as the information a system possesses. It is the sum total of facts, data, and representations of the world that have been stored. In a human, this is our memory: knowing that the capital of France is Paris, that water freezes at 0 degrees Celsius, or how a typical conversation about the weather goes. In a computer, this is its database, its stored files, and, crucially for AI, the patterns it has learned from its training data. Knowledge is essentially a static snapshot of information at a given point in time .
The Elusive Nature of Intelligence
Intelligence, on the other hand, is a much more dynamic and complex concept. It is not just about having information, but about what you can do with it. Drawing from the classic 1956 Dartmouth Conference, which birthed the field of AI, true intelligence was defined by abilities like:
- Abstract Thinking: The ability to work with symbols and concepts that aren't tied to concrete physical objects.
- Logical Reasoning: Using rules to draw conclusions from premises.
- Causal Analysis: Understanding not just that two things happen together, but that one causes the other.
- Self-Correction: The ability to recognize a mistake and adjust future behavior accordingly .
A common sense perspective adds further layers. Intelligence involves the ability to learn and remember, but not just through rote repetition. It means adapting to novel situations, solving problems you've never seen before, and generating creative new solutions. As one definition puts it, a hallmark of intelligence is "coming up with something new" to cope with environmental changes .
This leads to the crucial distinction: A system can be incredibly knowledgeable without being intelligent. Imagine a library. A library contains more knowledge than any single person could ever hold in their mind. You can find books on physics, philosophy, and poetry. But the library itself cannot use that knowledge to solve a problem, write a new poem, or reason about its own existence. It is a storehouse, not a thinker. The central claim of many critics today is that LLMs are like digital libraries—incredibly advanced ones—but libraries nonetheless.
Part 2: The Architecture of a Know-It-All - How LLMs Acquire Their Knowledge
To understand why critics say LLMs are all knowledge and no intelligence, we need to peek under the hood and see how they are built. At their core, LLMs are a type of neural network, but scaled to an almost unimaginable size.
The Foundation: Learning from Text
An LLM is "trained" on a colossal amount of text data—a significant portion of the entire public internet, including books, articles, Wikipedia, and Reddit threads. During this training process, the model is given a task that seems simple: predict the next word in a sentence. It is shown a phrase like "The cat sat on the..." and it must guess what word comes next.
It will guess wrong, millions of times. But each time it guesses, it subtly adjusts its internal settings (its millions or billions of parameters) to make its next guess slightly better. Over months of training and an astronomical number of calculations, the model becomes incredibly good at this game. It builds a complex, statistical map of human language—which words tend to follow which other words, in which contexts .
The Result: A Master of Style, Not Substance
The result is a machine that can generate fluent, grammatically correct, and stylistically appropriate text. It "knows" that after a sentence about the capital of France, the word "Paris" is statistically very likely. It has absorbed the knowledge contained in its training data and can retrieve and recombine it on command.
This is why an LLM can write a Shakespearean sonnet about cryptocurrency. It has learned the statistical patterns of a sonnet (rhyme scheme, iambic pentameter) from one part of its data and the vocabulary and concepts of cryptocurrency from another, and it can weave them together. It is a phenomenal feat of pattern matching and recombination.
The Static Brain: Frozen in Time
However, there is a critical catch. Once an LLM is trained and deployed, its "brain" goes static. The learning process stops. If you tell it a new fact today—for example, "My dog's name is Max"—it will remember that for the duration of your conversation (thanks to a short-term memory called the context window). But as soon as you start a new chat, that information is gone. The model's internal parameters, its weights, do not permanently update based on your input. To give it new permanent knowledge, you would have to retrain the entire massive model from scratch .
This is the first major strike against its intelligence. It is a system of pure knowledge acquisition during training, followed by a complete inability to learn from ongoing experience. It is a frozen snapshot of the internet from the point it was trained.
Part 3: The Case Against LLM Intelligence - Where They Fall Short
Armed with an understanding of how LLMs work, the arguments against their intelligence become clear. The critiques fall into several key areas, highlighting the fundamental differences between statistical pattern matching and genuine understanding.
Argument 1: No Understanding, Only Syntax
Philosopher Luciano Floridi famously argued that LLMs exhibit "agency without intelligence." They can act (by producing text) but do so without any understanding of the content. They manipulate the syntax of language (the structure and rules) without any access to the semantics (the meaning) .
A powerful illustration of this is philosopher John Searle's famous "Chinese Room Argument." Imagine a person who doesn't understand Chinese sitting in a room with a giant rulebook. People outside slide slips of paper with Chinese characters under the door. The person inside uses the rulebook to find the correct set of Chinese characters to slide back out. To the people outside, it looks like the room understands Chinese. But the person inside doesn't understand a word; they are just following rules. LLMs, critics argue, are the ultimate Chinese Room. They have a staggeringly complex rulebook (their parameters) for manipulating symbols (words), but they have no actual experience of the world to ground those symbols in meaning. As one expert put it, "LLMs are simply tools that emulate the communicative function of language" but lack the cognitive process of thinking .
Argument 2: The Embodiment Problem - No World, No Meaning
This lack of understanding leads to a deeper issue: the embodiment problem. Human intelligence is not a disembodied process happening in a void. It is shaped by, and grounded in, our physical existence in the world. We have bodies that interact with the environment. We experience "handiness"—the feel of a tool, the weight of a book, the coolness of water. We learn language in the context of these experiences. When a child learns the word "ball," they don't just see the word in a sentence; they see a round object, they touch it, they throw it, they watch it bounce.
LLMs have none of this. They are pure symbol processors, disconnected from any physical or social reality. As researcher Rasmus Gahrn-Andersen explains, they lack "being-in-the-world." This leads to profound failures when they are asked to generate not just text, but images or to reason about physical reality. For example, an AI image generator asked to create a picture of "engineers working on a runway, aided by drones" might place a construction crane right next to an active runway, a scenario that is both practically nonsensical and dangerously unsafe. It knows the words "engineer," "runway," and "crane," but it has no practical, embodied knowledge of how they actually relate to one another in the real world .
Argument 3: No Capacity for True Creativity
If intelligence involves "coming up with something new," how do LLMs fare? A recent analysis published in the Journal of Creative Behavior suggests the ceiling is surprisingly low. Because LLMs are fundamentally probabilistic systems, they work by remixing and recombining existing data. They are designed to find the most statistically likely next word, which inherently pushes them toward the average, the expected, and the formulaic.
While they can create convincing pastiches, their ability to generate truly original and effective output is capped. The study concluded that an LLM will always produce something average. "A skilled writer, artist or designer can occasionally produce something truly original and effective. An LLM never will." . They are trapped in the vocabulary and patterns of their training data, unable to make the kind of intuitive leap that defines human creativity and innovation.
Argument 4: A Frozen Point of View
Finally, the static nature of an LLM's knowledge is fundamentally unintelligent. Real intelligence requires the ability to constantly update one's worldview based on new information. A human child doesn't need to be completely "retrained" from birth to learn that a dog is called a "dog" and then later update that knowledge to distinguish between a "dog" and a "corgi." They continuously adapt.
LLMs cannot do this. Their knowledge is a snapshot of the past. They are like a brilliant scholar who died in 2021 and can never learn anything new about the world after that date. This is why they can be "tricked" by new information and why they suffer from catastrophic forgetting if you try to update them—they may learn a new fact but forget how to write a poem. As Professor Ma Yi of the University of Hong Kong notes, LLMs lack even the concept of a natural number, let alone the ability to autonomously acquire new knowledge and correct themselves, which was the original goal of AI set out in 1956 .
Part 4: Going Back to Basics - The Original Promise of Neural Networks
If LLMs, for all their power, represent a dead end on the path to true intelligence, where should we look instead? The answer may lie in revisiting the foundational principles of neural networks, which were originally conceived not just as pattern matchers, but as dynamic systems capable of learning in a much more fundamental way.
The Biological Inspiration
The very name "neural network" is a nod to its inspiration: the human brain. The brain is not a static database. It is a living, constantly changing organ. Its basic units, neurons, are connected in a vast, complex web. The key to learning and memory in the brain is plasticity—the ability of those connections to strengthen or weaken over time based on activity. The adage in neuroscience is, "Cells that fire together, wire together." . When you have a new experience, the connections between the neurons involved in that experience are physically altered. The brain literally rewires itself to encode new knowledge.
The Basic Computational Unit
Early AI pioneers like McCulloch and Pitts (1943) and Frank Rosenblatt (1958) tried to capture this essence in a simple mathematical model. They created the perceptron, the ancestor of all modern neural networks . The idea is beautifully simple:
- A neuron receives multiple input signals.
- Each input is multiplied by a weight, which represents the strength of that connection. A high weight means that input is very important.
- The neuron sums up all these weighted inputs.
- If that sum is above a certain threshold, the neuron "fires" and sends a signal on to the next layer of neurons .
Learning, in this model, is the process of adjusting those weights. In the beginning, the weights are random, so the network's output is garbage. But through training, it gradually finds the right set of weights that allows it to correctly transform an input (like a picture of a cat) into the correct output (the word "cat").
How Networks Learn: The Training Loop
This process of adjusting weights is governed by a beautiful feedback loop that is the core of deep learning:
- The Forward Pass: You feed the network an input, and it processes it through its layers to produce an output.
- The Loss Function: You compare that output to the correct answer. How wrong was it? The "loss" is a single number representing the magnitude of the error.
- Backpropagation: This is the magic trick. Because the network is built of differentiable math functions, you can calculate how much each individual weight contributed to the overall error. You can then work backward through the network, assigning blame (or credit) to each connection.
- The Optimizer: The optimizer takes from backpropagation and uses it to tweak all the weights by a tiny amount, in the direction that would reduce the error. Then, you repeat the entire process billions of times with new data .
This loop—predict, compare, adjust—is a genuine form of learning. It's not just storing information; it's fundamentally altering the system's own circuitry to improve its performance.
Part 5: Alternative Visions - Neural Networks That Truly Learn
The brilliance of the basic neural network model is that it is a blueprint for learning, not just knowledge storage. By returning to these principles and asking how they can be embodied in new ways, researchers are building systems that are far more dynamic and intelligent than today's monolithic LLMs. These examples show that the path to intelligence lies in systems that can adapt, evolve, and ground their knowledge in interaction.
Example 1: The Self-Learning Superconducting Network
At the National Institute of Standards and Technology (NIST), researchers have built a simulation of a neural network with a remarkable property: it can learn on its own, continuously. This network is built from superconducting components, which allow electricity to flow without resistance and operate at incredibly high speeds with very low energy consumption.
The key innovation is in the hardware itself. In a standard neural network, the "learning" (adjusting the weights) is done by a separate, external computer running a software algorithm. In the NIST design, the hardware that makes up the circuitry is capable of determining how to change its own weights. It performs a type of learning called reinforcement learning (learning by trial and error, like using a carrot and stick) without needing any external control.
This offers two massive advantages over standard LLMs. First, it can learn continually. As new data comes in, the circuit adapts itself. It doesn't need to be retrained from scratch. Second, it can automatically adjust to its own physical imperfections, making it far more robust. This is a true learning system, dynamically evolving in response to its environment .
Example 2: The DNA Neural Network That Develops Memories
At the California Institute of Technology (Caltech), scientists have taken the concept of neural networks and built one out of an entirely different medium: DNA. This "wet lab" neural network doesn't use silicon chips and digital signals. Instead, it uses strands of DNA and chemical reactions to perform computation.
In a groundbreaking 2025 study, researchers led by Lulu Qian demonstrated a DNA-based neural network that can learn. It does this by using "molecular wires" that can be chemically flipped on or off to store information. When the system is presented with a molecular example—in this case, a pattern representing a handwritten number—it "develops" its own memories. It physically builds up a record of what it has learned, encoded in the concentrations of specific DNA molecules.
The researchers describe this as the system building a physical record of its experiences, a process akin to the "cells that fire together, wire together" principle of the human brain. While still in its infancy, this research points toward a future where "smart" medicines could adapt to pathogens in real-time, or "smart" materials could learn from their environment and change their properties accordingly. This is not a static knowledge base; it is a chemical system whose very structure is its memory and learning process .
The Common Thread: Learning as Adaptation
What unites the NIST and Caltech research? Both move away from the idea of intelligence as a massive, static database of words. Instead, they embrace the original vision of neural networks: intelligence as a dynamic process of adaptation. Learning, in these systems, is not something that happens once during a training phase and then stops. It is an ongoing, fundamental property of the system's existence. They are designed to interact with their environment (whether electrical signals or chemical molecules) and change themselves in response.
Part 6: The Road Ahead - Towards a New Kind of AI
The debate between knowledge and intelligence is not just an academic exercise. It has profound implications for the future of technology and our understanding of ourselves.
The Future is Hybrid
It is unlikely that the future of AI will involve completely abandoning LLMs. They are too useful for what they are: powerful engines for knowledge retrieval, summarization, and content creation. The path forward is likely a hybrid one. We are already seeing the first steps of this in the development of LLM agents. These systems use an LLM as a core "reasoning engine," but they are also equipped with tools and memory that allow them to interact with the world. They can browse the web, run code, query databases, and remember past interactions. This moves the system from a passive knowledge repository to a more active, goal-driven entity .
Learning from Embodiment
The most profound advances, however, will likely come from taking the embodiment problem seriously. If intelligence requires a body and a world to interact with, then the most intelligent machines of the future may not be disembodied chatbots, but robots. A robot that has to navigate a messy kitchen, grasp a slippery fruit, and learn from its failures is engaged in a fundamentally different kind of intelligence than an LLM predicting the next word. It is grounding its learning in physical cause and effect.
Researchers like Yann LeCun, a Turing Award winner and former chief AI scientist at Meta, have long argued for pursuing "world models"—AI systems designed to understand the three-dimensional world by training on physical and sensory data, rather than just language. This approach recognizes that language is a symptom of intelligence, not its source .
A Call for Foundational Thinking
Professor Ma Yi's message to young scientists is a powerful one: don't just follow the crowd. The current frenzy around scaling up LLMs—throwing more data and more computing power at the problem—may be leading us down a blind alley. To achieve true intelligence, we need to go back to first principles. We need to ask the hard questions that the Dartmouth conference participants asked in 1956. What is intelligence? How does learning work? What is the role of the body and the world in shaping the mind?
The answers will not come from building larger statistical models of internet text. They will come from a deeper understanding of neural networks, not just as mathematical functions, but as a blueprint for building systems that can truly learn, adapt, and evolve—systems that don't just know, but understand. The journey back to the basics may be the only way forward to the intelligence we've always dreamed of creating.