The AI That Never Forgets? The Dendritron Transformer and the Search for Continuous Memory
A brain-inspired architecture proposes giving frozen language models a separate memory that can keep learning
Introduction
Large language models, or LLMs, often seem to remember. They can follow extended conversations, analyze documents, adjust their tone, and sometimes preserve user preferences across different sessions. Yet this apparent memory is not the same as permanent learning.
Most modern LLMs are built on Transformer architectures whose core parameters are trained in advance and then frozen. During a conversation, the model can use information placed inside its context window, but it does not normally transform each interaction into lasting internal knowledge. It may understand a correction today and repeat the same mistake tomorrow.
The Dendritron Transformer is an experimental proposal that attempts to change this. It combines a stable, frozen Transformer with a smaller trainable module inspired by the dendrites of biological neurons—the branching structures that perform local, nonlinear processing before signals are integrated by the cell.
The idea is simple but ambitious: preserve the broad knowledge of the original model while allowing a separate, specialized memory system to continue learning from new experiences.
In this architecture, the frozen Transformer provides stability, while the Dendritron module provides plasticity. The unresolved question is whether this balance can produce a genuinely adaptive AI without introducing new forms of forgetting, instability, manipulation, and loss of control.
The Illusion of Learning: Four Different Kinds of AI Memory
A Transformer-based large language model can appear to learn in several different ways. Yet these mechanisms are not equivalent. Some provide only temporary access to information, others store data outside the model, and only fine-tuning changes the neural network itself.
Context Windows: Temporary Working Memory
The context window contains the information currently available to the model. It includes the conversation, system instructions, retrieved documents, and other recent inputs.
For example, a user may say that they are vegetarian and planning a trip with a limited budget. Several messages later, the model can still use both constraints when recommending a restaurant. In a sufficiently long conversation, however, one of those details may be summarized, displaced, or no longer available.
The model appeared to remember, but the information was only temporarily present in its active context.
External Retrieval: Looking Information Up
Retrieval-augmented generation, or RAG, allows the system to search external files, databases, or websites before answering.
Suppose a user uploads a company handbook and asks how many remote-work days employees receive. The system locates the relevant section and uses it to answer the question.
This is useful because the source can be inspected, updated, or deleted. But the model has not necessarily incorporated the policy into its internal knowledge. Without the document, it may be unable to answer the same question later.
Retrieval is therefore closer to consulting a notebook than to remembering from experience.
Persistent Profiles: Memory Outside the Model
Some systems preserve user preferences across sessions. They may remember that a user prefers concise explanations, writes in British English, or has a dog named Buster.
These details are typically stored outside the model’s main neural network. When a new conversation begins, the system reintroduces the saved information as part of the prompt.
The result feels like personal memory, but the model itself has not been retrained. Its underlying parameters remain unchanged.
Fine-Tuning: Changing the Model Itself
Fine-tuning is different because it modifies the neural network’s internal weights.
A hospital, for example, might fine-tune a general-purpose model on radiology reports and medical terminology. After training, the model may understand specialized language without needing the same reference documents in every conversation.
The table below summarizes how these four mechanisms differ in where information is stored, how it is accessed, and whether the model itself actually learns.
| Mechanism | How it works | What it resembles | Main limitation |
|---|---|---|---|
| Context window | Keeps the current conversation and recent instructions available to the model | Short-term working memory | Earlier details may disappear when the conversation becomes too long or ends |
| External retrieval or RAG | Searches documents, databases, or websites and inserts relevant passages into the prompt | Consulting a searchable notebook | The model accesses the information but does not necessarily learn it |
| Persistent user profile | Stores selected preferences or personal details outside the neural network and reintroduces them in future prompts | External sticky notes | The model’s core parameters remain unchanged |
| Fine-tuning | Updates the model’s internal weights using new training data | Rewriting long-term knowledge | Expensive, difficult to reverse, and vulnerable to catastrophic forgetting |
Fine-tuning is the point at which apparent memory becomes genuine internal learning—but changing the model’s weights to learn something new can unintentionally erase abilities acquired earlier. This phenomenon is known as catastrophic forgetting.
What Catastrophic Forgetting Is — and Why It Happens
A neural network does not usually store each skill in a completely separate location. Different tasks often depend on overlapping groups of parameters.
Imagine that a model is first trained to recognize cats. During training, its weights are adjusted to detect features such as ears, fur, facial structure, and body shape.
The model is then trained exclusively on dogs. The new training updates many of the same weights because cat recognition and dog recognition depend on related visual features.
Those updates may improve dog recognition while partially destroying the internal representations previously used for cats.
The model does not simply add the new skill beside the old one. It may rewrite shared internal structures.
As a result, performance can follow this pattern:
| Training stage | Cat recognition | Dog recognition |
|---|---|---|
| After learning cats | High | Low |
| After learning dogs | Lower | High |
The model becomes better at the new task, but worse at the one it had already learned.
Catastrophic forgetting is related to overfitting, but the two are not the same. Overfitting occurs when a model adapts too closely to its current training data and fails to generalize to new examples. Catastrophic forgetting occurs when learning a new task damages knowledge acquired earlier.
The two problems can happen at the same time. Intensive training on a narrow set of dog images may cause the model to memorize peculiarities of that dataset while also overwriting representations previously used to recognize cats.
However, catastrophic forgetting does not require overfitting. A model may perform well on unseen dog images and still lose much of its earlier ability to recognize cats. Put simply, overfitting means learning the new task too narrowly; catastrophic forgetting means learning it at the expense of previous tasks.
The term catastrophic does not imply that every earlier ability disappears completely. It refers to a loss that can be sudden and disproportionately large compared with the amount of new learning involved.
This happens because optimization focuses on reducing error for the current task. Unless the system includes mechanisms to preserve earlier knowledge, it has no automatic reason to protect the internal representations that supported previous abilities. New learning can therefore improve present performance while quietly damaging the past.
A continually learning AI must therefore do two things at once:
- change enough to acquire new knowledge;
- preserve the internal structures required for earlier abilities.
That conflict is the foundation of the stability–plasticity dilemma and one of the central problems the Dendritron architecture attempts to address.
The Stability–Plasticity Dilemma
Continual learning depends on balancing two opposing requirements.
Plasticity is the ability to change. It allows a model to absorb new information, correct mistakes, and adapt to unfamiliar situations.
Stability is the ability to preserve useful knowledge. It prevents each new learning episode from rewriting everything that came before it.
Too much plasticity creates a system that learns rapidly but forgets constantly.
Too much stability creates a system that preserves old knowledge but becomes rigid, unable to revise mistakes or adapt to change.
Human cognition appears to maintain a dynamic balance between these two extremes. We acquire new skills without forgetting language, identity, or basic knowledge of the world. At the same time, our memories are not perfectly fixed: they are revised, weakened, reorganized, and sometimes deliberately forgotten.
Artificial intelligence has not yet reproduced this balance reliably.
The Dendritron approach attempts to separate the two functions. The large pretrained Transformer remains frozen and preserves its general capabilities. A smaller adaptive module handles new learning.
Instead of rewriting the entire system, the architecture modifies only its additional memory.
Why Dendrites Matter
The artificial neuron used in conventional neural networks is an extreme simplification of a biological neuron.
It receives several inputs, assigns a numerical weight to each one, combines them into a single value, and applies a nonlinear activation. Its power emerges when millions or billions of these simple units are connected across multiple layers.
This abstraction has been remarkably successful. It underlies most of all systems capable of recognizing images, generating language, predicting molecular structures, and controlling complex machines.
But biological neurons are far more sophisticated.
Dendrites are the branching structures through which neurons receive signals. They were once commonly treated as passive cables carrying information toward the cell body. Experimental neuroscience has shown that individual branches can perform local computation.
A dendritic branch can respond differently to different combinations of inputs. It may exhibit threshold effects, saturation, local electrical spikes, nonlinear summation, and branch-specific plasticity. Signals can therefore be transformed inside separate branches before they are combined by the neuron as a whole.
This matters mathematically.
A conventional single-layer artificial neuron cannot solve problems such as a XOR gate, in which the correct answer depends on a nonlinear relationship between inputs. Such problems normally require additional neurons or layers.
A neuron with several dendritic branches can apply separate nonlinear transformations to different groups of inputs before combining them. In principle, this allows one structured unit to represent interactions that a standard perceptron cannot express on its own.
The biological neuron may therefore behave less like a single artificial neuron and more like a small internal network.
An artificial dendritic architecture attempts to capture this principle without reproducing every detail of real cellular physiology. Different branches could specialize in different contexts, domains, users, or kinds of memory.
One branch might become responsive to medical information. Another might specialize in physics. Others could encode user preferences, recently acquired knowledge, or uncertain observations waiting for validation.
Because learning is localized, a new experience might modify only the relevant branches rather than the entire memory module. This could reduce interference with unrelated knowledge.
But biological inspiration is only a motivation. A model is not superior merely because it resembles the brain. Dendritic architectures must still outperform simpler alternatives under comparable conditions.
How the Dendritron Architecture Works
At a conceptual level, the architecture contains three elements.
1. The frozen Transformer core
The pretrained Transformer provides language competence, general knowledge, and broad reasoning abilities. Its central parameters remain unchanged during continual adaptation.
Freezing the core protects it from direct catastrophic forgetting.
2. The trainable Dendritron memory
The smaller adaptive module receives internal representations produced by the Transformer. Its dendritic branches learn corrections, associations, preferences, or specialized knowledge without rewriting the foundation model itself.
3. Selective activation
A routing mechanism may activate only the branches relevant to the current input. A medical question could engage one group of branches, while a programming task activates another.
This creates a form of conditional memory: not every branch participates in every computation.
The way this memory interacts with the Transformer can be organized into three broad stages.
Before attention: modifying internal representations
The Dendritron could adjust the hidden representations entering a Transformer layer. In this configuration, memory changes how the model represents the current information before attention is calculated.
This is similar to an adapter, but the additional computation is organized through dendritic branches rather than an ordinary small neural network.
Inside attention: influencing what the model focuses on
The memory module could modify the queries, keys, or values used by the attention mechanism.
This would allow past learning to influence which parts of the current context receive greater importance. The memory would not simply change the final answer; it would influence how the model interprets the input.
This approach is more powerful, but it also intervenes more deeply in the Transformer’s internal computation.
After attention: correcting the final output
The Dendritron could modify the final output scores before the model selects the next token.
This is the simplest form of integration. The original Transformer performs its normal computation, and the adaptive memory adds a final correction.
The advantage is efficiency and relative isolation. The limitation is that the memory can alter the answer but not deeply reshape the model’s internal reasoning.
A gating mechanism could also regulate the balance between the two systems. For familiar questions, the model might rely mainly on the frozen Transformer. For personalized or recently learned information, the adaptive memory could have greater influence.
How It Compares with Existing Adaptation Methods
The broader idea of freezing a large model and training only a smaller component is already well established. The Dendritron belongs to the wider field of parameter-efficient fine-tuning, or PEFT.
| Method | What changes | Main idea |
|---|---|---|
| Adapters | Small neural modules inserted into frozen layers | Add task-specific transformations without retraining the full model |
| LoRA (Low-Rank Adaptation) | Small low-rank matrices attached to existing weights | Approximate useful weight changes with far fewer trainable parameters |
| Prompt tuning | Trainable vectors added to the input | Guide model behavior without changing its original weights |
| Prefix tuning | Trainable representations added inside attention | Influence attention through learned prefixes |
| Mixture of experts | Different modules activated for different inputs | Route each task to specialized components |
| Dendritron | Branch-structured adaptive memory | Use local nonlinear computation and selective plasticity |
The Dendritron is therefore not novel simply because it attaches a trainable component to a frozen Transformer.
Its scientific value must come from the specific properties of dendritic organization.
Does branch-based computation learn with fewer parameters? Does it reduce interference between tasks? Can it localize memories more effectively? Does selective activation improve scalability? Can individual branches be audited, removed, or reset?
Those are the claims that require evidence.
What Would Count as Proof?
A system that learns one fact in a demonstration has not solved continual learning.
A credible evaluation would require a long sequence of tasks. After each new task, the model would need to be tested on everything it had learned previously.
Researchers would need to determine whether the architecture can:
- acquire new knowledge without extensive retraining;
- preserve performance on earlier tasks;
- transfer useful knowledge between related domains;
- avoid unlimited growth in memory and computation;
- remain stable when tasks arrive in different orders;
- distinguish reliable information from noise;
- resist malicious or contradictory training inputs.
The comparisons must also be fair.
A Dendritron module should be evaluated against ordinary adapters, LoRA, multilayer perceptrons, external retrieval systems, replay-based learning, and other continual-learning methods using comparable parameter counts and computational budgets.
Such comparisons can show whether the architecture outperforms existing alternatives, but not which of its internal components are responsible for the gains. That requires ablation studies, which reveal which parts of the architecture actually matter.
Researchers can remove or replace one component at a time. They might substitute dendritic branches with standard neural layers, disable selective routing, or eliminate local nonlinear processing.
Performance is then measured again. If removing a component reduces the gains, that component is probably important. If nothing changes, the added complexity may be unnecessary.
Otherwise, any advantage could simply result from adding more trainable computation rather than from the dendritic design itself.
The Hidden Cost of Continuous Learning
An AI that continues learning after deployment may become more useful, but it also becomes less predictable.
A frozen model is limited, yet relatively stable. A particular version can be tested, archived, compared, and reproduced.
A continually adapting model changes with experience.
Two identical copies could gradually become very different after interacting with different users, organizations, or information environments. Their behavior would depend not only on their original parameters but also on everything they had learned afterward.
This creates behavioral drift.
Each individual update may appear harmless. Yet thousands of small changes can accumulate, gradually altering the model’s priorities, tone, judgments, and responses in ways that were never explicitly designed.
The danger is not necessarily one dramatic failure. It is the slow movement of the system away from its original, tested behavior.
A model validated at launch may therefore no longer be the same model operating six months later. Its capabilities, biases, and failure modes may have changed through continuous adaptation.
Continuous learning transforms the model from a fixed technological product into an evolving process—one that must be monitored, audited, and governed throughout its entire operational life.
Memory Poisoning and Manipulation
A trainable memory also creates a new security surface.
A malicious user might repeatedly present false information, biased associations, or hidden instructions designed to influence future behavior.
In an ordinary prompt injection, the malicious instruction may disappear when the conversation ends. In a continually learning system, the same manipulation could be stored and become persistent.
This is memory poisoning.
An attacker might attempt to teach the model to trust an unreliable source, ignore certain safety constraints, favor a particular agenda, or respond differently when a hidden trigger appears.
Selective branches could limit the spread of the damage, but only if the system can identify that poisoning has occurred.
A safe memory architecture would require controls over what is allowed to become permanent. It would need provenance records, confidence estimates, reversible updates, independent validation, protected checkpoints, and mechanisms for detecting abnormal changes.
Continuous learning cannot mean indiscriminate learning.
Privacy and the Right to Be Forgotten
External memory is imperfect, but it can usually be inspected.
A stored document or database record can be found, edited, or deleted. Knowledge absorbed into neural parameters is much harder to locate.
Suppose the system learns a private fact about a user. Where is that fact stored? Which branch contains it? Has it spread across several branches? Can it be removed without damaging unrelated knowledge? How can the deletion be verified?
A modular dendritic architecture might make memory more localized and therefore easier to manage. Different users could have separate modules, and a compromised branch might theoretically be reset.
But this remains a hypothesis. Neural representations often become entangled even inside modular systems.
Users would need clear answers to several questions:
- Is the system learning from this interaction?
- Which information may be retained?
- Will it influence responses to other users?
- Can the memory be inspected?
- Can it be corrected or deleted?
- Can the system prove that deletion occurred?
An architecture that remembers permanently without meaningful control would turn personalization into a privacy problem.
Intelligence Also Requires Forgetting
The phrase “an AI that never forgets” sounds desirable because human beings often experience forgetting as a weakness.
But perfect memory may be neither natural nor intelligent.
Human memory is selective. We forget details, weaken unused associations, revise beliefs, and compress many experiences into broader concepts.
Forgetting reduces noise. It removes obsolete information, prevents overload, protects plasticity, and allows new interpretations to replace old ones.
A machine that stored every interaction with equal importance would accumulate misinformation, contradictions, irrelevant details, accidental correlations, and malicious inputs.
Over time, it might become less coherent rather than more capable.
The goal should therefore not be unlimited retention, but memory governance.
A mature artificial memory system would need to distinguish between information required only for the present conversation, temporary experiences worth monitoring, candidate memories awaiting validation, reliable knowledge suitable for consolidation, outdated information that should weaken, and dangerous or private information that should be erased.
Learning would become a controlled sequence of selection, verification, consolidation, revision, and forgetting.
The ability to forget correctly may be as important as the ability to remember.
A Parallel Approach: The Dragon Hatchling
The Dendritron is not the only attempt to move beyond the static memory of conventional Transformers. Another experimental architecture, known as the Dragon Hatchling, or BDH, explores the idea that memory should reside in changing connections rather than only in stored tokens.
The acronym BDH originally stood for Baby Dragon Hatchling, although the scientific paper generally refers to the architecture simply as Dragon Hatchling. The name reflects its experimental status: a system that has only recently “hatched,” but may point toward a more mature family of adaptive architectures.
BDH represents part of its working memory as temporary correlations between artificial neurons. These connections behave like fast synaptic weights, strengthening or weakening as the model processes new information.
This gives the architecture a form of dynamic memory during inference. Instead of preserving the entire past as an ever-growing sequence of tokens, BDH compresses previous information into an evolving associative state.
The resemblance to the Dendritron is clear. Both approaches separate relatively stable learned knowledge from a more plastic mechanism that changes during use. Both are inspired by the idea that biological intelligence depends not only on neural activity, but also on adaptable connections and local computation.
The crucial difference lies in what has been demonstrated. Public research on BDH primarily shows temporary adaptation and recurrent memory during inference. It does not yet prove that the system can safely consolidate new knowledge into permanent memory, preserve that knowledge across sessions, or avoid catastrophic forgetting over long periods.
BDH therefore reinforces the broader argument behind the Dendritron: the future of artificial intelligence may require architectures in which memory is an active and adaptive process rather than a passive archive of previous tokens.
At the same time, it supports the same cautionary conclusion. Dynamic memory is promising, but reliable lifelong learning remains unsolved.
The Dendritron Transformer addresses a real limitation of current artificial intelligence.
Transformer-based language models can process new information without permanently learning from it. Updating the entire model is expensive and risks catastrophic forgetting. External retrieval provides useful memory, but consulting stored information is not the same as incorporating it into the model’s internal knowledge.
A separate dendritic memory offers an interesting compromise.
The frozen Transformer preserves broad knowledge and general capabilities. The adaptive module learns from new experience. Branch-based processing may encourage specialization, selective activation, and more localized updates.
The Dragon Hatchling, or BDH, explores a related direction. Instead of relying only on an expanding token context, it maintains a dynamic internal state that can be interpreted as temporary synaptic connections changing during inference. This suggests that adaptive memory may be implemented inside the architecture without continuously retraining the entire model. Yet BDH primarily demonstrates dynamic working memory, not proven lifelong learning or permanent consolidation across sessions.
Together, Dendritron and BDH point toward a broader shift: memory may become an active, evolving component of the model rather than a passive archive of previous tokens.
But these architectures remain experimental hypotheses.
They must demonstrate that biologically inspired memory performs better than simpler alternatives. They must remain stable through long sequences of learning, resist manipulation, avoid uncontrolled growth, protect privacy, and support the correction or deletion of unwanted memories.
They must also solve a deeper problem: deciding when temporary adaptation should become permanent knowledge.
The central challenge is not simply to create an AI that remembers more.
It is to create one that can decide what deserves to become permanent, what should remain temporary, what must be questioned, and what should disappear.
The future of artificial intelligence may not belong to the machine that never forgets.
It may belong to the machine that learns what is worth remembering.
References
Ramón y Cajal, S. (1909–1911). Histologie du système nerveux de l’homme et des vertébrés. Paris: A. Maloine.
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133.
Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408.
Vaswani, A., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
Chavlis, S., & Poirazi, P. (2025). Dendrites endow artificial neural networks with accurate, robust and parameter-efficient learning. Nature Communications, 16, 943.
Pathway. (2026, March 17). Pathway’s BDH solves Sudoku Extreme with 97.4% accuracy, while leading LLMs are close to 0. https://pathway.com/research/beyond-transformers-sudoku-bench.
Editorial transparency note: This article, as with all articles published on this site, was conceived, directed, written, and reviewed by Prof. Maurício Veloso Brant Pinheiro. Artificial intelligence was used as an assistant for editorial refinement, formatting, image generation, SEO metadata, and publication workflow.
Copyright 2026 AI-Talks.org
