We Audited Our AI's 10,071 Memories: 91% Unverified

In short: Our audit reveals that while AI agents can readily accumulate a large volume of memories—reaching 10,071 across 284 projects—the genuine bottleneck is not storage infrastructure but the continuous work of management, curation, and trust establishment, with a striking 91% or 9,187 memories still pending verification and 76% or 7,624 sitting as stale candidates in need of attention.

AI Long-Term Memory Explained: The Overflowing Receipt Drawer#

AI long-term memory serves as the persistent store for facts, past interactions, and step-by-step procedures that agents draw upon across separate projects and sessions. Without strong curation processes, however, it quickly resembles a kitchen drawer into which every receipt from years of purchases gets tossed without sorting, date-checking, or verification of which ones still matter for taxes, warranties, or returns. New slips slide in effortlessly every day, yet the drawer grows into an undifferentiated pile where locating the correct receipt—or trusting that it remains valid—becomes increasingly difficult and time-consuming. The same dynamic appears in agent memory systems: ingestion happens automatically and at scale, but the downstream labor of organizing, confirming accuracy, and discarding what no longer applies lags far behind. This creates exactly the pattern our numbers expose.

What Does 91% Pending Verification Actually Cost AI Agents?#

Of the 10,071 total memories, 9,187 sit in the verify queue—roughly 91% of everything stored. These entries have been captured but never subjected to trust confirmation, cross-checking against other known information, or assessment of continued relevance. For agents that rely on this store to maintain continuity in long-running work such as game design or simulation projects, the practical effect is significant. Every unverified item carries latent risk: an agent might repeat a flawed assumption, surface an inaccurate detail as fact, or build subsequent reasoning on shaky ground. Because verification has not kept pace with accumulation, a growing verification debt now exists. Resolving it requires either substantial human review time or sophisticated automated validation layers that can propagate confidence across related entries. Until that debt shrinks, the memory system functions more as a high-volume suggestion box than a dependable knowledge foundation.

Why 76% Stale Candidates Signal an Aging Problem That Won't Fix Itself?#

Separately, 7,624 memories—about 76% of the total—have been flagged as stale candidates. These are older entries whose context may have shifted, whose supporting conditions may no longer hold, or whose original accuracy has simply degraded with time. AI memory does not refresh automatically the way a live database query might; once written, an item stays until an explicit process revisits it. In dynamic domains, yesterday's correct procedure or observed pattern can become misleading or inefficient today. High stale rates therefore translate into agents that may apply outdated rules, miss recent changes in project requirements, or retain low-value noise that crowds out fresher insights. The result is a quiet but compounding loss of adaptability. Much like human memory that retains childhood misconceptions unless actively updated through new evidence, agent memory accrues its own form of verification debt that only deliberate re-checking or pruning can reduce.

Memory Types and Project Skew: Where the Knowledge Actually Lives#

The composition of the store further shapes its usefulness. Semantic memories—core facts—account for roughly 60% or 6,025 entries. Episodic memories capturing specific experiences represent about 32% or 3,187 items. Procedural memories encoding repeatable steps make up only 9% or 859 entries. This distribution tilts heavily toward static knowledge and away from executable know-how, which can limit how effectively agents translate stored information into consistent action. Distribution across projects is even more uneven. Two projects alone—custom-game with 1,081 memories and noname-rpg with 690—hold the large majority of the total, while most of the remaining 282 projects contain zero, one, or two memories each. The system therefore delivers rich context to a small number of initiatives and near-empty recall to many others, creating pockets of capability rather than broad, transferable intelligence.

The Core Design Lesson: Verification and Aging Management Matter More Than Capacity#

The audit makes one conclusion unavoidable. Storage capacity is no longer the limiting factor; the 10,071 memories already exist. The decisive engineering and operational challenge is building reliable systems for verification at ingestion time, continuous confidence scoring, freshness detection, and selective removal of what has aged out. Without these capabilities, any long-term memory system will steadily fill with untrusted and outdated material, forcing agents to operate with growing uncertainty. Prioritizing curation pipelines, automated validation, and aging policies turns raw accumulation into trustworthy, usable intelligence. Everything else is simply more paper in the drawer.

Note: Data from internal curator memory DB audit conducted on 2026-07-13.

Reference links

• Semantic memory — Wikipedia
• Episodic memory — Wikipedia
• Procedural memory — Wikipedia