- A conversational AI with long-term memory stores (vector and fact) had an extractor that over-produced junk milestones (~6.5/day), causing prompt monoculture with forced romantic declarations.
- Diagnosis used a trace inspector and golden set; a keyword-as-necessary-condition rule blocked 94% of historical junk while passing 100% of positive controls.
- Fix A stopped the tap: keyword gate, roleplay guard, tightened dictionaries, anti-multi-label, deployed behind a test suite.
- Fix B cleaned existing data via reclassification (never delete) using an LLM judge, after avoiding prompt contamination and unsound shortcuts.
- Measurement showed new milestones/day dropped 92% (6.5 to 0.5), entity labels per message reduced to 1, and forced milestones per prompt dropped from 2.0 to 0.65.
- A post-fix incident revealed a relevance failure: retyped facts were force-injected unconditionally, causing accurate but irrelevant memories, highlighting a gap in context-aware injection.
- Qualitative contrast: before fix, assistant drifted romantic; after fix, it stayed on-topic, recalled specifics correctly, and admitted gaps instead of fabricating.
- Remaining issues include always-include inflation, extractor gate leaks, date over-extraction, parallel junk populations, debugger persona limitations, and untested scale/volume degradation.
- Key takeaways: memory can be accurate but wrong (relevance vs truth), instrument before tuning, and reversibility enables honest shipping and measurement.