LLM Memory
a day ago
- LLM memory is limited by small context windows, making consistent long-form story generation difficult.
- Knowledge requires reference frames (temporal, spatial, or abstract) to be valid, as facts change over time or context.
- Vector embeddings map text to N-dimensional points but struggle with episodic memories and are hard to reason about.
- Knowledge graphs use linked nodes, with edges either labeled (semantic) or unlabeled, to represent relationships.
- Meta-documents store cached results from past queries or reasoning, helping to consolidate and reuse knowledge.
- Forgetting mechanisms like connection decay or LLM-based pruning prevent unbounded graph growth and maintain usefulness.
- Episodic memory captures agent experiences in narrative form, often connected in a timeline for later retrieval.
- Memory traversal can be agentic, using priority queues and sub-agents to efficiently search relevant documents.
- Tools like SQLite tables or scratchpads can complement core memory, but models may mismanage pruning without structure.
- Implicit memory control is preferred over explicit agent control to avoid overconfidence and loops.
- Future neural methods may subsume these techniques with fully learned memory systems.