COMPLETED Research

MemoryCompiler

Memory compression framework for long conversations based on compiler optimization theory

status COMPLETED
type Research
started 2024-10-01
completed 2025-01-01
stack Python Qwen 2.5 PyTorch HuggingFace Transformers

// DESCRIPTION

MemoryCompiler: Compiling Interaction Histories into Executable Knowledge

MemoryCompiler addresses the challenge of converting unstructured LLM interaction histories into structured, executable knowledge representations. Rather than storing raw conversation logs, the system "compiles" interactions into a typed knowledge graph with executable procedures, enabling agents to not just remember facts but replay and adapt learned workflows.

The compilation pipeline extracts entities, relationships, and procedural steps from conversation histories using a multi-pass LLM analysis, resolving coreferences and temporal dependencies. The output is a typed graph where nodes represent concepts and edges encode both semantic relationships and causal/procedural orderings. Procedural subgraphs can be "executed" by replaying the distilled steps with new parameters.

Evaluation on the WikiHow procedural task benchmark shows that compiled memory representations achieve 55% higher procedure reproduction accuracy compared to retrieval-augmented generation over raw histories. The compilation step also reduces storage requirements by 80% through deduplication and abstraction.

The project is complete and provides a robust foundation for building agents that learn reusable procedures from experience, bridging the gap between episodic memory and procedural knowledge in LLM-based systems.

// HIGHLIGHTS

  • 55% higher procedure reproduction accuracy vs. RAG over raw interaction histories
  • 80% storage reduction through knowledge compilation, deduplication, and abstraction
  • Typed knowledge graph with executable procedural subgraphs
  • Multi-pass LLM analysis pipeline for entity extraction, coreference resolution, and causal ordering
  • Foundation for agents that learn reusable procedures from experience