docs/Docs/Memory System
Core Concepts

The Three-Tier Memory System

Bastion implements a three-tier memory architecture inspired by cognitive science: working memory (STM), long-term memory (LTM), and forensic memory. Each tier serves a distinct purpose — fast retrieval, durable knowledge, and tamper-evident audit.

Memory Architecture Overview
Tier 1
Working Memory
In-memory LRU cache
< 1ms retrieval
Tier 2
Long-Term Memory
CockroachDB C-SPANN
15-30ms vector search
Tier 3
Forensic Memory
SHA-256 hash chain
Audit log + compliance

Tier 1 — Short-Term Memory (Working Cache)

The fastest tier. An in-memory LRU cache that holds recently and frequently accessed memories for sub-millisecond retrieval. When a memory is accessed 3+ times, it's automatically promoted from CockroachDB into the cache.

L1 Cache
In-memory dictionary keyed by memory_id. Evicts LRU when full (capacity: 1000). < 1ms retrieval.
L2 Storage
CockroachDB C-SPANN vector index. Persistent, distributed, ACID. 15-30ms retrieval.

The MemoryRouter sits between the application and storage, transparently routing searches across both tiers:

python
from bastion.cache_router import MemoryRouter

router = MemoryRouter(
    memory=bastion_memory,
    cache_size=1000,           # L1 capacity
    promotion_threshold=3,     # accesses before L1 promotion
    demotion_interval_seconds=300,  # cold-check interval
)

# Search merges L1 + L2 results
results = router.search("deploy pipeline", k=5)

# Stats show cache hit rate
stats = router.get_stats()
# → {"cache_size": 42, "hit_rate_percent": 78.5, ...}

Tier 2 — Long-Term Memory (Persistent Store)

The durable backbone. Every memory is stored in CockroachDB with SERIALIZABLE isolation, a SHA-256 hash chain, and 1024-dim vector embeddings for semantic search.

Memory Types

Inspired by cognitive science, Bastion classifies memories into three types:

📅
Episodic
"What happened" — events, experiences, conversations. Timestamped records of interactions and observations.
💡
Semantic
"What is true" — facts, knowledge, rules. Durable knowledge extracted from episodic memories via consolidation.
⚙️
Procedural
"How to do things" — workflows, decision patterns, learned procedures. Auto-detected from recurring task sequences.

The LTM Gateway

The LTMMemoryGatewayis the "money shot" — before running an expensive workflow, it checks if a similar analysis already exists in long-term memory. If a match above the 80% similarity threshold exists, it returns the cached result instantly.

python
from bastion.ltm_gateway import LTMMemoryGateway

gateway = LTMMemoryGateway(memory_engine, reuse_threshold=0.80)

# Check before expensive workflow
result = gateway.check_reuse("analyze Q2 revenue trends")
if result:
    print(f"Found {result.similarity:.1%} match — reusing cached analysis")
    # Skip the full pipeline, return cached insight
else:
    analysis = run_expensive_workflow(...)
    gateway.store_analysis("analyze Q2 revenue trends", analysis)

Multi-Agent Merge (CRDT)

When multiple agents share a memory store, conflicts are resolved using CRDTs (Conflict-free Replicated Data Types) with three merge strategies:

Last-Write-Wins
Highest vector clock wins. Fastest, no LLM needed. Best for: numeric counters, timestamps.
Element-Union (Add-Wins)
Merges sets by union. If one agent adds and another keeps, the add wins. Best for: tag lists, collections.
Semantic Merge
LLM-powered merge that understands meaning. Resolves factual contradictions intelligently. Best for: preferences, knowledge.

Tier 3 — Forensic Memory (Audit & Compliance)

The tamper-evident layer. Every memory write appends to an immutable audit log with SHA-256 hash chaining. If any record is modified, the chain breaks — detected instantly by the forensic report.

Hash Chain Integrity

sql
# Each memory block stores:
cryptographic_hash = SHA-256(content + previous_hash)
previous_hash      = SHA-256 of the last block

# Forensic verification (run via MCP tool):
SELECT memory_id, cryptographic_hash
FROM agent_audit
WHERE action = 'memory_store'
ORDER BY recorded_at ASC;

# If any record is modified:
#   Hash chain breaks → detected by forensic_report

Forensic Report

The forensic_report MCP tool queries live CockroachDB data to verify chain integrity, count audit entries, and check memory distribution — no mocks, real cluster queries:

Hash Chain Status
INTACT or BROKEN — verifies every hash links to its predecessor
Memory Distribution
Breakdown by type: episodic, semantic, procedural
Audit Log Entries
Total append-only records in the audit trail
Guard Statistics
Total checks, blocked count, block percentage
Pin / Access Stats
Pinned memories, avg access count, avg importance
Poison Detection
Count of memories flagged by OWASP ASI06 guard

AS OF SYSTEM TIME

CockroachDB's MVCC enables point-in-time queries — see exactly what your agent knew at any timestamp without maintaining separate snapshots:

sql
-- What did the agent know on Jan 15, 2026 at noon?
SELECT content, importance_score, created_at
FROM agent_memory
AS OF SYSTEM TIME '2026-01-15T12:00:00Z'
WHERE agent_id = 'my-agent'
ORDER BY importance_score DESC
LIMIT 10;

Sleep-Time Consolidation (Dreaming)

When agents are idle, background processes review recent episodic memories and consolidate learnings into durable knowledge — inspired by how the human brain consolidates memories during sleep.

Consolidation Pipeline
1. Review recent episodic memories
2. Extract patterns, lessons, recurring themes
3. Consolidate duplicates and near-matches
4. Promote high-value episodic → semantic
5. Prune low-value, expired, redundant memories
6. Log all actions in audit trail

Consolidation is triggered by:

✓ CockroachDB CDC changefeed (memory write threshold)
✓ Lambda scheduled event (every N minutes)
✓ Manual trigger via the dream() MCP tool
python
# Trigger consolidation manually
from bastion.dreaming import MemoryDreamer

dreamer = MemoryDreamer(memory_engine)
journal = dreamer.dream(agent_id="my-agent")

print(f"Consolidated {journal.memories_consolidated} memories")
print(f"Promoted {journal.memories_promoted} episodic → semantic")
print(f"Pruned {journal.memories_pruned} low-value memories")
print(f"Duration: {journal.duration_ms:.0f}ms"

Contradiction Detection

When a new memory is stored, Bastion scans existing memories for contradictions across three dimensions:

Negation
"User prefers English" vs "User does NOT prefer English" — direct factual opposition.
Temporal
"Server is in us-east-1" vs "Server moved to eu-west-1" — same entity, different time.
Semantic
"Budget is $50k" vs "Budget is $100k" — similar content, different claims.

Drift Detection

The MemoryDriftmodule monitors behavioral drift across five dimensions — detecting when an agent's behavior changes unexpectedly over time:

Semantic Similarity
Are new memories semantically different from the baseline?
Importance Distribution
Is the agent storing more/less important memories?
Trust Level
Are trust scores shifting over time?
Access Patterns
Are memories being accessed more/less frequently?
Content Length
Are memory descriptions getting longer/shorter?
See the schema in action
Explore the CockroachDB tables and query examples.
CockroachDB Features →