Add hierarchical memory management with multi-tier caching (RAM → SSD → Network)

[Copilot]

Implements multi-tier caching to scale beyond single-machine memory limits. Data automatically migrates between RAM, SSD, and network storage tiers based on access patterns.

Implementation

Storage tier abstraction (paper/storage_tier.py)

• Abstract StorageTier base class with automatic promotion/demotion
• RAMTier: In-memory cache with LRU eviction
• SSDTier: Disk-backed cache using NumPy .npy files
• NetworkTier: Simulated remote storage with configurable latency

Hierarchical buffer manager (paper/hierarchical_buffer.py)

• Orchestrates 3-tier hierarchy with automatic data movement
• Compatible with existing BufferManager interface
• Metrics: hits, misses, promotions, demotions, evictions per tier

Usage

from paper.hierarchical_buffer import HierarchicalBufferManager
from paper.backend import add

buffer_mgr = HierarchicalBufferManager(
    ram_capacity_tiles=64,
    ssd_capacity_tiles=256,
    network_capacity_tiles=1024
)

A = PaperMatrix("A.bin", shape=(8192, 8192))
B = PaperMatrix("B.bin", shape=(8192, 8192))
C = add(A, B, "C.bin", buffer_mgr)

buffer_mgr.print_metrics()  # View hit rates and tier utilization

Testing

27 new tests covering:

• Individual tier behavior (LRU eviction, capacity enforcement)
• Tier interactions (promotion, demotion, cascading)
• Matrix operations with hierarchical caching
• Thread safety and concurrent access

Demo: python demo_hierarchical.py

Cloud-native mapping

• RAM tier → Instance memory
• SSD tier → Persistent volumes / local SSD
• Network tier → Object storage (S3, Azure Blob, GCS)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Hierarchical Memory Management</issue_title>
<issue_description>Rationale: Multi-tier caching (RAM → SSD → Network) is critical for scaling beyond single-machine limits. This builds on your buffer pool architecture and aligns with modern cloud-native deployments.​

we need clear cut unit & system test for the functionality verification.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Hierarchical Memory Management #36

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[j143]

at this moment, this implementation may not be required. later when experiments show this is required! then we can add this.