🛡️ Write-Ahead Log (WAL) & Durability¶

Crash-resilient append-only write-ahead logging guaranteeing data durability, state recovery, and distributed sync across off-heap memory segments.

Why a Write-Ahead Log?¶

In native memory architectures, mutable cognitive state (vector projections, synaptic tags, association edge weights, and recall telemetry) is held in off-heap memory segments. While the operating system dirty-page flushing mechanism eventually writes dirty pages back to disk, a sudden JVM crash or power interruption could result in partial writes or page tearing.

The Spector Write-Ahead Log (WAL) resolves this by enforcing a strict durability invariant:

No memory mutation is reflected in off-heap structures until its immutable event has been durably appended and validated in the Write-Ahead Log.

graph LR
    subgraph "Write Path"
        OP["Client Operation<br/><i>remember / reinforce / forget</i>"] --> WAL["Append to WAL<br/><i>AppendMemory with CRC-32</i>"]
        WAL --> FSYNC["Storage Commit<br/><i>FileChannel Append</i>"]
        FSYNC --> SEG["Update Off-Heap Segment<br/><i>Bundle Memory Mapped Slices</i>"]
    end

    subgraph "Crash Recovery Path"
        START["JVM Restart / Node Init"] --> DISK["Scan WAL Chunk Files"]
        DISK --> CRC{"Verify CRC-32<br/>& Magic Bytes"}
        CRC -->|Corrupt| HALT["Isolate Corrupt Chunk"]
        CRC -->|Valid| REPLAY["Replay Events to Bundles<br/><i>Warm up in-memory state</i>"]
        REPLAY --> READY["Namespace Ready for Queries"]
    end

    style OP fill:#3498db,color:white
    style WAL fill:#2ecc71,color:white
    style FSYNC fill:#27ae60,color:white
    style SEG fill:#e67e22,color:white
    style REPLAY fill:#9b59b6,color:white

WAL File Architecture & Chunking¶

To prevent unbounded single-file growth and simplify point-in-time snapshots, the WAL utilizes a Chunked File Model:

namespaces/{namespace_id}/
├── wal/
│   ├── wal-000000.bin            # Archived chunk (frozen at 8MB)
│   ├── wal-000001.bin            # Archived chunk (frozen at 8MB)
│   └── wal-000002.bin            # Active write chunk (cursor appending)

1. Monotonic Sequences: Every WAL event is tagged with a strictly increasing 64-bit sequence counter (\(1, 2, 3, \dots, N\)), establishing a total global order across all operations in a namespace.

2. Chunk Rolling: When the active write chunk reaches the configured maximum file threshold (default: 8 MB), it is fsync-committed and closed. A new chunk is opened with the next sequential index.

3. Compaction & Archival: When an episodic partition bundle is sealed or when a baseline memory snapshot is saved, WAL chunks prior to the snapshot's high-water mark can be pruned or archived to cold object storage.

Binary Event Record Format¶

Each event in the WAL is serialized as an immutable, self-contained binary frame:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    magic (2B, 0x5741 'WA')    | ver(1B) |flags(1B)|type(1B)|    0x00
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|       id_length (2B)          | rsv(1B) |                      0x05
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+                    sequence_number (8B)                       +  0x08
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+                    timestamp_epoch_ms (8B)                    +  0x10
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      payload_length (4B)                      |  0x18
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   payload_crc32 (4B)                          |  0x1C
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      _reserved (4B)                           |  0x20
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    header_crc32 (4B)                          |  0x24
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+          variable_id (id_length bytes, UTF-8)                +  0x28
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+          variable_payload (payload_length bytes)              +
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|              8-byte alignment padding (0-7 bytes)             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Event Types¶

Integrity & Crash Recovery¶

When a namespace is opened or reopened after a shutdown:

sequenceDiagram
    participant Boot as Startup Orchestrator
    participant Disp as Recovery Dispatcher
    participant WAL as Chunk Scanner
    participant Kernel as Namespace Kernel

    Boot->>Disp: recoverNamespace(dir)
    Disp->>WAL: locateAllChunks()
    WAL-->>Disp: [wal-000000.bin, wal-000001.bin, ...]
    loop For Each Chunk in Monotonic Order
        Disp->>WAL: readNextEvent()
        WAL->>WAL: Verify Magic & CRC-32
        alt Checksum Matches
            Disp->>Kernel: dispatchReplay(event)
            Kernel->>Kernel: Re-apply mutation to bundle slices
        else Checksum Mismatch (Corrupt Record)
            WAL-->>Disp: Report Corruption Point
            Disp->>Disp: Truncate torn write at end of log
        end
    end
    Disp-->>Boot: Recovery Complete (HWM Restored)

1. Checksum Verification: The scanner validates the CRC-32 hash of every record before evaluating the payload. A damaged record caused by sudden power loss mid-write is detected instantly.

2. Torn Write Truncation: If the final record in the active chunk is truncated due to a power loss, the recovery engine rolls back to the last valid verified record boundary, preventing corruption of pre-allocated bundles.

3. State Re-establishment: Events are re-applied to the bundle memory slices, restoring engram counts, graph weights, and the sequence counter.

4. Warming Replay: Frequently accessed vector segments and index pointers are touched during replay, warming the OS page cache before external client queries begin.