Absolutely — let’s lay down **ENC-ASL-LOG v0.1**, a minimal deterministic encoding for the append-only ASL log. I’ll aim for something that is **platform-independent, reproducible, and maps cleanly to your index and artifact workflow**. --- # ENC-ASL-LOG v0.1 ## 1. Purpose This document specifies the **exact byte-level encoding** of the ASL append-only log. It is intended for: * C libraries * CLI tools * Memory-mapped readers * Interchange between hosts It defines **encoding**, **record structure**, and **persistence semantics**, but **not log semantics** — see ASL-STORE and ASL-CORE for ordering, snapshot, and visibility rules. --- ## 2. Encoding Principles 1. **Little-endian integers** (multi-byte) 2. **Packed structures**: no compiler padding 3. **Forward-compatible versioning** via `header.version` 4. **Checksums** for corruption detection 5. **Deterministic serialization**: same log content → same byte sequence --- ## 3. Log File Layout ``` +----------------+ | LogHeader | +----------------+ | LogRecord[ ] | +----------------+ | LogFooter | +----------------+ ``` * **LogHeader**: fixed-size, mandatory, begins file * **LogRecord[]**: append-only entries, variable number * **LogFooter**: optional, contains global checksum --- ## 4. LogHeader ```c #pragma pack(push,1) typedef struct { uint64_t magic; // Unique magic for ASL log uint16_t version; // Encoding version uint16_t flags; // Reserved uint32_t header_size; // Total header bytes including this struct uint64_t first_snapshot; // First snapshot referenced uint64_t last_snapshot; // Last snapshot referenced } LogHeader; #pragma pack(pop) ``` Notes: * `magic` ensures correct file type * `version` allows forward compatibility * `first_snapshot` and `last_snapshot` help range validation --- ## 5. LogRecord ```c #pragma pack(push,1) typedef enum { LOG_RECORD_ADD_INDEX_SEGMENT = 1, LOG_RECORD_SEAL_SEGMENT = 2, LOG_RECORD_TOMBSTONE = 3, LOG_RECORD_CUSTOM = 0x1000 } LogRecordType; typedef struct { uint64_t record_id; // Unique log entry ID uint64_t timestamp_ns; // Monotonic time of record creation uint64_t snapshot_id; // Snapshot for which entry applies uint32_t type; // LogRecordType uint32_t payload_size; // Size of payload bytes following this header } LogRecordHeader; #pragma pack(pop) ``` * Immediately after `LogRecordHeader`, `payload_size` bytes of **payload** follow. * Payload encodes type-specific information. ### 5.1 Payload Examples **Add Index Segment** ``` struct { uint64_t segment_file_id; // Corresponding ENC-ASL-CORE-INDEX segment uint64_t entry_count; // Number of index entries uint8_t reserved[16]; // Future fields }; ``` **Seal Segment** ``` struct { uint64_t segment_file_id; uint64_t seal_snapshot_id; uint64_t seal_time_ns; }; ``` **Tombstone** ``` struct { uint64_t artifact_hash_hi; uint64_t artifact_hash_mid; uint64_t artifact_hash_lo; uint32_t artifact_hash_tail; uint32_t flags; // reserved }; ``` **Custom** ``` Arbitrary byte sequence, self-describing. ``` --- ## 6. LogFooter ```c #pragma pack(push,1) typedef struct { uint64_t crc64; // CRC over entire log (excluding footer) uint64_t record_count;// Total records in log } LogFooter; #pragma pack(pop) ``` --- ## 7. Record Ordering & Durability * Entries are **strictly append-only**. * `record_id` must be strictly increasing. * `snapshot_id` reflects the visibility snapshot. * Readers **replay records in order** to derive CURRENT state. --- ## 8. Checksums & Integrity * All multi-byte fields are little-endian. * `LogFooter.crc64` is optional but recommended. * Per-record integrity may optionally include a **record CRC** as first 8 bytes of payload. --- ## 9. Versioning * `LogHeader.version` = 0x01 * New fields must not break existing layout. * Unsupported versions must be rejected by readers. --- ## 10. Relationship to Other Layers | Layer | Responsibility | | ------------------ | ----------------------------------------------- | | ASL-STORE | Lifecycle, visibility, snapshot enforcement | | ASL-CORE-INDEX | Index semantics (ArtifactKey → BlockID mapping) | | ENC-ASL-CORE-INDEX | Exact byte layout for index segments | | ENC-ASL-LOG | Exact byte layout for append-only log | --- ✅ This gives you a **minimal, deterministic log encoding**. You could now use it to: 1. Append a new log record for an index segment 2. Seal segments 3. Mark tombstones 4. Replay in order to reconstruct CURRENT --- If you want, I can **draft a minimal C writer/reader API** next that directly reads/writes this log, including helpers for `add_index_segment()`, `seal_segment()`, and `tombstone()` with payload serialization. Do you want me to do that next?