amduat-api/notes/ENC-ASL-LOG v0.1.md

Absolutely — let’s lay down ENC-ASL-LOG v0.1, a minimal deterministic encoding for the append-only ASL log. I’ll aim for something that is platform-independent, reproducible, and maps cleanly to your index and artifact workflow.

---

ENC-ASL-LOG v0.1

1. Purpose

This document specifies the exact byte-level encoding of the ASL append-only log.

It is intended for:

• C libraries
• CLI tools
• Memory-mapped readers
• Interchange between hosts

It defines encoding, record structure, and persistence semantics, but not log semantics — see ASL-STORE and ASL-CORE for ordering, snapshot, and visibility rules.

---

2. Encoding Principles

1. Little-endian integers (multi-byte)
2. Packed structures: no compiler padding
3. Forward-compatible versioning via header.version
4. Checksums for corruption detection
5. Deterministic serialization: same log content → same byte sequence

---

3. Log File Layout

+----------------+
| LogHeader      |
+----------------+
| LogRecord[ ]   |
+----------------+
| LogFooter      |
+----------------+

• LogHeader: fixed-size, mandatory, begins file
• LogRecord[]: append-only entries, variable number
• LogFooter: optional, contains global checksum

---

4. LogHeader

#pragma pack(push,1)
typedef struct {
    uint64_t magic;       // Unique magic for ASL log
    uint16_t version;     // Encoding version
    uint16_t flags;       // Reserved
    uint32_t header_size; // Total header bytes including this struct
    uint64_t first_snapshot; // First snapshot referenced
    uint64_t last_snapshot;  // Last snapshot referenced
} LogHeader;
#pragma pack(pop)

Notes:

• magic ensures correct file type
• version allows forward compatibility
• first_snapshot and last_snapshot help range validation

---

5. LogRecord

#pragma pack(push,1)
typedef enum {
    LOG_RECORD_ADD_INDEX_SEGMENT = 1,
    LOG_RECORD_SEAL_SEGMENT      = 2,
    LOG_RECORD_TOMBSTONE         = 3,
    LOG_RECORD_CUSTOM            = 0x1000
} LogRecordType;

typedef struct {
    uint64_t record_id;     // Unique log entry ID
    uint64_t timestamp_ns;  // Monotonic time of record creation
    uint64_t snapshot_id;   // Snapshot for which entry applies
    uint32_t type;          // LogRecordType
    uint32_t payload_size;  // Size of payload bytes following this header
} LogRecordHeader;
#pragma pack(pop)

• Immediately after LogRecordHeader, payload_size bytes of payload follow.
• Payload encodes type-specific information.

5.1 Payload Examples

Add Index Segment

struct {
    uint64_t segment_file_id; // Corresponding ENC-ASL-CORE-INDEX segment
    uint64_t entry_count;     // Number of index entries
    uint8_t  reserved[16];    // Future fields
};

Seal Segment

struct {
    uint64_t segment_file_id;
    uint64_t seal_snapshot_id;
    uint64_t seal_time_ns;
};

Tombstone

struct {
    uint64_t artifact_hash_hi;
    uint64_t artifact_hash_mid;
    uint64_t artifact_hash_lo;
    uint32_t artifact_hash_tail;
    uint32_t flags; // reserved
};

Custom

Arbitrary byte sequence, self-describing.

---

6. LogFooter

#pragma pack(push,1)
typedef struct {
    uint64_t crc64;      // CRC over entire log (excluding footer)
    uint64_t record_count;// Total records in log
} LogFooter;
#pragma pack(pop)

---

7. Record Ordering & Durability

• Entries are strictly append-only.
• record_id must be strictly increasing.
• snapshot_id reflects the visibility snapshot.
• Readers replay records in order to derive CURRENT state.

---

8. Checksums & Integrity

• All multi-byte fields are little-endian.
• LogFooter.crc64 is optional but recommended.
• Per-record integrity may optionally include a record CRC as first 8 bytes of payload.

---

9. Versioning

• LogHeader.version = 0x01
• New fields must not break existing layout.
• Unsupported versions must be rejected by readers.

---

10. Relationship to Other Layers

Layer	Responsibility
ASL-STORE	Lifecycle, visibility, snapshot enforcement
ASL-CORE-INDEX	Index semantics (ArtifactKey → BlockID mapping)
ENC-ASL-CORE-INDEX	Exact byte layout for index segments
ENC-ASL-LOG	Exact byte layout for append-only log

---

✅ This gives you a minimal, deterministic log encoding.

You could now use it to:

1. Append a new log record for an index segment
2. Seal segments
3. Mark tombstones
4. Replay in order to reconstruct CURRENT

---

If you want, I can draft a minimal C writer/reader API next that directly reads/writes this log, including helpers for add_index_segment(), seal_segment(), and tombstone() with payload serialization.

Do you want me to do that next?