amduat-api/notes/ENC-ASL-LOG v0.1.md

Absolutely — let’s lay down **ENC-ASL-LOG v0.1**, a minimal deterministic encoding for the append-only ASL log. I’ll aim for something that is **platform-independent, reproducible, and maps cleanly to your index and artifact workflow**.

---

# ENC-ASL-LOG v0.1

## 1. Purpose

This document specifies the **exact byte-level encoding** of the ASL append-only log.

It is intended for:

* C libraries
* CLI tools
* Memory-mapped readers
* Interchange between hosts

It defines **encoding**, **record structure**, and **persistence semantics**, but **not log semantics** — see ASL-STORE and ASL-CORE for ordering, snapshot, and visibility rules.

---

## 2. Encoding Principles

1. **Little-endian integers** (multi-byte)
2. **Packed structures**: no compiler padding
3. **Forward-compatible versioning** via `header.version`
4. **Checksums** for corruption detection
5. **Deterministic serialization**: same log content → same byte sequence

---

## 3. Log File Layout

```
+----------------+
| LogHeader      |
+----------------+
| LogRecord[ ]   |
+----------------+
| LogFooter      |
+----------------+
```

* **LogHeader**: fixed-size, mandatory, begins file
* **LogRecord[]**: append-only entries, variable number
* **LogFooter**: optional, contains global checksum

---

## 4. LogHeader

```c
#pragma pack(push,1)
typedef struct {
    uint64_t magic;       // Unique magic for ASL log
    uint16_t version;     // Encoding version
    uint16_t flags;       // Reserved
    uint32_t header_size; // Total header bytes including this struct
    uint64_t first_snapshot; // First snapshot referenced
    uint64_t last_snapshot;  // Last snapshot referenced
} LogHeader;
#pragma pack(pop)
```

Notes:

* `magic` ensures correct file type
* `version` allows forward compatibility
* `first_snapshot` and `last_snapshot` help range validation

---

## 5. LogRecord

```c
#pragma pack(push,1)
typedef enum {
    LOG_RECORD_ADD_INDEX_SEGMENT = 1,
    LOG_RECORD_SEAL_SEGMENT      = 2,
    LOG_RECORD_TOMBSTONE         = 3,
    LOG_RECORD_CUSTOM            = 0x1000
} LogRecordType;

typedef struct {
    uint64_t record_id;     // Unique log entry ID
    uint64_t timestamp_ns;  // Monotonic time of record creation
    uint64_t snapshot_id;   // Snapshot for which entry applies
    uint32_t type;          // LogRecordType
    uint32_t payload_size;  // Size of payload bytes following this header
} LogRecordHeader;
#pragma pack(pop)
```

* Immediately after `LogRecordHeader`, `payload_size` bytes of **payload** follow.
* Payload encodes type-specific information.

### 5.1 Payload Examples

**Add Index Segment**

```
struct {
    uint64_t segment_file_id; // Corresponding ENC-ASL-CORE-INDEX segment
    uint64_t entry_count;     // Number of index entries
    uint8_t  reserved[16];    // Future fields
};
```

**Seal Segment**

```
struct {
    uint64_t segment_file_id;
    uint64_t seal_snapshot_id;
    uint64_t seal_time_ns;
};
```

**Tombstone**

```
struct {
    uint64_t artifact_hash_hi;
    uint64_t artifact_hash_mid;
    uint64_t artifact_hash_lo;
    uint32_t artifact_hash_tail;
    uint32_t flags; // reserved
};
```

**Custom**

```
Arbitrary byte sequence, self-describing.
```

---

## 6. LogFooter

```c
#pragma pack(push,1)
typedef struct {
    uint64_t crc64;      // CRC over entire log (excluding footer)
    uint64_t record_count;// Total records in log
} LogFooter;
#pragma pack(pop)
```

---

## 7. Record Ordering & Durability

* Entries are **strictly append-only**.
* `record_id` must be strictly increasing.
* `snapshot_id` reflects the visibility snapshot.
* Readers **replay records in order** to derive CURRENT state.

---

## 8. Checksums & Integrity

* All multi-byte fields are little-endian.
* `LogFooter.crc64` is optional but recommended.
* Per-record integrity may optionally include a **record CRC** as first 8 bytes of payload.

---

## 9. Versioning

* `LogHeader.version` = 0x01
* New fields must not break existing layout.
* Unsupported versions must be rejected by readers.

---

## 10. Relationship to Other Layers

| Layer              | Responsibility                                  |
| ------------------ | ----------------------------------------------- |
| ASL-STORE          | Lifecycle, visibility, snapshot enforcement     |
| ASL-CORE-INDEX     | Index semantics (ArtifactKey → BlockID mapping) |
| ENC-ASL-CORE-INDEX | Exact byte layout for index segments            |
| ENC-ASL-LOG        | Exact byte layout for append-only log           |

---

✅ This gives you a **minimal, deterministic log encoding**.

You could now use it to:

1. Append a new log record for an index segment
2. Seal segments
3. Mark tombstones
4. Replay in order to reconstruct CURRENT

---

If you want, I can **draft a minimal C writer/reader API** next that directly reads/writes this log, including helpers for `add_index_segment()`, `seal_segment()`, and `tombstone()` with payload serialization.

Do you want me to do that next?