# Projection Checkpoint Schema Status: Draft Scope: Schema for projection checkpoint artifacts used by derived indexers (e.g., Elasticsearch) to bind schema versions, replay state, and lag metrics to authoritative ASL state. --- # 1. Purpose Projection checkpoints make derived indexes replayable, auditable, and bounded. They tie a projection schema version to a specific ASL log position and snapshot identity so that rebuilds are deterministic and lag can be measured. --- # 2. Canonical Checkpoint Artifact Checkpoint artifacts are ASL artifacts whose bytes encode the following fields. The encoding is deterministic and versioned. ## 2.1 Minimal Fields 1. `checkpoint_version` (u32) 2. `projection_name` (string) 3. `projection_schema_version` (u32) 4. `asl_snapshot_id` (u64, optional if not available) 5. `asl_log_position` (u64, required) 6. `created_at` (u64, epoch milliseconds) 7. `builder_id` (string, identifying the indexer instance) ## 2.2 Optional Fields 1. `artifact_count` (u64) 2. `edge_count` (u64) 3. `error_count` (u64) 4. `lag_millis` (u64, derived) 5. `note` (string) --- # 3. Identity and Storage 1. Checkpoints are stored as ASL artifacts using ASL/1-STORE semantics. 2. `projection_name` plus `projection_schema_version` must uniquely identify a projection format. 3. The checkpoint artifact reference is the authoritative identifier for the projection state. --- # 4. Lag Metric Binding Lag is defined as: ``` Lag = ASL_head_log_position - checkpoint.asl_log_position ``` The ASL head position is authoritative. The checkpoint log position is the last fully processed position. --- # 5. Replay Rules 1. Rebuild begins at `checkpoint.asl_log_position + 1`. 2. Any schema change requires full replay from the earliest available log position. 3. Checkpoints must be monotonic in `asl_log_position` for a given projection. --- # 6. Validation Requirements When loading a checkpoint, the indexer MUST verify: 1. `projection_schema_version` matches the current build. 2. `asl_log_position` is not ahead of the authoritative ASL head. 3. `checkpoint_version` is supported. 4. If present, `asl_snapshot_id` matches the expected snapshot boundary. --- # 7. Operational Guidance 1. Checkpoints should be written after a successful batch commit. 2. Checkpoints should be immutable; new checkpoints are separate artifacts. 3. Retain checkpoints for audit and rollback.