NOTE: Superseded by `tier1/tgk-1.md` (TGK/1). Kept for historical context. Absolutely — here’s a **draft for ENC-ASL-TGK-INDEX**, carefully merging ASL artifact indexes and TGK edge indexes while respecting the **separation of concerns** and **snapshot determinism**. This design keeps **ENC-ASL-CORE** and **ENC-TGK-CORE** authoritative, and only merges **index references and acceleration structures**. --- # ENC-ASL-TGK-INDEX ### Merged On-Disk Index for ASL Artifacts and TGK Edges --- ## 1. Purpose ENC-ASL-TGK-INDEX defines a **unified on-disk index** that: * References **ASL artifacts** (ENC-ASL-CORE) * References **TGK edges** (ENC-TGK-CORE) * Supports **routing keys, filters, sharding, SIMD acceleration** per ASL-INDEX-ACCEL * Preserves **snapshot safety, log-sequence ordering, and immutability** > Semantic data lives in the respective CORE layers; this index layer **only stores references**. --- ## 2. Layering Principle | Layer | Responsibility | | --------------------- | -------------------------------------------- | | ENC-ASL-CORE | Artifact structure and type tags | | ENC-TGK-CORE | Edge structure (`from[] → to[]`) | | TGK-INDEX / ASL-INDEX | Canonical & routing keys, index semantics | | ENC-ASL-TGK-INDEX | On-disk references and acceleration metadata | **Invariant:** This index never re-encodes artifacts or edges. --- ## 3. Segment Layout Segments are **append-only** and **snapshot-bound**: ``` +-----------------------------+ | Segment Header | +-----------------------------+ | Routing Filters | +-----------------------------+ | ASL Artifact Index Records | +-----------------------------+ | TGK Edge Index Records | +-----------------------------+ | Optional Acceleration Data | +-----------------------------+ | Segment Footer | +-----------------------------+ ``` * Segment atomicity enforced * Footer checksum guarantees integrity --- ## 4. Segment Header ```c struct asl_tgk_index_segment_header { uint32_t magic; // 'ATXI' uint16_t version; uint16_t flags; uint64_t segment_id; uint64_t logseq_min; uint64_t logseq_max; uint64_t asl_record_count; uint64_t tgk_record_count; uint64_t record_area_offset; uint64_t footer_offset; }; ``` * `logseq_*` enforce snapshot visibility * Separate counts for ASL and TGK entries --- ## 5. Routing Filters Filters may be **segmented by type**: * **ASL filters**: artifact hash + type tag * **TGK filters**: canonical edge ID + edge type key + optional role ```c struct asl_tgk_filter_header { uint16_t filter_type; // e.g., BLOOM, XOR uint16_t version; uint32_t flags; uint64_t size_bytes; // length of filter payload }; ``` * Filters are advisory; false positives allowed, false negatives forbidden * Must be deterministic per snapshot --- ## 6. ASL Artifact Index Record ```c struct asl_index_record { uint64_t logseq; uint64_t artifact_id; // ENC-ASL-CORE reference uint32_t type_tag; // optional uint8_t has_type_tag; // 0 or 1 uint16_t flags; // tombstone, reserved }; ``` * `artifact_id` = canonical identity * No artifact payload here --- ## 7. TGK Edge Index Record ```c struct tgk_index_record { uint64_t logseq; uint64_t tgk_edge_id; // ENC-TGK-CORE reference uint32_t edge_type_key; // optional uint8_t has_edge_type; uint8_t role; // optional from/to/both uint16_t flags; // tombstone, reserved }; ``` * `tgk_edge_id` = canonical TGK-CORE edge ID * No node lists stored in index --- ## 8. Optional Node-Projection Records For acceleration: ```c struct node_edge_ref { uint64_t logseq; uint64_t node_id; // from/to node uint64_t tgk_edge_id; uint8_t position; // from or to }; ``` * Fully derivable from TGK-CORE edges * Optional; purely for lookup speed --- ## 9. Sharding and SIMD * Shard assignment is **routing key based** (ASL artifact or TGK edge) * SIMD arrays may store precomputed routing keys for fast filter evaluation * Must follow ASL-INDEX-ACCEL invariants: deterministic, immutable, snapshot-safe --- ## 10. Snapshot Interaction At snapshot `S`: * Segment visible if `logseq_min ≤ S` * ASL or TGK record visible if `logseq ≤ S` * Tombstones shadow earlier records * Filters may be used as advisory before canonical verification --- ## 11. Segment Footer ```c struct asl_tgk_index_segment_footer { uint64_t checksum; // covers header, filters, records uint64_t asl_record_bytes; uint64_t tgk_record_bytes; uint64_t filter_bytes; }; ``` * Ensures atomicity and completeness --- ## 12. Normative Invariants 1. **ASL artifact identity = ENC-ASL-CORE artifact ID** 2. **TGK edge identity = ENC-TGK-CORE edge ID** 3. Edge type tag and artifact type tag **do not affect canonical identity** 4. Filters are advisory only; no false negatives 5. Sharding is observationally invisible 6. Index records are immutable once written 7. Snapshot visibility strictly follows `logseq` 8. Determinism guaranteed per snapshot --- ## 13. Summary ENC-ASL-TGK-INDEX merges ASL artifacts and TGK edges into a **single, snapshot-safe, acceleration-friendly index layer**: * Keeps core semantics authoritative * Enables high-performance lookups using routing, sharding, SIMD, and filters * Preserves immutability and determinism * Fully compatible with ASL-INDEX-ACCEL principles This design supports billions of references while avoiding semantic collisions between ASL and TGK layers. --- If you want, the next step could be **drafting a unified query execution model** over this merged index, connecting **artifact lookups** and **TGK graph traversals** in a snapshot-safe, deterministic way. Do you want me to do that next?