amduat/docs/es-indexing-plan.md

Elasticsearch Indexing Plan

Status: Draft Scope: Minimal phased ES deployment aligned with indexing contract.

---

Phase A — Artifact Metadata Projection (Minimal)

Goal

Accelerate search and metadata filtering only.

Index Name

amduat_artifacts_v1

Document Structure

1. _id: canonical ReferenceBytes hex.
2. reference: canonical hex.
3. projection_schema_version: integer.
4. kind: keyword.
5. enc_profile: keyword.
6. labels: keyword array.
7. created_at: date, only if intrinsic to artifact.
8. text: text, optional, deterministic extractor.
9. extractor_version: keyword, if text present.

Data Source

1. ASL authoritative log/index.
2. Deterministic decoding only.

Query Patterns Accelerated

1. Text search.
2. Filter by kind.
3. Filter by labels.
4. Narrow candidate set.

Verification Rule

All returned References MUST be confirmed via ASL/1-STORE before returning artifact data.

---

Phase B — Edge Projection (Optional)

Goal

Accelerate candidate neighbor discovery.

Index Name

amduat_edges_v1

Document Structure

1. _id: edge Reference hex.
2. edge_reference: canonical hex.
3. src_ref: canonical hex.
4. dst_ref: canonical hex.
5. edge_label: keyword.
6. hash_id: keyword or numeric.
7. digest: keyword.
8. projection_schema_version: integer.

Data Source

1. EdgeArtifacts only.
2. TGK profile rules.
3. No inferred edges.

Verification Rule

Returned edges MUST be validated via TGK/STORE/1 authoritative query, and canonical ordering MUST be applied before client response.

---

Phase C — Enterprise Projection Hardening

Additions

1. Projection checkpoint artifacts stored in ASL.
2. Schema version recorded in checkpoint.
3. Full rebuild test automation.
4. Lag metrics published.
5. Projection audit job comparing snapshot vs ES.

---

Operational Guarantees

1. ES may be disabled without breaking semantics.
2. Projection rebuild must be deterministic.
3. Schema changes require replay.
4. Checkpoint artifact defines replay boundary.

---

Observability Requirements

Expose:

1. ASL head log position.
2. Indexer checkpoint position.
3. Computed lag.
4. Projection error counts.
5. Rebuild duration.
6. Verification fallback rate.

---

Final Guardrail

Elasticsearch accelerates discovery.

Authoritative semantics remain in ASL/1-STORE and TGK/STORE/1.

Ordering, identity, and correctness are never delegated to ES.