121 lines
2.4 KiB
Markdown
121 lines
2.4 KiB
Markdown
|
# Elasticsearch Indexing Plan
|
||
|
|
|
||
|
|
Status: Draft
|
||
|
|
Scope: Minimal phased ES deployment aligned with indexing contract.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Phase A — Artifact Metadata Projection (Minimal)
|
||
|
|
|
||
|
|
## Goal
|
||
|
|
|
||
|
|
Accelerate search and metadata filtering only.
|
||
|
|
|
||
|
|
## Index Name
|
||
|
|
|
||
|
|
`amduat_artifacts_v1`
|
||
|
|
|
||
|
|
## Document Structure
|
||
|
|
|
||
|
|
1. `_id`: canonical `ReferenceBytes` hex.
|
||
|
|
2. `reference`: canonical hex.
|
||
|
|
3. `projection_schema_version`: integer.
|
||
|
|
4. `kind`: keyword.
|
||
|
|
5. `enc_profile`: keyword.
|
||
|
|
6. `labels`: keyword array.
|
||
|
|
7. `created_at`: date, only if intrinsic to artifact.
|
||
|
|
8. `text`: text, optional, deterministic extractor.
|
||
|
|
9. `extractor_version`: keyword, if `text` present.
|
||
|
|
|
||
|
|
## Data Source
|
||
|
|
|
||
|
|
1. ASL authoritative log/index.
|
||
|
|
2. Deterministic decoding only.
|
||
|
|
|
||
|
|
## Query Patterns Accelerated
|
||
|
|
|
||
|
|
1. Text search.
|
||
|
|
2. Filter by kind.
|
||
|
|
3. Filter by labels.
|
||
|
|
4. Narrow candidate set.
|
||
|
|
|
||
|
|
## Verification Rule
|
||
|
|
|
||
|
|
All returned References MUST be confirmed via ASL/1-STORE before returning artifact data.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Phase B — Edge Projection (Optional)
|
||
|
|
|
||
|
|
## Goal
|
||
|
|
|
||
|
|
Accelerate candidate neighbor discovery.
|
||
|
|
|
||
|
|
## Index Name
|
||
|
|
|
||
|
|
`amduat_edges_v1`
|
||
|
|
|
||
|
|
## Document Structure
|
||
|
|
|
||
|
|
1. `_id`: edge Reference hex.
|
||
|
|
2. `edge_reference`: canonical hex.
|
||
|
|
3. `src_ref`: canonical hex.
|
||
|
|
4. `dst_ref`: canonical hex.
|
||
|
|
5. `edge_label`: keyword.
|
||
|
|
6. `hash_id`: keyword or numeric.
|
||
|
|
7. `digest`: keyword.
|
||
|
|
8. `projection_schema_version`: integer.
|
||
|
|
|
||
|
|
## Data Source
|
||
|
|
|
||
|
|
1. EdgeArtifacts only.
|
||
|
|
2. TGK profile rules.
|
||
|
|
3. No inferred edges.
|
||
|
|
|
||
|
|
## Verification Rule
|
||
|
|
|
||
|
|
Returned edges MUST be validated via TGK/STORE/1 authoritative query, and canonical ordering MUST be applied before client response.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Phase C — Enterprise Projection Hardening
|
||
|
|
|
||
|
|
## Additions
|
||
|
|
|
||
|
|
1. Projection checkpoint artifacts stored in ASL.
|
||
|
|
2. Schema version recorded in checkpoint.
|
||
|
|
3. Full rebuild test automation.
|
||
|
|
4. Lag metrics published.
|
||
|
|
5. Projection audit job comparing snapshot vs ES.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Operational Guarantees
|
||
|
|
|
||
|
|
1. ES may be disabled without breaking semantics.
|
||
|
|
2. Projection rebuild must be deterministic.
|
||
|
|
3. Schema changes require replay.
|
||
|
|
4. Checkpoint artifact defines replay boundary.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Observability Requirements
|
||
|
|
|
||
|
|
Expose:
|
||
|
|
1. ASL head log position.
|
||
|
|
2. Indexer checkpoint position.
|
||
|
|
3. Computed lag.
|
||
|
|
4. Projection error counts.
|
||
|
|
5. Rebuild duration.
|
||
|
|
6. Verification fallback rate.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Final Guardrail
|
||
|
|
|
||
|
|
Elasticsearch accelerates discovery.
|
||
|
|
|
||
|
|
Authoritative semantics remain in ASL/1-STORE and TGK/STORE/1.
|
||
|
|
|
||
|
|
Ordering, identity, and correctness are never delegated to ES.
|