# Amduat Open-Source Backend Proposal ## Contract-First Backend Integration Architecture **Status:** Proposal **Scope:** Define how the OSS stack (MinIO, JanusGraph, Cassandra, Elasticsearch, Spark, Nessie, IPFS, Ollama) can be deployed as a backend implementation of Amduat while preserving semantic invariants. --- # Related Documents * `docs/implementation-plan.md` * `docs/phase0-1-execution-checklist.md` * `docs/decision-checklist.md` * `docs/conformance-harness-spec.md` * `docs/adapter-error-taxonomy.md` * `docs/tgk-ordering-enforcement.md` * `docs/indexing-requirements-and-projections.md` * `docs/es-indexing-plan.md` * `docs/projection-checkpoint-schema.md` * `docs/oss-backend-adapter-boundary.md` --- # 1. Executive Summary This document proposes integrating a scalable open-source infrastructure stack as a backend implementation of Amduat. The proposal preserves Amduat’s defining principle: > The semantic core is invariant. > Backends are interchangeable implementations of store/query surfaces. The OSS stack provides scalability, distribution, analytics, and optional acceleration layers without altering the deterministic semantics defined by the Amduat core specifications. --- # 2. Architectural Position ## 2.1 Semantic Core Core semantics are defined by: * `ASL/1-CORE` * `PEL/1-CORE` * `TGK/1-CORE` Canonical encodings are provided by: * `ENC/ASL1-CORE` (and related encoding profiles) Identity derivation is defined by: * `ASL/1-CORE` * `HASH/ASL1` These define: * Artifact identity and immutability * Core execution semantics * Core graph semantics * Deterministic projection rules These are deployment-invariant. ## 2.2 Store / Query Surfaces Operational interfaces are defined by: * `ASL/1-STORE` * `TGK/STORE/1` Optional provenance semantics: * `TGK/PROV/1` Storage and indexing layers are pluggable implementations of these surfaces. They do not redefine core semantics. --- # 3. No Semantic Drift Rule To preserve Amduat’s portability and determinism, all backend implementations must adhere to the following: 1. **No Backend-Specific Semantics** * No interpretation of artifacts beyond canonical encoding. * No implicit schema expansion. * No hidden type coercion. 2. **No Semantic Elevation of Projections** * Search indexes (e.g., Elasticsearch) * Distribution caches (e.g., IPFS) * Analytics views (e.g., Spark) * AI-generated outputs …are derived projections unless explicitly promoted by governance. 3. **Deterministic Reproducibility** * The logical TGK graph must be reproducible from ASL state. * Snapshot exports must reproduce identical graph projections. 4. **Surface Conformance Only** * The OSS stack may optimize storage and indexing. * It may not redefine core behavior. 5. **Observational Equivalence** * Any backend-specific optimization must be observationally equivalent to the core semantics and query results defined by the specifications. This rule guarantees backend substitution without semantic drift. --- # 4. ASL/1-STORE Implementation (Artifact Layer) ## 4.1 Contract Must implement: * `put(Artifact) -> Reference` * `get(Reference) -> Artifact | NotFound` Identity derivation follows: 1. Canonical encoding (`ENC/ASL1-CORE`) 2. Identity derivation rules (`ASL/1-CORE`) 3. Hash function (`HASH/ASL1`) In abstract form: ``` Reference = HASH( CANONICAL_ENCODE(Artifact) ) ``` Reference derivation is deterministic and caller-independent. ## 4.2 Immutability * Artifacts are immutable. * Rewriting the same Artifact yields the same Reference. * Mutation semantics do not exist at this layer. ## 4.3 Authority Rule The authoritative store in any deployment is the backend implementation that conforms to `ASL/1-STORE`. `ASL/1-STORE` is a specification; authority resides in whichever concrete backend implements it. Caches and projections derive from this authoritative ASL implementation unless explicitly promoted by governance. ## 4.4 Proposed Backend Primary implementation: * MinIO-backed object storage Optional: * IPFS as distribution/cache layer --- # 5. TGK/STORE/1 Implementation (Graph Query Surface) ## 5.1 Projection Model The TGK graph is a logical projection derived from: * Artifacts stored via ASL * Core graph semantics defined in `TGK/1-CORE` It is not an independently authoritative database. Graph state must be reproducible from ASL state. ## 5.2 Required Operations Backends must implement the required query operations and error model defined in `TGK/STORE/1`. This proposal references the specification directly to avoid semantic drift through re-enumeration. ## 5.3 Read-Only Nature * `GraphStoreInstance` is logically read-only. * Edges arise from writing EdgeArtifacts to ASL. * Deletion, compaction, or tombstone policies are backend or profile-level implementation concerns and are not part of `TGK/STORE/1`. ## 5.4 Proposed Backend * JanusGraph as query engine * Cassandra as storage backend * Optional Elasticsearch as search projection JanusGraph functions as an implementation detail of the `TGK/STORE/1` surface. --- # 6. TGK/PROV/1 (Optional Provenance Layer) `TGK/PROV/1` defines pure functions over a `ProvenanceGraph`. * It does not require a particular store. * It is evaluated against `TGK/1-CORE` graph semantics. * Implementations may use `TGK/STORE/1` queries to realize provenance functions. Provenance is layered above store semantics and does not alter core behavior. Implementation options: * Native traversal realization * Snapshot-based traversal in analytics engines * Hybrid realization --- # 7. Projection & Acceleration Layers Unless explicitly promoted by governance: * Elasticsearch * IPFS * Lakehouse datasets (MinIO + Nessie) * Spark analytics outputs * AI-derived artifacts …are derived projections. Authoritative truth resides in: * The deployed `ASL/1-STORE` implementation * Deterministic TGK projection derived from ASL state Projections may be eventually consistent. Audit-grade claims must reference reproducible ASL state or snapshot identifiers. --- # 8. Capability Tiers Capability tiers define operational envelope, not semantic differences. --- ## Tier 0 — Embedded / Edge Implements: * `ASL/1-STORE` * `TGK/STORE/1` * Optional minimal `TGK/PROV/1` Characteristics: * Single-node * Deterministic local projection * Strong local consistency * Optional sync/export Use cases: * Personal knowledge systems * Offline assistant systems --- ## Tier 1 — Platform / Scale-Out Implements: * `ASL/1-STORE` * `TGK/STORE/1` * Optional `TGK/PROV/1` Typical infrastructure: * Distributed object store (e.g., MinIO) * Distributed graph backend (e.g., JanusGraph over Cassandra) * Optional derived search index Characteristics: * Multi-node * Horizontal scaling * Operational consistency * Deterministic core preserved Use cases: * Community platforms * Shared knowledge systems --- ## Tier 2 — Enterprise / Audit-Grade Includes Tier 1 plus: * Snapshot export capability * Lakehouse projection * Versioned snapshots (e.g., Nessie) * Governance workflows * Promotion pipelines Characteristics: * Snapshot-based reproducibility * Audit traceability * Multi-environment promotion Use cases: * Enterprise backbone systems * Regulated environments --- # 9. Conformance Requirements Each backend must: * Pass `ASL/1-STORE` conformance tests * Pass `TGK/STORE/1` conformance and deterministic ordering tests * Demonstrate projection reproducibility from ASL state * Preserve encoding stability across versions Tier 2 must additionally: * Validate snapshot equivalence * Demonstrate replay determinism from ASL state Conformance is versioned alongside spec revisions. --- # 10. Design Outcome This architecture ensures: * One invariant semantic kernel * Backend substitution without semantic drift * Scaling through infrastructure choice * Deterministic reproducibility * Clear separation between authority and projection layers * Portability from embedded edge systems to enterprise-scale deployments The OSS stack is therefore a backend implementation of Amduat — not a semantic authority.