amduat/docs/federation-implementation-notes.md

# Federation Implementation Notes (Core)

Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-02-XX

## Purpose

These notes bind the federation semantics docs to concrete core-library
responsibilities, APIs, and data structures. The intent is to keep
federation logic inside the core substrate and keep daemon/frontends thin.

## Normative inputs

Core semantics and replay:
- tier1/asl-federation-1.md
- tier1/asl-federation-replay-1.md
- tier1/asl-store-index-1.md
- tier1/enc-asl-core-index-1.md

Admission and policy gating:
- tier1/asl-dap-1.md
- tier1/asl-policy-hash-1.md
- tier1/asl-domain-model-1.md
- tier1/asl-dam-1.md

Contextual alignment:
- tier1/asl-system-1.md
- tier1/asl-encrypted-blocks-1.md

## Scope (core library)

Federation MUST be implemented as core substrate logic:
- Deterministic federation view construction.
- Replay ordering and bounds per domain.
- Imported record metadata preservation (domain_id, visibility,
  cross_domain_source).
- Tombstone and shadowing behavior per domain.

The following are explicitly out of scope for core:
- Transport protocols (HTTP, IPC, gossip).
- Peer discovery and operational orchestration.
- Admin UX and deployment wiring.
- Admission workflows, auth, and retries/backoff.
- Cache policy knobs (fetch timing, eviction, prefetch).
- Operational concerns (metrics, admin endpoints).
- Policy evaluation and per-record filtering decisions.

## Layering note

Core provides deterministic federation semantics and view construction only.
Middle-layer components are responsible for transport, admission workflows,
policy evaluation (including per-record filtering), caching strategies, and
operational wiring.

Definition:
- Middle layer: the daemon/service boundary around core logic that owns
  network transport, admission workflows, and operational policy.

## Responsibilities

1) Federation registry
   - Tracks known domains, admission status, policy hash, and
     last admitted {SnapshotID, LogPrefix}.
   - Enforces DAP + policy compatibility prior to admitting remote state.

2) Federation view cache
   - Materializes a deterministic view from local state + admitted
     published records from other domains.
   - Stores imported records with origin metadata for replay.
   - Tracks per-domain replay high-water {domain_id, logseq}.

3) Resolver
   - Resolves ArtifactKey using local store + federation view.
   - Does not mutate store/index as part of GET semantics.

## Data model (suggested)

```c
typedef enum {
    AMDUAT_FED_REC_ARTIFACT = 0,
    AMDUAT_FED_REC_PER      = 1,
    AMDUAT_FED_REC_TGK_EDGE = 2,
    AMDUAT_FED_REC_TOMBSTONE = 3
} amduat_fed_record_type_t;

typedef struct {
    uint32_t domain_id;
    uint64_t snapshot_id;
    uint64_t log_prefix;
    uint64_t last_logseq;
    uint8_t  admitted;     // boolean
    uint8_t  policy_ok;    // boolean
    uint8_t  reserved[6];
    amduat_hash_id_t policy_hash_id;
    amduat_octets_t  policy_hash;
} amduat_fed_domain_state_t;

typedef struct {
    uint32_t domain_id;
    uint8_t  visibility;   // 0 internal, 1 published
    uint8_t  has_source;   // 0/1
    uint16_t reserved0;
    uint32_t source_domain;
} amduat_fed_record_meta_t;

typedef struct {
    amduat_fed_record_type_t type;
    union {
        amduat_asl_artifact_key_t artifact_key;
        amduat_asl_tgk_edge_key_t tgk_edge_key;
        amduat_asl_per_key_t      per_key;
        amduat_asl_artifact_key_t tombstone_key; // key being removed
    } id;
} amduat_fed_record_id_t;

typedef struct {
    amduat_fed_record_meta_t meta;
    amduat_fed_record_id_t id;
    amduat_asl_artifact_location_t loc;
    uint64_t logseq;
    uint64_t snapshot_id;
    uint64_t log_prefix;
} amduat_fed_index_record_t;

typedef struct {
    amduat_fed_record_id_t id;
    uint32_t reason_code;  // policy-specific; 0 if unknown
} amduat_fed_policy_deny_t;
```

Notes:
- Imported records MUST retain domain_id and cross-domain source metadata.
- Tombstones must retain domain_id/visibility for domain-local shadowing.
- Each record MUST include a record type and canonical identity for deterministic
  replay across artifacts, PERs, TGK edges, and tombstones.
- PER/TGK canonical identities are currently represented by ASL references
  (artifact IDs); no separate edge/PER key types exist yet.

## Core API sketch

```c
typedef struct amduat_fed_registry_t amduat_fed_registry_t;
typedef struct amduat_fed_view_t     amduat_fed_view_t;

amduat_fed_registry_t *amduat_fed_registry_open(...);
void amduat_fed_registry_close(amduat_fed_registry_t *);

// Admission + policy gating
bool amduat_fed_admit_domain(amduat_fed_registry_t *, domain_id, policy_hash, ...);
bool amduat_fed_set_domain_state(amduat_fed_registry_t *, domain_id,
                                 snapshot_id, log_prefix);

// Ingest published records for a domain (already transported).
// Each record MUST include its type and canonical identity in the id field.
bool amduat_fed_ingest_records(amduat_fed_registry_t *, domain_id,
                               const amduat_fed_index_record_t *records,
                               size_t count);

// Build or refresh a deterministic federation view.
amduat_fed_view_t *amduat_fed_view_build(amduat_fed_registry_t *,
                                         const amduat_asl_store_t *local_store,
                                         const amduat_asl_index_state_t *local_state);

// Resolve via local store + federated view.
amduat_asl_store_error_t amduat_fed_resolve(
    const amduat_fed_view_t *view,
    const amduat_asl_artifact_key_t *key,
    amduat_bytes_t *out_bytes);
```

Notes:
- Transport fetch is not part of resolve; it only consumes ingested records.
- The daemon can choose to fetch missing bytes when resolve reports a remote
  location but local bytes are absent.
- If per-record filtering is enabled, it is applied during ingest or view build
  and any denials are recorded in view metadata (e.g., a table of
  amduat_fed_policy_deny_t).

## Replay and view construction

Rules are as per ASL/FEDERATION-REPLAY/1:
- Records are ordered by (logseq asc, canonical identity tie-break).
- Replay is bounded by {SnapshotID, LogPrefix} per domain.
- Tombstones shadow only within their source domain.
- Imported entries keep domain_id + cross_domain_source.

The view is the union of:
1) Local domain internal + published state at local {SnapshotID, LogPrefix}.
2) Admitted foreign published state at each domain's {SnapshotID, LogPrefix}.

## Cache and storage

Federation view storage MAY be:
- In-memory (ephemeral), or
- On-disk index segments with federation metadata populated.

If remote bytes are fetched, they MUST be written to a cache store that is
logically separate from the authoritative local store (policy-controlled).

## Policy gating

- Admission gating is per-domain.
- Per-record filtering is optional and MUST be an explicit, deterministic
  policy layer if enabled.

## Error reporting

Core resolve should distinguish:
- NOT_FOUND (no record in local or federation view)
- FOUND_REMOTE_NO_BYTES (record exists in view but bytes missing locally)
- INTEGRITY_ERROR (hash mismatch on bytes)
- POLICY_DENIED (domain admitted but record filtered by policy)

Notes:
- When per-record filtering is enabled, POLICY_DENIED SHOULD surface at ingest
  or view-build time by excluding filtered records from the view and recording
  the denial in view metadata. Resolve MAY return POLICY_DENIED only when such a
  denial is recorded for the queried key; otherwise it MUST return NOT_FOUND.

## Tests (minimal)

1) Replay ordering determinism across two domains with interleaved logseq.
2) Tombstone shadowing is domain-local.
3) Imported record metadata preserved in view and survives rebuild.
4) Conflict: same ArtifactKey with different bytes across domains rejected.
5) Bound replay by {SnapshotID, LogPrefix} produces stable view.