amduat-api/README.md

# amduat-api

`amduat-api` builds `amduatd`, a minimal HTTP server over a Unix domain socket that exposes Amduat substrate operations for a **single ASL store root**.

## Build

```sh
cmake -S . -B build
cmake --build build -j
```

To build without the embedded UI:

```sh
cmake -S . -B build -DAMDUATD_ENABLE_UI=OFF
```

When the UI is enabled (default), `/v1/ui` serves the same embedded HTML as before.

## Core dependency

This repo vendors the core implementation as a git submodule at `vendor/amduat`.

```sh
git submodule update --init --recursive
```

## Quickstart (local)

Initialize a store:

```sh
./vendor/amduat/build/amduat-asl init --root .amduat-asl
```

Run the daemon (fs backend default):

```sh
./build/amduatd --root .amduat-asl --sock amduatd.sock
```

Run the daemon with the index-backed store:

```sh
./build/amduatd --root .amduat-asl --sock amduatd.sock --store-backend index
```

Note: `/v1/fed/records` and `/v1/fed/push/plan` require the index backend.

## Federation (dev)

Federation is opt-in and disabled by default. Enabling federation requires the
index-backed store and explicit flags:

```sh
./build/amduatd --root .amduat-asl --sock amduatd.sock --store-backend index \
  --fed-enable --fed-transport unix --fed-unix-sock peer.sock \
  --fed-domain-id 1 --fed-registry-ref <registry_ref_hex>
```

Flags:

- `--fed-enable` turns on the coordinator tick loop.
- `--fed-transport stub|unix` selects transport (`stub` by default).
- `--fed-unix-sock PATH` configures the unix transport socket path.
- `--fed-domain-id ID` sets the local domain id.
- `--fed-registry-ref REF` seeds the registry reference (hex ref).
- `--fed-require-space` rejects `/v1/fed/*` requests that do not resolve a space.

`X-Amduat-Space` is honored for `/v1/fed/*` requests the same way as other
endpoints. If `--space` is configured, unix transport requests will include the
same `X-Amduat-Space` header when contacting peers.

### Federation cursors

Federation cursors track deterministic, auditable sync checkpoints per
`(space, peer)`. Cursor heads are stored as ASL pointers that reference CAS
records (`fed/cursor` schema). A peer key should be a stable identifier for the
remote (for example a federation registry domain id rendered as a decimal
string).

Push cursors are separate from pull cursors and live under
`fed/push_cursor/<peer>/head` (space scoped).

To avoid cursor collisions across multiple mounts to the same peer, pass
`remote_space_id=<space_id>` on cursor-aware endpoints. When provided, cursor
heads use `fed/cursor/<peer>/<remote_space_id>/head` and
`fed/push_cursor/<peer>/<remote_space_id>/head`. When omitted, the legacy v1
cursor names remain in effect for backward compatibility.

Read the current cursor for a peer:

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/fed/cursor?peer=domain-2' \
  -H 'X-Amduat-Space: demo'
```

Scoped to a specific remote space:

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/fed/cursor?peer=domain-2&remote_space_id=beta' \
  -H 'X-Amduat-Space: demo'
```

Write a cursor update (CAS-safe; include `expected_ref` to enforce; omitting it
only succeeds when the cursor is absent):

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/cursor?peer=domain-2' \
  -H 'Content-Type: application/json' \
  -H 'X-Amduat-Space: demo' \
  -d '{"last_logseq":123,"last_record_hash":"<ref>","expected_ref":"<ref>"}'
```

Cursor values are intended to drive incremental log/index scanning when that
infrastructure is available; the cursor endpoints themselves do not require the
index backend.

### Federation pull plan

You can ask the daemon to compute a read-only plan of which remote records would
be pulled from a peer given the current cursor:

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/fed/pull/plan?peer=2&limit=128' \
  -H 'X-Amduat-Space: demo'
```

The plan does not write artifacts, records, or cursors. It is deterministic and
returns only identifiers (logseq/ref), plus the next cursor candidate if the
plan were applied successfully.
Append `&remote_space_id=<space_id>` to use mount-specific cursor keying.

Apply a bounded batch of remote records (advances the cursor only after
success):

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/pull?peer=2&limit=128' \
  -H 'X-Amduat-Space: demo'
```

`/v1/fed/pull` requires the index backend and will not advance the cursor on
partial failure.
Use `remote_space_id=<space_id>` to scope the cursor to a mount.

### Federation push plan (sender dry run)

Compute a read-only plan of what would be sent to a peer from the local log
since the push cursor (does not advance the cursor):

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/fed/push/plan?peer=2&limit=128' \
  -H 'X-Amduat-Space: demo'
```

`/v1/fed/push/plan` requires the index backend and uses a push cursor separate
from the pull cursor.
Append `&remote_space_id=<space_id>` to use mount-specific cursor keying.

### Federation push (sender apply)

Send local records to a peer (advances the push cursor only after all records
apply successfully on the peer):

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/push?peer=2&limit=128' \
  -H 'X-Amduat-Space: demo'
```

`/v1/fed/push` uses `/v1/fed/ingest` on the peer and only advances the push
cursor after the batch completes. It requires the index backend.
Use `remote_space_id=<space_id>` to scope the cursor to a mount.

### Federation sync until caught up

Use the bounded helpers to repeatedly apply pull/push until no work remains,
with a hard `max_rounds` cap to keep requests bounded:

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/pull/until?peer=2&limit=128&max_rounds=10' \
  -H 'X-Amduat-Space: demo'
```

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/push/until?peer=2&limit=128&max_rounds=10' \
  -H 'X-Amduat-Space: demo'
```

### Federation ingest (receiver)

`/v1/fed/ingest` applies a single incoming record (push receiver). The request
is space-scoped via `X-Amduat-Space` and requires federation to be enabled;
otherwise the daemon responds with 503.

For artifact, per, and tgk_edge records, send raw bytes and provide metadata via
query params:

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/ingest?record_type=artifact&ref=<ref>' \
  -H 'Content-Type: application/octet-stream' \
  -H 'X-Amduat-Space: demo' \
  --data-binary 'payload'
```

For tombstones, send a small JSON payload:

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/fed/ingest' \
  -H 'Content-Type: application/json' \
  -H 'X-Amduat-Space: demo' \
  -d '{"record_type":"tombstone","ref":"<ref>"}'
```

Notes:

- Record types: `artifact`, `per`, `tgk_edge`, `tombstone`.
- Size limit: 8 MiB per request.
- Tombstones use deterministic defaults: `scope=0`, `reason_code=0`.

Run the daemon with derivation indexing enabled:

```sh
./build/amduatd --root .amduat-asl --sock amduatd.sock --enable-derivation-index
```

## Space roots (GC)

`/v1/space/roots` enumerates the pointer heads that must be treated as GC roots
for the effective space, including federation cursor heads.

GC root sets MUST include federation cursors to avoid trimming artifacts still
reachable via replication state. Use the roots listing to build your root set
before running GC tooling.

Example:

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/space/roots' \
  -H 'X-Amduat-Space: demo'
```

## Space sync status

`/v1/space/sync/status` is a read-only summary of federation readiness and
per-peer cursor positions for the effective space. Peers are discovered from
cursor head pointers (pull and push) and returned in deterministic order.

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/space/sync/status' \
  -H 'X-Amduat-Space: demo'
```

The response groups cursor status per peer and per remote space id:
`peers:[{peer_key, remotes:[{remote_space_id, pull_cursor, push_cursor}]}]`.
`remote_space_id` is `null` for legacy v1 cursor heads.

## Space manifest

`/v1/space/manifest` returns the space manifest rooted at the deterministic
pointer head (`manifest/head` or `space/<space_id>/manifest/head`). The manifest
is stored in CAS as a record and returned with its ref plus a decoded,
deterministic JSON payload. If no manifest head is present, the endpoint
returns a 404.

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/space/manifest' \
  -H 'X-Amduat-Space: demo'
```

Create the manifest (only if no head exists yet):

```sh
curl --unix-socket amduatd.sock \
  -X PUT 'http://localhost/v1/space/manifest' \
  -H 'Content-Type: application/json' \
  -H 'X-Amduat-Space: demo' \
  --data-binary '{"version":1,"mounts":[]}'
```

Update with optimistic concurrency:

```sh
curl --unix-socket amduatd.sock \
  -X PUT 'http://localhost/v1/space/manifest' \
  -H 'Content-Type: application/json' \
  -H 'X-Amduat-Space: demo' \
  -H 'If-Match: <manifest_ref>' \
  --data-binary @manifest.json
```

`If-Match` can be replaced with `?expected_ref=<manifest_ref>` if needed.

## Space mount resolution

`/v1/space/mounts/resolve` returns a deterministic, local-only view of the
space manifest mounts with their local pull cursor state. It performs no
network I/O and does not mutate storage. Track mounts indicate intent; syncing
remains a separate concern.
If no manifest head is present, the endpoint returns a 404.
Track mounts report `local_tracking.cursor_namespace` (`v2` when using
`remote_space_id`-keyed cursors).

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/space/mounts/resolve' \
  -H 'X-Amduat-Space: demo'
```

## Space workspace snapshot

`/v1/space/workspace` returns a deterministic, read-only snapshot for the
effective space. It aggregates the manifest, mount resolution, per-mount cursor
status, store backend metadata, federation flags, and store capabilities into
one JSON response. It performs no network I/O and does not mutate storage.

This is a local snapshot that complements:
- `/v1/space/manifest` (manifest root + canonical manifest)
- `/v1/space/mounts/resolve` (resolved mounts + local tracking)
- `/v1/space/sync/status` (peer-wide cursor status)
- `/v1/space/mounts/sync/until` (active sync for track mounts)

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/space/workspace' \
  -H 'X-Amduat-Space: demo'
```

## Workspace UI

`/workspace` serves a minimal, human-facing page that consumes
`/v1/space/workspace` and `/v1/space/mounts/sync/until`. It is a convenience
view for inspection and manual sync control, not a stable API. For
programmatic use, call the `/v1/*` endpoints directly.

## Space mounts sync (track mounts)

`/v1/space/mounts/sync/until` runs the federation pull/until loop for every
`track` mount in the current manifest using v2 cursor keying
(`remote_space_id = mount.space_id`). It is bounded by `limit`, `max_rounds`,
and `max_mounts`, returns per-mount status, and continues after errors.
Requires federation enabled and the index backend. If no manifest head is
present, the endpoint returns a 404.

```sh
curl --unix-socket amduatd.sock -X POST \
  'http://localhost/v1/space/mounts/sync/until?limit=128&max_rounds=10&max_mounts=32' \
  -H 'X-Amduat-Space: demo'
```

To fail `/v1/pel/run` if the derivation index write fails:

```sh
./build/amduatd --root .amduat-asl --sock amduatd.sock --derivation-index-strict
```

Dev loop (build + restart):

```sh
./scripts/dev-restart.sh
```

## Federation smoke test

Run the end-to-end federation smoke test (starts two local daemons, verifies
pull replication A→B and push replication B→A, and checks cursors):

```sh
./scripts/test_fed_smoke.sh
```

The test requires the index backend and either `curl` with `--unix-socket`
support or the built-in `build/amduatd_http_unix` helper.

Query store meta:

```sh
curl --unix-socket amduatd.sock http://localhost/v1/meta
```

Browser UI (use `socat` to forward the Unix socket):

```sh
socat TCP-LISTEN:8080,fork,reuseaddr UNIX-CONNECT:amduatd.sock
```

Then open `http://localhost:8080/v1/ui` (concept editor).

Discover the store-backed API contract ref:

```sh
curl --unix-socket amduatd.sock 'http://localhost/v1/contract?format=ref'
```

Fetch the contract bytes (JSON):

```sh
curl --unix-socket amduatd.sock http://localhost/v1/contract
```

Upload raw bytes:

```sh
curl --unix-socket amduatd.sock -X POST http://localhost/v1/artifacts \
  -H 'Content-Type: application/octet-stream' \
  --data-binary 'hello'
```

Download raw bytes:

```sh
curl --unix-socket amduatd.sock http://localhost/v1/artifacts/<ref>
```

Download artifact framing (`ENC/ASL1-CORE v1`):

```sh
curl --unix-socket amduatd.sock \
  'http://localhost/v1/artifacts/<ref>?format=artifact' --output artifact.bin
```

Run a PEL program from store-backed refs (default `scheme_ref=dag`):

```sh
curl --unix-socket amduatd.sock -X POST http://localhost/v1/pel/run \
  -H 'Content-Type: application/json' \
  -d '{"program_ref":"<program_ref>","input_refs":["<input_ref_0>"],"params_ref":"<params_ref>"}'
```

When derivation indexing is enabled, successful PEL runs record derivations under
`<root>/index/derivations/by_artifact/` keyed by output refs (plus result/trace/receipt refs).

Define a PEL/PROGRAM-DAG/1 program (store-backed):

```sh
curl --unix-socket amduatd.sock -X POST http://localhost/v1/pel/programs \
  -H 'Content-Type: application/json' \
  -d '{"nodes":[{"id":1,"op":{"name":"pel.bytes.concat","version":1},"inputs":[{"external":{"input_index":0}},{"external":{"input_index":1}}],"params_hex":""}],"roots":[{"node_id":1,"output_index":0}]}'
```

Create a named concept and publish a ref (so you can use the name instead of hex refs):

```sh
curl --unix-socket amduatd.sock -X POST http://localhost/v1/concepts \
  -H 'Content-Type: application/json' \
  -d '{"name":"hello","ref":"<program_ref>"}'
```

Publish a new version:

```sh
curl --unix-socket amduatd.sock -X POST http://localhost/v1/concepts/hello/publish \
  -H 'Content-Type: application/json' \
  -d '{"ref":"<program_ref_v2>"}'
```

Resolve the latest ref:

```sh
curl --unix-socket amduatd.sock http://localhost/v1/resolve/hello
```

Inspect concepts:

```sh
curl --unix-socket amduatd.sock http://localhost/v1/concepts
curl --unix-socket amduatd.sock http://localhost/v1/concepts/hello
```

Artifact info (length + type tag):

```sh
curl --unix-socket amduatd.sock 'http://localhost/v1/artifacts/<ref>?format=info'
```

## Space selection

Requests can select a space via the `X-Amduat-Space` header:

```sh
curl --unix-socket amduatd.sock http://localhost/v1/concepts \
  -H 'X-Amduat-Space: demo'
```

Precedence rules:

- `X-Amduat-Space` header (if present)
- daemon `--space` default (if configured)
- unscoped names (no space)

When capability tokens are used, the requested space must match the token's
space (or the token must be unscoped), otherwise the request is rejected.

## Space doctor

Check deterministic invariants for the effective space:

```sh
curl --unix-socket amduatd.sock http://localhost/v1/space/doctor
curl --unix-socket amduatd.sock http://localhost/v1/space/doctor \
  -H 'X-Amduat-Space: demo'
```

When the daemon uses the `fs` store backend, index-only checks are reported as
`"skipped"`; the `index` backend runs them.

## Current endpoints

- `GET /v1/meta` → `{store_id, encoding_profile_id, hash_id, api_contract_ref}`
- `GET /v1/contract` → contract bytes (JSON) (+ `X-Amduat-Contract-Ref` header)
- `GET /v1/contract?format=ref` → `{ref}`
- `GET /v1/space/doctor` → deterministic space health checks
- `GET /v1/space/manifest` → `{effective_space, manifest_ref, manifest}`
- `PUT /v1/space/manifest` → `{effective_space, manifest_ref, updated, previous_ref?, manifest}`
- `GET /v1/space/mounts/resolve` → `{effective_space, manifest_ref, mounts}`
- `GET /v1/space/workspace` → `{effective_space, store_backend, federation, capabilities, manifest_ref, manifest, mounts}`
- `POST /v1/space/mounts/sync/until?limit=...&max_rounds=...&max_mounts=...` → `{effective_space, manifest_ref, limit, max_rounds, max_mounts, mounts_total, mounts_synced, ok, results}`
- `GET /v1/space/sync/status` → `{effective_space, store_backend, federation, peers}`
- `GET /v1/ui` → browser UI for authoring/running programs
- `GET /v1/fed/records?domain_id=...&from_logseq=...&limit=...` → `{domain_id, snapshot_id, log_prefix, next_logseq, records[]}` (published artifacts + tombstones + PER + TGK edges)
- `GET /v1/fed/cursor?peer=...&remote_space_id=...` → `{peer_key, space_id, last_logseq, last_record_hash, ref}` (`remote_space_id` optional)
- `POST /v1/fed/cursor?peer=...&remote_space_id=...` → `{ref}` (CAS update; `expected_ref` in body; `remote_space_id` optional)
- `GET /v1/fed/pull/plan?peer=...&limit=...&remote_space_id=...` → `{peer, effective_space, cursor, remote_scan, records, next_cursor_candidate, ...}`
- `GET /v1/fed/push/plan?peer=...&limit=...&remote_space_id=...` → `{peer, domain_id, effective_space, cursor, scan, records, required_artifacts, next_cursor_candidate}`
- `POST /v1/fed/pull?peer=...&limit=...&remote_space_id=...` → `{peer, effective_space, cursor_before, plan_summary, applied, cursor_after, errors}`
- `POST /v1/fed/push?peer=...&limit=...&remote_space_id=...` → `{peer, domain_id, effective_space, cursor_before, plan_summary, sent, cursor_after, errors}`
- `GET /v1/fed/artifacts/{ref}` → raw bytes for federation resolve
- `GET /v1/fed/status` → `{status, domain_id, registry_ref, last_tick_ms}`
- `POST /v1/artifacts`
  - raw bytes: `Content-Type: application/octet-stream` (+ optional `X-Amduat-Type-Tag: 0x...`)
  - artifact framing: `Content-Type: application/vnd.amduat.asl.artifact+v1`
- `GET|HEAD /v1/artifacts/{ref}`
  - raw bytes default
  - artifact framing: `?format=artifact`
- `POST /v1/concepts`
  - request: `{name, ref?}` (`name` is lowercase; `ref` publishes an initial version)
  - response: `{name, concept_ref}`
- `POST /v1/concepts/{name}/publish` → publishes a new version `{ref}`
- `GET /v1/concepts` → `{concepts:[{name, concept_ref}]}`
- `GET /v1/concepts/{name}` → `{name, concept_ref, latest_ref, versions[]}`
- `GET /v1/resolve/{name}` → `{ref}` (latest published)
- `POST /v1/pel/run`
  - request: `{program_ref, input_refs[], params_ref?, scheme_ref?}` (`program_ref`/`input_refs`/`params_ref` accept hex refs or concept names; omit `scheme_ref` to use `dag`)
  - request receipt (optional): `{receipt:{input_manifest_ref, environment_ref, evaluator_id, executor_ref, started_at, completed_at, sbom_ref?, parity_digest_hex?, executor_fingerprint_ref?, run_id_hex?, limits?, logs?, determinism_level?, rng_seed_hex?, signature_hex?}}`
  - response: `{result_ref, trace_ref?, receipt_ref?, output_refs[], status}`
- `POST /v1/pel/programs`
  - request: authoring JSON for `PEL/PROGRAM-DAG/1` (kernel ops only; `params_hex` is raw hex bytes)
  - response: `{program_ref}`
- `POST /v1/context_frames`

UI (human-facing, not an API contract):
- `GET /workspace` → minimal workspace snapshot + sync controls (uses `/v1/space/workspace`)

Receipt example (with v1.1 fields):

```json
{
  "program_ref": "ab12...",
  "input_refs": ["cd34..."],
  "receipt": {
    "input_manifest_ref": "ef56...",
    "environment_ref": "7890...",
    "evaluator_id": "local-amduatd",
    "executor_ref": "1122...",
    "started_at": 1712345678,
    "completed_at": 1712345688,
    "executor_fingerprint_ref": "3344...",
    "run_id_hex": "deadbeef",
    "limits": {
      "cpu_ms": 12,
      "wall_ms": 20,
      "max_rss_kib": 1024,
      "io_reads": 1,
      "io_writes": 0
    },
    "logs": [
      {"kind": 1, "log_ref": "5566...", "sha256_hex": "aabbcc"}
    ],
    "determinism_level": 2,
    "rng_seed_hex": "010203",
    "signature_hex": "bead"
  }
}
```

Federation records example:

```bash
curl --unix-socket amduatd.sock \
  'http://localhost/v1/fed/records?domain_id=1&from_logseq=0&limit=256'
```

```json
{
  "domain_id": 1,
  "snapshot_id": 42,
  "log_prefix": 1234,
  "next_logseq": 120,
  "records": [
    {
      "domain_id": 1,
      "type": 0,
      "ref": "ab12...",
      "logseq": 100,
      "snapshot_id": 42,
      "log_prefix": 1234,
      "visibility": 1,
      "has_source": false,
      "source_domain": 0
    }
  ]
}
```

Response example:

```json
{
  "result_ref": "aa11...",
  "trace_ref": "bb22...",
  "receipt_ref": "cc33...",
  "output_refs": ["dd44..."],
  "status": "OK"
}
```

## Notes

- This is intentionally a local-first surface (Unix socket, no TLS). HTTPS can be added later without changing the semantics.