Compare commits

..

57 commits

Author SHA1 Message Date
Carl Niklas Rydberg a433f92f13 Fixed but not 2026-02-08 09:54:37 +01:00
Carl Niklas Rydberg 88d9717513 Stop snapshot-anchor warning flood during state checks 2026-02-08 09:01:13 +01:00
Carl Niklas Rydberg 624bd29bf9 Recover index state and stale log heads after partial repairs 2026-02-08 08:55:39 +01:00
Carl Niklas Rydberg 6ab25361be Keep log verify retries portable across C feature sets 2026-02-08 08:47:25 +01:00
Carl Niklas Rydberg 8b2979e11d Use nanosleep for log verify retry delay 2026-02-08 08:46:34 +01:00
Carl Niklas Rydberg d0e53170f3 Gate log head CAS on readable chunk verification 2026-02-08 08:46:09 +01:00
Carl Niklas Rydberg 327812ca96 Harden pointer head writes against ENOENT races 2026-02-08 08:46:06 +01:00
Carl Niklas Rydberg 03d970c576 Fixed lock file error 2026-02-08 06:08:25 +01:00
Carl Niklas Rydberg c7a9e2f6aa Fix index-fs write locking and add index backend regressions 2026-02-08 00:06:42 +01:00
Carl Niklas Rydberg 9a2903072b Fix log encode size calc 2026-01-23 22:28:45 +01:00
Carl Niklas Rydberg 337466b073 amduat-pel: add edges dump command 2026-01-23 20:58:09 +01:00
Carl Niklas Rydberg d9122b53bb Add PEL collection view projection 2026-01-23 20:18:23 +01:00
Carl Niklas Rydberg 3e526975ce Fix build linkages for pel tools 2026-01-23 19:34:53 +01:00
Carl Niklas Rydberg 85c23e49eb Add generic records and collections 2026-01-23 19:15:16 +01:00
Carl Niklas Rydberg e2d26e53cd Add CAS-native append-only log 2026-01-23 19:04:49 +01:00
Carl Niklas Rydberg c4571c3bfb Add SID cache and read-first derivation 2026-01-23 18:31:21 +01:00
Carl Niklas Rydberg 4932fbd21c federation? 2026-01-21 19:54:32 +01:00
Carl Niklas Rydberg 0f38165804 Document federation API and deprecate replay_domain 2026-01-18 12:10:12 +01:00
Carl Niklas Rydberg 282d1eb959 Document ownership and add replay build alias 2026-01-18 12:07:43 +01:00
Carl Niklas Rydberg d8a6603ddc Improve federation API docs and errors 2026-01-18 12:03:18 +01:00
Carl Niklas Rydberg 5cc56b2ce8 Add federation integration example 2026-01-18 11:54:36 +01:00
Carl Niklas Rydberg ba78946a73 Add federation implementation references 2026-01-18 11:52:39 +01:00
Carl Niklas Rydberg 94c6c4be26 Update federation implementation notes 2026-01-18 11:44:30 +01:00
Carl Niklas Rydberg ed64c6ed89 Remove federation location metadata 2026-01-18 11:43:03 +01:00
Carl Niklas Rydberg 2931e35c69 Add federation locations and conflict checks 2026-01-18 11:25:39 +01:00
Carl Niklas Rydberg 489738c5ba Add federation view cache and resolve 2026-01-18 11:17:35 +01:00
Carl Niklas Rydberg 5a0a2f80c7 Add federation registry, replay, and ingest 2026-01-18 10:56:52 +01:00
Carl Niklas Rydberg f707244888 Add federation docs and implementation notes 2026-01-18 09:48:30 +01:00
Carl Niklas Rydberg 948a156f5c Add PEL program builder and derivation indexing 2026-01-18 09:21:25 +01:00
Carl Niklas Rydberg 8c5e593cec Added tmp and store to .gitignore 2026-01-18 09:20:41 +01:00
Carl Niklas Rydberg 7b9c55ea6d Adjust asl_store_index_fs timing 2026-01-18 07:41:03 +01:00
Carl Niklas Rydberg 3198d8ddbe Fix derivation index fs test 2026-01-18 07:36:59 +01:00
Carl Niklas Rydberg 1d04c32401 Add derivation index CLI 2026-01-18 07:27:48 +01:00
Carl Niklas Rydberg 8c5fa71388 Refine ASL indexes taxonomy 2026-01-18 06:55:00 +01:00
Carl Niklas Rydberg 7a3dcc3978 Add audit entries for index/log accel 2026-01-18 06:28:25 +01:00
Carl Niklas Rydberg 4f59bc7e79 Update audit coverage status 2026-01-18 06:24:09 +01:00
Carl Niklas Rydberg 745cf89eb7 Add ASL index accel routing key 2026-01-18 06:20:48 +01:00
Carl Niklas Rydberg 7878cd3702 Add TGK exec plan encoding 2026-01-18 06:13:07 +01:00
Carl Niklas Rydberg dde532d68f Improve ASL store index lifecycle 2026-01-18 05:54:31 +01:00
Carl Niklas Rydberg 0a118b9841 Tighten ASL index/log conformance checks 2026-01-18 05:04:55 +01:00
Carl Niklas Rydberg 4d2fb250cd Document FER/1 v1.1 TLVs and test helpers 2026-01-17 21:38:04 +01:00
Carl Niklas Rydberg b506cc6c7c Extend FER/1 receipts and TGK store support 2026-01-17 21:34:24 +01:00
Carl Niklas Rydberg 83cbe28ede asl_store_index_fs: add perf/stress tests and summaries 2026-01-17 19:49:12 +01:00
Carl Niklas Rydberg 017bc87e04 asl_store_index_fs: stream log snapshot/truncate 2026-01-17 17:58:59 +01:00
Carl Niklas Rydberg 0667cee17d Add bloom filters to ASL index segments 2026-01-17 16:55:46 +01:00
Carl Niklas Rydberg 06a96f25db Add snapshot manifests and auto snapshot policy 2026-01-17 16:43:47 +01:00
Carl Niklas Rydberg 556c65a54e Add ASL index/log inspection commands 2026-01-17 14:26:17 +01:00
Carl Niklas Rydberg ac1ce381a2 Stop tracking CTest log 2026-01-17 14:11:40 +01:00
Carl Niklas Rydberg 07ef551f56 Ignore CTest output 2026-01-17 14:10:24 +01:00
Carl Niklas Rydberg 347d2e4dda Add filesystem-backed ASL index store 2026-01-17 14:08:41 +01:00
Carl Niklas Rydberg 68371b1554 Add ASL core index codec and tests 2026-01-17 13:10:07 +01:00
Carl Niklas Rydberg b7b4b2f195 Add ASL log encoding/decoding 2026-01-17 12:51:32 +01:00
Carl Niklas Rydberg 0d810affb0 Add ASL index state API stubs 2026-01-17 12:45:13 +01:00
Carl Niklas Rydberg a91ab91e39 Add indexed ASL store ops and tests 2026-01-17 12:44:33 +01:00
Carl Niklas Rydberg 20f092606d Polish ASL index/log specs 2026-01-17 12:21:15 +01:00
Carl Niklas Rydberg c595e2370a Clarify ASL index/log semantics 2026-01-17 11:46:57 +01:00
Carl Niklas Rydberg 3886716799 Add core tier1 specs for ASL/TGK 2026-01-17 11:18:00 +01:00
112 changed files with 38434 additions and 51 deletions

4
.gitignore vendored
View file

@ -3,6 +3,8 @@ CMakeFiles/
CMakeCache.txt CMakeCache.txt
cmake_install.cmake cmake_install.cmake
Makefile Makefile
CTestTestfile.cmake
Testing/
*.o *.o
*.a *.a
*.so *.so
@ -12,3 +14,5 @@ Makefile
*.exe *.exe
*.pdb *.pdb
.DS_Store .DS_Store
tmp/
.amduat-asl

View file

@ -22,6 +22,62 @@ Verification notes:
- Prefer explicit commands and paths (e.g., `ctest --test-dir build`). - Prefer explicit commands and paths (e.g., `ctest --test-dir build`).
- If results are user-reported, note that explicitly. - If results are user-reported, note that explicitly.
Note: the filesystem ASL store (`asl_store_fs`) is a legacy convenience backend
and will be considered non-conformant to ASL index/log specs once the index/log
store is introduced. Audits for ASL index/log specs target the new backend only.
## Test Expectations (Planned)
These tests are planned to validate index/log behavior once implemented:
| Area | Example tests |
| --- | --- |
| Segment encoding | Round-trip encode/decode; CRC mismatch rejection; offset bounds checks |
| Log encoding | Hash-chain validation; unknown record type skip; truncated record rejection |
| Replay | Snapshot anchor + log replay determinism; segment seal visibility |
| Tombstones | Shadowing and lift across snapshots; domain-local shadowing rules |
| Visibility | CURRENT computed by `(SnapshotID, LogPosition)`; reverse seal-log order |
| Recovery | Crash with open segment; replay yields deterministic CURRENT |
## Spec Coverage (Implementation Status)
Status legend: ✅ implemented, 🟡 planned/in-progress, ⬜ not started.
| Spec | Status | Notes |
| --- | --- | --- |
| `ASL/1-CORE` | ✅ | Core artifact semantics implemented. |
| `ASL/1-STORE` | ✅ | Store semantics + fs backend. |
| `ENC/ASL1-CORE` | ✅ | Artifact/Reference encoding. |
| `HASH/ASL1` | ✅ | Hash registry + streaming API. |
| `PEL/1-CORE` | ✅ | Core execution semantics. |
| `PEL/1-SURF` | ✅ | Store-backed surface execution. |
| `PEL/PROGRAM-DAG/1` | ✅ | DAG scheme execution. |
| `PEL/PROGRAM-DAG-DESC/1` | ✅ | Scheme descriptor codec + wiring. |
| `ENC/PEL-PROGRAM-DAG/1` | ✅ | Program encoding. |
| `ENC/PEL1-RESULT/1` | ✅ | Result encoding. |
| `PEL/TRACE-DAG/1` | ✅ | Trace semantics + wiring. |
| `ENC/PEL-TRACE-DAG/1` | ✅ | Trace encoding. |
| `TGK/1-CORE` | ✅ | Edge semantics + validation. |
| `ENC/TGK1-EDGE/1` | ✅ | Edge encoding. |
| `TGK/STORE/1` | ✅ | Store semantics. |
| `TGK/PROV/1` | ✅ | Provenance operators. |
| `OPREG/PEL1-KERNEL` | ✅ | Kernel op registry. |
| `OPREG/PEL1-KERNEL-PARAMS/1` | ✅ | Kernel params encoding. |
| `AMDUAT20-STACK-OVERVIEW` | ✅ | Orientation surface aligned. |
| `ASL/1-CORE-INDEX` | ✅ | Index semantics + replay implemented. |
| `ASL/STORE-INDEX/1` | ✅ | Index/log store backend implemented (fs). |
| `ENC/ASL-CORE-INDEX/1` | ✅ | Segment encoding/decoding implemented. |
| `ASL/LOG/1` | ✅ | Log semantics implemented. |
| `ENC/ASL-LOG/1` | ✅ | Log encoding/decoding implemented. |
| `ASL/INDEX-ACCEL/1` | ✅ | Routing key + bloom/shard helpers implemented. |
| `ASL/INDEXES/1` | 🟡 | Taxonomy planned. |
| `ASL/TGK-EXEC-PLAN/1` | 🟡 | Encoding implemented; executor out of scope. |
| `ENC/ASL-TGK-EXEC-PLAN/1` | ✅ | Plan encoding implemented. |
| `ASL/FEDERATION/1` | ✅ | Core federation primitives implemented. |
| `ASL/FEDERATION-REPLAY/1` | ✅ | Deterministic replay and view construction implemented. |
| `ASL/SYSTEM/1` | 🟡 | Cross-cutting view planned. |
| `TGK/1` | 🟡 | Semantic layer planned. |
## Audit Plan ## Audit Plan
Status legend: ✅ completed, ⬜ pending. Status legend: ✅ completed, ⬜ pending.
@ -108,6 +164,24 @@ Status legend: ✅ completed, ⬜ pending.
- Tests: command not provided — pass (user reported “100% tests passed, 0 tests - Tests: command not provided — pass (user reported “100% tests passed, 0 tests
failed out of 11”). failed out of 11”).
## 2026-01-18 — ASL index/log stack (`tier1/asl-core-index-1.md`, `tier1/asl-store-index-1.md`, `tier1/enc-asl-core-index-1.md`, `tier1/asl-log-1.md`, `tier1/enc-asl-log-1.md`)
- Scope: index semantics, filesystem index/log store, segment encoding, and log encoding/decoding paths.
- Findings: N/A (implemented components already present).
- Resolution: recorded implementation status and aligned routing/shard helpers with index usage.
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
## 2026-01-18 — ASL/INDEX-ACCEL/1 (`tier1/asl-index-accel-1.md`)
- Scope: routing key derivation, shard selection contract, bloom advisory behavior.
- Findings: missing formal routing-key API and tests around acceleration helpers.
- Resolution: added routing-key/shard helpers and tests for routing-key layout, shard determinism, and bloom advisory behavior.
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
## 2026-01-18 — ENC/ASL-TGK-EXEC-PLAN/1 (`tier1/enc-asl-tgk-exec-plan-1.md`)
- Scope: execution plan encoding/decoding; validation of operator IDs/inputs.
- Findings: encoding layer missing.
- Resolution: implemented encode/decode/free API and round-trip validation tests.
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
## 2025-12-22 — PEL/PROGRAM-DAG/1 (`tier1/pel-program-dag-1.md`) ## 2025-12-22 — PEL/PROGRAM-DAG/1 (`tier1/pel-program-dag-1.md`)
- Scope: Exec_DAG semantics, structural validity, canonical order, diagnostics, - Scope: Exec_DAG semantics, structural validity, canonical order, diagnostics,
and scheme entrypoint correctness. and scheme entrypoint correctness.
@ -276,3 +350,15 @@ Status legend: ✅ completed, ⬜ pending.
embedded commit-message appendix; tightened wording throughout; bumped the embedded commit-message appendix; tightened wording throughout; bumped the
document version/date. document version/date.
- Tests: N/A (documentation-only change). - Tests: N/A (documentation-only change).
## 2026-02-XX — Federation core (`tier1/asl-federation-1.md`, `tier1/asl-federation-replay-1.md`)
- Scope: core federation registry, ingest validation, deterministic replay, view
construction, and resolve semantics.
- Findings: missing record typing and identity coverage for PER/TGK/tombstones;
policy gating needed explicit per-domain + optional per-record handling; view
build and resolve error reporting needed explicit codes and tests.
- Resolution: added federation registry storage, ingest validation, replay/view
build, resolve APIs, and tests for ordering, tombstone scoping, conflicts,
bounds, and metadata preservation; documented middle-layer boundary and
ref-only remote fetch guidance.
- Tests: user reported “100% tests passed, 0 tests failed out of 29”.

View file

@ -57,12 +57,19 @@ set(AMDUAT_UTIL_SRCS
src/internal/varint.c src/internal/varint.c
src/internal/endian.c src/internal/endian.c
src/internal/hex.c src/internal/hex.c
src/internal/log.c
) )
set(AMDUAT_ASL_SRCS set(AMDUAT_ASL_SRCS
src/kernel/asl/core.c src/kernel/asl/core.c
src/near_core/asl/collection_view.c
src/near_core/asl/none.c
src/near_core/asl/artifact_io.c src/near_core/asl/artifact_io.c
src/near_core/asl/io.c src/near_core/asl/io.c
src/near_core/asl/index_accel.c
src/near_core/asl/index_bloom.c
src/near_core/asl/index_snapshot.c
src/near_core/asl/index_replay.c
src/near_core/asl/parse.c src/near_core/asl/parse.c
src/near_core/asl/ref_io.c src/near_core/asl/ref_io.c
src/near_core/asl/store.c src/near_core/asl/store.c
@ -70,6 +77,18 @@ set(AMDUAT_ASL_SRCS
src/near_core/asl/registry.c src/near_core/asl/registry.c
) )
set(AMDUAT_ASL_LOG_STORE_SRCS
src/core/asl_log_store.c
)
set(AMDUAT_ASL_RECORD_SRCS
src/core/asl_record.c
)
set(AMDUAT_ASL_COLLECTION_SRCS
src/core/asl_collection.c
)
set(AMDUAT_HASH_ASL1_SRCS set(AMDUAT_HASH_ASL1_SRCS
src/near_core/hash/asl1.c src/near_core/hash/asl1.c
src/near_core/hash/sha256.c src/near_core/hash/sha256.c
@ -81,6 +100,9 @@ set(AMDUAT_ENC_SRCS
src/near_core/asl/ref_derive.c src/near_core/asl/ref_derive.c
src/near_core/enc/fer1_receipt.c src/near_core/enc/fer1_receipt.c
src/near_core/fer/receipt.c src/near_core/fer/receipt.c
src/near_core/enc/asl_log.c
src/near_core/enc/asl_core_index.c
src/near_core/enc/asl_tgk_exec_plan.c
src/near_core/enc/pel_program_dag.c src/near_core/enc/pel_program_dag.c
src/near_core/enc/pel_program_dag_desc.c src/near_core/enc/pel_program_dag_desc.c
src/near_core/enc/pel_trace_dag.c src/near_core/enc/pel_trace_dag.c
@ -98,6 +120,7 @@ set(AMDUAT_FORMAT_SRCS
set(AMDUAT_PEL_SRCS set(AMDUAT_PEL_SRCS
src/kernel/pel/core.c src/kernel/pel/core.c
src/core/derivation_sid.c
src/pel_stack/decode.c src/pel_stack/decode.c
src/pel_stack/surf/surf.c src/pel_stack/surf/surf.c
src/pel_stack/program_dag/program_dag.c src/pel_stack/program_dag/program_dag.c
@ -106,6 +129,7 @@ set(AMDUAT_PEL_SRCS
src/pel_stack/trace_dag/trace_dag.c src/pel_stack/trace_dag/trace_dag.c
src/pel_stack/queue/queue.c src/pel_stack/queue/queue.c
src/pel_stack/opreg/kernel.c src/pel_stack/opreg/kernel.c
src/pel_stack/opreg/kernel_collection.c
src/pel_stack/opreg/kernel_params.c src/pel_stack/opreg/kernel_params.c
) )
@ -115,12 +139,36 @@ set(AMDUAT_TGK_SRCS
src/tgk_stack/prov/prov.c src/tgk_stack/prov/prov.c
) )
set(AMDUAT_FED_SRCS
src/near_core/fed/registry.c
src/near_core/fed/replay.c
src/near_core/fed/ingest.c
src/near_core/fed/view.c
)
set(AMDUAT_ASL_STORE_FS_SRCS set(AMDUAT_ASL_STORE_FS_SRCS
src/adapters/asl_store_fs/asl_store_fs.c src/adapters/asl_store_fs/asl_store_fs.c
src/adapters/asl_store_fs/asl_store_fs_layout.c src/adapters/asl_store_fs/asl_store_fs_layout.c
src/adapters/asl_store_fs/asl_store_fs_meta.c src/adapters/asl_store_fs/asl_store_fs_meta.c
) )
set(AMDUAT_ASL_STORE_INDEX_FS_SRCS
src/adapters/asl_store_index_fs/asl_store_index_fs.c
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.c
)
set(AMDUAT_ASL_DERIVATION_INDEX_FS_SRCS
src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c
)
set(AMDUAT_ASL_MATERIALIZATION_CACHE_FS_SRCS
src/adapters/asl_materialization_cache_fs/asl_materialization_cache_fs.c
)
set(AMDUAT_ASL_POINTER_FS_SRCS
src/adapters/asl_pointer_fs/asl_pointer_fs.c
)
set(AMDUAT_TGK_STORE_MEM_SRCS set(AMDUAT_TGK_STORE_MEM_SRCS
src/adapters/tgk_store_mem/tgk_store_mem.c src/adapters/tgk_store_mem/tgk_store_mem.c
) )
@ -129,6 +177,10 @@ set(AMDUAT_TGK_STORE_FS_SRCS
src/adapters/tgk_store_fs/tgk_store_fs.c src/adapters/tgk_store_fs/tgk_store_fs.c
) )
set(AMDUAT_TGK_STORE_ASL_INDEX_FS_SRCS
src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c
)
amduat_add_lib(util SRCS ${AMDUAT_UTIL_SRCS}) amduat_add_lib(util SRCS ${AMDUAT_UTIL_SRCS})
amduat_add_lib(asl SRCS ${AMDUAT_ASL_SRCS}) amduat_add_lib(asl SRCS ${AMDUAT_ASL_SRCS})
@ -144,28 +196,64 @@ amduat_add_lib(format SRCS ${AMDUAT_FORMAT_SRCS})
amduat_link(format amduat_asl amduat_enc amduat_util) amduat_link(format amduat_asl amduat_enc amduat_util)
amduat_add_lib(pel SRCS ${AMDUAT_PEL_SRCS}) amduat_add_lib(pel SRCS ${AMDUAT_PEL_SRCS})
amduat_link(pel amduat_asl amduat_enc amduat_hash_asl1 amduat_util) amduat_link(pel amduat_asl_materialization_cache_fs amduat_asl_collection
amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(tgk SRCS ${AMDUAT_TGK_SRCS}) amduat_add_lib(tgk SRCS ${AMDUAT_TGK_SRCS})
amduat_link(tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util) amduat_link(tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(fed SRCS ${AMDUAT_FED_SRCS})
amduat_link(fed amduat_asl)
amduat_add_lib(asl_store_fs SRCS ${AMDUAT_ASL_STORE_FS_SRCS}) amduat_add_lib(asl_store_fs SRCS ${AMDUAT_ASL_STORE_FS_SRCS})
amduat_link(asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util) amduat_link(asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_store_fs_obj PRIVATE _POSIX_C_SOURCE=200809L) target_compile_definitions(amduat_asl_store_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_store_index_fs SRCS ${AMDUAT_ASL_STORE_INDEX_FS_SRCS})
amduat_link(asl_store_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_store_index_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_derivation_index_fs SRCS ${AMDUAT_ASL_DERIVATION_INDEX_FS_SRCS})
amduat_link(asl_derivation_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_derivation_index_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_materialization_cache_fs SRCS ${AMDUAT_ASL_MATERIALIZATION_CACHE_FS_SRCS})
amduat_link(asl_materialization_cache_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_materialization_cache_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_pointer_fs SRCS ${AMDUAT_ASL_POINTER_FS_SRCS})
amduat_link(asl_pointer_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_pointer_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_log_store SRCS ${AMDUAT_ASL_LOG_STORE_SRCS})
amduat_link(asl_log_store amduat_asl_pointer_fs amduat_asl amduat_enc amduat_util)
amduat_add_lib(asl_record SRCS ${AMDUAT_ASL_RECORD_SRCS})
amduat_link(asl_record amduat_asl amduat_enc amduat_util)
amduat_add_lib(asl_collection SRCS ${AMDUAT_ASL_COLLECTION_SRCS})
amduat_link(asl_collection amduat_asl_log_store amduat_asl_record amduat_asl_pointer_fs
amduat_asl amduat_enc amduat_util)
amduat_add_lib(tgk_store_mem SRCS ${AMDUAT_TGK_STORE_MEM_SRCS}) amduat_add_lib(tgk_store_mem SRCS ${AMDUAT_TGK_STORE_MEM_SRCS})
amduat_link(tgk_store_mem amduat_tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util) amduat_link(tgk_store_mem amduat_tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(tgk_store_fs SRCS ${AMDUAT_TGK_STORE_FS_SRCS}) amduat_add_lib(tgk_store_fs SRCS ${AMDUAT_TGK_STORE_FS_SRCS})
amduat_link(tgk_store_fs amduat_tgk_store_mem amduat_tgk amduat_asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util) amduat_link(tgk_store_fs amduat_tgk_store_mem amduat_tgk amduat_asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(tgk_store_asl_index_fs SRCS ${AMDUAT_TGK_STORE_ASL_INDEX_FS_SRCS})
target_include_directories(amduat_tgk_store_asl_index_fs_obj
PRIVATE ${AMDUAT_ROOT}/src/adapters/asl_store_index_fs
)
amduat_link(tgk_store_asl_index_fs amduat_tgk amduat_asl_store_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
add_executable(amduat_asl_cli src/tools/amduat_asl_cli.c) add_executable(amduat_asl_cli src/tools/amduat_asl_cli.c)
target_include_directories(amduat_asl_cli target_include_directories(amduat_asl_cli
PRIVATE ${AMDUAT_INTERNAL_DIR} PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR} PRIVATE ${AMDUAT_INCLUDE_DIR}
) )
target_link_libraries(amduat_asl_cli target_link_libraries(amduat_asl_cli
PRIVATE amduat_format amduat_asl_store_fs amduat_asl amduat_enc PRIVATE amduat_format amduat_asl_store_fs amduat_asl_store_index_fs
amduat_asl_derivation_index_fs amduat_asl amduat_enc
amduat_hash_asl1 amduat_util amduat_hash_asl1 amduat_util
) )
set_target_properties(amduat_asl_cli PROPERTIES OUTPUT_NAME amduat-asl) set_target_properties(amduat_asl_cli PROPERTIES OUTPUT_NAME amduat-asl)
@ -198,11 +286,24 @@ target_include_directories(amduat_pel_cli
PRIVATE ${AMDUAT_INCLUDE_DIR} PRIVATE ${AMDUAT_INCLUDE_DIR}
) )
target_link_libraries(amduat_pel_cli target_link_libraries(amduat_pel_cli
PRIVATE amduat_format amduat_pel amduat_asl_store_fs amduat_asl amduat_enc PRIVATE amduat_format amduat_pel amduat_asl_store_fs
amduat_asl_collection amduat_asl_record amduat_asl_log_store
amduat_asl_pointer_fs
amduat_asl_derivation_index_fs amduat_asl amduat_enc
amduat_hash_asl1 amduat_util amduat_hash_asl1 amduat_util
) )
set_target_properties(amduat_pel_cli PROPERTIES OUTPUT_NAME amduat-pel) set_target_properties(amduat_pel_cli PROPERTIES OUTPUT_NAME amduat-pel)
add_executable(amduat_pel_build src/tools/amduat_pel_build.c)
target_include_directories(amduat_pel_build
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_pel_build
PRIVATE amduat_pel amduat_asl_store_fs amduat_asl amduat_enc amduat_util
)
set_target_properties(amduat_pel_build PROPERTIES OUTPUT_NAME amduat-pel-build)
enable_testing() enable_testing()
add_executable(amduat_test_pel_program_dag tests/enc/test_pel_program_dag.c) add_executable(amduat_test_pel_program_dag tests/enc/test_pel_program_dag.c)
@ -225,6 +326,19 @@ target_link_libraries(amduat_test_pel_trace_dag
) )
add_test(NAME pel_trace_dag COMMAND amduat_test_pel_trace_dag) add_test(NAME pel_trace_dag COMMAND amduat_test_pel_trace_dag)
add_executable(amduat_test_pel_program_build_concat
tests/pel/test_pel_program_build_concat.c
)
target_include_directories(amduat_test_pel_program_build_concat
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel_program_build_concat
PRIVATE amduat_pel amduat_asl_store_fs amduat_asl amduat_enc
amduat_hash_asl1 amduat_util
)
add_test(NAME pel_program_build_concat COMMAND amduat_test_pel_program_build_concat)
add_executable(amduat_test_pel1_result tests/enc/test_pel1_result.c) add_executable(amduat_test_pel1_result tests/enc/test_pel1_result.c)
target_include_directories(amduat_test_pel1_result target_include_directories(amduat_test_pel1_result
PRIVATE ${AMDUAT_INTERNAL_DIR} PRIVATE ${AMDUAT_INTERNAL_DIR}
@ -247,6 +361,37 @@ target_link_libraries(amduat_test_pel1_result_invariants
add_test(NAME pel1_result_invariants add_test(NAME pel1_result_invariants
COMMAND amduat_test_pel1_result_invariants) COMMAND amduat_test_pel1_result_invariants)
add_executable(amduat_test_asl_log tests/enc/test_asl_log.c)
target_include_directories(amduat_test_asl_log
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_log
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME asl_log COMMAND amduat_test_asl_log)
add_executable(amduat_test_asl_core_index tests/enc/test_asl_core_index.c)
target_include_directories(amduat_test_asl_core_index
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_core_index
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME asl_core_index COMMAND amduat_test_asl_core_index)
add_executable(amduat_test_asl_tgk_exec_plan
tests/enc/test_asl_tgk_exec_plan.c)
target_include_directories(amduat_test_asl_tgk_exec_plan
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_tgk_exec_plan
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME asl_tgk_exec_plan COMMAND amduat_test_asl_tgk_exec_plan)
add_executable(amduat_test_tgk1_edge tests/enc/test_tgk1_edge.c) add_executable(amduat_test_tgk1_edge tests/enc/test_tgk1_edge.c)
target_include_directories(amduat_test_tgk1_edge target_include_directories(amduat_test_tgk1_edge
PRIVATE ${AMDUAT_INTERNAL_DIR} PRIVATE ${AMDUAT_INTERNAL_DIR}
@ -310,6 +455,115 @@ target_link_libraries(amduat_test_asl_ref_derive
) )
add_test(NAME asl_ref_derive COMMAND amduat_test_asl_ref_derive) add_test(NAME asl_ref_derive COMMAND amduat_test_asl_ref_derive)
add_executable(amduat_test_asl_store_indexed_ops
tests/asl/test_asl_store_indexed_ops.c)
target_include_directories(amduat_test_asl_store_indexed_ops
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_store_indexed_ops
PRIVATE amduat_asl
)
add_test(NAME asl_store_indexed_ops COMMAND amduat_test_asl_store_indexed_ops)
add_executable(amduat_test_asl_replay tests/asl/test_asl_replay.c)
target_include_directories(amduat_test_asl_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_replay
PRIVATE amduat_asl
)
add_test(NAME asl_replay COMMAND amduat_test_asl_replay)
add_executable(amduat_test_asl_index_replay
tests/asl/test_asl_index_replay.c)
target_include_directories(amduat_test_asl_index_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
PRIVATE ${AMDUAT_ROOT}/src/adapters/asl_store_index_fs
)
target_compile_definitions(amduat_test_asl_index_replay
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_index_replay
PRIVATE amduat_asl_store_index_fs
)
add_test(NAME asl_index_replay COMMAND amduat_test_asl_index_replay)
add_executable(amduat_test_asl_store_index_fs
tests/asl/test_asl_store_index_fs.c)
target_include_directories(amduat_test_asl_store_index_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_store_index_fs
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_store_index_fs
PRIVATE amduat_asl_store_index_fs amduat_format pthread
)
add_test(NAME asl_store_index_fs COMMAND amduat_test_asl_store_index_fs)
set_tests_properties(asl_store_index_fs PROPERTIES
ENVIRONMENT "AMDUAT_ASL_PERF_COUNT=100;AMDUAT_ASL_STRESS_SECS=5"
TIMEOUT 120
)
add_executable(amduat_test_asl_log_store_index_fs
tests/asl/test_asl_log_store_index_fs.c)
target_include_directories(amduat_test_asl_log_store_index_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_log_store_index_fs
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_log_store_index_fs
PRIVATE amduat_asl_log_store amduat_asl_store_index_fs amduat_format pthread
)
add_test(NAME asl_log_store_index_fs COMMAND amduat_test_asl_log_store_index_fs)
set_tests_properties(asl_log_store_index_fs PROPERTIES TIMEOUT 30)
add_executable(amduat_test_asl_index_put_get_consistency
tests/asl/test_asl_index_put_get_consistency.c)
target_include_directories(amduat_test_asl_index_put_get_consistency
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_index_put_get_consistency
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_index_put_get_consistency
PRIVATE amduat_asl_store_index_fs amduat_format pthread
)
add_test(NAME asl_index_put_get_consistency
COMMAND amduat_test_asl_index_put_get_consistency)
add_executable(amduat_test_asl_index_accel tests/asl/test_asl_index_accel.c)
target_include_directories(amduat_test_asl_index_accel
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_index_accel
PRIVATE amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME asl_index_accel COMMAND amduat_test_asl_index_accel)
add_executable(amduat_test_asl_derivation_index_fs
tests/asl/test_asl_derivation_index_fs.c)
target_include_directories(amduat_test_asl_derivation_index_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_derivation_index_fs
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_derivation_index_fs
PRIVATE amduat_asl_derivation_index_fs
)
add_test(NAME asl_derivation_index_fs
COMMAND amduat_test_asl_derivation_index_fs)
add_executable(amduat_test_pel_program_dag_exec add_executable(amduat_test_pel_program_dag_exec
tests/pel/test_pel_program_dag_exec.c) tests/pel/test_pel_program_dag_exec.c)
target_include_directories(amduat_test_pel_program_dag_exec target_include_directories(amduat_test_pel_program_dag_exec
@ -317,7 +571,7 @@ target_include_directories(amduat_test_pel_program_dag_exec
PRIVATE ${AMDUAT_INCLUDE_DIR} PRIVATE ${AMDUAT_INCLUDE_DIR}
) )
target_link_libraries(amduat_test_pel_program_dag_exec target_link_libraries(amduat_test_pel_program_dag_exec
PRIVATE amduat_pel PRIVATE amduat_pel amduat_asl_store_fs
) )
add_test(NAME pel_program_dag_exec COMMAND amduat_test_pel_program_dag_exec) add_test(NAME pel_program_dag_exec COMMAND amduat_test_pel_program_dag_exec)
@ -349,7 +603,7 @@ target_include_directories(amduat_test_pel_surf_run
PRIVATE ${AMDUAT_INCLUDE_DIR} PRIVATE ${AMDUAT_INCLUDE_DIR}
) )
target_link_libraries(amduat_test_pel_surf_run target_link_libraries(amduat_test_pel_surf_run
PRIVATE amduat_pel PRIVATE amduat_pel amduat_asl_store_fs
) )
add_test(NAME pel_surf_run COMMAND amduat_test_pel_surf_run) add_test(NAME pel_surf_run COMMAND amduat_test_pel_surf_run)
@ -365,3 +619,43 @@ target_link_libraries(amduat_test_pel_queue
PRIVATE amduat_pel PRIVATE amduat_pel
) )
add_test(NAME pel_queue COMMAND amduat_test_pel_queue) add_test(NAME pel_queue COMMAND amduat_test_pel_queue)
add_executable(amduat_test_fed_registry tests/fed/test_fed_registry.c)
target_include_directories(amduat_test_fed_registry
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_registry
PRIVATE amduat_fed
)
add_test(NAME fed_registry COMMAND amduat_test_fed_registry)
add_executable(amduat_test_fed_replay tests/fed/test_fed_replay.c)
target_include_directories(amduat_test_fed_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_replay
PRIVATE amduat_fed
)
add_test(NAME fed_replay COMMAND amduat_test_fed_replay)
add_executable(amduat_test_fed_ingest tests/fed/test_fed_ingest.c)
target_include_directories(amduat_test_fed_ingest
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_ingest
PRIVATE amduat_fed
)
add_test(NAME fed_ingest COMMAND amduat_test_fed_ingest)
add_executable(amduat_test_fed_view tests/fed/test_fed_view.c)
target_include_directories(amduat_test_fed_view
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_view
PRIVATE amduat_fed
)
add_test(NAME fed_view COMMAND amduat_test_fed_view)

View file

@ -65,6 +65,20 @@ status and refs are printed to stderr.
when not using `--output-raw`. when not using `--output-raw`.
- The filesystem ASL store layout expects digests at least 2 bytes long - The filesystem ASL store layout expects digests at least 2 bytes long
(two directory levels). Experimental shorter digests need a different store. (two directory levels). Experimental shorter digests need a different store.
- The filesystem ASL store (`amduat-asl ... --root`) is a legacy convenience
backend; once the index/log store is introduced it is considered
non-conformant to ASL index/log specs and should be used only for quickstart
demos.
- Compatibility & migration: existing `asl_store_fs` stores will not be
automatically upgraded. Plan to re-ingest artifacts into the index/log store
when it lands.
## Documentation
- Implementation clarifications: `docs/spec-clarifications.md`
- Spec coverage matrix: `AUDITS.md` (Spec Coverage section)
- Index/log API sketch: `docs/index-log-api-sketch.md`
- Federation core API: `include/amduat/fed/` (registry, ingest, replay, view)
## PEL reference ## PEL reference

View file

@ -1,3 +0,0 @@
Start testing: Dec 21 23:21 CET
----------------------------------------------------------
End testing: Dec 21 23:21 CET

View file

@ -0,0 +1,295 @@
# Federation Implementation Notes (Core)
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-02-XX
## Purpose
These notes bind the federation semantics docs to concrete core-library
responsibilities, APIs, and data structures. The intent is to keep
federation logic inside the core substrate and keep daemon/frontends thin.
## Normative inputs
Core semantics and replay:
- tier1/asl-federation-1.md
- tier1/asl-federation-replay-1.md
- tier1/asl-store-index-1.md
- tier1/enc-asl-core-index-1.md
Admission and policy gating:
- tier1/asl-dap-1.md
- tier1/asl-policy-hash-1.md
- tier1/asl-domain-model-1.md
- tier1/asl-dam-1.md
Contextual alignment:
- tier1/asl-system-1.md
- tier1/asl-encrypted-blocks-1.md
## Scope (core library)
Federation MUST be implemented as core substrate logic:
- Deterministic federation view construction.
- Replay ordering and bounds per domain.
- Imported record metadata preservation (domain_id, visibility,
cross_domain_source).
- Tombstone and shadowing behavior per domain.
The following are explicitly out of scope for core:
- Transport protocols (HTTP, IPC, gossip).
- Peer discovery and operational orchestration.
- Admin UX and deployment wiring.
- Admission workflows, auth, and retries/backoff.
- Cache policy knobs (fetch timing, eviction, prefetch).
- Operational concerns (metrics, admin endpoints).
- Policy evaluation and per-record filtering decisions.
## Layering note
Core provides deterministic federation semantics and view construction only.
Middle-layer components are responsible for transport, admission workflows,
policy evaluation (including per-record filtering), caching strategies, and
operational wiring.
Definition:
- Middle layer: the daemon/service boundary around core logic that owns
network transport, admission workflows, and operational policy.
## Responsibilities
1) Federation registry
- Tracks known domains, admission status, policy hash, and
last admitted {SnapshotID, LogPrefix}.
- Enforces DAP + policy compatibility prior to admitting remote state.
2) Federation view cache
- Materializes a deterministic view from local state + admitted
published records from other domains.
- Stores imported records with origin metadata for replay.
- Tracks per-domain replay high-water {domain_id, logseq}.
3) Resolver
- Resolves ArtifactKey using local store + federation view.
- Does not mutate store/index as part of GET semantics.
## Data model (suggested)
```c
typedef enum {
AMDUAT_FED_REC_ARTIFACT = 0,
AMDUAT_FED_REC_PER = 1,
AMDUAT_FED_REC_TGK_EDGE = 2,
AMDUAT_FED_REC_TOMBSTONE = 3
} amduat_fed_record_type_t;
typedef struct {
uint32_t domain_id;
uint64_t snapshot_id;
uint64_t log_prefix;
uint64_t last_logseq;
uint8_t admitted; // boolean
uint8_t policy_ok; // boolean
uint8_t reserved[6];
amduat_hash_id_t policy_hash_id;
amduat_octets_t policy_hash;
} amduat_fed_domain_state_t;
typedef struct {
uint32_t domain_id;
uint8_t visibility; // 0 internal, 1 published
uint8_t has_source; // 0/1
uint16_t reserved0;
uint32_t source_domain;
} amduat_fed_record_meta_t;
typedef struct {
amduat_fed_record_type_t type;
union {
amduat_asl_artifact_key_t artifact_key;
amduat_asl_tgk_edge_key_t tgk_edge_key;
amduat_asl_per_key_t per_key;
amduat_asl_artifact_key_t tombstone_key; // key being removed
} id;
} amduat_fed_record_id_t;
typedef struct {
amduat_fed_record_meta_t meta;
amduat_fed_record_id_t id;
amduat_asl_artifact_location_t loc;
uint64_t logseq;
uint64_t snapshot_id;
uint64_t log_prefix;
} amduat_fed_index_record_t;
typedef struct {
amduat_fed_record_id_t id;
uint32_t reason_code; // policy-specific; 0 if unknown
} amduat_fed_policy_deny_t;
```
Notes:
- Imported records MUST retain domain_id and cross-domain source metadata.
- Tombstones must retain domain_id/visibility for domain-local shadowing.
- Each record MUST include a record type and canonical identity for deterministic
replay across artifacts, PERs, TGK edges, and tombstones.
- PER/TGK canonical identities are currently represented by ASL references
(artifact IDs); no separate edge/PER key types exist yet.
## Core API sketch
```c
typedef struct amduat_fed_registry_t amduat_fed_registry_t;
typedef struct amduat_fed_view_t amduat_fed_view_t;
amduat_fed_registry_t *amduat_fed_registry_open(...);
void amduat_fed_registry_close(amduat_fed_registry_t *);
// Admission + policy gating
bool amduat_fed_admit_domain(amduat_fed_registry_t *, domain_id, policy_hash, ...);
bool amduat_fed_set_domain_state(amduat_fed_registry_t *, domain_id,
snapshot_id, log_prefix);
// Ingest published records for a domain (already transported).
// Each record MUST include its type and canonical identity in the id field.
bool amduat_fed_ingest_records(amduat_fed_registry_t *, domain_id,
const amduat_fed_index_record_t *records,
size_t count);
// Build or refresh a deterministic federation view.
amduat_fed_view_t *amduat_fed_view_build(amduat_fed_registry_t *,
const amduat_asl_store_t *local_store,
const amduat_asl_index_state_t *local_state);
// Resolve via local store + federated view.
amduat_asl_store_error_t amduat_fed_resolve(
const amduat_fed_view_t *view,
const amduat_asl_artifact_key_t *key,
amduat_bytes_t *out_bytes);
```
Notes:
- Transport fetch is not part of resolve; it only consumes ingested records.
- The daemon can choose to fetch missing bytes when resolve reports a remote
reference but local bytes are absent.
- If per-record filtering is enabled, it is applied during ingest or view build
and any denials are recorded in view metadata (e.g., a table of
amduat_fed_policy_deny_t).
## Replay and view construction
Rules are as per ASL/FEDERATION-REPLAY/1:
- Records are ordered by (logseq asc, canonical identity tie-break).
- Replay is bounded by {SnapshotID, LogPrefix} per domain.
- Tombstones shadow only within their source domain.
- Imported entries keep domain_id + cross_domain_source.
The view is the union of:
1) Local domain internal + published state at local {SnapshotID, LogPrefix}.
2) Admitted foreign published state at each domain's {SnapshotID, LogPrefix}.
## Cache and storage
Federation view storage MAY be:
- In-memory (ephemeral), or
- On-disk index segments with federation metadata populated.
If remote bytes are fetched, they MUST be written to a cache store that is
logically separate from the authoritative local store (policy-controlled).
## Federation API overview
Core federation primitives are exposed via:
- include/amduat/fed/registry.h
- include/amduat/fed/ingest.h
- include/amduat/fed/replay.h
- include/amduat/fed/view.h
## Integration example (core flow)
```c
amduat_fed_registry_store_t reg_store;
amduat_fed_registry_value_t reg_value;
amduat_fed_domain_state_t reg_states[8];
amduat_fed_registry_value_init(&reg_value, reg_states, 8);
amduat_fed_registry_store_init(&reg_store, local_store);
amduat_fed_registry_store_put(&reg_store, &reg_value, &reg_ref, NULL);
if (amduat_fed_ingest_validate(records, record_count, NULL, NULL) !=
AMDUAT_FED_INGEST_OK) {
return false;
}
if (!amduat_fed_view_build(records,
record_count,
local_domain_id,
bounds,
bounds_len,
denies,
denies_len,
&view)) {
return false;
}
resolve_err = amduat_fed_resolve(&view, local_store, ref, &artifact);
```
## Policy gating
- Admission gating is per-domain.
- Per-record filtering is optional and MUST be an explicit, deterministic
policy layer if enabled.
## Error reporting
Core resolve should distinguish:
- NOT_FOUND (no record in local or federation view)
- FOUND_REMOTE_NO_BYTES (record exists in view but bytes missing locally)
- INTEGRITY_ERROR (hash mismatch on bytes)
- POLICY_DENIED (domain admitted but record filtered by policy)
Notes:
- When per-record filtering is enabled, POLICY_DENIED SHOULD surface at ingest
or view-build time by excluding filtered records from the view and recording
the denial in view metadata. Resolve MAY return POLICY_DENIED only when such a
denial is recorded for the queried key; otherwise it MUST return NOT_FOUND.
## Remote fetch guidance
Core resolve returns only artifact references. The middle layer is responsible
for fetching remote bytes by reference over its transport and caching them
separately from the authoritative local store.
## Ownership and lifecycle
- `amduat_fed_registry_decode` allocates policy hash buffers; free with
`amduat_fed_registry_value_free`.
- `amduat_fed_registry_encode` returns allocated bytes; free with
`amduat_octets_free`.
- `amduat_fed_view_build` allocates view records and denies; free with
`amduat_fed_view_free`.
- `amduat_fed_replay_domain` allocates replay records; free with
`amduat_fed_replay_view_free`.
## Middle-layer fetch example (informative)
```c
err = amduat_fed_resolve(view, local_store, ref, &artifact);
if (err == AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES) {
if (fetch_remote_bytes(ref, &artifact) && cache_store_put(artifact)) {
err = amduat_fed_resolve(view, local_store, ref, &artifact);
}
}
```
## Tests (minimal)
1) Replay ordering determinism across two domains with interleaved logseq.
2) Tombstone shadowing is domain-local.
3) Imported record metadata preserved in view and survives rebuild.
4) Conflict: same ArtifactKey with different bytes across domains rejected.
5) Bound replay by {SnapshotID, LogPrefix} produces stable view.

View file

@ -0,0 +1,58 @@
# Index/Log API Surface (Sketch)
This document is a one-page sketch of the planned public API for ASL index/log
support. It is non-normative and intended to guide header design.
## ASL Index/Log Types (Draft)
```
typedef uint64_t amduat_asl_snapshot_id_t;
typedef uint64_t amduat_asl_log_position_t; // inclusive logseq upper bound
typedef struct {
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t log_position;
} amduat_asl_index_state_t;
```
## Core Store API (Draft)
```
// Initialization and config.
bool amduat_asl_store_index_init(...);
// PUT/GET with index state reporting.
amduat_asl_store_error_t amduat_asl_store_put_indexed(
amduat_asl_store_t *store,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_get_indexed(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact);
```
## Index/Log Introspection (Draft)
```
// Snapshot/log position queries.
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
amduat_asl_index_state_t *out_state);
// Segment and log inspection (read-only).
bool amduat_asl_log_scan(amduat_asl_store_t *store, ...);
bool amduat_asl_segment_scan(amduat_asl_store_t *store, ...);
```
## Expected Error Surfaces
* `AMDUAT_ASL_STORE_ERR_INTEGRITY` for malformed index segments or log records.
* `AMDUAT_ASL_STORE_ERR_IO` for underlying I/O faults.
* `AMDUAT_ASL_STORE_ERR_NOT_FOUND` for absent artifacts or missing segments.
* `AMDUAT_ASL_STORE_ERR_UNSUPPORTED` for unsupported encoding versions.
These are illustrative; exact error codes and mapping will be finalized when
headers are introduced.

201
docs/spec-clarifications.md Normal file
View file

@ -0,0 +1,201 @@
# Spec Clarifications
This document records implementation-level clarifications for draft Tier-1
specs. These notes do not change the specs; they document concrete choices for
the implementation in this repository.
## Glossary and Abbreviations
| Term | Meaning |
| --- | --- |
| CURRENT | Effective index state after replaying a log position on a snapshot. |
| LogPosition | Inclusive `logseq` upper bound for replay (not a byte offset). |
| SnapshotID | Opaque `uint64_t` identifier persisted in `SNAPSHOT_ANCHOR`. |
| Segment seal | Log record admitting a segment via `(segment_id, segment_hash)`. |
| Segment hash | SHA-256 over exact on-disk segment bytes, including footer. |
| Tombstone | Visibility policy record applied during replay. |
| Tombstone lift | Cancels a specific tombstone record for the same artifact. |
| Exec plan | Serialized plan format; executor out of scope for core library. |
## Snapshot and Log Identity (ASL/STORE-INDEX + ASL/LOG)
Decision:
- LogPosition is the log sequence number (`logseq`), not a byte offset.
- SnapshotID is an opaque store-assigned `uint64_t`, persisted in the
`SNAPSHOT_ANCHOR` payload.
Implications:
- `IndexState = (SnapshotID, LogPosition)` uses an inclusive logseq upper bound
when replaying `log[0:LogPosition]`.
- The log's record envelope already carries `logseq`, so snapshot anchors use
the anchor record's `logseq` as the snapshot log position.
- If no snapshot exists, treat SnapshotID as `0` and LogPosition as `0`.
Rationale:
- `ASL/LOG/1` defines replay and visibility in terms of `logseq` ordering.
- `ASL/TGK-EXEC-PLAN/1` orders results by `logseq` and uses `log_prefix` bounds.
- `ASL/STORE-INDEX/1` defines LogPosition as a monotonic integer position and
replay as `log[0:LogPosition]`, which maps directly to logseq.
References:
- `tier1/asl-log-1.md`
- `tier1/enc-asl-log-1.md`
- `tier1/asl-store-index-1.md`
- `tier1/asl-tgk-execution-plan-1.md`
- `tier1/enc-asl-tgk-exec-plan-1.md`
## Index Segment Identity and Seals (ASL/STORE-INDEX + ASL/LOG)
Decision:
- `segment_id` is a store-local, monotonic `uint64_t` assigned when a segment is
created (before writing records), and persisted by naming/metadata outside the
segment file.
- `segment_hash` is SHA-256 over the exact segment file bytes as stored on disk,
including header, records, digest bytes, extents, and footer.
Implications:
- The seal record (`SEGMENT_SEAL`) binds a specific persisted segment file to the
log via `(segment_id, segment_hash)`. Hashing occurs after the footer is
written so the hash commits to seal metadata (CRC, seal snapshot, timestamp).
- Replay uses `segment_id` to locate the segment file and verifies
`segment_hash` before admitting it as visible.
Rationale:
- `ENC/ASL-LOG/1` defines the seal payload as a segment ID plus a hash of the
segment bytes; the log is the visibility gate, so the hash must cover the
complete on-disk segment.
- `ENC/ASL-CORE-INDEX/1` does not embed a segment ID, so the ID must be an
external, store-managed handle (filename or catalog entry).
References:
- `tier1/asl-log-1.md`
- `tier1/enc-asl-log-1.md`
- `tier1/asl-store-index-1.md`
- `tier1/enc-asl-core-index-1.md`
## Tombstone Semantics (ASL/LOG + ASL/STORE-INDEX)
Decision:
- `scope` and `reason_code` are opaque metadata and do not affect shadowing.
- A `TOMBSTONE_LIFT` cancels only the referenced tombstone record for the same
artifact; other tombstones for that artifact remain effective.
Across snapshots:
- Snapshots capture the effective tombstone state as of the snapshot's `logseq`.
- Lifts recorded after a snapshot become effective only when replay reaches
their `logseq`.
References:
- `tier1/asl-log-1.md`
- `tier1/asl-store-index-1.md`
## Federation Fields (ENC/ASL-CORE-INDEX)
Decision:
- Version 3 encoders must always emit federation fields in both headers and
records. They are required, not optional, in v3.
- Decoders accept legacy versions that omit federation fields and apply default
local/internal values as defined in the encoding spec.
References:
- `tier1/enc-asl-core-index-1.md`
## Execution Plan Scope (ASL/TGK-EXEC-PLAN + ENC/ASL-TGK-EXEC-PLAN)
Decision:
- The implementation treats execution plans as a serialized/transport artifact
and semantic contract only. A plan executor is out of scope for the core
library.
References:
- `tier1/asl-tgk-execution-plan-1.md`
- `tier1/enc-asl-tgk-exec-plan-1.md`
## Publish/Unpublish Scope (ASL/LOG + ASL/SYSTEM)
Decision:
- `ARTIFACT_PUBLISH` and `ARTIFACT_UNPUBLISH` are treated as reserved record
types in the core replay path and do not alter ASL index state.
- Publishing is modeled as moving artifacts and index segments between stores,
advancing the destination store's snapshot/log.
Implications:
- Core replay ignores publish/unpublish records.
- Any visibility policy tied to publishing is handled by higher-level tooling
or system-layer orchestration, not ASL/1 core semantics.
References:
- `tier1/asl-log-1.md`
- `tier1/asl-system-1.md`
## Receipt Output Reference Fallback (FER/1 + PEL/1)
Decision:
- When a PEL run produces no output artifacts (e.g. failed execution), the
receipt's `output_ref` falls back to the stored PEL result artifact reference.
Implications:
- Receipts can be emitted for both successful and failed runs using a single
canonical output reference.
- Callers using `amduat_fer1_receipt_from_pel_run` should expect `output_ref`
to match `result_ref` when `output_refs_len == 0`.
References:
- `tier1/enc-fer1-receipt-1.md`
- `tier1/srs.md`
## FER/1 v1.1 Determinism and Validation (FER/1 + SRS)
Decision:
- `run_id` is a deterministic hash over stable inputs only and MUST exclude
timestamps, logs, or mutable metadata.
- Typed logs are optional; if present they MUST be ordered and size-bounded.
- Limits are a single required record when the `limits` TLV is present.
- Executor set verification is strict when a policy-provided set exists.
Concrete rules:
- `run_id = H("AMDUAT:RUN\0" || EncRef(function) || EncRef(input_manifest) ||
EncRef(environment) || EncRef(executor_fingerprint))`, where `EncRef` is
`ENC/ASL1-CORE` canonical bytes and `executor_fingerprint` is the canonical
digest reference. No other fields are included.
- `logs` (if present): order by `(kind, cid)` byte-lexicographically; cap to
64 entries; cap total log payload references to 1 MiB aggregate of capsule
bytes. Reject out-of-order or oversized sets.
- `limits` (if present): exactly one TLV containing all numeric fields
(`cpu_ms`, `wall_ms`, `max_rss_kib`, `io_reads`, `io_writes`) with fixed
units. Reject missing or duplicate fields.
- Executor set validation:
- If an expected executor set is supplied by policy, receipt executor_refs
MUST match it exactly (same members, byte-order, no extras).
- Otherwise, validate strict ordering and uniqueness, and require
`parity_len == executor_refs_len` with aligned ordering and `output_ref`
equality for every parity entry.
References:
- `tier1/srs.md`
- `tier1/enc-fer1-receipt-1.md`
## FER/1 v1.1 Encoding Notes (Implementation)
Decision:
- The v1.1 encoder appends a TLV extension block after the v1 base layout.
- Unknown or duplicate TLV tags are rejected during decode.
TLV tags (implementation):
- `0x0001` executor fingerprint reference (encoded reference bytes).
- `0x0002` run id (`U32` length + bytes).
- `0x0003` logs (`U32` count; per entry: `U32 kind`, encoded ref, `U32` sha256
length + bytes). Entries must be ordered by `(kind, ref)` byte order.
- `0x0004` limits (`U64` cpu_ms, `U64` wall_ms, `U64` max_rss_kib,
`U64` io_reads, `U64` io_writes).
- `0x0005` determinism (`U8` level, `U32` seed_len + seed bytes).
- `0x0006` signature (opaque bytes).
Helper usage:
- `amduat_fer1_receipt_from_pel_run_v1_1` emits v1.1 receipts and uses the
same output_ref fallback as v1: when no outputs exist, `output_ref` is the
stored PEL result reference.
References:
- `include/amduat/enc/fer1_receipt.h`
- `src/near_core/enc/fer1_receipt.c`

View file

@ -0,0 +1,59 @@
#ifndef AMDUAT_ASL_DERIVATION_INDEX_FS_H
#define AMDUAT_ASL_DERIVATION_INDEX_FS_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_DERIVATION_INDEX_FS_ROOT_MAX = 1024 };
typedef struct {
amduat_octets_t sid;
amduat_reference_t program_ref;
uint32_t output_index;
amduat_reference_t *input_refs;
size_t input_refs_len;
bool has_params_ref;
amduat_reference_t params_ref;
bool has_exec_profile;
amduat_octets_t exec_profile;
} amduat_asl_derivation_record_t;
typedef struct {
char root_path[AMDUAT_ASL_DERIVATION_INDEX_FS_ROOT_MAX];
} amduat_asl_derivation_index_fs_t;
bool amduat_asl_derivation_index_fs_init(
amduat_asl_derivation_index_fs_t *index,
const char *root_path);
amduat_asl_store_error_t amduat_asl_derivation_index_fs_add(
amduat_asl_derivation_index_fs_t *index,
amduat_reference_t artifact_ref,
const amduat_asl_derivation_record_t *record);
amduat_asl_store_error_t amduat_asl_derivation_index_fs_list(
amduat_asl_derivation_index_fs_t *index,
amduat_reference_t artifact_ref,
amduat_asl_derivation_record_t **out_records,
size_t *out_count);
void amduat_asl_derivation_record_free(
amduat_asl_derivation_record_t *record);
void amduat_asl_derivation_records_free(
amduat_asl_derivation_record_t *records,
size_t count);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_DERIVATION_INDEX_FS_H */

View file

@ -0,0 +1,40 @@
#ifndef AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H
#define AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX = 1024 };
typedef struct {
char root_path[AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX];
} amduat_asl_materialization_cache_fs_t;
bool amduat_asl_materialization_cache_fs_init(
amduat_asl_materialization_cache_fs_t *cache,
const char *root_path);
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_get(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
amduat_reference_t **out_refs,
size_t *out_refs_len);
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_put(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
const amduat_reference_t *refs,
size_t refs_len);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H */

View file

@ -0,0 +1,58 @@
#ifndef AMDUAT_ASL_POINTER_FS_H
#define AMDUAT_ASL_POINTER_FS_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_POINTER_FS_ROOT_MAX = 1024,
AMDUAT_ASL_POINTER_NAME_MAX = 512
};
typedef enum {
AMDUAT_ASL_POINTER_OK = 0,
AMDUAT_ASL_POINTER_ERR_NOT_FOUND = 1,
AMDUAT_ASL_POINTER_ERR_IO = 2,
AMDUAT_ASL_POINTER_ERR_INVALID_NAME = 3,
AMDUAT_ASL_POINTER_ERR_INTEGRITY = 4
} amduat_asl_pointer_error_t;
typedef struct {
char root_path[AMDUAT_ASL_POINTER_FS_ROOT_MAX];
} amduat_asl_pointer_store_t;
/* Pointers are generic control-plane names for artifacts (e.g. dataset heads).
* Example names:
* - space/123/dataset/calendar_types/head
* - space/123/collection/events/head
*/
bool amduat_asl_pointer_store_init(amduat_asl_pointer_store_t *ps,
const char *root_path);
amduat_asl_pointer_error_t amduat_asl_pointer_get(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool *out_exists,
amduat_reference_t *out_ref);
amduat_asl_pointer_error_t amduat_asl_pointer_cas(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool expected_exists,
const amduat_reference_t *expected_ref,
const amduat_reference_t *new_ref,
bool *out_swapped);
bool amduat_asl_pointer_name_is_valid(const char *name);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_POINTER_FS_H */

View file

@ -0,0 +1,92 @@
#ifndef AMDUAT_ASL_STORE_INDEX_FS_H
#define AMDUAT_ASL_STORE_INDEX_FS_H
#include "amduat/asl/store.h"
#include <pthread.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_STORE_INDEX_FS_ROOT_MAX = 1024 };
typedef struct {
bool enabled;
uint64_t max_pending_bytes;
uint64_t idle_time_ns;
} amduat_asl_store_index_fs_snapshot_policy_t;
typedef struct {
uint64_t max_segment_records;
uint64_t max_segment_bytes;
uint64_t small_artifact_threshold;
bool allow_deferred_visibility;
} amduat_asl_store_index_fs_segment_policy_t;
typedef struct {
uint32_t segment_domain_id;
uint8_t record_visibility;
} amduat_asl_store_index_fs_visibility_policy_t;
typedef struct {
amduat_asl_store_config_t config;
amduat_asl_store_index_fs_snapshot_policy_t snapshot_policy;
amduat_asl_store_index_fs_segment_policy_t segment_policy;
amduat_asl_store_index_fs_visibility_policy_t visibility_policy;
uint16_t shard_count;
char root_path[AMDUAT_ASL_STORE_INDEX_FS_ROOT_MAX];
uint64_t pending_snapshot_bytes;
uint64_t last_ingest_time_ns;
amduat_asl_snapshot_id_t next_snapshot_id;
bool snapshot_state_initialized;
void *open_segments;
pthread_mutex_t write_mutex;
uint32_t write_depth;
int write_lock_fd;
} amduat_asl_store_index_fs_t;
bool amduat_asl_store_index_fs_init(amduat_asl_store_index_fs_t *fs,
amduat_asl_store_config_t config,
const char *root_path);
void amduat_asl_store_index_fs_set_snapshot_policy(
amduat_asl_store_index_fs_t *fs,
amduat_asl_store_index_fs_snapshot_policy_t policy);
void amduat_asl_store_index_fs_set_segment_policy(
amduat_asl_store_index_fs_t *fs,
amduat_asl_store_index_fs_segment_policy_t policy);
void amduat_asl_store_index_fs_set_visibility_policy(
amduat_asl_store_index_fs_t *fs,
amduat_asl_store_index_fs_visibility_policy_t policy);
void amduat_asl_store_index_fs_set_shard_count(
amduat_asl_store_index_fs_t *fs,
uint16_t shard_count);
amduat_asl_store_error_t amduat_asl_store_index_fs_snapshot_create(
amduat_asl_store_index_fs_t *fs,
amduat_asl_snapshot_id_t snapshot_id,
amduat_asl_log_position_t *out_logseq,
uint8_t out_root_hash[32]);
amduat_asl_store_error_t amduat_asl_store_index_fs_flush(
amduat_asl_store_index_fs_t *fs,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_index_fs_gc(
amduat_asl_store_index_fs_t *fs,
const amduat_asl_index_state_t *state);
amduat_asl_store_ops_t amduat_asl_store_index_fs_ops(void);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_STORE_INDEX_FS_H */

View file

@ -0,0 +1,81 @@
#ifndef AMDUAT_ASL_COLLECTION_H
#define AMDUAT_ASL_COLLECTION_H
#include "amduat/asl/asl_pointer_fs.h"
#include "amduat/asl/core.h"
#include "amduat/asl/log_store.h"
#include "amduat/asl/record.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_ASL_COLLECTION_OK = 0,
AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME = 1,
AMDUAT_ASL_COLLECTION_ERR_IO = 2,
AMDUAT_ASL_COLLECTION_ERR_INTEGRITY = 3,
AMDUAT_ASL_COLLECTION_ERR_CAS_MISMATCH = 4
} amduat_asl_collection_error_t;
typedef struct {
amduat_asl_store_t *store;
amduat_asl_log_store_t log_store;
amduat_asl_pointer_store_t pointer_store;
} amduat_asl_collection_store_t;
typedef struct {
uint64_t snapshot_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_collection_snapshot_payload_t;
bool amduat_asl_collection_store_init(
amduat_asl_collection_store_t *collection_store,
const char *root_path,
amduat_asl_store_t *store);
amduat_asl_collection_error_t amduat_asl_collection_append(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
amduat_reference_t record_ref,
uint16_t kind,
amduat_octets_t actor,
uint64_t *out_offset);
amduat_asl_collection_error_t amduat_asl_collection_snapshot(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t up_to_offset,
amduat_reference_t *out_snapshot_ref,
bool *out_swapped);
amduat_asl_collection_error_t amduat_asl_collection_read(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t from_offset,
size_t limit,
amduat_reference_t **out_record_refs,
size_t *out_len,
uint64_t *out_next_offset,
bool *out_end);
void amduat_asl_collection_refs_free(amduat_reference_t *refs,
size_t refs_len);
bool amduat_asl_collection_snapshot_payload_decode_v1(
amduat_octets_t payload,
amduat_asl_collection_snapshot_payload_t *out_snapshot);
void amduat_asl_collection_snapshot_payload_free(
amduat_asl_collection_snapshot_payload_t *snapshot);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_COLLECTION_H */

View file

@ -0,0 +1,74 @@
#ifndef AMDUAT_ASL_COLLECTION_VIEW_H
#define AMDUAT_ASL_COLLECTION_VIEW_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_COLLECTION_VIEW_1 = 0x00000403u };
enum { AMDUAT_TYPE_TAG_ASL_COLLECTION_VIEW_1 = TYPE_TAG_ASL_COLLECTION_VIEW_1 };
enum { TYPE_TAG_ASL_SNAPSHOT_INFO_1 = 0x00000405u };
enum { AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1 = TYPE_TAG_ASL_SNAPSHOT_INFO_1 };
enum { TYPE_TAG_ASL_LOG_RANGE_1 = 0x00000406u };
enum { AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1 = TYPE_TAG_ASL_LOG_RANGE_1 };
typedef struct {
uint64_t snapshot_at_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_collection_snapshot_info_t;
typedef struct {
uint64_t start_offset;
uint64_t next_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_log_range_t;
typedef struct {
uint64_t computed_from_offset;
uint64_t computed_up_to_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_collection_view_t;
bool amduat_asl_collection_snapshot_info_encode_v1(
const amduat_asl_collection_snapshot_info_t *info,
amduat_octets_t *out_bytes);
bool amduat_asl_collection_snapshot_info_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_snapshot_info_t *out_info);
void amduat_asl_collection_snapshot_info_free(
amduat_asl_collection_snapshot_info_t *info);
bool amduat_asl_log_range_encode_v1(const amduat_asl_log_range_t *range,
amduat_octets_t *out_bytes);
bool amduat_asl_log_range_decode_v1(amduat_octets_t bytes,
amduat_asl_log_range_t *out_range);
void amduat_asl_log_range_free(amduat_asl_log_range_t *range);
bool amduat_asl_collection_view_encode_v1(
const amduat_asl_collection_view_t *view,
amduat_octets_t *out_bytes);
bool amduat_asl_collection_view_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_view_t *out_view);
void amduat_asl_collection_view_free(amduat_asl_collection_view_t *view);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_COLLECTION_VIEW_H */

View file

@ -0,0 +1,33 @@
#ifndef AMDUAT_ASL_INDEX_ACCEL_H
#define AMDUAT_ASL_INDEX_ACCEL_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
bool amduat_asl_index_accel_routing_key_from_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
amduat_octets_t *out_key);
bool amduat_asl_index_accel_routing_key_hash(amduat_octets_t key,
uint64_t *out_hash);
uint16_t amduat_asl_index_accel_shard_for_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
uint16_t shard_count);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_ACCEL_H */

View file

@ -0,0 +1,33 @@
#ifndef AMDUAT_ASL_INDEX_BLOOM_H
#define AMDUAT_ASL_INDEX_BLOOM_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_INDEX_BLOOM_BYTES = 256,
AMDUAT_ASL_INDEX_BLOOM_HASHES = 4
};
bool amduat_asl_index_bloom_init(amduat_octets_t *out_bloom);
bool amduat_asl_index_bloom_add(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest);
bool amduat_asl_index_bloom_maybe_contains(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_BLOOM_H */

View file

@ -0,0 +1,47 @@
#ifndef AMDUAT_ASL_INDEX_REPLAY_H
#define AMDUAT_ASL_INDEX_REPLAY_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include "amduat/enc/asl_log.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
uint64_t segment_id;
uint8_t segment_hash[32];
} amduat_asl_segment_seal_t;
typedef struct {
amduat_reference_t ref;
uint64_t tombstone_logseq;
} amduat_asl_tombstone_entry_t;
typedef struct {
amduat_asl_segment_seal_t *segments;
size_t segments_len;
amduat_asl_tombstone_entry_t *tombstones;
size_t tombstones_len;
amduat_asl_index_state_t state;
} amduat_asl_replay_state_t;
bool amduat_asl_replay_init(amduat_asl_replay_state_t *out);
void amduat_asl_replay_free(amduat_asl_replay_state_t *state);
bool amduat_asl_replay_apply_log(
const amduat_asl_log_record_t *records,
size_t record_count,
amduat_asl_log_position_t log_position,
amduat_asl_replay_state_t *state);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_REPLAY_H */

View file

@ -0,0 +1,48 @@
#ifndef AMDUAT_ASL_INDEX_SNAPSHOT_H
#define AMDUAT_ASL_INDEX_SNAPSHOT_H
#include "amduat/asl/index_replay.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION = 1,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE = 80,
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN = 8
};
typedef struct {
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t anchor_logseq;
amduat_asl_store_config_t config;
amduat_asl_segment_seal_t *segments;
size_t segments_len;
amduat_asl_tombstone_entry_t *tombstones;
size_t tombstones_len;
} amduat_asl_snapshot_manifest_t;
bool amduat_asl_snapshot_manifest_write(
const char *path,
const amduat_asl_snapshot_manifest_t *manifest,
uint8_t out_hash[32]);
bool amduat_asl_snapshot_manifest_read(
const char *path,
amduat_asl_snapshot_manifest_t *out_manifest,
uint8_t out_hash[32]);
void amduat_asl_snapshot_manifest_free(
amduat_asl_snapshot_manifest_t *manifest);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_SNAPSHOT_H */

View file

@ -0,0 +1,78 @@
#ifndef AMDUAT_ASL_LOG_STORE_H
#define AMDUAT_ASL_LOG_STORE_H
#include "amduat/asl/asl_pointer_fs.h"
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_LOG_CHUNK_1 = 0x00000401u };
enum { AMDUAT_TYPE_TAG_ASL_LOG_CHUNK_1 = TYPE_TAG_ASL_LOG_CHUNK_1 };
typedef struct {
uint16_t kind;
bool has_timestamp;
uint64_t timestamp;
amduat_reference_t payload_ref;
bool has_actor;
amduat_octets_t actor;
} amduat_asl_log_entry_t;
typedef struct {
bool has_prev;
amduat_reference_t prev_ref;
uint64_t base_offset;
uint32_t entry_count;
bool has_timestamp;
bool has_actor;
amduat_asl_log_entry_t *entries;
} amduat_asl_log_chunk_t;
typedef struct {
amduat_asl_store_t *store;
amduat_asl_pointer_store_t pointer_store;
} amduat_asl_log_store_t;
bool amduat_asl_log_store_init(amduat_asl_log_store_t *log_store,
const char *root_path,
amduat_asl_store_t *store,
const amduat_asl_pointer_store_t *pointer_store);
amduat_asl_store_error_t amduat_asl_log_append(
amduat_asl_log_store_t *log_store,
const char *log_name,
const amduat_asl_log_entry_t *entries,
size_t entries_len,
uint64_t *out_first_offset);
amduat_asl_store_error_t amduat_asl_log_read(
amduat_asl_log_store_t *log_store,
const char *log_name,
uint64_t from_offset,
size_t max_entries,
amduat_asl_log_entry_t **out_entries,
size_t *out_len,
uint64_t *out_next_offset,
bool *out_end);
void amduat_asl_log_entries_free(amduat_asl_log_entry_t *entries,
size_t entries_len);
amduat_asl_store_error_t amduat_asl_log_chunk_decode_v1(
amduat_octets_t bytes,
amduat_asl_log_chunk_t *out_chunk);
void amduat_asl_log_chunk_free(amduat_asl_log_chunk_t *chunk);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_LOG_STORE_H */

27
include/amduat/asl/none.h Normal file
View file

@ -0,0 +1,27 @@
#ifndef AMDUAT_ASL_NONE_H
#define AMDUAT_ASL_NONE_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_NONE_1 = 0x00000404u };
enum { AMDUAT_TYPE_TAG_ASL_NONE_1 = TYPE_TAG_ASL_NONE_1 };
bool amduat_asl_none_encode_v1(amduat_octets_t *out_bytes);
bool amduat_asl_none_decode_v1(amduat_octets_t bytes);
bool amduat_asl_none_is_artifact(const amduat_artifact_t *artifact);
bool amduat_asl_none_artifact(amduat_artifact_t *out_artifact);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_NONE_H */

View file

@ -0,0 +1,46 @@
#ifndef AMDUAT_ASL_RECORD_H
#define AMDUAT_ASL_RECORD_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_RECORD_1 = 0x00000402u };
enum { AMDUAT_TYPE_TAG_ASL_RECORD_1 = TYPE_TAG_ASL_RECORD_1 };
typedef struct {
amduat_octets_t schema;
amduat_octets_t payload;
} amduat_asl_record_t;
bool amduat_asl_record_encode_v1(const amduat_asl_record_t *record,
amduat_octets_t *out_bytes);
bool amduat_asl_record_decode_v1(amduat_octets_t bytes,
amduat_asl_record_t *out_record);
void amduat_asl_record_free(amduat_asl_record_t *record);
amduat_asl_store_error_t amduat_asl_record_store_put(
amduat_asl_store_t *store,
amduat_octets_t schema,
amduat_octets_t payload,
amduat_reference_t *out_ref);
amduat_asl_store_error_t amduat_asl_record_store_get(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_record_t *out_record);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_RECORD_H */

View file

@ -2,6 +2,7 @@
#define AMDUAT_ASL_STORE_H #define AMDUAT_ASL_STORE_H
#include "amduat/asl/core.h" #include "amduat/asl/core.h"
#include "amduat/enc/asl_log.h"
#include <stdbool.h> #include <stdbool.h>
#include <stddef.h> #include <stddef.h>
@ -25,6 +26,14 @@ typedef enum {
AMDUAT_ASL_STORE_ERR_IO = 4 AMDUAT_ASL_STORE_ERR_IO = 4
} amduat_asl_store_error_t; } amduat_asl_store_error_t;
typedef uint64_t amduat_asl_snapshot_id_t;
typedef uint64_t amduat_asl_log_position_t;
typedef struct {
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t log_position;
} amduat_asl_index_state_t;
typedef struct { typedef struct {
amduat_asl_store_error_t (*put)(void *ctx, amduat_asl_store_error_t (*put)(void *ctx,
amduat_artifact_t artifact, amduat_artifact_t artifact,
@ -32,6 +41,29 @@ typedef struct {
amduat_asl_store_error_t (*get)(void *ctx, amduat_asl_store_error_t (*get)(void *ctx,
amduat_reference_t ref, amduat_reference_t ref,
amduat_artifact_t *out_artifact); amduat_artifact_t *out_artifact);
amduat_asl_store_error_t (*put_indexed)(void *ctx,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*get_indexed)(void *ctx,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact);
amduat_asl_store_error_t (*tombstone)(void *ctx,
amduat_reference_t ref,
uint32_t scope,
uint32_t reason_code,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*tombstone_lift)(
void *ctx,
amduat_reference_t ref,
amduat_asl_log_position_t tombstone_logseq,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*log_scan)(
void *ctx,
amduat_asl_log_record_t **out_records,
size_t *out_count);
bool (*current_state)(void *ctx, amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*validate_config)( amduat_asl_store_error_t (*validate_config)(
void *ctx, void *ctx,
amduat_asl_store_config_t config); amduat_asl_store_config_t config);
@ -43,6 +75,12 @@ static inline void amduat_asl_store_ops_init(amduat_asl_store_ops_t *ops) {
} }
ops->put = NULL; ops->put = NULL;
ops->get = NULL; ops->get = NULL;
ops->put_indexed = NULL;
ops->get_indexed = NULL;
ops->tombstone = NULL;
ops->tombstone_lift = NULL;
ops->log_scan = NULL;
ops->current_state = NULL;
ops->validate_config = NULL; ops->validate_config = NULL;
} }
@ -65,6 +103,39 @@ amduat_asl_store_error_t amduat_asl_store_get(amduat_asl_store_t *store,
amduat_reference_t ref, amduat_reference_t ref,
amduat_artifact_t *out_artifact); amduat_artifact_t *out_artifact);
amduat_asl_store_error_t amduat_asl_store_put_indexed(
amduat_asl_store_t *store,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_get_indexed(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact);
amduat_asl_store_error_t amduat_asl_store_tombstone(
amduat_asl_store_t *store,
amduat_reference_t ref,
uint32_t scope,
uint32_t reason_code,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_tombstone_lift(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_log_position_t tombstone_logseq,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_log_scan(
amduat_asl_store_t *store,
amduat_asl_log_record_t **out_records,
size_t *out_count);
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
amduat_asl_index_state_t *out_state);
#ifdef __cplusplus #ifdef __cplusplus
} /* extern "C" */ } /* extern "C" */
#endif #endif

View file

@ -0,0 +1,101 @@
#ifndef AMDUAT_ENC_ASL_CORE_INDEX_H
#define AMDUAT_ENC_ASL_CORE_INDEX_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_CORE_INDEX_VERSION = 3,
AMDUAT_ASL_CORE_INDEX_HEADER_SIZE = 112,
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE = 48,
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE = 16,
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE = 24
};
enum {
AMDUAT_ASL_INDEX_FLAG_TOMBSTONE = 0x00000001u
};
typedef struct {
uint64_t magic;
uint16_t version;
uint16_t shard_id;
uint32_t header_size;
uint64_t snapshot_min;
uint64_t snapshot_max;
uint64_t record_count;
uint64_t records_offset;
uint64_t bloom_offset;
uint64_t bloom_size;
uint64_t digests_offset;
uint64_t digests_size;
uint64_t extents_offset;
uint64_t extent_count;
uint32_t segment_domain_id;
uint8_t segment_visibility;
uint8_t federation_version;
uint16_t reserved0;
uint64_t flags;
} amduat_asl_segment_header_t;
typedef struct {
uint32_t hash_id;
uint16_t digest_len;
uint16_t reserved0;
uint64_t digest_offset;
uint64_t extents_offset;
uint32_t extent_count;
uint32_t total_length;
uint32_t domain_id;
uint8_t visibility;
uint8_t has_cross_domain_source;
uint16_t reserved1;
uint32_t cross_domain_source;
uint32_t flags;
} amduat_asl_index_record_t;
typedef struct {
uint64_t block_id;
uint32_t offset;
uint32_t length;
} amduat_asl_extent_record_t;
typedef struct {
uint64_t crc64;
uint64_t seal_snapshot;
uint64_t seal_time_ns;
} amduat_asl_segment_footer_t;
typedef struct {
amduat_asl_segment_header_t header;
amduat_octets_t bloom;
amduat_asl_index_record_t *records;
size_t record_count;
amduat_octets_t digests;
amduat_asl_extent_record_t *extents;
size_t extent_count;
amduat_asl_segment_footer_t footer;
} amduat_asl_core_index_segment_t;
bool amduat_enc_asl_core_index_encode_v1(
const amduat_asl_core_index_segment_t *segment,
amduat_octets_t *out_bytes);
bool amduat_enc_asl_core_index_decode_v1(
amduat_octets_t bytes,
amduat_asl_core_index_segment_t *out_segment);
void amduat_enc_asl_core_index_free(amduat_asl_core_index_segment_t *segment);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_ASL_CORE_INDEX_H */

View file

@ -0,0 +1,53 @@
#ifndef AMDUAT_ENC_ASL_LOG_H
#define AMDUAT_ENC_ASL_LOG_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL = 0x01,
AMDUAT_ASL_LOG_RECORD_TOMBSTONE = 0x10,
AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT = 0x11,
AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR = 0x20,
AMDUAT_ASL_LOG_RECORD_ARTIFACT_PUBLISH = 0x30,
AMDUAT_ASL_LOG_RECORD_ARTIFACT_UNPUBLISH = 0x31
};
typedef struct {
uint64_t logseq;
uint32_t record_type;
amduat_octets_t payload;
uint8_t record_hash[32];
} amduat_asl_log_record_t;
bool amduat_enc_asl_log_encode_v1(const amduat_asl_log_record_t *records,
size_t record_count,
amduat_octets_t *out_bytes);
bool amduat_enc_asl_log_decode_v1(amduat_octets_t bytes,
amduat_asl_log_record_t **out_records,
size_t *out_count);
void amduat_enc_asl_log_free(amduat_asl_log_record_t *records,
size_t record_count);
/* Caller owns out_ref digest; free with amduat_reference_free. */
bool amduat_asl_log_decode_artifact_ref(amduat_octets_t payload,
amduat_reference_t *out_ref);
/* Caller owns out_bytes; free with amduat_octets_free. */
bool amduat_asl_log_encode_artifact_ref(amduat_reference_t ref,
amduat_octets_t *out_bytes);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_ASL_LOG_H */

View file

@ -0,0 +1,115 @@
#ifndef AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H
#define AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_TGK_EXEC_PLAN_VERSION = 1 };
typedef enum {
AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN = 0,
AMDUAT_ASL_TGK_EXEC_OP_INDEX_FILTER = 1,
AMDUAT_ASL_TGK_EXEC_OP_MERGE = 2,
AMDUAT_ASL_TGK_EXEC_OP_PROJECTION = 3,
AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL = 4,
AMDUAT_ASL_TGK_EXEC_OP_AGGREGATION = 5,
AMDUAT_ASL_TGK_EXEC_OP_LIMIT_OFFSET = 6,
AMDUAT_ASL_TGK_EXEC_OP_SHARD_DISPATCH = 7,
AMDUAT_ASL_TGK_EXEC_OP_SIMD_FILTER = 8,
AMDUAT_ASL_TGK_EXEC_OP_TOMBSTONE_SHADOW = 9
} amduat_asl_tgk_exec_operator_type_t;
typedef enum {
AMDUAT_ASL_TGK_EXEC_OP_FLAG_NONE = 0x00,
AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL = 0x01,
AMDUAT_ASL_TGK_EXEC_OP_FLAG_OPTIONAL = 0x02
} amduat_asl_tgk_exec_operator_flags_t;
typedef struct {
uint64_t logseq_min;
uint64_t logseq_max;
} amduat_asl_tgk_exec_snapshot_range_t;
typedef struct {
struct {
uint8_t is_asl_segment;
uint64_t segment_start_id;
uint64_t segment_end_id;
} segment_scan;
struct {
uint32_t artifact_type_tag;
uint8_t has_type_tag;
uint32_t edge_type_key;
uint8_t has_edge_type;
uint8_t role;
} index_filter;
struct {
uint8_t deterministic;
} merge;
struct {
uint8_t project_artifact_id;
uint8_t project_tgk_edge_id;
uint8_t project_node_id;
uint8_t project_type_tag;
} projection;
struct {
uint64_t start_node_id;
uint32_t traversal_depth;
uint8_t direction;
} tgk_traversal;
struct {
uint8_t agg_count;
uint8_t agg_union;
uint8_t agg_sum;
} aggregation;
struct {
uint64_t limit;
uint64_t offset;
} limit_offset;
} amduat_asl_tgk_exec_operator_params_t;
typedef struct {
uint32_t op_id;
amduat_asl_tgk_exec_operator_type_t op_type;
amduat_asl_tgk_exec_operator_flags_t flags;
amduat_asl_tgk_exec_snapshot_range_t snapshot;
amduat_asl_tgk_exec_operator_params_t params;
uint32_t input_count;
uint32_t inputs[8];
} amduat_asl_tgk_exec_operator_def_t;
typedef struct {
uint32_t plan_version;
uint32_t operator_count;
amduat_asl_tgk_exec_operator_def_t *operators;
} amduat_asl_tgk_exec_plan_t;
bool amduat_enc_asl_tgk_exec_plan_encode_v1(
const amduat_asl_tgk_exec_plan_t *plan,
amduat_octets_t *out_bytes);
bool amduat_enc_asl_tgk_exec_plan_decode_v1(
amduat_octets_t bytes,
amduat_asl_tgk_exec_plan_t *out_plan);
void amduat_enc_asl_tgk_exec_plan_free(
amduat_asl_tgk_exec_plan_t *plan);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H */

View file

@ -11,6 +11,10 @@ extern "C" {
#endif #endif
enum { FER1_RECEIPT_ENC_V1 = 0x0301u }; enum { FER1_RECEIPT_ENC_V1 = 0x0301u };
enum { FER1_RECEIPT_ENC_V1_1 = 0x0302u };
enum { AMDUAT_FER1_VERSION_1 = 0x0001u };
enum { AMDUAT_FER1_VERSION_1_1 = 0x0101u };
enum { TYPE_TAG_FER1_RECEIPT_1 = 0x00000301u }; enum { TYPE_TAG_FER1_RECEIPT_1 = 0x00000301u };
enum { AMDUAT_TYPE_TAG_FER1_RECEIPT_1 = TYPE_TAG_FER1_RECEIPT_1 }; enum { AMDUAT_TYPE_TAG_FER1_RECEIPT_1 = TYPE_TAG_FER1_RECEIPT_1 };
@ -23,6 +27,20 @@ typedef struct {
amduat_octets_t parity_digest; amduat_octets_t parity_digest;
} amduat_fer1_parity_entry_t; } amduat_fer1_parity_entry_t;
typedef struct {
uint32_t kind;
amduat_reference_t log_ref;
amduat_octets_t sha256;
} amduat_fer1_log_entry_t;
typedef struct {
uint64_t cpu_ms;
uint64_t wall_ms;
uint64_t max_rss_kib;
uint64_t io_reads;
uint64_t io_writes;
} amduat_fer1_limits_t;
typedef struct { typedef struct {
uint16_t fer1_version; uint16_t fer1_version;
amduat_reference_t function_ref; amduat_reference_t function_ref;
@ -36,14 +54,34 @@ typedef struct {
size_t parity_len; size_t parity_len;
uint64_t started_at; uint64_t started_at;
uint64_t completed_at; uint64_t completed_at;
bool has_executor_fingerprint_ref;
amduat_reference_t executor_fingerprint_ref;
bool has_run_id;
amduat_octets_t run_id;
bool has_limits;
amduat_fer1_limits_t limits;
amduat_fer1_log_entry_t *logs;
size_t logs_len;
bool has_determinism;
uint8_t determinism_level;
bool has_rng_seed;
amduat_octets_t rng_seed;
bool has_signature;
amduat_octets_t signature;
} amduat_fer1_receipt_t; } amduat_fer1_receipt_t;
bool amduat_enc_fer1_receipt_encode_v1( bool amduat_enc_fer1_receipt_encode_v1(
const amduat_fer1_receipt_t *receipt, const amduat_fer1_receipt_t *receipt,
amduat_octets_t *out_bytes); amduat_octets_t *out_bytes);
bool amduat_enc_fer1_receipt_encode_v1_1(
const amduat_fer1_receipt_t *receipt,
amduat_octets_t *out_bytes);
bool amduat_enc_fer1_receipt_decode_v1( bool amduat_enc_fer1_receipt_decode_v1(
amduat_octets_t bytes, amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt); amduat_fer1_receipt_t *out_receipt);
bool amduat_enc_fer1_receipt_decode_v1_1(
amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt);
void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt); void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt);
#ifdef __cplusplus #ifdef __cplusplus

View file

@ -0,0 +1,28 @@
#ifndef AMDUAT_FED_INGEST_H
#define AMDUAT_FED_INGEST_H
#include "amduat/fed/replay.h"
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FED_INGEST_OK = 0,
AMDUAT_FED_INGEST_ERR_INVALID = 1,
AMDUAT_FED_INGEST_ERR_CONFLICT = 2
} amduat_fed_ingest_error_t;
amduat_fed_ingest_error_t amduat_fed_ingest_validate(
const amduat_fed_record_t *records,
size_t count,
size_t *out_error_index,
size_t *out_conflict_index);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_INGEST_H */

View file

@ -0,0 +1,84 @@
#ifndef AMDUAT_FED_REGISTRY_H
#define AMDUAT_FED_REGISTRY_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
uint32_t domain_id;
uint64_t snapshot_id;
uint64_t log_prefix;
uint64_t last_logseq;
uint8_t admitted;
uint8_t policy_ok;
uint8_t reserved[6];
amduat_hash_id_t policy_hash_id;
amduat_octets_t policy_hash; /* Empty when unknown; caller-owned bytes. */
} amduat_fed_domain_state_t;
typedef struct {
amduat_fed_domain_state_t *states;
size_t len;
size_t cap;
bool owns_states;
} amduat_fed_registry_value_t;
void amduat_fed_registry_value_init(amduat_fed_registry_value_t *value,
amduat_fed_domain_state_t *states,
size_t cap);
bool amduat_fed_registry_value_insert(amduat_fed_registry_value_t *value,
amduat_fed_domain_state_t state);
const amduat_fed_domain_state_t *amduat_fed_registry_value_lookup(
const amduat_fed_registry_value_t *value,
uint32_t domain_id);
void amduat_fed_registry_value_free(amduat_fed_registry_value_t *value);
/* Encode/Decode allocate buffers; caller frees via amduat_octets_free. */
bool amduat_fed_registry_encode(const amduat_fed_registry_value_t *value,
amduat_octets_t *out_bytes);
/* Decode allocates policy_hash bytes; free via amduat_fed_registry_value_free. */
bool amduat_fed_registry_decode(amduat_octets_t bytes,
amduat_fed_registry_value_t *out_value);
typedef enum {
AMDUAT_FED_REGISTRY_OK = 0,
AMDUAT_FED_REGISTRY_ERR_CODEC = 1,
AMDUAT_FED_REGISTRY_ERR_STORE = 2
} amduat_fed_registry_error_t;
typedef struct {
amduat_asl_store_t *store;
} amduat_fed_registry_store_t;
void amduat_fed_registry_store_init(amduat_fed_registry_store_t *reg,
amduat_asl_store_t *store);
amduat_fed_registry_error_t amduat_fed_registry_store_put(
amduat_fed_registry_store_t *reg,
const amduat_fed_registry_value_t *value,
amduat_reference_t *out_ref,
amduat_asl_store_error_t *out_store_err);
amduat_fed_registry_error_t amduat_fed_registry_store_get(
amduat_fed_registry_store_t *reg,
amduat_reference_t ref,
amduat_fed_registry_value_t *out_value,
amduat_asl_store_error_t *out_store_err);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_REGISTRY_H */

View file

@ -0,0 +1,72 @@
#ifndef AMDUAT_FED_REPLAY_H
#define AMDUAT_FED_REPLAY_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FED_REC_ARTIFACT = 0,
AMDUAT_FED_REC_PER = 1,
AMDUAT_FED_REC_TGK_EDGE = 2,
AMDUAT_FED_REC_TOMBSTONE = 3
} amduat_fed_record_type_t;
typedef struct {
uint32_t domain_id;
uint8_t visibility;
uint8_t has_source;
uint16_t reserved0;
uint32_t source_domain;
} amduat_fed_record_meta_t;
typedef struct {
amduat_fed_record_type_t type;
amduat_reference_t ref; /* PER/TGK identities are ASL references. */
} amduat_fed_record_id_t;
typedef struct {
amduat_fed_record_meta_t meta;
amduat_fed_record_id_t id;
uint64_t logseq;
uint64_t snapshot_id;
uint64_t log_prefix;
} amduat_fed_record_t;
typedef struct {
amduat_fed_record_t *records;
size_t len;
} amduat_fed_replay_view_t;
/* Caller frees record ids with amduat_fed_replay_view_free. */
bool amduat_fed_record_validate(const amduat_fed_record_t *record);
bool amduat_fed_replay_build(const amduat_fed_record_t *records,
size_t count,
uint32_t domain_id,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_replay_view_t *out_view);
/* Backwards-compatible alias for amduat_fed_replay_build. */
/* Deprecated: use amduat_fed_replay_build. */
bool amduat_fed_replay_domain(const amduat_fed_record_t *records,
size_t count,
uint32_t domain_id,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_replay_view_t *out_view);
void amduat_fed_replay_view_free(amduat_fed_replay_view_t *view);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_REPLAY_H */

72
include/amduat/fed/view.h Normal file
View file

@ -0,0 +1,72 @@
#ifndef AMDUAT_FED_VIEW_H
#define AMDUAT_FED_VIEW_H
#include "amduat/asl/store.h"
#include "amduat/fed/replay.h"
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FED_RESOLVE_OK = 0,
AMDUAT_FED_RESOLVE_NOT_FOUND = 1,
AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES = 2,
AMDUAT_FED_RESOLVE_INTEGRITY_ERROR = 3,
AMDUAT_FED_RESOLVE_POLICY_DENIED = 4,
AMDUAT_FED_RESOLVE_STORE_ERROR = 5
} amduat_fed_resolve_error_t;
typedef struct {
uint32_t domain_id;
uint64_t snapshot_id;
uint64_t log_prefix;
} amduat_fed_view_bounds_t;
typedef struct {
amduat_fed_record_id_t id;
uint32_t reason_code;
} amduat_fed_policy_deny_t;
typedef struct {
uint32_t local_domain_id;
amduat_fed_record_t *records;
size_t len;
amduat_fed_policy_deny_t *denies;
size_t denies_len;
} amduat_fed_view_t;
typedef enum {
AMDUAT_FED_VIEW_OK = 0,
AMDUAT_FED_VIEW_ERR_INVALID = 1,
AMDUAT_FED_VIEW_ERR_CONFLICT = 2,
AMDUAT_FED_VIEW_ERR_OOM = 3
} amduat_fed_view_error_t;
/* Caller frees records/denies with amduat_fed_view_free. */
amduat_fed_view_error_t amduat_fed_view_build(
const amduat_fed_record_t *records,
size_t count,
uint32_t local_domain_id,
const amduat_fed_view_bounds_t *bounds,
size_t bounds_len,
const amduat_fed_policy_deny_t *denies,
size_t denies_len,
amduat_fed_view_t *out_view);
void amduat_fed_view_free(amduat_fed_view_t *view);
amduat_fed_resolve_error_t amduat_fed_resolve(
const amduat_fed_view_t *view,
amduat_asl_store_t *local_store,
amduat_reference_t ref,
amduat_artifact_t *out_artifact);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_VIEW_H */

View file

@ -2,6 +2,8 @@
#define AMDUAT_FER_RECEIPT_H #define AMDUAT_FER_RECEIPT_H
#include "amduat/asl/core.h" #include "amduat/asl/core.h"
#include "amduat/enc/fer1_receipt.h"
#include "amduat/pel/run.h"
#include "amduat/pel/surf.h" #include "amduat/pel/surf.h"
#include <stdbool.h> #include <stdbool.h>
@ -24,6 +26,46 @@ bool amduat_fer1_receipt_from_pel_result(
uint64_t completed_at, uint64_t completed_at,
amduat_artifact_t *out_artifact); amduat_artifact_t *out_artifact);
bool amduat_fer1_receipt_from_pel_run(
const amduat_pel_run_result_t *pel_run,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
amduat_artifact_t *out_artifact);
bool amduat_fer1_receipt_from_pel_run_v1_1(
const amduat_pel_run_result_t *pel_run,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
bool has_executor_fingerprint_ref,
amduat_reference_t executor_fingerprint_ref,
bool has_run_id,
amduat_octets_t run_id,
bool has_limits,
amduat_fer1_limits_t limits,
const amduat_fer1_log_entry_t *logs,
size_t logs_len,
bool has_determinism,
uint8_t determinism_level,
bool has_rng_seed,
amduat_octets_t rng_seed,
bool has_signature,
amduat_octets_t signature,
amduat_artifact_t *out_artifact);
#ifdef __cplusplus #ifdef __cplusplus
} /* extern "C" */ } /* extern "C" */
#endif #endif

View file

@ -0,0 +1,31 @@
#ifndef AMDUAT_PEL_DERIVATION_SID_H
#define AMDUAT_PEL_DERIVATION_SID_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
amduat_reference_t program_ref;
const amduat_reference_t *input_refs;
size_t input_refs_len;
bool has_params_ref;
amduat_reference_t params_ref;
bool has_exec_profile;
amduat_octets_t exec_profile;
} amduat_pel_derivation_sid_input_t;
bool amduat_pel_derivation_sid_compute(
const amduat_pel_derivation_sid_input_t *in,
amduat_octets_t *out_sid);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_PEL_DERIVATION_SID_H */

View file

@ -17,6 +17,11 @@ extern "C" {
#define AMDUAT_PEL_KERNEL_OP_HASH_ASL1_NAME "pel.bytes.hash.asl1" #define AMDUAT_PEL_KERNEL_OP_HASH_ASL1_NAME "pel.bytes.hash.asl1"
#define AMDUAT_PEL_KERNEL_OP_PARAMS_NAME "pel.bytes.params" #define AMDUAT_PEL_KERNEL_OP_PARAMS_NAME "pel.bytes.params"
#define AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME "pel.format.encode" #define AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME "pel.format.encode"
#define AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME \
"collection.snapshot_decode_v1"
#define AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME "log.read_range_v1"
#define AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME \
"collection.merge_refs_v1"
enum { enum {
AMDUAT_PEL_KERNEL_OP_CODE_CONCAT = 0x0001u, AMDUAT_PEL_KERNEL_OP_CODE_CONCAT = 0x0001u,
@ -24,7 +29,10 @@ enum {
AMDUAT_PEL_KERNEL_OP_CODE_CONST = 0x0003u, AMDUAT_PEL_KERNEL_OP_CODE_CONST = 0x0003u,
AMDUAT_PEL_KERNEL_OP_CODE_HASH_ASL1 = 0x0004u, AMDUAT_PEL_KERNEL_OP_CODE_HASH_ASL1 = 0x0004u,
AMDUAT_PEL_KERNEL_OP_CODE_PARAMS = 0x0005u, AMDUAT_PEL_KERNEL_OP_CODE_PARAMS = 0x0005u,
AMDUAT_PEL_KERNEL_OP_CODE_FORMAT_ENCODE = 0x0006u AMDUAT_PEL_KERNEL_OP_CODE_FORMAT_ENCODE = 0x0006u,
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_SNAPSHOT_DECODE = 0x0007u,
AMDUAT_PEL_KERNEL_OP_CODE_LOG_READ_RANGE = 0x0008u,
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_MERGE_REFS = 0x0009u
}; };
enum { enum {
@ -35,7 +43,13 @@ enum {
AMDUAT_PEL_KERNEL_STATUS_FORMAT_SCHEMA_INVALID = 0x00030001u, AMDUAT_PEL_KERNEL_STATUS_FORMAT_SCHEMA_INVALID = 0x00030001u,
AMDUAT_PEL_KERNEL_STATUS_FORMAT_REGISTRY_INVALID = 0x00030002u, AMDUAT_PEL_KERNEL_STATUS_FORMAT_REGISTRY_INVALID = 0x00030002u,
AMDUAT_PEL_KERNEL_STATUS_FORMAT_MISSING_FIELD = 0x00030003u, AMDUAT_PEL_KERNEL_STATUS_FORMAT_MISSING_FIELD = 0x00030003u,
AMDUAT_PEL_KERNEL_STATUS_FORMAT_TYPE_UNSUPPORTED = 0x00030004u AMDUAT_PEL_KERNEL_STATUS_FORMAT_TYPE_UNSUPPORTED = 0x00030004u,
AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID = 0x00040001u,
AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID = 0x00040002u,
AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID = 0x00040003u,
AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID = 0x00040004u,
AMDUAT_PEL_KERNEL_STATUS_COLLECTION_VIEW_INVALID = 0x00040005u,
AMDUAT_PEL_KERNEL_STATUS_REF_RESOLVE_FAILED = 0x00040006u
}; };
typedef enum { typedef enum {
@ -44,7 +58,10 @@ typedef enum {
AMDUAT_PEL_KERNEL_OP_CONST = 3, AMDUAT_PEL_KERNEL_OP_CONST = 3,
AMDUAT_PEL_KERNEL_OP_HASH_ASL1 = 4, AMDUAT_PEL_KERNEL_OP_HASH_ASL1 = 4,
AMDUAT_PEL_KERNEL_OP_PARAMS = 5, AMDUAT_PEL_KERNEL_OP_PARAMS = 5,
AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE = 6 AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE = 6,
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE = 7,
AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE = 8,
AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS = 9
} amduat_pel_kernel_op_kind_t; } amduat_pel_kernel_op_kind_t;
typedef struct { typedef struct {
@ -58,6 +75,11 @@ typedef struct {
struct amduat_pel_kernel_params; struct amduat_pel_kernel_params;
typedef struct amduat_pel_kernel_params amduat_pel_kernel_params_t; typedef struct amduat_pel_kernel_params amduat_pel_kernel_params_t;
typedef bool (*amduat_pel_kernel_artifact_resolver_t)(
amduat_reference_t ref,
amduat_artifact_t *out_artifact,
void *ctx);
const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup( const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
amduat_octets_t name, amduat_octets_t name,
uint32_t version); uint32_t version);
@ -67,6 +89,15 @@ const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_descs(
const char *amduat_pel_kernel_op_name(amduat_pel_kernel_op_kind_t kind); const char *amduat_pel_kernel_op_name(amduat_pel_kernel_op_kind_t kind);
void amduat_pel_kernel_set_artifact_resolver(
amduat_pel_kernel_artifact_resolver_t resolver,
void *ctx);
void amduat_pel_kernel_clear_artifact_resolver(void);
bool amduat_pel_kernel_resolve_artifact(amduat_reference_t ref,
amduat_artifact_t *out_artifact);
bool amduat_pel_kernel_op_eval( bool amduat_pel_kernel_op_eval(
const amduat_pel_kernel_op_desc_t *desc, const amduat_pel_kernel_op_desc_t *desc,
const amduat_artifact_t *inputs, const amduat_artifact_t *inputs,

View file

@ -0,0 +1,38 @@
#ifndef AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H
#define AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H
/* TGK/1 projection over ASL index/log (filesystem backend). */
#include "amduat/asl/asl_store_index_fs.h"
#include "amduat/tgk/store.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
amduat_asl_store_index_fs_t *asl_fs;
amduat_asl_store_t asl_store;
amduat_tgk_store_config_t config;
amduat_asl_index_state_t pinned_state;
bool pinned;
bool use_shards;
} amduat_tgk_store_asl_index_fs_t;
bool amduat_tgk_store_asl_index_fs_init(
amduat_tgk_store_asl_index_fs_t *store,
amduat_tgk_store_config_t config,
amduat_asl_store_index_fs_t *asl_fs);
void amduat_tgk_store_asl_index_fs_free(
amduat_tgk_store_asl_index_fs_t *store);
amduat_tgk_store_ops_t amduat_tgk_store_asl_index_fs_ops(void);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H */

23
include/amduat/util/log.h Normal file
View file

@ -0,0 +1,23 @@
#ifndef AMDUAT_UTIL_LOG_H
#define AMDUAT_UTIL_LOG_H
#include <stdarg.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_LOG_ERROR = 0,
AMDUAT_LOG_WARN = 1,
AMDUAT_LOG_INFO = 2,
AMDUAT_LOG_DEBUG = 3
} amduat_log_level_t;
void amduat_log(amduat_log_level_t level, const char *fmt, ...);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_UTIL_LOG_H */

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,716 @@
#include "amduat/asl/asl_materialization_cache_fs.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/util/log.h"
#include "amduat/util/hex.h"
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
enum {
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN = 8,
AMDUAT_ASL_MATERIALIZATION_VERSION = 1
};
static const uint8_t k_amduat_asl_materialization_magic[
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN] = {'A', 'S', 'L', 'M',
'A', 'T', '1', '\0'};
enum {
AMDUAT_ASL_MATERIALIZATION_FLAG_TIMESTAMP = 1u << 0,
AMDUAT_ASL_MATERIALIZATION_FLAG_PEL_VERSION = 1u << 1
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_materialization_cursor_t;
static void amduat_asl_materialization_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_materialization_read_u32_le(
amduat_asl_materialization_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4u;
return true;
}
static bool amduat_asl_materialization_read_u64_le(
amduat_asl_materialization_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8u;
return true;
}
static bool amduat_asl_materialization_read_u16_le(
amduat_asl_materialization_cursor_t *cur,
uint16_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 2u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
cur->offset += 2u;
return true;
}
static bool amduat_asl_materialization_read_u8(
amduat_asl_materialization_cursor_t *cur,
uint8_t *out) {
if (cur->len - cur->offset < 1u) {
return false;
}
*out = cur->data[cur->offset++];
return true;
}
static bool amduat_asl_materialization_join_path(const char *base,
const char *segment,
char **out_path) {
size_t base_len;
size_t segment_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || segment[0] == '\0') {
return false;
}
base_len = strlen(base);
segment_len = strlen(segment);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + segment_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, segment_len);
offset += segment_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool amduat_asl_materialization_ensure_directory(const char *path) {
struct stat st;
if (path == NULL || path[0] == '\0') {
return false;
}
if (stat(path, &st) == 0) {
return S_ISDIR(st.st_mode);
}
if (errno != ENOENT) {
return false;
}
if (mkdir(path, 0755) != 0) {
if (errno == EEXIST) {
return stat(path, &st) == 0 && S_ISDIR(st.st_mode);
}
return false;
}
return true;
}
static bool amduat_asl_materialization_hex_byte(uint8_t value, char out[3]) {
static const char k_hex[] = "0123456789abcdef";
out[0] = k_hex[value >> 4u];
out[1] = k_hex[value & 0x0fu];
out[2] = '\0';
return true;
}
static char *amduat_asl_materialization_parent_dir(const char *path) {
const char *slash;
size_t len;
char *dir;
if (path == NULL) {
return NULL;
}
slash = strrchr(path, '/');
if (slash == NULL || slash == path) {
return NULL;
}
len = (size_t)(slash - path);
dir = (char *)malloc(len + 1u);
if (dir == NULL) {
return NULL;
}
memcpy(dir, path, len);
dir[len] = '\0';
return dir;
}
static bool amduat_asl_materialization_fsync_directory(const char *path) {
int fd;
int flags = O_RDONLY;
if (path == NULL) {
return false;
}
#ifdef O_DIRECTORY
flags |= O_DIRECTORY;
#endif
fd = open(path, flags);
if (fd < 0) {
return false;
}
if (fsync(fd) != 0) {
close(fd);
return false;
}
return close(fd) == 0;
}
static bool amduat_asl_materialization_build_sid_path(
const char *root_path,
amduat_octets_t sid,
bool ensure_dirs,
char **out_path) {
char *index_path = NULL;
char *materializations_path = NULL;
char *by_sid_path = NULL;
char *prefix_a_path = NULL;
char *prefix_b_path = NULL;
char *sid_hex = NULL;
char prefix_a[3];
char prefix_b[3];
bool ok = false;
if (root_path == NULL || out_path == NULL) {
return false;
}
*out_path = NULL;
if (sid.len < 2u || sid.data == NULL) {
return false;
}
if (!amduat_hex_encode_alloc(sid.data, sid.len, &sid_hex)) {
return false;
}
amduat_asl_materialization_hex_byte(sid.data[0], prefix_a);
amduat_asl_materialization_hex_byte(sid.data[1], prefix_b);
if (!amduat_asl_materialization_join_path(root_path, "index", &index_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(index_path, "materializations",
&materializations_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(materializations_path, "by_sid",
&by_sid_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(by_sid_path, prefix_a,
&prefix_a_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(prefix_a_path, prefix_b,
&prefix_b_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(prefix_b_path, sid_hex,
out_path)) {
goto cleanup;
}
if (ensure_dirs) {
if (!amduat_asl_materialization_ensure_directory(index_path) ||
!amduat_asl_materialization_ensure_directory(materializations_path) ||
!amduat_asl_materialization_ensure_directory(by_sid_path) ||
!amduat_asl_materialization_ensure_directory(prefix_a_path) ||
!amduat_asl_materialization_ensure_directory(prefix_b_path)) {
free(*out_path);
*out_path = NULL;
goto cleanup;
}
}
ok = true;
cleanup:
free(index_path);
free(materializations_path);
free(by_sid_path);
free(prefix_a_path);
free(prefix_b_path);
free(sid_hex);
return ok;
}
bool amduat_asl_materialization_cache_fs_init(
amduat_asl_materialization_cache_fs_t *cache,
const char *root_path) {
size_t len;
if (cache == NULL || root_path == NULL) {
return false;
}
len = strlen(root_path);
if (len == 0u || len >= AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX) {
return false;
}
memcpy(cache->root_path, root_path, len);
cache->root_path[len] = '\0';
return true;
}
static amduat_asl_store_error_t amduat_asl_materialization_decode_refs(
amduat_asl_materialization_cursor_t *cur,
uint32_t count,
amduat_reference_t **out_refs,
size_t *out_refs_len) {
amduat_reference_t *refs;
uint32_t i;
if (out_refs == NULL || out_refs_len == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
*out_refs = NULL;
*out_refs_len = 0u;
if (count == 0u) {
return AMDUAT_ASL_STORE_OK;
}
if (count > SIZE_MAX / sizeof(*refs)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
refs = (amduat_reference_t *)calloc(count, sizeof(*refs));
if (refs == NULL) {
return AMDUAT_ASL_STORE_ERR_IO;
}
for (i = 0u; i < count; ++i) {
uint32_t ref_len;
amduat_octets_t ref_bytes;
if (!amduat_asl_materialization_read_u32_le(cur, &ref_len)) {
goto decode_error;
}
if (ref_len < 2u || cur->len - cur->offset < ref_len) {
goto decode_error;
}
ref_bytes = amduat_octets(cur->data + cur->offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, &refs[i])) {
goto decode_error;
}
cur->offset += ref_len;
}
*out_refs = refs;
*out_refs_len = count;
return AMDUAT_ASL_STORE_OK;
decode_error:
for (uint32_t j = 0u; j < i; ++j) {
amduat_reference_free(&refs[j]);
}
free(refs);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
static void amduat_asl_materialization_refs_free(amduat_reference_t *refs,
size_t refs_len) {
size_t i;
if (refs == NULL) {
return;
}
for (i = 0u; i < refs_len; ++i) {
amduat_reference_free(&refs[i]);
}
free(refs);
}
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_get(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
amduat_reference_t **out_refs,
size_t *out_refs_len) {
char *path = NULL;
FILE *fp = NULL;
uint8_t *buffer = NULL;
long file_size;
amduat_asl_materialization_cursor_t cur;
uint8_t flags;
uint32_t version;
uint32_t sid_len;
uint32_t output_count;
amduat_asl_store_error_t err = AMDUAT_ASL_STORE_ERR_NOT_FOUND;
if (out_refs == NULL || out_refs_len == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
*out_refs = NULL;
*out_refs_len = 0u;
if (cache == NULL || sid.len == 0u || sid.data == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
if (!amduat_asl_materialization_build_sid_path(cache->root_path, sid, false,
&path)) {
return AMDUAT_ASL_STORE_ERR_IO;
}
fp = fopen(path, "rb");
if (fp == NULL) {
free(path);
return errno == ENOENT ? AMDUAT_ASL_STORE_ERR_NOT_FOUND
: AMDUAT_ASL_STORE_ERR_IO;
}
if (fseek(fp, 0, SEEK_END) != 0) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
file_size = ftell(fp);
if (file_size < 0) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
if (fseek(fp, 0, SEEK_SET) != 0) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
buffer = (uint8_t *)malloc((size_t)file_size);
if (buffer == NULL) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
if (fread(buffer, 1u, (size_t)file_size, fp) != (size_t)file_size) {
free(buffer);
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
fclose(fp);
free(path);
cur.data = buffer;
cur.len = (size_t)file_size;
cur.offset = 0u;
if (cur.len < AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN + 4u) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (memcmp(cur.data, k_amduat_asl_materialization_magic,
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN) != 0) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
cur.offset += AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN;
if (!amduat_asl_materialization_read_u32_le(&cur, &version) ||
version != AMDUAT_ASL_MATERIALIZATION_VERSION) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (!amduat_asl_materialization_read_u32_le(&cur, &sid_len)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (sid_len != sid.len || cur.len - cur.offset < sid_len) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (memcmp(cur.data + cur.offset, sid.data, sid.len) != 0) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
cur.offset += sid_len;
if (!amduat_asl_materialization_read_u32_le(&cur, &output_count)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
err = amduat_asl_materialization_decode_refs(&cur, output_count, out_refs,
out_refs_len);
if (err != AMDUAT_ASL_STORE_OK) {
goto cleanup;
}
if (!amduat_asl_materialization_read_u8(&cur, &flags)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (flags & AMDUAT_ASL_MATERIALIZATION_FLAG_TIMESTAMP) {
uint64_t ignored;
if (!amduat_asl_materialization_read_u64_le(&cur, &ignored)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
}
if (flags & AMDUAT_ASL_MATERIALIZATION_FLAG_PEL_VERSION) {
uint16_t ignored;
if (!amduat_asl_materialization_read_u16_le(&cur, &ignored)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
}
if (cur.offset != cur.len) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
err = AMDUAT_ASL_STORE_OK;
cleanup:
if (err != AMDUAT_ASL_STORE_OK) {
amduat_asl_materialization_refs_free(*out_refs, *out_refs_len);
*out_refs = NULL;
*out_refs_len = 0u;
}
free(buffer);
return err;
}
static bool amduat_asl_materialization_refs_equal(
const amduat_reference_t *a,
size_t a_len,
const amduat_reference_t *b,
size_t b_len) {
size_t i;
if (a_len != b_len) {
return false;
}
for (i = 0; i < a_len; ++i) {
if (!amduat_reference_eq(a[i], b[i])) {
return false;
}
}
return true;
}
static amduat_asl_store_error_t amduat_asl_materialization_write_record(
FILE *fp,
amduat_octets_t sid,
const amduat_reference_t *refs,
size_t refs_len) {
uint8_t header[AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN + 4u];
uint8_t flags = 0u;
uint32_t count_u32;
size_t i;
if (sid.len > UINT32_MAX || refs_len > UINT32_MAX) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
memcpy(header, k_amduat_asl_materialization_magic,
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN);
amduat_asl_materialization_store_u32_le(
header + AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN,
AMDUAT_ASL_MATERIALIZATION_VERSION);
if (fwrite(header, 1u, sizeof(header), fp) != sizeof(header)) {
return AMDUAT_ASL_STORE_ERR_IO;
}
amduat_asl_materialization_store_u32_le(header, (uint32_t)sid.len);
if (fwrite(header, 1u, 4u, fp) != 4u) {
return AMDUAT_ASL_STORE_ERR_IO;
}
if (sid.len != 0u &&
fwrite(sid.data, 1u, sid.len, fp) != sid.len) {
return AMDUAT_ASL_STORE_ERR_IO;
}
count_u32 = (uint32_t)refs_len;
amduat_asl_materialization_store_u32_le(header, count_u32);
if (fwrite(header, 1u, 4u, fp) != 4u) {
return AMDUAT_ASL_STORE_ERR_IO;
}
for (i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
uint8_t len_buf[4u];
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
amduat_asl_materialization_store_u32_le(len_buf,
(uint32_t)ref_bytes.len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
(ref_bytes.len != 0u &&
fwrite(ref_bytes.data, 1u, ref_bytes.len, fp) != ref_bytes.len)) {
free((void *)ref_bytes.data);
return AMDUAT_ASL_STORE_ERR_IO;
}
free((void *)ref_bytes.data);
}
if (fwrite(&flags, 1u, 1u, fp) != 1u) {
return AMDUAT_ASL_STORE_ERR_IO;
}
return AMDUAT_ASL_STORE_OK;
}
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_put(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
const amduat_reference_t *refs,
size_t refs_len) {
amduat_reference_t *existing_refs = NULL;
size_t existing_len = 0u;
amduat_asl_store_error_t err;
char *path = NULL;
char *tmp_path = NULL;
FILE *fp = NULL;
size_t tmp_len;
if (cache == NULL || sid.len == 0u || sid.data == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
if (refs_len != 0u && refs == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
err = amduat_asl_materialization_cache_fs_get(cache, sid,
&existing_refs, &existing_len);
if (err == AMDUAT_ASL_STORE_OK) {
bool same = amduat_asl_materialization_refs_equal(
existing_refs, existing_len, refs, refs_len);
amduat_asl_materialization_refs_free(existing_refs, existing_len);
if (!same) {
amduat_log(AMDUAT_LOG_ERROR,
"materialization cache SID collision with mismatched outputs");
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
return AMDUAT_ASL_STORE_OK;
}
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
return err;
}
if (!amduat_asl_materialization_build_sid_path(cache->root_path, sid, true,
&path)) {
return AMDUAT_ASL_STORE_ERR_IO;
}
tmp_len = strlen(path) + 5u;
tmp_path = (char *)malloc(tmp_len);
if (tmp_path == NULL) {
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
snprintf(tmp_path, tmp_len, "%s.tmp", path);
fp = fopen(tmp_path, "wb");
if (fp == NULL) {
free(tmp_path);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
/*
* Crash consistency: write + fsync temp file, rename, then fsync parent
* directory. Failure only affects cache persistence, not correctness.
*/
err = amduat_asl_materialization_write_record(fp, sid, refs, refs_len);
if (fflush(fp) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
if (err == AMDUAT_ASL_STORE_OK && fsync(fileno(fp)) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
if (fclose(fp) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
fp = NULL;
if (err == AMDUAT_ASL_STORE_OK && rename(tmp_path, path) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
if (err == AMDUAT_ASL_STORE_OK) {
char *parent_dir = amduat_asl_materialization_parent_dir(path);
if (parent_dir != NULL) {
if (!amduat_asl_materialization_fsync_directory(parent_dir)) {
amduat_log(AMDUAT_LOG_WARN,
"materialization cache fsync dir failed for %s",
parent_dir);
err = AMDUAT_ASL_STORE_ERR_IO;
}
free(parent_dir);
}
}
if (err != AMDUAT_ASL_STORE_OK) {
(void)remove(tmp_path);
}
#ifndef NDEBUG
if (err == AMDUAT_ASL_STORE_OK) {
amduat_reference_t *check_refs = NULL;
size_t check_len = 0u;
amduat_asl_store_error_t check_err =
amduat_asl_materialization_cache_fs_get(cache, sid, &check_refs,
&check_len);
assert(check_err == AMDUAT_ASL_STORE_OK);
assert(amduat_asl_materialization_refs_equal(
check_refs, check_len, refs, refs_len));
amduat_asl_materialization_refs_free(check_refs, check_len);
}
#endif
free(tmp_path);
free(path);
return err;
}

View file

@ -0,0 +1,753 @@
#include "amduat/asl/asl_pointer_fs.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/util/log.h"
#include <errno.h>
#include <fcntl.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
enum {
AMDUAT_ASL_POINTER_MAGIC_LEN = 8,
AMDUAT_ASL_POINTER_VERSION = 1
};
static const uint8_t k_amduat_asl_pointer_magic[AMDUAT_ASL_POINTER_MAGIC_LEN] = {
'A', 'S', 'L', 'P', 'T', 'R', '1', '\0'
};
enum { AMDUAT_ASL_POINTER_FLAG_HAS_EXPECTED = 1u << 0,
AMDUAT_ASL_POINTER_FLAG_HAS_PREV = 1u << 1 };
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_pointer_cursor_t;
static void amduat_asl_pointer_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_pointer_read_u32_le(amduat_asl_pointer_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4u;
return true;
}
static bool amduat_asl_pointer_read_u8(amduat_asl_pointer_cursor_t *cur,
uint8_t *out) {
if (cur->len - cur->offset < 1u) {
return false;
}
*out = cur->data[cur->offset++];
return true;
}
static bool amduat_asl_pointer_join_path(const char *base,
const char *segment,
char **out_path) {
size_t base_len;
size_t segment_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || segment[0] == '\0') {
return false;
}
base_len = strlen(base);
segment_len = strlen(segment);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + segment_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, segment_len);
offset += segment_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool amduat_asl_pointer_ensure_directory(const char *path) {
struct stat st;
if (path == NULL || path[0] == '\0') {
return false;
}
if (stat(path, &st) == 0) {
return S_ISDIR(st.st_mode);
}
if (errno != ENOENT) {
return false;
}
if (mkdir(path, 0755) != 0) {
if (errno == EEXIST) {
return stat(path, &st) == 0 && S_ISDIR(st.st_mode);
}
return false;
}
return true;
}
static char *amduat_asl_pointer_parent_dir(const char *path) {
const char *slash;
size_t len;
char *dir;
if (path == NULL) {
return NULL;
}
slash = strrchr(path, '/');
if (slash == NULL || slash == path) {
return NULL;
}
len = (size_t)(slash - path);
dir = (char *)malloc(len + 1u);
if (dir == NULL) {
return NULL;
}
memcpy(dir, path, len);
dir[len] = '\0';
return dir;
}
static bool amduat_asl_pointer_fsync_directory(const char *path) {
int fd;
int flags = O_RDONLY;
if (path == NULL) {
return false;
}
#ifdef O_DIRECTORY
flags |= O_DIRECTORY;
#endif
fd = open(path, flags);
if (fd < 0) {
return false;
}
if (fsync(fd) != 0) {
close(fd);
return false;
}
return close(fd) == 0;
}
bool amduat_asl_pointer_name_is_valid(const char *name) {
size_t len;
size_t seg_start = 0u;
if (name == NULL) {
return false;
}
len = strlen(name);
if (len == 0u || len > AMDUAT_ASL_POINTER_NAME_MAX) {
return false;
}
if (name[0] == '/' || name[len - 1u] == '/') {
return false;
}
for (size_t i = 0u; i < len; ++i) {
char c = name[i];
bool ok = (c >= 'A' && c <= 'Z') ||
(c >= 'a' && c <= 'z') ||
(c >= '0' && c <= '9') ||
c == '.' || c == '_' || c == '-' || c == '/';
if (!ok) {
return false;
}
if (c == '/') {
size_t seg_len = i - seg_start;
if (seg_len == 0u) {
return false;
}
if (seg_len == 2u && name[seg_start] == '.' &&
name[seg_start + 1u] == '.') {
return false;
}
seg_start = i + 1u;
}
}
{
size_t seg_len = len - seg_start;
if (seg_len == 0u) {
return false;
}
if (seg_len == 2u && name[seg_start] == '.' &&
name[seg_start + 1u] == '.') {
return false;
}
}
return true;
}
static bool amduat_asl_pointer_build_head_path(const char *root_path,
const char *name,
bool ensure_dirs,
char **out_path) {
char *pointers_path = NULL;
char *cursor_path = NULL;
char *head_path = NULL;
const char *segment_start;
const char *segment_end;
bool ok = false;
if (root_path == NULL || name == NULL || out_path == NULL) {
return false;
}
*out_path = NULL;
if (!amduat_asl_pointer_join_path(root_path, "pointers", &pointers_path)) {
return false;
}
if (ensure_dirs &&
!amduat_asl_pointer_ensure_directory(pointers_path)) {
free(pointers_path);
return false;
}
cursor_path = pointers_path;
pointers_path = NULL;
segment_start = name;
while (*segment_start != '\0') {
segment_end = strchr(segment_start, '/');
if (segment_end == NULL) {
segment_end = segment_start + strlen(segment_start);
}
{
size_t seg_len = (size_t)(segment_end - segment_start);
char *segment = (char *)malloc(seg_len + 1u);
char *next_path = NULL;
if (segment == NULL) {
goto cleanup;
}
memcpy(segment, segment_start, seg_len);
segment[seg_len] = '\0';
if (!amduat_asl_pointer_join_path(cursor_path, segment, &next_path)) {
free(segment);
goto cleanup;
}
free(segment);
if (ensure_dirs &&
!amduat_asl_pointer_ensure_directory(next_path)) {
free(next_path);
goto cleanup;
}
free(cursor_path);
cursor_path = next_path;
}
if (*segment_end == '\0') {
break;
}
segment_start = segment_end + 1u;
}
if (!amduat_asl_pointer_join_path(cursor_path, "head", &head_path)) {
goto cleanup;
}
*out_path = head_path;
head_path = NULL;
ok = true;
cleanup:
free(pointers_path);
free(cursor_path);
free(head_path);
return ok;
}
bool amduat_asl_pointer_store_init(amduat_asl_pointer_store_t *ps,
const char *root_path) {
size_t len;
if (ps == NULL || root_path == NULL) {
return false;
}
len = strlen(root_path);
if (len == 0u || len >= AMDUAT_ASL_POINTER_FS_ROOT_MAX) {
return false;
}
memcpy(ps->root_path, root_path, len);
ps->root_path[len] = '\0';
return true;
}
static amduat_asl_pointer_error_t amduat_asl_pointer_read_head(
const char *path,
const char *name,
bool *out_exists,
amduat_reference_t *out_ref,
amduat_reference_t *out_prev_ref,
bool *out_has_prev) {
FILE *fp;
uint8_t *buffer;
long file_size;
amduat_asl_pointer_cursor_t cur;
uint32_t version;
uint32_t name_len;
uint32_t ref_len;
uint32_t prev_len;
uint8_t flags;
amduat_octets_t ref_bytes;
amduat_octets_t prev_bytes;
if (out_exists == NULL || out_ref == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
*out_exists = false;
memset(out_ref, 0, sizeof(*out_ref));
if (out_prev_ref != NULL) {
memset(out_prev_ref, 0, sizeof(*out_prev_ref));
}
if (out_has_prev != NULL) {
*out_has_prev = false;
}
fp = fopen(path, "rb");
if (fp == NULL) {
return errno == ENOENT ? AMDUAT_ASL_POINTER_ERR_NOT_FOUND
: AMDUAT_ASL_POINTER_ERR_IO;
}
if (fseek(fp, 0, SEEK_END) != 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
file_size = ftell(fp);
if (file_size < 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
if (file_size == 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_NOT_FOUND;
}
if (fseek(fp, 0, SEEK_SET) != 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
buffer = (uint8_t *)malloc((size_t)file_size);
if (buffer == NULL) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
if (fread(buffer, 1u, (size_t)file_size, fp) != (size_t)file_size) {
free(buffer);
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
fclose(fp);
cur.data = buffer;
cur.len = (size_t)file_size;
cur.offset = 0u;
if (cur.len < AMDUAT_ASL_POINTER_MAGIC_LEN + 4u + 1u) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (memcmp(cur.data, k_amduat_asl_pointer_magic,
AMDUAT_ASL_POINTER_MAGIC_LEN) != 0) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += AMDUAT_ASL_POINTER_MAGIC_LEN;
if (!amduat_asl_pointer_read_u32_le(&cur, &version) ||
version != AMDUAT_ASL_POINTER_VERSION) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_read_u8(&cur, &flags)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_read_u32_le(&cur, &name_len)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (cur.len - cur.offset < name_len) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (strlen(name) != name_len ||
memcmp(cur.data + cur.offset, name, name_len) != 0) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += name_len;
if (!amduat_asl_pointer_read_u32_le(&cur, &ref_len)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (cur.len - cur.offset < ref_len || ref_len < 2u) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
ref_bytes = amduat_octets(cur.data + cur.offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, out_ref)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += ref_len;
if (!amduat_asl_pointer_read_u32_le(&cur, &prev_len)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (prev_len != 0u) {
if (!(flags & AMDUAT_ASL_POINTER_FLAG_HAS_PREV)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (cur.len - cur.offset < prev_len || prev_len < 2u) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
prev_bytes = amduat_octets(cur.data + cur.offset, prev_len);
if (out_prev_ref != NULL &&
!amduat_enc_asl1_core_decode_reference_v1(prev_bytes, out_prev_ref)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += prev_len;
if (out_has_prev != NULL) {
*out_has_prev = true;
}
}
if (cur.offset != cur.len) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
*out_exists = true;
free(buffer);
return AMDUAT_ASL_POINTER_OK;
}
static amduat_asl_pointer_error_t amduat_asl_pointer_write_head(
const char *path,
const char *name,
const amduat_reference_t *ref,
const amduat_reference_t *prev_ref,
bool has_prev) {
char *tmp_path;
char *parent_dir = NULL;
size_t tmp_len;
int tmp_fd;
FILE *fp;
uint8_t header[AMDUAT_ASL_POINTER_MAGIC_LEN + 4u + 1u];
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
amduat_octets_t prev_bytes = amduat_octets(NULL, 0u);
uint32_t name_len;
uint32_t ref_len;
uint32_t prev_len;
uint8_t flags = 0u;
amduat_asl_pointer_error_t err = AMDUAT_ASL_POINTER_OK;
if (ref == NULL || name == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
name_len = (uint32_t)strlen(name);
if (!amduat_enc_asl1_core_encode_reference_v1(*ref, &ref_bytes)) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
ref_len = (uint32_t)ref_bytes.len;
if (has_prev) {
if (prev_ref == NULL ||
!amduat_enc_asl1_core_encode_reference_v1(*prev_ref, &prev_bytes)) {
free((void *)ref_bytes.data);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
prev_len = (uint32_t)prev_bytes.len;
flags |= AMDUAT_ASL_POINTER_FLAG_HAS_PREV;
} else {
prev_len = 0u;
}
tmp_len = strlen(path) + sizeof(".tmp.XXXXXX");
tmp_path = (char *)malloc(tmp_len);
if (tmp_path == NULL) {
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
parent_dir = amduat_asl_pointer_parent_dir(path);
if (parent_dir == NULL || !amduat_asl_pointer_ensure_directory(parent_dir)) {
free(parent_dir);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
snprintf(tmp_path, tmp_len, "%s.tmp.XXXXXX", path);
tmp_fd = mkstemp(tmp_path);
if (tmp_fd < 0 && errno == ENOENT &&
amduat_asl_pointer_ensure_directory(parent_dir)) {
tmp_fd = mkstemp(tmp_path);
}
if (tmp_fd < 0) {
free(parent_dir);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
fp = fdopen(tmp_fd, "wb");
if (fp == NULL) {
close(tmp_fd);
(void)remove(tmp_path);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
memcpy(header, k_amduat_asl_pointer_magic, AMDUAT_ASL_POINTER_MAGIC_LEN);
amduat_asl_pointer_store_u32_le(
header + AMDUAT_ASL_POINTER_MAGIC_LEN, AMDUAT_ASL_POINTER_VERSION);
header[AMDUAT_ASL_POINTER_MAGIC_LEN + 4u] = flags;
if (fwrite(header, 1u, sizeof(header), fp) != sizeof(header)) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK) {
uint8_t len_buf[4u];
amduat_asl_pointer_store_u32_le(len_buf, name_len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
fwrite(name, 1u, name_len, fp) != name_len) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK) {
uint8_t len_buf[4u];
amduat_asl_pointer_store_u32_le(len_buf, ref_len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
fwrite(ref_bytes.data, 1u, ref_len, fp) != ref_len) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK) {
uint8_t len_buf[4u];
amduat_asl_pointer_store_u32_le(len_buf, prev_len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK && prev_len != 0u &&
fwrite(prev_bytes.data, 1u, prev_len, fp) != prev_len) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK && fflush(fp) != 0) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK && fsync(fileno(fp)) != 0) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (fclose(fp) != 0) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK && rename(tmp_path, path) != 0) {
if (errno == ENOENT && amduat_asl_pointer_ensure_directory(parent_dir) &&
rename(tmp_path, path) == 0) {
/* Recovered after recreating parent directory. */
} else {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK) {
if (!amduat_asl_pointer_fsync_directory(parent_dir)) {
amduat_log(AMDUAT_LOG_WARN,
"pointer fsync dir failed for %s", parent_dir);
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err != AMDUAT_ASL_POINTER_OK) {
(void)remove(tmp_path);
}
free(parent_dir);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return err;
}
amduat_asl_pointer_error_t amduat_asl_pointer_get(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool *out_exists,
amduat_reference_t *out_ref) {
char *head_path = NULL;
amduat_asl_pointer_error_t err;
if (ps == NULL || name == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return AMDUAT_ASL_POINTER_ERR_INVALID_NAME;
}
if (!amduat_asl_pointer_build_head_path(ps->root_path, name, false,
&head_path)) {
return AMDUAT_ASL_POINTER_ERR_IO;
}
err = amduat_asl_pointer_read_head(head_path, name, out_exists, out_ref,
NULL, NULL);
if (err == AMDUAT_ASL_POINTER_ERR_NOT_FOUND) {
*out_exists = false;
err = AMDUAT_ASL_POINTER_OK;
}
free(head_path);
return err;
}
amduat_asl_pointer_error_t amduat_asl_pointer_cas(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool expected_exists,
const amduat_reference_t *expected_ref,
const amduat_reference_t *new_ref,
bool *out_swapped) {
char *head_path = NULL;
int lock_fd = -1;
struct flock lock;
bool exists = false;
amduat_reference_t current_ref;
amduat_reference_t prev_ref;
bool has_prev = false;
char *lock_path = NULL;
amduat_asl_pointer_error_t err;
if (out_swapped == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
*out_swapped = false;
if (ps == NULL || name == NULL || new_ref == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return AMDUAT_ASL_POINTER_ERR_INVALID_NAME;
}
if (expected_exists && expected_ref == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_build_head_path(ps->root_path, name, true,
&head_path)) {
return AMDUAT_ASL_POINTER_ERR_IO;
}
lock_path = (char *)malloc(strlen(head_path) + sizeof(".lock"));
if (lock_path == NULL) {
free(head_path);
return AMDUAT_ASL_POINTER_ERR_IO;
}
snprintf(lock_path, strlen(head_path) + sizeof(".lock"), "%s.lock",
head_path);
/* Lock a stable sidecar file so lock ownership survives head-file renames. */
lock_fd = open(lock_path, O_RDWR | O_CREAT, 0644);
if (lock_fd < 0) {
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_ERR_IO;
}
memset(&lock, 0, sizeof(lock));
lock.l_type = F_WRLCK;
lock.l_whence = SEEK_SET;
lock.l_start = 0;
lock.l_len = 0;
if (fcntl(lock_fd, F_SETLKW, &lock) != 0) {
close(lock_fd);
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_ERR_IO;
}
err = amduat_asl_pointer_read_head(head_path, name, &exists, &current_ref,
&prev_ref, &has_prev);
if (err == AMDUAT_ASL_POINTER_ERR_NOT_FOUND) {
exists = false;
err = AMDUAT_ASL_POINTER_OK;
}
if (err != AMDUAT_ASL_POINTER_OK) {
close(lock_fd);
free(lock_path);
free(head_path);
return err;
}
if (expected_exists != exists) {
close(lock_fd);
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_OK;
}
if (expected_exists && !amduat_reference_eq(*expected_ref, current_ref)) {
close(lock_fd);
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_OK;
}
err = amduat_asl_pointer_write_head(head_path, name, new_ref,
expected_exists ? &current_ref : NULL,
expected_exists);
if (err == AMDUAT_ASL_POINTER_OK) {
*out_swapped = true;
}
if (exists) {
amduat_reference_free(&current_ref);
}
if (has_prev) {
amduat_reference_free(&prev_ref);
}
close(lock_fd);
free(lock_path);
free(head_path);
return err;
}

View file

@ -489,6 +489,30 @@ static amduat_asl_store_error_t amduat_asl_store_fs_put_impl(
return AMDUAT_ASL_STORE_OK; return AMDUAT_ASL_STORE_OK;
} }
static void amduat_asl_store_fs_fill_index_state(
amduat_asl_index_state_t *out_state) {
if (out_state == NULL) {
return;
}
out_state->snapshot_id = 0u;
out_state->log_position = 0u;
}
static amduat_asl_store_error_t amduat_asl_store_fs_put_indexed_impl(
void *ctx,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t err;
err = amduat_asl_store_fs_put_impl(ctx, artifact, out_ref);
if (err != AMDUAT_ASL_STORE_OK) {
return err;
}
amduat_asl_store_fs_fill_index_state(out_state);
return AMDUAT_ASL_STORE_OK;
}
static amduat_asl_store_error_t amduat_asl_store_fs_validate_config( static amduat_asl_store_error_t amduat_asl_store_fs_validate_config(
void *ctx, void *ctx,
amduat_asl_store_config_t config) { amduat_asl_store_config_t config) {
@ -511,6 +535,16 @@ static amduat_asl_store_error_t amduat_asl_store_fs_validate_config(
return AMDUAT_ASL_STORE_OK; return AMDUAT_ASL_STORE_OK;
} }
static bool amduat_asl_store_fs_current_state_impl(
void *ctx,
amduat_asl_index_state_t *out_state) {
if (ctx == NULL) {
return false;
}
amduat_asl_store_fs_fill_index_state(out_state);
return true;
}
static amduat_asl_store_error_t amduat_asl_store_fs_get_impl( static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
void *ctx, void *ctx,
amduat_reference_t ref, amduat_reference_t ref,
@ -672,6 +706,15 @@ static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
return AMDUAT_ASL_STORE_OK; return AMDUAT_ASL_STORE_OK;
} }
static amduat_asl_store_error_t amduat_asl_store_fs_get_indexed_impl(
void *ctx,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact) {
(void)state;
return amduat_asl_store_fs_get_impl(ctx, ref, out_artifact);
}
bool amduat_asl_store_fs_init(amduat_asl_store_fs_t *fs, bool amduat_asl_store_fs_init(amduat_asl_store_fs_t *fs,
amduat_asl_store_config_t config, amduat_asl_store_config_t config,
const char *root_path) { const char *root_path) {
@ -696,6 +739,9 @@ amduat_asl_store_ops_t amduat_asl_store_fs_ops(void) {
amduat_asl_store_ops_init(&ops); amduat_asl_store_ops_init(&ops);
ops.put = amduat_asl_store_fs_put_impl; ops.put = amduat_asl_store_fs_put_impl;
ops.get = amduat_asl_store_fs_get_impl; ops.get = amduat_asl_store_fs_get_impl;
ops.put_indexed = amduat_asl_store_fs_put_indexed_impl;
ops.get_indexed = amduat_asl_store_fs_get_indexed_impl;
ops.current_state = amduat_asl_store_fs_current_state_impl;
ops.validate_config = amduat_asl_store_fs_validate_config; ops.validate_config = amduat_asl_store_fs_validate_config;
return ops; return ops;
} }

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,415 @@
#include "asl_store_index_fs_layout.h"
#include <inttypes.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static bool amduat_asl_store_index_fs_layout_join(const char *base,
const char *suffix,
char **out_path) {
size_t base_len;
size_t suffix_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || suffix == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || suffix[0] == '\0') {
return false;
}
base_len = strlen(base);
suffix_len = strlen(suffix);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + suffix_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, suffix, suffix_len);
offset += suffix_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool amduat_asl_store_index_fs_layout_format_id(const char *prefix,
uint64_t id,
const char *suffix,
char **out_name) {
int needed;
char *buffer;
if (prefix == NULL || suffix == NULL || out_name == NULL) {
return false;
}
needed = snprintf(NULL, 0, "%s%016" PRIx64 "%s", prefix, id, suffix);
if (needed <= 0) {
return false;
}
buffer = (char *)malloc((size_t)needed + 1u);
if (buffer == NULL) {
return false;
}
snprintf(buffer, (size_t)needed + 1u, "%s%016" PRIx64 "%s", prefix, id,
suffix);
*out_name = buffer;
return true;
}
bool amduat_asl_store_index_fs_layout_build_index_path(const char *root_path,
char **out_path) {
return amduat_asl_store_index_fs_layout_join(root_path, "index", out_path);
}
bool amduat_asl_store_index_fs_layout_build_segments_path(
const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "segments", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_blocks_path(const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "blocks", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_log_path(const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "log.asl", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shards_path(
const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "shards", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *shards_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shards_path(root_path,
&shards_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("shard-",
(uint64_t)shard_id,
"",
&name)) {
free(shards_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(shards_path, name, out_path);
free(name);
free(shards_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_segments_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *shard_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_path(root_path,
shard_id,
&shard_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(shard_path, "segments", out_path);
free(shard_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_blocks_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *shard_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_path(root_path,
shard_id,
&shard_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(shard_path, "blocks", out_path);
free(shard_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_segment_meta_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *segments_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_segments_path(root_path,
shard_id,
&segments_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path,
"next_id",
out_path);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_segment_path(
const char *root_path,
uint16_t shard_id,
uint64_t segment_id,
char **out_path) {
char *segments_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_segments_path(root_path,
shard_id,
&segments_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("segment-",
segment_id,
".asl",
&name)) {
free(segments_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path, name, out_path);
free(name);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_block_path(
const char *root_path,
uint16_t shard_id,
uint64_t block_id,
char **out_path) {
char *blocks_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_blocks_path(root_path,
shard_id,
&blocks_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("block-",
block_id,
".asl",
&name)) {
free(blocks_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(blocks_path, name, out_path);
free(name);
free(blocks_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_snapshots_path(
const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "snapshots", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_snapshot_manifest_path(
const char *root_path,
uint64_t snapshot_id,
char **out_path) {
char *snapshots_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_snapshots_path(root_path,
&snapshots_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("snap-",
snapshot_id,
".bin",
&name)) {
free(snapshots_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(snapshots_path, name, out_path);
free(name);
free(snapshots_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_snapshot_latest_path(
const char *root_path,
char **out_path) {
char *snapshots_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_snapshots_path(root_path,
&snapshots_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(snapshots_path,
"latest",
out_path);
free(snapshots_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_segment_meta_path(
const char *root_path,
char **out_path) {
char *segments_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
&segments_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path,
"next_id",
out_path);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_segment_summary_path(
const char *root_path,
char **out_path) {
char *segments_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
&segments_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path,
"summary",
out_path);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_segment_path(
const char *root_path,
uint64_t segment_id,
char **out_path) {
char *segments_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
&segments_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("segment-",
segment_id,
".asl",
&name)) {
free(segments_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path, name, out_path);
free(name);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_block_path(
const char *root_path,
uint64_t block_id,
char **out_path) {
char *blocks_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_blocks_path(root_path,
&blocks_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("block-",
block_id,
".asl",
&name)) {
free(blocks_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(blocks_path, name, out_path);
free(name);
free(blocks_path);
return ok;
}

View file

@ -0,0 +1,95 @@
#ifndef AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H
#define AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
bool amduat_asl_store_index_fs_layout_build_index_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segments_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_blocks_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_log_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shards_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_segments_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_blocks_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_segment_meta_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_segment_path(
const char *root_path,
uint16_t shard_id,
uint64_t segment_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_block_path(
const char *root_path,
uint16_t shard_id,
uint64_t block_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_snapshots_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_snapshot_manifest_path(
const char *root_path,
uint64_t snapshot_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_snapshot_latest_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segment_meta_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segment_summary_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segment_path(
const char *root_path,
uint64_t segment_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_block_path(
const char *root_path,
uint64_t block_id,
char **out_path);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H */

File diff suppressed because it is too large Load diff

607
src/core/asl_collection.c Normal file
View file

@ -0,0 +1,607 @@
#include "amduat/asl/collection.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/util/log.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_COLLECTION_MAGIC_LEN = 8,
AMDUAT_ASL_COLLECTION_VERSION = 1,
AMDUAT_ASL_COLLECTION_READ_BATCH = 1024u
};
static const uint8_t k_amduat_asl_collection_magic[
AMDUAT_ASL_COLLECTION_MAGIC_LEN] = {
'A', 'S', 'L', 'C', 'O', 'L', '1', '\0'
};
static void amduat_asl_collection_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_collection_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_collection_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_collection_read_u64_le(const uint8_t *data,
size_t len,
size_t *offset,
uint64_t *out) {
if (len - *offset < 8u) {
return false;
}
*out = (uint64_t)data[*offset] |
((uint64_t)data[*offset + 1u] << 8) |
((uint64_t)data[*offset + 2u] << 16) |
((uint64_t)data[*offset + 3u] << 24) |
((uint64_t)data[*offset + 4u] << 32) |
((uint64_t)data[*offset + 5u] << 40) |
((uint64_t)data[*offset + 6u] << 48) |
((uint64_t)data[*offset + 7u] << 56);
*offset += 8u;
return true;
}
static bool amduat_asl_collection_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_collection_build_log_name(const char *name,
char **out_name) {
size_t name_len;
size_t total_len;
char *buffer;
size_t offset;
if (name == NULL || out_name == NULL) {
return false;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return false;
}
name_len = strlen(name);
total_len = 11u + name_len + 4u + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, "collection/", 11u);
offset += 11u;
memcpy(buffer + offset, name, name_len);
offset += name_len;
memcpy(buffer + offset, "/log", 4u);
offset += 4u;
buffer[offset] = '\0';
*out_name = buffer;
return true;
}
static bool amduat_asl_collection_build_head_name(const char *name,
char **out_name) {
size_t name_len;
size_t total_len;
char *buffer;
size_t offset;
if (name == NULL || out_name == NULL) {
return false;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return false;
}
name_len = strlen(name);
total_len = 11u + name_len + 5u + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, "collection/", 11u);
offset += 11u;
memcpy(buffer + offset, name, name_len);
offset += name_len;
memcpy(buffer + offset, "/head", 5u);
offset += 5u;
buffer[offset] = '\0';
*out_name = buffer;
return true;
}
static bool amduat_asl_collection_encode_snapshot_payload(
uint64_t snapshot_offset,
const amduat_reference_t *refs,
size_t refs_len,
amduat_octets_t *out_payload) {
size_t total_len = 0u;
uint8_t *buffer;
size_t offset = 0u;
if (out_payload == NULL) {
return false;
}
out_payload->data = NULL;
out_payload->len = 0u;
if (refs_len > UINT32_MAX) {
return false;
}
if (!amduat_asl_collection_add_size(
&total_len, AMDUAT_ASL_COLLECTION_MAGIC_LEN + 4u + 8u + 4u)) {
return false;
}
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return false;
}
if (!amduat_asl_collection_add_size(&total_len, 4u + ref_bytes.len)) {
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_collection_magic,
AMDUAT_ASL_COLLECTION_MAGIC_LEN);
offset += AMDUAT_ASL_COLLECTION_MAGIC_LEN;
amduat_asl_collection_store_u32_le(buffer + offset,
AMDUAT_ASL_COLLECTION_VERSION);
offset += 4u;
amduat_asl_collection_store_u64_le(buffer + offset, snapshot_offset);
offset += 8u;
amduat_asl_collection_store_u32_le(buffer + offset, (uint32_t)refs_len);
offset += 4u;
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
free(buffer);
return false;
}
amduat_asl_collection_store_u32_le(buffer + offset,
(uint32_t)ref_bytes.len);
offset += 4u;
memcpy(buffer + offset, ref_bytes.data, ref_bytes.len);
offset += ref_bytes.len;
free((void *)ref_bytes.data);
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_payload = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_collection_store_init(
amduat_asl_collection_store_t *collection_store,
const char *root_path,
amduat_asl_store_t *store) {
if (collection_store == NULL || root_path == NULL || store == NULL) {
return false;
}
collection_store->store = store;
if (!amduat_asl_pointer_store_init(&collection_store->pointer_store,
root_path)) {
return false;
}
if (!amduat_asl_log_store_init(&collection_store->log_store, root_path,
store, &collection_store->pointer_store)) {
return false;
}
return true;
}
amduat_asl_collection_error_t amduat_asl_collection_append(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
amduat_reference_t record_ref,
uint16_t kind,
amduat_octets_t actor,
uint64_t *out_offset) {
char *log_name = NULL;
amduat_asl_log_entry_t entry;
amduat_asl_store_error_t err;
if (collection_store == NULL || collection_store->store == NULL ||
collection_name == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
}
if (!amduat_asl_collection_build_log_name(collection_name, &log_name)) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
memset(&entry, 0, sizeof(entry));
entry.kind = kind;
entry.payload_ref = record_ref;
entry.has_actor = actor.len != 0u;
entry.actor = actor;
err = amduat_asl_log_append(&collection_store->log_store, log_name,
&entry, 1u, out_offset);
free(log_name);
if (err != AMDUAT_ASL_STORE_OK) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
return AMDUAT_ASL_COLLECTION_OK;
}
amduat_asl_collection_error_t amduat_asl_collection_snapshot(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t up_to_offset,
amduat_reference_t *out_snapshot_ref,
bool *out_swapped) {
char *log_name = NULL;
char *head_name = NULL;
amduat_reference_t *refs = NULL;
size_t refs_len = 0u;
uint64_t from = 0u;
bool end = false;
uint64_t next_offset = 0u;
amduat_asl_record_t snapshot_record;
amduat_octets_t payload = amduat_octets(NULL, 0u);
amduat_reference_t snapshot_ref;
amduat_asl_pointer_error_t ptr_err;
bool head_exists = false;
amduat_reference_t head_ref;
bool swapped = false;
if (out_snapshot_ref != NULL) {
out_snapshot_ref->hash_id = 0u;
out_snapshot_ref->digest = amduat_octets(NULL, 0u);
}
if (out_swapped != NULL) {
*out_swapped = false;
}
if (collection_store == NULL || collection_store->store == NULL ||
collection_name == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
}
if (!amduat_asl_collection_build_log_name(collection_name, &log_name) ||
!amduat_asl_collection_build_head_name(collection_name, &head_name)) {
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
while (!end && (up_to_offset == UINT64_MAX || from <= up_to_offset)) {
size_t limit = AMDUAT_ASL_COLLECTION_READ_BATCH;
amduat_asl_log_entry_t *entries = NULL;
size_t entries_len = 0u;
amduat_asl_store_error_t err = amduat_asl_log_read(
&collection_store->log_store, log_name, from, limit, &entries,
&entries_len, &next_offset, &end);
if (err != AMDUAT_ASL_STORE_OK) {
free(log_name);
free(head_name);
amduat_asl_collection_refs_free(refs, refs_len);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
for (size_t i = 0u; i < entries_len; ++i) {
uint64_t offset = from + i;
if (up_to_offset != UINT64_MAX && offset > up_to_offset) {
end = true;
break;
}
amduat_reference_t *next =
(amduat_reference_t *)realloc(
refs, (refs_len + 1u) * sizeof(*refs));
if (next == NULL) {
amduat_asl_log_entries_free(entries, entries_len);
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
refs = next;
refs[refs_len] = entries[i].payload_ref;
if (!amduat_octets_clone(entries[i].payload_ref.digest,
&refs[refs_len].digest)) {
amduat_asl_log_entries_free(entries, entries_len);
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
refs_len++;
}
amduat_asl_log_entries_free(entries, entries_len);
from = next_offset;
if (entries_len == 0u) {
break;
}
}
{
uint64_t snapshot_offset = from;
if (up_to_offset != UINT64_MAX &&
up_to_offset < UINT64_MAX &&
snapshot_offset > up_to_offset + 1u) {
snapshot_offset = up_to_offset + 1u;
}
if (!amduat_asl_collection_encode_snapshot_payload(
snapshot_offset, refs, refs_len, &payload)) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
}
if (payload.len == 0u || payload.data == NULL) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
{
amduat_asl_store_error_t err = amduat_asl_record_store_put(
collection_store->store,
amduat_octets("collection/snapshot",
strlen("collection/snapshot")),
payload,
&snapshot_ref);
amduat_octets_free(&payload);
if (err != AMDUAT_ASL_STORE_OK) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
}
ptr_err = amduat_asl_pointer_get(&collection_store->pointer_store, head_name,
&head_exists, &head_ref);
if (ptr_err != AMDUAT_ASL_POINTER_OK) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
ptr_err = amduat_asl_pointer_cas(&collection_store->pointer_store, head_name,
head_exists,
head_exists ? &head_ref : NULL,
&snapshot_ref, &swapped);
if (head_exists) {
amduat_reference_free(&head_ref);
}
if (ptr_err != AMDUAT_ASL_POINTER_OK) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
if (out_swapped != NULL) {
*out_swapped = swapped;
}
if (out_snapshot_ref != NULL) {
*out_snapshot_ref = snapshot_ref;
}
if (!swapped) {
amduat_log(AMDUAT_LOG_DEBUG, "collection snapshot CAS mismatch");
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_CAS_MISMATCH;
}
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_OK;
}
amduat_asl_collection_error_t amduat_asl_collection_read(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t from_offset,
size_t limit,
amduat_reference_t **out_record_refs,
size_t *out_len,
uint64_t *out_next_offset,
bool *out_end) {
char *log_name = NULL;
amduat_asl_log_entry_t *entries = NULL;
size_t entries_len = 0u;
amduat_asl_store_error_t err;
if (out_record_refs == NULL || out_len == NULL ||
out_next_offset == NULL || out_end == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
*out_record_refs = NULL;
*out_len = 0u;
if (collection_store == NULL || collection_store->store == NULL ||
collection_name == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
}
if (!amduat_asl_collection_build_log_name(collection_name, &log_name)) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
err = amduat_asl_log_read(&collection_store->log_store, log_name,
from_offset, limit, &entries, &entries_len,
out_next_offset, out_end);
free(log_name);
if (err != AMDUAT_ASL_STORE_OK) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
if (entries_len != 0u) {
amduat_reference_t *refs =
(amduat_reference_t *)calloc(entries_len, sizeof(*refs));
if (refs == NULL) {
amduat_asl_log_entries_free(entries, entries_len);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
for (size_t i = 0u; i < entries_len; ++i) {
refs[i].hash_id = entries[i].payload_ref.hash_id;
if (!amduat_octets_clone(entries[i].payload_ref.digest,
&refs[i].digest)) {
amduat_asl_collection_refs_free(refs, i);
amduat_asl_log_entries_free(entries, entries_len);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
}
*out_record_refs = refs;
*out_len = entries_len;
}
amduat_asl_log_entries_free(entries, entries_len);
return AMDUAT_ASL_COLLECTION_OK;
}
void amduat_asl_collection_refs_free(amduat_reference_t *refs,
size_t refs_len) {
if (refs == NULL) {
return;
}
for (size_t i = 0u; i < refs_len; ++i) {
amduat_reference_free(&refs[i]);
}
free(refs);
}
bool amduat_asl_collection_snapshot_payload_decode_v1(
amduat_octets_t payload,
amduat_asl_collection_snapshot_payload_t *out_snapshot) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t entry_count = 0u;
if (out_snapshot == NULL) {
return false;
}
out_snapshot->snapshot_offset = 0u;
out_snapshot->refs = NULL;
out_snapshot->refs_len = 0u;
if (payload.len < AMDUAT_ASL_COLLECTION_MAGIC_LEN + 4u + 8u + 4u) {
return false;
}
if (payload.data == NULL) {
return false;
}
if (memcmp(payload.data, k_amduat_asl_collection_magic,
AMDUAT_ASL_COLLECTION_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_COLLECTION_MAGIC_LEN;
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
&offset, &version) ||
version != AMDUAT_ASL_COLLECTION_VERSION) {
return false;
}
if (!amduat_asl_collection_read_u64_le(payload.data, payload.len,
&offset,
&out_snapshot->snapshot_offset)) {
return false;
}
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
&offset, &entry_count)) {
return false;
}
if (entry_count == 0u) {
return offset == payload.len;
}
if (entry_count > SIZE_MAX / sizeof(amduat_reference_t)) {
return false;
}
out_snapshot->refs =
(amduat_reference_t *)calloc(entry_count, sizeof(*out_snapshot->refs));
if (out_snapshot->refs == NULL) {
return false;
}
out_snapshot->refs_len = entry_count;
for (uint32_t i = 0u; i < entry_count; ++i) {
uint32_t ref_len = 0u;
amduat_octets_t ref_bytes;
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
&offset, &ref_len)) {
goto fail;
}
if (payload.len - offset < ref_len) {
goto fail;
}
ref_bytes = amduat_octets(payload.data + offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(
ref_bytes, &out_snapshot->refs[i])) {
goto fail;
}
offset += ref_len;
}
return offset == payload.len;
fail:
amduat_asl_collection_snapshot_payload_free(out_snapshot);
return false;
}
void amduat_asl_collection_snapshot_payload_free(
amduat_asl_collection_snapshot_payload_t *snapshot) {
if (snapshot == NULL || snapshot->refs == NULL) {
return;
}
for (size_t i = 0u; i < snapshot->refs_len; ++i) {
amduat_reference_free(&snapshot->refs[i]);
}
free(snapshot->refs);
snapshot->refs = NULL;
snapshot->refs_len = 0u;
}

1021
src/core/asl_log_store.c Normal file

File diff suppressed because it is too large Load diff

268
src/core/asl_record.c Normal file
View file

@ -0,0 +1,268 @@
#include "amduat/asl/record.h"
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_RECORD_MAGIC_LEN = 8,
AMDUAT_ASL_RECORD_VERSION = 1
};
static const uint8_t k_amduat_asl_record_magic[AMDUAT_ASL_RECORD_MAGIC_LEN] = {
'A', 'S', 'L', 'R', 'E', 'C', '1', '\0'
};
static void amduat_asl_record_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_record_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_record_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_record_schema_is_ascii(amduat_octets_t schema) {
if (schema.len == 0u || schema.data == NULL) {
return false;
}
for (size_t i = 0u; i < schema.len; ++i) {
uint8_t c = schema.data[i];
if (c < 0x20u || c > 0x7eu) {
return false;
}
}
return true;
}
bool amduat_asl_record_encode_v1(const amduat_asl_record_t *record,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t offset = 0u;
uint32_t schema_len;
uint32_t payload_len;
uint8_t flags = 0u;
if (out_bytes == NULL || record == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_record_schema_is_ascii(record->schema)) {
return false;
}
if (record->payload.len != 0u && record->payload.data == NULL) {
return false;
}
if (record->schema.len > UINT32_MAX || record->payload.len > UINT32_MAX) {
return false;
}
schema_len = (uint32_t)record->schema.len;
payload_len = (uint32_t)record->payload.len;
if (!amduat_asl_record_add_size(&total_len,
AMDUAT_ASL_RECORD_MAGIC_LEN + 4u) ||
!amduat_asl_record_add_size(&total_len, 4u + schema_len) ||
!amduat_asl_record_add_size(&total_len, 4u + payload_len) ||
!amduat_asl_record_add_size(&total_len, 1u)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_record_magic,
AMDUAT_ASL_RECORD_MAGIC_LEN);
offset += AMDUAT_ASL_RECORD_MAGIC_LEN;
amduat_asl_record_store_u32_le(buffer + offset, AMDUAT_ASL_RECORD_VERSION);
offset += 4u;
amduat_asl_record_store_u32_le(buffer + offset, schema_len);
offset += 4u;
memcpy(buffer + offset, record->schema.data, schema_len);
offset += schema_len;
amduat_asl_record_store_u32_le(buffer + offset, payload_len);
offset += 4u;
if (payload_len != 0u) {
memcpy(buffer + offset, record->payload.data, payload_len);
offset += payload_len;
}
buffer[offset++] = flags;
if (offset != total_len) {
free(buffer);
return false;
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_asl_record_decode_v1(amduat_octets_t bytes,
amduat_asl_record_t *out_record) {
size_t offset = 0u;
uint32_t version;
uint32_t schema_len;
uint32_t payload_len;
uint8_t flags;
uint8_t *schema_bytes;
uint8_t *payload_bytes;
if (out_record == NULL) {
return false;
}
out_record->schema = amduat_octets(NULL, 0u);
out_record->payload = amduat_octets(NULL, 0u);
if (bytes.len < AMDUAT_ASL_RECORD_MAGIC_LEN + 4u + 1u) {
return false;
}
if (bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_record_magic,
AMDUAT_ASL_RECORD_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_RECORD_MAGIC_LEN;
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
&version) ||
version != AMDUAT_ASL_RECORD_VERSION) {
return false;
}
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
&schema_len)) {
return false;
}
if (bytes.len - offset < schema_len) {
return false;
}
schema_bytes = NULL;
if (schema_len != 0u) {
schema_bytes = (uint8_t *)malloc(schema_len);
if (schema_bytes == NULL) {
return false;
}
memcpy(schema_bytes, bytes.data + offset, schema_len);
}
offset += schema_len;
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
&payload_len)) {
free(schema_bytes);
return false;
}
if (bytes.len - offset < payload_len + 1u) {
free(schema_bytes);
return false;
}
payload_bytes = NULL;
if (payload_len != 0u) {
payload_bytes = (uint8_t *)malloc(payload_len);
if (payload_bytes == NULL) {
free(schema_bytes);
return false;
}
memcpy(payload_bytes, bytes.data + offset, payload_len);
}
offset += payload_len;
flags = bytes.data[offset++];
(void)flags;
if (offset != bytes.len) {
free(schema_bytes);
free(payload_bytes);
return false;
}
out_record->schema = amduat_octets(schema_bytes, schema_len);
out_record->payload = amduat_octets(payload_bytes, payload_len);
return true;
}
void amduat_asl_record_free(amduat_asl_record_t *record) {
if (record == NULL) {
return;
}
amduat_octets_free(&record->schema);
amduat_octets_free(&record->payload);
}
amduat_asl_store_error_t amduat_asl_record_store_put(
amduat_asl_store_t *store,
amduat_octets_t schema,
amduat_octets_t payload,
amduat_reference_t *out_ref) {
amduat_asl_record_t record;
amduat_octets_t encoded = amduat_octets(NULL, 0u);
amduat_artifact_t artifact;
amduat_asl_store_error_t err;
if (store == NULL || out_ref == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
record.schema = schema;
record.payload = payload;
if (!amduat_asl_record_encode_v1(&record, &encoded)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
artifact = amduat_artifact_with_type(
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_RECORD_1));
err = amduat_asl_store_put(store, artifact, out_ref);
free((void *)encoded.data);
return err;
}
amduat_asl_store_error_t amduat_asl_record_store_get(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_record_t *out_record) {
amduat_artifact_t artifact;
amduat_asl_store_error_t err;
if (store == NULL || out_record == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
err = amduat_asl_store_get(store, ref, &artifact);
if (err != AMDUAT_ASL_STORE_OK) {
return err;
}
if (!artifact.has_type_tag ||
artifact.type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
amduat_artifact_free(&artifact);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
if (!amduat_asl_record_decode_v1(artifact.bytes, out_record)) {
amduat_artifact_free(&artifact);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
amduat_artifact_free(&artifact);
return AMDUAT_ASL_STORE_OK;
}

161
src/core/derivation_sid.c Normal file
View file

@ -0,0 +1,161 @@
#include "amduat/pel/derivation_sid.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/hash/asl1.h"
#include <assert.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
static bool amduat_derivation_sid_check_ref(amduat_reference_t ref) {
const amduat_hash_asl1_desc_t *desc;
size_t digest_len;
if (ref.digest.len != 0 && ref.digest.data == NULL) {
return false;
}
if (amduat_hash_asl1_is_reserved(ref.hash_id)) {
return false;
}
desc = amduat_hash_asl1_desc_lookup(ref.hash_id);
digest_len = ref.digest.len;
if (desc != NULL && desc->digest_len != 0) {
assert(desc->digest_len == digest_len);
if (desc->digest_len != digest_len) {
return false;
}
}
return true;
}
bool amduat_pel_derivation_sid_compute(
const amduat_pel_derivation_sid_input_t *in,
amduat_octets_t *out_sid) {
amduat_hash_asl1_stream_t stream;
uint8_t marker;
size_t i;
if (out_sid == NULL) {
return false;
}
out_sid->data = NULL;
out_sid->len = 0;
if (in == NULL) {
return false;
}
if (in->input_refs_len != 0 && in->input_refs == NULL) {
return false;
}
if (in->has_params_ref && !amduat_derivation_sid_check_ref(in->params_ref)) {
return false;
}
if (!amduat_derivation_sid_check_ref(in->program_ref)) {
return false;
}
for (i = 0; i < in->input_refs_len; ++i) {
if (!amduat_derivation_sid_check_ref(in->input_refs[i])) {
return false;
}
}
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
return false;
}
{
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
bool ok;
/* ReferenceBytes are self-delimiting because hash_id implies digest_len. */
ok = amduat_enc_asl1_core_encode_reference_v1(in->program_ref, &ref_bytes);
if (!ok ||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
amduat_hash_asl1_stream_destroy(&stream);
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
for (i = 0; i < in->input_refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
bool ok = amduat_enc_asl1_core_encode_reference_v1(in->input_refs[i],
&ref_bytes);
if (!ok ||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
amduat_hash_asl1_stream_destroy(&stream);
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
marker = in->has_params_ref ? 0x01u : 0x00u;
if (!amduat_hash_asl1_stream_update(&stream,
amduat_octets(&marker, 1u))) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (in->has_params_ref) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
bool ok = amduat_enc_asl1_core_encode_reference_v1(in->params_ref,
&ref_bytes);
if (!ok ||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
amduat_hash_asl1_stream_destroy(&stream);
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
/* ExecProfile is currently absent; presence must be explicitly marked. */
marker = in->has_exec_profile ? 0x01u : 0x00u;
if (!amduat_hash_asl1_stream_update(&stream,
amduat_octets(&marker, 1u))) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (in->has_exec_profile) {
if (in->exec_profile.len != 0 && in->exec_profile.data == NULL) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (!amduat_hash_asl1_stream_update(&stream, in->exec_profile)) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
}
{
const amduat_hash_asl1_desc_t *desc =
amduat_hash_asl1_desc_lookup(AMDUAT_HASH_ASL1_ID_SHA256);
uint8_t *digest;
size_t digest_len;
if (desc == NULL || desc->digest_len == 0u) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
digest_len = desc->digest_len;
digest = (uint8_t *)malloc(digest_len);
if (digest == NULL) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (!amduat_hash_asl1_stream_final(&stream, digest, digest_len)) {
free(digest);
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
amduat_hash_asl1_stream_destroy(&stream);
*out_sid = amduat_octets(digest, digest_len);
}
return true;
}

41
src/internal/log.c Normal file
View file

@ -0,0 +1,41 @@
#include "amduat/util/log.h"
#include <stdbool.h>
#include <stdio.h>
static const char *amduat_log_level_label(amduat_log_level_t level) {
switch (level) {
case AMDUAT_LOG_ERROR:
return "ERROR";
case AMDUAT_LOG_WARN:
return "WARN";
case AMDUAT_LOG_INFO:
return "INFO";
case AMDUAT_LOG_DEBUG:
return "DEBUG";
default:
return "LOG";
}
}
static bool amduat_log_debug_enabled(void) {
#ifdef AMDUAT_LOG_ENABLE_DEBUG
return true;
#else
return false;
#endif
}
void amduat_log(amduat_log_level_t level, const char *fmt, ...) {
va_list ap;
if (level == AMDUAT_LOG_DEBUG && !amduat_log_debug_enabled()) {
return;
}
fprintf(stderr, "%s: ", amduat_log_level_label(level));
va_start(ap, fmt);
vfprintf(stderr, fmt, ap);
va_end(ap);
fputc('\n', stderr);
}

View file

@ -0,0 +1,520 @@
#include "amduat/asl/collection_view.h"
#include "amduat/enc/asl1_core_codec.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN = 8,
AMDUAT_ASL_COLLECTION_VIEW_VERSION = 1,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN = 8,
AMDUAT_ASL_SNAPSHOT_INFO_VERSION = 1,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN = 8,
AMDUAT_ASL_LOG_RANGE_VERSION = 1
};
static const uint8_t k_amduat_asl_collection_view_magic[
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN] = {
'A', 'S', 'L', 'C', 'V', 'W', '1', '\0'
};
static const uint8_t k_amduat_asl_snapshot_info_magic[
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN] = {
'A', 'S', 'L', 'S', 'N', 'P', '1', '\0'
};
static const uint8_t k_amduat_asl_log_range_magic[
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN] = {
'A', 'S', 'L', 'R', 'N', 'G', '1', '\0'
};
static void amduat_asl_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_read_u64_le(const uint8_t *data,
size_t len,
size_t *offset,
uint64_t *out) {
if (len - *offset < 8u) {
return false;
}
*out = (uint64_t)data[*offset] |
((uint64_t)data[*offset + 1u] << 8) |
((uint64_t)data[*offset + 2u] << 16) |
((uint64_t)data[*offset + 3u] << 24) |
((uint64_t)data[*offset + 4u] << 32) |
((uint64_t)data[*offset + 5u] << 40) |
((uint64_t)data[*offset + 6u] << 48) |
((uint64_t)data[*offset + 7u] << 56);
*offset += 8u;
return true;
}
static bool amduat_asl_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_encode_refs(const amduat_reference_t *refs,
size_t refs_len,
size_t *out_total_len) {
size_t total = 0u;
if (refs_len > UINT32_MAX) {
return false;
}
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total, 4u + ref_bytes.len)) {
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
*out_total_len = total;
return true;
}
static bool amduat_asl_write_refs(uint8_t *buffer,
size_t total_len,
size_t *offset,
const amduat_reference_t *refs,
size_t refs_len) {
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return false;
}
if (total_len - *offset < 4u + ref_bytes.len) {
free((void *)ref_bytes.data);
return false;
}
amduat_asl_store_u32_le(buffer + *offset, (uint32_t)ref_bytes.len);
*offset += 4u;
memcpy(buffer + *offset, ref_bytes.data, ref_bytes.len);
*offset += ref_bytes.len;
free((void *)ref_bytes.data);
}
return true;
}
static bool amduat_asl_decode_refs(amduat_octets_t bytes,
size_t *offset,
uint32_t refs_len,
amduat_reference_t **out_refs,
size_t *out_refs_len) {
amduat_reference_t *refs = NULL;
if (out_refs == NULL || out_refs_len == NULL) {
return false;
}
*out_refs = NULL;
*out_refs_len = 0u;
if (refs_len == 0u) {
return true;
}
if (refs_len > SIZE_MAX / sizeof(*refs)) {
return false;
}
refs = (amduat_reference_t *)calloc(refs_len, sizeof(*refs));
if (refs == NULL) {
return false;
}
for (uint32_t i = 0u; i < refs_len; ++i) {
uint32_t ref_len = 0u;
amduat_octets_t ref_bytes;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, offset, &ref_len)) {
goto fail;
}
if (bytes.len - *offset < ref_len) {
goto fail;
}
ref_bytes = amduat_octets(bytes.data + *offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, &refs[i])) {
goto fail;
}
*offset += ref_len;
}
*out_refs = refs;
*out_refs_len = refs_len;
return true;
fail:
for (uint32_t i = 0u; i < refs_len; ++i) {
amduat_reference_free(&refs[i]);
}
free(refs);
return false;
}
bool amduat_asl_collection_snapshot_info_encode_v1(
const amduat_asl_collection_snapshot_info_t *info,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t refs_len_bytes = 0u;
size_t offset = 0u;
if (out_bytes == NULL || info == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_add_size(&total_len,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN + 4u + 8u + 4u)) {
return false;
}
if (!amduat_asl_encode_refs(info->refs, info->refs_len, &refs_len_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_snapshot_info_magic,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN);
offset += AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN;
amduat_asl_store_u32_le(buffer + offset, AMDUAT_ASL_SNAPSHOT_INFO_VERSION);
offset += 4u;
amduat_asl_store_u64_le(buffer + offset, info->snapshot_at_offset);
offset += 8u;
amduat_asl_store_u32_le(buffer + offset, (uint32_t)info->refs_len);
offset += 4u;
if (!amduat_asl_write_refs(buffer, total_len, &offset,
info->refs, info->refs_len)) {
free(buffer);
return false;
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_collection_snapshot_info_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_snapshot_info_t *out_info) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t refs_len = 0u;
if (out_info == NULL) {
return false;
}
out_info->snapshot_at_offset = 0u;
out_info->refs = NULL;
out_info->refs_len = 0u;
if (bytes.len < AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN + 4u + 8u + 4u ||
bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_snapshot_info_magic,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_SNAPSHOT_INFO_VERSION) {
return false;
}
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_info->snapshot_at_offset)) {
return false;
}
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
return false;
}
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
&out_info->refs, &out_info->refs_len)) {
return false;
}
return offset == bytes.len;
}
void amduat_asl_collection_snapshot_info_free(
amduat_asl_collection_snapshot_info_t *info) {
if (info == NULL || info->refs == NULL) {
return;
}
for (size_t i = 0u; i < info->refs_len; ++i) {
amduat_reference_free(&info->refs[i]);
}
free(info->refs);
info->refs = NULL;
info->refs_len = 0u;
}
bool amduat_asl_log_range_encode_v1(const amduat_asl_log_range_t *range,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t refs_len_bytes = 0u;
size_t offset = 0u;
if (out_bytes == NULL || range == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_add_size(&total_len,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN + 4u + 8u + 8u + 4u)) {
return false;
}
if (!amduat_asl_encode_refs(range->refs, range->refs_len, &refs_len_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_log_range_magic,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN);
offset += AMDUAT_ASL_LOG_RANGE_MAGIC_LEN;
amduat_asl_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_RANGE_VERSION);
offset += 4u;
amduat_asl_store_u64_le(buffer + offset, range->start_offset);
offset += 8u;
amduat_asl_store_u64_le(buffer + offset, range->next_offset);
offset += 8u;
amduat_asl_store_u32_le(buffer + offset, (uint32_t)range->refs_len);
offset += 4u;
if (!amduat_asl_write_refs(buffer, total_len, &offset,
range->refs, range->refs_len)) {
free(buffer);
return false;
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_log_range_decode_v1(amduat_octets_t bytes,
amduat_asl_log_range_t *out_range) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t refs_len = 0u;
if (out_range == NULL) {
return false;
}
out_range->start_offset = 0u;
out_range->next_offset = 0u;
out_range->refs = NULL;
out_range->refs_len = 0u;
if (bytes.len <
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN + 4u + 8u + 8u + 4u ||
bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_log_range_magic,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_LOG_RANGE_MAGIC_LEN;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_LOG_RANGE_VERSION) {
return false;
}
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_range->start_offset) ||
!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_range->next_offset)) {
return false;
}
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
return false;
}
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
&out_range->refs, &out_range->refs_len)) {
return false;
}
return offset == bytes.len;
}
void amduat_asl_log_range_free(amduat_asl_log_range_t *range) {
if (range == NULL || range->refs == NULL) {
return;
}
for (size_t i = 0u; i < range->refs_len; ++i) {
amduat_reference_free(&range->refs[i]);
}
free(range->refs);
range->refs = NULL;
range->refs_len = 0u;
}
bool amduat_asl_collection_view_encode_v1(
const amduat_asl_collection_view_t *view,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t refs_len_bytes = 0u;
size_t offset = 0u;
if (out_bytes == NULL || view == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_add_size(&total_len,
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN + 4u +
8u + 8u + 4u)) {
return false;
}
if (!amduat_asl_encode_refs(view->refs, view->refs_len, &refs_len_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_collection_view_magic,
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN);
offset += AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN;
amduat_asl_store_u32_le(buffer + offset,
AMDUAT_ASL_COLLECTION_VIEW_VERSION);
offset += 4u;
amduat_asl_store_u64_le(buffer + offset, view->computed_from_offset);
offset += 8u;
amduat_asl_store_u64_le(buffer + offset, view->computed_up_to_offset);
offset += 8u;
amduat_asl_store_u32_le(buffer + offset, (uint32_t)view->refs_len);
offset += 4u;
if (!amduat_asl_write_refs(buffer, total_len, &offset,
view->refs, view->refs_len)) {
free(buffer);
return false;
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_collection_view_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_view_t *out_view) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t refs_len = 0u;
if (out_view == NULL) {
return false;
}
out_view->computed_from_offset = 0u;
out_view->computed_up_to_offset = 0u;
out_view->refs = NULL;
out_view->refs_len = 0u;
if (bytes.len <
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN + 4u + 8u + 8u + 4u ||
bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_collection_view_magic,
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_COLLECTION_VIEW_VERSION) {
return false;
}
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_view->computed_from_offset) ||
!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_view->computed_up_to_offset)) {
return false;
}
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
return false;
}
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
&out_view->refs, &out_view->refs_len)) {
return false;
}
return offset == bytes.len;
}
void amduat_asl_collection_view_free(amduat_asl_collection_view_t *view) {
if (view == NULL || view->refs == NULL) {
return;
}
for (size_t i = 0u; i < view->refs_len; ++i) {
amduat_reference_free(&view->refs[i]);
}
free(view->refs);
view->refs = NULL;
view->refs_len = 0u;
}

View file

@ -0,0 +1,158 @@
#include "amduat/asl/index_accel.h"
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE = 4,
AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE = 5
};
static uint64_t amduat_asl_index_accel_fnv1a64_update(uint64_t hash,
const uint8_t *data,
size_t len) {
size_t i;
if (data == NULL || len == 0u) {
return hash;
}
for (i = 0u; i < len; ++i) {
hash ^= data[i];
hash *= 1099511628211ull;
}
return hash;
}
static void amduat_asl_index_accel_store_u16_le(uint8_t *out,
uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_index_accel_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_index_accel_hash_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
uint64_t *out_hash) {
uint8_t header[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE];
uint8_t tag_buf[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE];
uint16_t digest_len;
uint64_t hash;
if (out_hash == NULL) {
return false;
}
if (ref.digest.len != 0u && ref.digest.data == NULL) {
return false;
}
if (ref.digest.len > UINT16_MAX) {
return false;
}
digest_len = (uint16_t)ref.digest.len;
amduat_asl_index_accel_store_u16_le(header, ref.hash_id);
amduat_asl_index_accel_store_u16_le(header + 2u, digest_len);
tag_buf[0] = has_type_tag ? 1u : 0u;
amduat_asl_index_accel_store_u32_le(
tag_buf + 1u, has_type_tag ? type_tag.tag_id : 0u);
hash = 14695981039346656037ull;
hash = amduat_asl_index_accel_fnv1a64_update(hash, header, sizeof(header));
hash = amduat_asl_index_accel_fnv1a64_update(
hash, ref.digest.data, ref.digest.len);
hash = amduat_asl_index_accel_fnv1a64_update(hash, tag_buf, sizeof(tag_buf));
*out_hash = hash;
return true;
}
bool amduat_asl_index_accel_routing_key_from_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
amduat_octets_t *out_key) {
size_t total_len;
uint8_t *buffer;
uint8_t tag_flag;
uint16_t digest_len;
if (out_key == NULL) {
return false;
}
*out_key = amduat_octets(NULL, 0u);
if (ref.digest.len != 0u && ref.digest.data == NULL) {
return false;
}
if (ref.digest.len > UINT16_MAX) {
return false;
}
digest_len = (uint16_t)ref.digest.len;
total_len = AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE +
ref.digest.len + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE;
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
amduat_asl_index_accel_store_u16_le(buffer, ref.hash_id);
amduat_asl_index_accel_store_u16_le(buffer + 2u, digest_len);
if (ref.digest.len != 0u) {
memcpy(buffer + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE,
ref.digest.data,
ref.digest.len);
}
tag_flag = has_type_tag ? 1u : 0u;
buffer[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE + ref.digest.len] =
tag_flag;
amduat_asl_index_accel_store_u32_le(
buffer + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE + ref.digest.len +
1u,
has_type_tag ? type_tag.tag_id : 0u);
*out_key = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_index_accel_routing_key_hash(amduat_octets_t key,
uint64_t *out_hash) {
uint64_t hash;
if (out_hash == NULL) {
return false;
}
if (key.len != 0u && key.data == NULL) {
return false;
}
hash = 14695981039346656037ull;
hash = amduat_asl_index_accel_fnv1a64_update(hash, key.data, key.len);
*out_hash = hash;
return true;
}
uint16_t amduat_asl_index_accel_shard_for_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
uint16_t shard_count) {
uint64_t hash = 0u;
if (shard_count == 0u) {
return 0u;
}
if (!amduat_asl_index_accel_hash_ref(ref, has_type_tag, type_tag, &hash)) {
return 0u;
}
return (uint16_t)(hash % shard_count);
}

View file

@ -0,0 +1,152 @@
#include "amduat/asl/index_bloom.h"
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
static uint64_t amduat_asl_index_bloom_hash_update(uint64_t hash,
const uint8_t *data,
size_t len) {
size_t i;
if (data == NULL || len == 0u) {
return hash;
}
for (i = 0u; i < len; ++i) {
hash ^= (uint64_t)data[i];
hash *= 1099511628211ull;
}
return hash;
}
static bool amduat_asl_index_bloom_hash_pair(amduat_hash_id_t hash_id,
amduat_octets_t digest,
uint64_t *out_h1,
uint64_t *out_h2) {
uint64_t hash;
uint8_t header[4];
uint16_t digest_len;
if (out_h1 == NULL || out_h2 == NULL) {
return false;
}
if (digest.len != 0u && digest.data == NULL) {
return false;
}
if (digest.len > UINT16_MAX) {
return false;
}
digest_len = (uint16_t)digest.len;
header[0] = (uint8_t)(hash_id & 0xffu);
header[1] = (uint8_t)((hash_id >> 8) & 0xffu);
header[2] = (uint8_t)(digest_len & 0xffu);
header[3] = (uint8_t)((digest_len >> 8) & 0xffu);
hash = 14695981039346656037ull ^ 0x9e3779b97f4a7c15ull;
hash = amduat_asl_index_bloom_hash_update(hash, header, sizeof(header));
hash = amduat_asl_index_bloom_hash_update(hash, digest.data, digest.len);
*out_h1 = hash;
hash = 14695981039346656037ull ^ 0xbf58476d1ce4e5b9ull;
hash = amduat_asl_index_bloom_hash_update(hash, header, sizeof(header));
hash = amduat_asl_index_bloom_hash_update(hash, digest.data, digest.len);
if (hash == 0u) {
hash = 0x94d049bb133111ebull;
}
*out_h2 = hash;
return true;
}
static void amduat_asl_index_bloom_set_bit(uint8_t *data, size_t bit) {
data[bit >> 3] |= (uint8_t)(1u << (bit & 7u));
}
static bool amduat_asl_index_bloom_test_bit(const uint8_t *data, size_t bit) {
return (data[bit >> 3] & (uint8_t)(1u << (bit & 7u))) != 0u;
}
bool amduat_asl_index_bloom_init(amduat_octets_t *out_bloom) {
uint8_t *data;
if (out_bloom == NULL) {
return false;
}
*out_bloom = amduat_octets(NULL, 0u);
if (AMDUAT_ASL_INDEX_BLOOM_BYTES == 0u) {
return true;
}
data = (uint8_t *)calloc(1u, AMDUAT_ASL_INDEX_BLOOM_BYTES);
if (data == NULL) {
return false;
}
*out_bloom = amduat_octets(data, AMDUAT_ASL_INDEX_BLOOM_BYTES);
return true;
}
bool amduat_asl_index_bloom_add(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest) {
uint64_t h1;
uint64_t h2;
size_t i;
size_t bit_count;
uint8_t *data;
if (bloom.len == 0u) {
return false;
}
if (bloom.data == NULL) {
return false;
}
bit_count = bloom.len * 8u;
if (bit_count == 0u) {
return false;
}
if (!amduat_asl_index_bloom_hash_pair(hash_id, digest, &h1, &h2)) {
return false;
}
data = (uint8_t *)bloom.data;
for (i = 0u; i < AMDUAT_ASL_INDEX_BLOOM_HASHES; ++i) {
uint64_t mix = h1 + (uint64_t)i * h2;
size_t bit = (size_t)(mix % (uint64_t)bit_count);
amduat_asl_index_bloom_set_bit(data, bit);
}
return true;
}
bool amduat_asl_index_bloom_maybe_contains(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest) {
uint64_t h1;
uint64_t h2;
size_t i;
size_t bit_count;
if (bloom.len == 0u || bloom.data == NULL) {
return true;
}
bit_count = bloom.len * 8u;
if (bit_count == 0u) {
return true;
}
if (!amduat_asl_index_bloom_hash_pair(hash_id, digest, &h1, &h2)) {
return true;
}
for (i = 0u; i < AMDUAT_ASL_INDEX_BLOOM_HASHES; ++i) {
uint64_t mix = h1 + (uint64_t)i * h2;
size_t bit = (size_t)(mix % (uint64_t)bit_count);
if (!amduat_asl_index_bloom_test_bit(bloom.data, bit)) {
return false;
}
}
return true;
}

View file

@ -0,0 +1,341 @@
#include "amduat/asl/index_replay.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_replay_cursor_t;
static bool amduat_asl_replay_read_u16_le(amduat_asl_replay_cursor_t *cur,
uint16_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 2) {
return false;
}
data = cur->data + cur->offset;
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
cur->offset += 2;
return true;
}
static bool amduat_asl_replay_read_u32_le(amduat_asl_replay_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4;
return true;
}
static bool amduat_asl_replay_read_u64_le(amduat_asl_replay_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8;
return true;
}
static bool amduat_asl_replay_parse_artifact_ref(
amduat_asl_replay_cursor_t *cur,
amduat_reference_t *out_ref) {
uint32_t hash_id_raw;
uint16_t digest_len;
uint16_t reserved0;
const uint8_t *digest;
if (!amduat_asl_replay_read_u32_le(cur, &hash_id_raw) ||
!amduat_asl_replay_read_u16_le(cur, &digest_len) ||
!amduat_asl_replay_read_u16_le(cur, &reserved0)) {
return false;
}
if (hash_id_raw > UINT16_MAX || digest_len == 0 || reserved0 != 0) {
return false;
}
if (cur->len - cur->offset < digest_len) {
return false;
}
digest = cur->data + cur->offset;
cur->offset += digest_len;
*out_ref = amduat_reference((amduat_hash_id_t)hash_id_raw,
amduat_octets(digest, digest_len));
return true;
}
static bool amduat_asl_replay_parse_segment_seal(
amduat_octets_t payload,
amduat_asl_segment_seal_t *out) {
amduat_asl_replay_cursor_t cur;
uint64_t segment_id;
if (payload.len != 8u + 32u || payload.data == NULL || out == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_read_u64_le(&cur, &segment_id)) {
return false;
}
if (cur.len - cur.offset < 32) {
return false;
}
out->segment_id = segment_id;
memcpy(out->segment_hash, cur.data + cur.offset, 32);
return true;
}
static bool amduat_asl_replay_parse_tombstone(
amduat_octets_t payload,
amduat_reference_t *out_ref) {
amduat_asl_replay_cursor_t cur;
uint32_t scope;
uint32_t reason;
if (payload.len == 0 || payload.data == NULL || out_ref == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_parse_artifact_ref(&cur, out_ref)) {
return false;
}
if (!amduat_asl_replay_read_u32_le(&cur, &scope) ||
!amduat_asl_replay_read_u32_le(&cur, &reason)) {
return false;
}
(void)scope;
(void)reason;
return cur.offset == cur.len;
}
static bool amduat_asl_replay_parse_tombstone_lift(
amduat_octets_t payload,
amduat_reference_t *out_ref,
uint64_t *out_logseq) {
amduat_asl_replay_cursor_t cur;
uint64_t tombstone_logseq;
if (payload.len == 0 || payload.data == NULL || out_ref == NULL ||
out_logseq == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_parse_artifact_ref(&cur, out_ref) ||
!amduat_asl_replay_read_u64_le(&cur, &tombstone_logseq)) {
return false;
}
*out_logseq = tombstone_logseq;
return cur.offset == cur.len;
}
static bool amduat_asl_replay_parse_snapshot_anchor(
amduat_octets_t payload,
amduat_asl_snapshot_id_t *out_snapshot_id) {
amduat_asl_replay_cursor_t cur;
uint64_t snapshot_id;
if (payload.len != 8u + 32u || payload.data == NULL ||
out_snapshot_id == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_read_u64_le(&cur, &snapshot_id)) {
return false;
}
*out_snapshot_id = snapshot_id;
return true;
}
static bool amduat_asl_replay_update_segment(
amduat_asl_replay_state_t *state,
const amduat_asl_segment_seal_t *seal) {
size_t i;
amduat_asl_segment_seal_t *next;
for (i = 0; i < state->segments_len; ++i) {
if (state->segments[i].segment_id == seal->segment_id) {
memcpy(state->segments[i].segment_hash, seal->segment_hash, 32);
return true;
}
}
next = (amduat_asl_segment_seal_t *)realloc(
state->segments,
(state->segments_len + 1u) * sizeof(*state->segments));
if (next == NULL) {
return false;
}
state->segments = next;
state->segments[state->segments_len] = *seal;
state->segments_len += 1u;
return true;
}
static bool amduat_asl_replay_add_tombstone(
amduat_asl_replay_state_t *state,
amduat_reference_t ref,
uint64_t logseq) {
amduat_asl_tombstone_entry_t *next;
amduat_reference_t stored;
if (!amduat_reference_clone(ref, &stored)) {
return false;
}
next = (amduat_asl_tombstone_entry_t *)realloc(
state->tombstones,
(state->tombstones_len + 1u) * sizeof(*state->tombstones));
if (next == NULL) {
amduat_reference_free(&stored);
return false;
}
state->tombstones = next;
state->tombstones[state->tombstones_len].ref = stored;
state->tombstones[state->tombstones_len].tombstone_logseq = logseq;
state->tombstones_len += 1u;
return true;
}
static void amduat_asl_replay_remove_tombstone(
amduat_asl_replay_state_t *state,
amduat_reference_t ref,
uint64_t tombstone_logseq) {
size_t i;
for (i = 0; i < state->tombstones_len; ++i) {
if (state->tombstones[i].tombstone_logseq != tombstone_logseq) {
continue;
}
if (!amduat_reference_eq(state->tombstones[i].ref, ref)) {
continue;
}
amduat_reference_free(&state->tombstones[i].ref);
if (i + 1u < state->tombstones_len) {
state->tombstones[i] = state->tombstones[state->tombstones_len - 1u];
}
state->tombstones_len -= 1u;
return;
}
}
bool amduat_asl_replay_init(amduat_asl_replay_state_t *out) {
if (out == NULL) {
return false;
}
out->segments = NULL;
out->segments_len = 0;
out->tombstones = NULL;
out->tombstones_len = 0;
out->state.snapshot_id = 0;
out->state.log_position = 0;
return true;
}
void amduat_asl_replay_free(amduat_asl_replay_state_t *state) {
size_t i;
if (state == NULL) {
return;
}
free(state->segments);
state->segments = NULL;
state->segments_len = 0;
if (state->tombstones != NULL) {
for (i = 0; i < state->tombstones_len; ++i) {
amduat_reference_free(&state->tombstones[i].ref);
}
}
free(state->tombstones);
state->tombstones = NULL;
state->tombstones_len = 0;
state->state.snapshot_id = 0;
state->state.log_position = 0;
}
bool amduat_asl_replay_apply_log(
const amduat_asl_log_record_t *records,
size_t record_count,
amduat_asl_log_position_t log_position,
amduat_asl_replay_state_t *state) {
size_t i;
if (state == NULL) {
return false;
}
if (record_count != 0 && records == NULL) {
return false;
}
for (i = 0; i < record_count; ++i) {
const amduat_asl_log_record_t *record = &records[i];
amduat_asl_segment_seal_t seal;
amduat_reference_t ref;
uint64_t tombstone_logseq;
amduat_asl_snapshot_id_t snapshot_id;
if (record->logseq > log_position) {
break;
}
switch (record->record_type) {
case AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL:
if (!amduat_asl_replay_parse_segment_seal(record->payload, &seal)) {
return false;
}
if (!amduat_asl_replay_update_segment(state, &seal)) {
return false;
}
break;
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE:
if (!amduat_asl_replay_parse_tombstone(record->payload, &ref)) {
return false;
}
if (!amduat_asl_replay_add_tombstone(state, ref, record->logseq)) {
return false;
}
break;
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT:
if (!amduat_asl_replay_parse_tombstone_lift(record->payload,
&ref,
&tombstone_logseq)) {
return false;
}
amduat_asl_replay_remove_tombstone(state, ref, tombstone_logseq);
break;
case AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR:
if (!amduat_asl_replay_parse_snapshot_anchor(record->payload,
&snapshot_id)) {
return false;
}
state->state.snapshot_id = snapshot_id;
break;
default:
break;
}
}
state->state.log_position = log_position;
return true;
}

View file

@ -0,0 +1,487 @@
#include "amduat/asl/index_snapshot.h"
#include "amduat/asl/io.h"
#include "amduat/hash/asl1.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE = 40
};
static const uint8_t k_amduat_asl_snapshot_manifest_magic[
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN] = {'A', 'S', 'L', 'S',
'N', 'A', 'P', '1'};
static void amduat_asl_snapshot_store_u16_le(uint8_t *out,
uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_snapshot_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_snapshot_store_u64_le(uint8_t *out,
uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static uint16_t amduat_asl_snapshot_load_u16_le(const uint8_t *data) {
return (uint16_t)data[0] | ((uint16_t)data[1] << 8);
}
static uint32_t amduat_asl_snapshot_load_u32_le(const uint8_t *data) {
return (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
}
static uint64_t amduat_asl_snapshot_load_u64_le(const uint8_t *data) {
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
}
static bool amduat_asl_snapshot_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static int amduat_asl_snapshot_compare_segments(const void *lhs,
const void *rhs) {
const amduat_asl_segment_seal_t *left =
*(const amduat_asl_segment_seal_t *const *)lhs;
const amduat_asl_segment_seal_t *right =
*(const amduat_asl_segment_seal_t *const *)rhs;
if (left->segment_id < right->segment_id) {
return -1;
}
if (left->segment_id > right->segment_id) {
return 1;
}
return memcmp(left->segment_hash, right->segment_hash,
sizeof(left->segment_hash));
}
static int amduat_asl_snapshot_compare_tombstones(const void *lhs,
const void *rhs) {
const amduat_asl_tombstone_entry_t *left =
*(const amduat_asl_tombstone_entry_t *const *)lhs;
const amduat_asl_tombstone_entry_t *right =
*(const amduat_asl_tombstone_entry_t *const *)rhs;
size_t min_len;
int cmp;
if (left->ref.hash_id < right->ref.hash_id) {
return -1;
}
if (left->ref.hash_id > right->ref.hash_id) {
return 1;
}
if (left->ref.digest.len < right->ref.digest.len) {
return -1;
}
if (left->ref.digest.len > right->ref.digest.len) {
return 1;
}
min_len = left->ref.digest.len;
if (min_len != 0u) {
cmp = memcmp(left->ref.digest.data, right->ref.digest.data, min_len);
if (cmp != 0) {
return cmp;
}
}
if (left->tombstone_logseq < right->tombstone_logseq) {
return -1;
}
if (left->tombstone_logseq > right->tombstone_logseq) {
return 1;
}
return 0;
}
static bool amduat_asl_snapshot_hash_manifest(const uint8_t *bytes,
size_t len,
uint8_t out_hash[32]) {
if (out_hash == NULL) {
return true;
}
return amduat_hash_asl1_digest(AMDUAT_HASH_ASL1_ID_SHA256,
amduat_octets(bytes, len), out_hash, 32);
}
void amduat_asl_snapshot_manifest_free(
amduat_asl_snapshot_manifest_t *manifest) {
size_t i;
if (manifest == NULL) {
return;
}
free(manifest->segments);
manifest->segments = NULL;
manifest->segments_len = 0;
if (manifest->tombstones != NULL) {
for (i = 0; i < manifest->tombstones_len; ++i) {
amduat_reference_free(&manifest->tombstones[i].ref);
}
}
free(manifest->tombstones);
manifest->tombstones = NULL;
manifest->tombstones_len = 0;
}
bool amduat_asl_snapshot_manifest_write(
const char *path,
const amduat_asl_snapshot_manifest_t *manifest,
uint8_t out_hash[32]) {
size_t i;
size_t total_len;
size_t offset;
uint8_t *buffer;
amduat_asl_segment_seal_t **segment_order;
amduat_asl_tombstone_entry_t **tombstone_order;
if (path == NULL || manifest == NULL) {
return false;
}
if (manifest->segments_len != 0u && manifest->segments == NULL) {
return false;
}
if (manifest->tombstones_len != 0u && manifest->tombstones == NULL) {
return false;
}
if (manifest->segments_len > SIZE_MAX / AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
return false;
}
total_len = 0;
if (!amduat_asl_snapshot_add_size(&total_len,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE)) {
return false;
}
if (!amduat_asl_snapshot_add_size(
&total_len,
manifest->segments_len *
AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE)) {
return false;
}
for (i = 0; i < manifest->tombstones_len; ++i) {
const amduat_asl_tombstone_entry_t *entry = &manifest->tombstones[i];
if (entry->ref.digest.len == 0 ||
(entry->ref.digest.len != 0u && entry->ref.digest.data == NULL)) {
return false;
}
if (entry->ref.digest.len > UINT16_MAX) {
return false;
}
if (!amduat_asl_snapshot_add_size(&total_len,
16u + entry->ref.digest.len)) {
return false;
}
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
segment_order = NULL;
tombstone_order = NULL;
if (manifest->segments_len != 0u) {
segment_order = (amduat_asl_segment_seal_t **)calloc(
manifest->segments_len, sizeof(*segment_order));
if (segment_order == NULL) {
free(buffer);
return false;
}
for (i = 0; i < manifest->segments_len; ++i) {
segment_order[i] = &manifest->segments[i];
}
qsort(segment_order, manifest->segments_len, sizeof(*segment_order),
amduat_asl_snapshot_compare_segments);
}
if (manifest->tombstones_len != 0u) {
tombstone_order = (amduat_asl_tombstone_entry_t **)calloc(
manifest->tombstones_len, sizeof(*tombstone_order));
if (tombstone_order == NULL) {
free(segment_order);
free(buffer);
return false;
}
for (i = 0; i < manifest->tombstones_len; ++i) {
tombstone_order[i] = &manifest->tombstones[i];
}
qsort(tombstone_order, manifest->tombstones_len,
sizeof(*tombstone_order), amduat_asl_snapshot_compare_tombstones);
}
offset = 0;
memcpy(buffer + offset, k_amduat_asl_snapshot_manifest_magic,
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN);
offset += AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN;
amduat_asl_snapshot_store_u16_le(buffer + offset,
AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION);
offset += 2;
amduat_asl_snapshot_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_snapshot_store_u32_le(buffer + offset,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE);
offset += 4;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->snapshot_id);
offset += 8;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->anchor_logseq);
offset += 8;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->segments_len);
offset += 8;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->tombstones_len);
offset += 8;
amduat_asl_snapshot_store_u32_le(buffer + offset,
manifest->config.encoding_profile_id);
offset += 4;
amduat_asl_snapshot_store_u32_le(buffer + offset,
manifest->config.hash_id);
offset += 4;
memset(buffer + offset, 0,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE - offset);
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE;
for (i = 0; i < manifest->segments_len; ++i) {
const amduat_asl_segment_seal_t *entry = segment_order[i];
amduat_asl_snapshot_store_u64_le(buffer + offset, entry->segment_id);
offset += 8;
memcpy(buffer + offset, entry->segment_hash, sizeof(entry->segment_hash));
offset += sizeof(entry->segment_hash);
}
for (i = 0; i < manifest->tombstones_len; ++i) {
const amduat_asl_tombstone_entry_t *entry = tombstone_order[i];
amduat_asl_snapshot_store_u32_le(buffer + offset, entry->ref.hash_id);
offset += 4;
amduat_asl_snapshot_store_u16_le(buffer + offset,
(uint16_t)entry->ref.digest.len);
offset += 2;
amduat_asl_snapshot_store_u16_le(buffer + offset, 0);
offset += 2;
memcpy(buffer + offset, entry->ref.digest.data, entry->ref.digest.len);
offset += entry->ref.digest.len;
amduat_asl_snapshot_store_u64_le(buffer + offset,
entry->tombstone_logseq);
offset += 8;
}
free(segment_order);
free(tombstone_order);
if (offset != total_len) {
free(buffer);
return false;
}
if (!amduat_asl_snapshot_hash_manifest(buffer, total_len, out_hash)) {
free(buffer);
return false;
}
if (!amduat_asl_write_path(path, buffer, total_len)) {
free(buffer);
return false;
}
free(buffer);
return true;
}
bool amduat_asl_snapshot_manifest_read(
const char *path,
amduat_asl_snapshot_manifest_t *out_manifest,
uint8_t out_hash[32]) {
uint8_t *buffer;
size_t len;
size_t offset;
uint16_t version;
uint16_t reserved0;
uint32_t header_size;
uint64_t segment_count;
uint64_t tombstone_count;
size_t i;
if (path == NULL || out_manifest == NULL) {
return false;
}
memset(out_manifest, 0, sizeof(*out_manifest));
buffer = NULL;
len = 0u;
if (!amduat_asl_read_path(path, &buffer, &len)) {
return false;
}
if (len < AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE) {
free(buffer);
return false;
}
if (!amduat_asl_snapshot_hash_manifest(buffer, len, out_hash)) {
free(buffer);
return false;
}
if (memcmp(buffer, k_amduat_asl_snapshot_manifest_magic,
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN) != 0) {
free(buffer);
return false;
}
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN;
version = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
reserved0 = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
header_size = amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
if (version != AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION || reserved0 != 0 ||
header_size != AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE) {
free(buffer);
return false;
}
out_manifest->snapshot_id = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
out_manifest->anchor_logseq =
amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
segment_count = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
tombstone_count = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
out_manifest->config.encoding_profile_id =
amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
out_manifest->config.hash_id =
amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE;
if (segment_count > SIZE_MAX || tombstone_count > SIZE_MAX) {
free(buffer);
return false;
}
out_manifest->segments_len = (size_t)segment_count;
if (out_manifest->segments_len != 0u) {
if (out_manifest->segments_len >
SIZE_MAX / AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
free(buffer);
return false;
}
out_manifest->segments = (amduat_asl_segment_seal_t *)calloc(
out_manifest->segments_len, sizeof(*out_manifest->segments));
if (out_manifest->segments == NULL) {
free(buffer);
return false;
}
}
for (i = 0; i < out_manifest->segments_len; ++i) {
amduat_asl_segment_seal_t *entry = &out_manifest->segments[i];
if (len - offset < AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
entry->segment_id = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
memcpy(entry->segment_hash, buffer + offset, sizeof(entry->segment_hash));
offset += sizeof(entry->segment_hash);
}
out_manifest->tombstones_len = (size_t)tombstone_count;
if (out_manifest->tombstones_len != 0u) {
out_manifest->tombstones = (amduat_asl_tombstone_entry_t *)calloc(
out_manifest->tombstones_len, sizeof(*out_manifest->tombstones));
if (out_manifest->tombstones == NULL) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
}
for (i = 0; i < out_manifest->tombstones_len; ++i) {
amduat_asl_tombstone_entry_t *entry = &out_manifest->tombstones[i];
uint32_t hash_id;
uint16_t digest_len;
uint16_t tombstone_reserved;
amduat_reference_t ref;
uint64_t tombstone_logseq;
if (len - offset < 16) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
hash_id = amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
if (hash_id > UINT16_MAX) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
digest_len = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
tombstone_reserved = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
if (tombstone_reserved != 0 || digest_len == 0) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
if (len - offset < (size_t)digest_len + 8u) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
ref = amduat_reference((amduat_hash_id_t)hash_id,
amduat_octets(buffer + offset, digest_len));
offset += digest_len;
tombstone_logseq = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
if (!amduat_reference_clone(ref, &entry->ref)) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
entry->tombstone_logseq = tombstone_logseq;
}
if (offset != len) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
free(buffer);
return true;
}

110
src/near_core/asl/none.c Normal file
View file

@ -0,0 +1,110 @@
#include "amduat/asl/none.h"
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_NONE_MAGIC_LEN = 8,
AMDUAT_ASL_NONE_VERSION = 1
};
static const uint8_t k_amduat_asl_none_magic[AMDUAT_ASL_NONE_MAGIC_LEN] = {
'A', 'S', 'L', 'N', 'O', 'N', '1', '\0'
};
static void amduat_asl_none_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_none_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
bool amduat_asl_none_encode_v1(amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t offset = 0u;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
buffer = (uint8_t *)malloc(AMDUAT_ASL_NONE_MAGIC_LEN + 4u);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_none_magic, AMDUAT_ASL_NONE_MAGIC_LEN);
offset += AMDUAT_ASL_NONE_MAGIC_LEN;
amduat_asl_none_store_u32_le(buffer + offset, AMDUAT_ASL_NONE_VERSION);
offset += 4u;
out_bytes->data = buffer;
out_bytes->len = offset;
return true;
}
bool amduat_asl_none_decode_v1(amduat_octets_t bytes) {
size_t offset = 0u;
uint32_t version = 0u;
if (bytes.len != AMDUAT_ASL_NONE_MAGIC_LEN + 4u) {
return false;
}
if (bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_none_magic,
AMDUAT_ASL_NONE_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_NONE_MAGIC_LEN;
if (!amduat_asl_none_read_u32_le(bytes.data, bytes.len, &offset, &version)) {
return false;
}
if (version != AMDUAT_ASL_NONE_VERSION) {
return false;
}
return offset == bytes.len;
}
bool amduat_asl_none_is_artifact(const amduat_artifact_t *artifact) {
if (artifact == NULL) {
return false;
}
if (!artifact->has_type_tag ||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_NONE_1) {
return false;
}
return amduat_asl_none_decode_v1(artifact->bytes);
}
bool amduat_asl_none_artifact(amduat_artifact_t *out_artifact) {
amduat_octets_t bytes = amduat_octets(NULL, 0u);
if (out_artifact == NULL) {
return false;
}
if (!amduat_asl_none_encode_v1(&bytes)) {
return false;
}
*out_artifact =
amduat_artifact_with_type(bytes, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_NONE_1));
return true;
}

View file

@ -60,3 +60,122 @@ amduat_asl_store_error_t amduat_asl_store_get(amduat_asl_store_t *store,
} }
return store->ops.get(store->ctx, ref, out_artifact); return store->ops.get(store->ctx, ref, out_artifact);
} }
amduat_asl_store_error_t amduat_asl_store_put_indexed(
amduat_asl_store_t *store,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
amduat_asl_store_error_t store_err;
if (store == NULL || store->ops.put_indexed == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
store_err = store->ops.put_indexed(store->ctx, artifact, out_ref, out_state);
if (store_err != AMDUAT_ASL_STORE_OK) {
return store_err;
}
if (out_ref != NULL && out_ref->hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
return AMDUAT_ASL_STORE_OK;
}
amduat_asl_store_error_t amduat_asl_store_get_indexed(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.get_indexed == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
if (ref.hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.get_indexed(store->ctx, ref, state, out_artifact);
}
amduat_asl_store_error_t amduat_asl_store_tombstone(
amduat_asl_store_t *store,
amduat_reference_t ref,
uint32_t scope,
uint32_t reason_code,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.tombstone == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
if (ref.hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.tombstone(store->ctx, ref, scope, reason_code, out_state);
}
amduat_asl_store_error_t amduat_asl_store_tombstone_lift(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_log_position_t tombstone_logseq,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.tombstone_lift == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
if (ref.hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.tombstone_lift(store->ctx,
ref,
tombstone_logseq,
out_state);
}
amduat_asl_store_error_t amduat_asl_log_scan(
amduat_asl_store_t *store,
amduat_asl_log_record_t **out_records,
size_t *out_count) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.log_scan == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.log_scan(store->ctx, out_records, out_count);
}
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.current_state == NULL) {
return false;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return false;
}
return store->ops.current_state(store->ctx, out_state);
}

View file

@ -0,0 +1,846 @@
#include "amduat/enc/asl_core_index.h"
#include "amduat/hash/asl1.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_CORE_INDEX_MAGIC = 0x33305844494c5341ull
};
static void amduat_asl_core_index_store_u16_le(uint8_t *out,
uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_core_index_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_core_index_store_u64_le(uint8_t *out,
uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static uint16_t amduat_asl_core_index_load_u16_le(const uint8_t *data) {
return (uint16_t)data[0] | ((uint16_t)data[1] << 8);
}
static uint32_t amduat_asl_core_index_load_u32_le(const uint8_t *data) {
return (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
}
static uint64_t amduat_asl_core_index_load_u64_le(const uint8_t *data) {
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
}
static bool amduat_asl_core_index_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_core_index_is_aligned8(uint64_t value) {
return (value & 7u) == 0u;
}
static uint64_t amduat_asl_core_index_crc64(const uint8_t *data, size_t len) {
uint64_t crc = 0u;
size_t i;
for (i = 0; i < len; ++i) {
uint64_t bit;
uint8_t value = data[i];
crc ^= ((uint64_t)value) << 56;
for (bit = 0; bit < 8; ++bit) {
if (crc & 0x8000000000000000ull) {
crc = (crc << 1) ^ 0x42f0e1eba9ea3693ull;
} else {
crc <<= 1;
}
}
}
return crc;
}
static bool amduat_asl_core_index_validate_record(
const amduat_asl_index_record_t *record,
const amduat_asl_extent_record_t *extents,
size_t extent_count,
size_t *extent_cursor,
size_t *digest_cursor,
size_t digests_len,
uint8_t *max_visibility) {
size_t i;
size_t start;
uint64_t total_len;
bool is_tombstone;
const amduat_hash_asl1_desc_t *hash_desc;
if (record->reserved0 != 0 || record->reserved1 != 0) {
return false;
}
if ((record->flags & ~AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0) {
return false;
}
if (record->visibility > 1) {
return false;
}
if (record->has_cross_domain_source > 1) {
return false;
}
if (record->has_cross_domain_source == 0 &&
record->cross_domain_source != 0) {
return false;
}
if (record->digest_len == 0) {
return false;
}
if ((record->digest_len % 8u) != 0u) {
return false;
}
if (record->hash_id <= UINT16_MAX) {
hash_desc = amduat_hash_asl1_desc_lookup(
(amduat_hash_id_t)record->hash_id);
if (hash_desc != NULL && hash_desc->digest_len != 0 &&
record->digest_len != hash_desc->digest_len) {
return false;
}
}
if ((size_t)record->digest_len > digests_len - *digest_cursor) {
return false;
}
*digest_cursor += record->digest_len;
is_tombstone = (record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
if (is_tombstone) {
if (record->extent_count != 0 || record->total_length != 0) {
return false;
}
if (record->extents_offset != 0) {
return false;
}
if (record->visibility > *max_visibility) {
*max_visibility = record->visibility;
}
return true;
}
if (record->extent_count == 0) {
return false;
}
if ((size_t)record->extent_count > extent_count - *extent_cursor) {
return false;
}
start = *extent_cursor;
total_len = 0;
for (i = 0; i < record->extent_count; ++i) {
const amduat_asl_extent_record_t *extent = &extents[start + i];
if (extent->length == 0) {
return false;
}
total_len += extent->length;
if (total_len > UINT32_MAX) {
return false;
}
}
if ((uint32_t)total_len != record->total_length) {
return false;
}
*extent_cursor += record->extent_count;
if (record->visibility > *max_visibility) {
*max_visibility = record->visibility;
}
return true;
}
bool amduat_enc_asl_core_index_encode_v1(
const amduat_asl_core_index_segment_t *segment,
amduat_octets_t *out_bytes) {
size_t i;
size_t offset;
size_t digest_cursor;
size_t extent_cursor;
size_t record_count;
size_t extent_count;
size_t total_len;
uint64_t header_size;
uint64_t bloom_offset;
uint64_t bloom_size;
uint64_t records_offset;
uint64_t records_bytes;
uint64_t digests_offset;
uint64_t digests_size;
uint64_t extents_offset;
uint64_t extents_bytes;
uint64_t footer_offset;
uint8_t *buffer;
uint8_t max_visibility;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (segment == NULL) {
return false;
}
record_count = segment->record_count;
extent_count = segment->extent_count;
if (record_count != 0 && segment->records == NULL) {
return false;
}
if (extent_count != 0 && segment->extents == NULL) {
return false;
}
if (segment->digests.len != 0 && segment->digests.data == NULL) {
return false;
}
if (segment->bloom.len != 0 && segment->bloom.data == NULL) {
return false;
}
if (segment->header.flags != 0 || segment->header.federation_version != 0 ||
segment->header.reserved0 != 0) {
return false;
}
if (segment->header.snapshot_min > segment->header.snapshot_max) {
return false;
}
if (segment->header.segment_visibility > 1) {
return false;
}
digest_cursor = 0;
extent_cursor = 0;
max_visibility = 0;
for (i = 0; i < record_count; ++i) {
if (!amduat_asl_core_index_validate_record(
&segment->records[i],
segment->extents,
extent_count,
&extent_cursor,
&digest_cursor,
segment->digests.len,
&max_visibility)) {
return false;
}
}
if (digest_cursor != segment->digests.len) {
return false;
}
if (extent_cursor != extent_count) {
return false;
}
if (segment->header.segment_visibility != max_visibility) {
return false;
}
if (segment->bloom.len != 0 && (segment->bloom.len % 8u) != 0u) {
return false;
}
if ((segment->digests.len % 8u) != 0u) {
return false;
}
if (record_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_RECORD_SIZE) {
return false;
}
if (extent_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
return false;
}
header_size = AMDUAT_ASL_CORE_INDEX_HEADER_SIZE;
bloom_size = segment->bloom.len;
if (bloom_size == 0) {
bloom_offset = 0;
} else {
bloom_offset = header_size;
}
records_offset = header_size + bloom_size;
if (!amduat_asl_core_index_is_aligned8(records_offset)) {
return false;
}
records_bytes = (uint64_t)record_count *
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
digests_offset = records_offset + records_bytes;
if (digests_offset < records_offset ||
!amduat_asl_core_index_is_aligned8(digests_offset)) {
return false;
}
digests_size = segment->digests.len;
extents_offset = digests_offset + digests_size;
if (extents_offset < digests_offset ||
!amduat_asl_core_index_is_aligned8(extents_offset)) {
return false;
}
extents_bytes = (uint64_t)extent_count *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
footer_offset = extents_offset + extents_bytes;
if (footer_offset < extents_offset) {
return false;
}
if (footer_offset > SIZE_MAX) {
return false;
}
total_len = 0;
if (!amduat_asl_core_index_add_size(&total_len, (size_t)footer_offset) ||
!amduat_asl_core_index_add_size(&total_len,
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0;
amduat_asl_core_index_store_u64_le(buffer + offset,
AMDUAT_ASL_CORE_INDEX_MAGIC);
offset += 8;
amduat_asl_core_index_store_u16_le(buffer + offset,
AMDUAT_ASL_CORE_INDEX_VERSION);
offset += 2;
amduat_asl_core_index_store_u16_le(buffer + offset,
segment->header.shard_id);
offset += 2;
amduat_asl_core_index_store_u32_le(buffer + offset,
AMDUAT_ASL_CORE_INDEX_HEADER_SIZE);
offset += 4;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->header.snapshot_min);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->header.snapshot_max);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, record_count);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, records_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, bloom_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, bloom_size);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, digests_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, digests_size);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, extents_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, extent_count);
offset += 8;
amduat_asl_core_index_store_u32_le(buffer + offset,
segment->header.segment_domain_id);
offset += 4;
buffer[offset++] = segment->header.segment_visibility;
buffer[offset++] = 0;
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_core_index_store_u64_le(buffer + offset, 0);
offset += 8;
if (offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
free(buffer);
return false;
}
if (bloom_size != 0) {
memcpy(buffer + (size_t)bloom_offset, segment->bloom.data,
(size_t)bloom_size);
}
digest_cursor = 0;
extent_cursor = 0;
offset = (size_t)records_offset;
for (i = 0; i < record_count; ++i) {
const amduat_asl_index_record_t *record = &segment->records[i];
uint64_t digest_offset = digests_offset + digest_cursor;
uint64_t extents_offset_out = 0;
bool is_tombstone =
(record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
if (!is_tombstone) {
extents_offset_out = extents_offset +
(uint64_t)extent_cursor *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
extent_cursor += record->extent_count;
}
amduat_asl_core_index_store_u32_le(buffer + offset, record->hash_id);
offset += 4;
amduat_asl_core_index_store_u16_le(buffer + offset, record->digest_len);
offset += 2;
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_core_index_store_u64_le(buffer + offset, digest_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, extents_offset_out);
offset += 8;
amduat_asl_core_index_store_u32_le(buffer + offset,
record->extent_count);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset,
record->total_length);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset, record->domain_id);
offset += 4;
buffer[offset++] = record->visibility;
buffer[offset++] = record->has_cross_domain_source;
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_core_index_store_u32_le(buffer + offset,
record->cross_domain_source);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset, record->flags);
offset += 4;
digest_cursor += record->digest_len;
}
if (segment->digests.len != 0) {
memcpy(buffer + (size_t)digests_offset, segment->digests.data,
segment->digests.len);
}
offset = (size_t)extents_offset;
for (i = 0; i < extent_count; ++i) {
const amduat_asl_extent_record_t *extent = &segment->extents[i];
amduat_asl_core_index_store_u64_le(buffer + offset, extent->block_id);
offset += 8;
amduat_asl_core_index_store_u32_le(buffer + offset, extent->offset);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset, extent->length);
offset += 4;
}
{
uint64_t crc =
amduat_asl_core_index_crc64(buffer, (size_t)footer_offset);
offset = (size_t)footer_offset;
amduat_asl_core_index_store_u64_le(buffer + offset, crc);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->footer.seal_snapshot);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->footer.seal_time_ns);
offset += 8;
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_asl_core_index_decode_v1(
amduat_octets_t bytes,
amduat_asl_core_index_segment_t *out_segment) {
const uint8_t *data;
amduat_asl_segment_header_t header;
amduat_asl_segment_footer_t footer;
size_t record_count;
size_t extent_count;
size_t i;
uint64_t footer_offset;
uint64_t records_bytes;
uint64_t extents_bytes;
uint8_t max_visibility;
bool legacy_defaults;
if (out_segment == NULL) {
return false;
}
memset(out_segment, 0, sizeof(*out_segment));
if (bytes.len != 0 && bytes.data == NULL) {
return false;
}
if (bytes.len < AMDUAT_ASL_CORE_INDEX_HEADER_SIZE +
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE) {
return false;
}
data = bytes.data;
header.magic = amduat_asl_core_index_load_u64_le(data);
header.version = amduat_asl_core_index_load_u16_le(data + 8);
header.shard_id = amduat_asl_core_index_load_u16_le(data + 10);
header.header_size = amduat_asl_core_index_load_u32_le(data + 12);
header.snapshot_min = amduat_asl_core_index_load_u64_le(data + 16);
header.snapshot_max = amduat_asl_core_index_load_u64_le(data + 24);
header.record_count = amduat_asl_core_index_load_u64_le(data + 32);
header.records_offset = amduat_asl_core_index_load_u64_le(data + 40);
header.bloom_offset = amduat_asl_core_index_load_u64_le(data + 48);
header.bloom_size = amduat_asl_core_index_load_u64_le(data + 56);
header.digests_offset = amduat_asl_core_index_load_u64_le(data + 64);
header.digests_size = amduat_asl_core_index_load_u64_le(data + 72);
header.extents_offset = amduat_asl_core_index_load_u64_le(data + 80);
header.extent_count = amduat_asl_core_index_load_u64_le(data + 88);
header.segment_domain_id = amduat_asl_core_index_load_u32_le(data + 96);
header.segment_visibility = data[100];
header.federation_version = data[101];
header.reserved0 = amduat_asl_core_index_load_u16_le(data + 102);
header.flags = amduat_asl_core_index_load_u64_le(data + 104);
if (header.magic != AMDUAT_ASL_CORE_INDEX_MAGIC) {
return false;
}
if (header.version < 1 || header.version > AMDUAT_ASL_CORE_INDEX_VERSION) {
return false;
}
if (header.header_size != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
return false;
}
if (header.flags != 0) {
return false;
}
if (header.snapshot_min > header.snapshot_max) {
return false;
}
legacy_defaults = header.version < AMDUAT_ASL_CORE_INDEX_VERSION;
if (!legacy_defaults) {
if (header.federation_version != 0 || header.reserved0 != 0) {
return false;
}
if (header.segment_visibility > 1) {
return false;
}
}
if (header.record_count > SIZE_MAX) {
return false;
}
record_count = (size_t)header.record_count;
if (record_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_RECORD_SIZE) {
return false;
}
records_bytes = header.record_count *
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
if (header.extent_count > SIZE_MAX) {
return false;
}
extent_count = (size_t)header.extent_count;
if (extent_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
return false;
}
extents_bytes = header.extent_count *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
if (header.bloom_size == 0) {
if (header.bloom_offset != 0) {
return false;
}
} else {
if (header.bloom_offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.bloom_size)) {
return false;
}
}
if (header.records_offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE +
header.bloom_size) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.records_offset)) {
return false;
}
if (header.digests_offset != header.records_offset + records_bytes) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.digests_offset)) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.digests_size)) {
return false;
}
if (header.extents_offset != header.digests_offset + header.digests_size) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.extents_offset)) {
return false;
}
footer_offset = header.extents_offset + extents_bytes;
if (footer_offset < header.extents_offset) {
return false;
}
if (footer_offset > SIZE_MAX) {
return false;
}
if (footer_offset + AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE != bytes.len) {
return false;
}
footer.crc64 = amduat_asl_core_index_load_u64_le(
data + (size_t)footer_offset);
footer.seal_snapshot = amduat_asl_core_index_load_u64_le(
data + (size_t)footer_offset + 8);
footer.seal_time_ns = amduat_asl_core_index_load_u64_le(
data + (size_t)footer_offset + 16);
if (amduat_asl_core_index_crc64(data, (size_t)footer_offset) !=
footer.crc64) {
return false;
}
if (header.bloom_size != 0) {
if (!amduat_octets_clone(
amduat_octets(data + (size_t)header.bloom_offset,
(size_t)header.bloom_size),
&out_segment->bloom)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
if (!amduat_octets_clone(
amduat_octets(data + (size_t)header.digests_offset,
(size_t)header.digests_size),
&out_segment->digests)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record_count != 0) {
out_segment->records = (amduat_asl_index_record_t *)calloc(
record_count, sizeof(*out_segment->records));
if (out_segment->records == NULL) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
out_segment->record_count = record_count;
if (extent_count != 0) {
out_segment->extents = (amduat_asl_extent_record_t *)calloc(
extent_count, sizeof(*out_segment->extents));
if (out_segment->extents == NULL) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
out_segment->extent_count = extent_count;
for (i = 0; i < extent_count; ++i) {
size_t base = (size_t)header.extents_offset +
i * AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
amduat_asl_extent_record_t *extent = &out_segment->extents[i];
extent->block_id = amduat_asl_core_index_load_u64_le(data + base);
extent->offset = amduat_asl_core_index_load_u32_le(data + base + 8);
extent->length = amduat_asl_core_index_load_u32_le(data + base + 12);
if (extent->length == 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
max_visibility = 0;
for (i = 0; i < record_count; ++i) {
size_t base = (size_t)header.records_offset +
i * AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
amduat_asl_index_record_t *record = &out_segment->records[i];
uint64_t extents_offset;
size_t extent_index;
size_t j;
uint64_t total_len;
bool is_tombstone;
record->hash_id = amduat_asl_core_index_load_u32_le(data + base);
record->digest_len = amduat_asl_core_index_load_u16_le(data + base + 4);
record->reserved0 = amduat_asl_core_index_load_u16_le(data + base + 6);
record->digest_offset = amduat_asl_core_index_load_u64_le(data + base + 8);
record->extents_offset = amduat_asl_core_index_load_u64_le(
data + base + 16);
record->extent_count = amduat_asl_core_index_load_u32_le(data + base + 24);
record->total_length = amduat_asl_core_index_load_u32_le(data + base + 28);
record->domain_id = amduat_asl_core_index_load_u32_le(data + base + 32);
record->visibility = data[base + 36];
record->has_cross_domain_source = data[base + 37];
record->reserved1 = amduat_asl_core_index_load_u16_le(data + base + 38);
record->cross_domain_source =
amduat_asl_core_index_load_u32_le(data + base + 40);
record->flags = amduat_asl_core_index_load_u32_le(data + base + 44);
if (!legacy_defaults) {
if (record->reserved0 != 0 || record->reserved1 != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->visibility > 1 || record->has_cross_domain_source > 1) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->has_cross_domain_source == 0 &&
record->cross_domain_source != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
} else {
record->domain_id = 0;
record->visibility = 0;
record->has_cross_domain_source = 0;
record->cross_domain_source = 0;
}
if ((record->flags & ~AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->hash_id <= UINT16_MAX) {
const amduat_hash_asl1_desc_t *hash_desc =
amduat_hash_asl1_desc_lookup((amduat_hash_id_t)record->hash_id);
if (hash_desc != NULL && hash_desc->digest_len != 0 &&
record->digest_len != hash_desc->digest_len) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
if (record->digest_len == 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->digest_len > header.digests_size) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (!legacy_defaults &&
!amduat_asl_core_index_is_aligned8(record->digest_offset)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->digest_offset < header.digests_offset) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->digest_offset >
header.digests_offset + header.digests_size -
record->digest_len) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
is_tombstone =
(record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
if (is_tombstone) {
if (record->extent_count != 0 || record->total_length != 0 ||
record->extents_offset != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->visibility > max_visibility) {
max_visibility = record->visibility;
}
continue;
}
if (record->extent_count == 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
extents_offset = record->extents_offset;
if (extents_offset < header.extents_offset) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (!amduat_asl_core_index_is_aligned8(extents_offset)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->extent_count >
UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if ((extents_offset - header.extents_offset) %
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE !=
0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (extents_offset + (uint64_t)record->extent_count *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE >
header.extents_offset + extents_bytes) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
extent_index = (size_t)((extents_offset - header.extents_offset) /
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE);
total_len = 0;
for (j = 0; j < record->extent_count; ++j) {
total_len += out_segment->extents[extent_index + j].length;
if (total_len > UINT32_MAX) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
if ((uint32_t)total_len != record->total_length) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->visibility > max_visibility) {
max_visibility = record->visibility;
}
}
if (!legacy_defaults) {
if (max_visibility != header.segment_visibility) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
} else {
header.segment_domain_id = 0;
header.segment_visibility = 0;
header.federation_version = 0;
header.reserved0 = 0;
}
out_segment->header = header;
out_segment->footer = footer;
return true;
}
void amduat_enc_asl_core_index_free(amduat_asl_core_index_segment_t *segment) {
if (segment == NULL) {
return;
}
free(segment->records);
segment->records = NULL;
segment->record_count = 0;
free(segment->extents);
segment->extents = NULL;
segment->extent_count = 0;
amduat_octets_free(&segment->digests);
amduat_octets_free(&segment->bloom);
}

470
src/near_core/enc/asl_log.c Normal file
View file

@ -0,0 +1,470 @@
#include "amduat/enc/asl_log.h"
#include "amduat/hash/asl1.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_LOG_MAGIC_LEN = 8,
AMDUAT_ASL_LOG_HASH_LEN = 32,
AMDUAT_ASL_LOG_HEADER_LEN = 24,
AMDUAT_ASL_LOG_VERSION = 1
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_log_cursor_t;
static const uint8_t k_amduat_asl_log_magic[AMDUAT_ASL_LOG_MAGIC_LEN] = {
'A', 'S', 'L', 'L', 'O', 'G', '0', '1'};
static void amduat_asl_log_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_log_store_u16_le(uint8_t *out, uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_log_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_log_read_u32_le(amduat_asl_log_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4;
return true;
}
static bool amduat_asl_log_read_u16_le(amduat_asl_log_cursor_t *cur,
uint16_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 2) {
return false;
}
data = cur->data + cur->offset;
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
cur->offset += 2;
return true;
}
static bool amduat_asl_log_read_u64_le(amduat_asl_log_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8;
return true;
}
static bool amduat_asl_log_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_log_is_known_record_type(uint32_t record_type) {
switch (record_type) {
case AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL:
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE:
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT:
case AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR:
case AMDUAT_ASL_LOG_RECORD_ARTIFACT_PUBLISH:
case AMDUAT_ASL_LOG_RECORD_ARTIFACT_UNPUBLISH:
return true;
default:
return false;
}
}
bool amduat_asl_log_decode_artifact_ref(amduat_octets_t payload,
amduat_reference_t *out_ref) {
amduat_asl_log_cursor_t cur;
uint32_t hash_id_raw;
uint16_t digest_len;
uint16_t reserved0;
if (out_ref == NULL || payload.data == NULL || payload.len < 8u) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_log_read_u32_le(&cur, &hash_id_raw) ||
!amduat_asl_log_read_u16_le(&cur, &digest_len) ||
!amduat_asl_log_read_u16_le(&cur, &reserved0)) {
return false;
}
if (hash_id_raw > UINT16_MAX || digest_len == 0 || reserved0 != 0) {
return false;
}
if (cur.len - cur.offset < digest_len) {
return false;
}
{
amduat_octets_t digest = amduat_octets(cur.data + cur.offset, digest_len);
amduat_reference_t tmp =
amduat_reference((amduat_hash_id_t)hash_id_raw, digest);
return amduat_reference_clone(tmp, out_ref);
}
}
bool amduat_asl_log_encode_artifact_ref(amduat_reference_t ref,
amduat_octets_t *out_bytes) {
uint8_t *buf;
size_t len;
if (out_bytes == NULL) {
return false;
}
*out_bytes = amduat_octets(NULL, 0u);
if (ref.hash_id == 0 || ref.digest.data == NULL || ref.digest.len == 0) {
return false;
}
if (ref.digest.len > UINT16_MAX) {
return false;
}
len = 8u + ref.digest.len;
buf = (uint8_t *)malloc(len);
if (buf == NULL) {
return false;
}
amduat_asl_log_store_u32_le(buf, ref.hash_id);
amduat_asl_log_store_u16_le(buf + 4u, (uint16_t)ref.digest.len);
amduat_asl_log_store_u16_le(buf + 6u, 0u);
memcpy(buf + 8u, ref.digest.data, ref.digest.len);
*out_bytes = amduat_octets(buf, len);
return true;
}
static bool amduat_asl_log_hash_record(const uint8_t prev_hash[32],
uint64_t logseq,
uint32_t record_type,
uint32_t payload_len,
const uint8_t *payload,
uint8_t out_hash[32]) {
amduat_hash_asl1_stream_t stream;
uint8_t logseq_bytes[8];
uint8_t type_bytes[4];
uint8_t payload_len_bytes[4];
bool ok;
if (payload_len != 0 && payload == NULL) {
return false;
}
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
return false;
}
amduat_asl_log_store_u64_le(logseq_bytes, logseq);
amduat_asl_log_store_u32_le(type_bytes, record_type);
amduat_asl_log_store_u32_le(payload_len_bytes, payload_len);
ok = amduat_hash_asl1_stream_update(&stream,
amduat_octets(prev_hash,
AMDUAT_ASL_LOG_HASH_LEN)) &&
amduat_hash_asl1_stream_update(&stream,
amduat_octets(logseq_bytes,
sizeof(logseq_bytes))) &&
amduat_hash_asl1_stream_update(&stream,
amduat_octets(type_bytes,
sizeof(type_bytes))) &&
amduat_hash_asl1_stream_update(
&stream,
amduat_octets(payload_len_bytes, sizeof(payload_len_bytes)));
if (ok && payload_len != 0) {
ok = amduat_hash_asl1_stream_update(
&stream, amduat_octets(payload, payload_len));
}
if (ok) {
ok = amduat_hash_asl1_stream_final(&stream, out_hash,
AMDUAT_ASL_LOG_HASH_LEN);
}
amduat_hash_asl1_stream_destroy(&stream);
return ok;
}
void amduat_enc_asl_log_free(amduat_asl_log_record_t *records,
size_t record_count) {
size_t i;
if (records == NULL) {
return;
}
for (i = 0; i < record_count; ++i) {
amduat_octets_free(&records[i].payload);
}
free(records);
}
bool amduat_enc_asl_log_encode_v1(const amduat_asl_log_record_t *records,
size_t record_count,
amduat_octets_t *out_bytes) {
size_t total_len;
size_t i;
size_t offset;
uint8_t *buffer;
uint8_t prev_hash[AMDUAT_ASL_LOG_HASH_LEN];
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (record_count != 0 && records == NULL) {
return false;
}
total_len = AMDUAT_ASL_LOG_HEADER_LEN;
for (i = 0; i < record_count; ++i) {
size_t payload_len = records[i].payload.len;
if (payload_len != 0 && records[i].payload.data == NULL) {
return false;
}
if (payload_len > UINT32_MAX) {
return false;
}
if (!amduat_asl_log_add_size(&total_len, 8 + 4 + 4)) {
return false;
}
if (!amduat_asl_log_add_size(&total_len, payload_len)) {
return false;
}
if (!amduat_asl_log_add_size(&total_len, AMDUAT_ASL_LOG_HASH_LEN)) {
return false;
}
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0;
memcpy(buffer + offset, k_amduat_asl_log_magic, AMDUAT_ASL_LOG_MAGIC_LEN);
offset += AMDUAT_ASL_LOG_MAGIC_LEN;
amduat_asl_log_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_VERSION);
offset += 4;
amduat_asl_log_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_HEADER_LEN);
offset += 4;
amduat_asl_log_store_u64_le(buffer + offset, 0);
offset += 8;
memset(prev_hash, 0, sizeof(prev_hash));
for (i = 0; i < record_count; ++i) {
const amduat_asl_log_record_t *record = &records[i];
uint32_t payload_len = (uint32_t)record->payload.len;
uint8_t record_hash[AMDUAT_ASL_LOG_HASH_LEN];
amduat_asl_log_store_u64_le(buffer + offset, record->logseq);
offset += 8;
amduat_asl_log_store_u32_le(buffer + offset, record->record_type);
offset += 4;
amduat_asl_log_store_u32_le(buffer + offset, payload_len);
offset += 4;
if (payload_len != 0) {
memcpy(buffer + offset, record->payload.data, payload_len);
offset += payload_len;
}
if (!amduat_asl_log_hash_record(prev_hash,
record->logseq,
record->record_type,
payload_len,
record->payload.data,
record_hash)) {
free(buffer);
return false;
}
memcpy(buffer + offset, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
offset += AMDUAT_ASL_LOG_HASH_LEN;
memcpy(prev_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_asl_log_decode_v1(amduat_octets_t bytes,
amduat_asl_log_record_t **out_records,
size_t *out_count) {
amduat_asl_log_cursor_t cur;
uint32_t version;
uint32_t header_size;
uint64_t flags;
uint8_t prev_hash[AMDUAT_ASL_LOG_HASH_LEN];
amduat_asl_log_record_t *records;
size_t record_count;
size_t record_capacity;
uint64_t last_logseq;
if (out_records == NULL || out_count == NULL) {
return false;
}
*out_records = NULL;
*out_count = 0;
if (bytes.len != 0 && bytes.data == NULL) {
return false;
}
if (bytes.len < AMDUAT_ASL_LOG_HEADER_LEN) {
return false;
}
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0;
if (memcmp(cur.data, k_amduat_asl_log_magic, AMDUAT_ASL_LOG_MAGIC_LEN) != 0) {
return false;
}
cur.offset += AMDUAT_ASL_LOG_MAGIC_LEN;
if (!amduat_asl_log_read_u32_le(&cur, &version)) {
return false;
}
if (version != AMDUAT_ASL_LOG_VERSION) {
return false;
}
if (!amduat_asl_log_read_u32_le(&cur, &header_size)) {
return false;
}
if (!amduat_asl_log_read_u64_le(&cur, &flags)) {
return false;
}
if (flags != 0) {
return false;
}
if (header_size < AMDUAT_ASL_LOG_HEADER_LEN) {
return false;
}
if (bytes.len < header_size) {
return false;
}
cur.offset = header_size;
records = NULL;
record_count = 0;
record_capacity = 0;
last_logseq = 0u;
memset(prev_hash, 0, sizeof(prev_hash));
while (cur.offset < cur.len) {
uint64_t logseq;
uint32_t record_type;
uint32_t payload_len;
const uint8_t *payload;
const uint8_t *record_hash;
uint8_t expected_hash[AMDUAT_ASL_LOG_HASH_LEN];
if (!amduat_asl_log_read_u64_le(&cur, &logseq) ||
!amduat_asl_log_read_u32_le(&cur, &record_type) ||
!amduat_asl_log_read_u32_le(&cur, &payload_len)) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
if (record_count != 0u && logseq <= last_logseq) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
if (cur.len - cur.offset < payload_len + AMDUAT_ASL_LOG_HASH_LEN) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
payload = cur.data + cur.offset;
cur.offset += payload_len;
record_hash = cur.data + cur.offset;
cur.offset += AMDUAT_ASL_LOG_HASH_LEN;
if (!amduat_asl_log_hash_record(prev_hash,
logseq,
record_type,
payload_len,
payload,
expected_hash)) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
if (memcmp(expected_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN) != 0) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
memcpy(prev_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
last_logseq = logseq;
if (amduat_asl_log_is_known_record_type(record_type)) {
amduat_asl_log_record_t *slot;
amduat_octets_t payload_bytes;
if (record_count == record_capacity) {
size_t new_capacity = record_capacity == 0 ? 4 : record_capacity * 2;
amduat_asl_log_record_t *next =
(amduat_asl_log_record_t *)realloc(
records, new_capacity * sizeof(*records));
if (next == NULL) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
records = next;
record_capacity = new_capacity;
}
payload_bytes = amduat_octets(payload, payload_len);
slot = &records[record_count];
slot->logseq = logseq;
slot->record_type = record_type;
slot->payload = amduat_octets(NULL, 0u);
if (!amduat_octets_clone(payload_bytes, &slot->payload)) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
memcpy(slot->record_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
record_count++;
}
}
*out_records = records;
*out_count = record_count;
return true;
}

View file

@ -0,0 +1,494 @@
#include "amduat/enc/asl_tgk_exec_plan.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE = 8,
AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP = 8,
AMDUAT_ASL_TGK_EXEC_PLAN_PARAMS_SIZE = 65,
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE = 129,
AMDUAT_ASL_TGK_EXEC_PLAN_OP_TYPE_MAX =
AMDUAT_ASL_TGK_EXEC_OP_TOMBSTONE_SHADOW,
AMDUAT_ASL_TGK_EXEC_PLAN_OP_FLAG_MASK =
AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL |
AMDUAT_ASL_TGK_EXEC_OP_FLAG_OPTIONAL
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_tgk_exec_plan_cursor_t;
static void amduat_asl_tgk_exec_plan_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_tgk_exec_plan_store_u64_le(uint8_t *out,
uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_tgk_exec_plan_read_u32_le(
amduat_asl_tgk_exec_plan_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4u;
return true;
}
static bool amduat_asl_tgk_exec_plan_read_u64_le(
amduat_asl_tgk_exec_plan_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8u;
return true;
}
static bool amduat_asl_tgk_exec_plan_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_tgk_exec_plan_is_bool(uint8_t value) {
return value == 0u || value == 1u;
}
static bool amduat_asl_tgk_exec_plan_validate_params(
amduat_asl_tgk_exec_operator_type_t op_type,
const amduat_asl_tgk_exec_operator_params_t *params) {
if (!amduat_asl_tgk_exec_plan_is_bool(params->segment_scan.is_asl_segment)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->index_filter.has_type_tag)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->index_filter.has_edge_type)) {
return false;
}
if (params->index_filter.role > 3u) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->merge.deterministic)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(
params->projection.project_artifact_id) ||
!amduat_asl_tgk_exec_plan_is_bool(
params->projection.project_tgk_edge_id) ||
!amduat_asl_tgk_exec_plan_is_bool(params->projection.project_node_id) ||
!amduat_asl_tgk_exec_plan_is_bool(params->projection.project_type_tag)) {
return false;
}
if (params->tgk_traversal.direction > 3u) {
return false;
}
if (op_type == AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL &&
params->tgk_traversal.direction == 0u) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_count) ||
!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_union) ||
!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_sum)) {
return false;
}
return true;
}
static bool amduat_asl_tgk_exec_plan_has_op_id(
const amduat_asl_tgk_exec_plan_t *plan,
uint32_t op_id) {
uint32_t i;
for (i = 0; i < plan->operator_count; ++i) {
if (plan->operators[i].op_id == op_id) {
return true;
}
}
return false;
}
static bool amduat_asl_tgk_exec_plan_unique_op_ids(
const amduat_asl_tgk_exec_plan_t *plan) {
uint32_t i;
uint32_t j;
for (i = 0; i < plan->operator_count; ++i) {
for (j = i + 1u; j < plan->operator_count; ++j) {
if (plan->operators[i].op_id == plan->operators[j].op_id) {
return false;
}
}
}
return true;
}
static bool amduat_asl_tgk_exec_plan_validate_operator(
const amduat_asl_tgk_exec_plan_t *plan,
const amduat_asl_tgk_exec_operator_def_t *op) {
uint32_t i;
if ((uint32_t)op->op_type > AMDUAT_ASL_TGK_EXEC_PLAN_OP_TYPE_MAX) {
return false;
}
if (((uint32_t)op->flags & ~AMDUAT_ASL_TGK_EXEC_PLAN_OP_FLAG_MASK) != 0u) {
return false;
}
if (op->input_count > AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP) {
return false;
}
if (!amduat_asl_tgk_exec_plan_validate_params(op->op_type, &op->params)) {
return false;
}
for (i = 0; i < op->input_count; ++i) {
if (!amduat_asl_tgk_exec_plan_has_op_id(plan, op->inputs[i])) {
return false;
}
}
return true;
}
static void amduat_asl_tgk_exec_plan_encode_params(
const amduat_asl_tgk_exec_operator_params_t *params,
uint8_t *buffer,
size_t *offset) {
buffer[(*offset)++] = params->segment_scan.is_asl_segment;
amduat_asl_tgk_exec_plan_store_u64_le(
buffer + *offset, params->segment_scan.segment_start_id);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u64_le(
buffer + *offset, params->segment_scan.segment_end_id);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u32_le(
buffer + *offset, params->index_filter.artifact_type_tag);
*offset += 4u;
buffer[(*offset)++] = params->index_filter.has_type_tag;
amduat_asl_tgk_exec_plan_store_u32_le(
buffer + *offset, params->index_filter.edge_type_key);
*offset += 4u;
buffer[(*offset)++] = params->index_filter.has_edge_type;
buffer[(*offset)++] = params->index_filter.role;
buffer[(*offset)++] = params->merge.deterministic;
buffer[(*offset)++] = params->projection.project_artifact_id;
buffer[(*offset)++] = params->projection.project_tgk_edge_id;
buffer[(*offset)++] = params->projection.project_node_id;
buffer[(*offset)++] = params->projection.project_type_tag;
amduat_asl_tgk_exec_plan_store_u64_le(
buffer + *offset, params->tgk_traversal.start_node_id);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u32_le(
buffer + *offset, params->tgk_traversal.traversal_depth);
*offset += 4u;
buffer[(*offset)++] = params->tgk_traversal.direction;
buffer[(*offset)++] = params->aggregation.agg_count;
buffer[(*offset)++] = params->aggregation.agg_union;
buffer[(*offset)++] = params->aggregation.agg_sum;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + *offset,
params->limit_offset.limit);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + *offset,
params->limit_offset.offset);
*offset += 8u;
}
static bool amduat_asl_tgk_exec_plan_decode_params(
amduat_asl_tgk_exec_plan_cursor_t *cur,
amduat_asl_tgk_exec_operator_params_t *params) {
if (cur->len - cur->offset < AMDUAT_ASL_TGK_EXEC_PLAN_PARAMS_SIZE) {
return false;
}
params->segment_scan.is_asl_segment = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->segment_scan.segment_start_id) ||
!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->segment_scan.segment_end_id)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_read_u32_le(
cur, &params->index_filter.artifact_type_tag)) {
return false;
}
params->index_filter.has_type_tag = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u32_le(
cur, &params->index_filter.edge_type_key)) {
return false;
}
params->index_filter.has_edge_type = cur->data[cur->offset++];
params->index_filter.role = cur->data[cur->offset++];
params->merge.deterministic = cur->data[cur->offset++];
params->projection.project_artifact_id = cur->data[cur->offset++];
params->projection.project_tgk_edge_id = cur->data[cur->offset++];
params->projection.project_node_id = cur->data[cur->offset++];
params->projection.project_type_tag = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->tgk_traversal.start_node_id) ||
!amduat_asl_tgk_exec_plan_read_u32_le(
cur, &params->tgk_traversal.traversal_depth)) {
return false;
}
params->tgk_traversal.direction = cur->data[cur->offset++];
params->aggregation.agg_count = cur->data[cur->offset++];
params->aggregation.agg_union = cur->data[cur->offset++];
params->aggregation.agg_sum = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->limit_offset.limit) ||
!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->limit_offset.offset)) {
return false;
}
return true;
}
bool amduat_enc_asl_tgk_exec_plan_encode_v1(
const amduat_asl_tgk_exec_plan_t *plan,
amduat_octets_t *out_bytes) {
size_t total_len;
size_t offset;
uint8_t *buffer;
uint32_t i;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (plan == NULL) {
return false;
}
if (plan->operator_count != 0u && plan->operators == NULL) {
return false;
}
if (plan->operator_count > SIZE_MAX / AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE) {
return false;
}
total_len = AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE;
if (!amduat_asl_tgk_exec_plan_add_size(
&total_len,
(size_t)plan->operator_count *
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE)) {
return false;
}
if (plan->operator_count > UINT32_MAX) {
return false;
}
for (i = 0; i < plan->operator_count; ++i) {
if (!amduat_asl_tgk_exec_plan_validate_operator(plan,
&plan->operators[i])) {
return false;
}
}
if (!amduat_asl_tgk_exec_plan_unique_op_ids(plan)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, plan->plan_version);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
plan->operator_count);
offset += 4u;
for (i = 0; i < plan->operator_count; ++i) {
const amduat_asl_tgk_exec_operator_def_t *op = &plan->operators[i];
uint32_t j;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->op_id);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
(uint32_t)op->op_type);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
(uint32_t)op->flags);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + offset,
op->snapshot.logseq_min);
offset += 8u;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + offset,
op->snapshot.logseq_max);
offset += 8u;
amduat_asl_tgk_exec_plan_encode_params(&op->params, buffer, &offset);
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->input_count);
offset += 4u;
for (j = 0; j < AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP; ++j) {
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->inputs[j]);
offset += 4u;
}
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_asl_tgk_exec_plan_decode_v1(
amduat_octets_t bytes,
amduat_asl_tgk_exec_plan_t *out_plan) {
amduat_asl_tgk_exec_plan_cursor_t cur;
uint32_t operator_count;
size_t expected_len;
amduat_asl_tgk_exec_operator_def_t *operators;
uint32_t i;
if (out_plan == NULL) {
return false;
}
out_plan->plan_version = 0u;
out_plan->operator_count = 0u;
out_plan->operators = NULL;
if (bytes.len < AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE ||
bytes.data == NULL) {
return false;
}
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0u;
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &out_plan->plan_version) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &operator_count)) {
return false;
}
if (operator_count > SIZE_MAX / AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE) {
return false;
}
expected_len = AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE +
(size_t)operator_count *
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE;
if (expected_len != bytes.len) {
return false;
}
if (operator_count == 0u) {
out_plan->operator_count = 0u;
return true;
}
operators = (amduat_asl_tgk_exec_operator_def_t *)calloc(
operator_count, sizeof(*operators));
if (operators == NULL) {
return false;
}
for (i = 0; i < operator_count; ++i) {
amduat_asl_tgk_exec_operator_def_t *op = &operators[i];
uint32_t j;
uint32_t op_type_raw;
uint32_t flags_raw;
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->op_id) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op_type_raw) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &flags_raw) ||
!amduat_asl_tgk_exec_plan_read_u64_le(&cur,
&op->snapshot.logseq_min) ||
!amduat_asl_tgk_exec_plan_read_u64_le(&cur,
&op->snapshot.logseq_max)) {
free(operators);
return false;
}
op->op_type = (amduat_asl_tgk_exec_operator_type_t)op_type_raw;
op->flags = (amduat_asl_tgk_exec_operator_flags_t)flags_raw;
if (!amduat_asl_tgk_exec_plan_decode_params(&cur, &op->params) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->input_count)) {
free(operators);
return false;
}
if (op->input_count > AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP) {
free(operators);
return false;
}
for (j = 0; j < AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP; ++j) {
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->inputs[j])) {
free(operators);
return false;
}
}
}
out_plan->operator_count = operator_count;
out_plan->operators = operators;
if (!amduat_asl_tgk_exec_plan_unique_op_ids(out_plan)) {
amduat_enc_asl_tgk_exec_plan_free(out_plan);
return false;
}
for (i = 0; i < operator_count; ++i) {
if (!amduat_asl_tgk_exec_plan_validate_operator(out_plan,
&operators[i])) {
amduat_enc_asl_tgk_exec_plan_free(out_plan);
return false;
}
}
return true;
}
void amduat_enc_asl_tgk_exec_plan_free(amduat_asl_tgk_exec_plan_t *plan) {
if (plan == NULL) {
return;
}
free(plan->operators);
plan->operators = NULL;
plan->operator_count = 0u;
plan->plan_version = 0u;
}

View file

@ -8,6 +8,19 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
enum {
AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT = 0x0001u,
AMDUAT_FER1_TLV_RUN_ID = 0x0002u,
AMDUAT_FER1_TLV_LOGS = 0x0003u,
AMDUAT_FER1_TLV_LIMITS = 0x0004u,
AMDUAT_FER1_TLV_DETERMINISM = 0x0005u,
AMDUAT_FER1_TLV_SIGNATURE = 0x0006u
};
enum {
AMDUAT_FER1_LOGS_MAX = 64
};
typedef struct { typedef struct {
const uint8_t *data; const uint8_t *data;
size_t len; size_t len;
@ -177,6 +190,14 @@ static bool amduat_write_encoded_ref(uint8_t *buffer,
return true; return true;
} }
static int amduat_log_entry_cmp(const amduat_fer1_log_entry_t *a,
const amduat_fer1_log_entry_t *b) {
if (a->kind != b->kind) {
return a->kind < b->kind ? -1 : 1;
}
return amduat_reference_cmp(a->log_ref, b->log_ref);
}
static bool amduat_read_encoded_ref(amduat_cursor_t *cur, static bool amduat_read_encoded_ref(amduat_cursor_t *cur,
amduat_reference_t *out_ref) { amduat_reference_t *out_ref) {
uint32_t ref_len_u32; uint32_t ref_len_u32;
@ -231,10 +252,42 @@ void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt) {
free(receipt->parity); free(receipt->parity);
} }
if (receipt->has_executor_fingerprint_ref) {
amduat_reference_free(&receipt->executor_fingerprint_ref);
}
if (receipt->has_run_id) {
amduat_octets_free(&receipt->run_id);
}
if (receipt->logs != NULL) {
for (i = 0; i < receipt->logs_len; ++i) {
amduat_reference_free(&receipt->logs[i].log_ref);
amduat_octets_free(&receipt->logs[i].sha256);
}
free(receipt->logs);
}
if (receipt->has_rng_seed) {
amduat_octets_free(&receipt->rng_seed);
}
if (receipt->has_signature) {
amduat_octets_free(&receipt->signature);
}
receipt->executor_refs = NULL; receipt->executor_refs = NULL;
receipt->executor_refs_len = 0; receipt->executor_refs_len = 0;
receipt->parity = NULL; receipt->parity = NULL;
receipt->parity_len = 0; receipt->parity_len = 0;
receipt->has_executor_fingerprint_ref = false;
receipt->has_run_id = false;
receipt->run_id = amduat_octets(NULL, 0u);
receipt->has_limits = false;
receipt->logs = NULL;
receipt->logs_len = 0;
receipt->has_determinism = false;
receipt->determinism_level = 0u;
receipt->has_rng_seed = false;
receipt->rng_seed = amduat_octets(NULL, 0u);
receipt->has_signature = false;
receipt->signature = amduat_octets(NULL, 0u);
} }
bool amduat_enc_fer1_receipt_encode_v1( bool amduat_enc_fer1_receipt_encode_v1(
@ -444,6 +497,431 @@ bool amduat_enc_fer1_receipt_encode_v1(
return true; return true;
} }
bool amduat_enc_fer1_receipt_encode_v1_1(
const amduat_fer1_receipt_t *receipt,
amduat_octets_t *out_bytes) {
size_t total_len = 0;
size_t offset = 0;
size_t ext_len = 0;
uint8_t *buffer;
size_t i;
if (receipt == NULL || out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (receipt->fer1_version != AMDUAT_FER1_VERSION_1_1) {
return false;
}
if (receipt->evaluator_id.len != 0 && receipt->evaluator_id.data == NULL) {
return false;
}
if (receipt->executor_refs_len != 0 && receipt->executor_refs == NULL) {
return false;
}
if (receipt->parity_len != 0 && receipt->parity == NULL) {
return false;
}
if (receipt->executor_refs_len != receipt->parity_len) {
return false;
}
if (receipt->evaluator_id.len > UINT32_MAX) {
return false;
}
if (receipt->started_at > receipt->completed_at) {
return false;
}
if (receipt->has_run_id &&
receipt->run_id.len != 0 &&
receipt->run_id.data == NULL) {
return false;
}
if (receipt->has_run_id && receipt->run_id.len > UINT32_MAX) {
return false;
}
if (receipt->has_rng_seed &&
receipt->rng_seed.len != 0 &&
receipt->rng_seed.data == NULL) {
return false;
}
if (receipt->has_rng_seed && receipt->rng_seed.len > UINT32_MAX) {
return false;
}
if (receipt->has_signature &&
receipt->signature.len != 0 &&
receipt->signature.data == NULL) {
return false;
}
if (receipt->has_signature && receipt->signature.len > UINT32_MAX) {
return false;
}
if (receipt->logs_len != 0 && receipt->logs == NULL) {
return false;
}
if (receipt->logs_len > AMDUAT_FER1_LOGS_MAX) {
return false;
}
for (i = 1; i < receipt->executor_refs_len; ++i) {
if (amduat_reference_cmp(receipt->executor_refs[i - 1],
receipt->executor_refs[i]) > 0) {
return false;
}
}
for (i = 0; i < receipt->parity_len; ++i) {
if (!amduat_reference_eq(receipt->parity[i].executor_ref,
receipt->executor_refs[i])) {
return false;
}
if (i > 0 &&
amduat_reference_cmp(receipt->parity[i - 1].executor_ref,
receipt->parity[i].executor_ref) > 0) {
return false;
}
}
for (i = 0; i < receipt->logs_len; ++i) {
const amduat_fer1_log_entry_t *entry = &receipt->logs[i];
if (entry->sha256.len != 0 && entry->sha256.data == NULL) {
return false;
}
if (entry->sha256.len > UINT32_MAX) {
return false;
}
if (i > 0 &&
amduat_log_entry_cmp(&receipt->logs[i - 1], entry) > 0) {
return false;
}
}
{
size_t enc_len;
if (!amduat_add_size(&total_len, 2) ||
!amduat_encoded_ref_len(receipt->function_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_encoded_ref_len(receipt->input_manifest_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_encoded_ref_len(receipt->environment_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4 + receipt->evaluator_id.len)) {
return false;
}
{
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->output_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4)) {
return false;
}
for (i = 0; i < receipt->executor_refs_len; ++i) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->executor_refs[i], &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4)) {
return false;
}
for (i = 0; i < receipt->parity_len; ++i) {
size_t enc_len;
const amduat_fer1_parity_entry_t *entry = &receipt->parity[i];
if (entry->parity_digest.len != 0 &&
entry->parity_digest.data == NULL) {
return false;
}
if (entry->parity_digest.len > UINT32_MAX) {
return false;
}
if (!amduat_reference_eq(entry->output_ref, receipt->output_ref)) {
return false;
}
if (!amduat_encoded_ref_len(entry->executor_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_encoded_ref_len(entry->output_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_add_size(&total_len, 1)) {
return false;
}
if (entry->has_sbom_ref) {
if (!amduat_encoded_ref_len(entry->sbom_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4 + entry->parity_digest.len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 8 + 8)) {
return false;
}
if (receipt->has_executor_fingerprint_ref) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->executor_fingerprint_ref, &enc_len)) {
return false;
}
if (!amduat_add_size(&ext_len, 2 + 4 + enc_len)) {
return false;
}
}
if (receipt->has_run_id) {
if (!amduat_add_size(&ext_len, 2 + 4 + 4 + receipt->run_id.len)) {
return false;
}
}
if (receipt->has_limits) {
if (!amduat_add_size(&ext_len, 2 + 4 + (5u * 8u))) {
return false;
}
}
if (receipt->logs_len != 0) {
size_t logs_len = 4;
for (i = 0; i < receipt->logs_len; ++i) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->logs[i].log_ref, &enc_len)) {
return false;
}
if (!amduat_add_size(&logs_len, 4) ||
!amduat_add_size(&logs_len, enc_len) ||
!amduat_add_size(&logs_len, 4 + receipt->logs[i].sha256.len)) {
return false;
}
}
if (!amduat_add_size(&ext_len, 2 + 4 + logs_len)) {
return false;
}
}
if (receipt->has_determinism) {
if (!amduat_add_size(&ext_len,
2 + 4 + 1 + 4 + receipt->rng_seed.len)) {
return false;
}
}
if (receipt->has_signature) {
if (!amduat_add_size(&ext_len, 2 + 4 + receipt->signature.len)) {
return false;
}
}
if (ext_len > UINT32_MAX) {
return false;
}
if (!amduat_add_size(&total_len, 4 + ext_len)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
amduat_store_u16_be(buffer + offset, receipt->fer1_version);
offset += 2;
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->function_ref) ||
!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->input_manifest_ref) ||
!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->environment_ref)) {
free(buffer);
return false;
}
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->evaluator_id.len);
offset += 4;
if (receipt->evaluator_id.len != 0) {
memcpy(buffer + offset, receipt->evaluator_id.data,
receipt->evaluator_id.len);
offset += receipt->evaluator_id.len;
}
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->output_ref)) {
free(buffer);
return false;
}
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->executor_refs_len);
offset += 4;
for (i = 0; i < receipt->executor_refs_len; ++i) {
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->executor_refs[i])) {
free(buffer);
return false;
}
}
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->parity_len);
offset += 4;
for (i = 0; i < receipt->parity_len; ++i) {
const amduat_fer1_parity_entry_t *entry = &receipt->parity[i];
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->executor_ref) ||
!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->output_ref)) {
free(buffer);
return false;
}
buffer[offset++] = entry->has_sbom_ref ? 0x01u : 0x00u;
if (entry->has_sbom_ref) {
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->sbom_ref)) {
free(buffer);
return false;
}
}
amduat_store_u32_be(buffer + offset, (uint32_t)entry->parity_digest.len);
offset += 4;
if (entry->parity_digest.len != 0) {
memcpy(buffer + offset, entry->parity_digest.data,
entry->parity_digest.len);
offset += entry->parity_digest.len;
}
}
amduat_store_u64_be(buffer + offset, receipt->started_at);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->completed_at);
offset += 8;
amduat_store_u32_be(buffer + offset, (uint32_t)ext_len);
offset += 4;
if (receipt->has_executor_fingerprint_ref) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->executor_fingerprint_ref, &enc_len)) {
free(buffer);
return false;
}
amduat_store_u16_be(buffer + offset,
AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT);
offset += 2;
amduat_store_u32_be(buffer + offset, (uint32_t)enc_len);
offset += 4;
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->executor_fingerprint_ref)) {
free(buffer);
return false;
}
}
if (receipt->has_run_id) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_RUN_ID);
offset += 2;
amduat_store_u32_be(buffer + offset,
(uint32_t)(4u + receipt->run_id.len));
offset += 4;
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->run_id.len);
offset += 4;
if (receipt->run_id.len != 0) {
memcpy(buffer + offset, receipt->run_id.data, receipt->run_id.len);
offset += receipt->run_id.len;
}
}
if (receipt->logs_len != 0) {
size_t logs_len = 4;
for (i = 0; i < receipt->logs_len; ++i) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->logs[i].log_ref, &enc_len)) {
free(buffer);
return false;
}
logs_len += 4 + enc_len + 4 + receipt->logs[i].sha256.len;
}
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_LOGS);
offset += 2;
amduat_store_u32_be(buffer + offset, (uint32_t)logs_len);
offset += 4;
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->logs_len);
offset += 4;
for (i = 0; i < receipt->logs_len; ++i) {
const amduat_fer1_log_entry_t *entry = &receipt->logs[i];
amduat_store_u32_be(buffer + offset, entry->kind);
offset += 4;
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->log_ref)) {
free(buffer);
return false;
}
amduat_store_u32_be(buffer + offset, (uint32_t)entry->sha256.len);
offset += 4;
if (entry->sha256.len != 0) {
memcpy(buffer + offset, entry->sha256.data, entry->sha256.len);
offset += entry->sha256.len;
}
}
}
if (receipt->has_limits) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_LIMITS);
offset += 2;
amduat_store_u32_be(buffer + offset, (uint32_t)(5u * 8u));
offset += 4;
amduat_store_u64_be(buffer + offset, receipt->limits.cpu_ms);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.wall_ms);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.max_rss_kib);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.io_reads);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.io_writes);
offset += 8;
}
if (receipt->has_determinism) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_DETERMINISM);
offset += 2;
amduat_store_u32_be(buffer + offset,
(uint32_t)(1u + 4u + receipt->rng_seed.len));
offset += 4;
buffer[offset++] = receipt->determinism_level;
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->rng_seed.len);
offset += 4;
if (receipt->rng_seed.len != 0) {
memcpy(buffer + offset, receipt->rng_seed.data, receipt->rng_seed.len);
offset += receipt->rng_seed.len;
}
}
if (receipt->has_signature) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_SIGNATURE);
offset += 2;
amduat_store_u32_be(buffer + offset,
(uint32_t)receipt->signature.len);
offset += 4;
if (receipt->signature.len != 0) {
memcpy(buffer + offset, receipt->signature.data, receipt->signature.len);
offset += receipt->signature.len;
}
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_fer1_receipt_decode_v1( bool amduat_enc_fer1_receipt_decode_v1(
amduat_octets_t bytes, amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt) { amduat_fer1_receipt_t *out_receipt) {
@ -627,3 +1105,375 @@ bool amduat_enc_fer1_receipt_decode_v1(
return true; return true;
} }
bool amduat_enc_fer1_receipt_decode_v1_1(
amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt) {
amduat_cursor_t cur;
uint16_t fer1_version;
uint32_t len_u32;
uint32_t executor_count;
uint32_t parity_count;
uint32_t ext_len;
size_t i;
if (out_receipt == NULL) {
return false;
}
if (bytes.len != 0 && bytes.data == NULL) {
return false;
}
memset(out_receipt, 0, sizeof(*out_receipt));
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0;
if (!amduat_read_u16(&cur, &fer1_version)) {
return false;
}
if (fer1_version != AMDUAT_FER1_VERSION_1_1) {
return false;
}
out_receipt->fer1_version = fer1_version;
if (!amduat_read_encoded_ref(&cur, &out_receipt->function_ref) ||
!amduat_read_encoded_ref(&cur, &out_receipt->input_manifest_ref) ||
!amduat_read_encoded_ref(&cur, &out_receipt->environment_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u32(&cur, &len_u32)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (cur.len - cur.offset < len_u32) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (len_u32 != 0) {
amduat_octets_t src = amduat_octets(cur.data + cur.offset, len_u32);
if (!amduat_octets_clone(src, &out_receipt->evaluator_id)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
cur.offset += len_u32;
}
if (!amduat_read_encoded_ref(&cur, &out_receipt->output_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u32(&cur, &executor_count)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (executor_count != 0) {
if (executor_count > SIZE_MAX / sizeof(amduat_reference_t)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->executor_refs =
(amduat_reference_t *)calloc(executor_count,
sizeof(amduat_reference_t));
if (out_receipt->executor_refs == NULL) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->executor_refs_len = executor_count;
for (i = 0; i < out_receipt->executor_refs_len; ++i) {
if (!amduat_read_encoded_ref(&cur, &out_receipt->executor_refs[i])) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
for (i = 1; i < out_receipt->executor_refs_len; ++i) {
if (amduat_reference_cmp(out_receipt->executor_refs[i - 1],
out_receipt->executor_refs[i]) > 0) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (!amduat_read_u32(&cur, &parity_count)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (parity_count > SIZE_MAX / sizeof(amduat_fer1_parity_entry_t)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (parity_count != 0) {
out_receipt->parity =
(amduat_fer1_parity_entry_t *)calloc(parity_count,
sizeof(amduat_fer1_parity_entry_t));
if (out_receipt->parity == NULL) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->parity_len = parity_count;
for (i = 0; i < out_receipt->parity_len; ++i) {
amduat_fer1_parity_entry_t *entry = &out_receipt->parity[i];
uint8_t has_sbom_ref;
if (!amduat_read_encoded_ref(&cur, &entry->executor_ref) ||
!amduat_read_encoded_ref(&cur, &entry->output_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u8(&cur, &has_sbom_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (has_sbom_ref != 0x00u && has_sbom_ref != 0x01u) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
entry->has_sbom_ref = has_sbom_ref == 0x01u;
if (entry->has_sbom_ref) {
if (!amduat_read_encoded_ref(&cur, &entry->sbom_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (!amduat_read_u32(&cur, &len_u32)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (cur.len - cur.offset < len_u32) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (len_u32 != 0) {
amduat_octets_t src = amduat_octets(cur.data + cur.offset, len_u32);
if (!amduat_octets_clone(src, &entry->parity_digest)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
cur.offset += len_u32;
}
}
for (i = 0; i < out_receipt->parity_len; ++i) {
if (!amduat_reference_eq(out_receipt->parity[i].executor_ref,
out_receipt->executor_refs[i])) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (i > 0 &&
amduat_reference_cmp(out_receipt->parity[i - 1].executor_ref,
out_receipt->parity[i].executor_ref) > 0) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_reference_eq(out_receipt->parity[i].output_ref,
out_receipt->output_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (!amduat_read_u64(&cur, &out_receipt->started_at) ||
!amduat_read_u64(&cur, &out_receipt->completed_at)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (out_receipt->started_at > out_receipt->completed_at) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u32(&cur, &ext_len)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (cur.len - cur.offset < ext_len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
{
size_t end_offset = cur.offset + ext_len;
while (cur.offset < end_offset) {
uint16_t tag;
uint32_t tag_len;
amduat_cursor_t tlv_cur;
if (!amduat_read_u16(&cur, &tag) ||
!amduat_read_u32(&cur, &tag_len)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (tag_len > end_offset - cur.offset) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
tlv_cur.data = cur.data + cur.offset;
tlv_cur.len = tag_len;
tlv_cur.offset = 0;
switch (tag) {
case AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT:
if (out_receipt->has_executor_fingerprint_ref ||
!amduat_read_encoded_ref(&tlv_cur,
&out_receipt->executor_fingerprint_ref) ||
tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->has_executor_fingerprint_ref = true;
break;
case AMDUAT_FER1_TLV_RUN_ID: {
uint32_t run_len;
if (out_receipt->has_run_id ||
!amduat_read_u32(&tlv_cur, &run_len) ||
tlv_cur.len - tlv_cur.offset < run_len ||
tlv_cur.offset + run_len != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (run_len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, run_len);
if (!amduat_octets_clone(src, &out_receipt->run_id)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->has_run_id = true;
break;
}
case AMDUAT_FER1_TLV_LOGS: {
uint32_t count;
if (!amduat_read_u32(&tlv_cur, &count) ||
count > AMDUAT_FER1_LOGS_MAX ||
count > SIZE_MAX / sizeof(amduat_fer1_log_entry_t)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (count != 0) {
out_receipt->logs =
(amduat_fer1_log_entry_t *)calloc(
count, sizeof(amduat_fer1_log_entry_t));
if (out_receipt->logs == NULL) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->logs_len = count;
for (i = 0; i < out_receipt->logs_len; ++i) {
amduat_fer1_log_entry_t *entry = &out_receipt->logs[i];
uint32_t sha_len;
if (!amduat_read_u32(&tlv_cur, &entry->kind) ||
!amduat_read_encoded_ref(&tlv_cur, &entry->log_ref) ||
!amduat_read_u32(&tlv_cur, &sha_len) ||
tlv_cur.len - tlv_cur.offset < sha_len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (sha_len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, sha_len);
if (!amduat_octets_clone(src, &entry->sha256)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
tlv_cur.offset += sha_len;
if (i > 0 &&
amduat_log_entry_cmp(&out_receipt->logs[i - 1],
entry) > 0) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
break;
}
case AMDUAT_FER1_TLV_LIMITS:
if (out_receipt->has_limits ||
tlv_cur.len != 5u * 8u ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.cpu_ms) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.wall_ms) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.max_rss_kib) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.io_reads) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.io_writes) ||
tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->has_limits = true;
break;
case AMDUAT_FER1_TLV_DETERMINISM: {
uint8_t level;
uint32_t seed_len;
if (out_receipt->has_determinism ||
!amduat_read_u8(&tlv_cur, &level) ||
!amduat_read_u32(&tlv_cur, &seed_len) ||
tlv_cur.len - tlv_cur.offset < seed_len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (seed_len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, seed_len);
if (!amduat_octets_clone(src, &out_receipt->rng_seed)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->has_rng_seed = true;
}
tlv_cur.offset += seed_len;
if (tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->determinism_level = level;
out_receipt->has_determinism = true;
break;
}
case AMDUAT_FER1_TLV_SIGNATURE: {
if (out_receipt->has_signature) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (tlv_cur.len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, tlv_cur.len);
if (!amduat_octets_clone(src, &out_receipt->signature)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
tlv_cur.offset = tlv_cur.len;
out_receipt->has_signature = true;
break;
}
default:
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
cur.offset += tag_len;
}
if (cur.offset != end_offset) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (cur.offset != cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
return true;
}

View file

@ -0,0 +1,72 @@
#include "amduat/fed/ingest.h"
#include <stdbool.h>
static bool amduat_fed_record_id_eq(const amduat_fed_record_id_t *a,
const amduat_fed_record_id_t *b) {
if (a->type != b->type) {
return false;
}
return amduat_reference_eq(a->ref, b->ref);
}
static bool amduat_fed_record_equivalent(const amduat_fed_record_t *a,
const amduat_fed_record_t *b) {
return a->meta.domain_id == b->meta.domain_id &&
a->meta.visibility == b->meta.visibility &&
a->meta.has_source == b->meta.has_source &&
a->meta.source_domain == b->meta.source_domain &&
a->logseq == b->logseq &&
a->snapshot_id == b->snapshot_id &&
a->log_prefix == b->log_prefix;
}
amduat_fed_ingest_error_t amduat_fed_ingest_validate(
const amduat_fed_record_t *records,
size_t count,
size_t *out_error_index,
size_t *out_conflict_index) {
size_t i;
size_t j;
if (out_error_index != NULL) {
*out_error_index = (size_t)-1;
}
if (out_conflict_index != NULL) {
*out_conflict_index = (size_t)-1;
}
if (records == NULL && count != 0u) {
return AMDUAT_FED_INGEST_ERR_INVALID;
}
for (i = 0; i < count; ++i) {
const amduat_fed_record_t *record = &records[i];
if (!amduat_fed_record_validate(record)) {
if (out_error_index != NULL) {
*out_error_index = i;
}
return AMDUAT_FED_INGEST_ERR_INVALID;
}
for (j = 0; j < i; ++j) {
const amduat_fed_record_t *other = &records[j];
if (!amduat_fed_record_id_eq(&record->id, &other->id)) {
continue;
}
if (!amduat_fed_record_equivalent(record, other)) {
if (out_error_index != NULL) {
*out_error_index = i;
}
if (out_conflict_index != NULL) {
*out_conflict_index = j;
}
return AMDUAT_FED_INGEST_ERR_CONFLICT;
}
}
}
return AMDUAT_FED_INGEST_OK;
}

View file

@ -0,0 +1,416 @@
#include "amduat/fed/registry.h"
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_FED_REGISTRY_VERSION = 1,
AMDUAT_FED_REGISTRY_HEADER_LEN = 8,
AMDUAT_FED_REGISTRY_ENTRY_FIXED_LEN = 42
};
static void amduat_fed_store_u16_be(uint8_t *out, uint16_t value) {
out[0] = (uint8_t)((value >> 8) & 0xffu);
out[1] = (uint8_t)(value & 0xffu);
}
static void amduat_fed_store_u32_be(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)((value >> 24) & 0xffu);
out[1] = (uint8_t)((value >> 16) & 0xffu);
out[2] = (uint8_t)((value >> 8) & 0xffu);
out[3] = (uint8_t)(value & 0xffu);
}
static void amduat_fed_store_u64_be(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)((value >> 56) & 0xffu);
out[1] = (uint8_t)((value >> 48) & 0xffu);
out[2] = (uint8_t)((value >> 40) & 0xffu);
out[3] = (uint8_t)((value >> 32) & 0xffu);
out[4] = (uint8_t)((value >> 24) & 0xffu);
out[5] = (uint8_t)((value >> 16) & 0xffu);
out[6] = (uint8_t)((value >> 8) & 0xffu);
out[7] = (uint8_t)(value & 0xffu);
}
static uint16_t amduat_fed_load_u16_be(const uint8_t *data) {
return ((uint16_t)data[0] << 8) | (uint16_t)data[1];
}
static uint32_t amduat_fed_load_u32_be(const uint8_t *data) {
return ((uint32_t)data[0] << 24) | ((uint32_t)data[1] << 16) |
((uint32_t)data[2] << 8) | (uint32_t)data[3];
}
static uint64_t amduat_fed_load_u64_be(const uint8_t *data) {
return ((uint64_t)data[0] << 56) | ((uint64_t)data[1] << 48) |
((uint64_t)data[2] << 40) | ((uint64_t)data[3] << 32) |
((uint64_t)data[4] << 24) | ((uint64_t)data[5] << 16) |
((uint64_t)data[6] << 8) | (uint64_t)data[7];
}
void amduat_fed_registry_value_init(amduat_fed_registry_value_t *value,
amduat_fed_domain_state_t *states,
size_t cap) {
if (value == NULL) {
return;
}
value->states = states;
value->len = 0;
value->cap = cap;
value->owns_states = false;
}
bool amduat_fed_registry_value_insert(amduat_fed_registry_value_t *value,
amduat_fed_domain_state_t state) {
size_t left;
size_t right;
if (value == NULL || value->states == NULL) {
return false;
}
if (value->len >= value->cap) {
return false;
}
left = 0;
right = value->len;
while (left < right) {
size_t mid;
uint32_t current_id;
mid = left + (right - left) / 2;
current_id = value->states[mid].domain_id;
if (state.domain_id == current_id) {
return false;
}
if (state.domain_id < current_id) {
right = mid;
} else {
left = mid + 1;
}
}
if (left < value->len) {
memmove(&value->states[left + 1],
&value->states[left],
(value->len - left) * sizeof(*value->states));
}
value->states[left] = state;
value->len += 1;
return true;
}
const amduat_fed_domain_state_t *amduat_fed_registry_value_lookup(
const amduat_fed_registry_value_t *value,
uint32_t domain_id) {
size_t left;
size_t right;
if (value == NULL || value->states == NULL) {
return NULL;
}
left = 0;
right = value->len;
while (left < right) {
size_t mid;
uint32_t current_id;
mid = left + (right - left) / 2;
current_id = value->states[mid].domain_id;
if (domain_id == current_id) {
return &value->states[mid];
}
if (domain_id < current_id) {
right = mid;
} else {
left = mid + 1;
}
}
return NULL;
}
void amduat_fed_registry_value_free(amduat_fed_registry_value_t *value) {
size_t i;
if (value == NULL) {
return;
}
if (value->states != NULL) {
for (i = 0; i < value->len; ++i) {
amduat_octets_free(&value->states[i].policy_hash);
}
}
if (value->owns_states && value->states != NULL) {
free(value->states);
value->states = NULL;
value->cap = 0;
}
value->len = 0;
value->owns_states = false;
}
bool amduat_fed_registry_encode(const amduat_fed_registry_value_t *value,
amduat_octets_t *out_bytes) {
size_t i;
size_t total_len;
size_t offset;
uint8_t *buffer;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (value == NULL || value->states == NULL) {
return false;
}
if (value->len > UINT32_MAX) {
return false;
}
total_len = AMDUAT_FED_REGISTRY_HEADER_LEN;
for (i = 0; i < value->len; ++i) {
const amduat_fed_domain_state_t *state;
size_t entry_len;
state = &value->states[i];
if (state->policy_hash.len > UINT32_MAX) {
return false;
}
if (state->policy_hash.len != 0u && state->policy_hash.data == NULL) {
return false;
}
entry_len = AMDUAT_FED_REGISTRY_ENTRY_FIXED_LEN + state->policy_hash.len;
if (entry_len > SIZE_MAX - total_len) {
return false;
}
total_len += entry_len;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0;
amduat_fed_store_u32_be(buffer + offset, AMDUAT_FED_REGISTRY_VERSION);
offset += 4;
amduat_fed_store_u32_be(buffer + offset, (uint32_t)value->len);
offset += 4;
for (i = 0; i < value->len; ++i) {
const amduat_fed_domain_state_t *state;
state = &value->states[i];
amduat_fed_store_u32_be(buffer + offset, state->domain_id);
offset += 4;
amduat_fed_store_u64_be(buffer + offset, state->snapshot_id);
offset += 8;
amduat_fed_store_u64_be(buffer + offset, state->log_prefix);
offset += 8;
amduat_fed_store_u64_be(buffer + offset, state->last_logseq);
offset += 8;
buffer[offset++] = state->admitted;
buffer[offset++] = state->policy_ok;
memcpy(buffer + offset, state->reserved, sizeof(state->reserved));
offset += sizeof(state->reserved);
amduat_fed_store_u16_be(buffer + offset, state->policy_hash_id);
offset += 2;
amduat_fed_store_u32_be(buffer + offset,
(uint32_t)state->policy_hash.len);
offset += 4;
if (state->policy_hash.len != 0u) {
memcpy(buffer + offset,
state->policy_hash.data,
state->policy_hash.len);
offset += state->policy_hash.len;
}
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_fed_registry_decode(amduat_octets_t bytes,
amduat_fed_registry_value_t *out_value) {
uint32_t version;
uint32_t count;
size_t offset;
size_t i;
amduat_fed_registry_value_t value;
if (out_value == NULL) {
return false;
}
if (bytes.len != 0u && bytes.data == NULL) {
return false;
}
if (bytes.len < AMDUAT_FED_REGISTRY_HEADER_LEN) {
return false;
}
offset = 0;
version = amduat_fed_load_u32_be(bytes.data + offset);
offset += 4;
if (version != AMDUAT_FED_REGISTRY_VERSION) {
return false;
}
count = amduat_fed_load_u32_be(bytes.data + offset);
offset += 4;
value = *out_value;
if (value.states == NULL) {
if (count != 0u) {
value.states =
(amduat_fed_domain_state_t *)calloc(count, sizeof(*value.states));
if (value.states == NULL) {
return false;
}
}
value.cap = count;
value.owns_states = true;
} else {
if (value.cap < count) {
return false;
}
value.owns_states = false;
}
value.len = 0;
for (i = 0; i < count; ++i) {
amduat_fed_domain_state_t state;
uint32_t policy_hash_len;
if (bytes.len - offset < AMDUAT_FED_REGISTRY_ENTRY_FIXED_LEN) {
amduat_fed_registry_value_free(&value);
return false;
}
memset(&state, 0, sizeof(state));
state.domain_id = amduat_fed_load_u32_be(bytes.data + offset);
offset += 4;
state.snapshot_id = amduat_fed_load_u64_be(bytes.data + offset);
offset += 8;
state.log_prefix = amduat_fed_load_u64_be(bytes.data + offset);
offset += 8;
state.last_logseq = amduat_fed_load_u64_be(bytes.data + offset);
offset += 8;
state.admitted = bytes.data[offset++];
state.policy_ok = bytes.data[offset++];
memcpy(state.reserved, bytes.data + offset, sizeof(state.reserved));
offset += sizeof(state.reserved);
state.policy_hash_id = amduat_fed_load_u16_be(bytes.data + offset);
offset += 2;
policy_hash_len = amduat_fed_load_u32_be(bytes.data + offset);
offset += 4;
if (policy_hash_len > bytes.len - offset) {
amduat_fed_registry_value_free(&value);
return false;
}
if (policy_hash_len != 0u) {
uint8_t *policy_bytes;
policy_bytes = (uint8_t *)malloc(policy_hash_len);
if (policy_bytes == NULL) {
amduat_fed_registry_value_free(&value);
return false;
}
memcpy(policy_bytes, bytes.data + offset, policy_hash_len);
state.policy_hash = amduat_octets(policy_bytes, policy_hash_len);
offset += policy_hash_len;
} else {
state.policy_hash = amduat_octets(NULL, 0);
}
value.states[i] = state;
value.len += 1;
}
*out_value = value;
return true;
}
void amduat_fed_registry_store_init(amduat_fed_registry_store_t *reg,
amduat_asl_store_t *store) {
if (reg == NULL) {
return;
}
reg->store = store;
}
amduat_fed_registry_error_t amduat_fed_registry_store_put(
amduat_fed_registry_store_t *reg,
const amduat_fed_registry_value_t *value,
amduat_reference_t *out_ref,
amduat_asl_store_error_t *out_store_err) {
amduat_octets_t bytes;
amduat_artifact_t artifact;
amduat_asl_store_error_t store_err;
if (out_store_err != NULL) {
*out_store_err = AMDUAT_ASL_STORE_OK;
}
if (reg == NULL || reg->store == NULL) {
return AMDUAT_FED_REGISTRY_ERR_STORE;
}
if (!amduat_fed_registry_encode(value, &bytes)) {
return AMDUAT_FED_REGISTRY_ERR_CODEC;
}
artifact = amduat_artifact(bytes);
store_err = amduat_asl_store_put(reg->store, artifact, out_ref);
amduat_octets_free(&bytes);
if (store_err != AMDUAT_ASL_STORE_OK) {
if (out_store_err != NULL) {
*out_store_err = store_err;
}
return AMDUAT_FED_REGISTRY_ERR_STORE;
}
return AMDUAT_FED_REGISTRY_OK;
}
amduat_fed_registry_error_t amduat_fed_registry_store_get(
amduat_fed_registry_store_t *reg,
amduat_reference_t ref,
amduat_fed_registry_value_t *out_value,
amduat_asl_store_error_t *out_store_err) {
amduat_artifact_t artifact;
amduat_asl_store_error_t store_err;
bool ok;
if (out_store_err != NULL) {
*out_store_err = AMDUAT_ASL_STORE_OK;
}
if (reg == NULL || reg->store == NULL || out_value == NULL) {
return AMDUAT_FED_REGISTRY_ERR_STORE;
}
store_err = amduat_asl_store_get(reg->store, ref, &artifact);
if (store_err != AMDUAT_ASL_STORE_OK) {
if (out_store_err != NULL) {
*out_store_err = store_err;
}
return AMDUAT_FED_REGISTRY_ERR_STORE;
}
ok = amduat_fed_registry_decode(artifact.bytes, out_value);
amduat_artifact_free(&artifact);
if (!ok) {
return AMDUAT_FED_REGISTRY_ERR_CODEC;
}
return AMDUAT_FED_REGISTRY_OK;
}

309
src/near_core/fed/replay.c Normal file
View file

@ -0,0 +1,309 @@
#include "amduat/fed/replay.h"
#include <stdlib.h>
#include <string.h>
static int amduat_fed_octets_cmp(amduat_octets_t a, amduat_octets_t b) {
size_t min_len;
int cmp;
min_len = a.len < b.len ? a.len : b.len;
if (min_len > 0) {
cmp = memcmp(a.data, b.data, min_len);
if (cmp != 0) {
return cmp;
}
}
if (a.len < b.len) {
return -1;
}
if (a.len > b.len) {
return 1;
}
return 0;
}
static int amduat_fed_record_id_cmp(const amduat_fed_record_id_t *a,
const amduat_fed_record_id_t *b) {
if (a->type != b->type) {
return (a->type < b->type) ? -1 : 1;
}
if (a->ref.hash_id != b->ref.hash_id) {
return (a->ref.hash_id < b->ref.hash_id) ? -1 : 1;
}
return amduat_fed_octets_cmp(a->ref.digest, b->ref.digest);
}
static int amduat_fed_record_cmp(const void *lhs, const void *rhs) {
const amduat_fed_record_t *a = (const amduat_fed_record_t *)lhs;
const amduat_fed_record_t *b = (const amduat_fed_record_t *)rhs;
if (a->logseq != b->logseq) {
return (a->logseq < b->logseq) ? -1 : 1;
}
return amduat_fed_record_id_cmp(&a->id, &b->id);
}
static bool amduat_fed_record_clone(const amduat_fed_record_t *src,
amduat_fed_record_t *out) {
if (src == NULL || out == NULL) {
return false;
}
*out = *src;
if (!amduat_reference_clone(src->id.ref, &out->id.ref)) {
return false;
}
return true;
}
static void amduat_fed_record_free(amduat_fed_record_t *record) {
if (record == NULL) {
return;
}
amduat_reference_free(&record->id.ref);
memset(record, 0, sizeof(*record));
}
static bool amduat_fed_bounds_ok(const amduat_fed_record_t *record,
uint64_t snapshot_id,
uint64_t log_prefix) {
if (record->snapshot_id < snapshot_id) {
return true;
}
if (record->snapshot_id > snapshot_id) {
return false;
}
return record->logseq <= log_prefix;
}
static bool amduat_fed_ref_eq(amduat_reference_t a, amduat_reference_t b) {
return amduat_reference_eq(a, b);
}
static bool amduat_fed_tombstone_matches(const amduat_fed_record_t *tombstone,
const amduat_fed_record_t *record) {
if (tombstone->id.type != AMDUAT_FED_REC_TOMBSTONE) {
return false;
}
if (record->id.type == AMDUAT_FED_REC_TOMBSTONE) {
return false;
}
return amduat_fed_ref_eq(tombstone->id.ref, record->id.ref);
}
static bool amduat_fed_tombstone_list_has(
const amduat_reference_t *refs,
size_t refs_len,
amduat_reference_t candidate) {
size_t i;
for (i = 0; i < refs_len; ++i) {
if (amduat_fed_ref_eq(refs[i], candidate)) {
return true;
}
}
return false;
}
static bool amduat_fed_tombstone_list_push(amduat_reference_t **refs,
size_t *len,
size_t *cap,
amduat_reference_t ref) {
amduat_reference_t *next;
if (*len == *cap) {
size_t next_cap = (*cap == 0u) ? 4u : (*cap * 2u);
next = (amduat_reference_t *)realloc(*refs,
next_cap * sizeof(*next));
if (next == NULL) {
return false;
}
*refs = next;
*cap = next_cap;
}
if (!amduat_reference_clone(ref, &(*refs)[*len])) {
return false;
}
(*len)++;
return true;
}
bool amduat_fed_record_validate(const amduat_fed_record_t *record) {
if (record == NULL) {
return false;
}
if (record->meta.visibility > 1u) {
return false;
}
if (record->meta.has_source > 1u) {
return false;
}
if (record->meta.has_source == 0u && record->meta.source_domain != 0u) {
return false;
}
if (record->id.type < AMDUAT_FED_REC_ARTIFACT ||
record->id.type > AMDUAT_FED_REC_TOMBSTONE) {
return false;
}
if (record->id.ref.digest.len != 0u &&
record->id.ref.digest.data == NULL) {
return false;
}
return true;
}
bool amduat_fed_replay_build(const amduat_fed_record_t *records,
size_t count,
uint32_t domain_id,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_replay_view_t *out_view) {
amduat_fed_record_t *scratch;
amduat_reference_t *tombstones;
size_t tombstones_len;
size_t tombstones_cap;
size_t scratch_len;
size_t i;
size_t out_len;
bool ok;
if (out_view == NULL) {
return false;
}
out_view->records = NULL;
out_view->len = 0;
if (records == NULL && count != 0u) {
return false;
}
if (count == 0u) {
return true;
}
scratch = (amduat_fed_record_t *)calloc(count, sizeof(*scratch));
if (scratch == NULL) {
return false;
}
tombstones = NULL;
tombstones_len = 0;
tombstones_cap = 0;
scratch_len = 0;
for (i = 0; i < count; ++i) {
const amduat_fed_record_t *record = &records[i];
if (record->meta.domain_id != domain_id) {
continue;
}
if (!amduat_fed_record_validate(record)) {
continue;
}
if (!amduat_fed_bounds_ok(record, snapshot_id, log_prefix)) {
continue;
}
if (!amduat_fed_record_clone(record, &scratch[scratch_len])) {
while (scratch_len > 0) {
scratch_len--;
amduat_fed_record_free(&scratch[scratch_len]);
}
free(scratch);
return false;
}
scratch_len++;
}
qsort(scratch, scratch_len, sizeof(*scratch), amduat_fed_record_cmp);
ok = true;
out_len = 0;
for (i = 0; i < scratch_len; ++i) {
amduat_fed_record_t current = scratch[i];
size_t j;
bool tombstoned;
scratch[i].id.ref.digest.data = NULL;
scratch[i].id.ref.digest.len = 0u;
if (current.id.type == AMDUAT_FED_REC_TOMBSTONE) {
for (j = 0; j < out_len; ++j) {
if (amduat_fed_tombstone_matches(&current, &scratch[j])) {
amduat_fed_record_free(&scratch[j]);
memmove(&scratch[j],
&scratch[j + 1],
(out_len - j - 1) * sizeof(*scratch));
out_len--;
j--;
}
}
if (!amduat_fed_tombstone_list_push(&tombstones,
&tombstones_len,
&tombstones_cap,
current.id.ref)) {
amduat_fed_record_free(&current);
ok = false;
break;
}
amduat_fed_record_free(&current);
continue;
}
tombstoned = amduat_fed_tombstone_list_has(tombstones,
tombstones_len,
current.id.ref);
if (tombstoned) {
amduat_fed_record_free(&current);
continue;
}
scratch[out_len++] = current;
}
for (i = 0; i < tombstones_len; ++i) {
amduat_reference_free(&tombstones[i]);
}
free(tombstones);
if (!ok) {
for (i = 0; i < scratch_len; ++i) {
amduat_fed_record_free(&scratch[i]);
}
free(scratch);
return false;
}
out_view->records = scratch;
out_view->len = out_len;
return true;
}
bool amduat_fed_replay_domain(const amduat_fed_record_t *records,
size_t count,
uint32_t domain_id,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_replay_view_t *out_view) {
return amduat_fed_replay_build(records,
count,
domain_id,
snapshot_id,
log_prefix,
out_view);
}
void amduat_fed_replay_view_free(amduat_fed_replay_view_t *view) {
size_t i;
if (view == NULL) {
return;
}
if (view->records != NULL) {
for (i = 0; i < view->len; ++i) {
amduat_fed_record_free(&view->records[i]);
}
free(view->records);
}
view->records = NULL;
view->len = 0;
}

368
src/near_core/fed/view.c Normal file
View file

@ -0,0 +1,368 @@
#include "amduat/fed/view.h"
#include <stdlib.h>
#include <string.h>
static bool amduat_fed_record_clone(const amduat_fed_record_t *src,
amduat_fed_record_t *out) {
if (src == NULL || out == NULL) {
return false;
}
*out = *src;
if (!amduat_reference_clone(src->id.ref, &out->id.ref)) {
return false;
}
return true;
}
static void amduat_fed_record_free(amduat_fed_record_t *record) {
if (record == NULL) {
return;
}
amduat_reference_free(&record->id.ref);
memset(record, 0, sizeof(*record));
}
static bool amduat_fed_record_equivalent(const amduat_fed_record_t *a,
const amduat_fed_record_t *b) {
return a->meta.domain_id == b->meta.domain_id &&
a->meta.visibility == b->meta.visibility &&
a->meta.has_source == b->meta.has_source &&
a->meta.source_domain == b->meta.source_domain &&
a->id.type == b->id.type &&
amduat_reference_eq(a->id.ref, b->id.ref) &&
a->logseq == b->logseq &&
a->snapshot_id == b->snapshot_id &&
a->log_prefix == b->log_prefix;
}
static const amduat_fed_record_t *amduat_fed_view_find(
const amduat_fed_view_t *view,
const amduat_fed_record_t *record) {
size_t i;
for (i = 0; i < view->len; ++i) {
const amduat_fed_record_t *existing = &view->records[i];
if (existing->id.type != record->id.type) {
continue;
}
if (!amduat_reference_eq(existing->id.ref, record->id.ref)) {
continue;
}
return existing;
}
return NULL;
}
static bool amduat_fed_policy_deny_clone(const amduat_fed_policy_deny_t *src,
amduat_fed_policy_deny_t *out) {
if (src == NULL || out == NULL) {
return false;
}
*out = *src;
if (!amduat_reference_clone(src->id.ref, &out->id.ref)) {
return false;
}
return true;
}
static void amduat_fed_policy_deny_free(amduat_fed_policy_deny_t *deny) {
if (deny == NULL) {
return;
}
amduat_reference_free(&deny->id.ref);
memset(deny, 0, sizeof(*deny));
}
static amduat_fed_view_error_t amduat_fed_view_append(
amduat_fed_view_t *view,
const amduat_fed_record_t *records,
size_t count) {
size_t i;
size_t base_len;
if (count == 0u) {
return AMDUAT_FED_VIEW_OK;
}
if (view->len > SIZE_MAX - count) {
return AMDUAT_FED_VIEW_ERR_OOM;
}
if (view->records == NULL) {
view->records = (amduat_fed_record_t *)calloc(count, sizeof(*view->records));
if (view->records == NULL) {
return AMDUAT_FED_VIEW_ERR_OOM;
}
} else {
amduat_fed_record_t *next =
(amduat_fed_record_t *)realloc(view->records,
(view->len + count) *
sizeof(*view->records));
if (next == NULL) {
return AMDUAT_FED_VIEW_ERR_OOM;
}
view->records = next;
}
base_len = view->len;
for (i = 0; i < count; ++i) {
const amduat_fed_record_t *existing = amduat_fed_view_find(view,
&records[i]);
if (existing != NULL) {
if (!amduat_fed_record_equivalent(existing, &records[i])) {
size_t j;
for (j = base_len; j < view->len; ++j) {
amduat_fed_record_free(&view->records[j]);
}
view->len = base_len;
return AMDUAT_FED_VIEW_ERR_CONFLICT;
}
continue;
}
if (!amduat_fed_record_clone(&records[i],
&view->records[view->len])) {
size_t j;
for (j = base_len; j < view->len; ++j) {
amduat_fed_record_free(&view->records[j]);
}
view->len = base_len;
return AMDUAT_FED_VIEW_ERR_OOM;
}
view->len++;
}
return AMDUAT_FED_VIEW_OK;
}
amduat_fed_view_error_t amduat_fed_view_build(
const amduat_fed_record_t *records,
size_t count,
uint32_t local_domain_id,
const amduat_fed_view_bounds_t *bounds,
size_t bounds_len,
const amduat_fed_policy_deny_t *denies,
size_t denies_len,
amduat_fed_view_t *out_view) {
size_t i;
amduat_fed_view_t view;
if (out_view == NULL) {
return AMDUAT_FED_VIEW_ERR_INVALID;
}
out_view->records = NULL;
out_view->len = 0;
out_view->local_domain_id = local_domain_id;
out_view->denies = NULL;
out_view->denies_len = 0;
if (records == NULL && count != 0u) {
return AMDUAT_FED_VIEW_ERR_INVALID;
}
view.records = NULL;
view.len = 0;
view.local_domain_id = local_domain_id;
view.denies = NULL;
view.denies_len = 0;
if (denies_len != 0u && denies == NULL) {
return AMDUAT_FED_VIEW_ERR_INVALID;
}
if (denies_len != 0u) {
view.denies = (amduat_fed_policy_deny_t *)calloc(denies_len,
sizeof(*view.denies));
if (view.denies == NULL) {
return AMDUAT_FED_VIEW_ERR_OOM;
}
for (i = 0; i < denies_len; ++i) {
if (!amduat_fed_policy_deny_clone(&denies[i], &view.denies[i])) {
size_t j;
for (j = 0; j < i; ++j) {
amduat_fed_policy_deny_free(&view.denies[j]);
}
free(view.denies);
return AMDUAT_FED_VIEW_ERR_OOM;
}
}
view.denies_len = denies_len;
}
for (i = 0; i < bounds_len; ++i) {
amduat_fed_replay_view_t domain_view;
amduat_fed_view_bounds_t bound = bounds[i];
size_t j;
size_t domain_count = 0;
amduat_fed_record_t *domain_records = NULL;
bool ok;
for (j = 0; j < count; ++j) {
const amduat_fed_record_t *record = &records[j];
if (record->meta.domain_id != bound.domain_id) {
continue;
}
if (bound.domain_id != local_domain_id && record->meta.visibility != 1u) {
continue;
}
domain_count++;
}
if (domain_count == 0u) {
continue;
}
domain_records =
(amduat_fed_record_t *)calloc(domain_count, sizeof(*domain_records));
if (domain_records == NULL) {
amduat_fed_view_free(&view);
return AMDUAT_FED_VIEW_ERR_OOM;
}
domain_count = 0;
for (j = 0; j < count; ++j) {
const amduat_fed_record_t *record = &records[j];
if (record->meta.domain_id != bound.domain_id) {
continue;
}
if (bound.domain_id != local_domain_id && record->meta.visibility != 1u) {
continue;
}
if (!amduat_fed_record_clone(record, &domain_records[domain_count])) {
size_t k;
for (k = 0; k < domain_count; ++k) {
amduat_fed_record_free(&domain_records[k]);
}
free(domain_records);
amduat_fed_view_free(&view);
return AMDUAT_FED_VIEW_ERR_OOM;
}
domain_count++;
}
ok = amduat_fed_replay_build(domain_records,
domain_count,
bound.domain_id,
bound.snapshot_id,
bound.log_prefix,
&domain_view);
for (j = 0; j < domain_count; ++j) {
amduat_fed_record_free(&domain_records[j]);
}
free(domain_records);
if (!ok) {
amduat_fed_view_free(&view);
return AMDUAT_FED_VIEW_ERR_INVALID;
}
{
amduat_fed_view_error_t append_err;
append_err = amduat_fed_view_append(&view,
domain_view.records,
domain_view.len);
if (append_err != AMDUAT_FED_VIEW_OK) {
amduat_fed_replay_view_free(&domain_view);
amduat_fed_view_free(&view);
return append_err;
}
}
amduat_fed_replay_view_free(&domain_view);
}
*out_view = view;
return AMDUAT_FED_VIEW_OK;
}
void amduat_fed_view_free(amduat_fed_view_t *view) {
size_t i;
if (view == NULL) {
return;
}
if (view->records != NULL) {
for (i = 0; i < view->len; ++i) {
amduat_fed_record_free(&view->records[i]);
}
free(view->records);
}
if (view->denies != NULL) {
for (i = 0; i < view->denies_len; ++i) {
amduat_fed_policy_deny_free(&view->denies[i]);
}
free(view->denies);
}
view->records = NULL;
view->len = 0;
view->local_domain_id = 0;
view->denies = NULL;
view->denies_len = 0;
}
static const amduat_fed_record_t *amduat_fed_view_lookup(
const amduat_fed_view_t *view,
amduat_reference_t ref) {
size_t i;
if (view == NULL) {
return NULL;
}
for (i = 0; i < view->len; ++i) {
if (amduat_reference_eq(view->records[i].id.ref, ref)) {
return &view->records[i];
}
}
return NULL;
}
static bool amduat_fed_view_policy_denied(const amduat_fed_view_t *view,
amduat_reference_t ref) {
size_t i;
if (view == NULL) {
return false;
}
for (i = 0; i < view->denies_len; ++i) {
if (amduat_reference_eq(view->denies[i].id.ref, ref)) {
return true;
}
}
return false;
}
amduat_fed_resolve_error_t amduat_fed_resolve(
const amduat_fed_view_t *view,
amduat_asl_store_t *local_store,
amduat_reference_t ref,
amduat_artifact_t *out_artifact) {
amduat_asl_store_error_t store_err;
const amduat_fed_record_t *record;
if (local_store == NULL || out_artifact == NULL) {
return AMDUAT_FED_RESOLVE_STORE_ERROR;
}
store_err = amduat_asl_store_get(local_store, ref, out_artifact);
if (store_err == AMDUAT_ASL_STORE_OK) {
return AMDUAT_FED_RESOLVE_OK;
}
if (store_err == AMDUAT_ASL_STORE_ERR_INTEGRITY) {
return AMDUAT_FED_RESOLVE_INTEGRITY_ERROR;
}
if (store_err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
return AMDUAT_FED_RESOLVE_STORE_ERROR;
}
if (amduat_fed_view_policy_denied(view, ref)) {
return AMDUAT_FED_RESOLVE_POLICY_DENIED;
}
record = amduat_fed_view_lookup(view, ref);
if (record == NULL) {
return AMDUAT_FED_RESOLVE_NOT_FOUND;
}
if (record->meta.domain_id != view->local_domain_id) {
return AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES;
}
return AMDUAT_FED_RESOLVE_NOT_FOUND;
}

View file

@ -1,11 +1,13 @@
#include "amduat/fer/receipt.h" #include "amduat/fer/receipt.h"
#include "amduat/enc/fer1_receipt.h" #include "amduat/enc/fer1_receipt.h"
#include "amduat/pel/run.h"
#include <string.h> #include <string.h>
bool amduat_fer1_receipt_from_pel_result( static bool amduat_fer1_receipt_from_pel_result_with_output_ref(
const amduat_pel_surface_execution_result_t *pel_result, const amduat_pel_surface_execution_result_t *pel_result,
amduat_reference_t output_ref,
amduat_reference_t input_manifest_ref, amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref, amduat_reference_t environment_ref,
amduat_octets_t evaluator_id, amduat_octets_t evaluator_id,
@ -29,9 +31,6 @@ bool amduat_fer1_receipt_from_pel_result(
if (pel_result == NULL) { if (pel_result == NULL) {
return false; return false;
} }
if (pel_result->output_refs_len != 1 || pel_result->output_refs == NULL) {
return false;
}
if (evaluator_id.len != 0 && evaluator_id.data == NULL) { if (evaluator_id.len != 0 && evaluator_id.data == NULL) {
return false; return false;
} }
@ -45,7 +44,7 @@ bool amduat_fer1_receipt_from_pel_result(
receipt.input_manifest_ref = input_manifest_ref; receipt.input_manifest_ref = input_manifest_ref;
receipt.environment_ref = environment_ref; receipt.environment_ref = environment_ref;
receipt.evaluator_id = evaluator_id; receipt.evaluator_id = evaluator_id;
receipt.output_ref = pel_result->output_refs[0]; receipt.output_ref = output_ref;
receipt.started_at = started_at; receipt.started_at = started_at;
receipt.completed_at = completed_at; receipt.completed_at = completed_at;
@ -73,3 +72,206 @@ bool amduat_fer1_receipt_from_pel_result(
amduat_type_tag(AMDUAT_TYPE_TAG_FER1_RECEIPT_1)); amduat_type_tag(AMDUAT_TYPE_TAG_FER1_RECEIPT_1));
return true; return true;
} }
bool amduat_fer1_receipt_from_pel_result(
const amduat_pel_surface_execution_result_t *pel_result,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
amduat_artifact_t *out_artifact) {
if (pel_result == NULL) {
return false;
}
if (pel_result->output_refs_len != 1 || pel_result->output_refs == NULL) {
return false;
}
return amduat_fer1_receipt_from_pel_result_with_output_ref(
pel_result,
pel_result->output_refs[0],
input_manifest_ref,
environment_ref,
evaluator_id,
executor_ref,
has_sbom_ref,
sbom_ref,
parity_digest,
started_at,
completed_at,
out_artifact);
}
bool amduat_fer1_receipt_from_pel_run(
const amduat_pel_run_result_t *pel_run,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
amduat_artifact_t *out_artifact) {
amduat_reference_t output_ref;
if (pel_run == NULL || !pel_run->has_result_value) {
return false;
}
if (pel_run->output_refs_len == 1 && pel_run->output_refs != NULL) {
output_ref = pel_run->output_refs[0];
} else if (pel_run->output_refs_len == 0) {
output_ref = pel_run->result_ref;
} else {
return false;
}
return amduat_fer1_receipt_from_pel_result_with_output_ref(
&pel_run->result_value,
output_ref,
input_manifest_ref,
environment_ref,
evaluator_id,
executor_ref,
has_sbom_ref,
sbom_ref,
parity_digest,
started_at,
completed_at,
out_artifact);
}
bool amduat_fer1_receipt_from_pel_run_v1_1(
const amduat_pel_run_result_t *pel_run,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
bool has_executor_fingerprint_ref,
amduat_reference_t executor_fingerprint_ref,
bool has_run_id,
amduat_octets_t run_id,
bool has_limits,
amduat_fer1_limits_t limits,
const amduat_fer1_log_entry_t *logs,
size_t logs_len,
bool has_determinism,
uint8_t determinism_level,
bool has_rng_seed,
amduat_octets_t rng_seed,
bool has_signature,
amduat_octets_t signature,
amduat_artifact_t *out_artifact) {
amduat_fer1_receipt_t receipt;
amduat_fer1_parity_entry_t parity;
amduat_reference_t executor_refs[1];
amduat_reference_t output_ref;
amduat_octets_t receipt_bytes;
if (out_artifact == NULL) {
return false;
}
*out_artifact = amduat_artifact(amduat_octets(NULL, 0u));
if (pel_run == NULL || !pel_run->has_result_value) {
return false;
}
if (pel_run->output_refs_len == 1 && pel_run->output_refs != NULL) {
output_ref = pel_run->output_refs[0];
} else if (pel_run->output_refs_len == 0) {
output_ref = pel_run->result_ref;
} else {
return false;
}
if (evaluator_id.len != 0 && evaluator_id.data == NULL) {
return false;
}
if (parity_digest.len != 0 && parity_digest.data == NULL) {
return false;
}
if (has_run_id && run_id.len != 0 && run_id.data == NULL) {
return false;
}
if (has_rng_seed && rng_seed.len != 0 && rng_seed.data == NULL) {
return false;
}
if (has_signature && signature.len != 0 && signature.data == NULL) {
return false;
}
if (logs_len != 0 && logs == NULL) {
return false;
}
memset(&receipt, 0, sizeof(receipt));
receipt.fer1_version = AMDUAT_FER1_VERSION_1_1;
receipt.function_ref = pel_run->result_value.program_ref;
receipt.input_manifest_ref = input_manifest_ref;
receipt.environment_ref = environment_ref;
receipt.evaluator_id = evaluator_id;
receipt.output_ref = output_ref;
receipt.started_at = started_at;
receipt.completed_at = completed_at;
executor_refs[0] = executor_ref;
receipt.executor_refs = executor_refs;
receipt.executor_refs_len = 1;
memset(&parity, 0, sizeof(parity));
parity.executor_ref = executor_ref;
parity.output_ref = receipt.output_ref;
parity.has_sbom_ref = has_sbom_ref;
if (has_sbom_ref) {
parity.sbom_ref = sbom_ref;
}
parity.parity_digest = parity_digest;
receipt.parity = &parity;
receipt.parity_len = 1;
receipt.has_executor_fingerprint_ref = has_executor_fingerprint_ref;
if (has_executor_fingerprint_ref) {
receipt.executor_fingerprint_ref = executor_fingerprint_ref;
}
receipt.has_run_id = has_run_id;
if (has_run_id) {
receipt.run_id = run_id;
}
receipt.has_limits = has_limits;
if (has_limits) {
receipt.limits = limits;
}
receipt.logs = (amduat_fer1_log_entry_t *)logs;
receipt.logs_len = logs_len;
receipt.has_determinism = has_determinism;
if (has_determinism) {
receipt.determinism_level = determinism_level;
}
receipt.has_rng_seed = has_rng_seed;
if (has_rng_seed) {
receipt.rng_seed = rng_seed;
}
receipt.has_signature = has_signature;
if (has_signature) {
receipt.signature = signature;
}
if (!amduat_enc_fer1_receipt_encode_v1_1(&receipt, &receipt_bytes)) {
return false;
}
*out_artifact = amduat_artifact_with_type(
amduat_octets(receipt_bytes.data, receipt_bytes.len),
amduat_type_tag(AMDUAT_TYPE_TAG_FER1_RECEIPT_1));
return true;
}

View file

@ -95,6 +95,16 @@ const char *amduat_format_pel_kernel_kind_name(
return "const"; return "const";
case AMDUAT_PEL_KERNEL_OP_HASH_ASL1: case AMDUAT_PEL_KERNEL_OP_HASH_ASL1:
return "hash_asl1"; return "hash_asl1";
case AMDUAT_PEL_KERNEL_OP_PARAMS:
return "params";
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
return "format_encode";
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
return "collection_snapshot_decode";
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
return "log_read_range";
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
return "collection_merge_refs";
default: default:
return "unknown"; return "unknown";
} }

View file

@ -1,5 +1,6 @@
#include "amduat/pel/opreg_kernel.h" #include "amduat/pel/opreg_kernel.h"
#include "amduat/pel/opreg_kernel_params.h" #include "amduat/pel/opreg_kernel_params.h"
#include "kernel_collection.h"
#include "amduat/hash/asl1.h" #include "amduat/hash/asl1.h"
#include <stdbool.h> #include <stdbool.h>
@ -64,8 +65,32 @@ static const amduat_pel_kernel_op_desc_t k_kernel_descs[] = {
3, 3,
1 1
}, },
{
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE,
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_SNAPSHOT_DECODE,
1,
1,
1
},
{
AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE,
AMDUAT_PEL_KERNEL_OP_CODE_LOG_READ_RANGE,
3,
3,
1
},
{
AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS,
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_MERGE_REFS,
4,
4,
1
},
}; };
static amduat_pel_kernel_artifact_resolver_t g_resolver = NULL;
static void *g_resolver_ctx = NULL;
const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup( const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
amduat_octets_t name, amduat_octets_t name,
uint32_t version) { uint32_t version) {
@ -90,6 +115,16 @@ const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME)) { if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME)) {
return &k_kernel_descs[5]; return &k_kernel_descs[5];
} }
if (amduat_name_eq(name,
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME)) {
return &k_kernel_descs[6];
}
if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME)) {
return &k_kernel_descs[7];
}
if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME)) {
return &k_kernel_descs[8];
}
return NULL; return NULL;
} }
@ -115,11 +150,37 @@ const char *amduat_pel_kernel_op_name(amduat_pel_kernel_op_kind_t kind) {
return AMDUAT_PEL_KERNEL_OP_PARAMS_NAME; return AMDUAT_PEL_KERNEL_OP_PARAMS_NAME;
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE: case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
return AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME; return AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME;
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
return AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME;
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
return AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME;
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
return AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME;
default: default:
return NULL; return NULL;
} }
} }
void amduat_pel_kernel_set_artifact_resolver(
amduat_pel_kernel_artifact_resolver_t resolver,
void *ctx) {
g_resolver = resolver;
g_resolver_ctx = ctx;
}
void amduat_pel_kernel_clear_artifact_resolver(void) {
g_resolver = NULL;
g_resolver_ctx = NULL;
}
bool amduat_pel_kernel_resolve_artifact(amduat_reference_t ref,
amduat_artifact_t *out_artifact) {
if (g_resolver == NULL) {
return false;
}
return g_resolver(ref, out_artifact, g_resolver_ctx);
}
static bool amduat_type_tags_match(const amduat_artifact_t *a, static bool amduat_type_tags_match(const amduat_artifact_t *a,
const amduat_artifact_t *b) { const amduat_artifact_t *b) {
if (a->has_type_tag != b->has_type_tag) { if (a->has_type_tag != b->has_type_tag) {
@ -1537,6 +1598,15 @@ bool amduat_pel_kernel_op_eval(
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE: case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
return amduat_kernel_format_encode(inputs, inputs_len, out_outputs, return amduat_kernel_format_encode(inputs, inputs_len, out_outputs,
out_outputs_len, out_status_code); out_outputs_len, out_status_code);
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
return amduat_kernel_collection_snapshot_decode(
inputs, inputs_len, out_outputs, out_outputs_len, out_status_code);
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
return amduat_kernel_log_read_range(
inputs, inputs_len, out_outputs, out_outputs_len, out_status_code);
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
return amduat_kernel_collection_merge_refs(
inputs, inputs_len, out_outputs, out_outputs_len, out_status_code);
default: default:
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL; *out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
return false; return false;

View file

@ -0,0 +1,615 @@
#include "kernel_collection.h"
#include "amduat/asl/collection.h"
#include "amduat/asl/collection_view.h"
#include "amduat/asl/log_store.h"
#include "amduat/asl/none.h"
#include "amduat/asl/record.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
static bool amduat_alloc_outputs(size_t count,
amduat_artifact_t **out_outputs) {
amduat_artifact_t *outputs;
if (out_outputs == NULL) {
return false;
}
outputs = (amduat_artifact_t *)calloc(count, sizeof(*outputs));
if (outputs == NULL) {
return false;
}
*out_outputs = outputs;
return true;
}
static bool amduat_load_u64_le(amduat_octets_t bytes,
size_t offset,
uint64_t *out) {
const uint8_t *data;
if (out == NULL) {
return false;
}
if (bytes.len - offset < 8u || bytes.data == NULL) {
return false;
}
data = bytes.data + offset;
*out = (uint64_t)data[0] |
((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) |
((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) |
((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) |
((uint64_t)data[7] << 56);
return true;
}
static bool amduat_load_u32_le(amduat_octets_t bytes,
size_t offset,
uint32_t *out) {
const uint8_t *data;
if (out == NULL) {
return false;
}
if (bytes.len - offset < 4u || bytes.data == NULL) {
return false;
}
data = bytes.data + offset;
*out = (uint32_t)data[0] |
((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) |
((uint32_t)data[3] << 24);
return true;
}
static bool amduat_decode_params(const amduat_artifact_t *artifact,
uint64_t *out_from,
uint32_t *out_limit) {
const char k_params_schema[] = "pel/params";
amduat_asl_record_t record;
bool ok = false;
if (artifact == NULL || out_from == NULL || out_limit == NULL) {
return false;
}
if (!artifact->has_type_tag ||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
return false;
}
if (!amduat_asl_record_decode_v1(artifact->bytes, &record)) {
return false;
}
if (record.schema.len != strlen(k_params_schema) ||
record.schema.data == NULL ||
memcmp(record.schema.data, k_params_schema,
record.schema.len) != 0) {
amduat_asl_record_free(&record);
return false;
}
if (record.payload.len != 12u) {
amduat_asl_record_free(&record);
return false;
}
if (!amduat_load_u64_le(record.payload, 0u, out_from) ||
!amduat_load_u32_le(record.payload, 8u, out_limit)) {
amduat_asl_record_free(&record);
return false;
}
ok = true;
amduat_asl_record_free(&record);
return ok;
}
static bool amduat_log_chunk_from_artifact(
const amduat_artifact_t *artifact,
amduat_asl_log_chunk_t *out_chunk) {
if (artifact == NULL || out_chunk == NULL) {
return false;
}
if (!artifact->has_type_tag ||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_LOG_CHUNK_1) {
return false;
}
return amduat_asl_log_chunk_decode_v1(artifact->bytes, out_chunk) ==
AMDUAT_ASL_STORE_OK;
}
bool amduat_kernel_collection_snapshot_decode(
const amduat_artifact_t *inputs,
size_t inputs_len,
amduat_artifact_t **out_outputs,
size_t *out_outputs_len,
uint32_t *out_status_code) {
amduat_asl_collection_snapshot_payload_t snapshot;
amduat_asl_collection_snapshot_info_t info;
amduat_asl_record_t record;
amduat_octets_t encoded = amduat_octets(NULL, 0u);
bool ok = false;
if (inputs == NULL || inputs_len != 1u) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
}
return false;
}
memset(&snapshot, 0, sizeof(snapshot));
memset(&info, 0, sizeof(info));
memset(&record, 0, sizeof(record));
if (amduat_asl_none_is_artifact(&inputs[0])) {
info.snapshot_at_offset = 0u;
info.refs = NULL;
info.refs_len = 0u;
} else {
if (!inputs[0].has_type_tag ||
inputs[0].type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
}
return false;
}
if (!amduat_asl_record_decode_v1(inputs[0].bytes, &record)) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
}
return false;
}
if (record.schema.len != strlen("collection/snapshot") ||
record.schema.data == NULL ||
memcmp(record.schema.data, "collection/snapshot",
record.schema.len) != 0) {
amduat_asl_record_free(&record);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
}
return false;
}
if (!amduat_asl_collection_snapshot_payload_decode_v1(record.payload,
&snapshot)) {
amduat_asl_record_free(&record);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
}
return false;
}
info.snapshot_at_offset = snapshot.snapshot_offset;
info.refs = snapshot.refs;
info.refs_len = snapshot.refs_len;
amduat_asl_record_free(&record);
}
if (!amduat_asl_collection_snapshot_info_encode_v1(&info, &encoded)) {
amduat_asl_collection_snapshot_payload_free(&snapshot);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
if (!amduat_alloc_outputs(1u, out_outputs)) {
amduat_asl_collection_snapshot_payload_free(&snapshot);
free((void *)encoded.data);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
(*out_outputs)[0] = amduat_artifact_with_type(
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1));
if (out_outputs_len != NULL) {
*out_outputs_len = 1u;
}
ok = true;
amduat_asl_collection_snapshot_payload_free(&snapshot);
return ok;
}
bool amduat_kernel_log_read_range(
const amduat_artifact_t *inputs,
size_t inputs_len,
amduat_artifact_t **out_outputs,
size_t *out_outputs_len,
uint32_t *out_status_code) {
amduat_asl_log_range_t range;
amduat_octets_t encoded = amduat_octets(NULL, 0u);
uint64_t from_offset = 0u;
uint32_t limit = 0u;
uint64_t limit_from_offset = 0u;
uint32_t limit_value = 0u;
bool ok = false;
if (inputs == NULL || inputs_len != 3u) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
}
return false;
}
if (!amduat_decode_params(&inputs[1], &from_offset, &limit_value)) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID;
}
return false;
}
if (!amduat_decode_params(&inputs[2], &limit_from_offset, &limit)) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID;
}
return false;
}
(void)limit_from_offset;
(void)limit_value;
memset(&range, 0, sizeof(range));
range.start_offset = from_offset;
range.next_offset = from_offset;
range.refs = NULL;
range.refs_len = 0u;
if (limit == 0u || amduat_asl_none_is_artifact(&inputs[0])) {
goto encode_range;
}
{
amduat_asl_log_chunk_t *chunks = NULL;
size_t chunks_len = 0u;
amduat_asl_log_chunk_t head_chunk;
uint64_t head_end_offset = 0u;
if (!amduat_log_chunk_from_artifact(&inputs[0], &head_chunk)) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID;
}
return false;
}
if (head_chunk.base_offset >
UINT64_MAX - (uint64_t)head_chunk.entry_count) {
amduat_asl_log_chunk_free(&head_chunk);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID;
}
return false;
}
head_end_offset = head_chunk.base_offset + head_chunk.entry_count;
{
amduat_asl_log_chunk_t *next =
(amduat_asl_log_chunk_t *)realloc(
chunks, (chunks_len + 1u) * sizeof(*chunks));
if (next == NULL) {
amduat_asl_log_chunk_free(&head_chunk);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
chunks = next;
chunks[chunks_len++] = head_chunk;
}
if (from_offset >= head_end_offset) {
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
range.next_offset = from_offset;
goto encode_range;
}
while (true) {
amduat_asl_log_chunk_t chunk;
amduat_artifact_t prev_artifact;
if (chunks[chunks_len - 1u].base_offset <= from_offset ||
!chunks[chunks_len - 1u].has_prev) {
break;
}
memset(&prev_artifact, 0, sizeof(prev_artifact));
if (!amduat_pel_kernel_resolve_artifact(
chunks[chunks_len - 1u].prev_ref, &prev_artifact)) {
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_REF_RESOLVE_FAILED;
}
return false;
}
if (!amduat_log_chunk_from_artifact(&prev_artifact, &chunk)) {
amduat_artifact_free(&prev_artifact);
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID;
}
return false;
}
amduat_artifact_free(&prev_artifact);
{
amduat_asl_log_chunk_t *next =
(amduat_asl_log_chunk_t *)realloc(
chunks, (chunks_len + 1u) * sizeof(*chunks));
if (next == NULL) {
amduat_asl_log_chunk_free(&chunk);
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
chunks = next;
chunks[chunks_len++] = chunk;
}
}
if (limit > SIZE_MAX) {
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID;
}
return false;
}
if (limit != 0u) {
range.refs = (amduat_reference_t *)calloc(limit, sizeof(*range.refs));
if (range.refs == NULL) {
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
}
for (size_t i = 0u; i < chunks_len / 2u; ++i) {
amduat_asl_log_chunk_t tmp = chunks[i];
chunks[i] = chunks[chunks_len - 1u - i];
chunks[chunks_len - 1u - i] = tmp;
}
range.refs_len = 0u;
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_t *chunk = &chunks[i];
for (uint32_t j = 0u; j < chunk->entry_count; ++j) {
uint64_t offset = chunk->base_offset + j;
if (offset < from_offset) {
continue;
}
if (range.refs_len == limit) {
goto done_collect;
}
if (!amduat_reference_clone(chunk->entries[j].payload_ref,
&range.refs[range.refs_len])) {
for (size_t k = 0u; k < chunks_len; ++k) {
amduat_asl_log_chunk_free(&chunks[k]);
}
free(chunks);
amduat_asl_log_range_free(&range);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
range.refs_len++;
range.next_offset = offset + 1u;
}
}
done_collect:
if (range.refs_len < limit) {
range.next_offset = head_end_offset;
}
for (size_t i = 0u; i < chunks_len; ++i) {
amduat_asl_log_chunk_free(&chunks[i]);
}
free(chunks);
}
encode_range:
if (!amduat_asl_log_range_encode_v1(&range, &encoded)) {
amduat_asl_log_range_free(&range);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
if (!amduat_alloc_outputs(1u, out_outputs)) {
amduat_asl_log_range_free(&range);
free((void *)encoded.data);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
(*out_outputs)[0] = amduat_artifact_with_type(
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1));
if (out_outputs_len != NULL) {
*out_outputs_len = 1u;
}
ok = true;
amduat_asl_log_range_free(&range);
return ok;
}
bool amduat_kernel_collection_merge_refs(
const amduat_artifact_t *inputs,
size_t inputs_len,
amduat_artifact_t **out_outputs,
size_t *out_outputs_len,
uint32_t *out_status_code) {
amduat_asl_collection_snapshot_info_t snapshot;
amduat_asl_log_range_t log_range;
amduat_asl_collection_view_t view;
amduat_octets_t encoded = amduat_octets(NULL, 0u);
uint64_t from_offset = 0u;
uint32_t limit = 0u;
uint64_t limit_from_offset = 0u;
uint32_t limit_value = 0u;
size_t total_refs = 0u;
size_t start_index = 0u;
size_t end_index = 0u;
bool ok = false;
if (inputs == NULL || inputs_len != 4u) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
}
return false;
}
if (!amduat_decode_params(&inputs[2], &from_offset, &limit_value) ||
!amduat_decode_params(&inputs[3], &limit_from_offset, &limit)) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID;
}
return false;
}
(void)limit_from_offset;
(void)limit_value;
memset(&snapshot, 0, sizeof(snapshot));
memset(&log_range, 0, sizeof(log_range));
memset(&view, 0, sizeof(view));
if (!inputs[0].has_type_tag ||
inputs[0].type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1 ||
!amduat_asl_collection_snapshot_info_decode_v1(inputs[0].bytes,
&snapshot)) {
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
}
return false;
}
if (!inputs[1].has_type_tag ||
inputs[1].type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1 ||
!amduat_asl_log_range_decode_v1(inputs[1].bytes, &log_range)) {
amduat_asl_collection_snapshot_info_free(&snapshot);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID;
}
return false;
}
if (snapshot.refs_len > SIZE_MAX - log_range.refs_len) {
amduat_asl_collection_snapshot_info_free(&snapshot);
amduat_asl_log_range_free(&log_range);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_VIEW_INVALID;
}
return false;
}
total_refs = snapshot.refs_len + log_range.refs_len;
if (from_offset > SIZE_MAX) {
start_index = total_refs;
} else {
start_index = (size_t)from_offset;
if (start_index > total_refs) {
start_index = total_refs;
}
}
if (limit > SIZE_MAX || start_index > SIZE_MAX - (size_t)limit) {
end_index = total_refs;
} else {
end_index = start_index + (size_t)limit;
if (end_index > total_refs) {
end_index = total_refs;
}
}
view.refs_len = end_index - start_index;
view.refs = NULL;
if (view.refs_len != 0u) {
view.refs = (amduat_reference_t *)calloc(view.refs_len, sizeof(*view.refs));
if (view.refs == NULL) {
amduat_asl_collection_snapshot_info_free(&snapshot);
amduat_asl_log_range_free(&log_range);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
}
for (size_t i = 0u; i < view.refs_len; ++i) {
size_t idx = start_index + i;
const amduat_reference_t *src =
(idx < snapshot.refs_len)
? &snapshot.refs[idx]
: &log_range.refs[idx - snapshot.refs_len];
if (!amduat_reference_clone(*src, &view.refs[i])) {
amduat_asl_collection_view_free(&view);
amduat_asl_collection_snapshot_info_free(&snapshot);
amduat_asl_log_range_free(&log_range);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
}
view.computed_from_offset = from_offset;
if (log_range.refs_len < limit) {
view.computed_up_to_offset = log_range.next_offset;
} else if (from_offset > UINT64_MAX - (uint64_t)view.refs_len) {
view.computed_up_to_offset = UINT64_MAX;
} else {
view.computed_up_to_offset = from_offset + (uint64_t)view.refs_len;
}
if (!amduat_asl_collection_view_encode_v1(&view, &encoded)) {
amduat_asl_collection_view_free(&view);
amduat_asl_collection_snapshot_info_free(&snapshot);
amduat_asl_log_range_free(&log_range);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
if (!amduat_alloc_outputs(1u, out_outputs)) {
amduat_asl_collection_view_free(&view);
amduat_asl_collection_snapshot_info_free(&snapshot);
amduat_asl_log_range_free(&log_range);
free((void *)encoded.data);
if (out_status_code) {
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
}
return false;
}
(*out_outputs)[0] = amduat_artifact_with_type(
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_COLLECTION_VIEW_1));
if (out_outputs_len != NULL) {
*out_outputs_len = 1u;
}
ok = true;
amduat_asl_collection_view_free(&view);
amduat_asl_collection_snapshot_info_free(&snapshot);
amduat_asl_log_range_free(&log_range);
return ok;
}

View file

@ -0,0 +1,29 @@
#ifndef AMDUAT_PEL_OPREG_KERNEL_COLLECTION_H
#define AMDUAT_PEL_OPREG_KERNEL_COLLECTION_H
#include "amduat/pel/opreg_kernel.h"
#include <stddef.h>
bool amduat_kernel_collection_snapshot_decode(
const amduat_artifact_t *inputs,
size_t inputs_len,
amduat_artifact_t **out_outputs,
size_t *out_outputs_len,
uint32_t *out_status_code);
bool amduat_kernel_log_read_range(
const amduat_artifact_t *inputs,
size_t inputs_len,
amduat_artifact_t **out_outputs,
size_t *out_outputs_len,
uint32_t *out_status_code);
bool amduat_kernel_collection_merge_refs(
const amduat_artifact_t *inputs,
size_t inputs_len,
amduat_artifact_t **out_outputs,
size_t *out_outputs_len,
uint32_t *out_status_code);
#endif /* AMDUAT_PEL_OPREG_KERNEL_COLLECTION_H */

View file

@ -133,6 +133,12 @@ bool amduat_pel_kernel_params_decode(
return amduat_decode_unit(params_bytes); return amduat_decode_unit(params_bytes);
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE: case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
return amduat_decode_unit(params_bytes); return amduat_decode_unit(params_bytes);
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
return amduat_decode_unit(params_bytes);
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
return amduat_decode_unit(params_bytes);
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
return amduat_decode_unit(params_bytes);
default: default:
return false; return false;
} }

View file

@ -1,12 +1,18 @@
#include "amduat/pel/surf.h" #include "amduat/pel/surf.h"
#include "amduat/asl/asl_materialization_cache_fs.h"
#include "amduat/asl/asl_store_fs.h"
#include "amduat/enc/pel1_result.h" #include "amduat/enc/pel1_result.h"
#include "amduat/enc/pel_program_dag.h" #include "amduat/enc/pel_program_dag.h"
#include "amduat/enc/pel_trace_dag.h" #include "amduat/enc/pel_trace_dag.h"
#include "amduat/pel/derivation_sid.h"
#include "amduat/pel/opreg_kernel.h"
#include "amduat/util/log.h"
#include "pel_program_dag_internal.h" #include "pel_program_dag_internal.h"
#include "amduat/pel/program_dag_desc.h" #include "amduat/pel/program_dag_desc.h"
#include <stdarg.h> #include <stdarg.h>
#include <assert.h>
#include <stdbool.h> #include <stdbool.h>
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <stdint.h>
@ -30,6 +36,17 @@ void amduat_pel_surf_free_ref(amduat_reference_t *ref) {
amduat_reference_free(ref); amduat_reference_free(ref);
} }
static bool amduat_pel_surf_resolve_artifact(amduat_reference_t ref,
amduat_artifact_t *out_artifact,
void *ctx) {
amduat_asl_store_t *store = (amduat_asl_store_t *)ctx;
if (store == NULL || out_artifact == NULL) {
return false;
}
return amduat_asl_store_get(store, ref, out_artifact) ==
AMDUAT_ASL_STORE_OK;
}
static void amduat_init_core_result(amduat_pel_execution_result_value_t *result, static void amduat_init_core_result(amduat_pel_execution_result_value_t *result,
amduat_reference_t scheme_ref, amduat_reference_t scheme_ref,
amduat_pel_execution_status_t status, amduat_pel_execution_status_t status,
@ -217,6 +234,37 @@ static void amduat_trace_nodes_free(amduat_pel_node_trace_dag_t *nodes,
free(nodes); free(nodes);
} }
static const char k_materialized_hit_op_name[] = "materialized_hit";
static bool amduat_pel_surf_store_fs_ctx(
const amduat_asl_store_t *store,
amduat_asl_store_fs_t **out_fs) {
amduat_asl_store_ops_t fs_ops;
if (out_fs == NULL) {
return false;
}
*out_fs = NULL;
if (store == NULL) {
return false;
}
fs_ops = amduat_asl_store_fs_ops();
if (store->ops.put != fs_ops.put ||
store->ops.get != fs_ops.get ||
store->ops.put_indexed != fs_ops.put_indexed ||
store->ops.get_indexed != fs_ops.get_indexed ||
store->ops.tombstone != fs_ops.tombstone ||
store->ops.tombstone_lift != fs_ops.tombstone_lift ||
store->ops.log_scan != fs_ops.log_scan ||
store->ops.current_state != fs_ops.current_state ||
store->ops.validate_config != fs_ops.validate_config) {
return false;
}
*out_fs = (amduat_asl_store_fs_t *)store->ctx;
return *out_fs != NULL;
}
static bool amduat_store_trace( static bool amduat_store_trace(
amduat_asl_store_t *store, amduat_asl_store_t *store,
amduat_reference_t scheme_ref, amduat_reference_t scheme_ref,
@ -230,6 +278,7 @@ static bool amduat_store_trace(
const amduat_pel_execution_result_value_t *core_result, const amduat_pel_execution_result_value_t *core_result,
const amduat_pel_program_t *program, const amduat_pel_program_t *program,
const amduat_pel_program_dag_trace_t *trace_eval, const amduat_pel_program_dag_trace_t *trace_eval,
bool materialized_hit,
amduat_reference_t *out_trace_ref) { amduat_reference_t *out_trace_ref) {
amduat_pel_trace_dag_value_t trace; amduat_pel_trace_dag_value_t trace;
amduat_pel_node_trace_dag_t *node_traces; amduat_pel_node_trace_dag_t *node_traces;
@ -311,6 +360,29 @@ static bool amduat_store_trace(
} }
} }
if (materialized_hit) {
amduat_pel_node_trace_dag_t *next;
/* Emit a synthetic trace entry to mark a materialization cache hit. */
next = (amduat_pel_node_trace_dag_t *)realloc(
node_traces, (node_count + 1u) * sizeof(*node_traces));
if (next == NULL) {
amduat_trace_nodes_free(node_traces, node_count);
return false;
}
node_traces = next;
memset(&node_traces[node_count], 0, sizeof(*node_traces));
/* UINT32_MAX is a reserved sentinel; decoders treat node_id as opaque. */
node_traces[node_count].node_id = UINT32_MAX;
node_traces[node_count].op_name = amduat_octets(
(const uint8_t *)k_materialized_hit_op_name,
sizeof(k_materialized_hit_op_name) - 1u);
node_traces[node_count].op_version = 1u;
node_traces[node_count].status = AMDUAT_PEL_NODE_TRACE_SKIPPED;
node_traces[node_count].status_code = 0u;
node_count += 1u;
}
memset(&trace, 0, sizeof(trace)); memset(&trace, 0, sizeof(trace));
trace.pel1_version = 1; trace.pel1_version = 1;
trace.scheme_ref = scheme_ref; trace.scheme_ref = scheme_ref;
@ -367,6 +439,8 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
size_t outputs_len; size_t outputs_len;
amduat_reference_t *output_refs; amduat_reference_t *output_refs;
size_t output_refs_len; size_t output_refs_len;
amduat_asl_materialization_cache_fs_t mat_cache;
amduat_octets_t sid;
amduat_reference_t trace_ref; amduat_reference_t trace_ref;
amduat_reference_t exec_result_ref; amduat_reference_t exec_result_ref;
size_t i; size_t i;
@ -375,6 +449,10 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
bool exec_invoked = false; bool exec_invoked = false;
bool trace_ok = false; bool trace_ok = false;
bool has_exec_result_ref = false; bool has_exec_result_ref = false;
bool mat_cache_ready = false;
bool materialized_hit = false;
bool cache_miss_due_missing = false;
bool executed = false;
if (store == NULL || out_output_refs == NULL || if (store == NULL || out_output_refs == NULL ||
out_output_refs_len == NULL || out_result_ref == NULL) { out_output_refs_len == NULL || out_result_ref == NULL) {
@ -392,6 +470,7 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
trace_ref.digest = amduat_octets(NULL, 0); trace_ref.digest = amduat_octets(NULL, 0);
exec_result_ref.hash_id = 0; exec_result_ref.hash_id = 0;
exec_result_ref.digest = amduat_octets(NULL, 0); exec_result_ref.digest = amduat_octets(NULL, 0);
sid = amduat_octets(NULL, 0);
if (!amduat_reference_eq(scheme_ref, if (!amduat_reference_eq(scheme_ref,
amduat_pel_program_dag_scheme_ref())) { amduat_pel_program_dag_scheme_ref())) {
@ -490,6 +569,75 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
has_params_artifact = true; has_params_artifact = true;
} }
/*
* Cache safety assumptions:
* - PEL DAG execution is deterministic for ProgramRef + Inputs + Params.
* - ExecProfile is currently absent and encoded as a marker only.
* - The store is single-tenant; SID collisions are treated as integrity errors.
* - This code path assumes amduat_asl_store_fs_t as the backing store.
*/
{
amduat_pel_derivation_sid_input_t sid_input;
memset(&sid_input, 0, sizeof(sid_input));
sid_input.program_ref = program_ref;
sid_input.input_refs = input_refs;
sid_input.input_refs_len = input_refs_len;
sid_input.has_params_ref = has_params_ref;
if (has_params_ref) {
sid_input.params_ref = params_ref;
}
sid_input.has_exec_profile = false;
sid_input.exec_profile = amduat_octets(NULL, 0u);
if (amduat_pel_derivation_sid_compute(&sid_input, &sid)) {
amduat_asl_store_fs_t *fs = NULL;
if (amduat_pel_surf_store_fs_ctx(store, &fs) &&
amduat_asl_materialization_cache_fs_init(&mat_cache,
fs->root_path)) {
mat_cache_ready = true;
}
}
}
if (mat_cache_ready) {
amduat_reference_t *cached_refs = NULL;
size_t cached_len = 0;
amduat_asl_store_error_t cache_err;
cache_err = amduat_asl_materialization_cache_fs_get(&mat_cache, sid,
&cached_refs,
&cached_len);
if (cache_err == AMDUAT_ASL_STORE_OK) {
bool cache_valid = true;
for (i = 0; i < cached_len; ++i) {
amduat_artifact_t cached_artifact;
if (amduat_asl_store_get(store, cached_refs[i],
&cached_artifact) !=
AMDUAT_ASL_STORE_OK) {
cache_valid = false;
break;
}
amduat_artifact_free(&cached_artifact);
}
if (cache_valid) {
output_refs = cached_refs;
output_refs_len = cached_len;
materialized_hit = true;
exec_invoked = true;
amduat_init_core_result(&core_result, scheme_ref,
AMDUAT_PEL_EXEC_STATUS_OK,
AMDUAT_PEL_EXEC_ERROR_NONE, 0);
outputs = NULL;
outputs_len = 0;
amduat_artifact_free(&program_artifact);
goto cache_hit;
}
cache_miss_due_missing = true;
amduat_pel_surf_free_refs(cached_refs, cached_len);
}
}
exec_invoked = true; exec_invoked = true;
outputs = NULL; outputs = NULL;
outputs_len = 0; outputs_len = 0;
@ -555,9 +703,12 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
program_decoded = true; program_decoded = true;
const amduat_artifact_t *params_arg = const amduat_artifact_t *params_arg =
has_params_artifact ? &params_artifact : NULL; has_params_artifact ? &params_artifact : NULL;
amduat_pel_kernel_set_artifact_resolver(
amduat_pel_surf_resolve_artifact, store);
if (!amduat_pel_program_dag_exec_trace( if (!amduat_pel_program_dag_exec_trace(
&program, input_artifacts, input_refs_len, params_arg, &outputs, &program, input_artifacts, input_refs_len, params_arg, &outputs,
&outputs_len, &core_result, &trace_eval)) { &outputs_len, &core_result, &trace_eval)) {
amduat_pel_kernel_clear_artifact_resolver();
amduat_artifact_free(&program_artifact); amduat_artifact_free(&program_artifact);
amduat_enc_pel_program_dag_free(&program); amduat_enc_pel_program_dag_free(&program);
for (i = 0; i < input_refs_len; ++i) { for (i = 0; i < input_refs_len; ++i) {
@ -570,6 +721,8 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
amduat_pel_program_dag_free_outputs(outputs, outputs_len); amduat_pel_program_dag_free_outputs(outputs, outputs_len);
return false; return false;
} }
amduat_pel_kernel_clear_artifact_resolver();
executed = true;
} }
} }
} }
@ -593,6 +746,27 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
} }
} }
if (mat_cache_ready) {
(void)amduat_asl_materialization_cache_fs_put(&mat_cache, sid,
output_refs, output_refs_len);
}
cache_hit:
#ifndef NDEBUG
if (cache_miss_due_missing) {
assert(!materialized_hit);
}
#endif
if (materialized_hit) {
amduat_log(AMDUAT_LOG_DEBUG, "pel surf: materialization hit");
} else if (executed) {
if (cache_miss_due_missing) {
amduat_log(AMDUAT_LOG_DEBUG,
"pel surf: executed (cache miss: missing outputs)");
} else {
amduat_log(AMDUAT_LOG_DEBUG, "pel surf: executed");
}
}
trace_ok = false; trace_ok = false;
if (exec_invoked) { if (exec_invoked) {
if (!amduat_store_surface_result( if (!amduat_store_surface_result(
@ -607,7 +781,9 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
has_params_ref, params_ref, has_exec_result_ref, exec_result_ref, has_params_ref, params_ref, has_exec_result_ref, exec_result_ref,
&core_result, &core_result,
program_decoded ? &program : NULL, program_decoded ? &program : NULL,
program_decoded ? &trace_eval : NULL, &trace_ref); program_decoded ? &trace_eval : NULL,
materialized_hit,
&trace_ref);
if (!trace_ok) { if (!trace_ok) {
goto cleanup; goto cleanup;
} }
@ -637,6 +813,7 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
amduat_pel_program_dag_free_outputs(outputs, outputs_len); amduat_pel_program_dag_free_outputs(outputs, outputs_len);
amduat_pel_execution_result_free(&core_result); amduat_pel_execution_result_free(&core_result);
amduat_pel_surf_free_ref(&exec_result_ref); amduat_pel_surf_free_ref(&exec_result_ref);
amduat_octets_free(&sid);
return true; return true;
cleanup: cleanup:
@ -655,5 +832,6 @@ cleanup:
amduat_pel_program_dag_free_outputs(outputs, outputs_len); amduat_pel_program_dag_free_outputs(outputs, outputs_len);
amduat_pel_execution_result_free(&core_result); amduat_pel_execution_result_free(&core_result);
amduat_pel_surf_free_ref(&exec_result_ref); amduat_pel_surf_free_ref(&exec_result_ref);
amduat_octets_free(&sid);
return false; return false;
} }

File diff suppressed because it is too large Load diff

1012
src/tools/amduat_pel_build.c Normal file

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -53,7 +53,8 @@ static void amduat_pel_run_print_usage(FILE *stream) {
"\n" "\n"
"notes:\n" "notes:\n"
" REF values are ASL reference hex strings.\n" " REF values are ASL reference hex strings.\n"
" --output-raw writes output bytes to stdout and exec result to stderr.\n", " --output-raw writes output bytes to stdout and exec result to stderr.\n"
" Uses the PEL/PROGRAM-DAG/1 scheme ref.\n",
AMDUAT_PEL_RUN_DEFAULT_ROOT); AMDUAT_PEL_RUN_DEFAULT_ROOT);
} }

View file

@ -69,7 +69,10 @@ static void amduat_pel_seed_print_usage(FILE *stream) {
" --root %s\n" " --root %s\n"
" --seed const\n" " --seed const\n"
" --ref-format hex\n" " --ref-format hex\n"
" --output -\n", " --output -\n"
"\n"
"notes:\n"
" --list prints available seeds and exits\n",
AMDUAT_PEL_SEED_DEFAULT_ROOT); AMDUAT_PEL_SEED_DEFAULT_ROOT);
} }

View file

@ -0,0 +1,231 @@
#include "amduat/asl/asl_derivation_index_fs.h"
#include <dirent.h>
#include <errno.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
static char *make_temp_root(void) {
char *templ;
const char template_prefix[] =
"/tmp/amduat_test_asl_derivation_index_fs_XXXXXX";
templ = (char *)malloc(sizeof(template_prefix));
if (templ == NULL) {
return NULL;
}
memcpy(templ, template_prefix, sizeof(template_prefix));
if (mkdtemp(templ) == NULL) {
free(templ);
return NULL;
}
return templ;
}
static bool remove_tree(const char *path) {
DIR *dir;
struct dirent *entry;
struct stat st;
if (path == NULL) {
return false;
}
if (lstat(path, &st) != 0) {
return errno == ENOENT;
}
if (!S_ISDIR(st.st_mode)) {
return unlink(path) == 0;
}
dir = opendir(path);
if (dir == NULL) {
return false;
}
while ((entry = readdir(dir)) != NULL) {
char child[2048];
if (strcmp(entry->d_name, ".") == 0 ||
strcmp(entry->d_name, "..") == 0) {
continue;
}
snprintf(child, sizeof(child), "%s/%s", path, entry->d_name);
if (!remove_tree(child)) {
closedir(dir);
return false;
}
}
closedir(dir);
return rmdir(path) == 0;
}
static amduat_reference_t make_ref(uint8_t seed, size_t len) {
uint8_t *bytes = (uint8_t *)malloc(len);
if (bytes == NULL) {
return amduat_reference(0u, amduat_octets(NULL, 0u));
}
for (size_t i = 0; i < len; ++i) {
bytes[i] = (uint8_t)(seed + i);
}
return amduat_reference(0x1234u, amduat_octets(bytes, len));
}
static int test_round_trip(void) {
amduat_asl_derivation_index_fs_t index;
amduat_asl_derivation_record_t record;
amduat_asl_derivation_record_t record2;
amduat_asl_derivation_record_t *records = NULL;
size_t records_len = 0u;
amduat_reference_t artifact_ref;
amduat_reference_t inputs[2];
uint8_t sid_bytes[32];
uint8_t exec_bytes[4] = {0x01, 0x02, 0x03, 0x04};
char *root;
int exit_code = 1;
root = make_temp_root();
if (root == NULL) {
fprintf(stderr, "temp root failed\n");
return 1;
}
if (!amduat_asl_derivation_index_fs_init(&index, root)) {
fprintf(stderr, "index init failed\n");
goto cleanup;
}
artifact_ref = make_ref(0x10, 4);
record.program_ref = make_ref(0x20, 4);
record.output_index = 0u;
inputs[0] = make_ref(0x30, 4);
inputs[1] = make_ref(0x40, 4);
memset(sid_bytes, 0xa5, sizeof(sid_bytes));
record.sid = amduat_octets(sid_bytes, sizeof(sid_bytes));
record.input_refs = inputs;
record.input_refs_len = 2u;
record.has_params_ref = true;
record.params_ref = make_ref(0x50, 4);
record.has_exec_profile = true;
record.exec_profile = amduat_octets(exec_bytes, sizeof(exec_bytes));
record2 = record;
record2.sid = amduat_octets(sid_bytes, sizeof(sid_bytes));
record2.output_index = 1u;
record2.has_params_ref = false;
record2.has_exec_profile = false;
{
amduat_asl_store_error_t err =
amduat_asl_derivation_index_fs_add(&index, artifact_ref, &record);
if (err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "add record failed: %d\n", err);
goto cleanup_records;
}
}
{
amduat_asl_store_error_t err =
amduat_asl_derivation_index_fs_add(&index, artifact_ref, &record2);
if (err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "add record2 failed: %d\n", err);
goto cleanup_records;
}
}
if (amduat_asl_derivation_index_fs_list(&index,
artifact_ref,
&records,
&records_len) !=
AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "list failed\n");
goto cleanup_records;
}
if (records_len != 2u) {
fprintf(stderr, "record count mismatch\n");
goto cleanup_records_list;
}
if (records[0].input_refs_len != 2u ||
records[0].has_params_ref != true ||
records[0].has_exec_profile != true ||
records[0].output_index != 0u) {
fprintf(stderr, "record fields mismatch\n");
goto cleanup_records_list;
}
if (records[1].has_params_ref != false ||
records[1].has_exec_profile != false ||
records[1].output_index != 1u) {
fprintf(stderr, "record2 flags mismatch\n");
goto cleanup_records_list;
}
exit_code = 0;
cleanup_records_list:
amduat_asl_derivation_records_free(records, records_len);
free(records);
cleanup_records:
amduat_reference_free(&artifact_ref);
amduat_reference_free(&record.program_ref);
amduat_reference_free(&record.params_ref);
amduat_reference_free(&inputs[0]);
amduat_reference_free(&inputs[1]);
cleanup:
if (!remove_tree(root)) {
fprintf(stderr, "cleanup failed\n");
exit_code = 1;
}
free(root);
return exit_code;
}
static int test_not_found(void) {
amduat_asl_derivation_index_fs_t index;
amduat_reference_t artifact_ref;
amduat_asl_derivation_record_t *records = NULL;
size_t records_len = 0u;
char *root;
int exit_code = 1;
root = make_temp_root();
if (root == NULL) {
fprintf(stderr, "temp root failed\n");
return 1;
}
if (!amduat_asl_derivation_index_fs_init(&index, root)) {
fprintf(stderr, "index init failed\n");
goto cleanup;
}
artifact_ref = make_ref(0x70, 4);
if (amduat_asl_derivation_index_fs_list(&index,
artifact_ref,
&records,
&records_len) !=
AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
fprintf(stderr, "expected not found\n");
amduat_reference_free(&artifact_ref);
goto cleanup;
}
amduat_reference_free(&artifact_ref);
exit_code = 0;
cleanup:
if (!remove_tree(root)) {
fprintf(stderr, "cleanup failed\n");
exit_code = 1;
}
free(root);
return exit_code;
}
int main(void) {
if (test_round_trip() != 0) {
return 1;
}
return test_not_found();
}

View file

@ -0,0 +1,154 @@
#include "amduat/asl/index_accel.h"
#include "amduat/asl/index_bloom.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static int test_routing_key_layout(void) {
uint8_t digest_bytes[2] = {0xaa, 0xbb};
amduat_reference_t ref =
amduat_reference(0x1234u, amduat_octets(digest_bytes, sizeof(digest_bytes)));
amduat_octets_t key;
int exit_code = 1;
if (!amduat_asl_index_accel_routing_key_from_ref(
ref, true, amduat_type_tag(0x11223344u), &key)) {
fprintf(stderr, "routing key build failed\n");
return exit_code;
}
if (key.len != 2u + 2u + sizeof(digest_bytes) + 1u + 4u) {
fprintf(stderr, "routing key length mismatch\n");
goto cleanup;
}
if (key.data[0] != 0x34u || key.data[1] != 0x12u ||
key.data[2] != 0x02u || key.data[3] != 0x00u) {
fprintf(stderr, "routing key header mismatch\n");
goto cleanup;
}
if (key.data[4] != 0xaau || key.data[5] != 0xbbu) {
fprintf(stderr, "routing key digest mismatch\n");
goto cleanup;
}
if (key.data[6] != 0x01u) {
fprintf(stderr, "routing key type tag flag mismatch\n");
goto cleanup;
}
if (key.data[7] != 0x44u || key.data[8] != 0x33u ||
key.data[9] != 0x22u || key.data[10] != 0x11u) {
fprintf(stderr, "routing key type tag bytes mismatch\n");
goto cleanup;
}
exit_code = 0;
cleanup:
amduat_octets_free(&key);
return exit_code;
}
static int test_routing_key_absence(void) {
uint8_t digest_bytes[1] = {0x7f};
amduat_reference_t ref =
amduat_reference(0x0001u, amduat_octets(digest_bytes, sizeof(digest_bytes)));
amduat_octets_t key;
int exit_code = 1;
if (!amduat_asl_index_accel_routing_key_from_ref(
ref, false, amduat_type_tag(0xdeadbeefu), &key)) {
fprintf(stderr, "routing key build failed\n");
return exit_code;
}
if (key.data[4] != 0x7fu) {
fprintf(stderr, "routing key digest mismatch\n");
goto cleanup;
}
if (key.data[5] != 0x00u) {
fprintf(stderr, "routing key absence flag mismatch\n");
goto cleanup;
}
if (key.data[6] != 0x00u || key.data[7] != 0x00u ||
key.data[8] != 0x00u || key.data[9] != 0x00u) {
fprintf(stderr, "routing key absence tag mismatch\n");
goto cleanup;
}
exit_code = 0;
cleanup:
amduat_octets_free(&key);
return exit_code;
}
static int test_shard_determinism(void) {
uint8_t digest_bytes[3] = {0x01, 0x02, 0x03};
amduat_reference_t ref =
amduat_reference(0x00f0u, amduat_octets(digest_bytes, sizeof(digest_bytes)));
uint16_t shard_a;
uint16_t shard_b;
shard_a = amduat_asl_index_accel_shard_for_ref(
ref, false, amduat_type_tag(0u), 8u);
shard_b = amduat_asl_index_accel_shard_for_ref(
ref, false, amduat_type_tag(0u), 8u);
if (shard_a != shard_b) {
fprintf(stderr, "shard selection mismatch\n");
return 1;
}
if (shard_a >= 8u) {
fprintf(stderr, "shard out of range\n");
return 1;
}
return 0;
}
static int test_bloom_advisory(void) {
uint8_t digest_bytes[4] = {0x10, 0x20, 0x30, 0x40};
amduat_octets_t bloom;
amduat_octets_t empty = amduat_octets(NULL, 0u);
int exit_code = 1;
if (!amduat_asl_index_bloom_init(&bloom)) {
fprintf(stderr, "bloom init failed\n");
return exit_code;
}
if (!amduat_asl_index_bloom_add(
bloom, 0x0001u, amduat_octets(digest_bytes, sizeof(digest_bytes)))) {
fprintf(stderr, "bloom add failed\n");
goto cleanup;
}
if (!amduat_asl_index_bloom_maybe_contains(
bloom, 0x0001u, amduat_octets(digest_bytes, sizeof(digest_bytes)))) {
fprintf(stderr, "bloom false negative\n");
goto cleanup;
}
if (!amduat_asl_index_bloom_maybe_contains(
empty, 0x0002u, amduat_octets(NULL, 0u))) {
fprintf(stderr, "empty bloom should be advisory\n");
goto cleanup;
}
exit_code = 0;
cleanup:
amduat_octets_free(&bloom);
return exit_code;
}
int main(void) {
if (test_routing_key_layout() != 0) {
return 1;
}
if (test_routing_key_absence() != 0) {
return 1;
}
if (test_shard_determinism() != 0) {
return 1;
}
return test_bloom_advisory();
}

View file

@ -0,0 +1,176 @@
#include "amduat/asl/asl_store_index_fs.h"
#include "amduat/asl/store.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/hash/asl1.h"
#include <dirent.h>
#include <errno.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
static bool join_path(const char *base, const char *segment, char **out_path) {
size_t base_len;
size_t seg_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
base_len = strlen(base);
seg_len = strlen(segment);
if (base_len == 0u || seg_len == 0u) {
return false;
}
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + seg_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, seg_len);
offset += seg_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool remove_tree(const char *path) {
struct stat st;
DIR *dir;
struct dirent *entry;
if (path == NULL) {
return false;
}
if (lstat(path, &st) != 0) {
return errno == ENOENT;
}
if (!S_ISDIR(st.st_mode)) {
return unlink(path) == 0;
}
dir = opendir(path);
if (dir == NULL) {
return false;
}
while ((entry = readdir(dir)) != NULL) {
char *child = NULL;
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0) {
continue;
}
if (!join_path(path, entry->d_name, &child)) {
closedir(dir);
return false;
}
if (!remove_tree(child)) {
free(child);
closedir(dir);
return false;
}
free(child);
}
if (closedir(dir) != 0) {
return false;
}
return rmdir(path) == 0;
}
static char *make_temp_root(void) {
char *tmpl = NULL;
char *root = NULL;
tmpl = strdup("/tmp/amduat-index-putget-XXXXXX");
if (tmpl == NULL) {
return NULL;
}
if (mkdtemp(tmpl) == NULL) {
free(tmpl);
return NULL;
}
root = tmpl;
return root;
}
int main(void) {
amduat_asl_store_config_t config;
amduat_asl_store_index_fs_t fs;
amduat_asl_store_t store;
char *root = NULL;
int rc = 1;
size_t i;
root = make_temp_root();
if (root == NULL) {
fprintf(stderr, "temp root creation failed\n");
return 1;
}
memset(&config, 0, sizeof(config));
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
if (!amduat_asl_store_index_fs_init(&fs, config, root)) {
fprintf(stderr, "index fs init failed\n");
goto cleanup;
}
amduat_asl_store_init(&store, config, amduat_asl_store_index_fs_ops(), &fs);
for (i = 0u; i < 128u; ++i) {
char payload_buf[64];
size_t payload_len;
amduat_artifact_t artifact;
amduat_reference_t ref;
amduat_artifact_t loaded;
amduat_asl_store_error_t err;
payload_len = (size_t)snprintf(payload_buf, sizeof(payload_buf),
"payload-%zu", i);
if (payload_len == 0u || payload_len >= sizeof(payload_buf)) {
fprintf(stderr, "payload format failed at %zu\n", i);
goto cleanup;
}
artifact = amduat_artifact(
amduat_octets((const uint8_t *)payload_buf, payload_len));
ref = amduat_reference(0u, amduat_octets(NULL, 0u));
err = amduat_asl_store_put(&store, artifact, &ref);
if (err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "put failed at %zu: %d\n", i, (int)err);
goto cleanup;
}
loaded = amduat_artifact(amduat_octets(NULL, 0u));
err = amduat_asl_store_get(&store, ref, &loaded);
if (err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "get after put failed at %zu: %d\n", i, (int)err);
amduat_reference_free(&ref);
goto cleanup;
}
amduat_artifact_free(&loaded);
amduat_reference_free(&ref);
}
rc = 0;
cleanup:
if (root != NULL) {
if (!remove_tree(root)) {
fprintf(stderr, "warning: cleanup failed for %s\n", root);
}
free(root);
}
return rc;
}

View file

@ -0,0 +1,932 @@
#include "amduat/asl/asl_store_index_fs.h"
#include "amduat/asl/index_replay.h"
#include "amduat/asl/ref_derive.h"
#include "amduat/asl/store.h"
#include "amduat/enc/asl_core_index.h"
#include "amduat/enc/asl_log.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/hash/asl1.h"
#include "asl_store_index_fs_layout.h"
#include <dirent.h>
#include <errno.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
static void store_u16_le(uint8_t *out, uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static size_t build_artifact_ref(uint8_t *out,
size_t out_cap,
amduat_hash_id_t hash_id,
const uint8_t *digest,
size_t digest_len) {
size_t total = 4 + 2 + 2 + digest_len;
if (out_cap < total) {
return 0;
}
store_u32_le(out, (uint32_t)hash_id);
store_u16_le(out + 4, (uint16_t)digest_len);
store_u16_le(out + 6, 0);
memcpy(out + 8, digest, digest_len);
return total;
}
static size_t build_tombstone_payload(uint8_t *out,
size_t out_cap,
amduat_hash_id_t hash_id,
const uint8_t *digest,
size_t digest_len) {
size_t offset = 0;
size_t ref_len = build_artifact_ref(out,
out_cap,
hash_id,
digest,
digest_len);
if (ref_len == 0 || out_cap < ref_len + 8) {
return 0;
}
offset += ref_len;
store_u32_le(out + offset, 0);
offset += 4;
store_u32_le(out + offset, 0);
offset += 4;
return offset;
}
static size_t build_tombstone_lift_payload(uint8_t *out,
size_t out_cap,
amduat_hash_id_t hash_id,
const uint8_t *digest,
size_t digest_len,
uint64_t tombstone_logseq) {
size_t offset = 0;
size_t ref_len = build_artifact_ref(out,
out_cap,
hash_id,
digest,
digest_len);
if (ref_len == 0 || out_cap < ref_len + 8) {
return 0;
}
offset += ref_len;
store_u64_le(out + offset, tombstone_logseq);
offset += 8;
return offset;
}
static size_t build_segment_seal_payload(uint8_t *out,
size_t out_cap,
uint64_t segment_id,
const uint8_t hash[32]) {
if (out_cap < 8 + 32) {
return 0;
}
store_u64_le(out, segment_id);
memcpy(out + 8, hash, 32);
return 8 + 32;
}
static size_t build_snapshot_anchor_payload(uint8_t *out,
size_t out_cap,
uint64_t snapshot_id,
const uint8_t root_hash[32]) {
if (out_cap < 8 + 32) {
return 0;
}
store_u64_le(out, snapshot_id);
memcpy(out + 8, root_hash, 32);
return 8 + 32;
}
static bool join_path(const char *base, const char *segment, char **out_path) {
size_t base_len;
size_t seg_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || segment[0] == '\0') {
return false;
}
base_len = strlen(base);
seg_len = strlen(segment);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + seg_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, seg_len);
offset += seg_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool remove_tree(const char *path) {
struct stat st;
DIR *dir;
struct dirent *entry;
if (path == NULL) {
return false;
}
if (lstat(path, &st) != 0) {
return errno == ENOENT;
}
if (!S_ISDIR(st.st_mode)) {
return unlink(path) == 0;
}
dir = opendir(path);
if (dir == NULL) {
return false;
}
while ((entry = readdir(dir)) != NULL) {
char *child = NULL;
if (strcmp(entry->d_name, ".") == 0 ||
strcmp(entry->d_name, "..") == 0) {
continue;
}
if (!join_path(path, entry->d_name, &child)) {
closedir(dir);
return false;
}
if (!remove_tree(child)) {
free(child);
closedir(dir);
return false;
}
free(child);
}
if (closedir(dir) != 0) {
return false;
}
return rmdir(path) == 0;
}
static char *make_temp_root(void) {
char *templ;
const char template_prefix[] = "/tmp/amduat_test_asl_index_replay_XXXXXX";
templ = (char *)malloc(sizeof(template_prefix));
if (templ == NULL) {
return NULL;
}
memcpy(templ, template_prefix, sizeof(template_prefix));
if (mkdtemp(templ) == NULL) {
free(templ);
return NULL;
}
return templ;
}
static bool ensure_dir(const char *path) {
if (mkdir(path, 0700) == 0) {
return true;
}
return errno == EEXIST;
}
static bool write_file(const char *path, const uint8_t *data, size_t len) {
FILE *file;
size_t written;
file = fopen(path, "wb");
if (file == NULL) {
return false;
}
written = 0u;
if (len != 0) {
written = fwrite(data, 1u, len, file);
}
if (written != len) {
fclose(file);
return false;
}
return fclose(file) == 0;
}
static bool read_file(const char *path, uint8_t **out_bytes, size_t *out_len) {
FILE *file;
long size;
uint8_t *buffer;
if (out_bytes == NULL || out_len == NULL) {
return false;
}
*out_bytes = NULL;
*out_len = 0u;
file = fopen(path, "rb");
if (file == NULL) {
return false;
}
if (fseek(file, 0, SEEK_END) != 0) {
fclose(file);
return false;
}
size = ftell(file);
if (size < 0) {
fclose(file);
return false;
}
if (fseek(file, 0, SEEK_SET) != 0) {
fclose(file);
return false;
}
buffer = (uint8_t *)malloc((size_t)size);
if (buffer == NULL) {
fclose(file);
return false;
}
if (size != 0 && fread(buffer, 1u, (size_t)size, file) != (size_t)size) {
free(buffer);
fclose(file);
return false;
}
if (fclose(file) != 0) {
free(buffer);
return false;
}
*out_bytes = buffer;
*out_len = (size_t)size;
return true;
}
static bool prepare_index_tree(const char *root) {
char *index_path = NULL;
char *segments_path = NULL;
char *blocks_path = NULL;
bool ok = false;
if (!amduat_asl_store_index_fs_layout_build_index_path(root,
&index_path) ||
!amduat_asl_store_index_fs_layout_build_segments_path(root,
&segments_path) ||
!amduat_asl_store_index_fs_layout_build_blocks_path(root,
&blocks_path)) {
goto cleanup;
}
if (!ensure_dir(index_path) ||
!ensure_dir(segments_path) ||
!ensure_dir(blocks_path)) {
goto cleanup;
}
ok = true;
cleanup:
free(index_path);
free(segments_path);
free(blocks_path);
return ok;
}
static bool write_log(const char *root,
amduat_asl_log_record_t *records,
size_t record_count) {
char *log_path = NULL;
amduat_octets_t bytes;
bool ok = false;
if (!amduat_enc_asl_log_encode_v1(records, record_count, &bytes)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_build_log_path(root, &log_path)) {
free((void *)bytes.data);
return false;
}
ok = write_file(log_path, bytes.data, bytes.len);
free((void *)bytes.data);
free(log_path);
return ok;
}
static bool load_log_records(const char *root,
amduat_asl_log_record_t **out_records,
size_t *out_count) {
char *log_path = NULL;
uint8_t *bytes = NULL;
size_t len = 0;
bool ok = false;
if (!amduat_asl_store_index_fs_layout_build_log_path(root, &log_path)) {
return false;
}
if (!read_file(log_path, &bytes, &len)) {
free(log_path);
return false;
}
ok = amduat_enc_asl_log_decode_v1(amduat_octets(bytes, len),
out_records,
out_count);
free(bytes);
free(log_path);
return ok;
}
static bool write_block_file(const char *root,
uint64_t block_id,
amduat_octets_t bytes) {
char *block_path = NULL;
bool ok = false;
if (!amduat_asl_store_index_fs_layout_build_block_path(root,
block_id,
&block_path)) {
return false;
}
ok = write_file(block_path, bytes.data, bytes.len);
free(block_path);
return ok;
}
static bool write_segment_file(const char *root,
uint64_t segment_id,
const amduat_asl_core_index_segment_t *segment,
uint8_t out_hash[32]) {
char *segment_path = NULL;
amduat_octets_t encoded;
bool ok = false;
if (!amduat_enc_asl_core_index_encode_v1(segment, &encoded)) {
return false;
}
if (!amduat_hash_asl1_digest(AMDUAT_HASH_ASL1_ID_SHA256,
encoded,
out_hash,
32)) {
free((void *)encoded.data);
return false;
}
if (!amduat_asl_store_index_fs_layout_build_segment_path(root,
segment_id,
&segment_path)) {
free((void *)encoded.data);
return false;
}
ok = write_file(segment_path, encoded.data, encoded.len);
free((void *)encoded.data);
free(segment_path);
return ok;
}
static bool build_segment_for_artifact(
amduat_asl_core_index_segment_t *segment,
amduat_asl_index_record_t *record,
amduat_asl_extent_record_t *extent,
amduat_reference_t ref,
amduat_octets_t artifact_bytes,
uint64_t snapshot_id,
uint64_t block_id) {
if (segment == NULL || record == NULL || extent == NULL) {
return false;
}
memset(segment, 0, sizeof(*segment));
memset(record, 0, sizeof(*record));
memset(extent, 0, sizeof(*extent));
segment->header.snapshot_min = snapshot_id;
segment->header.snapshot_max = snapshot_id;
segment->header.segment_domain_id = 0;
segment->header.segment_visibility = 1;
segment->header.federation_version = 0;
segment->header.flags = 0;
segment->header.reserved0 = 0;
record->hash_id = ref.hash_id;
record->digest_len = (uint16_t)ref.digest.len;
record->extent_count = 1;
record->total_length = (uint32_t)artifact_bytes.len;
record->domain_id = 0;
record->visibility = 1;
record->has_cross_domain_source = 0;
record->cross_domain_source = 0;
record->flags = 0;
extent->block_id = block_id;
extent->offset = 0;
extent->length = (uint32_t)artifact_bytes.len;
segment->records = record;
segment->record_count = 1;
segment->digests = ref.digest;
segment->extents = extent;
segment->extent_count = 1;
segment->footer.seal_snapshot = snapshot_id;
segment->footer.seal_time_ns = snapshot_id;
return true;
}
static bool write_artifact_segment(const char *root,
amduat_artifact_t artifact,
uint64_t segment_id,
uint64_t block_id,
uint64_t snapshot_id,
uint8_t out_hash[32],
amduat_reference_t *out_ref) {
amduat_reference_t ref;
amduat_octets_t artifact_bytes;
amduat_asl_core_index_segment_t segment;
amduat_asl_index_record_t record;
amduat_asl_extent_record_t extent;
if (out_ref == NULL) {
return false;
}
if (!amduat_asl_ref_derive(artifact,
AMDUAT_ENC_ASL1_CORE_V1,
AMDUAT_HASH_ASL1_ID_SHA256,
&ref,
&artifact_bytes)) {
return false;
}
if (!write_block_file(root, block_id, artifact_bytes)) {
amduat_octets_free(&artifact_bytes);
amduat_reference_free(&ref);
return false;
}
if (!build_segment_for_artifact(&segment,
&record,
&extent,
ref,
artifact_bytes,
snapshot_id,
block_id) ||
!write_segment_file(root, segment_id, &segment, out_hash)) {
amduat_octets_free(&artifact_bytes);
amduat_reference_free(&ref);
return false;
}
amduat_octets_free(&artifact_bytes);
*out_ref = ref;
return true;
}
static bool replay_state_equal(const amduat_asl_replay_state_t *lhs,
const amduat_asl_replay_state_t *rhs) {
size_t i;
if (lhs->segments_len != rhs->segments_len ||
lhs->tombstones_len != rhs->tombstones_len ||
lhs->state.log_position != rhs->state.log_position) {
return false;
}
for (i = 0; i < lhs->segments_len; ++i) {
const amduat_asl_segment_seal_t *left = &lhs->segments[i];
const amduat_asl_segment_seal_t *right = &rhs->segments[i];
if (left->segment_id != right->segment_id ||
memcmp(left->segment_hash, right->segment_hash,
sizeof(left->segment_hash)) != 0) {
return false;
}
}
for (i = 0; i < lhs->tombstones_len; ++i) {
const amduat_asl_tombstone_entry_t *left = &lhs->tombstones[i];
const amduat_asl_tombstone_entry_t *right = &rhs->tombstones[i];
if (left->tombstone_logseq != right->tombstone_logseq ||
!amduat_reference_eq(left->ref, right->ref)) {
return false;
}
}
return true;
}
static int test_replay_determinism(void) {
char *root;
amduat_artifact_t artifact_a;
amduat_artifact_t artifact_b;
amduat_reference_t ref_a;
amduat_reference_t ref_b;
uint8_t hash_a[32];
uint8_t hash_b[32];
uint8_t seal_payload_a[8 + 32];
uint8_t seal_payload_b[8 + 32];
size_t seal_len_a;
size_t seal_len_b;
amduat_asl_log_record_t records[2];
amduat_asl_log_record_t *loaded_records = NULL;
size_t loaded_count = 0;
amduat_asl_replay_state_t first;
amduat_asl_replay_state_t second;
uint8_t payload_a[4] = {0x01, 0x02, 0x03, 0x04};
uint8_t payload_b[4] = {0x10, 0x20, 0x30, 0x40};
int exit_code = 1;
ref_a = amduat_reference(0u, amduat_octets(NULL, 0u));
ref_b = amduat_reference(0u, amduat_octets(NULL, 0u));
root = make_temp_root();
if (root == NULL) {
fprintf(stderr, "temp root failed\n");
return 1;
}
if (!prepare_index_tree(root)) {
fprintf(stderr, "prepare index tree failed\n");
goto cleanup;
}
artifact_a = amduat_artifact(amduat_octets(payload_a, sizeof(payload_a)));
artifact_b = amduat_artifact(amduat_octets(payload_b, sizeof(payload_b)));
if (!write_artifact_segment(root, artifact_a, 1, 1, 1, hash_a, &ref_a) ||
!write_artifact_segment(root, artifact_b, 2, 2, 2, hash_b, &ref_b)) {
fprintf(stderr, "write artifact segments failed\n");
goto cleanup;
}
seal_len_a = build_segment_seal_payload(seal_payload_a,
sizeof(seal_payload_a),
1,
hash_a);
seal_len_b = build_segment_seal_payload(seal_payload_b,
sizeof(seal_payload_b),
2,
hash_b);
if (seal_len_a == 0 || seal_len_b == 0) {
fprintf(stderr, "seal payload build failed\n");
goto cleanup;
}
memset(records, 0, sizeof(records));
records[0].logseq = 10;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[0].payload = amduat_octets(seal_payload_a, seal_len_a);
records[1].logseq = 20;
records[1].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[1].payload = amduat_octets(seal_payload_b, seal_len_b);
if (!write_log(root, records, 2)) {
fprintf(stderr, "write log failed\n");
goto cleanup;
}
if (!load_log_records(root, &loaded_records, &loaded_count)) {
fprintf(stderr, "log decode failed\n");
goto cleanup;
}
if (!amduat_asl_replay_init(&first) ||
!amduat_asl_replay_init(&second)) {
fprintf(stderr, "replay init failed\n");
goto cleanup;
}
if (!amduat_asl_replay_apply_log(loaded_records,
loaded_count,
20,
&first) ||
!amduat_asl_replay_apply_log(loaded_records,
loaded_count,
20,
&second)) {
fprintf(stderr, "replay apply failed\n");
amduat_asl_replay_free(&first);
amduat_asl_replay_free(&second);
goto cleanup;
}
if (!replay_state_equal(&first, &second)) {
fprintf(stderr, "replay state mismatch\n");
amduat_asl_replay_free(&first);
amduat_asl_replay_free(&second);
goto cleanup;
}
amduat_asl_replay_free(&first);
amduat_asl_replay_free(&second);
exit_code = 0;
cleanup:
if (loaded_records != NULL) {
amduat_enc_asl_log_free(loaded_records, loaded_count);
}
amduat_reference_free(&ref_a);
amduat_reference_free(&ref_b);
if (!remove_tree(root)) {
fprintf(stderr, "cleanup failed\n");
exit_code = 1;
}
free(root);
return exit_code;
}
static int test_tombstone_lift_boundary(void) {
char *root;
amduat_artifact_t artifact;
amduat_reference_t ref;
uint8_t hash[32];
uint8_t seal_payload[8 + 32];
uint8_t anchor_payload[8 + 32];
uint8_t tombstone_payload[4 + 2 + 2 + 32 + 4 + 4];
uint8_t lift_payload[4 + 2 + 2 + 32 + 8];
size_t seal_len;
size_t anchor_len;
size_t tombstone_len;
size_t lift_len;
amduat_asl_log_record_t records[4];
amduat_asl_store_config_t config;
amduat_asl_store_index_fs_t fs;
amduat_asl_store_t store;
amduat_asl_index_state_t cutoff;
amduat_artifact_t loaded;
amduat_asl_store_error_t err;
uint8_t payload[5] = {0x11, 0x22, 0x33, 0x44, 0x55};
int exit_code = 1;
ref = amduat_reference(0u, amduat_octets(NULL, 0u));
root = make_temp_root();
if (root == NULL) {
fprintf(stderr, "temp root failed\n");
return 1;
}
if (!prepare_index_tree(root)) {
fprintf(stderr, "prepare index tree failed\n");
goto cleanup;
}
artifact = amduat_artifact(amduat_octets(payload, sizeof(payload)));
if (!write_artifact_segment(root, artifact, 5, 5, 5, hash, &ref)) {
fprintf(stderr, "write artifact segment failed\n");
goto cleanup;
}
seal_len = build_segment_seal_payload(seal_payload,
sizeof(seal_payload),
5,
hash);
anchor_len = build_snapshot_anchor_payload(anchor_payload,
sizeof(anchor_payload),
5,
hash);
tombstone_len = build_tombstone_payload(tombstone_payload,
sizeof(tombstone_payload),
ref.hash_id,
ref.digest.data,
ref.digest.len);
lift_len = build_tombstone_lift_payload(lift_payload,
sizeof(lift_payload),
ref.hash_id,
ref.digest.data,
ref.digest.len,
20);
if (seal_len == 0 || anchor_len == 0 || tombstone_len == 0 ||
lift_len == 0) {
fprintf(stderr, "log payload build failed\n");
goto cleanup;
}
memset(records, 0, sizeof(records));
records[0].logseq = 10;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[0].payload = amduat_octets(seal_payload, seal_len);
records[1].logseq = 20;
records[1].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
records[1].payload = amduat_octets(tombstone_payload, tombstone_len);
records[2].logseq = 30;
records[2].record_type = AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR;
records[2].payload = amduat_octets(anchor_payload, anchor_len);
records[3].logseq = 40;
records[3].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT;
records[3].payload = amduat_octets(lift_payload, lift_len);
if (!write_log(root, records, 4)) {
fprintf(stderr, "write log failed\n");
goto cleanup;
}
memset(&config, 0, sizeof(config));
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
if (!amduat_asl_store_index_fs_init(&fs, config, root)) {
fprintf(stderr, "index fs init failed\n");
goto cleanup;
}
amduat_asl_store_init(&store, config, amduat_asl_store_index_fs_ops(), &fs);
cutoff.snapshot_id = 0;
cutoff.log_position = 30;
loaded = amduat_artifact(amduat_octets(NULL, 0u));
err = amduat_asl_store_get_indexed(&store, ref, cutoff, &loaded);
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
fprintf(stderr, "tombstone cutoff expected not found: %d\n", err);
amduat_artifact_free(&loaded);
goto cleanup;
}
cutoff.log_position = 40;
loaded = amduat_artifact(amduat_octets(NULL, 0u));
err = amduat_asl_store_get_indexed(&store, ref, cutoff, &loaded);
if (err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "lift lookup failed: %d\n", err);
amduat_artifact_free(&loaded);
goto cleanup;
}
if (!amduat_artifact_eq(artifact, loaded)) {
fprintf(stderr, "lifted artifact mismatch\n");
amduat_artifact_free(&loaded);
goto cleanup;
}
amduat_artifact_free(&loaded);
exit_code = 0;
cleanup:
amduat_reference_free(&ref);
if (!remove_tree(root)) {
fprintf(stderr, "cleanup failed\n");
exit_code = 1;
}
free(root);
return exit_code;
}
static int test_crash_recovery_unsealed(void) {
char *root;
amduat_artifact_t artifact_a;
amduat_artifact_t artifact_b;
amduat_reference_t ref_a;
amduat_reference_t ref_b;
uint8_t hash_a[32];
uint8_t hash_b[32];
uint8_t seal_payload[8 + 32];
size_t seal_len;
amduat_asl_log_record_t records[1];
amduat_asl_log_record_t *loaded_records = NULL;
size_t loaded_count = 0;
amduat_asl_replay_state_t replay_state;
amduat_asl_store_config_t config;
amduat_asl_store_index_fs_t fs;
amduat_asl_store_t store;
amduat_asl_index_state_t current;
amduat_artifact_t loaded;
amduat_asl_store_error_t err;
uint8_t payload_a[3] = {0x61, 0x62, 0x63};
uint8_t payload_b[3] = {0x64, 0x65, 0x66};
int exit_code = 1;
ref_a = amduat_reference(0u, amduat_octets(NULL, 0u));
ref_b = amduat_reference(0u, amduat_octets(NULL, 0u));
root = make_temp_root();
if (root == NULL) {
fprintf(stderr, "temp root failed\n");
return 1;
}
if (!prepare_index_tree(root)) {
fprintf(stderr, "prepare index tree failed\n");
goto cleanup;
}
artifact_a = amduat_artifact(amduat_octets(payload_a, sizeof(payload_a)));
artifact_b = amduat_artifact(amduat_octets(payload_b, sizeof(payload_b)));
if (!write_artifact_segment(root, artifact_a, 1, 1, 1, hash_a, &ref_a) ||
!write_artifact_segment(root, artifact_b, 9, 9, 9, hash_b, &ref_b)) {
fprintf(stderr, "write artifact segments failed\n");
goto cleanup;
}
seal_len = build_segment_seal_payload(seal_payload,
sizeof(seal_payload),
1,
hash_a);
if (seal_len == 0) {
fprintf(stderr, "seal payload build failed\n");
goto cleanup;
}
memset(records, 0, sizeof(records));
records[0].logseq = 10;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[0].payload = amduat_octets(seal_payload, seal_len);
if (!write_log(root, records, 1)) {
fprintf(stderr, "write log failed\n");
goto cleanup;
}
memset(&config, 0, sizeof(config));
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
if (!amduat_asl_store_index_fs_init(&fs, config, root)) {
fprintf(stderr, "index fs init failed\n");
goto cleanup;
}
amduat_asl_store_init(&store, config, amduat_asl_store_index_fs_ops(), &fs);
if (!amduat_asl_index_current_state(&store, &current)) {
fprintf(stderr, "current state failed\n");
goto cleanup;
}
loaded = amduat_artifact(amduat_octets(NULL, 0u));
err = amduat_asl_store_get_indexed(&store, ref_b, current, &loaded);
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
fprintf(stderr, "unsealed segment should be ignored: %d\n", err);
amduat_artifact_free(&loaded);
goto cleanup;
}
if (!load_log_records(root, &loaded_records, &loaded_count)) {
fprintf(stderr, "log decode failed\n");
goto cleanup;
}
if (!amduat_asl_replay_init(&replay_state)) {
fprintf(stderr, "replay init failed\n");
goto cleanup;
}
if (!amduat_asl_replay_apply_log(loaded_records,
loaded_count,
current.log_position,
&replay_state)) {
fprintf(stderr, "replay apply failed\n");
amduat_asl_replay_free(&replay_state);
goto cleanup;
}
if (replay_state.segments_len != 1 ||
replay_state.segments[0].segment_id != 1 ||
memcmp(replay_state.segments[0].segment_hash, hash_a,
sizeof(hash_a)) != 0) {
fprintf(stderr, "replay should only include sealed segment\n");
amduat_asl_replay_free(&replay_state);
goto cleanup;
}
amduat_asl_replay_free(&replay_state);
exit_code = 0;
cleanup:
if (loaded_records != NULL) {
amduat_enc_asl_log_free(loaded_records, loaded_count);
}
amduat_reference_free(&ref_a);
amduat_reference_free(&ref_b);
if (!remove_tree(root)) {
fprintf(stderr, "cleanup failed\n");
exit_code = 1;
}
free(root);
return exit_code;
}
int main(void) {
if (test_replay_determinism() != 0) {
return 1;
}
if (test_tombstone_lift_boundary() != 0) {
return 1;
}
if (test_crash_recovery_unsealed() != 0) {
return 1;
}
return 0;
}

View file

@ -0,0 +1,525 @@
#include "amduat/asl/log_store.h"
#include "amduat/asl/asl_store_index_fs.h"
#include "amduat/asl/store.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/hash/asl1.h"
#include <dirent.h>
#include <errno.h>
#include <inttypes.h>
#include <pthread.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
static bool join_path(const char *base, const char *segment, char **out_path) {
size_t base_len;
size_t seg_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
base_len = strlen(base);
seg_len = strlen(segment);
if (base_len == 0u || seg_len == 0u) {
return false;
}
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + seg_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, seg_len);
offset += seg_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool remove_tree(const char *path) {
struct stat st;
DIR *dir;
struct dirent *entry;
if (path == NULL) {
return false;
}
if (lstat(path, &st) != 0) {
return errno == ENOENT;
}
if (!S_ISDIR(st.st_mode)) {
return unlink(path) == 0;
}
dir = opendir(path);
if (dir == NULL) {
return false;
}
while ((entry = readdir(dir)) != NULL) {
char *child = NULL;
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0) {
continue;
}
if (!join_path(path, entry->d_name, &child)) {
closedir(dir);
return false;
}
if (!remove_tree(child)) {
free(child);
closedir(dir);
return false;
}
free(child);
}
if (closedir(dir) != 0) {
return false;
}
return rmdir(path) == 0;
}
static bool refs_equal(amduat_reference_t a, amduat_reference_t b) {
if (a.hash_id != b.hash_id) {
return false;
}
if (a.digest.len != b.digest.len) {
return false;
}
if (a.digest.len == 0u) {
return true;
}
if (a.digest.data == NULL || b.digest.data == NULL) {
return false;
}
return memcmp(a.digest.data, b.digest.data, a.digest.len) == 0;
}
typedef struct {
amduat_asl_store_t *store;
amduat_asl_log_store_t *log_store;
const char *log_name;
uint16_t kind;
size_t iterations;
amduat_asl_store_error_t first_err;
uint64_t *observed_offsets;
} append_worker_t;
typedef struct {
amduat_asl_store_t *store;
uint16_t kind;
size_t iterations;
amduat_asl_store_error_t first_err;
} put_worker_t;
static void *append_worker_main(void *opaque) {
append_worker_t *worker = (append_worker_t *)opaque;
size_t i;
if (worker == NULL || worker->store == NULL || worker->log_store == NULL ||
worker->log_name == NULL) {
return NULL;
}
worker->first_err = AMDUAT_ASL_STORE_OK;
for (i = 0u; i < worker->iterations; ++i) {
char payload_buf[64];
size_t payload_len;
amduat_artifact_t payload_artifact;
amduat_reference_t payload_ref;
amduat_asl_log_entry_t entry;
amduat_asl_store_error_t err;
uint64_t first_offset = 0u;
payload_len = (size_t)snprintf(payload_buf, sizeof(payload_buf),
"worker-%u-%zu", (unsigned)worker->kind, i);
if (payload_len == 0u || payload_len >= sizeof(payload_buf)) {
worker->first_err = AMDUAT_ASL_STORE_ERR_IO;
return NULL;
}
payload_artifact =
amduat_artifact(amduat_octets((const uint8_t *)payload_buf, payload_len));
payload_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
err = amduat_asl_store_put(worker->store, payload_artifact, &payload_ref);
if (err != AMDUAT_ASL_STORE_OK) {
worker->first_err = err;
return NULL;
}
memset(&entry, 0, sizeof(entry));
entry.kind = worker->kind;
entry.payload_ref = payload_ref;
err = amduat_asl_log_append(worker->log_store,
worker->log_name,
&entry,
1u,
&first_offset);
amduat_reference_free(&payload_ref);
if (err != AMDUAT_ASL_STORE_OK) {
worker->first_err = err;
return NULL;
}
if (worker->observed_offsets != NULL && i < worker->iterations) {
worker->observed_offsets[i] = first_offset;
}
}
return NULL;
}
static void *put_worker_main(void *opaque) {
put_worker_t *worker = (put_worker_t *)opaque;
size_t i;
if (worker == NULL || worker->store == NULL) {
return NULL;
}
worker->first_err = AMDUAT_ASL_STORE_OK;
for (i = 0u; i < worker->iterations; ++i) {
char payload_buf[64];
size_t payload_len;
amduat_artifact_t payload_artifact;
amduat_reference_t payload_ref;
amduat_asl_store_error_t err;
payload_len = (size_t)snprintf(payload_buf, sizeof(payload_buf),
"put-only-%u-%zu", (unsigned)worker->kind, i);
if (payload_len == 0u || payload_len >= sizeof(payload_buf)) {
worker->first_err = AMDUAT_ASL_STORE_ERR_IO;
return NULL;
}
payload_artifact =
amduat_artifact(amduat_octets((const uint8_t *)payload_buf, payload_len));
payload_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
err = amduat_asl_store_put(worker->store, payload_artifact, &payload_ref);
amduat_reference_free(&payload_ref);
if (err != AMDUAT_ASL_STORE_OK) {
worker->first_err = err;
return NULL;
}
}
return NULL;
}
int main(void) {
amduat_asl_store_config_t config;
amduat_asl_store_index_fs_t fs_a;
amduat_asl_store_index_fs_t fs_b;
amduat_asl_store_t store_a;
amduat_asl_store_t store_b;
amduat_asl_log_store_t log_a;
amduat_asl_log_store_t log_b;
amduat_artifact_t payload_artifact_a;
amduat_artifact_t payload_artifact_b;
amduat_reference_t payload_ref_a;
amduat_reference_t payload_ref_b;
amduat_asl_log_entry_t entry_a;
amduat_asl_log_entry_t entry_b;
amduat_asl_log_entry_t *out_entries = NULL;
size_t out_len = 0u;
uint64_t out_next_offset = 0u;
bool out_end = false;
uint64_t first_offset = 0u;
const char *log_name = "collection/space/app1/daemon/edges/log";
enum { WORKER_ITERS = 32 };
append_worker_t worker_a;
append_worker_t worker_b;
append_worker_t mixed_append_worker;
put_worker_t mixed_put_worker;
pthread_t thread_a;
pthread_t thread_b;
pthread_t mixed_append_thread;
pthread_t mixed_put_thread;
bool started_a = false;
bool started_b = false;
bool started_mixed_append = false;
bool started_mixed_put = false;
uint64_t worker_a_offsets[WORKER_ITERS];
uint64_t worker_b_offsets[WORKER_ITERS];
char *root = NULL;
int rc = 1;
root = strdup("/tmp/amduat-log-index-XXXXXX");
if (root == NULL) {
fprintf(stderr, "alloc root template failed\n");
return 1;
}
if (mkdtemp(root) == NULL) {
fprintf(stderr, "mkdtemp failed\n");
goto cleanup;
}
memset(&config, 0, sizeof(config));
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
if (!amduat_asl_store_index_fs_init(&fs_a, config, root) ||
!amduat_asl_store_index_fs_init(&fs_b, config, root)) {
fprintf(stderr, "index fs init failed\n");
goto cleanup;
}
amduat_asl_store_init(&store_a, config, amduat_asl_store_index_fs_ops(), &fs_a);
amduat_asl_store_init(&store_b, config, amduat_asl_store_index_fs_ops(), &fs_b);
if (!amduat_asl_log_store_init(&log_a, root, &store_a, NULL) ||
!amduat_asl_log_store_init(&log_b, root, &store_b, NULL)) {
fprintf(stderr, "log store init failed\n");
goto cleanup;
}
payload_artifact_a = amduat_artifact(amduat_octets((const uint8_t *)"payload-a", 9u));
payload_ref_a = amduat_reference(0u, amduat_octets(NULL, 0u));
if (amduat_asl_store_put(&store_a, payload_artifact_a, &payload_ref_a) !=
AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "put payload a failed\n");
goto cleanup;
}
memset(&entry_a, 0, sizeof(entry_a));
entry_a.kind = 1u;
entry_a.payload_ref = payload_ref_a;
if (amduat_asl_log_append(&log_a, log_name, &entry_a, 1u, &first_offset) !=
AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "first append failed\n");
goto cleanup;
}
if (first_offset != 0u) {
fprintf(stderr, "first offset mismatch: %" PRIu64 "\n", first_offset);
goto cleanup;
}
payload_artifact_b = amduat_artifact(amduat_octets((const uint8_t *)"payload-b", 9u));
payload_ref_b = amduat_reference(0u, amduat_octets(NULL, 0u));
if (amduat_asl_store_put(&store_b, payload_artifact_b, &payload_ref_b) !=
AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "put payload b failed\n");
goto cleanup;
}
memset(&entry_b, 0, sizeof(entry_b));
entry_b.kind = 2u;
entry_b.payload_ref = payload_ref_b;
if (amduat_asl_log_append(&log_b, log_name, &entry_b, 1u, &first_offset) !=
AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "second append failed\n");
goto cleanup;
}
if (first_offset != 1u) {
fprintf(stderr, "second first_offset mismatch: %" PRIu64 "\n", first_offset);
goto cleanup;
}
if (amduat_asl_log_read(&log_a,
log_name,
0u,
8u,
&out_entries,
&out_len,
&out_next_offset,
&out_end) != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "log read failed\n");
goto cleanup;
}
if (out_len != 2u || out_next_offset != 2u || !out_end) {
fprintf(stderr, "readback shape mismatch len=%zu next=%" PRIu64 " end=%d\n",
out_len, out_next_offset, out_end ? 1 : 0);
goto cleanup;
}
if (out_entries[0].kind != 1u || !refs_equal(out_entries[0].payload_ref, payload_ref_a)) {
fprintf(stderr, "entry0 mismatch\n");
goto cleanup;
}
if (out_entries[1].kind != 2u || !refs_equal(out_entries[1].payload_ref, payload_ref_b)) {
fprintf(stderr, "entry1 mismatch\n");
goto cleanup;
}
amduat_asl_log_entries_free(out_entries, out_len);
out_entries = NULL;
out_len = 0u;
memset(&worker_a, 0, sizeof(worker_a));
worker_a.store = &store_a;
worker_a.log_store = &log_a;
worker_a.log_name = log_name;
worker_a.kind = 10u;
worker_a.iterations = WORKER_ITERS;
worker_a.observed_offsets = worker_a_offsets;
memset(&worker_b, 0, sizeof(worker_b));
worker_b.store = &store_b;
worker_b.log_store = &log_b;
worker_b.log_name = log_name;
worker_b.kind = 11u;
worker_b.iterations = WORKER_ITERS;
worker_b.observed_offsets = worker_b_offsets;
if (pthread_create(&thread_a, NULL, append_worker_main, &worker_a) != 0) {
fprintf(stderr, "pthread_create failed\n");
goto cleanup;
}
started_a = true;
if (pthread_create(&thread_b, NULL, append_worker_main, &worker_b) != 0) {
fprintf(stderr, "pthread_create failed\n");
goto cleanup;
}
started_b = true;
(void)pthread_join(thread_a, NULL);
(void)pthread_join(thread_b, NULL);
started_a = false;
started_b = false;
if (worker_a.first_err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "worker_a failed err=%d\n", (int)worker_a.first_err);
goto cleanup;
}
if (worker_b.first_err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "worker_b failed err=%d\n", (int)worker_b.first_err);
goto cleanup;
}
{
bool seen[2u + 2u * WORKER_ITERS];
size_t i;
memset(seen, 0, sizeof(seen));
for (i = 0u; i < WORKER_ITERS; ++i) {
if (worker_a_offsets[i] < 2u ||
worker_a_offsets[i] >= (uint64_t)(2u + 2u * WORKER_ITERS)) {
fprintf(stderr, "worker_a offset out of range: %" PRIu64 "\n",
worker_a_offsets[i]);
goto cleanup;
}
if (seen[worker_a_offsets[i]]) {
fprintf(stderr, "duplicate offset from worker_a: %" PRIu64 "\n",
worker_a_offsets[i]);
goto cleanup;
}
seen[worker_a_offsets[i]] = true;
}
for (i = 0u; i < WORKER_ITERS; ++i) {
if (worker_b_offsets[i] < 2u ||
worker_b_offsets[i] >= (uint64_t)(2u + 2u * WORKER_ITERS)) {
fprintf(stderr, "worker_b offset out of range: %" PRIu64 "\n",
worker_b_offsets[i]);
goto cleanup;
}
if (seen[worker_b_offsets[i]]) {
fprintf(stderr, "duplicate offset from worker_b: %" PRIu64 "\n",
worker_b_offsets[i]);
goto cleanup;
}
seen[worker_b_offsets[i]] = true;
}
for (i = 2u; i < (2u + 2u * WORKER_ITERS); ++i) {
if (!seen[i]) {
fprintf(stderr, "missing worker offset: %zu\n", i);
goto cleanup;
}
}
}
if (amduat_asl_log_read(&log_a,
log_name,
0u,
4096u,
&out_entries,
&out_len,
&out_next_offset,
&out_end) != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "log read after stress failed\n");
goto cleanup;
}
if (out_len != (2u + 2u * WORKER_ITERS) || out_next_offset != out_len ||
!out_end) {
fprintf(stderr,
"stress read mismatch len=%zu next=%" PRIu64 " expected=%u end=%d\n",
out_len, out_next_offset, (unsigned)(2u + 2u * WORKER_ITERS),
out_end ? 1 : 0);
goto cleanup;
}
amduat_asl_log_entries_free(out_entries, out_len);
out_entries = NULL;
out_len = 0u;
memset(&mixed_append_worker, 0, sizeof(mixed_append_worker));
mixed_append_worker.store = &store_a;
mixed_append_worker.log_store = &log_a;
mixed_append_worker.log_name = log_name;
mixed_append_worker.kind = 12u;
mixed_append_worker.iterations = WORKER_ITERS;
memset(&mixed_put_worker, 0, sizeof(mixed_put_worker));
mixed_put_worker.store = &store_b;
mixed_put_worker.kind = 13u;
mixed_put_worker.iterations = WORKER_ITERS * 4u;
if (pthread_create(&mixed_append_thread, NULL, append_worker_main,
&mixed_append_worker) != 0) {
fprintf(stderr, "pthread_create mixed append failed\n");
goto cleanup;
}
started_mixed_append = true;
if (pthread_create(&mixed_put_thread, NULL, put_worker_main,
&mixed_put_worker) != 0) {
fprintf(stderr, "pthread_create mixed put failed\n");
goto cleanup;
}
started_mixed_put = true;
(void)pthread_join(mixed_append_thread, NULL);
(void)pthread_join(mixed_put_thread, NULL);
started_mixed_append = false;
started_mixed_put = false;
if (mixed_append_worker.first_err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "mixed append worker failed err=%d\n",
(int)mixed_append_worker.first_err);
goto cleanup;
}
if (mixed_put_worker.first_err != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "mixed put worker failed err=%d\n",
(int)mixed_put_worker.first_err);
goto cleanup;
}
rc = 0;
cleanup:
if (started_a) {
(void)pthread_join(thread_a, NULL);
}
if (started_b) {
(void)pthread_join(thread_b, NULL);
}
if (started_mixed_append) {
(void)pthread_join(mixed_append_thread, NULL);
}
if (started_mixed_put) {
(void)pthread_join(mixed_put_thread, NULL);
}
amduat_asl_log_entries_free(out_entries, out_len);
amduat_reference_free(&payload_ref_a);
amduat_reference_free(&payload_ref_b);
if (root != NULL) {
if (!remove_tree(root)) {
fprintf(stderr, "warning: cleanup failed for %s\n", root);
}
free(root);
}
return rc;
}

291
tests/asl/test_asl_replay.c Normal file
View file

@ -0,0 +1,291 @@
#include "amduat/asl/index_replay.h"
#include "amduat/hash/asl1.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
static void store_u16_le(uint8_t *out, uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static size_t build_artifact_ref(uint8_t *out,
size_t out_cap,
amduat_hash_id_t hash_id,
const uint8_t *digest,
size_t digest_len) {
size_t total = 4 + 2 + 2 + digest_len;
if (out_cap < total) {
return 0;
}
store_u32_le(out, (uint32_t)hash_id);
store_u16_le(out + 4, (uint16_t)digest_len);
store_u16_le(out + 6, 0);
memcpy(out + 8, digest, digest_len);
return total;
}
static size_t build_tombstone_payload(uint8_t *out,
size_t out_cap,
amduat_hash_id_t hash_id,
const uint8_t *digest,
size_t digest_len) {
size_t offset = 0;
size_t ref_len = build_artifact_ref(out,
out_cap,
hash_id,
digest,
digest_len);
if (ref_len == 0 || out_cap < ref_len + 8) {
return 0;
}
offset += ref_len;
store_u32_le(out + offset, 0);
offset += 4;
store_u32_le(out + offset, 0);
offset += 4;
return offset;
}
static size_t build_tombstone_lift_payload(uint8_t *out,
size_t out_cap,
amduat_hash_id_t hash_id,
const uint8_t *digest,
size_t digest_len,
uint64_t tombstone_logseq) {
size_t offset = 0;
size_t ref_len = build_artifact_ref(out,
out_cap,
hash_id,
digest,
digest_len);
if (ref_len == 0 || out_cap < ref_len + 8) {
return 0;
}
offset += ref_len;
store_u64_le(out + offset, tombstone_logseq);
offset += 8;
return offset;
}
static size_t build_segment_seal_payload(uint8_t *out,
size_t out_cap,
uint64_t segment_id,
const uint8_t hash[32]) {
if (out_cap < 8 + 32) {
return 0;
}
store_u64_le(out, segment_id);
memcpy(out + 8, hash, 32);
return 8 + 32;
}
static int test_tombstone_lift_cutoff(void) {
amduat_asl_log_record_t records[2];
amduat_asl_replay_state_t state;
uint8_t digest[32];
uint8_t tombstone_payload[4 + 2 + 2 + 32 + 4 + 4];
uint8_t lift_payload[4 + 2 + 2 + 32 + 8];
size_t tombstone_len;
size_t lift_len;
amduat_reference_t ref;
memset(digest, 0x5a, sizeof(digest));
tombstone_len = build_tombstone_payload(tombstone_payload,
sizeof(tombstone_payload),
AMDUAT_HASH_ASL1_ID_SHA256,
digest,
sizeof(digest));
lift_len = build_tombstone_lift_payload(lift_payload,
sizeof(lift_payload),
AMDUAT_HASH_ASL1_ID_SHA256,
digest,
sizeof(digest),
20);
if (tombstone_len == 0 || lift_len == 0) {
fprintf(stderr, "payload build failed\n");
return 1;
}
memset(records, 0, sizeof(records));
records[0].logseq = 20;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
records[0].payload = amduat_octets(tombstone_payload, tombstone_len);
records[1].logseq = 40;
records[1].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT;
records[1].payload = amduat_octets(lift_payload, lift_len);
if (!amduat_asl_replay_init(&state)) {
fprintf(stderr, "replay init failed\n");
return 1;
}
if (!amduat_asl_replay_apply_log(records, 2, 30, &state)) {
fprintf(stderr, "replay apply failed\n");
amduat_asl_replay_free(&state);
return 1;
}
if (state.tombstones_len != 1) {
fprintf(stderr, "tombstone not applied\n");
amduat_asl_replay_free(&state);
return 1;
}
ref = amduat_reference(AMDUAT_HASH_ASL1_ID_SHA256,
amduat_octets(digest, sizeof(digest)));
if (state.tombstones[0].tombstone_logseq != 20 ||
!amduat_reference_eq(state.tombstones[0].ref, ref)) {
fprintf(stderr, "tombstone mismatch\n");
amduat_asl_replay_free(&state);
return 1;
}
amduat_asl_replay_free(&state);
if (!amduat_asl_replay_init(&state)) {
fprintf(stderr, "replay init failed\n");
return 1;
}
if (!amduat_asl_replay_apply_log(records, 2, 40, &state)) {
fprintf(stderr, "replay apply failed\n");
amduat_asl_replay_free(&state);
return 1;
}
if (state.tombstones_len != 0) {
fprintf(stderr, "tombstone lift failed\n");
amduat_asl_replay_free(&state);
return 1;
}
amduat_asl_replay_free(&state);
return 0;
}
static int test_unknown_record_skip(void) {
amduat_asl_log_record_t records[2];
amduat_asl_replay_state_t state;
uint8_t unknown_payload[1] = {0x01};
uint8_t seal_payload[8 + 32];
uint8_t seal_hash[32];
size_t seal_len;
memset(seal_hash, 0xab, sizeof(seal_hash));
seal_len = build_segment_seal_payload(seal_payload,
sizeof(seal_payload),
9,
seal_hash);
if (seal_len == 0) {
fprintf(stderr, "seal payload build failed\n");
return 1;
}
memset(records, 0, sizeof(records));
records[0].logseq = 10;
records[0].record_type = 0x99;
records[0].payload = amduat_octets(unknown_payload, sizeof(unknown_payload));
records[1].logseq = 11;
records[1].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[1].payload = amduat_octets(seal_payload, seal_len);
if (!amduat_asl_replay_init(&state)) {
fprintf(stderr, "replay init failed\n");
return 1;
}
if (!amduat_asl_replay_apply_log(records, 2, 11, &state)) {
fprintf(stderr, "replay apply failed\n");
amduat_asl_replay_free(&state);
return 1;
}
if (state.segments_len != 1) {
fprintf(stderr, "segment seal missing\n");
amduat_asl_replay_free(&state);
return 1;
}
amduat_asl_replay_free(&state);
return 0;
}
static int test_multiple_seals_latest(void) {
amduat_asl_log_record_t records[2];
amduat_asl_replay_state_t state;
uint8_t seal_payload_a[8 + 32];
uint8_t seal_payload_b[8 + 32];
uint8_t hash_a[32];
uint8_t hash_b[32];
size_t seal_len_a;
size_t seal_len_b;
memset(hash_a, 0x11, sizeof(hash_a));
memset(hash_b, 0x22, sizeof(hash_b));
seal_len_a = build_segment_seal_payload(seal_payload_a,
sizeof(seal_payload_a),
5,
hash_a);
seal_len_b = build_segment_seal_payload(seal_payload_b,
sizeof(seal_payload_b),
5,
hash_b);
if (seal_len_a == 0 || seal_len_b == 0) {
fprintf(stderr, "seal payload build failed\n");
return 1;
}
memset(records, 0, sizeof(records));
records[0].logseq = 10;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[0].payload = amduat_octets(seal_payload_a, seal_len_a);
records[1].logseq = 20;
records[1].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[1].payload = amduat_octets(seal_payload_b, seal_len_b);
if (!amduat_asl_replay_init(&state)) {
fprintf(stderr, "replay init failed\n");
return 1;
}
if (!amduat_asl_replay_apply_log(records, 2, 20, &state)) {
fprintf(stderr, "replay apply failed\n");
amduat_asl_replay_free(&state);
return 1;
}
if (state.segments_len != 1) {
fprintf(stderr, "segment seal count mismatch\n");
amduat_asl_replay_free(&state);
return 1;
}
if (memcmp(state.segments[0].segment_hash, hash_b, sizeof(hash_b)) != 0) {
fprintf(stderr, "segment seal not updated\n");
amduat_asl_replay_free(&state);
return 1;
}
amduat_asl_replay_free(&state);
return 0;
}
int main(void) {
if (test_tombstone_lift_cutoff() != 0) {
return 1;
}
if (test_unknown_record_skip() != 0) {
return 1;
}
if (test_multiple_seals_latest() != 0) {
return 1;
}
return 0;
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,68 @@
#include "amduat/asl/store.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/hash/asl1.h"
#include <stdio.h>
static int test_indexed_ops_unsupported(void) {
amduat_asl_store_ops_t ops;
amduat_asl_store_config_t config;
amduat_asl_store_t store;
amduat_asl_store_error_t err;
amduat_asl_index_state_t state;
amduat_artifact_t artifact;
amduat_reference_t ref;
amduat_asl_store_ops_init(&ops);
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
amduat_asl_store_init(&store, config, ops, NULL);
artifact = amduat_artifact(amduat_octets(NULL, 0u));
ref = amduat_reference(0u, amduat_octets(NULL, 0u));
state.snapshot_id = 0u;
state.log_position = 0u;
err = amduat_asl_store_put_indexed(&store, artifact, &ref, &state);
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
fprintf(stderr, "put_indexed missing ops should be unsupported: %d\n", err);
return 1;
}
err = amduat_asl_store_get_indexed(&store, ref, state, &artifact);
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
fprintf(stderr, "get_indexed missing ops should be unsupported: %d\n", err);
return 1;
}
if (amduat_asl_index_current_state(&store, &state)) {
fprintf(stderr, "current_state missing ops should be false\n");
return 1;
}
err = amduat_asl_store_tombstone(&store, ref, 0u, 0u, &state);
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
fprintf(stderr, "tombstone missing ops should be unsupported: %d\n", err);
return 1;
}
err = amduat_asl_store_tombstone_lift(&store, ref, 1u, &state);
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
fprintf(stderr,
"tombstone_lift missing ops should be unsupported: %d\n",
err);
return 1;
}
err = amduat_asl_log_scan(&store, NULL, NULL);
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
fprintf(stderr, "log_scan missing ops should be unsupported: %d\n", err);
return 1;
}
return 0;
}
int main(void) {
return test_indexed_ops_unsupported();
}

View file

@ -0,0 +1,336 @@
#include "amduat/enc/asl_core_index.h"
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static void store_u16_le(uint8_t *out, uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static uint64_t load_u64_le(const uint8_t *data) {
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
}
static uint64_t crc64_ecma(const uint8_t *data, size_t len) {
uint64_t crc = 0u;
size_t i;
for (i = 0; i < len; ++i) {
uint64_t bit;
uint8_t value = data[i];
crc ^= ((uint64_t)value) << 56;
for (bit = 0; bit < 8; ++bit) {
if (crc & 0x8000000000000000ull) {
crc = (crc << 1) ^ 0x42f0e1eba9ea3693ull;
} else {
crc <<= 1;
}
}
}
return crc;
}
static void update_crc(amduat_octets_t bytes) {
size_t footer_offset = bytes.len - AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE;
uint64_t crc = crc64_ecma(bytes.data, footer_offset);
store_u64_le((uint8_t *)bytes.data + footer_offset, crc);
}
static void build_simple_segment(amduat_asl_core_index_segment_t *segment,
amduat_asl_index_record_t *record,
amduat_asl_extent_record_t *extent,
uint8_t digest_bytes[8]) {
memset(segment, 0, sizeof(*segment));
memset(record, 0, sizeof(*record));
memset(extent, 0, sizeof(*extent));
digest_bytes[0] = 0x01;
digest_bytes[1] = 0x02;
digest_bytes[2] = 0x03;
digest_bytes[3] = 0x04;
digest_bytes[4] = 0x05;
digest_bytes[5] = 0x06;
digest_bytes[6] = 0x07;
digest_bytes[7] = 0x08;
segment->header.snapshot_min = 5;
segment->header.snapshot_max = 6;
segment->header.segment_domain_id = 7;
segment->header.segment_visibility = 1;
segment->header.federation_version = 0;
segment->header.flags = 0;
segment->header.reserved0 = 0;
record->hash_id = 9;
record->digest_len = 8;
record->extent_count = 1;
record->total_length = 3;
record->domain_id = 7;
record->visibility = 1;
record->has_cross_domain_source = 0;
record->cross_domain_source = 0;
record->flags = 0;
extent->block_id = 123;
extent->offset = 0;
extent->length = 3;
segment->records = record;
segment->record_count = 1;
segment->digests = amduat_octets(digest_bytes, 8);
segment->extents = extent;
segment->extent_count = 1;
segment->footer.seal_snapshot = 10;
segment->footer.seal_time_ns = 11;
}
static int test_round_trip(void) {
amduat_asl_core_index_segment_t segment;
amduat_asl_index_record_t record;
amduat_asl_extent_record_t extent;
uint8_t digest_bytes[8];
amduat_octets_t encoded;
amduat_asl_core_index_segment_t decoded;
int exit_code = 1;
build_simple_segment(&segment, &record, &extent, digest_bytes);
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
if (!amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
fprintf(stderr, "decode failed\n");
goto cleanup;
}
if (decoded.header.version != AMDUAT_ASL_CORE_INDEX_VERSION ||
decoded.header.header_size != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
fprintf(stderr, "header mismatch\n");
goto cleanup_decoded;
}
if (decoded.record_count != 1 || decoded.extent_count != 1) {
fprintf(stderr, "count mismatch\n");
goto cleanup_decoded;
}
if (decoded.header.segment_domain_id != 7 ||
decoded.header.segment_visibility != 1) {
fprintf(stderr, "segment federation mismatch\n");
goto cleanup_decoded;
}
if (decoded.records[0].hash_id != record.hash_id ||
decoded.records[0].digest_len != record.digest_len ||
decoded.records[0].total_length != record.total_length ||
decoded.records[0].domain_id != record.domain_id ||
decoded.records[0].visibility != record.visibility ||
decoded.records[0].flags != record.flags) {
fprintf(stderr, "record mismatch\n");
goto cleanup_decoded;
}
if (decoded.digests.len != sizeof(digest_bytes) ||
memcmp(decoded.digests.data, digest_bytes, sizeof(digest_bytes)) != 0) {
fprintf(stderr, "digest mismatch\n");
goto cleanup_decoded;
}
if (decoded.extents[0].block_id != extent.block_id ||
decoded.extents[0].offset != extent.offset ||
decoded.extents[0].length != extent.length) {
fprintf(stderr, "extent mismatch\n");
goto cleanup_decoded;
}
if (decoded.footer.seal_snapshot != segment.footer.seal_snapshot ||
decoded.footer.seal_time_ns != segment.footer.seal_time_ns) {
fprintf(stderr, "footer mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_asl_core_index_free(&decoded);
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_crc_mismatch(void) {
amduat_asl_core_index_segment_t segment;
amduat_asl_index_record_t record;
amduat_asl_extent_record_t extent;
uint8_t digest_bytes[8];
amduat_octets_t encoded;
amduat_asl_core_index_segment_t decoded;
uint64_t digests_offset;
int exit_code = 1;
build_simple_segment(&segment, &record, &extent, digest_bytes);
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
digests_offset = load_u64_le(encoded.data + 64);
((uint8_t *)encoded.data)[digests_offset] ^= 0xffu;
if (amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
fprintf(stderr, "decode unexpectedly succeeded\n");
amduat_enc_asl_core_index_free(&decoded);
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_invalid_offsets(void) {
amduat_asl_core_index_segment_t segment;
amduat_asl_index_record_t record;
amduat_asl_extent_record_t extent;
uint8_t digest_bytes[8];
amduat_octets_t encoded;
amduat_asl_core_index_segment_t decoded;
int exit_code = 1;
build_simple_segment(&segment, &record, &extent, digest_bytes);
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
store_u64_le((uint8_t *)encoded.data + 72, 0);
update_crc(encoded);
if (amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
fprintf(stderr, "decode unexpectedly succeeded\n");
amduat_enc_asl_core_index_free(&decoded);
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_invalid_federation_fields(void) {
amduat_asl_core_index_segment_t segment;
amduat_asl_index_record_t record;
amduat_asl_extent_record_t extent;
uint8_t digest_bytes[8];
amduat_octets_t encoded;
amduat_asl_core_index_segment_t decoded;
uint64_t records_offset;
int exit_code = 1;
build_simple_segment(&segment, &record, &extent, digest_bytes);
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
records_offset = load_u64_le(encoded.data + 40);
((uint8_t *)encoded.data)[records_offset + 36] = 2;
update_crc(encoded);
if (amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
fprintf(stderr, "decode unexpectedly succeeded\n");
amduat_enc_asl_core_index_free(&decoded);
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_legacy_defaults(void) {
amduat_asl_core_index_segment_t segment;
amduat_asl_index_record_t record;
amduat_asl_extent_record_t extent;
uint8_t digest_bytes[8];
amduat_octets_t encoded;
amduat_asl_core_index_segment_t decoded;
uint64_t records_offset;
int exit_code = 1;
build_simple_segment(&segment, &record, &extent, digest_bytes);
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
store_u16_le((uint8_t *)encoded.data + 8, 2);
((uint8_t *)encoded.data)[100] = 1;
records_offset = load_u64_le(encoded.data + 40);
((uint8_t *)encoded.data)[records_offset + 36] = 1;
update_crc(encoded);
if (!amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
fprintf(stderr, "decode failed\n");
goto cleanup;
}
if (decoded.header.segment_visibility != 0 ||
decoded.records[0].visibility != 0 ||
decoded.records[0].domain_id != 0 ||
decoded.records[0].has_cross_domain_source != 0 ||
decoded.records[0].cross_domain_source != 0) {
fprintf(stderr, "legacy defaults not applied\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_asl_core_index_free(&decoded);
cleanup:
free((void *)encoded.data);
return exit_code;
}
int main(void) {
if (test_round_trip() != 0) {
return 1;
}
if (test_crc_mismatch() != 0) {
return 1;
}
if (test_invalid_offsets() != 0) {
return 1;
}
if (test_invalid_federation_fields() != 0) {
return 1;
}
if (test_legacy_defaults() != 0) {
return 1;
}
return 0;
}

217
tests/enc/test_asl_log.c Normal file
View file

@ -0,0 +1,217 @@
#include "amduat/enc/asl_log.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static bool octets_equal(amduat_octets_t a, amduat_octets_t b) {
if (a.len != b.len) {
return false;
}
if (a.len == 0) {
return true;
}
return memcmp(a.data, b.data, a.len) == 0;
}
static int test_round_trip(void) {
uint8_t payload_a[] = {0x01, 0x02, 0x03};
uint8_t payload_b[] = {0x10, 0x20};
amduat_asl_log_record_t records[2];
amduat_octets_t encoded;
amduat_asl_log_record_t *decoded = NULL;
size_t decoded_len = 0;
int exit_code = 1;
memset(records, 0, sizeof(records));
records[0].logseq = 1;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[0].payload = amduat_octets(payload_a, sizeof(payload_a));
records[1].logseq = 2;
records[1].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
records[1].payload = amduat_octets(payload_b, sizeof(payload_b));
if (!amduat_enc_asl_log_encode_v1(records, 2, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
if (!amduat_enc_asl_log_decode_v1(encoded, &decoded, &decoded_len)) {
fprintf(stderr, "decode failed\n");
goto cleanup;
}
if (decoded_len != 2) {
fprintf(stderr, "decoded length mismatch\n");
goto cleanup_decoded;
}
if (decoded[0].logseq != records[0].logseq ||
decoded[1].logseq != records[1].logseq) {
fprintf(stderr, "decoded logseq mismatch\n");
goto cleanup_decoded;
}
if (decoded[0].record_type != records[0].record_type ||
decoded[1].record_type != records[1].record_type) {
fprintf(stderr, "decoded record type mismatch\n");
goto cleanup_decoded;
}
if (!octets_equal(decoded[0].payload, records[0].payload) ||
!octets_equal(decoded[1].payload, records[1].payload)) {
fprintf(stderr, "decoded payload mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_asl_log_free(decoded, decoded_len);
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_hash_chain_mutation(void) {
uint8_t payload_a[] = {0xaa, 0xbb, 0xcc};
amduat_asl_log_record_t record;
amduat_octets_t encoded;
amduat_asl_log_record_t *decoded = NULL;
size_t decoded_len = 0;
int exit_code = 1;
size_t payload_offset;
memset(&record, 0, sizeof(record));
record.logseq = 1;
record.record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
record.payload = amduat_octets(payload_a, sizeof(payload_a));
if (!amduat_enc_asl_log_encode_v1(&record, 1, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
payload_offset = 24 + 8 + 4 + 4;
if (encoded.len <= payload_offset) {
fprintf(stderr, "encoded payload offset invalid\n");
goto cleanup;
}
((uint8_t *)encoded.data)[payload_offset] ^= 0xffu;
if (amduat_enc_asl_log_decode_v1(encoded, &decoded, &decoded_len)) {
fprintf(stderr, "decode unexpectedly succeeded\n");
amduat_enc_asl_log_free(decoded, decoded_len);
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_unknown_record_type(void) {
uint8_t payload_a[] = {0x11};
uint8_t payload_b[] = {0x22};
uint8_t payload_c[] = {0x33};
amduat_asl_log_record_t records[3];
amduat_octets_t encoded;
amduat_asl_log_record_t *decoded = NULL;
size_t decoded_len = 0;
int exit_code = 1;
memset(records, 0, sizeof(records));
records[0].logseq = 1;
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
records[0].payload = amduat_octets(payload_a, sizeof(payload_a));
records[1].logseq = 2;
records[1].record_type = 0x99;
records[1].payload = amduat_octets(payload_b, sizeof(payload_b));
records[2].logseq = 3;
records[2].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
records[2].payload = amduat_octets(payload_c, sizeof(payload_c));
if (!amduat_enc_asl_log_encode_v1(records, 3, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
if (!amduat_enc_asl_log_decode_v1(encoded, &decoded, &decoded_len)) {
fprintf(stderr, "decode failed\n");
goto cleanup;
}
if (decoded_len != 2) {
fprintf(stderr, "unknown record not skipped\n");
goto cleanup_decoded;
}
if (decoded[0].logseq != records[0].logseq ||
decoded[1].logseq != records[2].logseq) {
fprintf(stderr, "decoded logseq mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_asl_log_free(decoded, decoded_len);
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_truncated_payload(void) {
uint8_t payload_a[] = {0xaa, 0xbb, 0xcc, 0xdd};
amduat_asl_log_record_t record;
amduat_octets_t encoded;
amduat_asl_log_record_t *decoded = NULL;
size_t decoded_len = 0;
int exit_code = 1;
memset(&record, 0, sizeof(record));
record.logseq = 42;
record.record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
record.payload = amduat_octets(payload_a, sizeof(payload_a));
if (!amduat_enc_asl_log_encode_v1(&record, 1, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
if (encoded.len == 0) {
fprintf(stderr, "encoded length invalid\n");
goto cleanup;
}
if (amduat_enc_asl_log_decode_v1(
amduat_octets(encoded.data, encoded.len - 1),
&decoded,
&decoded_len)) {
fprintf(stderr, "decode unexpectedly succeeded\n");
amduat_enc_asl_log_free(decoded, decoded_len);
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
int main(void) {
if (test_round_trip() != 0) {
return 1;
}
if (test_hash_chain_mutation() != 0) {
return 1;
}
if (test_unknown_record_type() != 0) {
return 1;
}
if (test_truncated_payload() != 0) {
return 1;
}
return 0;
}

View file

@ -0,0 +1,231 @@
#include "amduat/enc/asl_tgk_exec_plan.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
enum {
k_plan_header_size = 8,
k_params_size = 65,
k_operator_size = 129,
k_input_offset = 97
};
static void store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void fill_default_params(amduat_asl_tgk_exec_operator_params_t *params) {
memset(params, 0, sizeof(*params));
params->tgk_traversal.direction = 1u;
}
static int test_round_trip(void) {
amduat_asl_tgk_exec_operator_def_t operators[2];
amduat_asl_tgk_exec_plan_t plan;
amduat_octets_t encoded;
amduat_asl_tgk_exec_plan_t decoded;
int exit_code = 1;
memset(operators, 0, sizeof(operators));
fill_default_params(&operators[0].params);
fill_default_params(&operators[1].params);
operators[0].op_id = 1u;
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
operators[0].flags = AMDUAT_ASL_TGK_EXEC_OP_FLAG_NONE;
operators[0].snapshot.logseq_min = 0u;
operators[0].snapshot.logseq_max = 10u;
operators[0].input_count = 0u;
operators[1].op_id = 2u;
operators[1].op_type = AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL;
operators[1].flags = AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL;
operators[1].snapshot.logseq_min = 1u;
operators[1].snapshot.logseq_max = 10u;
operators[1].params.tgk_traversal.direction = 2u;
operators[1].params.tgk_traversal.traversal_depth = 3u;
operators[1].params.tgk_traversal.start_node_id = 42u;
operators[1].input_count = 1u;
operators[1].inputs[0] = 1u;
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
plan.operator_count = 2u;
plan.operators = operators;
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
if (!amduat_enc_asl_tgk_exec_plan_decode_v1(encoded, &decoded)) {
fprintf(stderr, "decode failed\n");
goto cleanup;
}
if (decoded.plan_version != plan.plan_version ||
decoded.operator_count != plan.operator_count) {
fprintf(stderr, "header mismatch\n");
goto cleanup_decoded;
}
if (decoded.operators[1].op_type != operators[1].op_type ||
decoded.operators[1].params.tgk_traversal.direction != 2u ||
decoded.operators[1].params.tgk_traversal.traversal_depth != 3u ||
decoded.operators[1].params.tgk_traversal.start_node_id != 42u) {
fprintf(stderr, "operator fields mismatch\n");
goto cleanup_decoded;
}
if (decoded.operators[1].input_count != 1u ||
decoded.operators[1].inputs[0] != 1u) {
fprintf(stderr, "inputs mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_asl_tgk_exec_plan_free(&decoded);
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_invalid_input_ref(void) {
amduat_asl_tgk_exec_operator_def_t operators[1];
amduat_asl_tgk_exec_plan_t plan;
amduat_octets_t encoded;
int exit_code = 1;
size_t op_offset;
size_t input_offset;
memset(operators, 0, sizeof(operators));
fill_default_params(&operators[0].params);
operators[0].op_id = 1u;
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
operators[0].input_count = 0u;
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
plan.operator_count = 1u;
plan.operators = operators;
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
op_offset = k_plan_header_size;
if (encoded.len < op_offset + k_input_offset + 4u) {
fprintf(stderr, "encoded buffer too small\n");
goto cleanup;
}
store_u32_le((uint8_t *)encoded.data + op_offset + k_params_size + 28u, 1u);
input_offset = op_offset + k_input_offset;
store_u32_le((uint8_t *)encoded.data + input_offset, 99u);
if (amduat_enc_asl_tgk_exec_plan_decode_v1(encoded, &(amduat_asl_tgk_exec_plan_t){0})) {
fprintf(stderr, "decode unexpectedly succeeded\n");
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_truncated_decode(void) {
amduat_asl_tgk_exec_operator_def_t operators[1];
amduat_asl_tgk_exec_plan_t plan;
amduat_octets_t encoded;
amduat_octets_t truncated;
int exit_code = 1;
memset(operators, 0, sizeof(operators));
fill_default_params(&operators[0].params);
operators[0].op_id = 1u;
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
plan.operator_count = 1u;
plan.operators = operators;
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
truncated = amduat_octets(encoded.data, encoded.len - 1u);
if (amduat_enc_asl_tgk_exec_plan_decode_v1(truncated,
&(amduat_asl_tgk_exec_plan_t){0})) {
fprintf(stderr, "decode unexpectedly succeeded\n");
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_invalid_input_count(void) {
amduat_asl_tgk_exec_operator_def_t operators[1];
amduat_asl_tgk_exec_plan_t plan;
amduat_octets_t encoded;
int exit_code = 1;
size_t op_offset;
size_t input_count_offset;
memset(operators, 0, sizeof(operators));
fill_default_params(&operators[0].params);
operators[0].op_id = 1u;
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
plan.operator_count = 1u;
plan.operators = operators;
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
op_offset = k_plan_header_size;
input_count_offset = op_offset + k_params_size + 28u;
if (encoded.len < input_count_offset + 4u) {
fprintf(stderr, "encoded buffer too small\n");
goto cleanup;
}
store_u32_le((uint8_t *)encoded.data + input_count_offset, 9u);
if (amduat_enc_asl_tgk_exec_plan_decode_v1(encoded,
&(amduat_asl_tgk_exec_plan_t){0})) {
fprintf(stderr, "decode unexpectedly succeeded\n");
goto cleanup;
}
exit_code = 0;
cleanup:
free((void *)encoded.data);
return exit_code;
}
int main(void) {
if (test_round_trip() != 0) {
return 1;
}
if (test_invalid_input_ref() != 0) {
return 1;
}
if (test_truncated_decode() != 0) {
return 1;
}
return test_invalid_input_count();
}

View file

@ -1,5 +1,6 @@
#include "amduat/enc/fer1_receipt.h" #include "amduat/enc/fer1_receipt.h"
#include "amduat/fer/receipt.h" #include "amduat/fer/receipt.h"
#include "amduat/pel/run.h"
#include <stdbool.h> #include <stdbool.h>
#include <stdint.h> #include <stdint.h>
@ -38,16 +39,15 @@ static const uint8_t k_expected_receipt_bytes[] = {
0x01, 0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x60, 0x60, 0x60, 0x60, 0x60, 0x01, 0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x60, 0x60, 0x60, 0x60, 0x60,
0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60,
0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60,
0x60, 0x60, 0x60, 0x60, 0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, 0x00, 0x60, 0x60, 0x60, 0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, 0x00, 0x00,
0x00, 0x00, 0x22, 0x00, 0x01, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x00, 0x22, 0x00, 0x01, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
0x51, 0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x44, 0x44, 0x44, 0x44, 0x44, 0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44,
0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44,
0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44,
0x44, 0x44, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x44, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x14
0x00, 0x14,
}; };
static const uint8_t k_expected_receipt_helper_bytes[] = { static const uint8_t k_expected_receipt_helper_bytes[] = {
@ -103,6 +103,7 @@ static bool bytes_equal(amduat_octets_t bytes,
static int test_receipt_round_trip(void) { static int test_receipt_round_trip(void) {
amduat_fer1_receipt_t receipt; amduat_fer1_receipt_t receipt;
amduat_octets_t encoded; amduat_octets_t encoded;
amduat_octets_t mutated;
amduat_fer1_receipt_t decoded; amduat_fer1_receipt_t decoded;
amduat_reference_t executor_refs[2]; amduat_reference_t executor_refs[2];
amduat_fer1_parity_entry_t parity[2]; amduat_fer1_parity_entry_t parity[2];
@ -261,6 +262,640 @@ static int test_receipt_helper(void) {
return 0; return 0;
} }
static int test_receipt_helper_failed_run(void) {
amduat_pel_run_result_t pel_run;
amduat_artifact_t artifact;
amduat_fer1_receipt_t decoded;
uint8_t f0[32], i0[32], e0[32], r0[32];
uint8_t ex0[32];
int exit_code = 1;
memset(&pel_run, 0, sizeof(pel_run));
pel_run.result_ref = make_ref(0x77, r0);
pel_run.output_refs = NULL;
pel_run.output_refs_len = 0;
pel_run.has_result_value = true;
pel_run.result_value.pel1_version = 1;
pel_run.result_value.program_ref = make_ref(0x11, f0);
if (!amduat_fer1_receipt_from_pel_run(
&pel_run,
make_ref(0x22, i0),
make_ref(0x33, e0),
amduat_octets("tester", 6),
make_ref(0x50, ex0),
false,
amduat_reference(0, amduat_octets(NULL, 0)),
amduat_octets(NULL, 0),
10,
20,
&artifact)) {
fprintf(stderr, "failed run helper failed\n");
return exit_code;
}
if (!amduat_enc_fer1_receipt_decode_v1(artifact.bytes, &decoded)) {
fprintf(stderr, "failed run helper decode failed\n");
amduat_artifact_free(&artifact);
return exit_code;
}
if (!amduat_reference_eq(decoded.output_ref, pel_run.result_ref) ||
!amduat_reference_eq(decoded.function_ref,
pel_run.result_value.program_ref)) {
fprintf(stderr, "failed run helper decoded refs mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_fer1_receipt_free(&decoded);
amduat_artifact_free(&artifact);
return exit_code;
}
static int test_receipt_round_trip_v1_1(void) {
amduat_fer1_receipt_t receipt;
amduat_octets_t encoded;
amduat_fer1_receipt_t decoded;
amduat_octets_t mutated;
amduat_reference_t executor_refs[1];
amduat_fer1_parity_entry_t parity[1];
amduat_fer1_log_entry_t logs[2];
uint8_t f0[32], i0[32], e0[32], o0[32];
uint8_t ex0[32], fp0[32], lr0[32], lr1[32];
uint8_t digest0[] = {0xaa, 0xbb, 0xcc};
uint8_t run_id[] = {0x01, 0x02, 0x03, 0x04};
uint8_t rng_seed[] = {0x09, 0x08, 0x07};
uint8_t signature[] = {0xde, 0xad, 0xbe, 0xef};
int exit_code = 1;
memset(&receipt, 0, sizeof(receipt));
receipt.fer1_version = AMDUAT_FER1_VERSION_1_1;
receipt.function_ref = make_ref(0x11, f0);
receipt.input_manifest_ref = make_ref(0x22, i0);
receipt.environment_ref = make_ref(0x33, e0);
receipt.evaluator_id = amduat_octets("tester", 6);
receipt.output_ref = make_ref(0x44, o0);
receipt.started_at = 10;
receipt.completed_at = 20;
executor_refs[0] = make_ref(0x50, ex0);
receipt.executor_refs = executor_refs;
receipt.executor_refs_len = 1;
memset(parity, 0, sizeof(parity));
parity[0].executor_ref = executor_refs[0];
parity[0].output_ref = receipt.output_ref;
parity[0].has_sbom_ref = false;
parity[0].parity_digest = amduat_octets(digest0, sizeof(digest0));
receipt.parity = parity;
receipt.parity_len = 1;
receipt.has_executor_fingerprint_ref = true;
receipt.executor_fingerprint_ref = make_ref(0x66, fp0);
receipt.has_run_id = true;
receipt.run_id = amduat_octets(run_id, sizeof(run_id));
receipt.has_limits = true;
receipt.limits.cpu_ms = 1;
receipt.limits.wall_ms = 2;
receipt.limits.max_rss_kib = 3;
receipt.limits.io_reads = 4;
receipt.limits.io_writes = 5;
memset(logs, 0, sizeof(logs));
logs[0].kind = 1;
logs[0].log_ref = make_ref(0x70, lr0);
logs[0].sha256 = amduat_octets(digest0, sizeof(digest0));
logs[1].kind = 2;
logs[1].log_ref = make_ref(0x71, lr1);
logs[1].sha256 = amduat_octets(NULL, 0);
receipt.logs = logs;
receipt.logs_len = 2;
receipt.has_determinism = true;
receipt.determinism_level = 2;
receipt.has_rng_seed = true;
receipt.rng_seed = amduat_octets(rng_seed, sizeof(rng_seed));
receipt.has_signature = true;
receipt.signature = amduat_octets(signature, sizeof(signature));
if (!amduat_enc_fer1_receipt_encode_v1_1(&receipt, &encoded)) {
fprintf(stderr, "encode v1.1 failed\n");
return exit_code;
}
if (!amduat_enc_fer1_receipt_decode_v1_1(encoded, &decoded)) {
fprintf(stderr, "decode v1.1 failed\n");
goto cleanup;
}
if (decoded.fer1_version != AMDUAT_FER1_VERSION_1_1 ||
!amduat_reference_eq(decoded.function_ref, receipt.function_ref) ||
!amduat_reference_eq(decoded.input_manifest_ref,
receipt.input_manifest_ref) ||
!amduat_reference_eq(decoded.environment_ref,
receipt.environment_ref) ||
!amduat_reference_eq(decoded.output_ref, receipt.output_ref)) {
fprintf(stderr, "decoded v1.1 refs mismatch\n");
goto cleanup_decoded;
}
if (!decoded.has_executor_fingerprint_ref ||
!amduat_reference_eq(decoded.executor_fingerprint_ref,
receipt.executor_fingerprint_ref) ||
!decoded.has_run_id ||
!amduat_octets_eq(decoded.run_id, receipt.run_id) ||
!decoded.has_determinism ||
decoded.determinism_level != receipt.determinism_level ||
!decoded.has_rng_seed ||
!amduat_octets_eq(decoded.rng_seed, receipt.rng_seed) ||
!decoded.has_signature ||
!amduat_octets_eq(decoded.signature, receipt.signature) ||
!decoded.has_limits ||
decoded.limits.cpu_ms != receipt.limits.cpu_ms ||
decoded.limits.wall_ms != receipt.limits.wall_ms ||
decoded.limits.max_rss_kib != receipt.limits.max_rss_kib ||
decoded.limits.io_reads != receipt.limits.io_reads ||
decoded.limits.io_writes != receipt.limits.io_writes) {
fprintf(stderr, "decoded v1.1 fields mismatch\n");
goto cleanup_decoded;
}
if (decoded.logs_len != receipt.logs_len ||
!amduat_reference_eq(decoded.logs[0].log_ref, logs[0].log_ref) ||
decoded.logs[0].kind != logs[0].kind ||
!amduat_octets_eq(decoded.logs[0].sha256, logs[0].sha256) ||
!amduat_reference_eq(decoded.logs[1].log_ref, logs[1].log_ref) ||
decoded.logs[1].kind != logs[1].kind ||
decoded.logs[1].sha256.len != 0) {
fprintf(stderr, "decoded v1.1 logs mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_fer1_receipt_free(&decoded);
cleanup:
free((void *)encoded.data);
return exit_code;
}
static int test_receipt_v1_1_reject_duplicate_tag(void) {
amduat_fer1_receipt_t receipt;
amduat_octets_t encoded;
amduat_fer1_receipt_t decoded;
amduat_octets_t mutated;
amduat_reference_t executor_refs[1];
amduat_fer1_parity_entry_t parity[1];
amduat_fer1_log_entry_t logs[1];
uint8_t f0[32], i0[32], e0[32], o0[32];
uint8_t ex0[32], fp0[32], lr0[32];
uint8_t digest0[] = {0xaa, 0xbb, 0xcc};
uint8_t run_id[] = {0x01, 0x02, 0x03, 0x04};
size_t offset = 0;
size_t ext_len = 0;
size_t ext_offset = 0;
size_t tlv_offset = 0;
uint32_t len_u32 = 0;
size_t count = 0;
size_t i = 0;
int exit_code = 1;
memset(&receipt, 0, sizeof(receipt));
receipt.fer1_version = AMDUAT_FER1_VERSION_1_1;
receipt.function_ref = make_ref(0x11, f0);
receipt.input_manifest_ref = make_ref(0x22, i0);
receipt.environment_ref = make_ref(0x33, e0);
receipt.evaluator_id = amduat_octets("tester", 6);
receipt.output_ref = make_ref(0x44, o0);
receipt.started_at = 10;
receipt.completed_at = 20;
executor_refs[0] = make_ref(0x50, ex0);
receipt.executor_refs = executor_refs;
receipt.executor_refs_len = 1;
memset(parity, 0, sizeof(parity));
parity[0].executor_ref = executor_refs[0];
parity[0].output_ref = receipt.output_ref;
parity[0].has_sbom_ref = false;
parity[0].parity_digest = amduat_octets(digest0, sizeof(digest0));
receipt.parity = parity;
receipt.parity_len = 1;
receipt.has_executor_fingerprint_ref = true;
receipt.executor_fingerprint_ref = make_ref(0x66, fp0);
receipt.has_run_id = true;
receipt.run_id = amduat_octets(run_id, sizeof(run_id));
receipt.has_limits = true;
receipt.limits.cpu_ms = 1;
receipt.limits.wall_ms = 2;
receipt.limits.max_rss_kib = 3;
receipt.limits.io_reads = 4;
receipt.limits.io_writes = 5;
memset(logs, 0, sizeof(logs));
logs[0].kind = 1;
logs[0].log_ref = make_ref(0x70, lr0);
logs[0].sha256 = amduat_octets(digest0, sizeof(digest0));
receipt.logs = logs;
receipt.logs_len = 1;
if (!amduat_enc_fer1_receipt_encode_v1_1(&receipt, &encoded)) {
fprintf(stderr, "encode v1.1 failed\n");
return exit_code;
}
if (encoded.len < 2) {
fprintf(stderr, "encoded v1.1 too short\n");
goto cleanup;
}
mutated = amduat_octets(NULL, 0u);
if (encoded.len != 0) {
uint8_t *buffer = (uint8_t *)malloc(encoded.len);
if (buffer == NULL) {
fprintf(stderr, "encoded v1.1 alloc failed\n");
goto cleanup;
}
memcpy(buffer, encoded.data, encoded.len);
mutated = amduat_octets(buffer, encoded.len);
}
if (mutated.data == NULL || mutated.len != encoded.len) {
fprintf(stderr, "encoded v1.1 clone failed\n");
goto cleanup;
}
if (encoded.len - offset < 2) {
fprintf(stderr, "encoded v1.1 header too short\n");
goto cleanup;
}
offset += 2;
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 function_ref missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + len_u32;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 function_ref truncated\n");
goto cleanup;
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 input_manifest_ref missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + len_u32;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 input_manifest_ref truncated\n");
goto cleanup;
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 environment_ref missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + len_u32;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 environment_ref truncated\n");
goto cleanup;
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 evaluator_id missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + len_u32;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 evaluator_id truncated\n");
goto cleanup;
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 output_ref missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + len_u32;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 output_ref truncated\n");
goto cleanup;
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 executor_count missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4;
count = (size_t)len_u32;
for (i = 0; i < count; ++i) {
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 executor_ref missing\n");
goto cleanup;
}
uint32_t ref_len = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + ref_len;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 executor_ref truncated\n");
goto cleanup;
}
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 parity_count missing\n");
goto cleanup;
}
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4;
count = (size_t)len_u32;
for (i = 0; i < count; ++i) {
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 parity executor_ref missing\n");
goto cleanup;
}
uint32_t ref_len = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + ref_len;
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 parity output_ref missing\n");
goto cleanup;
}
ref_len = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + ref_len;
if (offset >= encoded.len) {
fprintf(stderr, "encoded v1.1 parity truncated\n");
goto cleanup;
}
if (encoded.data[offset] == 0x01u) {
offset += 1;
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 parity sbom missing\n");
goto cleanup;
}
ref_len = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + ref_len;
} else {
offset += 1;
}
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 parity digest missing\n");
goto cleanup;
}
uint32_t digest_len = (uint32_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
offset += 4 + digest_len;
if (offset > encoded.len) {
fprintf(stderr, "encoded v1.1 parity digest truncated\n");
goto cleanup;
}
}
if (offset + 16 > encoded.len) {
fprintf(stderr, "encoded v1.1 timestamps missing\n");
goto cleanup;
}
offset += 16;
if (offset + 4 > encoded.len) {
fprintf(stderr, "encoded v1.1 ext_len missing\n");
goto cleanup;
}
ext_len = (size_t)((encoded.data[offset] << 24) |
(encoded.data[offset + 1] << 16) |
(encoded.data[offset + 2] << 8) |
encoded.data[offset + 3]);
ext_offset = offset + 4;
if (ext_offset + ext_len > encoded.len) {
fprintf(stderr, "encoded v1.1 ext payload truncated\n");
goto cleanup;
}
tlv_offset = ext_offset;
if (tlv_offset + 6 > encoded.len) {
fprintf(stderr, "encoded v1.1 tlv header missing\n");
goto cleanup;
}
tlv_offset += 2;
len_u32 = (uint32_t)((encoded.data[tlv_offset] << 24) |
(encoded.data[tlv_offset + 1] << 16) |
(encoded.data[tlv_offset + 2] << 8) |
encoded.data[tlv_offset + 3]);
tlv_offset += 4 + len_u32;
if (tlv_offset + 2 > encoded.len) {
fprintf(stderr, "encoded v1.1 second tlv missing\n");
goto cleanup;
}
((uint8_t *)mutated.data)[tlv_offset] = 0x00u;
((uint8_t *)mutated.data)[tlv_offset + 1] = 0x01u;
if (amduat_enc_fer1_receipt_decode_v1_1(mutated, &decoded)) {
fprintf(stderr, "duplicate tlv accepted\n");
amduat_enc_fer1_receipt_free(&decoded);
goto cleanup;
}
exit_code = 0;
cleanup:
if (mutated.data != NULL) {
free((void *)mutated.data);
}
free((void *)encoded.data);
return exit_code;
}
static int test_receipt_helper_v1_1(void) {
amduat_pel_run_result_t pel_run;
amduat_artifact_t artifact;
amduat_fer1_receipt_t decoded;
amduat_fer1_log_entry_t log_entry;
amduat_fer1_limits_t limits;
uint8_t f0[32], i0[32], e0[32], o0[32], ex0[32], fp0[32], lr0[32];
uint8_t run_id[] = {0x01, 0x02, 0x03, 0x04};
uint8_t rng_seed[] = {0x09, 0x08, 0x07};
uint8_t signature[] = {0xde, 0xad, 0xbe, 0xef};
uint8_t digest0[] = {0xaa, 0xbb, 0xcc};
int exit_code = 1;
memset(&pel_run, 0, sizeof(pel_run));
pel_run.result_ref = make_ref(0x77, o0);
pel_run.output_refs = &pel_run.result_ref;
pel_run.output_refs_len = 1;
pel_run.has_result_value = true;
pel_run.result_value.pel1_version = 1;
pel_run.result_value.program_ref = make_ref(0x11, f0);
memset(&limits, 0, sizeof(limits));
limits.cpu_ms = 1;
limits.wall_ms = 2;
limits.max_rss_kib = 3;
limits.io_reads = 4;
limits.io_writes = 5;
memset(&log_entry, 0, sizeof(log_entry));
log_entry.kind = 1;
log_entry.log_ref = make_ref(0x70, lr0);
log_entry.sha256 = amduat_octets(digest0, sizeof(digest0));
if (!amduat_fer1_receipt_from_pel_run_v1_1(
&pel_run,
make_ref(0x22, i0),
make_ref(0x33, e0),
amduat_octets("tester", 6),
make_ref(0x50, ex0),
false,
amduat_reference(0, amduat_octets(NULL, 0)),
amduat_octets(NULL, 0),
10,
20,
true,
make_ref(0x66, fp0),
true,
amduat_octets(run_id, sizeof(run_id)),
true,
limits,
&log_entry,
1,
true,
2,
true,
amduat_octets(rng_seed, sizeof(rng_seed)),
true,
amduat_octets(signature, sizeof(signature)),
&artifact)) {
fprintf(stderr, "v1.1 helper failed\n");
return exit_code;
}
if (!amduat_enc_fer1_receipt_decode_v1_1(artifact.bytes, &decoded)) {
fprintf(stderr, "v1.1 helper decode failed\n");
amduat_artifact_free(&artifact);
return exit_code;
}
if (!decoded.has_run_id || !decoded.has_limits || !decoded.has_determinism ||
!decoded.has_signature ||
!amduat_reference_eq(decoded.output_ref, pel_run.result_ref)) {
fprintf(stderr, "v1.1 helper decoded fields mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_fer1_receipt_free(&decoded);
amduat_artifact_free(&artifact);
return exit_code;
}
static int test_receipt_helper_v1_1_failed_run(void) {
amduat_pel_run_result_t pel_run;
amduat_artifact_t artifact;
amduat_fer1_receipt_t decoded;
uint8_t f0[32], i0[32], e0[32], r0[32], ex0[32];
int exit_code = 1;
memset(&pel_run, 0, sizeof(pel_run));
pel_run.result_ref = make_ref(0x77, r0);
pel_run.output_refs = NULL;
pel_run.output_refs_len = 0;
pel_run.has_result_value = true;
pel_run.result_value.pel1_version = 1;
pel_run.result_value.program_ref = make_ref(0x11, f0);
if (!amduat_fer1_receipt_from_pel_run_v1_1(
&pel_run,
make_ref(0x22, i0),
make_ref(0x33, e0),
amduat_octets("tester", 6),
make_ref(0x50, ex0),
false,
amduat_reference(0, amduat_octets(NULL, 0)),
amduat_octets(NULL, 0),
10,
20,
false,
amduat_reference(0, amduat_octets(NULL, 0)),
false,
amduat_octets(NULL, 0),
false,
(amduat_fer1_limits_t){0},
NULL,
0,
false,
0,
false,
amduat_octets(NULL, 0),
false,
amduat_octets(NULL, 0),
&artifact)) {
fprintf(stderr, "v1.1 failed run helper failed\n");
return exit_code;
}
if (!amduat_enc_fer1_receipt_decode_v1_1(artifact.bytes, &decoded)) {
fprintf(stderr, "v1.1 failed run helper decode failed\n");
amduat_artifact_free(&artifact);
return exit_code;
}
if (!amduat_reference_eq(decoded.output_ref, pel_run.result_ref)) {
fprintf(stderr, "v1.1 failed run output_ref mismatch\n");
goto cleanup_decoded;
}
exit_code = 0;
cleanup_decoded:
amduat_enc_fer1_receipt_free(&decoded);
amduat_artifact_free(&artifact);
return exit_code;
}
int main(void) { int main(void) {
if (test_receipt_round_trip() != 0) { if (test_receipt_round_trip() != 0) {
return 1; return 1;
@ -271,5 +906,20 @@ int main(void) {
if (test_receipt_helper() != 0) { if (test_receipt_helper() != 0) {
return 1; return 1;
} }
if (test_receipt_helper_failed_run() != 0) {
return 1;
}
if (test_receipt_round_trip_v1_1() != 0) {
return 1;
}
if (test_receipt_v1_1_reject_duplicate_tag() != 0) {
return 1;
}
if (test_receipt_helper_v1_1() != 0) {
return 1;
}
if (test_receipt_helper_v1_1_failed_run() != 0) {
return 1;
}
return 0; return 0;
} }

View file

@ -0,0 +1,81 @@
#include "amduat/fed/ingest.h"
#include <stdio.h>
#include <string.h>
static amduat_reference_t make_ref(amduat_hash_id_t hash_id,
const uint8_t *bytes,
size_t len) {
return amduat_reference(hash_id, amduat_octets(bytes, len));
}
static amduat_fed_record_t make_record(uint32_t domain_id,
uint64_t logseq,
amduat_fed_record_type_t type,
amduat_reference_t ref) {
amduat_fed_record_t record;
memset(&record, 0, sizeof(record));
record.meta.domain_id = domain_id;
record.meta.visibility = 1;
record.meta.has_source = 0;
record.id.type = type;
record.id.ref = ref;
record.logseq = logseq;
record.snapshot_id = 1;
record.log_prefix = 10;
return record;
}
static int test_invalid_record(void) {
uint8_t key[] = {0x01};
amduat_fed_record_t record;
size_t error_index = 0;
record = make_record(1, 1, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, key, sizeof(key)));
record.meta.visibility = 2;
if (amduat_fed_ingest_validate(&record, 1, &error_index, NULL) !=
AMDUAT_FED_INGEST_ERR_INVALID) {
fprintf(stderr, "expected invalid error\n");
return 1;
}
if (error_index != 0) {
fprintf(stderr, "expected error index 0\n");
return 1;
}
return 0;
}
static int test_conflicting_duplicate(void) {
uint8_t key[] = {0x02};
amduat_reference_t ref = make_ref(1, key, sizeof(key));
amduat_fed_record_t records[2];
size_t error_index = 0;
size_t conflict_index = 0;
records[0] = make_record(1, 1, AMDUAT_FED_REC_ARTIFACT, ref);
records[1] = make_record(1, 2, AMDUAT_FED_REC_ARTIFACT, ref);
if (amduat_fed_ingest_validate(records, 2, &error_index, &conflict_index) !=
AMDUAT_FED_INGEST_ERR_CONFLICT) {
fprintf(stderr, "expected conflict error\n");
return 1;
}
if (error_index != 1 || conflict_index != 0) {
fprintf(stderr, "unexpected conflict indices\n");
return 1;
}
return 0;
}
int main(void) {
if (test_invalid_record() != 0) {
return 1;
}
if (test_conflicting_duplicate() != 0) {
return 1;
}
return 0;
}

View file

@ -0,0 +1,93 @@
#include "amduat/fed/registry.h"
#include <stdio.h>
#include <string.h>
static bool octets_equal(amduat_octets_t a, amduat_octets_t b) {
if (a.len != b.len) {
return false;
}
if (a.len == 0) {
return true;
}
return memcmp(a.data, b.data, a.len) == 0;
}
static int test_round_trip(void) {
uint8_t policy_a[] = {0xde, 0xad, 0xbe, 0xef};
amduat_fed_domain_state_t states[2];
amduat_fed_registry_value_t value;
amduat_fed_registry_value_t decoded;
amduat_octets_t encoded;
int exit_code = 1;
memset(states, 0, sizeof(states));
amduat_fed_registry_value_init(&value, states, 2);
states[0].domain_id = 42;
states[0].snapshot_id = 10;
states[0].log_prefix = 8;
states[0].last_logseq = 7;
states[0].admitted = 1;
states[0].policy_ok = 1;
states[0].policy_hash_id = 1;
states[0].policy_hash = amduat_octets(policy_a, sizeof(policy_a));
states[1].domain_id = 7;
states[1].snapshot_id = 3;
states[1].log_prefix = 2;
states[1].last_logseq = 2;
states[1].admitted = 0;
states[1].policy_ok = 0;
states[1].policy_hash_id = 0;
states[1].policy_hash = amduat_octets(NULL, 0);
if (!amduat_fed_registry_value_insert(&value, states[0]) ||
!amduat_fed_registry_value_insert(&value, states[1])) {
fprintf(stderr, "insert failed\n");
return exit_code;
}
if (!amduat_fed_registry_encode(&value, &encoded)) {
fprintf(stderr, "encode failed\n");
return exit_code;
}
memset(&decoded, 0, sizeof(decoded));
if (!amduat_fed_registry_decode(encoded, &decoded)) {
fprintf(stderr, "decode failed\n");
amduat_octets_free(&encoded);
return exit_code;
}
if (decoded.len != 2) {
fprintf(stderr, "decoded length mismatch\n");
goto cleanup;
}
if (decoded.states[0].domain_id != 7 ||
decoded.states[1].domain_id != 42) {
fprintf(stderr, "decoded order mismatch\n");
goto cleanup;
}
if (!octets_equal(decoded.states[1].policy_hash,
amduat_octets(policy_a, sizeof(policy_a)))) {
fprintf(stderr, "decoded policy hash mismatch\n");
goto cleanup;
}
exit_code = 0;
cleanup:
amduat_octets_free(&encoded);
amduat_fed_registry_value_free(&decoded);
return exit_code;
}
int main(void) {
if (test_round_trip() != 0) {
return 1;
}
return 0;
}

188
tests/fed/test_fed_replay.c Normal file
View file

@ -0,0 +1,188 @@
#include "amduat/fed/replay.h"
#include <stdio.h>
#include <string.h>
static amduat_reference_t make_ref(amduat_hash_id_t hash_id,
const uint8_t *bytes,
size_t len) {
return amduat_reference(hash_id, amduat_octets(bytes, len));
}
static amduat_fed_record_t make_record(uint32_t domain_id,
uint64_t logseq,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_record_type_t type,
amduat_reference_t ref) {
amduat_fed_record_t record;
memset(&record, 0, sizeof(record));
record.meta.domain_id = domain_id;
record.meta.visibility = 1;
record.meta.has_source = 0;
record.id.type = type;
record.id.ref = ref;
record.logseq = logseq;
record.snapshot_id = snapshot_id;
record.log_prefix = log_prefix;
return record;
}
static int test_ordering(void) {
uint8_t d0[] = {0x00};
uint8_t d1[] = {0x01};
uint8_t d2[] = {0x02};
amduat_fed_record_t records[3];
amduat_fed_replay_view_t view;
records[0] = make_record(1, 5, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d1, sizeof(d1)));
records[1] = make_record(1, 5, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d0, sizeof(d0)));
records[2] = make_record(1, 4, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d2, sizeof(d2)));
if (!amduat_fed_replay_build(records, 3, 1, 1, 10, &view)) {
fprintf(stderr, "replay failed\n");
return 1;
}
if (view.len != 3) {
fprintf(stderr, "ordering length mismatch\n");
amduat_fed_replay_view_free(&view);
return 1;
}
if (view.records[0].logseq != 4 ||
view.records[1].id.ref.digest.data[0] != 0x00 ||
view.records[2].id.ref.digest.data[0] != 0x01) {
fprintf(stderr, "ordering mismatch\n");
amduat_fed_replay_view_free(&view);
return 1;
}
amduat_fed_replay_view_free(&view);
return 0;
}
static int test_tombstone_scope(void) {
uint8_t key[] = {0xaa};
amduat_reference_t ref = make_ref(1, key, sizeof(key));
amduat_fed_record_t records[3];
amduat_fed_replay_view_t view;
records[0] = make_record(1, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT, ref);
records[1] = make_record(1, 2, 1, 10, AMDUAT_FED_REC_TOMBSTONE, ref);
records[2] = make_record(2, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT, ref);
if (!amduat_fed_replay_build(records, 3, 1, 1, 10, &view)) {
fprintf(stderr, "replay domain 1 failed\n");
return 1;
}
if (view.len != 0) {
fprintf(stderr, "tombstone scope mismatch (domain 1)\n");
amduat_fed_replay_view_free(&view);
return 1;
}
amduat_fed_replay_view_free(&view);
if (!amduat_fed_replay_build(records, 3, 2, 1, 10, &view)) {
fprintf(stderr, "replay domain 2 failed\n");
return 1;
}
if (view.len != 1) {
fprintf(stderr, "tombstone scope mismatch (domain 2)\n");
amduat_fed_replay_view_free(&view);
return 1;
}
amduat_fed_replay_view_free(&view);
return 0;
}
static int test_bounds(void) {
uint8_t key[] = {0xbb};
amduat_reference_t ref = make_ref(1, key, sizeof(key));
amduat_fed_record_t records[2];
amduat_fed_replay_view_t view;
records[0] = make_record(3, 1, 2, 10, AMDUAT_FED_REC_ARTIFACT, ref);
records[1] = make_record(3, 5, 1, 4, AMDUAT_FED_REC_ARTIFACT, ref);
if (!amduat_fed_replay_build(records, 2, 3, 1, 4, &view)) {
fprintf(stderr, "replay bounds failed\n");
return 1;
}
if (view.len != 0) {
fprintf(stderr, "bounds mismatch\n");
amduat_fed_replay_view_free(&view);
return 1;
}
amduat_fed_replay_view_free(&view);
return 0;
}
static int test_multi_domain_ordering(void) {
uint8_t d0[] = {0x00};
uint8_t d1[] = {0x01};
uint8_t d2[] = {0x02};
amduat_fed_record_t records[4];
amduat_fed_replay_view_t view_a;
amduat_fed_replay_view_t view_b;
records[0] = make_record(1, 2, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d2, sizeof(d2)));
records[1] = make_record(1, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d1, sizeof(d1)));
records[2] = make_record(2, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d0, sizeof(d0)));
records[3] = make_record(2, 2, 1, 10, AMDUAT_FED_REC_ARTIFACT,
make_ref(1, d2, sizeof(d2)));
if (!amduat_fed_replay_build(records, 4, 1, 1, 10, &view_a)) {
fprintf(stderr, "replay domain 1 failed\n");
return 1;
}
if (!amduat_fed_replay_build(records, 4, 2, 1, 10, &view_b)) {
fprintf(stderr, "replay domain 2 failed\n");
amduat_fed_replay_view_free(&view_a);
return 1;
}
if (view_a.len != 2 || view_b.len != 2) {
fprintf(stderr, "multi-domain lengths mismatch\n");
amduat_fed_replay_view_free(&view_a);
amduat_fed_replay_view_free(&view_b);
return 1;
}
if (view_a.records[0].logseq != 1 || view_a.records[1].logseq != 2) {
fprintf(stderr, "domain 1 ordering mismatch\n");
amduat_fed_replay_view_free(&view_a);
amduat_fed_replay_view_free(&view_b);
return 1;
}
if (view_b.records[0].logseq != 1 || view_b.records[1].logseq != 2) {
fprintf(stderr, "domain 2 ordering mismatch\n");
amduat_fed_replay_view_free(&view_a);
amduat_fed_replay_view_free(&view_b);
return 1;
}
amduat_fed_replay_view_free(&view_a);
amduat_fed_replay_view_free(&view_b);
return 0;
}
int main(void) {
if (test_ordering() != 0) {
return 1;
}
if (test_tombstone_scope() != 0) {
return 1;
}
if (test_bounds() != 0) {
return 1;
}
if (test_multi_domain_ordering() != 0) {
return 1;
}
return 0;
}

241
tests/fed/test_fed_view.c Normal file
View file

@ -0,0 +1,241 @@
#include "amduat/fed/view.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct {
amduat_reference_t ref;
amduat_artifact_t artifact;
} stub_store_entry_t;
typedef struct {
amduat_asl_store_config_t config;
stub_store_entry_t entries[4];
size_t entries_len;
} stub_store_t;
static void stub_store_init(stub_store_t *store) {
memset(store, 0, sizeof(*store));
store->config.hash_id = 1;
}
static amduat_asl_store_error_t stub_store_get(void *ctx,
amduat_reference_t ref,
amduat_artifact_t *out) {
stub_store_t *store = (stub_store_t *)ctx;
size_t i;
for (i = 0; i < store->entries_len; ++i) {
if (amduat_reference_eq(store->entries[i].ref, ref)) {
amduat_artifact_t src = store->entries[i].artifact;
uint8_t *payload = NULL;
if (src.bytes.len != 0u) {
payload = (uint8_t *)malloc(src.bytes.len);
if (payload == NULL) {
return AMDUAT_ASL_STORE_ERR_IO;
}
memcpy(payload, src.bytes.data, src.bytes.len);
}
out->bytes = amduat_octets(payload, src.bytes.len);
out->has_type_tag = src.has_type_tag;
out->type_tag = src.type_tag;
return AMDUAT_ASL_STORE_OK;
}
}
return AMDUAT_ASL_STORE_ERR_NOT_FOUND;
}
static amduat_reference_t make_ref(amduat_hash_id_t hash_id,
const uint8_t *bytes,
size_t len) {
return amduat_reference(hash_id, amduat_octets(bytes, len));
}
static amduat_fed_record_t make_record(uint32_t domain_id,
uint8_t visibility,
uint64_t logseq,
amduat_reference_t ref) {
amduat_fed_record_t record;
memset(&record, 0, sizeof(record));
record.meta.domain_id = domain_id;
record.meta.visibility = visibility;
record.meta.has_source = 0;
record.id.type = AMDUAT_FED_REC_ARTIFACT;
record.id.ref = ref;
record.logseq = logseq;
record.snapshot_id = 1;
record.log_prefix = 10;
return record;
}
static int test_view_and_resolve(void) {
uint8_t a_bytes[] = {0x01};
uint8_t b_bytes[] = {0x02};
uint8_t c_bytes[] = {0x03};
uint8_t d_bytes[] = {0x04};
amduat_reference_t ref_a = make_ref(1, a_bytes, sizeof(a_bytes));
amduat_reference_t ref_b = make_ref(1, b_bytes, sizeof(b_bytes));
amduat_reference_t ref_c = make_ref(1, c_bytes, sizeof(c_bytes));
amduat_reference_t ref_d = make_ref(1, d_bytes, sizeof(d_bytes));
amduat_fed_record_t records[4];
amduat_fed_view_bounds_t bounds[2];
amduat_fed_view_t view;
amduat_fed_policy_deny_t denies[1];
stub_store_t stub;
amduat_asl_store_ops_t ops;
amduat_asl_store_t store;
amduat_artifact_t artifact;
records[0] = make_record(1, 0, 1, ref_a);
records[1] = make_record(1, 1, 2, ref_b);
records[2] = make_record(2, 1, 1, ref_c);
records[3] = make_record(2, 0, 1, ref_d);
bounds[0].domain_id = 1;
bounds[0].snapshot_id = 1;
bounds[0].log_prefix = 10;
bounds[1].domain_id = 2;
bounds[1].snapshot_id = 1;
bounds[1].log_prefix = 10;
denies[0].id.type = AMDUAT_FED_REC_ARTIFACT;
denies[0].id.ref = ref_d;
denies[0].reason_code = 0;
if (amduat_fed_view_build(records, 4, 1, bounds, 2, denies, 1, &view) !=
AMDUAT_FED_VIEW_OK) {
fprintf(stderr, "view build failed\n");
return 1;
}
if (view.len != 3) {
fprintf(stderr, "view size mismatch\n");
amduat_fed_view_free(&view);
return 1;
}
stub_store_init(&stub);
stub.entries[0].ref = ref_a;
stub.entries[0].artifact = amduat_artifact(amduat_octets(a_bytes,
sizeof(a_bytes)));
stub.entries_len = 1;
amduat_asl_store_ops_init(&ops);
ops.get = stub_store_get;
amduat_asl_store_init(&store, stub.config, ops, &stub);
if (amduat_fed_resolve(&view, &store, ref_a, &artifact) !=
AMDUAT_FED_RESOLVE_OK) {
fprintf(stderr, "resolve local failed\n");
amduat_fed_view_free(&view);
return 1;
}
amduat_artifact_free(&artifact);
if (amduat_fed_resolve(&view, &store, ref_c, &artifact) !=
AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES) {
fprintf(stderr, "resolve remote mismatch\n");
amduat_fed_view_free(&view);
return 1;
}
if (amduat_fed_resolve(&view, &store, ref_d, &artifact) !=
AMDUAT_FED_RESOLVE_POLICY_DENIED) {
fprintf(stderr, "resolve policy denied mismatch\n");
amduat_fed_view_free(&view);
return 1;
}
amduat_fed_view_free(&view);
return 0;
}
static int test_view_conflict(void) {
uint8_t key[] = {0x09};
amduat_reference_t ref = make_ref(1, key, sizeof(key));
amduat_fed_record_t records[2];
amduat_fed_view_bounds_t bounds[2];
amduat_fed_view_t view;
records[0] = make_record(1, 1, 1, ref);
records[1] = make_record(2, 1, 1, ref);
bounds[0].domain_id = 1;
bounds[0].snapshot_id = 1;
bounds[0].log_prefix = 10;
bounds[1].domain_id = 2;
bounds[1].snapshot_id = 1;
bounds[1].log_prefix = 10;
if (amduat_fed_view_build(records, 2, 1, bounds, 2, NULL, 0, &view) !=
AMDUAT_FED_VIEW_ERR_CONFLICT) {
fprintf(stderr, "expected conflict\n");
amduat_fed_view_free(&view);
return 1;
}
return 0;
}
static int test_view_rebuild_metadata(void) {
uint8_t key[] = {0x08};
amduat_reference_t ref = make_ref(1, key, sizeof(key));
amduat_fed_record_t records[1];
amduat_fed_view_bounds_t bounds[1];
amduat_fed_view_t view_a;
amduat_fed_view_t view_b;
records[0] = make_record(1, 1, 1, ref);
records[0].meta.has_source = 1;
records[0].meta.source_domain = 5;
bounds[0].domain_id = 1;
bounds[0].snapshot_id = 1;
bounds[0].log_prefix = 10;
if (amduat_fed_view_build(records, 1, 1, bounds, 1, NULL, 0, &view_a) !=
AMDUAT_FED_VIEW_OK) {
fprintf(stderr, "view rebuild build A failed\n");
return 1;
}
if (amduat_fed_view_build(records, 1, 1, bounds, 1, NULL, 0, &view_b) !=
AMDUAT_FED_VIEW_OK) {
fprintf(stderr, "view rebuild build B failed\n");
amduat_fed_view_free(&view_a);
return 1;
}
if (view_a.len != 1 || view_b.len != 1) {
fprintf(stderr, "view rebuild length mismatch\n");
amduat_fed_view_free(&view_a);
amduat_fed_view_free(&view_b);
return 1;
}
if (view_a.records[0].meta.has_source !=
view_b.records[0].meta.has_source ||
view_a.records[0].meta.source_domain !=
view_b.records[0].meta.source_domain ||
view_a.records[0].meta.visibility != view_b.records[0].meta.visibility) {
fprintf(stderr, "view rebuild metadata mismatch\n");
amduat_fed_view_free(&view_a);
amduat_fed_view_free(&view_b);
return 1;
}
amduat_fed_view_free(&view_a);
amduat_fed_view_free(&view_b);
return 0;
}
int main(void) {
if (test_view_and_resolve() != 0) {
return 1;
}
if (test_view_conflict() != 0) {
return 1;
}
if (test_view_rebuild_metadata() != 0) {
return 1;
}
return 0;
}

View file

@ -0,0 +1,348 @@
#include "amduat/asl/ref_derive.h"
#include "amduat/asl/store.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/enc/pel1_result.h"
#include "amduat/enc/pel_program_dag.h"
#include "amduat/hash/asl1.h"
#include "amduat/pel/program_dag.h"
#include "amduat/pel/program_dag_desc.h"
#include "amduat/pel/run.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct {
amduat_reference_t ref;
amduat_artifact_t artifact;
} stub_store_entry_t;
typedef struct {
stub_store_entry_t *entries;
size_t len;
size_t cap;
amduat_asl_store_config_t config;
} stub_store_t;
static void stub_store_init(stub_store_t *store) {
if (store == NULL) {
return;
}
store->entries = NULL;
store->len = 0;
store->cap = 0;
store->config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
store->config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
}
static void stub_store_free(stub_store_t *store) {
size_t i;
if (store == NULL) {
return;
}
for (i = 0; i < store->len; ++i) {
stub_store_entry_t *entry = &store->entries[i];
free((void *)entry->ref.digest.data);
entry->ref.digest.data = NULL;
entry->ref.digest.len = 0;
free((void *)entry->artifact.bytes.data);
entry->artifact.bytes.data = NULL;
entry->artifact.bytes.len = 0;
}
free(store->entries);
store->entries = NULL;
store->len = 0;
store->cap = 0;
}
static bool stub_store_add_entry(stub_store_t *store,
amduat_reference_t stored_ref,
amduat_artifact_t stored_artifact) {
stub_store_entry_t *entries;
size_t new_cap;
if (store->len == store->cap) {
new_cap = store->cap == 0 ? 8 : store->cap * 2;
entries = (stub_store_entry_t *)realloc(store->entries,
new_cap * sizeof(*entries));
if (entries == NULL) {
return false;
}
store->entries = entries;
store->cap = new_cap;
}
store->entries[store->len++] =
(stub_store_entry_t){stored_ref, stored_artifact};
return true;
}
static amduat_asl_store_error_t stub_store_put(
void *ctx,
amduat_artifact_t artifact,
amduat_reference_t *out_ref) {
stub_store_t *store;
amduat_reference_t derived_ref;
amduat_reference_t stored_ref;
amduat_artifact_t stored_artifact;
size_t i;
if (ctx == NULL || out_ref == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
store = (stub_store_t *)ctx;
out_ref->hash_id = 0;
out_ref->digest = amduat_octets(NULL, 0);
derived_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
if (!amduat_asl_ref_derive(artifact,
store->config.encoding_profile_id,
store->config.hash_id,
&derived_ref,
NULL)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
*out_ref = derived_ref;
for (i = 0; i < store->len; ++i) {
if (amduat_reference_eq(store->entries[i].ref, *out_ref)) {
return AMDUAT_ASL_STORE_OK;
}
}
if (!amduat_reference_clone(*out_ref, &stored_ref)) {
amduat_reference_free(out_ref);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
stored_artifact.bytes = amduat_octets(NULL, 0u);
if (artifact.bytes.len != 0) {
uint8_t *payload = (uint8_t *)malloc(artifact.bytes.len);
if (payload == NULL) {
amduat_reference_free(out_ref);
amduat_reference_free(&stored_ref);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
memcpy(payload, artifact.bytes.data, artifact.bytes.len);
stored_artifact.bytes = amduat_octets(payload, artifact.bytes.len);
}
stored_artifact.has_type_tag = artifact.has_type_tag;
stored_artifact.type_tag = artifact.type_tag;
if (!stub_store_add_entry(store, stored_ref, stored_artifact)) {
amduat_reference_free(out_ref);
amduat_reference_free(&stored_ref);
free((void *)stored_artifact.bytes.data);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
return AMDUAT_ASL_STORE_OK;
}
static amduat_asl_store_error_t stub_store_get(
void *ctx,
amduat_reference_t ref,
amduat_artifact_t *out_artifact) {
stub_store_t *store;
size_t i;
if (ctx == NULL || out_artifact == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
store = (stub_store_t *)ctx;
out_artifact->bytes = amduat_octets(NULL, 0u);
out_artifact->has_type_tag = false;
out_artifact->type_tag = amduat_type_tag(0);
for (i = 0; i < store->len; ++i) {
stub_store_entry_t *entry = &store->entries[i];
if (amduat_reference_eq(entry->ref, ref)) {
uint8_t *payload = NULL;
if (entry->artifact.bytes.len != 0u) {
payload = (uint8_t *)malloc(entry->artifact.bytes.len);
if (payload == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
memcpy(payload, entry->artifact.bytes.data,
entry->artifact.bytes.len);
}
out_artifact->bytes = amduat_octets(payload,
entry->artifact.bytes.len);
out_artifact->has_type_tag = entry->artifact.has_type_tag;
out_artifact->type_tag = entry->artifact.type_tag;
return AMDUAT_ASL_STORE_OK;
}
}
return AMDUAT_ASL_STORE_ERR_NOT_FOUND;
}
static int test_program_concat_two_inputs(void) {
stub_store_t stub;
amduat_asl_store_ops_t ops;
amduat_asl_store_t store;
amduat_pel_program_t program;
amduat_pel_node_t *nodes;
amduat_pel_dag_input_t *inputs;
amduat_pel_root_ref_t *roots;
amduat_octets_t encoded;
amduat_type_tag_t program_tag;
amduat_asl_encoding_profile_id_t program_profile;
amduat_artifact_t program_artifact;
amduat_reference_t program_ref;
amduat_artifact_t input_artifacts[2];
amduat_reference_t input_refs[2];
amduat_pel_run_result_t run_result;
amduat_artifact_t output_artifact;
int exit_code = 0;
stub_store_init(&stub);
amduat_asl_store_ops_init(&ops);
ops.put = stub_store_put;
ops.get = stub_store_get;
amduat_asl_store_init(&store, stub.config, ops, &stub);
nodes = (amduat_pel_node_t *)calloc(1u, sizeof(*nodes));
inputs = (amduat_pel_dag_input_t *)calloc(2u, sizeof(*inputs));
roots = (amduat_pel_root_ref_t *)calloc(1u, sizeof(*roots));
if (nodes == NULL || inputs == NULL || roots == NULL) {
fprintf(stderr, "out of memory\n");
exit_code = 1;
goto cleanup;
}
inputs[0].kind = AMDUAT_PEL_DAG_INPUT_EXTERNAL;
inputs[0].value.external.input_index = 0u;
inputs[1].kind = AMDUAT_PEL_DAG_INPUT_EXTERNAL;
inputs[1].value.external.input_index = 1u;
nodes[0].id = 1u;
nodes[0].op.name =
amduat_octets("pel.bytes.concat", strlen("pel.bytes.concat"));
nodes[0].op.version = 1u;
nodes[0].inputs = inputs;
nodes[0].inputs_len = 2u;
nodes[0].params = amduat_octets(NULL, 0u);
roots[0].node_id = 1u;
roots[0].output_index = 0u;
program.nodes = nodes;
program.nodes_len = 1u;
program.roots = roots;
program.roots_len = 1u;
if (!amduat_pel_program_dag_validate(&program)) {
fprintf(stderr, "program validation failed\n");
exit_code = 1;
goto cleanup;
}
encoded = amduat_octets(NULL, 0u);
if (!amduat_enc_pel_program_dag_encode_v1(&program, &encoded)) {
fprintf(stderr, "program encode failed\n");
exit_code = 1;
goto cleanup;
}
if (!amduat_pel_program_dag_desc_get_program_binding(&program_tag,
&program_profile)) {
fprintf(stderr, "program binding failed\n");
free((void *)encoded.data);
exit_code = 1;
goto cleanup;
}
(void)program_profile;
program_artifact = amduat_artifact_with_type(encoded, program_tag);
program_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
if (amduat_asl_store_put(&store, program_artifact,
&program_ref) != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "store put program failed\n");
free((void *)encoded.data);
exit_code = 1;
goto cleanup;
}
free((void *)encoded.data);
input_artifacts[0] =
amduat_artifact(amduat_octets("foo", 3u));
input_artifacts[1] =
amduat_artifact(amduat_octets("bar", 3u));
input_refs[0] = amduat_reference(0u, amduat_octets(NULL, 0u));
input_refs[1] = amduat_reference(0u, amduat_octets(NULL, 0u));
if (amduat_asl_store_put(&store, input_artifacts[0],
&input_refs[0]) != AMDUAT_ASL_STORE_OK ||
amduat_asl_store_put(&store, input_artifacts[1],
&input_refs[1]) != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "store put inputs failed\n");
exit_code = 1;
goto cleanup;
}
memset(&run_result, 0, sizeof(run_result));
if (!amduat_pel_surf_run_with_result(
&store,
amduat_pel_program_dag_scheme_ref(),
program_ref,
input_refs,
2u,
false,
amduat_reference(0u, amduat_octets(NULL, 0u)),
&run_result)) {
fprintf(stderr, "surf run failed\n");
exit_code = 1;
goto cleanup;
}
if (!run_result.has_result_value ||
run_result.result_value.core_result.status != AMDUAT_PEL_EXEC_STATUS_OK) {
fprintf(stderr, "execution status not OK\n");
exit_code = 1;
goto cleanup_run;
}
if (run_result.output_refs_len != 1u) {
fprintf(stderr, "unexpected output_refs_len=%zu\n",
run_result.output_refs_len);
exit_code = 1;
goto cleanup_run;
}
memset(&output_artifact, 0, sizeof(output_artifact));
if (amduat_asl_store_get(&store,
run_result.output_refs[0],
&output_artifact) != AMDUAT_ASL_STORE_OK) {
fprintf(stderr, "store get output failed\n");
exit_code = 1;
goto cleanup_run;
}
if (output_artifact.bytes.len != 6u ||
memcmp(output_artifact.bytes.data, "foobar", 6u) != 0) {
fprintf(stderr, "unexpected output bytes\n");
exit_code = 1;
}
free((void *)output_artifact.bytes.data);
cleanup_run:
if (run_result.has_result_value) {
amduat_enc_pel1_result_free(&run_result.result_value);
}
if (run_result.output_refs != NULL) {
amduat_pel_surf_free_refs(run_result.output_refs,
run_result.output_refs_len);
}
amduat_pel_surf_free_ref(&run_result.result_ref);
cleanup:
free(nodes);
free(inputs);
free(roots);
stub_store_free(&stub);
return exit_code;
}
int main(void) {
return test_program_concat_two_inputs();
}

248
tier1/asl-core-index-1.md Normal file
View file

@ -0,0 +1,248 @@
# ASL/1-CORE-INDEX — Semantic Index Model
Status: Draft
Owner: Niklas Rydberg
Version: 0.1.0
SoT: No
Last Updated: 2025-11-16
Linked Phase Pack: N/A
Tags: [deterministic, index, semantics]
<!-- Source: /amduat-api/tier1/asl-core-index.md | Canonical: /amduat/tier1/asl-core-index-1.md -->
**Document ID:** `ASL/1-CORE-INDEX`
**Layer:** L0.5 — Semantic mapping over ASL/1-CORE values (no storage / encoding / lifecycle)
**Depends on (normative):**
* `ASL/1-CORE`
* `ASL/1-STORE`
**Informative references:**
* `ASL/STORE-INDEX/1` — store lifecycle and replay contracts
* `ENC/ASL-CORE-INDEX/1` — bytes-on-disk encoding profile
* `ASL/INDEX-ACCEL/1` — acceleration semantics (routing, filters, sharding)
* `ASL/LOG/1` — append-only semantic log (segment visibility)
* `TGK/1` — TGK edge visibility and traversal alignment
* `ASL/SYSTEM/1` — unified system view (PEL/TGK/federation alignment)
© 2025 Niklas Rydberg.
## License
Except where otherwise noted, this document (text and diagrams) is licensed under
the Creative Commons Attribution 4.0 International License (CC BY 4.0).
The identifier registries and mapping tables (e.g. TypeTag IDs, HashId
assignments, EdgeTypeId tables) are additionally made available under CC0 1.0
Universal (CC0) to enable unrestricted reuse in implementations and derivative
specifications.
Code examples in this document are provided under the Apache License 2.0 unless
explicitly stated otherwise. Test vectors, where present, are dedicated to the
public domain under CC0 1.0.
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/1-CORE-INDEX defines **semantic meaning only**. It does not define storage formats, on-disk encoding, or operational lifecycle. Those belong to ASL-STORE-INDEX, ASL/LOG/1, and ENC-ASL-CORE-INDEX.
---
## 1. Purpose & Non-Goals
### 1.1 Purpose
ASL/1-CORE-INDEX defines the **semantic model** for indexing artifacts:
* It specifies what it means to map an artifact identity to a byte location.
* It defines visibility, immutability, and shadowing semantics.
* It ensures deterministic lookup for a fixed snapshot and log position.
### 1.2 Non-goals
ASL/1-CORE-INDEX explicitly does **not** define:
* On-disk layouts, segment files, or memory representations.
* Block allocation, packing, GC, or lifecycle rules.
* Snapshot implementation details, checkpoints, or log storage.
* Performance optimizations (bloom filters, sharding, SIMD).
* Federation, provenance, or execution semantics.
---
## 2. Terminology
* **Artifact** — ASL/1 immutable value defined in ASL/1-CORE.
* **Reference** — ASL/1 content address of an Artifact (hash_id + digest).
* **StoreConfig**`{ encoding_profile, hash_id }` fixed per StoreSnapshot (ASL/1-STORE).
* **Block** — immutable storage unit containing artifact bytes.
* **BlockID** — opaque identifier for a block.
* **ArtifactExtent**`(BlockID, offset, length)` identifying a byte slice within a block.
* **ArtifactLocation** — ordered list of `ArtifactExtent` values that, when concatenated, produce the artifact bytes.
* **Degenerate store** — a store that treats each artifact as its own block,
with a single extent covering the entire blob.
* **Snapshot** — a checkpointed StoreSnapshot (ASL/1-STORE) used as a base state.
* **Append-Only Log** — ordered sequence of index-visible mutations after a snapshot.
* **CURRENT** — effective state after replaying a log position on a snapshot.
---
## 3. Core Mapping Semantics
### 3.1 Index Mapping
The index defines a semantic mapping:
```
Reference -> ArtifactLocation
```
For any visible `Reference`, there is exactly one `ArtifactLocation` at a given CURRENT state.
### 3.2 Determinism
For a fixed `{StoreConfig, Snapshot, LogPosition}`, lookup results MUST be deterministic. No nondeterministic input may affect index semantics.
### 3.3 StoreConfig Consistency
All references in an index view are interpreted under a fixed StoreConfig. Implementations MAY store only the digest portion in the index when `hash_id` is fixed by StoreConfig, but the semantic key is always a full `Reference`. Encoding profiles MUST allow variable-length digests; the digest length MUST be either explicit in the encoding or derivable from `hash_id` and StoreConfig.
---
## 4. ArtifactLocation Semantics
* An ArtifactLocation is an **ordered list** of ArtifactExtents.
* Each extent references immutable bytes within a block.
* The artifact bytes are defined by **concatenating extents in order**.
* A visible ArtifactLocation MUST be **non-empty** and MUST fully cover the artifact byte sequence with no gaps or extra bytes.
* Tombstone entries are visible but MUST have no ArtifactLocation; they only shadow prior entries.
* Extents MUST have `length > 0` and MUST reference valid byte ranges within their blocks.
* Extents MAY refer to the same BlockID multiple times, but the ordered concatenation MUST be deterministic and exact.
* An ArtifactLocation is valid only while all referenced blocks are retained.
* ASL/1-CORE-INDEX does not define how blocks are allocated or sealed; it only requires that referenced bytes are immutable for the lifetime of the mapping.
* In a degenerate store, an ArtifactLocation consists of a single extent that
spans the full blob in its dedicated block.
---
## 5. Visibility Model
An index entry is **visible** at CURRENT if and only if:
1. The entry is admitted by the store's visibility mechanism as defined in
`ASL/STORE-INDEX/1` (e.g., via sealed segments and an append-only log), for
the given snapshot/log position.
2. The referenced bytes are immutable (e.g., the underlying block is sealed by
store rules).
Visibility is binary; entries are either visible or not visible.
**Implementation note:** A store MAY implement a degenerate visibility
mechanism (e.g., a single implicit segment that is always sealed and a trivial
log position), which is sufficient for simple filesystem-backed stores such as
`asl_store_fs`.
---
## 6. Snapshot and Log Semantics
Snapshots provide a base mapping of sealed segments; the append-only log admits later segment seals and policy records that define subsequent changes.
The index state for a given CURRENT is defined as:
```
Index(CURRENT) = Index(snapshot) + replay(log_position)
```
Replay is strictly ordered, deterministic, and idempotent. Snapshot and log entries are semantically equivalent once replayed.
---
## 7. Immutability and Shadowing
### 7.1 Immutability
* Index entries are never mutated.
* Once visible, an entrys meaning does not change.
* Referenced bytes are immutable for the lifetime of the entry.
### 7.2 Shadowing
* Later entries MAY shadow earlier entries with the same Reference.
* Precedence is determined solely by log order.
* Snapshot boundaries do not alter shadowing semantics.
---
## 8. Tombstones (Optional)
Tombstones MAY be used to invalidate prior mappings.
* A tombstone shadows earlier entries for the same Reference.
* Tombstones are visibility policy records (see `ASL/LOG/1`) and are applied
during replay; they are not required to appear as index entries.
* If an encoding chooses to materialize tombstones in index segments, they MUST
have no `ArtifactLocation` and MUST follow the same visibility rules as other
entries.
---
## 9. Determinism Guarantees
For fixed:
* StoreConfig
* Snapshot
* Log prefix
ASL/1-CORE-INDEX guarantees:
* Deterministic lookup results
* Deterministic shadowing resolution
* Deterministic visibility
---
## 10. Normative Invariants
Conforming implementations MUST enforce:
1. No visibility without a sealed segment whose seal record is log-admitted (or snapshot-anchored).
2. No mutation of visible index entries.
3. Referenced bytes remain immutable for the entrys lifetime.
4. Shadowing follows strict log order.
5. Snapshot + log replay uniquely defines CURRENT.
6. Visible ArtifactLocations are non-empty and byte-exact (no gaps, no overrun), except for tombstones which have no ArtifactLocation.
Violation of any invariant constitutes index corruption.
---
## 11. Relationship to Other Specifications
| Layer | Responsibility |
| ------------------ | ---------------------------------------------------------- |
| ASL/1-CORE | Artifact semantics and identity |
| ASL/1-STORE | StoreSnapshot and put/get logical model |
| ASL/1-CORE-INDEX | Semantic mapping of Reference → ArtifactLocation |
| ASL-STORE-INDEX | Lifecycle, replay, and visibility contracts |
| ENC-ASL-CORE-INDEX | On-disk encoding for index segments and records |
---
## 12. Summary
ASL/1-CORE-INDEX specifies the semantic meaning of the index:
* It maps artifact References to byte locations deterministically.
* It defines visibility and shadowing rules across snapshot + log replay.
* It guarantees immutability and deterministic lookup.
It answers one question:
> *Given a Reference and a CURRENT state, where are the bytes?*

148
tier1/asl-dam-1.md Normal file
View file

@ -0,0 +1,148 @@
# ASL/DAM/1 -- Domain Authority Manifest
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-01-17
Tags: [authority, trust, policy, domains]
**Document ID:** `ASL/DAM/1`
**Layer:** L2 -- Authority semantics (no encoding)
**Depends on (normative):**
* `ASL/POLICY-HASH/1`
**Informative references:**
* `ASL/OCS/1` -- offline certificate system
* `ASL/OFFLINE-ROOT-TRUST/1` -- offline root policy
* `PER/SIGNATURE/1` -- PER signature verification
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/DAM/1 defines the **logical structure and semantics** of the Domain Authority Manifest. It does not define an encoding.
---
## 1. Purpose
The Domain Authority Manifest (DAM) defines **who may assert authority** on behalf of a domain.
It binds domain identity to principals, roles, and a policy hash.
---
## 2. Core Concepts
* **Principal**: a cryptographic public key
* **Role**: capability granted to a principal
* **Policy hash**: canonical hash binding policy constraints to a domain
* **Authority scope**: explicit actions this DAM authorizes within the domain
---
## 3. Roles (Minimal Set)
| Role | Capability |
| ---------- | ---------- |
| produce | Create internal artifacts |
| execute | Emit PERs |
| publish | Publish artifacts/snapshots |
| federate | Export published state |
| audit | Verify only, no mutations |
Roles are **capabilities**, not identities.
---
## 4. Logical Schema
```text
DomainAuthorityManifest {
domain_id : DomainID
version : u32
root_key : PublicKey
principals[] : PrincipalEntry
policy_hash : Hash
scope[] : AuthorityScope
}
PrincipalEntry {
principal_id : Hash
public_key : PublicKey
roles[] : Role
}
```
The DAM is immutable once published. Rotation is performed by publishing a new DAM.
### 4.1 Authority Scope
The optional `scope[]` constrains what this DAM authorizes. If omitted, scope is
assumed to include all roles defined in this document.
```text
AuthorityScope = {
"produce",
"execute",
"publish",
"federate",
"audit"
}
```
---
## 5. Admission and Pinning (Normative)
Before a DAM is trusted, a receiving domain MUST:
1. Admit the domain (see `ASL/DAP/1`).
2. Pin the DAM artifact to a snapshot.
3. Pin the DAM's `policy_hash` for that snapshot lineage.
---
## 6. Validation Rules (Normative)
A node MUST reject any action unless:
1. The DAM artifact is visible in the relevant snapshot.
2. The DAM hash matches the snapshot reference (if recorded).
3. The action is signed by a principal listed in the DAM.
4. The principal has the required role.
5. The DAM `root_key` is certified by the offline root trust chain.
6. The action is within the DAM's declared `scope[]` (if present).
---
## 7. Policy Binding
The DAM `policy_hash` binds the domain to a specific policy document. If policy changes, a new DAM MUST be published and referenced by new snapshots.
---
## 8. Idempotency and Rotation
* Replaying the same snapshot + DAM MUST yield identical authority decisions.
* Rotation is done by publishing a new DAM and referencing it in new snapshots.
* Old snapshots remain valid with the DAM they reference.
---
## 9. Non-Goals
* Encoding format
* Key rotation workflow
* Live revocation
---
## 10. Summary
ASL/DAM/1 defines the minimal authority document for a domain, binding principals and roles to a policy hash under an offline root trust chain.

165
tier1/asl-dap-1.md Normal file
View file

@ -0,0 +1,165 @@
# ASL/DAP/1 -- Domain Admission Protocol
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-01-17
Tags: [admission, trust, federation]
**Document ID:** `ASL/DAP/1`
**Layer:** L2 -- Admission semantics (no transport)
**Depends on (normative):**
* `ASL/DAM/1`
* `ASL/POLICY-HASH/1`
**Informative references:**
* `ASL/OCS/1` -- offline certificate system
* `ASL/OFFLINE-ROOT-TRUST/1`
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/DAP/1 defines **admission semantics** for recognizing external domains. It does not define transport.
---
## 1. Purpose
The Domain Admission Protocol defines how a receiving domain evaluates and admits an applicant domain before any federation or trust is granted.
---
## 2. Actors
| Actor | Role |
| ----- | ---- |
| Applicant Domain | Domain requesting admission |
| Receiving Domain | Domain evaluating admission |
| Operator | Optional human/policy agent |
---
## 3. Admission Object Model
An admission request MUST include:
* Domain Authority Manifest (DAM)
* Proof of possession of the DAM root key
* Requested admission scope
* Optional courtesy lease request
No artifacts, blocks, or logs are required for admission.
---
## 4. Admission Record (Normative)
The receiving domain MUST persist an admission record that is snapshot-pinned:
```text
AdmissionRecord {
domain_id
dam_hash
policy_hash
admitted_scope[]
decision
decision_epoch
}
```
The `decision_epoch` is a monotonically increasing local counter (not wall time).
---
## 5. Admission Flow (Normative)
1. **Submission**
* Applicant sends DAM and proof of root-key possession.
2. **Structural validation**
* Receiving domain verifies DAM schema and signature.
* Policy hash integrity MUST be verified.
3. **Policy compatibility**
* Receiving domain evaluates requested scope and policy alignment.
4. **Decision**
* Outcomes: ACCEPTED, ACCEPTED_LIMITED, DEFERRED, REJECTED.
---
## 6. Admission Guarantees
If accepted:
* DomainID is recognized by the receiving domain.
* Root key and policy hash are pinned.
* Admission scope is enforced for federation.
Admission does not imply trust in artifacts beyond the granted scope.
---
## 7. Scope Enforcement (Normative)
* Admission scope MUST gate federation view construction and replay admission.
* A receiving domain MUST NOT admit state outside the granted scope.
* Scope changes require a new admission decision and updated AdmissionRecord.
---
## 8. Courtesy Lease (Optional)
A courtesy lease is a **bounded, revocable grant of resources** without semantic trust.
```text
CourtesyLease {
lease_id
domain_id
granted_by_domain
resources: {
storage_bytes
block_count
snapshot_count
}
duration: {
start_time
end_time
}
constraints: {
encrypted_only: boolean
no_federation: boolean
no_public_indexing: boolean
}
}
```
Courtesy storage MAY be deleted upon lease expiry. Courtesy does not grant federation or publication rights.
---
## 9. Non-Goals
* Transport format
* PKI integration
* Live revocation or liveness checks
---
## 10. Summary
ASL/DAP/1 defines a deterministic admission process for domains, with optional courtesy leasing for rescue and bootstrap scenarios.

170
tier1/asl-domain-model-1.md Normal file
View file

@ -0,0 +1,170 @@
# ASL/DOMAIN-MODEL/1 -- Domain Topology and Publication Semantics
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-01-17
Tags: [domains, authority, publication, federation]
**Document ID:** `ASL/DOMAIN-MODEL/1`
**Layer:** L2 -- Domain topology and delegation semantics (no transport)
**Depends on (normative):**
* `ASL/DAM/1`
* `ASL/DAP/1`
* `ASL/POLICY-HASH/1`
* `ASL/FEDERATION/1`
* `ASL/LOG/1`
**Informative references:**
* `ASL/OCS/1` -- offline certificate system
* `ASL/OFFLINE-ROOT-TRUST/1`
* `ASL/SYSTEM/1` -- unified system view
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/DOMAIN-MODEL/1 defines domain topology and publication semantics. It does not define transport, storage layout, or encoding.
---
## 1. Purpose
This document defines how domains relate, how authority is delegated, and how publication is made safe and explicit.
It provides a stable mental model for personal, group, and shared domains without introducing implicit trust.
---
## 2. First Principles (Normative)
1. **No implicit inheritance.** Domains are not hierarchical by default; trust is explicit.
2. **Authority is explicit.** Authority is defined by DAM + certificates; it is never implied by naming or topology.
3. **Publication is explicit.** Visibility is controlled by index metadata and policy, not by storage or naming.
4. **Determinism is preserved.** All cross-domain visibility MUST be replayable from snapshots and logs.
---
## 3. Domain Roles (Common, Personal, Working)
These roles are semantic patterns; a deployment MAY create many instances.
### 3.1 Common Domain
A shared, conservative domain for stable artifacts and schemas.
Properties:
* High trust threshold
* Read-mostly, slow-changing
* Publishes broadly
### 3.2 Personal Domain
A domain anchored to a single personal identity and authority.
Properties:
* Root of local agency
* Owns offline roots and DAM
* Decides what to publish and to whom
### 3.3 Working / Ephemeral Domains
Task-focused domains created under delegated authority.
Properties:
* Narrow policy and scope
* Often short-lived
* Results MAY be promoted to personal or common domains
---
## 4. Delegation and Authority (Normative)
Delegation is explicit and certificate-based.
Rules:
* A domain MAY delegate authority to a new domain by issuing a `domain_root` certificate (ASL/OCS/1).
* Delegation MUST be recorded in the receiving domain's DAM and policy hash.
* Delegation does not create inheritance: the delegating domain does not gain automatic visibility into the new domain.
---
## 5. Publication and Visibility (Normative)
Publication is a visibility decision, not a storage action.
Rules:
* Only published artifacts are eligible for cross-domain visibility.
* Publication state MUST be encoded in index metadata (ENC-ASL-CORE-INDEX).
* Blocks and storage layouts MUST NOT be treated as publication units.
* Publication of snapshots (or snapshot hashes) is allowed but MUST NOT imply data publication.
---
## 6. Cross-Domain Trust and Admission (Normative)
Trust is established by explicit admission and policy compatibility.
Rules:
* A receiving domain MUST admit an external domain (ASL/DAP/1) before including its state.
* Policy hash compatibility MUST be checked before accepting published artifacts.
* A domain MAY pin a trusted foreign domain without reciprocal trust.
---
## 7. Safe Publication Patterns (Informative)
### 7.1 Personal to Personal Archive
```
personal/rescue -> personal/archive
```
* Publish explicitly from the working domain to an archival domain.
* Only published artifacts are visible across the boundary.
### 7.2 Personal to Group Domain
```
personal/project -> group/shared
```
* Requires admission by the group domain and policy compatibility.
* No unilateral publishing into the group domain.
### 7.3 Personal to Public Domain
```
personal/public -> common/public
```
* One-way trust is permitted.
* Public domain pins the personal domain; the personal domain need not pin the public domain.
---
## 8. Non-Goals
ASL/DOMAIN-MODEL/1 does not define:
* Transport or replication protocols
* Encoding formats
* Storage layouts or filesystem assumptions
* Governance workflows beyond admission and policy compatibility
Middle layer (informative): the daemon/service boundary around core logic that
owns network transport, admission workflows, and operational policy.
Implementation note (informative): core federation primitives live in
`include/amduat/fed/{registry,ingest,replay,view}.h`.

View file

@ -0,0 +1,113 @@
# ASL/ENCRYPTED-BLOCKS/1 -- Encrypted Block Storage Across Domains
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-01-17
Tags: [encryption, blocks, federation, storage]
**Document ID:** `ASL/ENCRYPTED-BLOCKS/1`
**Layer:** L2 -- Encrypted storage semantics (no transport)
**Depends on (normative):**
* `ASL-STORE-INDEX`
* `ASL/FEDERATION/1`
* `ASL/LOG/1`
**Informative references:**
* `ASL/DOMAIN-MODEL/1`
* `ASL/POLICY-HASH/1`
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/ENCRYPTED-BLOCKS/1 defines semantics for storing encrypted blocks across domains. It does not define encryption algorithms, key management, or transport.
---
## 1. Purpose
This document defines how encrypted blocks may be stored in a foreign domain without transferring semantic authority or decryption capability.
---
## 2. Core Principle (Normative)
A domain MAY store encrypted blocks for another domain, but MUST NOT assert semantic meaning for those bytes.
Meaning is owned by the domain that holds the decryption keys and index entries.
---
## 3. Encryption Model (Normative)
### 3.1 Block Encryption
Before sealing, a block MAY be encrypted:
```
plaintext_block
-> encrypt(K)
-> ciphertext_block
-> BlockID = H(ciphertext_block)
```
Rules:
* Encryption occurs before sealing.
* `BlockID` is computed over ciphertext bytes.
* Deterministic encryption is NOT required.
### 3.2 Key Ownership
* Encryption keys are owned by the originating domain.
* Keys MUST NOT be federated or embedded in index metadata.
* Decryption metadata MUST remain local to the originating domain.
---
## 4. Foreign Domain Storage (Normative)
A foreign domain storing encrypted blocks:
* Treats ciphertext blocks as opaque bytes.
* MAY retain or GC blocks under its local policy.
* MUST NOT create semantic index entries for those blocks.
---
## 5. Originating Domain References (Normative)
The originating domain:
* Maintains index entries referencing the ciphertext `BlockID`.
* Applies normal visibility, log, and snapshot rules.
* Uses local decryption metadata to materialize plaintext.
---
## 6. Cross-Domain References (Informative)
Two references are distinct:
* **Storage reference:** foreign domain stores ciphertext blocks.
* **Semantic reference:** originating domain records artifact visibility and meaning.
Foreign storage does not imply federation of semantics.
---
## 7. Non-Goals
ASL/ENCRYPTED-BLOCKS/1 does not define:
* Key exchange or key discovery
* Encryption algorithm choices
* Transport or replication protocols
* Storage layout or block packing rules

192
tier1/asl-federation-1.md Normal file
View file

@ -0,0 +1,192 @@
# ASL/FEDERATION/1 -- Core Federation Semantics
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-01-17
Tags: [federation, domains, visibility, determinism]
**Document ID:** `ASL/FEDERATION/1`
**Layer:** L2 -- Federation semantics (no transport, no encodings)
**Depends on (normative):**
* `ASL/1-CORE`
* `ASL/1-CORE-INDEX`
* `ASL/LOG/1`
* `ASL-STORE-INDEX`
**Informative references:**
* `ENC-ASL-CORE-INDEX` -- domain/visibility fields (`tier1/enc-asl-core-index.md`)
* `ASL/SYSTEM/1` -- unified system view
* `ASL/FEDERATION-REPLAY/1` -- cross-node deterministic replay
* `ASL/DAM/1` -- Domain Authority Manifest
* `ASL/POLICY-HASH/1` -- policy hash
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/FEDERATION/1 defines **semantic rules** for multi-domain visibility and cross-domain references. It does not define transport, replication, or encodings.
---
## 1. Purpose
ASL/FEDERATION/1 defines the **multi-domain model** for ASL-based systems:
* Domain isolation and visibility rules
* Published vs internal state
* Cross-domain reference constraints
* Snapshot identity and deterministic reconstruction
---
## 2. Core Concepts
| Term | Definition |
| -------------------- | ---------- |
| Domain | Logical namespace with its own ASL store, log, and snapshot lineage. |
| Internal state | Artifacts/snapshots visible only within the domain. |
| Published state | Artifacts/snapshots visible to other domains. |
| Federated snapshot | Snapshot with visibility state that may be imported by other domains. |
| Cross-domain reference | Reference to a published artifact from another domain. |
| Federation view | A deterministic view constructed by combining local internal state with admitted published state from other domains. |
---
## 3. Domain Semantics
1. **Domain isolation**
* Each domain has its own store, index, and log.
* Internal state is invisible outside the domain.
2. **Published state**
* Published artifacts and snapshots are visible to other domains.
* Published artifacts MUST satisfy ASL immutability and snapshot safety rules.
3. **Cross-domain references**
* Only published artifacts MAY be referenced by other domains.
* Cross-domain references are read-only and immutable.
* The consuming domain indexes imported artifacts using normal ASL index semantics.
* Imported entries MUST preserve origin metadata (domain identity and visibility) for deterministic replay.
---
## 4. Snapshot Identity
* Snapshot IDs are unique per domain.
* Federated snapshot identity is `(DomainID, SnapshotID)`.
* A federated snapshot MAY include cross-domain references only to published artifacts.
* Replay of federated state MUST be bounded by the source domain's `{SnapshotID, LogPrefix}`.
---
## 5. Visibility Rules
| Object | Internal Domain | Other Domains |
| ----------------------------------- | --------------- | ------------------- |
| Internal artifact | visible | hidden |
| Published artifact | visible | visible (read-only) |
| Internal snapshot | visible | hidden |
| Published snapshot | visible | visible |
| Block supporting published artifact | visible | visible |
| Block supporting internal artifact | visible | hidden |
* Index entries follow the same visibility rules.
* Determinism is defined per-domain and per-snapshot view.
---
## 6. Federation View Construction
To construct a federation view for a domain:
1. Start with the local domain's internal + published state at `{SnapshotID, LogPrefix}`.
2. For each admitted foreign domain, include only published state that is:
* Visible under that domain's `{SnapshotID, LogPrefix}`, and
* Allowed by the receiving domain's admission and policy rules.
3. Apply normal ASL index shadowing and tombstone rules within each domain's log order.
Federation MUST NOT bypass ASL/LOG/1 ordering or ASL/1-CORE-INDEX semantics.
---
## 7. Cross-Domain Operations
1. **Import published artifacts**
* A domain MAY import a published artifact from another domain.
* Imported artifacts MUST be treated as immutable.
* Import MUST be gated by admission and policy compatibility (see `ASL/DAP/1` and `ASL/POLICY-HASH/1`).
2. **Export published artifacts**
* Internal artifacts MAY be promoted to published state.
* Promotion MUST be snapshot-bound and log-ordered.
* Publication MUST respect the domain's policy hash and DAM roles.
3. **Federation log propagation**
* Each domain maintains its own append-only log.
* Federation MAY replicate published log-visible state.
---
## 8. Provenance and Traceability
* Execution receipts MAY include cross-domain inputs.
* Provenance MUST preserve origin domain and snapshot identity.
* Deterministic replay MUST be possible given `{Snapshot, LogPrefix}` for each domain.
---
## 9. Normative Invariants
1. **Determinism:** Reconstructing CURRENT in a domain yields the same visible state given the same snapshot and log prefix.
2. **Immutability:** Published artifacts and snapshots are immutable.
3. **Domain isolation:** Internal artifacts and snapshots are never visible to other domains.
4. **Federation safety:** Cross-domain references are read-only.
5. **Snapshot integrity:** Federated snapshots reference only published artifacts.
---
## 10. Integration with Other Layers
| Layer | Role in Federation |
| ------------------ | ------------------ |
| ASL/1-CORE | Artifact immutability and identity |
| ASL/1-CORE-INDEX | Semantic mapping and shadowing |
| ASL-STORE-INDEX | Sealing, retention, snapshot pinning |
| ASL/LOG/1 | Log-ordered visibility |
| ENC-ASL-CORE-INDEX | Domain/visibility metadata in records |
---
## 11. Non-Goals
* Transport protocols
* Network replication
* Witness signatures
* Domain admission and trust policy
Middle layer (informative): the daemon/service boundary around core logic that
owns network transport, admission workflows, and operational policy.
Implementation note (informative): core federation primitives live in
`include/amduat/fed/{registry,ingest,replay,view}.h`.
---
## 12. Summary
ASL/FEDERATION/1 defines the semantic rules for multi-domain visibility and cross-domain reference. It keeps federation deterministic, snapshot-safe, and read-only across domain boundaries.

View file

@ -0,0 +1,170 @@
# ASL/FEDERATION-REPLAY/1 -- Cross-Node Deterministic Replay
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-01-17
Tags: [federation, replay, determinism, tgk, pel]
**Document ID:** `ASL/FEDERATION-REPLAY/1`
**Layer:** L2 -- Federation replay semantics (no transport)
**Depends on (normative):**
* `ASL/FEDERATION/1`
* `ASL/LOG/1`
* `ASL/1-CORE-INDEX`
* `TGK/1`
**Informative references:**
* `ASL/SYSTEM/1` -- unified system view
* `ENC-ASL-CORE-INDEX` -- domain metadata
* `ASL/DAP/1` -- admission gating
* `ASL/POLICY-HASH/1` -- policy compatibility
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/FEDERATION-REPLAY/1 defines **deterministic replay rules** for federated propagation. It does not define network protocols or encodings.
---
## 1. Purpose
This document defines how artifacts, PERs, and TGK edges are propagated and replayed across federation nodes while preserving deterministic reconstruction.
---
## 2. Core Inputs
* **ArtifactKey**: canonical identifier for artifacts and PERs
* **SnapshotID**: snapshot boundary for replay
* **logseq**: append-only ordering within a domain
* **Canonical Edge Key**: TGK edge identity
---
## 3. Replay Record Requirements
Each propagated record MUST be replayable without external context. Records MUST carry:
* `domain_id` (source domain)
* `record_type` (artifact, PER, TGK edge, tombstone)
* `logseq` (source-domain ordering)
* `snapshot_id` and `log_prefix` bounds for which the record is visible
* Canonical identity (ArtifactKey or Canonical Edge Key)
* Visibility metadata (internal/published, cross-domain source when applicable)
Records MAY include optional integrity fields (hashes, signatures), but replay MUST
remain deterministic without them.
---
## 4. Propagation Rules
### 4.1 Artifacts and PERs
* Artifacts and PERs are propagated with their `ArtifactKey` and `logseq`.
* Only artifacts visible under a published snapshot MAY be propagated.
* Duplicate ArtifactKeys MUST be de-duplicated by identity.
* Imported entries MUST preserve origin metadata (domain identity and visibility).
### 4.2 TGK Edges
* TGK edges are propagated with their canonical edge identity and `logseq`.
* Edge propagation MUST preserve the same snapshot/log visibility constraints as artifacts.
---
## 5. Deterministic Replay Ordering
Replay order MUST be deterministic across nodes:
1. Sort by `logseq` ascending
2. Tie-break by canonical identity (ArtifactKey or Canonical Edge Key)
This ordering MUST be applied identically by all receivers.
---
## 6. Snapshot Bounds
* Replay MUST be bounded by `{SnapshotID, LogPrefix}`.
* Records with `logseq` greater than the replay prefix MUST be ignored.
* Replay MUST use the source domain's `{SnapshotID, LogPrefix}` as the bound for imported state.
---
## 7. Federation View Construction
Receivers construct a federation view by combining:
1. Local domain state at `{SnapshotID, LogPrefix}`.
2. Admitted foreign published state bounded by the source domain's `{SnapshotID, LogPrefix}`.
Admission and policy compatibility MUST be enforced before any foreign state is admitted.
---
## 8. Tombstones and Shadowing
* Tombstones MUST be replayed in log order and apply only within their source domain.
* A tombstone from domain A MUST NOT shadow artifacts from domain B.
* Shadowing is resolved per-domain using ASL/LOG/1 order and ASL/1-CORE-INDEX semantics.
---
## 9. Conflict Handling
1. **ArtifactKey collision**
* If bytes match existing artifact: discard duplicate.
* If bytes differ: reject and flag conflict.
2. **TGK edge conflicts**
* Multiple edges with the same canonical identity are resolved by log order and tombstone rules.
3. **PER conflicts**
* PERs with identical inputs and program identity but divergent outputs MUST be rejected.
---
## 10. Replay State and Idempotency
Replay MUST be idempotent:
* Re-applying the same record set MUST NOT change the resulting state.
* Receivers SHOULD track `{domain_id, logseq}` high-water marks per peer.
* Checkpointing MUST be aligned to `{SnapshotID, LogPrefix}` boundaries.
---
## 11. Provenance and Audit
Receivers SHOULD maintain:
* Last applied `logseq` per peer
* Snapshot provenance tables for applied records
This supports deterministic audit and replay verification.
---
## 12. Non-Goals
* Transport protocol selection
* Streaming formats
* Compression or batching
---
## 13. Summary
ASL/FEDERATION-REPLAY/1 defines a deterministic replay ordering and conflict rules to ensure federation is reproducible across nodes and snapshots.

296
tier1/asl-index-accel-1.md Normal file
View file

@ -0,0 +1,296 @@
# ASL/INDEX-ACCEL/1 — Index Acceleration Semantics
Status: Draft
Owner: Niklas Rydberg
Version: 0.1.0
SoT: No
Last Updated: 2025-11-16
Linked Phase Pack: N/A
Tags: [deterministic, index, acceleration]
<!-- Source: /amduat-api/tier1/asl-index-accel-1.md | Canonical: /amduat/tier1/asl-index-accel-1.md -->
**Document ID:** `ASL/INDEX-ACCEL/1`
**Layer:** L1 — Acceleration rules over index semantics (no storage / encoding)
**Depends on (normative):**
* `ASL/1-CORE-INDEX`
* `ASL/LOG/1`
**Informative references:**
* `ASL/STORE-INDEX/1` — store lifecycle and replay contracts
* `ENC/ASL-CORE-INDEX/1` — bytes-on-disk encoding profile
* `TGK/1` — TGK semantics and visibility alignment
* `TGK/1-CORE` — EdgeBody and EdgeTypeId definitions
© 2025 Niklas Rydberg.
## License
Except where otherwise noted, this document (text and diagrams) is licensed under
the Creative Commons Attribution 4.0 International License (CC BY 4.0).
The identifier registries and mapping tables (e.g. TypeTag IDs, HashId
assignments, EdgeTypeId tables) are additionally made available under CC0 1.0
Universal (CC0) to enable unrestricted reuse in implementations and derivative
specifications.
Code examples in this document are provided under the Apache License 2.0 unless
explicitly stated otherwise. Test vectors, where present, are dedicated to the
public domain under CC0 1.0.
---
## 0. Conventions
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
ASL/INDEX-ACCEL/1 defines **acceleration semantics only**. It MUST NOT change index meaning defined by ASL/1-CORE-INDEX.
---
## 1. Purpose
ASL/INDEX-ACCEL/1 defines **acceleration mechanisms** used by ASL-based indexes, including:
* Routing keys
* Sharding
* Filters (Bloom, XOR, Ribbon, etc.)
* SIMD execution
* Hash recasting
All mechanisms defined herein are **observationally invisible** to ASL/1-CORE-INDEX semantics.
---
## 2. Scope
Applies to:
* Artifact indexes (ASL)
* Projection and graph indexes (e.g., TGK)
* Any index layered on ASL/1-CORE-INDEX semantics
Does **not** define:
* Artifact or edge identity
* Snapshot semantics
* Storage lifecycle
* Encoding details
---
## 3. Canonical Key vs Routing Key
### 3.1 Canonical Key
The **Canonical Key** uniquely identifies an indexable entity.
Examples:
* Artifact: `Reference`
* TGK Edge: canonical key defined by `TGK/1` and `TGK/1-CORE` (opaque here)
Properties:
* Defines semantic identity
* Used for equality, shadowing, and tombstones
* Stable and immutable
* Fully compared on index match
### 3.2 Routing Key
The **Routing Key** is a **derived, advisory key** used exclusively for acceleration.
Properties:
* Derived deterministically from Canonical Key and optional attributes
* MAY be used for sharding, filters, SIMD layouts
* MUST NOT affect index semantics
* MUST be verified by full Canonical Key comparison on match
Formal rule:
```
CanonicalKey determines correctness
RoutingKey determines performance
```
---
## 4. Filter Semantics
### 4.1 Advisory Nature
All filters are **advisory only**.
Rules:
* False positives are permitted
* False negatives are forbidden
* Filter behavior MUST NOT affect correctness
Invariant:
```
Filter miss => key is definitely absent
Filter hit => key may be present
```
### 4.2 Filter Inputs
Filters operate over **Routing Keys**, not Canonical Keys.
A Routing Key MAY incorporate:
* Hash of Canonical Key
* Artifact type tag (if present)
* TGK `EdgeTypeId` or other immutable classification attributes (TGK/1-CORE)
* Direction, role, or other immutable classification attributes
Absence of optional attributes MUST be encoded explicitly.
### 4.3 Filter Construction
* Filters are built only over **sealed, immutable segments**
* Filters are immutable once built
* Filter construction MUST be deterministic
* Filter state MUST be covered by segment checksums
* Filters SHOULD be snapshot-scoped or versioned with their segment to avoid
unbounded false-positive accumulation over time
---
## 5. Sharding Semantics
### 5.1 Observational Invisibility
Sharding is a **mechanical partitioning** of the index.
Invariant:
```
LogicalIndex = union(all shards)
```
Rules:
* Shards MUST NOT affect lookup results
* Shard count and boundaries may change over time
* Rebalancing MUST preserve lookup semantics
### 5.2 Shard Assignment
Shard assignment MAY be based on:
* Hash of Canonical Key
* Routing Key
* Composite routing strategies
Shard selection MUST be deterministic per snapshot.
---
## 6. Hashing and Hash Recasting
### 6.1 Hashing
Hashes MAY be used for routing, filtering, or SIMD layout.
Hashes MUST NOT be treated as identity.
### 6.2 Hash Recasting
Hash recasting (changing hash functions or seeds) is permitted if:
1. It is deterministic
2. It does not change Canonical Keys
3. It does not affect index semantics
Recasting is equivalent to rebuilding acceleration structures.
---
## 7. SIMD Execution
SIMD operations MAY be used to:
* Evaluate filters
* Compare routing keys
* Accelerate scans
Rules:
* SIMD must operate only on immutable data
* SIMD must not short-circuit semantic checks
* SIMD must preserve deterministic behavior
---
## 8. Multi-Dimensional Routing Examples (Normative)
### 8.1 Artifact Index
* Canonical Key: `Reference`
* Routing Key components:
* `H(Reference)`
* `type_tag` (if present)
* `has_typetag`
### 8.2 TGK Edge Index
* Canonical Key: defined by `TGK/1` and `TGK/1-CORE` (opaque here)
* Routing Key components:
* `H(CanonicalEdgeKey)`
* `EdgeTypeId` (if present in the TGK profile)
* Direction or role (optional)
---
## 9. Snapshot Interaction
Acceleration structures:
* MUST respect snapshot visibility rules
* MUST operate over the same sealed segments visible to the snapshot
* MUST NOT bypass tombstones or shadowing
Snapshot cuts apply **after** routing and filtering.
---
## 10. Normative Invariants
1. Canonical Keys define identity and correctness
2. Routing Keys are advisory only
3. Filters may never introduce false negatives
4. Sharding is observationally invisible
5. Hashes are not identity
6. SIMD is an execution strategy, not a semantic construct
7. All acceleration is deterministic per snapshot
---
## 11. Non-Goals
ASL/INDEX-ACCEL/1 does not define:
* Specific filter algorithms
* Memory layout
* CPU instruction selection
* Encoding formats
* Federation policies
---
## 12. Summary
ASL/INDEX-ACCEL/1 establishes a strict contract:
> All acceleration exists to make the index faster, never different.
It formalizes Canonical vs Routing keys and constrains filters, sharding, hashing, and SIMD so that correctness is preserved under all optimizations.

Some files were not shown because too many files have changed in this diff Show more