Compare commits
57 commits
burgen-fel
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a433f92f13 | ||
|
|
88d9717513 | ||
|
|
624bd29bf9 | ||
|
|
6ab25361be | ||
|
|
8b2979e11d | ||
|
|
d0e53170f3 | ||
|
|
327812ca96 | ||
|
|
03d970c576 | ||
|
|
c7a9e2f6aa | ||
|
|
9a2903072b | ||
|
|
337466b073 | ||
|
|
d9122b53bb | ||
|
|
3e526975ce | ||
|
|
85c23e49eb | ||
|
|
e2d26e53cd | ||
|
|
c4571c3bfb | ||
|
|
4932fbd21c | ||
|
|
0f38165804 | ||
|
|
282d1eb959 | ||
|
|
d8a6603ddc | ||
|
|
5cc56b2ce8 | ||
|
|
ba78946a73 | ||
|
|
94c6c4be26 | ||
|
|
ed64c6ed89 | ||
|
|
2931e35c69 | ||
|
|
489738c5ba | ||
|
|
5a0a2f80c7 | ||
|
|
f707244888 | ||
|
|
948a156f5c | ||
|
|
8c5e593cec | ||
|
|
7b9c55ea6d | ||
|
|
3198d8ddbe | ||
|
|
1d04c32401 | ||
|
|
8c5fa71388 | ||
|
|
7a3dcc3978 | ||
|
|
4f59bc7e79 | ||
|
|
745cf89eb7 | ||
|
|
7878cd3702 | ||
|
|
dde532d68f | ||
|
|
0a118b9841 | ||
|
|
4d2fb250cd | ||
|
|
b506cc6c7c | ||
|
|
83cbe28ede | ||
|
|
017bc87e04 | ||
|
|
0667cee17d | ||
|
|
06a96f25db | ||
|
|
556c65a54e | ||
|
|
ac1ce381a2 | ||
|
|
07ef551f56 | ||
|
|
347d2e4dda | ||
|
|
68371b1554 | ||
|
|
b7b4b2f195 | ||
|
|
0d810affb0 | ||
|
|
a91ab91e39 | ||
|
|
20f092606d | ||
|
|
c595e2370a | ||
|
|
3886716799 |
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -3,6 +3,8 @@ CMakeFiles/
|
|||
CMakeCache.txt
|
||||
cmake_install.cmake
|
||||
Makefile
|
||||
CTestTestfile.cmake
|
||||
Testing/
|
||||
*.o
|
||||
*.a
|
||||
*.so
|
||||
|
|
@ -12,3 +14,5 @@ Makefile
|
|||
*.exe
|
||||
*.pdb
|
||||
.DS_Store
|
||||
tmp/
|
||||
.amduat-asl
|
||||
86
AUDITS.md
86
AUDITS.md
|
|
@ -22,6 +22,62 @@ Verification notes:
|
|||
- Prefer explicit commands and paths (e.g., `ctest --test-dir build`).
|
||||
- If results are user-reported, note that explicitly.
|
||||
|
||||
Note: the filesystem ASL store (`asl_store_fs`) is a legacy convenience backend
|
||||
and will be considered non-conformant to ASL index/log specs once the index/log
|
||||
store is introduced. Audits for ASL index/log specs target the new backend only.
|
||||
|
||||
## Test Expectations (Planned)
|
||||
|
||||
These tests are planned to validate index/log behavior once implemented:
|
||||
|
||||
| Area | Example tests |
|
||||
| --- | --- |
|
||||
| Segment encoding | Round-trip encode/decode; CRC mismatch rejection; offset bounds checks |
|
||||
| Log encoding | Hash-chain validation; unknown record type skip; truncated record rejection |
|
||||
| Replay | Snapshot anchor + log replay determinism; segment seal visibility |
|
||||
| Tombstones | Shadowing and lift across snapshots; domain-local shadowing rules |
|
||||
| Visibility | CURRENT computed by `(SnapshotID, LogPosition)`; reverse seal-log order |
|
||||
| Recovery | Crash with open segment; replay yields deterministic CURRENT |
|
||||
|
||||
## Spec Coverage (Implementation Status)
|
||||
|
||||
Status legend: ✅ implemented, 🟡 planned/in-progress, ⬜ not started.
|
||||
|
||||
| Spec | Status | Notes |
|
||||
| --- | --- | --- |
|
||||
| `ASL/1-CORE` | ✅ | Core artifact semantics implemented. |
|
||||
| `ASL/1-STORE` | ✅ | Store semantics + fs backend. |
|
||||
| `ENC/ASL1-CORE` | ✅ | Artifact/Reference encoding. |
|
||||
| `HASH/ASL1` | ✅ | Hash registry + streaming API. |
|
||||
| `PEL/1-CORE` | ✅ | Core execution semantics. |
|
||||
| `PEL/1-SURF` | ✅ | Store-backed surface execution. |
|
||||
| `PEL/PROGRAM-DAG/1` | ✅ | DAG scheme execution. |
|
||||
| `PEL/PROGRAM-DAG-DESC/1` | ✅ | Scheme descriptor codec + wiring. |
|
||||
| `ENC/PEL-PROGRAM-DAG/1` | ✅ | Program encoding. |
|
||||
| `ENC/PEL1-RESULT/1` | ✅ | Result encoding. |
|
||||
| `PEL/TRACE-DAG/1` | ✅ | Trace semantics + wiring. |
|
||||
| `ENC/PEL-TRACE-DAG/1` | ✅ | Trace encoding. |
|
||||
| `TGK/1-CORE` | ✅ | Edge semantics + validation. |
|
||||
| `ENC/TGK1-EDGE/1` | ✅ | Edge encoding. |
|
||||
| `TGK/STORE/1` | ✅ | Store semantics. |
|
||||
| `TGK/PROV/1` | ✅ | Provenance operators. |
|
||||
| `OPREG/PEL1-KERNEL` | ✅ | Kernel op registry. |
|
||||
| `OPREG/PEL1-KERNEL-PARAMS/1` | ✅ | Kernel params encoding. |
|
||||
| `AMDUAT20-STACK-OVERVIEW` | ✅ | Orientation surface aligned. |
|
||||
| `ASL/1-CORE-INDEX` | ✅ | Index semantics + replay implemented. |
|
||||
| `ASL/STORE-INDEX/1` | ✅ | Index/log store backend implemented (fs). |
|
||||
| `ENC/ASL-CORE-INDEX/1` | ✅ | Segment encoding/decoding implemented. |
|
||||
| `ASL/LOG/1` | ✅ | Log semantics implemented. |
|
||||
| `ENC/ASL-LOG/1` | ✅ | Log encoding/decoding implemented. |
|
||||
| `ASL/INDEX-ACCEL/1` | ✅ | Routing key + bloom/shard helpers implemented. |
|
||||
| `ASL/INDEXES/1` | 🟡 | Taxonomy planned. |
|
||||
| `ASL/TGK-EXEC-PLAN/1` | 🟡 | Encoding implemented; executor out of scope. |
|
||||
| `ENC/ASL-TGK-EXEC-PLAN/1` | ✅ | Plan encoding implemented. |
|
||||
| `ASL/FEDERATION/1` | ✅ | Core federation primitives implemented. |
|
||||
| `ASL/FEDERATION-REPLAY/1` | ✅ | Deterministic replay and view construction implemented. |
|
||||
| `ASL/SYSTEM/1` | 🟡 | Cross-cutting view planned. |
|
||||
| `TGK/1` | 🟡 | Semantic layer planned. |
|
||||
|
||||
## Audit Plan
|
||||
|
||||
Status legend: ✅ completed, ⬜ pending.
|
||||
|
|
@ -108,6 +164,24 @@ Status legend: ✅ completed, ⬜ pending.
|
|||
- Tests: command not provided — pass (user reported “100% tests passed, 0 tests
|
||||
failed out of 11”).
|
||||
|
||||
## 2026-01-18 — ASL index/log stack (`tier1/asl-core-index-1.md`, `tier1/asl-store-index-1.md`, `tier1/enc-asl-core-index-1.md`, `tier1/asl-log-1.md`, `tier1/enc-asl-log-1.md`)
|
||||
- Scope: index semantics, filesystem index/log store, segment encoding, and log encoding/decoding paths.
|
||||
- Findings: N/A (implemented components already present).
|
||||
- Resolution: recorded implementation status and aligned routing/shard helpers with index usage.
|
||||
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
|
||||
|
||||
## 2026-01-18 — ASL/INDEX-ACCEL/1 (`tier1/asl-index-accel-1.md`)
|
||||
- Scope: routing key derivation, shard selection contract, bloom advisory behavior.
|
||||
- Findings: missing formal routing-key API and tests around acceleration helpers.
|
||||
- Resolution: added routing-key/shard helpers and tests for routing-key layout, shard determinism, and bloom advisory behavior.
|
||||
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
|
||||
|
||||
## 2026-01-18 — ENC/ASL-TGK-EXEC-PLAN/1 (`tier1/enc-asl-tgk-exec-plan-1.md`)
|
||||
- Scope: execution plan encoding/decoding; validation of operator IDs/inputs.
|
||||
- Findings: encoding layer missing.
|
||||
- Resolution: implemented encode/decode/free API and round-trip validation tests.
|
||||
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
|
||||
|
||||
## 2025-12-22 — PEL/PROGRAM-DAG/1 (`tier1/pel-program-dag-1.md`)
|
||||
- Scope: Exec_DAG semantics, structural validity, canonical order, diagnostics,
|
||||
and scheme entrypoint correctness.
|
||||
|
|
@ -276,3 +350,15 @@ Status legend: ✅ completed, ⬜ pending.
|
|||
embedded commit-message appendix; tightened wording throughout; bumped the
|
||||
document version/date.
|
||||
- Tests: N/A (documentation-only change).
|
||||
|
||||
## 2026-02-XX — Federation core (`tier1/asl-federation-1.md`, `tier1/asl-federation-replay-1.md`)
|
||||
- Scope: core federation registry, ingest validation, deterministic replay, view
|
||||
construction, and resolve semantics.
|
||||
- Findings: missing record typing and identity coverage for PER/TGK/tombstones;
|
||||
policy gating needed explicit per-domain + optional per-record handling; view
|
||||
build and resolve error reporting needed explicit codes and tests.
|
||||
- Resolution: added federation registry storage, ingest validation, replay/view
|
||||
build, resolve APIs, and tests for ordering, tombstone scoping, conflicts,
|
||||
bounds, and metadata preservation; documented middle-layer boundary and
|
||||
ref-only remote fetch guidance.
|
||||
- Tests: user reported “100% tests passed, 0 tests failed out of 29”.
|
||||
|
|
|
|||
304
CMakeLists.txt
304
CMakeLists.txt
|
|
@ -57,12 +57,19 @@ set(AMDUAT_UTIL_SRCS
|
|||
src/internal/varint.c
|
||||
src/internal/endian.c
|
||||
src/internal/hex.c
|
||||
src/internal/log.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_SRCS
|
||||
src/kernel/asl/core.c
|
||||
src/near_core/asl/collection_view.c
|
||||
src/near_core/asl/none.c
|
||||
src/near_core/asl/artifact_io.c
|
||||
src/near_core/asl/io.c
|
||||
src/near_core/asl/index_accel.c
|
||||
src/near_core/asl/index_bloom.c
|
||||
src/near_core/asl/index_snapshot.c
|
||||
src/near_core/asl/index_replay.c
|
||||
src/near_core/asl/parse.c
|
||||
src/near_core/asl/ref_io.c
|
||||
src/near_core/asl/store.c
|
||||
|
|
@ -70,6 +77,18 @@ set(AMDUAT_ASL_SRCS
|
|||
src/near_core/asl/registry.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_LOG_STORE_SRCS
|
||||
src/core/asl_log_store.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_RECORD_SRCS
|
||||
src/core/asl_record.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_COLLECTION_SRCS
|
||||
src/core/asl_collection.c
|
||||
)
|
||||
|
||||
set(AMDUAT_HASH_ASL1_SRCS
|
||||
src/near_core/hash/asl1.c
|
||||
src/near_core/hash/sha256.c
|
||||
|
|
@ -81,6 +100,9 @@ set(AMDUAT_ENC_SRCS
|
|||
src/near_core/asl/ref_derive.c
|
||||
src/near_core/enc/fer1_receipt.c
|
||||
src/near_core/fer/receipt.c
|
||||
src/near_core/enc/asl_log.c
|
||||
src/near_core/enc/asl_core_index.c
|
||||
src/near_core/enc/asl_tgk_exec_plan.c
|
||||
src/near_core/enc/pel_program_dag.c
|
||||
src/near_core/enc/pel_program_dag_desc.c
|
||||
src/near_core/enc/pel_trace_dag.c
|
||||
|
|
@ -98,6 +120,7 @@ set(AMDUAT_FORMAT_SRCS
|
|||
|
||||
set(AMDUAT_PEL_SRCS
|
||||
src/kernel/pel/core.c
|
||||
src/core/derivation_sid.c
|
||||
src/pel_stack/decode.c
|
||||
src/pel_stack/surf/surf.c
|
||||
src/pel_stack/program_dag/program_dag.c
|
||||
|
|
@ -106,6 +129,7 @@ set(AMDUAT_PEL_SRCS
|
|||
src/pel_stack/trace_dag/trace_dag.c
|
||||
src/pel_stack/queue/queue.c
|
||||
src/pel_stack/opreg/kernel.c
|
||||
src/pel_stack/opreg/kernel_collection.c
|
||||
src/pel_stack/opreg/kernel_params.c
|
||||
)
|
||||
|
||||
|
|
@ -115,12 +139,36 @@ set(AMDUAT_TGK_SRCS
|
|||
src/tgk_stack/prov/prov.c
|
||||
)
|
||||
|
||||
set(AMDUAT_FED_SRCS
|
||||
src/near_core/fed/registry.c
|
||||
src/near_core/fed/replay.c
|
||||
src/near_core/fed/ingest.c
|
||||
src/near_core/fed/view.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_STORE_FS_SRCS
|
||||
src/adapters/asl_store_fs/asl_store_fs.c
|
||||
src/adapters/asl_store_fs/asl_store_fs_layout.c
|
||||
src/adapters/asl_store_fs/asl_store_fs_meta.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_STORE_INDEX_FS_SRCS
|
||||
src/adapters/asl_store_index_fs/asl_store_index_fs.c
|
||||
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_DERIVATION_INDEX_FS_SRCS
|
||||
src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_MATERIALIZATION_CACHE_FS_SRCS
|
||||
src/adapters/asl_materialization_cache_fs/asl_materialization_cache_fs.c
|
||||
)
|
||||
|
||||
set(AMDUAT_ASL_POINTER_FS_SRCS
|
||||
src/adapters/asl_pointer_fs/asl_pointer_fs.c
|
||||
)
|
||||
|
||||
set(AMDUAT_TGK_STORE_MEM_SRCS
|
||||
src/adapters/tgk_store_mem/tgk_store_mem.c
|
||||
)
|
||||
|
|
@ -129,6 +177,10 @@ set(AMDUAT_TGK_STORE_FS_SRCS
|
|||
src/adapters/tgk_store_fs/tgk_store_fs.c
|
||||
)
|
||||
|
||||
set(AMDUAT_TGK_STORE_ASL_INDEX_FS_SRCS
|
||||
src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c
|
||||
)
|
||||
|
||||
amduat_add_lib(util SRCS ${AMDUAT_UTIL_SRCS})
|
||||
|
||||
amduat_add_lib(asl SRCS ${AMDUAT_ASL_SRCS})
|
||||
|
|
@ -144,28 +196,64 @@ amduat_add_lib(format SRCS ${AMDUAT_FORMAT_SRCS})
|
|||
amduat_link(format amduat_asl amduat_enc amduat_util)
|
||||
|
||||
amduat_add_lib(pel SRCS ${AMDUAT_PEL_SRCS})
|
||||
amduat_link(pel amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
amduat_link(pel amduat_asl_materialization_cache_fs amduat_asl_collection
|
||||
amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
|
||||
amduat_add_lib(tgk SRCS ${AMDUAT_TGK_SRCS})
|
||||
amduat_link(tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
|
||||
amduat_add_lib(fed SRCS ${AMDUAT_FED_SRCS})
|
||||
amduat_link(fed amduat_asl)
|
||||
|
||||
amduat_add_lib(asl_store_fs SRCS ${AMDUAT_ASL_STORE_FS_SRCS})
|
||||
amduat_link(asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
target_compile_definitions(amduat_asl_store_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
|
||||
|
||||
amduat_add_lib(asl_store_index_fs SRCS ${AMDUAT_ASL_STORE_INDEX_FS_SRCS})
|
||||
amduat_link(asl_store_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
target_compile_definitions(amduat_asl_store_index_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
|
||||
|
||||
amduat_add_lib(asl_derivation_index_fs SRCS ${AMDUAT_ASL_DERIVATION_INDEX_FS_SRCS})
|
||||
amduat_link(asl_derivation_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
target_compile_definitions(amduat_asl_derivation_index_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
|
||||
|
||||
amduat_add_lib(asl_materialization_cache_fs SRCS ${AMDUAT_ASL_MATERIALIZATION_CACHE_FS_SRCS})
|
||||
amduat_link(asl_materialization_cache_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
target_compile_definitions(amduat_asl_materialization_cache_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
|
||||
|
||||
amduat_add_lib(asl_pointer_fs SRCS ${AMDUAT_ASL_POINTER_FS_SRCS})
|
||||
amduat_link(asl_pointer_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
target_compile_definitions(amduat_asl_pointer_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
|
||||
|
||||
amduat_add_lib(asl_log_store SRCS ${AMDUAT_ASL_LOG_STORE_SRCS})
|
||||
amduat_link(asl_log_store amduat_asl_pointer_fs amduat_asl amduat_enc amduat_util)
|
||||
|
||||
amduat_add_lib(asl_record SRCS ${AMDUAT_ASL_RECORD_SRCS})
|
||||
amduat_link(asl_record amduat_asl amduat_enc amduat_util)
|
||||
|
||||
amduat_add_lib(asl_collection SRCS ${AMDUAT_ASL_COLLECTION_SRCS})
|
||||
amduat_link(asl_collection amduat_asl_log_store amduat_asl_record amduat_asl_pointer_fs
|
||||
amduat_asl amduat_enc amduat_util)
|
||||
amduat_add_lib(tgk_store_mem SRCS ${AMDUAT_TGK_STORE_MEM_SRCS})
|
||||
amduat_link(tgk_store_mem amduat_tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
|
||||
amduat_add_lib(tgk_store_fs SRCS ${AMDUAT_TGK_STORE_FS_SRCS})
|
||||
amduat_link(tgk_store_fs amduat_tgk_store_mem amduat_tgk amduat_asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
|
||||
amduat_add_lib(tgk_store_asl_index_fs SRCS ${AMDUAT_TGK_STORE_ASL_INDEX_FS_SRCS})
|
||||
target_include_directories(amduat_tgk_store_asl_index_fs_obj
|
||||
PRIVATE ${AMDUAT_ROOT}/src/adapters/asl_store_index_fs
|
||||
)
|
||||
amduat_link(tgk_store_asl_index_fs amduat_tgk amduat_asl_store_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
|
||||
|
||||
add_executable(amduat_asl_cli src/tools/amduat_asl_cli.c)
|
||||
target_include_directories(amduat_asl_cli
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_asl_cli
|
||||
PRIVATE amduat_format amduat_asl_store_fs amduat_asl amduat_enc
|
||||
PRIVATE amduat_format amduat_asl_store_fs amduat_asl_store_index_fs
|
||||
amduat_asl_derivation_index_fs amduat_asl amduat_enc
|
||||
amduat_hash_asl1 amduat_util
|
||||
)
|
||||
set_target_properties(amduat_asl_cli PROPERTIES OUTPUT_NAME amduat-asl)
|
||||
|
|
@ -198,11 +286,24 @@ target_include_directories(amduat_pel_cli
|
|||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_pel_cli
|
||||
PRIVATE amduat_format amduat_pel amduat_asl_store_fs amduat_asl amduat_enc
|
||||
PRIVATE amduat_format amduat_pel amduat_asl_store_fs
|
||||
amduat_asl_collection amduat_asl_record amduat_asl_log_store
|
||||
amduat_asl_pointer_fs
|
||||
amduat_asl_derivation_index_fs amduat_asl amduat_enc
|
||||
amduat_hash_asl1 amduat_util
|
||||
)
|
||||
set_target_properties(amduat_pel_cli PROPERTIES OUTPUT_NAME amduat-pel)
|
||||
|
||||
add_executable(amduat_pel_build src/tools/amduat_pel_build.c)
|
||||
target_include_directories(amduat_pel_build
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_pel_build
|
||||
PRIVATE amduat_pel amduat_asl_store_fs amduat_asl amduat_enc amduat_util
|
||||
)
|
||||
set_target_properties(amduat_pel_build PROPERTIES OUTPUT_NAME amduat-pel-build)
|
||||
|
||||
enable_testing()
|
||||
|
||||
add_executable(amduat_test_pel_program_dag tests/enc/test_pel_program_dag.c)
|
||||
|
|
@ -225,6 +326,19 @@ target_link_libraries(amduat_test_pel_trace_dag
|
|||
)
|
||||
add_test(NAME pel_trace_dag COMMAND amduat_test_pel_trace_dag)
|
||||
|
||||
add_executable(amduat_test_pel_program_build_concat
|
||||
tests/pel/test_pel_program_build_concat.c
|
||||
)
|
||||
target_include_directories(amduat_test_pel_program_build_concat
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_pel_program_build_concat
|
||||
PRIVATE amduat_pel amduat_asl_store_fs amduat_asl amduat_enc
|
||||
amduat_hash_asl1 amduat_util
|
||||
)
|
||||
add_test(NAME pel_program_build_concat COMMAND amduat_test_pel_program_build_concat)
|
||||
|
||||
add_executable(amduat_test_pel1_result tests/enc/test_pel1_result.c)
|
||||
target_include_directories(amduat_test_pel1_result
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
|
|
@ -247,6 +361,37 @@ target_link_libraries(amduat_test_pel1_result_invariants
|
|||
add_test(NAME pel1_result_invariants
|
||||
COMMAND amduat_test_pel1_result_invariants)
|
||||
|
||||
add_executable(amduat_test_asl_log tests/enc/test_asl_log.c)
|
||||
target_include_directories(amduat_test_asl_log
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_log
|
||||
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
|
||||
)
|
||||
add_test(NAME asl_log COMMAND amduat_test_asl_log)
|
||||
|
||||
add_executable(amduat_test_asl_core_index tests/enc/test_asl_core_index.c)
|
||||
target_include_directories(amduat_test_asl_core_index
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_core_index
|
||||
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
|
||||
)
|
||||
add_test(NAME asl_core_index COMMAND amduat_test_asl_core_index)
|
||||
|
||||
add_executable(amduat_test_asl_tgk_exec_plan
|
||||
tests/enc/test_asl_tgk_exec_plan.c)
|
||||
target_include_directories(amduat_test_asl_tgk_exec_plan
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_tgk_exec_plan
|
||||
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
|
||||
)
|
||||
add_test(NAME asl_tgk_exec_plan COMMAND amduat_test_asl_tgk_exec_plan)
|
||||
|
||||
add_executable(amduat_test_tgk1_edge tests/enc/test_tgk1_edge.c)
|
||||
target_include_directories(amduat_test_tgk1_edge
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
|
|
@ -310,6 +455,115 @@ target_link_libraries(amduat_test_asl_ref_derive
|
|||
)
|
||||
add_test(NAME asl_ref_derive COMMAND amduat_test_asl_ref_derive)
|
||||
|
||||
add_executable(amduat_test_asl_store_indexed_ops
|
||||
tests/asl/test_asl_store_indexed_ops.c)
|
||||
target_include_directories(amduat_test_asl_store_indexed_ops
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_store_indexed_ops
|
||||
PRIVATE amduat_asl
|
||||
)
|
||||
add_test(NAME asl_store_indexed_ops COMMAND amduat_test_asl_store_indexed_ops)
|
||||
|
||||
add_executable(amduat_test_asl_replay tests/asl/test_asl_replay.c)
|
||||
target_include_directories(amduat_test_asl_replay
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_replay
|
||||
PRIVATE amduat_asl
|
||||
)
|
||||
add_test(NAME asl_replay COMMAND amduat_test_asl_replay)
|
||||
|
||||
add_executable(amduat_test_asl_index_replay
|
||||
tests/asl/test_asl_index_replay.c)
|
||||
target_include_directories(amduat_test_asl_index_replay
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
PRIVATE ${AMDUAT_ROOT}/src/adapters/asl_store_index_fs
|
||||
)
|
||||
target_compile_definitions(amduat_test_asl_index_replay
|
||||
PRIVATE _POSIX_C_SOURCE=200809L
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_index_replay
|
||||
PRIVATE amduat_asl_store_index_fs
|
||||
)
|
||||
add_test(NAME asl_index_replay COMMAND amduat_test_asl_index_replay)
|
||||
|
||||
add_executable(amduat_test_asl_store_index_fs
|
||||
tests/asl/test_asl_store_index_fs.c)
|
||||
target_include_directories(amduat_test_asl_store_index_fs
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_compile_definitions(amduat_test_asl_store_index_fs
|
||||
PRIVATE _POSIX_C_SOURCE=200809L
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_store_index_fs
|
||||
PRIVATE amduat_asl_store_index_fs amduat_format pthread
|
||||
)
|
||||
add_test(NAME asl_store_index_fs COMMAND amduat_test_asl_store_index_fs)
|
||||
set_tests_properties(asl_store_index_fs PROPERTIES
|
||||
ENVIRONMENT "AMDUAT_ASL_PERF_COUNT=100;AMDUAT_ASL_STRESS_SECS=5"
|
||||
TIMEOUT 120
|
||||
)
|
||||
|
||||
add_executable(amduat_test_asl_log_store_index_fs
|
||||
tests/asl/test_asl_log_store_index_fs.c)
|
||||
target_include_directories(amduat_test_asl_log_store_index_fs
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_compile_definitions(amduat_test_asl_log_store_index_fs
|
||||
PRIVATE _POSIX_C_SOURCE=200809L
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_log_store_index_fs
|
||||
PRIVATE amduat_asl_log_store amduat_asl_store_index_fs amduat_format pthread
|
||||
)
|
||||
add_test(NAME asl_log_store_index_fs COMMAND amduat_test_asl_log_store_index_fs)
|
||||
set_tests_properties(asl_log_store_index_fs PROPERTIES TIMEOUT 30)
|
||||
|
||||
add_executable(amduat_test_asl_index_put_get_consistency
|
||||
tests/asl/test_asl_index_put_get_consistency.c)
|
||||
target_include_directories(amduat_test_asl_index_put_get_consistency
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_compile_definitions(amduat_test_asl_index_put_get_consistency
|
||||
PRIVATE _POSIX_C_SOURCE=200809L
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_index_put_get_consistency
|
||||
PRIVATE amduat_asl_store_index_fs amduat_format pthread
|
||||
)
|
||||
add_test(NAME asl_index_put_get_consistency
|
||||
COMMAND amduat_test_asl_index_put_get_consistency)
|
||||
|
||||
add_executable(amduat_test_asl_index_accel tests/asl/test_asl_index_accel.c)
|
||||
target_include_directories(amduat_test_asl_index_accel
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_index_accel
|
||||
PRIVATE amduat_asl amduat_hash_asl1 amduat_util
|
||||
)
|
||||
add_test(NAME asl_index_accel COMMAND amduat_test_asl_index_accel)
|
||||
|
||||
add_executable(amduat_test_asl_derivation_index_fs
|
||||
tests/asl/test_asl_derivation_index_fs.c)
|
||||
target_include_directories(amduat_test_asl_derivation_index_fs
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_compile_definitions(amduat_test_asl_derivation_index_fs
|
||||
PRIVATE _POSIX_C_SOURCE=200809L
|
||||
)
|
||||
target_link_libraries(amduat_test_asl_derivation_index_fs
|
||||
PRIVATE amduat_asl_derivation_index_fs
|
||||
)
|
||||
add_test(NAME asl_derivation_index_fs
|
||||
COMMAND amduat_test_asl_derivation_index_fs)
|
||||
|
||||
add_executable(amduat_test_pel_program_dag_exec
|
||||
tests/pel/test_pel_program_dag_exec.c)
|
||||
target_include_directories(amduat_test_pel_program_dag_exec
|
||||
|
|
@ -317,7 +571,7 @@ target_include_directories(amduat_test_pel_program_dag_exec
|
|||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_pel_program_dag_exec
|
||||
PRIVATE amduat_pel
|
||||
PRIVATE amduat_pel amduat_asl_store_fs
|
||||
)
|
||||
add_test(NAME pel_program_dag_exec COMMAND amduat_test_pel_program_dag_exec)
|
||||
|
||||
|
|
@ -349,7 +603,7 @@ target_include_directories(amduat_test_pel_surf_run
|
|||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_pel_surf_run
|
||||
PRIVATE amduat_pel
|
||||
PRIVATE amduat_pel amduat_asl_store_fs
|
||||
)
|
||||
add_test(NAME pel_surf_run COMMAND amduat_test_pel_surf_run)
|
||||
|
||||
|
|
@ -365,3 +619,43 @@ target_link_libraries(amduat_test_pel_queue
|
|||
PRIVATE amduat_pel
|
||||
)
|
||||
add_test(NAME pel_queue COMMAND amduat_test_pel_queue)
|
||||
|
||||
add_executable(amduat_test_fed_registry tests/fed/test_fed_registry.c)
|
||||
target_include_directories(amduat_test_fed_registry
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_fed_registry
|
||||
PRIVATE amduat_fed
|
||||
)
|
||||
add_test(NAME fed_registry COMMAND amduat_test_fed_registry)
|
||||
|
||||
add_executable(amduat_test_fed_replay tests/fed/test_fed_replay.c)
|
||||
target_include_directories(amduat_test_fed_replay
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_fed_replay
|
||||
PRIVATE amduat_fed
|
||||
)
|
||||
add_test(NAME fed_replay COMMAND amduat_test_fed_replay)
|
||||
|
||||
add_executable(amduat_test_fed_ingest tests/fed/test_fed_ingest.c)
|
||||
target_include_directories(amduat_test_fed_ingest
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_fed_ingest
|
||||
PRIVATE amduat_fed
|
||||
)
|
||||
add_test(NAME fed_ingest COMMAND amduat_test_fed_ingest)
|
||||
|
||||
add_executable(amduat_test_fed_view tests/fed/test_fed_view.c)
|
||||
target_include_directories(amduat_test_fed_view
|
||||
PRIVATE ${AMDUAT_INTERNAL_DIR}
|
||||
PRIVATE ${AMDUAT_INCLUDE_DIR}
|
||||
)
|
||||
target_link_libraries(amduat_test_fed_view
|
||||
PRIVATE amduat_fed
|
||||
)
|
||||
add_test(NAME fed_view COMMAND amduat_test_fed_view)
|
||||
|
|
|
|||
14
README.md
14
README.md
|
|
@ -65,6 +65,20 @@ status and refs are printed to stderr.
|
|||
when not using `--output-raw`.
|
||||
- The filesystem ASL store layout expects digests at least 2 bytes long
|
||||
(two directory levels). Experimental shorter digests need a different store.
|
||||
- The filesystem ASL store (`amduat-asl ... --root`) is a legacy convenience
|
||||
backend; once the index/log store is introduced it is considered
|
||||
non-conformant to ASL index/log specs and should be used only for quickstart
|
||||
demos.
|
||||
- Compatibility & migration: existing `asl_store_fs` stores will not be
|
||||
automatically upgraded. Plan to re-ingest artifacts into the index/log store
|
||||
when it lands.
|
||||
|
||||
## Documentation
|
||||
|
||||
- Implementation clarifications: `docs/spec-clarifications.md`
|
||||
- Spec coverage matrix: `AUDITS.md` (Spec Coverage section)
|
||||
- Index/log API sketch: `docs/index-log-api-sketch.md`
|
||||
- Federation core API: `include/amduat/fed/` (registry, ingest, replay, view)
|
||||
|
||||
## PEL reference
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +0,0 @@
|
|||
Start testing: Dec 21 23:21 CET
|
||||
----------------------------------------------------------
|
||||
End testing: Dec 21 23:21 CET
|
||||
295
docs/federation-implementation-notes.md
Normal file
295
docs/federation-implementation-notes.md
Normal file
|
|
@ -0,0 +1,295 @@
|
|||
# Federation Implementation Notes (Core)
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-02-XX
|
||||
|
||||
## Purpose
|
||||
|
||||
These notes bind the federation semantics docs to concrete core-library
|
||||
responsibilities, APIs, and data structures. The intent is to keep
|
||||
federation logic inside the core substrate and keep daemon/frontends thin.
|
||||
|
||||
## Normative inputs
|
||||
|
||||
Core semantics and replay:
|
||||
- tier1/asl-federation-1.md
|
||||
- tier1/asl-federation-replay-1.md
|
||||
- tier1/asl-store-index-1.md
|
||||
- tier1/enc-asl-core-index-1.md
|
||||
|
||||
Admission and policy gating:
|
||||
- tier1/asl-dap-1.md
|
||||
- tier1/asl-policy-hash-1.md
|
||||
- tier1/asl-domain-model-1.md
|
||||
- tier1/asl-dam-1.md
|
||||
|
||||
Contextual alignment:
|
||||
- tier1/asl-system-1.md
|
||||
- tier1/asl-encrypted-blocks-1.md
|
||||
|
||||
## Scope (core library)
|
||||
|
||||
Federation MUST be implemented as core substrate logic:
|
||||
- Deterministic federation view construction.
|
||||
- Replay ordering and bounds per domain.
|
||||
- Imported record metadata preservation (domain_id, visibility,
|
||||
cross_domain_source).
|
||||
- Tombstone and shadowing behavior per domain.
|
||||
|
||||
The following are explicitly out of scope for core:
|
||||
- Transport protocols (HTTP, IPC, gossip).
|
||||
- Peer discovery and operational orchestration.
|
||||
- Admin UX and deployment wiring.
|
||||
- Admission workflows, auth, and retries/backoff.
|
||||
- Cache policy knobs (fetch timing, eviction, prefetch).
|
||||
- Operational concerns (metrics, admin endpoints).
|
||||
- Policy evaluation and per-record filtering decisions.
|
||||
|
||||
## Layering note
|
||||
|
||||
Core provides deterministic federation semantics and view construction only.
|
||||
Middle-layer components are responsible for transport, admission workflows,
|
||||
policy evaluation (including per-record filtering), caching strategies, and
|
||||
operational wiring.
|
||||
|
||||
Definition:
|
||||
- Middle layer: the daemon/service boundary around core logic that owns
|
||||
network transport, admission workflows, and operational policy.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
1) Federation registry
|
||||
- Tracks known domains, admission status, policy hash, and
|
||||
last admitted {SnapshotID, LogPrefix}.
|
||||
- Enforces DAP + policy compatibility prior to admitting remote state.
|
||||
|
||||
2) Federation view cache
|
||||
- Materializes a deterministic view from local state + admitted
|
||||
published records from other domains.
|
||||
- Stores imported records with origin metadata for replay.
|
||||
- Tracks per-domain replay high-water {domain_id, logseq}.
|
||||
|
||||
3) Resolver
|
||||
- Resolves ArtifactKey using local store + federation view.
|
||||
- Does not mutate store/index as part of GET semantics.
|
||||
|
||||
## Data model (suggested)
|
||||
|
||||
```c
|
||||
typedef enum {
|
||||
AMDUAT_FED_REC_ARTIFACT = 0,
|
||||
AMDUAT_FED_REC_PER = 1,
|
||||
AMDUAT_FED_REC_TGK_EDGE = 2,
|
||||
AMDUAT_FED_REC_TOMBSTONE = 3
|
||||
} amduat_fed_record_type_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t domain_id;
|
||||
uint64_t snapshot_id;
|
||||
uint64_t log_prefix;
|
||||
uint64_t last_logseq;
|
||||
uint8_t admitted; // boolean
|
||||
uint8_t policy_ok; // boolean
|
||||
uint8_t reserved[6];
|
||||
amduat_hash_id_t policy_hash_id;
|
||||
amduat_octets_t policy_hash;
|
||||
} amduat_fed_domain_state_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t domain_id;
|
||||
uint8_t visibility; // 0 internal, 1 published
|
||||
uint8_t has_source; // 0/1
|
||||
uint16_t reserved0;
|
||||
uint32_t source_domain;
|
||||
} amduat_fed_record_meta_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_type_t type;
|
||||
union {
|
||||
amduat_asl_artifact_key_t artifact_key;
|
||||
amduat_asl_tgk_edge_key_t tgk_edge_key;
|
||||
amduat_asl_per_key_t per_key;
|
||||
amduat_asl_artifact_key_t tombstone_key; // key being removed
|
||||
} id;
|
||||
} amduat_fed_record_id_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_meta_t meta;
|
||||
amduat_fed_record_id_t id;
|
||||
amduat_asl_artifact_location_t loc;
|
||||
uint64_t logseq;
|
||||
uint64_t snapshot_id;
|
||||
uint64_t log_prefix;
|
||||
} amduat_fed_index_record_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_id_t id;
|
||||
uint32_t reason_code; // policy-specific; 0 if unknown
|
||||
} amduat_fed_policy_deny_t;
|
||||
```
|
||||
|
||||
Notes:
|
||||
- Imported records MUST retain domain_id and cross-domain source metadata.
|
||||
- Tombstones must retain domain_id/visibility for domain-local shadowing.
|
||||
- Each record MUST include a record type and canonical identity for deterministic
|
||||
replay across artifacts, PERs, TGK edges, and tombstones.
|
||||
- PER/TGK canonical identities are currently represented by ASL references
|
||||
(artifact IDs); no separate edge/PER key types exist yet.
|
||||
|
||||
## Core API sketch
|
||||
|
||||
```c
|
||||
typedef struct amduat_fed_registry_t amduat_fed_registry_t;
|
||||
typedef struct amduat_fed_view_t amduat_fed_view_t;
|
||||
|
||||
amduat_fed_registry_t *amduat_fed_registry_open(...);
|
||||
void amduat_fed_registry_close(amduat_fed_registry_t *);
|
||||
|
||||
// Admission + policy gating
|
||||
bool amduat_fed_admit_domain(amduat_fed_registry_t *, domain_id, policy_hash, ...);
|
||||
bool amduat_fed_set_domain_state(amduat_fed_registry_t *, domain_id,
|
||||
snapshot_id, log_prefix);
|
||||
|
||||
// Ingest published records for a domain (already transported).
|
||||
// Each record MUST include its type and canonical identity in the id field.
|
||||
bool amduat_fed_ingest_records(amduat_fed_registry_t *, domain_id,
|
||||
const amduat_fed_index_record_t *records,
|
||||
size_t count);
|
||||
|
||||
// Build or refresh a deterministic federation view.
|
||||
amduat_fed_view_t *amduat_fed_view_build(amduat_fed_registry_t *,
|
||||
const amduat_asl_store_t *local_store,
|
||||
const amduat_asl_index_state_t *local_state);
|
||||
|
||||
// Resolve via local store + federated view.
|
||||
amduat_asl_store_error_t amduat_fed_resolve(
|
||||
const amduat_fed_view_t *view,
|
||||
const amduat_asl_artifact_key_t *key,
|
||||
amduat_bytes_t *out_bytes);
|
||||
```
|
||||
|
||||
Notes:
|
||||
- Transport fetch is not part of resolve; it only consumes ingested records.
|
||||
- The daemon can choose to fetch missing bytes when resolve reports a remote
|
||||
reference but local bytes are absent.
|
||||
- If per-record filtering is enabled, it is applied during ingest or view build
|
||||
and any denials are recorded in view metadata (e.g., a table of
|
||||
amduat_fed_policy_deny_t).
|
||||
|
||||
## Replay and view construction
|
||||
|
||||
Rules are as per ASL/FEDERATION-REPLAY/1:
|
||||
- Records are ordered by (logseq asc, canonical identity tie-break).
|
||||
- Replay is bounded by {SnapshotID, LogPrefix} per domain.
|
||||
- Tombstones shadow only within their source domain.
|
||||
- Imported entries keep domain_id + cross_domain_source.
|
||||
|
||||
The view is the union of:
|
||||
1) Local domain internal + published state at local {SnapshotID, LogPrefix}.
|
||||
2) Admitted foreign published state at each domain's {SnapshotID, LogPrefix}.
|
||||
|
||||
## Cache and storage
|
||||
|
||||
Federation view storage MAY be:
|
||||
- In-memory (ephemeral), or
|
||||
- On-disk index segments with federation metadata populated.
|
||||
|
||||
If remote bytes are fetched, they MUST be written to a cache store that is
|
||||
logically separate from the authoritative local store (policy-controlled).
|
||||
|
||||
## Federation API overview
|
||||
|
||||
Core federation primitives are exposed via:
|
||||
- include/amduat/fed/registry.h
|
||||
- include/amduat/fed/ingest.h
|
||||
- include/amduat/fed/replay.h
|
||||
- include/amduat/fed/view.h
|
||||
|
||||
## Integration example (core flow)
|
||||
|
||||
```c
|
||||
amduat_fed_registry_store_t reg_store;
|
||||
amduat_fed_registry_value_t reg_value;
|
||||
amduat_fed_domain_state_t reg_states[8];
|
||||
|
||||
amduat_fed_registry_value_init(®_value, reg_states, 8);
|
||||
amduat_fed_registry_store_init(®_store, local_store);
|
||||
amduat_fed_registry_store_put(®_store, ®_value, ®_ref, NULL);
|
||||
|
||||
if (amduat_fed_ingest_validate(records, record_count, NULL, NULL) !=
|
||||
AMDUAT_FED_INGEST_OK) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_fed_view_build(records,
|
||||
record_count,
|
||||
local_domain_id,
|
||||
bounds,
|
||||
bounds_len,
|
||||
denies,
|
||||
denies_len,
|
||||
&view)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
resolve_err = amduat_fed_resolve(&view, local_store, ref, &artifact);
|
||||
```
|
||||
|
||||
## Policy gating
|
||||
|
||||
- Admission gating is per-domain.
|
||||
- Per-record filtering is optional and MUST be an explicit, deterministic
|
||||
policy layer if enabled.
|
||||
|
||||
## Error reporting
|
||||
|
||||
Core resolve should distinguish:
|
||||
- NOT_FOUND (no record in local or federation view)
|
||||
- FOUND_REMOTE_NO_BYTES (record exists in view but bytes missing locally)
|
||||
- INTEGRITY_ERROR (hash mismatch on bytes)
|
||||
- POLICY_DENIED (domain admitted but record filtered by policy)
|
||||
|
||||
Notes:
|
||||
- When per-record filtering is enabled, POLICY_DENIED SHOULD surface at ingest
|
||||
or view-build time by excluding filtered records from the view and recording
|
||||
the denial in view metadata. Resolve MAY return POLICY_DENIED only when such a
|
||||
denial is recorded for the queried key; otherwise it MUST return NOT_FOUND.
|
||||
|
||||
## Remote fetch guidance
|
||||
|
||||
Core resolve returns only artifact references. The middle layer is responsible
|
||||
for fetching remote bytes by reference over its transport and caching them
|
||||
separately from the authoritative local store.
|
||||
|
||||
## Ownership and lifecycle
|
||||
|
||||
- `amduat_fed_registry_decode` allocates policy hash buffers; free with
|
||||
`amduat_fed_registry_value_free`.
|
||||
- `amduat_fed_registry_encode` returns allocated bytes; free with
|
||||
`amduat_octets_free`.
|
||||
- `amduat_fed_view_build` allocates view records and denies; free with
|
||||
`amduat_fed_view_free`.
|
||||
- `amduat_fed_replay_domain` allocates replay records; free with
|
||||
`amduat_fed_replay_view_free`.
|
||||
|
||||
## Middle-layer fetch example (informative)
|
||||
|
||||
```c
|
||||
err = amduat_fed_resolve(view, local_store, ref, &artifact);
|
||||
if (err == AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES) {
|
||||
if (fetch_remote_bytes(ref, &artifact) && cache_store_put(artifact)) {
|
||||
err = amduat_fed_resolve(view, local_store, ref, &artifact);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Tests (minimal)
|
||||
|
||||
1) Replay ordering determinism across two domains with interleaved logseq.
|
||||
2) Tombstone shadowing is domain-local.
|
||||
3) Imported record metadata preserved in view and survives rebuild.
|
||||
4) Conflict: same ArtifactKey with different bytes across domains rejected.
|
||||
5) Bound replay by {SnapshotID, LogPrefix} produces stable view.
|
||||
58
docs/index-log-api-sketch.md
Normal file
58
docs/index-log-api-sketch.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
# Index/Log API Surface (Sketch)
|
||||
|
||||
This document is a one-page sketch of the planned public API for ASL index/log
|
||||
support. It is non-normative and intended to guide header design.
|
||||
|
||||
## ASL Index/Log Types (Draft)
|
||||
|
||||
```
|
||||
typedef uint64_t amduat_asl_snapshot_id_t;
|
||||
typedef uint64_t amduat_asl_log_position_t; // inclusive logseq upper bound
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_snapshot_id_t snapshot_id;
|
||||
amduat_asl_log_position_t log_position;
|
||||
} amduat_asl_index_state_t;
|
||||
```
|
||||
|
||||
## Core Store API (Draft)
|
||||
|
||||
```
|
||||
// Initialization and config.
|
||||
bool amduat_asl_store_index_init(...);
|
||||
|
||||
// PUT/GET with index state reporting.
|
||||
amduat_asl_store_error_t amduat_asl_store_put_indexed(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_artifact_t artifact,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_get_indexed(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_index_state_t state,
|
||||
amduat_artifact_t *out_artifact);
|
||||
```
|
||||
|
||||
## Index/Log Introspection (Draft)
|
||||
|
||||
```
|
||||
// Snapshot/log position queries.
|
||||
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
// Segment and log inspection (read-only).
|
||||
bool amduat_asl_log_scan(amduat_asl_store_t *store, ...);
|
||||
bool amduat_asl_segment_scan(amduat_asl_store_t *store, ...);
|
||||
```
|
||||
|
||||
## Expected Error Surfaces
|
||||
|
||||
* `AMDUAT_ASL_STORE_ERR_INTEGRITY` for malformed index segments or log records.
|
||||
* `AMDUAT_ASL_STORE_ERR_IO` for underlying I/O faults.
|
||||
* `AMDUAT_ASL_STORE_ERR_NOT_FOUND` for absent artifacts or missing segments.
|
||||
* `AMDUAT_ASL_STORE_ERR_UNSUPPORTED` for unsupported encoding versions.
|
||||
|
||||
These are illustrative; exact error codes and mapping will be finalized when
|
||||
headers are introduced.
|
||||
201
docs/spec-clarifications.md
Normal file
201
docs/spec-clarifications.md
Normal file
|
|
@ -0,0 +1,201 @@
|
|||
# Spec Clarifications
|
||||
|
||||
This document records implementation-level clarifications for draft Tier-1
|
||||
specs. These notes do not change the specs; they document concrete choices for
|
||||
the implementation in this repository.
|
||||
|
||||
## Glossary and Abbreviations
|
||||
|
||||
| Term | Meaning |
|
||||
| --- | --- |
|
||||
| CURRENT | Effective index state after replaying a log position on a snapshot. |
|
||||
| LogPosition | Inclusive `logseq` upper bound for replay (not a byte offset). |
|
||||
| SnapshotID | Opaque `uint64_t` identifier persisted in `SNAPSHOT_ANCHOR`. |
|
||||
| Segment seal | Log record admitting a segment via `(segment_id, segment_hash)`. |
|
||||
| Segment hash | SHA-256 over exact on-disk segment bytes, including footer. |
|
||||
| Tombstone | Visibility policy record applied during replay. |
|
||||
| Tombstone lift | Cancels a specific tombstone record for the same artifact. |
|
||||
| Exec plan | Serialized plan format; executor out of scope for core library. |
|
||||
|
||||
## Snapshot and Log Identity (ASL/STORE-INDEX + ASL/LOG)
|
||||
|
||||
Decision:
|
||||
- LogPosition is the log sequence number (`logseq`), not a byte offset.
|
||||
- SnapshotID is an opaque store-assigned `uint64_t`, persisted in the
|
||||
`SNAPSHOT_ANCHOR` payload.
|
||||
|
||||
Implications:
|
||||
- `IndexState = (SnapshotID, LogPosition)` uses an inclusive logseq upper bound
|
||||
when replaying `log[0:LogPosition]`.
|
||||
- The log's record envelope already carries `logseq`, so snapshot anchors use
|
||||
the anchor record's `logseq` as the snapshot log position.
|
||||
- If no snapshot exists, treat SnapshotID as `0` and LogPosition as `0`.
|
||||
|
||||
Rationale:
|
||||
- `ASL/LOG/1` defines replay and visibility in terms of `logseq` ordering.
|
||||
- `ASL/TGK-EXEC-PLAN/1` orders results by `logseq` and uses `log_prefix` bounds.
|
||||
- `ASL/STORE-INDEX/1` defines LogPosition as a monotonic integer position and
|
||||
replay as `log[0:LogPosition]`, which maps directly to logseq.
|
||||
|
||||
References:
|
||||
- `tier1/asl-log-1.md`
|
||||
- `tier1/enc-asl-log-1.md`
|
||||
- `tier1/asl-store-index-1.md`
|
||||
- `tier1/asl-tgk-execution-plan-1.md`
|
||||
- `tier1/enc-asl-tgk-exec-plan-1.md`
|
||||
|
||||
## Index Segment Identity and Seals (ASL/STORE-INDEX + ASL/LOG)
|
||||
|
||||
Decision:
|
||||
- `segment_id` is a store-local, monotonic `uint64_t` assigned when a segment is
|
||||
created (before writing records), and persisted by naming/metadata outside the
|
||||
segment file.
|
||||
- `segment_hash` is SHA-256 over the exact segment file bytes as stored on disk,
|
||||
including header, records, digest bytes, extents, and footer.
|
||||
|
||||
Implications:
|
||||
- The seal record (`SEGMENT_SEAL`) binds a specific persisted segment file to the
|
||||
log via `(segment_id, segment_hash)`. Hashing occurs after the footer is
|
||||
written so the hash commits to seal metadata (CRC, seal snapshot, timestamp).
|
||||
- Replay uses `segment_id` to locate the segment file and verifies
|
||||
`segment_hash` before admitting it as visible.
|
||||
|
||||
Rationale:
|
||||
- `ENC/ASL-LOG/1` defines the seal payload as a segment ID plus a hash of the
|
||||
segment bytes; the log is the visibility gate, so the hash must cover the
|
||||
complete on-disk segment.
|
||||
- `ENC/ASL-CORE-INDEX/1` does not embed a segment ID, so the ID must be an
|
||||
external, store-managed handle (filename or catalog entry).
|
||||
|
||||
References:
|
||||
- `tier1/asl-log-1.md`
|
||||
- `tier1/enc-asl-log-1.md`
|
||||
- `tier1/asl-store-index-1.md`
|
||||
- `tier1/enc-asl-core-index-1.md`
|
||||
|
||||
## Tombstone Semantics (ASL/LOG + ASL/STORE-INDEX)
|
||||
|
||||
Decision:
|
||||
- `scope` and `reason_code` are opaque metadata and do not affect shadowing.
|
||||
- A `TOMBSTONE_LIFT` cancels only the referenced tombstone record for the same
|
||||
artifact; other tombstones for that artifact remain effective.
|
||||
|
||||
Across snapshots:
|
||||
- Snapshots capture the effective tombstone state as of the snapshot's `logseq`.
|
||||
- Lifts recorded after a snapshot become effective only when replay reaches
|
||||
their `logseq`.
|
||||
|
||||
References:
|
||||
- `tier1/asl-log-1.md`
|
||||
- `tier1/asl-store-index-1.md`
|
||||
|
||||
## Federation Fields (ENC/ASL-CORE-INDEX)
|
||||
|
||||
Decision:
|
||||
- Version 3 encoders must always emit federation fields in both headers and
|
||||
records. They are required, not optional, in v3.
|
||||
- Decoders accept legacy versions that omit federation fields and apply default
|
||||
local/internal values as defined in the encoding spec.
|
||||
|
||||
References:
|
||||
- `tier1/enc-asl-core-index-1.md`
|
||||
|
||||
## Execution Plan Scope (ASL/TGK-EXEC-PLAN + ENC/ASL-TGK-EXEC-PLAN)
|
||||
|
||||
Decision:
|
||||
- The implementation treats execution plans as a serialized/transport artifact
|
||||
and semantic contract only. A plan executor is out of scope for the core
|
||||
library.
|
||||
|
||||
References:
|
||||
- `tier1/asl-tgk-execution-plan-1.md`
|
||||
- `tier1/enc-asl-tgk-exec-plan-1.md`
|
||||
|
||||
## Publish/Unpublish Scope (ASL/LOG + ASL/SYSTEM)
|
||||
|
||||
Decision:
|
||||
- `ARTIFACT_PUBLISH` and `ARTIFACT_UNPUBLISH` are treated as reserved record
|
||||
types in the core replay path and do not alter ASL index state.
|
||||
- Publishing is modeled as moving artifacts and index segments between stores,
|
||||
advancing the destination store's snapshot/log.
|
||||
|
||||
Implications:
|
||||
- Core replay ignores publish/unpublish records.
|
||||
- Any visibility policy tied to publishing is handled by higher-level tooling
|
||||
or system-layer orchestration, not ASL/1 core semantics.
|
||||
|
||||
References:
|
||||
- `tier1/asl-log-1.md`
|
||||
- `tier1/asl-system-1.md`
|
||||
|
||||
## Receipt Output Reference Fallback (FER/1 + PEL/1)
|
||||
|
||||
Decision:
|
||||
- When a PEL run produces no output artifacts (e.g. failed execution), the
|
||||
receipt's `output_ref` falls back to the stored PEL result artifact reference.
|
||||
|
||||
Implications:
|
||||
- Receipts can be emitted for both successful and failed runs using a single
|
||||
canonical output reference.
|
||||
- Callers using `amduat_fer1_receipt_from_pel_run` should expect `output_ref`
|
||||
to match `result_ref` when `output_refs_len == 0`.
|
||||
|
||||
References:
|
||||
- `tier1/enc-fer1-receipt-1.md`
|
||||
- `tier1/srs.md`
|
||||
|
||||
## FER/1 v1.1 Determinism and Validation (FER/1 + SRS)
|
||||
|
||||
Decision:
|
||||
- `run_id` is a deterministic hash over stable inputs only and MUST exclude
|
||||
timestamps, logs, or mutable metadata.
|
||||
- Typed logs are optional; if present they MUST be ordered and size-bounded.
|
||||
- Limits are a single required record when the `limits` TLV is present.
|
||||
- Executor set verification is strict when a policy-provided set exists.
|
||||
|
||||
Concrete rules:
|
||||
- `run_id = H("AMDUAT:RUN\0" || EncRef(function) || EncRef(input_manifest) ||
|
||||
EncRef(environment) || EncRef(executor_fingerprint))`, where `EncRef` is
|
||||
`ENC/ASL1-CORE` canonical bytes and `executor_fingerprint` is the canonical
|
||||
digest reference. No other fields are included.
|
||||
- `logs` (if present): order by `(kind, cid)` byte-lexicographically; cap to
|
||||
64 entries; cap total log payload references to 1 MiB aggregate of capsule
|
||||
bytes. Reject out-of-order or oversized sets.
|
||||
- `limits` (if present): exactly one TLV containing all numeric fields
|
||||
(`cpu_ms`, `wall_ms`, `max_rss_kib`, `io_reads`, `io_writes`) with fixed
|
||||
units. Reject missing or duplicate fields.
|
||||
- Executor set validation:
|
||||
- If an expected executor set is supplied by policy, receipt executor_refs
|
||||
MUST match it exactly (same members, byte-order, no extras).
|
||||
- Otherwise, validate strict ordering and uniqueness, and require
|
||||
`parity_len == executor_refs_len` with aligned ordering and `output_ref`
|
||||
equality for every parity entry.
|
||||
|
||||
References:
|
||||
- `tier1/srs.md`
|
||||
- `tier1/enc-fer1-receipt-1.md`
|
||||
|
||||
## FER/1 v1.1 Encoding Notes (Implementation)
|
||||
|
||||
Decision:
|
||||
- The v1.1 encoder appends a TLV extension block after the v1 base layout.
|
||||
- Unknown or duplicate TLV tags are rejected during decode.
|
||||
|
||||
TLV tags (implementation):
|
||||
- `0x0001` executor fingerprint reference (encoded reference bytes).
|
||||
- `0x0002` run id (`U32` length + bytes).
|
||||
- `0x0003` logs (`U32` count; per entry: `U32 kind`, encoded ref, `U32` sha256
|
||||
length + bytes). Entries must be ordered by `(kind, ref)` byte order.
|
||||
- `0x0004` limits (`U64` cpu_ms, `U64` wall_ms, `U64` max_rss_kib,
|
||||
`U64` io_reads, `U64` io_writes).
|
||||
- `0x0005` determinism (`U8` level, `U32` seed_len + seed bytes).
|
||||
- `0x0006` signature (opaque bytes).
|
||||
|
||||
Helper usage:
|
||||
- `amduat_fer1_receipt_from_pel_run_v1_1` emits v1.1 receipts and uses the
|
||||
same output_ref fallback as v1: when no outputs exist, `output_ref` is the
|
||||
stored PEL result reference.
|
||||
|
||||
References:
|
||||
- `include/amduat/enc/fer1_receipt.h`
|
||||
- `src/near_core/enc/fer1_receipt.c`
|
||||
59
include/amduat/asl/asl_derivation_index_fs.h
Normal file
59
include/amduat/asl/asl_derivation_index_fs.h
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
#ifndef AMDUAT_ASL_DERIVATION_INDEX_FS_H
|
||||
#define AMDUAT_ASL_DERIVATION_INDEX_FS_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { AMDUAT_ASL_DERIVATION_INDEX_FS_ROOT_MAX = 1024 };
|
||||
|
||||
typedef struct {
|
||||
amduat_octets_t sid;
|
||||
amduat_reference_t program_ref;
|
||||
uint32_t output_index;
|
||||
amduat_reference_t *input_refs;
|
||||
size_t input_refs_len;
|
||||
bool has_params_ref;
|
||||
amduat_reference_t params_ref;
|
||||
bool has_exec_profile;
|
||||
amduat_octets_t exec_profile;
|
||||
} amduat_asl_derivation_record_t;
|
||||
|
||||
typedef struct {
|
||||
char root_path[AMDUAT_ASL_DERIVATION_INDEX_FS_ROOT_MAX];
|
||||
} amduat_asl_derivation_index_fs_t;
|
||||
|
||||
bool amduat_asl_derivation_index_fs_init(
|
||||
amduat_asl_derivation_index_fs_t *index,
|
||||
const char *root_path);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_derivation_index_fs_add(
|
||||
amduat_asl_derivation_index_fs_t *index,
|
||||
amduat_reference_t artifact_ref,
|
||||
const amduat_asl_derivation_record_t *record);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_derivation_index_fs_list(
|
||||
amduat_asl_derivation_index_fs_t *index,
|
||||
amduat_reference_t artifact_ref,
|
||||
amduat_asl_derivation_record_t **out_records,
|
||||
size_t *out_count);
|
||||
|
||||
void amduat_asl_derivation_record_free(
|
||||
amduat_asl_derivation_record_t *record);
|
||||
|
||||
void amduat_asl_derivation_records_free(
|
||||
amduat_asl_derivation_record_t *records,
|
||||
size_t count);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_DERIVATION_INDEX_FS_H */
|
||||
40
include/amduat/asl/asl_materialization_cache_fs.h
Normal file
40
include/amduat/asl/asl_materialization_cache_fs.h
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
#ifndef AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H
|
||||
#define AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX = 1024 };
|
||||
|
||||
typedef struct {
|
||||
char root_path[AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX];
|
||||
} amduat_asl_materialization_cache_fs_t;
|
||||
|
||||
bool amduat_asl_materialization_cache_fs_init(
|
||||
amduat_asl_materialization_cache_fs_t *cache,
|
||||
const char *root_path);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_get(
|
||||
amduat_asl_materialization_cache_fs_t *cache,
|
||||
amduat_octets_t sid,
|
||||
amduat_reference_t **out_refs,
|
||||
size_t *out_refs_len);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_put(
|
||||
amduat_asl_materialization_cache_fs_t *cache,
|
||||
amduat_octets_t sid,
|
||||
const amduat_reference_t *refs,
|
||||
size_t refs_len);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H */
|
||||
58
include/amduat/asl/asl_pointer_fs.h
Normal file
58
include/amduat/asl/asl_pointer_fs.h
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
#ifndef AMDUAT_ASL_POINTER_FS_H
|
||||
#define AMDUAT_ASL_POINTER_FS_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_POINTER_FS_ROOT_MAX = 1024,
|
||||
AMDUAT_ASL_POINTER_NAME_MAX = 512
|
||||
};
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_ASL_POINTER_OK = 0,
|
||||
AMDUAT_ASL_POINTER_ERR_NOT_FOUND = 1,
|
||||
AMDUAT_ASL_POINTER_ERR_IO = 2,
|
||||
AMDUAT_ASL_POINTER_ERR_INVALID_NAME = 3,
|
||||
AMDUAT_ASL_POINTER_ERR_INTEGRITY = 4
|
||||
} amduat_asl_pointer_error_t;
|
||||
|
||||
typedef struct {
|
||||
char root_path[AMDUAT_ASL_POINTER_FS_ROOT_MAX];
|
||||
} amduat_asl_pointer_store_t;
|
||||
|
||||
/* Pointers are generic control-plane names for artifacts (e.g. dataset heads).
|
||||
* Example names:
|
||||
* - space/123/dataset/calendar_types/head
|
||||
* - space/123/collection/events/head
|
||||
*/
|
||||
bool amduat_asl_pointer_store_init(amduat_asl_pointer_store_t *ps,
|
||||
const char *root_path);
|
||||
|
||||
amduat_asl_pointer_error_t amduat_asl_pointer_get(
|
||||
const amduat_asl_pointer_store_t *ps,
|
||||
const char *name,
|
||||
bool *out_exists,
|
||||
amduat_reference_t *out_ref);
|
||||
|
||||
amduat_asl_pointer_error_t amduat_asl_pointer_cas(
|
||||
const amduat_asl_pointer_store_t *ps,
|
||||
const char *name,
|
||||
bool expected_exists,
|
||||
const amduat_reference_t *expected_ref,
|
||||
const amduat_reference_t *new_ref,
|
||||
bool *out_swapped);
|
||||
|
||||
bool amduat_asl_pointer_name_is_valid(const char *name);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_POINTER_FS_H */
|
||||
92
include/amduat/asl/asl_store_index_fs.h
Normal file
92
include/amduat/asl/asl_store_index_fs.h
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
#ifndef AMDUAT_ASL_STORE_INDEX_FS_H
|
||||
#define AMDUAT_ASL_STORE_INDEX_FS_H
|
||||
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <pthread.h>
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { AMDUAT_ASL_STORE_INDEX_FS_ROOT_MAX = 1024 };
|
||||
|
||||
typedef struct {
|
||||
bool enabled;
|
||||
uint64_t max_pending_bytes;
|
||||
uint64_t idle_time_ns;
|
||||
} amduat_asl_store_index_fs_snapshot_policy_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t max_segment_records;
|
||||
uint64_t max_segment_bytes;
|
||||
uint64_t small_artifact_threshold;
|
||||
bool allow_deferred_visibility;
|
||||
} amduat_asl_store_index_fs_segment_policy_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t segment_domain_id;
|
||||
uint8_t record_visibility;
|
||||
} amduat_asl_store_index_fs_visibility_policy_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_store_index_fs_snapshot_policy_t snapshot_policy;
|
||||
amduat_asl_store_index_fs_segment_policy_t segment_policy;
|
||||
amduat_asl_store_index_fs_visibility_policy_t visibility_policy;
|
||||
uint16_t shard_count;
|
||||
char root_path[AMDUAT_ASL_STORE_INDEX_FS_ROOT_MAX];
|
||||
uint64_t pending_snapshot_bytes;
|
||||
uint64_t last_ingest_time_ns;
|
||||
amduat_asl_snapshot_id_t next_snapshot_id;
|
||||
bool snapshot_state_initialized;
|
||||
void *open_segments;
|
||||
pthread_mutex_t write_mutex;
|
||||
uint32_t write_depth;
|
||||
int write_lock_fd;
|
||||
} amduat_asl_store_index_fs_t;
|
||||
|
||||
bool amduat_asl_store_index_fs_init(amduat_asl_store_index_fs_t *fs,
|
||||
amduat_asl_store_config_t config,
|
||||
const char *root_path);
|
||||
|
||||
void amduat_asl_store_index_fs_set_snapshot_policy(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
amduat_asl_store_index_fs_snapshot_policy_t policy);
|
||||
|
||||
void amduat_asl_store_index_fs_set_segment_policy(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
amduat_asl_store_index_fs_segment_policy_t policy);
|
||||
|
||||
void amduat_asl_store_index_fs_set_visibility_policy(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
amduat_asl_store_index_fs_visibility_policy_t policy);
|
||||
|
||||
void amduat_asl_store_index_fs_set_shard_count(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
uint16_t shard_count);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_index_fs_snapshot_create(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
amduat_asl_snapshot_id_t snapshot_id,
|
||||
amduat_asl_log_position_t *out_logseq,
|
||||
uint8_t out_root_hash[32]);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_index_fs_flush(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_index_fs_gc(
|
||||
amduat_asl_store_index_fs_t *fs,
|
||||
const amduat_asl_index_state_t *state);
|
||||
|
||||
amduat_asl_store_ops_t amduat_asl_store_index_fs_ops(void);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_STORE_INDEX_FS_H */
|
||||
81
include/amduat/asl/collection.h
Normal file
81
include/amduat/asl/collection.h
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
#ifndef AMDUAT_ASL_COLLECTION_H
|
||||
#define AMDUAT_ASL_COLLECTION_H
|
||||
|
||||
#include "amduat/asl/asl_pointer_fs.h"
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/log_store.h"
|
||||
#include "amduat/asl/record.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_ASL_COLLECTION_OK = 0,
|
||||
AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME = 1,
|
||||
AMDUAT_ASL_COLLECTION_ERR_IO = 2,
|
||||
AMDUAT_ASL_COLLECTION_ERR_INTEGRITY = 3,
|
||||
AMDUAT_ASL_COLLECTION_ERR_CAS_MISMATCH = 4
|
||||
} amduat_asl_collection_error_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_t *store;
|
||||
amduat_asl_log_store_t log_store;
|
||||
amduat_asl_pointer_store_t pointer_store;
|
||||
} amduat_asl_collection_store_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t snapshot_offset;
|
||||
amduat_reference_t *refs;
|
||||
size_t refs_len;
|
||||
} amduat_asl_collection_snapshot_payload_t;
|
||||
|
||||
bool amduat_asl_collection_store_init(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *root_path,
|
||||
amduat_asl_store_t *store);
|
||||
|
||||
amduat_asl_collection_error_t amduat_asl_collection_append(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *collection_name,
|
||||
amduat_reference_t record_ref,
|
||||
uint16_t kind,
|
||||
amduat_octets_t actor,
|
||||
uint64_t *out_offset);
|
||||
|
||||
amduat_asl_collection_error_t amduat_asl_collection_snapshot(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *collection_name,
|
||||
uint64_t up_to_offset,
|
||||
amduat_reference_t *out_snapshot_ref,
|
||||
bool *out_swapped);
|
||||
|
||||
amduat_asl_collection_error_t amduat_asl_collection_read(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *collection_name,
|
||||
uint64_t from_offset,
|
||||
size_t limit,
|
||||
amduat_reference_t **out_record_refs,
|
||||
size_t *out_len,
|
||||
uint64_t *out_next_offset,
|
||||
bool *out_end);
|
||||
|
||||
void amduat_asl_collection_refs_free(amduat_reference_t *refs,
|
||||
size_t refs_len);
|
||||
|
||||
bool amduat_asl_collection_snapshot_payload_decode_v1(
|
||||
amduat_octets_t payload,
|
||||
amduat_asl_collection_snapshot_payload_t *out_snapshot);
|
||||
|
||||
void amduat_asl_collection_snapshot_payload_free(
|
||||
amduat_asl_collection_snapshot_payload_t *snapshot);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_COLLECTION_H */
|
||||
74
include/amduat/asl/collection_view.h
Normal file
74
include/amduat/asl/collection_view.h
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
#ifndef AMDUAT_ASL_COLLECTION_VIEW_H
|
||||
#define AMDUAT_ASL_COLLECTION_VIEW_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { TYPE_TAG_ASL_COLLECTION_VIEW_1 = 0x00000403u };
|
||||
enum { AMDUAT_TYPE_TAG_ASL_COLLECTION_VIEW_1 = TYPE_TAG_ASL_COLLECTION_VIEW_1 };
|
||||
enum { TYPE_TAG_ASL_SNAPSHOT_INFO_1 = 0x00000405u };
|
||||
enum { AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1 = TYPE_TAG_ASL_SNAPSHOT_INFO_1 };
|
||||
enum { TYPE_TAG_ASL_LOG_RANGE_1 = 0x00000406u };
|
||||
enum { AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1 = TYPE_TAG_ASL_LOG_RANGE_1 };
|
||||
|
||||
typedef struct {
|
||||
uint64_t snapshot_at_offset;
|
||||
amduat_reference_t *refs;
|
||||
size_t refs_len;
|
||||
} amduat_asl_collection_snapshot_info_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t start_offset;
|
||||
uint64_t next_offset;
|
||||
amduat_reference_t *refs;
|
||||
size_t refs_len;
|
||||
} amduat_asl_log_range_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t computed_from_offset;
|
||||
uint64_t computed_up_to_offset;
|
||||
amduat_reference_t *refs;
|
||||
size_t refs_len;
|
||||
} amduat_asl_collection_view_t;
|
||||
|
||||
bool amduat_asl_collection_snapshot_info_encode_v1(
|
||||
const amduat_asl_collection_snapshot_info_t *info,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_asl_collection_snapshot_info_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_collection_snapshot_info_t *out_info);
|
||||
|
||||
void amduat_asl_collection_snapshot_info_free(
|
||||
amduat_asl_collection_snapshot_info_t *info);
|
||||
|
||||
bool amduat_asl_log_range_encode_v1(const amduat_asl_log_range_t *range,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_asl_log_range_decode_v1(amduat_octets_t bytes,
|
||||
amduat_asl_log_range_t *out_range);
|
||||
|
||||
void amduat_asl_log_range_free(amduat_asl_log_range_t *range);
|
||||
|
||||
bool amduat_asl_collection_view_encode_v1(
|
||||
const amduat_asl_collection_view_t *view,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_asl_collection_view_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_collection_view_t *out_view);
|
||||
|
||||
void amduat_asl_collection_view_free(amduat_asl_collection_view_t *view);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_COLLECTION_VIEW_H */
|
||||
33
include/amduat/asl/index_accel.h
Normal file
33
include/amduat/asl/index_accel.h
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
#ifndef AMDUAT_ASL_INDEX_ACCEL_H
|
||||
#define AMDUAT_ASL_INDEX_ACCEL_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
bool amduat_asl_index_accel_routing_key_from_ref(
|
||||
amduat_reference_t ref,
|
||||
bool has_type_tag,
|
||||
amduat_type_tag_t type_tag,
|
||||
amduat_octets_t *out_key);
|
||||
|
||||
bool amduat_asl_index_accel_routing_key_hash(amduat_octets_t key,
|
||||
uint64_t *out_hash);
|
||||
|
||||
uint16_t amduat_asl_index_accel_shard_for_ref(
|
||||
amduat_reference_t ref,
|
||||
bool has_type_tag,
|
||||
amduat_type_tag_t type_tag,
|
||||
uint16_t shard_count);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_INDEX_ACCEL_H */
|
||||
33
include/amduat/asl/index_bloom.h
Normal file
33
include/amduat/asl/index_bloom.h
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
#ifndef AMDUAT_ASL_INDEX_BLOOM_H
|
||||
#define AMDUAT_ASL_INDEX_BLOOM_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_INDEX_BLOOM_BYTES = 256,
|
||||
AMDUAT_ASL_INDEX_BLOOM_HASHES = 4
|
||||
};
|
||||
|
||||
bool amduat_asl_index_bloom_init(amduat_octets_t *out_bloom);
|
||||
|
||||
bool amduat_asl_index_bloom_add(amduat_octets_t bloom,
|
||||
amduat_hash_id_t hash_id,
|
||||
amduat_octets_t digest);
|
||||
|
||||
bool amduat_asl_index_bloom_maybe_contains(amduat_octets_t bloom,
|
||||
amduat_hash_id_t hash_id,
|
||||
amduat_octets_t digest);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_INDEX_BLOOM_H */
|
||||
47
include/amduat/asl/index_replay.h
Normal file
47
include/amduat/asl/index_replay.h
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
#ifndef AMDUAT_ASL_INDEX_REPLAY_H
|
||||
#define AMDUAT_ASL_INDEX_REPLAY_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/store.h"
|
||||
#include "amduat/enc/asl_log.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef struct {
|
||||
uint64_t segment_id;
|
||||
uint8_t segment_hash[32];
|
||||
} amduat_asl_segment_seal_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_reference_t ref;
|
||||
uint64_t tombstone_logseq;
|
||||
} amduat_asl_tombstone_entry_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_segment_seal_t *segments;
|
||||
size_t segments_len;
|
||||
amduat_asl_tombstone_entry_t *tombstones;
|
||||
size_t tombstones_len;
|
||||
amduat_asl_index_state_t state;
|
||||
} amduat_asl_replay_state_t;
|
||||
|
||||
bool amduat_asl_replay_init(amduat_asl_replay_state_t *out);
|
||||
void amduat_asl_replay_free(amduat_asl_replay_state_t *state);
|
||||
|
||||
bool amduat_asl_replay_apply_log(
|
||||
const amduat_asl_log_record_t *records,
|
||||
size_t record_count,
|
||||
amduat_asl_log_position_t log_position,
|
||||
amduat_asl_replay_state_t *state);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_INDEX_REPLAY_H */
|
||||
48
include/amduat/asl/index_snapshot.h
Normal file
48
include/amduat/asl/index_snapshot.h
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
#ifndef AMDUAT_ASL_INDEX_SNAPSHOT_H
|
||||
#define AMDUAT_ASL_INDEX_SNAPSHOT_H
|
||||
|
||||
#include "amduat/asl/index_replay.h"
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION = 1,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE = 80,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN = 8
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_snapshot_id_t snapshot_id;
|
||||
amduat_asl_log_position_t anchor_logseq;
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_segment_seal_t *segments;
|
||||
size_t segments_len;
|
||||
amduat_asl_tombstone_entry_t *tombstones;
|
||||
size_t tombstones_len;
|
||||
} amduat_asl_snapshot_manifest_t;
|
||||
|
||||
bool amduat_asl_snapshot_manifest_write(
|
||||
const char *path,
|
||||
const amduat_asl_snapshot_manifest_t *manifest,
|
||||
uint8_t out_hash[32]);
|
||||
|
||||
bool amduat_asl_snapshot_manifest_read(
|
||||
const char *path,
|
||||
amduat_asl_snapshot_manifest_t *out_manifest,
|
||||
uint8_t out_hash[32]);
|
||||
|
||||
void amduat_asl_snapshot_manifest_free(
|
||||
amduat_asl_snapshot_manifest_t *manifest);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_INDEX_SNAPSHOT_H */
|
||||
78
include/amduat/asl/log_store.h
Normal file
78
include/amduat/asl/log_store.h
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
#ifndef AMDUAT_ASL_LOG_STORE_H
|
||||
#define AMDUAT_ASL_LOG_STORE_H
|
||||
|
||||
#include "amduat/asl/asl_pointer_fs.h"
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { TYPE_TAG_ASL_LOG_CHUNK_1 = 0x00000401u };
|
||||
enum { AMDUAT_TYPE_TAG_ASL_LOG_CHUNK_1 = TYPE_TAG_ASL_LOG_CHUNK_1 };
|
||||
|
||||
typedef struct {
|
||||
uint16_t kind;
|
||||
bool has_timestamp;
|
||||
uint64_t timestamp;
|
||||
amduat_reference_t payload_ref;
|
||||
bool has_actor;
|
||||
amduat_octets_t actor;
|
||||
} amduat_asl_log_entry_t;
|
||||
|
||||
typedef struct {
|
||||
bool has_prev;
|
||||
amduat_reference_t prev_ref;
|
||||
uint64_t base_offset;
|
||||
uint32_t entry_count;
|
||||
bool has_timestamp;
|
||||
bool has_actor;
|
||||
amduat_asl_log_entry_t *entries;
|
||||
} amduat_asl_log_chunk_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_t *store;
|
||||
amduat_asl_pointer_store_t pointer_store;
|
||||
} amduat_asl_log_store_t;
|
||||
|
||||
bool amduat_asl_log_store_init(amduat_asl_log_store_t *log_store,
|
||||
const char *root_path,
|
||||
amduat_asl_store_t *store,
|
||||
const amduat_asl_pointer_store_t *pointer_store);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_log_append(
|
||||
amduat_asl_log_store_t *log_store,
|
||||
const char *log_name,
|
||||
const amduat_asl_log_entry_t *entries,
|
||||
size_t entries_len,
|
||||
uint64_t *out_first_offset);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_log_read(
|
||||
amduat_asl_log_store_t *log_store,
|
||||
const char *log_name,
|
||||
uint64_t from_offset,
|
||||
size_t max_entries,
|
||||
amduat_asl_log_entry_t **out_entries,
|
||||
size_t *out_len,
|
||||
uint64_t *out_next_offset,
|
||||
bool *out_end);
|
||||
|
||||
void amduat_asl_log_entries_free(amduat_asl_log_entry_t *entries,
|
||||
size_t entries_len);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_log_chunk_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_log_chunk_t *out_chunk);
|
||||
|
||||
void amduat_asl_log_chunk_free(amduat_asl_log_chunk_t *chunk);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_LOG_STORE_H */
|
||||
27
include/amduat/asl/none.h
Normal file
27
include/amduat/asl/none.h
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
#ifndef AMDUAT_ASL_NONE_H
|
||||
#define AMDUAT_ASL_NONE_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { TYPE_TAG_ASL_NONE_1 = 0x00000404u };
|
||||
enum { AMDUAT_TYPE_TAG_ASL_NONE_1 = TYPE_TAG_ASL_NONE_1 };
|
||||
|
||||
bool amduat_asl_none_encode_v1(amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_asl_none_decode_v1(amduat_octets_t bytes);
|
||||
|
||||
bool amduat_asl_none_is_artifact(const amduat_artifact_t *artifact);
|
||||
|
||||
bool amduat_asl_none_artifact(amduat_artifact_t *out_artifact);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_NONE_H */
|
||||
46
include/amduat/asl/record.h
Normal file
46
include/amduat/asl/record.h
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
#ifndef AMDUAT_ASL_RECORD_H
|
||||
#define AMDUAT_ASL_RECORD_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { TYPE_TAG_ASL_RECORD_1 = 0x00000402u };
|
||||
enum { AMDUAT_TYPE_TAG_ASL_RECORD_1 = TYPE_TAG_ASL_RECORD_1 };
|
||||
|
||||
typedef struct {
|
||||
amduat_octets_t schema;
|
||||
amduat_octets_t payload;
|
||||
} amduat_asl_record_t;
|
||||
|
||||
bool amduat_asl_record_encode_v1(const amduat_asl_record_t *record,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_asl_record_decode_v1(amduat_octets_t bytes,
|
||||
amduat_asl_record_t *out_record);
|
||||
|
||||
void amduat_asl_record_free(amduat_asl_record_t *record);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_record_store_put(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_octets_t schema,
|
||||
amduat_octets_t payload,
|
||||
amduat_reference_t *out_ref);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_record_store_get(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_record_t *out_record);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_RECORD_H */
|
||||
|
|
@ -2,6 +2,7 @@
|
|||
#define AMDUAT_ASL_STORE_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/enc/asl_log.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
|
|
@ -25,6 +26,14 @@ typedef enum {
|
|||
AMDUAT_ASL_STORE_ERR_IO = 4
|
||||
} amduat_asl_store_error_t;
|
||||
|
||||
typedef uint64_t amduat_asl_snapshot_id_t;
|
||||
typedef uint64_t amduat_asl_log_position_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_snapshot_id_t snapshot_id;
|
||||
amduat_asl_log_position_t log_position;
|
||||
} amduat_asl_index_state_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_error_t (*put)(void *ctx,
|
||||
amduat_artifact_t artifact,
|
||||
|
|
@ -32,6 +41,29 @@ typedef struct {
|
|||
amduat_asl_store_error_t (*get)(void *ctx,
|
||||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact);
|
||||
amduat_asl_store_error_t (*put_indexed)(void *ctx,
|
||||
amduat_artifact_t artifact,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
amduat_asl_store_error_t (*get_indexed)(void *ctx,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_index_state_t state,
|
||||
amduat_artifact_t *out_artifact);
|
||||
amduat_asl_store_error_t (*tombstone)(void *ctx,
|
||||
amduat_reference_t ref,
|
||||
uint32_t scope,
|
||||
uint32_t reason_code,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
amduat_asl_store_error_t (*tombstone_lift)(
|
||||
void *ctx,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_log_position_t tombstone_logseq,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
amduat_asl_store_error_t (*log_scan)(
|
||||
void *ctx,
|
||||
amduat_asl_log_record_t **out_records,
|
||||
size_t *out_count);
|
||||
bool (*current_state)(void *ctx, amduat_asl_index_state_t *out_state);
|
||||
amduat_asl_store_error_t (*validate_config)(
|
||||
void *ctx,
|
||||
amduat_asl_store_config_t config);
|
||||
|
|
@ -43,6 +75,12 @@ static inline void amduat_asl_store_ops_init(amduat_asl_store_ops_t *ops) {
|
|||
}
|
||||
ops->put = NULL;
|
||||
ops->get = NULL;
|
||||
ops->put_indexed = NULL;
|
||||
ops->get_indexed = NULL;
|
||||
ops->tombstone = NULL;
|
||||
ops->tombstone_lift = NULL;
|
||||
ops->log_scan = NULL;
|
||||
ops->current_state = NULL;
|
||||
ops->validate_config = NULL;
|
||||
}
|
||||
|
||||
|
|
@ -65,6 +103,39 @@ amduat_asl_store_error_t amduat_asl_store_get(amduat_asl_store_t *store,
|
|||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_put_indexed(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_artifact_t artifact,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_get_indexed(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_index_state_t state,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_tombstone(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
uint32_t scope,
|
||||
uint32_t reason_code,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_tombstone_lift(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_log_position_t tombstone_logseq,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_log_scan(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_asl_log_record_t **out_records,
|
||||
size_t *out_count);
|
||||
|
||||
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
|
||||
amduat_asl_index_state_t *out_state);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
|
|
|||
101
include/amduat/enc/asl_core_index.h
Normal file
101
include/amduat/enc/asl_core_index.h
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
#ifndef AMDUAT_ENC_ASL_CORE_INDEX_H
|
||||
#define AMDUAT_ENC_ASL_CORE_INDEX_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_CORE_INDEX_VERSION = 3,
|
||||
AMDUAT_ASL_CORE_INDEX_HEADER_SIZE = 112,
|
||||
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE = 48,
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE = 16,
|
||||
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE = 24
|
||||
};
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_INDEX_FLAG_TOMBSTONE = 0x00000001u
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
uint64_t magic;
|
||||
uint16_t version;
|
||||
uint16_t shard_id;
|
||||
uint32_t header_size;
|
||||
uint64_t snapshot_min;
|
||||
uint64_t snapshot_max;
|
||||
uint64_t record_count;
|
||||
uint64_t records_offset;
|
||||
uint64_t bloom_offset;
|
||||
uint64_t bloom_size;
|
||||
uint64_t digests_offset;
|
||||
uint64_t digests_size;
|
||||
uint64_t extents_offset;
|
||||
uint64_t extent_count;
|
||||
uint32_t segment_domain_id;
|
||||
uint8_t segment_visibility;
|
||||
uint8_t federation_version;
|
||||
uint16_t reserved0;
|
||||
uint64_t flags;
|
||||
} amduat_asl_segment_header_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t hash_id;
|
||||
uint16_t digest_len;
|
||||
uint16_t reserved0;
|
||||
uint64_t digest_offset;
|
||||
uint64_t extents_offset;
|
||||
uint32_t extent_count;
|
||||
uint32_t total_length;
|
||||
uint32_t domain_id;
|
||||
uint8_t visibility;
|
||||
uint8_t has_cross_domain_source;
|
||||
uint16_t reserved1;
|
||||
uint32_t cross_domain_source;
|
||||
uint32_t flags;
|
||||
} amduat_asl_index_record_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t block_id;
|
||||
uint32_t offset;
|
||||
uint32_t length;
|
||||
} amduat_asl_extent_record_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t crc64;
|
||||
uint64_t seal_snapshot;
|
||||
uint64_t seal_time_ns;
|
||||
} amduat_asl_segment_footer_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_segment_header_t header;
|
||||
amduat_octets_t bloom;
|
||||
amduat_asl_index_record_t *records;
|
||||
size_t record_count;
|
||||
amduat_octets_t digests;
|
||||
amduat_asl_extent_record_t *extents;
|
||||
size_t extent_count;
|
||||
amduat_asl_segment_footer_t footer;
|
||||
} amduat_asl_core_index_segment_t;
|
||||
|
||||
bool amduat_enc_asl_core_index_encode_v1(
|
||||
const amduat_asl_core_index_segment_t *segment,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_enc_asl_core_index_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_core_index_segment_t *out_segment);
|
||||
|
||||
void amduat_enc_asl_core_index_free(amduat_asl_core_index_segment_t *segment);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ENC_ASL_CORE_INDEX_H */
|
||||
53
include/amduat/enc/asl_log.h
Normal file
53
include/amduat/enc/asl_log.h
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
#ifndef AMDUAT_ENC_ASL_LOG_H
|
||||
#define AMDUAT_ENC_ASL_LOG_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL = 0x01,
|
||||
AMDUAT_ASL_LOG_RECORD_TOMBSTONE = 0x10,
|
||||
AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT = 0x11,
|
||||
AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR = 0x20,
|
||||
AMDUAT_ASL_LOG_RECORD_ARTIFACT_PUBLISH = 0x30,
|
||||
AMDUAT_ASL_LOG_RECORD_ARTIFACT_UNPUBLISH = 0x31
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
uint64_t logseq;
|
||||
uint32_t record_type;
|
||||
amduat_octets_t payload;
|
||||
uint8_t record_hash[32];
|
||||
} amduat_asl_log_record_t;
|
||||
|
||||
bool amduat_enc_asl_log_encode_v1(const amduat_asl_log_record_t *records,
|
||||
size_t record_count,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_enc_asl_log_decode_v1(amduat_octets_t bytes,
|
||||
amduat_asl_log_record_t **out_records,
|
||||
size_t *out_count);
|
||||
|
||||
void amduat_enc_asl_log_free(amduat_asl_log_record_t *records,
|
||||
size_t record_count);
|
||||
|
||||
/* Caller owns out_ref digest; free with amduat_reference_free. */
|
||||
bool amduat_asl_log_decode_artifact_ref(amduat_octets_t payload,
|
||||
amduat_reference_t *out_ref);
|
||||
|
||||
/* Caller owns out_bytes; free with amduat_octets_free. */
|
||||
bool amduat_asl_log_encode_artifact_ref(amduat_reference_t ref,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ENC_ASL_LOG_H */
|
||||
115
include/amduat/enc/asl_tgk_exec_plan.h
Normal file
115
include/amduat/enc/asl_tgk_exec_plan.h
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
#ifndef AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H
|
||||
#define AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { AMDUAT_ASL_TGK_EXEC_PLAN_VERSION = 1 };
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN = 0,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_INDEX_FILTER = 1,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_MERGE = 2,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_PROJECTION = 3,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL = 4,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_AGGREGATION = 5,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_LIMIT_OFFSET = 6,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_SHARD_DISPATCH = 7,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_SIMD_FILTER = 8,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_TOMBSTONE_SHADOW = 9
|
||||
} amduat_asl_tgk_exec_operator_type_t;
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_ASL_TGK_EXEC_OP_FLAG_NONE = 0x00,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL = 0x01,
|
||||
AMDUAT_ASL_TGK_EXEC_OP_FLAG_OPTIONAL = 0x02
|
||||
} amduat_asl_tgk_exec_operator_flags_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t logseq_min;
|
||||
uint64_t logseq_max;
|
||||
} amduat_asl_tgk_exec_snapshot_range_t;
|
||||
|
||||
typedef struct {
|
||||
struct {
|
||||
uint8_t is_asl_segment;
|
||||
uint64_t segment_start_id;
|
||||
uint64_t segment_end_id;
|
||||
} segment_scan;
|
||||
|
||||
struct {
|
||||
uint32_t artifact_type_tag;
|
||||
uint8_t has_type_tag;
|
||||
uint32_t edge_type_key;
|
||||
uint8_t has_edge_type;
|
||||
uint8_t role;
|
||||
} index_filter;
|
||||
|
||||
struct {
|
||||
uint8_t deterministic;
|
||||
} merge;
|
||||
|
||||
struct {
|
||||
uint8_t project_artifact_id;
|
||||
uint8_t project_tgk_edge_id;
|
||||
uint8_t project_node_id;
|
||||
uint8_t project_type_tag;
|
||||
} projection;
|
||||
|
||||
struct {
|
||||
uint64_t start_node_id;
|
||||
uint32_t traversal_depth;
|
||||
uint8_t direction;
|
||||
} tgk_traversal;
|
||||
|
||||
struct {
|
||||
uint8_t agg_count;
|
||||
uint8_t agg_union;
|
||||
uint8_t agg_sum;
|
||||
} aggregation;
|
||||
|
||||
struct {
|
||||
uint64_t limit;
|
||||
uint64_t offset;
|
||||
} limit_offset;
|
||||
} amduat_asl_tgk_exec_operator_params_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t op_id;
|
||||
amduat_asl_tgk_exec_operator_type_t op_type;
|
||||
amduat_asl_tgk_exec_operator_flags_t flags;
|
||||
amduat_asl_tgk_exec_snapshot_range_t snapshot;
|
||||
amduat_asl_tgk_exec_operator_params_t params;
|
||||
uint32_t input_count;
|
||||
uint32_t inputs[8];
|
||||
} amduat_asl_tgk_exec_operator_def_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t plan_version;
|
||||
uint32_t operator_count;
|
||||
amduat_asl_tgk_exec_operator_def_t *operators;
|
||||
} amduat_asl_tgk_exec_plan_t;
|
||||
|
||||
bool amduat_enc_asl_tgk_exec_plan_encode_v1(
|
||||
const amduat_asl_tgk_exec_plan_t *plan,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
bool amduat_enc_asl_tgk_exec_plan_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_tgk_exec_plan_t *out_plan);
|
||||
|
||||
void amduat_enc_asl_tgk_exec_plan_free(
|
||||
amduat_asl_tgk_exec_plan_t *plan);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H */
|
||||
|
|
@ -11,6 +11,10 @@ extern "C" {
|
|||
#endif
|
||||
|
||||
enum { FER1_RECEIPT_ENC_V1 = 0x0301u };
|
||||
enum { FER1_RECEIPT_ENC_V1_1 = 0x0302u };
|
||||
|
||||
enum { AMDUAT_FER1_VERSION_1 = 0x0001u };
|
||||
enum { AMDUAT_FER1_VERSION_1_1 = 0x0101u };
|
||||
|
||||
enum { TYPE_TAG_FER1_RECEIPT_1 = 0x00000301u };
|
||||
enum { AMDUAT_TYPE_TAG_FER1_RECEIPT_1 = TYPE_TAG_FER1_RECEIPT_1 };
|
||||
|
|
@ -23,6 +27,20 @@ typedef struct {
|
|||
amduat_octets_t parity_digest;
|
||||
} amduat_fer1_parity_entry_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t kind;
|
||||
amduat_reference_t log_ref;
|
||||
amduat_octets_t sha256;
|
||||
} amduat_fer1_log_entry_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t cpu_ms;
|
||||
uint64_t wall_ms;
|
||||
uint64_t max_rss_kib;
|
||||
uint64_t io_reads;
|
||||
uint64_t io_writes;
|
||||
} amduat_fer1_limits_t;
|
||||
|
||||
typedef struct {
|
||||
uint16_t fer1_version;
|
||||
amduat_reference_t function_ref;
|
||||
|
|
@ -36,14 +54,34 @@ typedef struct {
|
|||
size_t parity_len;
|
||||
uint64_t started_at;
|
||||
uint64_t completed_at;
|
||||
bool has_executor_fingerprint_ref;
|
||||
amduat_reference_t executor_fingerprint_ref;
|
||||
bool has_run_id;
|
||||
amduat_octets_t run_id;
|
||||
bool has_limits;
|
||||
amduat_fer1_limits_t limits;
|
||||
amduat_fer1_log_entry_t *logs;
|
||||
size_t logs_len;
|
||||
bool has_determinism;
|
||||
uint8_t determinism_level;
|
||||
bool has_rng_seed;
|
||||
amduat_octets_t rng_seed;
|
||||
bool has_signature;
|
||||
amduat_octets_t signature;
|
||||
} amduat_fer1_receipt_t;
|
||||
|
||||
bool amduat_enc_fer1_receipt_encode_v1(
|
||||
const amduat_fer1_receipt_t *receipt,
|
||||
amduat_octets_t *out_bytes);
|
||||
bool amduat_enc_fer1_receipt_encode_v1_1(
|
||||
const amduat_fer1_receipt_t *receipt,
|
||||
amduat_octets_t *out_bytes);
|
||||
bool amduat_enc_fer1_receipt_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_fer1_receipt_t *out_receipt);
|
||||
bool amduat_enc_fer1_receipt_decode_v1_1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_fer1_receipt_t *out_receipt);
|
||||
void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt);
|
||||
|
||||
#ifdef __cplusplus
|
||||
|
|
|
|||
28
include/amduat/fed/ingest.h
Normal file
28
include/amduat/fed/ingest.h
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
#ifndef AMDUAT_FED_INGEST_H
|
||||
#define AMDUAT_FED_INGEST_H
|
||||
|
||||
#include "amduat/fed/replay.h"
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_FED_INGEST_OK = 0,
|
||||
AMDUAT_FED_INGEST_ERR_INVALID = 1,
|
||||
AMDUAT_FED_INGEST_ERR_CONFLICT = 2
|
||||
} amduat_fed_ingest_error_t;
|
||||
|
||||
amduat_fed_ingest_error_t amduat_fed_ingest_validate(
|
||||
const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
size_t *out_error_index,
|
||||
size_t *out_conflict_index);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_FED_INGEST_H */
|
||||
84
include/amduat/fed/registry.h
Normal file
84
include/amduat/fed/registry.h
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
#ifndef AMDUAT_FED_REGISTRY_H
|
||||
#define AMDUAT_FED_REGISTRY_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/asl/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef struct {
|
||||
uint32_t domain_id;
|
||||
uint64_t snapshot_id;
|
||||
uint64_t log_prefix;
|
||||
uint64_t last_logseq;
|
||||
uint8_t admitted;
|
||||
uint8_t policy_ok;
|
||||
uint8_t reserved[6];
|
||||
amduat_hash_id_t policy_hash_id;
|
||||
amduat_octets_t policy_hash; /* Empty when unknown; caller-owned bytes. */
|
||||
} amduat_fed_domain_state_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_domain_state_t *states;
|
||||
size_t len;
|
||||
size_t cap;
|
||||
bool owns_states;
|
||||
} amduat_fed_registry_value_t;
|
||||
|
||||
void amduat_fed_registry_value_init(amduat_fed_registry_value_t *value,
|
||||
amduat_fed_domain_state_t *states,
|
||||
size_t cap);
|
||||
|
||||
bool amduat_fed_registry_value_insert(amduat_fed_registry_value_t *value,
|
||||
amduat_fed_domain_state_t state);
|
||||
|
||||
const amduat_fed_domain_state_t *amduat_fed_registry_value_lookup(
|
||||
const amduat_fed_registry_value_t *value,
|
||||
uint32_t domain_id);
|
||||
|
||||
void amduat_fed_registry_value_free(amduat_fed_registry_value_t *value);
|
||||
|
||||
/* Encode/Decode allocate buffers; caller frees via amduat_octets_free. */
|
||||
bool amduat_fed_registry_encode(const amduat_fed_registry_value_t *value,
|
||||
amduat_octets_t *out_bytes);
|
||||
|
||||
/* Decode allocates policy_hash bytes; free via amduat_fed_registry_value_free. */
|
||||
bool amduat_fed_registry_decode(amduat_octets_t bytes,
|
||||
amduat_fed_registry_value_t *out_value);
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_FED_REGISTRY_OK = 0,
|
||||
AMDUAT_FED_REGISTRY_ERR_CODEC = 1,
|
||||
AMDUAT_FED_REGISTRY_ERR_STORE = 2
|
||||
} amduat_fed_registry_error_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_t *store;
|
||||
} amduat_fed_registry_store_t;
|
||||
|
||||
void amduat_fed_registry_store_init(amduat_fed_registry_store_t *reg,
|
||||
amduat_asl_store_t *store);
|
||||
|
||||
amduat_fed_registry_error_t amduat_fed_registry_store_put(
|
||||
amduat_fed_registry_store_t *reg,
|
||||
const amduat_fed_registry_value_t *value,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_store_error_t *out_store_err);
|
||||
|
||||
amduat_fed_registry_error_t amduat_fed_registry_store_get(
|
||||
amduat_fed_registry_store_t *reg,
|
||||
amduat_reference_t ref,
|
||||
amduat_fed_registry_value_t *out_value,
|
||||
amduat_asl_store_error_t *out_store_err);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_FED_REGISTRY_H */
|
||||
72
include/amduat/fed/replay.h
Normal file
72
include/amduat/fed/replay.h
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
#ifndef AMDUAT_FED_REPLAY_H
|
||||
#define AMDUAT_FED_REPLAY_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_FED_REC_ARTIFACT = 0,
|
||||
AMDUAT_FED_REC_PER = 1,
|
||||
AMDUAT_FED_REC_TGK_EDGE = 2,
|
||||
AMDUAT_FED_REC_TOMBSTONE = 3
|
||||
} amduat_fed_record_type_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t domain_id;
|
||||
uint8_t visibility;
|
||||
uint8_t has_source;
|
||||
uint16_t reserved0;
|
||||
uint32_t source_domain;
|
||||
} amduat_fed_record_meta_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_type_t type;
|
||||
amduat_reference_t ref; /* PER/TGK identities are ASL references. */
|
||||
} amduat_fed_record_id_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_meta_t meta;
|
||||
amduat_fed_record_id_t id;
|
||||
uint64_t logseq;
|
||||
uint64_t snapshot_id;
|
||||
uint64_t log_prefix;
|
||||
} amduat_fed_record_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_t *records;
|
||||
size_t len;
|
||||
} amduat_fed_replay_view_t;
|
||||
|
||||
/* Caller frees record ids with amduat_fed_replay_view_free. */
|
||||
bool amduat_fed_record_validate(const amduat_fed_record_t *record);
|
||||
|
||||
bool amduat_fed_replay_build(const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
uint32_t domain_id,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t log_prefix,
|
||||
amduat_fed_replay_view_t *out_view);
|
||||
|
||||
/* Backwards-compatible alias for amduat_fed_replay_build. */
|
||||
/* Deprecated: use amduat_fed_replay_build. */
|
||||
bool amduat_fed_replay_domain(const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
uint32_t domain_id,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t log_prefix,
|
||||
amduat_fed_replay_view_t *out_view);
|
||||
|
||||
void amduat_fed_replay_view_free(amduat_fed_replay_view_t *view);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_FED_REPLAY_H */
|
||||
72
include/amduat/fed/view.h
Normal file
72
include/amduat/fed/view.h
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
#ifndef AMDUAT_FED_VIEW_H
|
||||
#define AMDUAT_FED_VIEW_H
|
||||
|
||||
#include "amduat/asl/store.h"
|
||||
#include "amduat/fed/replay.h"
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_FED_RESOLVE_OK = 0,
|
||||
AMDUAT_FED_RESOLVE_NOT_FOUND = 1,
|
||||
AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES = 2,
|
||||
AMDUAT_FED_RESOLVE_INTEGRITY_ERROR = 3,
|
||||
AMDUAT_FED_RESOLVE_POLICY_DENIED = 4,
|
||||
AMDUAT_FED_RESOLVE_STORE_ERROR = 5
|
||||
} amduat_fed_resolve_error_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t domain_id;
|
||||
uint64_t snapshot_id;
|
||||
uint64_t log_prefix;
|
||||
} amduat_fed_view_bounds_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_fed_record_id_t id;
|
||||
uint32_t reason_code;
|
||||
} amduat_fed_policy_deny_t;
|
||||
|
||||
typedef struct {
|
||||
uint32_t local_domain_id;
|
||||
amduat_fed_record_t *records;
|
||||
size_t len;
|
||||
amduat_fed_policy_deny_t *denies;
|
||||
size_t denies_len;
|
||||
} amduat_fed_view_t;
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_FED_VIEW_OK = 0,
|
||||
AMDUAT_FED_VIEW_ERR_INVALID = 1,
|
||||
AMDUAT_FED_VIEW_ERR_CONFLICT = 2,
|
||||
AMDUAT_FED_VIEW_ERR_OOM = 3
|
||||
} amduat_fed_view_error_t;
|
||||
|
||||
/* Caller frees records/denies with amduat_fed_view_free. */
|
||||
amduat_fed_view_error_t amduat_fed_view_build(
|
||||
const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
uint32_t local_domain_id,
|
||||
const amduat_fed_view_bounds_t *bounds,
|
||||
size_t bounds_len,
|
||||
const amduat_fed_policy_deny_t *denies,
|
||||
size_t denies_len,
|
||||
amduat_fed_view_t *out_view);
|
||||
|
||||
void amduat_fed_view_free(amduat_fed_view_t *view);
|
||||
|
||||
amduat_fed_resolve_error_t amduat_fed_resolve(
|
||||
const amduat_fed_view_t *view,
|
||||
amduat_asl_store_t *local_store,
|
||||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_FED_VIEW_H */
|
||||
|
|
@ -2,6 +2,8 @@
|
|||
#define AMDUAT_FER_RECEIPT_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
#include "amduat/enc/fer1_receipt.h"
|
||||
#include "amduat/pel/run.h"
|
||||
#include "amduat/pel/surf.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
|
@ -24,6 +26,46 @@ bool amduat_fer1_receipt_from_pel_result(
|
|||
uint64_t completed_at,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
bool amduat_fer1_receipt_from_pel_run(
|
||||
const amduat_pel_run_result_t *pel_run,
|
||||
amduat_reference_t input_manifest_ref,
|
||||
amduat_reference_t environment_ref,
|
||||
amduat_octets_t evaluator_id,
|
||||
amduat_reference_t executor_ref,
|
||||
bool has_sbom_ref,
|
||||
amduat_reference_t sbom_ref,
|
||||
amduat_octets_t parity_digest,
|
||||
uint64_t started_at,
|
||||
uint64_t completed_at,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
bool amduat_fer1_receipt_from_pel_run_v1_1(
|
||||
const amduat_pel_run_result_t *pel_run,
|
||||
amduat_reference_t input_manifest_ref,
|
||||
amduat_reference_t environment_ref,
|
||||
amduat_octets_t evaluator_id,
|
||||
amduat_reference_t executor_ref,
|
||||
bool has_sbom_ref,
|
||||
amduat_reference_t sbom_ref,
|
||||
amduat_octets_t parity_digest,
|
||||
uint64_t started_at,
|
||||
uint64_t completed_at,
|
||||
bool has_executor_fingerprint_ref,
|
||||
amduat_reference_t executor_fingerprint_ref,
|
||||
bool has_run_id,
|
||||
amduat_octets_t run_id,
|
||||
bool has_limits,
|
||||
amduat_fer1_limits_t limits,
|
||||
const amduat_fer1_log_entry_t *logs,
|
||||
size_t logs_len,
|
||||
bool has_determinism,
|
||||
uint8_t determinism_level,
|
||||
bool has_rng_seed,
|
||||
amduat_octets_t rng_seed,
|
||||
bool has_signature,
|
||||
amduat_octets_t signature,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
|
|
|||
31
include/amduat/pel/derivation_sid.h
Normal file
31
include/amduat/pel/derivation_sid.h
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
#ifndef AMDUAT_PEL_DERIVATION_SID_H
|
||||
#define AMDUAT_PEL_DERIVATION_SID_H
|
||||
|
||||
#include "amduat/asl/core.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef struct {
|
||||
amduat_reference_t program_ref;
|
||||
const amduat_reference_t *input_refs;
|
||||
size_t input_refs_len;
|
||||
bool has_params_ref;
|
||||
amduat_reference_t params_ref;
|
||||
bool has_exec_profile;
|
||||
amduat_octets_t exec_profile;
|
||||
} amduat_pel_derivation_sid_input_t;
|
||||
|
||||
bool amduat_pel_derivation_sid_compute(
|
||||
const amduat_pel_derivation_sid_input_t *in,
|
||||
amduat_octets_t *out_sid);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_PEL_DERIVATION_SID_H */
|
||||
|
|
@ -17,6 +17,11 @@ extern "C" {
|
|||
#define AMDUAT_PEL_KERNEL_OP_HASH_ASL1_NAME "pel.bytes.hash.asl1"
|
||||
#define AMDUAT_PEL_KERNEL_OP_PARAMS_NAME "pel.bytes.params"
|
||||
#define AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME "pel.format.encode"
|
||||
#define AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME \
|
||||
"collection.snapshot_decode_v1"
|
||||
#define AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME "log.read_range_v1"
|
||||
#define AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME \
|
||||
"collection.merge_refs_v1"
|
||||
|
||||
enum {
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_CONCAT = 0x0001u,
|
||||
|
|
@ -24,7 +29,10 @@ enum {
|
|||
AMDUAT_PEL_KERNEL_OP_CODE_CONST = 0x0003u,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_HASH_ASL1 = 0x0004u,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_PARAMS = 0x0005u,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_FORMAT_ENCODE = 0x0006u
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_FORMAT_ENCODE = 0x0006u,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_SNAPSHOT_DECODE = 0x0007u,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_LOG_READ_RANGE = 0x0008u,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_MERGE_REFS = 0x0009u
|
||||
};
|
||||
|
||||
enum {
|
||||
|
|
@ -35,7 +43,13 @@ enum {
|
|||
AMDUAT_PEL_KERNEL_STATUS_FORMAT_SCHEMA_INVALID = 0x00030001u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_FORMAT_REGISTRY_INVALID = 0x00030002u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_FORMAT_MISSING_FIELD = 0x00030003u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_FORMAT_TYPE_UNSUPPORTED = 0x00030004u
|
||||
AMDUAT_PEL_KERNEL_STATUS_FORMAT_TYPE_UNSUPPORTED = 0x00030004u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID = 0x00040001u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID = 0x00040002u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID = 0x00040003u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID = 0x00040004u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_COLLECTION_VIEW_INVALID = 0x00040005u,
|
||||
AMDUAT_PEL_KERNEL_STATUS_REF_RESOLVE_FAILED = 0x00040006u
|
||||
};
|
||||
|
||||
typedef enum {
|
||||
|
|
@ -44,7 +58,10 @@ typedef enum {
|
|||
AMDUAT_PEL_KERNEL_OP_CONST = 3,
|
||||
AMDUAT_PEL_KERNEL_OP_HASH_ASL1 = 4,
|
||||
AMDUAT_PEL_KERNEL_OP_PARAMS = 5,
|
||||
AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE = 6
|
||||
AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE = 6,
|
||||
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE = 7,
|
||||
AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE = 8,
|
||||
AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS = 9
|
||||
} amduat_pel_kernel_op_kind_t;
|
||||
|
||||
typedef struct {
|
||||
|
|
@ -58,6 +75,11 @@ typedef struct {
|
|||
struct amduat_pel_kernel_params;
|
||||
typedef struct amduat_pel_kernel_params amduat_pel_kernel_params_t;
|
||||
|
||||
typedef bool (*amduat_pel_kernel_artifact_resolver_t)(
|
||||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact,
|
||||
void *ctx);
|
||||
|
||||
const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
|
||||
amduat_octets_t name,
|
||||
uint32_t version);
|
||||
|
|
@ -67,6 +89,15 @@ const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_descs(
|
|||
|
||||
const char *amduat_pel_kernel_op_name(amduat_pel_kernel_op_kind_t kind);
|
||||
|
||||
void amduat_pel_kernel_set_artifact_resolver(
|
||||
amduat_pel_kernel_artifact_resolver_t resolver,
|
||||
void *ctx);
|
||||
|
||||
void amduat_pel_kernel_clear_artifact_resolver(void);
|
||||
|
||||
bool amduat_pel_kernel_resolve_artifact(amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact);
|
||||
|
||||
bool amduat_pel_kernel_op_eval(
|
||||
const amduat_pel_kernel_op_desc_t *desc,
|
||||
const amduat_artifact_t *inputs,
|
||||
|
|
|
|||
38
include/amduat/tgk/tgk_store_asl_index_fs.h
Normal file
38
include/amduat/tgk/tgk_store_asl_index_fs.h
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
#ifndef AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H
|
||||
#define AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H
|
||||
|
||||
/* TGK/1 projection over ASL index/log (filesystem backend). */
|
||||
|
||||
#include "amduat/asl/asl_store_index_fs.h"
|
||||
#include "amduat/tgk/store.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_index_fs_t *asl_fs;
|
||||
amduat_asl_store_t asl_store;
|
||||
amduat_tgk_store_config_t config;
|
||||
amduat_asl_index_state_t pinned_state;
|
||||
bool pinned;
|
||||
bool use_shards;
|
||||
} amduat_tgk_store_asl_index_fs_t;
|
||||
|
||||
bool amduat_tgk_store_asl_index_fs_init(
|
||||
amduat_tgk_store_asl_index_fs_t *store,
|
||||
amduat_tgk_store_config_t config,
|
||||
amduat_asl_store_index_fs_t *asl_fs);
|
||||
|
||||
void amduat_tgk_store_asl_index_fs_free(
|
||||
amduat_tgk_store_asl_index_fs_t *store);
|
||||
|
||||
amduat_tgk_store_ops_t amduat_tgk_store_asl_index_fs_ops(void);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H */
|
||||
23
include/amduat/util/log.h
Normal file
23
include/amduat/util/log.h
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
#ifndef AMDUAT_UTIL_LOG_H
|
||||
#define AMDUAT_UTIL_LOG_H
|
||||
|
||||
#include <stdarg.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
AMDUAT_LOG_ERROR = 0,
|
||||
AMDUAT_LOG_WARN = 1,
|
||||
AMDUAT_LOG_INFO = 2,
|
||||
AMDUAT_LOG_DEBUG = 3
|
||||
} amduat_log_level_t;
|
||||
|
||||
void amduat_log(amduat_log_level_t level, const char *fmt, ...);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_UTIL_LOG_H */
|
||||
1048
src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c
Normal file
1048
src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,716 @@
|
|||
#include "amduat/asl/asl_materialization_cache_fs.h"
|
||||
|
||||
#include "amduat/enc/asl1_core_codec.h"
|
||||
#include "amduat/util/log.h"
|
||||
#include "amduat/util/hex.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_MATERIALIZATION_VERSION = 1
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_materialization_magic[
|
||||
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN] = {'A', 'S', 'L', 'M',
|
||||
'A', 'T', '1', '\0'};
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_MATERIALIZATION_FLAG_TIMESTAMP = 1u << 0,
|
||||
AMDUAT_ASL_MATERIALIZATION_FLAG_PEL_VERSION = 1u << 1
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
const uint8_t *data;
|
||||
size_t len;
|
||||
size_t offset;
|
||||
} amduat_asl_materialization_cursor_t;
|
||||
|
||||
static void amduat_asl_materialization_store_u32_le(uint8_t *out,
|
||||
uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_read_u32_le(
|
||||
amduat_asl_materialization_cursor_t *cur,
|
||||
uint32_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
cur->offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_read_u64_le(
|
||||
amduat_asl_materialization_cursor_t *cur,
|
||||
uint64_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 8u) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
cur->offset += 8u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_read_u16_le(
|
||||
amduat_asl_materialization_cursor_t *cur,
|
||||
uint16_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 2u) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
|
||||
cur->offset += 2u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_read_u8(
|
||||
amduat_asl_materialization_cursor_t *cur,
|
||||
uint8_t *out) {
|
||||
if (cur->len - cur->offset < 1u) {
|
||||
return false;
|
||||
}
|
||||
*out = cur->data[cur->offset++];
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_join_path(const char *base,
|
||||
const char *segment,
|
||||
char **out_path) {
|
||||
size_t base_len;
|
||||
size_t segment_len;
|
||||
bool needs_sep;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (base == NULL || segment == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (base[0] == '\0' || segment[0] == '\0') {
|
||||
return false;
|
||||
}
|
||||
|
||||
base_len = strlen(base);
|
||||
segment_len = strlen(segment);
|
||||
needs_sep = base[base_len - 1u] != '/';
|
||||
total_len = base_len + (needs_sep ? 1u : 0u) + segment_len + 1u;
|
||||
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, base, base_len);
|
||||
offset += base_len;
|
||||
if (needs_sep) {
|
||||
buffer[offset++] = '/';
|
||||
}
|
||||
memcpy(buffer + offset, segment, segment_len);
|
||||
offset += segment_len;
|
||||
buffer[offset] = '\0';
|
||||
|
||||
*out_path = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_ensure_directory(const char *path) {
|
||||
struct stat st;
|
||||
|
||||
if (path == NULL || path[0] == '\0') {
|
||||
return false;
|
||||
}
|
||||
if (stat(path, &st) == 0) {
|
||||
return S_ISDIR(st.st_mode);
|
||||
}
|
||||
if (errno != ENOENT) {
|
||||
return false;
|
||||
}
|
||||
if (mkdir(path, 0755) != 0) {
|
||||
if (errno == EEXIST) {
|
||||
return stat(path, &st) == 0 && S_ISDIR(st.st_mode);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_hex_byte(uint8_t value, char out[3]) {
|
||||
static const char k_hex[] = "0123456789abcdef";
|
||||
out[0] = k_hex[value >> 4u];
|
||||
out[1] = k_hex[value & 0x0fu];
|
||||
out[2] = '\0';
|
||||
return true;
|
||||
}
|
||||
|
||||
static char *amduat_asl_materialization_parent_dir(const char *path) {
|
||||
const char *slash;
|
||||
size_t len;
|
||||
char *dir;
|
||||
|
||||
if (path == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
slash = strrchr(path, '/');
|
||||
if (slash == NULL || slash == path) {
|
||||
return NULL;
|
||||
}
|
||||
len = (size_t)(slash - path);
|
||||
dir = (char *)malloc(len + 1u);
|
||||
if (dir == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
memcpy(dir, path, len);
|
||||
dir[len] = '\0';
|
||||
return dir;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_fsync_directory(const char *path) {
|
||||
int fd;
|
||||
int flags = O_RDONLY;
|
||||
|
||||
if (path == NULL) {
|
||||
return false;
|
||||
}
|
||||
#ifdef O_DIRECTORY
|
||||
flags |= O_DIRECTORY;
|
||||
#endif
|
||||
fd = open(path, flags);
|
||||
if (fd < 0) {
|
||||
return false;
|
||||
}
|
||||
if (fsync(fd) != 0) {
|
||||
close(fd);
|
||||
return false;
|
||||
}
|
||||
return close(fd) == 0;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_build_sid_path(
|
||||
const char *root_path,
|
||||
amduat_octets_t sid,
|
||||
bool ensure_dirs,
|
||||
char **out_path) {
|
||||
char *index_path = NULL;
|
||||
char *materializations_path = NULL;
|
||||
char *by_sid_path = NULL;
|
||||
char *prefix_a_path = NULL;
|
||||
char *prefix_b_path = NULL;
|
||||
char *sid_hex = NULL;
|
||||
char prefix_a[3];
|
||||
char prefix_b[3];
|
||||
bool ok = false;
|
||||
|
||||
if (root_path == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_path = NULL;
|
||||
|
||||
if (sid.len < 2u || sid.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_hex_encode_alloc(sid.data, sid.len, &sid_hex)) {
|
||||
return false;
|
||||
}
|
||||
amduat_asl_materialization_hex_byte(sid.data[0], prefix_a);
|
||||
amduat_asl_materialization_hex_byte(sid.data[1], prefix_b);
|
||||
|
||||
if (!amduat_asl_materialization_join_path(root_path, "index", &index_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_materialization_join_path(index_path, "materializations",
|
||||
&materializations_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_materialization_join_path(materializations_path, "by_sid",
|
||||
&by_sid_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_materialization_join_path(by_sid_path, prefix_a,
|
||||
&prefix_a_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_materialization_join_path(prefix_a_path, prefix_b,
|
||||
&prefix_b_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_materialization_join_path(prefix_b_path, sid_hex,
|
||||
out_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (ensure_dirs) {
|
||||
if (!amduat_asl_materialization_ensure_directory(index_path) ||
|
||||
!amduat_asl_materialization_ensure_directory(materializations_path) ||
|
||||
!amduat_asl_materialization_ensure_directory(by_sid_path) ||
|
||||
!amduat_asl_materialization_ensure_directory(prefix_a_path) ||
|
||||
!amduat_asl_materialization_ensure_directory(prefix_b_path)) {
|
||||
free(*out_path);
|
||||
*out_path = NULL;
|
||||
goto cleanup;
|
||||
}
|
||||
}
|
||||
|
||||
ok = true;
|
||||
|
||||
cleanup:
|
||||
free(index_path);
|
||||
free(materializations_path);
|
||||
free(by_sid_path);
|
||||
free(prefix_a_path);
|
||||
free(prefix_b_path);
|
||||
free(sid_hex);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_materialization_cache_fs_init(
|
||||
amduat_asl_materialization_cache_fs_t *cache,
|
||||
const char *root_path) {
|
||||
size_t len;
|
||||
|
||||
if (cache == NULL || root_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
len = strlen(root_path);
|
||||
if (len == 0u || len >= AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX) {
|
||||
return false;
|
||||
}
|
||||
memcpy(cache->root_path, root_path, len);
|
||||
cache->root_path[len] = '\0';
|
||||
return true;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t amduat_asl_materialization_decode_refs(
|
||||
amduat_asl_materialization_cursor_t *cur,
|
||||
uint32_t count,
|
||||
amduat_reference_t **out_refs,
|
||||
size_t *out_refs_len) {
|
||||
amduat_reference_t *refs;
|
||||
uint32_t i;
|
||||
|
||||
if (out_refs == NULL || out_refs_len == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
*out_refs = NULL;
|
||||
*out_refs_len = 0u;
|
||||
|
||||
if (count == 0u) {
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
if (count > SIZE_MAX / sizeof(*refs)) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
refs = (amduat_reference_t *)calloc(count, sizeof(*refs));
|
||||
if (refs == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
for (i = 0u; i < count; ++i) {
|
||||
uint32_t ref_len;
|
||||
amduat_octets_t ref_bytes;
|
||||
|
||||
if (!amduat_asl_materialization_read_u32_le(cur, &ref_len)) {
|
||||
goto decode_error;
|
||||
}
|
||||
if (ref_len < 2u || cur->len - cur->offset < ref_len) {
|
||||
goto decode_error;
|
||||
}
|
||||
ref_bytes = amduat_octets(cur->data + cur->offset, ref_len);
|
||||
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, &refs[i])) {
|
||||
goto decode_error;
|
||||
}
|
||||
cur->offset += ref_len;
|
||||
}
|
||||
|
||||
*out_refs = refs;
|
||||
*out_refs_len = count;
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
|
||||
decode_error:
|
||||
for (uint32_t j = 0u; j < i; ++j) {
|
||||
amduat_reference_free(&refs[j]);
|
||||
}
|
||||
free(refs);
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
static void amduat_asl_materialization_refs_free(amduat_reference_t *refs,
|
||||
size_t refs_len) {
|
||||
size_t i;
|
||||
|
||||
if (refs == NULL) {
|
||||
return;
|
||||
}
|
||||
for (i = 0u; i < refs_len; ++i) {
|
||||
amduat_reference_free(&refs[i]);
|
||||
}
|
||||
free(refs);
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_get(
|
||||
amduat_asl_materialization_cache_fs_t *cache,
|
||||
amduat_octets_t sid,
|
||||
amduat_reference_t **out_refs,
|
||||
size_t *out_refs_len) {
|
||||
char *path = NULL;
|
||||
FILE *fp = NULL;
|
||||
uint8_t *buffer = NULL;
|
||||
long file_size;
|
||||
amduat_asl_materialization_cursor_t cur;
|
||||
uint8_t flags;
|
||||
uint32_t version;
|
||||
uint32_t sid_len;
|
||||
uint32_t output_count;
|
||||
amduat_asl_store_error_t err = AMDUAT_ASL_STORE_ERR_NOT_FOUND;
|
||||
|
||||
if (out_refs == NULL || out_refs_len == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
*out_refs = NULL;
|
||||
*out_refs_len = 0u;
|
||||
|
||||
if (cache == NULL || sid.len == 0u || sid.data == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
if (!amduat_asl_materialization_build_sid_path(cache->root_path, sid, false,
|
||||
&path)) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
fp = fopen(path, "rb");
|
||||
if (fp == NULL) {
|
||||
free(path);
|
||||
return errno == ENOENT ? AMDUAT_ASL_STORE_ERR_NOT_FOUND
|
||||
: AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (fseek(fp, 0, SEEK_END) != 0) {
|
||||
fclose(fp);
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
file_size = ftell(fp);
|
||||
if (file_size < 0) {
|
||||
fclose(fp);
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (fseek(fp, 0, SEEK_SET) != 0) {
|
||||
fclose(fp);
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc((size_t)file_size);
|
||||
if (buffer == NULL) {
|
||||
fclose(fp);
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (fread(buffer, 1u, (size_t)file_size, fp) != (size_t)file_size) {
|
||||
free(buffer);
|
||||
fclose(fp);
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
fclose(fp);
|
||||
free(path);
|
||||
|
||||
cur.data = buffer;
|
||||
cur.len = (size_t)file_size;
|
||||
cur.offset = 0u;
|
||||
|
||||
if (cur.len < AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN + 4u) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
if (memcmp(cur.data, k_amduat_asl_materialization_magic,
|
||||
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN) != 0) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
cur.offset += AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN;
|
||||
if (!amduat_asl_materialization_read_u32_le(&cur, &version) ||
|
||||
version != AMDUAT_ASL_MATERIALIZATION_VERSION) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_materialization_read_u32_le(&cur, &sid_len)) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
if (sid_len != sid.len || cur.len - cur.offset < sid_len) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
if (memcmp(cur.data + cur.offset, sid.data, sid.len) != 0) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
cur.offset += sid_len;
|
||||
if (!amduat_asl_materialization_read_u32_le(&cur, &output_count)) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
err = amduat_asl_materialization_decode_refs(&cur, output_count, out_refs,
|
||||
out_refs_len);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!amduat_asl_materialization_read_u8(&cur, &flags)) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
if (flags & AMDUAT_ASL_MATERIALIZATION_FLAG_TIMESTAMP) {
|
||||
uint64_t ignored;
|
||||
if (!amduat_asl_materialization_read_u64_le(&cur, &ignored)) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
}
|
||||
if (flags & AMDUAT_ASL_MATERIALIZATION_FLAG_PEL_VERSION) {
|
||||
uint16_t ignored;
|
||||
if (!amduat_asl_materialization_read_u16_le(&cur, &ignored)) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
}
|
||||
|
||||
if (cur.offset != cur.len) {
|
||||
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
err = AMDUAT_ASL_STORE_OK;
|
||||
|
||||
cleanup:
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
amduat_asl_materialization_refs_free(*out_refs, *out_refs_len);
|
||||
*out_refs = NULL;
|
||||
*out_refs_len = 0u;
|
||||
}
|
||||
free(buffer);
|
||||
return err;
|
||||
}
|
||||
|
||||
static bool amduat_asl_materialization_refs_equal(
|
||||
const amduat_reference_t *a,
|
||||
size_t a_len,
|
||||
const amduat_reference_t *b,
|
||||
size_t b_len) {
|
||||
size_t i;
|
||||
|
||||
if (a_len != b_len) {
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < a_len; ++i) {
|
||||
if (!amduat_reference_eq(a[i], b[i])) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t amduat_asl_materialization_write_record(
|
||||
FILE *fp,
|
||||
amduat_octets_t sid,
|
||||
const amduat_reference_t *refs,
|
||||
size_t refs_len) {
|
||||
uint8_t header[AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN + 4u];
|
||||
uint8_t flags = 0u;
|
||||
uint32_t count_u32;
|
||||
size_t i;
|
||||
|
||||
if (sid.len > UINT32_MAX || refs_len > UINT32_MAX) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
memcpy(header, k_amduat_asl_materialization_magic,
|
||||
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN);
|
||||
amduat_asl_materialization_store_u32_le(
|
||||
header + AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN,
|
||||
AMDUAT_ASL_MATERIALIZATION_VERSION);
|
||||
if (fwrite(header, 1u, sizeof(header), fp) != sizeof(header)) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
amduat_asl_materialization_store_u32_le(header, (uint32_t)sid.len);
|
||||
if (fwrite(header, 1u, 4u, fp) != 4u) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (sid.len != 0u &&
|
||||
fwrite(sid.data, 1u, sid.len, fp) != sid.len) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
count_u32 = (uint32_t)refs_len;
|
||||
amduat_asl_materialization_store_u32_le(header, count_u32);
|
||||
if (fwrite(header, 1u, 4u, fp) != 4u) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
for (i = 0u; i < refs_len; ++i) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
uint8_t len_buf[4u];
|
||||
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
amduat_asl_materialization_store_u32_le(len_buf,
|
||||
(uint32_t)ref_bytes.len);
|
||||
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
|
||||
(ref_bytes.len != 0u &&
|
||||
fwrite(ref_bytes.data, 1u, ref_bytes.len, fp) != ref_bytes.len)) {
|
||||
free((void *)ref_bytes.data);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
|
||||
if (fwrite(&flags, 1u, 1u, fp) != 1u) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_put(
|
||||
amduat_asl_materialization_cache_fs_t *cache,
|
||||
amduat_octets_t sid,
|
||||
const amduat_reference_t *refs,
|
||||
size_t refs_len) {
|
||||
amduat_reference_t *existing_refs = NULL;
|
||||
size_t existing_len = 0u;
|
||||
amduat_asl_store_error_t err;
|
||||
char *path = NULL;
|
||||
char *tmp_path = NULL;
|
||||
FILE *fp = NULL;
|
||||
size_t tmp_len;
|
||||
|
||||
if (cache == NULL || sid.len == 0u || sid.data == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
if (refs_len != 0u && refs == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
err = amduat_asl_materialization_cache_fs_get(cache, sid,
|
||||
&existing_refs, &existing_len);
|
||||
if (err == AMDUAT_ASL_STORE_OK) {
|
||||
bool same = amduat_asl_materialization_refs_equal(
|
||||
existing_refs, existing_len, refs, refs_len);
|
||||
amduat_asl_materialization_refs_free(existing_refs, existing_len);
|
||||
if (!same) {
|
||||
amduat_log(AMDUAT_LOG_ERROR,
|
||||
"materialization cache SID collision with mismatched outputs");
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
|
||||
return err;
|
||||
}
|
||||
|
||||
if (!amduat_asl_materialization_build_sid_path(cache->root_path, sid, true,
|
||||
&path)) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
tmp_len = strlen(path) + 5u;
|
||||
tmp_path = (char *)malloc(tmp_len);
|
||||
if (tmp_path == NULL) {
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
snprintf(tmp_path, tmp_len, "%s.tmp", path);
|
||||
|
||||
fp = fopen(tmp_path, "wb");
|
||||
if (fp == NULL) {
|
||||
free(tmp_path);
|
||||
free(path);
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
|
||||
/*
|
||||
* Crash consistency: write + fsync temp file, rename, then fsync parent
|
||||
* directory. Failure only affects cache persistence, not correctness.
|
||||
*/
|
||||
err = amduat_asl_materialization_write_record(fp, sid, refs, refs_len);
|
||||
if (fflush(fp) != 0) {
|
||||
err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (err == AMDUAT_ASL_STORE_OK && fsync(fileno(fp)) != 0) {
|
||||
err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (fclose(fp) != 0) {
|
||||
err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
fp = NULL;
|
||||
|
||||
if (err == AMDUAT_ASL_STORE_OK && rename(tmp_path, path) != 0) {
|
||||
err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
if (err == AMDUAT_ASL_STORE_OK) {
|
||||
char *parent_dir = amduat_asl_materialization_parent_dir(path);
|
||||
if (parent_dir != NULL) {
|
||||
if (!amduat_asl_materialization_fsync_directory(parent_dir)) {
|
||||
amduat_log(AMDUAT_LOG_WARN,
|
||||
"materialization cache fsync dir failed for %s",
|
||||
parent_dir);
|
||||
err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
free(parent_dir);
|
||||
}
|
||||
}
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
(void)remove(tmp_path);
|
||||
}
|
||||
|
||||
#ifndef NDEBUG
|
||||
if (err == AMDUAT_ASL_STORE_OK) {
|
||||
amduat_reference_t *check_refs = NULL;
|
||||
size_t check_len = 0u;
|
||||
amduat_asl_store_error_t check_err =
|
||||
amduat_asl_materialization_cache_fs_get(cache, sid, &check_refs,
|
||||
&check_len);
|
||||
assert(check_err == AMDUAT_ASL_STORE_OK);
|
||||
assert(amduat_asl_materialization_refs_equal(
|
||||
check_refs, check_len, refs, refs_len));
|
||||
amduat_asl_materialization_refs_free(check_refs, check_len);
|
||||
}
|
||||
#endif
|
||||
|
||||
free(tmp_path);
|
||||
free(path);
|
||||
return err;
|
||||
}
|
||||
753
src/adapters/asl_pointer_fs/asl_pointer_fs.c
Normal file
753
src/adapters/asl_pointer_fs/asl_pointer_fs.c
Normal file
|
|
@ -0,0 +1,753 @@
|
|||
#include "amduat/asl/asl_pointer_fs.h"
|
||||
|
||||
#include "amduat/enc/asl1_core_codec.h"
|
||||
#include "amduat/util/log.h"
|
||||
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_POINTER_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_POINTER_VERSION = 1
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_pointer_magic[AMDUAT_ASL_POINTER_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'P', 'T', 'R', '1', '\0'
|
||||
};
|
||||
|
||||
enum { AMDUAT_ASL_POINTER_FLAG_HAS_EXPECTED = 1u << 0,
|
||||
AMDUAT_ASL_POINTER_FLAG_HAS_PREV = 1u << 1 };
|
||||
|
||||
typedef struct {
|
||||
const uint8_t *data;
|
||||
size_t len;
|
||||
size_t offset;
|
||||
} amduat_asl_pointer_cursor_t;
|
||||
|
||||
static void amduat_asl_pointer_store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_pointer_read_u32_le(amduat_asl_pointer_cursor_t *cur,
|
||||
uint32_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
cur->offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_pointer_read_u8(amduat_asl_pointer_cursor_t *cur,
|
||||
uint8_t *out) {
|
||||
if (cur->len - cur->offset < 1u) {
|
||||
return false;
|
||||
}
|
||||
*out = cur->data[cur->offset++];
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_pointer_join_path(const char *base,
|
||||
const char *segment,
|
||||
char **out_path) {
|
||||
size_t base_len;
|
||||
size_t segment_len;
|
||||
bool needs_sep;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (base == NULL || segment == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (base[0] == '\0' || segment[0] == '\0') {
|
||||
return false;
|
||||
}
|
||||
|
||||
base_len = strlen(base);
|
||||
segment_len = strlen(segment);
|
||||
needs_sep = base[base_len - 1u] != '/';
|
||||
total_len = base_len + (needs_sep ? 1u : 0u) + segment_len + 1u;
|
||||
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, base, base_len);
|
||||
offset += base_len;
|
||||
if (needs_sep) {
|
||||
buffer[offset++] = '/';
|
||||
}
|
||||
memcpy(buffer + offset, segment, segment_len);
|
||||
offset += segment_len;
|
||||
buffer[offset] = '\0';
|
||||
|
||||
*out_path = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_pointer_ensure_directory(const char *path) {
|
||||
struct stat st;
|
||||
|
||||
if (path == NULL || path[0] == '\0') {
|
||||
return false;
|
||||
}
|
||||
if (stat(path, &st) == 0) {
|
||||
return S_ISDIR(st.st_mode);
|
||||
}
|
||||
if (errno != ENOENT) {
|
||||
return false;
|
||||
}
|
||||
if (mkdir(path, 0755) != 0) {
|
||||
if (errno == EEXIST) {
|
||||
return stat(path, &st) == 0 && S_ISDIR(st.st_mode);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static char *amduat_asl_pointer_parent_dir(const char *path) {
|
||||
const char *slash;
|
||||
size_t len;
|
||||
char *dir;
|
||||
|
||||
if (path == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
slash = strrchr(path, '/');
|
||||
if (slash == NULL || slash == path) {
|
||||
return NULL;
|
||||
}
|
||||
len = (size_t)(slash - path);
|
||||
dir = (char *)malloc(len + 1u);
|
||||
if (dir == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
memcpy(dir, path, len);
|
||||
dir[len] = '\0';
|
||||
return dir;
|
||||
}
|
||||
|
||||
static bool amduat_asl_pointer_fsync_directory(const char *path) {
|
||||
int fd;
|
||||
int flags = O_RDONLY;
|
||||
|
||||
if (path == NULL) {
|
||||
return false;
|
||||
}
|
||||
#ifdef O_DIRECTORY
|
||||
flags |= O_DIRECTORY;
|
||||
#endif
|
||||
fd = open(path, flags);
|
||||
if (fd < 0) {
|
||||
return false;
|
||||
}
|
||||
if (fsync(fd) != 0) {
|
||||
close(fd);
|
||||
return false;
|
||||
}
|
||||
return close(fd) == 0;
|
||||
}
|
||||
|
||||
bool amduat_asl_pointer_name_is_valid(const char *name) {
|
||||
size_t len;
|
||||
size_t seg_start = 0u;
|
||||
|
||||
if (name == NULL) {
|
||||
return false;
|
||||
}
|
||||
len = strlen(name);
|
||||
if (len == 0u || len > AMDUAT_ASL_POINTER_NAME_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (name[0] == '/' || name[len - 1u] == '/') {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (size_t i = 0u; i < len; ++i) {
|
||||
char c = name[i];
|
||||
bool ok = (c >= 'A' && c <= 'Z') ||
|
||||
(c >= 'a' && c <= 'z') ||
|
||||
(c >= '0' && c <= '9') ||
|
||||
c == '.' || c == '_' || c == '-' || c == '/';
|
||||
if (!ok) {
|
||||
return false;
|
||||
}
|
||||
if (c == '/') {
|
||||
size_t seg_len = i - seg_start;
|
||||
if (seg_len == 0u) {
|
||||
return false;
|
||||
}
|
||||
if (seg_len == 2u && name[seg_start] == '.' &&
|
||||
name[seg_start + 1u] == '.') {
|
||||
return false;
|
||||
}
|
||||
seg_start = i + 1u;
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
size_t seg_len = len - seg_start;
|
||||
if (seg_len == 0u) {
|
||||
return false;
|
||||
}
|
||||
if (seg_len == 2u && name[seg_start] == '.' &&
|
||||
name[seg_start + 1u] == '.') {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_pointer_build_head_path(const char *root_path,
|
||||
const char *name,
|
||||
bool ensure_dirs,
|
||||
char **out_path) {
|
||||
char *pointers_path = NULL;
|
||||
char *cursor_path = NULL;
|
||||
char *head_path = NULL;
|
||||
const char *segment_start;
|
||||
const char *segment_end;
|
||||
bool ok = false;
|
||||
|
||||
if (root_path == NULL || name == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_path = NULL;
|
||||
|
||||
if (!amduat_asl_pointer_join_path(root_path, "pointers", &pointers_path)) {
|
||||
return false;
|
||||
}
|
||||
if (ensure_dirs &&
|
||||
!amduat_asl_pointer_ensure_directory(pointers_path)) {
|
||||
free(pointers_path);
|
||||
return false;
|
||||
}
|
||||
|
||||
cursor_path = pointers_path;
|
||||
pointers_path = NULL;
|
||||
|
||||
segment_start = name;
|
||||
while (*segment_start != '\0') {
|
||||
segment_end = strchr(segment_start, '/');
|
||||
if (segment_end == NULL) {
|
||||
segment_end = segment_start + strlen(segment_start);
|
||||
}
|
||||
{
|
||||
size_t seg_len = (size_t)(segment_end - segment_start);
|
||||
char *segment = (char *)malloc(seg_len + 1u);
|
||||
char *next_path = NULL;
|
||||
if (segment == NULL) {
|
||||
goto cleanup;
|
||||
}
|
||||
memcpy(segment, segment_start, seg_len);
|
||||
segment[seg_len] = '\0';
|
||||
if (!amduat_asl_pointer_join_path(cursor_path, segment, &next_path)) {
|
||||
free(segment);
|
||||
goto cleanup;
|
||||
}
|
||||
free(segment);
|
||||
if (ensure_dirs &&
|
||||
!amduat_asl_pointer_ensure_directory(next_path)) {
|
||||
free(next_path);
|
||||
goto cleanup;
|
||||
}
|
||||
free(cursor_path);
|
||||
cursor_path = next_path;
|
||||
}
|
||||
if (*segment_end == '\0') {
|
||||
break;
|
||||
}
|
||||
segment_start = segment_end + 1u;
|
||||
}
|
||||
|
||||
if (!amduat_asl_pointer_join_path(cursor_path, "head", &head_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
*out_path = head_path;
|
||||
head_path = NULL;
|
||||
ok = true;
|
||||
|
||||
cleanup:
|
||||
free(pointers_path);
|
||||
free(cursor_path);
|
||||
free(head_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_pointer_store_init(amduat_asl_pointer_store_t *ps,
|
||||
const char *root_path) {
|
||||
size_t len;
|
||||
|
||||
if (ps == NULL || root_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
len = strlen(root_path);
|
||||
if (len == 0u || len >= AMDUAT_ASL_POINTER_FS_ROOT_MAX) {
|
||||
return false;
|
||||
}
|
||||
memcpy(ps->root_path, root_path, len);
|
||||
ps->root_path[len] = '\0';
|
||||
return true;
|
||||
}
|
||||
|
||||
static amduat_asl_pointer_error_t amduat_asl_pointer_read_head(
|
||||
const char *path,
|
||||
const char *name,
|
||||
bool *out_exists,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_reference_t *out_prev_ref,
|
||||
bool *out_has_prev) {
|
||||
FILE *fp;
|
||||
uint8_t *buffer;
|
||||
long file_size;
|
||||
amduat_asl_pointer_cursor_t cur;
|
||||
uint32_t version;
|
||||
uint32_t name_len;
|
||||
uint32_t ref_len;
|
||||
uint32_t prev_len;
|
||||
uint8_t flags;
|
||||
amduat_octets_t ref_bytes;
|
||||
amduat_octets_t prev_bytes;
|
||||
|
||||
if (out_exists == NULL || out_ref == NULL) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
*out_exists = false;
|
||||
memset(out_ref, 0, sizeof(*out_ref));
|
||||
if (out_prev_ref != NULL) {
|
||||
memset(out_prev_ref, 0, sizeof(*out_prev_ref));
|
||||
}
|
||||
if (out_has_prev != NULL) {
|
||||
*out_has_prev = false;
|
||||
}
|
||||
|
||||
fp = fopen(path, "rb");
|
||||
if (fp == NULL) {
|
||||
return errno == ENOENT ? AMDUAT_ASL_POINTER_ERR_NOT_FOUND
|
||||
: AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (fseek(fp, 0, SEEK_END) != 0) {
|
||||
fclose(fp);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
file_size = ftell(fp);
|
||||
if (file_size < 0) {
|
||||
fclose(fp);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (file_size == 0) {
|
||||
fclose(fp);
|
||||
return AMDUAT_ASL_POINTER_ERR_NOT_FOUND;
|
||||
}
|
||||
if (fseek(fp, 0, SEEK_SET) != 0) {
|
||||
fclose(fp);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc((size_t)file_size);
|
||||
if (buffer == NULL) {
|
||||
fclose(fp);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (fread(buffer, 1u, (size_t)file_size, fp) != (size_t)file_size) {
|
||||
free(buffer);
|
||||
fclose(fp);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
fclose(fp);
|
||||
|
||||
cur.data = buffer;
|
||||
cur.len = (size_t)file_size;
|
||||
cur.offset = 0u;
|
||||
|
||||
if (cur.len < AMDUAT_ASL_POINTER_MAGIC_LEN + 4u + 1u) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (memcmp(cur.data, k_amduat_asl_pointer_magic,
|
||||
AMDUAT_ASL_POINTER_MAGIC_LEN) != 0) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
cur.offset += AMDUAT_ASL_POINTER_MAGIC_LEN;
|
||||
if (!amduat_asl_pointer_read_u32_le(&cur, &version) ||
|
||||
version != AMDUAT_ASL_POINTER_VERSION) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_read_u8(&cur, &flags)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_read_u32_le(&cur, &name_len)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (cur.len - cur.offset < name_len) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (strlen(name) != name_len ||
|
||||
memcmp(cur.data + cur.offset, name, name_len) != 0) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
cur.offset += name_len;
|
||||
if (!amduat_asl_pointer_read_u32_le(&cur, &ref_len)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (cur.len - cur.offset < ref_len || ref_len < 2u) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
ref_bytes = amduat_octets(cur.data + cur.offset, ref_len);
|
||||
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, out_ref)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
cur.offset += ref_len;
|
||||
|
||||
if (!amduat_asl_pointer_read_u32_le(&cur, &prev_len)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (prev_len != 0u) {
|
||||
if (!(flags & AMDUAT_ASL_POINTER_FLAG_HAS_PREV)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (cur.len - cur.offset < prev_len || prev_len < 2u) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
prev_bytes = amduat_octets(cur.data + cur.offset, prev_len);
|
||||
if (out_prev_ref != NULL &&
|
||||
!amduat_enc_asl1_core_decode_reference_v1(prev_bytes, out_prev_ref)) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
cur.offset += prev_len;
|
||||
if (out_has_prev != NULL) {
|
||||
*out_has_prev = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (cur.offset != cur.len) {
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
*out_exists = true;
|
||||
free(buffer);
|
||||
return AMDUAT_ASL_POINTER_OK;
|
||||
}
|
||||
|
||||
static amduat_asl_pointer_error_t amduat_asl_pointer_write_head(
|
||||
const char *path,
|
||||
const char *name,
|
||||
const amduat_reference_t *ref,
|
||||
const amduat_reference_t *prev_ref,
|
||||
bool has_prev) {
|
||||
char *tmp_path;
|
||||
char *parent_dir = NULL;
|
||||
size_t tmp_len;
|
||||
int tmp_fd;
|
||||
FILE *fp;
|
||||
uint8_t header[AMDUAT_ASL_POINTER_MAGIC_LEN + 4u + 1u];
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
amduat_octets_t prev_bytes = amduat_octets(NULL, 0u);
|
||||
uint32_t name_len;
|
||||
uint32_t ref_len;
|
||||
uint32_t prev_len;
|
||||
uint8_t flags = 0u;
|
||||
amduat_asl_pointer_error_t err = AMDUAT_ASL_POINTER_OK;
|
||||
|
||||
if (ref == NULL || name == NULL) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
name_len = (uint32_t)strlen(name);
|
||||
if (!amduat_enc_asl1_core_encode_reference_v1(*ref, &ref_bytes)) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
ref_len = (uint32_t)ref_bytes.len;
|
||||
|
||||
if (has_prev) {
|
||||
if (prev_ref == NULL ||
|
||||
!amduat_enc_asl1_core_encode_reference_v1(*prev_ref, &prev_bytes)) {
|
||||
free((void *)ref_bytes.data);
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
prev_len = (uint32_t)prev_bytes.len;
|
||||
flags |= AMDUAT_ASL_POINTER_FLAG_HAS_PREV;
|
||||
} else {
|
||||
prev_len = 0u;
|
||||
}
|
||||
|
||||
tmp_len = strlen(path) + sizeof(".tmp.XXXXXX");
|
||||
tmp_path = (char *)malloc(tmp_len);
|
||||
if (tmp_path == NULL) {
|
||||
free((void *)ref_bytes.data);
|
||||
free((void *)prev_bytes.data);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
parent_dir = amduat_asl_pointer_parent_dir(path);
|
||||
if (parent_dir == NULL || !amduat_asl_pointer_ensure_directory(parent_dir)) {
|
||||
free(parent_dir);
|
||||
free(tmp_path);
|
||||
free((void *)ref_bytes.data);
|
||||
free((void *)prev_bytes.data);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
snprintf(tmp_path, tmp_len, "%s.tmp.XXXXXX", path);
|
||||
tmp_fd = mkstemp(tmp_path);
|
||||
if (tmp_fd < 0 && errno == ENOENT &&
|
||||
amduat_asl_pointer_ensure_directory(parent_dir)) {
|
||||
tmp_fd = mkstemp(tmp_path);
|
||||
}
|
||||
if (tmp_fd < 0) {
|
||||
free(parent_dir);
|
||||
free(tmp_path);
|
||||
free((void *)ref_bytes.data);
|
||||
free((void *)prev_bytes.data);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
fp = fdopen(tmp_fd, "wb");
|
||||
if (fp == NULL) {
|
||||
close(tmp_fd);
|
||||
(void)remove(tmp_path);
|
||||
free(tmp_path);
|
||||
free((void *)ref_bytes.data);
|
||||
free((void *)prev_bytes.data);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
memcpy(header, k_amduat_asl_pointer_magic, AMDUAT_ASL_POINTER_MAGIC_LEN);
|
||||
amduat_asl_pointer_store_u32_le(
|
||||
header + AMDUAT_ASL_POINTER_MAGIC_LEN, AMDUAT_ASL_POINTER_VERSION);
|
||||
header[AMDUAT_ASL_POINTER_MAGIC_LEN + 4u] = flags;
|
||||
if (fwrite(header, 1u, sizeof(header), fp) != sizeof(header)) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (err == AMDUAT_ASL_POINTER_OK) {
|
||||
uint8_t len_buf[4u];
|
||||
amduat_asl_pointer_store_u32_le(len_buf, name_len);
|
||||
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
|
||||
fwrite(name, 1u, name_len, fp) != name_len) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
}
|
||||
if (err == AMDUAT_ASL_POINTER_OK) {
|
||||
uint8_t len_buf[4u];
|
||||
amduat_asl_pointer_store_u32_le(len_buf, ref_len);
|
||||
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
|
||||
fwrite(ref_bytes.data, 1u, ref_len, fp) != ref_len) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
}
|
||||
if (err == AMDUAT_ASL_POINTER_OK) {
|
||||
uint8_t len_buf[4u];
|
||||
amduat_asl_pointer_store_u32_le(len_buf, prev_len);
|
||||
if (fwrite(len_buf, 1u, 4u, fp) != 4u) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (err == AMDUAT_ASL_POINTER_OK && prev_len != 0u &&
|
||||
fwrite(prev_bytes.data, 1u, prev_len, fp) != prev_len) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
}
|
||||
|
||||
if (err == AMDUAT_ASL_POINTER_OK && fflush(fp) != 0) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (err == AMDUAT_ASL_POINTER_OK && fsync(fileno(fp)) != 0) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
if (fclose(fp) != 0) {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
if (err == AMDUAT_ASL_POINTER_OK && rename(tmp_path, path) != 0) {
|
||||
if (errno == ENOENT && amduat_asl_pointer_ensure_directory(parent_dir) &&
|
||||
rename(tmp_path, path) == 0) {
|
||||
/* Recovered after recreating parent directory. */
|
||||
} else {
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
}
|
||||
if (err == AMDUAT_ASL_POINTER_OK) {
|
||||
if (!amduat_asl_pointer_fsync_directory(parent_dir)) {
|
||||
amduat_log(AMDUAT_LOG_WARN,
|
||||
"pointer fsync dir failed for %s", parent_dir);
|
||||
err = AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
}
|
||||
if (err != AMDUAT_ASL_POINTER_OK) {
|
||||
(void)remove(tmp_path);
|
||||
}
|
||||
|
||||
free(parent_dir);
|
||||
free(tmp_path);
|
||||
free((void *)ref_bytes.data);
|
||||
free((void *)prev_bytes.data);
|
||||
return err;
|
||||
}
|
||||
|
||||
amduat_asl_pointer_error_t amduat_asl_pointer_get(
|
||||
const amduat_asl_pointer_store_t *ps,
|
||||
const char *name,
|
||||
bool *out_exists,
|
||||
amduat_reference_t *out_ref) {
|
||||
char *head_path = NULL;
|
||||
amduat_asl_pointer_error_t err;
|
||||
|
||||
if (ps == NULL || name == NULL) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(name)) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INVALID_NAME;
|
||||
}
|
||||
if (!amduat_asl_pointer_build_head_path(ps->root_path, name, false,
|
||||
&head_path)) {
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
err = amduat_asl_pointer_read_head(head_path, name, out_exists, out_ref,
|
||||
NULL, NULL);
|
||||
if (err == AMDUAT_ASL_POINTER_ERR_NOT_FOUND) {
|
||||
*out_exists = false;
|
||||
err = AMDUAT_ASL_POINTER_OK;
|
||||
}
|
||||
free(head_path);
|
||||
return err;
|
||||
}
|
||||
|
||||
amduat_asl_pointer_error_t amduat_asl_pointer_cas(
|
||||
const amduat_asl_pointer_store_t *ps,
|
||||
const char *name,
|
||||
bool expected_exists,
|
||||
const amduat_reference_t *expected_ref,
|
||||
const amduat_reference_t *new_ref,
|
||||
bool *out_swapped) {
|
||||
char *head_path = NULL;
|
||||
int lock_fd = -1;
|
||||
struct flock lock;
|
||||
bool exists = false;
|
||||
amduat_reference_t current_ref;
|
||||
amduat_reference_t prev_ref;
|
||||
bool has_prev = false;
|
||||
char *lock_path = NULL;
|
||||
amduat_asl_pointer_error_t err;
|
||||
|
||||
if (out_swapped == NULL) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
*out_swapped = false;
|
||||
if (ps == NULL || name == NULL || new_ref == NULL) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(name)) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INVALID_NAME;
|
||||
}
|
||||
if (expected_exists && expected_ref == NULL) {
|
||||
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
if (!amduat_asl_pointer_build_head_path(ps->root_path, name, true,
|
||||
&head_path)) {
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
lock_path = (char *)malloc(strlen(head_path) + sizeof(".lock"));
|
||||
if (lock_path == NULL) {
|
||||
free(head_path);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
snprintf(lock_path, strlen(head_path) + sizeof(".lock"), "%s.lock",
|
||||
head_path);
|
||||
|
||||
/* Lock a stable sidecar file so lock ownership survives head-file renames. */
|
||||
lock_fd = open(lock_path, O_RDWR | O_CREAT, 0644);
|
||||
if (lock_fd < 0) {
|
||||
free(lock_path);
|
||||
free(head_path);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
memset(&lock, 0, sizeof(lock));
|
||||
lock.l_type = F_WRLCK;
|
||||
lock.l_whence = SEEK_SET;
|
||||
lock.l_start = 0;
|
||||
lock.l_len = 0;
|
||||
if (fcntl(lock_fd, F_SETLKW, &lock) != 0) {
|
||||
close(lock_fd);
|
||||
free(lock_path);
|
||||
free(head_path);
|
||||
return AMDUAT_ASL_POINTER_ERR_IO;
|
||||
}
|
||||
|
||||
err = amduat_asl_pointer_read_head(head_path, name, &exists, ¤t_ref,
|
||||
&prev_ref, &has_prev);
|
||||
if (err == AMDUAT_ASL_POINTER_ERR_NOT_FOUND) {
|
||||
exists = false;
|
||||
err = AMDUAT_ASL_POINTER_OK;
|
||||
}
|
||||
if (err != AMDUAT_ASL_POINTER_OK) {
|
||||
close(lock_fd);
|
||||
free(lock_path);
|
||||
free(head_path);
|
||||
return err;
|
||||
}
|
||||
|
||||
if (expected_exists != exists) {
|
||||
close(lock_fd);
|
||||
free(lock_path);
|
||||
free(head_path);
|
||||
return AMDUAT_ASL_POINTER_OK;
|
||||
}
|
||||
if (expected_exists && !amduat_reference_eq(*expected_ref, current_ref)) {
|
||||
close(lock_fd);
|
||||
free(lock_path);
|
||||
free(head_path);
|
||||
return AMDUAT_ASL_POINTER_OK;
|
||||
}
|
||||
|
||||
err = amduat_asl_pointer_write_head(head_path, name, new_ref,
|
||||
expected_exists ? ¤t_ref : NULL,
|
||||
expected_exists);
|
||||
if (err == AMDUAT_ASL_POINTER_OK) {
|
||||
*out_swapped = true;
|
||||
}
|
||||
|
||||
if (exists) {
|
||||
amduat_reference_free(¤t_ref);
|
||||
}
|
||||
if (has_prev) {
|
||||
amduat_reference_free(&prev_ref);
|
||||
}
|
||||
close(lock_fd);
|
||||
free(lock_path);
|
||||
free(head_path);
|
||||
return err;
|
||||
}
|
||||
|
|
@ -489,6 +489,30 @@ static amduat_asl_store_error_t amduat_asl_store_fs_put_impl(
|
|||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
static void amduat_asl_store_fs_fill_index_state(
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
if (out_state == NULL) {
|
||||
return;
|
||||
}
|
||||
out_state->snapshot_id = 0u;
|
||||
out_state->log_position = 0u;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t amduat_asl_store_fs_put_indexed_impl(
|
||||
void *ctx,
|
||||
amduat_artifact_t artifact,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
err = amduat_asl_store_fs_put_impl(ctx, artifact, out_ref);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
return err;
|
||||
}
|
||||
amduat_asl_store_fs_fill_index_state(out_state);
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t amduat_asl_store_fs_validate_config(
|
||||
void *ctx,
|
||||
amduat_asl_store_config_t config) {
|
||||
|
|
@ -511,6 +535,16 @@ static amduat_asl_store_error_t amduat_asl_store_fs_validate_config(
|
|||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
static bool amduat_asl_store_fs_current_state_impl(
|
||||
void *ctx,
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
if (ctx == NULL) {
|
||||
return false;
|
||||
}
|
||||
amduat_asl_store_fs_fill_index_state(out_state);
|
||||
return true;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
|
||||
void *ctx,
|
||||
amduat_reference_t ref,
|
||||
|
|
@ -672,6 +706,15 @@ static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
|
|||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t amduat_asl_store_fs_get_indexed_impl(
|
||||
void *ctx,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_index_state_t state,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
(void)state;
|
||||
return amduat_asl_store_fs_get_impl(ctx, ref, out_artifact);
|
||||
}
|
||||
|
||||
bool amduat_asl_store_fs_init(amduat_asl_store_fs_t *fs,
|
||||
amduat_asl_store_config_t config,
|
||||
const char *root_path) {
|
||||
|
|
@ -696,6 +739,9 @@ amduat_asl_store_ops_t amduat_asl_store_fs_ops(void) {
|
|||
amduat_asl_store_ops_init(&ops);
|
||||
ops.put = amduat_asl_store_fs_put_impl;
|
||||
ops.get = amduat_asl_store_fs_get_impl;
|
||||
ops.put_indexed = amduat_asl_store_fs_put_indexed_impl;
|
||||
ops.get_indexed = amduat_asl_store_fs_get_indexed_impl;
|
||||
ops.current_state = amduat_asl_store_fs_current_state_impl;
|
||||
ops.validate_config = amduat_asl_store_fs_validate_config;
|
||||
return ops;
|
||||
}
|
||||
|
|
|
|||
5295
src/adapters/asl_store_index_fs/asl_store_index_fs.c
Normal file
5295
src/adapters/asl_store_index_fs/asl_store_index_fs.c
Normal file
File diff suppressed because it is too large
Load diff
415
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.c
Normal file
415
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.c
Normal file
|
|
@ -0,0 +1,415 @@
|
|||
#include "asl_store_index_fs_layout.h"
|
||||
|
||||
#include <inttypes.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static bool amduat_asl_store_index_fs_layout_join(const char *base,
|
||||
const char *suffix,
|
||||
char **out_path) {
|
||||
size_t base_len;
|
||||
size_t suffix_len;
|
||||
bool needs_sep;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (base == NULL || suffix == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (base[0] == '\0' || suffix[0] == '\0') {
|
||||
return false;
|
||||
}
|
||||
|
||||
base_len = strlen(base);
|
||||
suffix_len = strlen(suffix);
|
||||
needs_sep = base[base_len - 1u] != '/';
|
||||
total_len = base_len + (needs_sep ? 1u : 0u) + suffix_len + 1u;
|
||||
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, base, base_len);
|
||||
offset += base_len;
|
||||
if (needs_sep) {
|
||||
buffer[offset++] = '/';
|
||||
}
|
||||
memcpy(buffer + offset, suffix, suffix_len);
|
||||
offset += suffix_len;
|
||||
buffer[offset] = '\0';
|
||||
|
||||
*out_path = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_store_index_fs_layout_format_id(const char *prefix,
|
||||
uint64_t id,
|
||||
const char *suffix,
|
||||
char **out_name) {
|
||||
int needed;
|
||||
char *buffer;
|
||||
|
||||
if (prefix == NULL || suffix == NULL || out_name == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
needed = snprintf(NULL, 0, "%s%016" PRIx64 "%s", prefix, id, suffix);
|
||||
if (needed <= 0) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (char *)malloc((size_t)needed + 1u);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
snprintf(buffer, (size_t)needed + 1u, "%s%016" PRIx64 "%s", prefix, id,
|
||||
suffix);
|
||||
*out_name = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_index_path(const char *root_path,
|
||||
char **out_path) {
|
||||
return amduat_asl_store_index_fs_layout_join(root_path, "index", out_path);
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segments_path(
|
||||
const char *root_path,
|
||||
char **out_path) {
|
||||
char *index_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
|
||||
&index_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(index_path, "segments", out_path);
|
||||
free(index_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_blocks_path(const char *root_path,
|
||||
char **out_path) {
|
||||
char *index_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
|
||||
&index_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(index_path, "blocks", out_path);
|
||||
free(index_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_log_path(const char *root_path,
|
||||
char **out_path) {
|
||||
char *index_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
|
||||
&index_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(index_path, "log.asl", out_path);
|
||||
free(index_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shards_path(
|
||||
const char *root_path,
|
||||
char **out_path) {
|
||||
char *index_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
|
||||
&index_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(index_path, "shards", out_path);
|
||||
free(index_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path) {
|
||||
char *shards_path;
|
||||
char *name;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_shards_path(root_path,
|
||||
&shards_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_store_index_fs_layout_format_id("shard-",
|
||||
(uint64_t)shard_id,
|
||||
"",
|
||||
&name)) {
|
||||
free(shards_path);
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(shards_path, name, out_path);
|
||||
free(name);
|
||||
free(shards_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_segments_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path) {
|
||||
char *shard_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_shard_path(root_path,
|
||||
shard_id,
|
||||
&shard_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(shard_path, "segments", out_path);
|
||||
free(shard_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_blocks_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path) {
|
||||
char *shard_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_shard_path(root_path,
|
||||
shard_id,
|
||||
&shard_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(shard_path, "blocks", out_path);
|
||||
free(shard_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_segment_meta_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path) {
|
||||
char *segments_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_shard_segments_path(root_path,
|
||||
shard_id,
|
||||
&segments_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(segments_path,
|
||||
"next_id",
|
||||
out_path);
|
||||
free(segments_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_segment_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
uint64_t segment_id,
|
||||
char **out_path) {
|
||||
char *segments_path;
|
||||
char *name;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_shard_segments_path(root_path,
|
||||
shard_id,
|
||||
&segments_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_store_index_fs_layout_format_id("segment-",
|
||||
segment_id,
|
||||
".asl",
|
||||
&name)) {
|
||||
free(segments_path);
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = amduat_asl_store_index_fs_layout_join(segments_path, name, out_path);
|
||||
free(name);
|
||||
free(segments_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_block_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
uint64_t block_id,
|
||||
char **out_path) {
|
||||
char *blocks_path;
|
||||
char *name;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_shard_blocks_path(root_path,
|
||||
shard_id,
|
||||
&blocks_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_store_index_fs_layout_format_id("block-",
|
||||
block_id,
|
||||
".asl",
|
||||
&name)) {
|
||||
free(blocks_path);
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = amduat_asl_store_index_fs_layout_join(blocks_path, name, out_path);
|
||||
free(name);
|
||||
free(blocks_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_snapshots_path(
|
||||
const char *root_path,
|
||||
char **out_path) {
|
||||
char *index_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
|
||||
&index_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(index_path, "snapshots", out_path);
|
||||
free(index_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_snapshot_manifest_path(
|
||||
const char *root_path,
|
||||
uint64_t snapshot_id,
|
||||
char **out_path) {
|
||||
char *snapshots_path;
|
||||
char *name;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_snapshots_path(root_path,
|
||||
&snapshots_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_store_index_fs_layout_format_id("snap-",
|
||||
snapshot_id,
|
||||
".bin",
|
||||
&name)) {
|
||||
free(snapshots_path);
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = amduat_asl_store_index_fs_layout_join(snapshots_path, name, out_path);
|
||||
free(name);
|
||||
free(snapshots_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_snapshot_latest_path(
|
||||
const char *root_path,
|
||||
char **out_path) {
|
||||
char *snapshots_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_snapshots_path(root_path,
|
||||
&snapshots_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(snapshots_path,
|
||||
"latest",
|
||||
out_path);
|
||||
free(snapshots_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segment_meta_path(
|
||||
const char *root_path,
|
||||
char **out_path) {
|
||||
char *segments_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
|
||||
&segments_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(segments_path,
|
||||
"next_id",
|
||||
out_path);
|
||||
free(segments_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segment_summary_path(
|
||||
const char *root_path,
|
||||
char **out_path) {
|
||||
char *segments_path;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
|
||||
&segments_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = amduat_asl_store_index_fs_layout_join(segments_path,
|
||||
"summary",
|
||||
out_path);
|
||||
free(segments_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segment_path(
|
||||
const char *root_path,
|
||||
uint64_t segment_id,
|
||||
char **out_path) {
|
||||
char *segments_path;
|
||||
char *name;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
|
||||
&segments_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_store_index_fs_layout_format_id("segment-",
|
||||
segment_id,
|
||||
".asl",
|
||||
&name)) {
|
||||
free(segments_path);
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = amduat_asl_store_index_fs_layout_join(segments_path, name, out_path);
|
||||
free(name);
|
||||
free(segments_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_block_path(
|
||||
const char *root_path,
|
||||
uint64_t block_id,
|
||||
char **out_path) {
|
||||
char *blocks_path;
|
||||
char *name;
|
||||
bool ok;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_blocks_path(root_path,
|
||||
&blocks_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_store_index_fs_layout_format_id("block-",
|
||||
block_id,
|
||||
".asl",
|
||||
&name)) {
|
||||
free(blocks_path);
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = amduat_asl_store_index_fs_layout_join(blocks_path, name, out_path);
|
||||
free(name);
|
||||
free(blocks_path);
|
||||
return ok;
|
||||
}
|
||||
95
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.h
Normal file
95
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.h
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
#ifndef AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H
|
||||
#define AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_index_path(const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segments_path(const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_blocks_path(const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_log_path(const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shards_path(
|
||||
const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_segments_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_blocks_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_segment_meta_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_segment_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
uint64_t segment_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_shard_block_path(
|
||||
const char *root_path,
|
||||
uint16_t shard_id,
|
||||
uint64_t block_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_snapshots_path(
|
||||
const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_snapshot_manifest_path(
|
||||
const char *root_path,
|
||||
uint64_t snapshot_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_snapshot_latest_path(
|
||||
const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segment_meta_path(
|
||||
const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segment_summary_path(
|
||||
const char *root_path,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_segment_path(
|
||||
const char *root_path,
|
||||
uint64_t segment_id,
|
||||
char **out_path);
|
||||
|
||||
bool amduat_asl_store_index_fs_layout_build_block_path(
|
||||
const char *root_path,
|
||||
uint64_t block_id,
|
||||
char **out_path);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
|
||||
#endif /* AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H */
|
||||
1699
src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c
Normal file
1699
src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c
Normal file
File diff suppressed because it is too large
Load diff
607
src/core/asl_collection.c
Normal file
607
src/core/asl_collection.c
Normal file
|
|
@ -0,0 +1,607 @@
|
|||
#include "amduat/asl/collection.h"
|
||||
|
||||
#include "amduat/enc/asl1_core_codec.h"
|
||||
#include "amduat/util/log.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_COLLECTION_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_COLLECTION_VERSION = 1,
|
||||
AMDUAT_ASL_COLLECTION_READ_BATCH = 1024u
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_collection_magic[
|
||||
AMDUAT_ASL_COLLECTION_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'C', 'O', 'L', '1', '\0'
|
||||
};
|
||||
|
||||
static void amduat_asl_collection_store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_collection_store_u64_le(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_collection_read_u32_le(const uint8_t *data,
|
||||
size_t len,
|
||||
size_t *offset,
|
||||
uint32_t *out) {
|
||||
if (len - *offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
*out = (uint32_t)data[*offset] |
|
||||
((uint32_t)data[*offset + 1u] << 8) |
|
||||
((uint32_t)data[*offset + 2u] << 16) |
|
||||
((uint32_t)data[*offset + 3u] << 24);
|
||||
*offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_collection_read_u64_le(const uint8_t *data,
|
||||
size_t len,
|
||||
size_t *offset,
|
||||
uint64_t *out) {
|
||||
if (len - *offset < 8u) {
|
||||
return false;
|
||||
}
|
||||
*out = (uint64_t)data[*offset] |
|
||||
((uint64_t)data[*offset + 1u] << 8) |
|
||||
((uint64_t)data[*offset + 2u] << 16) |
|
||||
((uint64_t)data[*offset + 3u] << 24) |
|
||||
((uint64_t)data[*offset + 4u] << 32) |
|
||||
((uint64_t)data[*offset + 5u] << 40) |
|
||||
((uint64_t)data[*offset + 6u] << 48) |
|
||||
((uint64_t)data[*offset + 7u] << 56);
|
||||
*offset += 8u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_collection_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_collection_build_log_name(const char *name,
|
||||
char **out_name) {
|
||||
size_t name_len;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (name == NULL || out_name == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(name)) {
|
||||
return false;
|
||||
}
|
||||
name_len = strlen(name);
|
||||
total_len = 11u + name_len + 4u + 1u;
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, "collection/", 11u);
|
||||
offset += 11u;
|
||||
memcpy(buffer + offset, name, name_len);
|
||||
offset += name_len;
|
||||
memcpy(buffer + offset, "/log", 4u);
|
||||
offset += 4u;
|
||||
buffer[offset] = '\0';
|
||||
*out_name = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_collection_build_head_name(const char *name,
|
||||
char **out_name) {
|
||||
size_t name_len;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (name == NULL || out_name == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(name)) {
|
||||
return false;
|
||||
}
|
||||
name_len = strlen(name);
|
||||
total_len = 11u + name_len + 5u + 1u;
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, "collection/", 11u);
|
||||
offset += 11u;
|
||||
memcpy(buffer + offset, name, name_len);
|
||||
offset += name_len;
|
||||
memcpy(buffer + offset, "/head", 5u);
|
||||
offset += 5u;
|
||||
buffer[offset] = '\0';
|
||||
*out_name = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_collection_encode_snapshot_payload(
|
||||
uint64_t snapshot_offset,
|
||||
const amduat_reference_t *refs,
|
||||
size_t refs_len,
|
||||
amduat_octets_t *out_payload) {
|
||||
size_t total_len = 0u;
|
||||
uint8_t *buffer;
|
||||
size_t offset = 0u;
|
||||
|
||||
if (out_payload == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_payload->data = NULL;
|
||||
out_payload->len = 0u;
|
||||
|
||||
if (refs_len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_collection_add_size(
|
||||
&total_len, AMDUAT_ASL_COLLECTION_MAGIC_LEN + 4u + 8u + 4u)) {
|
||||
return false;
|
||||
}
|
||||
for (size_t i = 0u; i < refs_len; ++i) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_collection_add_size(&total_len, 4u + ref_bytes.len)) {
|
||||
free((void *)ref_bytes.data);
|
||||
return false;
|
||||
}
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(buffer + offset, k_amduat_asl_collection_magic,
|
||||
AMDUAT_ASL_COLLECTION_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_COLLECTION_MAGIC_LEN;
|
||||
amduat_asl_collection_store_u32_le(buffer + offset,
|
||||
AMDUAT_ASL_COLLECTION_VERSION);
|
||||
offset += 4u;
|
||||
amduat_asl_collection_store_u64_le(buffer + offset, snapshot_offset);
|
||||
offset += 8u;
|
||||
amduat_asl_collection_store_u32_le(buffer + offset, (uint32_t)refs_len);
|
||||
offset += 4u;
|
||||
|
||||
for (size_t i = 0u; i < refs_len; ++i) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
amduat_asl_collection_store_u32_le(buffer + offset,
|
||||
(uint32_t)ref_bytes.len);
|
||||
offset += 4u;
|
||||
memcpy(buffer + offset, ref_bytes.data, ref_bytes.len);
|
||||
offset += ref_bytes.len;
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
|
||||
if (offset != total_len) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
*out_payload = amduat_octets(buffer, total_len);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_collection_store_init(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *root_path,
|
||||
amduat_asl_store_t *store) {
|
||||
if (collection_store == NULL || root_path == NULL || store == NULL) {
|
||||
return false;
|
||||
}
|
||||
collection_store->store = store;
|
||||
if (!amduat_asl_pointer_store_init(&collection_store->pointer_store,
|
||||
root_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_log_store_init(&collection_store->log_store, root_path,
|
||||
store, &collection_store->pointer_store)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
amduat_asl_collection_error_t amduat_asl_collection_append(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *collection_name,
|
||||
amduat_reference_t record_ref,
|
||||
uint16_t kind,
|
||||
amduat_octets_t actor,
|
||||
uint64_t *out_offset) {
|
||||
char *log_name = NULL;
|
||||
amduat_asl_log_entry_t entry;
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
if (collection_store == NULL || collection_store->store == NULL ||
|
||||
collection_name == NULL) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
|
||||
}
|
||||
if (!amduat_asl_collection_build_log_name(collection_name, &log_name)) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
|
||||
memset(&entry, 0, sizeof(entry));
|
||||
entry.kind = kind;
|
||||
entry.payload_ref = record_ref;
|
||||
entry.has_actor = actor.len != 0u;
|
||||
entry.actor = actor;
|
||||
|
||||
err = amduat_asl_log_append(&collection_store->log_store, log_name,
|
||||
&entry, 1u, out_offset);
|
||||
free(log_name);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
return AMDUAT_ASL_COLLECTION_OK;
|
||||
}
|
||||
|
||||
amduat_asl_collection_error_t amduat_asl_collection_snapshot(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *collection_name,
|
||||
uint64_t up_to_offset,
|
||||
amduat_reference_t *out_snapshot_ref,
|
||||
bool *out_swapped) {
|
||||
char *log_name = NULL;
|
||||
char *head_name = NULL;
|
||||
amduat_reference_t *refs = NULL;
|
||||
size_t refs_len = 0u;
|
||||
uint64_t from = 0u;
|
||||
bool end = false;
|
||||
uint64_t next_offset = 0u;
|
||||
amduat_asl_record_t snapshot_record;
|
||||
amduat_octets_t payload = amduat_octets(NULL, 0u);
|
||||
amduat_reference_t snapshot_ref;
|
||||
amduat_asl_pointer_error_t ptr_err;
|
||||
bool head_exists = false;
|
||||
amduat_reference_t head_ref;
|
||||
bool swapped = false;
|
||||
|
||||
if (out_snapshot_ref != NULL) {
|
||||
out_snapshot_ref->hash_id = 0u;
|
||||
out_snapshot_ref->digest = amduat_octets(NULL, 0u);
|
||||
}
|
||||
if (out_swapped != NULL) {
|
||||
*out_swapped = false;
|
||||
}
|
||||
if (collection_store == NULL || collection_store->store == NULL ||
|
||||
collection_name == NULL) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
|
||||
}
|
||||
|
||||
if (!amduat_asl_collection_build_log_name(collection_name, &log_name) ||
|
||||
!amduat_asl_collection_build_head_name(collection_name, &head_name)) {
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
|
||||
while (!end && (up_to_offset == UINT64_MAX || from <= up_to_offset)) {
|
||||
size_t limit = AMDUAT_ASL_COLLECTION_READ_BATCH;
|
||||
amduat_asl_log_entry_t *entries = NULL;
|
||||
size_t entries_len = 0u;
|
||||
amduat_asl_store_error_t err = amduat_asl_log_read(
|
||||
&collection_store->log_store, log_name, from, limit, &entries,
|
||||
&entries_len, &next_offset, &end);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
for (size_t i = 0u; i < entries_len; ++i) {
|
||||
uint64_t offset = from + i;
|
||||
if (up_to_offset != UINT64_MAX && offset > up_to_offset) {
|
||||
end = true;
|
||||
break;
|
||||
}
|
||||
amduat_reference_t *next =
|
||||
(amduat_reference_t *)realloc(
|
||||
refs, (refs_len + 1u) * sizeof(*refs));
|
||||
if (next == NULL) {
|
||||
amduat_asl_log_entries_free(entries, entries_len);
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
refs = next;
|
||||
refs[refs_len] = entries[i].payload_ref;
|
||||
if (!amduat_octets_clone(entries[i].payload_ref.digest,
|
||||
&refs[refs_len].digest)) {
|
||||
amduat_asl_log_entries_free(entries, entries_len);
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
refs_len++;
|
||||
}
|
||||
amduat_asl_log_entries_free(entries, entries_len);
|
||||
from = next_offset;
|
||||
if (entries_len == 0u) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
uint64_t snapshot_offset = from;
|
||||
if (up_to_offset != UINT64_MAX &&
|
||||
up_to_offset < UINT64_MAX &&
|
||||
snapshot_offset > up_to_offset + 1u) {
|
||||
snapshot_offset = up_to_offset + 1u;
|
||||
}
|
||||
if (!amduat_asl_collection_encode_snapshot_payload(
|
||||
snapshot_offset, refs, refs_len, &payload)) {
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
|
||||
}
|
||||
}
|
||||
|
||||
if (payload.len == 0u || payload.data == NULL) {
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
{
|
||||
amduat_asl_store_error_t err = amduat_asl_record_store_put(
|
||||
collection_store->store,
|
||||
amduat_octets("collection/snapshot",
|
||||
strlen("collection/snapshot")),
|
||||
payload,
|
||||
&snapshot_ref);
|
||||
amduat_octets_free(&payload);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
}
|
||||
|
||||
ptr_err = amduat_asl_pointer_get(&collection_store->pointer_store, head_name,
|
||||
&head_exists, &head_ref);
|
||||
if (ptr_err != AMDUAT_ASL_POINTER_OK) {
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
ptr_err = amduat_asl_pointer_cas(&collection_store->pointer_store, head_name,
|
||||
head_exists,
|
||||
head_exists ? &head_ref : NULL,
|
||||
&snapshot_ref, &swapped);
|
||||
if (head_exists) {
|
||||
amduat_reference_free(&head_ref);
|
||||
}
|
||||
if (ptr_err != AMDUAT_ASL_POINTER_OK) {
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
|
||||
if (out_swapped != NULL) {
|
||||
*out_swapped = swapped;
|
||||
}
|
||||
if (out_snapshot_ref != NULL) {
|
||||
*out_snapshot_ref = snapshot_ref;
|
||||
}
|
||||
if (!swapped) {
|
||||
amduat_log(AMDUAT_LOG_DEBUG, "collection snapshot CAS mismatch");
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_CAS_MISMATCH;
|
||||
}
|
||||
|
||||
amduat_asl_collection_refs_free(refs, refs_len);
|
||||
free(log_name);
|
||||
free(head_name);
|
||||
return AMDUAT_ASL_COLLECTION_OK;
|
||||
}
|
||||
|
||||
amduat_asl_collection_error_t amduat_asl_collection_read(
|
||||
amduat_asl_collection_store_t *collection_store,
|
||||
const char *collection_name,
|
||||
uint64_t from_offset,
|
||||
size_t limit,
|
||||
amduat_reference_t **out_record_refs,
|
||||
size_t *out_len,
|
||||
uint64_t *out_next_offset,
|
||||
bool *out_end) {
|
||||
char *log_name = NULL;
|
||||
amduat_asl_log_entry_t *entries = NULL;
|
||||
size_t entries_len = 0u;
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
if (out_record_refs == NULL || out_len == NULL ||
|
||||
out_next_offset == NULL || out_end == NULL) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
|
||||
}
|
||||
*out_record_refs = NULL;
|
||||
*out_len = 0u;
|
||||
|
||||
if (collection_store == NULL || collection_store->store == NULL ||
|
||||
collection_name == NULL) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
|
||||
}
|
||||
if (!amduat_asl_collection_build_log_name(collection_name, &log_name)) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
|
||||
err = amduat_asl_log_read(&collection_store->log_store, log_name,
|
||||
from_offset, limit, &entries, &entries_len,
|
||||
out_next_offset, out_end);
|
||||
free(log_name);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
|
||||
if (entries_len != 0u) {
|
||||
amduat_reference_t *refs =
|
||||
(amduat_reference_t *)calloc(entries_len, sizeof(*refs));
|
||||
if (refs == NULL) {
|
||||
amduat_asl_log_entries_free(entries, entries_len);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
for (size_t i = 0u; i < entries_len; ++i) {
|
||||
refs[i].hash_id = entries[i].payload_ref.hash_id;
|
||||
if (!amduat_octets_clone(entries[i].payload_ref.digest,
|
||||
&refs[i].digest)) {
|
||||
amduat_asl_collection_refs_free(refs, i);
|
||||
amduat_asl_log_entries_free(entries, entries_len);
|
||||
return AMDUAT_ASL_COLLECTION_ERR_IO;
|
||||
}
|
||||
}
|
||||
*out_record_refs = refs;
|
||||
*out_len = entries_len;
|
||||
}
|
||||
|
||||
amduat_asl_log_entries_free(entries, entries_len);
|
||||
return AMDUAT_ASL_COLLECTION_OK;
|
||||
}
|
||||
|
||||
void amduat_asl_collection_refs_free(amduat_reference_t *refs,
|
||||
size_t refs_len) {
|
||||
if (refs == NULL) {
|
||||
return;
|
||||
}
|
||||
for (size_t i = 0u; i < refs_len; ++i) {
|
||||
amduat_reference_free(&refs[i]);
|
||||
}
|
||||
free(refs);
|
||||
}
|
||||
|
||||
bool amduat_asl_collection_snapshot_payload_decode_v1(
|
||||
amduat_octets_t payload,
|
||||
amduat_asl_collection_snapshot_payload_t *out_snapshot) {
|
||||
size_t offset = 0u;
|
||||
uint32_t version = 0u;
|
||||
uint32_t entry_count = 0u;
|
||||
|
||||
if (out_snapshot == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_snapshot->snapshot_offset = 0u;
|
||||
out_snapshot->refs = NULL;
|
||||
out_snapshot->refs_len = 0u;
|
||||
|
||||
if (payload.len < AMDUAT_ASL_COLLECTION_MAGIC_LEN + 4u + 8u + 4u) {
|
||||
return false;
|
||||
}
|
||||
if (payload.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (memcmp(payload.data, k_amduat_asl_collection_magic,
|
||||
AMDUAT_ASL_COLLECTION_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
offset += AMDUAT_ASL_COLLECTION_MAGIC_LEN;
|
||||
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
|
||||
&offset, &version) ||
|
||||
version != AMDUAT_ASL_COLLECTION_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_collection_read_u64_le(payload.data, payload.len,
|
||||
&offset,
|
||||
&out_snapshot->snapshot_offset)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
|
||||
&offset, &entry_count)) {
|
||||
return false;
|
||||
}
|
||||
if (entry_count == 0u) {
|
||||
return offset == payload.len;
|
||||
}
|
||||
if (entry_count > SIZE_MAX / sizeof(amduat_reference_t)) {
|
||||
return false;
|
||||
}
|
||||
out_snapshot->refs =
|
||||
(amduat_reference_t *)calloc(entry_count, sizeof(*out_snapshot->refs));
|
||||
if (out_snapshot->refs == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_snapshot->refs_len = entry_count;
|
||||
|
||||
for (uint32_t i = 0u; i < entry_count; ++i) {
|
||||
uint32_t ref_len = 0u;
|
||||
amduat_octets_t ref_bytes;
|
||||
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
|
||||
&offset, &ref_len)) {
|
||||
goto fail;
|
||||
}
|
||||
if (payload.len - offset < ref_len) {
|
||||
goto fail;
|
||||
}
|
||||
ref_bytes = amduat_octets(payload.data + offset, ref_len);
|
||||
if (!amduat_enc_asl1_core_decode_reference_v1(
|
||||
ref_bytes, &out_snapshot->refs[i])) {
|
||||
goto fail;
|
||||
}
|
||||
offset += ref_len;
|
||||
}
|
||||
|
||||
return offset == payload.len;
|
||||
|
||||
fail:
|
||||
amduat_asl_collection_snapshot_payload_free(out_snapshot);
|
||||
return false;
|
||||
}
|
||||
|
||||
void amduat_asl_collection_snapshot_payload_free(
|
||||
amduat_asl_collection_snapshot_payload_t *snapshot) {
|
||||
if (snapshot == NULL || snapshot->refs == NULL) {
|
||||
return;
|
||||
}
|
||||
for (size_t i = 0u; i < snapshot->refs_len; ++i) {
|
||||
amduat_reference_free(&snapshot->refs[i]);
|
||||
}
|
||||
free(snapshot->refs);
|
||||
snapshot->refs = NULL;
|
||||
snapshot->refs_len = 0u;
|
||||
}
|
||||
1021
src/core/asl_log_store.c
Normal file
1021
src/core/asl_log_store.c
Normal file
File diff suppressed because it is too large
Load diff
268
src/core/asl_record.c
Normal file
268
src/core/asl_record.c
Normal file
|
|
@ -0,0 +1,268 @@
|
|||
#include "amduat/asl/record.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_RECORD_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_RECORD_VERSION = 1
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_record_magic[AMDUAT_ASL_RECORD_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'R', 'E', 'C', '1', '\0'
|
||||
};
|
||||
|
||||
static void amduat_asl_record_store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_record_read_u32_le(const uint8_t *data,
|
||||
size_t len,
|
||||
size_t *offset,
|
||||
uint32_t *out) {
|
||||
if (len - *offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
*out = (uint32_t)data[*offset] |
|
||||
((uint32_t)data[*offset + 1u] << 8) |
|
||||
((uint32_t)data[*offset + 2u] << 16) |
|
||||
((uint32_t)data[*offset + 3u] << 24);
|
||||
*offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_record_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_record_schema_is_ascii(amduat_octets_t schema) {
|
||||
if (schema.len == 0u || schema.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
for (size_t i = 0u; i < schema.len; ++i) {
|
||||
uint8_t c = schema.data[i];
|
||||
if (c < 0x20u || c > 0x7eu) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_record_encode_v1(const amduat_asl_record_t *record,
|
||||
amduat_octets_t *out_bytes) {
|
||||
uint8_t *buffer;
|
||||
size_t total_len = 0u;
|
||||
size_t offset = 0u;
|
||||
uint32_t schema_len;
|
||||
uint32_t payload_len;
|
||||
uint8_t flags = 0u;
|
||||
|
||||
if (out_bytes == NULL || record == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0u;
|
||||
|
||||
if (!amduat_asl_record_schema_is_ascii(record->schema)) {
|
||||
return false;
|
||||
}
|
||||
if (record->payload.len != 0u && record->payload.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (record->schema.len > UINT32_MAX || record->payload.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
schema_len = (uint32_t)record->schema.len;
|
||||
payload_len = (uint32_t)record->payload.len;
|
||||
|
||||
if (!amduat_asl_record_add_size(&total_len,
|
||||
AMDUAT_ASL_RECORD_MAGIC_LEN + 4u) ||
|
||||
!amduat_asl_record_add_size(&total_len, 4u + schema_len) ||
|
||||
!amduat_asl_record_add_size(&total_len, 4u + payload_len) ||
|
||||
!amduat_asl_record_add_size(&total_len, 1u)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(buffer + offset, k_amduat_asl_record_magic,
|
||||
AMDUAT_ASL_RECORD_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_RECORD_MAGIC_LEN;
|
||||
amduat_asl_record_store_u32_le(buffer + offset, AMDUAT_ASL_RECORD_VERSION);
|
||||
offset += 4u;
|
||||
|
||||
amduat_asl_record_store_u32_le(buffer + offset, schema_len);
|
||||
offset += 4u;
|
||||
memcpy(buffer + offset, record->schema.data, schema_len);
|
||||
offset += schema_len;
|
||||
|
||||
amduat_asl_record_store_u32_le(buffer + offset, payload_len);
|
||||
offset += 4u;
|
||||
if (payload_len != 0u) {
|
||||
memcpy(buffer + offset, record->payload.data, payload_len);
|
||||
offset += payload_len;
|
||||
}
|
||||
|
||||
buffer[offset++] = flags;
|
||||
|
||||
if (offset != total_len) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = total_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_record_decode_v1(amduat_octets_t bytes,
|
||||
amduat_asl_record_t *out_record) {
|
||||
size_t offset = 0u;
|
||||
uint32_t version;
|
||||
uint32_t schema_len;
|
||||
uint32_t payload_len;
|
||||
uint8_t flags;
|
||||
uint8_t *schema_bytes;
|
||||
uint8_t *payload_bytes;
|
||||
|
||||
if (out_record == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_record->schema = amduat_octets(NULL, 0u);
|
||||
out_record->payload = amduat_octets(NULL, 0u);
|
||||
|
||||
if (bytes.len < AMDUAT_ASL_RECORD_MAGIC_LEN + 4u + 1u) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (memcmp(bytes.data, k_amduat_asl_record_magic,
|
||||
AMDUAT_ASL_RECORD_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
offset += AMDUAT_ASL_RECORD_MAGIC_LEN;
|
||||
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
|
||||
&version) ||
|
||||
version != AMDUAT_ASL_RECORD_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
|
||||
&schema_len)) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len - offset < schema_len) {
|
||||
return false;
|
||||
}
|
||||
schema_bytes = NULL;
|
||||
if (schema_len != 0u) {
|
||||
schema_bytes = (uint8_t *)malloc(schema_len);
|
||||
if (schema_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
memcpy(schema_bytes, bytes.data + offset, schema_len);
|
||||
}
|
||||
offset += schema_len;
|
||||
|
||||
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
|
||||
&payload_len)) {
|
||||
free(schema_bytes);
|
||||
return false;
|
||||
}
|
||||
if (bytes.len - offset < payload_len + 1u) {
|
||||
free(schema_bytes);
|
||||
return false;
|
||||
}
|
||||
payload_bytes = NULL;
|
||||
if (payload_len != 0u) {
|
||||
payload_bytes = (uint8_t *)malloc(payload_len);
|
||||
if (payload_bytes == NULL) {
|
||||
free(schema_bytes);
|
||||
return false;
|
||||
}
|
||||
memcpy(payload_bytes, bytes.data + offset, payload_len);
|
||||
}
|
||||
offset += payload_len;
|
||||
flags = bytes.data[offset++];
|
||||
(void)flags;
|
||||
|
||||
if (offset != bytes.len) {
|
||||
free(schema_bytes);
|
||||
free(payload_bytes);
|
||||
return false;
|
||||
}
|
||||
|
||||
out_record->schema = amduat_octets(schema_bytes, schema_len);
|
||||
out_record->payload = amduat_octets(payload_bytes, payload_len);
|
||||
return true;
|
||||
}
|
||||
|
||||
void amduat_asl_record_free(amduat_asl_record_t *record) {
|
||||
if (record == NULL) {
|
||||
return;
|
||||
}
|
||||
amduat_octets_free(&record->schema);
|
||||
amduat_octets_free(&record->payload);
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_record_store_put(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_octets_t schema,
|
||||
amduat_octets_t payload,
|
||||
amduat_reference_t *out_ref) {
|
||||
amduat_asl_record_t record;
|
||||
amduat_octets_t encoded = amduat_octets(NULL, 0u);
|
||||
amduat_artifact_t artifact;
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
if (store == NULL || out_ref == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
record.schema = schema;
|
||||
record.payload = payload;
|
||||
if (!amduat_asl_record_encode_v1(&record, &encoded)) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
artifact = amduat_artifact_with_type(
|
||||
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_RECORD_1));
|
||||
err = amduat_asl_store_put(store, artifact, out_ref);
|
||||
free((void *)encoded.data);
|
||||
return err;
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_record_store_get(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_record_t *out_record) {
|
||||
amduat_artifact_t artifact;
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
if (store == NULL || out_record == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
err = amduat_asl_store_get(store, ref, &artifact);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
return err;
|
||||
}
|
||||
if (!artifact.has_type_tag ||
|
||||
artifact.type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
|
||||
amduat_artifact_free(&artifact);
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
if (!amduat_asl_record_decode_v1(artifact.bytes, out_record)) {
|
||||
amduat_artifact_free(&artifact);
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
amduat_artifact_free(&artifact);
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
161
src/core/derivation_sid.c
Normal file
161
src/core/derivation_sid.c
Normal file
|
|
@ -0,0 +1,161 @@
|
|||
#include "amduat/pel/derivation_sid.h"
|
||||
|
||||
#include "amduat/enc/asl1_core_codec.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static bool amduat_derivation_sid_check_ref(amduat_reference_t ref) {
|
||||
const amduat_hash_asl1_desc_t *desc;
|
||||
size_t digest_len;
|
||||
|
||||
if (ref.digest.len != 0 && ref.digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (amduat_hash_asl1_is_reserved(ref.hash_id)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
desc = amduat_hash_asl1_desc_lookup(ref.hash_id);
|
||||
digest_len = ref.digest.len;
|
||||
if (desc != NULL && desc->digest_len != 0) {
|
||||
assert(desc->digest_len == digest_len);
|
||||
if (desc->digest_len != digest_len) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_pel_derivation_sid_compute(
|
||||
const amduat_pel_derivation_sid_input_t *in,
|
||||
amduat_octets_t *out_sid) {
|
||||
amduat_hash_asl1_stream_t stream;
|
||||
uint8_t marker;
|
||||
size_t i;
|
||||
|
||||
if (out_sid == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_sid->data = NULL;
|
||||
out_sid->len = 0;
|
||||
|
||||
if (in == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (in->input_refs_len != 0 && in->input_refs == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (in->has_params_ref && !amduat_derivation_sid_check_ref(in->params_ref)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_derivation_sid_check_ref(in->program_ref)) {
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < in->input_refs_len; ++i) {
|
||||
if (!amduat_derivation_sid_check_ref(in->input_refs[i])) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
{
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
bool ok;
|
||||
|
||||
/* ReferenceBytes are self-delimiting because hash_id implies digest_len. */
|
||||
ok = amduat_enc_asl1_core_encode_reference_v1(in->program_ref, &ref_bytes);
|
||||
if (!ok ||
|
||||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
free((void *)ref_bytes.data);
|
||||
return false;
|
||||
}
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
|
||||
for (i = 0; i < in->input_refs_len; ++i) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
bool ok = amduat_enc_asl1_core_encode_reference_v1(in->input_refs[i],
|
||||
&ref_bytes);
|
||||
if (!ok ||
|
||||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
free((void *)ref_bytes.data);
|
||||
return false;
|
||||
}
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
|
||||
marker = in->has_params_ref ? 0x01u : 0x00u;
|
||||
if (!amduat_hash_asl1_stream_update(&stream,
|
||||
amduat_octets(&marker, 1u))) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
if (in->has_params_ref) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
bool ok = amduat_enc_asl1_core_encode_reference_v1(in->params_ref,
|
||||
&ref_bytes);
|
||||
if (!ok ||
|
||||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
free((void *)ref_bytes.data);
|
||||
return false;
|
||||
}
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
|
||||
/* ExecProfile is currently absent; presence must be explicitly marked. */
|
||||
marker = in->has_exec_profile ? 0x01u : 0x00u;
|
||||
if (!amduat_hash_asl1_stream_update(&stream,
|
||||
amduat_octets(&marker, 1u))) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
if (in->has_exec_profile) {
|
||||
if (in->exec_profile.len != 0 && in->exec_profile.data == NULL) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
if (!amduat_hash_asl1_stream_update(&stream, in->exec_profile)) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
const amduat_hash_asl1_desc_t *desc =
|
||||
amduat_hash_asl1_desc_lookup(AMDUAT_HASH_ASL1_ID_SHA256);
|
||||
uint8_t *digest;
|
||||
size_t digest_len;
|
||||
|
||||
if (desc == NULL || desc->digest_len == 0u) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
digest_len = desc->digest_len;
|
||||
digest = (uint8_t *)malloc(digest_len);
|
||||
if (digest == NULL) {
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
if (!amduat_hash_asl1_stream_final(&stream, digest, digest_len)) {
|
||||
free(digest);
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return false;
|
||||
}
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
*out_sid = amduat_octets(digest, digest_len);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
41
src/internal/log.c
Normal file
41
src/internal/log.c
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
#include "amduat/util/log.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdio.h>
|
||||
|
||||
static const char *amduat_log_level_label(amduat_log_level_t level) {
|
||||
switch (level) {
|
||||
case AMDUAT_LOG_ERROR:
|
||||
return "ERROR";
|
||||
case AMDUAT_LOG_WARN:
|
||||
return "WARN";
|
||||
case AMDUAT_LOG_INFO:
|
||||
return "INFO";
|
||||
case AMDUAT_LOG_DEBUG:
|
||||
return "DEBUG";
|
||||
default:
|
||||
return "LOG";
|
||||
}
|
||||
}
|
||||
|
||||
static bool amduat_log_debug_enabled(void) {
|
||||
#ifdef AMDUAT_LOG_ENABLE_DEBUG
|
||||
return true;
|
||||
#else
|
||||
return false;
|
||||
#endif
|
||||
}
|
||||
|
||||
void amduat_log(amduat_log_level_t level, const char *fmt, ...) {
|
||||
va_list ap;
|
||||
|
||||
if (level == AMDUAT_LOG_DEBUG && !amduat_log_debug_enabled()) {
|
||||
return;
|
||||
}
|
||||
|
||||
fprintf(stderr, "%s: ", amduat_log_level_label(level));
|
||||
va_start(ap, fmt);
|
||||
vfprintf(stderr, fmt, ap);
|
||||
va_end(ap);
|
||||
fputc('\n', stderr);
|
||||
}
|
||||
520
src/near_core/asl/collection_view.c
Normal file
520
src/near_core/asl/collection_view.c
Normal file
|
|
@ -0,0 +1,520 @@
|
|||
#include "amduat/asl/collection_view.h"
|
||||
|
||||
#include "amduat/enc/asl1_core_codec.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_COLLECTION_VIEW_VERSION = 1,
|
||||
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_SNAPSHOT_INFO_VERSION = 1,
|
||||
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_LOG_RANGE_VERSION = 1
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_collection_view_magic[
|
||||
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'C', 'V', 'W', '1', '\0'
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_snapshot_info_magic[
|
||||
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'S', 'N', 'P', '1', '\0'
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_log_range_magic[
|
||||
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'R', 'N', 'G', '1', '\0'
|
||||
};
|
||||
|
||||
static void amduat_asl_store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_store_u64_le(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_read_u32_le(const uint8_t *data,
|
||||
size_t len,
|
||||
size_t *offset,
|
||||
uint32_t *out) {
|
||||
if (len - *offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
*out = (uint32_t)data[*offset] |
|
||||
((uint32_t)data[*offset + 1u] << 8) |
|
||||
((uint32_t)data[*offset + 2u] << 16) |
|
||||
((uint32_t)data[*offset + 3u] << 24);
|
||||
*offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_read_u64_le(const uint8_t *data,
|
||||
size_t len,
|
||||
size_t *offset,
|
||||
uint64_t *out) {
|
||||
if (len - *offset < 8u) {
|
||||
return false;
|
||||
}
|
||||
*out = (uint64_t)data[*offset] |
|
||||
((uint64_t)data[*offset + 1u] << 8) |
|
||||
((uint64_t)data[*offset + 2u] << 16) |
|
||||
((uint64_t)data[*offset + 3u] << 24) |
|
||||
((uint64_t)data[*offset + 4u] << 32) |
|
||||
((uint64_t)data[*offset + 5u] << 40) |
|
||||
((uint64_t)data[*offset + 6u] << 48) |
|
||||
((uint64_t)data[*offset + 7u] << 56);
|
||||
*offset += 8u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_encode_refs(const amduat_reference_t *refs,
|
||||
size_t refs_len,
|
||||
size_t *out_total_len) {
|
||||
size_t total = 0u;
|
||||
|
||||
if (refs_len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
for (size_t i = 0u; i < refs_len; ++i) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_add_size(&total, 4u + ref_bytes.len)) {
|
||||
free((void *)ref_bytes.data);
|
||||
return false;
|
||||
}
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
*out_total_len = total;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_write_refs(uint8_t *buffer,
|
||||
size_t total_len,
|
||||
size_t *offset,
|
||||
const amduat_reference_t *refs,
|
||||
size_t refs_len) {
|
||||
for (size_t i = 0u; i < refs_len; ++i) {
|
||||
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
|
||||
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
|
||||
return false;
|
||||
}
|
||||
if (total_len - *offset < 4u + ref_bytes.len) {
|
||||
free((void *)ref_bytes.data);
|
||||
return false;
|
||||
}
|
||||
amduat_asl_store_u32_le(buffer + *offset, (uint32_t)ref_bytes.len);
|
||||
*offset += 4u;
|
||||
memcpy(buffer + *offset, ref_bytes.data, ref_bytes.len);
|
||||
*offset += ref_bytes.len;
|
||||
free((void *)ref_bytes.data);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_decode_refs(amduat_octets_t bytes,
|
||||
size_t *offset,
|
||||
uint32_t refs_len,
|
||||
amduat_reference_t **out_refs,
|
||||
size_t *out_refs_len) {
|
||||
amduat_reference_t *refs = NULL;
|
||||
|
||||
if (out_refs == NULL || out_refs_len == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_refs = NULL;
|
||||
*out_refs_len = 0u;
|
||||
|
||||
if (refs_len == 0u) {
|
||||
return true;
|
||||
}
|
||||
if (refs_len > SIZE_MAX / sizeof(*refs)) {
|
||||
return false;
|
||||
}
|
||||
refs = (amduat_reference_t *)calloc(refs_len, sizeof(*refs));
|
||||
if (refs == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (uint32_t i = 0u; i < refs_len; ++i) {
|
||||
uint32_t ref_len = 0u;
|
||||
amduat_octets_t ref_bytes;
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, offset, &ref_len)) {
|
||||
goto fail;
|
||||
}
|
||||
if (bytes.len - *offset < ref_len) {
|
||||
goto fail;
|
||||
}
|
||||
ref_bytes = amduat_octets(bytes.data + *offset, ref_len);
|
||||
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, &refs[i])) {
|
||||
goto fail;
|
||||
}
|
||||
*offset += ref_len;
|
||||
}
|
||||
|
||||
*out_refs = refs;
|
||||
*out_refs_len = refs_len;
|
||||
return true;
|
||||
|
||||
fail:
|
||||
for (uint32_t i = 0u; i < refs_len; ++i) {
|
||||
amduat_reference_free(&refs[i]);
|
||||
}
|
||||
free(refs);
|
||||
return false;
|
||||
}
|
||||
|
||||
bool amduat_asl_collection_snapshot_info_encode_v1(
|
||||
const amduat_asl_collection_snapshot_info_t *info,
|
||||
amduat_octets_t *out_bytes) {
|
||||
uint8_t *buffer;
|
||||
size_t total_len = 0u;
|
||||
size_t refs_len_bytes = 0u;
|
||||
size_t offset = 0u;
|
||||
|
||||
if (out_bytes == NULL || info == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0u;
|
||||
|
||||
if (!amduat_asl_add_size(&total_len,
|
||||
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN + 4u + 8u + 4u)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_encode_refs(info->refs, info->refs_len, &refs_len_bytes)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(buffer + offset, k_amduat_asl_snapshot_info_magic,
|
||||
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN;
|
||||
amduat_asl_store_u32_le(buffer + offset, AMDUAT_ASL_SNAPSHOT_INFO_VERSION);
|
||||
offset += 4u;
|
||||
amduat_asl_store_u64_le(buffer + offset, info->snapshot_at_offset);
|
||||
offset += 8u;
|
||||
amduat_asl_store_u32_le(buffer + offset, (uint32_t)info->refs_len);
|
||||
offset += 4u;
|
||||
if (!amduat_asl_write_refs(buffer, total_len, &offset,
|
||||
info->refs, info->refs_len)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (offset != total_len) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
*out_bytes = amduat_octets(buffer, total_len);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_collection_snapshot_info_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_collection_snapshot_info_t *out_info) {
|
||||
size_t offset = 0u;
|
||||
uint32_t version = 0u;
|
||||
uint32_t refs_len = 0u;
|
||||
|
||||
if (out_info == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_info->snapshot_at_offset = 0u;
|
||||
out_info->refs = NULL;
|
||||
out_info->refs_len = 0u;
|
||||
|
||||
if (bytes.len < AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN + 4u + 8u + 4u ||
|
||||
bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (memcmp(bytes.data, k_amduat_asl_snapshot_info_magic,
|
||||
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
offset += AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN;
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
|
||||
version != AMDUAT_ASL_SNAPSHOT_INFO_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
|
||||
&out_info->snapshot_at_offset)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
|
||||
&out_info->refs, &out_info->refs_len)) {
|
||||
return false;
|
||||
}
|
||||
return offset == bytes.len;
|
||||
}
|
||||
|
||||
void amduat_asl_collection_snapshot_info_free(
|
||||
amduat_asl_collection_snapshot_info_t *info) {
|
||||
if (info == NULL || info->refs == NULL) {
|
||||
return;
|
||||
}
|
||||
for (size_t i = 0u; i < info->refs_len; ++i) {
|
||||
amduat_reference_free(&info->refs[i]);
|
||||
}
|
||||
free(info->refs);
|
||||
info->refs = NULL;
|
||||
info->refs_len = 0u;
|
||||
}
|
||||
|
||||
bool amduat_asl_log_range_encode_v1(const amduat_asl_log_range_t *range,
|
||||
amduat_octets_t *out_bytes) {
|
||||
uint8_t *buffer;
|
||||
size_t total_len = 0u;
|
||||
size_t refs_len_bytes = 0u;
|
||||
size_t offset = 0u;
|
||||
|
||||
if (out_bytes == NULL || range == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0u;
|
||||
|
||||
if (!amduat_asl_add_size(&total_len,
|
||||
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN + 4u + 8u + 8u + 4u)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_encode_refs(range->refs, range->refs_len, &refs_len_bytes)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(buffer + offset, k_amduat_asl_log_range_magic,
|
||||
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_LOG_RANGE_MAGIC_LEN;
|
||||
amduat_asl_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_RANGE_VERSION);
|
||||
offset += 4u;
|
||||
amduat_asl_store_u64_le(buffer + offset, range->start_offset);
|
||||
offset += 8u;
|
||||
amduat_asl_store_u64_le(buffer + offset, range->next_offset);
|
||||
offset += 8u;
|
||||
amduat_asl_store_u32_le(buffer + offset, (uint32_t)range->refs_len);
|
||||
offset += 4u;
|
||||
if (!amduat_asl_write_refs(buffer, total_len, &offset,
|
||||
range->refs, range->refs_len)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (offset != total_len) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
*out_bytes = amduat_octets(buffer, total_len);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_log_range_decode_v1(amduat_octets_t bytes,
|
||||
amduat_asl_log_range_t *out_range) {
|
||||
size_t offset = 0u;
|
||||
uint32_t version = 0u;
|
||||
uint32_t refs_len = 0u;
|
||||
|
||||
if (out_range == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_range->start_offset = 0u;
|
||||
out_range->next_offset = 0u;
|
||||
out_range->refs = NULL;
|
||||
out_range->refs_len = 0u;
|
||||
|
||||
if (bytes.len <
|
||||
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN + 4u + 8u + 8u + 4u ||
|
||||
bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (memcmp(bytes.data, k_amduat_asl_log_range_magic,
|
||||
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
offset += AMDUAT_ASL_LOG_RANGE_MAGIC_LEN;
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
|
||||
version != AMDUAT_ASL_LOG_RANGE_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
|
||||
&out_range->start_offset) ||
|
||||
!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
|
||||
&out_range->next_offset)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
|
||||
&out_range->refs, &out_range->refs_len)) {
|
||||
return false;
|
||||
}
|
||||
return offset == bytes.len;
|
||||
}
|
||||
|
||||
void amduat_asl_log_range_free(amduat_asl_log_range_t *range) {
|
||||
if (range == NULL || range->refs == NULL) {
|
||||
return;
|
||||
}
|
||||
for (size_t i = 0u; i < range->refs_len; ++i) {
|
||||
amduat_reference_free(&range->refs[i]);
|
||||
}
|
||||
free(range->refs);
|
||||
range->refs = NULL;
|
||||
range->refs_len = 0u;
|
||||
}
|
||||
|
||||
bool amduat_asl_collection_view_encode_v1(
|
||||
const amduat_asl_collection_view_t *view,
|
||||
amduat_octets_t *out_bytes) {
|
||||
uint8_t *buffer;
|
||||
size_t total_len = 0u;
|
||||
size_t refs_len_bytes = 0u;
|
||||
size_t offset = 0u;
|
||||
|
||||
if (out_bytes == NULL || view == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0u;
|
||||
|
||||
if (!amduat_asl_add_size(&total_len,
|
||||
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN + 4u +
|
||||
8u + 8u + 4u)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_encode_refs(view->refs, view->refs_len, &refs_len_bytes)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(buffer + offset, k_amduat_asl_collection_view_magic,
|
||||
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN;
|
||||
amduat_asl_store_u32_le(buffer + offset,
|
||||
AMDUAT_ASL_COLLECTION_VIEW_VERSION);
|
||||
offset += 4u;
|
||||
amduat_asl_store_u64_le(buffer + offset, view->computed_from_offset);
|
||||
offset += 8u;
|
||||
amduat_asl_store_u64_le(buffer + offset, view->computed_up_to_offset);
|
||||
offset += 8u;
|
||||
amduat_asl_store_u32_le(buffer + offset, (uint32_t)view->refs_len);
|
||||
offset += 4u;
|
||||
if (!amduat_asl_write_refs(buffer, total_len, &offset,
|
||||
view->refs, view->refs_len)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (offset != total_len) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
*out_bytes = amduat_octets(buffer, total_len);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_collection_view_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_collection_view_t *out_view) {
|
||||
size_t offset = 0u;
|
||||
uint32_t version = 0u;
|
||||
uint32_t refs_len = 0u;
|
||||
|
||||
if (out_view == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_view->computed_from_offset = 0u;
|
||||
out_view->computed_up_to_offset = 0u;
|
||||
out_view->refs = NULL;
|
||||
out_view->refs_len = 0u;
|
||||
|
||||
if (bytes.len <
|
||||
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN + 4u + 8u + 8u + 4u ||
|
||||
bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (memcmp(bytes.data, k_amduat_asl_collection_view_magic,
|
||||
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
offset += AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN;
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
|
||||
version != AMDUAT_ASL_COLLECTION_VIEW_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
|
||||
&out_view->computed_from_offset) ||
|
||||
!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
|
||||
&out_view->computed_up_to_offset)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
|
||||
&out_view->refs, &out_view->refs_len)) {
|
||||
return false;
|
||||
}
|
||||
return offset == bytes.len;
|
||||
}
|
||||
|
||||
void amduat_asl_collection_view_free(amduat_asl_collection_view_t *view) {
|
||||
if (view == NULL || view->refs == NULL) {
|
||||
return;
|
||||
}
|
||||
for (size_t i = 0u; i < view->refs_len; ++i) {
|
||||
amduat_reference_free(&view->refs[i]);
|
||||
}
|
||||
free(view->refs);
|
||||
view->refs = NULL;
|
||||
view->refs_len = 0u;
|
||||
}
|
||||
158
src/near_core/asl/index_accel.c
Normal file
158
src/near_core/asl/index_accel.c
Normal file
|
|
@ -0,0 +1,158 @@
|
|||
#include "amduat/asl/index_accel.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE = 4,
|
||||
AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE = 5
|
||||
};
|
||||
|
||||
static uint64_t amduat_asl_index_accel_fnv1a64_update(uint64_t hash,
|
||||
const uint8_t *data,
|
||||
size_t len) {
|
||||
size_t i;
|
||||
|
||||
if (data == NULL || len == 0u) {
|
||||
return hash;
|
||||
}
|
||||
for (i = 0u; i < len; ++i) {
|
||||
hash ^= data[i];
|
||||
hash *= 1099511628211ull;
|
||||
}
|
||||
return hash;
|
||||
}
|
||||
|
||||
static void amduat_asl_index_accel_store_u16_le(uint8_t *out,
|
||||
uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_index_accel_store_u32_le(uint8_t *out,
|
||||
uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_index_accel_hash_ref(
|
||||
amduat_reference_t ref,
|
||||
bool has_type_tag,
|
||||
amduat_type_tag_t type_tag,
|
||||
uint64_t *out_hash) {
|
||||
uint8_t header[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE];
|
||||
uint8_t tag_buf[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE];
|
||||
uint16_t digest_len;
|
||||
uint64_t hash;
|
||||
|
||||
if (out_hash == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (ref.digest.len != 0u && ref.digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (ref.digest.len > UINT16_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
digest_len = (uint16_t)ref.digest.len;
|
||||
amduat_asl_index_accel_store_u16_le(header, ref.hash_id);
|
||||
amduat_asl_index_accel_store_u16_le(header + 2u, digest_len);
|
||||
|
||||
tag_buf[0] = has_type_tag ? 1u : 0u;
|
||||
amduat_asl_index_accel_store_u32_le(
|
||||
tag_buf + 1u, has_type_tag ? type_tag.tag_id : 0u);
|
||||
|
||||
hash = 14695981039346656037ull;
|
||||
hash = amduat_asl_index_accel_fnv1a64_update(hash, header, sizeof(header));
|
||||
hash = amduat_asl_index_accel_fnv1a64_update(
|
||||
hash, ref.digest.data, ref.digest.len);
|
||||
hash = amduat_asl_index_accel_fnv1a64_update(hash, tag_buf, sizeof(tag_buf));
|
||||
*out_hash = hash;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_index_accel_routing_key_from_ref(
|
||||
amduat_reference_t ref,
|
||||
bool has_type_tag,
|
||||
amduat_type_tag_t type_tag,
|
||||
amduat_octets_t *out_key) {
|
||||
size_t total_len;
|
||||
uint8_t *buffer;
|
||||
uint8_t tag_flag;
|
||||
uint16_t digest_len;
|
||||
|
||||
if (out_key == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_key = amduat_octets(NULL, 0u);
|
||||
|
||||
if (ref.digest.len != 0u && ref.digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (ref.digest.len > UINT16_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
digest_len = (uint16_t)ref.digest.len;
|
||||
total_len = AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE +
|
||||
ref.digest.len + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE;
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_asl_index_accel_store_u16_le(buffer, ref.hash_id);
|
||||
amduat_asl_index_accel_store_u16_le(buffer + 2u, digest_len);
|
||||
if (ref.digest.len != 0u) {
|
||||
memcpy(buffer + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE,
|
||||
ref.digest.data,
|
||||
ref.digest.len);
|
||||
}
|
||||
tag_flag = has_type_tag ? 1u : 0u;
|
||||
buffer[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE + ref.digest.len] =
|
||||
tag_flag;
|
||||
amduat_asl_index_accel_store_u32_le(
|
||||
buffer + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE + ref.digest.len +
|
||||
1u,
|
||||
has_type_tag ? type_tag.tag_id : 0u);
|
||||
|
||||
*out_key = amduat_octets(buffer, total_len);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_index_accel_routing_key_hash(amduat_octets_t key,
|
||||
uint64_t *out_hash) {
|
||||
uint64_t hash;
|
||||
|
||||
if (out_hash == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (key.len != 0u && key.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
hash = 14695981039346656037ull;
|
||||
hash = amduat_asl_index_accel_fnv1a64_update(hash, key.data, key.len);
|
||||
*out_hash = hash;
|
||||
return true;
|
||||
}
|
||||
|
||||
uint16_t amduat_asl_index_accel_shard_for_ref(
|
||||
amduat_reference_t ref,
|
||||
bool has_type_tag,
|
||||
amduat_type_tag_t type_tag,
|
||||
uint16_t shard_count) {
|
||||
uint64_t hash = 0u;
|
||||
|
||||
if (shard_count == 0u) {
|
||||
return 0u;
|
||||
}
|
||||
if (!amduat_asl_index_accel_hash_ref(ref, has_type_tag, type_tag, &hash)) {
|
||||
return 0u;
|
||||
}
|
||||
return (uint16_t)(hash % shard_count);
|
||||
}
|
||||
152
src/near_core/asl/index_bloom.c
Normal file
152
src/near_core/asl/index_bloom.c
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
#include "amduat/asl/index_bloom.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <stdint.h>
|
||||
|
||||
static uint64_t amduat_asl_index_bloom_hash_update(uint64_t hash,
|
||||
const uint8_t *data,
|
||||
size_t len) {
|
||||
size_t i;
|
||||
|
||||
if (data == NULL || len == 0u) {
|
||||
return hash;
|
||||
}
|
||||
|
||||
for (i = 0u; i < len; ++i) {
|
||||
hash ^= (uint64_t)data[i];
|
||||
hash *= 1099511628211ull;
|
||||
}
|
||||
return hash;
|
||||
}
|
||||
|
||||
static bool amduat_asl_index_bloom_hash_pair(amduat_hash_id_t hash_id,
|
||||
amduat_octets_t digest,
|
||||
uint64_t *out_h1,
|
||||
uint64_t *out_h2) {
|
||||
uint64_t hash;
|
||||
uint8_t header[4];
|
||||
uint16_t digest_len;
|
||||
|
||||
if (out_h1 == NULL || out_h2 == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (digest.len != 0u && digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (digest.len > UINT16_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
digest_len = (uint16_t)digest.len;
|
||||
header[0] = (uint8_t)(hash_id & 0xffu);
|
||||
header[1] = (uint8_t)((hash_id >> 8) & 0xffu);
|
||||
header[2] = (uint8_t)(digest_len & 0xffu);
|
||||
header[3] = (uint8_t)((digest_len >> 8) & 0xffu);
|
||||
|
||||
hash = 14695981039346656037ull ^ 0x9e3779b97f4a7c15ull;
|
||||
hash = amduat_asl_index_bloom_hash_update(hash, header, sizeof(header));
|
||||
hash = amduat_asl_index_bloom_hash_update(hash, digest.data, digest.len);
|
||||
*out_h1 = hash;
|
||||
|
||||
hash = 14695981039346656037ull ^ 0xbf58476d1ce4e5b9ull;
|
||||
hash = amduat_asl_index_bloom_hash_update(hash, header, sizeof(header));
|
||||
hash = amduat_asl_index_bloom_hash_update(hash, digest.data, digest.len);
|
||||
if (hash == 0u) {
|
||||
hash = 0x94d049bb133111ebull;
|
||||
}
|
||||
*out_h2 = hash;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static void amduat_asl_index_bloom_set_bit(uint8_t *data, size_t bit) {
|
||||
data[bit >> 3] |= (uint8_t)(1u << (bit & 7u));
|
||||
}
|
||||
|
||||
static bool amduat_asl_index_bloom_test_bit(const uint8_t *data, size_t bit) {
|
||||
return (data[bit >> 3] & (uint8_t)(1u << (bit & 7u))) != 0u;
|
||||
}
|
||||
|
||||
bool amduat_asl_index_bloom_init(amduat_octets_t *out_bloom) {
|
||||
uint8_t *data;
|
||||
|
||||
if (out_bloom == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_bloom = amduat_octets(NULL, 0u);
|
||||
|
||||
if (AMDUAT_ASL_INDEX_BLOOM_BYTES == 0u) {
|
||||
return true;
|
||||
}
|
||||
|
||||
data = (uint8_t *)calloc(1u, AMDUAT_ASL_INDEX_BLOOM_BYTES);
|
||||
if (data == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_bloom = amduat_octets(data, AMDUAT_ASL_INDEX_BLOOM_BYTES);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_index_bloom_add(amduat_octets_t bloom,
|
||||
amduat_hash_id_t hash_id,
|
||||
amduat_octets_t digest) {
|
||||
uint64_t h1;
|
||||
uint64_t h2;
|
||||
size_t i;
|
||||
size_t bit_count;
|
||||
uint8_t *data;
|
||||
|
||||
if (bloom.len == 0u) {
|
||||
return false;
|
||||
}
|
||||
if (bloom.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
bit_count = bloom.len * 8u;
|
||||
if (bit_count == 0u) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_index_bloom_hash_pair(hash_id, digest, &h1, &h2)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
data = (uint8_t *)bloom.data;
|
||||
for (i = 0u; i < AMDUAT_ASL_INDEX_BLOOM_HASHES; ++i) {
|
||||
uint64_t mix = h1 + (uint64_t)i * h2;
|
||||
size_t bit = (size_t)(mix % (uint64_t)bit_count);
|
||||
amduat_asl_index_bloom_set_bit(data, bit);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_index_bloom_maybe_contains(amduat_octets_t bloom,
|
||||
amduat_hash_id_t hash_id,
|
||||
amduat_octets_t digest) {
|
||||
uint64_t h1;
|
||||
uint64_t h2;
|
||||
size_t i;
|
||||
size_t bit_count;
|
||||
|
||||
if (bloom.len == 0u || bloom.data == NULL) {
|
||||
return true;
|
||||
}
|
||||
bit_count = bloom.len * 8u;
|
||||
if (bit_count == 0u) {
|
||||
return true;
|
||||
}
|
||||
if (!amduat_asl_index_bloom_hash_pair(hash_id, digest, &h1, &h2)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
for (i = 0u; i < AMDUAT_ASL_INDEX_BLOOM_HASHES; ++i) {
|
||||
uint64_t mix = h1 + (uint64_t)i * h2;
|
||||
size_t bit = (size_t)(mix % (uint64_t)bit_count);
|
||||
if (!amduat_asl_index_bloom_test_bit(bloom.data, bit)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
341
src/near_core/asl/index_replay.c
Normal file
341
src/near_core/asl/index_replay.c
Normal file
|
|
@ -0,0 +1,341 @@
|
|||
#include "amduat/asl/index_replay.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
typedef struct {
|
||||
const uint8_t *data;
|
||||
size_t len;
|
||||
size_t offset;
|
||||
} amduat_asl_replay_cursor_t;
|
||||
|
||||
static bool amduat_asl_replay_read_u16_le(amduat_asl_replay_cursor_t *cur,
|
||||
uint16_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 2) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
|
||||
cur->offset += 2;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_read_u32_le(amduat_asl_replay_cursor_t *cur,
|
||||
uint32_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 4) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
cur->offset += 4;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_read_u64_le(amduat_asl_replay_cursor_t *cur,
|
||||
uint64_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 8) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
cur->offset += 8;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_parse_artifact_ref(
|
||||
amduat_asl_replay_cursor_t *cur,
|
||||
amduat_reference_t *out_ref) {
|
||||
uint32_t hash_id_raw;
|
||||
uint16_t digest_len;
|
||||
uint16_t reserved0;
|
||||
const uint8_t *digest;
|
||||
|
||||
if (!amduat_asl_replay_read_u32_le(cur, &hash_id_raw) ||
|
||||
!amduat_asl_replay_read_u16_le(cur, &digest_len) ||
|
||||
!amduat_asl_replay_read_u16_le(cur, &reserved0)) {
|
||||
return false;
|
||||
}
|
||||
if (hash_id_raw > UINT16_MAX || digest_len == 0 || reserved0 != 0) {
|
||||
return false;
|
||||
}
|
||||
if (cur->len - cur->offset < digest_len) {
|
||||
return false;
|
||||
}
|
||||
digest = cur->data + cur->offset;
|
||||
cur->offset += digest_len;
|
||||
*out_ref = amduat_reference((amduat_hash_id_t)hash_id_raw,
|
||||
amduat_octets(digest, digest_len));
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_parse_segment_seal(
|
||||
amduat_octets_t payload,
|
||||
amduat_asl_segment_seal_t *out) {
|
||||
amduat_asl_replay_cursor_t cur;
|
||||
uint64_t segment_id;
|
||||
|
||||
if (payload.len != 8u + 32u || payload.data == NULL || out == NULL) {
|
||||
return false;
|
||||
}
|
||||
cur.data = payload.data;
|
||||
cur.len = payload.len;
|
||||
cur.offset = 0;
|
||||
if (!amduat_asl_replay_read_u64_le(&cur, &segment_id)) {
|
||||
return false;
|
||||
}
|
||||
if (cur.len - cur.offset < 32) {
|
||||
return false;
|
||||
}
|
||||
out->segment_id = segment_id;
|
||||
memcpy(out->segment_hash, cur.data + cur.offset, 32);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_parse_tombstone(
|
||||
amduat_octets_t payload,
|
||||
amduat_reference_t *out_ref) {
|
||||
amduat_asl_replay_cursor_t cur;
|
||||
uint32_t scope;
|
||||
uint32_t reason;
|
||||
|
||||
if (payload.len == 0 || payload.data == NULL || out_ref == NULL) {
|
||||
return false;
|
||||
}
|
||||
cur.data = payload.data;
|
||||
cur.len = payload.len;
|
||||
cur.offset = 0;
|
||||
if (!amduat_asl_replay_parse_artifact_ref(&cur, out_ref)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_replay_read_u32_le(&cur, &scope) ||
|
||||
!amduat_asl_replay_read_u32_le(&cur, &reason)) {
|
||||
return false;
|
||||
}
|
||||
(void)scope;
|
||||
(void)reason;
|
||||
return cur.offset == cur.len;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_parse_tombstone_lift(
|
||||
amduat_octets_t payload,
|
||||
amduat_reference_t *out_ref,
|
||||
uint64_t *out_logseq) {
|
||||
amduat_asl_replay_cursor_t cur;
|
||||
uint64_t tombstone_logseq;
|
||||
|
||||
if (payload.len == 0 || payload.data == NULL || out_ref == NULL ||
|
||||
out_logseq == NULL) {
|
||||
return false;
|
||||
}
|
||||
cur.data = payload.data;
|
||||
cur.len = payload.len;
|
||||
cur.offset = 0;
|
||||
if (!amduat_asl_replay_parse_artifact_ref(&cur, out_ref) ||
|
||||
!amduat_asl_replay_read_u64_le(&cur, &tombstone_logseq)) {
|
||||
return false;
|
||||
}
|
||||
*out_logseq = tombstone_logseq;
|
||||
return cur.offset == cur.len;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_parse_snapshot_anchor(
|
||||
amduat_octets_t payload,
|
||||
amduat_asl_snapshot_id_t *out_snapshot_id) {
|
||||
amduat_asl_replay_cursor_t cur;
|
||||
uint64_t snapshot_id;
|
||||
|
||||
if (payload.len != 8u + 32u || payload.data == NULL ||
|
||||
out_snapshot_id == NULL) {
|
||||
return false;
|
||||
}
|
||||
cur.data = payload.data;
|
||||
cur.len = payload.len;
|
||||
cur.offset = 0;
|
||||
if (!amduat_asl_replay_read_u64_le(&cur, &snapshot_id)) {
|
||||
return false;
|
||||
}
|
||||
*out_snapshot_id = snapshot_id;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_update_segment(
|
||||
amduat_asl_replay_state_t *state,
|
||||
const amduat_asl_segment_seal_t *seal) {
|
||||
size_t i;
|
||||
amduat_asl_segment_seal_t *next;
|
||||
|
||||
for (i = 0; i < state->segments_len; ++i) {
|
||||
if (state->segments[i].segment_id == seal->segment_id) {
|
||||
memcpy(state->segments[i].segment_hash, seal->segment_hash, 32);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
next = (amduat_asl_segment_seal_t *)realloc(
|
||||
state->segments,
|
||||
(state->segments_len + 1u) * sizeof(*state->segments));
|
||||
if (next == NULL) {
|
||||
return false;
|
||||
}
|
||||
state->segments = next;
|
||||
state->segments[state->segments_len] = *seal;
|
||||
state->segments_len += 1u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_replay_add_tombstone(
|
||||
amduat_asl_replay_state_t *state,
|
||||
amduat_reference_t ref,
|
||||
uint64_t logseq) {
|
||||
amduat_asl_tombstone_entry_t *next;
|
||||
amduat_reference_t stored;
|
||||
|
||||
if (!amduat_reference_clone(ref, &stored)) {
|
||||
return false;
|
||||
}
|
||||
next = (amduat_asl_tombstone_entry_t *)realloc(
|
||||
state->tombstones,
|
||||
(state->tombstones_len + 1u) * sizeof(*state->tombstones));
|
||||
if (next == NULL) {
|
||||
amduat_reference_free(&stored);
|
||||
return false;
|
||||
}
|
||||
state->tombstones = next;
|
||||
state->tombstones[state->tombstones_len].ref = stored;
|
||||
state->tombstones[state->tombstones_len].tombstone_logseq = logseq;
|
||||
state->tombstones_len += 1u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static void amduat_asl_replay_remove_tombstone(
|
||||
amduat_asl_replay_state_t *state,
|
||||
amduat_reference_t ref,
|
||||
uint64_t tombstone_logseq) {
|
||||
size_t i;
|
||||
|
||||
for (i = 0; i < state->tombstones_len; ++i) {
|
||||
if (state->tombstones[i].tombstone_logseq != tombstone_logseq) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_reference_eq(state->tombstones[i].ref, ref)) {
|
||||
continue;
|
||||
}
|
||||
amduat_reference_free(&state->tombstones[i].ref);
|
||||
if (i + 1u < state->tombstones_len) {
|
||||
state->tombstones[i] = state->tombstones[state->tombstones_len - 1u];
|
||||
}
|
||||
state->tombstones_len -= 1u;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
bool amduat_asl_replay_init(amduat_asl_replay_state_t *out) {
|
||||
if (out == NULL) {
|
||||
return false;
|
||||
}
|
||||
out->segments = NULL;
|
||||
out->segments_len = 0;
|
||||
out->tombstones = NULL;
|
||||
out->tombstones_len = 0;
|
||||
out->state.snapshot_id = 0;
|
||||
out->state.log_position = 0;
|
||||
return true;
|
||||
}
|
||||
|
||||
void amduat_asl_replay_free(amduat_asl_replay_state_t *state) {
|
||||
size_t i;
|
||||
|
||||
if (state == NULL) {
|
||||
return;
|
||||
}
|
||||
free(state->segments);
|
||||
state->segments = NULL;
|
||||
state->segments_len = 0;
|
||||
if (state->tombstones != NULL) {
|
||||
for (i = 0; i < state->tombstones_len; ++i) {
|
||||
amduat_reference_free(&state->tombstones[i].ref);
|
||||
}
|
||||
}
|
||||
free(state->tombstones);
|
||||
state->tombstones = NULL;
|
||||
state->tombstones_len = 0;
|
||||
state->state.snapshot_id = 0;
|
||||
state->state.log_position = 0;
|
||||
}
|
||||
|
||||
bool amduat_asl_replay_apply_log(
|
||||
const amduat_asl_log_record_t *records,
|
||||
size_t record_count,
|
||||
amduat_asl_log_position_t log_position,
|
||||
amduat_asl_replay_state_t *state) {
|
||||
size_t i;
|
||||
|
||||
if (state == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (record_count != 0 && records == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
const amduat_asl_log_record_t *record = &records[i];
|
||||
amduat_asl_segment_seal_t seal;
|
||||
amduat_reference_t ref;
|
||||
uint64_t tombstone_logseq;
|
||||
amduat_asl_snapshot_id_t snapshot_id;
|
||||
|
||||
if (record->logseq > log_position) {
|
||||
break;
|
||||
}
|
||||
|
||||
switch (record->record_type) {
|
||||
case AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL:
|
||||
if (!amduat_asl_replay_parse_segment_seal(record->payload, &seal)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_replay_update_segment(state, &seal)) {
|
||||
return false;
|
||||
}
|
||||
break;
|
||||
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE:
|
||||
if (!amduat_asl_replay_parse_tombstone(record->payload, &ref)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_replay_add_tombstone(state, ref, record->logseq)) {
|
||||
return false;
|
||||
}
|
||||
break;
|
||||
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT:
|
||||
if (!amduat_asl_replay_parse_tombstone_lift(record->payload,
|
||||
&ref,
|
||||
&tombstone_logseq)) {
|
||||
return false;
|
||||
}
|
||||
amduat_asl_replay_remove_tombstone(state, ref, tombstone_logseq);
|
||||
break;
|
||||
case AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR:
|
||||
if (!amduat_asl_replay_parse_snapshot_anchor(record->payload,
|
||||
&snapshot_id)) {
|
||||
return false;
|
||||
}
|
||||
state->state.snapshot_id = snapshot_id;
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
state->state.log_position = log_position;
|
||||
return true;
|
||||
}
|
||||
487
src/near_core/asl/index_snapshot.c
Normal file
487
src/near_core/asl/index_snapshot.c
Normal file
|
|
@ -0,0 +1,487 @@
|
|||
#include "amduat/asl/index_snapshot.h"
|
||||
|
||||
#include "amduat/asl/io.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE = 40
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_snapshot_manifest_magic[
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN] = {'A', 'S', 'L', 'S',
|
||||
'N', 'A', 'P', '1'};
|
||||
|
||||
static void amduat_asl_snapshot_store_u16_le(uint8_t *out,
|
||||
uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_snapshot_store_u32_le(uint8_t *out,
|
||||
uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_snapshot_store_u64_le(uint8_t *out,
|
||||
uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static uint16_t amduat_asl_snapshot_load_u16_le(const uint8_t *data) {
|
||||
return (uint16_t)data[0] | ((uint16_t)data[1] << 8);
|
||||
}
|
||||
|
||||
static uint32_t amduat_asl_snapshot_load_u32_le(const uint8_t *data) {
|
||||
return (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
}
|
||||
|
||||
static uint64_t amduat_asl_snapshot_load_u64_le(const uint8_t *data) {
|
||||
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
}
|
||||
|
||||
static bool amduat_asl_snapshot_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static int amduat_asl_snapshot_compare_segments(const void *lhs,
|
||||
const void *rhs) {
|
||||
const amduat_asl_segment_seal_t *left =
|
||||
*(const amduat_asl_segment_seal_t *const *)lhs;
|
||||
const amduat_asl_segment_seal_t *right =
|
||||
*(const amduat_asl_segment_seal_t *const *)rhs;
|
||||
|
||||
if (left->segment_id < right->segment_id) {
|
||||
return -1;
|
||||
}
|
||||
if (left->segment_id > right->segment_id) {
|
||||
return 1;
|
||||
}
|
||||
return memcmp(left->segment_hash, right->segment_hash,
|
||||
sizeof(left->segment_hash));
|
||||
}
|
||||
|
||||
static int amduat_asl_snapshot_compare_tombstones(const void *lhs,
|
||||
const void *rhs) {
|
||||
const amduat_asl_tombstone_entry_t *left =
|
||||
*(const amduat_asl_tombstone_entry_t *const *)lhs;
|
||||
const amduat_asl_tombstone_entry_t *right =
|
||||
*(const amduat_asl_tombstone_entry_t *const *)rhs;
|
||||
size_t min_len;
|
||||
int cmp;
|
||||
|
||||
if (left->ref.hash_id < right->ref.hash_id) {
|
||||
return -1;
|
||||
}
|
||||
if (left->ref.hash_id > right->ref.hash_id) {
|
||||
return 1;
|
||||
}
|
||||
if (left->ref.digest.len < right->ref.digest.len) {
|
||||
return -1;
|
||||
}
|
||||
if (left->ref.digest.len > right->ref.digest.len) {
|
||||
return 1;
|
||||
}
|
||||
min_len = left->ref.digest.len;
|
||||
if (min_len != 0u) {
|
||||
cmp = memcmp(left->ref.digest.data, right->ref.digest.data, min_len);
|
||||
if (cmp != 0) {
|
||||
return cmp;
|
||||
}
|
||||
}
|
||||
if (left->tombstone_logseq < right->tombstone_logseq) {
|
||||
return -1;
|
||||
}
|
||||
if (left->tombstone_logseq > right->tombstone_logseq) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static bool amduat_asl_snapshot_hash_manifest(const uint8_t *bytes,
|
||||
size_t len,
|
||||
uint8_t out_hash[32]) {
|
||||
if (out_hash == NULL) {
|
||||
return true;
|
||||
}
|
||||
return amduat_hash_asl1_digest(AMDUAT_HASH_ASL1_ID_SHA256,
|
||||
amduat_octets(bytes, len), out_hash, 32);
|
||||
}
|
||||
|
||||
void amduat_asl_snapshot_manifest_free(
|
||||
amduat_asl_snapshot_manifest_t *manifest) {
|
||||
size_t i;
|
||||
|
||||
if (manifest == NULL) {
|
||||
return;
|
||||
}
|
||||
free(manifest->segments);
|
||||
manifest->segments = NULL;
|
||||
manifest->segments_len = 0;
|
||||
if (manifest->tombstones != NULL) {
|
||||
for (i = 0; i < manifest->tombstones_len; ++i) {
|
||||
amduat_reference_free(&manifest->tombstones[i].ref);
|
||||
}
|
||||
}
|
||||
free(manifest->tombstones);
|
||||
manifest->tombstones = NULL;
|
||||
manifest->tombstones_len = 0;
|
||||
}
|
||||
|
||||
bool amduat_asl_snapshot_manifest_write(
|
||||
const char *path,
|
||||
const amduat_asl_snapshot_manifest_t *manifest,
|
||||
uint8_t out_hash[32]) {
|
||||
size_t i;
|
||||
size_t total_len;
|
||||
size_t offset;
|
||||
uint8_t *buffer;
|
||||
amduat_asl_segment_seal_t **segment_order;
|
||||
amduat_asl_tombstone_entry_t **tombstone_order;
|
||||
|
||||
if (path == NULL || manifest == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (manifest->segments_len != 0u && manifest->segments == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (manifest->tombstones_len != 0u && manifest->tombstones == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (manifest->segments_len > SIZE_MAX / AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
|
||||
return false;
|
||||
}
|
||||
|
||||
total_len = 0;
|
||||
if (!amduat_asl_snapshot_add_size(&total_len,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_snapshot_add_size(
|
||||
&total_len,
|
||||
manifest->segments_len *
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (i = 0; i < manifest->tombstones_len; ++i) {
|
||||
const amduat_asl_tombstone_entry_t *entry = &manifest->tombstones[i];
|
||||
if (entry->ref.digest.len == 0 ||
|
||||
(entry->ref.digest.len != 0u && entry->ref.digest.data == NULL)) {
|
||||
return false;
|
||||
}
|
||||
if (entry->ref.digest.len > UINT16_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_snapshot_add_size(&total_len,
|
||||
16u + entry->ref.digest.len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
segment_order = NULL;
|
||||
tombstone_order = NULL;
|
||||
if (manifest->segments_len != 0u) {
|
||||
segment_order = (amduat_asl_segment_seal_t **)calloc(
|
||||
manifest->segments_len, sizeof(*segment_order));
|
||||
if (segment_order == NULL) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < manifest->segments_len; ++i) {
|
||||
segment_order[i] = &manifest->segments[i];
|
||||
}
|
||||
qsort(segment_order, manifest->segments_len, sizeof(*segment_order),
|
||||
amduat_asl_snapshot_compare_segments);
|
||||
}
|
||||
|
||||
if (manifest->tombstones_len != 0u) {
|
||||
tombstone_order = (amduat_asl_tombstone_entry_t **)calloc(
|
||||
manifest->tombstones_len, sizeof(*tombstone_order));
|
||||
if (tombstone_order == NULL) {
|
||||
free(segment_order);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < manifest->tombstones_len; ++i) {
|
||||
tombstone_order[i] = &manifest->tombstones[i];
|
||||
}
|
||||
qsort(tombstone_order, manifest->tombstones_len,
|
||||
sizeof(*tombstone_order), amduat_asl_snapshot_compare_tombstones);
|
||||
}
|
||||
|
||||
offset = 0;
|
||||
memcpy(buffer + offset, k_amduat_asl_snapshot_manifest_magic,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN;
|
||||
amduat_asl_snapshot_store_u16_le(buffer + offset,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION);
|
||||
offset += 2;
|
||||
amduat_asl_snapshot_store_u16_le(buffer + offset, 0);
|
||||
offset += 2;
|
||||
amduat_asl_snapshot_store_u32_le(buffer + offset,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE);
|
||||
offset += 4;
|
||||
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->snapshot_id);
|
||||
offset += 8;
|
||||
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->anchor_logseq);
|
||||
offset += 8;
|
||||
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->segments_len);
|
||||
offset += 8;
|
||||
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->tombstones_len);
|
||||
offset += 8;
|
||||
amduat_asl_snapshot_store_u32_le(buffer + offset,
|
||||
manifest->config.encoding_profile_id);
|
||||
offset += 4;
|
||||
amduat_asl_snapshot_store_u32_le(buffer + offset,
|
||||
manifest->config.hash_id);
|
||||
offset += 4;
|
||||
memset(buffer + offset, 0,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE - offset);
|
||||
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE;
|
||||
|
||||
for (i = 0; i < manifest->segments_len; ++i) {
|
||||
const amduat_asl_segment_seal_t *entry = segment_order[i];
|
||||
amduat_asl_snapshot_store_u64_le(buffer + offset, entry->segment_id);
|
||||
offset += 8;
|
||||
memcpy(buffer + offset, entry->segment_hash, sizeof(entry->segment_hash));
|
||||
offset += sizeof(entry->segment_hash);
|
||||
}
|
||||
|
||||
for (i = 0; i < manifest->tombstones_len; ++i) {
|
||||
const amduat_asl_tombstone_entry_t *entry = tombstone_order[i];
|
||||
amduat_asl_snapshot_store_u32_le(buffer + offset, entry->ref.hash_id);
|
||||
offset += 4;
|
||||
amduat_asl_snapshot_store_u16_le(buffer + offset,
|
||||
(uint16_t)entry->ref.digest.len);
|
||||
offset += 2;
|
||||
amduat_asl_snapshot_store_u16_le(buffer + offset, 0);
|
||||
offset += 2;
|
||||
memcpy(buffer + offset, entry->ref.digest.data, entry->ref.digest.len);
|
||||
offset += entry->ref.digest.len;
|
||||
amduat_asl_snapshot_store_u64_le(buffer + offset,
|
||||
entry->tombstone_logseq);
|
||||
offset += 8;
|
||||
}
|
||||
|
||||
free(segment_order);
|
||||
free(tombstone_order);
|
||||
|
||||
if (offset != total_len) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_snapshot_hash_manifest(buffer, total_len, out_hash)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_write_path(path, buffer, total_len)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
free(buffer);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_snapshot_manifest_read(
|
||||
const char *path,
|
||||
amduat_asl_snapshot_manifest_t *out_manifest,
|
||||
uint8_t out_hash[32]) {
|
||||
uint8_t *buffer;
|
||||
size_t len;
|
||||
size_t offset;
|
||||
uint16_t version;
|
||||
uint16_t reserved0;
|
||||
uint32_t header_size;
|
||||
uint64_t segment_count;
|
||||
uint64_t tombstone_count;
|
||||
size_t i;
|
||||
|
||||
if (path == NULL || out_manifest == NULL) {
|
||||
return false;
|
||||
}
|
||||
memset(out_manifest, 0, sizeof(*out_manifest));
|
||||
|
||||
buffer = NULL;
|
||||
len = 0u;
|
||||
if (!amduat_asl_read_path(path, &buffer, &len)) {
|
||||
return false;
|
||||
}
|
||||
if (len < AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_snapshot_hash_manifest(buffer, len, out_hash)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (memcmp(buffer, k_amduat_asl_snapshot_manifest_magic,
|
||||
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN) != 0) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN;
|
||||
version = amduat_asl_snapshot_load_u16_le(buffer + offset);
|
||||
offset += 2;
|
||||
reserved0 = amduat_asl_snapshot_load_u16_le(buffer + offset);
|
||||
offset += 2;
|
||||
header_size = amduat_asl_snapshot_load_u32_le(buffer + offset);
|
||||
offset += 4;
|
||||
|
||||
if (version != AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION || reserved0 != 0 ||
|
||||
header_size != AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
out_manifest->snapshot_id = amduat_asl_snapshot_load_u64_le(buffer + offset);
|
||||
offset += 8;
|
||||
out_manifest->anchor_logseq =
|
||||
amduat_asl_snapshot_load_u64_le(buffer + offset);
|
||||
offset += 8;
|
||||
segment_count = amduat_asl_snapshot_load_u64_le(buffer + offset);
|
||||
offset += 8;
|
||||
tombstone_count = amduat_asl_snapshot_load_u64_le(buffer + offset);
|
||||
offset += 8;
|
||||
out_manifest->config.encoding_profile_id =
|
||||
amduat_asl_snapshot_load_u32_le(buffer + offset);
|
||||
offset += 4;
|
||||
out_manifest->config.hash_id =
|
||||
amduat_asl_snapshot_load_u32_le(buffer + offset);
|
||||
offset += 4;
|
||||
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE;
|
||||
|
||||
if (segment_count > SIZE_MAX || tombstone_count > SIZE_MAX) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
out_manifest->segments_len = (size_t)segment_count;
|
||||
if (out_manifest->segments_len != 0u) {
|
||||
if (out_manifest->segments_len >
|
||||
SIZE_MAX / AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
out_manifest->segments = (amduat_asl_segment_seal_t *)calloc(
|
||||
out_manifest->segments_len, sizeof(*out_manifest->segments));
|
||||
if (out_manifest->segments == NULL) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
for (i = 0; i < out_manifest->segments_len; ++i) {
|
||||
amduat_asl_segment_seal_t *entry = &out_manifest->segments[i];
|
||||
if (len - offset < AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
entry->segment_id = amduat_asl_snapshot_load_u64_le(buffer + offset);
|
||||
offset += 8;
|
||||
memcpy(entry->segment_hash, buffer + offset, sizeof(entry->segment_hash));
|
||||
offset += sizeof(entry->segment_hash);
|
||||
}
|
||||
|
||||
out_manifest->tombstones_len = (size_t)tombstone_count;
|
||||
if (out_manifest->tombstones_len != 0u) {
|
||||
out_manifest->tombstones = (amduat_asl_tombstone_entry_t *)calloc(
|
||||
out_manifest->tombstones_len, sizeof(*out_manifest->tombstones));
|
||||
if (out_manifest->tombstones == NULL) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
for (i = 0; i < out_manifest->tombstones_len; ++i) {
|
||||
amduat_asl_tombstone_entry_t *entry = &out_manifest->tombstones[i];
|
||||
uint32_t hash_id;
|
||||
uint16_t digest_len;
|
||||
uint16_t tombstone_reserved;
|
||||
amduat_reference_t ref;
|
||||
uint64_t tombstone_logseq;
|
||||
|
||||
if (len - offset < 16) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
hash_id = amduat_asl_snapshot_load_u32_le(buffer + offset);
|
||||
offset += 4;
|
||||
if (hash_id > UINT16_MAX) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
digest_len = amduat_asl_snapshot_load_u16_le(buffer + offset);
|
||||
offset += 2;
|
||||
tombstone_reserved = amduat_asl_snapshot_load_u16_le(buffer + offset);
|
||||
offset += 2;
|
||||
if (tombstone_reserved != 0 || digest_len == 0) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
if (len - offset < (size_t)digest_len + 8u) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
ref = amduat_reference((amduat_hash_id_t)hash_id,
|
||||
amduat_octets(buffer + offset, digest_len));
|
||||
offset += digest_len;
|
||||
tombstone_logseq = amduat_asl_snapshot_load_u64_le(buffer + offset);
|
||||
offset += 8;
|
||||
if (!amduat_reference_clone(ref, &entry->ref)) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
entry->tombstone_logseq = tombstone_logseq;
|
||||
}
|
||||
|
||||
if (offset != len) {
|
||||
amduat_asl_snapshot_manifest_free(out_manifest);
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
free(buffer);
|
||||
return true;
|
||||
}
|
||||
110
src/near_core/asl/none.c
Normal file
110
src/near_core/asl/none.c
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
#include "amduat/asl/none.h"
|
||||
|
||||
#include <stdint.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_NONE_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_NONE_VERSION = 1
|
||||
};
|
||||
|
||||
static const uint8_t k_amduat_asl_none_magic[AMDUAT_ASL_NONE_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'N', 'O', 'N', '1', '\0'
|
||||
};
|
||||
|
||||
static void amduat_asl_none_store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_none_read_u32_le(const uint8_t *data,
|
||||
size_t len,
|
||||
size_t *offset,
|
||||
uint32_t *out) {
|
||||
if (len - *offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
*out = (uint32_t)data[*offset] |
|
||||
((uint32_t)data[*offset + 1u] << 8) |
|
||||
((uint32_t)data[*offset + 2u] << 16) |
|
||||
((uint32_t)data[*offset + 3u] << 24);
|
||||
*offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_none_encode_v1(amduat_octets_t *out_bytes) {
|
||||
uint8_t *buffer;
|
||||
size_t offset = 0u;
|
||||
|
||||
if (out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0u;
|
||||
|
||||
buffer = (uint8_t *)malloc(AMDUAT_ASL_NONE_MAGIC_LEN + 4u);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(buffer + offset, k_amduat_asl_none_magic, AMDUAT_ASL_NONE_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_NONE_MAGIC_LEN;
|
||||
amduat_asl_none_store_u32_le(buffer + offset, AMDUAT_ASL_NONE_VERSION);
|
||||
offset += 4u;
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = offset;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_asl_none_decode_v1(amduat_octets_t bytes) {
|
||||
size_t offset = 0u;
|
||||
uint32_t version = 0u;
|
||||
|
||||
if (bytes.len != AMDUAT_ASL_NONE_MAGIC_LEN + 4u) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (memcmp(bytes.data, k_amduat_asl_none_magic,
|
||||
AMDUAT_ASL_NONE_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
offset += AMDUAT_ASL_NONE_MAGIC_LEN;
|
||||
if (!amduat_asl_none_read_u32_le(bytes.data, bytes.len, &offset, &version)) {
|
||||
return false;
|
||||
}
|
||||
if (version != AMDUAT_ASL_NONE_VERSION) {
|
||||
return false;
|
||||
}
|
||||
return offset == bytes.len;
|
||||
}
|
||||
|
||||
bool amduat_asl_none_is_artifact(const amduat_artifact_t *artifact) {
|
||||
if (artifact == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (!artifact->has_type_tag ||
|
||||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_NONE_1) {
|
||||
return false;
|
||||
}
|
||||
return amduat_asl_none_decode_v1(artifact->bytes);
|
||||
}
|
||||
|
||||
bool amduat_asl_none_artifact(amduat_artifact_t *out_artifact) {
|
||||
amduat_octets_t bytes = amduat_octets(NULL, 0u);
|
||||
|
||||
if (out_artifact == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_none_encode_v1(&bytes)) {
|
||||
return false;
|
||||
}
|
||||
*out_artifact =
|
||||
amduat_artifact_with_type(bytes, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_NONE_1));
|
||||
return true;
|
||||
}
|
||||
|
|
@ -60,3 +60,122 @@ amduat_asl_store_error_t amduat_asl_store_get(amduat_asl_store_t *store,
|
|||
}
|
||||
return store->ops.get(store->ctx, ref, out_artifact);
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_put_indexed(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_artifact_t artifact,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
amduat_asl_store_error_t cfg_err;
|
||||
amduat_asl_store_error_t store_err;
|
||||
|
||||
if (store == NULL || store->ops.put_indexed == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
cfg_err = amduat_asl_store_validate_config(store);
|
||||
if (cfg_err != AMDUAT_ASL_STORE_OK) {
|
||||
return cfg_err;
|
||||
}
|
||||
store_err = store->ops.put_indexed(store->ctx, artifact, out_ref, out_state);
|
||||
if (store_err != AMDUAT_ASL_STORE_OK) {
|
||||
return store_err;
|
||||
}
|
||||
if (out_ref != NULL && out_ref->hash_id != store->config.hash_id) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_get_indexed(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_index_state_t state,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
amduat_asl_store_error_t cfg_err;
|
||||
|
||||
if (store == NULL || store->ops.get_indexed == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
if (ref.hash_id != store->config.hash_id) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
cfg_err = amduat_asl_store_validate_config(store);
|
||||
if (cfg_err != AMDUAT_ASL_STORE_OK) {
|
||||
return cfg_err;
|
||||
}
|
||||
return store->ops.get_indexed(store->ctx, ref, state, out_artifact);
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_tombstone(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
uint32_t scope,
|
||||
uint32_t reason_code,
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
amduat_asl_store_error_t cfg_err;
|
||||
|
||||
if (store == NULL || store->ops.tombstone == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
if (ref.hash_id != store->config.hash_id) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
cfg_err = amduat_asl_store_validate_config(store);
|
||||
if (cfg_err != AMDUAT_ASL_STORE_OK) {
|
||||
return cfg_err;
|
||||
}
|
||||
return store->ops.tombstone(store->ctx, ref, scope, reason_code, out_state);
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_store_tombstone_lift(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t ref,
|
||||
amduat_asl_log_position_t tombstone_logseq,
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
amduat_asl_store_error_t cfg_err;
|
||||
|
||||
if (store == NULL || store->ops.tombstone_lift == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
if (ref.hash_id != store->config.hash_id) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
cfg_err = amduat_asl_store_validate_config(store);
|
||||
if (cfg_err != AMDUAT_ASL_STORE_OK) {
|
||||
return cfg_err;
|
||||
}
|
||||
return store->ops.tombstone_lift(store->ctx,
|
||||
ref,
|
||||
tombstone_logseq,
|
||||
out_state);
|
||||
}
|
||||
|
||||
amduat_asl_store_error_t amduat_asl_log_scan(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_asl_log_record_t **out_records,
|
||||
size_t *out_count) {
|
||||
amduat_asl_store_error_t cfg_err;
|
||||
|
||||
if (store == NULL || store->ops.log_scan == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
|
||||
}
|
||||
cfg_err = amduat_asl_store_validate_config(store);
|
||||
if (cfg_err != AMDUAT_ASL_STORE_OK) {
|
||||
return cfg_err;
|
||||
}
|
||||
return store->ops.log_scan(store->ctx, out_records, out_count);
|
||||
}
|
||||
|
||||
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
|
||||
amduat_asl_index_state_t *out_state) {
|
||||
amduat_asl_store_error_t cfg_err;
|
||||
|
||||
if (store == NULL || store->ops.current_state == NULL) {
|
||||
return false;
|
||||
}
|
||||
cfg_err = amduat_asl_store_validate_config(store);
|
||||
if (cfg_err != AMDUAT_ASL_STORE_OK) {
|
||||
return false;
|
||||
}
|
||||
return store->ops.current_state(store->ctx, out_state);
|
||||
}
|
||||
|
|
|
|||
846
src/near_core/enc/asl_core_index.c
Normal file
846
src/near_core/enc/asl_core_index.c
Normal file
|
|
@ -0,0 +1,846 @@
|
|||
#include "amduat/enc/asl_core_index.h"
|
||||
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_CORE_INDEX_MAGIC = 0x33305844494c5341ull
|
||||
};
|
||||
|
||||
static void amduat_asl_core_index_store_u16_le(uint8_t *out,
|
||||
uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_core_index_store_u32_le(uint8_t *out,
|
||||
uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_core_index_store_u64_le(uint8_t *out,
|
||||
uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static uint16_t amduat_asl_core_index_load_u16_le(const uint8_t *data) {
|
||||
return (uint16_t)data[0] | ((uint16_t)data[1] << 8);
|
||||
}
|
||||
|
||||
static uint32_t amduat_asl_core_index_load_u32_le(const uint8_t *data) {
|
||||
return (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
}
|
||||
|
||||
static uint64_t amduat_asl_core_index_load_u64_le(const uint8_t *data) {
|
||||
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
}
|
||||
|
||||
static bool amduat_asl_core_index_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_core_index_is_aligned8(uint64_t value) {
|
||||
return (value & 7u) == 0u;
|
||||
}
|
||||
|
||||
static uint64_t amduat_asl_core_index_crc64(const uint8_t *data, size_t len) {
|
||||
uint64_t crc = 0u;
|
||||
size_t i;
|
||||
|
||||
for (i = 0; i < len; ++i) {
|
||||
uint64_t bit;
|
||||
uint8_t value = data[i];
|
||||
crc ^= ((uint64_t)value) << 56;
|
||||
for (bit = 0; bit < 8; ++bit) {
|
||||
if (crc & 0x8000000000000000ull) {
|
||||
crc = (crc << 1) ^ 0x42f0e1eba9ea3693ull;
|
||||
} else {
|
||||
crc <<= 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
return crc;
|
||||
}
|
||||
|
||||
static bool amduat_asl_core_index_validate_record(
|
||||
const amduat_asl_index_record_t *record,
|
||||
const amduat_asl_extent_record_t *extents,
|
||||
size_t extent_count,
|
||||
size_t *extent_cursor,
|
||||
size_t *digest_cursor,
|
||||
size_t digests_len,
|
||||
uint8_t *max_visibility) {
|
||||
size_t i;
|
||||
size_t start;
|
||||
uint64_t total_len;
|
||||
bool is_tombstone;
|
||||
const amduat_hash_asl1_desc_t *hash_desc;
|
||||
|
||||
if (record->reserved0 != 0 || record->reserved1 != 0) {
|
||||
return false;
|
||||
}
|
||||
if ((record->flags & ~AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0) {
|
||||
return false;
|
||||
}
|
||||
if (record->visibility > 1) {
|
||||
return false;
|
||||
}
|
||||
if (record->has_cross_domain_source > 1) {
|
||||
return false;
|
||||
}
|
||||
if (record->has_cross_domain_source == 0 &&
|
||||
record->cross_domain_source != 0) {
|
||||
return false;
|
||||
}
|
||||
if (record->digest_len == 0) {
|
||||
return false;
|
||||
}
|
||||
if ((record->digest_len % 8u) != 0u) {
|
||||
return false;
|
||||
}
|
||||
if (record->hash_id <= UINT16_MAX) {
|
||||
hash_desc = amduat_hash_asl1_desc_lookup(
|
||||
(amduat_hash_id_t)record->hash_id);
|
||||
if (hash_desc != NULL && hash_desc->digest_len != 0 &&
|
||||
record->digest_len != hash_desc->digest_len) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if ((size_t)record->digest_len > digests_len - *digest_cursor) {
|
||||
return false;
|
||||
}
|
||||
*digest_cursor += record->digest_len;
|
||||
|
||||
is_tombstone = (record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
|
||||
if (is_tombstone) {
|
||||
if (record->extent_count != 0 || record->total_length != 0) {
|
||||
return false;
|
||||
}
|
||||
if (record->extents_offset != 0) {
|
||||
return false;
|
||||
}
|
||||
if (record->visibility > *max_visibility) {
|
||||
*max_visibility = record->visibility;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
if (record->extent_count == 0) {
|
||||
return false;
|
||||
}
|
||||
if ((size_t)record->extent_count > extent_count - *extent_cursor) {
|
||||
return false;
|
||||
}
|
||||
|
||||
start = *extent_cursor;
|
||||
total_len = 0;
|
||||
for (i = 0; i < record->extent_count; ++i) {
|
||||
const amduat_asl_extent_record_t *extent = &extents[start + i];
|
||||
if (extent->length == 0) {
|
||||
return false;
|
||||
}
|
||||
total_len += extent->length;
|
||||
if (total_len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if ((uint32_t)total_len != record->total_length) {
|
||||
return false;
|
||||
}
|
||||
*extent_cursor += record->extent_count;
|
||||
|
||||
if (record->visibility > *max_visibility) {
|
||||
*max_visibility = record->visibility;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_asl_core_index_encode_v1(
|
||||
const amduat_asl_core_index_segment_t *segment,
|
||||
amduat_octets_t *out_bytes) {
|
||||
size_t i;
|
||||
size_t offset;
|
||||
size_t digest_cursor;
|
||||
size_t extent_cursor;
|
||||
size_t record_count;
|
||||
size_t extent_count;
|
||||
size_t total_len;
|
||||
uint64_t header_size;
|
||||
uint64_t bloom_offset;
|
||||
uint64_t bloom_size;
|
||||
uint64_t records_offset;
|
||||
uint64_t records_bytes;
|
||||
uint64_t digests_offset;
|
||||
uint64_t digests_size;
|
||||
uint64_t extents_offset;
|
||||
uint64_t extents_bytes;
|
||||
uint64_t footer_offset;
|
||||
uint8_t *buffer;
|
||||
uint8_t max_visibility;
|
||||
|
||||
if (out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0;
|
||||
|
||||
if (segment == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
record_count = segment->record_count;
|
||||
extent_count = segment->extent_count;
|
||||
|
||||
if (record_count != 0 && segment->records == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (extent_count != 0 && segment->extents == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (segment->digests.len != 0 && segment->digests.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (segment->bloom.len != 0 && segment->bloom.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (segment->header.flags != 0 || segment->header.federation_version != 0 ||
|
||||
segment->header.reserved0 != 0) {
|
||||
return false;
|
||||
}
|
||||
if (segment->header.snapshot_min > segment->header.snapshot_max) {
|
||||
return false;
|
||||
}
|
||||
if (segment->header.segment_visibility > 1) {
|
||||
return false;
|
||||
}
|
||||
|
||||
digest_cursor = 0;
|
||||
extent_cursor = 0;
|
||||
max_visibility = 0;
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
if (!amduat_asl_core_index_validate_record(
|
||||
&segment->records[i],
|
||||
segment->extents,
|
||||
extent_count,
|
||||
&extent_cursor,
|
||||
&digest_cursor,
|
||||
segment->digests.len,
|
||||
&max_visibility)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (digest_cursor != segment->digests.len) {
|
||||
return false;
|
||||
}
|
||||
if (extent_cursor != extent_count) {
|
||||
return false;
|
||||
}
|
||||
if (segment->header.segment_visibility != max_visibility) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (segment->bloom.len != 0 && (segment->bloom.len % 8u) != 0u) {
|
||||
return false;
|
||||
}
|
||||
if ((segment->digests.len % 8u) != 0u) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (record_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_RECORD_SIZE) {
|
||||
return false;
|
||||
}
|
||||
if (extent_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
|
||||
return false;
|
||||
}
|
||||
|
||||
header_size = AMDUAT_ASL_CORE_INDEX_HEADER_SIZE;
|
||||
bloom_size = segment->bloom.len;
|
||||
if (bloom_size == 0) {
|
||||
bloom_offset = 0;
|
||||
} else {
|
||||
bloom_offset = header_size;
|
||||
}
|
||||
records_offset = header_size + bloom_size;
|
||||
if (!amduat_asl_core_index_is_aligned8(records_offset)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
records_bytes = (uint64_t)record_count *
|
||||
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
|
||||
digests_offset = records_offset + records_bytes;
|
||||
if (digests_offset < records_offset ||
|
||||
!amduat_asl_core_index_is_aligned8(digests_offset)) {
|
||||
return false;
|
||||
}
|
||||
digests_size = segment->digests.len;
|
||||
extents_offset = digests_offset + digests_size;
|
||||
if (extents_offset < digests_offset ||
|
||||
!amduat_asl_core_index_is_aligned8(extents_offset)) {
|
||||
return false;
|
||||
}
|
||||
extents_bytes = (uint64_t)extent_count *
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
|
||||
footer_offset = extents_offset + extents_bytes;
|
||||
if (footer_offset < extents_offset) {
|
||||
return false;
|
||||
}
|
||||
if (footer_offset > SIZE_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
total_len = 0;
|
||||
if (!amduat_asl_core_index_add_size(&total_len, (size_t)footer_offset) ||
|
||||
!amduat_asl_core_index_add_size(&total_len,
|
||||
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset,
|
||||
AMDUAT_ASL_CORE_INDEX_MAGIC);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u16_le(buffer + offset,
|
||||
AMDUAT_ASL_CORE_INDEX_VERSION);
|
||||
offset += 2;
|
||||
amduat_asl_core_index_store_u16_le(buffer + offset,
|
||||
segment->header.shard_id);
|
||||
offset += 2;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset,
|
||||
AMDUAT_ASL_CORE_INDEX_HEADER_SIZE);
|
||||
offset += 4;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset,
|
||||
segment->header.snapshot_min);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset,
|
||||
segment->header.snapshot_max);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, record_count);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, records_offset);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, bloom_offset);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, bloom_size);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, digests_offset);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, digests_size);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, extents_offset);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, extent_count);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset,
|
||||
segment->header.segment_domain_id);
|
||||
offset += 4;
|
||||
buffer[offset++] = segment->header.segment_visibility;
|
||||
buffer[offset++] = 0;
|
||||
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
|
||||
offset += 2;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, 0);
|
||||
offset += 8;
|
||||
|
||||
if (offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (bloom_size != 0) {
|
||||
memcpy(buffer + (size_t)bloom_offset, segment->bloom.data,
|
||||
(size_t)bloom_size);
|
||||
}
|
||||
|
||||
digest_cursor = 0;
|
||||
extent_cursor = 0;
|
||||
offset = (size_t)records_offset;
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
const amduat_asl_index_record_t *record = &segment->records[i];
|
||||
uint64_t digest_offset = digests_offset + digest_cursor;
|
||||
uint64_t extents_offset_out = 0;
|
||||
bool is_tombstone =
|
||||
(record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
|
||||
|
||||
if (!is_tombstone) {
|
||||
extents_offset_out = extents_offset +
|
||||
(uint64_t)extent_cursor *
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
|
||||
extent_cursor += record->extent_count;
|
||||
}
|
||||
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset, record->hash_id);
|
||||
offset += 4;
|
||||
amduat_asl_core_index_store_u16_le(buffer + offset, record->digest_len);
|
||||
offset += 2;
|
||||
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
|
||||
offset += 2;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, digest_offset);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, extents_offset_out);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset,
|
||||
record->extent_count);
|
||||
offset += 4;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset,
|
||||
record->total_length);
|
||||
offset += 4;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset, record->domain_id);
|
||||
offset += 4;
|
||||
buffer[offset++] = record->visibility;
|
||||
buffer[offset++] = record->has_cross_domain_source;
|
||||
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
|
||||
offset += 2;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset,
|
||||
record->cross_domain_source);
|
||||
offset += 4;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset, record->flags);
|
||||
offset += 4;
|
||||
|
||||
digest_cursor += record->digest_len;
|
||||
}
|
||||
|
||||
if (segment->digests.len != 0) {
|
||||
memcpy(buffer + (size_t)digests_offset, segment->digests.data,
|
||||
segment->digests.len);
|
||||
}
|
||||
|
||||
offset = (size_t)extents_offset;
|
||||
for (i = 0; i < extent_count; ++i) {
|
||||
const amduat_asl_extent_record_t *extent = &segment->extents[i];
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, extent->block_id);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset, extent->offset);
|
||||
offset += 4;
|
||||
amduat_asl_core_index_store_u32_le(buffer + offset, extent->length);
|
||||
offset += 4;
|
||||
}
|
||||
|
||||
{
|
||||
uint64_t crc =
|
||||
amduat_asl_core_index_crc64(buffer, (size_t)footer_offset);
|
||||
offset = (size_t)footer_offset;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset, crc);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset,
|
||||
segment->footer.seal_snapshot);
|
||||
offset += 8;
|
||||
amduat_asl_core_index_store_u64_le(buffer + offset,
|
||||
segment->footer.seal_time_ns);
|
||||
offset += 8;
|
||||
}
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = total_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_asl_core_index_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_core_index_segment_t *out_segment) {
|
||||
const uint8_t *data;
|
||||
amduat_asl_segment_header_t header;
|
||||
amduat_asl_segment_footer_t footer;
|
||||
size_t record_count;
|
||||
size_t extent_count;
|
||||
size_t i;
|
||||
uint64_t footer_offset;
|
||||
uint64_t records_bytes;
|
||||
uint64_t extents_bytes;
|
||||
uint8_t max_visibility;
|
||||
bool legacy_defaults;
|
||||
|
||||
if (out_segment == NULL) {
|
||||
return false;
|
||||
}
|
||||
memset(out_segment, 0, sizeof(*out_segment));
|
||||
|
||||
if (bytes.len != 0 && bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len < AMDUAT_ASL_CORE_INDEX_HEADER_SIZE +
|
||||
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE) {
|
||||
return false;
|
||||
}
|
||||
|
||||
data = bytes.data;
|
||||
header.magic = amduat_asl_core_index_load_u64_le(data);
|
||||
header.version = amduat_asl_core_index_load_u16_le(data + 8);
|
||||
header.shard_id = amduat_asl_core_index_load_u16_le(data + 10);
|
||||
header.header_size = amduat_asl_core_index_load_u32_le(data + 12);
|
||||
header.snapshot_min = amduat_asl_core_index_load_u64_le(data + 16);
|
||||
header.snapshot_max = amduat_asl_core_index_load_u64_le(data + 24);
|
||||
header.record_count = amduat_asl_core_index_load_u64_le(data + 32);
|
||||
header.records_offset = amduat_asl_core_index_load_u64_le(data + 40);
|
||||
header.bloom_offset = amduat_asl_core_index_load_u64_le(data + 48);
|
||||
header.bloom_size = amduat_asl_core_index_load_u64_le(data + 56);
|
||||
header.digests_offset = amduat_asl_core_index_load_u64_le(data + 64);
|
||||
header.digests_size = amduat_asl_core_index_load_u64_le(data + 72);
|
||||
header.extents_offset = amduat_asl_core_index_load_u64_le(data + 80);
|
||||
header.extent_count = amduat_asl_core_index_load_u64_le(data + 88);
|
||||
header.segment_domain_id = amduat_asl_core_index_load_u32_le(data + 96);
|
||||
header.segment_visibility = data[100];
|
||||
header.federation_version = data[101];
|
||||
header.reserved0 = amduat_asl_core_index_load_u16_le(data + 102);
|
||||
header.flags = amduat_asl_core_index_load_u64_le(data + 104);
|
||||
|
||||
if (header.magic != AMDUAT_ASL_CORE_INDEX_MAGIC) {
|
||||
return false;
|
||||
}
|
||||
if (header.version < 1 || header.version > AMDUAT_ASL_CORE_INDEX_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (header.header_size != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
|
||||
return false;
|
||||
}
|
||||
if (header.flags != 0) {
|
||||
return false;
|
||||
}
|
||||
if (header.snapshot_min > header.snapshot_max) {
|
||||
return false;
|
||||
}
|
||||
|
||||
legacy_defaults = header.version < AMDUAT_ASL_CORE_INDEX_VERSION;
|
||||
if (!legacy_defaults) {
|
||||
if (header.federation_version != 0 || header.reserved0 != 0) {
|
||||
return false;
|
||||
}
|
||||
if (header.segment_visibility > 1) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (header.record_count > SIZE_MAX) {
|
||||
return false;
|
||||
}
|
||||
record_count = (size_t)header.record_count;
|
||||
if (record_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_RECORD_SIZE) {
|
||||
return false;
|
||||
}
|
||||
records_bytes = header.record_count *
|
||||
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
|
||||
|
||||
if (header.extent_count > SIZE_MAX) {
|
||||
return false;
|
||||
}
|
||||
extent_count = (size_t)header.extent_count;
|
||||
if (extent_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
|
||||
return false;
|
||||
}
|
||||
extents_bytes = header.extent_count *
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
|
||||
|
||||
if (header.bloom_size == 0) {
|
||||
if (header.bloom_offset != 0) {
|
||||
return false;
|
||||
}
|
||||
} else {
|
||||
if (header.bloom_offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_core_index_is_aligned8(header.bloom_size)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (header.records_offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE +
|
||||
header.bloom_size) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_core_index_is_aligned8(header.records_offset)) {
|
||||
return false;
|
||||
}
|
||||
if (header.digests_offset != header.records_offset + records_bytes) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_core_index_is_aligned8(header.digests_offset)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_core_index_is_aligned8(header.digests_size)) {
|
||||
return false;
|
||||
}
|
||||
if (header.extents_offset != header.digests_offset + header.digests_size) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_core_index_is_aligned8(header.extents_offset)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
footer_offset = header.extents_offset + extents_bytes;
|
||||
if (footer_offset < header.extents_offset) {
|
||||
return false;
|
||||
}
|
||||
if (footer_offset > SIZE_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (footer_offset + AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE != bytes.len) {
|
||||
return false;
|
||||
}
|
||||
|
||||
footer.crc64 = amduat_asl_core_index_load_u64_le(
|
||||
data + (size_t)footer_offset);
|
||||
footer.seal_snapshot = amduat_asl_core_index_load_u64_le(
|
||||
data + (size_t)footer_offset + 8);
|
||||
footer.seal_time_ns = amduat_asl_core_index_load_u64_le(
|
||||
data + (size_t)footer_offset + 16);
|
||||
|
||||
if (amduat_asl_core_index_crc64(data, (size_t)footer_offset) !=
|
||||
footer.crc64) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (header.bloom_size != 0) {
|
||||
if (!amduat_octets_clone(
|
||||
amduat_octets(data + (size_t)header.bloom_offset,
|
||||
(size_t)header.bloom_size),
|
||||
&out_segment->bloom)) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_octets_clone(
|
||||
amduat_octets(data + (size_t)header.digests_offset,
|
||||
(size_t)header.digests_size),
|
||||
&out_segment->digests)) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (record_count != 0) {
|
||||
out_segment->records = (amduat_asl_index_record_t *)calloc(
|
||||
record_count, sizeof(*out_segment->records));
|
||||
if (out_segment->records == NULL) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
out_segment->record_count = record_count;
|
||||
|
||||
if (extent_count != 0) {
|
||||
out_segment->extents = (amduat_asl_extent_record_t *)calloc(
|
||||
extent_count, sizeof(*out_segment->extents));
|
||||
if (out_segment->extents == NULL) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
out_segment->extent_count = extent_count;
|
||||
|
||||
for (i = 0; i < extent_count; ++i) {
|
||||
size_t base = (size_t)header.extents_offset +
|
||||
i * AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
|
||||
amduat_asl_extent_record_t *extent = &out_segment->extents[i];
|
||||
extent->block_id = amduat_asl_core_index_load_u64_le(data + base);
|
||||
extent->offset = amduat_asl_core_index_load_u32_le(data + base + 8);
|
||||
extent->length = amduat_asl_core_index_load_u32_le(data + base + 12);
|
||||
if (extent->length == 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
max_visibility = 0;
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
size_t base = (size_t)header.records_offset +
|
||||
i * AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
|
||||
amduat_asl_index_record_t *record = &out_segment->records[i];
|
||||
uint64_t extents_offset;
|
||||
size_t extent_index;
|
||||
size_t j;
|
||||
uint64_t total_len;
|
||||
bool is_tombstone;
|
||||
|
||||
record->hash_id = amduat_asl_core_index_load_u32_le(data + base);
|
||||
record->digest_len = amduat_asl_core_index_load_u16_le(data + base + 4);
|
||||
record->reserved0 = amduat_asl_core_index_load_u16_le(data + base + 6);
|
||||
record->digest_offset = amduat_asl_core_index_load_u64_le(data + base + 8);
|
||||
record->extents_offset = amduat_asl_core_index_load_u64_le(
|
||||
data + base + 16);
|
||||
record->extent_count = amduat_asl_core_index_load_u32_le(data + base + 24);
|
||||
record->total_length = amduat_asl_core_index_load_u32_le(data + base + 28);
|
||||
record->domain_id = amduat_asl_core_index_load_u32_le(data + base + 32);
|
||||
record->visibility = data[base + 36];
|
||||
record->has_cross_domain_source = data[base + 37];
|
||||
record->reserved1 = amduat_asl_core_index_load_u16_le(data + base + 38);
|
||||
record->cross_domain_source =
|
||||
amduat_asl_core_index_load_u32_le(data + base + 40);
|
||||
record->flags = amduat_asl_core_index_load_u32_le(data + base + 44);
|
||||
|
||||
if (!legacy_defaults) {
|
||||
if (record->reserved0 != 0 || record->reserved1 != 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->visibility > 1 || record->has_cross_domain_source > 1) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->has_cross_domain_source == 0 &&
|
||||
record->cross_domain_source != 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
} else {
|
||||
record->domain_id = 0;
|
||||
record->visibility = 0;
|
||||
record->has_cross_domain_source = 0;
|
||||
record->cross_domain_source = 0;
|
||||
}
|
||||
|
||||
if ((record->flags & ~AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->hash_id <= UINT16_MAX) {
|
||||
const amduat_hash_asl1_desc_t *hash_desc =
|
||||
amduat_hash_asl1_desc_lookup((amduat_hash_id_t)record->hash_id);
|
||||
if (hash_desc != NULL && hash_desc->digest_len != 0 &&
|
||||
record->digest_len != hash_desc->digest_len) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (record->digest_len == 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->digest_len > header.digests_size) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (!legacy_defaults &&
|
||||
!amduat_asl_core_index_is_aligned8(record->digest_offset)) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->digest_offset < header.digests_offset) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->digest_offset >
|
||||
header.digests_offset + header.digests_size -
|
||||
record->digest_len) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
|
||||
is_tombstone =
|
||||
(record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
|
||||
if (is_tombstone) {
|
||||
if (record->extent_count != 0 || record->total_length != 0 ||
|
||||
record->extents_offset != 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->visibility > max_visibility) {
|
||||
max_visibility = record->visibility;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (record->extent_count == 0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
extents_offset = record->extents_offset;
|
||||
if (extents_offset < header.extents_offset) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_core_index_is_aligned8(extents_offset)) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (record->extent_count >
|
||||
UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if ((extents_offset - header.extents_offset) %
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE !=
|
||||
0) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
if (extents_offset + (uint64_t)record->extent_count *
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE >
|
||||
header.extents_offset + extents_bytes) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
extent_index = (size_t)((extents_offset - header.extents_offset) /
|
||||
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE);
|
||||
total_len = 0;
|
||||
for (j = 0; j < record->extent_count; ++j) {
|
||||
total_len += out_segment->extents[extent_index + j].length;
|
||||
if (total_len > UINT32_MAX) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if ((uint32_t)total_len != record->total_length) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (record->visibility > max_visibility) {
|
||||
max_visibility = record->visibility;
|
||||
}
|
||||
}
|
||||
|
||||
if (!legacy_defaults) {
|
||||
if (max_visibility != header.segment_visibility) {
|
||||
amduat_enc_asl_core_index_free(out_segment);
|
||||
return false;
|
||||
}
|
||||
} else {
|
||||
header.segment_domain_id = 0;
|
||||
header.segment_visibility = 0;
|
||||
header.federation_version = 0;
|
||||
header.reserved0 = 0;
|
||||
}
|
||||
|
||||
out_segment->header = header;
|
||||
out_segment->footer = footer;
|
||||
return true;
|
||||
}
|
||||
|
||||
void amduat_enc_asl_core_index_free(amduat_asl_core_index_segment_t *segment) {
|
||||
if (segment == NULL) {
|
||||
return;
|
||||
}
|
||||
free(segment->records);
|
||||
segment->records = NULL;
|
||||
segment->record_count = 0;
|
||||
free(segment->extents);
|
||||
segment->extents = NULL;
|
||||
segment->extent_count = 0;
|
||||
amduat_octets_free(&segment->digests);
|
||||
amduat_octets_free(&segment->bloom);
|
||||
}
|
||||
470
src/near_core/enc/asl_log.c
Normal file
470
src/near_core/enc/asl_log.c
Normal file
|
|
@ -0,0 +1,470 @@
|
|||
#include "amduat/enc/asl_log.h"
|
||||
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_LOG_MAGIC_LEN = 8,
|
||||
AMDUAT_ASL_LOG_HASH_LEN = 32,
|
||||
AMDUAT_ASL_LOG_HEADER_LEN = 24,
|
||||
AMDUAT_ASL_LOG_VERSION = 1
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
const uint8_t *data;
|
||||
size_t len;
|
||||
size_t offset;
|
||||
} amduat_asl_log_cursor_t;
|
||||
|
||||
static const uint8_t k_amduat_asl_log_magic[AMDUAT_ASL_LOG_MAGIC_LEN] = {
|
||||
'A', 'S', 'L', 'L', 'O', 'G', '0', '1'};
|
||||
|
||||
static void amduat_asl_log_store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_log_store_u16_le(uint8_t *out, uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_log_store_u64_le(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_log_read_u32_le(amduat_asl_log_cursor_t *cur,
|
||||
uint32_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 4) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
cur->offset += 4;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_log_read_u16_le(amduat_asl_log_cursor_t *cur,
|
||||
uint16_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 2) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
|
||||
cur->offset += 2;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_log_read_u64_le(amduat_asl_log_cursor_t *cur,
|
||||
uint64_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 8) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
cur->offset += 8;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_log_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_log_is_known_record_type(uint32_t record_type) {
|
||||
switch (record_type) {
|
||||
case AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL:
|
||||
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE:
|
||||
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT:
|
||||
case AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR:
|
||||
case AMDUAT_ASL_LOG_RECORD_ARTIFACT_PUBLISH:
|
||||
case AMDUAT_ASL_LOG_RECORD_ARTIFACT_UNPUBLISH:
|
||||
return true;
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
bool amduat_asl_log_decode_artifact_ref(amduat_octets_t payload,
|
||||
amduat_reference_t *out_ref) {
|
||||
amduat_asl_log_cursor_t cur;
|
||||
uint32_t hash_id_raw;
|
||||
uint16_t digest_len;
|
||||
uint16_t reserved0;
|
||||
|
||||
if (out_ref == NULL || payload.data == NULL || payload.len < 8u) {
|
||||
return false;
|
||||
}
|
||||
cur.data = payload.data;
|
||||
cur.len = payload.len;
|
||||
cur.offset = 0;
|
||||
|
||||
if (!amduat_asl_log_read_u32_le(&cur, &hash_id_raw) ||
|
||||
!amduat_asl_log_read_u16_le(&cur, &digest_len) ||
|
||||
!amduat_asl_log_read_u16_le(&cur, &reserved0)) {
|
||||
return false;
|
||||
}
|
||||
if (hash_id_raw > UINT16_MAX || digest_len == 0 || reserved0 != 0) {
|
||||
return false;
|
||||
}
|
||||
if (cur.len - cur.offset < digest_len) {
|
||||
return false;
|
||||
}
|
||||
{
|
||||
amduat_octets_t digest = amduat_octets(cur.data + cur.offset, digest_len);
|
||||
amduat_reference_t tmp =
|
||||
amduat_reference((amduat_hash_id_t)hash_id_raw, digest);
|
||||
return amduat_reference_clone(tmp, out_ref);
|
||||
}
|
||||
}
|
||||
|
||||
bool amduat_asl_log_encode_artifact_ref(amduat_reference_t ref,
|
||||
amduat_octets_t *out_bytes) {
|
||||
uint8_t *buf;
|
||||
size_t len;
|
||||
|
||||
if (out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_bytes = amduat_octets(NULL, 0u);
|
||||
if (ref.hash_id == 0 || ref.digest.data == NULL || ref.digest.len == 0) {
|
||||
return false;
|
||||
}
|
||||
if (ref.digest.len > UINT16_MAX) {
|
||||
return false;
|
||||
}
|
||||
len = 8u + ref.digest.len;
|
||||
buf = (uint8_t *)malloc(len);
|
||||
if (buf == NULL) {
|
||||
return false;
|
||||
}
|
||||
amduat_asl_log_store_u32_le(buf, ref.hash_id);
|
||||
amduat_asl_log_store_u16_le(buf + 4u, (uint16_t)ref.digest.len);
|
||||
amduat_asl_log_store_u16_le(buf + 6u, 0u);
|
||||
memcpy(buf + 8u, ref.digest.data, ref.digest.len);
|
||||
*out_bytes = amduat_octets(buf, len);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_log_hash_record(const uint8_t prev_hash[32],
|
||||
uint64_t logseq,
|
||||
uint32_t record_type,
|
||||
uint32_t payload_len,
|
||||
const uint8_t *payload,
|
||||
uint8_t out_hash[32]) {
|
||||
amduat_hash_asl1_stream_t stream;
|
||||
uint8_t logseq_bytes[8];
|
||||
uint8_t type_bytes[4];
|
||||
uint8_t payload_len_bytes[4];
|
||||
bool ok;
|
||||
|
||||
if (payload_len != 0 && payload == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_asl_log_store_u64_le(logseq_bytes, logseq);
|
||||
amduat_asl_log_store_u32_le(type_bytes, record_type);
|
||||
amduat_asl_log_store_u32_le(payload_len_bytes, payload_len);
|
||||
|
||||
ok = amduat_hash_asl1_stream_update(&stream,
|
||||
amduat_octets(prev_hash,
|
||||
AMDUAT_ASL_LOG_HASH_LEN)) &&
|
||||
amduat_hash_asl1_stream_update(&stream,
|
||||
amduat_octets(logseq_bytes,
|
||||
sizeof(logseq_bytes))) &&
|
||||
amduat_hash_asl1_stream_update(&stream,
|
||||
amduat_octets(type_bytes,
|
||||
sizeof(type_bytes))) &&
|
||||
amduat_hash_asl1_stream_update(
|
||||
&stream,
|
||||
amduat_octets(payload_len_bytes, sizeof(payload_len_bytes)));
|
||||
|
||||
if (ok && payload_len != 0) {
|
||||
ok = amduat_hash_asl1_stream_update(
|
||||
&stream, amduat_octets(payload, payload_len));
|
||||
}
|
||||
|
||||
if (ok) {
|
||||
ok = amduat_hash_asl1_stream_final(&stream, out_hash,
|
||||
AMDUAT_ASL_LOG_HASH_LEN);
|
||||
}
|
||||
|
||||
amduat_hash_asl1_stream_destroy(&stream);
|
||||
return ok;
|
||||
}
|
||||
|
||||
void amduat_enc_asl_log_free(amduat_asl_log_record_t *records,
|
||||
size_t record_count) {
|
||||
size_t i;
|
||||
|
||||
if (records == NULL) {
|
||||
return;
|
||||
}
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
amduat_octets_free(&records[i].payload);
|
||||
}
|
||||
free(records);
|
||||
}
|
||||
|
||||
bool amduat_enc_asl_log_encode_v1(const amduat_asl_log_record_t *records,
|
||||
size_t record_count,
|
||||
amduat_octets_t *out_bytes) {
|
||||
size_t total_len;
|
||||
size_t i;
|
||||
size_t offset;
|
||||
uint8_t *buffer;
|
||||
uint8_t prev_hash[AMDUAT_ASL_LOG_HASH_LEN];
|
||||
|
||||
if (out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0;
|
||||
|
||||
if (record_count != 0 && records == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
total_len = AMDUAT_ASL_LOG_HEADER_LEN;
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
size_t payload_len = records[i].payload.len;
|
||||
if (payload_len != 0 && records[i].payload.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (payload_len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_log_add_size(&total_len, 8 + 4 + 4)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_log_add_size(&total_len, payload_len)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_log_add_size(&total_len, AMDUAT_ASL_LOG_HASH_LEN)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0;
|
||||
memcpy(buffer + offset, k_amduat_asl_log_magic, AMDUAT_ASL_LOG_MAGIC_LEN);
|
||||
offset += AMDUAT_ASL_LOG_MAGIC_LEN;
|
||||
amduat_asl_log_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_VERSION);
|
||||
offset += 4;
|
||||
amduat_asl_log_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_HEADER_LEN);
|
||||
offset += 4;
|
||||
amduat_asl_log_store_u64_le(buffer + offset, 0);
|
||||
offset += 8;
|
||||
|
||||
memset(prev_hash, 0, sizeof(prev_hash));
|
||||
for (i = 0; i < record_count; ++i) {
|
||||
const amduat_asl_log_record_t *record = &records[i];
|
||||
uint32_t payload_len = (uint32_t)record->payload.len;
|
||||
uint8_t record_hash[AMDUAT_ASL_LOG_HASH_LEN];
|
||||
|
||||
amduat_asl_log_store_u64_le(buffer + offset, record->logseq);
|
||||
offset += 8;
|
||||
amduat_asl_log_store_u32_le(buffer + offset, record->record_type);
|
||||
offset += 4;
|
||||
amduat_asl_log_store_u32_le(buffer + offset, payload_len);
|
||||
offset += 4;
|
||||
if (payload_len != 0) {
|
||||
memcpy(buffer + offset, record->payload.data, payload_len);
|
||||
offset += payload_len;
|
||||
}
|
||||
|
||||
if (!amduat_asl_log_hash_record(prev_hash,
|
||||
record->logseq,
|
||||
record->record_type,
|
||||
payload_len,
|
||||
record->payload.data,
|
||||
record_hash)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
memcpy(buffer + offset, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
|
||||
offset += AMDUAT_ASL_LOG_HASH_LEN;
|
||||
memcpy(prev_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
|
||||
}
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = total_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_asl_log_decode_v1(amduat_octets_t bytes,
|
||||
amduat_asl_log_record_t **out_records,
|
||||
size_t *out_count) {
|
||||
amduat_asl_log_cursor_t cur;
|
||||
uint32_t version;
|
||||
uint32_t header_size;
|
||||
uint64_t flags;
|
||||
uint8_t prev_hash[AMDUAT_ASL_LOG_HASH_LEN];
|
||||
amduat_asl_log_record_t *records;
|
||||
size_t record_count;
|
||||
size_t record_capacity;
|
||||
uint64_t last_logseq;
|
||||
|
||||
if (out_records == NULL || out_count == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_records = NULL;
|
||||
*out_count = 0;
|
||||
|
||||
if (bytes.len != 0 && bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len < AMDUAT_ASL_LOG_HEADER_LEN) {
|
||||
return false;
|
||||
}
|
||||
|
||||
cur.data = bytes.data;
|
||||
cur.len = bytes.len;
|
||||
cur.offset = 0;
|
||||
|
||||
if (memcmp(cur.data, k_amduat_asl_log_magic, AMDUAT_ASL_LOG_MAGIC_LEN) != 0) {
|
||||
return false;
|
||||
}
|
||||
cur.offset += AMDUAT_ASL_LOG_MAGIC_LEN;
|
||||
|
||||
if (!amduat_asl_log_read_u32_le(&cur, &version)) {
|
||||
return false;
|
||||
}
|
||||
if (version != AMDUAT_ASL_LOG_VERSION) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_log_read_u32_le(&cur, &header_size)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_log_read_u64_le(&cur, &flags)) {
|
||||
return false;
|
||||
}
|
||||
if (flags != 0) {
|
||||
return false;
|
||||
}
|
||||
if (header_size < AMDUAT_ASL_LOG_HEADER_LEN) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len < header_size) {
|
||||
return false;
|
||||
}
|
||||
cur.offset = header_size;
|
||||
|
||||
records = NULL;
|
||||
record_count = 0;
|
||||
record_capacity = 0;
|
||||
last_logseq = 0u;
|
||||
memset(prev_hash, 0, sizeof(prev_hash));
|
||||
|
||||
while (cur.offset < cur.len) {
|
||||
uint64_t logseq;
|
||||
uint32_t record_type;
|
||||
uint32_t payload_len;
|
||||
const uint8_t *payload;
|
||||
const uint8_t *record_hash;
|
||||
uint8_t expected_hash[AMDUAT_ASL_LOG_HASH_LEN];
|
||||
|
||||
if (!amduat_asl_log_read_u64_le(&cur, &logseq) ||
|
||||
!amduat_asl_log_read_u32_le(&cur, &record_type) ||
|
||||
!amduat_asl_log_read_u32_le(&cur, &payload_len)) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
if (record_count != 0u && logseq <= last_logseq) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
if (cur.len - cur.offset < payload_len + AMDUAT_ASL_LOG_HASH_LEN) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
|
||||
payload = cur.data + cur.offset;
|
||||
cur.offset += payload_len;
|
||||
record_hash = cur.data + cur.offset;
|
||||
cur.offset += AMDUAT_ASL_LOG_HASH_LEN;
|
||||
|
||||
if (!amduat_asl_log_hash_record(prev_hash,
|
||||
logseq,
|
||||
record_type,
|
||||
payload_len,
|
||||
payload,
|
||||
expected_hash)) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
if (memcmp(expected_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN) != 0) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
memcpy(prev_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
|
||||
last_logseq = logseq;
|
||||
|
||||
if (amduat_asl_log_is_known_record_type(record_type)) {
|
||||
amduat_asl_log_record_t *slot;
|
||||
amduat_octets_t payload_bytes;
|
||||
|
||||
if (record_count == record_capacity) {
|
||||
size_t new_capacity = record_capacity == 0 ? 4 : record_capacity * 2;
|
||||
amduat_asl_log_record_t *next =
|
||||
(amduat_asl_log_record_t *)realloc(
|
||||
records, new_capacity * sizeof(*records));
|
||||
if (next == NULL) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
records = next;
|
||||
record_capacity = new_capacity;
|
||||
}
|
||||
|
||||
payload_bytes = amduat_octets(payload, payload_len);
|
||||
slot = &records[record_count];
|
||||
slot->logseq = logseq;
|
||||
slot->record_type = record_type;
|
||||
slot->payload = amduat_octets(NULL, 0u);
|
||||
if (!amduat_octets_clone(payload_bytes, &slot->payload)) {
|
||||
amduat_enc_asl_log_free(records, record_count);
|
||||
return false;
|
||||
}
|
||||
memcpy(slot->record_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
|
||||
record_count++;
|
||||
}
|
||||
}
|
||||
|
||||
*out_records = records;
|
||||
*out_count = record_count;
|
||||
return true;
|
||||
}
|
||||
494
src/near_core/enc/asl_tgk_exec_plan.c
Normal file
494
src/near_core/enc/asl_tgk_exec_plan.c
Normal file
|
|
@ -0,0 +1,494 @@
|
|||
#include "amduat/enc/asl_tgk_exec_plan.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE = 8,
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP = 8,
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_PARAMS_SIZE = 65,
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE = 129,
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_OP_TYPE_MAX =
|
||||
AMDUAT_ASL_TGK_EXEC_OP_TOMBSTONE_SHADOW,
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_OP_FLAG_MASK =
|
||||
AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL |
|
||||
AMDUAT_ASL_TGK_EXEC_OP_FLAG_OPTIONAL
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
const uint8_t *data;
|
||||
size_t len;
|
||||
size_t offset;
|
||||
} amduat_asl_tgk_exec_plan_cursor_t;
|
||||
|
||||
static void amduat_asl_tgk_exec_plan_store_u32_le(uint8_t *out,
|
||||
uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_asl_tgk_exec_plan_store_u64_le(uint8_t *out,
|
||||
uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_read_u32_le(
|
||||
amduat_asl_tgk_exec_plan_cursor_t *cur,
|
||||
uint32_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 4u) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
|
||||
cur->offset += 4u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_read_u64_le(
|
||||
amduat_asl_tgk_exec_plan_cursor_t *cur,
|
||||
uint64_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (cur->len - cur->offset < 8u) {
|
||||
return false;
|
||||
}
|
||||
data = cur->data + cur->offset;
|
||||
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
cur->offset += 8u;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_add_size(size_t *acc, size_t add) {
|
||||
if (*acc > SIZE_MAX - add) {
|
||||
return false;
|
||||
}
|
||||
*acc += add;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_is_bool(uint8_t value) {
|
||||
return value == 0u || value == 1u;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_validate_params(
|
||||
amduat_asl_tgk_exec_operator_type_t op_type,
|
||||
const amduat_asl_tgk_exec_operator_params_t *params) {
|
||||
if (!amduat_asl_tgk_exec_plan_is_bool(params->segment_scan.is_asl_segment)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_is_bool(params->index_filter.has_type_tag)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_is_bool(params->index_filter.has_edge_type)) {
|
||||
return false;
|
||||
}
|
||||
if (params->index_filter.role > 3u) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_is_bool(params->merge.deterministic)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_is_bool(
|
||||
params->projection.project_artifact_id) ||
|
||||
!amduat_asl_tgk_exec_plan_is_bool(
|
||||
params->projection.project_tgk_edge_id) ||
|
||||
!amduat_asl_tgk_exec_plan_is_bool(params->projection.project_node_id) ||
|
||||
!amduat_asl_tgk_exec_plan_is_bool(params->projection.project_type_tag)) {
|
||||
return false;
|
||||
}
|
||||
if (params->tgk_traversal.direction > 3u) {
|
||||
return false;
|
||||
}
|
||||
if (op_type == AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL &&
|
||||
params->tgk_traversal.direction == 0u) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_count) ||
|
||||
!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_union) ||
|
||||
!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_sum)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_has_op_id(
|
||||
const amduat_asl_tgk_exec_plan_t *plan,
|
||||
uint32_t op_id) {
|
||||
uint32_t i;
|
||||
|
||||
for (i = 0; i < plan->operator_count; ++i) {
|
||||
if (plan->operators[i].op_id == op_id) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_unique_op_ids(
|
||||
const amduat_asl_tgk_exec_plan_t *plan) {
|
||||
uint32_t i;
|
||||
uint32_t j;
|
||||
|
||||
for (i = 0; i < plan->operator_count; ++i) {
|
||||
for (j = i + 1u; j < plan->operator_count; ++j) {
|
||||
if (plan->operators[i].op_id == plan->operators[j].op_id) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_validate_operator(
|
||||
const amduat_asl_tgk_exec_plan_t *plan,
|
||||
const amduat_asl_tgk_exec_operator_def_t *op) {
|
||||
uint32_t i;
|
||||
|
||||
if ((uint32_t)op->op_type > AMDUAT_ASL_TGK_EXEC_PLAN_OP_TYPE_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (((uint32_t)op->flags & ~AMDUAT_ASL_TGK_EXEC_PLAN_OP_FLAG_MASK) != 0u) {
|
||||
return false;
|
||||
}
|
||||
if (op->input_count > AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_validate_params(op->op_type, &op->params)) {
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < op->input_count; ++i) {
|
||||
if (!amduat_asl_tgk_exec_plan_has_op_id(plan, op->inputs[i])) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static void amduat_asl_tgk_exec_plan_encode_params(
|
||||
const amduat_asl_tgk_exec_operator_params_t *params,
|
||||
uint8_t *buffer,
|
||||
size_t *offset) {
|
||||
buffer[(*offset)++] = params->segment_scan.is_asl_segment;
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(
|
||||
buffer + *offset, params->segment_scan.segment_start_id);
|
||||
*offset += 8u;
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(
|
||||
buffer + *offset, params->segment_scan.segment_end_id);
|
||||
*offset += 8u;
|
||||
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(
|
||||
buffer + *offset, params->index_filter.artifact_type_tag);
|
||||
*offset += 4u;
|
||||
buffer[(*offset)++] = params->index_filter.has_type_tag;
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(
|
||||
buffer + *offset, params->index_filter.edge_type_key);
|
||||
*offset += 4u;
|
||||
buffer[(*offset)++] = params->index_filter.has_edge_type;
|
||||
buffer[(*offset)++] = params->index_filter.role;
|
||||
|
||||
buffer[(*offset)++] = params->merge.deterministic;
|
||||
|
||||
buffer[(*offset)++] = params->projection.project_artifact_id;
|
||||
buffer[(*offset)++] = params->projection.project_tgk_edge_id;
|
||||
buffer[(*offset)++] = params->projection.project_node_id;
|
||||
buffer[(*offset)++] = params->projection.project_type_tag;
|
||||
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(
|
||||
buffer + *offset, params->tgk_traversal.start_node_id);
|
||||
*offset += 8u;
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(
|
||||
buffer + *offset, params->tgk_traversal.traversal_depth);
|
||||
*offset += 4u;
|
||||
buffer[(*offset)++] = params->tgk_traversal.direction;
|
||||
|
||||
buffer[(*offset)++] = params->aggregation.agg_count;
|
||||
buffer[(*offset)++] = params->aggregation.agg_union;
|
||||
buffer[(*offset)++] = params->aggregation.agg_sum;
|
||||
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(buffer + *offset,
|
||||
params->limit_offset.limit);
|
||||
*offset += 8u;
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(buffer + *offset,
|
||||
params->limit_offset.offset);
|
||||
*offset += 8u;
|
||||
}
|
||||
|
||||
static bool amduat_asl_tgk_exec_plan_decode_params(
|
||||
amduat_asl_tgk_exec_plan_cursor_t *cur,
|
||||
amduat_asl_tgk_exec_operator_params_t *params) {
|
||||
if (cur->len - cur->offset < AMDUAT_ASL_TGK_EXEC_PLAN_PARAMS_SIZE) {
|
||||
return false;
|
||||
}
|
||||
|
||||
params->segment_scan.is_asl_segment = cur->data[cur->offset++];
|
||||
if (!amduat_asl_tgk_exec_plan_read_u64_le(
|
||||
cur, ¶ms->segment_scan.segment_start_id) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u64_le(
|
||||
cur, ¶ms->segment_scan.segment_end_id)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_read_u32_le(
|
||||
cur, ¶ms->index_filter.artifact_type_tag)) {
|
||||
return false;
|
||||
}
|
||||
params->index_filter.has_type_tag = cur->data[cur->offset++];
|
||||
if (!amduat_asl_tgk_exec_plan_read_u32_le(
|
||||
cur, ¶ms->index_filter.edge_type_key)) {
|
||||
return false;
|
||||
}
|
||||
params->index_filter.has_edge_type = cur->data[cur->offset++];
|
||||
params->index_filter.role = cur->data[cur->offset++];
|
||||
|
||||
params->merge.deterministic = cur->data[cur->offset++];
|
||||
|
||||
params->projection.project_artifact_id = cur->data[cur->offset++];
|
||||
params->projection.project_tgk_edge_id = cur->data[cur->offset++];
|
||||
params->projection.project_node_id = cur->data[cur->offset++];
|
||||
params->projection.project_type_tag = cur->data[cur->offset++];
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_read_u64_le(
|
||||
cur, ¶ms->tgk_traversal.start_node_id) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u32_le(
|
||||
cur, ¶ms->tgk_traversal.traversal_depth)) {
|
||||
return false;
|
||||
}
|
||||
params->tgk_traversal.direction = cur->data[cur->offset++];
|
||||
|
||||
params->aggregation.agg_count = cur->data[cur->offset++];
|
||||
params->aggregation.agg_union = cur->data[cur->offset++];
|
||||
params->aggregation.agg_sum = cur->data[cur->offset++];
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_read_u64_le(
|
||||
cur, ¶ms->limit_offset.limit) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u64_le(
|
||||
cur, ¶ms->limit_offset.offset)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_asl_tgk_exec_plan_encode_v1(
|
||||
const amduat_asl_tgk_exec_plan_t *plan,
|
||||
amduat_octets_t *out_bytes) {
|
||||
size_t total_len;
|
||||
size_t offset;
|
||||
uint8_t *buffer;
|
||||
uint32_t i;
|
||||
|
||||
if (out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0u;
|
||||
|
||||
if (plan == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (plan->operator_count != 0u && plan->operators == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (plan->operator_count > SIZE_MAX / AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE) {
|
||||
return false;
|
||||
}
|
||||
|
||||
total_len = AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE;
|
||||
if (!amduat_asl_tgk_exec_plan_add_size(
|
||||
&total_len,
|
||||
(size_t)plan->operator_count *
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (plan->operator_count > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (i = 0; i < plan->operator_count; ++i) {
|
||||
if (!amduat_asl_tgk_exec_plan_validate_operator(plan,
|
||||
&plan->operators[i])) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (!amduat_asl_tgk_exec_plan_unique_op_ids(plan)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0u;
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, plan->plan_version);
|
||||
offset += 4u;
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
|
||||
plan->operator_count);
|
||||
offset += 4u;
|
||||
|
||||
for (i = 0; i < plan->operator_count; ++i) {
|
||||
const amduat_asl_tgk_exec_operator_def_t *op = &plan->operators[i];
|
||||
uint32_t j;
|
||||
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->op_id);
|
||||
offset += 4u;
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
|
||||
(uint32_t)op->op_type);
|
||||
offset += 4u;
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
|
||||
(uint32_t)op->flags);
|
||||
offset += 4u;
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(buffer + offset,
|
||||
op->snapshot.logseq_min);
|
||||
offset += 8u;
|
||||
amduat_asl_tgk_exec_plan_store_u64_le(buffer + offset,
|
||||
op->snapshot.logseq_max);
|
||||
offset += 8u;
|
||||
|
||||
amduat_asl_tgk_exec_plan_encode_params(&op->params, buffer, &offset);
|
||||
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->input_count);
|
||||
offset += 4u;
|
||||
for (j = 0; j < AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP; ++j) {
|
||||
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->inputs[j]);
|
||||
offset += 4u;
|
||||
}
|
||||
}
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = total_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_asl_tgk_exec_plan_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_asl_tgk_exec_plan_t *out_plan) {
|
||||
amduat_asl_tgk_exec_plan_cursor_t cur;
|
||||
uint32_t operator_count;
|
||||
size_t expected_len;
|
||||
amduat_asl_tgk_exec_operator_def_t *operators;
|
||||
uint32_t i;
|
||||
|
||||
if (out_plan == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_plan->plan_version = 0u;
|
||||
out_plan->operator_count = 0u;
|
||||
out_plan->operators = NULL;
|
||||
|
||||
if (bytes.len < AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE ||
|
||||
bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
cur.data = bytes.data;
|
||||
cur.len = bytes.len;
|
||||
cur.offset = 0u;
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &out_plan->plan_version) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &operator_count)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (operator_count > SIZE_MAX / AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE) {
|
||||
return false;
|
||||
}
|
||||
|
||||
expected_len = AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE +
|
||||
(size_t)operator_count *
|
||||
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE;
|
||||
if (expected_len != bytes.len) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (operator_count == 0u) {
|
||||
out_plan->operator_count = 0u;
|
||||
return true;
|
||||
}
|
||||
|
||||
operators = (amduat_asl_tgk_exec_operator_def_t *)calloc(
|
||||
operator_count, sizeof(*operators));
|
||||
if (operators == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (i = 0; i < operator_count; ++i) {
|
||||
amduat_asl_tgk_exec_operator_def_t *op = &operators[i];
|
||||
uint32_t j;
|
||||
uint32_t op_type_raw;
|
||||
uint32_t flags_raw;
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->op_id) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op_type_raw) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &flags_raw) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u64_le(&cur,
|
||||
&op->snapshot.logseq_min) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u64_le(&cur,
|
||||
&op->snapshot.logseq_max)) {
|
||||
free(operators);
|
||||
return false;
|
||||
}
|
||||
op->op_type = (amduat_asl_tgk_exec_operator_type_t)op_type_raw;
|
||||
op->flags = (amduat_asl_tgk_exec_operator_flags_t)flags_raw;
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_decode_params(&cur, &op->params) ||
|
||||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->input_count)) {
|
||||
free(operators);
|
||||
return false;
|
||||
}
|
||||
if (op->input_count > AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP) {
|
||||
free(operators);
|
||||
return false;
|
||||
}
|
||||
for (j = 0; j < AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP; ++j) {
|
||||
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->inputs[j])) {
|
||||
free(operators);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
out_plan->operator_count = operator_count;
|
||||
out_plan->operators = operators;
|
||||
|
||||
if (!amduat_asl_tgk_exec_plan_unique_op_ids(out_plan)) {
|
||||
amduat_enc_asl_tgk_exec_plan_free(out_plan);
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < operator_count; ++i) {
|
||||
if (!amduat_asl_tgk_exec_plan_validate_operator(out_plan,
|
||||
&operators[i])) {
|
||||
amduat_enc_asl_tgk_exec_plan_free(out_plan);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void amduat_enc_asl_tgk_exec_plan_free(amduat_asl_tgk_exec_plan_t *plan) {
|
||||
if (plan == NULL) {
|
||||
return;
|
||||
}
|
||||
free(plan->operators);
|
||||
plan->operators = NULL;
|
||||
plan->operator_count = 0u;
|
||||
plan->plan_version = 0u;
|
||||
}
|
||||
|
|
@ -8,6 +8,19 @@
|
|||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT = 0x0001u,
|
||||
AMDUAT_FER1_TLV_RUN_ID = 0x0002u,
|
||||
AMDUAT_FER1_TLV_LOGS = 0x0003u,
|
||||
AMDUAT_FER1_TLV_LIMITS = 0x0004u,
|
||||
AMDUAT_FER1_TLV_DETERMINISM = 0x0005u,
|
||||
AMDUAT_FER1_TLV_SIGNATURE = 0x0006u
|
||||
};
|
||||
|
||||
enum {
|
||||
AMDUAT_FER1_LOGS_MAX = 64
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
const uint8_t *data;
|
||||
size_t len;
|
||||
|
|
@ -177,6 +190,14 @@ static bool amduat_write_encoded_ref(uint8_t *buffer,
|
|||
return true;
|
||||
}
|
||||
|
||||
static int amduat_log_entry_cmp(const amduat_fer1_log_entry_t *a,
|
||||
const amduat_fer1_log_entry_t *b) {
|
||||
if (a->kind != b->kind) {
|
||||
return a->kind < b->kind ? -1 : 1;
|
||||
}
|
||||
return amduat_reference_cmp(a->log_ref, b->log_ref);
|
||||
}
|
||||
|
||||
static bool amduat_read_encoded_ref(amduat_cursor_t *cur,
|
||||
amduat_reference_t *out_ref) {
|
||||
uint32_t ref_len_u32;
|
||||
|
|
@ -231,10 +252,42 @@ void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt) {
|
|||
free(receipt->parity);
|
||||
}
|
||||
|
||||
if (receipt->has_executor_fingerprint_ref) {
|
||||
amduat_reference_free(&receipt->executor_fingerprint_ref);
|
||||
}
|
||||
if (receipt->has_run_id) {
|
||||
amduat_octets_free(&receipt->run_id);
|
||||
}
|
||||
if (receipt->logs != NULL) {
|
||||
for (i = 0; i < receipt->logs_len; ++i) {
|
||||
amduat_reference_free(&receipt->logs[i].log_ref);
|
||||
amduat_octets_free(&receipt->logs[i].sha256);
|
||||
}
|
||||
free(receipt->logs);
|
||||
}
|
||||
if (receipt->has_rng_seed) {
|
||||
amduat_octets_free(&receipt->rng_seed);
|
||||
}
|
||||
if (receipt->has_signature) {
|
||||
amduat_octets_free(&receipt->signature);
|
||||
}
|
||||
|
||||
receipt->executor_refs = NULL;
|
||||
receipt->executor_refs_len = 0;
|
||||
receipt->parity = NULL;
|
||||
receipt->parity_len = 0;
|
||||
receipt->has_executor_fingerprint_ref = false;
|
||||
receipt->has_run_id = false;
|
||||
receipt->run_id = amduat_octets(NULL, 0u);
|
||||
receipt->has_limits = false;
|
||||
receipt->logs = NULL;
|
||||
receipt->logs_len = 0;
|
||||
receipt->has_determinism = false;
|
||||
receipt->determinism_level = 0u;
|
||||
receipt->has_rng_seed = false;
|
||||
receipt->rng_seed = amduat_octets(NULL, 0u);
|
||||
receipt->has_signature = false;
|
||||
receipt->signature = amduat_octets(NULL, 0u);
|
||||
}
|
||||
|
||||
bool amduat_enc_fer1_receipt_encode_v1(
|
||||
|
|
@ -444,6 +497,431 @@ bool amduat_enc_fer1_receipt_encode_v1(
|
|||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_fer1_receipt_encode_v1_1(
|
||||
const amduat_fer1_receipt_t *receipt,
|
||||
amduat_octets_t *out_bytes) {
|
||||
size_t total_len = 0;
|
||||
size_t offset = 0;
|
||||
size_t ext_len = 0;
|
||||
uint8_t *buffer;
|
||||
size_t i;
|
||||
|
||||
if (receipt == NULL || out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0;
|
||||
|
||||
if (receipt->fer1_version != AMDUAT_FER1_VERSION_1_1) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->evaluator_id.len != 0 && receipt->evaluator_id.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->executor_refs_len != 0 && receipt->executor_refs == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->parity_len != 0 && receipt->parity == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->executor_refs_len != receipt->parity_len) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->evaluator_id.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->started_at > receipt->completed_at) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->has_run_id &&
|
||||
receipt->run_id.len != 0 &&
|
||||
receipt->run_id.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->has_run_id && receipt->run_id.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->has_rng_seed &&
|
||||
receipt->rng_seed.len != 0 &&
|
||||
receipt->rng_seed.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->has_rng_seed && receipt->rng_seed.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->has_signature &&
|
||||
receipt->signature.len != 0 &&
|
||||
receipt->signature.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->has_signature && receipt->signature.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->logs_len != 0 && receipt->logs == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (receipt->logs_len > AMDUAT_FER1_LOGS_MAX) {
|
||||
return false;
|
||||
}
|
||||
for (i = 1; i < receipt->executor_refs_len; ++i) {
|
||||
if (amduat_reference_cmp(receipt->executor_refs[i - 1],
|
||||
receipt->executor_refs[i]) > 0) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
for (i = 0; i < receipt->parity_len; ++i) {
|
||||
if (!amduat_reference_eq(receipt->parity[i].executor_ref,
|
||||
receipt->executor_refs[i])) {
|
||||
return false;
|
||||
}
|
||||
if (i > 0 &&
|
||||
amduat_reference_cmp(receipt->parity[i - 1].executor_ref,
|
||||
receipt->parity[i].executor_ref) > 0) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
for (i = 0; i < receipt->logs_len; ++i) {
|
||||
const amduat_fer1_log_entry_t *entry = &receipt->logs[i];
|
||||
if (entry->sha256.len != 0 && entry->sha256.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (entry->sha256.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (i > 0 &&
|
||||
amduat_log_entry_cmp(&receipt->logs[i - 1], entry) > 0) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
size_t enc_len;
|
||||
if (!amduat_add_size(&total_len, 2) ||
|
||||
!amduat_encoded_ref_len(receipt->function_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len) ||
|
||||
!amduat_encoded_ref_len(receipt->input_manifest_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len) ||
|
||||
!amduat_encoded_ref_len(receipt->environment_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_add_size(&total_len, 4 + receipt->evaluator_id.len)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
{
|
||||
size_t enc_len;
|
||||
if (!amduat_encoded_ref_len(receipt->output_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_add_size(&total_len, 4)) {
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < receipt->executor_refs_len; ++i) {
|
||||
size_t enc_len;
|
||||
if (!amduat_encoded_ref_len(receipt->executor_refs[i], &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_add_size(&total_len, 4)) {
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < receipt->parity_len; ++i) {
|
||||
size_t enc_len;
|
||||
const amduat_fer1_parity_entry_t *entry = &receipt->parity[i];
|
||||
if (entry->parity_digest.len != 0 &&
|
||||
entry->parity_digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (entry->parity_digest.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_reference_eq(entry->output_ref, receipt->output_ref)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_encoded_ref_len(entry->executor_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len) ||
|
||||
!amduat_encoded_ref_len(entry->output_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len) ||
|
||||
!amduat_add_size(&total_len, 1)) {
|
||||
return false;
|
||||
}
|
||||
if (entry->has_sbom_ref) {
|
||||
if (!amduat_encoded_ref_len(entry->sbom_ref, &enc_len) ||
|
||||
!amduat_add_size(&total_len, enc_len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (!amduat_add_size(&total_len, 4 + entry->parity_digest.len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_add_size(&total_len, 8 + 8)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (receipt->has_executor_fingerprint_ref) {
|
||||
size_t enc_len;
|
||||
if (!amduat_encoded_ref_len(receipt->executor_fingerprint_ref, &enc_len)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_add_size(&ext_len, 2 + 4 + enc_len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_run_id) {
|
||||
if (!amduat_add_size(&ext_len, 2 + 4 + 4 + receipt->run_id.len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_limits) {
|
||||
if (!amduat_add_size(&ext_len, 2 + 4 + (5u * 8u))) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->logs_len != 0) {
|
||||
size_t logs_len = 4;
|
||||
for (i = 0; i < receipt->logs_len; ++i) {
|
||||
size_t enc_len;
|
||||
if (!amduat_encoded_ref_len(receipt->logs[i].log_ref, &enc_len)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_add_size(&logs_len, 4) ||
|
||||
!amduat_add_size(&logs_len, enc_len) ||
|
||||
!amduat_add_size(&logs_len, 4 + receipt->logs[i].sha256.len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (!amduat_add_size(&ext_len, 2 + 4 + logs_len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_determinism) {
|
||||
if (!amduat_add_size(&ext_len,
|
||||
2 + 4 + 1 + 4 + receipt->rng_seed.len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_signature) {
|
||||
if (!amduat_add_size(&ext_len, 2 + 4 + receipt->signature.len)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (ext_len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_add_size(&total_len, 4 + ext_len)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_store_u16_be(buffer + offset, receipt->fer1_version);
|
||||
offset += 2;
|
||||
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
receipt->function_ref) ||
|
||||
!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
receipt->input_manifest_ref) ||
|
||||
!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
receipt->environment_ref)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->evaluator_id.len);
|
||||
offset += 4;
|
||||
if (receipt->evaluator_id.len != 0) {
|
||||
memcpy(buffer + offset, receipt->evaluator_id.data,
|
||||
receipt->evaluator_id.len);
|
||||
offset += receipt->evaluator_id.len;
|
||||
}
|
||||
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
receipt->output_ref)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->executor_refs_len);
|
||||
offset += 4;
|
||||
for (i = 0; i < receipt->executor_refs_len; ++i) {
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
receipt->executor_refs[i])) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->parity_len);
|
||||
offset += 4;
|
||||
for (i = 0; i < receipt->parity_len; ++i) {
|
||||
const amduat_fer1_parity_entry_t *entry = &receipt->parity[i];
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
entry->executor_ref) ||
|
||||
!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
entry->output_ref)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
buffer[offset++] = entry->has_sbom_ref ? 0x01u : 0x00u;
|
||||
if (entry->has_sbom_ref) {
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
entry->sbom_ref)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)entry->parity_digest.len);
|
||||
offset += 4;
|
||||
if (entry->parity_digest.len != 0) {
|
||||
memcpy(buffer + offset, entry->parity_digest.data,
|
||||
entry->parity_digest.len);
|
||||
offset += entry->parity_digest.len;
|
||||
}
|
||||
}
|
||||
|
||||
amduat_store_u64_be(buffer + offset, receipt->started_at);
|
||||
offset += 8;
|
||||
amduat_store_u64_be(buffer + offset, receipt->completed_at);
|
||||
offset += 8;
|
||||
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)ext_len);
|
||||
offset += 4;
|
||||
|
||||
if (receipt->has_executor_fingerprint_ref) {
|
||||
size_t enc_len;
|
||||
if (!amduat_encoded_ref_len(receipt->executor_fingerprint_ref, &enc_len)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
amduat_store_u16_be(buffer + offset,
|
||||
AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT);
|
||||
offset += 2;
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)enc_len);
|
||||
offset += 4;
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
receipt->executor_fingerprint_ref)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_run_id) {
|
||||
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_RUN_ID);
|
||||
offset += 2;
|
||||
amduat_store_u32_be(buffer + offset,
|
||||
(uint32_t)(4u + receipt->run_id.len));
|
||||
offset += 4;
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->run_id.len);
|
||||
offset += 4;
|
||||
if (receipt->run_id.len != 0) {
|
||||
memcpy(buffer + offset, receipt->run_id.data, receipt->run_id.len);
|
||||
offset += receipt->run_id.len;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->logs_len != 0) {
|
||||
size_t logs_len = 4;
|
||||
for (i = 0; i < receipt->logs_len; ++i) {
|
||||
size_t enc_len;
|
||||
if (!amduat_encoded_ref_len(receipt->logs[i].log_ref, &enc_len)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
logs_len += 4 + enc_len + 4 + receipt->logs[i].sha256.len;
|
||||
}
|
||||
|
||||
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_LOGS);
|
||||
offset += 2;
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)logs_len);
|
||||
offset += 4;
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->logs_len);
|
||||
offset += 4;
|
||||
for (i = 0; i < receipt->logs_len; ++i) {
|
||||
const amduat_fer1_log_entry_t *entry = &receipt->logs[i];
|
||||
amduat_store_u32_be(buffer + offset, entry->kind);
|
||||
offset += 4;
|
||||
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
|
||||
entry->log_ref)) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)entry->sha256.len);
|
||||
offset += 4;
|
||||
if (entry->sha256.len != 0) {
|
||||
memcpy(buffer + offset, entry->sha256.data, entry->sha256.len);
|
||||
offset += entry->sha256.len;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_limits) {
|
||||
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_LIMITS);
|
||||
offset += 2;
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)(5u * 8u));
|
||||
offset += 4;
|
||||
amduat_store_u64_be(buffer + offset, receipt->limits.cpu_ms);
|
||||
offset += 8;
|
||||
amduat_store_u64_be(buffer + offset, receipt->limits.wall_ms);
|
||||
offset += 8;
|
||||
amduat_store_u64_be(buffer + offset, receipt->limits.max_rss_kib);
|
||||
offset += 8;
|
||||
amduat_store_u64_be(buffer + offset, receipt->limits.io_reads);
|
||||
offset += 8;
|
||||
amduat_store_u64_be(buffer + offset, receipt->limits.io_writes);
|
||||
offset += 8;
|
||||
}
|
||||
|
||||
if (receipt->has_determinism) {
|
||||
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_DETERMINISM);
|
||||
offset += 2;
|
||||
amduat_store_u32_be(buffer + offset,
|
||||
(uint32_t)(1u + 4u + receipt->rng_seed.len));
|
||||
offset += 4;
|
||||
buffer[offset++] = receipt->determinism_level;
|
||||
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->rng_seed.len);
|
||||
offset += 4;
|
||||
if (receipt->rng_seed.len != 0) {
|
||||
memcpy(buffer + offset, receipt->rng_seed.data, receipt->rng_seed.len);
|
||||
offset += receipt->rng_seed.len;
|
||||
}
|
||||
}
|
||||
|
||||
if (receipt->has_signature) {
|
||||
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_SIGNATURE);
|
||||
offset += 2;
|
||||
amduat_store_u32_be(buffer + offset,
|
||||
(uint32_t)receipt->signature.len);
|
||||
offset += 4;
|
||||
if (receipt->signature.len != 0) {
|
||||
memcpy(buffer + offset, receipt->signature.data, receipt->signature.len);
|
||||
offset += receipt->signature.len;
|
||||
}
|
||||
}
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = total_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_fer1_receipt_decode_v1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_fer1_receipt_t *out_receipt) {
|
||||
|
|
@ -627,3 +1105,375 @@ bool amduat_enc_fer1_receipt_decode_v1(
|
|||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_enc_fer1_receipt_decode_v1_1(
|
||||
amduat_octets_t bytes,
|
||||
amduat_fer1_receipt_t *out_receipt) {
|
||||
amduat_cursor_t cur;
|
||||
uint16_t fer1_version;
|
||||
uint32_t len_u32;
|
||||
uint32_t executor_count;
|
||||
uint32_t parity_count;
|
||||
uint32_t ext_len;
|
||||
size_t i;
|
||||
|
||||
if (out_receipt == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len != 0 && bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memset(out_receipt, 0, sizeof(*out_receipt));
|
||||
|
||||
cur.data = bytes.data;
|
||||
cur.len = bytes.len;
|
||||
cur.offset = 0;
|
||||
|
||||
if (!amduat_read_u16(&cur, &fer1_version)) {
|
||||
return false;
|
||||
}
|
||||
if (fer1_version != AMDUAT_FER1_VERSION_1_1) {
|
||||
return false;
|
||||
}
|
||||
out_receipt->fer1_version = fer1_version;
|
||||
|
||||
if (!amduat_read_encoded_ref(&cur, &out_receipt->function_ref) ||
|
||||
!amduat_read_encoded_ref(&cur, &out_receipt->input_manifest_ref) ||
|
||||
!amduat_read_encoded_ref(&cur, &out_receipt->environment_ref)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_read_u32(&cur, &len_u32)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (cur.len - cur.offset < len_u32) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (len_u32 != 0) {
|
||||
amduat_octets_t src = amduat_octets(cur.data + cur.offset, len_u32);
|
||||
if (!amduat_octets_clone(src, &out_receipt->evaluator_id)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
cur.offset += len_u32;
|
||||
}
|
||||
|
||||
if (!amduat_read_encoded_ref(&cur, &out_receipt->output_ref)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_read_u32(&cur, &executor_count)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (executor_count != 0) {
|
||||
if (executor_count > SIZE_MAX / sizeof(amduat_reference_t)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
out_receipt->executor_refs =
|
||||
(amduat_reference_t *)calloc(executor_count,
|
||||
sizeof(amduat_reference_t));
|
||||
if (out_receipt->executor_refs == NULL) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
out_receipt->executor_refs_len = executor_count;
|
||||
for (i = 0; i < out_receipt->executor_refs_len; ++i) {
|
||||
if (!amduat_read_encoded_ref(&cur, &out_receipt->executor_refs[i])) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
for (i = 1; i < out_receipt->executor_refs_len; ++i) {
|
||||
if (amduat_reference_cmp(out_receipt->executor_refs[i - 1],
|
||||
out_receipt->executor_refs[i]) > 0) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_read_u32(&cur, &parity_count)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (parity_count > SIZE_MAX / sizeof(amduat_fer1_parity_entry_t)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (parity_count != 0) {
|
||||
out_receipt->parity =
|
||||
(amduat_fer1_parity_entry_t *)calloc(parity_count,
|
||||
sizeof(amduat_fer1_parity_entry_t));
|
||||
if (out_receipt->parity == NULL) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
out_receipt->parity_len = parity_count;
|
||||
for (i = 0; i < out_receipt->parity_len; ++i) {
|
||||
amduat_fer1_parity_entry_t *entry = &out_receipt->parity[i];
|
||||
uint8_t has_sbom_ref;
|
||||
|
||||
if (!amduat_read_encoded_ref(&cur, &entry->executor_ref) ||
|
||||
!amduat_read_encoded_ref(&cur, &entry->output_ref)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (!amduat_read_u8(&cur, &has_sbom_ref)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (has_sbom_ref != 0x00u && has_sbom_ref != 0x01u) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
entry->has_sbom_ref = has_sbom_ref == 0x01u;
|
||||
if (entry->has_sbom_ref) {
|
||||
if (!amduat_read_encoded_ref(&cur, &entry->sbom_ref)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (!amduat_read_u32(&cur, &len_u32)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (cur.len - cur.offset < len_u32) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (len_u32 != 0) {
|
||||
amduat_octets_t src = amduat_octets(cur.data + cur.offset, len_u32);
|
||||
if (!amduat_octets_clone(src, &entry->parity_digest)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
cur.offset += len_u32;
|
||||
}
|
||||
}
|
||||
for (i = 0; i < out_receipt->parity_len; ++i) {
|
||||
if (!amduat_reference_eq(out_receipt->parity[i].executor_ref,
|
||||
out_receipt->executor_refs[i])) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (i > 0 &&
|
||||
amduat_reference_cmp(out_receipt->parity[i - 1].executor_ref,
|
||||
out_receipt->parity[i].executor_ref) > 0) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (!amduat_reference_eq(out_receipt->parity[i].output_ref,
|
||||
out_receipt->output_ref)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_read_u64(&cur, &out_receipt->started_at) ||
|
||||
!amduat_read_u64(&cur, &out_receipt->completed_at)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (out_receipt->started_at > out_receipt->completed_at) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_read_u32(&cur, &ext_len)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (cur.len - cur.offset < ext_len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
{
|
||||
size_t end_offset = cur.offset + ext_len;
|
||||
while (cur.offset < end_offset) {
|
||||
uint16_t tag;
|
||||
uint32_t tag_len;
|
||||
amduat_cursor_t tlv_cur;
|
||||
|
||||
if (!amduat_read_u16(&cur, &tag) ||
|
||||
!amduat_read_u32(&cur, &tag_len)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (tag_len > end_offset - cur.offset) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
tlv_cur.data = cur.data + cur.offset;
|
||||
tlv_cur.len = tag_len;
|
||||
tlv_cur.offset = 0;
|
||||
|
||||
switch (tag) {
|
||||
case AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT:
|
||||
if (out_receipt->has_executor_fingerprint_ref ||
|
||||
!amduat_read_encoded_ref(&tlv_cur,
|
||||
&out_receipt->executor_fingerprint_ref) ||
|
||||
tlv_cur.offset != tlv_cur.len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
out_receipt->has_executor_fingerprint_ref = true;
|
||||
break;
|
||||
case AMDUAT_FER1_TLV_RUN_ID: {
|
||||
uint32_t run_len;
|
||||
if (out_receipt->has_run_id ||
|
||||
!amduat_read_u32(&tlv_cur, &run_len) ||
|
||||
tlv_cur.len - tlv_cur.offset < run_len ||
|
||||
tlv_cur.offset + run_len != tlv_cur.len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (run_len != 0) {
|
||||
amduat_octets_t src =
|
||||
amduat_octets(tlv_cur.data + tlv_cur.offset, run_len);
|
||||
if (!amduat_octets_clone(src, &out_receipt->run_id)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
out_receipt->has_run_id = true;
|
||||
break;
|
||||
}
|
||||
case AMDUAT_FER1_TLV_LOGS: {
|
||||
uint32_t count;
|
||||
if (!amduat_read_u32(&tlv_cur, &count) ||
|
||||
count > AMDUAT_FER1_LOGS_MAX ||
|
||||
count > SIZE_MAX / sizeof(amduat_fer1_log_entry_t)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (count != 0) {
|
||||
out_receipt->logs =
|
||||
(amduat_fer1_log_entry_t *)calloc(
|
||||
count, sizeof(amduat_fer1_log_entry_t));
|
||||
if (out_receipt->logs == NULL) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
out_receipt->logs_len = count;
|
||||
for (i = 0; i < out_receipt->logs_len; ++i) {
|
||||
amduat_fer1_log_entry_t *entry = &out_receipt->logs[i];
|
||||
uint32_t sha_len;
|
||||
if (!amduat_read_u32(&tlv_cur, &entry->kind) ||
|
||||
!amduat_read_encoded_ref(&tlv_cur, &entry->log_ref) ||
|
||||
!amduat_read_u32(&tlv_cur, &sha_len) ||
|
||||
tlv_cur.len - tlv_cur.offset < sha_len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (sha_len != 0) {
|
||||
amduat_octets_t src =
|
||||
amduat_octets(tlv_cur.data + tlv_cur.offset, sha_len);
|
||||
if (!amduat_octets_clone(src, &entry->sha256)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
tlv_cur.offset += sha_len;
|
||||
if (i > 0 &&
|
||||
amduat_log_entry_cmp(&out_receipt->logs[i - 1],
|
||||
entry) > 0) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (tlv_cur.offset != tlv_cur.len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case AMDUAT_FER1_TLV_LIMITS:
|
||||
if (out_receipt->has_limits ||
|
||||
tlv_cur.len != 5u * 8u ||
|
||||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.cpu_ms) ||
|
||||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.wall_ms) ||
|
||||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.max_rss_kib) ||
|
||||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.io_reads) ||
|
||||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.io_writes) ||
|
||||
tlv_cur.offset != tlv_cur.len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
out_receipt->has_limits = true;
|
||||
break;
|
||||
case AMDUAT_FER1_TLV_DETERMINISM: {
|
||||
uint8_t level;
|
||||
uint32_t seed_len;
|
||||
if (out_receipt->has_determinism ||
|
||||
!amduat_read_u8(&tlv_cur, &level) ||
|
||||
!amduat_read_u32(&tlv_cur, &seed_len) ||
|
||||
tlv_cur.len - tlv_cur.offset < seed_len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (seed_len != 0) {
|
||||
amduat_octets_t src =
|
||||
amduat_octets(tlv_cur.data + tlv_cur.offset, seed_len);
|
||||
if (!amduat_octets_clone(src, &out_receipt->rng_seed)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
out_receipt->has_rng_seed = true;
|
||||
}
|
||||
tlv_cur.offset += seed_len;
|
||||
if (tlv_cur.offset != tlv_cur.len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
out_receipt->determinism_level = level;
|
||||
out_receipt->has_determinism = true;
|
||||
break;
|
||||
}
|
||||
case AMDUAT_FER1_TLV_SIGNATURE: {
|
||||
if (out_receipt->has_signature) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
if (tlv_cur.len != 0) {
|
||||
amduat_octets_t src =
|
||||
amduat_octets(tlv_cur.data + tlv_cur.offset, tlv_cur.len);
|
||||
if (!amduat_octets_clone(src, &out_receipt->signature)) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
tlv_cur.offset = tlv_cur.len;
|
||||
out_receipt->has_signature = true;
|
||||
break;
|
||||
}
|
||||
default:
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
cur.offset += tag_len;
|
||||
}
|
||||
if (cur.offset != end_offset) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (cur.offset != cur.len) {
|
||||
amduat_enc_fer1_receipt_free(out_receipt);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
|
|
|||
72
src/near_core/fed/ingest.c
Normal file
72
src/near_core/fed/ingest.c
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
#include "amduat/fed/ingest.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
static bool amduat_fed_record_id_eq(const amduat_fed_record_id_t *a,
|
||||
const amduat_fed_record_id_t *b) {
|
||||
if (a->type != b->type) {
|
||||
return false;
|
||||
}
|
||||
return amduat_reference_eq(a->ref, b->ref);
|
||||
}
|
||||
|
||||
static bool amduat_fed_record_equivalent(const amduat_fed_record_t *a,
|
||||
const amduat_fed_record_t *b) {
|
||||
return a->meta.domain_id == b->meta.domain_id &&
|
||||
a->meta.visibility == b->meta.visibility &&
|
||||
a->meta.has_source == b->meta.has_source &&
|
||||
a->meta.source_domain == b->meta.source_domain &&
|
||||
a->logseq == b->logseq &&
|
||||
a->snapshot_id == b->snapshot_id &&
|
||||
a->log_prefix == b->log_prefix;
|
||||
}
|
||||
|
||||
amduat_fed_ingest_error_t amduat_fed_ingest_validate(
|
||||
const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
size_t *out_error_index,
|
||||
size_t *out_conflict_index) {
|
||||
size_t i;
|
||||
size_t j;
|
||||
|
||||
if (out_error_index != NULL) {
|
||||
*out_error_index = (size_t)-1;
|
||||
}
|
||||
if (out_conflict_index != NULL) {
|
||||
*out_conflict_index = (size_t)-1;
|
||||
}
|
||||
|
||||
if (records == NULL && count != 0u) {
|
||||
return AMDUAT_FED_INGEST_ERR_INVALID;
|
||||
}
|
||||
|
||||
for (i = 0; i < count; ++i) {
|
||||
const amduat_fed_record_t *record = &records[i];
|
||||
|
||||
if (!amduat_fed_record_validate(record)) {
|
||||
if (out_error_index != NULL) {
|
||||
*out_error_index = i;
|
||||
}
|
||||
return AMDUAT_FED_INGEST_ERR_INVALID;
|
||||
}
|
||||
|
||||
for (j = 0; j < i; ++j) {
|
||||
const amduat_fed_record_t *other = &records[j];
|
||||
|
||||
if (!amduat_fed_record_id_eq(&record->id, &other->id)) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_fed_record_equivalent(record, other)) {
|
||||
if (out_error_index != NULL) {
|
||||
*out_error_index = i;
|
||||
}
|
||||
if (out_conflict_index != NULL) {
|
||||
*out_conflict_index = j;
|
||||
}
|
||||
return AMDUAT_FED_INGEST_ERR_CONFLICT;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return AMDUAT_FED_INGEST_OK;
|
||||
}
|
||||
416
src/near_core/fed/registry.c
Normal file
416
src/near_core/fed/registry.c
Normal file
|
|
@ -0,0 +1,416 @@
|
|||
#include "amduat/fed/registry.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
AMDUAT_FED_REGISTRY_VERSION = 1,
|
||||
AMDUAT_FED_REGISTRY_HEADER_LEN = 8,
|
||||
AMDUAT_FED_REGISTRY_ENTRY_FIXED_LEN = 42
|
||||
};
|
||||
|
||||
static void amduat_fed_store_u16_be(uint8_t *out, uint16_t value) {
|
||||
out[0] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[1] = (uint8_t)(value & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_fed_store_u32_be(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[3] = (uint8_t)(value & 0xffu);
|
||||
}
|
||||
|
||||
static void amduat_fed_store_u64_be(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)((value >> 56) & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[7] = (uint8_t)(value & 0xffu);
|
||||
}
|
||||
|
||||
static uint16_t amduat_fed_load_u16_be(const uint8_t *data) {
|
||||
return ((uint16_t)data[0] << 8) | (uint16_t)data[1];
|
||||
}
|
||||
|
||||
static uint32_t amduat_fed_load_u32_be(const uint8_t *data) {
|
||||
return ((uint32_t)data[0] << 24) | ((uint32_t)data[1] << 16) |
|
||||
((uint32_t)data[2] << 8) | (uint32_t)data[3];
|
||||
}
|
||||
|
||||
static uint64_t amduat_fed_load_u64_be(const uint8_t *data) {
|
||||
return ((uint64_t)data[0] << 56) | ((uint64_t)data[1] << 48) |
|
||||
((uint64_t)data[2] << 40) | ((uint64_t)data[3] << 32) |
|
||||
((uint64_t)data[4] << 24) | ((uint64_t)data[5] << 16) |
|
||||
((uint64_t)data[6] << 8) | (uint64_t)data[7];
|
||||
}
|
||||
|
||||
void amduat_fed_registry_value_init(amduat_fed_registry_value_t *value,
|
||||
amduat_fed_domain_state_t *states,
|
||||
size_t cap) {
|
||||
if (value == NULL) {
|
||||
return;
|
||||
}
|
||||
value->states = states;
|
||||
value->len = 0;
|
||||
value->cap = cap;
|
||||
value->owns_states = false;
|
||||
}
|
||||
|
||||
bool amduat_fed_registry_value_insert(amduat_fed_registry_value_t *value,
|
||||
amduat_fed_domain_state_t state) {
|
||||
size_t left;
|
||||
size_t right;
|
||||
|
||||
if (value == NULL || value->states == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (value->len >= value->cap) {
|
||||
return false;
|
||||
}
|
||||
|
||||
left = 0;
|
||||
right = value->len;
|
||||
while (left < right) {
|
||||
size_t mid;
|
||||
uint32_t current_id;
|
||||
|
||||
mid = left + (right - left) / 2;
|
||||
current_id = value->states[mid].domain_id;
|
||||
if (state.domain_id == current_id) {
|
||||
return false;
|
||||
}
|
||||
if (state.domain_id < current_id) {
|
||||
right = mid;
|
||||
} else {
|
||||
left = mid + 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (left < value->len) {
|
||||
memmove(&value->states[left + 1],
|
||||
&value->states[left],
|
||||
(value->len - left) * sizeof(*value->states));
|
||||
}
|
||||
value->states[left] = state;
|
||||
value->len += 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
const amduat_fed_domain_state_t *amduat_fed_registry_value_lookup(
|
||||
const amduat_fed_registry_value_t *value,
|
||||
uint32_t domain_id) {
|
||||
size_t left;
|
||||
size_t right;
|
||||
|
||||
if (value == NULL || value->states == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
left = 0;
|
||||
right = value->len;
|
||||
while (left < right) {
|
||||
size_t mid;
|
||||
uint32_t current_id;
|
||||
|
||||
mid = left + (right - left) / 2;
|
||||
current_id = value->states[mid].domain_id;
|
||||
if (domain_id == current_id) {
|
||||
return &value->states[mid];
|
||||
}
|
||||
if (domain_id < current_id) {
|
||||
right = mid;
|
||||
} else {
|
||||
left = mid + 1;
|
||||
}
|
||||
}
|
||||
|
||||
return NULL;
|
||||
}
|
||||
|
||||
void amduat_fed_registry_value_free(amduat_fed_registry_value_t *value) {
|
||||
size_t i;
|
||||
|
||||
if (value == NULL) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (value->states != NULL) {
|
||||
for (i = 0; i < value->len; ++i) {
|
||||
amduat_octets_free(&value->states[i].policy_hash);
|
||||
}
|
||||
}
|
||||
|
||||
if (value->owns_states && value->states != NULL) {
|
||||
free(value->states);
|
||||
value->states = NULL;
|
||||
value->cap = 0;
|
||||
}
|
||||
|
||||
value->len = 0;
|
||||
value->owns_states = false;
|
||||
}
|
||||
|
||||
bool amduat_fed_registry_encode(const amduat_fed_registry_value_t *value,
|
||||
amduat_octets_t *out_bytes) {
|
||||
size_t i;
|
||||
size_t total_len;
|
||||
size_t offset;
|
||||
uint8_t *buffer;
|
||||
|
||||
if (out_bytes == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_bytes->data = NULL;
|
||||
out_bytes->len = 0;
|
||||
|
||||
if (value == NULL || value->states == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (value->len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
total_len = AMDUAT_FED_REGISTRY_HEADER_LEN;
|
||||
for (i = 0; i < value->len; ++i) {
|
||||
const amduat_fed_domain_state_t *state;
|
||||
size_t entry_len;
|
||||
|
||||
state = &value->states[i];
|
||||
if (state->policy_hash.len > UINT32_MAX) {
|
||||
return false;
|
||||
}
|
||||
if (state->policy_hash.len != 0u && state->policy_hash.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
entry_len = AMDUAT_FED_REGISTRY_ENTRY_FIXED_LEN + state->policy_hash.len;
|
||||
if (entry_len > SIZE_MAX - total_len) {
|
||||
return false;
|
||||
}
|
||||
total_len += entry_len;
|
||||
}
|
||||
|
||||
buffer = (uint8_t *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0;
|
||||
amduat_fed_store_u32_be(buffer + offset, AMDUAT_FED_REGISTRY_VERSION);
|
||||
offset += 4;
|
||||
amduat_fed_store_u32_be(buffer + offset, (uint32_t)value->len);
|
||||
offset += 4;
|
||||
|
||||
for (i = 0; i < value->len; ++i) {
|
||||
const amduat_fed_domain_state_t *state;
|
||||
|
||||
state = &value->states[i];
|
||||
amduat_fed_store_u32_be(buffer + offset, state->domain_id);
|
||||
offset += 4;
|
||||
amduat_fed_store_u64_be(buffer + offset, state->snapshot_id);
|
||||
offset += 8;
|
||||
amduat_fed_store_u64_be(buffer + offset, state->log_prefix);
|
||||
offset += 8;
|
||||
amduat_fed_store_u64_be(buffer + offset, state->last_logseq);
|
||||
offset += 8;
|
||||
buffer[offset++] = state->admitted;
|
||||
buffer[offset++] = state->policy_ok;
|
||||
memcpy(buffer + offset, state->reserved, sizeof(state->reserved));
|
||||
offset += sizeof(state->reserved);
|
||||
amduat_fed_store_u16_be(buffer + offset, state->policy_hash_id);
|
||||
offset += 2;
|
||||
amduat_fed_store_u32_be(buffer + offset,
|
||||
(uint32_t)state->policy_hash.len);
|
||||
offset += 4;
|
||||
if (state->policy_hash.len != 0u) {
|
||||
memcpy(buffer + offset,
|
||||
state->policy_hash.data,
|
||||
state->policy_hash.len);
|
||||
offset += state->policy_hash.len;
|
||||
}
|
||||
}
|
||||
|
||||
out_bytes->data = buffer;
|
||||
out_bytes->len = total_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_fed_registry_decode(amduat_octets_t bytes,
|
||||
amduat_fed_registry_value_t *out_value) {
|
||||
uint32_t version;
|
||||
uint32_t count;
|
||||
size_t offset;
|
||||
size_t i;
|
||||
amduat_fed_registry_value_t value;
|
||||
|
||||
if (out_value == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len != 0u && bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len < AMDUAT_FED_REGISTRY_HEADER_LEN) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0;
|
||||
version = amduat_fed_load_u32_be(bytes.data + offset);
|
||||
offset += 4;
|
||||
if (version != AMDUAT_FED_REGISTRY_VERSION) {
|
||||
return false;
|
||||
}
|
||||
count = amduat_fed_load_u32_be(bytes.data + offset);
|
||||
offset += 4;
|
||||
|
||||
value = *out_value;
|
||||
if (value.states == NULL) {
|
||||
if (count != 0u) {
|
||||
value.states =
|
||||
(amduat_fed_domain_state_t *)calloc(count, sizeof(*value.states));
|
||||
if (value.states == NULL) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
value.cap = count;
|
||||
value.owns_states = true;
|
||||
} else {
|
||||
if (value.cap < count) {
|
||||
return false;
|
||||
}
|
||||
value.owns_states = false;
|
||||
}
|
||||
value.len = 0;
|
||||
|
||||
for (i = 0; i < count; ++i) {
|
||||
amduat_fed_domain_state_t state;
|
||||
uint32_t policy_hash_len;
|
||||
|
||||
if (bytes.len - offset < AMDUAT_FED_REGISTRY_ENTRY_FIXED_LEN) {
|
||||
amduat_fed_registry_value_free(&value);
|
||||
return false;
|
||||
}
|
||||
|
||||
memset(&state, 0, sizeof(state));
|
||||
state.domain_id = amduat_fed_load_u32_be(bytes.data + offset);
|
||||
offset += 4;
|
||||
state.snapshot_id = amduat_fed_load_u64_be(bytes.data + offset);
|
||||
offset += 8;
|
||||
state.log_prefix = amduat_fed_load_u64_be(bytes.data + offset);
|
||||
offset += 8;
|
||||
state.last_logseq = amduat_fed_load_u64_be(bytes.data + offset);
|
||||
offset += 8;
|
||||
state.admitted = bytes.data[offset++];
|
||||
state.policy_ok = bytes.data[offset++];
|
||||
memcpy(state.reserved, bytes.data + offset, sizeof(state.reserved));
|
||||
offset += sizeof(state.reserved);
|
||||
state.policy_hash_id = amduat_fed_load_u16_be(bytes.data + offset);
|
||||
offset += 2;
|
||||
policy_hash_len = amduat_fed_load_u32_be(bytes.data + offset);
|
||||
offset += 4;
|
||||
|
||||
if (policy_hash_len > bytes.len - offset) {
|
||||
amduat_fed_registry_value_free(&value);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (policy_hash_len != 0u) {
|
||||
uint8_t *policy_bytes;
|
||||
|
||||
policy_bytes = (uint8_t *)malloc(policy_hash_len);
|
||||
if (policy_bytes == NULL) {
|
||||
amduat_fed_registry_value_free(&value);
|
||||
return false;
|
||||
}
|
||||
memcpy(policy_bytes, bytes.data + offset, policy_hash_len);
|
||||
state.policy_hash = amduat_octets(policy_bytes, policy_hash_len);
|
||||
offset += policy_hash_len;
|
||||
} else {
|
||||
state.policy_hash = amduat_octets(NULL, 0);
|
||||
}
|
||||
|
||||
value.states[i] = state;
|
||||
value.len += 1;
|
||||
}
|
||||
|
||||
*out_value = value;
|
||||
return true;
|
||||
}
|
||||
|
||||
void amduat_fed_registry_store_init(amduat_fed_registry_store_t *reg,
|
||||
amduat_asl_store_t *store) {
|
||||
if (reg == NULL) {
|
||||
return;
|
||||
}
|
||||
reg->store = store;
|
||||
}
|
||||
|
||||
amduat_fed_registry_error_t amduat_fed_registry_store_put(
|
||||
amduat_fed_registry_store_t *reg,
|
||||
const amduat_fed_registry_value_t *value,
|
||||
amduat_reference_t *out_ref,
|
||||
amduat_asl_store_error_t *out_store_err) {
|
||||
amduat_octets_t bytes;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_asl_store_error_t store_err;
|
||||
|
||||
if (out_store_err != NULL) {
|
||||
*out_store_err = AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
if (reg == NULL || reg->store == NULL) {
|
||||
return AMDUAT_FED_REGISTRY_ERR_STORE;
|
||||
}
|
||||
if (!amduat_fed_registry_encode(value, &bytes)) {
|
||||
return AMDUAT_FED_REGISTRY_ERR_CODEC;
|
||||
}
|
||||
|
||||
artifact = amduat_artifact(bytes);
|
||||
store_err = amduat_asl_store_put(reg->store, artifact, out_ref);
|
||||
amduat_octets_free(&bytes);
|
||||
if (store_err != AMDUAT_ASL_STORE_OK) {
|
||||
if (out_store_err != NULL) {
|
||||
*out_store_err = store_err;
|
||||
}
|
||||
return AMDUAT_FED_REGISTRY_ERR_STORE;
|
||||
}
|
||||
|
||||
return AMDUAT_FED_REGISTRY_OK;
|
||||
}
|
||||
|
||||
amduat_fed_registry_error_t amduat_fed_registry_store_get(
|
||||
amduat_fed_registry_store_t *reg,
|
||||
amduat_reference_t ref,
|
||||
amduat_fed_registry_value_t *out_value,
|
||||
amduat_asl_store_error_t *out_store_err) {
|
||||
amduat_artifact_t artifact;
|
||||
amduat_asl_store_error_t store_err;
|
||||
bool ok;
|
||||
|
||||
if (out_store_err != NULL) {
|
||||
*out_store_err = AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
if (reg == NULL || reg->store == NULL || out_value == NULL) {
|
||||
return AMDUAT_FED_REGISTRY_ERR_STORE;
|
||||
}
|
||||
|
||||
store_err = amduat_asl_store_get(reg->store, ref, &artifact);
|
||||
if (store_err != AMDUAT_ASL_STORE_OK) {
|
||||
if (out_store_err != NULL) {
|
||||
*out_store_err = store_err;
|
||||
}
|
||||
return AMDUAT_FED_REGISTRY_ERR_STORE;
|
||||
}
|
||||
|
||||
ok = amduat_fed_registry_decode(artifact.bytes, out_value);
|
||||
amduat_artifact_free(&artifact);
|
||||
if (!ok) {
|
||||
return AMDUAT_FED_REGISTRY_ERR_CODEC;
|
||||
}
|
||||
|
||||
return AMDUAT_FED_REGISTRY_OK;
|
||||
}
|
||||
309
src/near_core/fed/replay.c
Normal file
309
src/near_core/fed/replay.c
Normal file
|
|
@ -0,0 +1,309 @@
|
|||
#include "amduat/fed/replay.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static int amduat_fed_octets_cmp(amduat_octets_t a, amduat_octets_t b) {
|
||||
size_t min_len;
|
||||
int cmp;
|
||||
|
||||
min_len = a.len < b.len ? a.len : b.len;
|
||||
if (min_len > 0) {
|
||||
cmp = memcmp(a.data, b.data, min_len);
|
||||
if (cmp != 0) {
|
||||
return cmp;
|
||||
}
|
||||
}
|
||||
if (a.len < b.len) {
|
||||
return -1;
|
||||
}
|
||||
if (a.len > b.len) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int amduat_fed_record_id_cmp(const amduat_fed_record_id_t *a,
|
||||
const amduat_fed_record_id_t *b) {
|
||||
if (a->type != b->type) {
|
||||
return (a->type < b->type) ? -1 : 1;
|
||||
}
|
||||
if (a->ref.hash_id != b->ref.hash_id) {
|
||||
return (a->ref.hash_id < b->ref.hash_id) ? -1 : 1;
|
||||
}
|
||||
return amduat_fed_octets_cmp(a->ref.digest, b->ref.digest);
|
||||
}
|
||||
|
||||
static int amduat_fed_record_cmp(const void *lhs, const void *rhs) {
|
||||
const amduat_fed_record_t *a = (const amduat_fed_record_t *)lhs;
|
||||
const amduat_fed_record_t *b = (const amduat_fed_record_t *)rhs;
|
||||
|
||||
if (a->logseq != b->logseq) {
|
||||
return (a->logseq < b->logseq) ? -1 : 1;
|
||||
}
|
||||
return amduat_fed_record_id_cmp(&a->id, &b->id);
|
||||
}
|
||||
|
||||
static bool amduat_fed_record_clone(const amduat_fed_record_t *src,
|
||||
amduat_fed_record_t *out) {
|
||||
if (src == NULL || out == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out = *src;
|
||||
if (!amduat_reference_clone(src->id.ref, &out->id.ref)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static void amduat_fed_record_free(amduat_fed_record_t *record) {
|
||||
if (record == NULL) {
|
||||
return;
|
||||
}
|
||||
amduat_reference_free(&record->id.ref);
|
||||
memset(record, 0, sizeof(*record));
|
||||
}
|
||||
|
||||
static bool amduat_fed_bounds_ok(const amduat_fed_record_t *record,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t log_prefix) {
|
||||
if (record->snapshot_id < snapshot_id) {
|
||||
return true;
|
||||
}
|
||||
if (record->snapshot_id > snapshot_id) {
|
||||
return false;
|
||||
}
|
||||
return record->logseq <= log_prefix;
|
||||
}
|
||||
|
||||
static bool amduat_fed_ref_eq(amduat_reference_t a, amduat_reference_t b) {
|
||||
return amduat_reference_eq(a, b);
|
||||
}
|
||||
|
||||
static bool amduat_fed_tombstone_matches(const amduat_fed_record_t *tombstone,
|
||||
const amduat_fed_record_t *record) {
|
||||
if (tombstone->id.type != AMDUAT_FED_REC_TOMBSTONE) {
|
||||
return false;
|
||||
}
|
||||
if (record->id.type == AMDUAT_FED_REC_TOMBSTONE) {
|
||||
return false;
|
||||
}
|
||||
return amduat_fed_ref_eq(tombstone->id.ref, record->id.ref);
|
||||
}
|
||||
|
||||
static bool amduat_fed_tombstone_list_has(
|
||||
const amduat_reference_t *refs,
|
||||
size_t refs_len,
|
||||
amduat_reference_t candidate) {
|
||||
size_t i;
|
||||
|
||||
for (i = 0; i < refs_len; ++i) {
|
||||
if (amduat_fed_ref_eq(refs[i], candidate)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool amduat_fed_tombstone_list_push(amduat_reference_t **refs,
|
||||
size_t *len,
|
||||
size_t *cap,
|
||||
amduat_reference_t ref) {
|
||||
amduat_reference_t *next;
|
||||
|
||||
if (*len == *cap) {
|
||||
size_t next_cap = (*cap == 0u) ? 4u : (*cap * 2u);
|
||||
next = (amduat_reference_t *)realloc(*refs,
|
||||
next_cap * sizeof(*next));
|
||||
if (next == NULL) {
|
||||
return false;
|
||||
}
|
||||
*refs = next;
|
||||
*cap = next_cap;
|
||||
}
|
||||
|
||||
if (!amduat_reference_clone(ref, &(*refs)[*len])) {
|
||||
return false;
|
||||
}
|
||||
(*len)++;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_fed_record_validate(const amduat_fed_record_t *record) {
|
||||
if (record == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (record->meta.visibility > 1u) {
|
||||
return false;
|
||||
}
|
||||
if (record->meta.has_source > 1u) {
|
||||
return false;
|
||||
}
|
||||
if (record->meta.has_source == 0u && record->meta.source_domain != 0u) {
|
||||
return false;
|
||||
}
|
||||
if (record->id.type < AMDUAT_FED_REC_ARTIFACT ||
|
||||
record->id.type > AMDUAT_FED_REC_TOMBSTONE) {
|
||||
return false;
|
||||
}
|
||||
if (record->id.ref.digest.len != 0u &&
|
||||
record->id.ref.digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_fed_replay_build(const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
uint32_t domain_id,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t log_prefix,
|
||||
amduat_fed_replay_view_t *out_view) {
|
||||
amduat_fed_record_t *scratch;
|
||||
amduat_reference_t *tombstones;
|
||||
size_t tombstones_len;
|
||||
size_t tombstones_cap;
|
||||
size_t scratch_len;
|
||||
size_t i;
|
||||
size_t out_len;
|
||||
bool ok;
|
||||
|
||||
if (out_view == NULL) {
|
||||
return false;
|
||||
}
|
||||
out_view->records = NULL;
|
||||
out_view->len = 0;
|
||||
|
||||
if (records == NULL && count != 0u) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (count == 0u) {
|
||||
return true;
|
||||
}
|
||||
|
||||
scratch = (amduat_fed_record_t *)calloc(count, sizeof(*scratch));
|
||||
if (scratch == NULL) {
|
||||
return false;
|
||||
}
|
||||
tombstones = NULL;
|
||||
tombstones_len = 0;
|
||||
tombstones_cap = 0;
|
||||
|
||||
scratch_len = 0;
|
||||
for (i = 0; i < count; ++i) {
|
||||
const amduat_fed_record_t *record = &records[i];
|
||||
|
||||
if (record->meta.domain_id != domain_id) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_fed_record_validate(record)) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_fed_bounds_ok(record, snapshot_id, log_prefix)) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_fed_record_clone(record, &scratch[scratch_len])) {
|
||||
while (scratch_len > 0) {
|
||||
scratch_len--;
|
||||
amduat_fed_record_free(&scratch[scratch_len]);
|
||||
}
|
||||
free(scratch);
|
||||
return false;
|
||||
}
|
||||
scratch_len++;
|
||||
}
|
||||
|
||||
qsort(scratch, scratch_len, sizeof(*scratch), amduat_fed_record_cmp);
|
||||
|
||||
ok = true;
|
||||
out_len = 0;
|
||||
for (i = 0; i < scratch_len; ++i) {
|
||||
amduat_fed_record_t current = scratch[i];
|
||||
size_t j;
|
||||
bool tombstoned;
|
||||
|
||||
scratch[i].id.ref.digest.data = NULL;
|
||||
scratch[i].id.ref.digest.len = 0u;
|
||||
|
||||
if (current.id.type == AMDUAT_FED_REC_TOMBSTONE) {
|
||||
for (j = 0; j < out_len; ++j) {
|
||||
if (amduat_fed_tombstone_matches(¤t, &scratch[j])) {
|
||||
amduat_fed_record_free(&scratch[j]);
|
||||
memmove(&scratch[j],
|
||||
&scratch[j + 1],
|
||||
(out_len - j - 1) * sizeof(*scratch));
|
||||
out_len--;
|
||||
j--;
|
||||
}
|
||||
}
|
||||
if (!amduat_fed_tombstone_list_push(&tombstones,
|
||||
&tombstones_len,
|
||||
&tombstones_cap,
|
||||
current.id.ref)) {
|
||||
amduat_fed_record_free(¤t);
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
amduat_fed_record_free(¤t);
|
||||
continue;
|
||||
}
|
||||
|
||||
tombstoned = amduat_fed_tombstone_list_has(tombstones,
|
||||
tombstones_len,
|
||||
current.id.ref);
|
||||
if (tombstoned) {
|
||||
amduat_fed_record_free(¤t);
|
||||
continue;
|
||||
}
|
||||
|
||||
scratch[out_len++] = current;
|
||||
}
|
||||
|
||||
for (i = 0; i < tombstones_len; ++i) {
|
||||
amduat_reference_free(&tombstones[i]);
|
||||
}
|
||||
free(tombstones);
|
||||
|
||||
if (!ok) {
|
||||
for (i = 0; i < scratch_len; ++i) {
|
||||
amduat_fed_record_free(&scratch[i]);
|
||||
}
|
||||
free(scratch);
|
||||
return false;
|
||||
}
|
||||
|
||||
out_view->records = scratch;
|
||||
out_view->len = out_len;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_fed_replay_domain(const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
uint32_t domain_id,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t log_prefix,
|
||||
amduat_fed_replay_view_t *out_view) {
|
||||
return amduat_fed_replay_build(records,
|
||||
count,
|
||||
domain_id,
|
||||
snapshot_id,
|
||||
log_prefix,
|
||||
out_view);
|
||||
}
|
||||
|
||||
void amduat_fed_replay_view_free(amduat_fed_replay_view_t *view) {
|
||||
size_t i;
|
||||
|
||||
if (view == NULL) {
|
||||
return;
|
||||
}
|
||||
if (view->records != NULL) {
|
||||
for (i = 0; i < view->len; ++i) {
|
||||
amduat_fed_record_free(&view->records[i]);
|
||||
}
|
||||
free(view->records);
|
||||
}
|
||||
view->records = NULL;
|
||||
view->len = 0;
|
||||
}
|
||||
368
src/near_core/fed/view.c
Normal file
368
src/near_core/fed/view.c
Normal file
|
|
@ -0,0 +1,368 @@
|
|||
#include "amduat/fed/view.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static bool amduat_fed_record_clone(const amduat_fed_record_t *src,
|
||||
amduat_fed_record_t *out) {
|
||||
if (src == NULL || out == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out = *src;
|
||||
if (!amduat_reference_clone(src->id.ref, &out->id.ref)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static void amduat_fed_record_free(amduat_fed_record_t *record) {
|
||||
if (record == NULL) {
|
||||
return;
|
||||
}
|
||||
amduat_reference_free(&record->id.ref);
|
||||
memset(record, 0, sizeof(*record));
|
||||
}
|
||||
|
||||
static bool amduat_fed_record_equivalent(const amduat_fed_record_t *a,
|
||||
const amduat_fed_record_t *b) {
|
||||
return a->meta.domain_id == b->meta.domain_id &&
|
||||
a->meta.visibility == b->meta.visibility &&
|
||||
a->meta.has_source == b->meta.has_source &&
|
||||
a->meta.source_domain == b->meta.source_domain &&
|
||||
a->id.type == b->id.type &&
|
||||
amduat_reference_eq(a->id.ref, b->id.ref) &&
|
||||
a->logseq == b->logseq &&
|
||||
a->snapshot_id == b->snapshot_id &&
|
||||
a->log_prefix == b->log_prefix;
|
||||
}
|
||||
|
||||
static const amduat_fed_record_t *amduat_fed_view_find(
|
||||
const amduat_fed_view_t *view,
|
||||
const amduat_fed_record_t *record) {
|
||||
size_t i;
|
||||
|
||||
for (i = 0; i < view->len; ++i) {
|
||||
const amduat_fed_record_t *existing = &view->records[i];
|
||||
|
||||
if (existing->id.type != record->id.type) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_reference_eq(existing->id.ref, record->id.ref)) {
|
||||
continue;
|
||||
}
|
||||
return existing;
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static bool amduat_fed_policy_deny_clone(const amduat_fed_policy_deny_t *src,
|
||||
amduat_fed_policy_deny_t *out) {
|
||||
if (src == NULL || out == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out = *src;
|
||||
if (!amduat_reference_clone(src->id.ref, &out->id.ref)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static void amduat_fed_policy_deny_free(amduat_fed_policy_deny_t *deny) {
|
||||
if (deny == NULL) {
|
||||
return;
|
||||
}
|
||||
amduat_reference_free(&deny->id.ref);
|
||||
memset(deny, 0, sizeof(*deny));
|
||||
}
|
||||
|
||||
static amduat_fed_view_error_t amduat_fed_view_append(
|
||||
amduat_fed_view_t *view,
|
||||
const amduat_fed_record_t *records,
|
||||
size_t count) {
|
||||
size_t i;
|
||||
size_t base_len;
|
||||
|
||||
if (count == 0u) {
|
||||
return AMDUAT_FED_VIEW_OK;
|
||||
}
|
||||
if (view->len > SIZE_MAX - count) {
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
if (view->records == NULL) {
|
||||
view->records = (amduat_fed_record_t *)calloc(count, sizeof(*view->records));
|
||||
if (view->records == NULL) {
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
} else {
|
||||
amduat_fed_record_t *next =
|
||||
(amduat_fed_record_t *)realloc(view->records,
|
||||
(view->len + count) *
|
||||
sizeof(*view->records));
|
||||
if (next == NULL) {
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
view->records = next;
|
||||
}
|
||||
|
||||
base_len = view->len;
|
||||
for (i = 0; i < count; ++i) {
|
||||
const amduat_fed_record_t *existing = amduat_fed_view_find(view,
|
||||
&records[i]);
|
||||
|
||||
if (existing != NULL) {
|
||||
if (!amduat_fed_record_equivalent(existing, &records[i])) {
|
||||
size_t j;
|
||||
for (j = base_len; j < view->len; ++j) {
|
||||
amduat_fed_record_free(&view->records[j]);
|
||||
}
|
||||
view->len = base_len;
|
||||
return AMDUAT_FED_VIEW_ERR_CONFLICT;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!amduat_fed_record_clone(&records[i],
|
||||
&view->records[view->len])) {
|
||||
size_t j;
|
||||
for (j = base_len; j < view->len; ++j) {
|
||||
amduat_fed_record_free(&view->records[j]);
|
||||
}
|
||||
view->len = base_len;
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
view->len++;
|
||||
}
|
||||
return AMDUAT_FED_VIEW_OK;
|
||||
}
|
||||
|
||||
amduat_fed_view_error_t amduat_fed_view_build(
|
||||
const amduat_fed_record_t *records,
|
||||
size_t count,
|
||||
uint32_t local_domain_id,
|
||||
const amduat_fed_view_bounds_t *bounds,
|
||||
size_t bounds_len,
|
||||
const amduat_fed_policy_deny_t *denies,
|
||||
size_t denies_len,
|
||||
amduat_fed_view_t *out_view) {
|
||||
size_t i;
|
||||
amduat_fed_view_t view;
|
||||
|
||||
if (out_view == NULL) {
|
||||
return AMDUAT_FED_VIEW_ERR_INVALID;
|
||||
}
|
||||
out_view->records = NULL;
|
||||
out_view->len = 0;
|
||||
out_view->local_domain_id = local_domain_id;
|
||||
out_view->denies = NULL;
|
||||
out_view->denies_len = 0;
|
||||
|
||||
if (records == NULL && count != 0u) {
|
||||
return AMDUAT_FED_VIEW_ERR_INVALID;
|
||||
}
|
||||
|
||||
view.records = NULL;
|
||||
view.len = 0;
|
||||
view.local_domain_id = local_domain_id;
|
||||
view.denies = NULL;
|
||||
view.denies_len = 0;
|
||||
|
||||
if (denies_len != 0u && denies == NULL) {
|
||||
return AMDUAT_FED_VIEW_ERR_INVALID;
|
||||
}
|
||||
if (denies_len != 0u) {
|
||||
view.denies = (amduat_fed_policy_deny_t *)calloc(denies_len,
|
||||
sizeof(*view.denies));
|
||||
if (view.denies == NULL) {
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
for (i = 0; i < denies_len; ++i) {
|
||||
if (!amduat_fed_policy_deny_clone(&denies[i], &view.denies[i])) {
|
||||
size_t j;
|
||||
for (j = 0; j < i; ++j) {
|
||||
amduat_fed_policy_deny_free(&view.denies[j]);
|
||||
}
|
||||
free(view.denies);
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
}
|
||||
view.denies_len = denies_len;
|
||||
}
|
||||
|
||||
for (i = 0; i < bounds_len; ++i) {
|
||||
amduat_fed_replay_view_t domain_view;
|
||||
amduat_fed_view_bounds_t bound = bounds[i];
|
||||
size_t j;
|
||||
size_t domain_count = 0;
|
||||
amduat_fed_record_t *domain_records = NULL;
|
||||
bool ok;
|
||||
|
||||
for (j = 0; j < count; ++j) {
|
||||
const amduat_fed_record_t *record = &records[j];
|
||||
|
||||
if (record->meta.domain_id != bound.domain_id) {
|
||||
continue;
|
||||
}
|
||||
if (bound.domain_id != local_domain_id && record->meta.visibility != 1u) {
|
||||
continue;
|
||||
}
|
||||
domain_count++;
|
||||
}
|
||||
|
||||
if (domain_count == 0u) {
|
||||
continue;
|
||||
}
|
||||
|
||||
domain_records =
|
||||
(amduat_fed_record_t *)calloc(domain_count, sizeof(*domain_records));
|
||||
if (domain_records == NULL) {
|
||||
amduat_fed_view_free(&view);
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
|
||||
domain_count = 0;
|
||||
for (j = 0; j < count; ++j) {
|
||||
const amduat_fed_record_t *record = &records[j];
|
||||
|
||||
if (record->meta.domain_id != bound.domain_id) {
|
||||
continue;
|
||||
}
|
||||
if (bound.domain_id != local_domain_id && record->meta.visibility != 1u) {
|
||||
continue;
|
||||
}
|
||||
if (!amduat_fed_record_clone(record, &domain_records[domain_count])) {
|
||||
size_t k;
|
||||
for (k = 0; k < domain_count; ++k) {
|
||||
amduat_fed_record_free(&domain_records[k]);
|
||||
}
|
||||
free(domain_records);
|
||||
amduat_fed_view_free(&view);
|
||||
return AMDUAT_FED_VIEW_ERR_OOM;
|
||||
}
|
||||
domain_count++;
|
||||
}
|
||||
|
||||
ok = amduat_fed_replay_build(domain_records,
|
||||
domain_count,
|
||||
bound.domain_id,
|
||||
bound.snapshot_id,
|
||||
bound.log_prefix,
|
||||
&domain_view);
|
||||
for (j = 0; j < domain_count; ++j) {
|
||||
amduat_fed_record_free(&domain_records[j]);
|
||||
}
|
||||
free(domain_records);
|
||||
if (!ok) {
|
||||
amduat_fed_view_free(&view);
|
||||
return AMDUAT_FED_VIEW_ERR_INVALID;
|
||||
}
|
||||
{
|
||||
amduat_fed_view_error_t append_err;
|
||||
|
||||
append_err = amduat_fed_view_append(&view,
|
||||
domain_view.records,
|
||||
domain_view.len);
|
||||
if (append_err != AMDUAT_FED_VIEW_OK) {
|
||||
amduat_fed_replay_view_free(&domain_view);
|
||||
amduat_fed_view_free(&view);
|
||||
return append_err;
|
||||
}
|
||||
}
|
||||
amduat_fed_replay_view_free(&domain_view);
|
||||
}
|
||||
|
||||
*out_view = view;
|
||||
return AMDUAT_FED_VIEW_OK;
|
||||
}
|
||||
|
||||
void amduat_fed_view_free(amduat_fed_view_t *view) {
|
||||
size_t i;
|
||||
|
||||
if (view == NULL) {
|
||||
return;
|
||||
}
|
||||
if (view->records != NULL) {
|
||||
for (i = 0; i < view->len; ++i) {
|
||||
amduat_fed_record_free(&view->records[i]);
|
||||
}
|
||||
free(view->records);
|
||||
}
|
||||
if (view->denies != NULL) {
|
||||
for (i = 0; i < view->denies_len; ++i) {
|
||||
amduat_fed_policy_deny_free(&view->denies[i]);
|
||||
}
|
||||
free(view->denies);
|
||||
}
|
||||
view->records = NULL;
|
||||
view->len = 0;
|
||||
view->local_domain_id = 0;
|
||||
view->denies = NULL;
|
||||
view->denies_len = 0;
|
||||
}
|
||||
|
||||
static const amduat_fed_record_t *amduat_fed_view_lookup(
|
||||
const amduat_fed_view_t *view,
|
||||
amduat_reference_t ref) {
|
||||
size_t i;
|
||||
|
||||
if (view == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
for (i = 0; i < view->len; ++i) {
|
||||
if (amduat_reference_eq(view->records[i].id.ref, ref)) {
|
||||
return &view->records[i];
|
||||
}
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static bool amduat_fed_view_policy_denied(const amduat_fed_view_t *view,
|
||||
amduat_reference_t ref) {
|
||||
size_t i;
|
||||
|
||||
if (view == NULL) {
|
||||
return false;
|
||||
}
|
||||
for (i = 0; i < view->denies_len; ++i) {
|
||||
if (amduat_reference_eq(view->denies[i].id.ref, ref)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_fed_resolve_error_t amduat_fed_resolve(
|
||||
const amduat_fed_view_t *view,
|
||||
amduat_asl_store_t *local_store,
|
||||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
amduat_asl_store_error_t store_err;
|
||||
const amduat_fed_record_t *record;
|
||||
|
||||
if (local_store == NULL || out_artifact == NULL) {
|
||||
return AMDUAT_FED_RESOLVE_STORE_ERROR;
|
||||
}
|
||||
|
||||
store_err = amduat_asl_store_get(local_store, ref, out_artifact);
|
||||
if (store_err == AMDUAT_ASL_STORE_OK) {
|
||||
return AMDUAT_FED_RESOLVE_OK;
|
||||
}
|
||||
if (store_err == AMDUAT_ASL_STORE_ERR_INTEGRITY) {
|
||||
return AMDUAT_FED_RESOLVE_INTEGRITY_ERROR;
|
||||
}
|
||||
if (store_err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
|
||||
return AMDUAT_FED_RESOLVE_STORE_ERROR;
|
||||
}
|
||||
|
||||
if (amduat_fed_view_policy_denied(view, ref)) {
|
||||
return AMDUAT_FED_RESOLVE_POLICY_DENIED;
|
||||
}
|
||||
|
||||
record = amduat_fed_view_lookup(view, ref);
|
||||
if (record == NULL) {
|
||||
return AMDUAT_FED_RESOLVE_NOT_FOUND;
|
||||
}
|
||||
if (record->meta.domain_id != view->local_domain_id) {
|
||||
return AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES;
|
||||
}
|
||||
return AMDUAT_FED_RESOLVE_NOT_FOUND;
|
||||
}
|
||||
|
|
@ -1,11 +1,13 @@
|
|||
#include "amduat/fer/receipt.h"
|
||||
|
||||
#include "amduat/enc/fer1_receipt.h"
|
||||
#include "amduat/pel/run.h"
|
||||
|
||||
#include <string.h>
|
||||
|
||||
bool amduat_fer1_receipt_from_pel_result(
|
||||
static bool amduat_fer1_receipt_from_pel_result_with_output_ref(
|
||||
const amduat_pel_surface_execution_result_t *pel_result,
|
||||
amduat_reference_t output_ref,
|
||||
amduat_reference_t input_manifest_ref,
|
||||
amduat_reference_t environment_ref,
|
||||
amduat_octets_t evaluator_id,
|
||||
|
|
@ -29,9 +31,6 @@ bool amduat_fer1_receipt_from_pel_result(
|
|||
if (pel_result == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (pel_result->output_refs_len != 1 || pel_result->output_refs == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (evaluator_id.len != 0 && evaluator_id.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
|
@ -45,7 +44,7 @@ bool amduat_fer1_receipt_from_pel_result(
|
|||
receipt.input_manifest_ref = input_manifest_ref;
|
||||
receipt.environment_ref = environment_ref;
|
||||
receipt.evaluator_id = evaluator_id;
|
||||
receipt.output_ref = pel_result->output_refs[0];
|
||||
receipt.output_ref = output_ref;
|
||||
receipt.started_at = started_at;
|
||||
receipt.completed_at = completed_at;
|
||||
|
||||
|
|
@ -73,3 +72,206 @@ bool amduat_fer1_receipt_from_pel_result(
|
|||
amduat_type_tag(AMDUAT_TYPE_TAG_FER1_RECEIPT_1));
|
||||
return true;
|
||||
}
|
||||
|
||||
bool amduat_fer1_receipt_from_pel_result(
|
||||
const amduat_pel_surface_execution_result_t *pel_result,
|
||||
amduat_reference_t input_manifest_ref,
|
||||
amduat_reference_t environment_ref,
|
||||
amduat_octets_t evaluator_id,
|
||||
amduat_reference_t executor_ref,
|
||||
bool has_sbom_ref,
|
||||
amduat_reference_t sbom_ref,
|
||||
amduat_octets_t parity_digest,
|
||||
uint64_t started_at,
|
||||
uint64_t completed_at,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
if (pel_result == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (pel_result->output_refs_len != 1 || pel_result->output_refs == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return amduat_fer1_receipt_from_pel_result_with_output_ref(
|
||||
pel_result,
|
||||
pel_result->output_refs[0],
|
||||
input_manifest_ref,
|
||||
environment_ref,
|
||||
evaluator_id,
|
||||
executor_ref,
|
||||
has_sbom_ref,
|
||||
sbom_ref,
|
||||
parity_digest,
|
||||
started_at,
|
||||
completed_at,
|
||||
out_artifact);
|
||||
}
|
||||
|
||||
bool amduat_fer1_receipt_from_pel_run(
|
||||
const amduat_pel_run_result_t *pel_run,
|
||||
amduat_reference_t input_manifest_ref,
|
||||
amduat_reference_t environment_ref,
|
||||
amduat_octets_t evaluator_id,
|
||||
amduat_reference_t executor_ref,
|
||||
bool has_sbom_ref,
|
||||
amduat_reference_t sbom_ref,
|
||||
amduat_octets_t parity_digest,
|
||||
uint64_t started_at,
|
||||
uint64_t completed_at,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
amduat_reference_t output_ref;
|
||||
|
||||
if (pel_run == NULL || !pel_run->has_result_value) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (pel_run->output_refs_len == 1 && pel_run->output_refs != NULL) {
|
||||
output_ref = pel_run->output_refs[0];
|
||||
} else if (pel_run->output_refs_len == 0) {
|
||||
output_ref = pel_run->result_ref;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
|
||||
return amduat_fer1_receipt_from_pel_result_with_output_ref(
|
||||
&pel_run->result_value,
|
||||
output_ref,
|
||||
input_manifest_ref,
|
||||
environment_ref,
|
||||
evaluator_id,
|
||||
executor_ref,
|
||||
has_sbom_ref,
|
||||
sbom_ref,
|
||||
parity_digest,
|
||||
started_at,
|
||||
completed_at,
|
||||
out_artifact);
|
||||
}
|
||||
|
||||
bool amduat_fer1_receipt_from_pel_run_v1_1(
|
||||
const amduat_pel_run_result_t *pel_run,
|
||||
amduat_reference_t input_manifest_ref,
|
||||
amduat_reference_t environment_ref,
|
||||
amduat_octets_t evaluator_id,
|
||||
amduat_reference_t executor_ref,
|
||||
bool has_sbom_ref,
|
||||
amduat_reference_t sbom_ref,
|
||||
amduat_octets_t parity_digest,
|
||||
uint64_t started_at,
|
||||
uint64_t completed_at,
|
||||
bool has_executor_fingerprint_ref,
|
||||
amduat_reference_t executor_fingerprint_ref,
|
||||
bool has_run_id,
|
||||
amduat_octets_t run_id,
|
||||
bool has_limits,
|
||||
amduat_fer1_limits_t limits,
|
||||
const amduat_fer1_log_entry_t *logs,
|
||||
size_t logs_len,
|
||||
bool has_determinism,
|
||||
uint8_t determinism_level,
|
||||
bool has_rng_seed,
|
||||
amduat_octets_t rng_seed,
|
||||
bool has_signature,
|
||||
amduat_octets_t signature,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
amduat_fer1_receipt_t receipt;
|
||||
amduat_fer1_parity_entry_t parity;
|
||||
amduat_reference_t executor_refs[1];
|
||||
amduat_reference_t output_ref;
|
||||
amduat_octets_t receipt_bytes;
|
||||
|
||||
if (out_artifact == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_artifact = amduat_artifact(amduat_octets(NULL, 0u));
|
||||
|
||||
if (pel_run == NULL || !pel_run->has_result_value) {
|
||||
return false;
|
||||
}
|
||||
if (pel_run->output_refs_len == 1 && pel_run->output_refs != NULL) {
|
||||
output_ref = pel_run->output_refs[0];
|
||||
} else if (pel_run->output_refs_len == 0) {
|
||||
output_ref = pel_run->result_ref;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
if (evaluator_id.len != 0 && evaluator_id.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (parity_digest.len != 0 && parity_digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (has_run_id && run_id.len != 0 && run_id.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (has_rng_seed && rng_seed.len != 0 && rng_seed.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (has_signature && signature.len != 0 && signature.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (logs_len != 0 && logs == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
memset(&receipt, 0, sizeof(receipt));
|
||||
receipt.fer1_version = AMDUAT_FER1_VERSION_1_1;
|
||||
receipt.function_ref = pel_run->result_value.program_ref;
|
||||
receipt.input_manifest_ref = input_manifest_ref;
|
||||
receipt.environment_ref = environment_ref;
|
||||
receipt.evaluator_id = evaluator_id;
|
||||
receipt.output_ref = output_ref;
|
||||
receipt.started_at = started_at;
|
||||
receipt.completed_at = completed_at;
|
||||
|
||||
executor_refs[0] = executor_ref;
|
||||
receipt.executor_refs = executor_refs;
|
||||
receipt.executor_refs_len = 1;
|
||||
|
||||
memset(&parity, 0, sizeof(parity));
|
||||
parity.executor_ref = executor_ref;
|
||||
parity.output_ref = receipt.output_ref;
|
||||
parity.has_sbom_ref = has_sbom_ref;
|
||||
if (has_sbom_ref) {
|
||||
parity.sbom_ref = sbom_ref;
|
||||
}
|
||||
parity.parity_digest = parity_digest;
|
||||
receipt.parity = &parity;
|
||||
receipt.parity_len = 1;
|
||||
|
||||
receipt.has_executor_fingerprint_ref = has_executor_fingerprint_ref;
|
||||
if (has_executor_fingerprint_ref) {
|
||||
receipt.executor_fingerprint_ref = executor_fingerprint_ref;
|
||||
}
|
||||
receipt.has_run_id = has_run_id;
|
||||
if (has_run_id) {
|
||||
receipt.run_id = run_id;
|
||||
}
|
||||
receipt.has_limits = has_limits;
|
||||
if (has_limits) {
|
||||
receipt.limits = limits;
|
||||
}
|
||||
receipt.logs = (amduat_fer1_log_entry_t *)logs;
|
||||
receipt.logs_len = logs_len;
|
||||
receipt.has_determinism = has_determinism;
|
||||
if (has_determinism) {
|
||||
receipt.determinism_level = determinism_level;
|
||||
}
|
||||
receipt.has_rng_seed = has_rng_seed;
|
||||
if (has_rng_seed) {
|
||||
receipt.rng_seed = rng_seed;
|
||||
}
|
||||
receipt.has_signature = has_signature;
|
||||
if (has_signature) {
|
||||
receipt.signature = signature;
|
||||
}
|
||||
|
||||
if (!amduat_enc_fer1_receipt_encode_v1_1(&receipt, &receipt_bytes)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
*out_artifact = amduat_artifact_with_type(
|
||||
amduat_octets(receipt_bytes.data, receipt_bytes.len),
|
||||
amduat_type_tag(AMDUAT_TYPE_TAG_FER1_RECEIPT_1));
|
||||
return true;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -95,6 +95,16 @@ const char *amduat_format_pel_kernel_kind_name(
|
|||
return "const";
|
||||
case AMDUAT_PEL_KERNEL_OP_HASH_ASL1:
|
||||
return "hash_asl1";
|
||||
case AMDUAT_PEL_KERNEL_OP_PARAMS:
|
||||
return "params";
|
||||
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
|
||||
return "format_encode";
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
|
||||
return "collection_snapshot_decode";
|
||||
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
|
||||
return "log_read_range";
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
|
||||
return "collection_merge_refs";
|
||||
default:
|
||||
return "unknown";
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
#include "amduat/pel/opreg_kernel.h"
|
||||
#include "amduat/pel/opreg_kernel_params.h"
|
||||
#include "kernel_collection.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
|
@ -64,8 +65,32 @@ static const amduat_pel_kernel_op_desc_t k_kernel_descs[] = {
|
|||
3,
|
||||
1
|
||||
},
|
||||
{
|
||||
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_SNAPSHOT_DECODE,
|
||||
1,
|
||||
1,
|
||||
1
|
||||
},
|
||||
{
|
||||
AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_LOG_READ_RANGE,
|
||||
3,
|
||||
3,
|
||||
1
|
||||
},
|
||||
{
|
||||
AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS,
|
||||
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_MERGE_REFS,
|
||||
4,
|
||||
4,
|
||||
1
|
||||
},
|
||||
};
|
||||
|
||||
static amduat_pel_kernel_artifact_resolver_t g_resolver = NULL;
|
||||
static void *g_resolver_ctx = NULL;
|
||||
|
||||
const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
|
||||
amduat_octets_t name,
|
||||
uint32_t version) {
|
||||
|
|
@ -90,6 +115,16 @@ const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
|
|||
if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME)) {
|
||||
return &k_kernel_descs[5];
|
||||
}
|
||||
if (amduat_name_eq(name,
|
||||
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME)) {
|
||||
return &k_kernel_descs[6];
|
||||
}
|
||||
if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME)) {
|
||||
return &k_kernel_descs[7];
|
||||
}
|
||||
if (amduat_name_eq(name, AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME)) {
|
||||
return &k_kernel_descs[8];
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
|
|
@ -115,11 +150,37 @@ const char *amduat_pel_kernel_op_name(amduat_pel_kernel_op_kind_t kind) {
|
|||
return AMDUAT_PEL_KERNEL_OP_PARAMS_NAME;
|
||||
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
|
||||
return AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME;
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
|
||||
return AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME;
|
||||
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
|
||||
return AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME;
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
|
||||
return AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME;
|
||||
default:
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
void amduat_pel_kernel_set_artifact_resolver(
|
||||
amduat_pel_kernel_artifact_resolver_t resolver,
|
||||
void *ctx) {
|
||||
g_resolver = resolver;
|
||||
g_resolver_ctx = ctx;
|
||||
}
|
||||
|
||||
void amduat_pel_kernel_clear_artifact_resolver(void) {
|
||||
g_resolver = NULL;
|
||||
g_resolver_ctx = NULL;
|
||||
}
|
||||
|
||||
bool amduat_pel_kernel_resolve_artifact(amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
if (g_resolver == NULL) {
|
||||
return false;
|
||||
}
|
||||
return g_resolver(ref, out_artifact, g_resolver_ctx);
|
||||
}
|
||||
|
||||
static bool amduat_type_tags_match(const amduat_artifact_t *a,
|
||||
const amduat_artifact_t *b) {
|
||||
if (a->has_type_tag != b->has_type_tag) {
|
||||
|
|
@ -1537,6 +1598,15 @@ bool amduat_pel_kernel_op_eval(
|
|||
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
|
||||
return amduat_kernel_format_encode(inputs, inputs_len, out_outputs,
|
||||
out_outputs_len, out_status_code);
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
|
||||
return amduat_kernel_collection_snapshot_decode(
|
||||
inputs, inputs_len, out_outputs, out_outputs_len, out_status_code);
|
||||
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
|
||||
return amduat_kernel_log_read_range(
|
||||
inputs, inputs_len, out_outputs, out_outputs_len, out_status_code);
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
|
||||
return amduat_kernel_collection_merge_refs(
|
||||
inputs, inputs_len, out_outputs, out_outputs_len, out_status_code);
|
||||
default:
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
|
||||
return false;
|
||||
|
|
|
|||
615
src/pel_stack/opreg/kernel_collection.c
Normal file
615
src/pel_stack/opreg/kernel_collection.c
Normal file
|
|
@ -0,0 +1,615 @@
|
|||
#include "kernel_collection.h"
|
||||
|
||||
#include "amduat/asl/collection.h"
|
||||
#include "amduat/asl/collection_view.h"
|
||||
#include "amduat/asl/log_store.h"
|
||||
#include "amduat/asl/none.h"
|
||||
#include "amduat/asl/record.h"
|
||||
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static bool amduat_alloc_outputs(size_t count,
|
||||
amduat_artifact_t **out_outputs) {
|
||||
amduat_artifact_t *outputs;
|
||||
|
||||
if (out_outputs == NULL) {
|
||||
return false;
|
||||
}
|
||||
outputs = (amduat_artifact_t *)calloc(count, sizeof(*outputs));
|
||||
if (outputs == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_outputs = outputs;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_load_u64_le(amduat_octets_t bytes,
|
||||
size_t offset,
|
||||
uint64_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (out == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len - offset < 8u || bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
data = bytes.data + offset;
|
||||
*out = (uint64_t)data[0] |
|
||||
((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) |
|
||||
((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) |
|
||||
((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) |
|
||||
((uint64_t)data[7] << 56);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_load_u32_le(amduat_octets_t bytes,
|
||||
size_t offset,
|
||||
uint32_t *out) {
|
||||
const uint8_t *data;
|
||||
|
||||
if (out == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (bytes.len - offset < 4u || bytes.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
data = bytes.data + offset;
|
||||
*out = (uint32_t)data[0] |
|
||||
((uint32_t)data[1] << 8) |
|
||||
((uint32_t)data[2] << 16) |
|
||||
((uint32_t)data[3] << 24);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool amduat_decode_params(const amduat_artifact_t *artifact,
|
||||
uint64_t *out_from,
|
||||
uint32_t *out_limit) {
|
||||
const char k_params_schema[] = "pel/params";
|
||||
amduat_asl_record_t record;
|
||||
bool ok = false;
|
||||
|
||||
if (artifact == NULL || out_from == NULL || out_limit == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (!artifact->has_type_tag ||
|
||||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_record_decode_v1(artifact->bytes, &record)) {
|
||||
return false;
|
||||
}
|
||||
if (record.schema.len != strlen(k_params_schema) ||
|
||||
record.schema.data == NULL ||
|
||||
memcmp(record.schema.data, k_params_schema,
|
||||
record.schema.len) != 0) {
|
||||
amduat_asl_record_free(&record);
|
||||
return false;
|
||||
}
|
||||
if (record.payload.len != 12u) {
|
||||
amduat_asl_record_free(&record);
|
||||
return false;
|
||||
}
|
||||
if (!amduat_load_u64_le(record.payload, 0u, out_from) ||
|
||||
!amduat_load_u32_le(record.payload, 8u, out_limit)) {
|
||||
amduat_asl_record_free(&record);
|
||||
return false;
|
||||
}
|
||||
ok = true;
|
||||
amduat_asl_record_free(&record);
|
||||
return ok;
|
||||
}
|
||||
|
||||
static bool amduat_log_chunk_from_artifact(
|
||||
const amduat_artifact_t *artifact,
|
||||
amduat_asl_log_chunk_t *out_chunk) {
|
||||
if (artifact == NULL || out_chunk == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (!artifact->has_type_tag ||
|
||||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_LOG_CHUNK_1) {
|
||||
return false;
|
||||
}
|
||||
return amduat_asl_log_chunk_decode_v1(artifact->bytes, out_chunk) ==
|
||||
AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
bool amduat_kernel_collection_snapshot_decode(
|
||||
const amduat_artifact_t *inputs,
|
||||
size_t inputs_len,
|
||||
amduat_artifact_t **out_outputs,
|
||||
size_t *out_outputs_len,
|
||||
uint32_t *out_status_code) {
|
||||
amduat_asl_collection_snapshot_payload_t snapshot;
|
||||
amduat_asl_collection_snapshot_info_t info;
|
||||
amduat_asl_record_t record;
|
||||
amduat_octets_t encoded = amduat_octets(NULL, 0u);
|
||||
bool ok = false;
|
||||
|
||||
if (inputs == NULL || inputs_len != 1u) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
memset(&snapshot, 0, sizeof(snapshot));
|
||||
memset(&info, 0, sizeof(info));
|
||||
memset(&record, 0, sizeof(record));
|
||||
|
||||
if (amduat_asl_none_is_artifact(&inputs[0])) {
|
||||
info.snapshot_at_offset = 0u;
|
||||
info.refs = NULL;
|
||||
info.refs_len = 0u;
|
||||
} else {
|
||||
if (!inputs[0].has_type_tag ||
|
||||
inputs[0].type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_record_decode_v1(inputs[0].bytes, &record)) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (record.schema.len != strlen("collection/snapshot") ||
|
||||
record.schema.data == NULL ||
|
||||
memcmp(record.schema.data, "collection/snapshot",
|
||||
record.schema.len) != 0) {
|
||||
amduat_asl_record_free(&record);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (!amduat_asl_collection_snapshot_payload_decode_v1(record.payload,
|
||||
&snapshot)) {
|
||||
amduat_asl_record_free(&record);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
info.snapshot_at_offset = snapshot.snapshot_offset;
|
||||
info.refs = snapshot.refs;
|
||||
info.refs_len = snapshot.refs_len;
|
||||
amduat_asl_record_free(&record);
|
||||
}
|
||||
|
||||
if (!amduat_asl_collection_snapshot_info_encode_v1(&info, &encoded)) {
|
||||
amduat_asl_collection_snapshot_payload_free(&snapshot);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_alloc_outputs(1u, out_outputs)) {
|
||||
amduat_asl_collection_snapshot_payload_free(&snapshot);
|
||||
free((void *)encoded.data);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
(*out_outputs)[0] = amduat_artifact_with_type(
|
||||
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1));
|
||||
if (out_outputs_len != NULL) {
|
||||
*out_outputs_len = 1u;
|
||||
}
|
||||
ok = true;
|
||||
|
||||
amduat_asl_collection_snapshot_payload_free(&snapshot);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_kernel_log_read_range(
|
||||
const amduat_artifact_t *inputs,
|
||||
size_t inputs_len,
|
||||
amduat_artifact_t **out_outputs,
|
||||
size_t *out_outputs_len,
|
||||
uint32_t *out_status_code) {
|
||||
amduat_asl_log_range_t range;
|
||||
amduat_octets_t encoded = amduat_octets(NULL, 0u);
|
||||
uint64_t from_offset = 0u;
|
||||
uint32_t limit = 0u;
|
||||
uint64_t limit_from_offset = 0u;
|
||||
uint32_t limit_value = 0u;
|
||||
bool ok = false;
|
||||
|
||||
if (inputs == NULL || inputs_len != 3u) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_decode_params(&inputs[1], &from_offset, &limit_value)) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (!amduat_decode_params(&inputs[2], &limit_from_offset, &limit)) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
(void)limit_from_offset;
|
||||
(void)limit_value;
|
||||
|
||||
memset(&range, 0, sizeof(range));
|
||||
range.start_offset = from_offset;
|
||||
range.next_offset = from_offset;
|
||||
range.refs = NULL;
|
||||
range.refs_len = 0u;
|
||||
|
||||
if (limit == 0u || amduat_asl_none_is_artifact(&inputs[0])) {
|
||||
goto encode_range;
|
||||
}
|
||||
|
||||
{
|
||||
amduat_asl_log_chunk_t *chunks = NULL;
|
||||
size_t chunks_len = 0u;
|
||||
amduat_asl_log_chunk_t head_chunk;
|
||||
uint64_t head_end_offset = 0u;
|
||||
if (!amduat_log_chunk_from_artifact(&inputs[0], &head_chunk)) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (head_chunk.base_offset >
|
||||
UINT64_MAX - (uint64_t)head_chunk.entry_count) {
|
||||
amduat_asl_log_chunk_free(&head_chunk);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
head_end_offset = head_chunk.base_offset + head_chunk.entry_count;
|
||||
|
||||
{
|
||||
amduat_asl_log_chunk_t *next =
|
||||
(amduat_asl_log_chunk_t *)realloc(
|
||||
chunks, (chunks_len + 1u) * sizeof(*chunks));
|
||||
if (next == NULL) {
|
||||
amduat_asl_log_chunk_free(&head_chunk);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
chunks = next;
|
||||
chunks[chunks_len++] = head_chunk;
|
||||
}
|
||||
|
||||
if (from_offset >= head_end_offset) {
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
range.next_offset = from_offset;
|
||||
goto encode_range;
|
||||
}
|
||||
|
||||
while (true) {
|
||||
amduat_asl_log_chunk_t chunk;
|
||||
amduat_artifact_t prev_artifact;
|
||||
|
||||
if (chunks[chunks_len - 1u].base_offset <= from_offset ||
|
||||
!chunks[chunks_len - 1u].has_prev) {
|
||||
break;
|
||||
}
|
||||
|
||||
memset(&prev_artifact, 0, sizeof(prev_artifact));
|
||||
if (!amduat_pel_kernel_resolve_artifact(
|
||||
chunks[chunks_len - 1u].prev_ref, &prev_artifact)) {
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_REF_RESOLVE_FAILED;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (!amduat_log_chunk_from_artifact(&prev_artifact, &chunk)) {
|
||||
amduat_artifact_free(&prev_artifact);
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
amduat_artifact_free(&prev_artifact);
|
||||
|
||||
{
|
||||
amduat_asl_log_chunk_t *next =
|
||||
(amduat_asl_log_chunk_t *)realloc(
|
||||
chunks, (chunks_len + 1u) * sizeof(*chunks));
|
||||
if (next == NULL) {
|
||||
amduat_asl_log_chunk_free(&chunk);
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
chunks = next;
|
||||
chunks[chunks_len++] = chunk;
|
||||
}
|
||||
}
|
||||
|
||||
if (limit > SIZE_MAX) {
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (limit != 0u) {
|
||||
range.refs = (amduat_reference_t *)calloc(limit, sizeof(*range.refs));
|
||||
if (range.refs == NULL) {
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
for (size_t i = 0u; i < chunks_len / 2u; ++i) {
|
||||
amduat_asl_log_chunk_t tmp = chunks[i];
|
||||
chunks[i] = chunks[chunks_len - 1u - i];
|
||||
chunks[chunks_len - 1u - i] = tmp;
|
||||
}
|
||||
|
||||
range.refs_len = 0u;
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_t *chunk = &chunks[i];
|
||||
for (uint32_t j = 0u; j < chunk->entry_count; ++j) {
|
||||
uint64_t offset = chunk->base_offset + j;
|
||||
if (offset < from_offset) {
|
||||
continue;
|
||||
}
|
||||
if (range.refs_len == limit) {
|
||||
goto done_collect;
|
||||
}
|
||||
if (!amduat_reference_clone(chunk->entries[j].payload_ref,
|
||||
&range.refs[range.refs_len])) {
|
||||
for (size_t k = 0u; k < chunks_len; ++k) {
|
||||
amduat_asl_log_chunk_free(&chunks[k]);
|
||||
}
|
||||
free(chunks);
|
||||
amduat_asl_log_range_free(&range);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
range.refs_len++;
|
||||
range.next_offset = offset + 1u;
|
||||
}
|
||||
}
|
||||
|
||||
done_collect:
|
||||
if (range.refs_len < limit) {
|
||||
range.next_offset = head_end_offset;
|
||||
}
|
||||
|
||||
for (size_t i = 0u; i < chunks_len; ++i) {
|
||||
amduat_asl_log_chunk_free(&chunks[i]);
|
||||
}
|
||||
free(chunks);
|
||||
}
|
||||
|
||||
encode_range:
|
||||
if (!amduat_asl_log_range_encode_v1(&range, &encoded)) {
|
||||
amduat_asl_log_range_free(&range);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_alloc_outputs(1u, out_outputs)) {
|
||||
amduat_asl_log_range_free(&range);
|
||||
free((void *)encoded.data);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
(*out_outputs)[0] = amduat_artifact_with_type(
|
||||
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1));
|
||||
if (out_outputs_len != NULL) {
|
||||
*out_outputs_len = 1u;
|
||||
}
|
||||
ok = true;
|
||||
amduat_asl_log_range_free(&range);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool amduat_kernel_collection_merge_refs(
|
||||
const amduat_artifact_t *inputs,
|
||||
size_t inputs_len,
|
||||
amduat_artifact_t **out_outputs,
|
||||
size_t *out_outputs_len,
|
||||
uint32_t *out_status_code) {
|
||||
amduat_asl_collection_snapshot_info_t snapshot;
|
||||
amduat_asl_log_range_t log_range;
|
||||
amduat_asl_collection_view_t view;
|
||||
amduat_octets_t encoded = amduat_octets(NULL, 0u);
|
||||
uint64_t from_offset = 0u;
|
||||
uint32_t limit = 0u;
|
||||
uint64_t limit_from_offset = 0u;
|
||||
uint32_t limit_value = 0u;
|
||||
size_t total_refs = 0u;
|
||||
size_t start_index = 0u;
|
||||
size_t end_index = 0u;
|
||||
bool ok = false;
|
||||
|
||||
if (inputs == NULL || inputs_len != 4u) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_INTERNAL;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_decode_params(&inputs[2], &from_offset, &limit_value) ||
|
||||
!amduat_decode_params(&inputs[3], &limit_from_offset, &limit)) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
(void)limit_from_offset;
|
||||
(void)limit_value;
|
||||
|
||||
memset(&snapshot, 0, sizeof(snapshot));
|
||||
memset(&log_range, 0, sizeof(log_range));
|
||||
memset(&view, 0, sizeof(view));
|
||||
|
||||
if (!inputs[0].has_type_tag ||
|
||||
inputs[0].type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1 ||
|
||||
!amduat_asl_collection_snapshot_info_decode_v1(inputs[0].bytes,
|
||||
&snapshot)) {
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
if (!inputs[1].has_type_tag ||
|
||||
inputs[1].type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1 ||
|
||||
!amduat_asl_log_range_decode_v1(inputs[1].bytes, &log_range)) {
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (snapshot.refs_len > SIZE_MAX - log_range.refs_len) {
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
amduat_asl_log_range_free(&log_range);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_COLLECTION_VIEW_INVALID;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
total_refs = snapshot.refs_len + log_range.refs_len;
|
||||
if (from_offset > SIZE_MAX) {
|
||||
start_index = total_refs;
|
||||
} else {
|
||||
start_index = (size_t)from_offset;
|
||||
if (start_index > total_refs) {
|
||||
start_index = total_refs;
|
||||
}
|
||||
}
|
||||
|
||||
if (limit > SIZE_MAX || start_index > SIZE_MAX - (size_t)limit) {
|
||||
end_index = total_refs;
|
||||
} else {
|
||||
end_index = start_index + (size_t)limit;
|
||||
if (end_index > total_refs) {
|
||||
end_index = total_refs;
|
||||
}
|
||||
}
|
||||
|
||||
view.refs_len = end_index - start_index;
|
||||
view.refs = NULL;
|
||||
if (view.refs_len != 0u) {
|
||||
view.refs = (amduat_reference_t *)calloc(view.refs_len, sizeof(*view.refs));
|
||||
if (view.refs == NULL) {
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
amduat_asl_log_range_free(&log_range);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
for (size_t i = 0u; i < view.refs_len; ++i) {
|
||||
size_t idx = start_index + i;
|
||||
const amduat_reference_t *src =
|
||||
(idx < snapshot.refs_len)
|
||||
? &snapshot.refs[idx]
|
||||
: &log_range.refs[idx - snapshot.refs_len];
|
||||
if (!amduat_reference_clone(*src, &view.refs[i])) {
|
||||
amduat_asl_collection_view_free(&view);
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
amduat_asl_log_range_free(&log_range);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
view.computed_from_offset = from_offset;
|
||||
if (log_range.refs_len < limit) {
|
||||
view.computed_up_to_offset = log_range.next_offset;
|
||||
} else if (from_offset > UINT64_MAX - (uint64_t)view.refs_len) {
|
||||
view.computed_up_to_offset = UINT64_MAX;
|
||||
} else {
|
||||
view.computed_up_to_offset = from_offset + (uint64_t)view.refs_len;
|
||||
}
|
||||
|
||||
if (!amduat_asl_collection_view_encode_v1(&view, &encoded)) {
|
||||
amduat_asl_collection_view_free(&view);
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
amduat_asl_log_range_free(&log_range);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_alloc_outputs(1u, out_outputs)) {
|
||||
amduat_asl_collection_view_free(&view);
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
amduat_asl_log_range_free(&log_range);
|
||||
free((void *)encoded.data);
|
||||
if (out_status_code) {
|
||||
*out_status_code = AMDUAT_PEL_KERNEL_STATUS_OOM;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
(*out_outputs)[0] = amduat_artifact_with_type(
|
||||
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_COLLECTION_VIEW_1));
|
||||
if (out_outputs_len != NULL) {
|
||||
*out_outputs_len = 1u;
|
||||
}
|
||||
ok = true;
|
||||
|
||||
amduat_asl_collection_view_free(&view);
|
||||
amduat_asl_collection_snapshot_info_free(&snapshot);
|
||||
amduat_asl_log_range_free(&log_range);
|
||||
return ok;
|
||||
}
|
||||
29
src/pel_stack/opreg/kernel_collection.h
Normal file
29
src/pel_stack/opreg/kernel_collection.h
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
#ifndef AMDUAT_PEL_OPREG_KERNEL_COLLECTION_H
|
||||
#define AMDUAT_PEL_OPREG_KERNEL_COLLECTION_H
|
||||
|
||||
#include "amduat/pel/opreg_kernel.h"
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
bool amduat_kernel_collection_snapshot_decode(
|
||||
const amduat_artifact_t *inputs,
|
||||
size_t inputs_len,
|
||||
amduat_artifact_t **out_outputs,
|
||||
size_t *out_outputs_len,
|
||||
uint32_t *out_status_code);
|
||||
|
||||
bool amduat_kernel_log_read_range(
|
||||
const amduat_artifact_t *inputs,
|
||||
size_t inputs_len,
|
||||
amduat_artifact_t **out_outputs,
|
||||
size_t *out_outputs_len,
|
||||
uint32_t *out_status_code);
|
||||
|
||||
bool amduat_kernel_collection_merge_refs(
|
||||
const amduat_artifact_t *inputs,
|
||||
size_t inputs_len,
|
||||
amduat_artifact_t **out_outputs,
|
||||
size_t *out_outputs_len,
|
||||
uint32_t *out_status_code);
|
||||
|
||||
#endif /* AMDUAT_PEL_OPREG_KERNEL_COLLECTION_H */
|
||||
|
|
@ -133,6 +133,12 @@ bool amduat_pel_kernel_params_decode(
|
|||
return amduat_decode_unit(params_bytes);
|
||||
case AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE:
|
||||
return amduat_decode_unit(params_bytes);
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE:
|
||||
return amduat_decode_unit(params_bytes);
|
||||
case AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE:
|
||||
return amduat_decode_unit(params_bytes);
|
||||
case AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS:
|
||||
return amduat_decode_unit(params_bytes);
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,12 +1,18 @@
|
|||
#include "amduat/pel/surf.h"
|
||||
|
||||
#include "amduat/asl/asl_materialization_cache_fs.h"
|
||||
#include "amduat/asl/asl_store_fs.h"
|
||||
#include "amduat/enc/pel1_result.h"
|
||||
#include "amduat/enc/pel_program_dag.h"
|
||||
#include "amduat/enc/pel_trace_dag.h"
|
||||
#include "amduat/pel/derivation_sid.h"
|
||||
#include "amduat/pel/opreg_kernel.h"
|
||||
#include "amduat/util/log.h"
|
||||
#include "pel_program_dag_internal.h"
|
||||
#include "amduat/pel/program_dag_desc.h"
|
||||
|
||||
#include <stdarg.h>
|
||||
#include <assert.h>
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
|
@ -30,6 +36,17 @@ void amduat_pel_surf_free_ref(amduat_reference_t *ref) {
|
|||
amduat_reference_free(ref);
|
||||
}
|
||||
|
||||
static bool amduat_pel_surf_resolve_artifact(amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact,
|
||||
void *ctx) {
|
||||
amduat_asl_store_t *store = (amduat_asl_store_t *)ctx;
|
||||
if (store == NULL || out_artifact == NULL) {
|
||||
return false;
|
||||
}
|
||||
return amduat_asl_store_get(store, ref, out_artifact) ==
|
||||
AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
static void amduat_init_core_result(amduat_pel_execution_result_value_t *result,
|
||||
amduat_reference_t scheme_ref,
|
||||
amduat_pel_execution_status_t status,
|
||||
|
|
@ -217,6 +234,37 @@ static void amduat_trace_nodes_free(amduat_pel_node_trace_dag_t *nodes,
|
|||
free(nodes);
|
||||
}
|
||||
|
||||
static const char k_materialized_hit_op_name[] = "materialized_hit";
|
||||
|
||||
static bool amduat_pel_surf_store_fs_ctx(
|
||||
const amduat_asl_store_t *store,
|
||||
amduat_asl_store_fs_t **out_fs) {
|
||||
amduat_asl_store_ops_t fs_ops;
|
||||
|
||||
if (out_fs == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_fs = NULL;
|
||||
if (store == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
fs_ops = amduat_asl_store_fs_ops();
|
||||
if (store->ops.put != fs_ops.put ||
|
||||
store->ops.get != fs_ops.get ||
|
||||
store->ops.put_indexed != fs_ops.put_indexed ||
|
||||
store->ops.get_indexed != fs_ops.get_indexed ||
|
||||
store->ops.tombstone != fs_ops.tombstone ||
|
||||
store->ops.tombstone_lift != fs_ops.tombstone_lift ||
|
||||
store->ops.log_scan != fs_ops.log_scan ||
|
||||
store->ops.current_state != fs_ops.current_state ||
|
||||
store->ops.validate_config != fs_ops.validate_config) {
|
||||
return false;
|
||||
}
|
||||
*out_fs = (amduat_asl_store_fs_t *)store->ctx;
|
||||
return *out_fs != NULL;
|
||||
}
|
||||
|
||||
static bool amduat_store_trace(
|
||||
amduat_asl_store_t *store,
|
||||
amduat_reference_t scheme_ref,
|
||||
|
|
@ -230,6 +278,7 @@ static bool amduat_store_trace(
|
|||
const amduat_pel_execution_result_value_t *core_result,
|
||||
const amduat_pel_program_t *program,
|
||||
const amduat_pel_program_dag_trace_t *trace_eval,
|
||||
bool materialized_hit,
|
||||
amduat_reference_t *out_trace_ref) {
|
||||
amduat_pel_trace_dag_value_t trace;
|
||||
amduat_pel_node_trace_dag_t *node_traces;
|
||||
|
|
@ -311,6 +360,29 @@ static bool amduat_store_trace(
|
|||
}
|
||||
}
|
||||
|
||||
if (materialized_hit) {
|
||||
amduat_pel_node_trace_dag_t *next;
|
||||
|
||||
/* Emit a synthetic trace entry to mark a materialization cache hit. */
|
||||
next = (amduat_pel_node_trace_dag_t *)realloc(
|
||||
node_traces, (node_count + 1u) * sizeof(*node_traces));
|
||||
if (next == NULL) {
|
||||
amduat_trace_nodes_free(node_traces, node_count);
|
||||
return false;
|
||||
}
|
||||
node_traces = next;
|
||||
memset(&node_traces[node_count], 0, sizeof(*node_traces));
|
||||
/* UINT32_MAX is a reserved sentinel; decoders treat node_id as opaque. */
|
||||
node_traces[node_count].node_id = UINT32_MAX;
|
||||
node_traces[node_count].op_name = amduat_octets(
|
||||
(const uint8_t *)k_materialized_hit_op_name,
|
||||
sizeof(k_materialized_hit_op_name) - 1u);
|
||||
node_traces[node_count].op_version = 1u;
|
||||
node_traces[node_count].status = AMDUAT_PEL_NODE_TRACE_SKIPPED;
|
||||
node_traces[node_count].status_code = 0u;
|
||||
node_count += 1u;
|
||||
}
|
||||
|
||||
memset(&trace, 0, sizeof(trace));
|
||||
trace.pel1_version = 1;
|
||||
trace.scheme_ref = scheme_ref;
|
||||
|
|
@ -367,6 +439,8 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
size_t outputs_len;
|
||||
amduat_reference_t *output_refs;
|
||||
size_t output_refs_len;
|
||||
amduat_asl_materialization_cache_fs_t mat_cache;
|
||||
amduat_octets_t sid;
|
||||
amduat_reference_t trace_ref;
|
||||
amduat_reference_t exec_result_ref;
|
||||
size_t i;
|
||||
|
|
@ -375,6 +449,10 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
bool exec_invoked = false;
|
||||
bool trace_ok = false;
|
||||
bool has_exec_result_ref = false;
|
||||
bool mat_cache_ready = false;
|
||||
bool materialized_hit = false;
|
||||
bool cache_miss_due_missing = false;
|
||||
bool executed = false;
|
||||
|
||||
if (store == NULL || out_output_refs == NULL ||
|
||||
out_output_refs_len == NULL || out_result_ref == NULL) {
|
||||
|
|
@ -392,6 +470,7 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
trace_ref.digest = amduat_octets(NULL, 0);
|
||||
exec_result_ref.hash_id = 0;
|
||||
exec_result_ref.digest = amduat_octets(NULL, 0);
|
||||
sid = amduat_octets(NULL, 0);
|
||||
|
||||
if (!amduat_reference_eq(scheme_ref,
|
||||
amduat_pel_program_dag_scheme_ref())) {
|
||||
|
|
@ -490,6 +569,75 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
has_params_artifact = true;
|
||||
}
|
||||
|
||||
/*
|
||||
* Cache safety assumptions:
|
||||
* - PEL DAG execution is deterministic for ProgramRef + Inputs + Params.
|
||||
* - ExecProfile is currently absent and encoded as a marker only.
|
||||
* - The store is single-tenant; SID collisions are treated as integrity errors.
|
||||
* - This code path assumes amduat_asl_store_fs_t as the backing store.
|
||||
*/
|
||||
{
|
||||
amduat_pel_derivation_sid_input_t sid_input;
|
||||
|
||||
memset(&sid_input, 0, sizeof(sid_input));
|
||||
sid_input.program_ref = program_ref;
|
||||
sid_input.input_refs = input_refs;
|
||||
sid_input.input_refs_len = input_refs_len;
|
||||
sid_input.has_params_ref = has_params_ref;
|
||||
if (has_params_ref) {
|
||||
sid_input.params_ref = params_ref;
|
||||
}
|
||||
sid_input.has_exec_profile = false;
|
||||
sid_input.exec_profile = amduat_octets(NULL, 0u);
|
||||
|
||||
if (amduat_pel_derivation_sid_compute(&sid_input, &sid)) {
|
||||
amduat_asl_store_fs_t *fs = NULL;
|
||||
if (amduat_pel_surf_store_fs_ctx(store, &fs) &&
|
||||
amduat_asl_materialization_cache_fs_init(&mat_cache,
|
||||
fs->root_path)) {
|
||||
mat_cache_ready = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (mat_cache_ready) {
|
||||
amduat_reference_t *cached_refs = NULL;
|
||||
size_t cached_len = 0;
|
||||
amduat_asl_store_error_t cache_err;
|
||||
|
||||
cache_err = amduat_asl_materialization_cache_fs_get(&mat_cache, sid,
|
||||
&cached_refs,
|
||||
&cached_len);
|
||||
if (cache_err == AMDUAT_ASL_STORE_OK) {
|
||||
bool cache_valid = true;
|
||||
for (i = 0; i < cached_len; ++i) {
|
||||
amduat_artifact_t cached_artifact;
|
||||
if (amduat_asl_store_get(store, cached_refs[i],
|
||||
&cached_artifact) !=
|
||||
AMDUAT_ASL_STORE_OK) {
|
||||
cache_valid = false;
|
||||
break;
|
||||
}
|
||||
amduat_artifact_free(&cached_artifact);
|
||||
}
|
||||
if (cache_valid) {
|
||||
output_refs = cached_refs;
|
||||
output_refs_len = cached_len;
|
||||
materialized_hit = true;
|
||||
exec_invoked = true;
|
||||
amduat_init_core_result(&core_result, scheme_ref,
|
||||
AMDUAT_PEL_EXEC_STATUS_OK,
|
||||
AMDUAT_PEL_EXEC_ERROR_NONE, 0);
|
||||
outputs = NULL;
|
||||
outputs_len = 0;
|
||||
amduat_artifact_free(&program_artifact);
|
||||
goto cache_hit;
|
||||
}
|
||||
cache_miss_due_missing = true;
|
||||
amduat_pel_surf_free_refs(cached_refs, cached_len);
|
||||
}
|
||||
}
|
||||
|
||||
exec_invoked = true;
|
||||
outputs = NULL;
|
||||
outputs_len = 0;
|
||||
|
|
@ -555,9 +703,12 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
program_decoded = true;
|
||||
const amduat_artifact_t *params_arg =
|
||||
has_params_artifact ? ¶ms_artifact : NULL;
|
||||
amduat_pel_kernel_set_artifact_resolver(
|
||||
amduat_pel_surf_resolve_artifact, store);
|
||||
if (!amduat_pel_program_dag_exec_trace(
|
||||
&program, input_artifacts, input_refs_len, params_arg, &outputs,
|
||||
&outputs_len, &core_result, &trace_eval)) {
|
||||
amduat_pel_kernel_clear_artifact_resolver();
|
||||
amduat_artifact_free(&program_artifact);
|
||||
amduat_enc_pel_program_dag_free(&program);
|
||||
for (i = 0; i < input_refs_len; ++i) {
|
||||
|
|
@ -570,6 +721,8 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
amduat_pel_program_dag_free_outputs(outputs, outputs_len);
|
||||
return false;
|
||||
}
|
||||
amduat_pel_kernel_clear_artifact_resolver();
|
||||
executed = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -593,6 +746,27 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
}
|
||||
}
|
||||
|
||||
if (mat_cache_ready) {
|
||||
(void)amduat_asl_materialization_cache_fs_put(&mat_cache, sid,
|
||||
output_refs, output_refs_len);
|
||||
}
|
||||
|
||||
cache_hit:
|
||||
#ifndef NDEBUG
|
||||
if (cache_miss_due_missing) {
|
||||
assert(!materialized_hit);
|
||||
}
|
||||
#endif
|
||||
if (materialized_hit) {
|
||||
amduat_log(AMDUAT_LOG_DEBUG, "pel surf: materialization hit");
|
||||
} else if (executed) {
|
||||
if (cache_miss_due_missing) {
|
||||
amduat_log(AMDUAT_LOG_DEBUG,
|
||||
"pel surf: executed (cache miss: missing outputs)");
|
||||
} else {
|
||||
amduat_log(AMDUAT_LOG_DEBUG, "pel surf: executed");
|
||||
}
|
||||
}
|
||||
trace_ok = false;
|
||||
if (exec_invoked) {
|
||||
if (!amduat_store_surface_result(
|
||||
|
|
@ -607,7 +781,9 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
has_params_ref, params_ref, has_exec_result_ref, exec_result_ref,
|
||||
&core_result,
|
||||
program_decoded ? &program : NULL,
|
||||
program_decoded ? &trace_eval : NULL, &trace_ref);
|
||||
program_decoded ? &trace_eval : NULL,
|
||||
materialized_hit,
|
||||
&trace_ref);
|
||||
if (!trace_ok) {
|
||||
goto cleanup;
|
||||
}
|
||||
|
|
@ -637,6 +813,7 @@ bool amduat_pel_surf_run(amduat_asl_store_t *store,
|
|||
amduat_pel_program_dag_free_outputs(outputs, outputs_len);
|
||||
amduat_pel_execution_result_free(&core_result);
|
||||
amduat_pel_surf_free_ref(&exec_result_ref);
|
||||
amduat_octets_free(&sid);
|
||||
return true;
|
||||
|
||||
cleanup:
|
||||
|
|
@ -655,5 +832,6 @@ cleanup:
|
|||
amduat_pel_program_dag_free_outputs(outputs, outputs_len);
|
||||
amduat_pel_execution_result_free(&core_result);
|
||||
amduat_pel_surf_free_ref(&exec_result_ref);
|
||||
amduat_octets_free(&sid);
|
||||
return false;
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
1012
src/tools/amduat_pel_build.c
Normal file
1012
src/tools/amduat_pel_build.c
Normal file
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -53,7 +53,8 @@ static void amduat_pel_run_print_usage(FILE *stream) {
|
|||
"\n"
|
||||
"notes:\n"
|
||||
" REF values are ASL reference hex strings.\n"
|
||||
" --output-raw writes output bytes to stdout and exec result to stderr.\n",
|
||||
" --output-raw writes output bytes to stdout and exec result to stderr.\n"
|
||||
" Uses the PEL/PROGRAM-DAG/1 scheme ref.\n",
|
||||
AMDUAT_PEL_RUN_DEFAULT_ROOT);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -69,7 +69,10 @@ static void amduat_pel_seed_print_usage(FILE *stream) {
|
|||
" --root %s\n"
|
||||
" --seed const\n"
|
||||
" --ref-format hex\n"
|
||||
" --output -\n",
|
||||
" --output -\n"
|
||||
"\n"
|
||||
"notes:\n"
|
||||
" --list prints available seeds and exits\n",
|
||||
AMDUAT_PEL_SEED_DEFAULT_ROOT);
|
||||
}
|
||||
|
||||
|
|
|
|||
231
tests/asl/test_asl_derivation_index_fs.c
Normal file
231
tests/asl/test_asl_derivation_index_fs.c
Normal file
|
|
@ -0,0 +1,231 @@
|
|||
#include "amduat/asl/asl_derivation_index_fs.h"
|
||||
|
||||
#include <dirent.h>
|
||||
#include <errno.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static char *make_temp_root(void) {
|
||||
char *templ;
|
||||
const char template_prefix[] =
|
||||
"/tmp/amduat_test_asl_derivation_index_fs_XXXXXX";
|
||||
|
||||
templ = (char *)malloc(sizeof(template_prefix));
|
||||
if (templ == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
memcpy(templ, template_prefix, sizeof(template_prefix));
|
||||
if (mkdtemp(templ) == NULL) {
|
||||
free(templ);
|
||||
return NULL;
|
||||
}
|
||||
return templ;
|
||||
}
|
||||
|
||||
static bool remove_tree(const char *path) {
|
||||
DIR *dir;
|
||||
struct dirent *entry;
|
||||
struct stat st;
|
||||
|
||||
if (path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (lstat(path, &st) != 0) {
|
||||
return errno == ENOENT;
|
||||
}
|
||||
if (!S_ISDIR(st.st_mode)) {
|
||||
return unlink(path) == 0;
|
||||
}
|
||||
|
||||
dir = opendir(path);
|
||||
if (dir == NULL) {
|
||||
return false;
|
||||
}
|
||||
while ((entry = readdir(dir)) != NULL) {
|
||||
char child[2048];
|
||||
if (strcmp(entry->d_name, ".") == 0 ||
|
||||
strcmp(entry->d_name, "..") == 0) {
|
||||
continue;
|
||||
}
|
||||
snprintf(child, sizeof(child), "%s/%s", path, entry->d_name);
|
||||
if (!remove_tree(child)) {
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
closedir(dir);
|
||||
return rmdir(path) == 0;
|
||||
}
|
||||
|
||||
static amduat_reference_t make_ref(uint8_t seed, size_t len) {
|
||||
uint8_t *bytes = (uint8_t *)malloc(len);
|
||||
if (bytes == NULL) {
|
||||
return amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
}
|
||||
for (size_t i = 0; i < len; ++i) {
|
||||
bytes[i] = (uint8_t)(seed + i);
|
||||
}
|
||||
return amduat_reference(0x1234u, amduat_octets(bytes, len));
|
||||
}
|
||||
|
||||
static int test_round_trip(void) {
|
||||
amduat_asl_derivation_index_fs_t index;
|
||||
amduat_asl_derivation_record_t record;
|
||||
amduat_asl_derivation_record_t record2;
|
||||
amduat_asl_derivation_record_t *records = NULL;
|
||||
size_t records_len = 0u;
|
||||
amduat_reference_t artifact_ref;
|
||||
amduat_reference_t inputs[2];
|
||||
uint8_t sid_bytes[32];
|
||||
uint8_t exec_bytes[4] = {0x01, 0x02, 0x03, 0x04};
|
||||
char *root;
|
||||
int exit_code = 1;
|
||||
|
||||
root = make_temp_root();
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "temp root failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (!amduat_asl_derivation_index_fs_init(&index, root)) {
|
||||
fprintf(stderr, "index init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
artifact_ref = make_ref(0x10, 4);
|
||||
record.program_ref = make_ref(0x20, 4);
|
||||
record.output_index = 0u;
|
||||
inputs[0] = make_ref(0x30, 4);
|
||||
inputs[1] = make_ref(0x40, 4);
|
||||
memset(sid_bytes, 0xa5, sizeof(sid_bytes));
|
||||
record.sid = amduat_octets(sid_bytes, sizeof(sid_bytes));
|
||||
record.input_refs = inputs;
|
||||
record.input_refs_len = 2u;
|
||||
record.has_params_ref = true;
|
||||
record.params_ref = make_ref(0x50, 4);
|
||||
record.has_exec_profile = true;
|
||||
record.exec_profile = amduat_octets(exec_bytes, sizeof(exec_bytes));
|
||||
|
||||
record2 = record;
|
||||
record2.sid = amduat_octets(sid_bytes, sizeof(sid_bytes));
|
||||
record2.output_index = 1u;
|
||||
record2.has_params_ref = false;
|
||||
record2.has_exec_profile = false;
|
||||
|
||||
{
|
||||
amduat_asl_store_error_t err =
|
||||
amduat_asl_derivation_index_fs_add(&index, artifact_ref, &record);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "add record failed: %d\n", err);
|
||||
goto cleanup_records;
|
||||
}
|
||||
}
|
||||
{
|
||||
amduat_asl_store_error_t err =
|
||||
amduat_asl_derivation_index_fs_add(&index, artifact_ref, &record2);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "add record2 failed: %d\n", err);
|
||||
goto cleanup_records;
|
||||
}
|
||||
}
|
||||
|
||||
if (amduat_asl_derivation_index_fs_list(&index,
|
||||
artifact_ref,
|
||||
&records,
|
||||
&records_len) !=
|
||||
AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "list failed\n");
|
||||
goto cleanup_records;
|
||||
}
|
||||
if (records_len != 2u) {
|
||||
fprintf(stderr, "record count mismatch\n");
|
||||
goto cleanup_records_list;
|
||||
}
|
||||
if (records[0].input_refs_len != 2u ||
|
||||
records[0].has_params_ref != true ||
|
||||
records[0].has_exec_profile != true ||
|
||||
records[0].output_index != 0u) {
|
||||
fprintf(stderr, "record fields mismatch\n");
|
||||
goto cleanup_records_list;
|
||||
}
|
||||
if (records[1].has_params_ref != false ||
|
||||
records[1].has_exec_profile != false ||
|
||||
records[1].output_index != 1u) {
|
||||
fprintf(stderr, "record2 flags mismatch\n");
|
||||
goto cleanup_records_list;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_records_list:
|
||||
amduat_asl_derivation_records_free(records, records_len);
|
||||
free(records);
|
||||
cleanup_records:
|
||||
amduat_reference_free(&artifact_ref);
|
||||
amduat_reference_free(&record.program_ref);
|
||||
amduat_reference_free(&record.params_ref);
|
||||
amduat_reference_free(&inputs[0]);
|
||||
amduat_reference_free(&inputs[1]);
|
||||
cleanup:
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "cleanup failed\n");
|
||||
exit_code = 1;
|
||||
}
|
||||
free(root);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_not_found(void) {
|
||||
amduat_asl_derivation_index_fs_t index;
|
||||
amduat_reference_t artifact_ref;
|
||||
amduat_asl_derivation_record_t *records = NULL;
|
||||
size_t records_len = 0u;
|
||||
char *root;
|
||||
int exit_code = 1;
|
||||
|
||||
root = make_temp_root();
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "temp root failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (!amduat_asl_derivation_index_fs_init(&index, root)) {
|
||||
fprintf(stderr, "index init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
artifact_ref = make_ref(0x70, 4);
|
||||
if (amduat_asl_derivation_index_fs_list(&index,
|
||||
artifact_ref,
|
||||
&records,
|
||||
&records_len) !=
|
||||
AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
|
||||
fprintf(stderr, "expected not found\n");
|
||||
amduat_reference_free(&artifact_ref);
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_reference_free(&artifact_ref);
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "cleanup failed\n");
|
||||
exit_code = 1;
|
||||
}
|
||||
free(root);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_round_trip() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return test_not_found();
|
||||
}
|
||||
154
tests/asl/test_asl_index_accel.c
Normal file
154
tests/asl/test_asl_index_accel.c
Normal file
|
|
@ -0,0 +1,154 @@
|
|||
#include "amduat/asl/index_accel.h"
|
||||
#include "amduat/asl/index_bloom.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static int test_routing_key_layout(void) {
|
||||
uint8_t digest_bytes[2] = {0xaa, 0xbb};
|
||||
amduat_reference_t ref =
|
||||
amduat_reference(0x1234u, amduat_octets(digest_bytes, sizeof(digest_bytes)));
|
||||
amduat_octets_t key;
|
||||
int exit_code = 1;
|
||||
|
||||
if (!amduat_asl_index_accel_routing_key_from_ref(
|
||||
ref, true, amduat_type_tag(0x11223344u), &key)) {
|
||||
fprintf(stderr, "routing key build failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (key.len != 2u + 2u + sizeof(digest_bytes) + 1u + 4u) {
|
||||
fprintf(stderr, "routing key length mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (key.data[0] != 0x34u || key.data[1] != 0x12u ||
|
||||
key.data[2] != 0x02u || key.data[3] != 0x00u) {
|
||||
fprintf(stderr, "routing key header mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (key.data[4] != 0xaau || key.data[5] != 0xbbu) {
|
||||
fprintf(stderr, "routing key digest mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (key.data[6] != 0x01u) {
|
||||
fprintf(stderr, "routing key type tag flag mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (key.data[7] != 0x44u || key.data[8] != 0x33u ||
|
||||
key.data[9] != 0x22u || key.data[10] != 0x11u) {
|
||||
fprintf(stderr, "routing key type tag bytes mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
amduat_octets_free(&key);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_routing_key_absence(void) {
|
||||
uint8_t digest_bytes[1] = {0x7f};
|
||||
amduat_reference_t ref =
|
||||
amduat_reference(0x0001u, amduat_octets(digest_bytes, sizeof(digest_bytes)));
|
||||
amduat_octets_t key;
|
||||
int exit_code = 1;
|
||||
|
||||
if (!amduat_asl_index_accel_routing_key_from_ref(
|
||||
ref, false, amduat_type_tag(0xdeadbeefu), &key)) {
|
||||
fprintf(stderr, "routing key build failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (key.data[4] != 0x7fu) {
|
||||
fprintf(stderr, "routing key digest mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (key.data[5] != 0x00u) {
|
||||
fprintf(stderr, "routing key absence flag mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (key.data[6] != 0x00u || key.data[7] != 0x00u ||
|
||||
key.data[8] != 0x00u || key.data[9] != 0x00u) {
|
||||
fprintf(stderr, "routing key absence tag mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
amduat_octets_free(&key);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_shard_determinism(void) {
|
||||
uint8_t digest_bytes[3] = {0x01, 0x02, 0x03};
|
||||
amduat_reference_t ref =
|
||||
amduat_reference(0x00f0u, amduat_octets(digest_bytes, sizeof(digest_bytes)));
|
||||
uint16_t shard_a;
|
||||
uint16_t shard_b;
|
||||
|
||||
shard_a = amduat_asl_index_accel_shard_for_ref(
|
||||
ref, false, amduat_type_tag(0u), 8u);
|
||||
shard_b = amduat_asl_index_accel_shard_for_ref(
|
||||
ref, false, amduat_type_tag(0u), 8u);
|
||||
|
||||
if (shard_a != shard_b) {
|
||||
fprintf(stderr, "shard selection mismatch\n");
|
||||
return 1;
|
||||
}
|
||||
if (shard_a >= 8u) {
|
||||
fprintf(stderr, "shard out of range\n");
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_bloom_advisory(void) {
|
||||
uint8_t digest_bytes[4] = {0x10, 0x20, 0x30, 0x40};
|
||||
amduat_octets_t bloom;
|
||||
amduat_octets_t empty = amduat_octets(NULL, 0u);
|
||||
int exit_code = 1;
|
||||
|
||||
if (!amduat_asl_index_bloom_init(&bloom)) {
|
||||
fprintf(stderr, "bloom init failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
if (!amduat_asl_index_bloom_add(
|
||||
bloom, 0x0001u, amduat_octets(digest_bytes, sizeof(digest_bytes)))) {
|
||||
fprintf(stderr, "bloom add failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_index_bloom_maybe_contains(
|
||||
bloom, 0x0001u, amduat_octets(digest_bytes, sizeof(digest_bytes)))) {
|
||||
fprintf(stderr, "bloom false negative\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_index_bloom_maybe_contains(
|
||||
empty, 0x0002u, amduat_octets(NULL, 0u))) {
|
||||
fprintf(stderr, "empty bloom should be advisory\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
amduat_octets_free(&bloom);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_routing_key_layout() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_routing_key_absence() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_shard_determinism() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return test_bloom_advisory();
|
||||
}
|
||||
176
tests/asl/test_asl_index_put_get_consistency.c
Normal file
176
tests/asl/test_asl_index_put_get_consistency.c
Normal file
|
|
@ -0,0 +1,176 @@
|
|||
#include "amduat/asl/asl_store_index_fs.h"
|
||||
#include "amduat/asl/store.h"
|
||||
#include "amduat/enc/asl1_core.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <dirent.h>
|
||||
#include <errno.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static bool join_path(const char *base, const char *segment, char **out_path) {
|
||||
size_t base_len;
|
||||
size_t seg_len;
|
||||
bool needs_sep;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (base == NULL || segment == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
base_len = strlen(base);
|
||||
seg_len = strlen(segment);
|
||||
if (base_len == 0u || seg_len == 0u) {
|
||||
return false;
|
||||
}
|
||||
needs_sep = base[base_len - 1u] != '/';
|
||||
total_len = base_len + (needs_sep ? 1u : 0u) + seg_len + 1u;
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, base, base_len);
|
||||
offset += base_len;
|
||||
if (needs_sep) {
|
||||
buffer[offset++] = '/';
|
||||
}
|
||||
memcpy(buffer + offset, segment, seg_len);
|
||||
offset += seg_len;
|
||||
buffer[offset] = '\0';
|
||||
*out_path = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool remove_tree(const char *path) {
|
||||
struct stat st;
|
||||
DIR *dir;
|
||||
struct dirent *entry;
|
||||
|
||||
if (path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (lstat(path, &st) != 0) {
|
||||
return errno == ENOENT;
|
||||
}
|
||||
if (!S_ISDIR(st.st_mode)) {
|
||||
return unlink(path) == 0;
|
||||
}
|
||||
dir = opendir(path);
|
||||
if (dir == NULL) {
|
||||
return false;
|
||||
}
|
||||
while ((entry = readdir(dir)) != NULL) {
|
||||
char *child = NULL;
|
||||
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0) {
|
||||
continue;
|
||||
}
|
||||
if (!join_path(path, entry->d_name, &child)) {
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
if (!remove_tree(child)) {
|
||||
free(child);
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
free(child);
|
||||
}
|
||||
if (closedir(dir) != 0) {
|
||||
return false;
|
||||
}
|
||||
return rmdir(path) == 0;
|
||||
}
|
||||
|
||||
static char *make_temp_root(void) {
|
||||
char *tmpl = NULL;
|
||||
char *root = NULL;
|
||||
|
||||
tmpl = strdup("/tmp/amduat-index-putget-XXXXXX");
|
||||
if (tmpl == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
if (mkdtemp(tmpl) == NULL) {
|
||||
free(tmpl);
|
||||
return NULL;
|
||||
}
|
||||
root = tmpl;
|
||||
return root;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_store_index_fs_t fs;
|
||||
amduat_asl_store_t store;
|
||||
char *root = NULL;
|
||||
int rc = 1;
|
||||
size_t i;
|
||||
|
||||
root = make_temp_root();
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "temp root creation failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
memset(&config, 0, sizeof(config));
|
||||
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
|
||||
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
|
||||
if (!amduat_asl_store_index_fs_init(&fs, config, root)) {
|
||||
fprintf(stderr, "index fs init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_store_init(&store, config, amduat_asl_store_index_fs_ops(), &fs);
|
||||
|
||||
for (i = 0u; i < 128u; ++i) {
|
||||
char payload_buf[64];
|
||||
size_t payload_len;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_reference_t ref;
|
||||
amduat_artifact_t loaded;
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
payload_len = (size_t)snprintf(payload_buf, sizeof(payload_buf),
|
||||
"payload-%zu", i);
|
||||
if (payload_len == 0u || payload_len >= sizeof(payload_buf)) {
|
||||
fprintf(stderr, "payload format failed at %zu\n", i);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
artifact = amduat_artifact(
|
||||
amduat_octets((const uint8_t *)payload_buf, payload_len));
|
||||
ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_put(&store, artifact, &ref);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "put failed at %zu: %d\n", i, (int)err);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
loaded = amduat_artifact(amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_get(&store, ref, &loaded);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "get after put failed at %zu: %d\n", i, (int)err);
|
||||
amduat_reference_free(&ref);
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_artifact_free(&loaded);
|
||||
amduat_reference_free(&ref);
|
||||
}
|
||||
|
||||
rc = 0;
|
||||
|
||||
cleanup:
|
||||
if (root != NULL) {
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "warning: cleanup failed for %s\n", root);
|
||||
}
|
||||
free(root);
|
||||
}
|
||||
return rc;
|
||||
}
|
||||
932
tests/asl/test_asl_index_replay.c
Normal file
932
tests/asl/test_asl_index_replay.c
Normal file
|
|
@ -0,0 +1,932 @@
|
|||
#include "amduat/asl/asl_store_index_fs.h"
|
||||
#include "amduat/asl/index_replay.h"
|
||||
#include "amduat/asl/ref_derive.h"
|
||||
#include "amduat/asl/store.h"
|
||||
#include "amduat/enc/asl_core_index.h"
|
||||
#include "amduat/enc/asl_log.h"
|
||||
#include "amduat/enc/asl1_core.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
#include "asl_store_index_fs_layout.h"
|
||||
|
||||
#include <dirent.h>
|
||||
#include <errno.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static void store_u16_le(uint8_t *out, uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void store_u64_le(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static size_t build_artifact_ref(uint8_t *out,
|
||||
size_t out_cap,
|
||||
amduat_hash_id_t hash_id,
|
||||
const uint8_t *digest,
|
||||
size_t digest_len) {
|
||||
size_t total = 4 + 2 + 2 + digest_len;
|
||||
if (out_cap < total) {
|
||||
return 0;
|
||||
}
|
||||
store_u32_le(out, (uint32_t)hash_id);
|
||||
store_u16_le(out + 4, (uint16_t)digest_len);
|
||||
store_u16_le(out + 6, 0);
|
||||
memcpy(out + 8, digest, digest_len);
|
||||
return total;
|
||||
}
|
||||
|
||||
static size_t build_tombstone_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
amduat_hash_id_t hash_id,
|
||||
const uint8_t *digest,
|
||||
size_t digest_len) {
|
||||
size_t offset = 0;
|
||||
size_t ref_len = build_artifact_ref(out,
|
||||
out_cap,
|
||||
hash_id,
|
||||
digest,
|
||||
digest_len);
|
||||
if (ref_len == 0 || out_cap < ref_len + 8) {
|
||||
return 0;
|
||||
}
|
||||
offset += ref_len;
|
||||
store_u32_le(out + offset, 0);
|
||||
offset += 4;
|
||||
store_u32_le(out + offset, 0);
|
||||
offset += 4;
|
||||
return offset;
|
||||
}
|
||||
|
||||
static size_t build_tombstone_lift_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
amduat_hash_id_t hash_id,
|
||||
const uint8_t *digest,
|
||||
size_t digest_len,
|
||||
uint64_t tombstone_logseq) {
|
||||
size_t offset = 0;
|
||||
size_t ref_len = build_artifact_ref(out,
|
||||
out_cap,
|
||||
hash_id,
|
||||
digest,
|
||||
digest_len);
|
||||
if (ref_len == 0 || out_cap < ref_len + 8) {
|
||||
return 0;
|
||||
}
|
||||
offset += ref_len;
|
||||
store_u64_le(out + offset, tombstone_logseq);
|
||||
offset += 8;
|
||||
return offset;
|
||||
}
|
||||
|
||||
static size_t build_segment_seal_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
uint64_t segment_id,
|
||||
const uint8_t hash[32]) {
|
||||
if (out_cap < 8 + 32) {
|
||||
return 0;
|
||||
}
|
||||
store_u64_le(out, segment_id);
|
||||
memcpy(out + 8, hash, 32);
|
||||
return 8 + 32;
|
||||
}
|
||||
|
||||
static size_t build_snapshot_anchor_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
uint64_t snapshot_id,
|
||||
const uint8_t root_hash[32]) {
|
||||
if (out_cap < 8 + 32) {
|
||||
return 0;
|
||||
}
|
||||
store_u64_le(out, snapshot_id);
|
||||
memcpy(out + 8, root_hash, 32);
|
||||
return 8 + 32;
|
||||
}
|
||||
|
||||
static bool join_path(const char *base, const char *segment, char **out_path) {
|
||||
size_t base_len;
|
||||
size_t seg_len;
|
||||
bool needs_sep;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (base == NULL || segment == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (base[0] == '\0' || segment[0] == '\0') {
|
||||
return false;
|
||||
}
|
||||
|
||||
base_len = strlen(base);
|
||||
seg_len = strlen(segment);
|
||||
needs_sep = base[base_len - 1u] != '/';
|
||||
total_len = base_len + (needs_sep ? 1u : 0u) + seg_len + 1u;
|
||||
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, base, base_len);
|
||||
offset += base_len;
|
||||
if (needs_sep) {
|
||||
buffer[offset++] = '/';
|
||||
}
|
||||
memcpy(buffer + offset, segment, seg_len);
|
||||
offset += seg_len;
|
||||
buffer[offset] = '\0';
|
||||
|
||||
*out_path = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool remove_tree(const char *path) {
|
||||
struct stat st;
|
||||
DIR *dir;
|
||||
struct dirent *entry;
|
||||
|
||||
if (path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (lstat(path, &st) != 0) {
|
||||
return errno == ENOENT;
|
||||
}
|
||||
|
||||
if (!S_ISDIR(st.st_mode)) {
|
||||
return unlink(path) == 0;
|
||||
}
|
||||
|
||||
dir = opendir(path);
|
||||
if (dir == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
while ((entry = readdir(dir)) != NULL) {
|
||||
char *child = NULL;
|
||||
if (strcmp(entry->d_name, ".") == 0 ||
|
||||
strcmp(entry->d_name, "..") == 0) {
|
||||
continue;
|
||||
}
|
||||
if (!join_path(path, entry->d_name, &child)) {
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
if (!remove_tree(child)) {
|
||||
free(child);
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
free(child);
|
||||
}
|
||||
|
||||
if (closedir(dir) != 0) {
|
||||
return false;
|
||||
}
|
||||
return rmdir(path) == 0;
|
||||
}
|
||||
|
||||
static char *make_temp_root(void) {
|
||||
char *templ;
|
||||
const char template_prefix[] = "/tmp/amduat_test_asl_index_replay_XXXXXX";
|
||||
|
||||
templ = (char *)malloc(sizeof(template_prefix));
|
||||
if (templ == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
memcpy(templ, template_prefix, sizeof(template_prefix));
|
||||
if (mkdtemp(templ) == NULL) {
|
||||
free(templ);
|
||||
return NULL;
|
||||
}
|
||||
return templ;
|
||||
}
|
||||
|
||||
static bool ensure_dir(const char *path) {
|
||||
if (mkdir(path, 0700) == 0) {
|
||||
return true;
|
||||
}
|
||||
return errno == EEXIST;
|
||||
}
|
||||
|
||||
static bool write_file(const char *path, const uint8_t *data, size_t len) {
|
||||
FILE *file;
|
||||
size_t written;
|
||||
|
||||
file = fopen(path, "wb");
|
||||
if (file == NULL) {
|
||||
return false;
|
||||
}
|
||||
written = 0u;
|
||||
if (len != 0) {
|
||||
written = fwrite(data, 1u, len, file);
|
||||
}
|
||||
if (written != len) {
|
||||
fclose(file);
|
||||
return false;
|
||||
}
|
||||
return fclose(file) == 0;
|
||||
}
|
||||
|
||||
static bool read_file(const char *path, uint8_t **out_bytes, size_t *out_len) {
|
||||
FILE *file;
|
||||
long size;
|
||||
uint8_t *buffer;
|
||||
|
||||
if (out_bytes == NULL || out_len == NULL) {
|
||||
return false;
|
||||
}
|
||||
*out_bytes = NULL;
|
||||
*out_len = 0u;
|
||||
|
||||
file = fopen(path, "rb");
|
||||
if (file == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (fseek(file, 0, SEEK_END) != 0) {
|
||||
fclose(file);
|
||||
return false;
|
||||
}
|
||||
size = ftell(file);
|
||||
if (size < 0) {
|
||||
fclose(file);
|
||||
return false;
|
||||
}
|
||||
if (fseek(file, 0, SEEK_SET) != 0) {
|
||||
fclose(file);
|
||||
return false;
|
||||
}
|
||||
buffer = (uint8_t *)malloc((size_t)size);
|
||||
if (buffer == NULL) {
|
||||
fclose(file);
|
||||
return false;
|
||||
}
|
||||
if (size != 0 && fread(buffer, 1u, (size_t)size, file) != (size_t)size) {
|
||||
free(buffer);
|
||||
fclose(file);
|
||||
return false;
|
||||
}
|
||||
if (fclose(file) != 0) {
|
||||
free(buffer);
|
||||
return false;
|
||||
}
|
||||
*out_bytes = buffer;
|
||||
*out_len = (size_t)size;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool prepare_index_tree(const char *root) {
|
||||
char *index_path = NULL;
|
||||
char *segments_path = NULL;
|
||||
char *blocks_path = NULL;
|
||||
bool ok = false;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_index_path(root,
|
||||
&index_path) ||
|
||||
!amduat_asl_store_index_fs_layout_build_segments_path(root,
|
||||
&segments_path) ||
|
||||
!amduat_asl_store_index_fs_layout_build_blocks_path(root,
|
||||
&blocks_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!ensure_dir(index_path) ||
|
||||
!ensure_dir(segments_path) ||
|
||||
!ensure_dir(blocks_path)) {
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
ok = true;
|
||||
|
||||
cleanup:
|
||||
free(index_path);
|
||||
free(segments_path);
|
||||
free(blocks_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
static bool write_log(const char *root,
|
||||
amduat_asl_log_record_t *records,
|
||||
size_t record_count) {
|
||||
char *log_path = NULL;
|
||||
amduat_octets_t bytes;
|
||||
bool ok = false;
|
||||
|
||||
if (!amduat_enc_asl_log_encode_v1(records, record_count, &bytes)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_log_path(root, &log_path)) {
|
||||
free((void *)bytes.data);
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = write_file(log_path, bytes.data, bytes.len);
|
||||
free((void *)bytes.data);
|
||||
free(log_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
static bool load_log_records(const char *root,
|
||||
amduat_asl_log_record_t **out_records,
|
||||
size_t *out_count) {
|
||||
char *log_path = NULL;
|
||||
uint8_t *bytes = NULL;
|
||||
size_t len = 0;
|
||||
bool ok = false;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_log_path(root, &log_path)) {
|
||||
return false;
|
||||
}
|
||||
if (!read_file(log_path, &bytes, &len)) {
|
||||
free(log_path);
|
||||
return false;
|
||||
}
|
||||
ok = amduat_enc_asl_log_decode_v1(amduat_octets(bytes, len),
|
||||
out_records,
|
||||
out_count);
|
||||
free(bytes);
|
||||
free(log_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
static bool write_block_file(const char *root,
|
||||
uint64_t block_id,
|
||||
amduat_octets_t bytes) {
|
||||
char *block_path = NULL;
|
||||
bool ok = false;
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_block_path(root,
|
||||
block_id,
|
||||
&block_path)) {
|
||||
return false;
|
||||
}
|
||||
ok = write_file(block_path, bytes.data, bytes.len);
|
||||
free(block_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
static bool write_segment_file(const char *root,
|
||||
uint64_t segment_id,
|
||||
const amduat_asl_core_index_segment_t *segment,
|
||||
uint8_t out_hash[32]) {
|
||||
char *segment_path = NULL;
|
||||
amduat_octets_t encoded;
|
||||
bool ok = false;
|
||||
|
||||
if (!amduat_enc_asl_core_index_encode_v1(segment, &encoded)) {
|
||||
return false;
|
||||
}
|
||||
if (!amduat_hash_asl1_digest(AMDUAT_HASH_ASL1_ID_SHA256,
|
||||
encoded,
|
||||
out_hash,
|
||||
32)) {
|
||||
free((void *)encoded.data);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_store_index_fs_layout_build_segment_path(root,
|
||||
segment_id,
|
||||
&segment_path)) {
|
||||
free((void *)encoded.data);
|
||||
return false;
|
||||
}
|
||||
ok = write_file(segment_path, encoded.data, encoded.len);
|
||||
free((void *)encoded.data);
|
||||
free(segment_path);
|
||||
return ok;
|
||||
}
|
||||
|
||||
static bool build_segment_for_artifact(
|
||||
amduat_asl_core_index_segment_t *segment,
|
||||
amduat_asl_index_record_t *record,
|
||||
amduat_asl_extent_record_t *extent,
|
||||
amduat_reference_t ref,
|
||||
amduat_octets_t artifact_bytes,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t block_id) {
|
||||
if (segment == NULL || record == NULL || extent == NULL) {
|
||||
return false;
|
||||
}
|
||||
memset(segment, 0, sizeof(*segment));
|
||||
memset(record, 0, sizeof(*record));
|
||||
memset(extent, 0, sizeof(*extent));
|
||||
|
||||
segment->header.snapshot_min = snapshot_id;
|
||||
segment->header.snapshot_max = snapshot_id;
|
||||
segment->header.segment_domain_id = 0;
|
||||
segment->header.segment_visibility = 1;
|
||||
segment->header.federation_version = 0;
|
||||
segment->header.flags = 0;
|
||||
segment->header.reserved0 = 0;
|
||||
|
||||
record->hash_id = ref.hash_id;
|
||||
record->digest_len = (uint16_t)ref.digest.len;
|
||||
record->extent_count = 1;
|
||||
record->total_length = (uint32_t)artifact_bytes.len;
|
||||
record->domain_id = 0;
|
||||
record->visibility = 1;
|
||||
record->has_cross_domain_source = 0;
|
||||
record->cross_domain_source = 0;
|
||||
record->flags = 0;
|
||||
|
||||
extent->block_id = block_id;
|
||||
extent->offset = 0;
|
||||
extent->length = (uint32_t)artifact_bytes.len;
|
||||
|
||||
segment->records = record;
|
||||
segment->record_count = 1;
|
||||
segment->digests = ref.digest;
|
||||
segment->extents = extent;
|
||||
segment->extent_count = 1;
|
||||
segment->footer.seal_snapshot = snapshot_id;
|
||||
segment->footer.seal_time_ns = snapshot_id;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool write_artifact_segment(const char *root,
|
||||
amduat_artifact_t artifact,
|
||||
uint64_t segment_id,
|
||||
uint64_t block_id,
|
||||
uint64_t snapshot_id,
|
||||
uint8_t out_hash[32],
|
||||
amduat_reference_t *out_ref) {
|
||||
amduat_reference_t ref;
|
||||
amduat_octets_t artifact_bytes;
|
||||
amduat_asl_core_index_segment_t segment;
|
||||
amduat_asl_index_record_t record;
|
||||
amduat_asl_extent_record_t extent;
|
||||
|
||||
if (out_ref == NULL) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!amduat_asl_ref_derive(artifact,
|
||||
AMDUAT_ENC_ASL1_CORE_V1,
|
||||
AMDUAT_HASH_ASL1_ID_SHA256,
|
||||
&ref,
|
||||
&artifact_bytes)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!write_block_file(root, block_id, artifact_bytes)) {
|
||||
amduat_octets_free(&artifact_bytes);
|
||||
amduat_reference_free(&ref);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!build_segment_for_artifact(&segment,
|
||||
&record,
|
||||
&extent,
|
||||
ref,
|
||||
artifact_bytes,
|
||||
snapshot_id,
|
||||
block_id) ||
|
||||
!write_segment_file(root, segment_id, &segment, out_hash)) {
|
||||
amduat_octets_free(&artifact_bytes);
|
||||
amduat_reference_free(&ref);
|
||||
return false;
|
||||
}
|
||||
|
||||
amduat_octets_free(&artifact_bytes);
|
||||
*out_ref = ref;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool replay_state_equal(const amduat_asl_replay_state_t *lhs,
|
||||
const amduat_asl_replay_state_t *rhs) {
|
||||
size_t i;
|
||||
|
||||
if (lhs->segments_len != rhs->segments_len ||
|
||||
lhs->tombstones_len != rhs->tombstones_len ||
|
||||
lhs->state.log_position != rhs->state.log_position) {
|
||||
return false;
|
||||
}
|
||||
|
||||
for (i = 0; i < lhs->segments_len; ++i) {
|
||||
const amduat_asl_segment_seal_t *left = &lhs->segments[i];
|
||||
const amduat_asl_segment_seal_t *right = &rhs->segments[i];
|
||||
if (left->segment_id != right->segment_id ||
|
||||
memcmp(left->segment_hash, right->segment_hash,
|
||||
sizeof(left->segment_hash)) != 0) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
for (i = 0; i < lhs->tombstones_len; ++i) {
|
||||
const amduat_asl_tombstone_entry_t *left = &lhs->tombstones[i];
|
||||
const amduat_asl_tombstone_entry_t *right = &rhs->tombstones[i];
|
||||
if (left->tombstone_logseq != right->tombstone_logseq ||
|
||||
!amduat_reference_eq(left->ref, right->ref)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static int test_replay_determinism(void) {
|
||||
char *root;
|
||||
amduat_artifact_t artifact_a;
|
||||
amduat_artifact_t artifact_b;
|
||||
amduat_reference_t ref_a;
|
||||
amduat_reference_t ref_b;
|
||||
uint8_t hash_a[32];
|
||||
uint8_t hash_b[32];
|
||||
uint8_t seal_payload_a[8 + 32];
|
||||
uint8_t seal_payload_b[8 + 32];
|
||||
size_t seal_len_a;
|
||||
size_t seal_len_b;
|
||||
amduat_asl_log_record_t records[2];
|
||||
amduat_asl_log_record_t *loaded_records = NULL;
|
||||
size_t loaded_count = 0;
|
||||
amduat_asl_replay_state_t first;
|
||||
amduat_asl_replay_state_t second;
|
||||
uint8_t payload_a[4] = {0x01, 0x02, 0x03, 0x04};
|
||||
uint8_t payload_b[4] = {0x10, 0x20, 0x30, 0x40};
|
||||
int exit_code = 1;
|
||||
|
||||
ref_a = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
ref_b = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
root = make_temp_root();
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "temp root failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (!prepare_index_tree(root)) {
|
||||
fprintf(stderr, "prepare index tree failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
artifact_a = amduat_artifact(amduat_octets(payload_a, sizeof(payload_a)));
|
||||
artifact_b = amduat_artifact(amduat_octets(payload_b, sizeof(payload_b)));
|
||||
if (!write_artifact_segment(root, artifact_a, 1, 1, 1, hash_a, &ref_a) ||
|
||||
!write_artifact_segment(root, artifact_b, 2, 2, 2, hash_b, &ref_b)) {
|
||||
fprintf(stderr, "write artifact segments failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
seal_len_a = build_segment_seal_payload(seal_payload_a,
|
||||
sizeof(seal_payload_a),
|
||||
1,
|
||||
hash_a);
|
||||
seal_len_b = build_segment_seal_payload(seal_payload_b,
|
||||
sizeof(seal_payload_b),
|
||||
2,
|
||||
hash_b);
|
||||
if (seal_len_a == 0 || seal_len_b == 0) {
|
||||
fprintf(stderr, "seal payload build failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 10;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[0].payload = amduat_octets(seal_payload_a, seal_len_a);
|
||||
records[1].logseq = 20;
|
||||
records[1].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[1].payload = amduat_octets(seal_payload_b, seal_len_b);
|
||||
|
||||
if (!write_log(root, records, 2)) {
|
||||
fprintf(stderr, "write log failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (!load_log_records(root, &loaded_records, &loaded_count)) {
|
||||
fprintf(stderr, "log decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!amduat_asl_replay_init(&first) ||
|
||||
!amduat_asl_replay_init(&second)) {
|
||||
fprintf(stderr, "replay init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_replay_apply_log(loaded_records,
|
||||
loaded_count,
|
||||
20,
|
||||
&first) ||
|
||||
!amduat_asl_replay_apply_log(loaded_records,
|
||||
loaded_count,
|
||||
20,
|
||||
&second)) {
|
||||
fprintf(stderr, "replay apply failed\n");
|
||||
amduat_asl_replay_free(&first);
|
||||
amduat_asl_replay_free(&second);
|
||||
goto cleanup;
|
||||
}
|
||||
if (!replay_state_equal(&first, &second)) {
|
||||
fprintf(stderr, "replay state mismatch\n");
|
||||
amduat_asl_replay_free(&first);
|
||||
amduat_asl_replay_free(&second);
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_replay_free(&first);
|
||||
amduat_asl_replay_free(&second);
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
if (loaded_records != NULL) {
|
||||
amduat_enc_asl_log_free(loaded_records, loaded_count);
|
||||
}
|
||||
amduat_reference_free(&ref_a);
|
||||
amduat_reference_free(&ref_b);
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "cleanup failed\n");
|
||||
exit_code = 1;
|
||||
}
|
||||
free(root);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_tombstone_lift_boundary(void) {
|
||||
char *root;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_reference_t ref;
|
||||
uint8_t hash[32];
|
||||
uint8_t seal_payload[8 + 32];
|
||||
uint8_t anchor_payload[8 + 32];
|
||||
uint8_t tombstone_payload[4 + 2 + 2 + 32 + 4 + 4];
|
||||
uint8_t lift_payload[4 + 2 + 2 + 32 + 8];
|
||||
size_t seal_len;
|
||||
size_t anchor_len;
|
||||
size_t tombstone_len;
|
||||
size_t lift_len;
|
||||
amduat_asl_log_record_t records[4];
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_store_index_fs_t fs;
|
||||
amduat_asl_store_t store;
|
||||
amduat_asl_index_state_t cutoff;
|
||||
amduat_artifact_t loaded;
|
||||
amduat_asl_store_error_t err;
|
||||
uint8_t payload[5] = {0x11, 0x22, 0x33, 0x44, 0x55};
|
||||
int exit_code = 1;
|
||||
|
||||
ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
root = make_temp_root();
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "temp root failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!prepare_index_tree(root)) {
|
||||
fprintf(stderr, "prepare index tree failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
artifact = amduat_artifact(amduat_octets(payload, sizeof(payload)));
|
||||
if (!write_artifact_segment(root, artifact, 5, 5, 5, hash, &ref)) {
|
||||
fprintf(stderr, "write artifact segment failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
seal_len = build_segment_seal_payload(seal_payload,
|
||||
sizeof(seal_payload),
|
||||
5,
|
||||
hash);
|
||||
anchor_len = build_snapshot_anchor_payload(anchor_payload,
|
||||
sizeof(anchor_payload),
|
||||
5,
|
||||
hash);
|
||||
tombstone_len = build_tombstone_payload(tombstone_payload,
|
||||
sizeof(tombstone_payload),
|
||||
ref.hash_id,
|
||||
ref.digest.data,
|
||||
ref.digest.len);
|
||||
lift_len = build_tombstone_lift_payload(lift_payload,
|
||||
sizeof(lift_payload),
|
||||
ref.hash_id,
|
||||
ref.digest.data,
|
||||
ref.digest.len,
|
||||
20);
|
||||
if (seal_len == 0 || anchor_len == 0 || tombstone_len == 0 ||
|
||||
lift_len == 0) {
|
||||
fprintf(stderr, "log payload build failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 10;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[0].payload = amduat_octets(seal_payload, seal_len);
|
||||
records[1].logseq = 20;
|
||||
records[1].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
|
||||
records[1].payload = amduat_octets(tombstone_payload, tombstone_len);
|
||||
records[2].logseq = 30;
|
||||
records[2].record_type = AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR;
|
||||
records[2].payload = amduat_octets(anchor_payload, anchor_len);
|
||||
records[3].logseq = 40;
|
||||
records[3].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT;
|
||||
records[3].payload = amduat_octets(lift_payload, lift_len);
|
||||
|
||||
if (!write_log(root, records, 4)) {
|
||||
fprintf(stderr, "write log failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(&config, 0, sizeof(config));
|
||||
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
|
||||
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
|
||||
if (!amduat_asl_store_index_fs_init(&fs, config, root)) {
|
||||
fprintf(stderr, "index fs init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_store_init(&store, config, amduat_asl_store_index_fs_ops(), &fs);
|
||||
|
||||
cutoff.snapshot_id = 0;
|
||||
cutoff.log_position = 30;
|
||||
loaded = amduat_artifact(amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_get_indexed(&store, ref, cutoff, &loaded);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
|
||||
fprintf(stderr, "tombstone cutoff expected not found: %d\n", err);
|
||||
amduat_artifact_free(&loaded);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
cutoff.log_position = 40;
|
||||
loaded = amduat_artifact(amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_get_indexed(&store, ref, cutoff, &loaded);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "lift lookup failed: %d\n", err);
|
||||
amduat_artifact_free(&loaded);
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_artifact_eq(artifact, loaded)) {
|
||||
fprintf(stderr, "lifted artifact mismatch\n");
|
||||
amduat_artifact_free(&loaded);
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_artifact_free(&loaded);
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
amduat_reference_free(&ref);
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "cleanup failed\n");
|
||||
exit_code = 1;
|
||||
}
|
||||
free(root);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_crash_recovery_unsealed(void) {
|
||||
char *root;
|
||||
amduat_artifact_t artifact_a;
|
||||
amduat_artifact_t artifact_b;
|
||||
amduat_reference_t ref_a;
|
||||
amduat_reference_t ref_b;
|
||||
uint8_t hash_a[32];
|
||||
uint8_t hash_b[32];
|
||||
uint8_t seal_payload[8 + 32];
|
||||
size_t seal_len;
|
||||
amduat_asl_log_record_t records[1];
|
||||
amduat_asl_log_record_t *loaded_records = NULL;
|
||||
size_t loaded_count = 0;
|
||||
amduat_asl_replay_state_t replay_state;
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_store_index_fs_t fs;
|
||||
amduat_asl_store_t store;
|
||||
amduat_asl_index_state_t current;
|
||||
amduat_artifact_t loaded;
|
||||
amduat_asl_store_error_t err;
|
||||
uint8_t payload_a[3] = {0x61, 0x62, 0x63};
|
||||
uint8_t payload_b[3] = {0x64, 0x65, 0x66};
|
||||
int exit_code = 1;
|
||||
|
||||
ref_a = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
ref_b = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
root = make_temp_root();
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "temp root failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!prepare_index_tree(root)) {
|
||||
fprintf(stderr, "prepare index tree failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
artifact_a = amduat_artifact(amduat_octets(payload_a, sizeof(payload_a)));
|
||||
artifact_b = amduat_artifact(amduat_octets(payload_b, sizeof(payload_b)));
|
||||
if (!write_artifact_segment(root, artifact_a, 1, 1, 1, hash_a, &ref_a) ||
|
||||
!write_artifact_segment(root, artifact_b, 9, 9, 9, hash_b, &ref_b)) {
|
||||
fprintf(stderr, "write artifact segments failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
seal_len = build_segment_seal_payload(seal_payload,
|
||||
sizeof(seal_payload),
|
||||
1,
|
||||
hash_a);
|
||||
if (seal_len == 0) {
|
||||
fprintf(stderr, "seal payload build failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 10;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[0].payload = amduat_octets(seal_payload, seal_len);
|
||||
|
||||
if (!write_log(root, records, 1)) {
|
||||
fprintf(stderr, "write log failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(&config, 0, sizeof(config));
|
||||
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
|
||||
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
|
||||
if (!amduat_asl_store_index_fs_init(&fs, config, root)) {
|
||||
fprintf(stderr, "index fs init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_store_init(&store, config, amduat_asl_store_index_fs_ops(), &fs);
|
||||
|
||||
if (!amduat_asl_index_current_state(&store, ¤t)) {
|
||||
fprintf(stderr, "current state failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
loaded = amduat_artifact(amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_get_indexed(&store, ref_b, current, &loaded);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
|
||||
fprintf(stderr, "unsealed segment should be ignored: %d\n", err);
|
||||
amduat_artifact_free(&loaded);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!load_log_records(root, &loaded_records, &loaded_count)) {
|
||||
fprintf(stderr, "log decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_replay_init(&replay_state)) {
|
||||
fprintf(stderr, "replay init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (!amduat_asl_replay_apply_log(loaded_records,
|
||||
loaded_count,
|
||||
current.log_position,
|
||||
&replay_state)) {
|
||||
fprintf(stderr, "replay apply failed\n");
|
||||
amduat_asl_replay_free(&replay_state);
|
||||
goto cleanup;
|
||||
}
|
||||
if (replay_state.segments_len != 1 ||
|
||||
replay_state.segments[0].segment_id != 1 ||
|
||||
memcmp(replay_state.segments[0].segment_hash, hash_a,
|
||||
sizeof(hash_a)) != 0) {
|
||||
fprintf(stderr, "replay should only include sealed segment\n");
|
||||
amduat_asl_replay_free(&replay_state);
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_replay_free(&replay_state);
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
if (loaded_records != NULL) {
|
||||
amduat_enc_asl_log_free(loaded_records, loaded_count);
|
||||
}
|
||||
amduat_reference_free(&ref_a);
|
||||
amduat_reference_free(&ref_b);
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "cleanup failed\n");
|
||||
exit_code = 1;
|
||||
}
|
||||
free(root);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_replay_determinism() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_tombstone_lift_boundary() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_crash_recovery_unsealed() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
525
tests/asl/test_asl_log_store_index_fs.c
Normal file
525
tests/asl/test_asl_log_store_index_fs.c
Normal file
|
|
@ -0,0 +1,525 @@
|
|||
#include "amduat/asl/log_store.h"
|
||||
#include "amduat/asl/asl_store_index_fs.h"
|
||||
#include "amduat/asl/store.h"
|
||||
#include "amduat/enc/asl1_core.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <dirent.h>
|
||||
#include <errno.h>
|
||||
#include <inttypes.h>
|
||||
#include <pthread.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static bool join_path(const char *base, const char *segment, char **out_path) {
|
||||
size_t base_len;
|
||||
size_t seg_len;
|
||||
bool needs_sep;
|
||||
size_t total_len;
|
||||
char *buffer;
|
||||
size_t offset;
|
||||
|
||||
if (base == NULL || segment == NULL || out_path == NULL) {
|
||||
return false;
|
||||
}
|
||||
base_len = strlen(base);
|
||||
seg_len = strlen(segment);
|
||||
if (base_len == 0u || seg_len == 0u) {
|
||||
return false;
|
||||
}
|
||||
needs_sep = base[base_len - 1u] != '/';
|
||||
total_len = base_len + (needs_sep ? 1u : 0u) + seg_len + 1u;
|
||||
buffer = (char *)malloc(total_len);
|
||||
if (buffer == NULL) {
|
||||
return false;
|
||||
}
|
||||
offset = 0u;
|
||||
memcpy(buffer + offset, base, base_len);
|
||||
offset += base_len;
|
||||
if (needs_sep) {
|
||||
buffer[offset++] = '/';
|
||||
}
|
||||
memcpy(buffer + offset, segment, seg_len);
|
||||
offset += seg_len;
|
||||
buffer[offset] = '\0';
|
||||
*out_path = buffer;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool remove_tree(const char *path) {
|
||||
struct stat st;
|
||||
DIR *dir;
|
||||
struct dirent *entry;
|
||||
|
||||
if (path == NULL) {
|
||||
return false;
|
||||
}
|
||||
if (lstat(path, &st) != 0) {
|
||||
return errno == ENOENT;
|
||||
}
|
||||
if (!S_ISDIR(st.st_mode)) {
|
||||
return unlink(path) == 0;
|
||||
}
|
||||
dir = opendir(path);
|
||||
if (dir == NULL) {
|
||||
return false;
|
||||
}
|
||||
while ((entry = readdir(dir)) != NULL) {
|
||||
char *child = NULL;
|
||||
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0) {
|
||||
continue;
|
||||
}
|
||||
if (!join_path(path, entry->d_name, &child)) {
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
if (!remove_tree(child)) {
|
||||
free(child);
|
||||
closedir(dir);
|
||||
return false;
|
||||
}
|
||||
free(child);
|
||||
}
|
||||
if (closedir(dir) != 0) {
|
||||
return false;
|
||||
}
|
||||
return rmdir(path) == 0;
|
||||
}
|
||||
|
||||
static bool refs_equal(amduat_reference_t a, amduat_reference_t b) {
|
||||
if (a.hash_id != b.hash_id) {
|
||||
return false;
|
||||
}
|
||||
if (a.digest.len != b.digest.len) {
|
||||
return false;
|
||||
}
|
||||
if (a.digest.len == 0u) {
|
||||
return true;
|
||||
}
|
||||
if (a.digest.data == NULL || b.digest.data == NULL) {
|
||||
return false;
|
||||
}
|
||||
return memcmp(a.digest.data, b.digest.data, a.digest.len) == 0;
|
||||
}
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_t *store;
|
||||
amduat_asl_log_store_t *log_store;
|
||||
const char *log_name;
|
||||
uint16_t kind;
|
||||
size_t iterations;
|
||||
amduat_asl_store_error_t first_err;
|
||||
uint64_t *observed_offsets;
|
||||
} append_worker_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_t *store;
|
||||
uint16_t kind;
|
||||
size_t iterations;
|
||||
amduat_asl_store_error_t first_err;
|
||||
} put_worker_t;
|
||||
|
||||
static void *append_worker_main(void *opaque) {
|
||||
append_worker_t *worker = (append_worker_t *)opaque;
|
||||
size_t i;
|
||||
|
||||
if (worker == NULL || worker->store == NULL || worker->log_store == NULL ||
|
||||
worker->log_name == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
worker->first_err = AMDUAT_ASL_STORE_OK;
|
||||
|
||||
for (i = 0u; i < worker->iterations; ++i) {
|
||||
char payload_buf[64];
|
||||
size_t payload_len;
|
||||
amduat_artifact_t payload_artifact;
|
||||
amduat_reference_t payload_ref;
|
||||
amduat_asl_log_entry_t entry;
|
||||
amduat_asl_store_error_t err;
|
||||
uint64_t first_offset = 0u;
|
||||
|
||||
payload_len = (size_t)snprintf(payload_buf, sizeof(payload_buf),
|
||||
"worker-%u-%zu", (unsigned)worker->kind, i);
|
||||
if (payload_len == 0u || payload_len >= sizeof(payload_buf)) {
|
||||
worker->first_err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
payload_artifact =
|
||||
amduat_artifact(amduat_octets((const uint8_t *)payload_buf, payload_len));
|
||||
payload_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_put(worker->store, payload_artifact, &payload_ref);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
worker->first_err = err;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
memset(&entry, 0, sizeof(entry));
|
||||
entry.kind = worker->kind;
|
||||
entry.payload_ref = payload_ref;
|
||||
err = amduat_asl_log_append(worker->log_store,
|
||||
worker->log_name,
|
||||
&entry,
|
||||
1u,
|
||||
&first_offset);
|
||||
amduat_reference_free(&payload_ref);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
worker->first_err = err;
|
||||
return NULL;
|
||||
}
|
||||
if (worker->observed_offsets != NULL && i < worker->iterations) {
|
||||
worker->observed_offsets[i] = first_offset;
|
||||
}
|
||||
}
|
||||
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static void *put_worker_main(void *opaque) {
|
||||
put_worker_t *worker = (put_worker_t *)opaque;
|
||||
size_t i;
|
||||
|
||||
if (worker == NULL || worker->store == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
worker->first_err = AMDUAT_ASL_STORE_OK;
|
||||
|
||||
for (i = 0u; i < worker->iterations; ++i) {
|
||||
char payload_buf[64];
|
||||
size_t payload_len;
|
||||
amduat_artifact_t payload_artifact;
|
||||
amduat_reference_t payload_ref;
|
||||
amduat_asl_store_error_t err;
|
||||
|
||||
payload_len = (size_t)snprintf(payload_buf, sizeof(payload_buf),
|
||||
"put-only-%u-%zu", (unsigned)worker->kind, i);
|
||||
if (payload_len == 0u || payload_len >= sizeof(payload_buf)) {
|
||||
worker->first_err = AMDUAT_ASL_STORE_ERR_IO;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
payload_artifact =
|
||||
amduat_artifact(amduat_octets((const uint8_t *)payload_buf, payload_len));
|
||||
payload_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
err = amduat_asl_store_put(worker->store, payload_artifact, &payload_ref);
|
||||
amduat_reference_free(&payload_ref);
|
||||
if (err != AMDUAT_ASL_STORE_OK) {
|
||||
worker->first_err = err;
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
return NULL;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_store_index_fs_t fs_a;
|
||||
amduat_asl_store_index_fs_t fs_b;
|
||||
amduat_asl_store_t store_a;
|
||||
amduat_asl_store_t store_b;
|
||||
amduat_asl_log_store_t log_a;
|
||||
amduat_asl_log_store_t log_b;
|
||||
amduat_artifact_t payload_artifact_a;
|
||||
amduat_artifact_t payload_artifact_b;
|
||||
amduat_reference_t payload_ref_a;
|
||||
amduat_reference_t payload_ref_b;
|
||||
amduat_asl_log_entry_t entry_a;
|
||||
amduat_asl_log_entry_t entry_b;
|
||||
amduat_asl_log_entry_t *out_entries = NULL;
|
||||
size_t out_len = 0u;
|
||||
uint64_t out_next_offset = 0u;
|
||||
bool out_end = false;
|
||||
uint64_t first_offset = 0u;
|
||||
const char *log_name = "collection/space/app1/daemon/edges/log";
|
||||
enum { WORKER_ITERS = 32 };
|
||||
append_worker_t worker_a;
|
||||
append_worker_t worker_b;
|
||||
append_worker_t mixed_append_worker;
|
||||
put_worker_t mixed_put_worker;
|
||||
pthread_t thread_a;
|
||||
pthread_t thread_b;
|
||||
pthread_t mixed_append_thread;
|
||||
pthread_t mixed_put_thread;
|
||||
bool started_a = false;
|
||||
bool started_b = false;
|
||||
bool started_mixed_append = false;
|
||||
bool started_mixed_put = false;
|
||||
uint64_t worker_a_offsets[WORKER_ITERS];
|
||||
uint64_t worker_b_offsets[WORKER_ITERS];
|
||||
char *root = NULL;
|
||||
int rc = 1;
|
||||
|
||||
root = strdup("/tmp/amduat-log-index-XXXXXX");
|
||||
if (root == NULL) {
|
||||
fprintf(stderr, "alloc root template failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (mkdtemp(root) == NULL) {
|
||||
fprintf(stderr, "mkdtemp failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(&config, 0, sizeof(config));
|
||||
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
|
||||
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
|
||||
|
||||
if (!amduat_asl_store_index_fs_init(&fs_a, config, root) ||
|
||||
!amduat_asl_store_index_fs_init(&fs_b, config, root)) {
|
||||
fprintf(stderr, "index fs init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_store_init(&store_a, config, amduat_asl_store_index_fs_ops(), &fs_a);
|
||||
amduat_asl_store_init(&store_b, config, amduat_asl_store_index_fs_ops(), &fs_b);
|
||||
|
||||
if (!amduat_asl_log_store_init(&log_a, root, &store_a, NULL) ||
|
||||
!amduat_asl_log_store_init(&log_b, root, &store_b, NULL)) {
|
||||
fprintf(stderr, "log store init failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
payload_artifact_a = amduat_artifact(amduat_octets((const uint8_t *)"payload-a", 9u));
|
||||
payload_ref_a = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
if (amduat_asl_store_put(&store_a, payload_artifact_a, &payload_ref_a) !=
|
||||
AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "put payload a failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(&entry_a, 0, sizeof(entry_a));
|
||||
entry_a.kind = 1u;
|
||||
entry_a.payload_ref = payload_ref_a;
|
||||
if (amduat_asl_log_append(&log_a, log_name, &entry_a, 1u, &first_offset) !=
|
||||
AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "first append failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (first_offset != 0u) {
|
||||
fprintf(stderr, "first offset mismatch: %" PRIu64 "\n", first_offset);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
payload_artifact_b = amduat_artifact(amduat_octets((const uint8_t *)"payload-b", 9u));
|
||||
payload_ref_b = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
if (amduat_asl_store_put(&store_b, payload_artifact_b, &payload_ref_b) !=
|
||||
AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "put payload b failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(&entry_b, 0, sizeof(entry_b));
|
||||
entry_b.kind = 2u;
|
||||
entry_b.payload_ref = payload_ref_b;
|
||||
if (amduat_asl_log_append(&log_b, log_name, &entry_b, 1u, &first_offset) !=
|
||||
AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "second append failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (first_offset != 1u) {
|
||||
fprintf(stderr, "second first_offset mismatch: %" PRIu64 "\n", first_offset);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (amduat_asl_log_read(&log_a,
|
||||
log_name,
|
||||
0u,
|
||||
8u,
|
||||
&out_entries,
|
||||
&out_len,
|
||||
&out_next_offset,
|
||||
&out_end) != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "log read failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (out_len != 2u || out_next_offset != 2u || !out_end) {
|
||||
fprintf(stderr, "readback shape mismatch len=%zu next=%" PRIu64 " end=%d\n",
|
||||
out_len, out_next_offset, out_end ? 1 : 0);
|
||||
goto cleanup;
|
||||
}
|
||||
if (out_entries[0].kind != 1u || !refs_equal(out_entries[0].payload_ref, payload_ref_a)) {
|
||||
fprintf(stderr, "entry0 mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (out_entries[1].kind != 2u || !refs_equal(out_entries[1].payload_ref, payload_ref_b)) {
|
||||
fprintf(stderr, "entry1 mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_log_entries_free(out_entries, out_len);
|
||||
out_entries = NULL;
|
||||
out_len = 0u;
|
||||
|
||||
memset(&worker_a, 0, sizeof(worker_a));
|
||||
worker_a.store = &store_a;
|
||||
worker_a.log_store = &log_a;
|
||||
worker_a.log_name = log_name;
|
||||
worker_a.kind = 10u;
|
||||
worker_a.iterations = WORKER_ITERS;
|
||||
worker_a.observed_offsets = worker_a_offsets;
|
||||
|
||||
memset(&worker_b, 0, sizeof(worker_b));
|
||||
worker_b.store = &store_b;
|
||||
worker_b.log_store = &log_b;
|
||||
worker_b.log_name = log_name;
|
||||
worker_b.kind = 11u;
|
||||
worker_b.iterations = WORKER_ITERS;
|
||||
worker_b.observed_offsets = worker_b_offsets;
|
||||
|
||||
if (pthread_create(&thread_a, NULL, append_worker_main, &worker_a) != 0) {
|
||||
fprintf(stderr, "pthread_create failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
started_a = true;
|
||||
if (pthread_create(&thread_b, NULL, append_worker_main, &worker_b) != 0) {
|
||||
fprintf(stderr, "pthread_create failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
started_b = true;
|
||||
(void)pthread_join(thread_a, NULL);
|
||||
(void)pthread_join(thread_b, NULL);
|
||||
started_a = false;
|
||||
started_b = false;
|
||||
|
||||
if (worker_a.first_err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "worker_a failed err=%d\n", (int)worker_a.first_err);
|
||||
goto cleanup;
|
||||
}
|
||||
if (worker_b.first_err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "worker_b failed err=%d\n", (int)worker_b.first_err);
|
||||
goto cleanup;
|
||||
}
|
||||
{
|
||||
bool seen[2u + 2u * WORKER_ITERS];
|
||||
size_t i;
|
||||
memset(seen, 0, sizeof(seen));
|
||||
for (i = 0u; i < WORKER_ITERS; ++i) {
|
||||
if (worker_a_offsets[i] < 2u ||
|
||||
worker_a_offsets[i] >= (uint64_t)(2u + 2u * WORKER_ITERS)) {
|
||||
fprintf(stderr, "worker_a offset out of range: %" PRIu64 "\n",
|
||||
worker_a_offsets[i]);
|
||||
goto cleanup;
|
||||
}
|
||||
if (seen[worker_a_offsets[i]]) {
|
||||
fprintf(stderr, "duplicate offset from worker_a: %" PRIu64 "\n",
|
||||
worker_a_offsets[i]);
|
||||
goto cleanup;
|
||||
}
|
||||
seen[worker_a_offsets[i]] = true;
|
||||
}
|
||||
for (i = 0u; i < WORKER_ITERS; ++i) {
|
||||
if (worker_b_offsets[i] < 2u ||
|
||||
worker_b_offsets[i] >= (uint64_t)(2u + 2u * WORKER_ITERS)) {
|
||||
fprintf(stderr, "worker_b offset out of range: %" PRIu64 "\n",
|
||||
worker_b_offsets[i]);
|
||||
goto cleanup;
|
||||
}
|
||||
if (seen[worker_b_offsets[i]]) {
|
||||
fprintf(stderr, "duplicate offset from worker_b: %" PRIu64 "\n",
|
||||
worker_b_offsets[i]);
|
||||
goto cleanup;
|
||||
}
|
||||
seen[worker_b_offsets[i]] = true;
|
||||
}
|
||||
for (i = 2u; i < (2u + 2u * WORKER_ITERS); ++i) {
|
||||
if (!seen[i]) {
|
||||
fprintf(stderr, "missing worker offset: %zu\n", i);
|
||||
goto cleanup;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (amduat_asl_log_read(&log_a,
|
||||
log_name,
|
||||
0u,
|
||||
4096u,
|
||||
&out_entries,
|
||||
&out_len,
|
||||
&out_next_offset,
|
||||
&out_end) != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "log read after stress failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (out_len != (2u + 2u * WORKER_ITERS) || out_next_offset != out_len ||
|
||||
!out_end) {
|
||||
fprintf(stderr,
|
||||
"stress read mismatch len=%zu next=%" PRIu64 " expected=%u end=%d\n",
|
||||
out_len, out_next_offset, (unsigned)(2u + 2u * WORKER_ITERS),
|
||||
out_end ? 1 : 0);
|
||||
goto cleanup;
|
||||
}
|
||||
amduat_asl_log_entries_free(out_entries, out_len);
|
||||
out_entries = NULL;
|
||||
out_len = 0u;
|
||||
|
||||
memset(&mixed_append_worker, 0, sizeof(mixed_append_worker));
|
||||
mixed_append_worker.store = &store_a;
|
||||
mixed_append_worker.log_store = &log_a;
|
||||
mixed_append_worker.log_name = log_name;
|
||||
mixed_append_worker.kind = 12u;
|
||||
mixed_append_worker.iterations = WORKER_ITERS;
|
||||
|
||||
memset(&mixed_put_worker, 0, sizeof(mixed_put_worker));
|
||||
mixed_put_worker.store = &store_b;
|
||||
mixed_put_worker.kind = 13u;
|
||||
mixed_put_worker.iterations = WORKER_ITERS * 4u;
|
||||
|
||||
if (pthread_create(&mixed_append_thread, NULL, append_worker_main,
|
||||
&mixed_append_worker) != 0) {
|
||||
fprintf(stderr, "pthread_create mixed append failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
started_mixed_append = true;
|
||||
if (pthread_create(&mixed_put_thread, NULL, put_worker_main,
|
||||
&mixed_put_worker) != 0) {
|
||||
fprintf(stderr, "pthread_create mixed put failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
started_mixed_put = true;
|
||||
(void)pthread_join(mixed_append_thread, NULL);
|
||||
(void)pthread_join(mixed_put_thread, NULL);
|
||||
started_mixed_append = false;
|
||||
started_mixed_put = false;
|
||||
|
||||
if (mixed_append_worker.first_err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "mixed append worker failed err=%d\n",
|
||||
(int)mixed_append_worker.first_err);
|
||||
goto cleanup;
|
||||
}
|
||||
if (mixed_put_worker.first_err != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "mixed put worker failed err=%d\n",
|
||||
(int)mixed_put_worker.first_err);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
rc = 0;
|
||||
|
||||
cleanup:
|
||||
if (started_a) {
|
||||
(void)pthread_join(thread_a, NULL);
|
||||
}
|
||||
if (started_b) {
|
||||
(void)pthread_join(thread_b, NULL);
|
||||
}
|
||||
if (started_mixed_append) {
|
||||
(void)pthread_join(mixed_append_thread, NULL);
|
||||
}
|
||||
if (started_mixed_put) {
|
||||
(void)pthread_join(mixed_put_thread, NULL);
|
||||
}
|
||||
amduat_asl_log_entries_free(out_entries, out_len);
|
||||
amduat_reference_free(&payload_ref_a);
|
||||
amduat_reference_free(&payload_ref_b);
|
||||
if (root != NULL) {
|
||||
if (!remove_tree(root)) {
|
||||
fprintf(stderr, "warning: cleanup failed for %s\n", root);
|
||||
}
|
||||
free(root);
|
||||
}
|
||||
return rc;
|
||||
}
|
||||
291
tests/asl/test_asl_replay.c
Normal file
291
tests/asl/test_asl_replay.c
Normal file
|
|
@ -0,0 +1,291 @@
|
|||
#include "amduat/asl/index_replay.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
|
||||
static void store_u16_le(uint8_t *out, uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void store_u64_le(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static size_t build_artifact_ref(uint8_t *out,
|
||||
size_t out_cap,
|
||||
amduat_hash_id_t hash_id,
|
||||
const uint8_t *digest,
|
||||
size_t digest_len) {
|
||||
size_t total = 4 + 2 + 2 + digest_len;
|
||||
if (out_cap < total) {
|
||||
return 0;
|
||||
}
|
||||
store_u32_le(out, (uint32_t)hash_id);
|
||||
store_u16_le(out + 4, (uint16_t)digest_len);
|
||||
store_u16_le(out + 6, 0);
|
||||
memcpy(out + 8, digest, digest_len);
|
||||
return total;
|
||||
}
|
||||
|
||||
static size_t build_tombstone_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
amduat_hash_id_t hash_id,
|
||||
const uint8_t *digest,
|
||||
size_t digest_len) {
|
||||
size_t offset = 0;
|
||||
size_t ref_len = build_artifact_ref(out,
|
||||
out_cap,
|
||||
hash_id,
|
||||
digest,
|
||||
digest_len);
|
||||
if (ref_len == 0 || out_cap < ref_len + 8) {
|
||||
return 0;
|
||||
}
|
||||
offset += ref_len;
|
||||
store_u32_le(out + offset, 0);
|
||||
offset += 4;
|
||||
store_u32_le(out + offset, 0);
|
||||
offset += 4;
|
||||
return offset;
|
||||
}
|
||||
|
||||
static size_t build_tombstone_lift_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
amduat_hash_id_t hash_id,
|
||||
const uint8_t *digest,
|
||||
size_t digest_len,
|
||||
uint64_t tombstone_logseq) {
|
||||
size_t offset = 0;
|
||||
size_t ref_len = build_artifact_ref(out,
|
||||
out_cap,
|
||||
hash_id,
|
||||
digest,
|
||||
digest_len);
|
||||
if (ref_len == 0 || out_cap < ref_len + 8) {
|
||||
return 0;
|
||||
}
|
||||
offset += ref_len;
|
||||
store_u64_le(out + offset, tombstone_logseq);
|
||||
offset += 8;
|
||||
return offset;
|
||||
}
|
||||
|
||||
static size_t build_segment_seal_payload(uint8_t *out,
|
||||
size_t out_cap,
|
||||
uint64_t segment_id,
|
||||
const uint8_t hash[32]) {
|
||||
if (out_cap < 8 + 32) {
|
||||
return 0;
|
||||
}
|
||||
store_u64_le(out, segment_id);
|
||||
memcpy(out + 8, hash, 32);
|
||||
return 8 + 32;
|
||||
}
|
||||
|
||||
static int test_tombstone_lift_cutoff(void) {
|
||||
amduat_asl_log_record_t records[2];
|
||||
amduat_asl_replay_state_t state;
|
||||
uint8_t digest[32];
|
||||
uint8_t tombstone_payload[4 + 2 + 2 + 32 + 4 + 4];
|
||||
uint8_t lift_payload[4 + 2 + 2 + 32 + 8];
|
||||
size_t tombstone_len;
|
||||
size_t lift_len;
|
||||
amduat_reference_t ref;
|
||||
|
||||
memset(digest, 0x5a, sizeof(digest));
|
||||
tombstone_len = build_tombstone_payload(tombstone_payload,
|
||||
sizeof(tombstone_payload),
|
||||
AMDUAT_HASH_ASL1_ID_SHA256,
|
||||
digest,
|
||||
sizeof(digest));
|
||||
lift_len = build_tombstone_lift_payload(lift_payload,
|
||||
sizeof(lift_payload),
|
||||
AMDUAT_HASH_ASL1_ID_SHA256,
|
||||
digest,
|
||||
sizeof(digest),
|
||||
20);
|
||||
if (tombstone_len == 0 || lift_len == 0) {
|
||||
fprintf(stderr, "payload build failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 20;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
|
||||
records[0].payload = amduat_octets(tombstone_payload, tombstone_len);
|
||||
records[1].logseq = 40;
|
||||
records[1].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT;
|
||||
records[1].payload = amduat_octets(lift_payload, lift_len);
|
||||
|
||||
if (!amduat_asl_replay_init(&state)) {
|
||||
fprintf(stderr, "replay init failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!amduat_asl_replay_apply_log(records, 2, 30, &state)) {
|
||||
fprintf(stderr, "replay apply failed\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
if (state.tombstones_len != 1) {
|
||||
fprintf(stderr, "tombstone not applied\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
ref = amduat_reference(AMDUAT_HASH_ASL1_ID_SHA256,
|
||||
amduat_octets(digest, sizeof(digest)));
|
||||
if (state.tombstones[0].tombstone_logseq != 20 ||
|
||||
!amduat_reference_eq(state.tombstones[0].ref, ref)) {
|
||||
fprintf(stderr, "tombstone mismatch\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
amduat_asl_replay_free(&state);
|
||||
|
||||
if (!amduat_asl_replay_init(&state)) {
|
||||
fprintf(stderr, "replay init failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!amduat_asl_replay_apply_log(records, 2, 40, &state)) {
|
||||
fprintf(stderr, "replay apply failed\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
if (state.tombstones_len != 0) {
|
||||
fprintf(stderr, "tombstone lift failed\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
amduat_asl_replay_free(&state);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_unknown_record_skip(void) {
|
||||
amduat_asl_log_record_t records[2];
|
||||
amduat_asl_replay_state_t state;
|
||||
uint8_t unknown_payload[1] = {0x01};
|
||||
uint8_t seal_payload[8 + 32];
|
||||
uint8_t seal_hash[32];
|
||||
size_t seal_len;
|
||||
|
||||
memset(seal_hash, 0xab, sizeof(seal_hash));
|
||||
seal_len = build_segment_seal_payload(seal_payload,
|
||||
sizeof(seal_payload),
|
||||
9,
|
||||
seal_hash);
|
||||
if (seal_len == 0) {
|
||||
fprintf(stderr, "seal payload build failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 10;
|
||||
records[0].record_type = 0x99;
|
||||
records[0].payload = amduat_octets(unknown_payload, sizeof(unknown_payload));
|
||||
records[1].logseq = 11;
|
||||
records[1].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[1].payload = amduat_octets(seal_payload, seal_len);
|
||||
|
||||
if (!amduat_asl_replay_init(&state)) {
|
||||
fprintf(stderr, "replay init failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!amduat_asl_replay_apply_log(records, 2, 11, &state)) {
|
||||
fprintf(stderr, "replay apply failed\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
if (state.segments_len != 1) {
|
||||
fprintf(stderr, "segment seal missing\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
amduat_asl_replay_free(&state);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_multiple_seals_latest(void) {
|
||||
amduat_asl_log_record_t records[2];
|
||||
amduat_asl_replay_state_t state;
|
||||
uint8_t seal_payload_a[8 + 32];
|
||||
uint8_t seal_payload_b[8 + 32];
|
||||
uint8_t hash_a[32];
|
||||
uint8_t hash_b[32];
|
||||
size_t seal_len_a;
|
||||
size_t seal_len_b;
|
||||
|
||||
memset(hash_a, 0x11, sizeof(hash_a));
|
||||
memset(hash_b, 0x22, sizeof(hash_b));
|
||||
seal_len_a = build_segment_seal_payload(seal_payload_a,
|
||||
sizeof(seal_payload_a),
|
||||
5,
|
||||
hash_a);
|
||||
seal_len_b = build_segment_seal_payload(seal_payload_b,
|
||||
sizeof(seal_payload_b),
|
||||
5,
|
||||
hash_b);
|
||||
if (seal_len_a == 0 || seal_len_b == 0) {
|
||||
fprintf(stderr, "seal payload build failed\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 10;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[0].payload = amduat_octets(seal_payload_a, seal_len_a);
|
||||
records[1].logseq = 20;
|
||||
records[1].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[1].payload = amduat_octets(seal_payload_b, seal_len_b);
|
||||
|
||||
if (!amduat_asl_replay_init(&state)) {
|
||||
fprintf(stderr, "replay init failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!amduat_asl_replay_apply_log(records, 2, 20, &state)) {
|
||||
fprintf(stderr, "replay apply failed\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
if (state.segments_len != 1) {
|
||||
fprintf(stderr, "segment seal count mismatch\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
if (memcmp(state.segments[0].segment_hash, hash_b, sizeof(hash_b)) != 0) {
|
||||
fprintf(stderr, "segment seal not updated\n");
|
||||
amduat_asl_replay_free(&state);
|
||||
return 1;
|
||||
}
|
||||
amduat_asl_replay_free(&state);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_tombstone_lift_cutoff() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_unknown_record_skip() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_multiple_seals_latest() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
1620
tests/asl/test_asl_store_index_fs.c
Normal file
1620
tests/asl/test_asl_store_index_fs.c
Normal file
File diff suppressed because it is too large
Load diff
68
tests/asl/test_asl_store_indexed_ops.c
Normal file
68
tests/asl/test_asl_store_indexed_ops.c
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
#include "amduat/asl/store.h"
|
||||
#include "amduat/enc/asl1_core.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
|
||||
#include <stdio.h>
|
||||
|
||||
static int test_indexed_ops_unsupported(void) {
|
||||
amduat_asl_store_ops_t ops;
|
||||
amduat_asl_store_config_t config;
|
||||
amduat_asl_store_t store;
|
||||
amduat_asl_store_error_t err;
|
||||
amduat_asl_index_state_t state;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_reference_t ref;
|
||||
|
||||
amduat_asl_store_ops_init(&ops);
|
||||
config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
|
||||
config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
|
||||
amduat_asl_store_init(&store, config, ops, NULL);
|
||||
|
||||
artifact = amduat_artifact(amduat_octets(NULL, 0u));
|
||||
ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
state.snapshot_id = 0u;
|
||||
state.log_position = 0u;
|
||||
|
||||
err = amduat_asl_store_put_indexed(&store, artifact, &ref, &state);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
|
||||
fprintf(stderr, "put_indexed missing ops should be unsupported: %d\n", err);
|
||||
return 1;
|
||||
}
|
||||
|
||||
err = amduat_asl_store_get_indexed(&store, ref, state, &artifact);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
|
||||
fprintf(stderr, "get_indexed missing ops should be unsupported: %d\n", err);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (amduat_asl_index_current_state(&store, &state)) {
|
||||
fprintf(stderr, "current_state missing ops should be false\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
err = amduat_asl_store_tombstone(&store, ref, 0u, 0u, &state);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
|
||||
fprintf(stderr, "tombstone missing ops should be unsupported: %d\n", err);
|
||||
return 1;
|
||||
}
|
||||
|
||||
err = amduat_asl_store_tombstone_lift(&store, ref, 1u, &state);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
|
||||
fprintf(stderr,
|
||||
"tombstone_lift missing ops should be unsupported: %d\n",
|
||||
err);
|
||||
return 1;
|
||||
}
|
||||
|
||||
err = amduat_asl_log_scan(&store, NULL, NULL);
|
||||
if (err != AMDUAT_ASL_STORE_ERR_UNSUPPORTED) {
|
||||
fprintf(stderr, "log_scan missing ops should be unsupported: %d\n", err);
|
||||
return 1;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
return test_indexed_ops_unsupported();
|
||||
}
|
||||
336
tests/enc/test_asl_core_index.c
Normal file
336
tests/enc/test_asl_core_index.c
Normal file
|
|
@ -0,0 +1,336 @@
|
|||
#include "amduat/enc/asl_core_index.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static void store_u16_le(uint8_t *out, uint16_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
}
|
||||
|
||||
static void store_u64_le(uint8_t *out, uint64_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
out[4] = (uint8_t)((value >> 32) & 0xffu);
|
||||
out[5] = (uint8_t)((value >> 40) & 0xffu);
|
||||
out[6] = (uint8_t)((value >> 48) & 0xffu);
|
||||
out[7] = (uint8_t)((value >> 56) & 0xffu);
|
||||
}
|
||||
|
||||
static uint64_t load_u64_le(const uint8_t *data) {
|
||||
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
|
||||
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
|
||||
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
|
||||
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
|
||||
}
|
||||
|
||||
static uint64_t crc64_ecma(const uint8_t *data, size_t len) {
|
||||
uint64_t crc = 0u;
|
||||
size_t i;
|
||||
|
||||
for (i = 0; i < len; ++i) {
|
||||
uint64_t bit;
|
||||
uint8_t value = data[i];
|
||||
crc ^= ((uint64_t)value) << 56;
|
||||
for (bit = 0; bit < 8; ++bit) {
|
||||
if (crc & 0x8000000000000000ull) {
|
||||
crc = (crc << 1) ^ 0x42f0e1eba9ea3693ull;
|
||||
} else {
|
||||
crc <<= 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
return crc;
|
||||
}
|
||||
|
||||
static void update_crc(amduat_octets_t bytes) {
|
||||
size_t footer_offset = bytes.len - AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE;
|
||||
uint64_t crc = crc64_ecma(bytes.data, footer_offset);
|
||||
store_u64_le((uint8_t *)bytes.data + footer_offset, crc);
|
||||
}
|
||||
|
||||
static void build_simple_segment(amduat_asl_core_index_segment_t *segment,
|
||||
amduat_asl_index_record_t *record,
|
||||
amduat_asl_extent_record_t *extent,
|
||||
uint8_t digest_bytes[8]) {
|
||||
memset(segment, 0, sizeof(*segment));
|
||||
memset(record, 0, sizeof(*record));
|
||||
memset(extent, 0, sizeof(*extent));
|
||||
|
||||
digest_bytes[0] = 0x01;
|
||||
digest_bytes[1] = 0x02;
|
||||
digest_bytes[2] = 0x03;
|
||||
digest_bytes[3] = 0x04;
|
||||
digest_bytes[4] = 0x05;
|
||||
digest_bytes[5] = 0x06;
|
||||
digest_bytes[6] = 0x07;
|
||||
digest_bytes[7] = 0x08;
|
||||
|
||||
segment->header.snapshot_min = 5;
|
||||
segment->header.snapshot_max = 6;
|
||||
segment->header.segment_domain_id = 7;
|
||||
segment->header.segment_visibility = 1;
|
||||
segment->header.federation_version = 0;
|
||||
segment->header.flags = 0;
|
||||
segment->header.reserved0 = 0;
|
||||
|
||||
record->hash_id = 9;
|
||||
record->digest_len = 8;
|
||||
record->extent_count = 1;
|
||||
record->total_length = 3;
|
||||
record->domain_id = 7;
|
||||
record->visibility = 1;
|
||||
record->has_cross_domain_source = 0;
|
||||
record->cross_domain_source = 0;
|
||||
record->flags = 0;
|
||||
|
||||
extent->block_id = 123;
|
||||
extent->offset = 0;
|
||||
extent->length = 3;
|
||||
|
||||
segment->records = record;
|
||||
segment->record_count = 1;
|
||||
segment->digests = amduat_octets(digest_bytes, 8);
|
||||
segment->extents = extent;
|
||||
segment->extent_count = 1;
|
||||
segment->footer.seal_snapshot = 10;
|
||||
segment->footer.seal_time_ns = 11;
|
||||
}
|
||||
|
||||
static int test_round_trip(void) {
|
||||
amduat_asl_core_index_segment_t segment;
|
||||
amduat_asl_index_record_t record;
|
||||
amduat_asl_extent_record_t extent;
|
||||
uint8_t digest_bytes[8];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_core_index_segment_t decoded;
|
||||
int exit_code = 1;
|
||||
|
||||
build_simple_segment(&segment, &record, &extent, digest_bytes);
|
||||
|
||||
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded.header.version != AMDUAT_ASL_CORE_INDEX_VERSION ||
|
||||
decoded.header.header_size != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
|
||||
fprintf(stderr, "header mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.record_count != 1 || decoded.extent_count != 1) {
|
||||
fprintf(stderr, "count mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.header.segment_domain_id != 7 ||
|
||||
decoded.header.segment_visibility != 1) {
|
||||
fprintf(stderr, "segment federation mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.records[0].hash_id != record.hash_id ||
|
||||
decoded.records[0].digest_len != record.digest_len ||
|
||||
decoded.records[0].total_length != record.total_length ||
|
||||
decoded.records[0].domain_id != record.domain_id ||
|
||||
decoded.records[0].visibility != record.visibility ||
|
||||
decoded.records[0].flags != record.flags) {
|
||||
fprintf(stderr, "record mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.digests.len != sizeof(digest_bytes) ||
|
||||
memcmp(decoded.digests.data, digest_bytes, sizeof(digest_bytes)) != 0) {
|
||||
fprintf(stderr, "digest mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.extents[0].block_id != extent.block_id ||
|
||||
decoded.extents[0].offset != extent.offset ||
|
||||
decoded.extents[0].length != extent.length) {
|
||||
fprintf(stderr, "extent mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.footer.seal_snapshot != segment.footer.seal_snapshot ||
|
||||
decoded.footer.seal_time_ns != segment.footer.seal_time_ns) {
|
||||
fprintf(stderr, "footer mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_asl_core_index_free(&decoded);
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_crc_mismatch(void) {
|
||||
amduat_asl_core_index_segment_t segment;
|
||||
amduat_asl_index_record_t record;
|
||||
amduat_asl_extent_record_t extent;
|
||||
uint8_t digest_bytes[8];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_core_index_segment_t decoded;
|
||||
uint64_t digests_offset;
|
||||
int exit_code = 1;
|
||||
|
||||
build_simple_segment(&segment, &record, &extent, digest_bytes);
|
||||
|
||||
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
digests_offset = load_u64_le(encoded.data + 64);
|
||||
((uint8_t *)encoded.data)[digests_offset] ^= 0xffu;
|
||||
|
||||
if (amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
amduat_enc_asl_core_index_free(&decoded);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_invalid_offsets(void) {
|
||||
amduat_asl_core_index_segment_t segment;
|
||||
amduat_asl_index_record_t record;
|
||||
amduat_asl_extent_record_t extent;
|
||||
uint8_t digest_bytes[8];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_core_index_segment_t decoded;
|
||||
int exit_code = 1;
|
||||
|
||||
build_simple_segment(&segment, &record, &extent, digest_bytes);
|
||||
|
||||
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
store_u64_le((uint8_t *)encoded.data + 72, 0);
|
||||
update_crc(encoded);
|
||||
|
||||
if (amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
amduat_enc_asl_core_index_free(&decoded);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_invalid_federation_fields(void) {
|
||||
amduat_asl_core_index_segment_t segment;
|
||||
amduat_asl_index_record_t record;
|
||||
amduat_asl_extent_record_t extent;
|
||||
uint8_t digest_bytes[8];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_core_index_segment_t decoded;
|
||||
uint64_t records_offset;
|
||||
int exit_code = 1;
|
||||
|
||||
build_simple_segment(&segment, &record, &extent, digest_bytes);
|
||||
|
||||
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
records_offset = load_u64_le(encoded.data + 40);
|
||||
((uint8_t *)encoded.data)[records_offset + 36] = 2;
|
||||
update_crc(encoded);
|
||||
|
||||
if (amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
amduat_enc_asl_core_index_free(&decoded);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_legacy_defaults(void) {
|
||||
amduat_asl_core_index_segment_t segment;
|
||||
amduat_asl_index_record_t record;
|
||||
amduat_asl_extent_record_t extent;
|
||||
uint8_t digest_bytes[8];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_core_index_segment_t decoded;
|
||||
uint64_t records_offset;
|
||||
int exit_code = 1;
|
||||
|
||||
build_simple_segment(&segment, &record, &extent, digest_bytes);
|
||||
|
||||
if (!amduat_enc_asl_core_index_encode_v1(&segment, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
store_u16_le((uint8_t *)encoded.data + 8, 2);
|
||||
((uint8_t *)encoded.data)[100] = 1;
|
||||
records_offset = load_u64_le(encoded.data + 40);
|
||||
((uint8_t *)encoded.data)[records_offset + 36] = 1;
|
||||
update_crc(encoded);
|
||||
|
||||
if (!amduat_enc_asl_core_index_decode_v1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded.header.segment_visibility != 0 ||
|
||||
decoded.records[0].visibility != 0 ||
|
||||
decoded.records[0].domain_id != 0 ||
|
||||
decoded.records[0].has_cross_domain_source != 0 ||
|
||||
decoded.records[0].cross_domain_source != 0) {
|
||||
fprintf(stderr, "legacy defaults not applied\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_asl_core_index_free(&decoded);
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_round_trip() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_crc_mismatch() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_invalid_offsets() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_invalid_federation_fields() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_legacy_defaults() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
217
tests/enc/test_asl_log.c
Normal file
217
tests/enc/test_asl_log.c
Normal file
|
|
@ -0,0 +1,217 @@
|
|||
#include "amduat/enc/asl_log.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static bool octets_equal(amduat_octets_t a, amduat_octets_t b) {
|
||||
if (a.len != b.len) {
|
||||
return false;
|
||||
}
|
||||
if (a.len == 0) {
|
||||
return true;
|
||||
}
|
||||
return memcmp(a.data, b.data, a.len) == 0;
|
||||
}
|
||||
|
||||
static int test_round_trip(void) {
|
||||
uint8_t payload_a[] = {0x01, 0x02, 0x03};
|
||||
uint8_t payload_b[] = {0x10, 0x20};
|
||||
amduat_asl_log_record_t records[2];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_log_record_t *decoded = NULL;
|
||||
size_t decoded_len = 0;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 1;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[0].payload = amduat_octets(payload_a, sizeof(payload_a));
|
||||
records[1].logseq = 2;
|
||||
records[1].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
|
||||
records[1].payload = amduat_octets(payload_b, sizeof(payload_b));
|
||||
|
||||
if (!amduat_enc_asl_log_encode_v1(records, 2, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_asl_log_decode_v1(encoded, &decoded, &decoded_len)) {
|
||||
fprintf(stderr, "decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded_len != 2) {
|
||||
fprintf(stderr, "decoded length mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded[0].logseq != records[0].logseq ||
|
||||
decoded[1].logseq != records[1].logseq) {
|
||||
fprintf(stderr, "decoded logseq mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded[0].record_type != records[0].record_type ||
|
||||
decoded[1].record_type != records[1].record_type) {
|
||||
fprintf(stderr, "decoded record type mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (!octets_equal(decoded[0].payload, records[0].payload) ||
|
||||
!octets_equal(decoded[1].payload, records[1].payload)) {
|
||||
fprintf(stderr, "decoded payload mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_asl_log_free(decoded, decoded_len);
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_hash_chain_mutation(void) {
|
||||
uint8_t payload_a[] = {0xaa, 0xbb, 0xcc};
|
||||
amduat_asl_log_record_t record;
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_log_record_t *decoded = NULL;
|
||||
size_t decoded_len = 0;
|
||||
int exit_code = 1;
|
||||
size_t payload_offset;
|
||||
|
||||
memset(&record, 0, sizeof(record));
|
||||
record.logseq = 1;
|
||||
record.record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
record.payload = amduat_octets(payload_a, sizeof(payload_a));
|
||||
|
||||
if (!amduat_enc_asl_log_encode_v1(&record, 1, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
payload_offset = 24 + 8 + 4 + 4;
|
||||
if (encoded.len <= payload_offset) {
|
||||
fprintf(stderr, "encoded payload offset invalid\n");
|
||||
goto cleanup;
|
||||
}
|
||||
((uint8_t *)encoded.data)[payload_offset] ^= 0xffu;
|
||||
|
||||
if (amduat_enc_asl_log_decode_v1(encoded, &decoded, &decoded_len)) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
amduat_enc_asl_log_free(decoded, decoded_len);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_unknown_record_type(void) {
|
||||
uint8_t payload_a[] = {0x11};
|
||||
uint8_t payload_b[] = {0x22};
|
||||
uint8_t payload_c[] = {0x33};
|
||||
amduat_asl_log_record_t records[3];
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_log_record_t *decoded = NULL;
|
||||
size_t decoded_len = 0;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(records, 0, sizeof(records));
|
||||
records[0].logseq = 1;
|
||||
records[0].record_type = AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL;
|
||||
records[0].payload = amduat_octets(payload_a, sizeof(payload_a));
|
||||
records[1].logseq = 2;
|
||||
records[1].record_type = 0x99;
|
||||
records[1].payload = amduat_octets(payload_b, sizeof(payload_b));
|
||||
records[2].logseq = 3;
|
||||
records[2].record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
|
||||
records[2].payload = amduat_octets(payload_c, sizeof(payload_c));
|
||||
|
||||
if (!amduat_enc_asl_log_encode_v1(records, 3, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_asl_log_decode_v1(encoded, &decoded, &decoded_len)) {
|
||||
fprintf(stderr, "decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded_len != 2) {
|
||||
fprintf(stderr, "unknown record not skipped\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded[0].logseq != records[0].logseq ||
|
||||
decoded[1].logseq != records[2].logseq) {
|
||||
fprintf(stderr, "decoded logseq mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_asl_log_free(decoded, decoded_len);
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_truncated_payload(void) {
|
||||
uint8_t payload_a[] = {0xaa, 0xbb, 0xcc, 0xdd};
|
||||
amduat_asl_log_record_t record;
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_log_record_t *decoded = NULL;
|
||||
size_t decoded_len = 0;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(&record, 0, sizeof(record));
|
||||
record.logseq = 42;
|
||||
record.record_type = AMDUAT_ASL_LOG_RECORD_TOMBSTONE;
|
||||
record.payload = amduat_octets(payload_a, sizeof(payload_a));
|
||||
|
||||
if (!amduat_enc_asl_log_encode_v1(&record, 1, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (encoded.len == 0) {
|
||||
fprintf(stderr, "encoded length invalid\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (amduat_enc_asl_log_decode_v1(
|
||||
amduat_octets(encoded.data, encoded.len - 1),
|
||||
&decoded,
|
||||
&decoded_len)) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
amduat_enc_asl_log_free(decoded, decoded_len);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_round_trip() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_hash_chain_mutation() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_unknown_record_type() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_truncated_payload() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
231
tests/enc/test_asl_tgk_exec_plan.c
Normal file
231
tests/enc/test_asl_tgk_exec_plan.c
Normal file
|
|
@ -0,0 +1,231 @@
|
|||
#include "amduat/enc/asl_tgk_exec_plan.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
enum {
|
||||
k_plan_header_size = 8,
|
||||
k_params_size = 65,
|
||||
k_operator_size = 129,
|
||||
k_input_offset = 97
|
||||
};
|
||||
|
||||
static void store_u32_le(uint8_t *out, uint32_t value) {
|
||||
out[0] = (uint8_t)(value & 0xffu);
|
||||
out[1] = (uint8_t)((value >> 8) & 0xffu);
|
||||
out[2] = (uint8_t)((value >> 16) & 0xffu);
|
||||
out[3] = (uint8_t)((value >> 24) & 0xffu);
|
||||
}
|
||||
|
||||
static void fill_default_params(amduat_asl_tgk_exec_operator_params_t *params) {
|
||||
memset(params, 0, sizeof(*params));
|
||||
params->tgk_traversal.direction = 1u;
|
||||
}
|
||||
|
||||
static int test_round_trip(void) {
|
||||
amduat_asl_tgk_exec_operator_def_t operators[2];
|
||||
amduat_asl_tgk_exec_plan_t plan;
|
||||
amduat_octets_t encoded;
|
||||
amduat_asl_tgk_exec_plan_t decoded;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(operators, 0, sizeof(operators));
|
||||
fill_default_params(&operators[0].params);
|
||||
fill_default_params(&operators[1].params);
|
||||
|
||||
operators[0].op_id = 1u;
|
||||
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
|
||||
operators[0].flags = AMDUAT_ASL_TGK_EXEC_OP_FLAG_NONE;
|
||||
operators[0].snapshot.logseq_min = 0u;
|
||||
operators[0].snapshot.logseq_max = 10u;
|
||||
operators[0].input_count = 0u;
|
||||
|
||||
operators[1].op_id = 2u;
|
||||
operators[1].op_type = AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL;
|
||||
operators[1].flags = AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL;
|
||||
operators[1].snapshot.logseq_min = 1u;
|
||||
operators[1].snapshot.logseq_max = 10u;
|
||||
operators[1].params.tgk_traversal.direction = 2u;
|
||||
operators[1].params.tgk_traversal.traversal_depth = 3u;
|
||||
operators[1].params.tgk_traversal.start_node_id = 42u;
|
||||
operators[1].input_count = 1u;
|
||||
operators[1].inputs[0] = 1u;
|
||||
|
||||
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
|
||||
plan.operator_count = 2u;
|
||||
plan.operators = operators;
|
||||
|
||||
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_asl_tgk_exec_plan_decode_v1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded.plan_version != plan.plan_version ||
|
||||
decoded.operator_count != plan.operator_count) {
|
||||
fprintf(stderr, "header mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.operators[1].op_type != operators[1].op_type ||
|
||||
decoded.operators[1].params.tgk_traversal.direction != 2u ||
|
||||
decoded.operators[1].params.tgk_traversal.traversal_depth != 3u ||
|
||||
decoded.operators[1].params.tgk_traversal.start_node_id != 42u) {
|
||||
fprintf(stderr, "operator fields mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
if (decoded.operators[1].input_count != 1u ||
|
||||
decoded.operators[1].inputs[0] != 1u) {
|
||||
fprintf(stderr, "inputs mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_asl_tgk_exec_plan_free(&decoded);
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_invalid_input_ref(void) {
|
||||
amduat_asl_tgk_exec_operator_def_t operators[1];
|
||||
amduat_asl_tgk_exec_plan_t plan;
|
||||
amduat_octets_t encoded;
|
||||
int exit_code = 1;
|
||||
size_t op_offset;
|
||||
size_t input_offset;
|
||||
|
||||
memset(operators, 0, sizeof(operators));
|
||||
fill_default_params(&operators[0].params);
|
||||
operators[0].op_id = 1u;
|
||||
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
|
||||
operators[0].input_count = 0u;
|
||||
|
||||
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
|
||||
plan.operator_count = 1u;
|
||||
plan.operators = operators;
|
||||
|
||||
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
op_offset = k_plan_header_size;
|
||||
if (encoded.len < op_offset + k_input_offset + 4u) {
|
||||
fprintf(stderr, "encoded buffer too small\n");
|
||||
goto cleanup;
|
||||
}
|
||||
store_u32_le((uint8_t *)encoded.data + op_offset + k_params_size + 28u, 1u);
|
||||
input_offset = op_offset + k_input_offset;
|
||||
store_u32_le((uint8_t *)encoded.data + input_offset, 99u);
|
||||
|
||||
if (amduat_enc_asl_tgk_exec_plan_decode_v1(encoded, &(amduat_asl_tgk_exec_plan_t){0})) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_truncated_decode(void) {
|
||||
amduat_asl_tgk_exec_operator_def_t operators[1];
|
||||
amduat_asl_tgk_exec_plan_t plan;
|
||||
amduat_octets_t encoded;
|
||||
amduat_octets_t truncated;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(operators, 0, sizeof(operators));
|
||||
fill_default_params(&operators[0].params);
|
||||
operators[0].op_id = 1u;
|
||||
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
|
||||
|
||||
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
|
||||
plan.operator_count = 1u;
|
||||
plan.operators = operators;
|
||||
|
||||
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
truncated = amduat_octets(encoded.data, encoded.len - 1u);
|
||||
if (amduat_enc_asl_tgk_exec_plan_decode_v1(truncated,
|
||||
&(amduat_asl_tgk_exec_plan_t){0})) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_invalid_input_count(void) {
|
||||
amduat_asl_tgk_exec_operator_def_t operators[1];
|
||||
amduat_asl_tgk_exec_plan_t plan;
|
||||
amduat_octets_t encoded;
|
||||
int exit_code = 1;
|
||||
size_t op_offset;
|
||||
size_t input_count_offset;
|
||||
|
||||
memset(operators, 0, sizeof(operators));
|
||||
fill_default_params(&operators[0].params);
|
||||
operators[0].op_id = 1u;
|
||||
operators[0].op_type = AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN;
|
||||
|
||||
plan.plan_version = AMDUAT_ASL_TGK_EXEC_PLAN_VERSION;
|
||||
plan.operator_count = 1u;
|
||||
plan.operators = operators;
|
||||
|
||||
if (!amduat_enc_asl_tgk_exec_plan_encode_v1(&plan, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
op_offset = k_plan_header_size;
|
||||
input_count_offset = op_offset + k_params_size + 28u;
|
||||
if (encoded.len < input_count_offset + 4u) {
|
||||
fprintf(stderr, "encoded buffer too small\n");
|
||||
goto cleanup;
|
||||
}
|
||||
store_u32_le((uint8_t *)encoded.data + input_count_offset, 9u);
|
||||
|
||||
if (amduat_enc_asl_tgk_exec_plan_decode_v1(encoded,
|
||||
&(amduat_asl_tgk_exec_plan_t){0})) {
|
||||
fprintf(stderr, "decode unexpectedly succeeded\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_round_trip() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_invalid_input_ref() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_truncated_decode() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return test_invalid_input_count();
|
||||
}
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
#include "amduat/enc/fer1_receipt.h"
|
||||
#include "amduat/fer/receipt.h"
|
||||
#include "amduat/pel/run.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
|
|
@ -38,16 +39,15 @@ static const uint8_t k_expected_receipt_bytes[] = {
|
|||
0x01, 0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x60, 0x60, 0x60, 0x60, 0x60,
|
||||
0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60,
|
||||
0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60, 0x60,
|
||||
0x60, 0x60, 0x60, 0x60, 0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, 0x00,
|
||||
0x00, 0x00, 0x22, 0x00, 0x01, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
|
||||
0x60, 0x60, 0x60, 0x00, 0x00, 0x00, 0x03, 0xaa, 0xbb, 0xcc, 0x00, 0x00,
|
||||
0x00, 0x22, 0x00, 0x01, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
|
||||
0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
|
||||
0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51, 0x51,
|
||||
0x51, 0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x44, 0x44, 0x44, 0x44, 0x44,
|
||||
0x00, 0x00, 0x00, 0x22, 0x00, 0x01, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44,
|
||||
0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44,
|
||||
0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44, 0x44,
|
||||
0x44, 0x44, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x14,
|
||||
0x44, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x14
|
||||
};
|
||||
|
||||
static const uint8_t k_expected_receipt_helper_bytes[] = {
|
||||
|
|
@ -103,6 +103,7 @@ static bool bytes_equal(amduat_octets_t bytes,
|
|||
static int test_receipt_round_trip(void) {
|
||||
amduat_fer1_receipt_t receipt;
|
||||
amduat_octets_t encoded;
|
||||
amduat_octets_t mutated;
|
||||
amduat_fer1_receipt_t decoded;
|
||||
amduat_reference_t executor_refs[2];
|
||||
amduat_fer1_parity_entry_t parity[2];
|
||||
|
|
@ -261,6 +262,640 @@ static int test_receipt_helper(void) {
|
|||
return 0;
|
||||
}
|
||||
|
||||
static int test_receipt_helper_failed_run(void) {
|
||||
amduat_pel_run_result_t pel_run;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_fer1_receipt_t decoded;
|
||||
uint8_t f0[32], i0[32], e0[32], r0[32];
|
||||
uint8_t ex0[32];
|
||||
int exit_code = 1;
|
||||
|
||||
memset(&pel_run, 0, sizeof(pel_run));
|
||||
pel_run.result_ref = make_ref(0x77, r0);
|
||||
pel_run.output_refs = NULL;
|
||||
pel_run.output_refs_len = 0;
|
||||
pel_run.has_result_value = true;
|
||||
pel_run.result_value.pel1_version = 1;
|
||||
pel_run.result_value.program_ref = make_ref(0x11, f0);
|
||||
|
||||
if (!amduat_fer1_receipt_from_pel_run(
|
||||
&pel_run,
|
||||
make_ref(0x22, i0),
|
||||
make_ref(0x33, e0),
|
||||
amduat_octets("tester", 6),
|
||||
make_ref(0x50, ex0),
|
||||
false,
|
||||
amduat_reference(0, amduat_octets(NULL, 0)),
|
||||
amduat_octets(NULL, 0),
|
||||
10,
|
||||
20,
|
||||
&artifact)) {
|
||||
fprintf(stderr, "failed run helper failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_fer1_receipt_decode_v1(artifact.bytes, &decoded)) {
|
||||
fprintf(stderr, "failed run helper decode failed\n");
|
||||
amduat_artifact_free(&artifact);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_reference_eq(decoded.output_ref, pel_run.result_ref) ||
|
||||
!amduat_reference_eq(decoded.function_ref,
|
||||
pel_run.result_value.program_ref)) {
|
||||
fprintf(stderr, "failed run helper decoded refs mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_fer1_receipt_free(&decoded);
|
||||
amduat_artifact_free(&artifact);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_receipt_round_trip_v1_1(void) {
|
||||
amduat_fer1_receipt_t receipt;
|
||||
amduat_octets_t encoded;
|
||||
amduat_fer1_receipt_t decoded;
|
||||
amduat_octets_t mutated;
|
||||
amduat_reference_t executor_refs[1];
|
||||
amduat_fer1_parity_entry_t parity[1];
|
||||
amduat_fer1_log_entry_t logs[2];
|
||||
uint8_t f0[32], i0[32], e0[32], o0[32];
|
||||
uint8_t ex0[32], fp0[32], lr0[32], lr1[32];
|
||||
uint8_t digest0[] = {0xaa, 0xbb, 0xcc};
|
||||
uint8_t run_id[] = {0x01, 0x02, 0x03, 0x04};
|
||||
uint8_t rng_seed[] = {0x09, 0x08, 0x07};
|
||||
uint8_t signature[] = {0xde, 0xad, 0xbe, 0xef};
|
||||
int exit_code = 1;
|
||||
|
||||
memset(&receipt, 0, sizeof(receipt));
|
||||
receipt.fer1_version = AMDUAT_FER1_VERSION_1_1;
|
||||
receipt.function_ref = make_ref(0x11, f0);
|
||||
receipt.input_manifest_ref = make_ref(0x22, i0);
|
||||
receipt.environment_ref = make_ref(0x33, e0);
|
||||
receipt.evaluator_id = amduat_octets("tester", 6);
|
||||
receipt.output_ref = make_ref(0x44, o0);
|
||||
receipt.started_at = 10;
|
||||
receipt.completed_at = 20;
|
||||
|
||||
executor_refs[0] = make_ref(0x50, ex0);
|
||||
receipt.executor_refs = executor_refs;
|
||||
receipt.executor_refs_len = 1;
|
||||
|
||||
memset(parity, 0, sizeof(parity));
|
||||
parity[0].executor_ref = executor_refs[0];
|
||||
parity[0].output_ref = receipt.output_ref;
|
||||
parity[0].has_sbom_ref = false;
|
||||
parity[0].parity_digest = amduat_octets(digest0, sizeof(digest0));
|
||||
receipt.parity = parity;
|
||||
receipt.parity_len = 1;
|
||||
|
||||
receipt.has_executor_fingerprint_ref = true;
|
||||
receipt.executor_fingerprint_ref = make_ref(0x66, fp0);
|
||||
receipt.has_run_id = true;
|
||||
receipt.run_id = amduat_octets(run_id, sizeof(run_id));
|
||||
receipt.has_limits = true;
|
||||
receipt.limits.cpu_ms = 1;
|
||||
receipt.limits.wall_ms = 2;
|
||||
receipt.limits.max_rss_kib = 3;
|
||||
receipt.limits.io_reads = 4;
|
||||
receipt.limits.io_writes = 5;
|
||||
|
||||
memset(logs, 0, sizeof(logs));
|
||||
logs[0].kind = 1;
|
||||
logs[0].log_ref = make_ref(0x70, lr0);
|
||||
logs[0].sha256 = amduat_octets(digest0, sizeof(digest0));
|
||||
logs[1].kind = 2;
|
||||
logs[1].log_ref = make_ref(0x71, lr1);
|
||||
logs[1].sha256 = amduat_octets(NULL, 0);
|
||||
receipt.logs = logs;
|
||||
receipt.logs_len = 2;
|
||||
receipt.has_determinism = true;
|
||||
receipt.determinism_level = 2;
|
||||
receipt.has_rng_seed = true;
|
||||
receipt.rng_seed = amduat_octets(rng_seed, sizeof(rng_seed));
|
||||
receipt.has_signature = true;
|
||||
receipt.signature = amduat_octets(signature, sizeof(signature));
|
||||
|
||||
if (!amduat_enc_fer1_receipt_encode_v1_1(&receipt, &encoded)) {
|
||||
fprintf(stderr, "encode v1.1 failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_fer1_receipt_decode_v1_1(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode v1.1 failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded.fer1_version != AMDUAT_FER1_VERSION_1_1 ||
|
||||
!amduat_reference_eq(decoded.function_ref, receipt.function_ref) ||
|
||||
!amduat_reference_eq(decoded.input_manifest_ref,
|
||||
receipt.input_manifest_ref) ||
|
||||
!amduat_reference_eq(decoded.environment_ref,
|
||||
receipt.environment_ref) ||
|
||||
!amduat_reference_eq(decoded.output_ref, receipt.output_ref)) {
|
||||
fprintf(stderr, "decoded v1.1 refs mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
if (!decoded.has_executor_fingerprint_ref ||
|
||||
!amduat_reference_eq(decoded.executor_fingerprint_ref,
|
||||
receipt.executor_fingerprint_ref) ||
|
||||
!decoded.has_run_id ||
|
||||
!amduat_octets_eq(decoded.run_id, receipt.run_id) ||
|
||||
!decoded.has_determinism ||
|
||||
decoded.determinism_level != receipt.determinism_level ||
|
||||
!decoded.has_rng_seed ||
|
||||
!amduat_octets_eq(decoded.rng_seed, receipt.rng_seed) ||
|
||||
!decoded.has_signature ||
|
||||
!amduat_octets_eq(decoded.signature, receipt.signature) ||
|
||||
!decoded.has_limits ||
|
||||
decoded.limits.cpu_ms != receipt.limits.cpu_ms ||
|
||||
decoded.limits.wall_ms != receipt.limits.wall_ms ||
|
||||
decoded.limits.max_rss_kib != receipt.limits.max_rss_kib ||
|
||||
decoded.limits.io_reads != receipt.limits.io_reads ||
|
||||
decoded.limits.io_writes != receipt.limits.io_writes) {
|
||||
fprintf(stderr, "decoded v1.1 fields mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
if (decoded.logs_len != receipt.logs_len ||
|
||||
!amduat_reference_eq(decoded.logs[0].log_ref, logs[0].log_ref) ||
|
||||
decoded.logs[0].kind != logs[0].kind ||
|
||||
!amduat_octets_eq(decoded.logs[0].sha256, logs[0].sha256) ||
|
||||
!amduat_reference_eq(decoded.logs[1].log_ref, logs[1].log_ref) ||
|
||||
decoded.logs[1].kind != logs[1].kind ||
|
||||
decoded.logs[1].sha256.len != 0) {
|
||||
fprintf(stderr, "decoded v1.1 logs mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_fer1_receipt_free(&decoded);
|
||||
cleanup:
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_receipt_v1_1_reject_duplicate_tag(void) {
|
||||
amduat_fer1_receipt_t receipt;
|
||||
amduat_octets_t encoded;
|
||||
amduat_fer1_receipt_t decoded;
|
||||
amduat_octets_t mutated;
|
||||
amduat_reference_t executor_refs[1];
|
||||
amduat_fer1_parity_entry_t parity[1];
|
||||
amduat_fer1_log_entry_t logs[1];
|
||||
uint8_t f0[32], i0[32], e0[32], o0[32];
|
||||
uint8_t ex0[32], fp0[32], lr0[32];
|
||||
uint8_t digest0[] = {0xaa, 0xbb, 0xcc};
|
||||
uint8_t run_id[] = {0x01, 0x02, 0x03, 0x04};
|
||||
size_t offset = 0;
|
||||
size_t ext_len = 0;
|
||||
size_t ext_offset = 0;
|
||||
size_t tlv_offset = 0;
|
||||
uint32_t len_u32 = 0;
|
||||
size_t count = 0;
|
||||
size_t i = 0;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(&receipt, 0, sizeof(receipt));
|
||||
receipt.fer1_version = AMDUAT_FER1_VERSION_1_1;
|
||||
receipt.function_ref = make_ref(0x11, f0);
|
||||
receipt.input_manifest_ref = make_ref(0x22, i0);
|
||||
receipt.environment_ref = make_ref(0x33, e0);
|
||||
receipt.evaluator_id = amduat_octets("tester", 6);
|
||||
receipt.output_ref = make_ref(0x44, o0);
|
||||
receipt.started_at = 10;
|
||||
receipt.completed_at = 20;
|
||||
|
||||
executor_refs[0] = make_ref(0x50, ex0);
|
||||
receipt.executor_refs = executor_refs;
|
||||
receipt.executor_refs_len = 1;
|
||||
|
||||
memset(parity, 0, sizeof(parity));
|
||||
parity[0].executor_ref = executor_refs[0];
|
||||
parity[0].output_ref = receipt.output_ref;
|
||||
parity[0].has_sbom_ref = false;
|
||||
parity[0].parity_digest = amduat_octets(digest0, sizeof(digest0));
|
||||
receipt.parity = parity;
|
||||
receipt.parity_len = 1;
|
||||
|
||||
receipt.has_executor_fingerprint_ref = true;
|
||||
receipt.executor_fingerprint_ref = make_ref(0x66, fp0);
|
||||
receipt.has_run_id = true;
|
||||
receipt.run_id = amduat_octets(run_id, sizeof(run_id));
|
||||
receipt.has_limits = true;
|
||||
receipt.limits.cpu_ms = 1;
|
||||
receipt.limits.wall_ms = 2;
|
||||
receipt.limits.max_rss_kib = 3;
|
||||
receipt.limits.io_reads = 4;
|
||||
receipt.limits.io_writes = 5;
|
||||
|
||||
memset(logs, 0, sizeof(logs));
|
||||
logs[0].kind = 1;
|
||||
logs[0].log_ref = make_ref(0x70, lr0);
|
||||
logs[0].sha256 = amduat_octets(digest0, sizeof(digest0));
|
||||
receipt.logs = logs;
|
||||
receipt.logs_len = 1;
|
||||
|
||||
if (!amduat_enc_fer1_receipt_encode_v1_1(&receipt, &encoded)) {
|
||||
fprintf(stderr, "encode v1.1 failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (encoded.len < 2) {
|
||||
fprintf(stderr, "encoded v1.1 too short\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
mutated = amduat_octets(NULL, 0u);
|
||||
if (encoded.len != 0) {
|
||||
uint8_t *buffer = (uint8_t *)malloc(encoded.len);
|
||||
if (buffer == NULL) {
|
||||
fprintf(stderr, "encoded v1.1 alloc failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
memcpy(buffer, encoded.data, encoded.len);
|
||||
mutated = amduat_octets(buffer, encoded.len);
|
||||
}
|
||||
|
||||
if (mutated.data == NULL || mutated.len != encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 clone failed\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (encoded.len - offset < 2) {
|
||||
fprintf(stderr, "encoded v1.1 header too short\n");
|
||||
goto cleanup;
|
||||
}
|
||||
offset += 2;
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 function_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + len_u32;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 function_ref truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 input_manifest_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + len_u32;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 input_manifest_ref truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 environment_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + len_u32;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 environment_ref truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 evaluator_id missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + len_u32;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 evaluator_id truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 output_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + len_u32;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 output_ref truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 executor_count missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4;
|
||||
count = (size_t)len_u32;
|
||||
for (i = 0; i < count; ++i) {
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 executor_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
uint32_t ref_len = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + ref_len;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 executor_ref truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
}
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity_count missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
len_u32 = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4;
|
||||
count = (size_t)len_u32;
|
||||
for (i = 0; i < count; ++i) {
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity executor_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
uint32_t ref_len = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + ref_len;
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity output_ref missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
ref_len = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + ref_len;
|
||||
if (offset >= encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
if (encoded.data[offset] == 0x01u) {
|
||||
offset += 1;
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity sbom missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
ref_len = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + ref_len;
|
||||
} else {
|
||||
offset += 1;
|
||||
}
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity digest missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
uint32_t digest_len = (uint32_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
offset += 4 + digest_len;
|
||||
if (offset > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 parity digest truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
}
|
||||
|
||||
if (offset + 16 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 timestamps missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
offset += 16;
|
||||
|
||||
if (offset + 4 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 ext_len missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
ext_len = (size_t)((encoded.data[offset] << 24) |
|
||||
(encoded.data[offset + 1] << 16) |
|
||||
(encoded.data[offset + 2] << 8) |
|
||||
encoded.data[offset + 3]);
|
||||
ext_offset = offset + 4;
|
||||
if (ext_offset + ext_len > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 ext payload truncated\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
tlv_offset = ext_offset;
|
||||
if (tlv_offset + 6 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 tlv header missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
tlv_offset += 2;
|
||||
len_u32 = (uint32_t)((encoded.data[tlv_offset] << 24) |
|
||||
(encoded.data[tlv_offset + 1] << 16) |
|
||||
(encoded.data[tlv_offset + 2] << 8) |
|
||||
encoded.data[tlv_offset + 3]);
|
||||
tlv_offset += 4 + len_u32;
|
||||
if (tlv_offset + 2 > encoded.len) {
|
||||
fprintf(stderr, "encoded v1.1 second tlv missing\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
((uint8_t *)mutated.data)[tlv_offset] = 0x00u;
|
||||
((uint8_t *)mutated.data)[tlv_offset + 1] = 0x01u;
|
||||
|
||||
if (amduat_enc_fer1_receipt_decode_v1_1(mutated, &decoded)) {
|
||||
fprintf(stderr, "duplicate tlv accepted\n");
|
||||
amduat_enc_fer1_receipt_free(&decoded);
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
if (mutated.data != NULL) {
|
||||
free((void *)mutated.data);
|
||||
}
|
||||
free((void *)encoded.data);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_receipt_helper_v1_1(void) {
|
||||
amduat_pel_run_result_t pel_run;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_fer1_receipt_t decoded;
|
||||
amduat_fer1_log_entry_t log_entry;
|
||||
amduat_fer1_limits_t limits;
|
||||
uint8_t f0[32], i0[32], e0[32], o0[32], ex0[32], fp0[32], lr0[32];
|
||||
uint8_t run_id[] = {0x01, 0x02, 0x03, 0x04};
|
||||
uint8_t rng_seed[] = {0x09, 0x08, 0x07};
|
||||
uint8_t signature[] = {0xde, 0xad, 0xbe, 0xef};
|
||||
uint8_t digest0[] = {0xaa, 0xbb, 0xcc};
|
||||
int exit_code = 1;
|
||||
|
||||
memset(&pel_run, 0, sizeof(pel_run));
|
||||
pel_run.result_ref = make_ref(0x77, o0);
|
||||
pel_run.output_refs = &pel_run.result_ref;
|
||||
pel_run.output_refs_len = 1;
|
||||
pel_run.has_result_value = true;
|
||||
pel_run.result_value.pel1_version = 1;
|
||||
pel_run.result_value.program_ref = make_ref(0x11, f0);
|
||||
|
||||
memset(&limits, 0, sizeof(limits));
|
||||
limits.cpu_ms = 1;
|
||||
limits.wall_ms = 2;
|
||||
limits.max_rss_kib = 3;
|
||||
limits.io_reads = 4;
|
||||
limits.io_writes = 5;
|
||||
|
||||
memset(&log_entry, 0, sizeof(log_entry));
|
||||
log_entry.kind = 1;
|
||||
log_entry.log_ref = make_ref(0x70, lr0);
|
||||
log_entry.sha256 = amduat_octets(digest0, sizeof(digest0));
|
||||
|
||||
if (!amduat_fer1_receipt_from_pel_run_v1_1(
|
||||
&pel_run,
|
||||
make_ref(0x22, i0),
|
||||
make_ref(0x33, e0),
|
||||
amduat_octets("tester", 6),
|
||||
make_ref(0x50, ex0),
|
||||
false,
|
||||
amduat_reference(0, amduat_octets(NULL, 0)),
|
||||
amduat_octets(NULL, 0),
|
||||
10,
|
||||
20,
|
||||
true,
|
||||
make_ref(0x66, fp0),
|
||||
true,
|
||||
amduat_octets(run_id, sizeof(run_id)),
|
||||
true,
|
||||
limits,
|
||||
&log_entry,
|
||||
1,
|
||||
true,
|
||||
2,
|
||||
true,
|
||||
amduat_octets(rng_seed, sizeof(rng_seed)),
|
||||
true,
|
||||
amduat_octets(signature, sizeof(signature)),
|
||||
&artifact)) {
|
||||
fprintf(stderr, "v1.1 helper failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_fer1_receipt_decode_v1_1(artifact.bytes, &decoded)) {
|
||||
fprintf(stderr, "v1.1 helper decode failed\n");
|
||||
amduat_artifact_free(&artifact);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!decoded.has_run_id || !decoded.has_limits || !decoded.has_determinism ||
|
||||
!decoded.has_signature ||
|
||||
!amduat_reference_eq(decoded.output_ref, pel_run.result_ref)) {
|
||||
fprintf(stderr, "v1.1 helper decoded fields mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_fer1_receipt_free(&decoded);
|
||||
amduat_artifact_free(&artifact);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
static int test_receipt_helper_v1_1_failed_run(void) {
|
||||
amduat_pel_run_result_t pel_run;
|
||||
amduat_artifact_t artifact;
|
||||
amduat_fer1_receipt_t decoded;
|
||||
uint8_t f0[32], i0[32], e0[32], r0[32], ex0[32];
|
||||
int exit_code = 1;
|
||||
|
||||
memset(&pel_run, 0, sizeof(pel_run));
|
||||
pel_run.result_ref = make_ref(0x77, r0);
|
||||
pel_run.output_refs = NULL;
|
||||
pel_run.output_refs_len = 0;
|
||||
pel_run.has_result_value = true;
|
||||
pel_run.result_value.pel1_version = 1;
|
||||
pel_run.result_value.program_ref = make_ref(0x11, f0);
|
||||
|
||||
if (!amduat_fer1_receipt_from_pel_run_v1_1(
|
||||
&pel_run,
|
||||
make_ref(0x22, i0),
|
||||
make_ref(0x33, e0),
|
||||
amduat_octets("tester", 6),
|
||||
make_ref(0x50, ex0),
|
||||
false,
|
||||
amduat_reference(0, amduat_octets(NULL, 0)),
|
||||
amduat_octets(NULL, 0),
|
||||
10,
|
||||
20,
|
||||
false,
|
||||
amduat_reference(0, amduat_octets(NULL, 0)),
|
||||
false,
|
||||
amduat_octets(NULL, 0),
|
||||
false,
|
||||
(amduat_fer1_limits_t){0},
|
||||
NULL,
|
||||
0,
|
||||
false,
|
||||
0,
|
||||
false,
|
||||
amduat_octets(NULL, 0),
|
||||
false,
|
||||
amduat_octets(NULL, 0),
|
||||
&artifact)) {
|
||||
fprintf(stderr, "v1.1 failed run helper failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_enc_fer1_receipt_decode_v1_1(artifact.bytes, &decoded)) {
|
||||
fprintf(stderr, "v1.1 failed run helper decode failed\n");
|
||||
amduat_artifact_free(&artifact);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_reference_eq(decoded.output_ref, pel_run.result_ref)) {
|
||||
fprintf(stderr, "v1.1 failed run output_ref mismatch\n");
|
||||
goto cleanup_decoded;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup_decoded:
|
||||
amduat_enc_fer1_receipt_free(&decoded);
|
||||
amduat_artifact_free(&artifact);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_receipt_round_trip() != 0) {
|
||||
return 1;
|
||||
|
|
@ -271,5 +906,20 @@ int main(void) {
|
|||
if (test_receipt_helper() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_receipt_helper_failed_run() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_receipt_round_trip_v1_1() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_receipt_v1_1_reject_duplicate_tag() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_receipt_helper_v1_1() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_receipt_helper_v1_1_failed_run() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
81
tests/fed/test_fed_ingest.c
Normal file
81
tests/fed/test_fed_ingest.c
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
#include "amduat/fed/ingest.h"
|
||||
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
|
||||
static amduat_reference_t make_ref(amduat_hash_id_t hash_id,
|
||||
const uint8_t *bytes,
|
||||
size_t len) {
|
||||
return amduat_reference(hash_id, amduat_octets(bytes, len));
|
||||
}
|
||||
|
||||
static amduat_fed_record_t make_record(uint32_t domain_id,
|
||||
uint64_t logseq,
|
||||
amduat_fed_record_type_t type,
|
||||
amduat_reference_t ref) {
|
||||
amduat_fed_record_t record;
|
||||
|
||||
memset(&record, 0, sizeof(record));
|
||||
record.meta.domain_id = domain_id;
|
||||
record.meta.visibility = 1;
|
||||
record.meta.has_source = 0;
|
||||
record.id.type = type;
|
||||
record.id.ref = ref;
|
||||
record.logseq = logseq;
|
||||
record.snapshot_id = 1;
|
||||
record.log_prefix = 10;
|
||||
return record;
|
||||
}
|
||||
|
||||
static int test_invalid_record(void) {
|
||||
uint8_t key[] = {0x01};
|
||||
amduat_fed_record_t record;
|
||||
size_t error_index = 0;
|
||||
|
||||
record = make_record(1, 1, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, key, sizeof(key)));
|
||||
record.meta.visibility = 2;
|
||||
|
||||
if (amduat_fed_ingest_validate(&record, 1, &error_index, NULL) !=
|
||||
AMDUAT_FED_INGEST_ERR_INVALID) {
|
||||
fprintf(stderr, "expected invalid error\n");
|
||||
return 1;
|
||||
}
|
||||
if (error_index != 0) {
|
||||
fprintf(stderr, "expected error index 0\n");
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_conflicting_duplicate(void) {
|
||||
uint8_t key[] = {0x02};
|
||||
amduat_reference_t ref = make_ref(1, key, sizeof(key));
|
||||
amduat_fed_record_t records[2];
|
||||
size_t error_index = 0;
|
||||
size_t conflict_index = 0;
|
||||
|
||||
records[0] = make_record(1, 1, AMDUAT_FED_REC_ARTIFACT, ref);
|
||||
records[1] = make_record(1, 2, AMDUAT_FED_REC_ARTIFACT, ref);
|
||||
|
||||
if (amduat_fed_ingest_validate(records, 2, &error_index, &conflict_index) !=
|
||||
AMDUAT_FED_INGEST_ERR_CONFLICT) {
|
||||
fprintf(stderr, "expected conflict error\n");
|
||||
return 1;
|
||||
}
|
||||
if (error_index != 1 || conflict_index != 0) {
|
||||
fprintf(stderr, "unexpected conflict indices\n");
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_invalid_record() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_conflicting_duplicate() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
93
tests/fed/test_fed_registry.c
Normal file
93
tests/fed/test_fed_registry.c
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
#include "amduat/fed/registry.h"
|
||||
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
|
||||
static bool octets_equal(amduat_octets_t a, amduat_octets_t b) {
|
||||
if (a.len != b.len) {
|
||||
return false;
|
||||
}
|
||||
if (a.len == 0) {
|
||||
return true;
|
||||
}
|
||||
return memcmp(a.data, b.data, a.len) == 0;
|
||||
}
|
||||
|
||||
static int test_round_trip(void) {
|
||||
uint8_t policy_a[] = {0xde, 0xad, 0xbe, 0xef};
|
||||
amduat_fed_domain_state_t states[2];
|
||||
amduat_fed_registry_value_t value;
|
||||
amduat_fed_registry_value_t decoded;
|
||||
amduat_octets_t encoded;
|
||||
int exit_code = 1;
|
||||
|
||||
memset(states, 0, sizeof(states));
|
||||
amduat_fed_registry_value_init(&value, states, 2);
|
||||
|
||||
states[0].domain_id = 42;
|
||||
states[0].snapshot_id = 10;
|
||||
states[0].log_prefix = 8;
|
||||
states[0].last_logseq = 7;
|
||||
states[0].admitted = 1;
|
||||
states[0].policy_ok = 1;
|
||||
states[0].policy_hash_id = 1;
|
||||
states[0].policy_hash = amduat_octets(policy_a, sizeof(policy_a));
|
||||
|
||||
states[1].domain_id = 7;
|
||||
states[1].snapshot_id = 3;
|
||||
states[1].log_prefix = 2;
|
||||
states[1].last_logseq = 2;
|
||||
states[1].admitted = 0;
|
||||
states[1].policy_ok = 0;
|
||||
states[1].policy_hash_id = 0;
|
||||
states[1].policy_hash = amduat_octets(NULL, 0);
|
||||
|
||||
if (!amduat_fed_registry_value_insert(&value, states[0]) ||
|
||||
!amduat_fed_registry_value_insert(&value, states[1])) {
|
||||
fprintf(stderr, "insert failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (!amduat_fed_registry_encode(&value, &encoded)) {
|
||||
fprintf(stderr, "encode failed\n");
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
memset(&decoded, 0, sizeof(decoded));
|
||||
if (!amduat_fed_registry_decode(encoded, &decoded)) {
|
||||
fprintf(stderr, "decode failed\n");
|
||||
amduat_octets_free(&encoded);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
if (decoded.len != 2) {
|
||||
fprintf(stderr, "decoded length mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (decoded.states[0].domain_id != 7 ||
|
||||
decoded.states[1].domain_id != 42) {
|
||||
fprintf(stderr, "decoded order mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!octets_equal(decoded.states[1].policy_hash,
|
||||
amduat_octets(policy_a, sizeof(policy_a)))) {
|
||||
fprintf(stderr, "decoded policy hash mismatch\n");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
exit_code = 0;
|
||||
|
||||
cleanup:
|
||||
amduat_octets_free(&encoded);
|
||||
amduat_fed_registry_value_free(&decoded);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_round_trip() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
188
tests/fed/test_fed_replay.c
Normal file
188
tests/fed/test_fed_replay.c
Normal file
|
|
@ -0,0 +1,188 @@
|
|||
#include "amduat/fed/replay.h"
|
||||
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
|
||||
static amduat_reference_t make_ref(amduat_hash_id_t hash_id,
|
||||
const uint8_t *bytes,
|
||||
size_t len) {
|
||||
return amduat_reference(hash_id, amduat_octets(bytes, len));
|
||||
}
|
||||
|
||||
static amduat_fed_record_t make_record(uint32_t domain_id,
|
||||
uint64_t logseq,
|
||||
uint64_t snapshot_id,
|
||||
uint64_t log_prefix,
|
||||
amduat_fed_record_type_t type,
|
||||
amduat_reference_t ref) {
|
||||
amduat_fed_record_t record;
|
||||
|
||||
memset(&record, 0, sizeof(record));
|
||||
record.meta.domain_id = domain_id;
|
||||
record.meta.visibility = 1;
|
||||
record.meta.has_source = 0;
|
||||
record.id.type = type;
|
||||
record.id.ref = ref;
|
||||
record.logseq = logseq;
|
||||
record.snapshot_id = snapshot_id;
|
||||
record.log_prefix = log_prefix;
|
||||
return record;
|
||||
}
|
||||
|
||||
static int test_ordering(void) {
|
||||
uint8_t d0[] = {0x00};
|
||||
uint8_t d1[] = {0x01};
|
||||
uint8_t d2[] = {0x02};
|
||||
amduat_fed_record_t records[3];
|
||||
amduat_fed_replay_view_t view;
|
||||
|
||||
records[0] = make_record(1, 5, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d1, sizeof(d1)));
|
||||
records[1] = make_record(1, 5, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d0, sizeof(d0)));
|
||||
records[2] = make_record(1, 4, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d2, sizeof(d2)));
|
||||
|
||||
if (!amduat_fed_replay_build(records, 3, 1, 1, 10, &view)) {
|
||||
fprintf(stderr, "replay failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (view.len != 3) {
|
||||
fprintf(stderr, "ordering length mismatch\n");
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
if (view.records[0].logseq != 4 ||
|
||||
view.records[1].id.ref.digest.data[0] != 0x00 ||
|
||||
view.records[2].id.ref.digest.data[0] != 0x01) {
|
||||
fprintf(stderr, "ordering mismatch\n");
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_tombstone_scope(void) {
|
||||
uint8_t key[] = {0xaa};
|
||||
amduat_reference_t ref = make_ref(1, key, sizeof(key));
|
||||
amduat_fed_record_t records[3];
|
||||
amduat_fed_replay_view_t view;
|
||||
|
||||
records[0] = make_record(1, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT, ref);
|
||||
records[1] = make_record(1, 2, 1, 10, AMDUAT_FED_REC_TOMBSTONE, ref);
|
||||
records[2] = make_record(2, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT, ref);
|
||||
|
||||
if (!amduat_fed_replay_build(records, 3, 1, 1, 10, &view)) {
|
||||
fprintf(stderr, "replay domain 1 failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (view.len != 0) {
|
||||
fprintf(stderr, "tombstone scope mismatch (domain 1)\n");
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
amduat_fed_replay_view_free(&view);
|
||||
|
||||
if (!amduat_fed_replay_build(records, 3, 2, 1, 10, &view)) {
|
||||
fprintf(stderr, "replay domain 2 failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (view.len != 1) {
|
||||
fprintf(stderr, "tombstone scope mismatch (domain 2)\n");
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_bounds(void) {
|
||||
uint8_t key[] = {0xbb};
|
||||
amduat_reference_t ref = make_ref(1, key, sizeof(key));
|
||||
amduat_fed_record_t records[2];
|
||||
amduat_fed_replay_view_t view;
|
||||
|
||||
records[0] = make_record(3, 1, 2, 10, AMDUAT_FED_REC_ARTIFACT, ref);
|
||||
records[1] = make_record(3, 5, 1, 4, AMDUAT_FED_REC_ARTIFACT, ref);
|
||||
|
||||
if (!amduat_fed_replay_build(records, 2, 3, 1, 4, &view)) {
|
||||
fprintf(stderr, "replay bounds failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (view.len != 0) {
|
||||
fprintf(stderr, "bounds mismatch\n");
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
amduat_fed_replay_view_free(&view);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_multi_domain_ordering(void) {
|
||||
uint8_t d0[] = {0x00};
|
||||
uint8_t d1[] = {0x01};
|
||||
uint8_t d2[] = {0x02};
|
||||
amduat_fed_record_t records[4];
|
||||
amduat_fed_replay_view_t view_a;
|
||||
amduat_fed_replay_view_t view_b;
|
||||
|
||||
records[0] = make_record(1, 2, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d2, sizeof(d2)));
|
||||
records[1] = make_record(1, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d1, sizeof(d1)));
|
||||
records[2] = make_record(2, 1, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d0, sizeof(d0)));
|
||||
records[3] = make_record(2, 2, 1, 10, AMDUAT_FED_REC_ARTIFACT,
|
||||
make_ref(1, d2, sizeof(d2)));
|
||||
|
||||
if (!amduat_fed_replay_build(records, 4, 1, 1, 10, &view_a)) {
|
||||
fprintf(stderr, "replay domain 1 failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (!amduat_fed_replay_build(records, 4, 2, 1, 10, &view_b)) {
|
||||
fprintf(stderr, "replay domain 2 failed\n");
|
||||
amduat_fed_replay_view_free(&view_a);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (view_a.len != 2 || view_b.len != 2) {
|
||||
fprintf(stderr, "multi-domain lengths mismatch\n");
|
||||
amduat_fed_replay_view_free(&view_a);
|
||||
amduat_fed_replay_view_free(&view_b);
|
||||
return 1;
|
||||
}
|
||||
if (view_a.records[0].logseq != 1 || view_a.records[1].logseq != 2) {
|
||||
fprintf(stderr, "domain 1 ordering mismatch\n");
|
||||
amduat_fed_replay_view_free(&view_a);
|
||||
amduat_fed_replay_view_free(&view_b);
|
||||
return 1;
|
||||
}
|
||||
if (view_b.records[0].logseq != 1 || view_b.records[1].logseq != 2) {
|
||||
fprintf(stderr, "domain 2 ordering mismatch\n");
|
||||
amduat_fed_replay_view_free(&view_a);
|
||||
amduat_fed_replay_view_free(&view_b);
|
||||
return 1;
|
||||
}
|
||||
|
||||
amduat_fed_replay_view_free(&view_a);
|
||||
amduat_fed_replay_view_free(&view_b);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_ordering() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_tombstone_scope() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_bounds() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_multi_domain_ordering() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
241
tests/fed/test_fed_view.c
Normal file
241
tests/fed/test_fed_view.c
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
#include "amduat/fed/view.h"
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
typedef struct {
|
||||
amduat_reference_t ref;
|
||||
amduat_artifact_t artifact;
|
||||
} stub_store_entry_t;
|
||||
|
||||
typedef struct {
|
||||
amduat_asl_store_config_t config;
|
||||
stub_store_entry_t entries[4];
|
||||
size_t entries_len;
|
||||
} stub_store_t;
|
||||
|
||||
static void stub_store_init(stub_store_t *store) {
|
||||
memset(store, 0, sizeof(*store));
|
||||
store->config.hash_id = 1;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t stub_store_get(void *ctx,
|
||||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out) {
|
||||
stub_store_t *store = (stub_store_t *)ctx;
|
||||
size_t i;
|
||||
|
||||
for (i = 0; i < store->entries_len; ++i) {
|
||||
if (amduat_reference_eq(store->entries[i].ref, ref)) {
|
||||
amduat_artifact_t src = store->entries[i].artifact;
|
||||
uint8_t *payload = NULL;
|
||||
if (src.bytes.len != 0u) {
|
||||
payload = (uint8_t *)malloc(src.bytes.len);
|
||||
if (payload == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_IO;
|
||||
}
|
||||
memcpy(payload, src.bytes.data, src.bytes.len);
|
||||
}
|
||||
out->bytes = amduat_octets(payload, src.bytes.len);
|
||||
out->has_type_tag = src.has_type_tag;
|
||||
out->type_tag = src.type_tag;
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
}
|
||||
|
||||
return AMDUAT_ASL_STORE_ERR_NOT_FOUND;
|
||||
}
|
||||
|
||||
static amduat_reference_t make_ref(amduat_hash_id_t hash_id,
|
||||
const uint8_t *bytes,
|
||||
size_t len) {
|
||||
return amduat_reference(hash_id, amduat_octets(bytes, len));
|
||||
}
|
||||
|
||||
static amduat_fed_record_t make_record(uint32_t domain_id,
|
||||
uint8_t visibility,
|
||||
uint64_t logseq,
|
||||
amduat_reference_t ref) {
|
||||
amduat_fed_record_t record;
|
||||
|
||||
memset(&record, 0, sizeof(record));
|
||||
record.meta.domain_id = domain_id;
|
||||
record.meta.visibility = visibility;
|
||||
record.meta.has_source = 0;
|
||||
record.id.type = AMDUAT_FED_REC_ARTIFACT;
|
||||
record.id.ref = ref;
|
||||
record.logseq = logseq;
|
||||
record.snapshot_id = 1;
|
||||
record.log_prefix = 10;
|
||||
return record;
|
||||
}
|
||||
|
||||
static int test_view_and_resolve(void) {
|
||||
uint8_t a_bytes[] = {0x01};
|
||||
uint8_t b_bytes[] = {0x02};
|
||||
uint8_t c_bytes[] = {0x03};
|
||||
uint8_t d_bytes[] = {0x04};
|
||||
amduat_reference_t ref_a = make_ref(1, a_bytes, sizeof(a_bytes));
|
||||
amduat_reference_t ref_b = make_ref(1, b_bytes, sizeof(b_bytes));
|
||||
amduat_reference_t ref_c = make_ref(1, c_bytes, sizeof(c_bytes));
|
||||
amduat_reference_t ref_d = make_ref(1, d_bytes, sizeof(d_bytes));
|
||||
amduat_fed_record_t records[4];
|
||||
amduat_fed_view_bounds_t bounds[2];
|
||||
amduat_fed_view_t view;
|
||||
amduat_fed_policy_deny_t denies[1];
|
||||
stub_store_t stub;
|
||||
amduat_asl_store_ops_t ops;
|
||||
amduat_asl_store_t store;
|
||||
amduat_artifact_t artifact;
|
||||
|
||||
records[0] = make_record(1, 0, 1, ref_a);
|
||||
records[1] = make_record(1, 1, 2, ref_b);
|
||||
records[2] = make_record(2, 1, 1, ref_c);
|
||||
records[3] = make_record(2, 0, 1, ref_d);
|
||||
|
||||
bounds[0].domain_id = 1;
|
||||
bounds[0].snapshot_id = 1;
|
||||
bounds[0].log_prefix = 10;
|
||||
bounds[1].domain_id = 2;
|
||||
bounds[1].snapshot_id = 1;
|
||||
bounds[1].log_prefix = 10;
|
||||
|
||||
denies[0].id.type = AMDUAT_FED_REC_ARTIFACT;
|
||||
denies[0].id.ref = ref_d;
|
||||
denies[0].reason_code = 0;
|
||||
|
||||
if (amduat_fed_view_build(records, 4, 1, bounds, 2, denies, 1, &view) !=
|
||||
AMDUAT_FED_VIEW_OK) {
|
||||
fprintf(stderr, "view build failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (view.len != 3) {
|
||||
fprintf(stderr, "view size mismatch\n");
|
||||
amduat_fed_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
|
||||
stub_store_init(&stub);
|
||||
stub.entries[0].ref = ref_a;
|
||||
stub.entries[0].artifact = amduat_artifact(amduat_octets(a_bytes,
|
||||
sizeof(a_bytes)));
|
||||
stub.entries_len = 1;
|
||||
amduat_asl_store_ops_init(&ops);
|
||||
ops.get = stub_store_get;
|
||||
amduat_asl_store_init(&store, stub.config, ops, &stub);
|
||||
|
||||
if (amduat_fed_resolve(&view, &store, ref_a, &artifact) !=
|
||||
AMDUAT_FED_RESOLVE_OK) {
|
||||
fprintf(stderr, "resolve local failed\n");
|
||||
amduat_fed_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
amduat_artifact_free(&artifact);
|
||||
|
||||
if (amduat_fed_resolve(&view, &store, ref_c, &artifact) !=
|
||||
AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES) {
|
||||
fprintf(stderr, "resolve remote mismatch\n");
|
||||
amduat_fed_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (amduat_fed_resolve(&view, &store, ref_d, &artifact) !=
|
||||
AMDUAT_FED_RESOLVE_POLICY_DENIED) {
|
||||
fprintf(stderr, "resolve policy denied mismatch\n");
|
||||
amduat_fed_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
|
||||
amduat_fed_view_free(&view);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_view_conflict(void) {
|
||||
uint8_t key[] = {0x09};
|
||||
amduat_reference_t ref = make_ref(1, key, sizeof(key));
|
||||
amduat_fed_record_t records[2];
|
||||
amduat_fed_view_bounds_t bounds[2];
|
||||
amduat_fed_view_t view;
|
||||
|
||||
records[0] = make_record(1, 1, 1, ref);
|
||||
records[1] = make_record(2, 1, 1, ref);
|
||||
|
||||
bounds[0].domain_id = 1;
|
||||
bounds[0].snapshot_id = 1;
|
||||
bounds[0].log_prefix = 10;
|
||||
bounds[1].domain_id = 2;
|
||||
bounds[1].snapshot_id = 1;
|
||||
bounds[1].log_prefix = 10;
|
||||
|
||||
if (amduat_fed_view_build(records, 2, 1, bounds, 2, NULL, 0, &view) !=
|
||||
AMDUAT_FED_VIEW_ERR_CONFLICT) {
|
||||
fprintf(stderr, "expected conflict\n");
|
||||
amduat_fed_view_free(&view);
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_view_rebuild_metadata(void) {
|
||||
uint8_t key[] = {0x08};
|
||||
amduat_reference_t ref = make_ref(1, key, sizeof(key));
|
||||
amduat_fed_record_t records[1];
|
||||
amduat_fed_view_bounds_t bounds[1];
|
||||
amduat_fed_view_t view_a;
|
||||
amduat_fed_view_t view_b;
|
||||
|
||||
records[0] = make_record(1, 1, 1, ref);
|
||||
records[0].meta.has_source = 1;
|
||||
records[0].meta.source_domain = 5;
|
||||
|
||||
bounds[0].domain_id = 1;
|
||||
bounds[0].snapshot_id = 1;
|
||||
bounds[0].log_prefix = 10;
|
||||
|
||||
if (amduat_fed_view_build(records, 1, 1, bounds, 1, NULL, 0, &view_a) !=
|
||||
AMDUAT_FED_VIEW_OK) {
|
||||
fprintf(stderr, "view rebuild build A failed\n");
|
||||
return 1;
|
||||
}
|
||||
if (amduat_fed_view_build(records, 1, 1, bounds, 1, NULL, 0, &view_b) !=
|
||||
AMDUAT_FED_VIEW_OK) {
|
||||
fprintf(stderr, "view rebuild build B failed\n");
|
||||
amduat_fed_view_free(&view_a);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (view_a.len != 1 || view_b.len != 1) {
|
||||
fprintf(stderr, "view rebuild length mismatch\n");
|
||||
amduat_fed_view_free(&view_a);
|
||||
amduat_fed_view_free(&view_b);
|
||||
return 1;
|
||||
}
|
||||
if (view_a.records[0].meta.has_source !=
|
||||
view_b.records[0].meta.has_source ||
|
||||
view_a.records[0].meta.source_domain !=
|
||||
view_b.records[0].meta.source_domain ||
|
||||
view_a.records[0].meta.visibility != view_b.records[0].meta.visibility) {
|
||||
fprintf(stderr, "view rebuild metadata mismatch\n");
|
||||
amduat_fed_view_free(&view_a);
|
||||
amduat_fed_view_free(&view_b);
|
||||
return 1;
|
||||
}
|
||||
|
||||
amduat_fed_view_free(&view_a);
|
||||
amduat_fed_view_free(&view_b);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
if (test_view_and_resolve() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_view_conflict() != 0) {
|
||||
return 1;
|
||||
}
|
||||
if (test_view_rebuild_metadata() != 0) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
348
tests/pel/test_pel_program_build_concat.c
Normal file
348
tests/pel/test_pel_program_build_concat.c
Normal file
|
|
@ -0,0 +1,348 @@
|
|||
#include "amduat/asl/ref_derive.h"
|
||||
#include "amduat/asl/store.h"
|
||||
#include "amduat/enc/asl1_core.h"
|
||||
#include "amduat/enc/pel1_result.h"
|
||||
#include "amduat/enc/pel_program_dag.h"
|
||||
#include "amduat/hash/asl1.h"
|
||||
#include "amduat/pel/program_dag.h"
|
||||
#include "amduat/pel/program_dag_desc.h"
|
||||
#include "amduat/pel/run.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
typedef struct {
|
||||
amduat_reference_t ref;
|
||||
amduat_artifact_t artifact;
|
||||
} stub_store_entry_t;
|
||||
|
||||
typedef struct {
|
||||
stub_store_entry_t *entries;
|
||||
size_t len;
|
||||
size_t cap;
|
||||
amduat_asl_store_config_t config;
|
||||
} stub_store_t;
|
||||
|
||||
static void stub_store_init(stub_store_t *store) {
|
||||
if (store == NULL) {
|
||||
return;
|
||||
}
|
||||
store->entries = NULL;
|
||||
store->len = 0;
|
||||
store->cap = 0;
|
||||
store->config.encoding_profile_id = AMDUAT_ENC_ASL1_CORE_V1;
|
||||
store->config.hash_id = AMDUAT_HASH_ASL1_ID_SHA256;
|
||||
}
|
||||
|
||||
static void stub_store_free(stub_store_t *store) {
|
||||
size_t i;
|
||||
|
||||
if (store == NULL) {
|
||||
return;
|
||||
}
|
||||
for (i = 0; i < store->len; ++i) {
|
||||
stub_store_entry_t *entry = &store->entries[i];
|
||||
free((void *)entry->ref.digest.data);
|
||||
entry->ref.digest.data = NULL;
|
||||
entry->ref.digest.len = 0;
|
||||
free((void *)entry->artifact.bytes.data);
|
||||
entry->artifact.bytes.data = NULL;
|
||||
entry->artifact.bytes.len = 0;
|
||||
}
|
||||
free(store->entries);
|
||||
store->entries = NULL;
|
||||
store->len = 0;
|
||||
store->cap = 0;
|
||||
}
|
||||
|
||||
static bool stub_store_add_entry(stub_store_t *store,
|
||||
amduat_reference_t stored_ref,
|
||||
amduat_artifact_t stored_artifact) {
|
||||
stub_store_entry_t *entries;
|
||||
size_t new_cap;
|
||||
|
||||
if (store->len == store->cap) {
|
||||
new_cap = store->cap == 0 ? 8 : store->cap * 2;
|
||||
entries = (stub_store_entry_t *)realloc(store->entries,
|
||||
new_cap * sizeof(*entries));
|
||||
if (entries == NULL) {
|
||||
return false;
|
||||
}
|
||||
store->entries = entries;
|
||||
store->cap = new_cap;
|
||||
}
|
||||
|
||||
store->entries[store->len++] =
|
||||
(stub_store_entry_t){stored_ref, stored_artifact};
|
||||
return true;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t stub_store_put(
|
||||
void *ctx,
|
||||
amduat_artifact_t artifact,
|
||||
amduat_reference_t *out_ref) {
|
||||
stub_store_t *store;
|
||||
amduat_reference_t derived_ref;
|
||||
amduat_reference_t stored_ref;
|
||||
amduat_artifact_t stored_artifact;
|
||||
size_t i;
|
||||
|
||||
if (ctx == NULL || out_ref == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
store = (stub_store_t *)ctx;
|
||||
out_ref->hash_id = 0;
|
||||
out_ref->digest = amduat_octets(NULL, 0);
|
||||
|
||||
derived_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
if (!amduat_asl_ref_derive(artifact,
|
||||
store->config.encoding_profile_id,
|
||||
store->config.hash_id,
|
||||
&derived_ref,
|
||||
NULL)) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
*out_ref = derived_ref;
|
||||
|
||||
for (i = 0; i < store->len; ++i) {
|
||||
if (amduat_reference_eq(store->entries[i].ref, *out_ref)) {
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
}
|
||||
|
||||
if (!amduat_reference_clone(*out_ref, &stored_ref)) {
|
||||
amduat_reference_free(out_ref);
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
stored_artifact.bytes = amduat_octets(NULL, 0u);
|
||||
if (artifact.bytes.len != 0) {
|
||||
uint8_t *payload = (uint8_t *)malloc(artifact.bytes.len);
|
||||
if (payload == NULL) {
|
||||
amduat_reference_free(out_ref);
|
||||
amduat_reference_free(&stored_ref);
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
memcpy(payload, artifact.bytes.data, artifact.bytes.len);
|
||||
stored_artifact.bytes = amduat_octets(payload, artifact.bytes.len);
|
||||
}
|
||||
stored_artifact.has_type_tag = artifact.has_type_tag;
|
||||
stored_artifact.type_tag = artifact.type_tag;
|
||||
|
||||
if (!stub_store_add_entry(store, stored_ref, stored_artifact)) {
|
||||
amduat_reference_free(out_ref);
|
||||
amduat_reference_free(&stored_ref);
|
||||
free((void *)stored_artifact.bytes.data);
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
|
||||
static amduat_asl_store_error_t stub_store_get(
|
||||
void *ctx,
|
||||
amduat_reference_t ref,
|
||||
amduat_artifact_t *out_artifact) {
|
||||
stub_store_t *store;
|
||||
size_t i;
|
||||
|
||||
if (ctx == NULL || out_artifact == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
store = (stub_store_t *)ctx;
|
||||
out_artifact->bytes = amduat_octets(NULL, 0u);
|
||||
out_artifact->has_type_tag = false;
|
||||
out_artifact->type_tag = amduat_type_tag(0);
|
||||
|
||||
for (i = 0; i < store->len; ++i) {
|
||||
stub_store_entry_t *entry = &store->entries[i];
|
||||
if (amduat_reference_eq(entry->ref, ref)) {
|
||||
uint8_t *payload = NULL;
|
||||
if (entry->artifact.bytes.len != 0u) {
|
||||
payload = (uint8_t *)malloc(entry->artifact.bytes.len);
|
||||
if (payload == NULL) {
|
||||
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
|
||||
}
|
||||
memcpy(payload, entry->artifact.bytes.data,
|
||||
entry->artifact.bytes.len);
|
||||
}
|
||||
out_artifact->bytes = amduat_octets(payload,
|
||||
entry->artifact.bytes.len);
|
||||
out_artifact->has_type_tag = entry->artifact.has_type_tag;
|
||||
out_artifact->type_tag = entry->artifact.type_tag;
|
||||
return AMDUAT_ASL_STORE_OK;
|
||||
}
|
||||
}
|
||||
return AMDUAT_ASL_STORE_ERR_NOT_FOUND;
|
||||
}
|
||||
|
||||
static int test_program_concat_two_inputs(void) {
|
||||
stub_store_t stub;
|
||||
amduat_asl_store_ops_t ops;
|
||||
amduat_asl_store_t store;
|
||||
amduat_pel_program_t program;
|
||||
amduat_pel_node_t *nodes;
|
||||
amduat_pel_dag_input_t *inputs;
|
||||
amduat_pel_root_ref_t *roots;
|
||||
amduat_octets_t encoded;
|
||||
amduat_type_tag_t program_tag;
|
||||
amduat_asl_encoding_profile_id_t program_profile;
|
||||
amduat_artifact_t program_artifact;
|
||||
amduat_reference_t program_ref;
|
||||
amduat_artifact_t input_artifacts[2];
|
||||
amduat_reference_t input_refs[2];
|
||||
amduat_pel_run_result_t run_result;
|
||||
amduat_artifact_t output_artifact;
|
||||
int exit_code = 0;
|
||||
|
||||
stub_store_init(&stub);
|
||||
amduat_asl_store_ops_init(&ops);
|
||||
ops.put = stub_store_put;
|
||||
ops.get = stub_store_get;
|
||||
amduat_asl_store_init(&store, stub.config, ops, &stub);
|
||||
|
||||
nodes = (amduat_pel_node_t *)calloc(1u, sizeof(*nodes));
|
||||
inputs = (amduat_pel_dag_input_t *)calloc(2u, sizeof(*inputs));
|
||||
roots = (amduat_pel_root_ref_t *)calloc(1u, sizeof(*roots));
|
||||
if (nodes == NULL || inputs == NULL || roots == NULL) {
|
||||
fprintf(stderr, "out of memory\n");
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
inputs[0].kind = AMDUAT_PEL_DAG_INPUT_EXTERNAL;
|
||||
inputs[0].value.external.input_index = 0u;
|
||||
inputs[1].kind = AMDUAT_PEL_DAG_INPUT_EXTERNAL;
|
||||
inputs[1].value.external.input_index = 1u;
|
||||
|
||||
nodes[0].id = 1u;
|
||||
nodes[0].op.name =
|
||||
amduat_octets("pel.bytes.concat", strlen("pel.bytes.concat"));
|
||||
nodes[0].op.version = 1u;
|
||||
nodes[0].inputs = inputs;
|
||||
nodes[0].inputs_len = 2u;
|
||||
nodes[0].params = amduat_octets(NULL, 0u);
|
||||
|
||||
roots[0].node_id = 1u;
|
||||
roots[0].output_index = 0u;
|
||||
|
||||
program.nodes = nodes;
|
||||
program.nodes_len = 1u;
|
||||
program.roots = roots;
|
||||
program.roots_len = 1u;
|
||||
|
||||
if (!amduat_pel_program_dag_validate(&program)) {
|
||||
fprintf(stderr, "program validation failed\n");
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
encoded = amduat_octets(NULL, 0u);
|
||||
if (!amduat_enc_pel_program_dag_encode_v1(&program, &encoded)) {
|
||||
fprintf(stderr, "program encode failed\n");
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!amduat_pel_program_dag_desc_get_program_binding(&program_tag,
|
||||
&program_profile)) {
|
||||
fprintf(stderr, "program binding failed\n");
|
||||
free((void *)encoded.data);
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
(void)program_profile;
|
||||
|
||||
program_artifact = amduat_artifact_with_type(encoded, program_tag);
|
||||
program_ref = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
if (amduat_asl_store_put(&store, program_artifact,
|
||||
&program_ref) != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "store put program failed\n");
|
||||
free((void *)encoded.data);
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
free((void *)encoded.data);
|
||||
|
||||
input_artifacts[0] =
|
||||
amduat_artifact(amduat_octets("foo", 3u));
|
||||
input_artifacts[1] =
|
||||
amduat_artifact(amduat_octets("bar", 3u));
|
||||
|
||||
input_refs[0] = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
input_refs[1] = amduat_reference(0u, amduat_octets(NULL, 0u));
|
||||
if (amduat_asl_store_put(&store, input_artifacts[0],
|
||||
&input_refs[0]) != AMDUAT_ASL_STORE_OK ||
|
||||
amduat_asl_store_put(&store, input_artifacts[1],
|
||||
&input_refs[1]) != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "store put inputs failed\n");
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
memset(&run_result, 0, sizeof(run_result));
|
||||
if (!amduat_pel_surf_run_with_result(
|
||||
&store,
|
||||
amduat_pel_program_dag_scheme_ref(),
|
||||
program_ref,
|
||||
input_refs,
|
||||
2u,
|
||||
false,
|
||||
amduat_reference(0u, amduat_octets(NULL, 0u)),
|
||||
&run_result)) {
|
||||
fprintf(stderr, "surf run failed\n");
|
||||
exit_code = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!run_result.has_result_value ||
|
||||
run_result.result_value.core_result.status != AMDUAT_PEL_EXEC_STATUS_OK) {
|
||||
fprintf(stderr, "execution status not OK\n");
|
||||
exit_code = 1;
|
||||
goto cleanup_run;
|
||||
}
|
||||
if (run_result.output_refs_len != 1u) {
|
||||
fprintf(stderr, "unexpected output_refs_len=%zu\n",
|
||||
run_result.output_refs_len);
|
||||
exit_code = 1;
|
||||
goto cleanup_run;
|
||||
}
|
||||
|
||||
memset(&output_artifact, 0, sizeof(output_artifact));
|
||||
if (amduat_asl_store_get(&store,
|
||||
run_result.output_refs[0],
|
||||
&output_artifact) != AMDUAT_ASL_STORE_OK) {
|
||||
fprintf(stderr, "store get output failed\n");
|
||||
exit_code = 1;
|
||||
goto cleanup_run;
|
||||
}
|
||||
if (output_artifact.bytes.len != 6u ||
|
||||
memcmp(output_artifact.bytes.data, "foobar", 6u) != 0) {
|
||||
fprintf(stderr, "unexpected output bytes\n");
|
||||
exit_code = 1;
|
||||
}
|
||||
free((void *)output_artifact.bytes.data);
|
||||
|
||||
cleanup_run:
|
||||
if (run_result.has_result_value) {
|
||||
amduat_enc_pel1_result_free(&run_result.result_value);
|
||||
}
|
||||
if (run_result.output_refs != NULL) {
|
||||
amduat_pel_surf_free_refs(run_result.output_refs,
|
||||
run_result.output_refs_len);
|
||||
}
|
||||
amduat_pel_surf_free_ref(&run_result.result_ref);
|
||||
|
||||
cleanup:
|
||||
free(nodes);
|
||||
free(inputs);
|
||||
free(roots);
|
||||
stub_store_free(&stub);
|
||||
return exit_code;
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
return test_program_concat_two_inputs();
|
||||
}
|
||||
248
tier1/asl-core-index-1.md
Normal file
248
tier1/asl-core-index-1.md
Normal file
|
|
@ -0,0 +1,248 @@
|
|||
# ASL/1-CORE-INDEX — Semantic Index Model
|
||||
|
||||
Status: Draft
|
||||
Owner: Niklas Rydberg
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-11-16
|
||||
Linked Phase Pack: N/A
|
||||
Tags: [deterministic, index, semantics]
|
||||
|
||||
<!-- Source: /amduat-api/tier1/asl-core-index.md | Canonical: /amduat/tier1/asl-core-index-1.md -->
|
||||
|
||||
**Document ID:** `ASL/1-CORE-INDEX`
|
||||
**Layer:** L0.5 — Semantic mapping over ASL/1-CORE values (no storage / encoding / lifecycle)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/1-CORE`
|
||||
* `ASL/1-STORE`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/STORE-INDEX/1` — store lifecycle and replay contracts
|
||||
* `ENC/ASL-CORE-INDEX/1` — bytes-on-disk encoding profile
|
||||
* `ASL/INDEX-ACCEL/1` — acceleration semantics (routing, filters, sharding)
|
||||
* `ASL/LOG/1` — append-only semantic log (segment visibility)
|
||||
* `TGK/1` — TGK edge visibility and traversal alignment
|
||||
* `ASL/SYSTEM/1` — unified system view (PEL/TGK/federation alignment)
|
||||
|
||||
© 2025 Niklas Rydberg.
|
||||
|
||||
## License
|
||||
|
||||
Except where otherwise noted, this document (text and diagrams) is licensed under
|
||||
the Creative Commons Attribution 4.0 International License (CC BY 4.0).
|
||||
|
||||
The identifier registries and mapping tables (e.g. TypeTag IDs, HashId
|
||||
assignments, EdgeTypeId tables) are additionally made available under CC0 1.0
|
||||
Universal (CC0) to enable unrestricted reuse in implementations and derivative
|
||||
specifications.
|
||||
|
||||
Code examples in this document are provided under the Apache License 2.0 unless
|
||||
explicitly stated otherwise. Test vectors, where present, are dedicated to the
|
||||
public domain under CC0 1.0.
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/1-CORE-INDEX defines **semantic meaning only**. It does not define storage formats, on-disk encoding, or operational lifecycle. Those belong to ASL-STORE-INDEX, ASL/LOG/1, and ENC-ASL-CORE-INDEX.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose & Non-Goals
|
||||
|
||||
### 1.1 Purpose
|
||||
|
||||
ASL/1-CORE-INDEX defines the **semantic model** for indexing artifacts:
|
||||
|
||||
* It specifies what it means to map an artifact identity to a byte location.
|
||||
* It defines visibility, immutability, and shadowing semantics.
|
||||
* It ensures deterministic lookup for a fixed snapshot and log position.
|
||||
|
||||
### 1.2 Non-goals
|
||||
|
||||
ASL/1-CORE-INDEX explicitly does **not** define:
|
||||
|
||||
* On-disk layouts, segment files, or memory representations.
|
||||
* Block allocation, packing, GC, or lifecycle rules.
|
||||
* Snapshot implementation details, checkpoints, or log storage.
|
||||
* Performance optimizations (bloom filters, sharding, SIMD).
|
||||
* Federation, provenance, or execution semantics.
|
||||
|
||||
---
|
||||
|
||||
## 2. Terminology
|
||||
|
||||
* **Artifact** — ASL/1 immutable value defined in ASL/1-CORE.
|
||||
* **Reference** — ASL/1 content address of an Artifact (hash_id + digest).
|
||||
* **StoreConfig** — `{ encoding_profile, hash_id }` fixed per StoreSnapshot (ASL/1-STORE).
|
||||
* **Block** — immutable storage unit containing artifact bytes.
|
||||
* **BlockID** — opaque identifier for a block.
|
||||
* **ArtifactExtent** — `(BlockID, offset, length)` identifying a byte slice within a block.
|
||||
* **ArtifactLocation** — ordered list of `ArtifactExtent` values that, when concatenated, produce the artifact bytes.
|
||||
* **Degenerate store** — a store that treats each artifact as its own block,
|
||||
with a single extent covering the entire blob.
|
||||
* **Snapshot** — a checkpointed StoreSnapshot (ASL/1-STORE) used as a base state.
|
||||
* **Append-Only Log** — ordered sequence of index-visible mutations after a snapshot.
|
||||
* **CURRENT** — effective state after replaying a log position on a snapshot.
|
||||
|
||||
---
|
||||
|
||||
## 3. Core Mapping Semantics
|
||||
|
||||
### 3.1 Index Mapping
|
||||
|
||||
The index defines a semantic mapping:
|
||||
|
||||
```
|
||||
Reference -> ArtifactLocation
|
||||
```
|
||||
|
||||
For any visible `Reference`, there is exactly one `ArtifactLocation` at a given CURRENT state.
|
||||
|
||||
### 3.2 Determinism
|
||||
|
||||
For a fixed `{StoreConfig, Snapshot, LogPosition}`, lookup results MUST be deterministic. No nondeterministic input may affect index semantics.
|
||||
|
||||
### 3.3 StoreConfig Consistency
|
||||
|
||||
All references in an index view are interpreted under a fixed StoreConfig. Implementations MAY store only the digest portion in the index when `hash_id` is fixed by StoreConfig, but the semantic key is always a full `Reference`. Encoding profiles MUST allow variable-length digests; the digest length MUST be either explicit in the encoding or derivable from `hash_id` and StoreConfig.
|
||||
|
||||
---
|
||||
|
||||
## 4. ArtifactLocation Semantics
|
||||
|
||||
* An ArtifactLocation is an **ordered list** of ArtifactExtents.
|
||||
* Each extent references immutable bytes within a block.
|
||||
* The artifact bytes are defined by **concatenating extents in order**.
|
||||
* A visible ArtifactLocation MUST be **non-empty** and MUST fully cover the artifact byte sequence with no gaps or extra bytes.
|
||||
* Tombstone entries are visible but MUST have no ArtifactLocation; they only shadow prior entries.
|
||||
* Extents MUST have `length > 0` and MUST reference valid byte ranges within their blocks.
|
||||
* Extents MAY refer to the same BlockID multiple times, but the ordered concatenation MUST be deterministic and exact.
|
||||
* An ArtifactLocation is valid only while all referenced blocks are retained.
|
||||
* ASL/1-CORE-INDEX does not define how blocks are allocated or sealed; it only requires that referenced bytes are immutable for the lifetime of the mapping.
|
||||
* In a degenerate store, an ArtifactLocation consists of a single extent that
|
||||
spans the full blob in its dedicated block.
|
||||
|
||||
---
|
||||
|
||||
## 5. Visibility Model
|
||||
|
||||
An index entry is **visible** at CURRENT if and only if:
|
||||
|
||||
1. The entry is admitted by the store's visibility mechanism as defined in
|
||||
`ASL/STORE-INDEX/1` (e.g., via sealed segments and an append-only log), for
|
||||
the given snapshot/log position.
|
||||
2. The referenced bytes are immutable (e.g., the underlying block is sealed by
|
||||
store rules).
|
||||
|
||||
Visibility is binary; entries are either visible or not visible.
|
||||
|
||||
**Implementation note:** A store MAY implement a degenerate visibility
|
||||
mechanism (e.g., a single implicit segment that is always sealed and a trivial
|
||||
log position), which is sufficient for simple filesystem-backed stores such as
|
||||
`asl_store_fs`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Snapshot and Log Semantics
|
||||
|
||||
Snapshots provide a base mapping of sealed segments; the append-only log admits later segment seals and policy records that define subsequent changes.
|
||||
|
||||
The index state for a given CURRENT is defined as:
|
||||
|
||||
```
|
||||
Index(CURRENT) = Index(snapshot) + replay(log_position)
|
||||
```
|
||||
|
||||
Replay is strictly ordered, deterministic, and idempotent. Snapshot and log entries are semantically equivalent once replayed.
|
||||
|
||||
---
|
||||
|
||||
## 7. Immutability and Shadowing
|
||||
|
||||
### 7.1 Immutability
|
||||
|
||||
* Index entries are never mutated.
|
||||
* Once visible, an entry’s meaning does not change.
|
||||
* Referenced bytes are immutable for the lifetime of the entry.
|
||||
|
||||
### 7.2 Shadowing
|
||||
|
||||
* Later entries MAY shadow earlier entries with the same Reference.
|
||||
* Precedence is determined solely by log order.
|
||||
* Snapshot boundaries do not alter shadowing semantics.
|
||||
|
||||
---
|
||||
|
||||
## 8. Tombstones (Optional)
|
||||
|
||||
Tombstones MAY be used to invalidate prior mappings.
|
||||
|
||||
* A tombstone shadows earlier entries for the same Reference.
|
||||
* Tombstones are visibility policy records (see `ASL/LOG/1`) and are applied
|
||||
during replay; they are not required to appear as index entries.
|
||||
* If an encoding chooses to materialize tombstones in index segments, they MUST
|
||||
have no `ArtifactLocation` and MUST follow the same visibility rules as other
|
||||
entries.
|
||||
|
||||
---
|
||||
|
||||
## 9. Determinism Guarantees
|
||||
|
||||
For fixed:
|
||||
|
||||
* StoreConfig
|
||||
* Snapshot
|
||||
* Log prefix
|
||||
|
||||
ASL/1-CORE-INDEX guarantees:
|
||||
|
||||
* Deterministic lookup results
|
||||
* Deterministic shadowing resolution
|
||||
* Deterministic visibility
|
||||
|
||||
---
|
||||
|
||||
## 10. Normative Invariants
|
||||
|
||||
Conforming implementations MUST enforce:
|
||||
|
||||
1. No visibility without a sealed segment whose seal record is log-admitted (or snapshot-anchored).
|
||||
2. No mutation of visible index entries.
|
||||
3. Referenced bytes remain immutable for the entry’s lifetime.
|
||||
4. Shadowing follows strict log order.
|
||||
5. Snapshot + log replay uniquely defines CURRENT.
|
||||
6. Visible ArtifactLocations are non-empty and byte-exact (no gaps, no overrun), except for tombstones which have no ArtifactLocation.
|
||||
|
||||
Violation of any invariant constitutes index corruption.
|
||||
|
||||
---
|
||||
|
||||
## 11. Relationship to Other Specifications
|
||||
|
||||
| Layer | Responsibility |
|
||||
| ------------------ | ---------------------------------------------------------- |
|
||||
| ASL/1-CORE | Artifact semantics and identity |
|
||||
| ASL/1-STORE | StoreSnapshot and put/get logical model |
|
||||
| ASL/1-CORE-INDEX | Semantic mapping of Reference → ArtifactLocation |
|
||||
| ASL-STORE-INDEX | Lifecycle, replay, and visibility contracts |
|
||||
| ENC-ASL-CORE-INDEX | On-disk encoding for index segments and records |
|
||||
|
||||
---
|
||||
|
||||
## 12. Summary
|
||||
|
||||
ASL/1-CORE-INDEX specifies the semantic meaning of the index:
|
||||
|
||||
* It maps artifact References to byte locations deterministically.
|
||||
* It defines visibility and shadowing rules across snapshot + log replay.
|
||||
* It guarantees immutability and deterministic lookup.
|
||||
|
||||
It answers one question:
|
||||
|
||||
> *Given a Reference and a CURRENT state, where are the bytes?*
|
||||
148
tier1/asl-dam-1.md
Normal file
148
tier1/asl-dam-1.md
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
# ASL/DAM/1 -- Domain Authority Manifest
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-01-17
|
||||
Tags: [authority, trust, policy, domains]
|
||||
|
||||
**Document ID:** `ASL/DAM/1`
|
||||
**Layer:** L2 -- Authority semantics (no encoding)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/POLICY-HASH/1`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/OCS/1` -- offline certificate system
|
||||
* `ASL/OFFLINE-ROOT-TRUST/1` -- offline root policy
|
||||
* `PER/SIGNATURE/1` -- PER signature verification
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/DAM/1 defines the **logical structure and semantics** of the Domain Authority Manifest. It does not define an encoding.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
The Domain Authority Manifest (DAM) defines **who may assert authority** on behalf of a domain.
|
||||
It binds domain identity to principals, roles, and a policy hash.
|
||||
|
||||
---
|
||||
|
||||
## 2. Core Concepts
|
||||
|
||||
* **Principal**: a cryptographic public key
|
||||
* **Role**: capability granted to a principal
|
||||
* **Policy hash**: canonical hash binding policy constraints to a domain
|
||||
* **Authority scope**: explicit actions this DAM authorizes within the domain
|
||||
|
||||
---
|
||||
|
||||
## 3. Roles (Minimal Set)
|
||||
|
||||
| Role | Capability |
|
||||
| ---------- | ---------- |
|
||||
| produce | Create internal artifacts |
|
||||
| execute | Emit PERs |
|
||||
| publish | Publish artifacts/snapshots |
|
||||
| federate | Export published state |
|
||||
| audit | Verify only, no mutations |
|
||||
|
||||
Roles are **capabilities**, not identities.
|
||||
|
||||
---
|
||||
|
||||
## 4. Logical Schema
|
||||
|
||||
```text
|
||||
DomainAuthorityManifest {
|
||||
domain_id : DomainID
|
||||
version : u32
|
||||
root_key : PublicKey
|
||||
principals[] : PrincipalEntry
|
||||
policy_hash : Hash
|
||||
scope[] : AuthorityScope
|
||||
}
|
||||
|
||||
PrincipalEntry {
|
||||
principal_id : Hash
|
||||
public_key : PublicKey
|
||||
roles[] : Role
|
||||
}
|
||||
```
|
||||
|
||||
The DAM is immutable once published. Rotation is performed by publishing a new DAM.
|
||||
|
||||
### 4.1 Authority Scope
|
||||
|
||||
The optional `scope[]` constrains what this DAM authorizes. If omitted, scope is
|
||||
assumed to include all roles defined in this document.
|
||||
|
||||
```text
|
||||
AuthorityScope = {
|
||||
"produce",
|
||||
"execute",
|
||||
"publish",
|
||||
"federate",
|
||||
"audit"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Admission and Pinning (Normative)
|
||||
|
||||
Before a DAM is trusted, a receiving domain MUST:
|
||||
|
||||
1. Admit the domain (see `ASL/DAP/1`).
|
||||
2. Pin the DAM artifact to a snapshot.
|
||||
3. Pin the DAM's `policy_hash` for that snapshot lineage.
|
||||
|
||||
---
|
||||
|
||||
## 6. Validation Rules (Normative)
|
||||
|
||||
A node MUST reject any action unless:
|
||||
|
||||
1. The DAM artifact is visible in the relevant snapshot.
|
||||
2. The DAM hash matches the snapshot reference (if recorded).
|
||||
3. The action is signed by a principal listed in the DAM.
|
||||
4. The principal has the required role.
|
||||
5. The DAM `root_key` is certified by the offline root trust chain.
|
||||
6. The action is within the DAM's declared `scope[]` (if present).
|
||||
|
||||
---
|
||||
|
||||
## 7. Policy Binding
|
||||
|
||||
The DAM `policy_hash` binds the domain to a specific policy document. If policy changes, a new DAM MUST be published and referenced by new snapshots.
|
||||
|
||||
---
|
||||
|
||||
## 8. Idempotency and Rotation
|
||||
|
||||
* Replaying the same snapshot + DAM MUST yield identical authority decisions.
|
||||
* Rotation is done by publishing a new DAM and referencing it in new snapshots.
|
||||
* Old snapshots remain valid with the DAM they reference.
|
||||
|
||||
---
|
||||
|
||||
## 9. Non-Goals
|
||||
|
||||
* Encoding format
|
||||
* Key rotation workflow
|
||||
* Live revocation
|
||||
|
||||
---
|
||||
|
||||
## 10. Summary
|
||||
|
||||
ASL/DAM/1 defines the minimal authority document for a domain, binding principals and roles to a policy hash under an offline root trust chain.
|
||||
165
tier1/asl-dap-1.md
Normal file
165
tier1/asl-dap-1.md
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
# ASL/DAP/1 -- Domain Admission Protocol
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-01-17
|
||||
Tags: [admission, trust, federation]
|
||||
|
||||
**Document ID:** `ASL/DAP/1`
|
||||
**Layer:** L2 -- Admission semantics (no transport)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/DAM/1`
|
||||
* `ASL/POLICY-HASH/1`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/OCS/1` -- offline certificate system
|
||||
* `ASL/OFFLINE-ROOT-TRUST/1`
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/DAP/1 defines **admission semantics** for recognizing external domains. It does not define transport.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
The Domain Admission Protocol defines how a receiving domain evaluates and admits an applicant domain before any federation or trust is granted.
|
||||
|
||||
---
|
||||
|
||||
## 2. Actors
|
||||
|
||||
| Actor | Role |
|
||||
| ----- | ---- |
|
||||
| Applicant Domain | Domain requesting admission |
|
||||
| Receiving Domain | Domain evaluating admission |
|
||||
| Operator | Optional human/policy agent |
|
||||
|
||||
---
|
||||
|
||||
## 3. Admission Object Model
|
||||
|
||||
An admission request MUST include:
|
||||
|
||||
* Domain Authority Manifest (DAM)
|
||||
* Proof of possession of the DAM root key
|
||||
* Requested admission scope
|
||||
* Optional courtesy lease request
|
||||
|
||||
No artifacts, blocks, or logs are required for admission.
|
||||
|
||||
---
|
||||
|
||||
## 4. Admission Record (Normative)
|
||||
|
||||
The receiving domain MUST persist an admission record that is snapshot-pinned:
|
||||
|
||||
```text
|
||||
AdmissionRecord {
|
||||
domain_id
|
||||
dam_hash
|
||||
policy_hash
|
||||
admitted_scope[]
|
||||
decision
|
||||
decision_epoch
|
||||
}
|
||||
```
|
||||
|
||||
The `decision_epoch` is a monotonically increasing local counter (not wall time).
|
||||
|
||||
---
|
||||
|
||||
## 5. Admission Flow (Normative)
|
||||
|
||||
1. **Submission**
|
||||
|
||||
* Applicant sends DAM and proof of root-key possession.
|
||||
|
||||
2. **Structural validation**
|
||||
|
||||
* Receiving domain verifies DAM schema and signature.
|
||||
* Policy hash integrity MUST be verified.
|
||||
|
||||
3. **Policy compatibility**
|
||||
|
||||
* Receiving domain evaluates requested scope and policy alignment.
|
||||
|
||||
4. **Decision**
|
||||
|
||||
* Outcomes: ACCEPTED, ACCEPTED_LIMITED, DEFERRED, REJECTED.
|
||||
|
||||
---
|
||||
|
||||
## 6. Admission Guarantees
|
||||
|
||||
If accepted:
|
||||
|
||||
* DomainID is recognized by the receiving domain.
|
||||
* Root key and policy hash are pinned.
|
||||
* Admission scope is enforced for federation.
|
||||
|
||||
Admission does not imply trust in artifacts beyond the granted scope.
|
||||
|
||||
---
|
||||
|
||||
## 7. Scope Enforcement (Normative)
|
||||
|
||||
* Admission scope MUST gate federation view construction and replay admission.
|
||||
* A receiving domain MUST NOT admit state outside the granted scope.
|
||||
* Scope changes require a new admission decision and updated AdmissionRecord.
|
||||
|
||||
---
|
||||
|
||||
## 8. Courtesy Lease (Optional)
|
||||
|
||||
A courtesy lease is a **bounded, revocable grant of resources** without semantic trust.
|
||||
|
||||
```text
|
||||
CourtesyLease {
|
||||
lease_id
|
||||
domain_id
|
||||
granted_by_domain
|
||||
|
||||
resources: {
|
||||
storage_bytes
|
||||
block_count
|
||||
snapshot_count
|
||||
}
|
||||
|
||||
duration: {
|
||||
start_time
|
||||
end_time
|
||||
}
|
||||
|
||||
constraints: {
|
||||
encrypted_only: boolean
|
||||
no_federation: boolean
|
||||
no_public_indexing: boolean
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Courtesy storage MAY be deleted upon lease expiry. Courtesy does not grant federation or publication rights.
|
||||
|
||||
---
|
||||
|
||||
## 9. Non-Goals
|
||||
|
||||
* Transport format
|
||||
* PKI integration
|
||||
* Live revocation or liveness checks
|
||||
|
||||
---
|
||||
|
||||
## 10. Summary
|
||||
|
||||
ASL/DAP/1 defines a deterministic admission process for domains, with optional courtesy leasing for rescue and bootstrap scenarios.
|
||||
170
tier1/asl-domain-model-1.md
Normal file
170
tier1/asl-domain-model-1.md
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
# ASL/DOMAIN-MODEL/1 -- Domain Topology and Publication Semantics
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-01-17
|
||||
Tags: [domains, authority, publication, federation]
|
||||
|
||||
**Document ID:** `ASL/DOMAIN-MODEL/1`
|
||||
**Layer:** L2 -- Domain topology and delegation semantics (no transport)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/DAM/1`
|
||||
* `ASL/DAP/1`
|
||||
* `ASL/POLICY-HASH/1`
|
||||
* `ASL/FEDERATION/1`
|
||||
* `ASL/LOG/1`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/OCS/1` -- offline certificate system
|
||||
* `ASL/OFFLINE-ROOT-TRUST/1`
|
||||
* `ASL/SYSTEM/1` -- unified system view
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/DOMAIN-MODEL/1 defines domain topology and publication semantics. It does not define transport, storage layout, or encoding.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This document defines how domains relate, how authority is delegated, and how publication is made safe and explicit.
|
||||
It provides a stable mental model for personal, group, and shared domains without introducing implicit trust.
|
||||
|
||||
---
|
||||
|
||||
## 2. First Principles (Normative)
|
||||
|
||||
1. **No implicit inheritance.** Domains are not hierarchical by default; trust is explicit.
|
||||
2. **Authority is explicit.** Authority is defined by DAM + certificates; it is never implied by naming or topology.
|
||||
3. **Publication is explicit.** Visibility is controlled by index metadata and policy, not by storage or naming.
|
||||
4. **Determinism is preserved.** All cross-domain visibility MUST be replayable from snapshots and logs.
|
||||
|
||||
---
|
||||
|
||||
## 3. Domain Roles (Common, Personal, Working)
|
||||
|
||||
These roles are semantic patterns; a deployment MAY create many instances.
|
||||
|
||||
### 3.1 Common Domain
|
||||
|
||||
A shared, conservative domain for stable artifacts and schemas.
|
||||
|
||||
Properties:
|
||||
|
||||
* High trust threshold
|
||||
* Read-mostly, slow-changing
|
||||
* Publishes broadly
|
||||
|
||||
### 3.2 Personal Domain
|
||||
|
||||
A domain anchored to a single personal identity and authority.
|
||||
|
||||
Properties:
|
||||
|
||||
* Root of local agency
|
||||
* Owns offline roots and DAM
|
||||
* Decides what to publish and to whom
|
||||
|
||||
### 3.3 Working / Ephemeral Domains
|
||||
|
||||
Task-focused domains created under delegated authority.
|
||||
|
||||
Properties:
|
||||
|
||||
* Narrow policy and scope
|
||||
* Often short-lived
|
||||
* Results MAY be promoted to personal or common domains
|
||||
|
||||
---
|
||||
|
||||
## 4. Delegation and Authority (Normative)
|
||||
|
||||
Delegation is explicit and certificate-based.
|
||||
|
||||
Rules:
|
||||
|
||||
* A domain MAY delegate authority to a new domain by issuing a `domain_root` certificate (ASL/OCS/1).
|
||||
* Delegation MUST be recorded in the receiving domain's DAM and policy hash.
|
||||
* Delegation does not create inheritance: the delegating domain does not gain automatic visibility into the new domain.
|
||||
|
||||
---
|
||||
|
||||
## 5. Publication and Visibility (Normative)
|
||||
|
||||
Publication is a visibility decision, not a storage action.
|
||||
|
||||
Rules:
|
||||
|
||||
* Only published artifacts are eligible for cross-domain visibility.
|
||||
* Publication state MUST be encoded in index metadata (ENC-ASL-CORE-INDEX).
|
||||
* Blocks and storage layouts MUST NOT be treated as publication units.
|
||||
* Publication of snapshots (or snapshot hashes) is allowed but MUST NOT imply data publication.
|
||||
|
||||
---
|
||||
|
||||
## 6. Cross-Domain Trust and Admission (Normative)
|
||||
|
||||
Trust is established by explicit admission and policy compatibility.
|
||||
|
||||
Rules:
|
||||
|
||||
* A receiving domain MUST admit an external domain (ASL/DAP/1) before including its state.
|
||||
* Policy hash compatibility MUST be checked before accepting published artifacts.
|
||||
* A domain MAY pin a trusted foreign domain without reciprocal trust.
|
||||
|
||||
---
|
||||
|
||||
## 7. Safe Publication Patterns (Informative)
|
||||
|
||||
### 7.1 Personal to Personal Archive
|
||||
|
||||
```
|
||||
personal/rescue -> personal/archive
|
||||
```
|
||||
|
||||
* Publish explicitly from the working domain to an archival domain.
|
||||
* Only published artifacts are visible across the boundary.
|
||||
|
||||
### 7.2 Personal to Group Domain
|
||||
|
||||
```
|
||||
personal/project -> group/shared
|
||||
```
|
||||
|
||||
* Requires admission by the group domain and policy compatibility.
|
||||
* No unilateral publishing into the group domain.
|
||||
|
||||
### 7.3 Personal to Public Domain
|
||||
|
||||
```
|
||||
personal/public -> common/public
|
||||
```
|
||||
|
||||
* One-way trust is permitted.
|
||||
* Public domain pins the personal domain; the personal domain need not pin the public domain.
|
||||
|
||||
---
|
||||
|
||||
## 8. Non-Goals
|
||||
|
||||
ASL/DOMAIN-MODEL/1 does not define:
|
||||
|
||||
* Transport or replication protocols
|
||||
* Encoding formats
|
||||
* Storage layouts or filesystem assumptions
|
||||
* Governance workflows beyond admission and policy compatibility
|
||||
|
||||
Middle layer (informative): the daemon/service boundary around core logic that
|
||||
owns network transport, admission workflows, and operational policy.
|
||||
|
||||
Implementation note (informative): core federation primitives live in
|
||||
`include/amduat/fed/{registry,ingest,replay,view}.h`.
|
||||
113
tier1/asl-encrypted-blocks-1.md
Normal file
113
tier1/asl-encrypted-blocks-1.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# ASL/ENCRYPTED-BLOCKS/1 -- Encrypted Block Storage Across Domains
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-01-17
|
||||
Tags: [encryption, blocks, federation, storage]
|
||||
|
||||
**Document ID:** `ASL/ENCRYPTED-BLOCKS/1`
|
||||
**Layer:** L2 -- Encrypted storage semantics (no transport)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL-STORE-INDEX`
|
||||
* `ASL/FEDERATION/1`
|
||||
* `ASL/LOG/1`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/DOMAIN-MODEL/1`
|
||||
* `ASL/POLICY-HASH/1`
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/ENCRYPTED-BLOCKS/1 defines semantics for storing encrypted blocks across domains. It does not define encryption algorithms, key management, or transport.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This document defines how encrypted blocks may be stored in a foreign domain without transferring semantic authority or decryption capability.
|
||||
|
||||
---
|
||||
|
||||
## 2. Core Principle (Normative)
|
||||
|
||||
A domain MAY store encrypted blocks for another domain, but MUST NOT assert semantic meaning for those bytes.
|
||||
|
||||
Meaning is owned by the domain that holds the decryption keys and index entries.
|
||||
|
||||
---
|
||||
|
||||
## 3. Encryption Model (Normative)
|
||||
|
||||
### 3.1 Block Encryption
|
||||
|
||||
Before sealing, a block MAY be encrypted:
|
||||
|
||||
```
|
||||
plaintext_block
|
||||
-> encrypt(K)
|
||||
-> ciphertext_block
|
||||
-> BlockID = H(ciphertext_block)
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
* Encryption occurs before sealing.
|
||||
* `BlockID` is computed over ciphertext bytes.
|
||||
* Deterministic encryption is NOT required.
|
||||
|
||||
### 3.2 Key Ownership
|
||||
|
||||
* Encryption keys are owned by the originating domain.
|
||||
* Keys MUST NOT be federated or embedded in index metadata.
|
||||
* Decryption metadata MUST remain local to the originating domain.
|
||||
|
||||
---
|
||||
|
||||
## 4. Foreign Domain Storage (Normative)
|
||||
|
||||
A foreign domain storing encrypted blocks:
|
||||
|
||||
* Treats ciphertext blocks as opaque bytes.
|
||||
* MAY retain or GC blocks under its local policy.
|
||||
* MUST NOT create semantic index entries for those blocks.
|
||||
|
||||
---
|
||||
|
||||
## 5. Originating Domain References (Normative)
|
||||
|
||||
The originating domain:
|
||||
|
||||
* Maintains index entries referencing the ciphertext `BlockID`.
|
||||
* Applies normal visibility, log, and snapshot rules.
|
||||
* Uses local decryption metadata to materialize plaintext.
|
||||
|
||||
---
|
||||
|
||||
## 6. Cross-Domain References (Informative)
|
||||
|
||||
Two references are distinct:
|
||||
|
||||
* **Storage reference:** foreign domain stores ciphertext blocks.
|
||||
* **Semantic reference:** originating domain records artifact visibility and meaning.
|
||||
|
||||
Foreign storage does not imply federation of semantics.
|
||||
|
||||
---
|
||||
|
||||
## 7. Non-Goals
|
||||
|
||||
ASL/ENCRYPTED-BLOCKS/1 does not define:
|
||||
|
||||
* Key exchange or key discovery
|
||||
* Encryption algorithm choices
|
||||
* Transport or replication protocols
|
||||
* Storage layout or block packing rules
|
||||
192
tier1/asl-federation-1.md
Normal file
192
tier1/asl-federation-1.md
Normal file
|
|
@ -0,0 +1,192 @@
|
|||
# ASL/FEDERATION/1 -- Core Federation Semantics
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-01-17
|
||||
Tags: [federation, domains, visibility, determinism]
|
||||
|
||||
**Document ID:** `ASL/FEDERATION/1`
|
||||
**Layer:** L2 -- Federation semantics (no transport, no encodings)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/1-CORE`
|
||||
* `ASL/1-CORE-INDEX`
|
||||
* `ASL/LOG/1`
|
||||
* `ASL-STORE-INDEX`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ENC-ASL-CORE-INDEX` -- domain/visibility fields (`tier1/enc-asl-core-index.md`)
|
||||
* `ASL/SYSTEM/1` -- unified system view
|
||||
* `ASL/FEDERATION-REPLAY/1` -- cross-node deterministic replay
|
||||
* `ASL/DAM/1` -- Domain Authority Manifest
|
||||
* `ASL/POLICY-HASH/1` -- policy hash
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/FEDERATION/1 defines **semantic rules** for multi-domain visibility and cross-domain references. It does not define transport, replication, or encodings.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
ASL/FEDERATION/1 defines the **multi-domain model** for ASL-based systems:
|
||||
|
||||
* Domain isolation and visibility rules
|
||||
* Published vs internal state
|
||||
* Cross-domain reference constraints
|
||||
* Snapshot identity and deterministic reconstruction
|
||||
|
||||
---
|
||||
|
||||
## 2. Core Concepts
|
||||
|
||||
| Term | Definition |
|
||||
| -------------------- | ---------- |
|
||||
| Domain | Logical namespace with its own ASL store, log, and snapshot lineage. |
|
||||
| Internal state | Artifacts/snapshots visible only within the domain. |
|
||||
| Published state | Artifacts/snapshots visible to other domains. |
|
||||
| Federated snapshot | Snapshot with visibility state that may be imported by other domains. |
|
||||
| Cross-domain reference | Reference to a published artifact from another domain. |
|
||||
| Federation view | A deterministic view constructed by combining local internal state with admitted published state from other domains. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Domain Semantics
|
||||
|
||||
1. **Domain isolation**
|
||||
|
||||
* Each domain has its own store, index, and log.
|
||||
* Internal state is invisible outside the domain.
|
||||
|
||||
2. **Published state**
|
||||
|
||||
* Published artifacts and snapshots are visible to other domains.
|
||||
* Published artifacts MUST satisfy ASL immutability and snapshot safety rules.
|
||||
|
||||
3. **Cross-domain references**
|
||||
|
||||
* Only published artifacts MAY be referenced by other domains.
|
||||
* Cross-domain references are read-only and immutable.
|
||||
* The consuming domain indexes imported artifacts using normal ASL index semantics.
|
||||
* Imported entries MUST preserve origin metadata (domain identity and visibility) for deterministic replay.
|
||||
|
||||
---
|
||||
|
||||
## 4. Snapshot Identity
|
||||
|
||||
* Snapshot IDs are unique per domain.
|
||||
* Federated snapshot identity is `(DomainID, SnapshotID)`.
|
||||
* A federated snapshot MAY include cross-domain references only to published artifacts.
|
||||
* Replay of federated state MUST be bounded by the source domain's `{SnapshotID, LogPrefix}`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Visibility Rules
|
||||
|
||||
| Object | Internal Domain | Other Domains |
|
||||
| ----------------------------------- | --------------- | ------------------- |
|
||||
| Internal artifact | visible | hidden |
|
||||
| Published artifact | visible | visible (read-only) |
|
||||
| Internal snapshot | visible | hidden |
|
||||
| Published snapshot | visible | visible |
|
||||
| Block supporting published artifact | visible | visible |
|
||||
| Block supporting internal artifact | visible | hidden |
|
||||
|
||||
* Index entries follow the same visibility rules.
|
||||
* Determinism is defined per-domain and per-snapshot view.
|
||||
|
||||
---
|
||||
|
||||
## 6. Federation View Construction
|
||||
|
||||
To construct a federation view for a domain:
|
||||
|
||||
1. Start with the local domain's internal + published state at `{SnapshotID, LogPrefix}`.
|
||||
2. For each admitted foreign domain, include only published state that is:
|
||||
|
||||
* Visible under that domain's `{SnapshotID, LogPrefix}`, and
|
||||
* Allowed by the receiving domain's admission and policy rules.
|
||||
|
||||
3. Apply normal ASL index shadowing and tombstone rules within each domain's log order.
|
||||
|
||||
Federation MUST NOT bypass ASL/LOG/1 ordering or ASL/1-CORE-INDEX semantics.
|
||||
|
||||
---
|
||||
|
||||
## 7. Cross-Domain Operations
|
||||
|
||||
1. **Import published artifacts**
|
||||
|
||||
* A domain MAY import a published artifact from another domain.
|
||||
* Imported artifacts MUST be treated as immutable.
|
||||
* Import MUST be gated by admission and policy compatibility (see `ASL/DAP/1` and `ASL/POLICY-HASH/1`).
|
||||
|
||||
2. **Export published artifacts**
|
||||
|
||||
* Internal artifacts MAY be promoted to published state.
|
||||
* Promotion MUST be snapshot-bound and log-ordered.
|
||||
* Publication MUST respect the domain's policy hash and DAM roles.
|
||||
|
||||
3. **Federation log propagation**
|
||||
|
||||
* Each domain maintains its own append-only log.
|
||||
* Federation MAY replicate published log-visible state.
|
||||
|
||||
---
|
||||
|
||||
## 8. Provenance and Traceability
|
||||
|
||||
* Execution receipts MAY include cross-domain inputs.
|
||||
* Provenance MUST preserve origin domain and snapshot identity.
|
||||
* Deterministic replay MUST be possible given `{Snapshot, LogPrefix}` for each domain.
|
||||
|
||||
---
|
||||
|
||||
## 9. Normative Invariants
|
||||
|
||||
1. **Determinism:** Reconstructing CURRENT in a domain yields the same visible state given the same snapshot and log prefix.
|
||||
2. **Immutability:** Published artifacts and snapshots are immutable.
|
||||
3. **Domain isolation:** Internal artifacts and snapshots are never visible to other domains.
|
||||
4. **Federation safety:** Cross-domain references are read-only.
|
||||
5. **Snapshot integrity:** Federated snapshots reference only published artifacts.
|
||||
|
||||
---
|
||||
|
||||
## 10. Integration with Other Layers
|
||||
|
||||
| Layer | Role in Federation |
|
||||
| ------------------ | ------------------ |
|
||||
| ASL/1-CORE | Artifact immutability and identity |
|
||||
| ASL/1-CORE-INDEX | Semantic mapping and shadowing |
|
||||
| ASL-STORE-INDEX | Sealing, retention, snapshot pinning |
|
||||
| ASL/LOG/1 | Log-ordered visibility |
|
||||
| ENC-ASL-CORE-INDEX | Domain/visibility metadata in records |
|
||||
|
||||
---
|
||||
|
||||
## 11. Non-Goals
|
||||
|
||||
* Transport protocols
|
||||
* Network replication
|
||||
* Witness signatures
|
||||
* Domain admission and trust policy
|
||||
|
||||
Middle layer (informative): the daemon/service boundary around core logic that
|
||||
owns network transport, admission workflows, and operational policy.
|
||||
|
||||
Implementation note (informative): core federation primitives live in
|
||||
`include/amduat/fed/{registry,ingest,replay,view}.h`.
|
||||
|
||||
---
|
||||
|
||||
## 12. Summary
|
||||
|
||||
ASL/FEDERATION/1 defines the semantic rules for multi-domain visibility and cross-domain reference. It keeps federation deterministic, snapshot-safe, and read-only across domain boundaries.
|
||||
170
tier1/asl-federation-replay-1.md
Normal file
170
tier1/asl-federation-replay-1.md
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
# ASL/FEDERATION-REPLAY/1 -- Cross-Node Deterministic Replay
|
||||
|
||||
Status: Draft
|
||||
Owner: Architecture
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-01-17
|
||||
Tags: [federation, replay, determinism, tgk, pel]
|
||||
|
||||
**Document ID:** `ASL/FEDERATION-REPLAY/1`
|
||||
**Layer:** L2 -- Federation replay semantics (no transport)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/FEDERATION/1`
|
||||
* `ASL/LOG/1`
|
||||
* `ASL/1-CORE-INDEX`
|
||||
* `TGK/1`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/SYSTEM/1` -- unified system view
|
||||
* `ENC-ASL-CORE-INDEX` -- domain metadata
|
||||
* `ASL/DAP/1` -- admission gating
|
||||
* `ASL/POLICY-HASH/1` -- policy compatibility
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/FEDERATION-REPLAY/1 defines **deterministic replay rules** for federated propagation. It does not define network protocols or encodings.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This document defines how artifacts, PERs, and TGK edges are propagated and replayed across federation nodes while preserving deterministic reconstruction.
|
||||
|
||||
---
|
||||
|
||||
## 2. Core Inputs
|
||||
|
||||
* **ArtifactKey**: canonical identifier for artifacts and PERs
|
||||
* **SnapshotID**: snapshot boundary for replay
|
||||
* **logseq**: append-only ordering within a domain
|
||||
* **Canonical Edge Key**: TGK edge identity
|
||||
|
||||
---
|
||||
|
||||
## 3. Replay Record Requirements
|
||||
|
||||
Each propagated record MUST be replayable without external context. Records MUST carry:
|
||||
|
||||
* `domain_id` (source domain)
|
||||
* `record_type` (artifact, PER, TGK edge, tombstone)
|
||||
* `logseq` (source-domain ordering)
|
||||
* `snapshot_id` and `log_prefix` bounds for which the record is visible
|
||||
* Canonical identity (ArtifactKey or Canonical Edge Key)
|
||||
* Visibility metadata (internal/published, cross-domain source when applicable)
|
||||
|
||||
Records MAY include optional integrity fields (hashes, signatures), but replay MUST
|
||||
remain deterministic without them.
|
||||
|
||||
---
|
||||
|
||||
## 4. Propagation Rules
|
||||
|
||||
### 4.1 Artifacts and PERs
|
||||
|
||||
* Artifacts and PERs are propagated with their `ArtifactKey` and `logseq`.
|
||||
* Only artifacts visible under a published snapshot MAY be propagated.
|
||||
* Duplicate ArtifactKeys MUST be de-duplicated by identity.
|
||||
* Imported entries MUST preserve origin metadata (domain identity and visibility).
|
||||
|
||||
### 4.2 TGK Edges
|
||||
|
||||
* TGK edges are propagated with their canonical edge identity and `logseq`.
|
||||
* Edge propagation MUST preserve the same snapshot/log visibility constraints as artifacts.
|
||||
|
||||
---
|
||||
|
||||
## 5. Deterministic Replay Ordering
|
||||
|
||||
Replay order MUST be deterministic across nodes:
|
||||
|
||||
1. Sort by `logseq` ascending
|
||||
2. Tie-break by canonical identity (ArtifactKey or Canonical Edge Key)
|
||||
|
||||
This ordering MUST be applied identically by all receivers.
|
||||
|
||||
---
|
||||
|
||||
## 6. Snapshot Bounds
|
||||
|
||||
* Replay MUST be bounded by `{SnapshotID, LogPrefix}`.
|
||||
* Records with `logseq` greater than the replay prefix MUST be ignored.
|
||||
* Replay MUST use the source domain's `{SnapshotID, LogPrefix}` as the bound for imported state.
|
||||
|
||||
---
|
||||
|
||||
## 7. Federation View Construction
|
||||
|
||||
Receivers construct a federation view by combining:
|
||||
|
||||
1. Local domain state at `{SnapshotID, LogPrefix}`.
|
||||
2. Admitted foreign published state bounded by the source domain's `{SnapshotID, LogPrefix}`.
|
||||
|
||||
Admission and policy compatibility MUST be enforced before any foreign state is admitted.
|
||||
|
||||
---
|
||||
|
||||
## 8. Tombstones and Shadowing
|
||||
|
||||
* Tombstones MUST be replayed in log order and apply only within their source domain.
|
||||
* A tombstone from domain A MUST NOT shadow artifacts from domain B.
|
||||
* Shadowing is resolved per-domain using ASL/LOG/1 order and ASL/1-CORE-INDEX semantics.
|
||||
|
||||
---
|
||||
|
||||
## 9. Conflict Handling
|
||||
|
||||
1. **ArtifactKey collision**
|
||||
|
||||
* If bytes match existing artifact: discard duplicate.
|
||||
* If bytes differ: reject and flag conflict.
|
||||
|
||||
2. **TGK edge conflicts**
|
||||
|
||||
* Multiple edges with the same canonical identity are resolved by log order and tombstone rules.
|
||||
|
||||
3. **PER conflicts**
|
||||
|
||||
* PERs with identical inputs and program identity but divergent outputs MUST be rejected.
|
||||
|
||||
---
|
||||
|
||||
## 10. Replay State and Idempotency
|
||||
|
||||
Replay MUST be idempotent:
|
||||
|
||||
* Re-applying the same record set MUST NOT change the resulting state.
|
||||
* Receivers SHOULD track `{domain_id, logseq}` high-water marks per peer.
|
||||
* Checkpointing MUST be aligned to `{SnapshotID, LogPrefix}` boundaries.
|
||||
|
||||
---
|
||||
|
||||
## 11. Provenance and Audit
|
||||
|
||||
Receivers SHOULD maintain:
|
||||
|
||||
* Last applied `logseq` per peer
|
||||
* Snapshot provenance tables for applied records
|
||||
|
||||
This supports deterministic audit and replay verification.
|
||||
|
||||
---
|
||||
|
||||
## 12. Non-Goals
|
||||
|
||||
* Transport protocol selection
|
||||
* Streaming formats
|
||||
* Compression or batching
|
||||
|
||||
---
|
||||
|
||||
## 13. Summary
|
||||
|
||||
ASL/FEDERATION-REPLAY/1 defines a deterministic replay ordering and conflict rules to ensure federation is reproducible across nodes and snapshots.
|
||||
296
tier1/asl-index-accel-1.md
Normal file
296
tier1/asl-index-accel-1.md
Normal file
|
|
@ -0,0 +1,296 @@
|
|||
# ASL/INDEX-ACCEL/1 — Index Acceleration Semantics
|
||||
|
||||
Status: Draft
|
||||
Owner: Niklas Rydberg
|
||||
Version: 0.1.0
|
||||
SoT: No
|
||||
Last Updated: 2025-11-16
|
||||
Linked Phase Pack: N/A
|
||||
Tags: [deterministic, index, acceleration]
|
||||
|
||||
<!-- Source: /amduat-api/tier1/asl-index-accel-1.md | Canonical: /amduat/tier1/asl-index-accel-1.md -->
|
||||
|
||||
**Document ID:** `ASL/INDEX-ACCEL/1`
|
||||
**Layer:** L1 — Acceleration rules over index semantics (no storage / encoding)
|
||||
|
||||
**Depends on (normative):**
|
||||
|
||||
* `ASL/1-CORE-INDEX`
|
||||
* `ASL/LOG/1`
|
||||
|
||||
**Informative references:**
|
||||
|
||||
* `ASL/STORE-INDEX/1` — store lifecycle and replay contracts
|
||||
* `ENC/ASL-CORE-INDEX/1` — bytes-on-disk encoding profile
|
||||
* `TGK/1` — TGK semantics and visibility alignment
|
||||
* `TGK/1-CORE` — EdgeBody and EdgeTypeId definitions
|
||||
|
||||
© 2025 Niklas Rydberg.
|
||||
|
||||
## License
|
||||
|
||||
Except where otherwise noted, this document (text and diagrams) is licensed under
|
||||
the Creative Commons Attribution 4.0 International License (CC BY 4.0).
|
||||
|
||||
The identifier registries and mapping tables (e.g. TypeTag IDs, HashId
|
||||
assignments, EdgeTypeId tables) are additionally made available under CC0 1.0
|
||||
Universal (CC0) to enable unrestricted reuse in implementations and derivative
|
||||
specifications.
|
||||
|
||||
Code examples in this document are provided under the Apache License 2.0 unless
|
||||
explicitly stated otherwise. Test vectors, where present, are dedicated to the
|
||||
public domain under CC0 1.0.
|
||||
|
||||
---
|
||||
|
||||
## 0. Conventions
|
||||
|
||||
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, and **MAY** are to be interpreted as in RFC 2119.
|
||||
|
||||
ASL/INDEX-ACCEL/1 defines **acceleration semantics only**. It MUST NOT change index meaning defined by ASL/1-CORE-INDEX.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
ASL/INDEX-ACCEL/1 defines **acceleration mechanisms** used by ASL-based indexes, including:
|
||||
|
||||
* Routing keys
|
||||
* Sharding
|
||||
* Filters (Bloom, XOR, Ribbon, etc.)
|
||||
* SIMD execution
|
||||
* Hash recasting
|
||||
|
||||
All mechanisms defined herein are **observationally invisible** to ASL/1-CORE-INDEX semantics.
|
||||
|
||||
---
|
||||
|
||||
## 2. Scope
|
||||
|
||||
Applies to:
|
||||
|
||||
* Artifact indexes (ASL)
|
||||
* Projection and graph indexes (e.g., TGK)
|
||||
* Any index layered on ASL/1-CORE-INDEX semantics
|
||||
|
||||
Does **not** define:
|
||||
|
||||
* Artifact or edge identity
|
||||
* Snapshot semantics
|
||||
* Storage lifecycle
|
||||
* Encoding details
|
||||
|
||||
---
|
||||
|
||||
## 3. Canonical Key vs Routing Key
|
||||
|
||||
### 3.1 Canonical Key
|
||||
|
||||
The **Canonical Key** uniquely identifies an indexable entity.
|
||||
|
||||
Examples:
|
||||
|
||||
* Artifact: `Reference`
|
||||
* TGK Edge: canonical key defined by `TGK/1` and `TGK/1-CORE` (opaque here)
|
||||
|
||||
Properties:
|
||||
|
||||
* Defines semantic identity
|
||||
* Used for equality, shadowing, and tombstones
|
||||
* Stable and immutable
|
||||
* Fully compared on index match
|
||||
|
||||
### 3.2 Routing Key
|
||||
|
||||
The **Routing Key** is a **derived, advisory key** used exclusively for acceleration.
|
||||
|
||||
Properties:
|
||||
|
||||
* Derived deterministically from Canonical Key and optional attributes
|
||||
* MAY be used for sharding, filters, SIMD layouts
|
||||
* MUST NOT affect index semantics
|
||||
* MUST be verified by full Canonical Key comparison on match
|
||||
|
||||
Formal rule:
|
||||
|
||||
```
|
||||
CanonicalKey determines correctness
|
||||
RoutingKey determines performance
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Filter Semantics
|
||||
|
||||
### 4.1 Advisory Nature
|
||||
|
||||
All filters are **advisory only**.
|
||||
|
||||
Rules:
|
||||
|
||||
* False positives are permitted
|
||||
* False negatives are forbidden
|
||||
* Filter behavior MUST NOT affect correctness
|
||||
|
||||
Invariant:
|
||||
|
||||
```
|
||||
Filter miss => key is definitely absent
|
||||
Filter hit => key may be present
|
||||
```
|
||||
|
||||
### 4.2 Filter Inputs
|
||||
|
||||
Filters operate over **Routing Keys**, not Canonical Keys.
|
||||
|
||||
A Routing Key MAY incorporate:
|
||||
|
||||
* Hash of Canonical Key
|
||||
* Artifact type tag (if present)
|
||||
* TGK `EdgeTypeId` or other immutable classification attributes (TGK/1-CORE)
|
||||
* Direction, role, or other immutable classification attributes
|
||||
|
||||
Absence of optional attributes MUST be encoded explicitly.
|
||||
|
||||
### 4.3 Filter Construction
|
||||
|
||||
* Filters are built only over **sealed, immutable segments**
|
||||
* Filters are immutable once built
|
||||
* Filter construction MUST be deterministic
|
||||
* Filter state MUST be covered by segment checksums
|
||||
* Filters SHOULD be snapshot-scoped or versioned with their segment to avoid
|
||||
unbounded false-positive accumulation over time
|
||||
|
||||
---
|
||||
|
||||
## 5. Sharding Semantics
|
||||
|
||||
### 5.1 Observational Invisibility
|
||||
|
||||
Sharding is a **mechanical partitioning** of the index.
|
||||
|
||||
Invariant:
|
||||
|
||||
```
|
||||
LogicalIndex = union(all shards)
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
* Shards MUST NOT affect lookup results
|
||||
* Shard count and boundaries may change over time
|
||||
* Rebalancing MUST preserve lookup semantics
|
||||
|
||||
### 5.2 Shard Assignment
|
||||
|
||||
Shard assignment MAY be based on:
|
||||
|
||||
* Hash of Canonical Key
|
||||
* Routing Key
|
||||
* Composite routing strategies
|
||||
|
||||
Shard selection MUST be deterministic per snapshot.
|
||||
|
||||
---
|
||||
|
||||
## 6. Hashing and Hash Recasting
|
||||
|
||||
### 6.1 Hashing
|
||||
|
||||
Hashes MAY be used for routing, filtering, or SIMD layout.
|
||||
|
||||
Hashes MUST NOT be treated as identity.
|
||||
|
||||
### 6.2 Hash Recasting
|
||||
|
||||
Hash recasting (changing hash functions or seeds) is permitted if:
|
||||
|
||||
1. It is deterministic
|
||||
2. It does not change Canonical Keys
|
||||
3. It does not affect index semantics
|
||||
|
||||
Recasting is equivalent to rebuilding acceleration structures.
|
||||
|
||||
---
|
||||
|
||||
## 7. SIMD Execution
|
||||
|
||||
SIMD operations MAY be used to:
|
||||
|
||||
* Evaluate filters
|
||||
* Compare routing keys
|
||||
* Accelerate scans
|
||||
|
||||
Rules:
|
||||
|
||||
* SIMD must operate only on immutable data
|
||||
* SIMD must not short-circuit semantic checks
|
||||
* SIMD must preserve deterministic behavior
|
||||
|
||||
---
|
||||
|
||||
## 8. Multi-Dimensional Routing Examples (Normative)
|
||||
|
||||
### 8.1 Artifact Index
|
||||
|
||||
* Canonical Key: `Reference`
|
||||
* Routing Key components:
|
||||
|
||||
* `H(Reference)`
|
||||
* `type_tag` (if present)
|
||||
* `has_typetag`
|
||||
|
||||
### 8.2 TGK Edge Index
|
||||
|
||||
* Canonical Key: defined by `TGK/1` and `TGK/1-CORE` (opaque here)
|
||||
* Routing Key components:
|
||||
|
||||
* `H(CanonicalEdgeKey)`
|
||||
* `EdgeTypeId` (if present in the TGK profile)
|
||||
* Direction or role (optional)
|
||||
|
||||
---
|
||||
|
||||
## 9. Snapshot Interaction
|
||||
|
||||
Acceleration structures:
|
||||
|
||||
* MUST respect snapshot visibility rules
|
||||
* MUST operate over the same sealed segments visible to the snapshot
|
||||
* MUST NOT bypass tombstones or shadowing
|
||||
|
||||
Snapshot cuts apply **after** routing and filtering.
|
||||
|
||||
---
|
||||
|
||||
## 10. Normative Invariants
|
||||
|
||||
1. Canonical Keys define identity and correctness
|
||||
2. Routing Keys are advisory only
|
||||
3. Filters may never introduce false negatives
|
||||
4. Sharding is observationally invisible
|
||||
5. Hashes are not identity
|
||||
6. SIMD is an execution strategy, not a semantic construct
|
||||
7. All acceleration is deterministic per snapshot
|
||||
|
||||
---
|
||||
|
||||
## 11. Non-Goals
|
||||
|
||||
ASL/INDEX-ACCEL/1 does not define:
|
||||
|
||||
* Specific filter algorithms
|
||||
* Memory layout
|
||||
* CPU instruction selection
|
||||
* Encoding formats
|
||||
* Federation policies
|
||||
|
||||
---
|
||||
|
||||
## 12. Summary
|
||||
|
||||
ASL/INDEX-ACCEL/1 establishes a strict contract:
|
||||
|
||||
> All acceleration exists to make the index faster, never different.
|
||||
|
||||
It formalizes Canonical vs Routing keys and constrains filters, sharding, hashing, and SIMD so that correctness is preserved under all optimizations.
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue