niklas/amduat
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: niklas:main
Branches Tags
niklas:main
niklas:niklas/lock-file-fix-anchor
niklas:apiv2
niklas:federation
niklas:qbits
niklas:burgen-fels
..
compare: niklas:burgen-fels
Branches Tags
niklas:main
niklas:niklas/lock-file-fix-anchor
niklas:apiv2
niklas:federation
niklas:qbits
niklas:burgen-fels

No commits in common. "main" and "burgen-fels" have entirely different histories.
main ... burgen-fel

198 changed files with 91 additions and 54814 deletions
Show stats Expand all files Collapse all files

6
.gitignore vendored
View file

@ -3,8 +3,6 @@ CMakeFiles/
CMakeCache.txt
cmake_install.cmake
Makefile
CTestTestfile.cmake
Testing/
*.o
*.a
*.so
@ -14,7 +12,3 @@ Testing/
*.exe
*.pdb
.DS_Store
tmp/
.amduat-asl
.codex
.claude

86
AUDITS.md
View file

@ -22,62 +22,6 @@ Verification notes:
- Prefer explicit commands and paths (e.g., `ctest --test-dir build`).
- If results are user-reported, note that explicitly.
Note: the filesystem ASL store (`asl_store_fs`) is a legacy convenience backend
and will be considered non-conformant to ASL index/log specs once the index/log
store is introduced. Audits for ASL index/log specs target the new backend only.
## Test Expectations (Planned)
These tests are planned to validate index/log behavior once implemented:
| Area | Example tests |
| --- | --- |
| Segment encoding | Round-trip encode/decode; CRC mismatch rejection; offset bounds checks |
| Log encoding | Hash-chain validation; unknown record type skip; truncated record rejection |
| Replay | Snapshot anchor + log replay determinism; segment seal visibility |
| Tombstones | Shadowing and lift across snapshots; domain-local shadowing rules |
| Visibility | CURRENT computed by `(SnapshotID, LogPosition)`; reverse seal-log order |
| Recovery | Crash with open segment; replay yields deterministic CURRENT |
## Spec Coverage (Implementation Status)
Status legend: ✅ implemented, 🟡 planned/in-progress, ⬜ not started.
| Spec | Status | Notes |
| --- | --- | --- |
| `ASL/1-CORE` | ✅ | Core artifact semantics implemented. |
| `ASL/1-STORE` | ✅ | Store semantics + fs backend. |
| `ENC/ASL1-CORE` | ✅ | Artifact/Reference encoding. |
| `HASH/ASL1` | ✅ | Hash registry + streaming API. |
| `PEL/1-CORE` | ✅ | Core execution semantics. |
| `PEL/1-SURF` | ✅ | Store-backed surface execution. |
| `PEL/PROGRAM-DAG/1` | ✅ | DAG scheme execution. |
| `PEL/PROGRAM-DAG-DESC/1` | ✅ | Scheme descriptor codec + wiring. |
| `ENC/PEL-PROGRAM-DAG/1` | ✅ | Program encoding. |
| `ENC/PEL1-RESULT/1` | ✅ | Result encoding. |
| `PEL/TRACE-DAG/1` | ✅ | Trace semantics + wiring. |
| `ENC/PEL-TRACE-DAG/1` | ✅ | Trace encoding. |
| `TGK/1-CORE` | ✅ | Edge semantics + validation. |
| `ENC/TGK1-EDGE/1` | ✅ | Edge encoding. |
| `TGK/STORE/1` | ✅ | Store semantics. |
| `TGK/PROV/1` | ✅ | Provenance operators. |
| `OPREG/PEL1-KERNEL` | ✅ | Kernel op registry. |
| `OPREG/PEL1-KERNEL-PARAMS/1` | ✅ | Kernel params encoding. |
| `AMDUAT20-STACK-OVERVIEW` | ✅ | Orientation surface aligned. |
| `ASL/1-CORE-INDEX` | ✅ | Index semantics + replay implemented. |
| `ASL/STORE-INDEX/1` | ✅ | Index/log store backend implemented (fs). |
| `ENC/ASL-CORE-INDEX/1` | ✅ | Segment encoding/decoding implemented. |
| `ASL/LOG/1` | ✅ | Log semantics implemented. |
| `ENC/ASL-LOG/1` | ✅ | Log encoding/decoding implemented. |
| `ASL/INDEX-ACCEL/1` | ✅ | Routing key + bloom/shard helpers implemented. |
| `ASL/INDEXES/1` | 🟡 | Taxonomy planned. |
| `ASL/TGK-EXEC-PLAN/1` | 🟡 | Encoding implemented; executor out of scope. |
| `ENC/ASL-TGK-EXEC-PLAN/1` | ✅ | Plan encoding implemented. |
| `ASL/FEDERATION/1` | ✅ | Core federation primitives implemented. |
| `ASL/FEDERATION-REPLAY/1` | ✅ | Deterministic replay and view construction implemented. |
| `ASL/SYSTEM/1` | 🟡 | Cross-cutting view planned. |
| `TGK/1` | 🟡 | Semantic layer planned. |
## Audit Plan
Status legend: ✅ completed, ⬜ pending.
@ -164,24 +108,6 @@ Status legend: ✅ completed, ⬜ pending.
- Tests: command not provided — pass (user reported “100% tests passed, 0 tests
failed out of 11”).
## 2026-01-18 — ASL index/log stack (`tier1/asl-core-index-1.md`, `tier1/asl-store-index-1.md`, `tier1/enc-asl-core-index-1.md`, `tier1/asl-log-1.md`, `tier1/enc-asl-log-1.md`)
- Scope: index semantics, filesystem index/log store, segment encoding, and log encoding/decoding paths.
- Findings: N/A (implemented components already present).
- Resolution: recorded implementation status and aligned routing/shard helpers with index usage.
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
## 2026-01-18 — ASL/INDEX-ACCEL/1 (`tier1/asl-index-accel-1.md`)
- Scope: routing key derivation, shard selection contract, bloom advisory behavior.
- Findings: missing formal routing-key API and tests around acceleration helpers.
- Resolution: added routing-key/shard helpers and tests for routing-key layout, shard determinism, and bloom advisory behavior.
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
## 2026-01-18 — ENC/ASL-TGK-EXEC-PLAN/1 (`tier1/enc-asl-tgk-exec-plan-1.md`)
- Scope: execution plan encoding/decoding; validation of operator IDs/inputs.
- Findings: encoding layer missing.
- Resolution: implemented encode/decode/free API and round-trip validation tests.
- Tests: `ctest --test-dir build` (user reported “100% tests passed, 0 tests failed out of 23”).
## 2025-12-22 — PEL/PROGRAM-DAG/1 (`tier1/pel-program-dag-1.md`)
- Scope: Exec_DAG semantics, structural validity, canonical order, diagnostics,
and scheme entrypoint correctness.
@ -350,15 +276,3 @@ Status legend: ✅ completed, ⬜ pending.
embedded commit-message appendix; tightened wording throughout; bumped the
document version/date.
- Tests: N/A (documentation-only change).
## 2026-02-XX — Federation core (`tier1/asl-federation-1.md`, `tier1/asl-federation-replay-1.md`)
- Scope: core federation registry, ingest validation, deterministic replay, view
construction, and resolve semantics.
- Findings: missing record typing and identity coverage for PER/TGK/tombstones;
policy gating needed explicit per-domain + optional per-record handling; view
build and resolve error reporting needed explicit codes and tests.
- Resolution: added federation registry storage, ingest validation, replay/view
build, resolve APIs, and tests for ordering, tombstone scoping, conflicts,
bounds, and metadata preservation; documented middle-layer boundary and
ref-only remote fetch guidance.
- Tests: user reported “100% tests passed, 0 tests failed out of 29”.

500
CMakeLists.txt
View file

@ -57,42 +57,17 @@ set(AMDUAT_UTIL_SRCS
src/internal/varint.c
src/internal/endian.c
src/internal/hex.c
src/internal/log.c
src/internal/string.c
)
set(AMDUAT_ASL_SRCS
src/kernel/asl/core.c
src/near_core/asl/collection_view.c
src/near_core/asl/none.c
src/near_core/asl/artifact_io.c
src/near_core/asl/identity.c
src/near_core/asl/io.c
src/near_core/asl/index_accel.c
src/near_core/asl/index_bloom.c
src/near_core/asl/index_snapshot.c
src/near_core/asl/index_replay.c
src/near_core/asl/log_cursor.c
src/near_core/asl/parse.c
src/near_core/asl/projection_checkpoint.c
src/near_core/asl/ref_io.c
src/near_core/asl/store.c
src/near_core/asl/ref_text.c
src/near_core/asl/registry.c
src/near_core/bytestore.c
src/near_core/fps.c
)
set(AMDUAT_ASL_LOG_STORE_SRCS
src/core/asl_log_store.c
)
set(AMDUAT_ASL_RECORD_SRCS
src/core/asl_record.c
)
set(AMDUAT_ASL_COLLECTION_SRCS
src/core/asl_collection.c
)
set(AMDUAT_HASH_ASL1_SRCS
@ -106,11 +81,6 @@ set(AMDUAT_ENC_SRCS
src/near_core/asl/ref_derive.c
src/near_core/enc/fer1_receipt.c
src/near_core/fer/receipt.c
src/near_core/enc/asl_log.c
src/near_core/enc/asl_core_index.c
src/near_core/enc/asl_tgk_exec_plan.c
src/near_core/enc/pcb1.c
src/near_core/enc/fcs1.c
src/near_core/enc/pel_program_dag.c
src/near_core/enc/pel_program_dag_desc.c
src/near_core/enc/pel_trace_dag.c
@ -128,7 +98,6 @@ set(AMDUAT_FORMAT_SRCS
set(AMDUAT_PEL_SRCS
src/kernel/pel/core.c
src/core/derivation_sid.c
src/pel_stack/decode.c
src/pel_stack/surf/surf.c
src/pel_stack/program_dag/program_dag.c
@ -137,7 +106,6 @@ set(AMDUAT_PEL_SRCS
src/pel_stack/trace_dag/trace_dag.c
src/pel_stack/queue/queue.c
src/pel_stack/opreg/kernel.c
src/pel_stack/opreg/kernel_collection.c
src/pel_stack/opreg/kernel_params.c
)
@ -147,41 +115,12 @@ set(AMDUAT_TGK_SRCS
src/tgk_stack/prov/prov.c
)
set(AMDUAT_FED_SRCS
src/near_core/fed/registry.c
src/near_core/fed/replay.c
src/near_core/fed/ingest.c
src/near_core/fed/view.c
)
set(AMDUAT_ASL_STORE_FS_SRCS
src/adapters/asl_store_fs/asl_store_fs.c
src/adapters/asl_store_fs/asl_store_fs_layout.c
src/adapters/asl_store_fs/asl_store_fs_meta.c
)
set(AMDUAT_ASL_STORE_INDEX_FS_SRCS
src/adapters/asl_store_index_fs/asl_store_index_fs.c
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.c
)
set(AMDUAT_ASL_STORE_MINIO_SRCS
src/adapters/asl_store_minio/asl_store_minio.c
src/adapters/asl_store_minio/s3_sigv4.c
)
set(AMDUAT_ASL_DERIVATION_INDEX_FS_SRCS
src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c
)
set(AMDUAT_ASL_MATERIALIZATION_CACHE_FS_SRCS
src/adapters/asl_materialization_cache_fs/asl_materialization_cache_fs.c
)
set(AMDUAT_ASL_POINTER_FS_SRCS
src/adapters/asl_pointer_fs/asl_pointer_fs.c
)
set(AMDUAT_TGK_STORE_MEM_SRCS
src/adapters/tgk_store_mem/tgk_store_mem.c
)
@ -190,10 +129,6 @@ set(AMDUAT_TGK_STORE_FS_SRCS
src/adapters/tgk_store_fs/tgk_store_fs.c
)
set(AMDUAT_TGK_STORE_ASL_INDEX_FS_SRCS
src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c
)
amduat_add_lib(util SRCS ${AMDUAT_UTIL_SRCS})
amduat_add_lib(asl SRCS ${AMDUAT_ASL_SRCS})
@ -209,68 +144,28 @@ amduat_add_lib(format SRCS ${AMDUAT_FORMAT_SRCS})
amduat_link(format amduat_asl amduat_enc amduat_util)
amduat_add_lib(pel SRCS ${AMDUAT_PEL_SRCS})
amduat_link(pel amduat_asl_materialization_cache_fs amduat_asl_collection
amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_link(pel amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(tgk SRCS ${AMDUAT_TGK_SRCS})
amduat_link(tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(fed SRCS ${AMDUAT_FED_SRCS})
amduat_link(fed amduat_asl)
amduat_add_lib(asl_store_fs SRCS ${AMDUAT_ASL_STORE_FS_SRCS})
amduat_link(asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_store_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_store_index_fs SRCS ${AMDUAT_ASL_STORE_INDEX_FS_SRCS})
amduat_link(asl_store_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_store_index_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_store_minio SRCS ${AMDUAT_ASL_STORE_MINIO_SRCS})
amduat_link(asl_store_minio amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_store_minio_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_derivation_index_fs SRCS ${AMDUAT_ASL_DERIVATION_INDEX_FS_SRCS})
amduat_link(asl_derivation_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_derivation_index_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_materialization_cache_fs SRCS ${AMDUAT_ASL_MATERIALIZATION_CACHE_FS_SRCS})
amduat_link(asl_materialization_cache_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_materialization_cache_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_pointer_fs SRCS ${AMDUAT_ASL_POINTER_FS_SRCS})
amduat_link(asl_pointer_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
target_compile_definitions(amduat_asl_pointer_fs_obj PRIVATE _POSIX_C_SOURCE=200809L)
amduat_add_lib(asl_log_store SRCS ${AMDUAT_ASL_LOG_STORE_SRCS})
amduat_link(asl_log_store amduat_asl_pointer_fs amduat_asl amduat_enc amduat_util)
amduat_add_lib(asl_record SRCS ${AMDUAT_ASL_RECORD_SRCS})
amduat_link(asl_record amduat_asl amduat_enc amduat_util)
amduat_add_lib(asl_collection SRCS ${AMDUAT_ASL_COLLECTION_SRCS})
amduat_link(asl_collection amduat_asl_log_store amduat_asl_record amduat_asl_pointer_fs
amduat_asl amduat_enc amduat_util)
amduat_add_lib(tgk_store_mem SRCS ${AMDUAT_TGK_STORE_MEM_SRCS})
amduat_link(tgk_store_mem amduat_tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(tgk_store_fs SRCS ${AMDUAT_TGK_STORE_FS_SRCS})
amduat_link(tgk_store_fs amduat_tgk_store_mem amduat_tgk amduat_asl_store_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
amduat_add_lib(tgk_store_asl_index_fs SRCS ${AMDUAT_TGK_STORE_ASL_INDEX_FS_SRCS})
target_include_directories(amduat_tgk_store_asl_index_fs_obj
PRIVATE ${AMDUAT_ROOT}/src/adapters/asl_store_index_fs
)
amduat_link(tgk_store_asl_index_fs amduat_tgk amduat_asl_store_index_fs amduat_asl amduat_enc amduat_hash_asl1 amduat_util)
add_executable(amduat_asl_cli src/tools/amduat_asl_cli.c)
target_include_directories(amduat_asl_cli
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_asl_cli
PRIVATE amduat_format amduat_asl_store_fs amduat_asl_store_index_fs
amduat_asl_derivation_index_fs amduat_asl amduat_enc
PRIVATE amduat_format amduat_asl_store_fs amduat_asl amduat_enc
amduat_hash_asl1 amduat_util
)
set_target_properties(amduat_asl_cli PROPERTIES OUTPUT_NAME amduat-asl)
@ -303,61 +198,11 @@ target_include_directories(amduat_pel_cli
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_pel_cli
PRIVATE amduat_format amduat_pel amduat_asl_store_fs
amduat_asl_collection amduat_asl_record amduat_asl_log_store
amduat_asl_pointer_fs
amduat_asl_derivation_index_fs amduat_asl amduat_enc
PRIVATE amduat_format amduat_pel amduat_asl_store_fs amduat_asl amduat_enc
amduat_hash_asl1 amduat_util
)
set_target_properties(amduat_pel_cli PROPERTIES OUTPUT_NAME amduat-pel)
add_executable(amduat_pel_build src/tools/amduat_pel_build.c)
target_include_directories(amduat_pel_build
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_pel_build
PRIVATE amduat_pel amduat_asl_store_fs amduat_asl amduat_enc amduat_util
)
set_target_properties(amduat_pel_build PROPERTIES OUTPUT_NAME amduat-pel-build)
add_executable(amduat_scriptlab_adapter src/tools/amduat_scriptlab_adapter.c)
target_include_directories(amduat_scriptlab_adapter
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_scriptlab_adapter
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_scriptlab_adapter
PRIVATE amduat_tgk_store_asl_index_fs amduat_asl_store_index_fs
amduat_asl_collection amduat_asl_record amduat_asl_derivation_index_fs
amduat_pel amduat_tgk amduat_asl amduat_enc amduat_hash_asl1
amduat_util pthread
)
set_target_properties(amduat_scriptlab_adapter
PROPERTIES OUTPUT_NAME amduat-scriptlab-adapter)
add_executable(amduat_conformance_run
src/tools/amduat_conformance_run.c
tests/conformance/asl_store/asl_store_conformance.c
tests/conformance/tgk_store/tgk_store_conformance.c
)
target_include_directories(amduat_conformance_run
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
PRIVATE ${AMDUAT_ROOT}
PRIVATE ${AMDUAT_ROOT}/tests/conformance/asl_store
PRIVATE ${AMDUAT_ROOT}/tests/conformance/tgk_store
)
target_link_libraries(amduat_conformance_run
PRIVATE amduat_asl_store_fs amduat_asl_store_index_fs
amduat_asl_store_minio
amduat_tgk_store_mem amduat_tgk_store_fs amduat_tgk_store_asl_index_fs
amduat_tgk amduat_asl amduat_enc amduat_hash_asl1 amduat_util
amduat_format
)
enable_testing()
add_executable(amduat_test_pel_program_dag tests/enc/test_pel_program_dag.c)
@ -366,7 +211,7 @@ target_include_directories(amduat_test_pel_program_dag
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel_program_dag
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME pel_program_dag COMMAND amduat_test_pel_program_dag)
@ -376,30 +221,17 @@ target_include_directories(amduat_test_pel_trace_dag
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel_trace_dag
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME pel_trace_dag COMMAND amduat_test_pel_trace_dag)
add_executable(amduat_test_pel_program_build_concat
tests/pel/test_pel_program_build_concat.c
)
target_include_directories(amduat_test_pel_program_build_concat
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel_program_build_concat
PRIVATE amduat_pel amduat_asl_store_fs amduat_asl amduat_enc
amduat_hash_asl1 amduat_util
)
add_test(NAME pel_program_build_concat COMMAND amduat_test_pel_program_build_concat)
add_executable(amduat_test_pel1_result tests/enc/test_pel1_result.c)
target_include_directories(amduat_test_pel1_result
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel1_result
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME pel1_result COMMAND amduat_test_pel1_result)
@ -410,69 +242,18 @@ target_include_directories(amduat_test_pel1_result_invariants
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel1_result_invariants
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME pel1_result_invariants
COMMAND amduat_test_pel1_result_invariants)
add_executable(amduat_test_asl_log tests/enc/test_asl_log.c)
target_include_directories(amduat_test_asl_log
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_log
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME asl_log COMMAND amduat_test_asl_log)
add_executable(amduat_test_asl_core_index tests/enc/test_asl_core_index.c)
target_include_directories(amduat_test_asl_core_index
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_core_index
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME asl_core_index COMMAND amduat_test_asl_core_index)
add_executable(amduat_test_asl_tgk_exec_plan
tests/enc/test_asl_tgk_exec_plan.c)
target_include_directories(amduat_test_asl_tgk_exec_plan
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_tgk_exec_plan
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME asl_tgk_exec_plan COMMAND amduat_test_asl_tgk_exec_plan)
add_executable(amduat_test_fps tests/enc/test_fps.c)
target_include_directories(amduat_test_fps
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fps
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME fps COMMAND amduat_test_fps)
add_executable(amduat_test_fcs_pcb1 tests/enc/test_fcs_pcb1.c)
target_include_directories(amduat_test_fcs_pcb1
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fcs_pcb1
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME fcs_pcb1 COMMAND amduat_test_fcs_pcb1)
add_executable(amduat_test_tgk1_edge tests/enc/test_tgk1_edge.c)
target_include_directories(amduat_test_tgk1_edge
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_tgk1_edge
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME tgk1_edge COMMAND amduat_test_tgk1_edge)
@ -482,7 +263,7 @@ target_include_directories(amduat_test_fer1_receipt
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fer1_receipt
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
PRIVATE amduat_enc amduat_hash_asl1 amduat_asl amduat_util
)
add_test(NAME fer1_receipt COMMAND amduat_test_fer1_receipt)
@ -496,17 +277,6 @@ target_link_libraries(amduat_test_tgk_store_mem
)
add_test(NAME tgk_store_mem COMMAND amduat_test_tgk_store_mem)
add_executable(amduat_test_tgk_error_taxonomy
tests/tgk/test_tgk_error_taxonomy.c)
target_include_directories(amduat_test_tgk_error_taxonomy
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_tgk_error_taxonomy
PRIVATE amduat_tgk
)
add_test(NAME tgk_error_taxonomy COMMAND amduat_test_tgk_error_taxonomy)
add_executable(amduat_test_tgk_store_fs tests/tgk/test_tgk_store_fs.c)
target_include_directories(amduat_test_tgk_store_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
@ -540,197 +310,6 @@ target_link_libraries(amduat_test_asl_ref_derive
)
add_test(NAME asl_ref_derive COMMAND amduat_test_asl_ref_derive)
add_executable(amduat_test_asl_store_indexed_ops
tests/asl/test_asl_store_indexed_ops.c)
target_include_directories(amduat_test_asl_store_indexed_ops
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_store_indexed_ops
PRIVATE amduat_asl
)
add_test(NAME asl_store_indexed_ops COMMAND amduat_test_asl_store_indexed_ops)
add_executable(amduat_test_asl_replay tests/asl/test_asl_replay.c)
target_include_directories(amduat_test_asl_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_replay
PRIVATE amduat_asl
)
add_test(NAME asl_replay COMMAND amduat_test_asl_replay)
add_executable(amduat_test_asl_index_replay
tests/asl/test_asl_index_replay.c)
target_include_directories(amduat_test_asl_index_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
PRIVATE ${AMDUAT_ROOT}/src/adapters/asl_store_index_fs
)
target_compile_definitions(amduat_test_asl_index_replay
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_index_replay
PRIVATE amduat_asl_store_index_fs
)
add_test(NAME asl_index_replay COMMAND amduat_test_asl_index_replay)
add_executable(amduat_test_asl_store_index_fs
tests/asl/test_asl_store_index_fs.c)
target_include_directories(amduat_test_asl_store_index_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_store_index_fs
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_store_index_fs
PRIVATE amduat_asl_store_index_fs amduat_format pthread
)
add_test(NAME asl_store_index_fs COMMAND amduat_test_asl_store_index_fs)
set_tests_properties(asl_store_index_fs PROPERTIES
ENVIRONMENT "AMDUAT_ASL_PERF_COUNT=100;AMDUAT_ASL_STRESS_SECS=5"
TIMEOUT 120
)
add_executable(amduat_test_asl_log_store_index_fs
tests/asl/test_asl_log_store_index_fs.c)
target_include_directories(amduat_test_asl_log_store_index_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_log_store_index_fs
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_log_store_index_fs
PRIVATE amduat_asl_log_store amduat_asl_store_index_fs amduat_format pthread
)
add_test(NAME asl_log_store_index_fs COMMAND amduat_test_asl_log_store_index_fs)
set_tests_properties(asl_log_store_index_fs PROPERTIES TIMEOUT 30)
add_executable(amduat_test_asl_index_put_get_consistency
tests/asl/test_asl_index_put_get_consistency.c)
target_include_directories(amduat_test_asl_index_put_get_consistency
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_index_put_get_consistency
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_index_put_get_consistency
PRIVATE amduat_asl_store_index_fs amduat_format pthread
)
add_test(NAME asl_index_put_get_consistency
COMMAND amduat_test_asl_index_put_get_consistency)
add_executable(amduat_test_asl_index_accel tests/asl/test_asl_index_accel.c)
target_include_directories(amduat_test_asl_index_accel
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_index_accel
PRIVATE amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME asl_index_accel COMMAND amduat_test_asl_index_accel)
add_executable(amduat_test_asl_log_encoding
tests/asl/test_asl_log_encoding.c)
target_include_directories(amduat_test_asl_log_encoding
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_log_encoding
PRIVATE amduat_enc amduat_asl amduat_hash_asl1 amduat_util
)
add_test(NAME asl_log_encoding COMMAND amduat_test_asl_log_encoding)
add_executable(amduat_test_asl_error_taxonomy
tests/asl/test_asl_error_taxonomy.c)
target_include_directories(amduat_test_asl_error_taxonomy
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_asl_error_taxonomy
PRIVATE amduat_asl
)
add_test(NAME asl_error_taxonomy COMMAND amduat_test_asl_error_taxonomy)
add_executable(amduat_test_asl_derivation_index_fs
tests/asl/test_asl_derivation_index_fs.c)
target_include_directories(amduat_test_asl_derivation_index_fs
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_derivation_index_fs
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_derivation_index_fs
PRIVATE amduat_asl_derivation_index_fs
)
add_test(NAME asl_derivation_index_fs
COMMAND amduat_test_asl_derivation_index_fs)
add_executable(amduat_test_asl_projection_replay
tests/asl/test_asl_projection_replay.c)
target_include_directories(amduat_test_asl_projection_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_asl_projection_replay
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_projection_replay
PRIVATE amduat_asl_store_index_fs amduat_asl_record amduat_asl
amduat_enc amduat_hash_asl1 amduat_util pthread
)
add_test(NAME asl_projection_replay
COMMAND amduat_test_asl_projection_replay)
add_executable(amduat_test_asl_store_conformance
tests/conformance/asl_store/test_asl_store_conformance.c
tests/conformance/asl_store/asl_store_conformance.c
)
target_include_directories(amduat_test_asl_store_conformance
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
PRIVATE ${AMDUAT_ROOT}
PRIVATE ${AMDUAT_ROOT}/tests/conformance/asl_store
)
target_compile_definitions(amduat_test_asl_store_conformance
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_asl_store_conformance
PRIVATE amduat_asl_store_fs amduat_asl_store_index_fs
amduat_asl_store_minio
amduat_asl amduat_enc amduat_hash_asl1 amduat_util
)
add_test(NAME asl_store_conformance
COMMAND amduat_test_asl_store_conformance
)
add_executable(amduat_test_tgk_store_conformance
tests/conformance/tgk_store/test_tgk_store_conformance.c
tests/conformance/tgk_store/tgk_store_conformance.c
)
target_include_directories(amduat_test_tgk_store_conformance
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
PRIVATE ${AMDUAT_ROOT}
PRIVATE ${AMDUAT_ROOT}/tests/conformance/tgk_store
)
target_compile_definitions(amduat_test_tgk_store_conformance
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_tgk_store_conformance
PRIVATE amduat_tgk_store_mem amduat_tgk_store_fs amduat_tgk_store_asl_index_fs
amduat_asl_store_fs amduat_asl_store_index_fs amduat_tgk amduat_asl
amduat_enc amduat_hash_asl1 amduat_util amduat_format
)
add_test(NAME tgk_store_conformance
COMMAND amduat_test_tgk_store_conformance
)
add_executable(amduat_test_pel_program_dag_exec
tests/pel/test_pel_program_dag_exec.c)
target_include_directories(amduat_test_pel_program_dag_exec
@ -738,7 +317,7 @@ target_include_directories(amduat_test_pel_program_dag_exec
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel_program_dag_exec
PRIVATE amduat_pel amduat_asl_store_fs
PRIVATE amduat_pel
)
add_test(NAME pel_program_dag_exec COMMAND amduat_test_pel_program_dag_exec)
@ -770,7 +349,7 @@ target_include_directories(amduat_test_pel_surf_run
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_pel_surf_run
PRIVATE amduat_pel amduat_asl_store_fs
PRIVATE amduat_pel
)
add_test(NAME pel_surf_run COMMAND amduat_test_pel_surf_run)
@ -786,60 +365,3 @@ target_link_libraries(amduat_test_pel_queue
PRIVATE amduat_pel
)
add_test(NAME pel_queue COMMAND amduat_test_pel_queue)
add_executable(amduat_test_fed_registry tests/fed/test_fed_registry.c)
target_include_directories(amduat_test_fed_registry
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_registry
PRIVATE amduat_fed
)
add_test(NAME fed_registry COMMAND amduat_test_fed_registry)
add_executable(amduat_test_fed_replay tests/fed/test_fed_replay.c)
target_include_directories(amduat_test_fed_replay
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_replay
PRIVATE amduat_fed
)
add_test(NAME fed_replay COMMAND amduat_test_fed_replay)
add_executable(amduat_test_fed_ingest tests/fed/test_fed_ingest.c)
target_include_directories(amduat_test_fed_ingest
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_ingest
PRIVATE amduat_fed
)
add_test(NAME fed_ingest COMMAND amduat_test_fed_ingest)
add_executable(amduat_test_fed_view tests/fed/test_fed_view.c)
target_include_directories(amduat_test_fed_view
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_link_libraries(amduat_test_fed_view
PRIVATE amduat_fed
)
add_test(NAME fed_view COMMAND amduat_test_fed_view)
add_executable(amduat_test_scriptlab_adapter
tests/substrate/test_scriptlab_adapter.c)
target_include_directories(amduat_test_scriptlab_adapter
PRIVATE ${AMDUAT_INTERNAL_DIR}
PRIVATE ${AMDUAT_INCLUDE_DIR}
)
target_compile_definitions(amduat_test_scriptlab_adapter
PRIVATE _POSIX_C_SOURCE=200809L
)
target_link_libraries(amduat_test_scriptlab_adapter
PRIVATE amduat_tgk_store_asl_index_fs amduat_asl_store_index_fs
amduat_asl_collection amduat_asl_record amduat_asl_derivation_index_fs
amduat_pel amduat_tgk amduat_asl amduat_enc amduat_hash_asl1
amduat_util pthread
)
add_test(NAME scriptlab_adapter COMMAND amduat_test_scriptlab_adapter)

14
README.md
View file

@ -65,20 +65,6 @@ status and refs are printed to stderr.
when not using `--output-raw`.
- The filesystem ASL store layout expects digests at least 2 bytes long
(two directory levels). Experimental shorter digests need a different store.
- The filesystem ASL store (`amduat-asl ... --root`) is a legacy convenience
backend; once the index/log store is introduced it is considered
non-conformant to ASL index/log specs and should be used only for quickstart
demos.
- Compatibility & migration: existing `asl_store_fs` stores will not be
automatically upgraded. Plan to re-ingest artifacts into the index/log store
when it lands.
## Documentation
- Implementation clarifications: `docs/spec-clarifications.md`
- Spec coverage matrix: `AUDITS.md` (Spec Coverage section)
- Index/log API sketch: `docs/index-log-api-sketch.md`
- Federation core API: `include/amduat/fed/` (registry, ingest, replay, view)
## PEL reference

3
Testing/Temporary/LastTest.log Normal file
View file

@ -0,0 +1,3 @@
Start testing: Dec 21 23:21 CET
----------------------------------------------------------
End testing: Dec 21 23:21 CET

198
docs/SUBSTRATE_CONTRACT_V0_1.md
View file

@ -1,198 +0,0 @@
# Substrate Contract v0.1
Status: Draft (repo-grounded)
Scope: Defines the substrate surfaces and invariants implemented in this repo for v0.1.
This contract is grounded in the current Amduat code and specs. Each requirement
below lists file references that define the behavior. If a line is marked **NEW**,
it is introduced in this change and implemented in code (referenced below).
---
## 1. Artifact Model (ASL/1-CORE)
1. **Artifact** = bytes + optional type tag; identity is content-addressed and immutable.
Sources: `tier1/asl-1-core.md`, `include/amduat/asl/core.h`.
2. **Reference** = `(hash_id, digest)`; equality is hash-id + digest equality.
Sources: `tier1/asl-1-core.md`, `include/amduat/asl/core.h`.
3. **Reference derivation** uses canonical encoding plus the ASL1 domain
separator (`"AMDUAT:ASL1:ART\0"`) and hash family; the API exposes this via
`amduat_asl_ref_derive`.
Sources: `tier1/asl-1-core.md`, `include/amduat/asl/ref_derive.h`,
`include/amduat/asl/identity.h`.
4. **Hash family** in use for the stack is HASH/ASL1 (e.g., SHA-256), with
registry/version metadata exposed in the core API.
Sources: `tier1/hash-asl1.md`, `include/amduat/hash/asl1.h`.
---
## 2. Record Artifacts (ASL records)
1. ASL records are artifacts with type tag `TYPE_TAG_ASL_RECORD_1` and a
payload encoding defined by `amduat_asl_record_encode_v1`/`decode_v1`.
Sources: `include/amduat/asl/record.h`, `src/core/asl_record.c`.
2. **Record schema is required and ASCII**; payload may be empty but must be
well-formed per the encoder rules.
Sources: `src/core/asl_record.c`.
3. Records are stored and retrieved via `amduat_asl_record_store_put/get`, which
wrap ASL store `put/get` over the encoded record bytes.
Sources: `include/amduat/asl/record.h`, `src/core/asl_record.c`, `include/amduat/asl/store.h`.
---
## 3. Authoritative CAS + Index/Log (ASL/1-STORE)
1. The authoritative content-addressed store surface is `amduat_asl_store_t` with
`put/get` and indexed variants `put_indexed/get_indexed`.
Sources: `include/amduat/asl/store.h`.
2. Index/log state is represented as `amduat_asl_index_state_t` with
`snapshot_id` and `log_position`.
Sources: `include/amduat/asl/store.h`, `docs/index-log-api-sketch.md`.
3. Index/log record types are encoded as ASL log records, including artifact
publish/unpublish, snapshot anchors, and tombstones.
Sources: `include/amduat/enc/asl_log.h`, `tier1/asl-log-1.md`.
4. The filesystem index/log adapter implements the indexed store surface and
emits log records with monotonically increasing `logseq` values.
Sources: `include/amduat/asl/asl_store_index_fs.h`, `src/adapters/asl_store_index_fs/asl_store_index_fs.c`.
---
## 4. Derivation / Provenance Index
1. Derivation records bind an output artifact ref to a program ref, input refs,
output index, and optional params/exec-profile.
Sources: `include/amduat/asl/asl_derivation_index_fs.h`.
2. Deterministic derivation SIDs are computed from program + inputs (+ optional
params/profile) via `amduat_pel_derivation_sid_compute`.
Sources: `include/amduat/pel/derivation_sid.h`, `src/core/derivation_sid.c`.
3. The filesystem derivation index stores and lists derivation records by
artifact reference.
Sources: `include/amduat/asl/asl_derivation_index_fs.h`,
`src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c`.
---
## 5. TGK Projection (Edges as Artifacts)
1. TGK edges are artifacts encoded via `ENC/TGK1-EDGE/1` with type tag
`TYPE_TAG_TGK1_EDGE_V1` and edge bodies containing `from`, `to`, and `payload` refs.
Sources: `include/amduat/enc/tgk1_edge.h`, `include/amduat/tgk/core.h`.
2. The TGK store surface provides graph queries (edges_from/to/scan/neighbors)
over projected edges.
Sources: `include/amduat/tgk/store.h`, `tier1/tgk-store-1.md`.
3. `amduat_tgk_store_asl_index_fs` implements a TGK projection over the ASL
index/log store.
Sources: `include/amduat/tgk/tgk_store_asl_index_fs.h`,
`src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c`.
---
## 6. Deterministic Transforms (PEL)
1. PEL surface execution results are stored as artifacts with type tag
`TYPE_TAG_PEL1_RESULT_1`, encoding `PEL_ENC_EXECUTION_RESULT_V1`.
Sources: `include/amduat/enc/pel1_result.h`, `include/amduat/pel/surf.h`.
2. Trace DAGs and Program DAGs are encoded as artifacts with their respective
type tags and encoding profiles.
Sources: `include/amduat/enc/pel_trace_dag.h`, `include/amduat/enc/pel_program_dag.h`.
3. A minimal job record for deterministic transforms is the tuple:
`(program_ref, input_refs, params_ref?, exec_profile?, result_ref, trace_ref?)`.
Sources: `include/amduat/pel/surf.h`, `include/amduat/enc/pel1_result.h`,
`include/amduat/pel/derivation_sid.h`.
---
## 7. Projection Hooks + Checkpoints (for external projectors)
1. **Projection feed cursor** is the ASL log position (`amduat_asl_log_position_t`)
from `amduat_asl_index_state_t` and log records are read via the ASL log API.
Sources: `include/amduat/asl/store.h`, `include/amduat/enc/asl_log.h`.
2. **NEW:** `amduat_asl_log_read_since` provides a cursor-based view of log
records (records with `logseq` > cursor), enabling deterministic replay from
a stable log position.
Sources: `include/amduat/asl/log_cursor.h`, `src/near_core/asl/log_cursor.c`.
3. **NEW:** Projection checkpoints are encoded via
`amduat_asl_projection_checkpoint_encode_v1/decode_v1` and stored as ASL
record artifacts. The v0.1 schema is derived from `docs/projection-checkpoint-schema.md`.
Sources: `include/amduat/asl/projection_checkpoint.h`,
`src/near_core/asl/projection_checkpoint.c`, `docs/projection-checkpoint-schema.md`,
`include/amduat/asl/record.h`.
4. **NEW (v0.1 convention):** Checkpoint record schema string is
`projection/checkpoint` and the record payload is the v1 checkpoint encoding.
Sources: `src/tools/amduat_scriptlab_adapter.c`, `tests/substrate/test_scriptlab_adapter.c`,
`include/amduat/asl/record.h`.
See `docs/projector-pattern.md` for the canonical cursor→project→checkpoint loop
using `amduat_asl_log_read_since(store, after_logseq, limit)` and
`projection/checkpoint` ASL records.
---
## 8. Non-Authoritative Projections
1. External projections (e.g., Elasticsearch, IPFS, JanusGraph) are derived and
must remain observationally equivalent to the ASL/TGK substrate.
Sources: `docs/amduat-oss-backend-proposal.md`, `docs/indexing-requirements-and-projections.md`.
---
## 9. Definition of Done (v0.1)
1. CAS, index/log store, and derivation index are available via the C surfaces
and filesystem adapters.
Sources: `include/amduat/asl/store.h`, `include/amduat/asl/asl_store_index_fs.h`,
`include/amduat/asl/asl_derivation_index_fs.h`.
2. TGK projection is available over ASL index/log store.
Sources: `include/amduat/tgk/tgk_store_asl_index_fs.h`.
3. PEL execution artifacts are produced and encoded as artifacts and can be
linked via derivation SIDs.
Sources: `include/amduat/pel/surf.h`, `include/amduat/enc/pel1_result.h`,
`include/amduat/pel/derivation_sid.h`.
4. **NEW:** Projection feed cursor + checkpoint artifacts are implemented and
validated by the scriptlab adapter flow.
Sources: `include/amduat/asl/log_cursor.h`, `include/amduat/asl/projection_checkpoint.h`,
`tests/substrate/test_scriptlab_adapter.c`.
---
## 10. Layer 1 Dependencies (Allowed vs Disallowed)
**Layer 1 (higher API) MAY depend on:**
1. ASL core and store surfaces (`amduat/asl/core.h`, `amduat/asl/store.h`).
2. ASL index/log state + replay (`amduat/asl/index_replay.h`, `amduat/enc/asl_log.h`).
3. Derivation/provenance index (`amduat/asl/asl_derivation_index_fs.h`,
`amduat/pel/derivation_sid.h`).
4. PEL deterministic artifacts and surface results (`amduat/pel/surf.h`,
`amduat/enc/pel1_result.h`).
5. TGK edge artifacts + TGK store projection (`amduat/enc/tgk1_edge.h`,
`amduat/tgk/store.h`, `amduat/tgk/tgk_store_asl_index_fs.h`).
6. Projection replay primitives (cursor + checkpoint helpers)
(`amduat/asl/log_cursor.h`, `amduat/asl/projection_checkpoint.h`).
**Layer 1 MUST NOT depend on:**
1. CLI tools (`src/tools/*`) as APIs.
2. Any networking, auth, tenancy, or server layers (none are provided).
3. App/domain schemas beyond raw ASL records (records may carry schema bytes,
but core does not interpret them).

98
docs/adapter-error-taxonomy.md
View file

@ -1,98 +0,0 @@
# Adapter Error Taxonomy
Status: Draft
Scope: Canonical error mapping for ASL and TGK adapters and derived projections, used by conformance tests to prevent drift.
---
# 1. Purpose
This taxonomy defines a canonical internal error set. Adapters MUST map backend-specific errors into these categories so that conformance tests remain stable across backends.
---
# 2. Canonical Error Set
## 2.1 ASL Store Errors
Canonical enum (from `include/amduat/asl/store.h`):
1. `AMDUAT_ASL_STORE_ERR_INTEGRITY`
2. `AMDUAT_ASL_STORE_ERR_UNSUPPORTED`
3. `AMDUAT_ASL_STORE_ERR_NOT_FOUND`
4. `AMDUAT_ASL_STORE_ERR_IO`
## 2.2 TGK Graph Store Errors
Canonical enum (from `include/amduat/tgk/store.h`):
1. `GS_ERR_NOT_EDGE`
2. `GS_ERR_ARTIFACT_ERROR`
3. `GS_ERR_UNSUPPORTED`
4. `GS_ERR_INTEGRITY`
## 2.3 Projection Errors (Derived Indexers)
Canonical categories:
1. NotFound
2. PermissionDenied
3. TransientIO
4. InvalidDocument
5. InternalError
Projection errors must never be surfaced as authoritative ASL or TGK errors.
---
# 3. Mapping Rules
## 3.1 ASL/1-STORE Mapping
1. `ERR_NOT_FOUND` → `AMDUAT_ASL_STORE_ERR_NOT_FOUND`
2. `ERR_INTEGRITY` → `AMDUAT_ASL_STORE_ERR_INTEGRITY`
3. `ERR_UNSUPPORTED` → `AMDUAT_ASL_STORE_ERR_UNSUPPORTED`
4. Backend I/O or transport failure → `AMDUAT_ASL_STORE_ERR_IO`
Notes:
1. `ASL/1-STORE` does not standardize I/O or auth errors; those must map to `ERR_IO` for conformance tests.
2. A `put` collision on identical reference with different bytes MUST map to `ERR_INTEGRITY`.
## 3.2 TGK/STORE/1 Mapping
1. Reference resolves to non-edge artifact → `GS_ERR_NOT_EDGE`
2. Artifact-layer integrity failure during resolve → `GS_ERR_ARTIFACT_ERROR`
3. Unsupported profile/encoding → `GS_ERR_UNSUPPORTED`
4. Detected graph-level corruption → `GS_ERR_INTEGRITY`
Notes:
1. `GS_ERR_ARTIFACT_ERROR` is required for artifact-level conflicts; do not use `GS_ERR_INTEGRITY`.
2. Backend-specific errors must be normalized to the canonical TGK error set.
## 3.3 Projection Adapter Mapping
1. Index missing doc → NotFound
2. Authorization failure → PermissionDenied
3. Timeout, transport, or retryable failure → TransientIO
4. Schema or decode failure → InvalidDocument
5. Unknown failure → InternalError
Projection errors are operational only and must not be treated as ASL/TGK semantic failures.
---
# 4. Test Requirements
Conformance tests MUST include:
1. ASL: `get` on missing ref → `ERR_NOT_FOUND`.
2. ASL: `put` collision → `ERR_INTEGRITY`.
3. ASL: unsupported config → `ERR_UNSUPPORTED`.
4. ASL: I/O failure path → `ERR_IO`.
5. TGK: `resolve_edge` on non-edge → `GS_ERR_NOT_EDGE`.
6. TGK: artifact integrity conflict → `GS_ERR_ARTIFACT_ERROR`.
7. TGK: unsupported encoding/profile → `GS_ERR_UNSUPPORTED`.
---
# 5. Non-Goals
1. Backend-specific error codes.
2. Retry policy definitions.
3. Human-readable error messaging conventions.

50
docs/adr-031-asl-tgk-index-separation.md
View file

@ -1,50 +0,0 @@
# ADR-031 — ASL + TGK Index Separation
Status: Accepted
Date: 2026-02-21
Decision Owner: Architecture
## Context
ASL index/log provide artifact visibility, replay, and tombstone semantics.
TGK store provides graph query semantics over TGK edge artifacts. We must decide
whether to **merge** ASL and TGK indexes into one unified index structure, or
keep them separated with explicit cross-layer invariants.
## Decision
We **reject** a merged ASL+TGK index.
ASL index/log and TGK index remain **separate layers** with strict invariants
that keep them consistent.
## Rationale
* ASL index is about **artifact visibility and replay**; TGK index is about
**graph query acceleration**. Their update streams and invariants differ.
* Keeping them separate preserves clean layering and deterministic replay.
* TGK indexes can be rebuilt or swapped without changing ASL semantics.
* A merged index risks coupling graph query policies with storage visibility,
complicating federation and policy enforcement.
## Invariants (Normative)
1. **Visibility alignment:** TGK indexes MUST only index artifacts visible under
ASL log replay at the same `(snapshot_id, log_position)`.
2. **Tombstone enforcement:** TGK indexes MUST respect ASL tombstones and
tombstone lifts at their `logseq`.
3. **No new identity:** TGK nodes/edges MUST use ASL `Reference` identities;
TGK MUST NOT introduce a distinct object identity scheme.
4. **Deterministic derivation:** TGK index updates MUST be deterministically
derived from ASL-admitted records (segments/log). No side-channel writes.
5. **Replay equivalence:** Replaying ASL log + sealed segments MUST yield the
same TGK index state (up to encoding) as an online build.
6. **Snapshot consistency:** TGK queries at a snapshot MUST be consistent with
ASL `IndexState` for that snapshot.
## Consequences
* ASL and TGK index encodings remain separate specs.
* Conformance tests for ASL and TGK can evolve independently, with explicit
cross-layer checks for the invariants above.
* TGK index rebuilds remain possible without touching ASL log/segment formats.

351
docs/amduat-oss-backend-proposal.md
View file

@ -1,351 +0,0 @@
# Amduat Open-Source Backend Proposal
## Contract-First Backend Integration Architecture
**Status:** Proposal
**Scope:** Define how the OSS stack (MinIO, JanusGraph, Cassandra, Elasticsearch, Spark, Nessie, IPFS, Ollama) can be deployed as a backend implementation of Amduat while preserving semantic invariants.
---
# Related Documents
* `docs/implementation-plan.md`
* `docs/phase0-1-execution-checklist.md`
* `docs/decision-checklist.md`
* `docs/conformance-harness-spec.md`
* `docs/adapter-error-taxonomy.md`
* `docs/tgk-ordering-enforcement.md`
* `docs/indexing-requirements-and-projections.md`
* `docs/es-indexing-plan.md`
* `docs/projection-checkpoint-schema.md`
* `docs/oss-backend-adapter-boundary.md`
---
# 1. Executive Summary
This document proposes integrating a scalable open-source infrastructure stack as a backend implementation of Amduat.
The proposal preserves Amduat’s defining principle:
> The semantic core is invariant.
> Backends are interchangeable implementations of store/query surfaces.
The OSS stack provides scalability, distribution, analytics, and optional acceleration layers without altering the deterministic semantics defined by the Amduat core specifications.
---
# 2. Architectural Position
## 2.1 Semantic Core
Core semantics are defined by:
* `ASL/1-CORE`
* `PEL/1-CORE`
* `TGK/1-CORE`
Canonical encodings are provided by:
* `ENC/ASL1-CORE` (and related encoding profiles)
Identity derivation is defined by:
* `ASL/1-CORE`
* `HASH/ASL1`
These define:
* Artifact identity and immutability
* Core execution semantics
* Core graph semantics
* Deterministic projection rules
These are deployment-invariant.
## 2.2 Store / Query Surfaces
Operational interfaces are defined by:
* `ASL/1-STORE`
* `TGK/STORE/1`
Optional provenance semantics:
* `TGK/PROV/1`
Storage and indexing layers are pluggable implementations of these surfaces.
They do not redefine core semantics.
---
# 3. No Semantic Drift Rule
To preserve Amduat’s portability and determinism, all backend implementations must adhere to the following:
1. **No Backend-Specific Semantics**
* No interpretation of artifacts beyond canonical encoding.
* No implicit schema expansion.
* No hidden type coercion.
2. **No Semantic Elevation of Projections**
* Search indexes (e.g., Elasticsearch)
* Distribution caches (e.g., IPFS)
* Analytics views (e.g., Spark)
* AI-generated outputs
…are derived projections unless explicitly promoted by governance.
3. **Deterministic Reproducibility**
* The logical TGK graph must be reproducible from ASL state.
* Snapshot exports must reproduce identical graph projections.
4. **Surface Conformance Only**
* The OSS stack may optimize storage and indexing.
* It may not redefine core behavior.
5. **Observational Equivalence**
* Any backend-specific optimization must be observationally equivalent to the core semantics and query results defined by the specifications.
This rule guarantees backend substitution without semantic drift.
---
# 4. ASL/1-STORE Implementation (Artifact Layer)
## 4.1 Contract
Must implement:
* `put(Artifact) -> Reference`
* `get(Reference) -> Artifact | NotFound`
Identity derivation follows:
1. Canonical encoding (`ENC/ASL1-CORE`)
2. Identity derivation rules (`ASL/1-CORE`)
3. Hash function (`HASH/ASL1`)
In abstract form:
```
Reference = HASH( CANONICAL_ENCODE(Artifact) )
```
Reference derivation is deterministic and caller-independent.
## 4.2 Immutability
* Artifacts are immutable.
* Rewriting the same Artifact yields the same Reference.
* Mutation semantics do not exist at this layer.
## 4.3 Authority Rule
The authoritative store in any deployment is the backend implementation that conforms to `ASL/1-STORE`.
`ASL/1-STORE` is a specification; authority resides in whichever concrete backend implements it.
Caches and projections derive from this authoritative ASL implementation unless explicitly promoted by governance.
## 4.4 Proposed Backend
Primary implementation:
* MinIO-backed object storage
Optional:
* IPFS as distribution/cache layer
---
# 5. TGK/STORE/1 Implementation (Graph Query Surface)
## 5.1 Projection Model
The TGK graph is a logical projection derived from:
* Artifacts stored via ASL
* Core graph semantics defined in `TGK/1-CORE`
It is not an independently authoritative database.
Graph state must be reproducible from ASL state.
## 5.2 Required Operations
Backends must implement the required query operations and error model defined in `TGK/STORE/1`.
This proposal references the specification directly to avoid semantic drift through re-enumeration.
## 5.3 Read-Only Nature
* `GraphStoreInstance` is logically read-only.
* Edges arise from writing EdgeArtifacts to ASL.
* Deletion, compaction, or tombstone policies are backend or profile-level implementation concerns and are not part of `TGK/STORE/1`.
## 5.4 Proposed Backend
* JanusGraph as query engine
* Cassandra as storage backend
* Optional Elasticsearch as search projection
JanusGraph functions as an implementation detail of the `TGK/STORE/1` surface.
---
# 6. TGK/PROV/1 (Optional Provenance Layer)
`TGK/PROV/1` defines pure functions over a `ProvenanceGraph`.
* It does not require a particular store.
* It is evaluated against `TGK/1-CORE` graph semantics.
* Implementations may use `TGK/STORE/1` queries to realize provenance functions.
Provenance is layered above store semantics and does not alter core behavior.
Implementation options:
* Native traversal realization
* Snapshot-based traversal in analytics engines
* Hybrid realization
---
# 7. Projection & Acceleration Layers
Unless explicitly promoted by governance:
* Elasticsearch
* IPFS
* Lakehouse datasets (MinIO + Nessie)
* Spark analytics outputs
* AI-derived artifacts
…are derived projections.
Authoritative truth resides in:
* The deployed `ASL/1-STORE` implementation
* Deterministic TGK projection derived from ASL state
Projections may be eventually consistent.
Audit-grade claims must reference reproducible ASL state or snapshot identifiers.
---
# 8. Capability Tiers
Capability tiers define operational envelope, not semantic differences.
---
## Tier 0 — Embedded / Edge
Implements:
* `ASL/1-STORE`
* `TGK/STORE/1`
* Optional minimal `TGK/PROV/1`
Characteristics:
* Single-node
* Deterministic local projection
* Strong local consistency
* Optional sync/export
Use cases:
* Personal knowledge systems
* Offline assistant systems
---
## Tier 1 — Platform / Scale-Out
Implements:
* `ASL/1-STORE`
* `TGK/STORE/1`
* Optional `TGK/PROV/1`
Typical infrastructure:
* Distributed object store (e.g., MinIO)
* Distributed graph backend (e.g., JanusGraph over Cassandra)
* Optional derived search index
Characteristics:
* Multi-node
* Horizontal scaling
* Operational consistency
* Deterministic core preserved
Use cases:
* Community platforms
* Shared knowledge systems
---
## Tier 2 — Enterprise / Audit-Grade
Includes Tier 1 plus:
* Snapshot export capability
* Lakehouse projection
* Versioned snapshots (e.g., Nessie)
* Governance workflows
* Promotion pipelines
Characteristics:
* Snapshot-based reproducibility
* Audit traceability
* Multi-environment promotion
Use cases:
* Enterprise backbone systems
* Regulated environments
---
# 9. Conformance Requirements
Each backend must:
* Pass `ASL/1-STORE` conformance tests
* Pass `TGK/STORE/1` conformance and deterministic ordering tests
* Demonstrate projection reproducibility from ASL state
* Preserve encoding stability across versions
Tier 2 must additionally:
* Validate snapshot equivalence
* Demonstrate replay determinism from ASL state
Conformance is versioned alongside spec revisions.
---
# 10. Design Outcome
This architecture ensures:
* One invariant semantic kernel
* Backend substitution without semantic drift
* Scaling through infrastructure choice
* Deterministic reproducibility
* Clear separation between authority and projection layers
* Portability from embedded edge systems to enterprise-scale deployments
The OSS stack is therefore a backend implementation of Amduat — not a semantic authority.

165
docs/conformance-harness-spec.md
View file

@ -1,165 +0,0 @@
# Conformance Harness Specification
Status: Draft
Scope: Define the Phase 0 conformance harness goals, backend matrix, fixture format, and pass/fail gates for ASL and TGK store surfaces.
---
# 1. Purpose
The harness ensures that all backend adapters implement existing store semantics without drift.
It MUST:
1. Run the same test vectors across multiple backends.
2. Enforce deterministic ordering rules.
3. Validate replay equivalence.
4. Provide a single command entry point for CI.
---
# 2. Targets and Scope
## 2.1 Surfaces Under Test
1. ASL/1-STORE
2. TGK/STORE/1
Optional extensions:
1. ASL/STORE-INDEX/1 (index/log store)
2. TGK/PROV/1 (if provenance surface is exposed)
## 2.2 Conformance Mode
All tests are:
1. Deterministic.
2. Backend-agnostic.
3. Semantics-first (no timing assumptions).
---
# 3. Backend Matrix
The harness MUST be able to run each test suite against:
## 3.1 ASL Backends
1. `asl_store_fs`
2. `asl_store_index_fs`
3. Future: `asl_store_minio`
## 3.2 TGK Backends
1. `tgk_store_mem`
2. `tgk_store_fs`
3. `tgk_store_asl_index_fs`
4. Future: `tgk_store_janus`
---
# 4. Canonical Fixture Format
Fixtures are the source of truth for all conformance tests.
## 4.1 Requirements
1. Versioned fixture schema.
2. Stable ordering of artifacts and edges.
3. Explicit references encoded as canonical `ReferenceBytes` hex.
4. Deterministic content generation (no randomness at runtime).
## 4.2 Minimal Fields
For ASL fixtures:
1. `artifact_bytes` (hex or base64, fixed encoding).
2. `type_tag` (optional).
3. `expected_reference` (canonical Reference hex).
For TGK fixtures:
1. `edge_artifact_reference` (canonical Reference hex).
2. `edge_body` (decoded representation).
3. `edge_type_id`
4. `from_refs` and `to_refs` (ordered lists, canonical hex).
---
# 5. Test Suite Requirements
## 5.1 ASL/1-STORE Suite
1. Put/get round-trip.
2. Idempotent `put`.
3. `get` on missing ref yields `ERR_NOT_FOUND`.
4. Collision on same ref yields `ERR_INTEGRITY`.
5. Unsupported config yields `ERR_UNSUPPORTED`.
6. Optional I/O fault path yields `ERR_IO`.
## 5.2 TGK/STORE/1 Suite
1. `resolve_edge` for valid edge returns `EdgeBody`.
2. `resolve_edge` for non-edge returns `GS_ERR_NOT_EDGE`.
3. `resolve_edge` for artifact integrity failure yields `GS_ERR_ARTIFACT_ERROR`.
4. `edges_from`, `edges_to`, `edges_incident` return identical lists across backends.
5. Deterministic ordering per `TGK/STORE/1` canonical `(hash_id, digest)` rule.
6. Optional `neighbors` returns canonical node order.
7. Optional `scan_edges` enumerates all edges exactly once.
## 5.3 Replay Equivalence
1. Build a fixture ASL log/index state.
2. Replay into a fresh store.
3. Assert identical references and TGK results.
---
# 6. Ordering and Equivalence Rules
## 6.1 Edge Ordering
Edge lists MUST be sorted in ascending lexicographic order of:
`hash_id_bytes || digest_bytes`, with `hash_id` encoded as big-endian `u16`.
## 6.2 Node Ordering
Neighbor lists MUST be sorted in ascending lexicographic order of:
`hash_id_bytes || digest_bytes`, with `hash_id` encoded as big-endian `u16`.
## 6.3 Cross-Backend Equivalence
For a fixed snapshot and `GraphStoreConfig`, all backends MUST return:
1. Identical edge sets.
2. Identical ordered lists for adjacency queries.
3. Identical results for `resolve_edge`.
---
# 7. Harness Structure
Suggested structure:
1. `tests/conformance/asl_store/*.c`
2. `tests/conformance/tgk_store/*.c`
3. `tests/fixtures/*.json`
4. `tools/amduat_conformance_run`
The harness runner MUST:
1. Accept backend selector flags.
2. Accept fixture set selection.
3. Emit a machine-readable summary.
---
# 8. CI Gate
An adapter cannot be merged unless:
1. It passes its conformance suite.
2. Cross-backend equivalence is demonstrated.
3. Replay equivalence passes on at least one backend pair.
---
# 9. Out of Scope
1. Performance benchmarking.
2. Load testing.
3. Backend-specific tuning.
Those are separate gates after conformance is proven.

101
docs/decision-checklist.md
View file

@ -1,101 +0,0 @@
# Decision Checklist (Gating Artifact)
This file records decisions that must be resolved before moving forward with
backend integration. It is the gating artifact for Phase 0/1.
---
## Phase 0 — Conformance Harness + Determinism
### Conformance Scope
* Tier 0 MUST-pass specs (backend/store focus):
* `ASL/1-CORE`
* `ASL/1-STORE`
* `ENC/ASL1-CORE`
* `HASH/ASL1`
* `TGK/1-CORE`
* `ENC/TGK1-EDGE/1`
* `TGK/STORE/1`
* Tier 1 MUST-pass specs (index/log + optional provenance):
* `ASL/LOG/1`
* `ENC/ASL-LOG/1`
* `ASL/1-CORE-INDEX`
* `ENC/ASL-CORE-INDEX/1`
* `ASL/STORE-INDEX/1`
* `ASL/INDEX-ACCEL/1`
* `ASL/FEDERATION/1` and `ASL/FEDERATION-REPLAY/1` (if federation is in scope)
* `TGK/PROV/1` (if provenance operators are surfaced)
Notes:
* The legacy filesystem store (`asl_store_fs`) is not a Tier 1 conformance target once index/log is used.
* PEL specs are implemented and tested in this repo, but they are not required for the Phase 0/1 backend gate unless the surface is shipped as part of the backend.
### TGK/STORE Equivalence Definition
* Exact definition of “same result” across backends:
* For a fixed `GraphStoreConfig` and snapshot where `(Artifacts, config.tgk_profiles)` are equal, all `TGK/STORE/1` operations must return identical results per `TGK/STORE-DET/1`.
* `resolve_edge(ref)` returns the same `EdgeBody` or the same error code.
* `edges_from`, `edges_to`, `edges_incident` return identical `GraphEdgeView` lists (same elements in the same order).
* If implemented, `neighbors` returns identical `Node` lists (same elements in the same order).
* If implemented, `scan_edges` enumerates the same edges with identical per-page ordering; `PageToken` bytes may differ but must represent the same position in the canonical ordering.
* Ordering requirements (explicitly enumerated):
* Edge ordering is canonical and global: ascending lexicographic order of `order_key(edge_ref) = hash_id_bytes || digest_bytes`, with `hash_id` as big-endian `u16` (`TGK/STORE-EDGE-ORDER/1`).
* Neighbor ordering is canonical: ascending lexicographic order of `(hash_id, digest)` with the same `u16` big-endian encoding (`TGK/STORE-NEIGHBORS/1`).
* Pagination/scan stability rules:
* `scan_edges` must enumerate each matching edge exactly once and preserve the canonical per-page ordering.
* Snapshot stability across pages is optional; if not stable, the implementation must still preserve per-call ordering and enumeration semantics for the snapshot used by that call.
### Error Model Mapping Policy
* Canonical internal error taxonomy:
* ASL store errors: `AMDUAT_ASL_STORE_ERR_INTEGRITY`, `AMDUAT_ASL_STORE_ERR_UNSUPPORTED`, `AMDUAT_ASL_STORE_ERR_NOT_FOUND`, `AMDUAT_ASL_STORE_ERR_IO`.
* TGK store errors: `GS_ERR_NOT_EDGE`, `GS_ERR_ARTIFACT_ERROR`, `GS_ERR_UNSUPPORTED`, `GS_ERR_INTEGRITY`.
* Adapter-specific error mapping rules:
* `ASL/1-STORE` specifies `ERR_NOT_FOUND`, `ERR_INTEGRITY`, `ERR_UNSUPPORTED`. I/O/auth failures are out of scope for the spec; in this repo they map to `AMDUAT_ASL_STORE_ERR_IO`.
* `TGK/STORE/1` requires artifact-layer conflicts during `resolve_edge` to surface as `GS_ERR_ARTIFACT_ERROR` (not `GS_ERR_INTEGRITY`).
* `GS_ERR_NOT_EDGE` is returned when a reference resolves to an Artifact that is not a TGK edge under the active profile set.
* Required error tests:
* ASL: `get` on missing ref → `ERR_NOT_FOUND`.
* ASL: `put` collision (same ref, different artifact) → `ERR_INTEGRITY`.
* ASL: unsupported config or hash/profile mismatch → `ERR_UNSUPPORTED`.
* ASL: adapter I/O faults → `ERR_IO` (non-normative but required for harness).
* TGK: `resolve_edge` non-edge artifact → `GS_ERR_NOT_EDGE`.
* TGK: `resolve_edge` artifact-layer integrity failure → `GS_ERR_ARTIFACT_ERROR`.
* TGK: unsupported profile/encoding → `GS_ERR_UNSUPPORTED`.
---
## Phase 1 — ASL MinIO Adapter
### Object Key Layout
* Reference → bucket/key mapping: TBD (not specified by current specs/tests).
* Prefix sharding / directory fanout: TBD (not specified by current specs/tests).
### Metadata Policy
* Encoding/profile version tracking: TBD (not specified by current specs/tests).
* Metadata location (object metadata/tags/in-band): TBD (not specified by current specs/tests).
### Multipart Policy
* Multipart threshold: TBD (not specified by current specs/tests).
* Fidelity guarantees and validation: TBD (not specified by current specs/tests).
### Consistency Assumptions
* Read-after-write expectations: TBD (not specified by current specs/tests).
* Deployment modes covered by tests: TBD (not specified by current specs/tests).
### Security / Credentials
* Credential source and rotation policy: TBD (not specified by current specs/tests).
* Permission boundaries: TBD (not specified by current specs/tests).
---
## Status
* Phase 0 decisions: DRAFTED (spec-derived; requires owner confirmation)
* Phase 1 decisions: NOT STARTED

120
docs/es-indexing-plan.md
View file

@ -1,120 +0,0 @@
# Elasticsearch Indexing Plan
Status: Draft
Scope: Minimal phased ES deployment aligned with indexing contract.
---
# Phase A — Artifact Metadata Projection (Minimal)
## Goal
Accelerate search and metadata filtering only.
## Index Name
`amduat_artifacts_v1`
## Document Structure
1. `_id`: canonical `ReferenceBytes` hex.
2. `reference`: canonical hex.
3. `projection_schema_version`: integer.
4. `kind`: keyword.
5. `enc_profile`: keyword.
6. `labels`: keyword array.
7. `created_at`: date, only if intrinsic to artifact.
8. `text`: text, optional, deterministic extractor.
9. `extractor_version`: keyword, if `text` present.
## Data Source
1. ASL authoritative log/index.
2. Deterministic decoding only.
## Query Patterns Accelerated
1. Text search.
2. Filter by kind.
3. Filter by labels.
4. Narrow candidate set.
## Verification Rule
All returned References MUST be confirmed via ASL/1-STORE before returning artifact data.
---
# Phase B — Edge Projection (Optional)
## Goal
Accelerate candidate neighbor discovery.
## Index Name
`amduat_edges_v1`
## Document Structure
1. `_id`: edge Reference hex.
2. `edge_reference`: canonical hex.
3. `src_ref`: canonical hex.
4. `dst_ref`: canonical hex.
5. `edge_label`: keyword.
6. `hash_id`: keyword or numeric.
7. `digest`: keyword.
8. `projection_schema_version`: integer.
## Data Source
1. EdgeArtifacts only.
2. TGK profile rules.
3. No inferred edges.
## Verification Rule
Returned edges MUST be validated via TGK/STORE/1 authoritative query, and canonical ordering MUST be applied before client response.
---
# Phase C — Enterprise Projection Hardening
## Additions
1. Projection checkpoint artifacts stored in ASL.
2. Schema version recorded in checkpoint.
3. Full rebuild test automation.
4. Lag metrics published.
5. Projection audit job comparing snapshot vs ES.
---
# Operational Guarantees
1. ES may be disabled without breaking semantics.
2. Projection rebuild must be deterministic.
3. Schema changes require replay.
4. Checkpoint artifact defines replay boundary.
---
# Observability Requirements
Expose:
1. ASL head log position.
2. Indexer checkpoint position.
3. Computed lag.
4. Projection error counts.
5. Rebuild duration.
6. Verification fallback rate.
---
# Final Guardrail
Elasticsearch accelerates discovery.
Authoritative semantics remain in ASL/1-STORE and TGK/STORE/1.
Ordering, identity, and correctness are never delegated to ES.

295
docs/federation-implementation-notes.md
View file

@ -1,295 +0,0 @@
# Federation Implementation Notes (Core)
Status: Draft
Owner: Architecture
Version: 0.1.0
SoT: No
Last Updated: 2025-02-XX
## Purpose
These notes bind the federation semantics docs to concrete core-library
responsibilities, APIs, and data structures. The intent is to keep
federation logic inside the core substrate and keep daemon/frontends thin.
## Normative inputs
Core semantics and replay:
- tier1/asl-federation-1.md
- tier1/asl-federation-replay-1.md
- tier1/asl-store-index-1.md
- tier1/enc-asl-core-index-1.md
Admission and policy gating:
- tier1/asl-dap-1.md
- tier1/asl-policy-hash-1.md
- tier1/asl-domain-model-1.md
- tier1/asl-dam-1.md
Contextual alignment:
- tier1/asl-system-1.md
- tier1/asl-encrypted-blocks-1.md
## Scope (core library)
Federation MUST be implemented as core substrate logic:
- Deterministic federation view construction.
- Replay ordering and bounds per domain.
- Imported record metadata preservation (domain_id, visibility,
cross_domain_source).
- Tombstone and shadowing behavior per domain.
The following are explicitly out of scope for core:
- Transport protocols (HTTP, IPC, gossip).
- Peer discovery and operational orchestration.
- Admin UX and deployment wiring.
- Admission workflows, auth, and retries/backoff.
- Cache policy knobs (fetch timing, eviction, prefetch).
- Operational concerns (metrics, admin endpoints).
- Policy evaluation and per-record filtering decisions.
## Layering note
Core provides deterministic federation semantics and view construction only.
Middle-layer components are responsible for transport, admission workflows,
policy evaluation (including per-record filtering), caching strategies, and
operational wiring.
Definition:
- Middle layer: the daemon/service boundary around core logic that owns
network transport, admission workflows, and operational policy.
## Responsibilities
1) Federation registry
- Tracks known domains, admission status, policy hash, and
last admitted {SnapshotID, LogPrefix}.
- Enforces DAP + policy compatibility prior to admitting remote state.
2) Federation view cache
- Materializes a deterministic view from local state + admitted
published records from other domains.
- Stores imported records with origin metadata for replay.
- Tracks per-domain replay high-water {domain_id, logseq}.
3) Resolver
- Resolves ArtifactKey using local store + federation view.
- Does not mutate store/index as part of GET semantics.
## Data model (suggested)
```c
typedef enum {
AMDUAT_FED_REC_ARTIFACT = 0,
AMDUAT_FED_REC_PER = 1,
AMDUAT_FED_REC_TGK_EDGE = 2,
AMDUAT_FED_REC_TOMBSTONE = 3
} amduat_fed_record_type_t;
typedef struct {
uint32_t domain_id;
uint64_t snapshot_id;
uint64_t log_prefix;
uint64_t last_logseq;
uint8_t admitted; // boolean
uint8_t policy_ok; // boolean
uint8_t reserved[6];
amduat_hash_id_t policy_hash_id;
amduat_octets_t policy_hash;
} amduat_fed_domain_state_t;
typedef struct {
uint32_t domain_id;
uint8_t visibility; // 0 internal, 1 published
uint8_t has_source; // 0/1
uint16_t reserved0;
uint32_t source_domain;
} amduat_fed_record_meta_t;
typedef struct {
amduat_fed_record_type_t type;
union {
amduat_asl_artifact_key_t artifact_key;
amduat_asl_tgk_edge_key_t tgk_edge_key;
amduat_asl_per_key_t per_key;
amduat_asl_artifact_key_t tombstone_key; // key being removed
} id;
} amduat_fed_record_id_t;
typedef struct {
amduat_fed_record_meta_t meta;
amduat_fed_record_id_t id;
amduat_asl_artifact_location_t loc;
uint64_t logseq;
uint64_t snapshot_id;
uint64_t log_prefix;
} amduat_fed_index_record_t;
typedef struct {
amduat_fed_record_id_t id;
uint32_t reason_code; // policy-specific; 0 if unknown
} amduat_fed_policy_deny_t;
```
Notes:
- Imported records MUST retain domain_id and cross-domain source metadata.
- Tombstones must retain domain_id/visibility for domain-local shadowing.
- Each record MUST include a record type and canonical identity for deterministic
replay across artifacts, PERs, TGK edges, and tombstones.
- PER/TGK canonical identities are currently represented by ASL references
(artifact IDs); no separate edge/PER key types exist yet.
## Core API sketch
```c
typedef struct amduat_fed_registry_t amduat_fed_registry_t;
typedef struct amduat_fed_view_t amduat_fed_view_t;
amduat_fed_registry_t *amduat_fed_registry_open(...);
void amduat_fed_registry_close(amduat_fed_registry_t *);
// Admission + policy gating
bool amduat_fed_admit_domain(amduat_fed_registry_t *, domain_id, policy_hash, ...);
bool amduat_fed_set_domain_state(amduat_fed_registry_t *, domain_id,
snapshot_id, log_prefix);
// Ingest published records for a domain (already transported).
// Each record MUST include its type and canonical identity in the id field.
bool amduat_fed_ingest_records(amduat_fed_registry_t *, domain_id,
const amduat_fed_index_record_t *records,
size_t count);
// Build or refresh a deterministic federation view.
amduat_fed_view_t *amduat_fed_view_build(amduat_fed_registry_t *,
const amduat_asl_store_t *local_store,
const amduat_asl_index_state_t *local_state);
// Resolve via local store + federated view.
amduat_asl_store_error_t amduat_fed_resolve(
const amduat_fed_view_t *view,
const amduat_asl_artifact_key_t *key,
amduat_bytes_t *out_bytes);
```
Notes:
- Transport fetch is not part of resolve; it only consumes ingested records.
- The daemon can choose to fetch missing bytes when resolve reports a remote
reference but local bytes are absent.
- If per-record filtering is enabled, it is applied during ingest or view build
and any denials are recorded in view metadata (e.g., a table of
amduat_fed_policy_deny_t).
## Replay and view construction
Rules are as per ASL/FEDERATION-REPLAY/1:
- Records are ordered by (logseq asc, canonical identity tie-break).
- Replay is bounded by {SnapshotID, LogPrefix} per domain.
- Tombstones shadow only within their source domain.
- Imported entries keep domain_id + cross_domain_source.
The view is the union of:
1) Local domain internal + published state at local {SnapshotID, LogPrefix}.
2) Admitted foreign published state at each domain's {SnapshotID, LogPrefix}.
## Cache and storage
Federation view storage MAY be:
- In-memory (ephemeral), or
- On-disk index segments with federation metadata populated.
If remote bytes are fetched, they MUST be written to a cache store that is
logically separate from the authoritative local store (policy-controlled).
## Federation API overview
Core federation primitives are exposed via:
- include/amduat/fed/registry.h
- include/amduat/fed/ingest.h
- include/amduat/fed/replay.h
- include/amduat/fed/view.h
## Integration example (core flow)
```c
amduat_fed_registry_store_t reg_store;
amduat_fed_registry_value_t reg_value;
amduat_fed_domain_state_t reg_states[8];
amduat_fed_registry_value_init(&reg_value, reg_states, 8);
amduat_fed_registry_store_init(&reg_store, local_store);
amduat_fed_registry_store_put(&reg_store, &reg_value, &reg_ref, NULL);
if (amduat_fed_ingest_validate(records, record_count, NULL, NULL) !=
AMDUAT_FED_INGEST_OK) {
return false;
}
if (!amduat_fed_view_build(records,
record_count,
local_domain_id,
bounds,
bounds_len,
denies,
denies_len,
&view)) {
return false;
}
resolve_err = amduat_fed_resolve(&view, local_store, ref, &artifact);
```
## Policy gating
- Admission gating is per-domain.
- Per-record filtering is optional and MUST be an explicit, deterministic
policy layer if enabled.
## Error reporting
Core resolve should distinguish:
- NOT_FOUND (no record in local or federation view)
- FOUND_REMOTE_NO_BYTES (record exists in view but bytes missing locally)
- INTEGRITY_ERROR (hash mismatch on bytes)
- POLICY_DENIED (domain admitted but record filtered by policy)
Notes:
- When per-record filtering is enabled, POLICY_DENIED SHOULD surface at ingest
or view-build time by excluding filtered records from the view and recording
the denial in view metadata. Resolve MAY return POLICY_DENIED only when such a
denial is recorded for the queried key; otherwise it MUST return NOT_FOUND.
## Remote fetch guidance
Core resolve returns only artifact references. The middle layer is responsible
for fetching remote bytes by reference over its transport and caching them
separately from the authoritative local store.
## Ownership and lifecycle
- `amduat_fed_registry_decode` allocates policy hash buffers; free with
`amduat_fed_registry_value_free`.
- `amduat_fed_registry_encode` returns allocated bytes; free with
`amduat_octets_free`.
- `amduat_fed_view_build` allocates view records and denies; free with
`amduat_fed_view_free`.
- `amduat_fed_replay_domain` allocates replay records; free with
`amduat_fed_replay_view_free`.
## Middle-layer fetch example (informative)
```c
err = amduat_fed_resolve(view, local_store, ref, &artifact);
if (err == AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES) {
if (fetch_remote_bytes(ref, &artifact) && cache_store_put(artifact)) {
err = amduat_fed_resolve(view, local_store, ref, &artifact);
}
}
```
## Tests (minimal)
1) Replay ordering determinism across two domains with interleaved logseq.
2) Tombstone shadowing is domain-local.
3) Imported record metadata preserved in view and survives rebuild.
4) Conflict: same ArtifactKey with different bytes across domains rejected.
5) Bound replay by {SnapshotID, LogPrefix} produces stable view.

460
docs/implementation-plan.md
View file

@ -1,460 +0,0 @@
# Amduat Open-Source Backend
# Implementation Plan Proposal
**Status:** Architecture & Implementation Baseline v1
**Purpose:** Define phased implementation of the OSS backend stack as a spec-compliant implementation of Amduat store/query surfaces.
---
# 1. Goal and Scope
## 1.1 Purpose
This plan defines how to implement the open-source stack (MinIO, JanusGraph, Cassandra, Elasticsearch, Spark, Nessie, IPFS, Ollama) as a backend implementation of:
* `ASL/1-STORE`
* `TGK/STORE/1`
* Optional `TGK/PROV/1`
The plan ensures:
* Strict semantic conformance
* Deterministic reproducibility
* Backend substitutability
* Observational equivalence
## 1.2 Out of Scope
* Modifying core specifications
* Introducing new kernel ops
* Elevating projections to semantic authority
* Redefining identity or encoding rules
## 1.3 Target Tiers
This plan covers:
* Tier 0 — Embedded / Edge
* Tier 1 — Platform / Scale-Out
* Tier 2 — Enterprise / Audit-Grade
---
# 2. Semantic Invariants
All phases must preserve:
## 2.1 No Semantic Drift Rule
* No backend-specific interpretation of artifacts
* No implicit schema expansion
* No hidden type coercion
* No projection elevation without governance
* All optimizations must be observationally equivalent to spec-defined behavior
## 2.2 Non-Negotiable Properties
* Canonical encoding (`ENC/ASL1-CORE`)
* Deterministic identity (`ASL/1-CORE` + `HASH/ASL1`)
* Deterministic TGK projection (`TGK/1-CORE`)
* Store/query surface conformance
* Replay equivalence
---
# 3. Decision Checklist (Must Resolve Upfront)
These decisions must be explicitly documented before Phase 1 begins.
---
## 3.1 Conformance Tests
* Exact test suites for:
* `ASL/1-STORE`
* `TGK/STORE/1`
* Deterministic ordering checks
* Replay equivalence tests
* Acceptance criteria per tier
---
## 3.2 Backend Mapping
### ASL Mapping
* Artifact storage layout in MinIO
* Namespace strategy
* Object key derivation rules
* Versioning policy (if enabled)
### TGK Mapping
* Vertex/edge mapping strategy
* Deterministic ordering enforcement
* Cassandra schema design
* Index strategy in Elasticsearch (if used)
---
## 3.3 Consistency Model
Per tier:
* Write guarantees
* Read-after-write expectations
* Projection lag tolerance
* Eventual consistency boundaries
* Snapshot isolation model
---
## 3.4 Snapshot & Export Strategy
* Snapshot format (artifact set, graph export, columnar dataset)
* Snapshot identity model
* Export cadence (manual, periodic, event-triggered)
* Replay and equivalence validation rules
---
## 3.5 Governance & Projection Promotion
* Rules for elevating a projection to authoritative
* Audit requirements
* Approval workflow
---
## 3.6 Failure & Recovery
* Partial-write handling
* Corruption detection
* Divergence reconciliation
* Reprojection strategy
* Disaster recovery model
---
## 3.7 Pluggable Ops Registry
* Storage location
* Versioning strategy
* Discovery mechanism
* Trust policy
---
## 3.8 Execution Artifact Schemas
Define required fields for:
* ExecutionIntent
* ExecutionReceipt
* ExecutionOutput
* ExecutionObservation
Define canonical encoding and validation rules.
---
## 3.9 Security & Isolation
* Executor trust boundaries
* Secret management
* Artifact integrity validation
* Network isolation policies
---
## 3.10 Performance Targets
Define SLOs per tier:
* Ingest throughput
* Query latency
* Snapshot generation time
* Recovery time objectives
---
# 4. Architecture Mapping
---
## 4.1 ASL/1-STORE Implementation
### Responsibilities
* Canonical encoding validation
* Identity derivation
* Artifact immutability
* Retrieval guarantees
### Concrete Mapping
* MinIO object store
* Object key = Reference
* Metadata includes encoding profile version
* Optional replication policy
---
## 4.2 TGK/STORE/1 Implementation
### Responsibilities
* Deterministic graph projection
* Required query operations
* Error model conformance
* Deterministic ordering guarantees
### Concrete Mapping
* JanusGraph for query layer
* Cassandra for durable storage
* Explicit ordering enforcement in query results
* Elasticsearch as optional derived projection
---
## 4.3 Optional Projections
### Elasticsearch
* Full-text search
* Attribute filtering
### IPFS
* Artifact distribution and caching
### Lakehouse (MinIO + Nessie + Spark)
* Snapshot export
* Batch provenance analysis
* Audit workflows
All remain derived projections.
---
## 4.4 Provenance Strategy
`TGK/PROV/1` may be realized via:
* Native graph traversal
* Snapshot-based analytics
* Hybrid strategy
Must remain observationally equivalent.
---
# 5. Interfaces and Contracts
---
## 5.1 ASL Interface
* `put(Artifact) -> Reference`
* `get(Reference) -> Artifact`
Error model strictly per spec.
---
## 5.2 TGK Interface
* Required query operations per `TGK/STORE/1`
* Deterministic ordering rules
* Explicit error semantics
---
## 5.3 Serialization
* Canonical encoding at artifact boundary
* No alternative encoding permitted
* Versioned encoding profile tracking
---
# 6. Execution Model (Pluggable Ops)
---
## 6.1 Registry Structure
* External registry
* Versioned definitions
* Immutable op identifiers
* Policy-scoped availability
---
## 6.2 Executor Lifecycle
* Intent creation
* Dispatch
* Receipt capture
* Output capture
* Optional observation
---
## 6.3 Artifact Workflow
1. Write ExecutionIntent
2. Executor produces ExecutionReceipt
3. Executor produces ExecutionOutput
4. Optional ExecutionObservation
5. Link artifacts via EdgeArtifacts
All artifacts stored in ASL.
---
## 6.4 TGK Linking
* Intent → Receipt
* Receipt → Output
* Output → Observation
Graph projection ensures traceability.
---
# 7. Implementation Phases
---
## Phase 0 — Conformance Baseline
* Build ASL/TGK conformance harness
* Define canonical test datasets
* Establish replay checks
---
## Phase 1 — ASL Store
* MinIO integration
* Canonical encoding enforcement
* Hash-based identity derivation
* Conformance validation
---
## Phase 2 — TGK Projection
* Graph projection pipeline
* JanusGraph integration
* Deterministic ordering enforcement
* Conformance validation
---
## Phase 3 — Projections & Indexing
* Elasticsearch integration
* IPFS optional layer
* Monitoring of projection lag
---
## Phase 4 — Snapshot & Governance
* Snapshot export format
* Nessie integration
* Replay equivalence testing
* Governance workflows
---
## Phase 5 — Pluggable Ops & Executors
* Registry implementation
* Execution artifact schemas
* Executor runtime
* Traceability validation
---
# 8. Testing and Verification
Each phase must include:
* Deterministic replay checks
* Ordering validation tests
* Snapshot equivalence tests
* Cross-backend equivalence tests
No phase advances without passing conformance.
---
# 9. Operational Concerns
---
## 9.1 Deployment per Tier
Tier 0:
* Single-node embedded
Tier 1:
* Janus + Cassandra + ES
* MinIO
* Optional IPFS
Tier 2:
* Add Spark
* Add Nessie
* Governance workflows
---
## 9.2 Observability
* Artifact write metrics
* Graph projection metrics
* Projection lag monitoring
* Snapshot health indicators
---
## 9.3 Backup & Restore
* MinIO replication or erasure coding
* Cassandra repair strategy
* Snapshot archival policy
---
# 10. Open Questions and Risks
* Canonical test dataset size
* Cassandra compaction strategy
* Projection drift detection
* Snapshot storage growth
* Executor trust hardening
Each risk must have a mitigation strategy before Tier 2 rollout.
---
# Design Outcome
This implementation plan:
* Preserves core invariants
* Enables infrastructure scaling
* Prevents semantic leakage
* Structures phased, verifiable progress
* Makes the OSS stack a compliant backend implementation

58
docs/index-log-api-sketch.md
View file

@ -1,58 +0,0 @@
# Index/Log API Surface (Sketch)
This document is a one-page sketch of the planned public API for ASL index/log
support. It is non-normative and intended to guide header design.
## ASL Index/Log Types (Draft)
```
typedef uint64_t amduat_asl_snapshot_id_t;
typedef uint64_t amduat_asl_log_position_t; // inclusive logseq upper bound
typedef struct {
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t log_position;
} amduat_asl_index_state_t;
```
## Core Store API (Draft)
```
// Initialization and config.
bool amduat_asl_store_index_init(...);
// PUT/GET with index state reporting.
amduat_asl_store_error_t amduat_asl_store_put_indexed(
amduat_asl_store_t *store,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_get_indexed(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact);
```
## Index/Log Introspection (Draft)
```
// Snapshot/log position queries.
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
amduat_asl_index_state_t *out_state);
// Segment and log inspection (read-only).
bool amduat_asl_log_scan(amduat_asl_store_t *store, ...);
bool amduat_asl_segment_scan(amduat_asl_store_t *store, ...);
```
## Expected Error Surfaces
* `AMDUAT_ASL_STORE_ERR_INTEGRITY` for malformed index segments or log records.
* `AMDUAT_ASL_STORE_ERR_IO` for underlying I/O faults.
* `AMDUAT_ASL_STORE_ERR_NOT_FOUND` for absent artifacts or missing segments.
* `AMDUAT_ASL_STORE_ERR_UNSUPPORTED` for unsupported encoding versions.
These are illustrative; exact error codes and mapping will be finalized when
headers are introduced.

145
docs/indexing-requirements-and-projections.md
View file

@ -1,145 +0,0 @@
# Indexing Requirements and Projections
Status: Draft
Scope: Define indexing contract, authority boundaries, and projection invariants for derived search/index systems (e.g., Elasticsearch).
---
# API Usage Rule (Normative)
1. Elasticsearch returns candidate References only.
2. Authoritative confirmation MUST occur via ASL/1-STORE and/or TGK/STORE/1.
3. Any ordering visible to clients MUST be computed using the canonical `(hash_id, digest)` ordering defined in TGK/STORE/1.
4. Elasticsearch MUST NOT be treated as a source of semantic truth.
---
# 1. Decision Checklist (Must Resolve Before Implementation)
1. Which artifact types are searchable.
2. Which fields are eligible for text extraction.
3. Whether `created_at` is intrinsic or ingestion-derived.
4. Checkpoint source of truth (ASL log position required).
5. Verification policy (which APIs require authoritative confirmation).
6. Edge indexing included in Phase A or deferred.
7. Redaction policy per TypeTag.
8. Projection schema versioning strategy.
No indexing implementation proceeds without resolving these.
---
# 2. Authority Boundary
## 2.1 Authoritative Sources
1. Artifact identity and content: ASL/1-STORE.
2. Graph semantics and ordering: TGK/STORE/1.
3. Canonical encoding: ENC/ASL1-CORE.
4. Identity derivation: ASL/1-CORE and HASH/ASL1.
Elasticsearch is a derived projection only.
## 2.2 No Semantic Drift Rule
1. No backend-specific interpretation of artifacts.
2. No inferred edges.
3. No implicit ordering.
4. No hidden type coercion.
5. All optimizations must be observationally equivalent to authoritative results.
---
# 3. Reference Encoding Rule
All `reference` fields indexed in ES MUST use canonical `ReferenceBytes` hex encoding as defined by ENC/ASL1-CORE.
This guarantees:
1. Unambiguous equality.
2. Stable identity across backends.
3. No ambiguity between `(hash_id, digest)` representations.
No alternative encodings are permitted.
---
# 4. Ordering Rule
Elasticsearch ordering MUST NOT be treated as authoritative.
Any client-visible ordering MUST be computed via the canonical `(hash_id, digest)` ordering defined in TGK/STORE/1.
If ES provides candidate edges, authoritative logic must re-order them.
---
# 5. Projection Contract
## 5.1 Replayability
Index state MUST be fully replayable from ASL log/index state, EdgeArtifacts, and deterministic decoding rules.
## 5.2 Observational Equivalence
Given identical ASL state, ES projection rebuilt from scratch MUST produce identical documents within a schema version.
## 5.3 Schema Versioning
Each projection index MUST have:
1. An explicit schema version, e.g., `projection_schema_version`.
2. The version recorded in a checkpoint artifact.
3. A versioned rebuild strategy.
Schema changes require full replay.
---
# 6. Lag Metric Definition
Lag is formally defined as:
```
Lag = ASL_head_log_position - indexer_checkpoint_log_position
```
Where:
1. `ASL_head_log_position` is authoritative.
2. `indexer_checkpoint_log_position` is recorded in a projection checkpoint artifact.
Lag MUST be observable via metrics.
---
# 7. Decoding and Extraction Rules
1. Artifact decoding MUST use only the declared encoding profile.
2. No heuristic decoding permitted.
3. Text extraction MUST be deterministic and versioned.
4. Extractor version MUST be stored in index documents.
---
# 8. Edge Indexing Guardrail
If edges are indexed:
1. Edge documents MUST be derived only from EdgeArtifacts.
2. No inferred adjacency allowed.
3. Edge semantics MUST respect TGK profile rules.
4. Ordering MUST be applied by the authoritative layer.
---
# 9. Security and Redaction
1. Per-TypeTag redaction policy is mandatory.
2. No secrets indexed unless explicitly approved.
3. Redaction rules MUST be declared before indexing begins.
---
# 10. Conformance Gates
1. System must function with ES disabled.
2. Full rebuild equivalence test required.
3. Ordering correctness test required.
4. Cross-backend equivalence test required.

67
docs/minio-adapter.md
View file

@ -1,67 +0,0 @@
# MinIO ASL Store Adapter
This adapter provides an S3-compatible `ASL/1-STORE` backend for MinIO. It is
opt-in for conformance runs and does not require MinIO or any external services
at build time.
## Required environment
These are required when running conformance against the MinIO backend:
- `AMDUAT_MINIO_ENDPOINT` (e.g. `http://localhost:9000`)
- `AMDUAT_MINIO_BUCKET` (e.g. `amduat`)
- `AMDUAT_MINIO_ACCESS_KEY`
- `AMDUAT_MINIO_SECRET_KEY`
Optional configuration:
- `AMDUAT_MINIO_REGION` (default: `us-east-1`)
- `AMDUAT_MINIO_PREFIX` (default: `asl`)
- `AMDUAT_MINIO_FORCE_PATH_STYLE` (default: `1`)
- `AMDUAT_MINIO_TLS_VERIFY` (default: `1`)
- `AMDUAT_MINIO_TIMEOUT_MS` (default: `30000`)
- `AMDUAT_MINIO_RETRY_MAX` (default: `3`)
## Conformance runs
MinIO is skipped unless explicitly enabled.
Example (opt-in conformance run):
```sh
AMDUAT_CONFORMANCE_ENABLE_MINIO=1 \
AMDUAT_MINIO_ENDPOINT=http://localhost:9000 \
AMDUAT_MINIO_BUCKET=amduat \
AMDUAT_MINIO_ACCESS_KEY=minioadmin \
AMDUAT_MINIO_SECRET_KEY=minioadmin \
./build/amduat_conformance_run --suite asl --backend minio --workdir /tmp/amduat_conformance
```
If required environment variables are missing, the backend is reported as
`SKIPPED` with the missing variable names.
## Key layout
Objects are stored under a sharded key derived from the canonical reference
hex:
```
<prefix>/<r0r1>/<r2r3>/<full_reference_hex>
```
## Error mapping
| HTTP/S3 response | Canonical taxonomy | Store error |
| --- | --- | --- |
| 404 | NotFound | `AMDUAT_ASL_STORE_ERR_NOT_FOUND` |
| 401, 403 | PermissionDenied | `AMDUAT_ASL_STORE_ERR_IO` |
| 408, 429 | TransientIO | `AMDUAT_ASL_STORE_ERR_IO` |
| 5xx | TransientIO | `AMDUAT_ASL_STORE_ERR_IO` |
| 4xx (other) | InvalidArtifact | `AMDUAT_ASL_STORE_ERR_INTEGRITY` |
| Network/TLS/parse errors | InternalError/TransientIO | `AMDUAT_ASL_STORE_ERR_IO` |
## Limitations
- HTTPS is not supported yet (HTTP-only). Use `http://` endpoints for now.
- Multipart uploads, streaming artifacts, and very large objects are not
implemented in Phase 0.

274
docs/oss-backend-adapter-boundary.md
View file

@ -1,274 +0,0 @@
# OSS Backend Adapter Boundary
Status: Draft
Purpose: Define architectural boundaries, responsibilities, and conformance requirements for implementing the OSS backend stack (MinIO, JanusGraph, Cassandra, Elasticsearch, etc.) as spec-compliant adapters.
---
# 1. Architectural Position
The OSS backend is an implementation of store/query surfaces, not a semantic extension.
It implements:
1. ASL/1-STORE.
2. TGK/STORE/1.
3. Optional: TGK/PROV/1.
4. Optional: derived projections (ES, lakehouse).
It does not redefine:
1. Core semantics (ASL/1-CORE, PEL/1-CORE, TGK/1-CORE).
2. Canonical encoding (ENC/ASL1-CORE).
3. Identity derivation (ASL/1-CORE and HASH/ASL1).
---
# 2. Authority and Responsibility Model
## 2.1 Authoritative Components
Authoritative state resides in:
1. Deployed ASL/1-STORE implementation.
2. Deterministic TGK projection derived from ASL state.
Adapters must preserve this.
## 2.2 Derived Components
The following are derived projections:
1. Elasticsearch.
2. IPFS.
3. Lakehouse snapshots.
4. AI outputs.
Derived components must:
1. Be replayable.
2. Be observationally equivalent.
3. Never redefine semantics.
---
# 3. Adapter Modules
This document defines four adapter classes.
## 3.1 ASL Store Adapter (MinIO or S3)
### Purpose
Implements ASL/1-STORE.
### Responsibilities
1. Accept `put(Artifact) -> Reference`.
2. Enforce canonical encoding validation.
3. Persist artifact bytes under canonical identity.
4. Implement `get(Reference) -> Artifact | NotFound`.
5. Map backend errors to canonical error model.
### Non-Responsibilities
1. Computing identity (handled by core logic).
2. Interpreting artifact contents.
3. Modifying encoding.
4. Adding metadata not declared by spec.
### Determinism Requirements
1. Same Artifact → same Reference.
2. Same Reference → identical bytes returned.
3. No backend-specific mutation.
### Configuration Surface
1. Endpoint.
2. Credentials.
3. Bucket name.
4. Retry policy.
5. Multipart threshold.
6. Timeout policy.
---
## 3.2 TGK Store Adapter (JanusGraph plus Cassandra)
### Purpose
Implements TGK/STORE/1.
### Responsibilities
1. Provide required query operations defined in TGK/STORE/1.
2. Enforce canonical `(hash_id, digest)` ordering.
3. Maintain deterministic projection behavior.
4. Map errors to spec-defined error model.
### Non-Responsibilities
1. Creating semantic edges (done via EdgeArtifacts).
2. Redefining graph semantics.
3. Implicitly inferring relationships.
4. Providing authoritative ordering outside spec.
### Determinism Requirements
1. Same ASL state → same TGK answers.
2. Same queries → same ordered results.
3. Ordering must not depend on Janus internal traversal order.
### Ordering Enforcement Point
Ordering MUST be applied:
1. Explicitly in the adapter layer, or
2. In a canonical result transformation layer.
Never rely on backend default sort semantics.
---
## 3.3 Projection Adapter (Elasticsearch)
### Purpose
Accelerate search and filtering.
### Responsibilities
1. Project canonical artifact metadata.
2. Optionally project EdgeArtifacts.
3. Maintain replayable checkpoint.
4. Expose lag metric.
5. Enforce schema versioning.
### Non-Responsibilities
1. Authoritative artifact reads.
2. Authoritative graph queries.
3. Deterministic ordering.
### Verification Requirement
Any ES-derived candidate results must be:
1. Confirmed via ASL/1-STORE.
2. Ordered via TGK/STORE/1 logic before returning to clients.
---
## 3.4 Snapshot or Lakehouse Adapter (Future)
### Purpose
Batch export and audit workflows.
### Responsibilities
1. Snapshot export from authoritative state.
2. Versioned dataset generation.
3. Replay equivalence validation.
### Non-Responsibilities
1. Live semantic authority.
2. Mutation of ASL state.
---
# 4. Adapter Error Taxonomy
All adapters must map backend-specific errors into a canonical internal taxonomy:
1. NotFound.
2. InvalidArtifact.
3. PermissionDenied.
4. TransientIO.
5. InternalError.
No backend-specific error types may leak across the adapter boundary.
---
# 5. Conformance Gates
No adapter may be merged or promoted unless:
## 5.1 ASL Adapter
1. Passes ASL/1-STORE conformance suite.
2. Passes replay equivalence test.
3. Passes cross-backend equivalence test.
## 5.2 TGK Adapter
1. Passes TGK/STORE/1 required query suite.
2. Passes deterministic ordering test.
3. Passes cross-backend equivalence test.
## 5.3 Projection Adapter
1. System passes integration tests with projection disabled.
2. Rebuild equivalence test passes.
3. Lag metric exposed and validated.
---
# 6. Observability Contract
All adapters must expose:
1. Health endpoint.
2. Error metrics.
3. Operation latency metrics.
4. Determinism assertion failures.
5. Projection lag, if applicable.
---
# 7. Security Boundary
Adapters must:
1. Not embed secrets in artifacts.
2. Respect per-TypeTag redaction policies before indexing.
3. Enforce explicit credential configuration.
4. Avoid implicit trust of backend defaults.
---
# 8. Interaction with Conformance Harness
Adapter design is subordinate to the Phase 0 harness.
Implementation order:
1. Conformance harness operational.
2. Adapter scaffold.
3. Adapter passes harness.
4. Integration and load testing.
No adapter bypasses this sequence.
---
# 9. Design Outcome
This boundary ensures:
1. Thin adapters.
2. Zero semantic drift.
3. Deterministic reproducibility.
4. Backend substitutability.
5. Operational scalability without semantic mutation.
The OSS backend becomes:
An infrastructure implementation of store/query surfaces, not an evolution of the semantic core.
---
# Where You Are Now (Strategic Position)
You now have:
1. Semantic core locked.
2. Execution boundary defined.
3. Indexing contract defined.
4. Backend boundary defined.
5. Implementation phases defined.
You are ready to:
1. Design adapter scaffolds.
2. Build Phase 0 conformance harness.
3. Begin MinIO adapter implementation safely.

160
docs/phase0-1-execution-checklist.md
View file

@ -1,160 +0,0 @@
# Phase 0/1 Execution Checklist
## OSS Backend Enablement (MinIO-first)
### 0. Purpose
This checklist gates work so we do not start infrastructure integration until we can prove:
* spec conformance holds across multiple backends,
* deterministic ordering/replay invariants are testable,
* error-model mappings are pinned down.
---
## Phase 0 — Conformance Harness + Determinism Gates
### 0.1 Deliverables
1. **Multi-backend conformance runner** (same test vectors, multiple backends)
2. **Golden dataset generator** for ASL + TGK fixtures
3. **Deterministic equivalence suite**:
* ordering checks
* replay equivalence
* cross-backend observational equivalence (surface behavior)
### 0.2 Required decisions (turn into `docs/decision-checklist.md`)
* **Conformance test scope**: which spec assertions are MUST-pass for Tier 0 vs Tier 1
* **Equivalence definition**: what counts as “same result” for TGK/STORE operations (including ordering)
* **Error model mapping policy**: canonical internal error types vs adapter-specific errors
### 0.3 Conformance harness: what to build
**A) ASL/1-STORE conformance suite**
* Put/get round-trip for a set of canonical Artifacts
* Idempotent `put(Artifact)` returns same `Reference`
* `get(Reference)` exact artifact fidelity
* NotFound behavior and error codes
* Corruption detection behavior (if defined/expected by your layers)
* Optional: latency-independent invariants (no timing assumptions)
**B) TGK/STORE/1 conformance suite**
* Required operations per spec and error model
* Deterministic ordering assertions (as spec defines)
* “Same ASL state → same TGK answers” invariants
* Cross-backend equivalence: mem vs fs vs asl_index_fs must match for the same fixture set
**C) Replay determinism**
* Build a fixture log/index state
* Replay into a fresh store
* Assert equivalent outcomes (references returned, query results, ordering)
### 0.4 Acceptance criteria (Phase 0 gate)
* ✅ Same fixture set passes on at least **two existing backends**:
* `asl_store_fs` and `asl_store_index_fs`
* `tgk_store_mem` and `tgk_store_fs` (or `tgk_store_asl_index_fs`)
* ✅ Deterministic ordering checks are explicit and enforced
* ✅ A “backend contract test” can be run by CI with a single command
### 0.5 Where this touches the repo (suggested structure)
* `tests/conformance/asl_store/*.c`
* `tests/conformance/tgk_store/*.c`
* `tests/fixtures/*.bin|*.json|*.asl` (whatever your canonical fixture format is)
* `tools/amduat_conformance_run` (small runner, like your other tools)
---
## Phase 1 — ASL MinIO Adapter (Strict ASL/1-STORE)
### 1.1 Deliverables
1. `src/adapters/asl_store_s3_minio/asl_store_s3_minio.c` (name as you prefer)
2. Config + credentials plumbing for MinIO endpoint
3. Full Phase 0 conformance suite passing against MinIO
4. CI job (optional initially) that spins MinIO and runs conformance tests
### 1.2 Required decisions (Phase 1-specific)
* **Object key layout**: how `Reference` maps to bucket/key (e.g., prefix sharding)
* **Metadata policy**: do we store encoding/profile version as object metadata, tags, or in-band?
* **Multipart handling**: threshold + exact fidelity guarantees
* **Read-after-write expectation**: what you assume from MinIO deployment mode; what you test for
### 1.3 Adapter contract requirements (what must not change)
* `put(Artifact) -> Reference` computed by core rules; adapter must not accept a caller-provided reference
* `get(Reference) -> Artifact | NotFound` must return exact bytes for canonical decode
* Error mapping must be stable (auth errors, NotFound, transient IO)
### 1.4 Failure modes to explicitly test (Phase 1 gate)
* Bucket missing / permission denied
* NotFound for unknown reference
* Partial upload interruption (must not produce a “phantom success”)
* Corrupt object payload (if detectable at decode step, ensure it surfaces as decode error)
### 1.5 Acceptance criteria (Phase 1 gate)
* ✅ ASL/1-STORE conformance suite passes on MinIO
* ✅ Cross-backend equivalence: MinIO results identical to index/fs for the same fixture set
* ✅ Determinism: repeated `put` gives same reference, independent of environment
---
# Missing interfaces and tests to write (concrete list)
## A) Missing test harness capability (Phase 0)
* A single abstraction that can run the same tests against:
* `asl_store_fs`
* `asl_store_index_fs`
* future `asl_store_minio`
* Same for TGK store adapters
* A “golden fixture set” format (even if it’s just embedded byte arrays initially)
## B) Deterministic ordering assertions (Phase 0)
The repo has lots of tests, but not:
* a named suite that asserts **ordering** exactly as TGK/STORE defines
* cross-backend “same query, same ordering” checks
## C) Cross-backend equivalence harness (Phase 0)
You need an explicit mechanism that:
* seeds the same ASL artifact set into multiple ASL backends
* builds the same TGK projection inputs
* runs the same TGK queries and diffs results
## D) Adapter error normalization (Phase 0/1)
Define a small internal error taxonomy used by conformance tests so adapters can map:
* NotFound
* InvalidArtifact / decode error
* PermissionDenied
* TransientNetwork
* Internal/Unknown
Without this, MinIO will “pass” but be impossible to reason about operationally.
---
# What I would do next (lowest-risk sequence)
1. **Add `docs/decision-checklist.md`** and fill only Phase 0/1 decisions (don’t boil the ocean).
2. Implement **conformance runner** that can already test FS vs index/log backends (prove the harness works).
3. Add deterministic ordering + replay equivalence tests.
4. Only then implement `asl_store_minio`, using the harness as your guardrail.

253
docs/primitive-vs-pluggable-ops.md
View file

@ -1,253 +0,0 @@
# Primitive Ops vs Pluggable Ops
This section defines the execution boundary between deterministic kernel semantics and extensible domain execution.
It preserves:
* Kernel determinism
* Store-neutral traceability
* Backend substitutability
* Observational equivalence
---
# 1. Terminology
## 1.1 Kernel / Primitive Op
A **Kernel Op** (Primitive Op) is an operation defined in:
`OPREG/PEL1-KERNEL`
It is part of the `PEL/1-CORE` contract.
Kernel ops define fixed, deterministic semantics and are considered part of Amduat’s invariant semantic core.
---
## 1.2 Pluggable Op
A **Pluggable Op** is an operation defined outside `OPREG/PEL1-KERNEL`.
It is not part of `PEL/1-CORE`.
Its execution semantics:
* Are versioned independently
* May be effectful
* Are governed by profile or deployment policy
* Must emit execution artifacts for traceability
Pluggable ops extend behavior without extending the kernel.
---
# 2. Kernel / Primitive Ops
## 2.1 Properties
Kernel ops:
* Are defined in `OPREG/PEL1-KERNEL`
* Are pure and deterministic
* Have no side effects
* Depend only on explicit inputs
* Produce canonical artifacts
* Have stable semantics governed by spec revision
Kernel ops are portable across all backend tiers.
They cannot:
* Access external systems
* Read hidden state
* Depend on environmental configuration
* Produce nondeterministic outputs
---
## 2.2 Examples
* `bytes.concat`
* `bytes.slice`
* `hash.asl1_256`
These operations:
* Operate solely on canonical encoded inputs
* Produce deterministic outputs
* Are observationally equivalent across implementations
---
# 3. Pluggable Ops
Pluggable ops enable domain-specific execution without extending the kernel.
They may:
* Invoke LLMs
* Execute SSH commands
* Configure infrastructure
* Perform device operations
* Call external APIs
They are not part of the core semantic contract.
---
## 3.1 Properties
Pluggable ops:
* Are defined in external registries
* May be effectful
* May interact with external systems
* May be nondeterministic
* Must produce execution artifacts
* Must remain observationally separable from kernel semantics
They extend behavior while preserving traceability.
---
# 4. Execution Artifact Model
Every pluggable op MUST emit execution artifacts to ASL.
These artifacts preserve determinism at the semantic layer, even if the effect itself is nondeterministic.
## 4.1 Required Artifacts
At minimum:
### 1. ExecutionIntent
Declares:
* Target
* Parameters
* Configuration
* Version of pluggable op
Represents the desired action.
---
### 2. ExecutionReceipt
Confirms:
* Action dispatch
* Acceptance by executor
* Execution attempt metadata
Represents the attempt.
---
### 3. ExecutionOutput
Captures:
* Stdout/stderr
* Returned data
* Produced artifacts
* Exit codes
Represents the immediate result.
---
### 4. ExecutionObservation (Recommended)
Captures:
* Observed external state
* Post-condition checks
* Environment validation
Represents the verified outcome.
---
All execution artifacts:
* Are canonical ASL artifacts
* Are immutable
* Derive identity via `ENC/ASL1-CORE` + `ASL/1-CORE` + `HASH/ASL1`
* Participate in TGK projection
---
# 5. Traceability Requirements
All pluggable ops MUST be traceable in TGK by linking:
* `ExecutionIntent → ExecutionReceipt`
* `ExecutionReceipt → ExecutionOutput`
* Optional: `ExecutionOutput → ExecutionObservation`
These links are created via EdgeArtifacts written to ASL.
This ensures:
* Deterministic graph projection
* Provenance recoverability
* Replay safety
* Auditability
Even if external execution is nondeterministic, the trace is deterministic.
---
# 6. Governance Boundary
The boundary between kernel and pluggable ops is strict.
## 6.1 Kernel
* Small
* Deterministic
* Slow-moving
* Versioned with PEL core
* Subject to formal governance
## 6.2 Pluggable
* Extensible
* Fast-moving
* Policy-scoped
* Profile-scoped
* May be deployment-specific
No pluggable op may redefine kernel semantics.
---
# 7. Interaction with Backend Plug-in Model
This execution model aligns with the No Semantic Drift Rule:
* Kernel ops are invariant across all backends.
* Pluggable ops must emit artifacts.
* Backends must treat execution artifacts as ordinary ASL artifacts.
* TGK projection must remain deterministic.
Backend-specific optimizations must remain observationally equivalent to the artifact graph defined by execution artifacts.
---
# 8. Design Outcome
This separation:
* Preserves deterministic core semantics
* Enables real-world execution
* Prevents semantic leakage into infrastructure
* Maintains backend substitutability
* Ensures enterprise-grade auditability
* Supports edge and embedded deployments
Primitive ops define what Amduat *is*.
Pluggable ops define what Amduat *does*.

81
docs/projection-checkpoint-schema.md
View file

@ -1,81 +0,0 @@
# Projection Checkpoint Schema
Status: Draft
Scope: Schema for projection checkpoint artifacts used by derived indexers (e.g., Elasticsearch) to bind schema versions, replay state, and lag metrics to authoritative ASL state.
---
# 1. Purpose
Projection checkpoints make derived indexes replayable, auditable, and bounded. They tie a projection schema version to a specific ASL log position and snapshot identity so that rebuilds are deterministic and lag can be measured.
---
# 2. Canonical Checkpoint Artifact
Checkpoint artifacts are ASL artifacts whose bytes encode the following fields. The encoding is deterministic and versioned.
## 2.1 Minimal Fields
1. `checkpoint_version` (u32)
2. `projection_name` (string)
3. `projection_schema_version` (u32)
4. `asl_snapshot_id` (u64, optional if not available)
5. `asl_log_position` (u64, required)
6. `created_at` (u64, epoch milliseconds)
7. `builder_id` (string, identifying the indexer instance)
## 2.2 Optional Fields
1. `artifact_count` (u64)
2. `edge_count` (u64)
3. `error_count` (u64)
4. `lag_millis` (u64, derived)
5. `note` (string)
---
# 3. Identity and Storage
1. Checkpoints are stored as ASL artifacts using ASL/1-STORE semantics.
2. `projection_name` plus `projection_schema_version` must uniquely identify a projection format.
3. The checkpoint artifact reference is the authoritative identifier for the projection state.
---
# 4. Lag Metric Binding
Lag is defined as:
```
Lag = ASL_head_log_position - checkpoint.asl_log_position
```
The ASL head position is authoritative. The checkpoint log position is the last fully processed position.
---
# 5. Replay Rules
1. Rebuild begins at `checkpoint.asl_log_position + 1`.
2. Any schema change requires full replay from the earliest available log position.
3. Checkpoints must be monotonic in `asl_log_position` for a given projection.
---
# 6. Validation Requirements
When loading a checkpoint, the indexer MUST verify:
1. `projection_schema_version` matches the current build.
2. `asl_log_position` is not ahead of the authoritative ASL head.
3. `checkpoint_version` is supported.
4. If present, `asl_snapshot_id` matches the expected snapshot boundary.
---
# 7. Operational Guidance
1. Checkpoints should be written after a successful batch commit.
2. Checkpoints should be immutable; new checkpoints are separate artifacts.
3. Retain checkpoints for audit and rollback.

282
docs/projector-pattern.md
View file

@ -1,282 +0,0 @@
# Projector Pattern
**Status:** Draft
**Scope:** Define the minimal projector loop for derived projections in Amduat, using the repo’s cursor API and versioned checkpoint records stored as ASL records. This pattern applies to Elasticsearch indexing, lakehouse snapshots, IPFS exports, and other derived views.
---
## 1. Purpose
A **projector** is a deterministic process that:
1. reads authoritative ASL log/index state in a stable order,
2. applies a projection function to produce derived state (or derived side effects), and
3. persists progress as a **projection checkpoint record** stored in ASL.
Projectors accelerate discovery and analysis without becoming semantic authority.
---
## 2. Normative Rules
### 2.1 Authority
* **Authoritative truth** is the deployed `ASL/1-STORE` implementation plus deterministic TGK projection derived from ASL artifacts.
* Projectors produce **derived projections** only.
### 2.2 No Semantic Drift
* Projectors MUST NOT introduce backend-specific semantics.
* Projectors MUST NOT infer edges or meaning beyond declared encoding profiles and spec-defined decoding.
* Any optimization MUST be observationally equivalent to results obtainable from authoritative stores.
### 2.3 Disable-ability
* The system MUST remain correct when any projector is disabled.
* Projections may lag and may be eventually consistent.
---
## 3. Inputs, Outputs, and State
### 3.1 Inputs
A projector consumes:
* authoritative ASL index state (current head position)
* **ASL log records** read via cursor paging
* optional decoding of artifacts using only declared encoding profiles
**Important:** the ASL log is **not** a general “artifact publish/unpublish event stream.” Cursor replay iterates over **log records** (segment seals, tombstones, snapshot anchors, etc.), and any “what to project” logic must be derived from these records plus authoritative store state.
### 3.2 Outputs
A projector emits:
* derived side effects (e.g., write to Elasticsearch; export/pin to IPFS; write snapshot files)
* optional derived artifacts (if you store projection outputs back into ASL)
* **a projection checkpoint record** for restart/resume
### 3.3 Projector State Model
Projector state must be reconstructible from:
* authoritative ASL store + log/index state, and
* the last checkpoint record.
No hidden state is permitted.
---
## 4. Minimal Projector Loop
This is the canonical pattern. Implementations may wrap it in a service, cron job, or CLI, but the core loop remains the same.
### 4.1 Pseudocode (repo-aligned)
```c
// Inputs
amduat_asl_store_t *store = ...;
// Authoritative head
amduat_asl_index_state_t head = amduat_asl_index_current_state(store);
// Resume (optional)
projection_checkpoint_t ckpt;
bool have_ckpt = load_latest_checkpoint(store, /*projection_name*/ "...", &ckpt);
// Cursor start uses checkpoint log_position as after_logseq
uint64_t after_logseq = have_ckpt ? ckpt.log_position : 0;
for (;;) {
// Cursor reads log records (not “publish events”)
asl_log_page_t page = amduat_asl_log_read_since(store, after_logseq, /*limit*/ N);
if (page.count == 0) break;
// Project: interpret log records as needed, using only declared profiles for decoding.
projection_apply(page.records, page.count, /*derived sinks/state*/ ...);
// Advance deterministically to the returned next position
after_logseq = page.next_log_position;
// Persist checkpoint as an ASL record: schema = "projection/checkpoint"
projection_checkpoint_t next = (projection_checkpoint_t){
.projection_name = "...", // required
.projection_schema_version = SCHEMA_VER, // required
.log_position = after_logseq, // required
.builder_id = "...", // required
.created_at = 0, // required (0 allowed)
// optional: .snapshot_id = ..., counts, note, lag_millis, etc.
};
asl_record_t rec = projection_checkpoint_encode(&next); // schema "projection/checkpoint"
asl_put_record(store, &rec);
}
```
### 4.2 Determinism Requirements
* Given the same ASL store state and the same projection schema version, repeated runs MUST produce identical derived outputs (within the declared projection contract).
* Cursor paging boundaries MUST NOT affect results.
* A checkpoint MUST allow resume without changing derived outcomes.
### 4.3 Implementation Slice (Checklist + Stub)
**Checklist (minimal, repo-aligned):**
* Choose a stable `projection_name`, `projection_schema_version`, and `builder_id`.
* Read authoritative head state via `amduat_asl_index_current_state`.
* Page log records with `amduat_asl_log_read_since` (cursor-driven).
* Apply a deterministic projection step (even if it is just a counter in the stub).
* Encode a checkpoint via `amduat_asl_projection_checkpoint_encode_v1`.
* Store the checkpoint as an ASL record with schema `projection/checkpoint`.
* Verify the stored checkpoint decodes and matches the last processed `log_position`.
**Repo stub:** `tests/asl/test_asl_projection_replay.c` includes
`test_projector_loop_stub()` that runs the minimal loop and persists a checkpoint.
---
## 5. Projection Checkpoint Records
### 5.1 Storage Convention (Kernel Layer)
Projection checkpoints are stored as **ASL records** with schema string:
> `projection/checkpoint`
This is the kernel-layer storage convention for restart/resume progress tracking.
### 5.2 Required Fields (repo-aligned)
Current required fields are:
* `projection_name`
* `projection_schema_version`
* `log_position`
* `builder_id`
* `created_at` (may be `0`)
Optional fields may include (as supported by current implementation/schema):
* `snapshot_id`
* counts
* `note`
* `lag_millis`
(See `projection_checkpoint` implementation and schema doc.)
### 5.3 Lag Metric (repo-aligned)
Lag is computed against the authoritative ASL index head:
```text
Lag = head.log_position − checkpoint.log_position
```
Where `head.log_position` comes from `amduat_asl_index_current_state(store)`.
---
## 6. Projection Schema Versioning
The repository currently treats `projection_schema_version` as a versioned field, but does not define major/minor/patch semantics in core.
**Recommended policy (not a MUST):**
* Treat incompatible schema changes as requiring full replay.
* Record schema version in each checkpoint.
* If schema version changes, prefer rebuild or explicit migration procedures.
Any stricter version policy should live in governance / deployment policy docs, not kernel-level requirements.
---
## 7. Decoding and Extraction Rules
* Projectors MAY decode artifacts, but decoding MUST use **only the declared encoding profile**.
* No heuristic decoding is permitted.
* Any extraction (e.g., text extraction for ES) must be deterministic and should be versioned (extractor version treated as part of the projection schema contract).
---
## 8. Ordering and Verification Rules
### 8.1 TGK Ordering
Any ordering visible to clients MUST be computed using canonical ordering defined by `TGK/STORE/1` (e.g., `(hash_id, digest)`), not by derived store defaults (ES sorting, DB insertion order, etc.).
### 8.2 Verification
If a projector drives an API that returns candidates (e.g., ES search):
* Results MUST be treated as **candidates**
* Truth MUST be confirmed against authoritative `ASL/1-STORE` and/or `TGK/STORE/1` before returning semantically meaningful answers.
---
## 9. How Specific Backends Fit
### 9.1 Elasticsearch
* ES stores derived documents keyed by canonical Reference hex.
* Projector writes ES docs and persists checkpoint records (`projection/checkpoint`) in ASL.
* ES is never authoritative; APIs must verify.
### 9.2 Lakehouse Snapshots (Nessie/Spark)
* Projector produces snapshot datasets derived from authoritative sources.
* Snapshot identifiers should be tied to a checkpoint boundary (log_position) for reproducibility.
### 9.3 IPFS Anchoring
* IPFS pinning is an export/distribution projection.
* Projector should emit pin/export receipts as artifacts (recommended).
* IPFS availability is not semantic truth.
---
## 10. Operational Guidance
### 10.1 Failure Handling
* Projector must be restartable from the last checkpoint.
* Partial writes to derived stores must not corrupt checkpoint progression:
* either write side effects then checkpoint, or
* checkpoint only after side effects are durably confirmed (policy choice; document per projector)
### 10.2 Observability
Expose:
* head log_position
* checkpoint log_position
* lag
* projection error counts / dead-letter queue
* rebuild duration
* verification fallback rate (for ES-backed APIs)
---
## 11. Non-Goals
This pattern does not define:
* daemon orchestration
* distributed consensus
* tenancy/authz
* semantic promotion of projections (governance handles promotion explicitly)
---
## 12. Design Outcome
This projector pattern ensures:
* replayable derived systems,
* stable restart/resume via checkpoint records,
* deterministic behavior independent of paging,
* strict separation of truth vs acceleration,
* safe foundations for MinIO/Janus/ES/IPFS without semantic drift.

201
docs/spec-clarifications.md
View file

@ -1,201 +0,0 @@
# Spec Clarifications
This document records implementation-level clarifications for draft Tier-1
specs. These notes do not change the specs; they document concrete choices for
the implementation in this repository.
## Glossary and Abbreviations
| Term | Meaning |
| --- | --- |
| CURRENT | Effective index state after replaying a log position on a snapshot. |
| LogPosition | Inclusive `logseq` upper bound for replay (not a byte offset). |
| SnapshotID | Opaque `uint64_t` identifier persisted in `SNAPSHOT_ANCHOR`. |
| Segment seal | Log record admitting a segment via `(segment_id, segment_hash)`. |
| Segment hash | SHA-256 over exact on-disk segment bytes, including footer. |
| Tombstone | Visibility policy record applied during replay. |
| Tombstone lift | Cancels a specific tombstone record for the same artifact. |
| Exec plan | Serialized plan format; executor out of scope for core library. |
## Snapshot and Log Identity (ASL/STORE-INDEX + ASL/LOG)
Decision:
- LogPosition is the log sequence number (`logseq`), not a byte offset.
- SnapshotID is an opaque store-assigned `uint64_t`, persisted in the
`SNAPSHOT_ANCHOR` payload.
Implications:
- `IndexState = (SnapshotID, LogPosition)` uses an inclusive logseq upper bound
when replaying `log[0:LogPosition]`.
- The log's record envelope already carries `logseq`, so snapshot anchors use
the anchor record's `logseq` as the snapshot log position.
- If no snapshot exists, treat SnapshotID as `0` and LogPosition as `0`.
Rationale:
- `ASL/LOG/1` defines replay and visibility in terms of `logseq` ordering.
- `ASL/TGK-EXEC-PLAN/1` orders results by `logseq` and uses `log_prefix` bounds.
- `ASL/STORE-INDEX/1` defines LogPosition as a monotonic integer position and
replay as `log[0:LogPosition]`, which maps directly to logseq.
References:
- `tier1/asl-log-1.md`
- `tier1/enc-asl-log-1.md`
- `tier1/asl-store-index-1.md`
- `tier1/asl-tgk-execution-plan-1.md`
- `tier1/enc-asl-tgk-exec-plan-1.md`
## Index Segment Identity and Seals (ASL/STORE-INDEX + ASL/LOG)
Decision:
- `segment_id` is a store-local, monotonic `uint64_t` assigned when a segment is
created (before writing records), and persisted by naming/metadata outside the
segment file.
- `segment_hash` is SHA-256 over the exact segment file bytes as stored on disk,
including header, records, digest bytes, extents, and footer.
Implications:
- The seal record (`SEGMENT_SEAL`) binds a specific persisted segment file to the
log via `(segment_id, segment_hash)`. Hashing occurs after the footer is
written so the hash commits to seal metadata (CRC, seal snapshot, timestamp).
- Replay uses `segment_id` to locate the segment file and verifies
`segment_hash` before admitting it as visible.
Rationale:
- `ENC/ASL-LOG/1` defines the seal payload as a segment ID plus a hash of the
segment bytes; the log is the visibility gate, so the hash must cover the
complete on-disk segment.
- `ENC/ASL-CORE-INDEX/1` does not embed a segment ID, so the ID must be an
external, store-managed handle (filename or catalog entry).
References:
- `tier1/asl-log-1.md`
- `tier1/enc-asl-log-1.md`
- `tier1/asl-store-index-1.md`
- `tier1/enc-asl-core-index-1.md`
## Tombstone Semantics (ASL/LOG + ASL/STORE-INDEX)
Decision:
- `scope` and `reason_code` are opaque metadata and do not affect shadowing.
- A `TOMBSTONE_LIFT` cancels only the referenced tombstone record for the same
artifact; other tombstones for that artifact remain effective.
Across snapshots:
- Snapshots capture the effective tombstone state as of the snapshot's `logseq`.
- Lifts recorded after a snapshot become effective only when replay reaches
their `logseq`.
References:
- `tier1/asl-log-1.md`
- `tier1/asl-store-index-1.md`
## Federation Fields (ENC/ASL-CORE-INDEX)
Decision:
- Version 3 encoders must always emit federation fields in both headers and
records. They are required, not optional, in v3.
- Decoders accept legacy versions that omit federation fields and apply default
local/internal values as defined in the encoding spec.
References:
- `tier1/enc-asl-core-index-1.md`
## Execution Plan Scope (ASL/TGK-EXEC-PLAN + ENC/ASL-TGK-EXEC-PLAN)
Decision:
- The implementation treats execution plans as a serialized/transport artifact
and semantic contract only. A plan executor is out of scope for the core
library.
References:
- `tier1/asl-tgk-execution-plan-1.md`
- `tier1/enc-asl-tgk-exec-plan-1.md`
## Publish/Unpublish Scope (ASL/LOG + ASL/SYSTEM)
Decision:
- `ARTIFACT_PUBLISH` and `ARTIFACT_UNPUBLISH` are treated as reserved record
types in the core replay path and do not alter ASL index state.
- Publishing is modeled as moving artifacts and index segments between stores,
advancing the destination store's snapshot/log.
Implications:
- Core replay ignores publish/unpublish records.
- Any visibility policy tied to publishing is handled by higher-level tooling
or system-layer orchestration, not ASL/1 core semantics.
References:
- `tier1/asl-log-1.md`
- `tier1/asl-system-1.md`
## Receipt Output Reference Fallback (FER/1 + PEL/1)
Decision:
- When a PEL run produces no output artifacts (e.g. failed execution), the
receipt's `output_ref` falls back to the stored PEL result artifact reference.
Implications:
- Receipts can be emitted for both successful and failed runs using a single
canonical output reference.
- Callers using `amduat_fer1_receipt_from_pel_run` should expect `output_ref`
to match `result_ref` when `output_refs_len == 0`.
References:
- `tier1/enc-fer1-receipt-1.md`
- `tier1/srs.md`
## FER/1 v1.1 Determinism and Validation (FER/1 + SRS)
Decision:
- `run_id` is a deterministic hash over stable inputs only and MUST exclude
timestamps, logs, or mutable metadata.
- Typed logs are optional; if present they MUST be ordered and size-bounded.
- Limits are a single required record when the `limits` TLV is present.
- Executor set verification is strict when a policy-provided set exists.
Concrete rules:
- `run_id = H("AMDUAT:RUN\0" || EncRef(function) || EncRef(input_manifest) ||
EncRef(environment) || EncRef(executor_fingerprint))`, where `EncRef` is
`ENC/ASL1-CORE` canonical bytes and `executor_fingerprint` is the canonical
digest reference. No other fields are included.
- `logs` (if present): order by `(kind, cid)` byte-lexicographically; cap to
64 entries; cap total log payload references to 1 MiB aggregate of capsule
bytes. Reject out-of-order or oversized sets.
- `limits` (if present): exactly one TLV containing all numeric fields
(`cpu_ms`, `wall_ms`, `max_rss_kib`, `io_reads`, `io_writes`) with fixed
units. Reject missing or duplicate fields.
- Executor set validation:
- If an expected executor set is supplied by policy, receipt executor_refs
MUST match it exactly (same members, byte-order, no extras).
- Otherwise, validate strict ordering and uniqueness, and require
`parity_len == executor_refs_len` with aligned ordering and `output_ref`
equality for every parity entry.
References:
- `tier1/srs.md`
- `tier1/enc-fer1-receipt-1.md`
## FER/1 v1.1 Encoding Notes (Implementation)
Decision:
- The v1.1 encoder appends a TLV extension block after the v1 base layout.
- Unknown or duplicate TLV tags are rejected during decode.
TLV tags (implementation):
- `0x0001` executor fingerprint reference (encoded reference bytes).
- `0x0002` run id (`U32` length + bytes).
- `0x0003` logs (`U32` count; per entry: `U32 kind`, encoded ref, `U32` sha256
length + bytes). Entries must be ordered by `(kind, ref)` byte order.
- `0x0004` limits (`U64` cpu_ms, `U64` wall_ms, `U64` max_rss_kib,
`U64` io_reads, `U64` io_writes).
- `0x0005` determinism (`U8` level, `U32` seed_len + seed bytes).
- `0x0006` signature (opaque bytes).
Helper usage:
- `amduat_fer1_receipt_from_pel_run_v1_1` emits v1.1 receipts and uses the
same output_ref fallback as v1: when no outputs exist, `output_ref` is the
stored PEL result reference.
References:
- `include/amduat/enc/fer1_receipt.h`
- `src/near_core/enc/fer1_receipt.c`

76
docs/tgk-ordering-enforcement.md
View file

@ -1,76 +0,0 @@
# TGK Ordering Enforcement
Status: Draft
Scope: Canonical ordering rules for TGK/STORE/1 and where to enforce them in adapters and query pipelines.
---
# 1. Purpose
TGK/STORE/1 defines a canonical deterministic ordering for all edge and neighbor query results. This document records the rule and the enforcement points so implementations do not drift.
---
# 2. Canonical Ordering Rules
## 2.1 Edge Ordering
All adjacency and scan results MUST be ordered by:
1. For each edge `(edge_ref, body)`, form:
`order_key(edge_ref) = hash_id_bytes || digest_bytes`
2. `hash_id_bytes` is `edge_ref.hash_id` encoded as big-endian `u16`.
3. `digest_bytes` is the raw `edge_ref.digest`.
4. Sort edges in ascending lexicographic order of `order_key(edge_ref)`.
This is the normative ordering defined by TGK/STORE/1 and is equivalent to sorting by canonical `ReferenceBytes`.
## 2.2 Neighbor Ordering
If `neighbors` is implemented, neighbor results MUST be ordered by:
1. For each node `n`, form:
`node_order_key(n) = hash_id_bytes || digest_bytes`
2. `hash_id_bytes` is `n.hash_id` encoded as big-endian `u16`.
3. `digest_bytes` is the raw `n.digest`.
4. Sort neighbors in ascending lexicographic order of `node_order_key(n)`.
---
# 3. Enforcement Points
Ordering MUST be enforced by TGK/STORE/1 logic, not by backend defaults.
Acceptable enforcement points:
1. Adapter layer (preferred): apply ordering before returning `GraphEdgeView` lists.
2. Canonical result transformation layer: a shared sorting function used by all adapters.
Disallowed enforcement:
1. Relying on JanusGraph traversal order.
2. Relying on Elasticsearch sort order.
3. Any backend-specific implicit ordering.
---
# 4. Test Requirements
Conformance tests MUST verify:
1. Adjacency results are ordered per the canonical `(hash_id, digest)` rule.
2. `scan_edges` results follow the same ordering per page.
3. `neighbors` results use the canonical node ordering if the operation is exposed.
4. Cross-backend equivalence: the same snapshot yields identical ordered lists.
---
# 5. Implementation Notes
1. Sorting uses the canonical `ReferenceBytes` ordering for `edge_ref` and `node` references.
2. Ordering is snapshot-relative; it must be stable for a fixed snapshot.
3. If ES is used for candidate retrieval, the authoritative layer must re-apply ordering.

59
include/amduat/asl/asl_derivation_index_fs.h
View file

@ -1,59 +0,0 @@
#ifndef AMDUAT_ASL_DERIVATION_INDEX_FS_H
#define AMDUAT_ASL_DERIVATION_INDEX_FS_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_DERIVATION_INDEX_FS_ROOT_MAX = 1024 };
typedef struct {
amduat_octets_t sid;
amduat_reference_t program_ref;
uint32_t output_index;
amduat_reference_t *input_refs;
size_t input_refs_len;
bool has_params_ref;
amduat_reference_t params_ref;
bool has_exec_profile;
amduat_octets_t exec_profile;
} amduat_asl_derivation_record_t;
typedef struct {
char root_path[AMDUAT_ASL_DERIVATION_INDEX_FS_ROOT_MAX];
} amduat_asl_derivation_index_fs_t;
bool amduat_asl_derivation_index_fs_init(
amduat_asl_derivation_index_fs_t *index,
const char *root_path);
amduat_asl_store_error_t amduat_asl_derivation_index_fs_add(
amduat_asl_derivation_index_fs_t *index,
amduat_reference_t artifact_ref,
const amduat_asl_derivation_record_t *record);
amduat_asl_store_error_t amduat_asl_derivation_index_fs_list(
amduat_asl_derivation_index_fs_t *index,
amduat_reference_t artifact_ref,
amduat_asl_derivation_record_t **out_records,
size_t *out_count);
void amduat_asl_derivation_record_free(
amduat_asl_derivation_record_t *record);
void amduat_asl_derivation_records_free(
amduat_asl_derivation_record_t *records,
size_t count);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_DERIVATION_INDEX_FS_H */

40
include/amduat/asl/asl_materialization_cache_fs.h
View file

@ -1,40 +0,0 @@
#ifndef AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H
#define AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX = 1024 };
typedef struct {
char root_path[AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX];
} amduat_asl_materialization_cache_fs_t;
bool amduat_asl_materialization_cache_fs_init(
amduat_asl_materialization_cache_fs_t *cache,
const char *root_path);
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_get(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
amduat_reference_t **out_refs,
size_t *out_refs_len);
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_put(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
const amduat_reference_t *refs,
size_t refs_len);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_MATERIALIZATION_CACHE_FS_H */

58
include/amduat/asl/asl_pointer_fs.h
View file

@ -1,58 +0,0 @@
#ifndef AMDUAT_ASL_POINTER_FS_H
#define AMDUAT_ASL_POINTER_FS_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_POINTER_FS_ROOT_MAX = 1024,
AMDUAT_ASL_POINTER_NAME_MAX = 512
};
typedef enum {
AMDUAT_ASL_POINTER_OK = 0,
AMDUAT_ASL_POINTER_ERR_NOT_FOUND = 1,
AMDUAT_ASL_POINTER_ERR_IO = 2,
AMDUAT_ASL_POINTER_ERR_INVALID_NAME = 3,
AMDUAT_ASL_POINTER_ERR_INTEGRITY = 4
} amduat_asl_pointer_error_t;
typedef struct {
char root_path[AMDUAT_ASL_POINTER_FS_ROOT_MAX];
} amduat_asl_pointer_store_t;
/* Pointers are generic control-plane names for artifacts (e.g. dataset heads).
* Example names:
* - space/123/dataset/calendar_types/head
* - space/123/collection/events/head
*/
bool amduat_asl_pointer_store_init(amduat_asl_pointer_store_t *ps,
const char *root_path);
amduat_asl_pointer_error_t amduat_asl_pointer_get(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool *out_exists,
amduat_reference_t *out_ref);
amduat_asl_pointer_error_t amduat_asl_pointer_cas(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool expected_exists,
const amduat_reference_t *expected_ref,
const amduat_reference_t *new_ref,
bool *out_swapped);
bool amduat_asl_pointer_name_is_valid(const char *name);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_POINTER_FS_H */

113
include/amduat/asl/asl_store_index_fs.h
View file

@ -1,113 +0,0 @@
#ifndef AMDUAT_ASL_STORE_INDEX_FS_H
#define AMDUAT_ASL_STORE_INDEX_FS_H
#include "amduat/asl/store.h"
#include <pthread.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_STORE_INDEX_FS_ROOT_MAX = 1024 };
enum {
AMDUAT_ASL_STORE_INDEX_FS_NEG_CACHE_CAP = 256,
AMDUAT_ASL_STORE_INDEX_FS_NEG_CACHE_DIGEST_MAX = 64
};
typedef struct {
bool enabled;
uint64_t max_pending_bytes;
uint64_t idle_time_ns;
} amduat_asl_store_index_fs_snapshot_policy_t;
typedef struct {
uint64_t max_segment_records;
uint64_t max_segment_bytes;
uint64_t small_artifact_threshold;
bool allow_deferred_visibility;
} amduat_asl_store_index_fs_segment_policy_t;
typedef struct {
uint32_t segment_domain_id;
uint8_t record_visibility;
} amduat_asl_store_index_fs_visibility_policy_t;
typedef struct {
bool valid;
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t log_position;
amduat_hash_id_t hash_id;
uint16_t digest_len;
uint8_t digest[AMDUAT_ASL_STORE_INDEX_FS_NEG_CACHE_DIGEST_MAX];
} amduat_asl_store_index_fs_neg_cache_entry_t;
typedef struct {
amduat_asl_store_config_t config;
amduat_asl_store_index_fs_snapshot_policy_t snapshot_policy;
amduat_asl_store_index_fs_segment_policy_t segment_policy;
amduat_asl_store_index_fs_visibility_policy_t visibility_policy;
uint16_t shard_count;
char root_path[AMDUAT_ASL_STORE_INDEX_FS_ROOT_MAX];
uint64_t pending_snapshot_bytes;
uint64_t last_ingest_time_ns;
amduat_asl_snapshot_id_t next_snapshot_id;
bool snapshot_state_initialized;
bool log_tail_initialized;
uint64_t log_tail_logseq;
uint8_t log_tail_hash[32];
bool skip_exists_precheck;
void *open_segments;
amduat_asl_store_index_fs_neg_cache_entry_t
neg_cache[AMDUAT_ASL_STORE_INDEX_FS_NEG_CACHE_CAP];
uint16_t neg_cache_next;
pthread_mutex_t neg_cache_mutex;
pthread_mutex_t write_mutex;
uint32_t write_depth;
int write_lock_fd;
} amduat_asl_store_index_fs_t;
bool amduat_asl_store_index_fs_init(amduat_asl_store_index_fs_t *fs,
amduat_asl_store_config_t config,
const char *root_path);
void amduat_asl_store_index_fs_set_snapshot_policy(
amduat_asl_store_index_fs_t *fs,
amduat_asl_store_index_fs_snapshot_policy_t policy);
void amduat_asl_store_index_fs_set_segment_policy(
amduat_asl_store_index_fs_t *fs,
amduat_asl_store_index_fs_segment_policy_t policy);
void amduat_asl_store_index_fs_set_visibility_policy(
amduat_asl_store_index_fs_t *fs,
amduat_asl_store_index_fs_visibility_policy_t policy);
void amduat_asl_store_index_fs_set_shard_count(
amduat_asl_store_index_fs_t *fs,
uint16_t shard_count);
amduat_asl_store_error_t amduat_asl_store_index_fs_snapshot_create(
amduat_asl_store_index_fs_t *fs,
amduat_asl_snapshot_id_t snapshot_id,
amduat_asl_log_position_t *out_logseq,
uint8_t out_root_hash[32]);
amduat_asl_store_error_t amduat_asl_store_index_fs_flush(
amduat_asl_store_index_fs_t *fs,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_index_fs_gc(
amduat_asl_store_index_fs_t *fs,
const amduat_asl_index_state_t *state);
amduat_asl_store_ops_t amduat_asl_store_index_fs_ops(void);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_STORE_INDEX_FS_H */

67
include/amduat/asl/asl_store_minio.h
View file

@ -1,67 +0,0 @@
#ifndef AMDUAT_ASL_STORE_MINIO_H
#define AMDUAT_ASL_STORE_MINIO_H
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_STORE_MINIO_ENDPOINT_MAX = 1024,
AMDUAT_ASL_STORE_MINIO_HOST_MAX = 256,
AMDUAT_ASL_STORE_MINIO_BUCKET_MAX = 256,
AMDUAT_ASL_STORE_MINIO_ACCESS_KEY_MAX = 256,
AMDUAT_ASL_STORE_MINIO_SECRET_KEY_MAX = 256,
AMDUAT_ASL_STORE_MINIO_REGION_MAX = 64,
AMDUAT_ASL_STORE_MINIO_PREFIX_MAX = 128
};
typedef struct {
amduat_asl_store_config_t store_config;
const char *endpoint;
const char *bucket;
const char *access_key;
const char *secret_key;
const char *region;
const char *prefix;
bool force_path_style;
bool tls_verify;
uint32_t timeout_ms;
uint32_t retry_max;
} amduat_asl_store_minio_config_t;
typedef struct {
amduat_asl_store_config_t store_config;
char endpoint[AMDUAT_ASL_STORE_MINIO_ENDPOINT_MAX];
char host[AMDUAT_ASL_STORE_MINIO_HOST_MAX];
uint16_t port;
bool use_tls;
char bucket[AMDUAT_ASL_STORE_MINIO_BUCKET_MAX];
char access_key[AMDUAT_ASL_STORE_MINIO_ACCESS_KEY_MAX];
char secret_key[AMDUAT_ASL_STORE_MINIO_SECRET_KEY_MAX];
char region[AMDUAT_ASL_STORE_MINIO_REGION_MAX];
char prefix[AMDUAT_ASL_STORE_MINIO_PREFIX_MAX];
bool force_path_style;
bool tls_verify;
uint32_t timeout_ms;
uint32_t retry_max;
} amduat_asl_store_minio_t;
void amduat_asl_store_minio_config_defaults(
amduat_asl_store_minio_config_t *config);
bool amduat_asl_store_minio_init(amduat_asl_store_minio_t *minio,
amduat_asl_store_minio_config_t config);
amduat_asl_store_ops_t amduat_asl_store_minio_ops(void);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_STORE_MINIO_H */

81
include/amduat/asl/collection.h
View file

@ -1,81 +0,0 @@
#ifndef AMDUAT_ASL_COLLECTION_H
#define AMDUAT_ASL_COLLECTION_H
#include "amduat/asl/asl_pointer_fs.h"
#include "amduat/asl/core.h"
#include "amduat/asl/log_store.h"
#include "amduat/asl/record.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_ASL_COLLECTION_OK = 0,
AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME = 1,
AMDUAT_ASL_COLLECTION_ERR_IO = 2,
AMDUAT_ASL_COLLECTION_ERR_INTEGRITY = 3,
AMDUAT_ASL_COLLECTION_ERR_CAS_MISMATCH = 4
} amduat_asl_collection_error_t;
typedef struct {
amduat_asl_store_t *store;
amduat_asl_log_store_t log_store;
amduat_asl_pointer_store_t pointer_store;
} amduat_asl_collection_store_t;
typedef struct {
uint64_t snapshot_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_collection_snapshot_payload_t;
bool amduat_asl_collection_store_init(
amduat_asl_collection_store_t *collection_store,
const char *root_path,
amduat_asl_store_t *store);
amduat_asl_collection_error_t amduat_asl_collection_append(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
amduat_reference_t record_ref,
uint16_t kind,
amduat_octets_t actor,
uint64_t *out_offset);
amduat_asl_collection_error_t amduat_asl_collection_snapshot(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t up_to_offset,
amduat_reference_t *out_snapshot_ref,
bool *out_swapped);
amduat_asl_collection_error_t amduat_asl_collection_read(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t from_offset,
size_t limit,
amduat_reference_t **out_record_refs,
size_t *out_len,
uint64_t *out_next_offset,
bool *out_end);
void amduat_asl_collection_refs_free(amduat_reference_t *refs,
size_t refs_len);
bool amduat_asl_collection_snapshot_payload_decode_v1(
amduat_octets_t payload,
amduat_asl_collection_snapshot_payload_t *out_snapshot);
void amduat_asl_collection_snapshot_payload_free(
amduat_asl_collection_snapshot_payload_t *snapshot);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_COLLECTION_H */

74
include/amduat/asl/collection_view.h
View file

@ -1,74 +0,0 @@
#ifndef AMDUAT_ASL_COLLECTION_VIEW_H
#define AMDUAT_ASL_COLLECTION_VIEW_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_COLLECTION_VIEW_1 = 0x00000403u };
enum { AMDUAT_TYPE_TAG_ASL_COLLECTION_VIEW_1 = TYPE_TAG_ASL_COLLECTION_VIEW_1 };
enum { TYPE_TAG_ASL_SNAPSHOT_INFO_1 = 0x00000405u };
enum { AMDUAT_TYPE_TAG_ASL_SNAPSHOT_INFO_1 = TYPE_TAG_ASL_SNAPSHOT_INFO_1 };
enum { TYPE_TAG_ASL_LOG_RANGE_1 = 0x00000406u };
enum { AMDUAT_TYPE_TAG_ASL_LOG_RANGE_1 = TYPE_TAG_ASL_LOG_RANGE_1 };
typedef struct {
uint64_t snapshot_at_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_collection_snapshot_info_t;
typedef struct {
uint64_t start_offset;
uint64_t next_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_log_range_t;
typedef struct {
uint64_t computed_from_offset;
uint64_t computed_up_to_offset;
amduat_reference_t *refs;
size_t refs_len;
} amduat_asl_collection_view_t;
bool amduat_asl_collection_snapshot_info_encode_v1(
const amduat_asl_collection_snapshot_info_t *info,
amduat_octets_t *out_bytes);
bool amduat_asl_collection_snapshot_info_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_snapshot_info_t *out_info);
void amduat_asl_collection_snapshot_info_free(
amduat_asl_collection_snapshot_info_t *info);
bool amduat_asl_log_range_encode_v1(const amduat_asl_log_range_t *range,
amduat_octets_t *out_bytes);
bool amduat_asl_log_range_decode_v1(amduat_octets_t bytes,
amduat_asl_log_range_t *out_range);
void amduat_asl_log_range_free(amduat_asl_log_range_t *range);
bool amduat_asl_collection_view_encode_v1(
const amduat_asl_collection_view_t *view,
amduat_octets_t *out_bytes);
bool amduat_asl_collection_view_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_view_t *out_view);
void amduat_asl_collection_view_free(amduat_asl_collection_view_t *view);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_COLLECTION_VIEW_H */

28
include/amduat/asl/identity.h
View file

@ -1,28 +0,0 @@
#ifndef AMDUAT_ASL_IDENTITY_H
#define AMDUAT_ASL_IDENTITY_H
#include "amduat/asl/core.h"
#include "amduat/hash/asl1.h"
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
const char *hash_registry;
uint16_t hash_registry_version;
amduat_octets_t domain_separator;
} amduat_asl_identity_rules_t;
/* Domain separator for ASL/1 artifact identity hashing. */
amduat_octets_t amduat_asl_identity_domain_separator(void);
const amduat_asl_identity_rules_t *amduat_asl_identity_rules(void);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_IDENTITY_H */

33
include/amduat/asl/index_accel.h
View file

@ -1,33 +0,0 @@
#ifndef AMDUAT_ASL_INDEX_ACCEL_H
#define AMDUAT_ASL_INDEX_ACCEL_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
bool amduat_asl_index_accel_routing_key_from_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
amduat_octets_t *out_key);
bool amduat_asl_index_accel_routing_key_hash(amduat_octets_t key,
uint64_t *out_hash);
uint16_t amduat_asl_index_accel_shard_for_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
uint16_t shard_count);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_ACCEL_H */

33
include/amduat/asl/index_bloom.h
View file

@ -1,33 +0,0 @@
#ifndef AMDUAT_ASL_INDEX_BLOOM_H
#define AMDUAT_ASL_INDEX_BLOOM_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_INDEX_BLOOM_BYTES = 256,
AMDUAT_ASL_INDEX_BLOOM_HASHES = 4
};
bool amduat_asl_index_bloom_init(amduat_octets_t *out_bloom);
bool amduat_asl_index_bloom_add(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest);
bool amduat_asl_index_bloom_maybe_contains(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_BLOOM_H */

47
include/amduat/asl/index_replay.h
View file

@ -1,47 +0,0 @@
#ifndef AMDUAT_ASL_INDEX_REPLAY_H
#define AMDUAT_ASL_INDEX_REPLAY_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include "amduat/enc/asl_log.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
uint64_t segment_id;
uint8_t segment_hash[32];
} amduat_asl_segment_seal_t;
typedef struct {
amduat_reference_t ref;
uint64_t tombstone_logseq;
} amduat_asl_tombstone_entry_t;
typedef struct {
amduat_asl_segment_seal_t *segments;
size_t segments_len;
amduat_asl_tombstone_entry_t *tombstones;
size_t tombstones_len;
amduat_asl_index_state_t state;
} amduat_asl_replay_state_t;
bool amduat_asl_replay_init(amduat_asl_replay_state_t *out);
void amduat_asl_replay_free(amduat_asl_replay_state_t *state);
bool amduat_asl_replay_apply_log(
const amduat_asl_log_record_t *records,
size_t record_count,
amduat_asl_log_position_t log_position,
amduat_asl_replay_state_t *state);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_REPLAY_H */

48
include/amduat/asl/index_snapshot.h
View file

@ -1,48 +0,0 @@
#ifndef AMDUAT_ASL_INDEX_SNAPSHOT_H
#define AMDUAT_ASL_INDEX_SNAPSHOT_H
#include "amduat/asl/index_replay.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION = 1,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE = 80,
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN = 8
};
typedef struct {
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t anchor_logseq;
amduat_asl_store_config_t config;
amduat_asl_segment_seal_t *segments;
size_t segments_len;
amduat_asl_tombstone_entry_t *tombstones;
size_t tombstones_len;
} amduat_asl_snapshot_manifest_t;
bool amduat_asl_snapshot_manifest_write(
const char *path,
const amduat_asl_snapshot_manifest_t *manifest,
uint8_t out_hash[32]);
bool amduat_asl_snapshot_manifest_read(
const char *path,
amduat_asl_snapshot_manifest_t *out_manifest,
uint8_t out_hash[32]);
void amduat_asl_snapshot_manifest_free(
amduat_asl_snapshot_manifest_t *manifest);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_INDEX_SNAPSHOT_H */

30
include/amduat/asl/log_cursor.h
View file

@ -1,30 +0,0 @@
#ifndef AMDUAT_ASL_LOG_CURSOR_H
#define AMDUAT_ASL_LOG_CURSOR_H
#include "amduat/asl/store.h"
#include "amduat/enc/asl_log.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/*
* Read log records with logseq > after_logseq.
* Caller owns out_records; free with amduat_enc_asl_log_free.
*/
bool amduat_asl_log_read_since(amduat_asl_store_t *store,
amduat_asl_log_position_t after_logseq,
size_t limit,
amduat_asl_log_record_t **out_records,
size_t *out_count,
amduat_asl_log_position_t *out_next_logseq,
bool *out_end);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_LOG_CURSOR_H */

78
include/amduat/asl/log_store.h
View file

@ -1,78 +0,0 @@
#ifndef AMDUAT_ASL_LOG_STORE_H
#define AMDUAT_ASL_LOG_STORE_H
#include "amduat/asl/asl_pointer_fs.h"
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_LOG_CHUNK_1 = 0x00000401u };
enum { AMDUAT_TYPE_TAG_ASL_LOG_CHUNK_1 = TYPE_TAG_ASL_LOG_CHUNK_1 };
typedef struct {
uint16_t kind;
bool has_timestamp;
uint64_t timestamp;
amduat_reference_t payload_ref;
bool has_actor;
amduat_octets_t actor;
} amduat_asl_log_entry_t;
typedef struct {
bool has_prev;
amduat_reference_t prev_ref;
uint64_t base_offset;
uint32_t entry_count;
bool has_timestamp;
bool has_actor;
amduat_asl_log_entry_t *entries;
} amduat_asl_log_chunk_t;
typedef struct {
amduat_asl_store_t *store;
amduat_asl_pointer_store_t pointer_store;
} amduat_asl_log_store_t;
bool amduat_asl_log_store_init(amduat_asl_log_store_t *log_store,
const char *root_path,
amduat_asl_store_t *store,
const amduat_asl_pointer_store_t *pointer_store);
amduat_asl_store_error_t amduat_asl_log_append(
amduat_asl_log_store_t *log_store,
const char *log_name,
const amduat_asl_log_entry_t *entries,
size_t entries_len,
uint64_t *out_first_offset);
amduat_asl_store_error_t amduat_asl_log_read(
amduat_asl_log_store_t *log_store,
const char *log_name,
uint64_t from_offset,
size_t max_entries,
amduat_asl_log_entry_t **out_entries,
size_t *out_len,
uint64_t *out_next_offset,
bool *out_end);
void amduat_asl_log_entries_free(amduat_asl_log_entry_t *entries,
size_t entries_len);
amduat_asl_store_error_t amduat_asl_log_chunk_decode_v1(
amduat_octets_t bytes,
amduat_asl_log_chunk_t *out_chunk);
void amduat_asl_log_chunk_free(amduat_asl_log_chunk_t *chunk);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_LOG_STORE_H */

27
include/amduat/asl/none.h
View file

@ -1,27 +0,0 @@
#ifndef AMDUAT_ASL_NONE_H
#define AMDUAT_ASL_NONE_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_NONE_1 = 0x00000404u };
enum { AMDUAT_TYPE_TAG_ASL_NONE_1 = TYPE_TAG_ASL_NONE_1 };
bool amduat_asl_none_encode_v1(amduat_octets_t *out_bytes);
bool amduat_asl_none_decode_v1(amduat_octets_t bytes);
bool amduat_asl_none_is_artifact(const amduat_artifact_t *artifact);
bool amduat_asl_none_artifact(amduat_artifact_t *out_artifact);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_NONE_H */

54
include/amduat/asl/projection_checkpoint.h
View file

@ -1,54 +0,0 @@
#ifndef AMDUAT_ASL_PROJECTION_CHECKPOINT_H
#define AMDUAT_ASL_PROJECTION_CHECKPOINT_H
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_PROJECTION_CHECKPOINT_VERSION_V1 = 1
};
typedef struct {
uint32_t checkpoint_version;
amduat_octets_t projection_name;
uint32_t projection_schema_version;
bool has_snapshot_id;
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t log_position;
uint64_t created_at;
amduat_octets_t builder_id;
bool has_artifact_count;
uint64_t artifact_count;
bool has_edge_count;
uint64_t edge_count;
bool has_error_count;
uint64_t error_count;
bool has_lag_millis;
uint64_t lag_millis;
bool has_note;
amduat_octets_t note;
} amduat_asl_projection_checkpoint_v1_t;
bool amduat_asl_projection_checkpoint_encode_v1(
const amduat_asl_projection_checkpoint_v1_t *checkpoint,
amduat_octets_t *out_bytes);
bool amduat_asl_projection_checkpoint_decode_v1(
amduat_octets_t bytes,
amduat_asl_projection_checkpoint_v1_t *out_checkpoint);
void amduat_asl_projection_checkpoint_free(
amduat_asl_projection_checkpoint_v1_t *checkpoint);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_PROJECTION_CHECKPOINT_H */

46
include/amduat/asl/record.h
View file

@ -1,46 +0,0 @@
#ifndef AMDUAT_ASL_RECORD_H
#define AMDUAT_ASL_RECORD_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { TYPE_TAG_ASL_RECORD_1 = 0x00000402u };
enum { AMDUAT_TYPE_TAG_ASL_RECORD_1 = TYPE_TAG_ASL_RECORD_1 };
typedef struct {
amduat_octets_t schema;
amduat_octets_t payload;
} amduat_asl_record_t;
bool amduat_asl_record_encode_v1(const amduat_asl_record_t *record,
amduat_octets_t *out_bytes);
bool amduat_asl_record_decode_v1(amduat_octets_t bytes,
amduat_asl_record_t *out_record);
void amduat_asl_record_free(amduat_asl_record_t *record);
amduat_asl_store_error_t amduat_asl_record_store_put(
amduat_asl_store_t *store,
amduat_octets_t schema,
amduat_octets_t payload,
amduat_reference_t *out_ref);
amduat_asl_store_error_t amduat_asl_record_store_get(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_record_t *out_record);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_RECORD_H */

71
include/amduat/asl/store.h
View file

@ -2,7 +2,6 @@
#define AMDUAT_ASL_STORE_H
#include "amduat/asl/core.h"
#include "amduat/enc/asl_log.h"
#include <stdbool.h>
#include <stddef.h>
@ -26,14 +25,6 @@ typedef enum {
AMDUAT_ASL_STORE_ERR_IO = 4
} amduat_asl_store_error_t;
typedef uint64_t amduat_asl_snapshot_id_t;
typedef uint64_t amduat_asl_log_position_t;
typedef struct {
amduat_asl_snapshot_id_t snapshot_id;
amduat_asl_log_position_t log_position;
} amduat_asl_index_state_t;
typedef struct {
amduat_asl_store_error_t (*put)(void *ctx,
amduat_artifact_t artifact,
@ -41,29 +32,6 @@ typedef struct {
amduat_asl_store_error_t (*get)(void *ctx,
amduat_reference_t ref,
amduat_artifact_t *out_artifact);
amduat_asl_store_error_t (*put_indexed)(void *ctx,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*get_indexed)(void *ctx,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact);
amduat_asl_store_error_t (*tombstone)(void *ctx,
amduat_reference_t ref,
uint32_t scope,
uint32_t reason_code,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*tombstone_lift)(
void *ctx,
amduat_reference_t ref,
amduat_asl_log_position_t tombstone_logseq,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*log_scan)(
void *ctx,
amduat_asl_log_record_t **out_records,
size_t *out_count);
bool (*current_state)(void *ctx, amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t (*validate_config)(
void *ctx,
amduat_asl_store_config_t config);
@ -75,12 +43,6 @@ static inline void amduat_asl_store_ops_init(amduat_asl_store_ops_t *ops) {
}
ops->put = NULL;
ops->get = NULL;
ops->put_indexed = NULL;
ops->get_indexed = NULL;
ops->tombstone = NULL;
ops->tombstone_lift = NULL;
ops->log_scan = NULL;
ops->current_state = NULL;
ops->validate_config = NULL;
}
@ -103,39 +65,6 @@ amduat_asl_store_error_t amduat_asl_store_get(amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_artifact_t *out_artifact);
amduat_asl_store_error_t amduat_asl_store_put_indexed(
amduat_asl_store_t *store,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_get_indexed(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact);
amduat_asl_store_error_t amduat_asl_store_tombstone(
amduat_asl_store_t *store,
amduat_reference_t ref,
uint32_t scope,
uint32_t reason_code,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_store_tombstone_lift(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_log_position_t tombstone_logseq,
amduat_asl_index_state_t *out_state);
amduat_asl_store_error_t amduat_asl_log_scan(
amduat_asl_store_t *store,
amduat_asl_log_record_t **out_records,
size_t *out_count);
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
amduat_asl_index_state_t *out_state);
#ifdef __cplusplus
} /* extern "C" */
#endif

137
include/amduat/bytestore.h
View file

@ -1,137 +0,0 @@
#ifndef AMDUAT_BYTESTORE_H
#define AMDUAT_BYTESTORE_H
#include "amduat/asl/core.h"
#include "amduat/hash/asl1.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_BYTESTORE_OK = 0,
AMDUAT_BYTESTORE_ERR_POLICY_SIZE = 1,
AMDUAT_BYTESTORE_ERR_IDENTITY_MISMATCH = 2,
AMDUAT_BYTESTORE_ERR_STREAM_ORDER = 3,
AMDUAT_BYTESTORE_ERR_STREAM_TRUNCATED = 4,
AMDUAT_BYTESTORE_ERR_AREA_VIOLATION = 5,
AMDUAT_BYTESTORE_ERR_CRASH_SIMULATION = 6,
AMDUAT_BYTESTORE_ERR_STORE_MISSING = 7,
AMDUAT_BYTESTORE_ERR_UNSUPPORTED = 8,
AMDUAT_BYTESTORE_ERR_IO = 9,
AMDUAT_BYTESTORE_ERR_INVALID = 10
} amduat_bytestore_error_t;
typedef struct {
amduat_hash_id_t hash_id;
size_t max_object_size;
const char *public_root;
const char *secure_root;
} amduat_bytestore_config_t;
typedef struct {
bool present;
size_t size;
} amduat_bytestore_stat_t;
typedef struct {
/* Return 1 for a chunk, 0 for end-of-stream, -1 on error. */
int (*next)(void *ctx, amduat_octets_t *out_chunk);
void *ctx;
} amduat_bytestore_stream_t;
typedef struct {
amduat_bytestore_error_t (*put)(void *ctx,
amduat_octets_t payload,
amduat_reference_t *out_cid);
amduat_bytestore_error_t (*put_stream)(void *ctx,
amduat_bytestore_stream_t stream,
amduat_reference_t *out_cid);
amduat_bytestore_error_t (*import_cor)(void *ctx,
amduat_octets_t envelope,
amduat_reference_t *out_cid);
amduat_bytestore_error_t (*export_cor)(void *ctx,
amduat_reference_t cid,
amduat_octets_t *out_envelope);
amduat_bytestore_error_t (*get)(void *ctx,
amduat_reference_t cid,
amduat_octets_t *out_bytes);
amduat_bytestore_error_t (*stat)(void *ctx,
amduat_reference_t cid,
amduat_bytestore_stat_t *out_stat);
amduat_bytestore_error_t (*assert_area_isolation)(void *ctx,
const char *public_root,
const char *secure_root);
amduat_bytestore_error_t (*validate_config)(void *ctx,
amduat_bytestore_config_t config);
} amduat_bytestore_ops_t;
static inline void amduat_bytestore_ops_init(amduat_bytestore_ops_t *ops) {
if (ops == NULL) {
return;
}
ops->put = NULL;
ops->put_stream = NULL;
ops->import_cor = NULL;
ops->export_cor = NULL;
ops->get = NULL;
ops->stat = NULL;
ops->assert_area_isolation = NULL;
ops->validate_config = NULL;
}
typedef struct {
amduat_bytestore_config_t config;
amduat_bytestore_ops_t ops;
void *ctx;
} amduat_bytestore_t;
void amduat_bytestore_init(amduat_bytestore_t *store,
amduat_bytestore_config_t config,
amduat_bytestore_ops_t ops,
void *ctx);
amduat_bytestore_error_t amduat_bytestore_put(amduat_bytestore_t *store,
amduat_octets_t payload,
amduat_reference_t *out_cid);
amduat_bytestore_error_t amduat_bytestore_put_stream(
amduat_bytestore_t *store,
amduat_bytestore_stream_t stream,
amduat_reference_t *out_cid);
amduat_bytestore_error_t amduat_bytestore_import_cor(
amduat_bytestore_t *store,
amduat_octets_t envelope,
amduat_reference_t *out_cid);
amduat_bytestore_error_t amduat_bytestore_export_cor(
amduat_bytestore_t *store,
amduat_reference_t cid,
amduat_octets_t *out_envelope);
amduat_bytestore_error_t amduat_bytestore_get(amduat_bytestore_t *store,
amduat_reference_t cid,
amduat_octets_t *out_bytes);
amduat_bytestore_error_t amduat_bytestore_stat(
amduat_bytestore_t *store,
amduat_reference_t cid,
amduat_bytestore_stat_t *out_stat);
amduat_bytestore_error_t amduat_bytestore_assert_area_isolation(
amduat_bytestore_t *store,
const char *public_root,
const char *secure_root);
bool amduat_bytestore_cid_for_payload(amduat_hash_id_t hash_id,
amduat_octets_t payload,
amduat_reference_t *out_cid);
bool amduat_bytestore_cid_for_stream(amduat_hash_id_t hash_id,
amduat_bytestore_stream_t stream,
amduat_reference_t *out_cid);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_BYTESTORE_H */

101
include/amduat/enc/asl_core_index.h
View file

@ -1,101 +0,0 @@
#ifndef AMDUAT_ENC_ASL_CORE_INDEX_H
#define AMDUAT_ENC_ASL_CORE_INDEX_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_CORE_INDEX_VERSION = 3,
AMDUAT_ASL_CORE_INDEX_HEADER_SIZE = 112,
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE = 48,
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE = 16,
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE = 24
};
enum {
AMDUAT_ASL_INDEX_FLAG_TOMBSTONE = 0x00000001u
};
typedef struct {
uint64_t magic;
uint16_t version;
uint16_t shard_id;
uint32_t header_size;
uint64_t snapshot_min;
uint64_t snapshot_max;
uint64_t record_count;
uint64_t records_offset;
uint64_t bloom_offset;
uint64_t bloom_size;
uint64_t digests_offset;
uint64_t digests_size;
uint64_t extents_offset;
uint64_t extent_count;
uint32_t segment_domain_id;
uint8_t segment_visibility;
uint8_t federation_version;
uint16_t reserved0;
uint64_t flags;
} amduat_asl_segment_header_t;
typedef struct {
uint32_t hash_id;
uint16_t digest_len;
uint16_t reserved0;
uint64_t digest_offset;
uint64_t extents_offset;
uint32_t extent_count;
uint32_t total_length;
uint32_t domain_id;
uint8_t visibility;
uint8_t has_cross_domain_source;
uint16_t reserved1;
uint32_t cross_domain_source;
uint32_t flags;
} amduat_asl_index_record_t;
typedef struct {
uint64_t block_id;
uint32_t offset;
uint32_t length;
} amduat_asl_extent_record_t;
typedef struct {
uint64_t crc64;
uint64_t seal_snapshot;
uint64_t seal_time_ns;
} amduat_asl_segment_footer_t;
typedef struct {
amduat_asl_segment_header_t header;
amduat_octets_t bloom;
amduat_asl_index_record_t *records;
size_t record_count;
amduat_octets_t digests;
amduat_asl_extent_record_t *extents;
size_t extent_count;
amduat_asl_segment_footer_t footer;
} amduat_asl_core_index_segment_t;
bool amduat_enc_asl_core_index_encode_v1(
const amduat_asl_core_index_segment_t *segment,
amduat_octets_t *out_bytes);
bool amduat_enc_asl_core_index_decode_v1(
amduat_octets_t bytes,
amduat_asl_core_index_segment_t *out_segment);
void amduat_enc_asl_core_index_free(amduat_asl_core_index_segment_t *segment);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_ASL_CORE_INDEX_H */

53
include/amduat/enc/asl_log.h
View file

@ -1,53 +0,0 @@
#ifndef AMDUAT_ENC_ASL_LOG_H
#define AMDUAT_ENC_ASL_LOG_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum {
AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL = 0x01,
AMDUAT_ASL_LOG_RECORD_TOMBSTONE = 0x10,
AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT = 0x11,
AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR = 0x20,
AMDUAT_ASL_LOG_RECORD_ARTIFACT_PUBLISH = 0x30,
AMDUAT_ASL_LOG_RECORD_ARTIFACT_UNPUBLISH = 0x31
};
typedef struct {
uint64_t logseq;
uint32_t record_type;
amduat_octets_t payload;
uint8_t record_hash[32];
} amduat_asl_log_record_t;
bool amduat_enc_asl_log_encode_v1(const amduat_asl_log_record_t *records,
size_t record_count,
amduat_octets_t *out_bytes);
bool amduat_enc_asl_log_decode_v1(amduat_octets_t bytes,
amduat_asl_log_record_t **out_records,
size_t *out_count);
void amduat_enc_asl_log_free(amduat_asl_log_record_t *records,
size_t record_count);
/* Caller owns out_ref digest; free with amduat_reference_free. */
bool amduat_asl_log_decode_artifact_ref(amduat_octets_t payload,
amduat_reference_t *out_ref);
/* Caller owns out_bytes; free with amduat_octets_free. */
bool amduat_asl_log_encode_artifact_ref(amduat_reference_t ref,
amduat_octets_t *out_bytes);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_ASL_LOG_H */

115
include/amduat/enc/asl_tgk_exec_plan.h
View file

@ -1,115 +0,0 @@
#ifndef AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H
#define AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
enum { AMDUAT_ASL_TGK_EXEC_PLAN_VERSION = 1 };
typedef enum {
AMDUAT_ASL_TGK_EXEC_OP_SEGMENT_SCAN = 0,
AMDUAT_ASL_TGK_EXEC_OP_INDEX_FILTER = 1,
AMDUAT_ASL_TGK_EXEC_OP_MERGE = 2,
AMDUAT_ASL_TGK_EXEC_OP_PROJECTION = 3,
AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL = 4,
AMDUAT_ASL_TGK_EXEC_OP_AGGREGATION = 5,
AMDUAT_ASL_TGK_EXEC_OP_LIMIT_OFFSET = 6,
AMDUAT_ASL_TGK_EXEC_OP_SHARD_DISPATCH = 7,
AMDUAT_ASL_TGK_EXEC_OP_SIMD_FILTER = 8,
AMDUAT_ASL_TGK_EXEC_OP_TOMBSTONE_SHADOW = 9
} amduat_asl_tgk_exec_operator_type_t;
typedef enum {
AMDUAT_ASL_TGK_EXEC_OP_FLAG_NONE = 0x00,
AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL = 0x01,
AMDUAT_ASL_TGK_EXEC_OP_FLAG_OPTIONAL = 0x02
} amduat_asl_tgk_exec_operator_flags_t;
typedef struct {
uint64_t logseq_min;
uint64_t logseq_max;
} amduat_asl_tgk_exec_snapshot_range_t;
typedef struct {
struct {
uint8_t is_asl_segment;
uint64_t segment_start_id;
uint64_t segment_end_id;
} segment_scan;
struct {
uint32_t artifact_type_tag;
uint8_t has_type_tag;
uint32_t edge_type_key;
uint8_t has_edge_type;
uint8_t role;
} index_filter;
struct {
uint8_t deterministic;
} merge;
struct {
uint8_t project_artifact_id;
uint8_t project_tgk_edge_id;
uint8_t project_node_id;
uint8_t project_type_tag;
} projection;
struct {
uint64_t start_node_id;
uint32_t traversal_depth;
uint8_t direction;
} tgk_traversal;
struct {
uint8_t agg_count;
uint8_t agg_union;
uint8_t agg_sum;
} aggregation;
struct {
uint64_t limit;
uint64_t offset;
} limit_offset;
} amduat_asl_tgk_exec_operator_params_t;
typedef struct {
uint32_t op_id;
amduat_asl_tgk_exec_operator_type_t op_type;
amduat_asl_tgk_exec_operator_flags_t flags;
amduat_asl_tgk_exec_snapshot_range_t snapshot;
amduat_asl_tgk_exec_operator_params_t params;
uint32_t input_count;
uint32_t inputs[8];
} amduat_asl_tgk_exec_operator_def_t;
typedef struct {
uint32_t plan_version;
uint32_t operator_count;
amduat_asl_tgk_exec_operator_def_t *operators;
} amduat_asl_tgk_exec_plan_t;
bool amduat_enc_asl_tgk_exec_plan_encode_v1(
const amduat_asl_tgk_exec_plan_t *plan,
amduat_octets_t *out_bytes);
bool amduat_enc_asl_tgk_exec_plan_decode_v1(
amduat_octets_t bytes,
amduat_asl_tgk_exec_plan_t *out_plan);
void amduat_enc_asl_tgk_exec_plan_free(
amduat_asl_tgk_exec_plan_t *plan);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_ASL_TGK_EXEC_PLAN_H */

52
include/amduat/enc/fcs1.h
View file

@ -1,52 +0,0 @@
#ifndef AMDUAT_ENC_FCS1_H
#define AMDUAT_ENC_FCS1_H
#include "amduat/asl/core.h"
#include "amduat/enc/pcb1.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FCS1_OK = 0,
AMDUAT_FCS1_ERR_HEADER_INVALID = 1,
AMDUAT_FCS1_ERR_TAG_ORDER = 2,
AMDUAT_FCS1_ERR_UNKNOWN_TAG = 3,
AMDUAT_FCS1_ERR_PARAMETER_FORMAT = 4,
AMDUAT_FCS1_ERR_PCB_ARITY_MISMATCH = 5,
AMDUAT_FCS1_ERR_CYCLE_DETECTED = 6,
AMDUAT_FCS1_ERR_VARINT_NON_MINIMAL = 7,
AMDUAT_FCS1_ERR_TRAILING_BYTES = 8,
AMDUAT_FCS1_ERR_INVALID = 9
} amduat_fcs1_error_t;
typedef struct {
amduat_reference_t function_ptr;
amduat_reference_t parameter_block;
uint32_t arity;
} amduat_fcs1_descriptor_t;
typedef bool (*amduat_fcs1_resolve_fn)(void *ctx,
amduat_reference_t ref,
amduat_octets_t *out_bytes);
amduat_fcs1_error_t amduat_fcs1_decode_v1(amduat_octets_t bytes,
amduat_fcs1_descriptor_t *out_desc);
amduat_fcs1_error_t amduat_fcs1_validate_v1(
amduat_octets_t bytes,
amduat_fcs1_resolve_fn resolver,
void *resolver_ctx);
void amduat_fcs1_descriptor_free(amduat_fcs1_descriptor_t *desc);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_FCS1_H */

38
include/amduat/enc/fer1_receipt.h
View file

@ -11,10 +11,6 @@ extern "C" {
#endif
enum { FER1_RECEIPT_ENC_V1 = 0x0301u };
enum { FER1_RECEIPT_ENC_V1_1 = 0x0302u };
enum { AMDUAT_FER1_VERSION_1 = 0x0001u };
enum { AMDUAT_FER1_VERSION_1_1 = 0x0101u };
enum { TYPE_TAG_FER1_RECEIPT_1 = 0x00000301u };
enum { AMDUAT_TYPE_TAG_FER1_RECEIPT_1 = TYPE_TAG_FER1_RECEIPT_1 };
@ -27,20 +23,6 @@ typedef struct {
amduat_octets_t parity_digest;
} amduat_fer1_parity_entry_t;
typedef struct {
uint32_t kind;
amduat_reference_t log_ref;
amduat_octets_t sha256;
} amduat_fer1_log_entry_t;
typedef struct {
uint64_t cpu_ms;
uint64_t wall_ms;
uint64_t max_rss_kib;
uint64_t io_reads;
uint64_t io_writes;
} amduat_fer1_limits_t;
typedef struct {
uint16_t fer1_version;
amduat_reference_t function_ref;
@ -54,34 +36,14 @@ typedef struct {
size_t parity_len;
uint64_t started_at;
uint64_t completed_at;
bool has_executor_fingerprint_ref;
amduat_reference_t executor_fingerprint_ref;
bool has_run_id;
amduat_octets_t run_id;
bool has_limits;
amduat_fer1_limits_t limits;
amduat_fer1_log_entry_t *logs;
size_t logs_len;
bool has_determinism;
uint8_t determinism_level;
bool has_rng_seed;
amduat_octets_t rng_seed;
bool has_signature;
amduat_octets_t signature;
} amduat_fer1_receipt_t;
bool amduat_enc_fer1_receipt_encode_v1(
const amduat_fer1_receipt_t *receipt,
amduat_octets_t *out_bytes);
bool amduat_enc_fer1_receipt_encode_v1_1(
const amduat_fer1_receipt_t *receipt,
amduat_octets_t *out_bytes);
bool amduat_enc_fer1_receipt_decode_v1(
amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt);
bool amduat_enc_fer1_receipt_decode_v1_1(
amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt);
void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt);
#ifdef __cplusplus

38
include/amduat/enc/pcb1.h
View file

@ -1,38 +0,0 @@
#ifndef AMDUAT_ENC_PCB1_H
#define AMDUAT_ENC_PCB1_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_PCB1_OK = 0,
AMDUAT_PCB1_ERR_HEADER_INVALID = 1,
AMDUAT_PCB1_ERR_TAG_ORDER = 2,
AMDUAT_PCB1_ERR_UNKNOWN_TAG = 3,
AMDUAT_PCB1_ERR_MANIFEST_ORDER = 4,
AMDUAT_PCB1_ERR_DIGEST_MISMATCH = 5,
AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL = 6,
AMDUAT_PCB1_ERR_TRAILING_BYTES = 7,
AMDUAT_PCB1_ERR_INVALID = 8
} amduat_pcb1_error_t;
typedef struct {
uint32_t slot_count;
} amduat_pcb1_manifest_t;
amduat_pcb1_error_t amduat_pcb1_decode_v1(amduat_octets_t bytes,
amduat_pcb1_manifest_t *out_manifest,
amduat_octets_t *out_slot_data);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ENC_PCB1_H */

28
include/amduat/fed/ingest.h
View file

@ -1,28 +0,0 @@
#ifndef AMDUAT_FED_INGEST_H
#define AMDUAT_FED_INGEST_H
#include "amduat/fed/replay.h"
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FED_INGEST_OK = 0,
AMDUAT_FED_INGEST_ERR_INVALID = 1,
AMDUAT_FED_INGEST_ERR_CONFLICT = 2
} amduat_fed_ingest_error_t;
amduat_fed_ingest_error_t amduat_fed_ingest_validate(
const amduat_fed_record_t *records,
size_t count,
size_t *out_error_index,
size_t *out_conflict_index);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_INGEST_H */

84
include/amduat/fed/registry.h
View file

@ -1,84 +0,0 @@
#ifndef AMDUAT_FED_REGISTRY_H
#define AMDUAT_FED_REGISTRY_H
#include "amduat/asl/core.h"
#include "amduat/asl/store.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
uint32_t domain_id;
uint64_t snapshot_id;
uint64_t log_prefix;
uint64_t last_logseq;
uint8_t admitted;
uint8_t policy_ok;
uint8_t reserved[6];
amduat_hash_id_t policy_hash_id;
amduat_octets_t policy_hash; /* Empty when unknown; caller-owned bytes. */
} amduat_fed_domain_state_t;
typedef struct {
amduat_fed_domain_state_t *states;
size_t len;
size_t cap;
bool owns_states;
} amduat_fed_registry_value_t;
void amduat_fed_registry_value_init(amduat_fed_registry_value_t *value,
amduat_fed_domain_state_t *states,
size_t cap);
bool amduat_fed_registry_value_insert(amduat_fed_registry_value_t *value,
amduat_fed_domain_state_t state);
const amduat_fed_domain_state_t *amduat_fed_registry_value_lookup(
const amduat_fed_registry_value_t *value,
uint32_t domain_id);
void amduat_fed_registry_value_free(amduat_fed_registry_value_t *value);
/* Encode/Decode allocate buffers; caller frees via amduat_octets_free. */
bool amduat_fed_registry_encode(const amduat_fed_registry_value_t *value,
amduat_octets_t *out_bytes);
/* Decode allocates policy_hash bytes; free via amduat_fed_registry_value_free. */
bool amduat_fed_registry_decode(amduat_octets_t bytes,
amduat_fed_registry_value_t *out_value);
typedef enum {
AMDUAT_FED_REGISTRY_OK = 0,
AMDUAT_FED_REGISTRY_ERR_CODEC = 1,
AMDUAT_FED_REGISTRY_ERR_STORE = 2
} amduat_fed_registry_error_t;
typedef struct {
amduat_asl_store_t *store;
} amduat_fed_registry_store_t;
void amduat_fed_registry_store_init(amduat_fed_registry_store_t *reg,
amduat_asl_store_t *store);
amduat_fed_registry_error_t amduat_fed_registry_store_put(
amduat_fed_registry_store_t *reg,
const amduat_fed_registry_value_t *value,
amduat_reference_t *out_ref,
amduat_asl_store_error_t *out_store_err);
amduat_fed_registry_error_t amduat_fed_registry_store_get(
amduat_fed_registry_store_t *reg,
amduat_reference_t ref,
amduat_fed_registry_value_t *out_value,
amduat_asl_store_error_t *out_store_err);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_REGISTRY_H */

72
include/amduat/fed/replay.h
View file

@ -1,72 +0,0 @@
#ifndef AMDUAT_FED_REPLAY_H
#define AMDUAT_FED_REPLAY_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FED_REC_ARTIFACT = 0,
AMDUAT_FED_REC_PER = 1,
AMDUAT_FED_REC_TGK_EDGE = 2,
AMDUAT_FED_REC_TOMBSTONE = 3
} amduat_fed_record_type_t;
typedef struct {
uint32_t domain_id;
uint8_t visibility;
uint8_t has_source;
uint16_t reserved0;
uint32_t source_domain;
} amduat_fed_record_meta_t;
typedef struct {
amduat_fed_record_type_t type;
amduat_reference_t ref; /* PER/TGK identities are ASL references. */
} amduat_fed_record_id_t;
typedef struct {
amduat_fed_record_meta_t meta;
amduat_fed_record_id_t id;
uint64_t logseq;
uint64_t snapshot_id;
uint64_t log_prefix;
} amduat_fed_record_t;
typedef struct {
amduat_fed_record_t *records;
size_t len;
} amduat_fed_replay_view_t;
/* Caller frees record ids with amduat_fed_replay_view_free. */
bool amduat_fed_record_validate(const amduat_fed_record_t *record);
bool amduat_fed_replay_build(const amduat_fed_record_t *records,
size_t count,
uint32_t domain_id,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_replay_view_t *out_view);
/* Backwards-compatible alias for amduat_fed_replay_build. */
/* Deprecated: use amduat_fed_replay_build. */
bool amduat_fed_replay_domain(const amduat_fed_record_t *records,
size_t count,
uint32_t domain_id,
uint64_t snapshot_id,
uint64_t log_prefix,
amduat_fed_replay_view_t *out_view);
void amduat_fed_replay_view_free(amduat_fed_replay_view_t *view);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_REPLAY_H */

72
include/amduat/fed/view.h
View file

@ -1,72 +0,0 @@
#ifndef AMDUAT_FED_VIEW_H
#define AMDUAT_FED_VIEW_H
#include "amduat/asl/store.h"
#include "amduat/fed/replay.h"
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FED_RESOLVE_OK = 0,
AMDUAT_FED_RESOLVE_NOT_FOUND = 1,
AMDUAT_FED_RESOLVE_FOUND_REMOTE_NO_BYTES = 2,
AMDUAT_FED_RESOLVE_INTEGRITY_ERROR = 3,
AMDUAT_FED_RESOLVE_POLICY_DENIED = 4,
AMDUAT_FED_RESOLVE_STORE_ERROR = 5
} amduat_fed_resolve_error_t;
typedef struct {
uint32_t domain_id;
uint64_t snapshot_id;
uint64_t log_prefix;
} amduat_fed_view_bounds_t;
typedef struct {
amduat_fed_record_id_t id;
uint32_t reason_code;
} amduat_fed_policy_deny_t;
typedef struct {
uint32_t local_domain_id;
amduat_fed_record_t *records;
size_t len;
amduat_fed_policy_deny_t *denies;
size_t denies_len;
} amduat_fed_view_t;
typedef enum {
AMDUAT_FED_VIEW_OK = 0,
AMDUAT_FED_VIEW_ERR_INVALID = 1,
AMDUAT_FED_VIEW_ERR_CONFLICT = 2,
AMDUAT_FED_VIEW_ERR_OOM = 3
} amduat_fed_view_error_t;
/* Caller frees records/denies with amduat_fed_view_free. */
amduat_fed_view_error_t amduat_fed_view_build(
const amduat_fed_record_t *records,
size_t count,
uint32_t local_domain_id,
const amduat_fed_view_bounds_t *bounds,
size_t bounds_len,
const amduat_fed_policy_deny_t *denies,
size_t denies_len,
amduat_fed_view_t *out_view);
void amduat_fed_view_free(amduat_fed_view_t *view);
amduat_fed_resolve_error_t amduat_fed_resolve(
const amduat_fed_view_t *view,
amduat_asl_store_t *local_store,
amduat_reference_t ref,
amduat_artifact_t *out_artifact);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FED_VIEW_H */

42
include/amduat/fer/receipt.h
View file

@ -2,8 +2,6 @@
#define AMDUAT_FER_RECEIPT_H
#include "amduat/asl/core.h"
#include "amduat/enc/fer1_receipt.h"
#include "amduat/pel/run.h"
#include "amduat/pel/surf.h"
#include <stdbool.h>
@ -26,46 +24,6 @@ bool amduat_fer1_receipt_from_pel_result(
uint64_t completed_at,
amduat_artifact_t *out_artifact);
bool amduat_fer1_receipt_from_pel_run(
const amduat_pel_run_result_t *pel_run,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
amduat_artifact_t *out_artifact);
bool amduat_fer1_receipt_from_pel_run_v1_1(
const amduat_pel_run_result_t *pel_run,
amduat_reference_t input_manifest_ref,
amduat_reference_t environment_ref,
amduat_octets_t evaluator_id,
amduat_reference_t executor_ref,
bool has_sbom_ref,
amduat_reference_t sbom_ref,
amduat_octets_t parity_digest,
uint64_t started_at,
uint64_t completed_at,
bool has_executor_fingerprint_ref,
amduat_reference_t executor_fingerprint_ref,
bool has_run_id,
amduat_octets_t run_id,
bool has_limits,
amduat_fer1_limits_t limits,
const amduat_fer1_log_entry_t *logs,
size_t logs_len,
bool has_determinism,
uint8_t determinism_level,
bool has_rng_seed,
amduat_octets_t rng_seed,
bool has_signature,
amduat_octets_t signature,
amduat_artifact_t *out_artifact);
#ifdef __cplusplus
} /* extern "C" */
#endif

61
include/amduat/fps.h
View file

@ -1,61 +0,0 @@
#ifndef AMDUAT_FPS_H
#define AMDUAT_FPS_H
#include "amduat/asl/core.h"
#include "amduat/hash/asl1.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_FPS_OK = 0,
AMDUAT_FPS_ERR_IO_FAILURE = 1,
AMDUAT_FPS_ERR_NORMALIZATION = 2,
AMDUAT_FPS_ERR_CID_NOT_FOUND = 3,
AMDUAT_FPS_ERR_SLICE_RANGE = 4,
AMDUAT_FPS_ERR_EMPTY_INPUTS = 5,
AMDUAT_FPS_ERR_REV_ALIGNMENT = 6,
AMDUAT_FPS_ERR_INVALID_LEVEL = 7,
AMDUAT_FPS_ERR_SPLICE_RANGE = 8,
AMDUAT_FPS_ERR_INVALID = 9
} amduat_fps_error_t;
typedef enum {
AMDUAT_FPS_REVERSE_BITS = 0,
AMDUAT_FPS_REVERSE_BYTES = 1,
AMDUAT_FPS_REVERSE_WORDS = 2,
AMDUAT_FPS_REVERSE_LONGS = 3
} amduat_fps_reverse_level_t;
amduat_fps_error_t amduat_fps_slice(amduat_octets_t input,
size_t offset,
size_t length,
amduat_octets_t *out);
amduat_fps_error_t amduat_fps_concat(const amduat_octets_t *inputs,
size_t input_count,
amduat_octets_t *out);
amduat_fps_error_t amduat_fps_reverse(amduat_octets_t input,
amduat_fps_reverse_level_t level,
amduat_octets_t *out);
amduat_fps_error_t amduat_fps_splice(amduat_octets_t a,
size_t offset,
amduat_octets_t b,
amduat_octets_t *out);
bool amduat_fps_cid_for_payload(amduat_hash_id_t hash_id,
amduat_octets_t payload,
amduat_reference_t *out_cid);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_FPS_H */

14
include/amduat/hash/asl1.h
View file

@ -34,15 +34,6 @@ typedef struct {
} amduat_hash_asl1_stream_t;
enum { AMDUAT_HASH_ASL1_ID_SHA256 = 0x0001 };
enum {
AMDUAT_HASH_ASL1_VERSION_MAJOR = 0,
AMDUAT_HASH_ASL1_VERSION_MINOR = 2,
AMDUAT_HASH_ASL1_VERSION_PATCH = 5,
AMDUAT_HASH_ASL1_VERSION = 0x0205
};
#define AMDUAT_HASH_ASL1_FAMILY "ASL1"
#define AMDUAT_HASH_ASL1_REGISTRY "HASH/ASL1"
amduat_octets_t amduat_hash_asl1_key(amduat_hash_id_t hash_id, uint8_t out[2]);
@ -58,11 +49,6 @@ bool amduat_hash_asl1_digest(amduat_hash_id_t hash_id,
amduat_octets_t input,
uint8_t *out,
size_t out_len);
bool amduat_hash_asl1_digest_domain(amduat_hash_id_t hash_id,
amduat_octets_t domain_separator,
amduat_octets_t input,
uint8_t *out,
size_t out_len);
bool amduat_hash_asl1_stream_init(amduat_hash_id_t hash_id,
amduat_hash_asl1_stream_t *out);

31
include/amduat/pel/derivation_sid.h
View file

@ -1,31 +0,0 @@
#ifndef AMDUAT_PEL_DERIVATION_SID_H
#define AMDUAT_PEL_DERIVATION_SID_H
#include "amduat/asl/core.h"
#include <stdbool.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
amduat_reference_t program_ref;
const amduat_reference_t *input_refs;
size_t input_refs_len;
bool has_params_ref;
amduat_reference_t params_ref;
bool has_exec_profile;
amduat_octets_t exec_profile;
} amduat_pel_derivation_sid_input_t;
bool amduat_pel_derivation_sid_compute(
const amduat_pel_derivation_sid_input_t *in,
amduat_octets_t *out_sid);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_PEL_DERIVATION_SID_H */

37
include/amduat/pel/opreg_kernel.h
View file

@ -17,11 +17,6 @@ extern "C" {
#define AMDUAT_PEL_KERNEL_OP_HASH_ASL1_NAME "pel.bytes.hash.asl1"
#define AMDUAT_PEL_KERNEL_OP_PARAMS_NAME "pel.bytes.params"
#define AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE_NAME "pel.format.encode"
#define AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE_NAME \
"collection.snapshot_decode_v1"
#define AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE_NAME "log.read_range_v1"
#define AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS_NAME \
"collection.merge_refs_v1"
enum {
AMDUAT_PEL_KERNEL_OP_CODE_CONCAT = 0x0001u,
@ -29,10 +24,7 @@ enum {
AMDUAT_PEL_KERNEL_OP_CODE_CONST = 0x0003u,
AMDUAT_PEL_KERNEL_OP_CODE_HASH_ASL1 = 0x0004u,
AMDUAT_PEL_KERNEL_OP_CODE_PARAMS = 0x0005u,
AMDUAT_PEL_KERNEL_OP_CODE_FORMAT_ENCODE = 0x0006u,
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_SNAPSHOT_DECODE = 0x0007u,
AMDUAT_PEL_KERNEL_OP_CODE_LOG_READ_RANGE = 0x0008u,
AMDUAT_PEL_KERNEL_OP_CODE_COLLECTION_MERGE_REFS = 0x0009u
AMDUAT_PEL_KERNEL_OP_CODE_FORMAT_ENCODE = 0x0006u
};
enum {
@ -43,13 +35,7 @@ enum {
AMDUAT_PEL_KERNEL_STATUS_FORMAT_SCHEMA_INVALID = 0x00030001u,
AMDUAT_PEL_KERNEL_STATUS_FORMAT_REGISTRY_INVALID = 0x00030002u,
AMDUAT_PEL_KERNEL_STATUS_FORMAT_MISSING_FIELD = 0x00030003u,
AMDUAT_PEL_KERNEL_STATUS_FORMAT_TYPE_UNSUPPORTED = 0x00030004u,
AMDUAT_PEL_KERNEL_STATUS_PARAMS_INVALID = 0x00040001u,
AMDUAT_PEL_KERNEL_STATUS_COLLECTION_SNAPSHOT_INVALID = 0x00040002u,
AMDUAT_PEL_KERNEL_STATUS_LOG_CHUNK_INVALID = 0x00040003u,
AMDUAT_PEL_KERNEL_STATUS_LOG_RANGE_INVALID = 0x00040004u,
AMDUAT_PEL_KERNEL_STATUS_COLLECTION_VIEW_INVALID = 0x00040005u,
AMDUAT_PEL_KERNEL_STATUS_REF_RESOLVE_FAILED = 0x00040006u
AMDUAT_PEL_KERNEL_STATUS_FORMAT_TYPE_UNSUPPORTED = 0x00030004u
};
typedef enum {
@ -58,10 +44,7 @@ typedef enum {
AMDUAT_PEL_KERNEL_OP_CONST = 3,
AMDUAT_PEL_KERNEL_OP_HASH_ASL1 = 4,
AMDUAT_PEL_KERNEL_OP_PARAMS = 5,
AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE = 6,
AMDUAT_PEL_KERNEL_OP_COLLECTION_SNAPSHOT_DECODE = 7,
AMDUAT_PEL_KERNEL_OP_LOG_READ_RANGE = 8,
AMDUAT_PEL_KERNEL_OP_COLLECTION_MERGE_REFS = 9
AMDUAT_PEL_KERNEL_OP_FORMAT_ENCODE = 6
} amduat_pel_kernel_op_kind_t;
typedef struct {
@ -75,11 +58,6 @@ typedef struct {
struct amduat_pel_kernel_params;
typedef struct amduat_pel_kernel_params amduat_pel_kernel_params_t;
typedef bool (*amduat_pel_kernel_artifact_resolver_t)(
amduat_reference_t ref,
amduat_artifact_t *out_artifact,
void *ctx);
const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_lookup(
amduat_octets_t name,
uint32_t version);
@ -89,15 +67,6 @@ const amduat_pel_kernel_op_desc_t *amduat_pel_kernel_op_descs(
const char *amduat_pel_kernel_op_name(amduat_pel_kernel_op_kind_t kind);
void amduat_pel_kernel_set_artifact_resolver(
amduat_pel_kernel_artifact_resolver_t resolver,
void *ctx);
void amduat_pel_kernel_clear_artifact_resolver(void);
bool amduat_pel_kernel_resolve_artifact(amduat_reference_t ref,
amduat_artifact_t *out_artifact);
bool amduat_pel_kernel_op_eval(
const amduat_pel_kernel_op_desc_t *desc,
const amduat_artifact_t *inputs,

38
include/amduat/tgk/tgk_store_asl_index_fs.h
View file

@ -1,38 +0,0 @@
#ifndef AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H
#define AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H
/* TGK/1 projection over ASL index/log (filesystem backend). */
#include "amduat/asl/asl_store_index_fs.h"
#include "amduat/tgk/store.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
amduat_asl_store_index_fs_t *asl_fs;
amduat_asl_store_t asl_store;
amduat_tgk_store_config_t config;
amduat_asl_index_state_t pinned_state;
bool pinned;
bool use_shards;
} amduat_tgk_store_asl_index_fs_t;
bool amduat_tgk_store_asl_index_fs_init(
amduat_tgk_store_asl_index_fs_t *store,
amduat_tgk_store_config_t config,
amduat_asl_store_index_fs_t *asl_fs);
void amduat_tgk_store_asl_index_fs_free(
amduat_tgk_store_asl_index_fs_t *store);
amduat_tgk_store_ops_t amduat_tgk_store_asl_index_fs_ops(void);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_TGK_TGK_STORE_ASL_INDEX_FS_H */

23
include/amduat/util/log.h
View file

@ -1,23 +0,0 @@
#ifndef AMDUAT_UTIL_LOG_H
#define AMDUAT_UTIL_LOG_H
#include <stdarg.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
AMDUAT_LOG_ERROR = 0,
AMDUAT_LOG_WARN = 1,
AMDUAT_LOG_INFO = 2,
AMDUAT_LOG_DEBUG = 3
} amduat_log_level_t;
void amduat_log(amduat_log_level_t level, const char *fmt, ...);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_UTIL_LOG_H */

17
include/amduat/util/string.h
View file

@ -1,17 +0,0 @@
#ifndef AMDUAT_UTIL_STRING_H
#define AMDUAT_UTIL_STRING_H
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/* strdup replacement for strict C11 builds. */
char *amduat_strdup(const char *src);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_UTIL_STRING_H */

3
registry/README.md
View file

@ -24,9 +24,6 @@ are data nodes that can be hashed and referenced.
## JSONL manifests
Common conventions for all registry manifests are captured in
`registry/manifest.schema.md`.
Each registry has a `registry/<name>.jsonl` manifest. Each line is one entry.
The manifest is the source of truth for codegen and documentation tables.

2
registry/enc-asl1-core.jsonl
View file

@ -1 +1 @@
{"registry":"ENC/ASL1-CORE","profile_id":"0x0001","handle":"amduat.enc.asl1.core.profile.0001@1","name":"ASL_ENC_CORE_V1","version":"0x0106","status":"mandatory","spec_ref":"ENC/ASL1-CORE v1.0.6","notes":"Canonical encoding profile for ASL/1-CORE artifacts and references.","descriptor_sha256":"2d86c1367149e37d076a63fc91673dbc56fe08df83c94a11e323a0185948a87b"}
{"registry":"ENC/ASL1-CORE","profile_id":"0x0001","handle":"amduat.enc.asl1.core.profile.0001@1","name":"ASL_ENC_CORE_V1","version":"0x0105","status":"mandatory","spec_ref":"ENC/ASL1-CORE v1.0.5","notes":"Canonical encoding profile for ASL/1-CORE artifacts and references.","descriptor_sha256":"ed4ce5f50d4cb62d5e420da377ffcc0a767939f795b2c9ea00e57d79e12d93c2"}

43
registry/manifest.schema.md
View file

@ -1,43 +0,0 @@
# Registry Manifest Schema (core)
This file defines the common conventions for registry JSONL manifests in
`vendor/amduat/registry/`. Each registry then supplies a registry-specific
schema that fixes the key field(s) and the descriptor field ordering used for
`descriptor_sha256`.
## JSONL rules
- Each line is one JSON object (JSONL).
- Fields must match the registry-specific schema.
- Required fields must be present even if the value is `null`.
## Handle naming
Handles are stable identifiers in the `amduat` namespace:
```
amduat.<domain>.<registry>.<kind>.<id>@<version>
```
`id` is the canonical key for the registry (usually lowercase, zero-padded hex).
`version` increments only if the meaning of the key changes.
## Canonical descriptor digest
Each entry includes `descriptor_sha256`, the SHA-256 digest (lowercase hex) of
its canonical descriptor JSON object. The canonical descriptor object:
- Includes the descriptor fields defined by the registry-specific schema.
- Uses the exact field ordering specified by the registry-specific schema.
- Excludes the `descriptor_sha256` field itself.
- Uses standard JSON encoding with no extra whitespace.
- Is hashed over its UTF-8 byte sequence.
Registry-specific schema files define the precise field ordering and any
additional constraints on descriptor fields.
## Canonical key encoding
Each registry defines the byte encoding of its key. The encoding is used when
constructing ASL registry artifacts. See the registry-specific schema for the
exact key encoding (e.g., big-endian `u16` for `HASH/ASL1`).

1049
src/adapters/asl_derivation_index_fs/asl_derivation_index_fs.c
View file

File diff suppressed because it is too large Load diff

716
src/adapters/asl_materialization_cache_fs/asl_materialization_cache_fs.c
View file

@ -1,716 +0,0 @@
#include "amduat/asl/asl_materialization_cache_fs.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/util/log.h"
#include "amduat/util/hex.h"
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
enum {
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN = 8,
AMDUAT_ASL_MATERIALIZATION_VERSION = 1
};
static const uint8_t k_amduat_asl_materialization_magic[
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN] = {'A', 'S', 'L', 'M',
'A', 'T', '1', '\0'};
enum {
AMDUAT_ASL_MATERIALIZATION_FLAG_TIMESTAMP = 1u << 0,
AMDUAT_ASL_MATERIALIZATION_FLAG_PEL_VERSION = 1u << 1
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_materialization_cursor_t;
static void amduat_asl_materialization_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_materialization_read_u32_le(
amduat_asl_materialization_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4u;
return true;
}
static bool amduat_asl_materialization_read_u64_le(
amduat_asl_materialization_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8u;
return true;
}
static bool amduat_asl_materialization_read_u16_le(
amduat_asl_materialization_cursor_t *cur,
uint16_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 2u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
cur->offset += 2u;
return true;
}
static bool amduat_asl_materialization_read_u8(
amduat_asl_materialization_cursor_t *cur,
uint8_t *out) {
if (cur->len - cur->offset < 1u) {
return false;
}
*out = cur->data[cur->offset++];
return true;
}
static bool amduat_asl_materialization_join_path(const char *base,
const char *segment,
char **out_path) {
size_t base_len;
size_t segment_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || segment[0] == '\0') {
return false;
}
base_len = strlen(base);
segment_len = strlen(segment);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + segment_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, segment_len);
offset += segment_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool amduat_asl_materialization_ensure_directory(const char *path) {
struct stat st;
if (path == NULL || path[0] == '\0') {
return false;
}
if (stat(path, &st) == 0) {
return S_ISDIR(st.st_mode);
}
if (errno != ENOENT) {
return false;
}
if (mkdir(path, 0755) != 0) {
if (errno == EEXIST) {
return stat(path, &st) == 0 && S_ISDIR(st.st_mode);
}
return false;
}
return true;
}
static bool amduat_asl_materialization_hex_byte(uint8_t value, char out[3]) {
static const char k_hex[] = "0123456789abcdef";
out[0] = k_hex[value >> 4u];
out[1] = k_hex[value & 0x0fu];
out[2] = '\0';
return true;
}
static char *amduat_asl_materialization_parent_dir(const char *path) {
const char *slash;
size_t len;
char *dir;
if (path == NULL) {
return NULL;
}
slash = strrchr(path, '/');
if (slash == NULL || slash == path) {
return NULL;
}
len = (size_t)(slash - path);
dir = (char *)malloc(len + 1u);
if (dir == NULL) {
return NULL;
}
memcpy(dir, path, len);
dir[len] = '\0';
return dir;
}
static bool amduat_asl_materialization_fsync_directory(const char *path) {
int fd;
int flags = O_RDONLY;
if (path == NULL) {
return false;
}
#ifdef O_DIRECTORY
flags |= O_DIRECTORY;
#endif
fd = open(path, flags);
if (fd < 0) {
return false;
}
if (fsync(fd) != 0) {
close(fd);
return false;
}
return close(fd) == 0;
}
static bool amduat_asl_materialization_build_sid_path(
const char *root_path,
amduat_octets_t sid,
bool ensure_dirs,
char **out_path) {
char *index_path = NULL;
char *materializations_path = NULL;
char *by_sid_path = NULL;
char *prefix_a_path = NULL;
char *prefix_b_path = NULL;
char *sid_hex = NULL;
char prefix_a[3];
char prefix_b[3];
bool ok = false;
if (root_path == NULL || out_path == NULL) {
return false;
}
*out_path = NULL;
if (sid.len < 2u || sid.data == NULL) {
return false;
}
if (!amduat_hex_encode_alloc(sid.data, sid.len, &sid_hex)) {
return false;
}
amduat_asl_materialization_hex_byte(sid.data[0], prefix_a);
amduat_asl_materialization_hex_byte(sid.data[1], prefix_b);
if (!amduat_asl_materialization_join_path(root_path, "index", &index_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(index_path, "materializations",
&materializations_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(materializations_path, "by_sid",
&by_sid_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(by_sid_path, prefix_a,
&prefix_a_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(prefix_a_path, prefix_b,
&prefix_b_path)) {
goto cleanup;
}
if (!amduat_asl_materialization_join_path(prefix_b_path, sid_hex,
out_path)) {
goto cleanup;
}
if (ensure_dirs) {
if (!amduat_asl_materialization_ensure_directory(index_path) ||
!amduat_asl_materialization_ensure_directory(materializations_path) ||
!amduat_asl_materialization_ensure_directory(by_sid_path) ||
!amduat_asl_materialization_ensure_directory(prefix_a_path) ||
!amduat_asl_materialization_ensure_directory(prefix_b_path)) {
free(*out_path);
*out_path = NULL;
goto cleanup;
}
}
ok = true;
cleanup:
free(index_path);
free(materializations_path);
free(by_sid_path);
free(prefix_a_path);
free(prefix_b_path);
free(sid_hex);
return ok;
}
bool amduat_asl_materialization_cache_fs_init(
amduat_asl_materialization_cache_fs_t *cache,
const char *root_path) {
size_t len;
if (cache == NULL || root_path == NULL) {
return false;
}
len = strlen(root_path);
if (len == 0u || len >= AMDUAT_ASL_MATERIALIZATION_CACHE_FS_ROOT_MAX) {
return false;
}
memcpy(cache->root_path, root_path, len);
cache->root_path[len] = '\0';
return true;
}
static amduat_asl_store_error_t amduat_asl_materialization_decode_refs(
amduat_asl_materialization_cursor_t *cur,
uint32_t count,
amduat_reference_t **out_refs,
size_t *out_refs_len) {
amduat_reference_t *refs;
uint32_t i;
if (out_refs == NULL || out_refs_len == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
*out_refs = NULL;
*out_refs_len = 0u;
if (count == 0u) {
return AMDUAT_ASL_STORE_OK;
}
if (count > SIZE_MAX / sizeof(*refs)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
refs = (amduat_reference_t *)calloc(count, sizeof(*refs));
if (refs == NULL) {
return AMDUAT_ASL_STORE_ERR_IO;
}
for (i = 0u; i < count; ++i) {
uint32_t ref_len;
amduat_octets_t ref_bytes;
if (!amduat_asl_materialization_read_u32_le(cur, &ref_len)) {
goto decode_error;
}
if (ref_len < 2u || cur->len - cur->offset < ref_len) {
goto decode_error;
}
ref_bytes = amduat_octets(cur->data + cur->offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, &refs[i])) {
goto decode_error;
}
cur->offset += ref_len;
}
*out_refs = refs;
*out_refs_len = count;
return AMDUAT_ASL_STORE_OK;
decode_error:
for (uint32_t j = 0u; j < i; ++j) {
amduat_reference_free(&refs[j]);
}
free(refs);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
static void amduat_asl_materialization_refs_free(amduat_reference_t *refs,
size_t refs_len) {
size_t i;
if (refs == NULL) {
return;
}
for (i = 0u; i < refs_len; ++i) {
amduat_reference_free(&refs[i]);
}
free(refs);
}
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_get(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
amduat_reference_t **out_refs,
size_t *out_refs_len) {
char *path = NULL;
FILE *fp = NULL;
uint8_t *buffer = NULL;
long file_size;
amduat_asl_materialization_cursor_t cur;
uint8_t flags;
uint32_t version;
uint32_t sid_len;
uint32_t output_count;
amduat_asl_store_error_t err = AMDUAT_ASL_STORE_ERR_NOT_FOUND;
if (out_refs == NULL || out_refs_len == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
*out_refs = NULL;
*out_refs_len = 0u;
if (cache == NULL || sid.len == 0u || sid.data == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
if (!amduat_asl_materialization_build_sid_path(cache->root_path, sid, false,
&path)) {
return AMDUAT_ASL_STORE_ERR_IO;
}
fp = fopen(path, "rb");
if (fp == NULL) {
free(path);
return errno == ENOENT ? AMDUAT_ASL_STORE_ERR_NOT_FOUND
: AMDUAT_ASL_STORE_ERR_IO;
}
if (fseek(fp, 0, SEEK_END) != 0) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
file_size = ftell(fp);
if (file_size < 0) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
if (fseek(fp, 0, SEEK_SET) != 0) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
buffer = (uint8_t *)malloc((size_t)file_size);
if (buffer == NULL) {
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
if (fread(buffer, 1u, (size_t)file_size, fp) != (size_t)file_size) {
free(buffer);
fclose(fp);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
fclose(fp);
free(path);
cur.data = buffer;
cur.len = (size_t)file_size;
cur.offset = 0u;
if (cur.len < AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN + 4u) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (memcmp(cur.data, k_amduat_asl_materialization_magic,
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN) != 0) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
cur.offset += AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN;
if (!amduat_asl_materialization_read_u32_le(&cur, &version) ||
version != AMDUAT_ASL_MATERIALIZATION_VERSION) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (!amduat_asl_materialization_read_u32_le(&cur, &sid_len)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (sid_len != sid.len || cur.len - cur.offset < sid_len) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (memcmp(cur.data + cur.offset, sid.data, sid.len) != 0) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
cur.offset += sid_len;
if (!amduat_asl_materialization_read_u32_le(&cur, &output_count)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
err = amduat_asl_materialization_decode_refs(&cur, output_count, out_refs,
out_refs_len);
if (err != AMDUAT_ASL_STORE_OK) {
goto cleanup;
}
if (!amduat_asl_materialization_read_u8(&cur, &flags)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
if (flags & AMDUAT_ASL_MATERIALIZATION_FLAG_TIMESTAMP) {
uint64_t ignored;
if (!amduat_asl_materialization_read_u64_le(&cur, &ignored)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
}
if (flags & AMDUAT_ASL_MATERIALIZATION_FLAG_PEL_VERSION) {
uint16_t ignored;
if (!amduat_asl_materialization_read_u16_le(&cur, &ignored)) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
}
if (cur.offset != cur.len) {
err = AMDUAT_ASL_STORE_ERR_INTEGRITY;
goto cleanup;
}
err = AMDUAT_ASL_STORE_OK;
cleanup:
if (err != AMDUAT_ASL_STORE_OK) {
amduat_asl_materialization_refs_free(*out_refs, *out_refs_len);
*out_refs = NULL;
*out_refs_len = 0u;
}
free(buffer);
return err;
}
static bool amduat_asl_materialization_refs_equal(
const amduat_reference_t *a,
size_t a_len,
const amduat_reference_t *b,
size_t b_len) {
size_t i;
if (a_len != b_len) {
return false;
}
for (i = 0; i < a_len; ++i) {
if (!amduat_reference_eq(a[i], b[i])) {
return false;
}
}
return true;
}
static amduat_asl_store_error_t amduat_asl_materialization_write_record(
FILE *fp,
amduat_octets_t sid,
const amduat_reference_t *refs,
size_t refs_len) {
uint8_t header[AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN + 4u];
uint8_t flags = 0u;
uint32_t count_u32;
size_t i;
if (sid.len > UINT32_MAX || refs_len > UINT32_MAX) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
memcpy(header, k_amduat_asl_materialization_magic,
AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN);
amduat_asl_materialization_store_u32_le(
header + AMDUAT_ASL_MATERIALIZATION_MAGIC_LEN,
AMDUAT_ASL_MATERIALIZATION_VERSION);
if (fwrite(header, 1u, sizeof(header), fp) != sizeof(header)) {
return AMDUAT_ASL_STORE_ERR_IO;
}
amduat_asl_materialization_store_u32_le(header, (uint32_t)sid.len);
if (fwrite(header, 1u, 4u, fp) != 4u) {
return AMDUAT_ASL_STORE_ERR_IO;
}
if (sid.len != 0u &&
fwrite(sid.data, 1u, sid.len, fp) != sid.len) {
return AMDUAT_ASL_STORE_ERR_IO;
}
count_u32 = (uint32_t)refs_len;
amduat_asl_materialization_store_u32_le(header, count_u32);
if (fwrite(header, 1u, 4u, fp) != 4u) {
return AMDUAT_ASL_STORE_ERR_IO;
}
for (i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
uint8_t len_buf[4u];
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
amduat_asl_materialization_store_u32_le(len_buf,
(uint32_t)ref_bytes.len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
(ref_bytes.len != 0u &&
fwrite(ref_bytes.data, 1u, ref_bytes.len, fp) != ref_bytes.len)) {
free((void *)ref_bytes.data);
return AMDUAT_ASL_STORE_ERR_IO;
}
free((void *)ref_bytes.data);
}
if (fwrite(&flags, 1u, 1u, fp) != 1u) {
return AMDUAT_ASL_STORE_ERR_IO;
}
return AMDUAT_ASL_STORE_OK;
}
amduat_asl_store_error_t amduat_asl_materialization_cache_fs_put(
amduat_asl_materialization_cache_fs_t *cache,
amduat_octets_t sid,
const amduat_reference_t *refs,
size_t refs_len) {
amduat_reference_t *existing_refs = NULL;
size_t existing_len = 0u;
amduat_asl_store_error_t err;
char *path = NULL;
char *tmp_path = NULL;
FILE *fp = NULL;
size_t tmp_len;
if (cache == NULL || sid.len == 0u || sid.data == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
if (refs_len != 0u && refs == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
err = amduat_asl_materialization_cache_fs_get(cache, sid,
&existing_refs, &existing_len);
if (err == AMDUAT_ASL_STORE_OK) {
bool same = amduat_asl_materialization_refs_equal(
existing_refs, existing_len, refs, refs_len);
amduat_asl_materialization_refs_free(existing_refs, existing_len);
if (!same) {
amduat_log(AMDUAT_LOG_ERROR,
"materialization cache SID collision with mismatched outputs");
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
return AMDUAT_ASL_STORE_OK;
}
if (err != AMDUAT_ASL_STORE_ERR_NOT_FOUND) {
return err;
}
if (!amduat_asl_materialization_build_sid_path(cache->root_path, sid, true,
&path)) {
return AMDUAT_ASL_STORE_ERR_IO;
}
tmp_len = strlen(path) + 5u;
tmp_path = (char *)malloc(tmp_len);
if (tmp_path == NULL) {
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
snprintf(tmp_path, tmp_len, "%s.tmp", path);
fp = fopen(tmp_path, "wb");
if (fp == NULL) {
free(tmp_path);
free(path);
return AMDUAT_ASL_STORE_ERR_IO;
}
/*
* Crash consistency: write + fsync temp file, rename, then fsync parent
* directory. Failure only affects cache persistence, not correctness.
*/
err = amduat_asl_materialization_write_record(fp, sid, refs, refs_len);
if (fflush(fp) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
if (err == AMDUAT_ASL_STORE_OK && fsync(fileno(fp)) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
if (fclose(fp) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
fp = NULL;
if (err == AMDUAT_ASL_STORE_OK && rename(tmp_path, path) != 0) {
err = AMDUAT_ASL_STORE_ERR_IO;
}
if (err == AMDUAT_ASL_STORE_OK) {
char *parent_dir = amduat_asl_materialization_parent_dir(path);
if (parent_dir != NULL) {
if (!amduat_asl_materialization_fsync_directory(parent_dir)) {
amduat_log(AMDUAT_LOG_WARN,
"materialization cache fsync dir failed for %s",
parent_dir);
err = AMDUAT_ASL_STORE_ERR_IO;
}
free(parent_dir);
}
}
if (err != AMDUAT_ASL_STORE_OK) {
(void)remove(tmp_path);
}
#ifndef NDEBUG
if (err == AMDUAT_ASL_STORE_OK) {
amduat_reference_t *check_refs = NULL;
size_t check_len = 0u;
amduat_asl_store_error_t check_err =
amduat_asl_materialization_cache_fs_get(cache, sid, &check_refs,
&check_len);
assert(check_err == AMDUAT_ASL_STORE_OK);
assert(amduat_asl_materialization_refs_equal(
check_refs, check_len, refs, refs_len));
amduat_asl_materialization_refs_free(check_refs, check_len);
}
#endif
free(tmp_path);
free(path);
return err;
}

753
src/adapters/asl_pointer_fs/asl_pointer_fs.c
View file

@ -1,753 +0,0 @@
#include "amduat/asl/asl_pointer_fs.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/util/log.h"
#include <errno.h>
#include <fcntl.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
enum {
AMDUAT_ASL_POINTER_MAGIC_LEN = 8,
AMDUAT_ASL_POINTER_VERSION = 1
};
static const uint8_t k_amduat_asl_pointer_magic[AMDUAT_ASL_POINTER_MAGIC_LEN] = {
'A', 'S', 'L', 'P', 'T', 'R', '1', '\0'
};
enum { AMDUAT_ASL_POINTER_FLAG_HAS_EXPECTED = 1u << 0,
AMDUAT_ASL_POINTER_FLAG_HAS_PREV = 1u << 1 };
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_pointer_cursor_t;
static void amduat_asl_pointer_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_pointer_read_u32_le(amduat_asl_pointer_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4u;
return true;
}
static bool amduat_asl_pointer_read_u8(amduat_asl_pointer_cursor_t *cur,
uint8_t *out) {
if (cur->len - cur->offset < 1u) {
return false;
}
*out = cur->data[cur->offset++];
return true;
}
static bool amduat_asl_pointer_join_path(const char *base,
const char *segment,
char **out_path) {
size_t base_len;
size_t segment_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || segment == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || segment[0] == '\0') {
return false;
}
base_len = strlen(base);
segment_len = strlen(segment);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + segment_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, segment, segment_len);
offset += segment_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool amduat_asl_pointer_ensure_directory(const char *path) {
struct stat st;
if (path == NULL || path[0] == '\0') {
return false;
}
if (stat(path, &st) == 0) {
return S_ISDIR(st.st_mode);
}
if (errno != ENOENT) {
return false;
}
if (mkdir(path, 0755) != 0) {
if (errno == EEXIST) {
return stat(path, &st) == 0 && S_ISDIR(st.st_mode);
}
return false;
}
return true;
}
static char *amduat_asl_pointer_parent_dir(const char *path) {
const char *slash;
size_t len;
char *dir;
if (path == NULL) {
return NULL;
}
slash = strrchr(path, '/');
if (slash == NULL || slash == path) {
return NULL;
}
len = (size_t)(slash - path);
dir = (char *)malloc(len + 1u);
if (dir == NULL) {
return NULL;
}
memcpy(dir, path, len);
dir[len] = '\0';
return dir;
}
static bool amduat_asl_pointer_fsync_directory(const char *path) {
int fd;
int flags = O_RDONLY;
if (path == NULL) {
return false;
}
#ifdef O_DIRECTORY
flags |= O_DIRECTORY;
#endif
fd = open(path, flags);
if (fd < 0) {
return false;
}
if (fsync(fd) != 0) {
close(fd);
return false;
}
return close(fd) == 0;
}
bool amduat_asl_pointer_name_is_valid(const char *name) {
size_t len;
size_t seg_start = 0u;
if (name == NULL) {
return false;
}
len = strlen(name);
if (len == 0u || len > AMDUAT_ASL_POINTER_NAME_MAX) {
return false;
}
if (name[0] == '/' || name[len - 1u] == '/') {
return false;
}
for (size_t i = 0u; i < len; ++i) {
char c = name[i];
bool ok = (c >= 'A' && c <= 'Z') ||
(c >= 'a' && c <= 'z') ||
(c >= '0' && c <= '9') ||
c == '.' || c == '_' || c == '-' || c == '/' || c == ':';
if (!ok) {
return false;
}
if (c == '/') {
size_t seg_len = i - seg_start;
if (seg_len == 0u) {
return false;
}
if (seg_len == 2u && name[seg_start] == '.' &&
name[seg_start + 1u] == '.') {
return false;
}
seg_start = i + 1u;
}
}
{
size_t seg_len = len - seg_start;
if (seg_len == 0u) {
return false;
}
if (seg_len == 2u && name[seg_start] == '.' &&
name[seg_start + 1u] == '.') {
return false;
}
}
return true;
}
static bool amduat_asl_pointer_build_head_path(const char *root_path,
const char *name,
bool ensure_dirs,
char **out_path) {
char *pointers_path = NULL;
char *cursor_path = NULL;
char *head_path = NULL;
const char *segment_start;
const char *segment_end;
bool ok = false;
if (root_path == NULL || name == NULL || out_path == NULL) {
return false;
}
*out_path = NULL;
if (!amduat_asl_pointer_join_path(root_path, "pointers", &pointers_path)) {
return false;
}
if (ensure_dirs &&
!amduat_asl_pointer_ensure_directory(pointers_path)) {
free(pointers_path);
return false;
}
cursor_path = pointers_path;
pointers_path = NULL;
segment_start = name;
while (*segment_start != '\0') {
segment_end = strchr(segment_start, '/');
if (segment_end == NULL) {
segment_end = segment_start + strlen(segment_start);
}
{
size_t seg_len = (size_t)(segment_end - segment_start);
char *segment = (char *)malloc(seg_len + 1u);
char *next_path = NULL;
if (segment == NULL) {
goto cleanup;
}
memcpy(segment, segment_start, seg_len);
segment[seg_len] = '\0';
if (!amduat_asl_pointer_join_path(cursor_path, segment, &next_path)) {
free(segment);
goto cleanup;
}
free(segment);
if (ensure_dirs &&
!amduat_asl_pointer_ensure_directory(next_path)) {
free(next_path);
goto cleanup;
}
free(cursor_path);
cursor_path = next_path;
}
if (*segment_end == '\0') {
break;
}
segment_start = segment_end + 1u;
}
if (!amduat_asl_pointer_join_path(cursor_path, "head", &head_path)) {
goto cleanup;
}
*out_path = head_path;
head_path = NULL;
ok = true;
cleanup:
free(pointers_path);
free(cursor_path);
free(head_path);
return ok;
}
bool amduat_asl_pointer_store_init(amduat_asl_pointer_store_t *ps,
const char *root_path) {
size_t len;
if (ps == NULL || root_path == NULL) {
return false;
}
len = strlen(root_path);
if (len == 0u || len >= AMDUAT_ASL_POINTER_FS_ROOT_MAX) {
return false;
}
memcpy(ps->root_path, root_path, len);
ps->root_path[len] = '\0';
return true;
}
static amduat_asl_pointer_error_t amduat_asl_pointer_read_head(
const char *path,
const char *name,
bool *out_exists,
amduat_reference_t *out_ref,
amduat_reference_t *out_prev_ref,
bool *out_has_prev) {
FILE *fp;
uint8_t *buffer;
long file_size;
amduat_asl_pointer_cursor_t cur;
uint32_t version;
uint32_t name_len;
uint32_t ref_len;
uint32_t prev_len;
uint8_t flags;
amduat_octets_t ref_bytes;
amduat_octets_t prev_bytes;
if (out_exists == NULL || out_ref == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
*out_exists = false;
memset(out_ref, 0, sizeof(*out_ref));
if (out_prev_ref != NULL) {
memset(out_prev_ref, 0, sizeof(*out_prev_ref));
}
if (out_has_prev != NULL) {
*out_has_prev = false;
}
fp = fopen(path, "rb");
if (fp == NULL) {
return errno == ENOENT ? AMDUAT_ASL_POINTER_ERR_NOT_FOUND
: AMDUAT_ASL_POINTER_ERR_IO;
}
if (fseek(fp, 0, SEEK_END) != 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
file_size = ftell(fp);
if (file_size < 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
if (file_size == 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_NOT_FOUND;
}
if (fseek(fp, 0, SEEK_SET) != 0) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
buffer = (uint8_t *)malloc((size_t)file_size);
if (buffer == NULL) {
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
if (fread(buffer, 1u, (size_t)file_size, fp) != (size_t)file_size) {
free(buffer);
fclose(fp);
return AMDUAT_ASL_POINTER_ERR_IO;
}
fclose(fp);
cur.data = buffer;
cur.len = (size_t)file_size;
cur.offset = 0u;
if (cur.len < AMDUAT_ASL_POINTER_MAGIC_LEN + 4u + 1u) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (memcmp(cur.data, k_amduat_asl_pointer_magic,
AMDUAT_ASL_POINTER_MAGIC_LEN) != 0) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += AMDUAT_ASL_POINTER_MAGIC_LEN;
if (!amduat_asl_pointer_read_u32_le(&cur, &version) ||
version != AMDUAT_ASL_POINTER_VERSION) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_read_u8(&cur, &flags)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_read_u32_le(&cur, &name_len)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (cur.len - cur.offset < name_len) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (strlen(name) != name_len ||
memcmp(cur.data + cur.offset, name, name_len) != 0) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += name_len;
if (!amduat_asl_pointer_read_u32_le(&cur, &ref_len)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (cur.len - cur.offset < ref_len || ref_len < 2u) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
ref_bytes = amduat_octets(cur.data + cur.offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, out_ref)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += ref_len;
if (!amduat_asl_pointer_read_u32_le(&cur, &prev_len)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (prev_len != 0u) {
if (!(flags & AMDUAT_ASL_POINTER_FLAG_HAS_PREV)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (cur.len - cur.offset < prev_len || prev_len < 2u) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
prev_bytes = amduat_octets(cur.data + cur.offset, prev_len);
if (out_prev_ref != NULL &&
!amduat_enc_asl1_core_decode_reference_v1(prev_bytes, out_prev_ref)) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
cur.offset += prev_len;
if (out_has_prev != NULL) {
*out_has_prev = true;
}
}
if (cur.offset != cur.len) {
free(buffer);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
*out_exists = true;
free(buffer);
return AMDUAT_ASL_POINTER_OK;
}
static amduat_asl_pointer_error_t amduat_asl_pointer_write_head(
const char *path,
const char *name,
const amduat_reference_t *ref,
const amduat_reference_t *prev_ref,
bool has_prev) {
char *tmp_path;
char *parent_dir = NULL;
size_t tmp_len;
int tmp_fd;
FILE *fp;
uint8_t header[AMDUAT_ASL_POINTER_MAGIC_LEN + 4u + 1u];
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
amduat_octets_t prev_bytes = amduat_octets(NULL, 0u);
uint32_t name_len;
uint32_t ref_len;
uint32_t prev_len;
uint8_t flags = 0u;
amduat_asl_pointer_error_t err = AMDUAT_ASL_POINTER_OK;
if (ref == NULL || name == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
name_len = (uint32_t)strlen(name);
if (!amduat_enc_asl1_core_encode_reference_v1(*ref, &ref_bytes)) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
ref_len = (uint32_t)ref_bytes.len;
if (has_prev) {
if (prev_ref == NULL ||
!amduat_enc_asl1_core_encode_reference_v1(*prev_ref, &prev_bytes)) {
free((void *)ref_bytes.data);
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
prev_len = (uint32_t)prev_bytes.len;
flags |= AMDUAT_ASL_POINTER_FLAG_HAS_PREV;
} else {
prev_len = 0u;
}
tmp_len = strlen(path) + sizeof(".tmp.XXXXXX");
tmp_path = (char *)malloc(tmp_len);
if (tmp_path == NULL) {
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
parent_dir = amduat_asl_pointer_parent_dir(path);
if (parent_dir == NULL || !amduat_asl_pointer_ensure_directory(parent_dir)) {
free(parent_dir);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
snprintf(tmp_path, tmp_len, "%s.tmp.XXXXXX", path);
tmp_fd = mkstemp(tmp_path);
if (tmp_fd < 0 && errno == ENOENT &&
amduat_asl_pointer_ensure_directory(parent_dir)) {
tmp_fd = mkstemp(tmp_path);
}
if (tmp_fd < 0) {
free(parent_dir);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
fp = fdopen(tmp_fd, "wb");
if (fp == NULL) {
close(tmp_fd);
(void)remove(tmp_path);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return AMDUAT_ASL_POINTER_ERR_IO;
}
memcpy(header, k_amduat_asl_pointer_magic, AMDUAT_ASL_POINTER_MAGIC_LEN);
amduat_asl_pointer_store_u32_le(
header + AMDUAT_ASL_POINTER_MAGIC_LEN, AMDUAT_ASL_POINTER_VERSION);
header[AMDUAT_ASL_POINTER_MAGIC_LEN + 4u] = flags;
if (fwrite(header, 1u, sizeof(header), fp) != sizeof(header)) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK) {
uint8_t len_buf[4u];
amduat_asl_pointer_store_u32_le(len_buf, name_len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
fwrite(name, 1u, name_len, fp) != name_len) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK) {
uint8_t len_buf[4u];
amduat_asl_pointer_store_u32_le(len_buf, ref_len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u ||
fwrite(ref_bytes.data, 1u, ref_len, fp) != ref_len) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK) {
uint8_t len_buf[4u];
amduat_asl_pointer_store_u32_le(len_buf, prev_len);
if (fwrite(len_buf, 1u, 4u, fp) != 4u) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK && prev_len != 0u &&
fwrite(prev_bytes.data, 1u, prev_len, fp) != prev_len) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK && fflush(fp) != 0) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK && fsync(fileno(fp)) != 0) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (fclose(fp) != 0) {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
if (err == AMDUAT_ASL_POINTER_OK && rename(tmp_path, path) != 0) {
if (errno == ENOENT && amduat_asl_pointer_ensure_directory(parent_dir) &&
rename(tmp_path, path) == 0) {
/* Recovered after recreating parent directory. */
} else {
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err == AMDUAT_ASL_POINTER_OK) {
if (!amduat_asl_pointer_fsync_directory(parent_dir)) {
amduat_log(AMDUAT_LOG_WARN,
"pointer fsync dir failed for %s", parent_dir);
err = AMDUAT_ASL_POINTER_ERR_IO;
}
}
if (err != AMDUAT_ASL_POINTER_OK) {
(void)remove(tmp_path);
}
free(parent_dir);
free(tmp_path);
free((void *)ref_bytes.data);
free((void *)prev_bytes.data);
return err;
}
amduat_asl_pointer_error_t amduat_asl_pointer_get(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool *out_exists,
amduat_reference_t *out_ref) {
char *head_path = NULL;
amduat_asl_pointer_error_t err;
if (ps == NULL || name == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return AMDUAT_ASL_POINTER_ERR_INVALID_NAME;
}
if (!amduat_asl_pointer_build_head_path(ps->root_path, name, false,
&head_path)) {
return AMDUAT_ASL_POINTER_ERR_IO;
}
err = amduat_asl_pointer_read_head(head_path, name, out_exists, out_ref,
NULL, NULL);
if (err == AMDUAT_ASL_POINTER_ERR_NOT_FOUND) {
*out_exists = false;
err = AMDUAT_ASL_POINTER_OK;
}
free(head_path);
return err;
}
amduat_asl_pointer_error_t amduat_asl_pointer_cas(
const amduat_asl_pointer_store_t *ps,
const char *name,
bool expected_exists,
const amduat_reference_t *expected_ref,
const amduat_reference_t *new_ref,
bool *out_swapped) {
char *head_path = NULL;
int lock_fd = -1;
struct flock lock;
bool exists = false;
amduat_reference_t current_ref;
amduat_reference_t prev_ref;
bool has_prev = false;
char *lock_path = NULL;
amduat_asl_pointer_error_t err;
if (out_swapped == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
*out_swapped = false;
if (ps == NULL || name == NULL || new_ref == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return AMDUAT_ASL_POINTER_ERR_INVALID_NAME;
}
if (expected_exists && expected_ref == NULL) {
return AMDUAT_ASL_POINTER_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_build_head_path(ps->root_path, name, true,
&head_path)) {
return AMDUAT_ASL_POINTER_ERR_IO;
}
lock_path = (char *)malloc(strlen(head_path) + sizeof(".lock"));
if (lock_path == NULL) {
free(head_path);
return AMDUAT_ASL_POINTER_ERR_IO;
}
snprintf(lock_path, strlen(head_path) + sizeof(".lock"), "%s.lock",
head_path);
/* Lock a stable sidecar file so lock ownership survives head-file renames. */
lock_fd = open(lock_path, O_RDWR | O_CREAT, 0644);
if (lock_fd < 0) {
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_ERR_IO;
}
memset(&lock, 0, sizeof(lock));
lock.l_type = F_WRLCK;
lock.l_whence = SEEK_SET;
lock.l_start = 0;
lock.l_len = 0;
if (fcntl(lock_fd, F_SETLKW, &lock) != 0) {
close(lock_fd);
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_ERR_IO;
}
err = amduat_asl_pointer_read_head(head_path, name, &exists, &current_ref,
&prev_ref, &has_prev);
if (err == AMDUAT_ASL_POINTER_ERR_NOT_FOUND) {
exists = false;
err = AMDUAT_ASL_POINTER_OK;
}
if (err != AMDUAT_ASL_POINTER_OK) {
close(lock_fd);
free(lock_path);
free(head_path);
return err;
}
if (expected_exists != exists) {
close(lock_fd);
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_OK;
}
if (expected_exists && !amduat_reference_eq(*expected_ref, current_ref)) {
close(lock_fd);
free(lock_path);
free(head_path);
return AMDUAT_ASL_POINTER_OK;
}
err = amduat_asl_pointer_write_head(head_path, name, new_ref,
expected_exists ? &current_ref : NULL,
expected_exists);
if (err == AMDUAT_ASL_POINTER_OK) {
*out_swapped = true;
}
if (exists) {
amduat_reference_free(&current_ref);
}
if (has_prev) {
amduat_reference_free(&prev_ref);
}
close(lock_fd);
free(lock_path);
free(head_path);
return err;
}

51
src/adapters/asl_store_fs/asl_store_fs.c
View file

@ -2,7 +2,6 @@
#include "asl_store_fs_layout.h"
#include "amduat/asl/core.h"
#include "amduat/asl/identity.h"
#include "amduat/asl/ref_derive.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/enc/asl1_core_codec.h"
@ -490,30 +489,6 @@ static amduat_asl_store_error_t amduat_asl_store_fs_put_impl(
return AMDUAT_ASL_STORE_OK;
}
static void amduat_asl_store_fs_fill_index_state(
amduat_asl_index_state_t *out_state) {
if (out_state == NULL) {
return;
}
out_state->snapshot_id = 0u;
out_state->log_position = 0u;
}
static amduat_asl_store_error_t amduat_asl_store_fs_put_indexed_impl(
void *ctx,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t err;
err = amduat_asl_store_fs_put_impl(ctx, artifact, out_ref);
if (err != AMDUAT_ASL_STORE_OK) {
return err;
}
amduat_asl_store_fs_fill_index_state(out_state);
return AMDUAT_ASL_STORE_OK;
}
static amduat_asl_store_error_t amduat_asl_store_fs_validate_config(
void *ctx,
amduat_asl_store_config_t config) {
@ -536,16 +511,6 @@ static amduat_asl_store_error_t amduat_asl_store_fs_validate_config(
return AMDUAT_ASL_STORE_OK;
}
static bool amduat_asl_store_fs_current_state_impl(
void *ctx,
amduat_asl_index_state_t *out_state) {
if (ctx == NULL) {
return false;
}
amduat_asl_store_fs_fill_index_state(out_state);
return true;
}
static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
void *ctx,
amduat_reference_t ref,
@ -662,9 +627,7 @@ static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
}
stored_octets = amduat_octets(stored_bytes, stored_len);
if (!amduat_hash_asl1_digest_domain(
hash_desc->hash_id,
amduat_asl_identity_domain_separator(),
if (!amduat_hash_asl1_digest(hash_desc->hash_id,
stored_octets,
computed_digest,
hash_desc->digest_len)) {
@ -709,15 +672,6 @@ static amduat_asl_store_error_t amduat_asl_store_fs_get_impl(
return AMDUAT_ASL_STORE_OK;
}
static amduat_asl_store_error_t amduat_asl_store_fs_get_indexed_impl(
void *ctx,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact) {
(void)state;
return amduat_asl_store_fs_get_impl(ctx, ref, out_artifact);
}
bool amduat_asl_store_fs_init(amduat_asl_store_fs_t *fs,
amduat_asl_store_config_t config,
const char *root_path) {
@ -742,9 +696,6 @@ amduat_asl_store_ops_t amduat_asl_store_fs_ops(void) {
amduat_asl_store_ops_init(&ops);
ops.put = amduat_asl_store_fs_put_impl;
ops.get = amduat_asl_store_fs_get_impl;
ops.put_indexed = amduat_asl_store_fs_put_indexed_impl;
ops.get_indexed = amduat_asl_store_fs_get_indexed_impl;
ops.current_state = amduat_asl_store_fs_current_state_impl;
ops.validate_config = amduat_asl_store_fs_validate_config;
return ops;
}

5685
src/adapters/asl_store_index_fs/asl_store_index_fs.c
View file

File diff suppressed because it is too large Load diff

415
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.c
View file

@ -1,415 +0,0 @@
#include "asl_store_index_fs_layout.h"
#include <inttypes.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static bool amduat_asl_store_index_fs_layout_join(const char *base,
const char *suffix,
char **out_path) {
size_t base_len;
size_t suffix_len;
bool needs_sep;
size_t total_len;
char *buffer;
size_t offset;
if (base == NULL || suffix == NULL || out_path == NULL) {
return false;
}
if (base[0] == '\0' || suffix[0] == '\0') {
return false;
}
base_len = strlen(base);
suffix_len = strlen(suffix);
needs_sep = base[base_len - 1u] != '/';
total_len = base_len + (needs_sep ? 1u : 0u) + suffix_len + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, base, base_len);
offset += base_len;
if (needs_sep) {
buffer[offset++] = '/';
}
memcpy(buffer + offset, suffix, suffix_len);
offset += suffix_len;
buffer[offset] = '\0';
*out_path = buffer;
return true;
}
static bool amduat_asl_store_index_fs_layout_format_id(const char *prefix,
uint64_t id,
const char *suffix,
char **out_name) {
int needed;
char *buffer;
if (prefix == NULL || suffix == NULL || out_name == NULL) {
return false;
}
needed = snprintf(NULL, 0, "%s%016" PRIx64 "%s", prefix, id, suffix);
if (needed <= 0) {
return false;
}
buffer = (char *)malloc((size_t)needed + 1u);
if (buffer == NULL) {
return false;
}
snprintf(buffer, (size_t)needed + 1u, "%s%016" PRIx64 "%s", prefix, id,
suffix);
*out_name = buffer;
return true;
}
bool amduat_asl_store_index_fs_layout_build_index_path(const char *root_path,
char **out_path) {
return amduat_asl_store_index_fs_layout_join(root_path, "index", out_path);
}
bool amduat_asl_store_index_fs_layout_build_segments_path(
const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "segments", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_blocks_path(const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "blocks", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_log_path(const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "log.asl", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shards_path(
const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "shards", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *shards_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shards_path(root_path,
&shards_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("shard-",
(uint64_t)shard_id,
"",
&name)) {
free(shards_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(shards_path, name, out_path);
free(name);
free(shards_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_segments_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *shard_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_path(root_path,
shard_id,
&shard_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(shard_path, "segments", out_path);
free(shard_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_blocks_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *shard_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_path(root_path,
shard_id,
&shard_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(shard_path, "blocks", out_path);
free(shard_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_segment_meta_path(
const char *root_path,
uint16_t shard_id,
char **out_path) {
char *segments_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_segments_path(root_path,
shard_id,
&segments_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path,
"next_id",
out_path);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_segment_path(
const char *root_path,
uint16_t shard_id,
uint64_t segment_id,
char **out_path) {
char *segments_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_segments_path(root_path,
shard_id,
&segments_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("segment-",
segment_id,
".asl",
&name)) {
free(segments_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path, name, out_path);
free(name);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_shard_block_path(
const char *root_path,
uint16_t shard_id,
uint64_t block_id,
char **out_path) {
char *blocks_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_shard_blocks_path(root_path,
shard_id,
&blocks_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("block-",
block_id,
".asl",
&name)) {
free(blocks_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(blocks_path, name, out_path);
free(name);
free(blocks_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_snapshots_path(
const char *root_path,
char **out_path) {
char *index_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_index_path(root_path,
&index_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(index_path, "snapshots", out_path);
free(index_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_snapshot_manifest_path(
const char *root_path,
uint64_t snapshot_id,
char **out_path) {
char *snapshots_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_snapshots_path(root_path,
&snapshots_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("snap-",
snapshot_id,
".bin",
&name)) {
free(snapshots_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(snapshots_path, name, out_path);
free(name);
free(snapshots_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_snapshot_latest_path(
const char *root_path,
char **out_path) {
char *snapshots_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_snapshots_path(root_path,
&snapshots_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(snapshots_path,
"latest",
out_path);
free(snapshots_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_segment_meta_path(
const char *root_path,
char **out_path) {
char *segments_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
&segments_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path,
"next_id",
out_path);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_segment_summary_path(
const char *root_path,
char **out_path) {
char *segments_path;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
&segments_path)) {
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path,
"summary",
out_path);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_segment_path(
const char *root_path,
uint64_t segment_id,
char **out_path) {
char *segments_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_segments_path(root_path,
&segments_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("segment-",
segment_id,
".asl",
&name)) {
free(segments_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(segments_path, name, out_path);
free(name);
free(segments_path);
return ok;
}
bool amduat_asl_store_index_fs_layout_build_block_path(
const char *root_path,
uint64_t block_id,
char **out_path) {
char *blocks_path;
char *name;
bool ok;
if (!amduat_asl_store_index_fs_layout_build_blocks_path(root_path,
&blocks_path)) {
return false;
}
if (!amduat_asl_store_index_fs_layout_format_id("block-",
block_id,
".asl",
&name)) {
free(blocks_path);
return false;
}
ok = amduat_asl_store_index_fs_layout_join(blocks_path, name, out_path);
free(name);
free(blocks_path);
return ok;
}

95
src/adapters/asl_store_index_fs/asl_store_index_fs_layout.h
View file

@ -1,95 +0,0 @@
#ifndef AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H
#define AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
bool amduat_asl_store_index_fs_layout_build_index_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segments_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_blocks_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_log_path(const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shards_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_segments_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_blocks_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_segment_meta_path(
const char *root_path,
uint16_t shard_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_segment_path(
const char *root_path,
uint16_t shard_id,
uint64_t segment_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_shard_block_path(
const char *root_path,
uint16_t shard_id,
uint64_t block_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_snapshots_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_snapshot_manifest_path(
const char *root_path,
uint64_t snapshot_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_snapshot_latest_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segment_meta_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segment_summary_path(
const char *root_path,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_segment_path(
const char *root_path,
uint64_t segment_id,
char **out_path);
bool amduat_asl_store_index_fs_layout_build_block_path(
const char *root_path,
uint64_t block_id,
char **out_path);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ASL_STORE_INDEX_FS_LAYOUT_H */

1325
src/adapters/asl_store_minio/asl_store_minio.c
View file

File diff suppressed because it is too large Load diff

288
src/adapters/asl_store_minio/s3_sigv4.c
View file

@ -1,288 +0,0 @@
#include "s3_sigv4.h"
#include "amduat/asl/core.h"
#include "amduat/hash/asl1.h"
#include "amduat/util/hex.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
enum { AMDUAT_S3_SIGV4_SHA256_LEN = 32u, AMDUAT_S3_SIGV4_BLOCK_LEN = 64u };
static bool sha256_digest(const uint8_t *data, size_t len, uint8_t out[32]) {
amduat_octets_t input;
input.data = data;
input.len = len;
return amduat_hash_asl1_digest(AMDUAT_HASH_ASL1_ID_SHA256, input, out, 32u);
}
static bool sha256_hex(const uint8_t *data, size_t len, char out_hex[65]) {
uint8_t digest[AMDUAT_S3_SIGV4_SHA256_LEN];
if (!sha256_digest(data, len, digest)) {
return false;
}
if (!amduat_hex_encode_lower(digest, sizeof(digest), out_hex, 65u)) {
return false;
}
out_hex[64] = '\0';
return true;
}
static bool hmac_sha256(const uint8_t *key,
size_t key_len,
const uint8_t *msg,
size_t msg_len,
uint8_t out[32]) {
uint8_t key_block[AMDUAT_S3_SIGV4_BLOCK_LEN];
uint8_t ipad[AMDUAT_S3_SIGV4_BLOCK_LEN];
uint8_t opad[AMDUAT_S3_SIGV4_BLOCK_LEN];
uint8_t inner_hash[AMDUAT_S3_SIGV4_SHA256_LEN];
size_t i;
amduat_hash_asl1_stream_t stream;
memset(key_block, 0, sizeof(key_block));
if (key_len > sizeof(key_block)) {
if (!sha256_digest(key, key_len, key_block)) {
return false;
}
key_len = AMDUAT_S3_SIGV4_SHA256_LEN;
} else if (key_len > 0 && key != NULL) {
memcpy(key_block, key, key_len);
}
for (i = 0; i < sizeof(key_block); ++i) {
ipad[i] = (uint8_t)(key_block[i] ^ 0x36u);
opad[i] = (uint8_t)(key_block[i] ^ 0x5cu);
}
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
return false;
}
if (!amduat_hash_asl1_stream_update(
&stream, amduat_octets(ipad, sizeof(ipad))) ||
(msg_len > 0 &&
!amduat_hash_asl1_stream_update(
&stream, amduat_octets(msg, msg_len))) ||
!amduat_hash_asl1_stream_final(&stream, inner_hash, sizeof(inner_hash))) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
amduat_hash_asl1_stream_destroy(&stream);
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
return false;
}
if (!amduat_hash_asl1_stream_update(
&stream, amduat_octets(opad, sizeof(opad))) ||
!amduat_hash_asl1_stream_update(
&stream, amduat_octets(inner_hash, sizeof(inner_hash))) ||
!amduat_hash_asl1_stream_final(&stream, out, AMDUAT_S3_SIGV4_SHA256_LEN)) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
amduat_hash_asl1_stream_destroy(&stream);
return true;
}
bool amduat_s3_sigv4_build_authorization(const char *method,
const char *canonical_uri,
const char *canonical_query,
const char *host,
const char *payload_hash_hex,
const char *access_key,
const char *secret_key,
const char *region,
const char *amz_date,
char **out_authorization) {
const char *signed_headers = "host;x-amz-content-sha256;x-amz-date";
char date_stamp[9];
char *canonical_request = NULL;
char *string_to_sign = NULL;
char canonical_request_hash[65];
uint8_t k_date[AMDUAT_S3_SIGV4_SHA256_LEN];
uint8_t k_region[AMDUAT_S3_SIGV4_SHA256_LEN];
uint8_t k_service[AMDUAT_S3_SIGV4_SHA256_LEN];
uint8_t k_signing[AMDUAT_S3_SIGV4_SHA256_LEN];
uint8_t signature_bytes[AMDUAT_S3_SIGV4_SHA256_LEN];
char signature_hex[65];
char *auth_header = NULL;
size_t auth_len;
int rc;
if (out_authorization != NULL) {
*out_authorization = NULL;
}
if (method == NULL || canonical_uri == NULL || canonical_query == NULL ||
host == NULL || payload_hash_hex == NULL || access_key == NULL ||
secret_key == NULL || region == NULL || amz_date == NULL ||
out_authorization == NULL) {
return false;
}
if (strlen(amz_date) < 8u) {
return false;
}
memcpy(date_stamp, amz_date, 8u);
date_stamp[8] = '\0';
rc = snprintf(NULL,
0,
"%s\n%s\n%s\n"
"host:%s\n"
"x-amz-content-sha256:%s\n"
"x-amz-date:%s\n\n"
"%s\n"
"%s",
method,
canonical_uri,
canonical_query,
host,
payload_hash_hex,
amz_date,
signed_headers,
payload_hash_hex);
if (rc <= 0) {
return false;
}
canonical_request = (char *)malloc((size_t)rc + 1u);
if (canonical_request == NULL) {
return false;
}
snprintf(canonical_request,
(size_t)rc + 1u,
"%s\n%s\n%s\n"
"host:%s\n"
"x-amz-content-sha256:%s\n"
"x-amz-date:%s\n\n"
"%s\n"
"%s",
method,
canonical_uri,
canonical_query,
host,
payload_hash_hex,
amz_date,
signed_headers,
payload_hash_hex);
if (!sha256_hex((const uint8_t *)canonical_request,
strlen(canonical_request),
canonical_request_hash)) {
free(canonical_request);
return false;
}
rc = snprintf(NULL,
0,
"AWS4-HMAC-SHA256\n%s\n%s/%s/s3/aws4_request\n%s",
amz_date,
date_stamp,
region,
canonical_request_hash);
if (rc <= 0) {
free(canonical_request);
return false;
}
string_to_sign = (char *)malloc((size_t)rc + 1u);
if (string_to_sign == NULL) {
free(canonical_request);
return false;
}
snprintf(string_to_sign,
(size_t)rc + 1u,
"AWS4-HMAC-SHA256\n%s\n%s/%s/s3/aws4_request\n%s",
amz_date,
date_stamp,
region,
canonical_request_hash);
{
size_t secret_len = strlen(secret_key);
size_t key_len = 4u + secret_len;
char *key_prefix = (char *)malloc(key_len + 1u);
if (key_prefix == NULL) {
free(canonical_request);
free(string_to_sign);
return false;
}
memcpy(key_prefix, "AWS4", 4u);
memcpy(key_prefix + 4u, secret_key, secret_len);
key_prefix[key_len] = '\0';
if (!hmac_sha256((const uint8_t *)key_prefix,
key_len,
(const uint8_t *)date_stamp,
strlen(date_stamp),
k_date) ||
!hmac_sha256(k_date,
sizeof(k_date),
(const uint8_t *)region,
strlen(region),
k_region) ||
!hmac_sha256(k_region,
sizeof(k_region),
(const uint8_t *)"s3",
2u,
k_service) ||
!hmac_sha256(k_service,
sizeof(k_service),
(const uint8_t *)"aws4_request",
12u,
k_signing) ||
!hmac_sha256(k_signing,
sizeof(k_signing),
(const uint8_t *)string_to_sign,
strlen(string_to_sign),
signature_bytes)) {
free(key_prefix);
free(canonical_request);
free(string_to_sign);
return false;
}
free(key_prefix);
}
if (!amduat_hex_encode_lower(signature_bytes,
sizeof(signature_bytes),
signature_hex,
sizeof(signature_hex))) {
free(canonical_request);
free(string_to_sign);
return false;
}
auth_len = (size_t)snprintf(NULL,
0,
"AWS4-HMAC-SHA256 Credential=%s/%s/%s/s3/aws4_request, "
"SignedHeaders=%s, Signature=%s",
access_key,
date_stamp,
region,
signed_headers,
signature_hex);
auth_header = (char *)malloc(auth_len + 1u);
if (auth_header == NULL) {
free(canonical_request);
free(string_to_sign);
return false;
}
snprintf(auth_header,
auth_len + 1u,
"AWS4-HMAC-SHA256 Credential=%s/%s/%s/s3/aws4_request, "
"SignedHeaders=%s, Signature=%s",
access_key,
date_stamp,
region,
signed_headers,
signature_hex);
free(canonical_request);
free(string_to_sign);
*out_authorization = auth_header;
return true;
}

25
src/adapters/asl_store_minio/s3_sigv4.h
View file

@ -1,25 +0,0 @@
#ifndef AMDUAT_ADAPTERS_ASL_STORE_MINIO_S3_SIGV4_H
#define AMDUAT_ADAPTERS_ASL_STORE_MINIO_S3_SIGV4_H
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
bool amduat_s3_sigv4_build_authorization(const char *method,
const char *canonical_uri,
const char *canonical_query,
const char *host,
const char *payload_hash_hex,
const char *access_key,
const char *secret_key,
const char *region,
const char *amz_date,
char **out_authorization);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* AMDUAT_ADAPTERS_ASL_STORE_MINIO_S3_SIGV4_H */

1699
src/adapters/tgk_store_asl_index_fs/tgk_store_asl_index_fs.c
View file

File diff suppressed because it is too large Load diff

607
src/core/asl_collection.c
View file

@ -1,607 +0,0 @@
#include "amduat/asl/collection.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/util/log.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_COLLECTION_MAGIC_LEN = 8,
AMDUAT_ASL_COLLECTION_VERSION = 1,
AMDUAT_ASL_COLLECTION_READ_BATCH = 1024u
};
static const uint8_t k_amduat_asl_collection_magic[
AMDUAT_ASL_COLLECTION_MAGIC_LEN] = {
'A', 'S', 'L', 'C', 'O', 'L', '1', '\0'
};
static void amduat_asl_collection_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_collection_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_collection_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_collection_read_u64_le(const uint8_t *data,
size_t len,
size_t *offset,
uint64_t *out) {
if (len - *offset < 8u) {
return false;
}
*out = (uint64_t)data[*offset] |
((uint64_t)data[*offset + 1u] << 8) |
((uint64_t)data[*offset + 2u] << 16) |
((uint64_t)data[*offset + 3u] << 24) |
((uint64_t)data[*offset + 4u] << 32) |
((uint64_t)data[*offset + 5u] << 40) |
((uint64_t)data[*offset + 6u] << 48) |
((uint64_t)data[*offset + 7u] << 56);
*offset += 8u;
return true;
}
static bool amduat_asl_collection_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_collection_build_log_name(const char *name,
char **out_name) {
size_t name_len;
size_t total_len;
char *buffer;
size_t offset;
if (name == NULL || out_name == NULL) {
return false;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return false;
}
name_len = strlen(name);
total_len = 11u + name_len + 4u + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, "collection/", 11u);
offset += 11u;
memcpy(buffer + offset, name, name_len);
offset += name_len;
memcpy(buffer + offset, "/log", 4u);
offset += 4u;
buffer[offset] = '\0';
*out_name = buffer;
return true;
}
static bool amduat_asl_collection_build_head_name(const char *name,
char **out_name) {
size_t name_len;
size_t total_len;
char *buffer;
size_t offset;
if (name == NULL || out_name == NULL) {
return false;
}
if (!amduat_asl_pointer_name_is_valid(name)) {
return false;
}
name_len = strlen(name);
total_len = 11u + name_len + 5u + 1u;
buffer = (char *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
memcpy(buffer + offset, "collection/", 11u);
offset += 11u;
memcpy(buffer + offset, name, name_len);
offset += name_len;
memcpy(buffer + offset, "/head", 5u);
offset += 5u;
buffer[offset] = '\0';
*out_name = buffer;
return true;
}
static bool amduat_asl_collection_encode_snapshot_payload(
uint64_t snapshot_offset,
const amduat_reference_t *refs,
size_t refs_len,
amduat_octets_t *out_payload) {
size_t total_len = 0u;
uint8_t *buffer;
size_t offset = 0u;
if (out_payload == NULL) {
return false;
}
out_payload->data = NULL;
out_payload->len = 0u;
if (refs_len > UINT32_MAX) {
return false;
}
if (!amduat_asl_collection_add_size(
&total_len, AMDUAT_ASL_COLLECTION_MAGIC_LEN + 4u + 8u + 4u)) {
return false;
}
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return false;
}
if (!amduat_asl_collection_add_size(&total_len, 4u + ref_bytes.len)) {
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_collection_magic,
AMDUAT_ASL_COLLECTION_MAGIC_LEN);
offset += AMDUAT_ASL_COLLECTION_MAGIC_LEN;
amduat_asl_collection_store_u32_le(buffer + offset,
AMDUAT_ASL_COLLECTION_VERSION);
offset += 4u;
amduat_asl_collection_store_u64_le(buffer + offset, snapshot_offset);
offset += 8u;
amduat_asl_collection_store_u32_le(buffer + offset, (uint32_t)refs_len);
offset += 4u;
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
free(buffer);
return false;
}
amduat_asl_collection_store_u32_le(buffer + offset,
(uint32_t)ref_bytes.len);
offset += 4u;
memcpy(buffer + offset, ref_bytes.data, ref_bytes.len);
offset += ref_bytes.len;
free((void *)ref_bytes.data);
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_payload = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_collection_store_init(
amduat_asl_collection_store_t *collection_store,
const char *root_path,
amduat_asl_store_t *store) {
if (collection_store == NULL || root_path == NULL || store == NULL) {
return false;
}
collection_store->store = store;
if (!amduat_asl_pointer_store_init(&collection_store->pointer_store,
root_path)) {
return false;
}
if (!amduat_asl_log_store_init(&collection_store->log_store, root_path,
store, &collection_store->pointer_store)) {
return false;
}
return true;
}
amduat_asl_collection_error_t amduat_asl_collection_append(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
amduat_reference_t record_ref,
uint16_t kind,
amduat_octets_t actor,
uint64_t *out_offset) {
char *log_name = NULL;
amduat_asl_log_entry_t entry;
amduat_asl_store_error_t err;
if (collection_store == NULL || collection_store->store == NULL ||
collection_name == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
}
if (!amduat_asl_collection_build_log_name(collection_name, &log_name)) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
memset(&entry, 0, sizeof(entry));
entry.kind = kind;
entry.payload_ref = record_ref;
entry.has_actor = actor.len != 0u;
entry.actor = actor;
err = amduat_asl_log_append(&collection_store->log_store, log_name,
&entry, 1u, out_offset);
free(log_name);
if (err != AMDUAT_ASL_STORE_OK) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
return AMDUAT_ASL_COLLECTION_OK;
}
amduat_asl_collection_error_t amduat_asl_collection_snapshot(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t up_to_offset,
amduat_reference_t *out_snapshot_ref,
bool *out_swapped) {
char *log_name = NULL;
char *head_name = NULL;
amduat_reference_t *refs = NULL;
size_t refs_len = 0u;
uint64_t from = 0u;
bool end = false;
uint64_t next_offset = 0u;
amduat_asl_record_t snapshot_record;
amduat_octets_t payload = amduat_octets(NULL, 0u);
amduat_reference_t snapshot_ref;
amduat_asl_pointer_error_t ptr_err;
bool head_exists = false;
amduat_reference_t head_ref;
bool swapped = false;
if (out_snapshot_ref != NULL) {
out_snapshot_ref->hash_id = 0u;
out_snapshot_ref->digest = amduat_octets(NULL, 0u);
}
if (out_swapped != NULL) {
*out_swapped = false;
}
if (collection_store == NULL || collection_store->store == NULL ||
collection_name == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
}
if (!amduat_asl_collection_build_log_name(collection_name, &log_name) ||
!amduat_asl_collection_build_head_name(collection_name, &head_name)) {
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
while (!end && (up_to_offset == UINT64_MAX || from <= up_to_offset)) {
size_t limit = AMDUAT_ASL_COLLECTION_READ_BATCH;
amduat_asl_log_entry_t *entries = NULL;
size_t entries_len = 0u;
amduat_asl_store_error_t err = amduat_asl_log_read(
&collection_store->log_store, log_name, from, limit, &entries,
&entries_len, &next_offset, &end);
if (err != AMDUAT_ASL_STORE_OK) {
free(log_name);
free(head_name);
amduat_asl_collection_refs_free(refs, refs_len);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
for (size_t i = 0u; i < entries_len; ++i) {
uint64_t offset = from + i;
if (up_to_offset != UINT64_MAX && offset > up_to_offset) {
end = true;
break;
}
amduat_reference_t *next =
(amduat_reference_t *)realloc(
refs, (refs_len + 1u) * sizeof(*refs));
if (next == NULL) {
amduat_asl_log_entries_free(entries, entries_len);
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
refs = next;
refs[refs_len] = entries[i].payload_ref;
if (!amduat_octets_clone(entries[i].payload_ref.digest,
&refs[refs_len].digest)) {
amduat_asl_log_entries_free(entries, entries_len);
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
refs_len++;
}
amduat_asl_log_entries_free(entries, entries_len);
from = next_offset;
if (entries_len == 0u) {
break;
}
}
{
uint64_t snapshot_offset = from;
if (up_to_offset != UINT64_MAX &&
up_to_offset < UINT64_MAX &&
snapshot_offset > up_to_offset + 1u) {
snapshot_offset = up_to_offset + 1u;
}
if (!amduat_asl_collection_encode_snapshot_payload(
snapshot_offset, refs, refs_len, &payload)) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
}
if (payload.len == 0u || payload.data == NULL) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
{
amduat_asl_store_error_t err = amduat_asl_record_store_put(
collection_store->store,
amduat_octets("collection/snapshot",
strlen("collection/snapshot")),
payload,
&snapshot_ref);
amduat_octets_free(&payload);
if (err != AMDUAT_ASL_STORE_OK) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
}
ptr_err = amduat_asl_pointer_get(&collection_store->pointer_store, head_name,
&head_exists, &head_ref);
if (ptr_err != AMDUAT_ASL_POINTER_OK) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
ptr_err = amduat_asl_pointer_cas(&collection_store->pointer_store, head_name,
head_exists,
head_exists ? &head_ref : NULL,
&snapshot_ref, &swapped);
if (head_exists) {
amduat_reference_free(&head_ref);
}
if (ptr_err != AMDUAT_ASL_POINTER_OK) {
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
if (out_swapped != NULL) {
*out_swapped = swapped;
}
if (out_snapshot_ref != NULL) {
*out_snapshot_ref = snapshot_ref;
}
if (!swapped) {
amduat_log(AMDUAT_LOG_DEBUG, "collection snapshot CAS mismatch");
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_ERR_CAS_MISMATCH;
}
amduat_asl_collection_refs_free(refs, refs_len);
free(log_name);
free(head_name);
return AMDUAT_ASL_COLLECTION_OK;
}
amduat_asl_collection_error_t amduat_asl_collection_read(
amduat_asl_collection_store_t *collection_store,
const char *collection_name,
uint64_t from_offset,
size_t limit,
amduat_reference_t **out_record_refs,
size_t *out_len,
uint64_t *out_next_offset,
bool *out_end) {
char *log_name = NULL;
amduat_asl_log_entry_t *entries = NULL;
size_t entries_len = 0u;
amduat_asl_store_error_t err;
if (out_record_refs == NULL || out_len == NULL ||
out_next_offset == NULL || out_end == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
*out_record_refs = NULL;
*out_len = 0u;
if (collection_store == NULL || collection_store->store == NULL ||
collection_name == NULL) {
return AMDUAT_ASL_COLLECTION_ERR_INTEGRITY;
}
if (!amduat_asl_pointer_name_is_valid(collection_name)) {
return AMDUAT_ASL_COLLECTION_ERR_INVALID_NAME;
}
if (!amduat_asl_collection_build_log_name(collection_name, &log_name)) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
err = amduat_asl_log_read(&collection_store->log_store, log_name,
from_offset, limit, &entries, &entries_len,
out_next_offset, out_end);
free(log_name);
if (err != AMDUAT_ASL_STORE_OK) {
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
if (entries_len != 0u) {
amduat_reference_t *refs =
(amduat_reference_t *)calloc(entries_len, sizeof(*refs));
if (refs == NULL) {
amduat_asl_log_entries_free(entries, entries_len);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
for (size_t i = 0u; i < entries_len; ++i) {
refs[i].hash_id = entries[i].payload_ref.hash_id;
if (!amduat_octets_clone(entries[i].payload_ref.digest,
&refs[i].digest)) {
amduat_asl_collection_refs_free(refs, i);
amduat_asl_log_entries_free(entries, entries_len);
return AMDUAT_ASL_COLLECTION_ERR_IO;
}
}
*out_record_refs = refs;
*out_len = entries_len;
}
amduat_asl_log_entries_free(entries, entries_len);
return AMDUAT_ASL_COLLECTION_OK;
}
void amduat_asl_collection_refs_free(amduat_reference_t *refs,
size_t refs_len) {
if (refs == NULL) {
return;
}
for (size_t i = 0u; i < refs_len; ++i) {
amduat_reference_free(&refs[i]);
}
free(refs);
}
bool amduat_asl_collection_snapshot_payload_decode_v1(
amduat_octets_t payload,
amduat_asl_collection_snapshot_payload_t *out_snapshot) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t entry_count = 0u;
if (out_snapshot == NULL) {
return false;
}
out_snapshot->snapshot_offset = 0u;
out_snapshot->refs = NULL;
out_snapshot->refs_len = 0u;
if (payload.len < AMDUAT_ASL_COLLECTION_MAGIC_LEN + 4u + 8u + 4u) {
return false;
}
if (payload.data == NULL) {
return false;
}
if (memcmp(payload.data, k_amduat_asl_collection_magic,
AMDUAT_ASL_COLLECTION_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_COLLECTION_MAGIC_LEN;
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
&offset, &version) ||
version != AMDUAT_ASL_COLLECTION_VERSION) {
return false;
}
if (!amduat_asl_collection_read_u64_le(payload.data, payload.len,
&offset,
&out_snapshot->snapshot_offset)) {
return false;
}
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
&offset, &entry_count)) {
return false;
}
if (entry_count == 0u) {
return offset == payload.len;
}
if (entry_count > SIZE_MAX / sizeof(amduat_reference_t)) {
return false;
}
out_snapshot->refs =
(amduat_reference_t *)calloc(entry_count, sizeof(*out_snapshot->refs));
if (out_snapshot->refs == NULL) {
return false;
}
out_snapshot->refs_len = entry_count;
for (uint32_t i = 0u; i < entry_count; ++i) {
uint32_t ref_len = 0u;
amduat_octets_t ref_bytes;
if (!amduat_asl_collection_read_u32_le(payload.data, payload.len,
&offset, &ref_len)) {
goto fail;
}
if (payload.len - offset < ref_len) {
goto fail;
}
ref_bytes = amduat_octets(payload.data + offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(
ref_bytes, &out_snapshot->refs[i])) {
goto fail;
}
offset += ref_len;
}
return offset == payload.len;
fail:
amduat_asl_collection_snapshot_payload_free(out_snapshot);
return false;
}
void amduat_asl_collection_snapshot_payload_free(
amduat_asl_collection_snapshot_payload_t *snapshot) {
if (snapshot == NULL || snapshot->refs == NULL) {
return;
}
for (size_t i = 0u; i < snapshot->refs_len; ++i) {
amduat_reference_free(&snapshot->refs[i]);
}
free(snapshot->refs);
snapshot->refs = NULL;
snapshot->refs_len = 0u;
}

1021
src/core/asl_log_store.c
View file

File diff suppressed because it is too large Load diff

268
src/core/asl_record.c
View file

@ -1,268 +0,0 @@
#include "amduat/asl/record.h"
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_RECORD_MAGIC_LEN = 8,
AMDUAT_ASL_RECORD_VERSION = 1
};
static const uint8_t k_amduat_asl_record_magic[AMDUAT_ASL_RECORD_MAGIC_LEN] = {
'A', 'S', 'L', 'R', 'E', 'C', '1', '\0'
};
static void amduat_asl_record_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_record_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_record_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_record_schema_is_ascii(amduat_octets_t schema) {
if (schema.len == 0u || schema.data == NULL) {
return false;
}
for (size_t i = 0u; i < schema.len; ++i) {
uint8_t c = schema.data[i];
if (c < 0x20u || c > 0x7eu) {
return false;
}
}
return true;
}
bool amduat_asl_record_encode_v1(const amduat_asl_record_t *record,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t offset = 0u;
uint32_t schema_len;
uint32_t payload_len;
uint8_t flags = 0u;
if (out_bytes == NULL || record == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_record_schema_is_ascii(record->schema)) {
return false;
}
if (record->payload.len != 0u && record->payload.data == NULL) {
return false;
}
if (record->schema.len > UINT32_MAX || record->payload.len > UINT32_MAX) {
return false;
}
schema_len = (uint32_t)record->schema.len;
payload_len = (uint32_t)record->payload.len;
if (!amduat_asl_record_add_size(&total_len,
AMDUAT_ASL_RECORD_MAGIC_LEN + 4u) ||
!amduat_asl_record_add_size(&total_len, 4u + schema_len) ||
!amduat_asl_record_add_size(&total_len, 4u + payload_len) ||
!amduat_asl_record_add_size(&total_len, 1u)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_record_magic,
AMDUAT_ASL_RECORD_MAGIC_LEN);
offset += AMDUAT_ASL_RECORD_MAGIC_LEN;
amduat_asl_record_store_u32_le(buffer + offset, AMDUAT_ASL_RECORD_VERSION);
offset += 4u;
amduat_asl_record_store_u32_le(buffer + offset, schema_len);
offset += 4u;
memcpy(buffer + offset, record->schema.data, schema_len);
offset += schema_len;
amduat_asl_record_store_u32_le(buffer + offset, payload_len);
offset += 4u;
if (payload_len != 0u) {
memcpy(buffer + offset, record->payload.data, payload_len);
offset += payload_len;
}
buffer[offset++] = flags;
if (offset != total_len) {
free(buffer);
return false;
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_asl_record_decode_v1(amduat_octets_t bytes,
amduat_asl_record_t *out_record) {
size_t offset = 0u;
uint32_t version;
uint32_t schema_len;
uint32_t payload_len;
uint8_t flags;
uint8_t *schema_bytes;
uint8_t *payload_bytes;
if (out_record == NULL) {
return false;
}
out_record->schema = amduat_octets(NULL, 0u);
out_record->payload = amduat_octets(NULL, 0u);
if (bytes.len < AMDUAT_ASL_RECORD_MAGIC_LEN + 4u + 1u) {
return false;
}
if (bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_record_magic,
AMDUAT_ASL_RECORD_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_RECORD_MAGIC_LEN;
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
&version) ||
version != AMDUAT_ASL_RECORD_VERSION) {
return false;
}
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
&schema_len)) {
return false;
}
if (bytes.len - offset < schema_len) {
return false;
}
schema_bytes = NULL;
if (schema_len != 0u) {
schema_bytes = (uint8_t *)malloc(schema_len);
if (schema_bytes == NULL) {
return false;
}
memcpy(schema_bytes, bytes.data + offset, schema_len);
}
offset += schema_len;
if (!amduat_asl_record_read_u32_le(bytes.data, bytes.len, &offset,
&payload_len)) {
free(schema_bytes);
return false;
}
if (bytes.len - offset < payload_len + 1u) {
free(schema_bytes);
return false;
}
payload_bytes = NULL;
if (payload_len != 0u) {
payload_bytes = (uint8_t *)malloc(payload_len);
if (payload_bytes == NULL) {
free(schema_bytes);
return false;
}
memcpy(payload_bytes, bytes.data + offset, payload_len);
}
offset += payload_len;
flags = bytes.data[offset++];
(void)flags;
if (offset != bytes.len) {
free(schema_bytes);
free(payload_bytes);
return false;
}
out_record->schema = amduat_octets(schema_bytes, schema_len);
out_record->payload = amduat_octets(payload_bytes, payload_len);
return true;
}
void amduat_asl_record_free(amduat_asl_record_t *record) {
if (record == NULL) {
return;
}
amduat_octets_free(&record->schema);
amduat_octets_free(&record->payload);
}
amduat_asl_store_error_t amduat_asl_record_store_put(
amduat_asl_store_t *store,
amduat_octets_t schema,
amduat_octets_t payload,
amduat_reference_t *out_ref) {
amduat_asl_record_t record;
amduat_octets_t encoded = amduat_octets(NULL, 0u);
amduat_artifact_t artifact;
amduat_asl_store_error_t err;
if (store == NULL || out_ref == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
record.schema = schema;
record.payload = payload;
if (!amduat_asl_record_encode_v1(&record, &encoded)) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
artifact = amduat_artifact_with_type(
encoded, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_RECORD_1));
err = amduat_asl_store_put(store, artifact, out_ref);
free((void *)encoded.data);
return err;
}
amduat_asl_store_error_t amduat_asl_record_store_get(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_record_t *out_record) {
amduat_artifact_t artifact;
amduat_asl_store_error_t err;
if (store == NULL || out_record == NULL) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
err = amduat_asl_store_get(store, ref, &artifact);
if (err != AMDUAT_ASL_STORE_OK) {
return err;
}
if (!artifact.has_type_tag ||
artifact.type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_RECORD_1) {
amduat_artifact_free(&artifact);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
if (!amduat_asl_record_decode_v1(artifact.bytes, out_record)) {
amduat_artifact_free(&artifact);
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
amduat_artifact_free(&artifact);
return AMDUAT_ASL_STORE_OK;
}

161
src/core/derivation_sid.c
View file

@ -1,161 +0,0 @@
#include "amduat/pel/derivation_sid.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/hash/asl1.h"
#include <assert.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
static bool amduat_derivation_sid_check_ref(amduat_reference_t ref) {
const amduat_hash_asl1_desc_t *desc;
size_t digest_len;
if (ref.digest.len != 0 && ref.digest.data == NULL) {
return false;
}
if (amduat_hash_asl1_is_reserved(ref.hash_id)) {
return false;
}
desc = amduat_hash_asl1_desc_lookup(ref.hash_id);
digest_len = ref.digest.len;
if (desc != NULL && desc->digest_len != 0) {
assert(desc->digest_len == digest_len);
if (desc->digest_len != digest_len) {
return false;
}
}
return true;
}
bool amduat_pel_derivation_sid_compute(
const amduat_pel_derivation_sid_input_t *in,
amduat_octets_t *out_sid) {
amduat_hash_asl1_stream_t stream;
uint8_t marker;
size_t i;
if (out_sid == NULL) {
return false;
}
out_sid->data = NULL;
out_sid->len = 0;
if (in == NULL) {
return false;
}
if (in->input_refs_len != 0 && in->input_refs == NULL) {
return false;
}
if (in->has_params_ref && !amduat_derivation_sid_check_ref(in->params_ref)) {
return false;
}
if (!amduat_derivation_sid_check_ref(in->program_ref)) {
return false;
}
for (i = 0; i < in->input_refs_len; ++i) {
if (!amduat_derivation_sid_check_ref(in->input_refs[i])) {
return false;
}
}
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
return false;
}
{
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
bool ok;
/* ReferenceBytes are self-delimiting because hash_id implies digest_len. */
ok = amduat_enc_asl1_core_encode_reference_v1(in->program_ref, &ref_bytes);
if (!ok ||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
amduat_hash_asl1_stream_destroy(&stream);
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
for (i = 0; i < in->input_refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
bool ok = amduat_enc_asl1_core_encode_reference_v1(in->input_refs[i],
&ref_bytes);
if (!ok ||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
amduat_hash_asl1_stream_destroy(&stream);
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
marker = in->has_params_ref ? 0x01u : 0x00u;
if (!amduat_hash_asl1_stream_update(&stream,
amduat_octets(&marker, 1u))) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (in->has_params_ref) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
bool ok = amduat_enc_asl1_core_encode_reference_v1(in->params_ref,
&ref_bytes);
if (!ok ||
!amduat_hash_asl1_stream_update(&stream, ref_bytes)) {
amduat_hash_asl1_stream_destroy(&stream);
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
/* ExecProfile is currently absent; presence must be explicitly marked. */
marker = in->has_exec_profile ? 0x01u : 0x00u;
if (!amduat_hash_asl1_stream_update(&stream,
amduat_octets(&marker, 1u))) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (in->has_exec_profile) {
if (in->exec_profile.len != 0 && in->exec_profile.data == NULL) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (!amduat_hash_asl1_stream_update(&stream, in->exec_profile)) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
}
{
const amduat_hash_asl1_desc_t *desc =
amduat_hash_asl1_desc_lookup(AMDUAT_HASH_ASL1_ID_SHA256);
uint8_t *digest;
size_t digest_len;
if (desc == NULL || desc->digest_len == 0u) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
digest_len = desc->digest_len;
digest = (uint8_t *)malloc(digest_len);
if (digest == NULL) {
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
if (!amduat_hash_asl1_stream_final(&stream, digest, digest_len)) {
free(digest);
amduat_hash_asl1_stream_destroy(&stream);
return false;
}
amduat_hash_asl1_stream_destroy(&stream);
*out_sid = amduat_octets(digest, digest_len);
}
return true;
}

41
src/internal/log.c
View file

@ -1,41 +0,0 @@
#include "amduat/util/log.h"
#include <stdbool.h>
#include <stdio.h>
static const char *amduat_log_level_label(amduat_log_level_t level) {
switch (level) {
case AMDUAT_LOG_ERROR:
return "ERROR";
case AMDUAT_LOG_WARN:
return "WARN";
case AMDUAT_LOG_INFO:
return "INFO";
case AMDUAT_LOG_DEBUG:
return "DEBUG";
default:
return "LOG";
}
}
static bool amduat_log_debug_enabled(void) {
#ifdef AMDUAT_LOG_ENABLE_DEBUG
return true;
#else
return false;
#endif
}
void amduat_log(amduat_log_level_t level, const char *fmt, ...) {
va_list ap;
if (level == AMDUAT_LOG_DEBUG && !amduat_log_debug_enabled()) {
return;
}
fprintf(stderr, "%s: ", amduat_log_level_label(level));
va_start(ap, fmt);
vfprintf(stderr, fmt, ap);
va_end(ap);
fputc('\n', stderr);
}

20
src/internal/string.c
View file

@ -1,20 +0,0 @@
#include "amduat/util/string.h"
#include <stdlib.h>
#include <string.h>
char *amduat_strdup(const char *src) {
char *out;
size_t len;
if (src == NULL) {
return NULL;
}
len = strlen(src) + 1u;
out = (char *)malloc(len);
if (out == NULL) {
return NULL;
}
memcpy(out, src, len);
return out;
}

520
src/near_core/asl/collection_view.c
View file

@ -1,520 +0,0 @@
#include "amduat/asl/collection_view.h"
#include "amduat/enc/asl1_core_codec.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN = 8,
AMDUAT_ASL_COLLECTION_VIEW_VERSION = 1,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN = 8,
AMDUAT_ASL_SNAPSHOT_INFO_VERSION = 1,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN = 8,
AMDUAT_ASL_LOG_RANGE_VERSION = 1
};
static const uint8_t k_amduat_asl_collection_view_magic[
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN] = {
'A', 'S', 'L', 'C', 'V', 'W', '1', '\0'
};
static const uint8_t k_amduat_asl_snapshot_info_magic[
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN] = {
'A', 'S', 'L', 'S', 'N', 'P', '1', '\0'
};
static const uint8_t k_amduat_asl_log_range_magic[
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN] = {
'A', 'S', 'L', 'R', 'N', 'G', '1', '\0'
};
static void amduat_asl_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_read_u64_le(const uint8_t *data,
size_t len,
size_t *offset,
uint64_t *out) {
if (len - *offset < 8u) {
return false;
}
*out = (uint64_t)data[*offset] |
((uint64_t)data[*offset + 1u] << 8) |
((uint64_t)data[*offset + 2u] << 16) |
((uint64_t)data[*offset + 3u] << 24) |
((uint64_t)data[*offset + 4u] << 32) |
((uint64_t)data[*offset + 5u] << 40) |
((uint64_t)data[*offset + 6u] << 48) |
((uint64_t)data[*offset + 7u] << 56);
*offset += 8u;
return true;
}
static bool amduat_asl_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_encode_refs(const amduat_reference_t *refs,
size_t refs_len,
size_t *out_total_len) {
size_t total = 0u;
if (refs_len > UINT32_MAX) {
return false;
}
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total, 4u + ref_bytes.len)) {
free((void *)ref_bytes.data);
return false;
}
free((void *)ref_bytes.data);
}
*out_total_len = total;
return true;
}
static bool amduat_asl_write_refs(uint8_t *buffer,
size_t total_len,
size_t *offset,
const amduat_reference_t *refs,
size_t refs_len) {
for (size_t i = 0u; i < refs_len; ++i) {
amduat_octets_t ref_bytes = amduat_octets(NULL, 0u);
if (!amduat_enc_asl1_core_encode_reference_v1(refs[i], &ref_bytes)) {
return false;
}
if (total_len - *offset < 4u + ref_bytes.len) {
free((void *)ref_bytes.data);
return false;
}
amduat_asl_store_u32_le(buffer + *offset, (uint32_t)ref_bytes.len);
*offset += 4u;
memcpy(buffer + *offset, ref_bytes.data, ref_bytes.len);
*offset += ref_bytes.len;
free((void *)ref_bytes.data);
}
return true;
}
static bool amduat_asl_decode_refs(amduat_octets_t bytes,
size_t *offset,
uint32_t refs_len,
amduat_reference_t **out_refs,
size_t *out_refs_len) {
amduat_reference_t *refs = NULL;
if (out_refs == NULL || out_refs_len == NULL) {
return false;
}
*out_refs = NULL;
*out_refs_len = 0u;
if (refs_len == 0u) {
return true;
}
if (refs_len > SIZE_MAX / sizeof(*refs)) {
return false;
}
refs = (amduat_reference_t *)calloc(refs_len, sizeof(*refs));
if (refs == NULL) {
return false;
}
for (uint32_t i = 0u; i < refs_len; ++i) {
uint32_t ref_len = 0u;
amduat_octets_t ref_bytes;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, offset, &ref_len)) {
goto fail;
}
if (bytes.len - *offset < ref_len) {
goto fail;
}
ref_bytes = amduat_octets(bytes.data + *offset, ref_len);
if (!amduat_enc_asl1_core_decode_reference_v1(ref_bytes, &refs[i])) {
goto fail;
}
*offset += ref_len;
}
*out_refs = refs;
*out_refs_len = refs_len;
return true;
fail:
for (uint32_t i = 0u; i < refs_len; ++i) {
amduat_reference_free(&refs[i]);
}
free(refs);
return false;
}
bool amduat_asl_collection_snapshot_info_encode_v1(
const amduat_asl_collection_snapshot_info_t *info,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t refs_len_bytes = 0u;
size_t offset = 0u;
if (out_bytes == NULL || info == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_add_size(&total_len,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN + 4u + 8u + 4u)) {
return false;
}
if (!amduat_asl_encode_refs(info->refs, info->refs_len, &refs_len_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_snapshot_info_magic,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN);
offset += AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN;
amduat_asl_store_u32_le(buffer + offset, AMDUAT_ASL_SNAPSHOT_INFO_VERSION);
offset += 4u;
amduat_asl_store_u64_le(buffer + offset, info->snapshot_at_offset);
offset += 8u;
amduat_asl_store_u32_le(buffer + offset, (uint32_t)info->refs_len);
offset += 4u;
if (!amduat_asl_write_refs(buffer, total_len, &offset,
info->refs, info->refs_len)) {
free(buffer);
return false;
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_collection_snapshot_info_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_snapshot_info_t *out_info) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t refs_len = 0u;
if (out_info == NULL) {
return false;
}
out_info->snapshot_at_offset = 0u;
out_info->refs = NULL;
out_info->refs_len = 0u;
if (bytes.len < AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN + 4u + 8u + 4u ||
bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_snapshot_info_magic,
AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_SNAPSHOT_INFO_MAGIC_LEN;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_SNAPSHOT_INFO_VERSION) {
return false;
}
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_info->snapshot_at_offset)) {
return false;
}
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
return false;
}
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
&out_info->refs, &out_info->refs_len)) {
return false;
}
return offset == bytes.len;
}
void amduat_asl_collection_snapshot_info_free(
amduat_asl_collection_snapshot_info_t *info) {
if (info == NULL || info->refs == NULL) {
return;
}
for (size_t i = 0u; i < info->refs_len; ++i) {
amduat_reference_free(&info->refs[i]);
}
free(info->refs);
info->refs = NULL;
info->refs_len = 0u;
}
bool amduat_asl_log_range_encode_v1(const amduat_asl_log_range_t *range,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t refs_len_bytes = 0u;
size_t offset = 0u;
if (out_bytes == NULL || range == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_add_size(&total_len,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN + 4u + 8u + 8u + 4u)) {
return false;
}
if (!amduat_asl_encode_refs(range->refs, range->refs_len, &refs_len_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_log_range_magic,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN);
offset += AMDUAT_ASL_LOG_RANGE_MAGIC_LEN;
amduat_asl_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_RANGE_VERSION);
offset += 4u;
amduat_asl_store_u64_le(buffer + offset, range->start_offset);
offset += 8u;
amduat_asl_store_u64_le(buffer + offset, range->next_offset);
offset += 8u;
amduat_asl_store_u32_le(buffer + offset, (uint32_t)range->refs_len);
offset += 4u;
if (!amduat_asl_write_refs(buffer, total_len, &offset,
range->refs, range->refs_len)) {
free(buffer);
return false;
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_log_range_decode_v1(amduat_octets_t bytes,
amduat_asl_log_range_t *out_range) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t refs_len = 0u;
if (out_range == NULL) {
return false;
}
out_range->start_offset = 0u;
out_range->next_offset = 0u;
out_range->refs = NULL;
out_range->refs_len = 0u;
if (bytes.len <
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN + 4u + 8u + 8u + 4u ||
bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_log_range_magic,
AMDUAT_ASL_LOG_RANGE_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_LOG_RANGE_MAGIC_LEN;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_LOG_RANGE_VERSION) {
return false;
}
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_range->start_offset) ||
!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_range->next_offset)) {
return false;
}
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
return false;
}
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
&out_range->refs, &out_range->refs_len)) {
return false;
}
return offset == bytes.len;
}
void amduat_asl_log_range_free(amduat_asl_log_range_t *range) {
if (range == NULL || range->refs == NULL) {
return;
}
for (size_t i = 0u; i < range->refs_len; ++i) {
amduat_reference_free(&range->refs[i]);
}
free(range->refs);
range->refs = NULL;
range->refs_len = 0u;
}
bool amduat_asl_collection_view_encode_v1(
const amduat_asl_collection_view_t *view,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t refs_len_bytes = 0u;
size_t offset = 0u;
if (out_bytes == NULL || view == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (!amduat_asl_add_size(&total_len,
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN + 4u +
8u + 8u + 4u)) {
return false;
}
if (!amduat_asl_encode_refs(view->refs, view->refs_len, &refs_len_bytes)) {
return false;
}
if (!amduat_asl_add_size(&total_len, refs_len_bytes)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_collection_view_magic,
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN);
offset += AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN;
amduat_asl_store_u32_le(buffer + offset,
AMDUAT_ASL_COLLECTION_VIEW_VERSION);
offset += 4u;
amduat_asl_store_u64_le(buffer + offset, view->computed_from_offset);
offset += 8u;
amduat_asl_store_u64_le(buffer + offset, view->computed_up_to_offset);
offset += 8u;
amduat_asl_store_u32_le(buffer + offset, (uint32_t)view->refs_len);
offset += 4u;
if (!amduat_asl_write_refs(buffer, total_len, &offset,
view->refs, view->refs_len)) {
free(buffer);
return false;
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_collection_view_decode_v1(
amduat_octets_t bytes,
amduat_asl_collection_view_t *out_view) {
size_t offset = 0u;
uint32_t version = 0u;
uint32_t refs_len = 0u;
if (out_view == NULL) {
return false;
}
out_view->computed_from_offset = 0u;
out_view->computed_up_to_offset = 0u;
out_view->refs = NULL;
out_view->refs_len = 0u;
if (bytes.len <
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN + 4u + 8u + 8u + 4u ||
bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_collection_view_magic,
AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_COLLECTION_VIEW_MAGIC_LEN;
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_COLLECTION_VIEW_VERSION) {
return false;
}
if (!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_view->computed_from_offset) ||
!amduat_asl_read_u64_le(bytes.data, bytes.len, &offset,
&out_view->computed_up_to_offset)) {
return false;
}
if (!amduat_asl_read_u32_le(bytes.data, bytes.len, &offset, &refs_len)) {
return false;
}
if (!amduat_asl_decode_refs(bytes, &offset, refs_len,
&out_view->refs, &out_view->refs_len)) {
return false;
}
return offset == bytes.len;
}
void amduat_asl_collection_view_free(amduat_asl_collection_view_t *view) {
if (view == NULL || view->refs == NULL) {
return;
}
for (size_t i = 0u; i < view->refs_len; ++i) {
amduat_reference_free(&view->refs[i]);
}
free(view->refs);
view->refs = NULL;
view->refs_len = 0u;
}

18
src/near_core/asl/identity.c
View file

@ -1,18 +0,0 @@
#include "amduat/asl/identity.h"
static const uint8_t k_asl_identity_domain_sep[] = {
'A', 'M', 'D', 'U', 'A', 'T', ':', 'A',
'S', 'L', '1', ':', 'A', 'R', 'T', 0x00};
amduat_octets_t amduat_asl_identity_domain_separator(void) {
return amduat_octets(k_asl_identity_domain_sep,
sizeof(k_asl_identity_domain_sep));
}
const amduat_asl_identity_rules_t *amduat_asl_identity_rules(void) {
static const amduat_asl_identity_rules_t k_rules = {
AMDUAT_HASH_ASL1_REGISTRY,
AMDUAT_HASH_ASL1_VERSION,
{k_asl_identity_domain_sep, sizeof(k_asl_identity_domain_sep)}};
return &k_rules;
}

158
src/near_core/asl/index_accel.c
View file

@ -1,158 +0,0 @@
#include "amduat/asl/index_accel.h"
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE = 4,
AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE = 5
};
static uint64_t amduat_asl_index_accel_fnv1a64_update(uint64_t hash,
const uint8_t *data,
size_t len) {
size_t i;
if (data == NULL || len == 0u) {
return hash;
}
for (i = 0u; i < len; ++i) {
hash ^= data[i];
hash *= 1099511628211ull;
}
return hash;
}
static void amduat_asl_index_accel_store_u16_le(uint8_t *out,
uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_index_accel_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_index_accel_hash_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
uint64_t *out_hash) {
uint8_t header[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE];
uint8_t tag_buf[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE];
uint16_t digest_len;
uint64_t hash;
if (out_hash == NULL) {
return false;
}
if (ref.digest.len != 0u && ref.digest.data == NULL) {
return false;
}
if (ref.digest.len > UINT16_MAX) {
return false;
}
digest_len = (uint16_t)ref.digest.len;
amduat_asl_index_accel_store_u16_le(header, ref.hash_id);
amduat_asl_index_accel_store_u16_le(header + 2u, digest_len);
tag_buf[0] = has_type_tag ? 1u : 0u;
amduat_asl_index_accel_store_u32_le(
tag_buf + 1u, has_type_tag ? type_tag.tag_id : 0u);
hash = 14695981039346656037ull;
hash = amduat_asl_index_accel_fnv1a64_update(hash, header, sizeof(header));
hash = amduat_asl_index_accel_fnv1a64_update(
hash, ref.digest.data, ref.digest.len);
hash = amduat_asl_index_accel_fnv1a64_update(hash, tag_buf, sizeof(tag_buf));
*out_hash = hash;
return true;
}
bool amduat_asl_index_accel_routing_key_from_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
amduat_octets_t *out_key) {
size_t total_len;
uint8_t *buffer;
uint8_t tag_flag;
uint16_t digest_len;
if (out_key == NULL) {
return false;
}
*out_key = amduat_octets(NULL, 0u);
if (ref.digest.len != 0u && ref.digest.data == NULL) {
return false;
}
if (ref.digest.len > UINT16_MAX) {
return false;
}
digest_len = (uint16_t)ref.digest.len;
total_len = AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE +
ref.digest.len + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_TAG_SIZE;
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
amduat_asl_index_accel_store_u16_le(buffer, ref.hash_id);
amduat_asl_index_accel_store_u16_le(buffer + 2u, digest_len);
if (ref.digest.len != 0u) {
memcpy(buffer + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE,
ref.digest.data,
ref.digest.len);
}
tag_flag = has_type_tag ? 1u : 0u;
buffer[AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE + ref.digest.len] =
tag_flag;
amduat_asl_index_accel_store_u32_le(
buffer + AMDUAT_ASL_INDEX_ACCEL_ROUTING_KEY_HEADER_SIZE + ref.digest.len +
1u,
has_type_tag ? type_tag.tag_id : 0u);
*out_key = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_index_accel_routing_key_hash(amduat_octets_t key,
uint64_t *out_hash) {
uint64_t hash;
if (out_hash == NULL) {
return false;
}
if (key.len != 0u && key.data == NULL) {
return false;
}
hash = 14695981039346656037ull;
hash = amduat_asl_index_accel_fnv1a64_update(hash, key.data, key.len);
*out_hash = hash;
return true;
}
uint16_t amduat_asl_index_accel_shard_for_ref(
amduat_reference_t ref,
bool has_type_tag,
amduat_type_tag_t type_tag,
uint16_t shard_count) {
uint64_t hash = 0u;
if (shard_count == 0u) {
return 0u;
}
if (!amduat_asl_index_accel_hash_ref(ref, has_type_tag, type_tag, &hash)) {
return 0u;
}
return (uint16_t)(hash % shard_count);
}

152
src/near_core/asl/index_bloom.c
View file

@ -1,152 +0,0 @@
#include "amduat/asl/index_bloom.h"
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
static uint64_t amduat_asl_index_bloom_hash_update(uint64_t hash,
const uint8_t *data,
size_t len) {
size_t i;
if (data == NULL || len == 0u) {
return hash;
}
for (i = 0u; i < len; ++i) {
hash ^= (uint64_t)data[i];
hash *= 1099511628211ull;
}
return hash;
}
static bool amduat_asl_index_bloom_hash_pair(amduat_hash_id_t hash_id,
amduat_octets_t digest,
uint64_t *out_h1,
uint64_t *out_h2) {
uint64_t hash;
uint8_t header[4];
uint16_t digest_len;
if (out_h1 == NULL || out_h2 == NULL) {
return false;
}
if (digest.len != 0u && digest.data == NULL) {
return false;
}
if (digest.len > UINT16_MAX) {
return false;
}
digest_len = (uint16_t)digest.len;
header[0] = (uint8_t)(hash_id & 0xffu);
header[1] = (uint8_t)((hash_id >> 8) & 0xffu);
header[2] = (uint8_t)(digest_len & 0xffu);
header[3] = (uint8_t)((digest_len >> 8) & 0xffu);
hash = 14695981039346656037ull ^ 0x9e3779b97f4a7c15ull;
hash = amduat_asl_index_bloom_hash_update(hash, header, sizeof(header));
hash = amduat_asl_index_bloom_hash_update(hash, digest.data, digest.len);
*out_h1 = hash;
hash = 14695981039346656037ull ^ 0xbf58476d1ce4e5b9ull;
hash = amduat_asl_index_bloom_hash_update(hash, header, sizeof(header));
hash = amduat_asl_index_bloom_hash_update(hash, digest.data, digest.len);
if (hash == 0u) {
hash = 0x94d049bb133111ebull;
}
*out_h2 = hash;
return true;
}
static void amduat_asl_index_bloom_set_bit(uint8_t *data, size_t bit) {
data[bit >> 3] |= (uint8_t)(1u << (bit & 7u));
}
static bool amduat_asl_index_bloom_test_bit(const uint8_t *data, size_t bit) {
return (data[bit >> 3] & (uint8_t)(1u << (bit & 7u))) != 0u;
}
bool amduat_asl_index_bloom_init(amduat_octets_t *out_bloom) {
uint8_t *data;
if (out_bloom == NULL) {
return false;
}
*out_bloom = amduat_octets(NULL, 0u);
if (AMDUAT_ASL_INDEX_BLOOM_BYTES == 0u) {
return true;
}
data = (uint8_t *)calloc(1u, AMDUAT_ASL_INDEX_BLOOM_BYTES);
if (data == NULL) {
return false;
}
*out_bloom = amduat_octets(data, AMDUAT_ASL_INDEX_BLOOM_BYTES);
return true;
}
bool amduat_asl_index_bloom_add(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest) {
uint64_t h1;
uint64_t h2;
size_t i;
size_t bit_count;
uint8_t *data;
if (bloom.len == 0u) {
return false;
}
if (bloom.data == NULL) {
return false;
}
bit_count = bloom.len * 8u;
if (bit_count == 0u) {
return false;
}
if (!amduat_asl_index_bloom_hash_pair(hash_id, digest, &h1, &h2)) {
return false;
}
data = (uint8_t *)bloom.data;
for (i = 0u; i < AMDUAT_ASL_INDEX_BLOOM_HASHES; ++i) {
uint64_t mix = h1 + (uint64_t)i * h2;
size_t bit = (size_t)(mix % (uint64_t)bit_count);
amduat_asl_index_bloom_set_bit(data, bit);
}
return true;
}
bool amduat_asl_index_bloom_maybe_contains(amduat_octets_t bloom,
amduat_hash_id_t hash_id,
amduat_octets_t digest) {
uint64_t h1;
uint64_t h2;
size_t i;
size_t bit_count;
if (bloom.len == 0u || bloom.data == NULL) {
return true;
}
bit_count = bloom.len * 8u;
if (bit_count == 0u) {
return true;
}
if (!amduat_asl_index_bloom_hash_pair(hash_id, digest, &h1, &h2)) {
return true;
}
for (i = 0u; i < AMDUAT_ASL_INDEX_BLOOM_HASHES; ++i) {
uint64_t mix = h1 + (uint64_t)i * h2;
size_t bit = (size_t)(mix % (uint64_t)bit_count);
if (!amduat_asl_index_bloom_test_bit(bloom.data, bit)) {
return false;
}
}
return true;
}

341
src/near_core/asl/index_replay.c
View file

@ -1,341 +0,0 @@
#include "amduat/asl/index_replay.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_replay_cursor_t;
static bool amduat_asl_replay_read_u16_le(amduat_asl_replay_cursor_t *cur,
uint16_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 2) {
return false;
}
data = cur->data + cur->offset;
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
cur->offset += 2;
return true;
}
static bool amduat_asl_replay_read_u32_le(amduat_asl_replay_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4;
return true;
}
static bool amduat_asl_replay_read_u64_le(amduat_asl_replay_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8;
return true;
}
static bool amduat_asl_replay_parse_artifact_ref(
amduat_asl_replay_cursor_t *cur,
amduat_reference_t *out_ref) {
uint32_t hash_id_raw;
uint16_t digest_len;
uint16_t reserved0;
const uint8_t *digest;
if (!amduat_asl_replay_read_u32_le(cur, &hash_id_raw) ||
!amduat_asl_replay_read_u16_le(cur, &digest_len) ||
!amduat_asl_replay_read_u16_le(cur, &reserved0)) {
return false;
}
if (hash_id_raw > UINT16_MAX || digest_len == 0 || reserved0 != 0) {
return false;
}
if (cur->len - cur->offset < digest_len) {
return false;
}
digest = cur->data + cur->offset;
cur->offset += digest_len;
*out_ref = amduat_reference((amduat_hash_id_t)hash_id_raw,
amduat_octets(digest, digest_len));
return true;
}
static bool amduat_asl_replay_parse_segment_seal(
amduat_octets_t payload,
amduat_asl_segment_seal_t *out) {
amduat_asl_replay_cursor_t cur;
uint64_t segment_id;
if (payload.len != 8u + 32u || payload.data == NULL || out == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_read_u64_le(&cur, &segment_id)) {
return false;
}
if (cur.len - cur.offset < 32) {
return false;
}
out->segment_id = segment_id;
memcpy(out->segment_hash, cur.data + cur.offset, 32);
return true;
}
static bool amduat_asl_replay_parse_tombstone(
amduat_octets_t payload,
amduat_reference_t *out_ref) {
amduat_asl_replay_cursor_t cur;
uint32_t scope;
uint32_t reason;
if (payload.len == 0 || payload.data == NULL || out_ref == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_parse_artifact_ref(&cur, out_ref)) {
return false;
}
if (!amduat_asl_replay_read_u32_le(&cur, &scope) ||
!amduat_asl_replay_read_u32_le(&cur, &reason)) {
return false;
}
(void)scope;
(void)reason;
return cur.offset == cur.len;
}
static bool amduat_asl_replay_parse_tombstone_lift(
amduat_octets_t payload,
amduat_reference_t *out_ref,
uint64_t *out_logseq) {
amduat_asl_replay_cursor_t cur;
uint64_t tombstone_logseq;
if (payload.len == 0 || payload.data == NULL || out_ref == NULL ||
out_logseq == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_parse_artifact_ref(&cur, out_ref) ||
!amduat_asl_replay_read_u64_le(&cur, &tombstone_logseq)) {
return false;
}
*out_logseq = tombstone_logseq;
return cur.offset == cur.len;
}
static bool amduat_asl_replay_parse_snapshot_anchor(
amduat_octets_t payload,
amduat_asl_snapshot_id_t *out_snapshot_id) {
amduat_asl_replay_cursor_t cur;
uint64_t snapshot_id;
if (payload.len != 8u + 32u || payload.data == NULL ||
out_snapshot_id == NULL) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_replay_read_u64_le(&cur, &snapshot_id)) {
return false;
}
*out_snapshot_id = snapshot_id;
return true;
}
static bool amduat_asl_replay_update_segment(
amduat_asl_replay_state_t *state,
const amduat_asl_segment_seal_t *seal) {
size_t i;
amduat_asl_segment_seal_t *next;
for (i = 0; i < state->segments_len; ++i) {
if (state->segments[i].segment_id == seal->segment_id) {
memcpy(state->segments[i].segment_hash, seal->segment_hash, 32);
return true;
}
}
next = (amduat_asl_segment_seal_t *)realloc(
state->segments,
(state->segments_len + 1u) * sizeof(*state->segments));
if (next == NULL) {
return false;
}
state->segments = next;
state->segments[state->segments_len] = *seal;
state->segments_len += 1u;
return true;
}
static bool amduat_asl_replay_add_tombstone(
amduat_asl_replay_state_t *state,
amduat_reference_t ref,
uint64_t logseq) {
amduat_asl_tombstone_entry_t *next;
amduat_reference_t stored;
if (!amduat_reference_clone(ref, &stored)) {
return false;
}
next = (amduat_asl_tombstone_entry_t *)realloc(
state->tombstones,
(state->tombstones_len + 1u) * sizeof(*state->tombstones));
if (next == NULL) {
amduat_reference_free(&stored);
return false;
}
state->tombstones = next;
state->tombstones[state->tombstones_len].ref = stored;
state->tombstones[state->tombstones_len].tombstone_logseq = logseq;
state->tombstones_len += 1u;
return true;
}
static void amduat_asl_replay_remove_tombstone(
amduat_asl_replay_state_t *state,
amduat_reference_t ref,
uint64_t tombstone_logseq) {
size_t i;
for (i = 0; i < state->tombstones_len; ++i) {
if (state->tombstones[i].tombstone_logseq != tombstone_logseq) {
continue;
}
if (!amduat_reference_eq(state->tombstones[i].ref, ref)) {
continue;
}
amduat_reference_free(&state->tombstones[i].ref);
if (i + 1u < state->tombstones_len) {
state->tombstones[i] = state->tombstones[state->tombstones_len - 1u];
}
state->tombstones_len -= 1u;
return;
}
}
bool amduat_asl_replay_init(amduat_asl_replay_state_t *out) {
if (out == NULL) {
return false;
}
out->segments = NULL;
out->segments_len = 0;
out->tombstones = NULL;
out->tombstones_len = 0;
out->state.snapshot_id = 0;
out->state.log_position = 0;
return true;
}
void amduat_asl_replay_free(amduat_asl_replay_state_t *state) {
size_t i;
if (state == NULL) {
return;
}
free(state->segments);
state->segments = NULL;
state->segments_len = 0;
if (state->tombstones != NULL) {
for (i = 0; i < state->tombstones_len; ++i) {
amduat_reference_free(&state->tombstones[i].ref);
}
}
free(state->tombstones);
state->tombstones = NULL;
state->tombstones_len = 0;
state->state.snapshot_id = 0;
state->state.log_position = 0;
}
bool amduat_asl_replay_apply_log(
const amduat_asl_log_record_t *records,
size_t record_count,
amduat_asl_log_position_t log_position,
amduat_asl_replay_state_t *state) {
size_t i;
if (state == NULL) {
return false;
}
if (record_count != 0 && records == NULL) {
return false;
}
for (i = 0; i < record_count; ++i) {
const amduat_asl_log_record_t *record = &records[i];
amduat_asl_segment_seal_t seal;
amduat_reference_t ref;
uint64_t tombstone_logseq;
amduat_asl_snapshot_id_t snapshot_id;
if (record->logseq > log_position) {
break;
}
switch (record->record_type) {
case AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL:
if (!amduat_asl_replay_parse_segment_seal(record->payload, &seal)) {
return false;
}
if (!amduat_asl_replay_update_segment(state, &seal)) {
return false;
}
break;
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE:
if (!amduat_asl_replay_parse_tombstone(record->payload, &ref)) {
return false;
}
if (!amduat_asl_replay_add_tombstone(state, ref, record->logseq)) {
return false;
}
break;
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT:
if (!amduat_asl_replay_parse_tombstone_lift(record->payload,
&ref,
&tombstone_logseq)) {
return false;
}
amduat_asl_replay_remove_tombstone(state, ref, tombstone_logseq);
break;
case AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR:
if (!amduat_asl_replay_parse_snapshot_anchor(record->payload,
&snapshot_id)) {
return false;
}
state->state.snapshot_id = snapshot_id;
break;
default:
break;
}
}
state->state.log_position = log_position;
return true;
}

487
src/near_core/asl/index_snapshot.c
View file

@ -1,487 +0,0 @@
#include "amduat/asl/index_snapshot.h"
#include "amduat/asl/io.h"
#include "amduat/hash/asl1.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE = 40
};
static const uint8_t k_amduat_asl_snapshot_manifest_magic[
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN] = {'A', 'S', 'L', 'S',
'N', 'A', 'P', '1'};
static void amduat_asl_snapshot_store_u16_le(uint8_t *out,
uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_snapshot_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_snapshot_store_u64_le(uint8_t *out,
uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static uint16_t amduat_asl_snapshot_load_u16_le(const uint8_t *data) {
return (uint16_t)data[0] | ((uint16_t)data[1] << 8);
}
static uint32_t amduat_asl_snapshot_load_u32_le(const uint8_t *data) {
return (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
}
static uint64_t amduat_asl_snapshot_load_u64_le(const uint8_t *data) {
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
}
static bool amduat_asl_snapshot_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static int amduat_asl_snapshot_compare_segments(const void *lhs,
const void *rhs) {
const amduat_asl_segment_seal_t *left =
*(const amduat_asl_segment_seal_t *const *)lhs;
const amduat_asl_segment_seal_t *right =
*(const amduat_asl_segment_seal_t *const *)rhs;
if (left->segment_id < right->segment_id) {
return -1;
}
if (left->segment_id > right->segment_id) {
return 1;
}
return memcmp(left->segment_hash, right->segment_hash,
sizeof(left->segment_hash));
}
static int amduat_asl_snapshot_compare_tombstones(const void *lhs,
const void *rhs) {
const amduat_asl_tombstone_entry_t *left =
*(const amduat_asl_tombstone_entry_t *const *)lhs;
const amduat_asl_tombstone_entry_t *right =
*(const amduat_asl_tombstone_entry_t *const *)rhs;
size_t min_len;
int cmp;
if (left->ref.hash_id < right->ref.hash_id) {
return -1;
}
if (left->ref.hash_id > right->ref.hash_id) {
return 1;
}
if (left->ref.digest.len < right->ref.digest.len) {
return -1;
}
if (left->ref.digest.len > right->ref.digest.len) {
return 1;
}
min_len = left->ref.digest.len;
if (min_len != 0u) {
cmp = memcmp(left->ref.digest.data, right->ref.digest.data, min_len);
if (cmp != 0) {
return cmp;
}
}
if (left->tombstone_logseq < right->tombstone_logseq) {
return -1;
}
if (left->tombstone_logseq > right->tombstone_logseq) {
return 1;
}
return 0;
}
static bool amduat_asl_snapshot_hash_manifest(const uint8_t *bytes,
size_t len,
uint8_t out_hash[32]) {
if (out_hash == NULL) {
return true;
}
return amduat_hash_asl1_digest(AMDUAT_HASH_ASL1_ID_SHA256,
amduat_octets(bytes, len), out_hash, 32);
}
void amduat_asl_snapshot_manifest_free(
amduat_asl_snapshot_manifest_t *manifest) {
size_t i;
if (manifest == NULL) {
return;
}
free(manifest->segments);
manifest->segments = NULL;
manifest->segments_len = 0;
if (manifest->tombstones != NULL) {
for (i = 0; i < manifest->tombstones_len; ++i) {
amduat_reference_free(&manifest->tombstones[i].ref);
}
}
free(manifest->tombstones);
manifest->tombstones = NULL;
manifest->tombstones_len = 0;
}
bool amduat_asl_snapshot_manifest_write(
const char *path,
const amduat_asl_snapshot_manifest_t *manifest,
uint8_t out_hash[32]) {
size_t i;
size_t total_len;
size_t offset;
uint8_t *buffer;
amduat_asl_segment_seal_t **segment_order;
amduat_asl_tombstone_entry_t **tombstone_order;
if (path == NULL || manifest == NULL) {
return false;
}
if (manifest->segments_len != 0u && manifest->segments == NULL) {
return false;
}
if (manifest->tombstones_len != 0u && manifest->tombstones == NULL) {
return false;
}
if (manifest->segments_len > SIZE_MAX / AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
return false;
}
total_len = 0;
if (!amduat_asl_snapshot_add_size(&total_len,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE)) {
return false;
}
if (!amduat_asl_snapshot_add_size(
&total_len,
manifest->segments_len *
AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE)) {
return false;
}
for (i = 0; i < manifest->tombstones_len; ++i) {
const amduat_asl_tombstone_entry_t *entry = &manifest->tombstones[i];
if (entry->ref.digest.len == 0 ||
(entry->ref.digest.len != 0u && entry->ref.digest.data == NULL)) {
return false;
}
if (entry->ref.digest.len > UINT16_MAX) {
return false;
}
if (!amduat_asl_snapshot_add_size(&total_len,
16u + entry->ref.digest.len)) {
return false;
}
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
segment_order = NULL;
tombstone_order = NULL;
if (manifest->segments_len != 0u) {
segment_order = (amduat_asl_segment_seal_t **)calloc(
manifest->segments_len, sizeof(*segment_order));
if (segment_order == NULL) {
free(buffer);
return false;
}
for (i = 0; i < manifest->segments_len; ++i) {
segment_order[i] = &manifest->segments[i];
}
qsort(segment_order, manifest->segments_len, sizeof(*segment_order),
amduat_asl_snapshot_compare_segments);
}
if (manifest->tombstones_len != 0u) {
tombstone_order = (amduat_asl_tombstone_entry_t **)calloc(
manifest->tombstones_len, sizeof(*tombstone_order));
if (tombstone_order == NULL) {
free(segment_order);
free(buffer);
return false;
}
for (i = 0; i < manifest->tombstones_len; ++i) {
tombstone_order[i] = &manifest->tombstones[i];
}
qsort(tombstone_order, manifest->tombstones_len,
sizeof(*tombstone_order), amduat_asl_snapshot_compare_tombstones);
}
offset = 0;
memcpy(buffer + offset, k_amduat_asl_snapshot_manifest_magic,
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN);
offset += AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN;
amduat_asl_snapshot_store_u16_le(buffer + offset,
AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION);
offset += 2;
amduat_asl_snapshot_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_snapshot_store_u32_le(buffer + offset,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE);
offset += 4;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->snapshot_id);
offset += 8;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->anchor_logseq);
offset += 8;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->segments_len);
offset += 8;
amduat_asl_snapshot_store_u64_le(buffer + offset, manifest->tombstones_len);
offset += 8;
amduat_asl_snapshot_store_u32_le(buffer + offset,
manifest->config.encoding_profile_id);
offset += 4;
amduat_asl_snapshot_store_u32_le(buffer + offset,
manifest->config.hash_id);
offset += 4;
memset(buffer + offset, 0,
AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE - offset);
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE;
for (i = 0; i < manifest->segments_len; ++i) {
const amduat_asl_segment_seal_t *entry = segment_order[i];
amduat_asl_snapshot_store_u64_le(buffer + offset, entry->segment_id);
offset += 8;
memcpy(buffer + offset, entry->segment_hash, sizeof(entry->segment_hash));
offset += sizeof(entry->segment_hash);
}
for (i = 0; i < manifest->tombstones_len; ++i) {
const amduat_asl_tombstone_entry_t *entry = tombstone_order[i];
amduat_asl_snapshot_store_u32_le(buffer + offset, entry->ref.hash_id);
offset += 4;
amduat_asl_snapshot_store_u16_le(buffer + offset,
(uint16_t)entry->ref.digest.len);
offset += 2;
amduat_asl_snapshot_store_u16_le(buffer + offset, 0);
offset += 2;
memcpy(buffer + offset, entry->ref.digest.data, entry->ref.digest.len);
offset += entry->ref.digest.len;
amduat_asl_snapshot_store_u64_le(buffer + offset,
entry->tombstone_logseq);
offset += 8;
}
free(segment_order);
free(tombstone_order);
if (offset != total_len) {
free(buffer);
return false;
}
if (!amduat_asl_snapshot_hash_manifest(buffer, total_len, out_hash)) {
free(buffer);
return false;
}
if (!amduat_asl_write_path(path, buffer, total_len)) {
free(buffer);
return false;
}
free(buffer);
return true;
}
bool amduat_asl_snapshot_manifest_read(
const char *path,
amduat_asl_snapshot_manifest_t *out_manifest,
uint8_t out_hash[32]) {
uint8_t *buffer;
size_t len;
size_t offset;
uint16_t version;
uint16_t reserved0;
uint32_t header_size;
uint64_t segment_count;
uint64_t tombstone_count;
size_t i;
if (path == NULL || out_manifest == NULL) {
return false;
}
memset(out_manifest, 0, sizeof(*out_manifest));
buffer = NULL;
len = 0u;
if (!amduat_asl_read_path(path, &buffer, &len)) {
return false;
}
if (len < AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE) {
free(buffer);
return false;
}
if (!amduat_asl_snapshot_hash_manifest(buffer, len, out_hash)) {
free(buffer);
return false;
}
if (memcmp(buffer, k_amduat_asl_snapshot_manifest_magic,
AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN) != 0) {
free(buffer);
return false;
}
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_MAGIC_LEN;
version = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
reserved0 = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
header_size = amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
if (version != AMDUAT_ASL_SNAPSHOT_MANIFEST_VERSION || reserved0 != 0 ||
header_size != AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE) {
free(buffer);
return false;
}
out_manifest->snapshot_id = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
out_manifest->anchor_logseq =
amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
segment_count = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
tombstone_count = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
out_manifest->config.encoding_profile_id =
amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
out_manifest->config.hash_id =
amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
offset = AMDUAT_ASL_SNAPSHOT_MANIFEST_HEADER_SIZE;
if (segment_count > SIZE_MAX || tombstone_count > SIZE_MAX) {
free(buffer);
return false;
}
out_manifest->segments_len = (size_t)segment_count;
if (out_manifest->segments_len != 0u) {
if (out_manifest->segments_len >
SIZE_MAX / AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
free(buffer);
return false;
}
out_manifest->segments = (amduat_asl_segment_seal_t *)calloc(
out_manifest->segments_len, sizeof(*out_manifest->segments));
if (out_manifest->segments == NULL) {
free(buffer);
return false;
}
}
for (i = 0; i < out_manifest->segments_len; ++i) {
amduat_asl_segment_seal_t *entry = &out_manifest->segments[i];
if (len - offset < AMDUAT_ASL_SNAPSHOT_MANIFEST_SEGMENT_SIZE) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
entry->segment_id = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
memcpy(entry->segment_hash, buffer + offset, sizeof(entry->segment_hash));
offset += sizeof(entry->segment_hash);
}
out_manifest->tombstones_len = (size_t)tombstone_count;
if (out_manifest->tombstones_len != 0u) {
out_manifest->tombstones = (amduat_asl_tombstone_entry_t *)calloc(
out_manifest->tombstones_len, sizeof(*out_manifest->tombstones));
if (out_manifest->tombstones == NULL) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
}
for (i = 0; i < out_manifest->tombstones_len; ++i) {
amduat_asl_tombstone_entry_t *entry = &out_manifest->tombstones[i];
uint32_t hash_id;
uint16_t digest_len;
uint16_t tombstone_reserved;
amduat_reference_t ref;
uint64_t tombstone_logseq;
if (len - offset < 16) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
hash_id = amduat_asl_snapshot_load_u32_le(buffer + offset);
offset += 4;
if (hash_id > UINT16_MAX) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
digest_len = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
tombstone_reserved = amduat_asl_snapshot_load_u16_le(buffer + offset);
offset += 2;
if (tombstone_reserved != 0 || digest_len == 0) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
if (len - offset < (size_t)digest_len + 8u) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
ref = amduat_reference((amduat_hash_id_t)hash_id,
amduat_octets(buffer + offset, digest_len));
offset += digest_len;
tombstone_logseq = amduat_asl_snapshot_load_u64_le(buffer + offset);
offset += 8;
if (!amduat_reference_clone(ref, &entry->ref)) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
entry->tombstone_logseq = tombstone_logseq;
}
if (offset != len) {
amduat_asl_snapshot_manifest_free(out_manifest);
free(buffer);
return false;
}
free(buffer);
return true;
}

101
src/near_core/asl/log_cursor.c
View file

@ -1,101 +0,0 @@
#include "amduat/asl/log_cursor.h"
#include "amduat/enc/asl_log.h"
#include <stdlib.h>
#include <string.h>
static bool amduat_asl_log_cursor_copy_record(
const amduat_asl_log_record_t *src,
amduat_asl_log_record_t *out) {
if (src == NULL || out == NULL) {
return false;
}
*out = *src;
if (src->payload.len == 0u) {
out->payload = amduat_octets(NULL, 0u);
return true;
}
if (src->payload.data == NULL) {
return false;
}
{
uint8_t *buf = (uint8_t *)malloc(src->payload.len);
if (buf == NULL) {
return false;
}
memcpy(buf, src->payload.data, src->payload.len);
out->payload = amduat_octets(buf, src->payload.len);
}
return true;
}
bool amduat_asl_log_read_since(amduat_asl_store_t *store,
amduat_asl_log_position_t after_logseq,
size_t limit,
amduat_asl_log_record_t **out_records,
size_t *out_count,
amduat_asl_log_position_t *out_next_logseq,
bool *out_end) {
amduat_asl_log_record_t *records = NULL;
size_t record_count = 0u;
amduat_asl_store_error_t err;
size_t start = 0u;
size_t available;
size_t take;
if (out_records == NULL || out_count == NULL || out_next_logseq == NULL ||
out_end == NULL) {
return false;
}
*out_records = NULL;
*out_count = 0u;
*out_next_logseq = after_logseq;
*out_end = true;
if (store == NULL) {
return false;
}
err = amduat_asl_log_scan(store, &records, &record_count);
if (err != AMDUAT_ASL_STORE_OK) {
return false;
}
while (start < record_count && records[start].logseq <= after_logseq) {
start++;
}
available = record_count - start;
take = (limit == 0u || limit > available) ? available : limit;
if (take == 0u) {
*out_end = (start >= record_count);
amduat_enc_asl_log_free(records, record_count);
return true;
}
*out_records = (amduat_asl_log_record_t *)calloc(take, sizeof(**out_records));
if (*out_records == NULL) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
for (size_t i = 0u; i < take; ++i) {
if (!amduat_asl_log_cursor_copy_record(&records[start + i],
&(*out_records)[i])) {
amduat_enc_asl_log_free(*out_records, i);
free(*out_records);
*out_records = NULL;
*out_count = 0u;
amduat_enc_asl_log_free(records, record_count);
return false;
}
}
*out_count = take;
*out_next_logseq = (*out_records)[take - 1u].logseq;
*out_end = (start + take >= record_count);
amduat_enc_asl_log_free(records, record_count);
return true;
}

110
src/near_core/asl/none.c
View file

@ -1,110 +0,0 @@
#include "amduat/asl/none.h"
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_NONE_MAGIC_LEN = 8,
AMDUAT_ASL_NONE_VERSION = 1
};
static const uint8_t k_amduat_asl_none_magic[AMDUAT_ASL_NONE_MAGIC_LEN] = {
'A', 'S', 'L', 'N', 'O', 'N', '1', '\0'
};
static void amduat_asl_none_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static bool amduat_asl_none_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
bool amduat_asl_none_encode_v1(amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t offset = 0u;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
buffer = (uint8_t *)malloc(AMDUAT_ASL_NONE_MAGIC_LEN + 4u);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_none_magic, AMDUAT_ASL_NONE_MAGIC_LEN);
offset += AMDUAT_ASL_NONE_MAGIC_LEN;
amduat_asl_none_store_u32_le(buffer + offset, AMDUAT_ASL_NONE_VERSION);
offset += 4u;
out_bytes->data = buffer;
out_bytes->len = offset;
return true;
}
bool amduat_asl_none_decode_v1(amduat_octets_t bytes) {
size_t offset = 0u;
uint32_t version = 0u;
if (bytes.len != AMDUAT_ASL_NONE_MAGIC_LEN + 4u) {
return false;
}
if (bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, k_amduat_asl_none_magic,
AMDUAT_ASL_NONE_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_NONE_MAGIC_LEN;
if (!amduat_asl_none_read_u32_le(bytes.data, bytes.len, &offset, &version)) {
return false;
}
if (version != AMDUAT_ASL_NONE_VERSION) {
return false;
}
return offset == bytes.len;
}
bool amduat_asl_none_is_artifact(const amduat_artifact_t *artifact) {
if (artifact == NULL) {
return false;
}
if (!artifact->has_type_tag ||
artifact->type_tag.tag_id != AMDUAT_TYPE_TAG_ASL_NONE_1) {
return false;
}
return amduat_asl_none_decode_v1(artifact->bytes);
}
bool amduat_asl_none_artifact(amduat_artifact_t *out_artifact) {
amduat_octets_t bytes = amduat_octets(NULL, 0u);
if (out_artifact == NULL) {
return false;
}
if (!amduat_asl_none_encode_v1(&bytes)) {
return false;
}
*out_artifact =
amduat_artifact_with_type(bytes, amduat_type_tag(AMDUAT_TYPE_TAG_ASL_NONE_1));
return true;
}

472
src/near_core/asl/projection_checkpoint.c
View file

@ -1,472 +0,0 @@
#include "amduat/asl/projection_checkpoint.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN = 8,
AMDUAT_ASL_PROJECTION_CHECKPOINT_VERSION = 1,
AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_SNAPSHOT_ID = 1u << 0u,
AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_ARTIFACT_COUNT = 1u << 1u,
AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_EDGE_COUNT = 1u << 2u,
AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_ERROR_COUNT = 1u << 3u,
AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_LAG_MILLIS = 1u << 4u,
AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_NOTE = 1u << 5u
};
static const uint8_t k_amduat_asl_projection_checkpoint_magic[
AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN] = {
'A', 'S', 'L', 'C', 'K', 'P', '1', '\0'
};
static void amduat_asl_checkpoint_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_checkpoint_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_checkpoint_read_u32_le(const uint8_t *data,
size_t len,
size_t *offset,
uint32_t *out) {
if (len - *offset < 4u) {
return false;
}
*out = (uint32_t)data[*offset] |
((uint32_t)data[*offset + 1u] << 8) |
((uint32_t)data[*offset + 2u] << 16) |
((uint32_t)data[*offset + 3u] << 24);
*offset += 4u;
return true;
}
static bool amduat_asl_checkpoint_read_u64_le(const uint8_t *data,
size_t len,
size_t *offset,
uint64_t *out) {
if (len - *offset < 8u) {
return false;
}
*out = (uint64_t)data[*offset] |
((uint64_t)data[*offset + 1u] << 8) |
((uint64_t)data[*offset + 2u] << 16) |
((uint64_t)data[*offset + 3u] << 24) |
((uint64_t)data[*offset + 4u] << 32) |
((uint64_t)data[*offset + 5u] << 40) |
((uint64_t)data[*offset + 6u] << 48) |
((uint64_t)data[*offset + 7u] << 56);
*offset += 8u;
return true;
}
static bool amduat_asl_checkpoint_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_checkpoint_copy_octets(amduat_octets_t src,
amduat_octets_t *out) {
if (out == NULL) {
return false;
}
*out = amduat_octets(NULL, 0u);
if (src.len == 0u) {
return true;
}
if (src.data == NULL) {
return false;
}
{
uint8_t *buf = (uint8_t *)malloc(src.len);
if (buf == NULL) {
return false;
}
memcpy(buf, src.data, src.len);
*out = amduat_octets(buf, src.len);
}
return true;
}
bool amduat_asl_projection_checkpoint_encode_v1(
const amduat_asl_projection_checkpoint_v1_t *checkpoint,
amduat_octets_t *out_bytes) {
uint8_t *buffer;
size_t total_len = 0u;
size_t offset = 0u;
uint32_t flags = 0u;
if (checkpoint == NULL || out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (checkpoint->checkpoint_version != AMDUAT_ASL_PROJECTION_CHECKPOINT_VERSION) {
return false;
}
if (checkpoint->projection_name.len == 0u ||
checkpoint->projection_name.data == NULL) {
return false;
}
if (checkpoint->builder_id.len == 0u ||
checkpoint->builder_id.data == NULL) {
return false;
}
if (checkpoint->projection_name.len > UINT32_MAX ||
checkpoint->builder_id.len > UINT32_MAX ||
checkpoint->note.len > UINT32_MAX) {
return false;
}
if (checkpoint->has_snapshot_id) {
flags |= AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_SNAPSHOT_ID;
}
if (checkpoint->has_artifact_count) {
flags |= AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_ARTIFACT_COUNT;
}
if (checkpoint->has_edge_count) {
flags |= AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_EDGE_COUNT;
}
if (checkpoint->has_error_count) {
flags |= AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_ERROR_COUNT;
}
if (checkpoint->has_lag_millis) {
flags |= AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_LAG_MILLIS;
}
if (checkpoint->has_note) {
flags |= AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_NOTE;
}
if (!amduat_asl_checkpoint_add_size(
&total_len,
AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN + 4u + 4u)) {
return false;
}
if (!amduat_asl_checkpoint_add_size(&total_len,
4u + checkpoint->projection_name.len)) {
return false;
}
if (!amduat_asl_checkpoint_add_size(&total_len, 4u)) {
return false;
}
if (checkpoint->has_snapshot_id) {
if (!amduat_asl_checkpoint_add_size(&total_len, 8u)) {
return false;
}
}
if (!amduat_asl_checkpoint_add_size(&total_len, 8u + 8u)) {
return false;
}
if (!amduat_asl_checkpoint_add_size(&total_len,
4u + checkpoint->builder_id.len)) {
return false;
}
if (checkpoint->has_artifact_count &&
!amduat_asl_checkpoint_add_size(&total_len, 8u)) {
return false;
}
if (checkpoint->has_edge_count &&
!amduat_asl_checkpoint_add_size(&total_len, 8u)) {
return false;
}
if (checkpoint->has_error_count &&
!amduat_asl_checkpoint_add_size(&total_len, 8u)) {
return false;
}
if (checkpoint->has_lag_millis &&
!amduat_asl_checkpoint_add_size(&total_len, 8u)) {
return false;
}
if (checkpoint->has_note &&
!amduat_asl_checkpoint_add_size(&total_len, 4u + checkpoint->note.len)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
memcpy(buffer + offset, k_amduat_asl_projection_checkpoint_magic,
AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN);
offset += AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN;
amduat_asl_checkpoint_store_u32_le(buffer + offset,
AMDUAT_ASL_PROJECTION_CHECKPOINT_VERSION);
offset += 4u;
amduat_asl_checkpoint_store_u32_le(buffer + offset, flags);
offset += 4u;
amduat_asl_checkpoint_store_u32_le(buffer + offset,
(uint32_t)checkpoint->projection_name.len);
offset += 4u;
memcpy(buffer + offset,
checkpoint->projection_name.data,
checkpoint->projection_name.len);
offset += checkpoint->projection_name.len;
amduat_asl_checkpoint_store_u32_le(buffer + offset,
checkpoint->projection_schema_version);
offset += 4u;
if (checkpoint->has_snapshot_id) {
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->snapshot_id);
offset += 8u;
}
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->log_position);
offset += 8u;
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->created_at);
offset += 8u;
amduat_asl_checkpoint_store_u32_le(buffer + offset,
(uint32_t)checkpoint->builder_id.len);
offset += 4u;
memcpy(buffer + offset, checkpoint->builder_id.data,
checkpoint->builder_id.len);
offset += checkpoint->builder_id.len;
if (checkpoint->has_artifact_count) {
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->artifact_count);
offset += 8u;
}
if (checkpoint->has_edge_count) {
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->edge_count);
offset += 8u;
}
if (checkpoint->has_error_count) {
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->error_count);
offset += 8u;
}
if (checkpoint->has_lag_millis) {
amduat_asl_checkpoint_store_u64_le(buffer + offset,
checkpoint->lag_millis);
offset += 8u;
}
if (checkpoint->has_note) {
amduat_asl_checkpoint_store_u32_le(buffer + offset,
(uint32_t)checkpoint->note.len);
offset += 4u;
if (checkpoint->note.len != 0u) {
memcpy(buffer + offset, checkpoint->note.data, checkpoint->note.len);
offset += checkpoint->note.len;
}
}
if (offset != total_len) {
free(buffer);
return false;
}
*out_bytes = amduat_octets(buffer, total_len);
return true;
}
bool amduat_asl_projection_checkpoint_decode_v1(
amduat_octets_t bytes,
amduat_asl_projection_checkpoint_v1_t *out_checkpoint) {
size_t offset = 0u;
uint32_t version;
uint32_t flags;
uint32_t name_len;
uint32_t builder_len;
uint32_t note_len;
amduat_asl_projection_checkpoint_v1_t checkpoint;
if (out_checkpoint == NULL) {
return false;
}
memset(&checkpoint, 0, sizeof(checkpoint));
checkpoint.projection_name = amduat_octets(NULL, 0u);
checkpoint.builder_id = amduat_octets(NULL, 0u);
checkpoint.note = amduat_octets(NULL, 0u);
if (bytes.data == NULL || bytes.len < AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN + 8u) {
return false;
}
if (memcmp(bytes.data,
k_amduat_asl_projection_checkpoint_magic,
AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN) != 0) {
return false;
}
offset += AMDUAT_ASL_PROJECTION_CHECKPOINT_MAGIC_LEN;
if (!amduat_asl_checkpoint_read_u32_le(bytes.data, bytes.len, &offset, &version) ||
version != AMDUAT_ASL_PROJECTION_CHECKPOINT_VERSION) {
return false;
}
checkpoint.checkpoint_version = version;
if (!amduat_asl_checkpoint_read_u32_le(bytes.data, bytes.len, &offset, &flags)) {
return false;
}
if (!amduat_asl_checkpoint_read_u32_le(bytes.data, bytes.len, &offset, &name_len)) {
return false;
}
if (bytes.len - offset < name_len) {
return false;
}
if (name_len != 0u) {
checkpoint.projection_name = amduat_octets(NULL, 0u);
checkpoint.projection_name.len = name_len;
if (!amduat_asl_checkpoint_copy_octets(
amduat_octets(bytes.data + offset, name_len),
&checkpoint.projection_name)) {
return false;
}
}
offset += name_len;
if (!amduat_asl_checkpoint_read_u32_le(bytes.data, bytes.len, &offset,
&checkpoint.projection_schema_version)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
checkpoint.has_snapshot_id =
(flags & AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_SNAPSHOT_ID) != 0u;
if (checkpoint.has_snapshot_id) {
if (!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.snapshot_id)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
if (!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.log_position) ||
!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.created_at)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
if (!amduat_asl_checkpoint_read_u32_le(bytes.data, bytes.len, &offset,
&builder_len)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
if (bytes.len - offset < builder_len) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
if (builder_len != 0u) {
checkpoint.builder_id = amduat_octets(NULL, 0u);
checkpoint.builder_id.len = builder_len;
if (!amduat_asl_checkpoint_copy_octets(
amduat_octets(bytes.data + offset, builder_len),
&checkpoint.builder_id)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
offset += builder_len;
checkpoint.has_artifact_count =
(flags & AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_ARTIFACT_COUNT) != 0u;
if (checkpoint.has_artifact_count) {
if (!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.artifact_count)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
checkpoint.has_edge_count =
(flags & AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_EDGE_COUNT) != 0u;
if (checkpoint.has_edge_count) {
if (!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.edge_count)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
checkpoint.has_error_count =
(flags & AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_ERROR_COUNT) != 0u;
if (checkpoint.has_error_count) {
if (!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.error_count)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
checkpoint.has_lag_millis =
(flags & AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_LAG_MILLIS) != 0u;
if (checkpoint.has_lag_millis) {
if (!amduat_asl_checkpoint_read_u64_le(bytes.data, bytes.len, &offset,
&checkpoint.lag_millis)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
checkpoint.has_note =
(flags & AMDUAT_ASL_PROJECTION_CHECKPOINT_FLAG_NOTE) != 0u;
if (checkpoint.has_note) {
if (!amduat_asl_checkpoint_read_u32_le(bytes.data, bytes.len, &offset,
&note_len)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
if (bytes.len - offset < note_len) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
if (note_len != 0u) {
checkpoint.note = amduat_octets(NULL, 0u);
checkpoint.note.len = note_len;
if (!amduat_asl_checkpoint_copy_octets(
amduat_octets(bytes.data + offset, note_len),
&checkpoint.note)) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
}
offset += note_len;
}
if (offset != bytes.len) {
amduat_asl_projection_checkpoint_free(&checkpoint);
return false;
}
*out_checkpoint = checkpoint;
return true;
}
void amduat_asl_projection_checkpoint_free(
amduat_asl_projection_checkpoint_v1_t *checkpoint) {
if (checkpoint == NULL) {
return;
}
amduat_octets_free(&checkpoint->projection_name);
amduat_octets_free(&checkpoint->builder_id);
amduat_octets_free(&checkpoint->note);
checkpoint->projection_name = amduat_octets(NULL, 0u);
checkpoint->builder_id = amduat_octets(NULL, 0u);
checkpoint->note = amduat_octets(NULL, 0u);
}

4
src/near_core/asl/ref_derive.c
View file

@ -1,6 +1,5 @@
#include "amduat/asl/ref_derive.h"
#include "amduat/asl/identity.h"
#include "amduat/enc/asl1_core.h"
#include "amduat/enc/asl1_core_codec.h"
#include "amduat/hash/asl1.h"
@ -54,8 +53,7 @@ bool amduat_asl_ref_derive(amduat_artifact_t artifact,
return false;
}
if (!amduat_hash_asl1_digest_domain(hash_id,
amduat_asl_identity_domain_separator(),
if (!amduat_hash_asl1_digest(hash_id,
artifact_bytes,
digest,
hash_desc->digest_len)) {

119
src/near_core/asl/store.c
View file

@ -60,122 +60,3 @@ amduat_asl_store_error_t amduat_asl_store_get(amduat_asl_store_t *store,
}
return store->ops.get(store->ctx, ref, out_artifact);
}
amduat_asl_store_error_t amduat_asl_store_put_indexed(
amduat_asl_store_t *store,
amduat_artifact_t artifact,
amduat_reference_t *out_ref,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
amduat_asl_store_error_t store_err;
if (store == NULL || store->ops.put_indexed == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
store_err = store->ops.put_indexed(store->ctx, artifact, out_ref, out_state);
if (store_err != AMDUAT_ASL_STORE_OK) {
return store_err;
}
if (out_ref != NULL && out_ref->hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_INTEGRITY;
}
return AMDUAT_ASL_STORE_OK;
}
amduat_asl_store_error_t amduat_asl_store_get_indexed(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_index_state_t state,
amduat_artifact_t *out_artifact) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.get_indexed == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
if (ref.hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.get_indexed(store->ctx, ref, state, out_artifact);
}
amduat_asl_store_error_t amduat_asl_store_tombstone(
amduat_asl_store_t *store,
amduat_reference_t ref,
uint32_t scope,
uint32_t reason_code,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.tombstone == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
if (ref.hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.tombstone(store->ctx, ref, scope, reason_code, out_state);
}
amduat_asl_store_error_t amduat_asl_store_tombstone_lift(
amduat_asl_store_t *store,
amduat_reference_t ref,
amduat_asl_log_position_t tombstone_logseq,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.tombstone_lift == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
if (ref.hash_id != store->config.hash_id) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.tombstone_lift(store->ctx,
ref,
tombstone_logseq,
out_state);
}
amduat_asl_store_error_t amduat_asl_log_scan(
amduat_asl_store_t *store,
amduat_asl_log_record_t **out_records,
size_t *out_count) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.log_scan == NULL) {
return AMDUAT_ASL_STORE_ERR_UNSUPPORTED;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return cfg_err;
}
return store->ops.log_scan(store->ctx, out_records, out_count);
}
bool amduat_asl_index_current_state(amduat_asl_store_t *store,
amduat_asl_index_state_t *out_state) {
amduat_asl_store_error_t cfg_err;
if (store == NULL || store->ops.current_state == NULL) {
return false;
}
cfg_err = amduat_asl_store_validate_config(store);
if (cfg_err != AMDUAT_ASL_STORE_OK) {
return false;
}
return store->ops.current_state(store->ctx, out_state);
}

235
src/near_core/bytestore.c
View file

@ -1,235 +0,0 @@
#include "amduat/bytestore.h"
#include <stdlib.h>
#include <string.h>
static const char k_amduat_bytestore_domain_sep[] = "CAS:OBJ\0";
static bool amduat_bytestore_trim_trailing_slash(const char *input,
size_t *out_len) {
size_t len;
if (input == NULL || out_len == NULL) {
return false;
}
len = strlen(input);
while (len > 1u && input[len - 1u] == '/') {
len--;
}
*out_len = len;
return true;
}
static bool amduat_bytestore_path_is_prefix(const char *parent,
size_t parent_len,
const char *child,
size_t child_len) {
if (parent_len == 0u || child_len == 0u) {
return false;
}
if (child_len < parent_len) {
return false;
}
if (memcmp(parent, child, parent_len) != 0) {
return false;
}
if (child_len == parent_len) {
return true;
}
return child[parent_len] == '/';
}
void amduat_bytestore_init(amduat_bytestore_t *store,
amduat_bytestore_config_t config,
amduat_bytestore_ops_t ops,
void *ctx) {
if (store == NULL) {
return;
}
store->config = config;
store->ops = ops;
store->ctx = ctx;
}
amduat_bytestore_error_t amduat_bytestore_put(amduat_bytestore_t *store,
amduat_octets_t payload,
amduat_reference_t *out_cid) {
if (store == NULL || store->ops.put == NULL) {
return AMDUAT_BYTESTORE_ERR_UNSUPPORTED;
}
return store->ops.put(store->ctx, payload, out_cid);
}
amduat_bytestore_error_t amduat_bytestore_put_stream(
amduat_bytestore_t *store,
amduat_bytestore_stream_t stream,
amduat_reference_t *out_cid) {
if (store == NULL || store->ops.put_stream == NULL) {
return AMDUAT_BYTESTORE_ERR_UNSUPPORTED;
}
return store->ops.put_stream(store->ctx, stream, out_cid);
}
amduat_bytestore_error_t amduat_bytestore_import_cor(
amduat_bytestore_t *store,
amduat_octets_t envelope,
amduat_reference_t *out_cid) {
if (store == NULL || store->ops.import_cor == NULL) {
return AMDUAT_BYTESTORE_ERR_UNSUPPORTED;
}
return store->ops.import_cor(store->ctx, envelope, out_cid);
}
amduat_bytestore_error_t amduat_bytestore_export_cor(
amduat_bytestore_t *store,
amduat_reference_t cid,
amduat_octets_t *out_envelope) {
if (store == NULL || store->ops.export_cor == NULL) {
return AMDUAT_BYTESTORE_ERR_UNSUPPORTED;
}
return store->ops.export_cor(store->ctx, cid, out_envelope);
}
amduat_bytestore_error_t amduat_bytestore_get(amduat_bytestore_t *store,
amduat_reference_t cid,
amduat_octets_t *out_bytes) {
if (store == NULL || store->ops.get == NULL) {
return AMDUAT_BYTESTORE_ERR_UNSUPPORTED;
}
return store->ops.get(store->ctx, cid, out_bytes);
}
amduat_bytestore_error_t amduat_bytestore_stat(
amduat_bytestore_t *store,
amduat_reference_t cid,
amduat_bytestore_stat_t *out_stat) {
if (store == NULL || store->ops.stat == NULL) {
return AMDUAT_BYTESTORE_ERR_UNSUPPORTED;
}
return store->ops.stat(store->ctx, cid, out_stat);
}
amduat_bytestore_error_t amduat_bytestore_assert_area_isolation(
amduat_bytestore_t *store,
const char *public_root,
const char *secure_root) {
size_t public_len;
size_t secure_len;
if (store != NULL && store->ops.assert_area_isolation != NULL) {
return store->ops.assert_area_isolation(store->ctx, public_root,
secure_root);
}
if (!amduat_bytestore_trim_trailing_slash(public_root, &public_len) ||
!amduat_bytestore_trim_trailing_slash(secure_root, &secure_len)) {
return AMDUAT_BYTESTORE_ERR_INVALID;
}
if (public_len == 0u || secure_len == 0u) {
return AMDUAT_BYTESTORE_ERR_INVALID;
}
if (public_len == secure_len &&
memcmp(public_root, secure_root, public_len) == 0) {
return AMDUAT_BYTESTORE_ERR_AREA_VIOLATION;
}
if (amduat_bytestore_path_is_prefix(public_root, public_len, secure_root,
secure_len)) {
return AMDUAT_BYTESTORE_ERR_AREA_VIOLATION;
}
if (amduat_bytestore_path_is_prefix(secure_root, secure_len, public_root,
public_len)) {
return AMDUAT_BYTESTORE_ERR_AREA_VIOLATION;
}
return AMDUAT_BYTESTORE_OK;
}
bool amduat_bytestore_cid_for_payload(amduat_hash_id_t hash_id,
amduat_octets_t payload,
amduat_reference_t *out_cid) {
const amduat_hash_asl1_desc_t *desc;
uint8_t *digest;
amduat_octets_t domain_sep;
if (out_cid == NULL) {
return false;
}
*out_cid = amduat_reference(0, amduat_octets(NULL, 0u));
desc = amduat_hash_asl1_desc_lookup(hash_id);
if (desc == NULL || desc->digest_len == 0u) {
return false;
}
digest = (uint8_t *)malloc(desc->digest_len);
if (digest == NULL) {
return false;
}
domain_sep = amduat_octets(k_amduat_bytestore_domain_sep,
sizeof(k_amduat_bytestore_domain_sep) - 1u);
if (!amduat_hash_asl1_digest_domain(hash_id, domain_sep, payload, digest,
desc->digest_len)) {
free(digest);
return false;
}
*out_cid = amduat_reference(hash_id, amduat_octets(digest, desc->digest_len));
return true;
}
bool amduat_bytestore_cid_for_stream(amduat_hash_id_t hash_id,
amduat_bytestore_stream_t stream,
amduat_reference_t *out_cid) {
amduat_hash_asl1_stream_t hasher;
const amduat_hash_asl1_desc_t *desc;
uint8_t *digest;
amduat_octets_t domain_sep;
amduat_octets_t chunk;
int next_status;
if (out_cid == NULL || stream.next == NULL) {
return false;
}
*out_cid = amduat_reference(0, amduat_octets(NULL, 0u));
desc = amduat_hash_asl1_desc_lookup(hash_id);
if (desc == NULL || desc->digest_len == 0u) {
return false;
}
if (!amduat_hash_asl1_stream_init(hash_id, &hasher)) {
return false;
}
domain_sep = amduat_octets(k_amduat_bytestore_domain_sep,
sizeof(k_amduat_bytestore_domain_sep) - 1u);
if (!amduat_hash_asl1_stream_update(&hasher, domain_sep)) {
amduat_hash_asl1_stream_destroy(&hasher);
return false;
}
while (true) {
chunk = amduat_octets(NULL, 0u);
next_status = stream.next(stream.ctx, &chunk);
if (next_status == 0) {
break;
}
if (next_status < 0) {
amduat_hash_asl1_stream_destroy(&hasher);
return false;
}
if (chunk.len != 0u && chunk.data == NULL) {
amduat_hash_asl1_stream_destroy(&hasher);
return false;
}
if (chunk.len != 0u &&
!amduat_hash_asl1_stream_update(&hasher, chunk)) {
amduat_hash_asl1_stream_destroy(&hasher);
return false;
}
}
digest = (uint8_t *)malloc(desc->digest_len);
if (digest == NULL) {
amduat_hash_asl1_stream_destroy(&hasher);
return false;
}
if (!amduat_hash_asl1_stream_final(&hasher, digest, desc->digest_len)) {
free(digest);
amduat_hash_asl1_stream_destroy(&hasher);
return false;
}
amduat_hash_asl1_stream_destroy(&hasher);
*out_cid = amduat_reference(hash_id, amduat_octets(digest, desc->digest_len));
return true;
}

2
src/near_core/enc/asl1_core.c
View file

@ -1,7 +1,7 @@
#include "amduat/enc/asl1_core.h"
static amduat_enc_asl1_core_profile_desc_t g_enc_asl1_core_profiles[] = {
{AMDUAT_ENC_ASL1_CORE_V1, "ASL_ENC_CORE_V1", 0x0106, NULL}};
{AMDUAT_ENC_ASL1_CORE_V1, "ASL_ENC_CORE_V1", 0x0105, NULL}};
amduat_octets_t amduat_enc_asl1_core_key(
amduat_asl_encoding_profile_id_t profile_id,

846
src/near_core/enc/asl_core_index.c
View file

@ -1,846 +0,0 @@
#include "amduat/enc/asl_core_index.h"
#include "amduat/hash/asl1.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_CORE_INDEX_MAGIC = 0x33305844494c5341ull
};
static void amduat_asl_core_index_store_u16_le(uint8_t *out,
uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_core_index_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_core_index_store_u64_le(uint8_t *out,
uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static uint16_t amduat_asl_core_index_load_u16_le(const uint8_t *data) {
return (uint16_t)data[0] | ((uint16_t)data[1] << 8);
}
static uint32_t amduat_asl_core_index_load_u32_le(const uint8_t *data) {
return (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
}
static uint64_t amduat_asl_core_index_load_u64_le(const uint8_t *data) {
return (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
}
static bool amduat_asl_core_index_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_core_index_is_aligned8(uint64_t value) {
return (value & 7u) == 0u;
}
static uint64_t amduat_asl_core_index_crc64(const uint8_t *data, size_t len) {
uint64_t crc = 0u;
size_t i;
for (i = 0; i < len; ++i) {
uint64_t bit;
uint8_t value = data[i];
crc ^= ((uint64_t)value) << 56;
for (bit = 0; bit < 8; ++bit) {
if (crc & 0x8000000000000000ull) {
crc = (crc << 1) ^ 0x42f0e1eba9ea3693ull;
} else {
crc <<= 1;
}
}
}
return crc;
}
static bool amduat_asl_core_index_validate_record(
const amduat_asl_index_record_t *record,
const amduat_asl_extent_record_t *extents,
size_t extent_count,
size_t *extent_cursor,
size_t *digest_cursor,
size_t digests_len,
uint8_t *max_visibility) {
size_t i;
size_t start;
uint64_t total_len;
bool is_tombstone;
const amduat_hash_asl1_desc_t *hash_desc;
if (record->reserved0 != 0 || record->reserved1 != 0) {
return false;
}
if ((record->flags & ~AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0) {
return false;
}
if (record->visibility > 1) {
return false;
}
if (record->has_cross_domain_source > 1) {
return false;
}
if (record->has_cross_domain_source == 0 &&
record->cross_domain_source != 0) {
return false;
}
if (record->digest_len == 0) {
return false;
}
if ((record->digest_len % 8u) != 0u) {
return false;
}
if (record->hash_id <= UINT16_MAX) {
hash_desc = amduat_hash_asl1_desc_lookup(
(amduat_hash_id_t)record->hash_id);
if (hash_desc != NULL && hash_desc->digest_len != 0 &&
record->digest_len != hash_desc->digest_len) {
return false;
}
}
if ((size_t)record->digest_len > digests_len - *digest_cursor) {
return false;
}
*digest_cursor += record->digest_len;
is_tombstone = (record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
if (is_tombstone) {
if (record->extent_count != 0 || record->total_length != 0) {
return false;
}
if (record->extents_offset != 0) {
return false;
}
if (record->visibility > *max_visibility) {
*max_visibility = record->visibility;
}
return true;
}
if (record->extent_count == 0) {
return false;
}
if ((size_t)record->extent_count > extent_count - *extent_cursor) {
return false;
}
start = *extent_cursor;
total_len = 0;
for (i = 0; i < record->extent_count; ++i) {
const amduat_asl_extent_record_t *extent = &extents[start + i];
if (extent->length == 0) {
return false;
}
total_len += extent->length;
if (total_len > UINT32_MAX) {
return false;
}
}
if ((uint32_t)total_len != record->total_length) {
return false;
}
*extent_cursor += record->extent_count;
if (record->visibility > *max_visibility) {
*max_visibility = record->visibility;
}
return true;
}
bool amduat_enc_asl_core_index_encode_v1(
const amduat_asl_core_index_segment_t *segment,
amduat_octets_t *out_bytes) {
size_t i;
size_t offset;
size_t digest_cursor;
size_t extent_cursor;
size_t record_count;
size_t extent_count;
size_t total_len;
uint64_t header_size;
uint64_t bloom_offset;
uint64_t bloom_size;
uint64_t records_offset;
uint64_t records_bytes;
uint64_t digests_offset;
uint64_t digests_size;
uint64_t extents_offset;
uint64_t extents_bytes;
uint64_t footer_offset;
uint8_t *buffer;
uint8_t max_visibility;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (segment == NULL) {
return false;
}
record_count = segment->record_count;
extent_count = segment->extent_count;
if (record_count != 0 && segment->records == NULL) {
return false;
}
if (extent_count != 0 && segment->extents == NULL) {
return false;
}
if (segment->digests.len != 0 && segment->digests.data == NULL) {
return false;
}
if (segment->bloom.len != 0 && segment->bloom.data == NULL) {
return false;
}
if (segment->header.flags != 0 || segment->header.federation_version != 0 ||
segment->header.reserved0 != 0) {
return false;
}
if (segment->header.snapshot_min > segment->header.snapshot_max) {
return false;
}
if (segment->header.segment_visibility > 1) {
return false;
}
digest_cursor = 0;
extent_cursor = 0;
max_visibility = 0;
for (i = 0; i < record_count; ++i) {
if (!amduat_asl_core_index_validate_record(
&segment->records[i],
segment->extents,
extent_count,
&extent_cursor,
&digest_cursor,
segment->digests.len,
&max_visibility)) {
return false;
}
}
if (digest_cursor != segment->digests.len) {
return false;
}
if (extent_cursor != extent_count) {
return false;
}
if (segment->header.segment_visibility != max_visibility) {
return false;
}
if (segment->bloom.len != 0 && (segment->bloom.len % 8u) != 0u) {
return false;
}
if ((segment->digests.len % 8u) != 0u) {
return false;
}
if (record_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_RECORD_SIZE) {
return false;
}
if (extent_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
return false;
}
header_size = AMDUAT_ASL_CORE_INDEX_HEADER_SIZE;
bloom_size = segment->bloom.len;
if (bloom_size == 0) {
bloom_offset = 0;
} else {
bloom_offset = header_size;
}
records_offset = header_size + bloom_size;
if (!amduat_asl_core_index_is_aligned8(records_offset)) {
return false;
}
records_bytes = (uint64_t)record_count *
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
digests_offset = records_offset + records_bytes;
if (digests_offset < records_offset ||
!amduat_asl_core_index_is_aligned8(digests_offset)) {
return false;
}
digests_size = segment->digests.len;
extents_offset = digests_offset + digests_size;
if (extents_offset < digests_offset ||
!amduat_asl_core_index_is_aligned8(extents_offset)) {
return false;
}
extents_bytes = (uint64_t)extent_count *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
footer_offset = extents_offset + extents_bytes;
if (footer_offset < extents_offset) {
return false;
}
if (footer_offset > SIZE_MAX) {
return false;
}
total_len = 0;
if (!amduat_asl_core_index_add_size(&total_len, (size_t)footer_offset) ||
!amduat_asl_core_index_add_size(&total_len,
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0;
amduat_asl_core_index_store_u64_le(buffer + offset,
AMDUAT_ASL_CORE_INDEX_MAGIC);
offset += 8;
amduat_asl_core_index_store_u16_le(buffer + offset,
AMDUAT_ASL_CORE_INDEX_VERSION);
offset += 2;
amduat_asl_core_index_store_u16_le(buffer + offset,
segment->header.shard_id);
offset += 2;
amduat_asl_core_index_store_u32_le(buffer + offset,
AMDUAT_ASL_CORE_INDEX_HEADER_SIZE);
offset += 4;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->header.snapshot_min);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->header.snapshot_max);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, record_count);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, records_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, bloom_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, bloom_size);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, digests_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, digests_size);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, extents_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, extent_count);
offset += 8;
amduat_asl_core_index_store_u32_le(buffer + offset,
segment->header.segment_domain_id);
offset += 4;
buffer[offset++] = segment->header.segment_visibility;
buffer[offset++] = 0;
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_core_index_store_u64_le(buffer + offset, 0);
offset += 8;
if (offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
free(buffer);
return false;
}
if (bloom_size != 0) {
memcpy(buffer + (size_t)bloom_offset, segment->bloom.data,
(size_t)bloom_size);
}
digest_cursor = 0;
extent_cursor = 0;
offset = (size_t)records_offset;
for (i = 0; i < record_count; ++i) {
const amduat_asl_index_record_t *record = &segment->records[i];
uint64_t digest_offset = digests_offset + digest_cursor;
uint64_t extents_offset_out = 0;
bool is_tombstone =
(record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
if (!is_tombstone) {
extents_offset_out = extents_offset +
(uint64_t)extent_cursor *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
extent_cursor += record->extent_count;
}
amduat_asl_core_index_store_u32_le(buffer + offset, record->hash_id);
offset += 4;
amduat_asl_core_index_store_u16_le(buffer + offset, record->digest_len);
offset += 2;
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_core_index_store_u64_le(buffer + offset, digest_offset);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset, extents_offset_out);
offset += 8;
amduat_asl_core_index_store_u32_le(buffer + offset,
record->extent_count);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset,
record->total_length);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset, record->domain_id);
offset += 4;
buffer[offset++] = record->visibility;
buffer[offset++] = record->has_cross_domain_source;
amduat_asl_core_index_store_u16_le(buffer + offset, 0);
offset += 2;
amduat_asl_core_index_store_u32_le(buffer + offset,
record->cross_domain_source);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset, record->flags);
offset += 4;
digest_cursor += record->digest_len;
}
if (segment->digests.len != 0) {
memcpy(buffer + (size_t)digests_offset, segment->digests.data,
segment->digests.len);
}
offset = (size_t)extents_offset;
for (i = 0; i < extent_count; ++i) {
const amduat_asl_extent_record_t *extent = &segment->extents[i];
amduat_asl_core_index_store_u64_le(buffer + offset, extent->block_id);
offset += 8;
amduat_asl_core_index_store_u32_le(buffer + offset, extent->offset);
offset += 4;
amduat_asl_core_index_store_u32_le(buffer + offset, extent->length);
offset += 4;
}
{
uint64_t crc =
amduat_asl_core_index_crc64(buffer, (size_t)footer_offset);
offset = (size_t)footer_offset;
amduat_asl_core_index_store_u64_le(buffer + offset, crc);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->footer.seal_snapshot);
offset += 8;
amduat_asl_core_index_store_u64_le(buffer + offset,
segment->footer.seal_time_ns);
offset += 8;
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_asl_core_index_decode_v1(
amduat_octets_t bytes,
amduat_asl_core_index_segment_t *out_segment) {
const uint8_t *data;
amduat_asl_segment_header_t header;
amduat_asl_segment_footer_t footer;
size_t record_count;
size_t extent_count;
size_t i;
uint64_t footer_offset;
uint64_t records_bytes;
uint64_t extents_bytes;
uint8_t max_visibility;
bool legacy_defaults;
if (out_segment == NULL) {
return false;
}
memset(out_segment, 0, sizeof(*out_segment));
if (bytes.len != 0 && bytes.data == NULL) {
return false;
}
if (bytes.len < AMDUAT_ASL_CORE_INDEX_HEADER_SIZE +
AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE) {
return false;
}
data = bytes.data;
header.magic = amduat_asl_core_index_load_u64_le(data);
header.version = amduat_asl_core_index_load_u16_le(data + 8);
header.shard_id = amduat_asl_core_index_load_u16_le(data + 10);
header.header_size = amduat_asl_core_index_load_u32_le(data + 12);
header.snapshot_min = amduat_asl_core_index_load_u64_le(data + 16);
header.snapshot_max = amduat_asl_core_index_load_u64_le(data + 24);
header.record_count = amduat_asl_core_index_load_u64_le(data + 32);
header.records_offset = amduat_asl_core_index_load_u64_le(data + 40);
header.bloom_offset = amduat_asl_core_index_load_u64_le(data + 48);
header.bloom_size = amduat_asl_core_index_load_u64_le(data + 56);
header.digests_offset = amduat_asl_core_index_load_u64_le(data + 64);
header.digests_size = amduat_asl_core_index_load_u64_le(data + 72);
header.extents_offset = amduat_asl_core_index_load_u64_le(data + 80);
header.extent_count = amduat_asl_core_index_load_u64_le(data + 88);
header.segment_domain_id = amduat_asl_core_index_load_u32_le(data + 96);
header.segment_visibility = data[100];
header.federation_version = data[101];
header.reserved0 = amduat_asl_core_index_load_u16_le(data + 102);
header.flags = amduat_asl_core_index_load_u64_le(data + 104);
if (header.magic != AMDUAT_ASL_CORE_INDEX_MAGIC) {
return false;
}
if (header.version < 1 || header.version > AMDUAT_ASL_CORE_INDEX_VERSION) {
return false;
}
if (header.header_size != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
return false;
}
if (header.flags != 0) {
return false;
}
if (header.snapshot_min > header.snapshot_max) {
return false;
}
legacy_defaults = header.version < AMDUAT_ASL_CORE_INDEX_VERSION;
if (!legacy_defaults) {
if (header.federation_version != 0 || header.reserved0 != 0) {
return false;
}
if (header.segment_visibility > 1) {
return false;
}
}
if (header.record_count > SIZE_MAX) {
return false;
}
record_count = (size_t)header.record_count;
if (record_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_RECORD_SIZE) {
return false;
}
records_bytes = header.record_count *
AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
if (header.extent_count > SIZE_MAX) {
return false;
}
extent_count = (size_t)header.extent_count;
if (extent_count > UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
return false;
}
extents_bytes = header.extent_count *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
if (header.bloom_size == 0) {
if (header.bloom_offset != 0) {
return false;
}
} else {
if (header.bloom_offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.bloom_size)) {
return false;
}
}
if (header.records_offset != AMDUAT_ASL_CORE_INDEX_HEADER_SIZE +
header.bloom_size) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.records_offset)) {
return false;
}
if (header.digests_offset != header.records_offset + records_bytes) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.digests_offset)) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.digests_size)) {
return false;
}
if (header.extents_offset != header.digests_offset + header.digests_size) {
return false;
}
if (!amduat_asl_core_index_is_aligned8(header.extents_offset)) {
return false;
}
footer_offset = header.extents_offset + extents_bytes;
if (footer_offset < header.extents_offset) {
return false;
}
if (footer_offset > SIZE_MAX) {
return false;
}
if (footer_offset + AMDUAT_ASL_CORE_INDEX_FOOTER_SIZE != bytes.len) {
return false;
}
footer.crc64 = amduat_asl_core_index_load_u64_le(
data + (size_t)footer_offset);
footer.seal_snapshot = amduat_asl_core_index_load_u64_le(
data + (size_t)footer_offset + 8);
footer.seal_time_ns = amduat_asl_core_index_load_u64_le(
data + (size_t)footer_offset + 16);
if (amduat_asl_core_index_crc64(data, (size_t)footer_offset) !=
footer.crc64) {
return false;
}
if (header.bloom_size != 0) {
if (!amduat_octets_clone(
amduat_octets(data + (size_t)header.bloom_offset,
(size_t)header.bloom_size),
&out_segment->bloom)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
if (!amduat_octets_clone(
amduat_octets(data + (size_t)header.digests_offset,
(size_t)header.digests_size),
&out_segment->digests)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record_count != 0) {
out_segment->records = (amduat_asl_index_record_t *)calloc(
record_count, sizeof(*out_segment->records));
if (out_segment->records == NULL) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
out_segment->record_count = record_count;
if (extent_count != 0) {
out_segment->extents = (amduat_asl_extent_record_t *)calloc(
extent_count, sizeof(*out_segment->extents));
if (out_segment->extents == NULL) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
out_segment->extent_count = extent_count;
for (i = 0; i < extent_count; ++i) {
size_t base = (size_t)header.extents_offset +
i * AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE;
amduat_asl_extent_record_t *extent = &out_segment->extents[i];
extent->block_id = amduat_asl_core_index_load_u64_le(data + base);
extent->offset = amduat_asl_core_index_load_u32_le(data + base + 8);
extent->length = amduat_asl_core_index_load_u32_le(data + base + 12);
if (extent->length == 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
max_visibility = 0;
for (i = 0; i < record_count; ++i) {
size_t base = (size_t)header.records_offset +
i * AMDUAT_ASL_CORE_INDEX_RECORD_SIZE;
amduat_asl_index_record_t *record = &out_segment->records[i];
uint64_t extents_offset;
size_t extent_index;
size_t j;
uint64_t total_len;
bool is_tombstone;
record->hash_id = amduat_asl_core_index_load_u32_le(data + base);
record->digest_len = amduat_asl_core_index_load_u16_le(data + base + 4);
record->reserved0 = amduat_asl_core_index_load_u16_le(data + base + 6);
record->digest_offset = amduat_asl_core_index_load_u64_le(data + base + 8);
record->extents_offset = amduat_asl_core_index_load_u64_le(
data + base + 16);
record->extent_count = amduat_asl_core_index_load_u32_le(data + base + 24);
record->total_length = amduat_asl_core_index_load_u32_le(data + base + 28);
record->domain_id = amduat_asl_core_index_load_u32_le(data + base + 32);
record->visibility = data[base + 36];
record->has_cross_domain_source = data[base + 37];
record->reserved1 = amduat_asl_core_index_load_u16_le(data + base + 38);
record->cross_domain_source =
amduat_asl_core_index_load_u32_le(data + base + 40);
record->flags = amduat_asl_core_index_load_u32_le(data + base + 44);
if (!legacy_defaults) {
if (record->reserved0 != 0 || record->reserved1 != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->visibility > 1 || record->has_cross_domain_source > 1) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->has_cross_domain_source == 0 &&
record->cross_domain_source != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
} else {
record->domain_id = 0;
record->visibility = 0;
record->has_cross_domain_source = 0;
record->cross_domain_source = 0;
}
if ((record->flags & ~AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->hash_id <= UINT16_MAX) {
const amduat_hash_asl1_desc_t *hash_desc =
amduat_hash_asl1_desc_lookup((amduat_hash_id_t)record->hash_id);
if (hash_desc != NULL && hash_desc->digest_len != 0 &&
record->digest_len != hash_desc->digest_len) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
if (record->digest_len == 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->digest_len > header.digests_size) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (!legacy_defaults &&
!amduat_asl_core_index_is_aligned8(record->digest_offset)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->digest_offset < header.digests_offset) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->digest_offset >
header.digests_offset + header.digests_size -
record->digest_len) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
is_tombstone =
(record->flags & AMDUAT_ASL_INDEX_FLAG_TOMBSTONE) != 0;
if (is_tombstone) {
if (record->extent_count != 0 || record->total_length != 0 ||
record->extents_offset != 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->visibility > max_visibility) {
max_visibility = record->visibility;
}
continue;
}
if (record->extent_count == 0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
extents_offset = record->extents_offset;
if (extents_offset < header.extents_offset) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (!amduat_asl_core_index_is_aligned8(extents_offset)) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->extent_count >
UINT64_MAX / AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if ((extents_offset - header.extents_offset) %
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE !=
0) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (extents_offset + (uint64_t)record->extent_count *
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE >
header.extents_offset + extents_bytes) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
extent_index = (size_t)((extents_offset - header.extents_offset) /
AMDUAT_ASL_CORE_INDEX_EXTENT_SIZE);
total_len = 0;
for (j = 0; j < record->extent_count; ++j) {
total_len += out_segment->extents[extent_index + j].length;
if (total_len > UINT32_MAX) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
}
if ((uint32_t)total_len != record->total_length) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
if (record->visibility > max_visibility) {
max_visibility = record->visibility;
}
}
if (!legacy_defaults) {
if (max_visibility != header.segment_visibility) {
amduat_enc_asl_core_index_free(out_segment);
return false;
}
} else {
header.segment_domain_id = 0;
header.segment_visibility = 0;
header.federation_version = 0;
header.reserved0 = 0;
}
out_segment->header = header;
out_segment->footer = footer;
return true;
}
void amduat_enc_asl_core_index_free(amduat_asl_core_index_segment_t *segment) {
if (segment == NULL) {
return;
}
free(segment->records);
segment->records = NULL;
segment->record_count = 0;
free(segment->extents);
segment->extents = NULL;
segment->extent_count = 0;
amduat_octets_free(&segment->digests);
amduat_octets_free(&segment->bloom);
}

470
src/near_core/enc/asl_log.c
View file

@ -1,470 +0,0 @@
#include "amduat/enc/asl_log.h"
#include "amduat/hash/asl1.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_LOG_MAGIC_LEN = 8,
AMDUAT_ASL_LOG_HASH_LEN = 32,
AMDUAT_ASL_LOG_HEADER_LEN = 24,
AMDUAT_ASL_LOG_VERSION = 1
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_log_cursor_t;
static const uint8_t k_amduat_asl_log_magic[AMDUAT_ASL_LOG_MAGIC_LEN] = {
'A', 'S', 'L', 'L', 'O', 'G', '0', '1'};
static void amduat_asl_log_store_u32_le(uint8_t *out, uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_log_store_u16_le(uint8_t *out, uint16_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
}
static void amduat_asl_log_store_u64_le(uint8_t *out, uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_log_read_u32_le(amduat_asl_log_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4;
return true;
}
static bool amduat_asl_log_read_u16_le(amduat_asl_log_cursor_t *cur,
uint16_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 2) {
return false;
}
data = cur->data + cur->offset;
*out = (uint16_t)data[0] | ((uint16_t)data[1] << 8);
cur->offset += 2;
return true;
}
static bool amduat_asl_log_read_u64_le(amduat_asl_log_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8;
return true;
}
static bool amduat_asl_log_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_log_is_known_record_type(uint32_t record_type) {
switch (record_type) {
case AMDUAT_ASL_LOG_RECORD_SEGMENT_SEAL:
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE:
case AMDUAT_ASL_LOG_RECORD_TOMBSTONE_LIFT:
case AMDUAT_ASL_LOG_RECORD_SNAPSHOT_ANCHOR:
case AMDUAT_ASL_LOG_RECORD_ARTIFACT_PUBLISH:
case AMDUAT_ASL_LOG_RECORD_ARTIFACT_UNPUBLISH:
return true;
default:
return false;
}
}
bool amduat_asl_log_decode_artifact_ref(amduat_octets_t payload,
amduat_reference_t *out_ref) {
amduat_asl_log_cursor_t cur;
uint32_t hash_id_raw;
uint16_t digest_len;
uint16_t reserved0;
if (out_ref == NULL || payload.data == NULL || payload.len < 8u) {
return false;
}
cur.data = payload.data;
cur.len = payload.len;
cur.offset = 0;
if (!amduat_asl_log_read_u32_le(&cur, &hash_id_raw) ||
!amduat_asl_log_read_u16_le(&cur, &digest_len) ||
!amduat_asl_log_read_u16_le(&cur, &reserved0)) {
return false;
}
if (hash_id_raw > UINT16_MAX || digest_len == 0 || reserved0 != 0) {
return false;
}
if (cur.len - cur.offset < digest_len) {
return false;
}
{
amduat_octets_t digest = amduat_octets(cur.data + cur.offset, digest_len);
amduat_reference_t tmp =
amduat_reference((amduat_hash_id_t)hash_id_raw, digest);
return amduat_reference_clone(tmp, out_ref);
}
}
bool amduat_asl_log_encode_artifact_ref(amduat_reference_t ref,
amduat_octets_t *out_bytes) {
uint8_t *buf;
size_t len;
if (out_bytes == NULL) {
return false;
}
*out_bytes = amduat_octets(NULL, 0u);
if (ref.hash_id == 0 || ref.digest.data == NULL || ref.digest.len == 0) {
return false;
}
if (ref.digest.len > UINT16_MAX) {
return false;
}
len = 8u + ref.digest.len;
buf = (uint8_t *)malloc(len);
if (buf == NULL) {
return false;
}
amduat_asl_log_store_u32_le(buf, ref.hash_id);
amduat_asl_log_store_u16_le(buf + 4u, (uint16_t)ref.digest.len);
amduat_asl_log_store_u16_le(buf + 6u, 0u);
memcpy(buf + 8u, ref.digest.data, ref.digest.len);
*out_bytes = amduat_octets(buf, len);
return true;
}
static bool amduat_asl_log_hash_record(const uint8_t prev_hash[32],
uint64_t logseq,
uint32_t record_type,
uint32_t payload_len,
const uint8_t *payload,
uint8_t out_hash[32]) {
amduat_hash_asl1_stream_t stream;
uint8_t logseq_bytes[8];
uint8_t type_bytes[4];
uint8_t payload_len_bytes[4];
bool ok;
if (payload_len != 0 && payload == NULL) {
return false;
}
if (!amduat_hash_asl1_stream_init(AMDUAT_HASH_ASL1_ID_SHA256, &stream)) {
return false;
}
amduat_asl_log_store_u64_le(logseq_bytes, logseq);
amduat_asl_log_store_u32_le(type_bytes, record_type);
amduat_asl_log_store_u32_le(payload_len_bytes, payload_len);
ok = amduat_hash_asl1_stream_update(&stream,
amduat_octets(prev_hash,
AMDUAT_ASL_LOG_HASH_LEN)) &&
amduat_hash_asl1_stream_update(&stream,
amduat_octets(logseq_bytes,
sizeof(logseq_bytes))) &&
amduat_hash_asl1_stream_update(&stream,
amduat_octets(type_bytes,
sizeof(type_bytes))) &&
amduat_hash_asl1_stream_update(
&stream,
amduat_octets(payload_len_bytes, sizeof(payload_len_bytes)));
if (ok && payload_len != 0) {
ok = amduat_hash_asl1_stream_update(
&stream, amduat_octets(payload, payload_len));
}
if (ok) {
ok = amduat_hash_asl1_stream_final(&stream, out_hash,
AMDUAT_ASL_LOG_HASH_LEN);
}
amduat_hash_asl1_stream_destroy(&stream);
return ok;
}
void amduat_enc_asl_log_free(amduat_asl_log_record_t *records,
size_t record_count) {
size_t i;
if (records == NULL) {
return;
}
for (i = 0; i < record_count; ++i) {
amduat_octets_free(&records[i].payload);
}
free(records);
}
bool amduat_enc_asl_log_encode_v1(const amduat_asl_log_record_t *records,
size_t record_count,
amduat_octets_t *out_bytes) {
size_t total_len;
size_t i;
size_t offset;
uint8_t *buffer;
uint8_t prev_hash[AMDUAT_ASL_LOG_HASH_LEN];
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (record_count != 0 && records == NULL) {
return false;
}
total_len = AMDUAT_ASL_LOG_HEADER_LEN;
for (i = 0; i < record_count; ++i) {
size_t payload_len = records[i].payload.len;
if (payload_len != 0 && records[i].payload.data == NULL) {
return false;
}
if (payload_len > UINT32_MAX) {
return false;
}
if (!amduat_asl_log_add_size(&total_len, 8 + 4 + 4)) {
return false;
}
if (!amduat_asl_log_add_size(&total_len, payload_len)) {
return false;
}
if (!amduat_asl_log_add_size(&total_len, AMDUAT_ASL_LOG_HASH_LEN)) {
return false;
}
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0;
memcpy(buffer + offset, k_amduat_asl_log_magic, AMDUAT_ASL_LOG_MAGIC_LEN);
offset += AMDUAT_ASL_LOG_MAGIC_LEN;
amduat_asl_log_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_VERSION);
offset += 4;
amduat_asl_log_store_u32_le(buffer + offset, AMDUAT_ASL_LOG_HEADER_LEN);
offset += 4;
amduat_asl_log_store_u64_le(buffer + offset, 0);
offset += 8;
memset(prev_hash, 0, sizeof(prev_hash));
for (i = 0; i < record_count; ++i) {
const amduat_asl_log_record_t *record = &records[i];
uint32_t payload_len = (uint32_t)record->payload.len;
uint8_t record_hash[AMDUAT_ASL_LOG_HASH_LEN];
amduat_asl_log_store_u64_le(buffer + offset, record->logseq);
offset += 8;
amduat_asl_log_store_u32_le(buffer + offset, record->record_type);
offset += 4;
amduat_asl_log_store_u32_le(buffer + offset, payload_len);
offset += 4;
if (payload_len != 0) {
memcpy(buffer + offset, record->payload.data, payload_len);
offset += payload_len;
}
if (!amduat_asl_log_hash_record(prev_hash,
record->logseq,
record->record_type,
payload_len,
record->payload.data,
record_hash)) {
free(buffer);
return false;
}
memcpy(buffer + offset, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
offset += AMDUAT_ASL_LOG_HASH_LEN;
memcpy(prev_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_asl_log_decode_v1(amduat_octets_t bytes,
amduat_asl_log_record_t **out_records,
size_t *out_count) {
amduat_asl_log_cursor_t cur;
uint32_t version;
uint32_t header_size;
uint64_t flags;
uint8_t prev_hash[AMDUAT_ASL_LOG_HASH_LEN];
amduat_asl_log_record_t *records;
size_t record_count;
size_t record_capacity;
uint64_t last_logseq;
if (out_records == NULL || out_count == NULL) {
return false;
}
*out_records = NULL;
*out_count = 0;
if (bytes.len != 0 && bytes.data == NULL) {
return false;
}
if (bytes.len < AMDUAT_ASL_LOG_HEADER_LEN) {
return false;
}
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0;
if (memcmp(cur.data, k_amduat_asl_log_magic, AMDUAT_ASL_LOG_MAGIC_LEN) != 0) {
return false;
}
cur.offset += AMDUAT_ASL_LOG_MAGIC_LEN;
if (!amduat_asl_log_read_u32_le(&cur, &version)) {
return false;
}
if (version != AMDUAT_ASL_LOG_VERSION) {
return false;
}
if (!amduat_asl_log_read_u32_le(&cur, &header_size)) {
return false;
}
if (!amduat_asl_log_read_u64_le(&cur, &flags)) {
return false;
}
if (flags != 0) {
return false;
}
if (header_size < AMDUAT_ASL_LOG_HEADER_LEN) {
return false;
}
if (bytes.len < header_size) {
return false;
}
cur.offset = header_size;
records = NULL;
record_count = 0;
record_capacity = 0;
last_logseq = 0u;
memset(prev_hash, 0, sizeof(prev_hash));
while (cur.offset < cur.len) {
uint64_t logseq;
uint32_t record_type;
uint32_t payload_len;
const uint8_t *payload;
const uint8_t *record_hash;
uint8_t expected_hash[AMDUAT_ASL_LOG_HASH_LEN];
if (!amduat_asl_log_read_u64_le(&cur, &logseq) ||
!amduat_asl_log_read_u32_le(&cur, &record_type) ||
!amduat_asl_log_read_u32_le(&cur, &payload_len)) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
if (record_count != 0u && logseq <= last_logseq) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
if (cur.len - cur.offset < payload_len + AMDUAT_ASL_LOG_HASH_LEN) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
payload = cur.data + cur.offset;
cur.offset += payload_len;
record_hash = cur.data + cur.offset;
cur.offset += AMDUAT_ASL_LOG_HASH_LEN;
if (!amduat_asl_log_hash_record(prev_hash,
logseq,
record_type,
payload_len,
payload,
expected_hash)) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
if (memcmp(expected_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN) != 0) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
memcpy(prev_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
last_logseq = logseq;
if (amduat_asl_log_is_known_record_type(record_type)) {
amduat_asl_log_record_t *slot;
amduat_octets_t payload_bytes;
if (record_count == record_capacity) {
size_t new_capacity = record_capacity == 0 ? 4 : record_capacity * 2;
amduat_asl_log_record_t *next =
(amduat_asl_log_record_t *)realloc(
records, new_capacity * sizeof(*records));
if (next == NULL) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
records = next;
record_capacity = new_capacity;
}
payload_bytes = amduat_octets(payload, payload_len);
slot = &records[record_count];
slot->logseq = logseq;
slot->record_type = record_type;
slot->payload = amduat_octets(NULL, 0u);
if (!amduat_octets_clone(payload_bytes, &slot->payload)) {
amduat_enc_asl_log_free(records, record_count);
return false;
}
memcpy(slot->record_hash, record_hash, AMDUAT_ASL_LOG_HASH_LEN);
record_count++;
}
}
*out_records = records;
*out_count = record_count;
return true;
}

497
src/near_core/enc/asl_tgk_exec_plan.c
View file

@ -1,497 +0,0 @@
#include "amduat/enc/asl_tgk_exec_plan.h"
#include <limits.h>
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE = 8,
AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP = 8,
AMDUAT_ASL_TGK_EXEC_PLAN_PARAMS_SIZE = 65,
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE = 129,
AMDUAT_ASL_TGK_EXEC_PLAN_OP_TYPE_MAX =
AMDUAT_ASL_TGK_EXEC_OP_TOMBSTONE_SHADOW,
AMDUAT_ASL_TGK_EXEC_PLAN_OP_FLAG_MASK =
AMDUAT_ASL_TGK_EXEC_OP_FLAG_PARALLEL |
AMDUAT_ASL_TGK_EXEC_OP_FLAG_OPTIONAL
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_asl_tgk_exec_plan_cursor_t;
static void amduat_asl_tgk_exec_plan_store_u32_le(uint8_t *out,
uint32_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
}
static void amduat_asl_tgk_exec_plan_store_u64_le(uint8_t *out,
uint64_t value) {
out[0] = (uint8_t)(value & 0xffu);
out[1] = (uint8_t)((value >> 8) & 0xffu);
out[2] = (uint8_t)((value >> 16) & 0xffu);
out[3] = (uint8_t)((value >> 24) & 0xffu);
out[4] = (uint8_t)((value >> 32) & 0xffu);
out[5] = (uint8_t)((value >> 40) & 0xffu);
out[6] = (uint8_t)((value >> 48) & 0xffu);
out[7] = (uint8_t)((value >> 56) & 0xffu);
}
static bool amduat_asl_tgk_exec_plan_read_u32_le(
amduat_asl_tgk_exec_plan_cursor_t *cur,
uint32_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 4u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint32_t)data[0] | ((uint32_t)data[1] << 8) |
((uint32_t)data[2] << 16) | ((uint32_t)data[3] << 24);
cur->offset += 4u;
return true;
}
static bool amduat_asl_tgk_exec_plan_read_u64_le(
amduat_asl_tgk_exec_plan_cursor_t *cur,
uint64_t *out) {
const uint8_t *data;
if (cur->len - cur->offset < 8u) {
return false;
}
data = cur->data + cur->offset;
*out = (uint64_t)data[0] | ((uint64_t)data[1] << 8) |
((uint64_t)data[2] << 16) | ((uint64_t)data[3] << 24) |
((uint64_t)data[4] << 32) | ((uint64_t)data[5] << 40) |
((uint64_t)data[6] << 48) | ((uint64_t)data[7] << 56);
cur->offset += 8u;
return true;
}
static bool amduat_asl_tgk_exec_plan_add_size(size_t *acc, size_t add) {
if (*acc > SIZE_MAX - add) {
return false;
}
*acc += add;
return true;
}
static bool amduat_asl_tgk_exec_plan_is_bool(uint8_t value) {
return value == 0u || value == 1u;
}
static bool amduat_asl_tgk_exec_plan_validate_params(
amduat_asl_tgk_exec_operator_type_t op_type,
const amduat_asl_tgk_exec_operator_params_t *params) {
if (!amduat_asl_tgk_exec_plan_is_bool(params->segment_scan.is_asl_segment)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->index_filter.has_type_tag)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->index_filter.has_edge_type)) {
return false;
}
if (params->index_filter.role > 3u) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->merge.deterministic)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(
params->projection.project_artifact_id) ||
!amduat_asl_tgk_exec_plan_is_bool(
params->projection.project_tgk_edge_id) ||
!amduat_asl_tgk_exec_plan_is_bool(params->projection.project_node_id) ||
!amduat_asl_tgk_exec_plan_is_bool(params->projection.project_type_tag)) {
return false;
}
if (params->tgk_traversal.direction > 3u) {
return false;
}
if (op_type == AMDUAT_ASL_TGK_EXEC_OP_TGK_TRAVERSAL &&
params->tgk_traversal.direction == 0u) {
return false;
}
if (!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_count) ||
!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_union) ||
!amduat_asl_tgk_exec_plan_is_bool(params->aggregation.agg_sum)) {
return false;
}
return true;
}
static bool amduat_asl_tgk_exec_plan_has_op_id(
const amduat_asl_tgk_exec_plan_t *plan,
uint32_t op_id) {
uint32_t i;
for (i = 0; i < plan->operator_count; ++i) {
if (plan->operators[i].op_id == op_id) {
return true;
}
}
return false;
}
static bool amduat_asl_tgk_exec_plan_unique_op_ids(
const amduat_asl_tgk_exec_plan_t *plan) {
uint32_t i;
uint32_t j;
for (i = 0; i < plan->operator_count; ++i) {
for (j = i + 1u; j < plan->operator_count; ++j) {
if (plan->operators[i].op_id == plan->operators[j].op_id) {
return false;
}
}
}
return true;
}
static bool amduat_asl_tgk_exec_plan_validate_operator(
const amduat_asl_tgk_exec_plan_t *plan,
const amduat_asl_tgk_exec_operator_def_t *op) {
uint32_t i;
if ((uint32_t)op->op_type > AMDUAT_ASL_TGK_EXEC_PLAN_OP_TYPE_MAX) {
return false;
}
if (((uint32_t)op->flags & ~AMDUAT_ASL_TGK_EXEC_PLAN_OP_FLAG_MASK) != 0u) {
return false;
}
if (op->snapshot.logseq_min > op->snapshot.logseq_max) {
return false;
}
if (op->input_count > AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP) {
return false;
}
if (!amduat_asl_tgk_exec_plan_validate_params(op->op_type, &op->params)) {
return false;
}
for (i = 0; i < op->input_count; ++i) {
if (!amduat_asl_tgk_exec_plan_has_op_id(plan, op->inputs[i])) {
return false;
}
}
return true;
}
static void amduat_asl_tgk_exec_plan_encode_params(
const amduat_asl_tgk_exec_operator_params_t *params,
uint8_t *buffer,
size_t *offset) {
buffer[(*offset)++] = params->segment_scan.is_asl_segment;
amduat_asl_tgk_exec_plan_store_u64_le(
buffer + *offset, params->segment_scan.segment_start_id);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u64_le(
buffer + *offset, params->segment_scan.segment_end_id);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u32_le(
buffer + *offset, params->index_filter.artifact_type_tag);
*offset += 4u;
buffer[(*offset)++] = params->index_filter.has_type_tag;
amduat_asl_tgk_exec_plan_store_u32_le(
buffer + *offset, params->index_filter.edge_type_key);
*offset += 4u;
buffer[(*offset)++] = params->index_filter.has_edge_type;
buffer[(*offset)++] = params->index_filter.role;
buffer[(*offset)++] = params->merge.deterministic;
buffer[(*offset)++] = params->projection.project_artifact_id;
buffer[(*offset)++] = params->projection.project_tgk_edge_id;
buffer[(*offset)++] = params->projection.project_node_id;
buffer[(*offset)++] = params->projection.project_type_tag;
amduat_asl_tgk_exec_plan_store_u64_le(
buffer + *offset, params->tgk_traversal.start_node_id);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u32_le(
buffer + *offset, params->tgk_traversal.traversal_depth);
*offset += 4u;
buffer[(*offset)++] = params->tgk_traversal.direction;
buffer[(*offset)++] = params->aggregation.agg_count;
buffer[(*offset)++] = params->aggregation.agg_union;
buffer[(*offset)++] = params->aggregation.agg_sum;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + *offset,
params->limit_offset.limit);
*offset += 8u;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + *offset,
params->limit_offset.offset);
*offset += 8u;
}
static bool amduat_asl_tgk_exec_plan_decode_params(
amduat_asl_tgk_exec_plan_cursor_t *cur,
amduat_asl_tgk_exec_operator_params_t *params) {
if (cur->len - cur->offset < AMDUAT_ASL_TGK_EXEC_PLAN_PARAMS_SIZE) {
return false;
}
params->segment_scan.is_asl_segment = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->segment_scan.segment_start_id) ||
!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->segment_scan.segment_end_id)) {
return false;
}
if (!amduat_asl_tgk_exec_plan_read_u32_le(
cur, &params->index_filter.artifact_type_tag)) {
return false;
}
params->index_filter.has_type_tag = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u32_le(
cur, &params->index_filter.edge_type_key)) {
return false;
}
params->index_filter.has_edge_type = cur->data[cur->offset++];
params->index_filter.role = cur->data[cur->offset++];
params->merge.deterministic = cur->data[cur->offset++];
params->projection.project_artifact_id = cur->data[cur->offset++];
params->projection.project_tgk_edge_id = cur->data[cur->offset++];
params->projection.project_node_id = cur->data[cur->offset++];
params->projection.project_type_tag = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->tgk_traversal.start_node_id) ||
!amduat_asl_tgk_exec_plan_read_u32_le(
cur, &params->tgk_traversal.traversal_depth)) {
return false;
}
params->tgk_traversal.direction = cur->data[cur->offset++];
params->aggregation.agg_count = cur->data[cur->offset++];
params->aggregation.agg_union = cur->data[cur->offset++];
params->aggregation.agg_sum = cur->data[cur->offset++];
if (!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->limit_offset.limit) ||
!amduat_asl_tgk_exec_plan_read_u64_le(
cur, &params->limit_offset.offset)) {
return false;
}
return true;
}
bool amduat_enc_asl_tgk_exec_plan_encode_v1(
const amduat_asl_tgk_exec_plan_t *plan,
amduat_octets_t *out_bytes) {
size_t total_len;
size_t offset;
uint8_t *buffer;
uint32_t i;
if (out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0u;
if (plan == NULL) {
return false;
}
if (plan->operator_count != 0u && plan->operators == NULL) {
return false;
}
if (plan->operator_count > SIZE_MAX / AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE) {
return false;
}
total_len = AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE;
if (!amduat_asl_tgk_exec_plan_add_size(
&total_len,
(size_t)plan->operator_count *
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE)) {
return false;
}
if (plan->operator_count > UINT32_MAX) {
return false;
}
for (i = 0; i < plan->operator_count; ++i) {
if (!amduat_asl_tgk_exec_plan_validate_operator(plan,
&plan->operators[i])) {
return false;
}
}
if (!amduat_asl_tgk_exec_plan_unique_op_ids(plan)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
offset = 0u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, plan->plan_version);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
plan->operator_count);
offset += 4u;
for (i = 0; i < plan->operator_count; ++i) {
const amduat_asl_tgk_exec_operator_def_t *op = &plan->operators[i];
uint32_t j;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->op_id);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
(uint32_t)op->op_type);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset,
(uint32_t)op->flags);
offset += 4u;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + offset,
op->snapshot.logseq_min);
offset += 8u;
amduat_asl_tgk_exec_plan_store_u64_le(buffer + offset,
op->snapshot.logseq_max);
offset += 8u;
amduat_asl_tgk_exec_plan_encode_params(&op->params, buffer, &offset);
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->input_count);
offset += 4u;
for (j = 0; j < AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP; ++j) {
amduat_asl_tgk_exec_plan_store_u32_le(buffer + offset, op->inputs[j]);
offset += 4u;
}
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_asl_tgk_exec_plan_decode_v1(
amduat_octets_t bytes,
amduat_asl_tgk_exec_plan_t *out_plan) {
amduat_asl_tgk_exec_plan_cursor_t cur;
uint32_t operator_count;
size_t expected_len;
amduat_asl_tgk_exec_operator_def_t *operators;
uint32_t i;
if (out_plan == NULL) {
return false;
}
out_plan->plan_version = 0u;
out_plan->operator_count = 0u;
out_plan->operators = NULL;
if (bytes.len < AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE ||
bytes.data == NULL) {
return false;
}
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0u;
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &out_plan->plan_version) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &operator_count)) {
return false;
}
if (operator_count > SIZE_MAX / AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE) {
return false;
}
expected_len = AMDUAT_ASL_TGK_EXEC_PLAN_HEADER_SIZE +
(size_t)operator_count *
AMDUAT_ASL_TGK_EXEC_PLAN_OPERATOR_SIZE;
if (expected_len != bytes.len) {
return false;
}
if (operator_count == 0u) {
out_plan->operator_count = 0u;
return true;
}
operators = (amduat_asl_tgk_exec_operator_def_t *)calloc(
operator_count, sizeof(*operators));
if (operators == NULL) {
return false;
}
for (i = 0; i < operator_count; ++i) {
amduat_asl_tgk_exec_operator_def_t *op = &operators[i];
uint32_t j;
uint32_t op_type_raw;
uint32_t flags_raw;
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->op_id) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op_type_raw) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &flags_raw) ||
!amduat_asl_tgk_exec_plan_read_u64_le(&cur,
&op->snapshot.logseq_min) ||
!amduat_asl_tgk_exec_plan_read_u64_le(&cur,
&op->snapshot.logseq_max)) {
free(operators);
return false;
}
op->op_type = (amduat_asl_tgk_exec_operator_type_t)op_type_raw;
op->flags = (amduat_asl_tgk_exec_operator_flags_t)flags_raw;
if (!amduat_asl_tgk_exec_plan_decode_params(&cur, &op->params) ||
!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->input_count)) {
free(operators);
return false;
}
if (op->input_count > AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP) {
free(operators);
return false;
}
for (j = 0; j < AMDUAT_ASL_TGK_EXEC_PLAN_INPUT_CAP; ++j) {
if (!amduat_asl_tgk_exec_plan_read_u32_le(&cur, &op->inputs[j])) {
free(operators);
return false;
}
}
}
out_plan->operator_count = operator_count;
out_plan->operators = operators;
if (!amduat_asl_tgk_exec_plan_unique_op_ids(out_plan)) {
amduat_enc_asl_tgk_exec_plan_free(out_plan);
return false;
}
for (i = 0; i < operator_count; ++i) {
if (!amduat_asl_tgk_exec_plan_validate_operator(out_plan,
&operators[i])) {
amduat_enc_asl_tgk_exec_plan_free(out_plan);
return false;
}
}
return true;
}
void amduat_enc_asl_tgk_exec_plan_free(amduat_asl_tgk_exec_plan_t *plan) {
if (plan == NULL) {
return;
}
free(plan->operators);
plan->operators = NULL;
plan->operator_count = 0u;
plan->plan_version = 0u;
}

294
src/near_core/enc/fcs1.c
View file

@ -1,294 +0,0 @@
#include "amduat/enc/fcs1.h"
#include "amduat/enc/asl1_core_codec.h"
#include <string.h>
enum {
AMDUAT_FCS1_HEADER_LEN = 7,
AMDUAT_FCS1_TAG_FUNCTION = 0x30,
AMDUAT_FCS1_TAG_PARAMS = 0x31,
AMDUAT_FCS1_TAG_ARITY = 0x32,
AMDUAT_FCS1_TAG_LEGACY_MIN = 0x33,
AMDUAT_FCS1_TAG_LEGACY_MAX = 0x36
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_fcs1_cursor_t;
static bool amduat_fcs1_read_varint(amduat_fcs1_cursor_t *cur,
uint64_t *out,
bool *out_minimal) {
uint64_t value = 0u;
uint32_t shift = 0u;
size_t bytes = 0u;
if (cur == NULL || out == NULL || out_minimal == NULL) {
return false;
}
while (cur->offset < cur->len) {
uint8_t byte = cur->data[cur->offset++];
value |= ((uint64_t)(byte & 0x7fu)) << shift;
bytes++;
if ((byte & 0x80u) == 0u) {
break;
}
shift += 7u;
if (shift >= 64u) {
return false;
}
}
if (bytes == 0u || cur->offset > cur->len) {
return false;
}
{
size_t minimal_len = 1u;
uint64_t tmp = value;
while (tmp >= 0x80u) {
tmp >>= 7u;
minimal_len++;
}
*out_minimal = (minimal_len == bytes);
}
*out = value;
return true;
}
static bool amduat_fcs1_read_bytes(amduat_fcs1_cursor_t *cur,
amduat_octets_t *out,
bool *out_minimal) {
uint64_t len;
bool minimal;
if (!amduat_fcs1_read_varint(cur, &len, &minimal)) {
return false;
}
if (len > SIZE_MAX || cur->len - cur->offset < (size_t)len) {
return false;
}
*out = amduat_octets(cur->data + cur->offset, (size_t)len);
cur->offset += (size_t)len;
*out_minimal = minimal;
return true;
}
static bool amduat_fcs1_has_header(amduat_octets_t bytes) {
static const uint8_t magic[4] = {'F', 'C', 'S', '1'};
if (bytes.len < AMDUAT_FCS1_HEADER_LEN || bytes.data == NULL) {
return false;
}
if (memcmp(bytes.data, magic, sizeof(magic)) != 0) {
return false;
}
if (bytes.data[4] != 0x01u || bytes.data[5] != 0x00u ||
bytes.data[6] != 0x00u) {
return false;
}
return true;
}
void amduat_fcs1_descriptor_free(amduat_fcs1_descriptor_t *desc) {
if (desc == NULL) {
return;
}
amduat_reference_free(&desc->function_ptr);
amduat_reference_free(&desc->parameter_block);
desc->arity = 0u;
}
amduat_fcs1_error_t amduat_fcs1_decode_v1(amduat_octets_t bytes,
amduat_fcs1_descriptor_t *out_desc) {
amduat_fcs1_cursor_t cur;
uint8_t last_tag = 0u;
bool seen_function = false;
bool seen_params = false;
bool seen_arity = false;
if (out_desc == NULL) {
return AMDUAT_FCS1_ERR_INVALID;
}
out_desc->function_ptr = amduat_reference(0, amduat_octets(NULL, 0u));
out_desc->parameter_block = amduat_reference(0, amduat_octets(NULL, 0u));
out_desc->arity = 0u;
if (!amduat_fcs1_has_header(bytes)) {
return AMDUAT_FCS1_ERR_HEADER_INVALID;
}
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = AMDUAT_FCS1_HEADER_LEN;
while (cur.offset < cur.len) {
uint8_t tag = cur.data[cur.offset++];
bool minimal = true;
if (tag <= last_tag) {
return AMDUAT_FCS1_ERR_TAG_ORDER;
}
last_tag = tag;
if (tag == AMDUAT_FCS1_TAG_FUNCTION ||
tag == AMDUAT_FCS1_TAG_PARAMS) {
amduat_octets_t cid_bytes;
if (!amduat_fcs1_read_bytes(&cur, &cid_bytes, &minimal)) {
return AMDUAT_FCS1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_FCS1_ERR_VARINT_NON_MINIMAL;
}
if (tag == AMDUAT_FCS1_TAG_FUNCTION) {
if (seen_function) {
return AMDUAT_FCS1_ERR_TAG_ORDER;
}
if (!amduat_enc_asl1_core_decode_reference_v1(cid_bytes,
&out_desc->function_ptr)) {
return AMDUAT_FCS1_ERR_INVALID;
}
seen_function = true;
} else {
if (seen_params) {
return AMDUAT_FCS1_ERR_TAG_ORDER;
}
if (!amduat_enc_asl1_core_decode_reference_v1(cid_bytes,
&out_desc->parameter_block)) {
return AMDUAT_FCS1_ERR_INVALID;
}
seen_params = true;
}
} else if (tag == AMDUAT_FCS1_TAG_ARITY) {
uint64_t arity_raw = 0u;
if (seen_arity) {
return AMDUAT_FCS1_ERR_TAG_ORDER;
}
if (!amduat_fcs1_read_varint(&cur, &arity_raw, &minimal)) {
return AMDUAT_FCS1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_FCS1_ERR_VARINT_NON_MINIMAL;
}
if (arity_raw > UINT32_MAX) {
return AMDUAT_FCS1_ERR_INVALID;
}
out_desc->arity = (uint32_t)arity_raw;
seen_arity = true;
} else if (tag >= AMDUAT_FCS1_TAG_LEGACY_MIN &&
tag <= AMDUAT_FCS1_TAG_LEGACY_MAX) {
return AMDUAT_FCS1_ERR_UNKNOWN_TAG;
} else {
return AMDUAT_FCS1_ERR_UNKNOWN_TAG;
}
}
if (!seen_function || !seen_params || !seen_arity) {
return AMDUAT_FCS1_ERR_TAG_ORDER;
}
if (cur.offset != cur.len) {
return AMDUAT_FCS1_ERR_TRAILING_BYTES;
}
return AMDUAT_FCS1_OK;
}
static amduat_fcs1_error_t amduat_fcs1_validate_cycle(
amduat_reference_t ref,
amduat_fcs1_resolve_fn resolver,
void *resolver_ctx,
amduat_reference_t *stack,
size_t stack_len,
size_t stack_cap) {
amduat_octets_t bytes = amduat_octets(NULL, 0u);
amduat_fcs1_descriptor_t desc;
amduat_fcs1_error_t err;
if (resolver == NULL) {
return AMDUAT_FCS1_ERR_INVALID;
}
if (!resolver(resolver_ctx, ref, &bytes)) {
return AMDUAT_FCS1_OK;
}
if (!amduat_fcs1_has_header(bytes)) {
amduat_octets_free(&bytes);
return AMDUAT_FCS1_OK;
}
for (size_t i = 0u; i < stack_len; ++i) {
if (amduat_reference_eq(stack[i], ref)) {
amduat_octets_free(&bytes);
return AMDUAT_FCS1_ERR_CYCLE_DETECTED;
}
}
memset(&desc, 0, sizeof(desc));
err = amduat_fcs1_decode_v1(bytes, &desc);
amduat_octets_free(&bytes);
if (err != AMDUAT_FCS1_OK) {
amduat_fcs1_descriptor_free(&desc);
return err;
}
if (stack_len >= stack_cap) {
amduat_fcs1_descriptor_free(&desc);
return AMDUAT_FCS1_ERR_INVALID;
}
stack[stack_len] = desc.function_ptr;
err = amduat_fcs1_validate_cycle(desc.function_ptr,
resolver,
resolver_ctx,
stack,
stack_len + 1u,
stack_cap);
amduat_fcs1_descriptor_free(&desc);
return err;
}
amduat_fcs1_error_t amduat_fcs1_validate_v1(
amduat_octets_t bytes,
amduat_fcs1_resolve_fn resolver,
void *resolver_ctx) {
amduat_fcs1_descriptor_t desc;
amduat_octets_t pcb_bytes = amduat_octets(NULL, 0u);
amduat_pcb1_manifest_t manifest;
amduat_octets_t slot_data;
amduat_fcs1_error_t err;
amduat_pcb1_error_t pcb_err;
amduat_reference_t stack[16];
memset(&desc, 0, sizeof(desc));
err = amduat_fcs1_decode_v1(bytes, &desc);
if (err != AMDUAT_FCS1_OK) {
return err;
}
if (resolver == NULL ||
!resolver(resolver_ctx, desc.parameter_block, &pcb_bytes)) {
amduat_fcs1_descriptor_free(&desc);
return AMDUAT_FCS1_ERR_PARAMETER_FORMAT;
}
pcb_err = amduat_pcb1_decode_v1(pcb_bytes, &manifest, &slot_data);
amduat_octets_free(&pcb_bytes);
if (pcb_err != AMDUAT_PCB1_OK) {
amduat_fcs1_descriptor_free(&desc);
return AMDUAT_FCS1_ERR_PARAMETER_FORMAT;
}
(void)slot_data;
if (desc.arity != manifest.slot_count) {
amduat_fcs1_descriptor_free(&desc);
return AMDUAT_FCS1_ERR_PCB_ARITY_MISMATCH;
}
stack[0] = desc.function_ptr;
err = amduat_fcs1_validate_cycle(desc.function_ptr,
resolver,
resolver_ctx,
stack,
1u,
sizeof(stack) / sizeof(stack[0]));
amduat_fcs1_descriptor_free(&desc);
return err;
}

850
src/near_core/enc/fer1_receipt.c
View file

@ -8,19 +8,6 @@
#include <stdlib.h>
#include <string.h>
enum {
AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT = 0x0001u,
AMDUAT_FER1_TLV_RUN_ID = 0x0002u,
AMDUAT_FER1_TLV_LOGS = 0x0003u,
AMDUAT_FER1_TLV_LIMITS = 0x0004u,
AMDUAT_FER1_TLV_DETERMINISM = 0x0005u,
AMDUAT_FER1_TLV_SIGNATURE = 0x0006u
};
enum {
AMDUAT_FER1_LOGS_MAX = 64
};
typedef struct {
const uint8_t *data;
size_t len;
@ -190,14 +177,6 @@ static bool amduat_write_encoded_ref(uint8_t *buffer,
return true;
}
static int amduat_log_entry_cmp(const amduat_fer1_log_entry_t *a,
const amduat_fer1_log_entry_t *b) {
if (a->kind != b->kind) {
return a->kind < b->kind ? -1 : 1;
}
return amduat_reference_cmp(a->log_ref, b->log_ref);
}
static bool amduat_read_encoded_ref(amduat_cursor_t *cur,
amduat_reference_t *out_ref) {
uint32_t ref_len_u32;
@ -252,42 +231,10 @@ void amduat_enc_fer1_receipt_free(amduat_fer1_receipt_t *receipt) {
free(receipt->parity);
}
if (receipt->has_executor_fingerprint_ref) {
amduat_reference_free(&receipt->executor_fingerprint_ref);
}
if (receipt->has_run_id) {
amduat_octets_free(&receipt->run_id);
}
if (receipt->logs != NULL) {
for (i = 0; i < receipt->logs_len; ++i) {
amduat_reference_free(&receipt->logs[i].log_ref);
amduat_octets_free(&receipt->logs[i].sha256);
}
free(receipt->logs);
}
if (receipt->has_rng_seed) {
amduat_octets_free(&receipt->rng_seed);
}
if (receipt->has_signature) {
amduat_octets_free(&receipt->signature);
}
receipt->executor_refs = NULL;
receipt->executor_refs_len = 0;
receipt->parity = NULL;
receipt->parity_len = 0;
receipt->has_executor_fingerprint_ref = false;
receipt->has_run_id = false;
receipt->run_id = amduat_octets(NULL, 0u);
receipt->has_limits = false;
receipt->logs = NULL;
receipt->logs_len = 0;
receipt->has_determinism = false;
receipt->determinism_level = 0u;
receipt->has_rng_seed = false;
receipt->rng_seed = amduat_octets(NULL, 0u);
receipt->has_signature = false;
receipt->signature = amduat_octets(NULL, 0u);
}
bool amduat_enc_fer1_receipt_encode_v1(
@ -497,431 +444,6 @@ bool amduat_enc_fer1_receipt_encode_v1(
return true;
}
bool amduat_enc_fer1_receipt_encode_v1_1(
const amduat_fer1_receipt_t *receipt,
amduat_octets_t *out_bytes) {
size_t total_len = 0;
size_t offset = 0;
size_t ext_len = 0;
uint8_t *buffer;
size_t i;
if (receipt == NULL || out_bytes == NULL) {
return false;
}
out_bytes->data = NULL;
out_bytes->len = 0;
if (receipt->fer1_version != AMDUAT_FER1_VERSION_1_1) {
return false;
}
if (receipt->evaluator_id.len != 0 && receipt->evaluator_id.data == NULL) {
return false;
}
if (receipt->executor_refs_len != 0 && receipt->executor_refs == NULL) {
return false;
}
if (receipt->parity_len != 0 && receipt->parity == NULL) {
return false;
}
if (receipt->executor_refs_len != receipt->parity_len) {
return false;
}
if (receipt->evaluator_id.len > UINT32_MAX) {
return false;
}
if (receipt->started_at > receipt->completed_at) {
return false;
}
if (receipt->has_run_id &&
receipt->run_id.len != 0 &&
receipt->run_id.data == NULL) {
return false;
}
if (receipt->has_run_id && receipt->run_id.len > UINT32_MAX) {
return false;
}
if (receipt->has_rng_seed &&
receipt->rng_seed.len != 0 &&
receipt->rng_seed.data == NULL) {
return false;
}
if (receipt->has_rng_seed && receipt->rng_seed.len > UINT32_MAX) {
return false;
}
if (receipt->has_signature &&
receipt->signature.len != 0 &&
receipt->signature.data == NULL) {
return false;
}
if (receipt->has_signature && receipt->signature.len > UINT32_MAX) {
return false;
}
if (receipt->logs_len != 0 && receipt->logs == NULL) {
return false;
}
if (receipt->logs_len > AMDUAT_FER1_LOGS_MAX) {
return false;
}
for (i = 1; i < receipt->executor_refs_len; ++i) {
if (amduat_reference_cmp(receipt->executor_refs[i - 1],
receipt->executor_refs[i]) > 0) {
return false;
}
}
for (i = 0; i < receipt->parity_len; ++i) {
if (!amduat_reference_eq(receipt->parity[i].executor_ref,
receipt->executor_refs[i])) {
return false;
}
if (i > 0 &&
amduat_reference_cmp(receipt->parity[i - 1].executor_ref,
receipt->parity[i].executor_ref) > 0) {
return false;
}
}
for (i = 0; i < receipt->logs_len; ++i) {
const amduat_fer1_log_entry_t *entry = &receipt->logs[i];
if (entry->sha256.len != 0 && entry->sha256.data == NULL) {
return false;
}
if (entry->sha256.len > UINT32_MAX) {
return false;
}
if (i > 0 &&
amduat_log_entry_cmp(&receipt->logs[i - 1], entry) > 0) {
return false;
}
}
{
size_t enc_len;
if (!amduat_add_size(&total_len, 2) ||
!amduat_encoded_ref_len(receipt->function_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_encoded_ref_len(receipt->input_manifest_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_encoded_ref_len(receipt->environment_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4 + receipt->evaluator_id.len)) {
return false;
}
{
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->output_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4)) {
return false;
}
for (i = 0; i < receipt->executor_refs_len; ++i) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->executor_refs[i], &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4)) {
return false;
}
for (i = 0; i < receipt->parity_len; ++i) {
size_t enc_len;
const amduat_fer1_parity_entry_t *entry = &receipt->parity[i];
if (entry->parity_digest.len != 0 &&
entry->parity_digest.data == NULL) {
return false;
}
if (entry->parity_digest.len > UINT32_MAX) {
return false;
}
if (!amduat_reference_eq(entry->output_ref, receipt->output_ref)) {
return false;
}
if (!amduat_encoded_ref_len(entry->executor_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_encoded_ref_len(entry->output_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len) ||
!amduat_add_size(&total_len, 1)) {
return false;
}
if (entry->has_sbom_ref) {
if (!amduat_encoded_ref_len(entry->sbom_ref, &enc_len) ||
!amduat_add_size(&total_len, enc_len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 4 + entry->parity_digest.len)) {
return false;
}
}
if (!amduat_add_size(&total_len, 8 + 8)) {
return false;
}
if (receipt->has_executor_fingerprint_ref) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->executor_fingerprint_ref, &enc_len)) {
return false;
}
if (!amduat_add_size(&ext_len, 2 + 4 + enc_len)) {
return false;
}
}
if (receipt->has_run_id) {
if (!amduat_add_size(&ext_len, 2 + 4 + 4 + receipt->run_id.len)) {
return false;
}
}
if (receipt->has_limits) {
if (!amduat_add_size(&ext_len, 2 + 4 + (5u * 8u))) {
return false;
}
}
if (receipt->logs_len != 0) {
size_t logs_len = 4;
for (i = 0; i < receipt->logs_len; ++i) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->logs[i].log_ref, &enc_len)) {
return false;
}
if (!amduat_add_size(&logs_len, 4) ||
!amduat_add_size(&logs_len, enc_len) ||
!amduat_add_size(&logs_len, 4 + receipt->logs[i].sha256.len)) {
return false;
}
}
if (!amduat_add_size(&ext_len, 2 + 4 + logs_len)) {
return false;
}
}
if (receipt->has_determinism) {
if (!amduat_add_size(&ext_len,
2 + 4 + 1 + 4 + receipt->rng_seed.len)) {
return false;
}
}
if (receipt->has_signature) {
if (!amduat_add_size(&ext_len, 2 + 4 + receipt->signature.len)) {
return false;
}
}
if (ext_len > UINT32_MAX) {
return false;
}
if (!amduat_add_size(&total_len, 4 + ext_len)) {
return false;
}
buffer = (uint8_t *)malloc(total_len);
if (buffer == NULL) {
return false;
}
amduat_store_u16_be(buffer + offset, receipt->fer1_version);
offset += 2;
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->function_ref) ||
!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->input_manifest_ref) ||
!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->environment_ref)) {
free(buffer);
return false;
}
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->evaluator_id.len);
offset += 4;
if (receipt->evaluator_id.len != 0) {
memcpy(buffer + offset, receipt->evaluator_id.data,
receipt->evaluator_id.len);
offset += receipt->evaluator_id.len;
}
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->output_ref)) {
free(buffer);
return false;
}
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->executor_refs_len);
offset += 4;
for (i = 0; i < receipt->executor_refs_len; ++i) {
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->executor_refs[i])) {
free(buffer);
return false;
}
}
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->parity_len);
offset += 4;
for (i = 0; i < receipt->parity_len; ++i) {
const amduat_fer1_parity_entry_t *entry = &receipt->parity[i];
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->executor_ref) ||
!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->output_ref)) {
free(buffer);
return false;
}
buffer[offset++] = entry->has_sbom_ref ? 0x01u : 0x00u;
if (entry->has_sbom_ref) {
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->sbom_ref)) {
free(buffer);
return false;
}
}
amduat_store_u32_be(buffer + offset, (uint32_t)entry->parity_digest.len);
offset += 4;
if (entry->parity_digest.len != 0) {
memcpy(buffer + offset, entry->parity_digest.data,
entry->parity_digest.len);
offset += entry->parity_digest.len;
}
}
amduat_store_u64_be(buffer + offset, receipt->started_at);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->completed_at);
offset += 8;
amduat_store_u32_be(buffer + offset, (uint32_t)ext_len);
offset += 4;
if (receipt->has_executor_fingerprint_ref) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->executor_fingerprint_ref, &enc_len)) {
free(buffer);
return false;
}
amduat_store_u16_be(buffer + offset,
AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT);
offset += 2;
amduat_store_u32_be(buffer + offset, (uint32_t)enc_len);
offset += 4;
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
receipt->executor_fingerprint_ref)) {
free(buffer);
return false;
}
}
if (receipt->has_run_id) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_RUN_ID);
offset += 2;
amduat_store_u32_be(buffer + offset,
(uint32_t)(4u + receipt->run_id.len));
offset += 4;
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->run_id.len);
offset += 4;
if (receipt->run_id.len != 0) {
memcpy(buffer + offset, receipt->run_id.data, receipt->run_id.len);
offset += receipt->run_id.len;
}
}
if (receipt->logs_len != 0) {
size_t logs_len = 4;
for (i = 0; i < receipt->logs_len; ++i) {
size_t enc_len;
if (!amduat_encoded_ref_len(receipt->logs[i].log_ref, &enc_len)) {
free(buffer);
return false;
}
logs_len += 4 + enc_len + 4 + receipt->logs[i].sha256.len;
}
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_LOGS);
offset += 2;
amduat_store_u32_be(buffer + offset, (uint32_t)logs_len);
offset += 4;
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->logs_len);
offset += 4;
for (i = 0; i < receipt->logs_len; ++i) {
const amduat_fer1_log_entry_t *entry = &receipt->logs[i];
amduat_store_u32_be(buffer + offset, entry->kind);
offset += 4;
if (!amduat_write_encoded_ref(buffer, total_len, &offset,
entry->log_ref)) {
free(buffer);
return false;
}
amduat_store_u32_be(buffer + offset, (uint32_t)entry->sha256.len);
offset += 4;
if (entry->sha256.len != 0) {
memcpy(buffer + offset, entry->sha256.data, entry->sha256.len);
offset += entry->sha256.len;
}
}
}
if (receipt->has_limits) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_LIMITS);
offset += 2;
amduat_store_u32_be(buffer + offset, (uint32_t)(5u * 8u));
offset += 4;
amduat_store_u64_be(buffer + offset, receipt->limits.cpu_ms);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.wall_ms);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.max_rss_kib);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.io_reads);
offset += 8;
amduat_store_u64_be(buffer + offset, receipt->limits.io_writes);
offset += 8;
}
if (receipt->has_determinism) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_DETERMINISM);
offset += 2;
amduat_store_u32_be(buffer + offset,
(uint32_t)(1u + 4u + receipt->rng_seed.len));
offset += 4;
buffer[offset++] = receipt->determinism_level;
amduat_store_u32_be(buffer + offset, (uint32_t)receipt->rng_seed.len);
offset += 4;
if (receipt->rng_seed.len != 0) {
memcpy(buffer + offset, receipt->rng_seed.data, receipt->rng_seed.len);
offset += receipt->rng_seed.len;
}
}
if (receipt->has_signature) {
amduat_store_u16_be(buffer + offset, AMDUAT_FER1_TLV_SIGNATURE);
offset += 2;
amduat_store_u32_be(buffer + offset,
(uint32_t)receipt->signature.len);
offset += 4;
if (receipt->signature.len != 0) {
memcpy(buffer + offset, receipt->signature.data, receipt->signature.len);
offset += receipt->signature.len;
}
}
out_bytes->data = buffer;
out_bytes->len = total_len;
return true;
}
bool amduat_enc_fer1_receipt_decode_v1(
amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt) {
@ -1105,375 +627,3 @@ bool amduat_enc_fer1_receipt_decode_v1(
return true;
}
bool amduat_enc_fer1_receipt_decode_v1_1(
amduat_octets_t bytes,
amduat_fer1_receipt_t *out_receipt) {
amduat_cursor_t cur;
uint16_t fer1_version;
uint32_t len_u32;
uint32_t executor_count;
uint32_t parity_count;
uint32_t ext_len;
size_t i;
if (out_receipt == NULL) {
return false;
}
if (bytes.len != 0 && bytes.data == NULL) {
return false;
}
memset(out_receipt, 0, sizeof(*out_receipt));
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0;
if (!amduat_read_u16(&cur, &fer1_version)) {
return false;
}
if (fer1_version != AMDUAT_FER1_VERSION_1_1) {
return false;
}
out_receipt->fer1_version = fer1_version;
if (!amduat_read_encoded_ref(&cur, &out_receipt->function_ref) ||
!amduat_read_encoded_ref(&cur, &out_receipt->input_manifest_ref) ||
!amduat_read_encoded_ref(&cur, &out_receipt->environment_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u32(&cur, &len_u32)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (cur.len - cur.offset < len_u32) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (len_u32 != 0) {
amduat_octets_t src = amduat_octets(cur.data + cur.offset, len_u32);
if (!amduat_octets_clone(src, &out_receipt->evaluator_id)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
cur.offset += len_u32;
}
if (!amduat_read_encoded_ref(&cur, &out_receipt->output_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u32(&cur, &executor_count)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (executor_count != 0) {
if (executor_count > SIZE_MAX / sizeof(amduat_reference_t)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->executor_refs =
(amduat_reference_t *)calloc(executor_count,
sizeof(amduat_reference_t));
if (out_receipt->executor_refs == NULL) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->executor_refs_len = executor_count;
for (i = 0; i < out_receipt->executor_refs_len; ++i) {
if (!amduat_read_encoded_ref(&cur, &out_receipt->executor_refs[i])) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
for (i = 1; i < out_receipt->executor_refs_len; ++i) {
if (amduat_reference_cmp(out_receipt->executor_refs[i - 1],
out_receipt->executor_refs[i]) > 0) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (!amduat_read_u32(&cur, &parity_count)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (parity_count > SIZE_MAX / sizeof(amduat_fer1_parity_entry_t)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (parity_count != 0) {
out_receipt->parity =
(amduat_fer1_parity_entry_t *)calloc(parity_count,
sizeof(amduat_fer1_parity_entry_t));
if (out_receipt->parity == NULL) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->parity_len = parity_count;
for (i = 0; i < out_receipt->parity_len; ++i) {
amduat_fer1_parity_entry_t *entry = &out_receipt->parity[i];
uint8_t has_sbom_ref;
if (!amduat_read_encoded_ref(&cur, &entry->executor_ref) ||
!amduat_read_encoded_ref(&cur, &entry->output_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u8(&cur, &has_sbom_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (has_sbom_ref != 0x00u && has_sbom_ref != 0x01u) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
entry->has_sbom_ref = has_sbom_ref == 0x01u;
if (entry->has_sbom_ref) {
if (!amduat_read_encoded_ref(&cur, &entry->sbom_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (!amduat_read_u32(&cur, &len_u32)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (cur.len - cur.offset < len_u32) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (len_u32 != 0) {
amduat_octets_t src = amduat_octets(cur.data + cur.offset, len_u32);
if (!amduat_octets_clone(src, &entry->parity_digest)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
cur.offset += len_u32;
}
}
for (i = 0; i < out_receipt->parity_len; ++i) {
if (!amduat_reference_eq(out_receipt->parity[i].executor_ref,
out_receipt->executor_refs[i])) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (i > 0 &&
amduat_reference_cmp(out_receipt->parity[i - 1].executor_ref,
out_receipt->parity[i].executor_ref) > 0) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_reference_eq(out_receipt->parity[i].output_ref,
out_receipt->output_ref)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (!amduat_read_u64(&cur, &out_receipt->started_at) ||
!amduat_read_u64(&cur, &out_receipt->completed_at)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (out_receipt->started_at > out_receipt->completed_at) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (!amduat_read_u32(&cur, &ext_len)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (cur.len - cur.offset < ext_len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
{
size_t end_offset = cur.offset + ext_len;
while (cur.offset < end_offset) {
uint16_t tag;
uint32_t tag_len;
amduat_cursor_t tlv_cur;
if (!amduat_read_u16(&cur, &tag) ||
!amduat_read_u32(&cur, &tag_len)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (tag_len > end_offset - cur.offset) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
tlv_cur.data = cur.data + cur.offset;
tlv_cur.len = tag_len;
tlv_cur.offset = 0;
switch (tag) {
case AMDUAT_FER1_TLV_EXECUTOR_FINGERPRINT:
if (out_receipt->has_executor_fingerprint_ref ||
!amduat_read_encoded_ref(&tlv_cur,
&out_receipt->executor_fingerprint_ref) ||
tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->has_executor_fingerprint_ref = true;
break;
case AMDUAT_FER1_TLV_RUN_ID: {
uint32_t run_len;
if (out_receipt->has_run_id ||
!amduat_read_u32(&tlv_cur, &run_len) ||
tlv_cur.len - tlv_cur.offset < run_len ||
tlv_cur.offset + run_len != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (run_len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, run_len);
if (!amduat_octets_clone(src, &out_receipt->run_id)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->has_run_id = true;
break;
}
case AMDUAT_FER1_TLV_LOGS: {
uint32_t count;
if (!amduat_read_u32(&tlv_cur, &count) ||
count > AMDUAT_FER1_LOGS_MAX ||
count > SIZE_MAX / sizeof(amduat_fer1_log_entry_t)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (count != 0) {
out_receipt->logs =
(amduat_fer1_log_entry_t *)calloc(
count, sizeof(amduat_fer1_log_entry_t));
if (out_receipt->logs == NULL) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
out_receipt->logs_len = count;
for (i = 0; i < out_receipt->logs_len; ++i) {
amduat_fer1_log_entry_t *entry = &out_receipt->logs[i];
uint32_t sha_len;
if (!amduat_read_u32(&tlv_cur, &entry->kind) ||
!amduat_read_encoded_ref(&tlv_cur, &entry->log_ref) ||
!amduat_read_u32(&tlv_cur, &sha_len) ||
tlv_cur.len - tlv_cur.offset < sha_len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (sha_len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, sha_len);
if (!amduat_octets_clone(src, &entry->sha256)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
tlv_cur.offset += sha_len;
if (i > 0 &&
amduat_log_entry_cmp(&out_receipt->logs[i - 1],
entry) > 0) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
break;
}
case AMDUAT_FER1_TLV_LIMITS:
if (out_receipt->has_limits ||
tlv_cur.len != 5u * 8u ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.cpu_ms) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.wall_ms) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.max_rss_kib) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.io_reads) ||
!amduat_read_u64(&tlv_cur, &out_receipt->limits.io_writes) ||
tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->has_limits = true;
break;
case AMDUAT_FER1_TLV_DETERMINISM: {
uint8_t level;
uint32_t seed_len;
if (out_receipt->has_determinism ||
!amduat_read_u8(&tlv_cur, &level) ||
!amduat_read_u32(&tlv_cur, &seed_len) ||
tlv_cur.len - tlv_cur.offset < seed_len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (seed_len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, seed_len);
if (!amduat_octets_clone(src, &out_receipt->rng_seed)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->has_rng_seed = true;
}
tlv_cur.offset += seed_len;
if (tlv_cur.offset != tlv_cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
out_receipt->determinism_level = level;
out_receipt->has_determinism = true;
break;
}
case AMDUAT_FER1_TLV_SIGNATURE: {
if (out_receipt->has_signature) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
if (tlv_cur.len != 0) {
amduat_octets_t src =
amduat_octets(tlv_cur.data + tlv_cur.offset, tlv_cur.len);
if (!amduat_octets_clone(src, &out_receipt->signature)) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
tlv_cur.offset = tlv_cur.len;
out_receipt->has_signature = true;
break;
}
default:
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
cur.offset += tag_len;
}
if (cur.offset != end_offset) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
}
if (cur.offset != cur.len) {
amduat_enc_fer1_receipt_free(out_receipt);
return false;
}
return true;
}

256
src/near_core/enc/pcb1.c
View file

@ -1,256 +0,0 @@
#include "amduat/enc/pcb1.h"
#include <string.h>
enum {
AMDUAT_PCB1_HEADER_LEN = 7,
AMDUAT_PCB1_TAG_MANIFEST = 0x50,
AMDUAT_PCB1_TAG_DATA = 0x51,
AMDUAT_PCB1_DIGEST_LEN = 32
};
typedef struct {
const uint8_t *data;
size_t len;
size_t offset;
} amduat_pcb1_cursor_t;
static bool amduat_pcb1_read_varint(amduat_pcb1_cursor_t *cur,
uint64_t *out,
bool *out_minimal) {
uint64_t value = 0u;
uint32_t shift = 0u;
size_t start;
size_t bytes = 0u;
if (cur == NULL || out == NULL || out_minimal == NULL) {
return false;
}
start = cur->offset;
while (cur->offset < cur->len) {
uint8_t byte = cur->data[cur->offset++];
value |= ((uint64_t)(byte & 0x7fu)) << shift;
bytes++;
if ((byte & 0x80u) == 0u) {
break;
}
shift += 7u;
if (shift >= 64u) {
return false;
}
}
if (bytes == 0u || cur->offset > cur->len) {
return false;
}
{
size_t minimal_len = 1u;
uint64_t tmp = value;
while (tmp >= 0x80u) {
tmp >>= 7u;
minimal_len++;
}
*out_minimal = (minimal_len == bytes);
}
*out = value;
return true;
}
static bool amduat_pcb1_read_bytes(amduat_pcb1_cursor_t *cur,
amduat_octets_t *out,
bool *out_minimal) {
uint64_t len;
bool minimal;
if (!amduat_pcb1_read_varint(cur, &len, &minimal)) {
return false;
}
if (len > SIZE_MAX || cur->len - cur->offset < (size_t)len) {
return false;
}
*out = amduat_octets(cur->data + cur->offset, (size_t)len);
cur->offset += (size_t)len;
*out_minimal = minimal;
return true;
}
static amduat_pcb1_error_t amduat_pcb1_parse_manifest(
amduat_octets_t bytes,
amduat_pcb1_manifest_t *out_manifest) {
amduat_pcb1_cursor_t cur;
uint64_t slot_count = 0u;
uint64_t last_index = 0u;
bool minimal;
if (out_manifest == NULL) {
return AMDUAT_PCB1_ERR_INVALID;
}
out_manifest->slot_count = 0u;
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = 0u;
if (!amduat_pcb1_read_varint(&cur, &slot_count, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
if (slot_count > UINT32_MAX) {
return AMDUAT_PCB1_ERR_INVALID;
}
for (uint64_t i = 0u; i < slot_count; ++i) {
uint64_t index;
uint64_t name_len;
uint64_t type;
uint64_t digest_len;
amduat_octets_t name;
amduat_octets_t digest;
if (!amduat_pcb1_read_varint(&cur, &index, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
if (i > 0u && index <= last_index) {
return AMDUAT_PCB1_ERR_MANIFEST_ORDER;
}
last_index = index;
if (!amduat_pcb1_read_varint(&cur, &name_len, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
if (name_len > SIZE_MAX || cur.len - cur.offset < (size_t)name_len) {
return AMDUAT_PCB1_ERR_INVALID;
}
name = amduat_octets(cur.data + cur.offset, (size_t)name_len);
(void)name;
cur.offset += (size_t)name_len;
if (!amduat_pcb1_read_varint(&cur, &type, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
(void)type;
if (!amduat_pcb1_read_varint(&cur, &digest_len, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
if (digest_len != AMDUAT_PCB1_DIGEST_LEN) {
return AMDUAT_PCB1_ERR_DIGEST_MISMATCH;
}
if (cur.len - cur.offset < AMDUAT_PCB1_DIGEST_LEN) {
return AMDUAT_PCB1_ERR_INVALID;
}
digest = amduat_octets(cur.data + cur.offset, AMDUAT_PCB1_DIGEST_LEN);
(void)digest;
cur.offset += AMDUAT_PCB1_DIGEST_LEN;
}
if (cur.offset != cur.len) {
return AMDUAT_PCB1_ERR_TRAILING_BYTES;
}
out_manifest->slot_count = (uint32_t)slot_count;
return AMDUAT_PCB1_OK;
}
amduat_pcb1_error_t amduat_pcb1_decode_v1(amduat_octets_t bytes,
amduat_pcb1_manifest_t *out_manifest,
amduat_octets_t *out_slot_data) {
static const uint8_t magic[4] = {'P', 'C', 'B', '1'};
amduat_pcb1_cursor_t cur;
uint8_t version;
uint8_t flags;
uint8_t rsv;
uint8_t tag;
uint8_t last_tag = 0u;
bool minimal;
bool seen_manifest = false;
bool seen_data = false;
amduat_octets_t manifest_bytes = amduat_octets(NULL, 0u);
amduat_octets_t slot_data = amduat_octets(NULL, 0u);
if (out_manifest == NULL || out_slot_data == NULL) {
return AMDUAT_PCB1_ERR_INVALID;
}
out_manifest->slot_count = 0u;
*out_slot_data = amduat_octets(NULL, 0u);
if (bytes.len < AMDUAT_PCB1_HEADER_LEN || bytes.data == NULL) {
return AMDUAT_PCB1_ERR_HEADER_INVALID;
}
if (memcmp(bytes.data, magic, sizeof(magic)) != 0) {
return AMDUAT_PCB1_ERR_HEADER_INVALID;
}
version = bytes.data[4];
flags = bytes.data[5];
rsv = bytes.data[6];
if (version != 0x01u || flags != 0x00u || rsv != 0x00u) {
return AMDUAT_PCB1_ERR_HEADER_INVALID;
}
cur.data = bytes.data;
cur.len = bytes.len;
cur.offset = AMDUAT_PCB1_HEADER_LEN;
while (cur.offset < cur.len) {
tag = cur.data[cur.offset++];
if (tag <= last_tag) {
return AMDUAT_PCB1_ERR_TAG_ORDER;
}
last_tag = tag;
if (tag == AMDUAT_PCB1_TAG_MANIFEST) {
if (seen_manifest) {
return AMDUAT_PCB1_ERR_TAG_ORDER;
}
if (!amduat_pcb1_read_bytes(&cur, &manifest_bytes, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
seen_manifest = true;
} else if (tag == AMDUAT_PCB1_TAG_DATA) {
if (seen_data) {
return AMDUAT_PCB1_ERR_TAG_ORDER;
}
if (!amduat_pcb1_read_bytes(&cur, &slot_data, &minimal)) {
return AMDUAT_PCB1_ERR_INVALID;
}
if (!minimal) {
return AMDUAT_PCB1_ERR_VARINT_NON_MINIMAL;
}
seen_data = true;
} else {
return AMDUAT_PCB1_ERR_UNKNOWN_TAG;
}
}
if (!seen_manifest || !seen_data) {
return AMDUAT_PCB1_ERR_TAG_ORDER;
}
if (cur.offset != cur.len) {
return AMDUAT_PCB1_ERR_TRAILING_BYTES;
}
{
amduat_pcb1_error_t err =
amduat_pcb1_parse_manifest(manifest_bytes, out_manifest);
if (err != AMDUAT_PCB1_OK) {
return err;
}
}
*out_slot_data = slot_data;
return AMDUAT_PCB1_OK;
}

Some files were not shown because too many files have changed in this diff Show more