diff --git a/.agents/skills/lxmf-update/SKILL.md b/.agents/skills/lxmf-update/SKILL.md deleted file mode 100644 index 20396c3..0000000 --- a/.agents/skills/lxmf-update/SKILL.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -name: lxmf-update -description: Check upstream LXMF and RNS implementations for changes affecting the LXMF specification, run local verifier tools, and propose spec updates with citations. ---- - -# LXMF update skill - -Use this skill when asked to check whether the LXMF specification is current with upstream LXMF or RNS. - -## Sources of truth - -Prefer, in order: - -1. Upstream LXMF source code at a pinned release or commit. -2. Upstream RNS source code where LXMF behavior depends on Reticulum behavior. -3. Reproducible local traces and test vectors. -4. Upstream prose documentation, only when consistent with source behavior. - -## Required workflow - -1. Determine the currently pinned upstream LXMF and RNS versions from `SOURCES.md`. -2. Check whether newer upstream releases or commits exist. -3. Identify changed files relevant to: - - message packing - - fields and flags - - destination handling - - identity/signature behavior - - encryption behavior - - propagation - - delivery semantics - - storage format if externally observable -4. Run the verifier tools in `tools/`. -5. Compare observed behavior against `SPEC.md`. -6. Produce a report with: - - changed upstream files - - affected specification sections - - proposed edits - - test-vector impact - - unresolved ambiguities -7. Do not modify `SPEC.md` unless explicitly asked. diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index d4c9224..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,16 +0,0 @@ -# AGENTS.md - -## Repository purpose - -This repository attempts to define an implementation-derived specification for LXMF. - -Do not treat upstream prose documentation as authoritative when it conflicts with observed source behavior. Prefer source code, test vectors, and reproducible traces. - -## Required discipline - -- Distinguish normative behavior from observed implementation behavior. -- Cite exact upstream files, functions, classes, constants, and version tags. -- Do not invent protocol rules. -- When behavior is unclear, mark it as "undetermined" rather than guessing. -- Prefer Perl tools for extraction, comparison, and report generation. -- Keep generated examples reproducible from scripts under tools/. diff --git a/ChatGPT_1.md b/ChatGPT_1.md deleted file mode 100644 index c40e197..0000000 --- a/ChatGPT_1.md +++ /dev/null @@ -1,174 +0,0 @@ -Yes. For Codex, the closest equivalents are: - -1. **`AGENTS.md`** for persistent project-wide instructions. -2. **Codex Skills using `SKILL.md`** for reusable workflows, very similar in spirit to the Claude Code file you linked. -3. Optionally **`PLANS.md` / ExecPlan-style documents** for long-running specification work. - -The important correction is that Codex does now support **Skills with `SKILL.md`**, not just `AGENTS.md`. OpenAI’s Codex documentation says a skill is a directory containing a required `SKILL.md` file, with optional `scripts/`, `references/`, `assets/`, and related support files. Codex initially sees the skill name, description, and path, then loads the full `SKILL.md` only when it decides that skill applies. ([OpenAI Developers][1]) - -So, yes: you can do essentially the same thing that `thatSFguy` is doing, but I would structure it slightly differently for LXMF. - -## Recommended Codex layout - -For an LXMF specification repository, I would use something like: - -```text -lxmf-specification/ -├── AGENTS.md -├── SPEC.md -├── NOTES.md -├── SOURCES.md -├── tools/ -│ ├── extract_lxmf_constants.pl -│ ├── compare_upstream_lxmf.pl -│ ├── verify_examples.pl -│ └── make_test_vectors.pl -├── examples/ -│ ├── lxmf_message_minimal.hex -│ ├── lxmf_message_signed.hex -│ └── lxmf_propagation_example.hex -├── references/ -│ ├── lxmf_source_map.md -│ ├── rns_dependency_map.md -│ └── terminology.md -└── .agents/ - └── skills/ - └── lxmf-update/ - └── SKILL.md -``` - -## Role of each file - -### `AGENTS.md` - -Use this for always-on repository rules. Codex reads `AGENTS.md` before doing work, and the docs describe it as the mechanism for project-specific persistent guidance. ([OpenAI Developers][2]) - -Example use: - -```markdown -# AGENTS.md - -## Repository purpose - -This repository attempts to define an implementation-derived specification for LXMF. - -Do not treat upstream prose documentation as authoritative when it conflicts with observed source behavior. Prefer source code, test vectors, and reproducible traces. - -## Required discipline - -- Distinguish normative behavior from observed implementation behavior. -- Cite exact upstream files, functions, classes, constants, and version tags. -- Do not invent protocol rules. -- When behavior is unclear, mark it as "undetermined" rather than guessing. -- Prefer Perl tools for extraction, comparison, and report generation. -- Keep generated examples reproducible from scripts under tools/. -``` - -### `.agents/skills/lxmf-update/SKILL.md` - -Use this for the repeatable workflow: “check upstream LXMF/RNS, compare against our spec, update citations, run verifiers, propose diffs.” - -That is the closest Codex analog to the Claude Code `rns-update/SKILL.md` you found. - -A minimal Codex skill might look like: - -```markdown ---- -name: lxmf-update -description: Check upstream LXMF and RNS implementations for changes affecting the LXMF specification, run local verifier tools, and propose spec updates with citations. ---- - -# LXMF update skill - -Use this skill when asked to check whether the LXMF specification is current with upstream LXMF or RNS. - -## Sources of truth - -Prefer, in order: - -1. Upstream LXMF source code at a pinned release or commit. -2. Upstream RNS source code where LXMF behavior depends on Reticulum behavior. -3. Reproducible local traces and test vectors. -4. Upstream prose documentation, only when consistent with source behavior. - -## Required workflow - -1. Determine the currently pinned upstream LXMF and RNS versions from `SOURCES.md`. -2. Check whether newer upstream releases or commits exist. -3. Identify changed files relevant to: - - message packing - - fields and flags - - destination handling - - identity/signature behavior - - encryption behavior - - propagation - - delivery semantics - - storage format if externally observable -4. Run the verifier tools in `tools/`. -5. Compare observed behavior against `SPEC.md`. -6. Produce a report with: - - changed upstream files - - affected specification sections - - proposed edits - - test-vector impact - - unresolved ambiguities -7. Do not modify `SPEC.md` unless explicitly asked. -``` - -Codex supports skills in the CLI, IDE extension, and Codex app, and the documented repository skill location is `.agents/skills` under the current directory or repository root. ([OpenAI Developers][1]) - -### `PLANS.md` - -For your use case, I would also add a planning document, because “derive a protocol specification from implementation” is not a one-shot coding task. OpenAI’s Codex cookbook describes using `PLANS.md` / ExecPlans for long-running, multi-stage work, with `AGENTS.md` telling Codex when to use it. ([OpenAI Developers][3]) - -For LXMF, `PLANS.md` could define the required sections of each investigation: - -```markdown -# LXMF Specification Investigation Plan - -Each investigation must include: - -1. Question being answered. -2. Upstream files inspected. -3. Exact version or commit. -4. Relevant classes/functions/constants. -5. Observed behavior. -6. Whether behavior is normative, inferred, or unresolved. -7. Test vectors needed. -8. SPEC.md sections affected. -9. Open questions. -``` - -## My recommendation - -For your goal, I would not start by asking Codex to “write the LXMF specification.” That is too broad and likely to produce confident prose ahead of evidence. - -I would instead give Codex narrow tasks such as: - -```text -Using the lxmf-update skill, inspect upstream LXMF and determine the exact serialized structure of an LXMessage. Do not edit SPEC.md. Produce a report listing every field, its source-code origin, and whether its behavior is confirmed by a test vector. -``` - -Then: - -```text -Create Perl tooling under tools/ that extracts or verifies the LXMF message packing behavior described in the report. Generate at least one reproducible test vector. -``` - -Then: - -```text -Update SPEC.md section "LXMessage Wire Format" using only behavior confirmed by source inspection or verifier output. Mark all unverified behavior as unresolved. -``` - -## Bottom line - -Yes, you can accomplish the same kind of workflow with Codex. In fact, Codex gives you both mechanisms: - -`AGENTS.md` for standing repo rules, and `SKILL.md` under `.agents/skills/...` for a reusable “LXMF update / spec maintenance” workflow. OpenAI’s Codex docs explicitly describe customization as a combination of `AGENTS.md`, skills, MCP, memories, and subagents; for your use case, `AGENTS.md` plus a `SKILL.md` skill is the right starting point. ([OpenAI Developers][4]) - -[1]: https://developers.openai.com/codex/skills "Agent Skills – Codex | OpenAI Developers" -[2]: https://developers.openai.com/codex/guides/agents-md "Custom instructions with AGENTS.md – Codex | OpenAI Developers" -[3]: https://developers.openai.com/cookbook/articles/codex_exec_plans "Using PLANS.md for multi-hour problem solving" -[4]: https://developers.openai.com/codex/concepts/customization "Customization – Codex | OpenAI Developers" - diff --git a/Codex_1.md b/Codex_1.md deleted file mode 100644 index 049676a..0000000 --- a/Codex_1.md +++ /dev/null @@ -1,8 +0,0 @@ -This directory: /usr/local/src/reticulum/LXMF -contains my GitHub fork (https://github.com/jlpoolen/LXMF) of Mark Qvist's: https://github.com/markqvist/LXMF which, incidentally, is a public mirror of some git server served up through Reticulum -- the source is not provided. - -I created an empty project in my Forgejo server: LXMF-specification at https://salemdata.net/repo/jlpoole/LMXF-specification -and I added my Forgejo instance as a remote repository identified by "forgejo". - -I added some files based on a ChatGPT chat, see ChatGPT_1.md. -So, for your first task: Using the lxmf-update skill, inspect upstream LXMF and determine the exact serialized structure of an LXMessage. Do not edit SPEC.md. Produce a report listing every field, its source-code origin, and whether its behavior is confirmed by a test vector. diff --git a/Codex_Response_1.md b/Codex_Response_1.md deleted file mode 100644 index 8b45de7..0000000 --- a/Codex_Response_1.md +++ /dev/null @@ -1,83 +0,0 @@ -No, I don’t need `SPEC.md` or `SOURCES.md` for this first report. I used the checked-out LXMF source directly. I did not edit `SPEC.md` or any other file. - -**LXMessage Serialized Structure** - -Source inspected: LXMF commit `fab12ad9bf9f997797034950f289fe41a79dcf5a` - -Core `lxmf_bytes` layout: - -```text -destination_hash || source_hash || signature || msgpack_payload -``` - -Where `msgpack_payload` is: - -```text -[ - timestamp, - title, - content, - fields, - optional_stamp -] -``` - -The fifth element is only present when a normal message stamp is generated and appended. - -| Field | Serialized position | Type / size | Source origin | Test vector confirmed | -|---|---:|---|---|---| -| `destination_hash` | bytes `0..15` | 16 bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:383), length from [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:40) and RNS 128-bit truncated hashes in [Reticulum.py](/usr/local/src/reticulum/Reticulum/RNS/Reticulum.py:146) | No | -| `source_hash` | bytes `16..31` | 16 bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:384) | No | -| `signature` | bytes `32..95` | 64 bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:378), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:385); RNS signature length from [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:59) and [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:80) | No | -| `msgpack_payload` | bytes `96..end` | msgpack list | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362), packed at [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:381) | No | -| `timestamp` | payload index `0` | Python float, normally msgpack float64 on this RNS umsgpack | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:357), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362); float64 behavior from [umsgpack.py](/usr/local/src/reticulum/Reticulum/RNS/vendor/umsgpack.py:1177) and [umsgpack.py](/usr/local/src/reticulum/Reticulum/RNS/vendor/umsgpack.py:325) | No | -| `title` | payload index `1` | bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:130), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:193), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362) | No | -| `content` | payload index `2` | bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:135), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:202), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362) | No | -| `fields` | payload index `3` | dict, default `{}` | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:215), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362) | No | -| `stamp` | payload index `4`, optional | bytes, only if generated | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:371), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:373); extracted on unpack at [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:754) | No | - -**Hash And Signature** - -`message_id` / `hash` is: - -```text -SHA256(destination_hash || source_hash || msgpack([timestamp, title, content, fields])) -``` - -Source: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:364), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:368), RNS SHA-256 at [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:238). - -The signature is over: - -```text -destination_hash || source_hash || msgpack([timestamp, title, content, fields]) || message_id -``` - -Source: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:375), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:378), RNS signing at [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:767). - -Important: the optional normal `stamp` is excluded from both `message_id` and signature. Unpack removes payload index `4` before recomputing hash/signature material: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:754). - -**Outer Representations** - -Direct/link/resource delivery uses full `self.packed` unchanged: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:636), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:654). - -Opportunistic packet delivery omits the destination hash from the packet payload because the RNS packet destination already implies it: - -```text -source_hash || signature || msgpack_payload -``` - -Source: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:633). - -Propagated and paper messages wrap/encrypt everything after the destination hash: - -```text -destination_hash || encrypted(source_hash || signature || msgpack_payload) -``` - -Source: propagated [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:429), paper [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:449). Propagation transit additionally msgpacks `[time.time(), [lxmf_data]]`: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:436). - -**Notes** - -The README is not authoritative here: it describes payload order as timestamp, content, title, fields, but current code serializes timestamp, title, content, fields. - -No test vector confirmation exists in this checkout. There is no `tools/` directory and no tracked `SPEC.md` or `SOURCES.md`, so every “Test vector confirmed” entry above is marked `No`. diff --git a/Codex_Response_2.md b/Codex_Response_2.md deleted file mode 100644 index 8b45de7..0000000 --- a/Codex_Response_2.md +++ /dev/null @@ -1,83 +0,0 @@ -No, I don’t need `SPEC.md` or `SOURCES.md` for this first report. I used the checked-out LXMF source directly. I did not edit `SPEC.md` or any other file. - -**LXMessage Serialized Structure** - -Source inspected: LXMF commit `fab12ad9bf9f997797034950f289fe41a79dcf5a` - -Core `lxmf_bytes` layout: - -```text -destination_hash || source_hash || signature || msgpack_payload -``` - -Where `msgpack_payload` is: - -```text -[ - timestamp, - title, - content, - fields, - optional_stamp -] -``` - -The fifth element is only present when a normal message stamp is generated and appended. - -| Field | Serialized position | Type / size | Source origin | Test vector confirmed | -|---|---:|---|---|---| -| `destination_hash` | bytes `0..15` | 16 bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:383), length from [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:40) and RNS 128-bit truncated hashes in [Reticulum.py](/usr/local/src/reticulum/Reticulum/RNS/Reticulum.py:146) | No | -| `source_hash` | bytes `16..31` | 16 bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:384) | No | -| `signature` | bytes `32..95` | 64 bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:378), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:385); RNS signature length from [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:59) and [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:80) | No | -| `msgpack_payload` | bytes `96..end` | msgpack list | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362), packed at [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:381) | No | -| `timestamp` | payload index `0` | Python float, normally msgpack float64 on this RNS umsgpack | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:357), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362); float64 behavior from [umsgpack.py](/usr/local/src/reticulum/Reticulum/RNS/vendor/umsgpack.py:1177) and [umsgpack.py](/usr/local/src/reticulum/Reticulum/RNS/vendor/umsgpack.py:325) | No | -| `title` | payload index `1` | bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:130), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:193), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362) | No | -| `content` | payload index `2` | bytes | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:135), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:202), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362) | No | -| `fields` | payload index `3` | dict, default `{}` | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:215), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:362) | No | -| `stamp` | payload index `4`, optional | bytes, only if generated | [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:371), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:373); extracted on unpack at [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:754) | No | - -**Hash And Signature** - -`message_id` / `hash` is: - -```text -SHA256(destination_hash || source_hash || msgpack([timestamp, title, content, fields])) -``` - -Source: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:364), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:368), RNS SHA-256 at [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:238). - -The signature is over: - -```text -destination_hash || source_hash || msgpack([timestamp, title, content, fields]) || message_id -``` - -Source: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:375), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:378), RNS signing at [Identity.py](/usr/local/src/reticulum/Reticulum/RNS/Identity.py:767). - -Important: the optional normal `stamp` is excluded from both `message_id` and signature. Unpack removes payload index `4` before recomputing hash/signature material: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:754). - -**Outer Representations** - -Direct/link/resource delivery uses full `self.packed` unchanged: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:636), [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:654). - -Opportunistic packet delivery omits the destination hash from the packet payload because the RNS packet destination already implies it: - -```text -source_hash || signature || msgpack_payload -``` - -Source: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:633). - -Propagated and paper messages wrap/encrypt everything after the destination hash: - -```text -destination_hash || encrypted(source_hash || signature || msgpack_payload) -``` - -Source: propagated [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:429), paper [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:449). Propagation transit additionally msgpacks `[time.time(), [lxmf_data]]`: [LXMessage.py](/usr/local/src/reticulum/LXMF/LXMF/LXMessage.py:436). - -**Notes** - -The README is not authoritative here: it describes payload order as timestamp, content, title, fields, but current code serializes timestamp, title, content, fields. - -No test vector confirmation exists in this checkout. There is no `tools/` directory and no tracked `SPEC.md` or `SOURCES.md`, so every “Test vector confirmed” entry above is marked `No`. diff --git a/Codex_Response_3.md b/Codex_Response_3.md deleted file mode 100644 index 6362cd8..0000000 --- a/Codex_Response_3.md +++ /dev/null @@ -1,27 +0,0 @@ -Task #2 completed in `/usr/local/src/reticulum/LXMF`. - -Created: - -- `tools/make_test_vectors.pl` -- `tools/verify_examples.pl` -- `tools/LXMF/MessagePack.pm` -- `tools/README.md` -- `examples/lxmf_message_minimal.hex` -- `examples/lxmf_message_stamped.hex` -- `examples/lxmf_message_vectors.json` - -The dependency-free Perl tooling verifies: - -- Fixed hash/signature positions -- Payload order and MessagePack types -- Message ID calculation -- Optional stamp exclusion from message ID and signature input - -Validation passed: - -```text -Reticulum umsgpack byte-for-byte cross-check passed -All LXMessage test-vector checks passed. -``` - -The vectors regenerate deterministically. Signature bytes are documented placeholders; Ed25519 validation is not claimed. `SPEC.md` was not created or modified. diff --git a/Codex_Response_4.md b/Codex_Response_4.md deleted file mode 100644 index c1d9211..0000000 --- a/Codex_Response_4.md +++ /dev/null @@ -1,14 +0,0 @@ -Created [SPEC.md](/usr/local/src/reticulum/LXMF/SPEC.md) containing the evidence-limited `LXMessage Wire Format` section. - -It documents: - -- Core serialized layout and payload order -- Message ID and signature-input calculations -- Stamp exclusion behavior -- Delivery representations -- Exact source references and evidence commits -- Explicit unresolved behavior - -Verification: `perl tools/verify_examples.pl` passed all checks. - -Only `SPEC.md` is untracked; no other files were changed. diff --git a/PLANS.md b/PLANS.md deleted file mode 100644 index cbaedc0..0000000 --- a/PLANS.md +++ /dev/null @@ -1,13 +0,0 @@ -# LXMF Specification Investigation Plan - -Each investigation must include: - -1. Question being answered. -2. Upstream files inspected. -3. Exact version or commit. -4. Relevant classes/functions/constants. -5. Observed behavior. -6. Whether behavior is normative, inferred, or unresolved. -7. Test vectors needed. -8. SPEC.md sections affected. -9. Open questions. diff --git a/SPEC.md b/SPEC.md deleted file mode 100644 index a6dcc8e..0000000 --- a/SPEC.md +++ /dev/null @@ -1,153 +0,0 @@ -# LXMF Implementation-Derived Specification - -## LXMessage Wire Format - -This section describes behavior confirmed in the checked-in LXMF implementation -and by the reproducible vectors under `examples/`. It does not assign normative -requirements to independent implementations. - -Evidence baseline: - -- LXMF commit `d483d40d5cb994c6027802af1bf284dfeccc623b` -- Reticulum commit `2646f673ee061135d7c351ef44dce290ccd7e06e` - -### Evidence status - -- **Source-confirmed** means the behavior is directly implemented by the - checked-in LXMF or Reticulum source. -- **Vector-confirmed** means `perl tools/verify_examples.pl` confirms the - behavior for the deterministic minimal and stamped vectors. -- Behavior not covered by either form of evidence is marked **Unresolved**. - -### Core serialized message - -`LXMessage.pack()` constructs `lxmf_bytes` by concatenating: - -```text -destination_hash || source_hash || signature || packed_payload -``` - -| Part | Position | Confirmed representation | Evidence | -|---|---:|---|---| -| `destination_hash` | bytes `0..15` | 16 bytes | Source-confirmed: `LXMF/LXMessage.py:40`, `LXMF/LXMessage.py:383`; Vector-confirmed | -| `source_hash` | bytes `16..31` | 16 bytes | Source-confirmed: `LXMF/LXMessage.py:384`; Vector-confirmed | -| `signature` | bytes `32..95` | 64 bytes | Source-confirmed: `LXMF/LXMessage.py:41`, `LXMF/LXMessage.py:385`; Vector-confirmed position and length only | -| `packed_payload` | bytes `96..end` | MessagePack array | Source-confirmed: `LXMF/LXMessage.py:362`, `LXMF/LXMessage.py:381-386`; Vector-confirmed | - -The 16-byte hash length derives from -`RNS.Identity.TRUNCATED_HASHLENGTH // 8`. The checked-in Reticulum source sets -the truncated hash length to 128 bits. The 64-byte signature length derives -from `RNS.Identity.SIGLENGTH // 8`. See `RNS/Reticulum.py:146-147` and -`RNS/Identity.py:80-83` in the evidence-baseline Reticulum checkout. - -### Payload - -For messages generated by `LXMessage.pack()`, the payload is: - -```text -[ - timestamp, - title, - content, - fields -] -``` - -If a message stamp is available when packing, it is appended: - -```text -[ - timestamp, - title, - content, - fields, - stamp -] -``` - -| Index | Name | Confirmed behavior | Evidence | -|---:|---|---|---| -| `0` | `timestamp` | Set from `time.time()` when not already set, then packed as the first payload value | Source-confirmed: `LXMF/LXMessage.py:357`, `LXMF/LXMessage.py:362`; Vector-confirmed as MessagePack float64 for the deterministic vectors | -| `1` | `title` | Constructor string input is UTF-8 encoded to bytes; byte input is retained | Source-confirmed: `LXMF/LXMessage.py:130-133`, `LXMF/LXMessage.py:193-197`; Vector-confirmed as MessagePack binary | -| `2` | `content` | Constructor string input is UTF-8 encoded to bytes; byte input is retained | Source-confirmed: `LXMF/LXMessage.py:135-136`, `LXMF/LXMessage.py:202-206`; Vector-confirmed as MessagePack binary | -| `3` | `fields` | Constructor input must be a dictionary or `None`; `None` becomes an empty dictionary | Source-confirmed: `LXMF/LXMessage.py:138`, `LXMF/LXMessage.py:215-219`; Vector-confirmed for an empty MessagePack map | -| `4` | `stamp` | Optional value appended after the four base payload values | Source-confirmed: `LXMF/LXMessage.py:371-373`; Vector-confirmed as MessagePack binary in the stamped vector | - -The payload order above is source- and vector-confirmed. It differs from the -order stated in the upstream `README.md`, which lists content before title. - -### Message ID - -The message ID, also stored as `LXMessage.hash`, is: - -```text -SHA-256( - destination_hash || - source_hash || - msgpack([timestamp, title, content, fields]) -) -``` - -The optional stamp is excluded from the MessagePack payload used to calculate -the message ID. This is source-confirmed by both packing and unpacking behavior, -and vector-confirmed by the minimal and stamped vectors producing the same -message ID. See `LXMF/LXMessage.py:362-369` and -`LXMF/LXMessage.py:754-764`. Reticulum defines `full_hash()` as SHA-256 at -`RNS/Identity.py:238-246`. - -The message ID is not included in `lxmf_bytes`. - -### Signature input - -`LXMessage.pack()` requests a signature over: - -```text -destination_hash || -source_hash || -msgpack([timestamp, title, content, fields]) || -message_id -``` - -The optional stamp is excluded from the signature input. The construction of -the signature input is source-confirmed at `LXMF/LXMessage.py:375-378` and -`LXMF/LXMessage.py:762-764`, and vector-confirmed. - -The deterministic vectors contain placeholder signature bytes. They confirm -the signature position and input bytes, but do not confirm signing or signature -validation. - -### Delivery representations - -The core `lxmf_bytes` representation is modified or wrapped for some delivery -methods: - -| Delivery representation | Confirmed serialized data | Evidence | -|---|---|---| -| Direct packet or resource | Full `lxmf_bytes` | Source-confirmed: `LXMF/LXMessage.py:635-636`, `LXMF/LXMessage.py:653-654` | -| Opportunistic packet | `lxmf_bytes` without the leading destination hash | Source-confirmed: `LXMF/LXMessage.py:633-634` | -| Propagated message data | `destination_hash || encrypt(lxmf_bytes after destination_hash)` | Source-confirmed: `LXMF/LXMessage.py:429-436` | -| Propagation transfer wrapper | MessagePack `[time.time(), [propagated_message_data]]` | Source-confirmed: `LXMF/LXMessage.py:436` | -| Paper message data | `destination_hash || encrypt(lxmf_bytes after destination_hash)` | Source-confirmed: `LXMF/LXMessage.py:446-451` | -| Paper URI | URL-safe Base64 of paper message data, without `=` padding, prefixed by `lxm://` | Source-confirmed: `LXMF/LXMessage.py:698-707` | - -These delivery representations are not covered by the current deterministic -test vectors. - -### Unresolved behavior - -The following behavior is not established by the current source inspection and -test-vector coverage: - -- Normative requirements for independent LXMF implementations. -- Cross-implementation Ed25519 signature generation and validation. -- Accepted signature encodings beyond the fixed 64-byte position generated by - the checked-in implementation. -- A universal MessagePack float width for timestamps on every supported - platform. The deterministic vectors confirm float64 only for those vectors. -- Interoperable constraints on the contents, key types, value types, ordering, - and nesting depth of `fields`. -- Interoperable constraints on stamp length and encoding. -- Required handling of malformed, truncated, non-canonical, or payload arrays - containing fewer than four or more than five entries. -- Byte-for-byte vectors for opportunistic, propagated, paper, URI, encrypted, - or persisted-container representations. diff --git a/examples/lxmf_message_minimal.hex b/examples/lxmf_message_minimal.hex deleted file mode 100644 index 85f8d89..0000000 --- a/examples/lxmf_message_minimal.hex +++ /dev/null @@ -1 +0,0 @@ -000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f94cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f647980 diff --git a/examples/lxmf_message_stamped.hex b/examples/lxmf_message_stamped.hex deleted file mode 100644 index be6fc07..0000000 --- a/examples/lxmf_message_stamped.hex +++ /dev/null @@ -1 +0,0 @@ -000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f95cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f647980c420a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf diff --git a/examples/lxmf_message_vectors.json b/examples/lxmf_message_vectors.json deleted file mode 100644 index b60d155..0000000 --- a/examples/lxmf_message_vectors.json +++ /dev/null @@ -1,26 +0,0 @@ -{ - "content_hex" : "44657465726d696e6973746963204c584d4620626f6479", - "description" : "Deterministic LXMessage packing vectors; signature bytes are placeholders", - "destination_hash_hex" : "000102030405060708090a0b0c0d0e0f", - "fields" : {}, - "format" : "destination_hash || source_hash || signature || msgpack_payload", - "message_id_hex" : "55f497f6f2ea5d858eaff541a9fa77f9d5bdf57be75be4a94df65d1f9a7ffea3", - "minimal_lxmf_hex" : "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f94cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f647980", - "payload_order" : [ - "timestamp", - "title", - "content", - "fields", - "optional_stamp" - ], - "payload_with_stamp_hex" : "95cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f647980c420a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf", - "payload_without_stamp_hex" : "94cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f647980", - "signature_hex" : "202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f", - "signature_valid" : false, - "signed_part_hex" : "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f94cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f64798055f497f6f2ea5d858eaff541a9fa77f9d5bdf57be75be4a94df65d1f9a7ffea3", - "source_hash_hex" : "101112131415161718191a1b1c1d1e1f", - "stamp_hex" : "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf", - "stamped_lxmf_hex" : "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f95cb41d954fc40100000c40a54657374207469746c65c41744657465726d696e6973746963204c584d4620626f647980c420a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf", - "timestamp" : 1700000000.25, - "title_hex" : "54657374207469746c65" -} diff --git a/grep_results.txt b/grep_results.txt deleted file mode 100644 index 52ae092..0000000 --- a/grep_results.txt +++ /dev/null @@ -1,39 +0,0 @@ -./LXMF/Utilities/lxmd.py:DEFFERED_JOBS_DELAY = 10 -./LXMF/Utilities/lxmd.py:def create_default_config(configpath): -./LXMF/Utilities/lxmd.py:def apply_config(): -./LXMF/Utilities/lxmd.py:def lxmf_delivery(lxm): -./LXMF/Utilities/lxmd.py:def program_setup(configdir = None, rnsconfigdir = None, run_pn = False, on_inbound = None, verbosity = 0, quietness = 0, service = False): -./LXMF/Utilities/lxmd.py:def jobs(): -./LXMF/Utilities/lxmd.py:def deferred_start_jobs(): -./LXMF/Utilities/lxmd.py:def _request_sync(identity, destination_hash, remote_identity, timeout=15, exit_on_fail=False): -./LXMF/Utilities/lxmd.py:def request_sync(target, remote=None, configdir=None, rnsconfigdir=None, verbosity=0, quietness=0, timeout=15, identity_path=None): -./LXMF/Utilities/lxmd.py:def _request_unpeer(identity, destination_hash, remote_identity, timeout=15, exit_on_fail=False): -./LXMF/Utilities/lxmd.py:def request_unpeer(target, remote=None, configdir=None, rnsconfigdir=None, verbosity=0, quietness=0, timeout=15, identity_path=None): -./LXMF/Utilities/lxmd.py:def query_status(identity, remote_identity=None, timeout=5, exit_on_fail=False): -./LXMF/Utilities/lxmd.py:def get_status(remote=None, configdir=None, rnsconfigdir=None, verbosity=0, quietness=0, timeout=5, -./LXMF/Utilities/lxmd.py:def _get_target_identity(remote=None, timeout=5): -./LXMF/Utilities/lxmd.py:def _remote_init(configdir=None, rnsconfigdir=None, verbosity=0, quietness=0, identity_path=None): -./LXMF/Utilities/lxmd.py:def main(): -./LXMF/LXMF.py:def display_name_from_app_data(app_data=None): -./LXMF/LXMF.py:def stamp_cost_from_app_data(app_data=None): -./LXMF/LXMF.py:def compression_support_from_app_data(app_data=None): -./LXMF/LXMF.py:def pn_name_from_app_data(app_data=None): -./LXMF/LXMF.py:def pn_stamp_cost_from_app_data(app_data=None): -./LXMF/LXMF.py:def pn_announce_data_is_valid(data): -./LXMF/LXStamper.py:def worker_context(ctx, stamp_cost, workblock, message_id): -./LXMF/LXStamper.py:def stamp_workblock(material, expand_rounds=WORKBLOCK_EXPAND_ROUNDS): -./LXMF/LXStamper.py:def stamp_value(workblock, stamp): -./LXMF/LXStamper.py:def stamp_valid(stamp, target_cost, workblock): -./LXMF/LXStamper.py:def validate_peering_key(peering_id, peering_key, target_cost): -./LXMF/LXStamper.py:def validate_pn_stamp(transient_data, target_cost): -./LXMF/LXStamper.py:def validate_pn_stamps_job_simple(transient_list, target_cost): -./LXMF/LXStamper.py:def validate_pn_stamps_job_multip(transient_list, target_cost): -./LXMF/LXStamper.py:def validate_pn_stamps(transient_list, target_cost): -./LXMF/LXStamper.py:def generate_stamp(message_id, stamp_cost, expand_rounds=WORKBLOCK_EXPAND_ROUNDS): -./LXMF/LXStamper.py:def cancel_work(message_id): -./LXMF/LXStamper.py:def job_simple(stamp_cost, workblock, message_id): -./LXMF/LXStamper.py:def job_linux_managed(stamp_cost, workblock, message_id): -./LXMF/LXStamper.py:def job_linux(stamp_cost, workblock, message_id): -./LXMF/LXStamper.py:def job_android(stamp_cost, workblock, message_id): -./docs/example_receiver.py:def delivery_callback(message): -./docs/example_sender.py:def delivery_callback(message): diff --git a/tools/LXMF/MessagePack.pm b/tools/LXMF/MessagePack.pm deleted file mode 100644 index 5ce8c39..0000000 --- a/tools/LXMF/MessagePack.pm +++ /dev/null @@ -1,181 +0,0 @@ -package LXMF::MessagePack; - -use strict; -use warnings; -use Exporter 'import'; - -our @EXPORT_OK = qw( - pack_array - pack_bin - pack_float64 - pack_map - parse_item -); - -sub pack_float64 { - my ($value) = @_; - return "\xcb" . pack("d>", $value); -} - -sub pack_bin { - my ($value) = @_; - my $length = length($value); - - return "\xc4" . pack("C", $length) . $value if $length <= 0xff; - return "\xc5" . pack("n", $length) . $value if $length <= 0xffff; - return "\xc6" . pack("N", $length) . $value; -} - -sub pack_array { - my (@items) = @_; - my $length = scalar @items; - my $header; - - if ($length <= 15) { - $header = pack("C", 0x90 | $length); - } - elsif ($length <= 0xffff) { - $header = "\xdc" . pack("n", $length); - } - else { - $header = "\xdd" . pack("N", $length); - } - - return $header . join("", @items); -} - -sub pack_map { - my (@entries) = @_; - die "pack_map requires key/value byte-string pairs\n" if @entries % 2; - - my $length = @entries / 2; - my $header; - - if ($length <= 15) { - $header = pack("C", 0x80 | $length); - } - elsif ($length <= 0xffff) { - $header = "\xde" . pack("n", $length); - } - else { - $header = "\xdf" . pack("N", $length); - } - - return $header . join("", @entries); -} - -sub _take { - my ($bytes, $offset_ref, $length) = @_; - die "truncated MessagePack item\n" if $$offset_ref + $length > length($bytes); - my $value = substr($bytes, $$offset_ref, $length); - $$offset_ref += $length; - return $value; -} - -sub _parse_length { - my ($bytes, $offset_ref, $width) = @_; - my $raw = _take($bytes, $offset_ref, $width); - return unpack("C", $raw) if $width == 1; - return unpack("n", $raw) if $width == 2; - return unpack("N", $raw); -} - -sub parse_item { - my ($bytes, $offset_ref) = @_; - $offset_ref //= \my $offset; - $$offset_ref //= 0; - - my $start = $$offset_ref; - my $code = unpack("C", _take($bytes, $offset_ref, 1)); - my $item = { code => $code }; - - if ($code <= 0x7f) { - @$item{qw(type value)} = ("integer", $code); - } - elsif ($code >= 0xe0) { - @$item{qw(type value)} = ("integer", $code - 256); - } - elsif (($code & 0xf0) == 0x90) { - _parse_array($bytes, $offset_ref, $item, $code & 0x0f); - } - elsif (($code & 0xf0) == 0x80) { - _parse_map($bytes, $offset_ref, $item, $code & 0x0f); - } - elsif (($code & 0xe0) == 0xa0) { - my $length = $code & 0x1f; - @$item{qw(type value)} = ("string", _take($bytes, $offset_ref, $length)); - } - elsif ($code == 0xc0) { - $item->{type} = "nil"; - $item->{value} = undef; - } - elsif ($code == 0xc2 || $code == 0xc3) { - @$item{qw(type value)} = ("boolean", $code == 0xc3 ? 1 : 0); - } - elsif ($code == 0xca) { - @$item{qw(type value)} = ("float32", unpack("f>", _take($bytes, $offset_ref, 4))); - } - elsif ($code == 0xcb) { - @$item{qw(type value)} = ("float64", unpack("d>", _take($bytes, $offset_ref, 8))); - } - elsif ($code >= 0xc4 && $code <= 0xc6) { - my $width = 1 << ($code - 0xc4); - my $length = _parse_length($bytes, $offset_ref, $width); - @$item{qw(type value)} = ("binary", _take($bytes, $offset_ref, $length)); - } - elsif ($code >= 0xcc && $code <= 0xcf) { - my @formats = ("C", "n", "N", "Q>"); - my @widths = (1, 2, 4, 8); - my $index = $code - 0xcc; - @$item{qw(type value)} = ( - "integer", - unpack($formats[$index], _take($bytes, $offset_ref, $widths[$index])), - ); - } - elsif ($code >= 0xd0 && $code <= 0xd3) { - my @formats = ("c", "s>", "l>", "q>"); - my @widths = (1, 2, 4, 8); - my $index = $code - 0xd0; - @$item{qw(type value)} = ( - "integer", - unpack($formats[$index], _take($bytes, $offset_ref, $widths[$index])), - ); - } - elsif ($code == 0xd9 || $code == 0xda || $code == 0xdb) { - my $width = 1 << ($code - 0xd9); - my $length = _parse_length($bytes, $offset_ref, $width); - @$item{qw(type value)} = ("string", _take($bytes, $offset_ref, $length)); - } - elsif ($code == 0xdc || $code == 0xdd) { - my $width = $code == 0xdc ? 2 : 4; - _parse_array($bytes, $offset_ref, $item, _parse_length($bytes, $offset_ref, $width)); - } - elsif ($code == 0xde || $code == 0xdf) { - my $width = $code == 0xde ? 2 : 4; - _parse_map($bytes, $offset_ref, $item, _parse_length($bytes, $offset_ref, $width)); - } - else { - die sprintf("unsupported MessagePack code 0x%02x at offset %d\n", $code, $start); - } - - $item->{start} = $start; - $item->{end} = $$offset_ref; - $item->{raw} = substr($bytes, $start, $$offset_ref - $start); - return $item; -} - -sub _parse_array { - my ($bytes, $offset_ref, $item, $length) = @_; - $item->{type} = "array"; - $item->{items} = [map { parse_item($bytes, $offset_ref) } 1 .. $length]; -} - -sub _parse_map { - my ($bytes, $offset_ref, $item, $length) = @_; - $item->{type} = "map"; - $item->{entries} = [ - map { [parse_item($bytes, $offset_ref), parse_item($bytes, $offset_ref)] } 1 .. $length - ]; -} - -1; diff --git a/tools/README.md b/tools/README.md deleted file mode 100644 index 75538fa..0000000 --- a/tools/README.md +++ /dev/null @@ -1,32 +0,0 @@ -# LXMessage Test-Vector Tooling - -The scripts in this directory verify the implementation-derived LXMessage -packing behavior described in `Code_Response_1.md`. - -They use only Perl core modules. The local `LXMF::MessagePack` module implements -the MessagePack subset needed to generate the vectors and parse their complete -payloads. - -Generate deterministic vectors: - -```sh -perl tools/make_test_vectors.pl -``` - -Verify the vectors: - -```sh -perl tools/verify_examples.pl -``` - -The vectors confirm: - -- the 16-byte destination hash, 16-byte source hash, and 64-byte signature - positions; -- payload order `[timestamp, title, content, fields, optional_stamp]`; -- float64 timestamp and binary title/content/stamp encodings; -- the message ID calculation; -- exclusion of the optional stamp from the message ID and signature input. - -The signature bytes are deterministic placeholders. These vectors do not claim -to verify Ed25519 signing or signature validation. diff --git a/tools/make_test_vectors.pl b/tools/make_test_vectors.pl deleted file mode 100755 index 8d4385b..0000000 --- a/tools/make_test_vectors.pl +++ /dev/null @@ -1,76 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; -use Digest::SHA qw(sha256); -use FindBin qw($Bin); -use File::Path qw(make_path); -use JSON::PP; -use lib "$Bin"; -use LXMF::MessagePack qw(pack_array pack_bin pack_float64 pack_map); - -my $output_dir = "$Bin/../examples"; -make_path($output_dir); - -my $destination_hash = pack("C*", 0x00 .. 0x0f); -my $source_hash = pack("C*", 0x10 .. 0x1f); -my $signature = pack("C*", 0x20 .. 0x5f); -my $stamp = pack("C*", 0xa0 .. 0xbf); - -my $timestamp = 1_700_000_000.25; -my $title = "Test title"; -my $content = "Deterministic LXMF body"; -my $fields = pack_map(); - -my @required_items = ( - pack_float64($timestamp), - pack_bin($title), - pack_bin($content), - $fields, -); - -my $payload = pack_array(@required_items); -my $stamped_payload = pack_array(@required_items, pack_bin($stamp)); -my $message_id = sha256($destination_hash . $source_hash . $payload); -my $signed_part = $destination_hash . $source_hash . $payload . $message_id; - -my $minimal = $destination_hash . $source_hash . $signature . $payload; -my $stamped = $destination_hash . $source_hash . $signature . $stamped_payload; - -write_hex("$output_dir/lxmf_message_minimal.hex", $minimal); -write_hex("$output_dir/lxmf_message_stamped.hex", $stamped); - -my $manifest = { - description => "Deterministic LXMessage packing vectors; signature bytes are placeholders", - format => "destination_hash || source_hash || signature || msgpack_payload", - payload_order => [qw(timestamp title content fields optional_stamp)], - destination_hash_hex => unpack("H*", $destination_hash), - source_hash_hex => unpack("H*", $source_hash), - signature_hex => unpack("H*", $signature), - signature_valid => JSON::PP::false, - timestamp => $timestamp, - title_hex => unpack("H*", $title), - content_hex => unpack("H*", $content), - fields => {}, - stamp_hex => unpack("H*", $stamp), - payload_without_stamp_hex => unpack("H*", $payload), - payload_with_stamp_hex => unpack("H*", $stamped_payload), - message_id_hex => unpack("H*", $message_id), - signed_part_hex => unpack("H*", $signed_part), - minimal_lxmf_hex => unpack("H*", $minimal), - stamped_lxmf_hex => unpack("H*", $stamped), -}; - -open my $json_fh, ">:raw", "$output_dir/lxmf_message_vectors.json" - or die "cannot write vector manifest: $!\n"; -print {$json_fh} JSON::PP->new->canonical->pretty->encode($manifest); -close $json_fh; - -print "Generated deterministic LXMessage vectors in $output_dir\n"; - -sub write_hex { - my ($path, $bytes) = @_; - open my $fh, ">:raw", $path or die "cannot write $path: $!\n"; - print {$fh} unpack("H*", $bytes), "\n"; - close $fh; -} diff --git a/tools/verify_examples.pl b/tools/verify_examples.pl deleted file mode 100755 index 0f86e67..0000000 --- a/tools/verify_examples.pl +++ /dev/null @@ -1,88 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; -use Digest::SHA qw(sha256); -use FindBin qw($Bin); -use JSON::PP; -use lib "$Bin"; -use LXMF::MessagePack qw(pack_array parse_item); - -my $examples_dir = "$Bin/../examples"; -my $manifest = read_json("$examples_dir/lxmf_message_vectors.json"); -my $minimal = read_hex("$examples_dir/lxmf_message_minimal.hex"); -my $stamped = read_hex("$examples_dir/lxmf_message_stamped.hex"); - -verify_lxmessage("minimal", $minimal, $manifest, 4); -verify_lxmessage("stamped", $stamped, $manifest, 5); - -check(unpack("H*", $minimal) eq $manifest->{minimal_lxmf_hex}, "minimal full bytes match manifest"); -check(unpack("H*", $stamped) eq $manifest->{stamped_lxmf_hex}, "stamped full bytes match manifest"); - -print "All LXMessage test-vector checks passed.\n"; - -sub verify_lxmessage { - my ($name, $bytes, $expected, $expected_count) = @_; - - check(length($bytes) > 96, "$name vector has fixed prefix and payload"); - my $destination_hash = substr($bytes, 0, 16); - my $source_hash = substr($bytes, 16, 16); - my $signature = substr($bytes, 32, 64); - my $packed_payload = substr($bytes, 96); - - check(unpack("H*", $destination_hash) eq $expected->{destination_hash_hex}, "$name destination hash"); - check(unpack("H*", $source_hash) eq $expected->{source_hash_hex}, "$name source hash"); - check(unpack("H*", $signature) eq $expected->{signature_hex}, "$name signature position and length"); - - my $offset = 0; - my $payload = parse_item($packed_payload, \$offset); - check($offset == length($packed_payload), "$name payload consumes all remaining bytes"); - check($payload->{type} eq "array", "$name payload is a MessagePack array"); - check(@{$payload->{items}} == $expected_count, "$name payload has $expected_count entries"); - - my @items = @{$payload->{items}}; - check($items[0]{type} eq "float64", "$name payload[0] timestamp is float64"); - check($items[0]{value} == $expected->{timestamp}, "$name payload[0] timestamp value"); - check($items[1]{type} eq "binary", "$name payload[1] title is binary"); - check(unpack("H*", $items[1]{value}) eq $expected->{title_hex}, "$name payload[1] title value"); - check($items[2]{type} eq "binary", "$name payload[2] content is binary"); - check(unpack("H*", $items[2]{value}) eq $expected->{content_hex}, "$name payload[2] content value"); - check($items[3]{type} eq "map", "$name payload[3] fields is a map"); - check(@{$items[3]{entries}} == 0, "$name payload[3] fields map is empty"); - - my $unstamped_payload = pack_array(map { $_->{raw} } @items[0 .. 3]); - my $message_id = sha256($destination_hash . $source_hash . $unstamped_payload); - my $signed_part = $destination_hash . $source_hash . $unstamped_payload . $message_id; - - check(unpack("H*", $unstamped_payload) eq $expected->{payload_without_stamp_hex}, "$name canonical unstamped payload"); - check(unpack("H*", $message_id) eq $expected->{message_id_hex}, "$name message ID excludes optional stamp"); - check(unpack("H*", $signed_part) eq $expected->{signed_part_hex}, "$name signature input excludes optional stamp"); - - if ($expected_count == 5) { - check($items[4]{type} eq "binary", "$name payload[4] stamp is binary"); - check(unpack("H*", $items[4]{value}) eq $expected->{stamp_hex}, "$name payload[4] stamp value"); - } -} - -sub read_json { - my ($path) = @_; - open my $fh, "<:raw", $path or die "cannot read $path: $!\n"; - local $/; - return JSON::PP->new->decode(<$fh>); -} - -sub read_hex { - my ($path) = @_; - open my $fh, "<:raw", $path or die "cannot read $path: $!\n"; - local $/; - my $hex = <$fh>; - $hex =~ s/\s+//g; - die "$path contains non-hexadecimal data\n" if $hex =~ /[^0-9a-fA-F]/; - return pack("H*", $hex); -} - -sub check { - my ($condition, $description) = @_; - die "not ok - $description\n" unless $condition; - print "ok - $description\n"; -}