After completing Task 3 of initial 3 task punch list.

2026-06-06 09:47:20 -07:00 · 2026-06-06 09:47:20 -07:00 · 18c19f95b3
commit 18c19f95b3
parent d483d40d5c
2 changed files with 167 additions and 0 deletions
--- a/Codex_Response_4.md
+++ b/Codex_Response_4.md
@ -0,0 +1,14 @@
+Created [SPEC.md](/usr/local/src/reticulum/LXMF/SPEC.md) containing the evidence-limited `LXMessage Wire Format` section.
+
+It documents:
+
+- Core serialized layout and payload order
+- Message ID and signature-input calculations
+- Stamp exclusion behavior
+- Delivery representations
+- Exact source references and evidence commits
+- Explicit unresolved behavior
+
+Verification: `perl tools/verify_examples.pl` passed all checks.
+
+Only `SPEC.md` is untracked; no other files were changed.
--- a/SPEC.md
+++ b/SPEC.md
@ -0,0 +1,153 @@
+# LXMF Implementation-Derived Specification
+
+## LXMessage Wire Format
+
+This section describes behavior confirmed in the checked-in LXMF implementation
+and by the reproducible vectors under `examples/`. It does not assign normative
+requirements to independent implementations.
+
+Evidence baseline:
+
+- LXMF commit `d483d40d5cb994c6027802af1bf284dfeccc623b`
+- Reticulum commit `2646f673ee061135d7c351ef44dce290ccd7e06e`
+
+### Evidence status
+
+- **Source-confirmed** means the behavior is directly implemented by the
+  checked-in LXMF or Reticulum source.
+- **Vector-confirmed** means `perl tools/verify_examples.pl` confirms the
+  behavior for the deterministic minimal and stamped vectors.
+- Behavior not covered by either form of evidence is marked **Unresolved**.
+
+### Core serialized message
+
+`LXMessage.pack()` constructs `lxmf_bytes` by concatenating:
+
+```text
+destination_hash || source_hash || signature || packed_payload
+```
+
+| Part | Position | Confirmed representation | Evidence |
+|---|---:|---|---|
+| `destination_hash` | bytes `0..15` | 16 bytes | Source-confirmed: `LXMF/LXMessage.py:40`, `LXMF/LXMessage.py:383`; Vector-confirmed |
+| `source_hash` | bytes `16..31` | 16 bytes | Source-confirmed: `LXMF/LXMessage.py:384`; Vector-confirmed |
+| `signature` | bytes `32..95` | 64 bytes | Source-confirmed: `LXMF/LXMessage.py:41`, `LXMF/LXMessage.py:385`; Vector-confirmed position and length only |
+| `packed_payload` | bytes `96..end` | MessagePack array | Source-confirmed: `LXMF/LXMessage.py:362`, `LXMF/LXMessage.py:381-386`; Vector-confirmed |
+
+The 16-byte hash length derives from
+`RNS.Identity.TRUNCATED_HASHLENGTH // 8`. The checked-in Reticulum source sets
+the truncated hash length to 128 bits. The 64-byte signature length derives
+from `RNS.Identity.SIGLENGTH // 8`. See `RNS/Reticulum.py:146-147` and
+`RNS/Identity.py:80-83` in the evidence-baseline Reticulum checkout.
+
+### Payload
+
+For messages generated by `LXMessage.pack()`, the payload is:
+
+```text
+[
+  timestamp,
+  title,
+  content,
+  fields
+]
+```
+
+If a message stamp is available when packing, it is appended:
+
+```text
+[
+  timestamp,
+  title,
+  content,
+  fields,
+  stamp
+]
+```
+
+| Index | Name | Confirmed behavior | Evidence |
+|---:|---|---|---|
+| `0` | `timestamp` | Set from `time.time()` when not already set, then packed as the first payload value | Source-confirmed: `LXMF/LXMessage.py:357`, `LXMF/LXMessage.py:362`; Vector-confirmed as MessagePack float64 for the deterministic vectors |
+| `1` | `title` | Constructor string input is UTF-8 encoded to bytes; byte input is retained | Source-confirmed: `LXMF/LXMessage.py:130-133`, `LXMF/LXMessage.py:193-197`; Vector-confirmed as MessagePack binary |
+| `2` | `content` | Constructor string input is UTF-8 encoded to bytes; byte input is retained | Source-confirmed: `LXMF/LXMessage.py:135-136`, `LXMF/LXMessage.py:202-206`; Vector-confirmed as MessagePack binary |
+| `3` | `fields` | Constructor input must be a dictionary or `None`; `None` becomes an empty dictionary | Source-confirmed: `LXMF/LXMessage.py:138`, `LXMF/LXMessage.py:215-219`; Vector-confirmed for an empty MessagePack map |
+| `4` | `stamp` | Optional value appended after the four base payload values | Source-confirmed: `LXMF/LXMessage.py:371-373`; Vector-confirmed as MessagePack binary in the stamped vector |
+
+The payload order above is source- and vector-confirmed. It differs from the
+order stated in the upstream `README.md`, which lists content before title.
+
+### Message ID
+
+The message ID, also stored as `LXMessage.hash`, is:
+
+```text
+SHA-256(
+  destination_hash ||
+  source_hash ||
+  msgpack([timestamp, title, content, fields])
+)
+```
+
+The optional stamp is excluded from the MessagePack payload used to calculate
+the message ID. This is source-confirmed by both packing and unpacking behavior,
+and vector-confirmed by the minimal and stamped vectors producing the same
+message ID. See `LXMF/LXMessage.py:362-369` and
+`LXMF/LXMessage.py:754-764`. Reticulum defines `full_hash()` as SHA-256 at
+`RNS/Identity.py:238-246`.
+
+The message ID is not included in `lxmf_bytes`.
+
+### Signature input
+
+`LXMessage.pack()` requests a signature over:
+
+```text
+destination_hash ||
+source_hash ||
+msgpack([timestamp, title, content, fields]) ||
+message_id
+```
+
+The optional stamp is excluded from the signature input. The construction of
+the signature input is source-confirmed at `LXMF/LXMessage.py:375-378` and
+`LXMF/LXMessage.py:762-764`, and vector-confirmed.
+
+The deterministic vectors contain placeholder signature bytes. They confirm
+the signature position and input bytes, but do not confirm signing or signature
+validation.
+
+### Delivery representations
+
+The core `lxmf_bytes` representation is modified or wrapped for some delivery
+methods:
+
+| Delivery representation | Confirmed serialized data | Evidence |
+|---|---|---|
+| Direct packet or resource | Full `lxmf_bytes` | Source-confirmed: `LXMF/LXMessage.py:635-636`, `LXMF/LXMessage.py:653-654` |
+| Opportunistic packet | `lxmf_bytes` without the leading destination hash | Source-confirmed: `LXMF/LXMessage.py:633-634` |
+| Propagated message data | `destination_hash || encrypt(lxmf_bytes after destination_hash)` | Source-confirmed: `LXMF/LXMessage.py:429-436` |
+| Propagation transfer wrapper | MessagePack `[time.time(), [propagated_message_data]]` | Source-confirmed: `LXMF/LXMessage.py:436` |
+| Paper message data | `destination_hash || encrypt(lxmf_bytes after destination_hash)` | Source-confirmed: `LXMF/LXMessage.py:446-451` |
+| Paper URI | URL-safe Base64 of paper message data, without `=` padding, prefixed by `lxm://` | Source-confirmed: `LXMF/LXMessage.py:698-707` |
+
+These delivery representations are not covered by the current deterministic
+test vectors.
+
+### Unresolved behavior
+
+The following behavior is not established by the current source inspection and
+test-vector coverage:
+
+- Normative requirements for independent LXMF implementations.
+- Cross-implementation Ed25519 signature generation and validation.
+- Accepted signature encodings beyond the fixed 64-byte position generated by
+  the checked-in implementation.
+- A universal MessagePack float width for timestamps on every supported
+  platform. The deterministic vectors confirm float64 only for those vectors.
+- Interoperable constraints on the contents, key types, value types, ordering,
+  and nesting depth of `fields`.
+- Interoperable constraints on stamp length and encoding.
+- Required handling of malformed, truncated, non-canonical, or payload arrays
+  containing fewer than four or more than five entries.
+- Byte-for-byte vectors for opportunistic, propagated, paper, URI, encrypted,
+  or persisted-container representations.