diff --git a/Codex_Response_4.md b/Codex_Response_4.md new file mode 100644 index 0000000..c1d9211 --- /dev/null +++ b/Codex_Response_4.md @@ -0,0 +1,14 @@ +Created [SPEC.md](/usr/local/src/reticulum/LXMF/SPEC.md) containing the evidence-limited `LXMessage Wire Format` section. + +It documents: + +- Core serialized layout and payload order +- Message ID and signature-input calculations +- Stamp exclusion behavior +- Delivery representations +- Exact source references and evidence commits +- Explicit unresolved behavior + +Verification: `perl tools/verify_examples.pl` passed all checks. + +Only `SPEC.md` is untracked; no other files were changed. diff --git a/SPEC.md b/SPEC.md new file mode 100644 index 0000000..a6dcc8e --- /dev/null +++ b/SPEC.md @@ -0,0 +1,153 @@ +# LXMF Implementation-Derived Specification + +## LXMessage Wire Format + +This section describes behavior confirmed in the checked-in LXMF implementation +and by the reproducible vectors under `examples/`. It does not assign normative +requirements to independent implementations. + +Evidence baseline: + +- LXMF commit `d483d40d5cb994c6027802af1bf284dfeccc623b` +- Reticulum commit `2646f673ee061135d7c351ef44dce290ccd7e06e` + +### Evidence status + +- **Source-confirmed** means the behavior is directly implemented by the + checked-in LXMF or Reticulum source. +- **Vector-confirmed** means `perl tools/verify_examples.pl` confirms the + behavior for the deterministic minimal and stamped vectors. +- Behavior not covered by either form of evidence is marked **Unresolved**. + +### Core serialized message + +`LXMessage.pack()` constructs `lxmf_bytes` by concatenating: + +```text +destination_hash || source_hash || signature || packed_payload +``` + +| Part | Position | Confirmed representation | Evidence | +|---|---:|---|---| +| `destination_hash` | bytes `0..15` | 16 bytes | Source-confirmed: `LXMF/LXMessage.py:40`, `LXMF/LXMessage.py:383`; Vector-confirmed | +| `source_hash` | bytes `16..31` | 16 bytes | Source-confirmed: `LXMF/LXMessage.py:384`; Vector-confirmed | +| `signature` | bytes `32..95` | 64 bytes | Source-confirmed: `LXMF/LXMessage.py:41`, `LXMF/LXMessage.py:385`; Vector-confirmed position and length only | +| `packed_payload` | bytes `96..end` | MessagePack array | Source-confirmed: `LXMF/LXMessage.py:362`, `LXMF/LXMessage.py:381-386`; Vector-confirmed | + +The 16-byte hash length derives from +`RNS.Identity.TRUNCATED_HASHLENGTH // 8`. The checked-in Reticulum source sets +the truncated hash length to 128 bits. The 64-byte signature length derives +from `RNS.Identity.SIGLENGTH // 8`. See `RNS/Reticulum.py:146-147` and +`RNS/Identity.py:80-83` in the evidence-baseline Reticulum checkout. + +### Payload + +For messages generated by `LXMessage.pack()`, the payload is: + +```text +[ + timestamp, + title, + content, + fields +] +``` + +If a message stamp is available when packing, it is appended: + +```text +[ + timestamp, + title, + content, + fields, + stamp +] +``` + +| Index | Name | Confirmed behavior | Evidence | +|---:|---|---|---| +| `0` | `timestamp` | Set from `time.time()` when not already set, then packed as the first payload value | Source-confirmed: `LXMF/LXMessage.py:357`, `LXMF/LXMessage.py:362`; Vector-confirmed as MessagePack float64 for the deterministic vectors | +| `1` | `title` | Constructor string input is UTF-8 encoded to bytes; byte input is retained | Source-confirmed: `LXMF/LXMessage.py:130-133`, `LXMF/LXMessage.py:193-197`; Vector-confirmed as MessagePack binary | +| `2` | `content` | Constructor string input is UTF-8 encoded to bytes; byte input is retained | Source-confirmed: `LXMF/LXMessage.py:135-136`, `LXMF/LXMessage.py:202-206`; Vector-confirmed as MessagePack binary | +| `3` | `fields` | Constructor input must be a dictionary or `None`; `None` becomes an empty dictionary | Source-confirmed: `LXMF/LXMessage.py:138`, `LXMF/LXMessage.py:215-219`; Vector-confirmed for an empty MessagePack map | +| `4` | `stamp` | Optional value appended after the four base payload values | Source-confirmed: `LXMF/LXMessage.py:371-373`; Vector-confirmed as MessagePack binary in the stamped vector | + +The payload order above is source- and vector-confirmed. It differs from the +order stated in the upstream `README.md`, which lists content before title. + +### Message ID + +The message ID, also stored as `LXMessage.hash`, is: + +```text +SHA-256( + destination_hash || + source_hash || + msgpack([timestamp, title, content, fields]) +) +``` + +The optional stamp is excluded from the MessagePack payload used to calculate +the message ID. This is source-confirmed by both packing and unpacking behavior, +and vector-confirmed by the minimal and stamped vectors producing the same +message ID. See `LXMF/LXMessage.py:362-369` and +`LXMF/LXMessage.py:754-764`. Reticulum defines `full_hash()` as SHA-256 at +`RNS/Identity.py:238-246`. + +The message ID is not included in `lxmf_bytes`. + +### Signature input + +`LXMessage.pack()` requests a signature over: + +```text +destination_hash || +source_hash || +msgpack([timestamp, title, content, fields]) || +message_id +``` + +The optional stamp is excluded from the signature input. The construction of +the signature input is source-confirmed at `LXMF/LXMessage.py:375-378` and +`LXMF/LXMessage.py:762-764`, and vector-confirmed. + +The deterministic vectors contain placeholder signature bytes. They confirm +the signature position and input bytes, but do not confirm signing or signature +validation. + +### Delivery representations + +The core `lxmf_bytes` representation is modified or wrapped for some delivery +methods: + +| Delivery representation | Confirmed serialized data | Evidence | +|---|---|---| +| Direct packet or resource | Full `lxmf_bytes` | Source-confirmed: `LXMF/LXMessage.py:635-636`, `LXMF/LXMessage.py:653-654` | +| Opportunistic packet | `lxmf_bytes` without the leading destination hash | Source-confirmed: `LXMF/LXMessage.py:633-634` | +| Propagated message data | `destination_hash || encrypt(lxmf_bytes after destination_hash)` | Source-confirmed: `LXMF/LXMessage.py:429-436` | +| Propagation transfer wrapper | MessagePack `[time.time(), [propagated_message_data]]` | Source-confirmed: `LXMF/LXMessage.py:436` | +| Paper message data | `destination_hash || encrypt(lxmf_bytes after destination_hash)` | Source-confirmed: `LXMF/LXMessage.py:446-451` | +| Paper URI | URL-safe Base64 of paper message data, without `=` padding, prefixed by `lxm://` | Source-confirmed: `LXMF/LXMessage.py:698-707` | + +These delivery representations are not covered by the current deterministic +test vectors. + +### Unresolved behavior + +The following behavior is not established by the current source inspection and +test-vector coverage: + +- Normative requirements for independent LXMF implementations. +- Cross-implementation Ed25519 signature generation and validation. +- Accepted signature encodings beyond the fixed 64-byte position generated by + the checked-in implementation. +- A universal MessagePack float width for timestamps on every supported + platform. The deterministic vectors confirm float64 only for those vectors. +- Interoperable constraints on the contents, key types, value types, ordering, + and nesting depth of `fields`. +- Interoperable constraints on stamp length and encoding. +- Required handling of malformed, truncated, non-canonical, or payload arrays + containing fewer than four or more than five entries. +- Byte-for-byte vectors for opportunistic, propagated, paper, URI, encrypted, + or persisted-container representations.