Compare commits

...

10 commits

Author SHA1 Message Date
Rob
ae5738ea2f spec: §5.9.8 reactions + §5.9.9 reply-to (app extensions) + §5.5 raw-wire-bytes rule
Resolves #8. Documents the convergent fields[16] / fields[0x30]+[0x31]
shapes shared by reticulum-mobile-app, MeshChatX, and Columba.

§5.5 gains the normative line that msgpack_payload MUST be hashed from
raw wire bytes — without it, reactions/replies miss the relay rewrite
cache for any message whose fields don't round-trip byte-identically.

§5.9.8 (tap-back reactions on fields[16]) and §5.9.9 (reply-to on
fields[0x30] + optional fields[0x31]) carry > UNVERIFIED markers since
these keys are outside the upstream LXMF allocation range. Columba's
legacy fields[16] = {reply_to} shape is recorded as an inbound-only
tolerance branch with a finite lifetime — torlando-tech/columba#926
shipped the dual-key shape in v2.0.0-beta on 2026-05-23.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 08:36:39 -04:00
Rob
41d2fd61ee spec: §6.7.6 LINKIDENTIFY wire format + §11.6 prose
§6 specified the wire format of every Link control packet — LRRTT
(§6.4.2), KEEPALIVE (§6.7.1), LINKCLOSE (§6.7.3) — but had no
subsection for LINKIDENTIFY (`context = 0xFB`). §11.6 described it
only in prose, and called the payload "a signature over `link_id`" —
imprecise on two counts.

Per RNS 1.2.9 `RNS/Link.py:459-475` `Link.identify()`:
  - Payload is `public_key(64) || signature(64)` (128 bytes total),
    NOT just the signature.
  - `signature` is over `link_id || public_key`, NOT over `link_id`
    alone.

LINKIDENTIFY is a DATA packet on the active link, so it IS
link-encrypted like ordinary link DATA (unlike KEEPALIVE / link
PROOF, §6.7.1 / §6.5). Added explicit §6.7.6 subsection with the
wire layout, the responder-side parse path, and the two clean-room
pitfalls; corrected the §11.6 prose to match and point at the new
subsection.

Closes #12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:34:32 -04:00
Rob
7a85385bb4 spec: §6.5.1 — Link DATA proof is signed with an Ed25519 key, not the link-derived signing key
The §6.5.1 bullet said the Link DATA proof's signing key is "the
link-derived signing key" — misleading. The link's HKDF output
(§6.4.1) splits into a symmetric HMAC `signing_key` (for the link
Token form, §3.1) and an `encryption_key`; that signing_key cannot
produce an Ed25519 signature.

Per RNS 1.2.9, the Link DATA proof is signed with the link's Ed25519
`sig_prv` (`RNS/Link.py:1212-1213` `sign()` uses `self.sig_prv`):
  - responder side: `owner.identity.sig_prv` (long-term identity
    Ed25519 private key, `Link.py:279`)
  - initiator side: a fresh ephemeral Ed25519 keypair generated at
    link creation (`Ed25519PrivateKey.generate()`, `Link.py:286`)

§6.5.1 now states this explicitly and distinguishes it from the
symmetric §6.4.1 signing_key.

Closes #11.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:34:32 -04:00
Rob
dee108e787 spec: §6.7.1 — KEEPALIVE is not Token-encrypted and does not generate a PROOF receipt
The §6.7.1 prose said KEEPALIVE is Token-encrypted (wire body
`iv(16) || ciphertext || hmac(32)`) and generates a mandatory PROOF
receipt like other link DATA. Both claims contradict the reference.

Verified against RNS 1.2.9:
- `RNS/Packet.py:206-209` `pack()` puts `KEEPALIVE` in its
  not-encrypted branch — the wire body is the sentinel byte in the
  clear, alongside RESOURCE / RESOURCE_PRF / link PROOF / CACHE_REQUEST.
- `RNS/Link.py:988` gates the link-DATA proof path on
  `context == NONE`; KEEPALIVE takes its own branch at line 1149.

A clean-room receiver that expected `iv(16) || ciphertext || hmac(32)`
per the old text would fail to parse a real KEEPALIVE.

Closes #10.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:34:32 -04:00
Rob
375521c963
spec: §10.7 — an exhausted RESOURCE_REQ may carry parts
A conformant sender fulfils any bundled `requested_map_hashes` AND
sends the RESOURCE_HMU. Verified against RNS 1.2.9 (`Resource.py:982-1071`):
part fulfilment runs unconditionally for every REQ, and the HMU branch
runs in addition. The reference receiver (`request_next`) routinely
bundles parts into an exhausted REQ. §10.7 now states the correct
rule; part-less exhausted REQs are an allowed receiver-side
simplification. `playbook.md` §7 records the matching fwdsvc
conformance bug (since fixed in `reticulum-forwarding-service` PR #10).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:19:56 -04:00
Rob
4c1d3de421 spec: §4.6 — rrc.hub app_data is CBOR, not msgpack
Correction. RRC's wire codec is CBOR throughout (rrcd/codec.py uses
Python cbor2), so the rrcd announce app_data {"proto","v","hub"} is
a CBOR map, not a msgpack map as §4.6 first stated.

Added the concrete gotcha that motivated the correction: a CBOR
3-entry map starts 0xa3, which msgpack reads as fixstr-3 — so a naive
msgpack decode of the app_data yields the bogus string "epr" (the
CBOR text-string header of "proto" + its first two chars). Clients
must CBOR-decode rrc.hub app_data, keyed on the name_hash.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 14:47:14 -04:00
Rob
a9a5857122 spec: §4.6 — rrc.hub announce app_data shape
Reticulum Relay Chat hubs announce on the `rrc.hub` aspect, and the
app_data is NOT an LXMF [name, cost] array. The two hubs disagree:
rrcd (Python) emits a msgpack map {"proto","v","hub"} with the human
name under "hub"; the Go hub emits the name as plain UTF-8 bytes.
Documents both, the resolve order a client should use, and that the
authoritative name also arrives in the RRC WELCOME (B_WELCOME_HUB).

Confirmed from upstream source — rrcd service.py and
reticulum-relay-chat internal/service/service.go.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 13:47:05 -04:00
Rob
fd2951be82 spec: pin FIELD_FILE_ATTACHMENTS (0x05) value shape
§5.9.7 — FIELD_FILE_ATTACHMENTS is a list of [filename, file_bytes]
pairs (multiple attachments per message). Confirmed from upstream
Sideband source: core.py builds `fields[FIELD_FILE_ATTACHMENTS] =
[attachment]` with `attachment = [filename, filedata]`; ui/messages.py
reads `attachment[0]`/`attachment[1]` on receive and strips `../`
from the filename.

Removed FIELD_FILE_ATTACHMENTS from the §5.9 UNVERIFIED list and
added a §5.9.7 pointer to the field table. Documents the
sender-controlled-filename sanitisation requirement.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 12:40:05 -04:00
Rob
ef3d98963b docs(agent): add rule to track latest RNS and verify packages with rnid
§3 now requires confirming the latest upstream version each session and
verifying package signatures before install. Records that upstream is
migrating off GitHub (1.2.5 ≈ last GitHub release) toward rngit/rnpkg
self-hosting over Reticulum, and that signed wheels must be rnid-checked
against Mark Qvist's release identity rather than trusting a bare
`pip install` from PyPI (PyPI carries no .rsg).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 10:34:22 -04:00
Rob
9c64f9b805 docs(spec): add §17.5 — application protocols layered over Reticulum
Adds a reader-navigation subsection pointing RRC (Reticulum Relay Chat)
implementers at the RNS-layer sections they need. RRC carries its own
authoritative spec (rrc.kc1awv.net); this only maps its dependencies
onto §§ here and notes the skip-list (§5/§10/§11 unused). No RRC wire
format is restated and no normative claim is added.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 10:34:21 -04:00
3 changed files with 245 additions and 7 deletions

220
SPEC.md
View file

@ -34,6 +34,7 @@ Source citations refer to the standard `pip install rns lxmf` install layout (`R
- [4.3 `app_data` format for LXMF delivery destinations](#43-app_data-format-for-lxmf-delivery-destinations)
- [4.4 Announce filtering by `name_hash`](#44-announce-filtering-by-name_hash)
- [4.5 Announce validation rules (receive side)](#45-announce-validation-rules-receive-side)
- [4.6 `rrc.hub` announce app_data (Reticulum Relay Chat)](#46-rrchub-announce-app_data-reticulum-relay-chat)
- [5. LXMF wire format](#5-lxmf-wire-format)
- [5.1 Opportunistic delivery (single Reticulum DATA packet)](#51-opportunistic-delivery-single-reticulum-data-packet)
- [5.2 Direct delivery (over an established Reticulum Link)](#52-direct-delivery-over-an-established-reticulum-link)
@ -150,6 +151,7 @@ Source citations refer to the standard `pip install rns lxmf` install layout (`R
- [17.2 Section relevance by category](#172-section-relevance-by-category)
- [17.3 Worked example: §2.3 originator HEADER_1→HEADER_2 conversion](#173-worked-example-23-originator-header_1header_2-conversion)
- [17.4 Pragmatic implication](#174-pragmatic-implication)
- [17.5 Application protocols layered over Reticulum](#175-application-protocols-layered-over-reticulum)
- [18. Test vectors](#18-test-vectors)
- [19. Source map](#19-source-map)
@ -624,6 +626,42 @@ These are not wire-spec MUST rules but most working clients implement them; with
| `RNS/Interfaces/Interface.py:60-200` | ingress-limit constants, `should_ingress_limit`, `hold_announce`, `process_held_announces` |
| `RNS/Packet.py:83` | `PATH_RESPONSE = 0x0B` context constant |
### 4.6 `rrc.hub` announce app_data (Reticulum Relay Chat)
Reticulum Relay Chat hubs announce a destination on the `rrc.hub`
aspect — `name_hash = SHA256("rrc.hub")[:10] = ac9fd3a81e4036f86e1d`.
Unlike §4.3 (LXMF delivery), the `app_data` is **not** a msgpack
`[name, cost]` array, and the two hub implementations disagree on
its shape:
- **`rrcd`** — the Python reference hub. `app_data` is a **CBOR**
(RFC 8949) map: `{"proto": "rrc", "v": 1, "hub": <hub_name>}`
CBOR, *not* msgpack, because RRC's wire codec (`rrcd/codec.py`,
Python `cbor2`) is CBOR throughout. The human hub name is the
`"hub"` key's value (a text string); `"proto"` is always `"rrc"`
and `"v"` is the app_data schema version (`1`). Source: `rrcd`
`service.py` — `app_data = encode({"proto": "rrc", "v": 1, "hub":
self.config.hub_name})`, where `encode` is the CBOR encoder.
- **`reticulum-relay-chat`** — the Go hub. `app_data` is the hub name
as **plain UTF-8 bytes**, unwrapped. Source:
`internal/service/service.go` — `BuildAnnounce(id, "rrc.hub",
[]byte(s.cfg.Hub.Name), ...)`.
> ⚠️ **CBOR-vs-msgpack gotcha.** A CBOR 3-entry map begins with byte
> `0xa3`. In msgpack `0xa3` is `fixstr` of length 3 — so a client that
> blindly msgpack-decodes the `rrcd` app_data reads the next three
> bytes (`0x65 0x70 0x72`, the CBOR text-string header of `"proto"`
> plus its first two characters) as the 3-character string `"epr"`.
> Decode `rrc.hub` app_data with a CBOR decoder, keyed on the
> `rrc.hub` name_hash — do not feed it to the LXMF (msgpack) app_data
> parser.
A client listing RRC hubs should resolve the name as: the `"hub"`
value when `app_data` CBOR-decodes to a map; else a bare UTF-8 string
(the Go hub's shape); else a generic "RRC hub" label. The same name
is also delivered authoritatively in the RRC `WELCOME` body key
`B_WELCOME_HUB` once a session is established.
---
## 5. LXMF wire format
@ -674,6 +712,8 @@ signature = Ed25519_sign(signed_data, sender_identity.Ed25519_priv)
For opportunistic delivery, `destination_hash` is the recipient's destination hash (from the outer packet header, not from the LXMF body).
The `msgpack_payload` MUST be the exact bytes received on the wire — not a decode-then-re-encode — when computing `message_hash`. The same value is reused as the LXMF `message_id` (workblock input for stamps in §5.7, target identifier for reactions in §5.9.8 and replies in §5.9.9). A receiver that re-encodes will compute a divergent identifier for any message whose `fields` map does not round-trip byte-identically — most commonly reply messages — which breaks both the §5.6 path-1 raw-signature check and any relay rewrite-cache keyed on the value. For a stamped message (§5.7.1) the raw payload on the wire is a 5-element array; the hash is over the first 4 elements, which requires a byte-canonical re-pack per §5.6.1.
### 5.6 Signature verification — msgpack variant tolerance
Different msgpack encoders produce subtly different byte sequences for the same logical value (e.g. integer encoding choice, string vs bin selection). The signer signed over THEIR encoder's output. A receiver should try verifying against:
@ -962,7 +1002,7 @@ Sender and receiver agree on these keys; each value's structure is field-specifi
| `0x02` | `FIELD_TELEMETRY` | A single telemetry snapshot (Sideband telemetry — see Sideband for the inner format; LXMF is opaque to the contents). |
| `0x03` | `FIELD_TELEMETRY_STREAM` | A list of telemetry snapshots (history flush). |
| `0x04` | `FIELD_ICON_APPEARANCE` | Sender-supplied avatar / appearance hint. |
| `0x05` | `FIELD_FILE_ATTACHMENTS` | A list of attached files (multiple attachments per message). |
| `0x05` | `FIELD_FILE_ATTACHMENTS` | A list of attached files (multiple attachments per message). See §5.9.7 for the wire shape. |
| `0x06` | `FIELD_IMAGE` | Single embedded image — `[extension_string, image_bytes]`. See §5.9.2 for the wire shape. |
| `0x07` | `FIELD_AUDIO` | Single embedded audio clip — `[mode_byte, audio_bytes]`. Mode byte chooses the codec; see §5.9.3. |
| `0x08` | `FIELD_THREAD` | Conversation thread ID (links related messages). |
@ -979,7 +1019,7 @@ Sender and receiver agree on these keys; each value's structure is field-specifi
| `0xFE` | `FIELD_NON_SPECIFIC` | Development / unstructured payload — not for production. |
| `0xFF` | `FIELD_DEBUG` | Debug payload — not for production. |
> ⚠️ **UNVERIFIED:** the byte-level shape of `FIELD_EMBEDDED_LXMS`, `FIELD_TELEMETRY*`, `FIELD_FILE_ATTACHMENTS`, `FIELD_COMMANDS`, `FIELD_RESULTS`, `FIELD_GROUP`, `FIELD_EVENT`, and `FIELD_RNR_REFS` is not described here because no test vectors have been captured against upstream Sideband emissions for these. The constants are verified (see `tools/verify_lxmf_fields.py`) but the value structures are application-defined and not pinned by LXMF itself. Future PRs should add per-field byte layouts as test vectors arrive.
> ⚠️ **UNVERIFIED:** the byte-level shape of `FIELD_EMBEDDED_LXMS`, `FIELD_TELEMETRY*`, `FIELD_COMMANDS`, `FIELD_RESULTS`, `FIELD_GROUP`, `FIELD_EVENT`, and `FIELD_RNR_REFS` is not described here because no test vectors have been captured against upstream Sideband emissions for these. The constants are verified (see `tools/verify_lxmf_fields.py`) but the value structures are application-defined and not pinned by LXMF itself. Future PRs should add per-field byte layouts as test vectors arrive. (`FIELD_FILE_ATTACHMENTS` was on this list until 2026-05-18 — its shape is now documented in §5.9.7 from upstream Sideband source.)
#### 5.9.2 `FIELD_IMAGE` (`0x06`) value shape
@ -1061,6 +1101,75 @@ For announce-level capability negotiation:
|---|---|---|
| `0x00` | `SF_COMPRESSION` | Sender supports compressed message bodies (see §10.12) |
#### 5.9.7 `FIELD_FILE_ATTACHMENTS` (`0x05`) value shape
```
fields[0x05] = [ [filename(str-or-bytes), file_bytes(bytes)], ... ]
```
A list of attachments — one LXMF message may carry more than one
file. Each attachment is itself a 2-element list: element `[0]` is the
file name, element `[1]` is the raw file content. As with `FIELD_IMAGE`
(§5.9.2) the file name may arrive as msgpack `str` or `bin` depending
on the encoder — receivers must tolerate both (see §9.3).
The file name is **sender-controlled and untrusted**. Upstream Sideband
strips `../` from it on receive (`sbapp/ui/messages.py`:
`filename = str(attachment[0]).replace("../", "")`); receivers MUST
sanitise it — reject or strip path separators, `..` segments and
control characters — before display or save, and never let it
influence a write path. The extension is likewise untrusted: do not
auto-open or auto-execute an attachment based on its claimed type.
Files larger than a single Reticulum DATA packet are delivered as a
Resource over a Link, identically to large `FIELD_IMAGE` payloads
(§10).
Source: `markqvist/Sideband` `sbapp/sideband/core.py`
(`fields[LXMF.FIELD_FILE_ATTACHMENTS] = [attachment]`, where each
`attachment = [filename, filedata]`) and `sbapp/ui/messages.py`
(receive side indexes `attachment[0]` filename / `attachment[1]`
bytes). Confirmed from upstream source 2026-05-18; a captured wire
test vector would further pin the msgpack `str`-vs-`bin` choice.
#### 5.9.8 `fields[16]` (`0x10`): tap-back reactions (app extension)
> ⚠️ **UNVERIFIED:** this key is outside the upstream LXMF allocation range (§5.9.1) — there is no `FIELD_REACTION` constant in `LXMF/LXMF.py`. The shape below is a convergent app-extension across three shipping FOSS clients (`reticulum-mobile-app`, `Quad4-Software/MeshChatX`, `torlando-tech/columba` `release/v0.10.x` and later). Documented here because the convergence is stable enough for new implementations to interop. Replace with the upstream allocation if/when `markqvist/LXMF` blesses one.
```
fields[16] = {
"reaction_to": <target LXMF message_id, 64-char hex string>,
"emoji": <unicode>,
"sender": <reactor's identity hash, 32-char hex string>,
}
```
- The carrying LXMF has empty `content` and empty `title`; `fields[16]` IS the entire payload.
- `reaction_to` is the canonical LXMF `message_id` (= `message_hash` of §5.5; the same value used as the workblock input for stamps in §5.7). It MUST be derived from the recipient-side raw wire payload bytes per the §5.5 normative rule — a decode-re-encode diverges for any non-trivial `fields` map (including reply-carrying messages, §5.9.9) and the reaction misses the relay rewrite-cache for that target.
- `sender` is the reactor's **identity hash** (`SHA256(identity_public_key)[:16]`), NOT the lxmf.delivery `source_hash`. Both are 16-byte SHA-256 truncations and trivially easy to conflate, but receivers key reaction aggregation by `(emoji, sender)`, so emitting the destination hash mis-buckets against every (identity-hash-emitting) peer. All three convergent clients emit the identity hash (`python/reticulum_wrapper.py::send_reaction` in Columba `release/v0.10.x` and `v2.0.0-beta+`; `meshchat.py::send_reaction` in MeshChatX; `ReticulumEngine.kt::sendReaction` in `reticulum-mobile-app`).
- Receivers MUST aggregate by `(emoji, sender)` with dedup and SHOULD NOT render the reaction-carrying LXMF as a separate bubble.
- String-vs-bytes tolerance: msgpack values may arrive as `str` or `bin` depending on encoder. Receivers MUST accept both — same precedent as §5.6 for the LXMF signature.
- **Inner-map decode tolerance.** The `fields[16]` value is a msgpack map. Strongly-typed decoders may surface it as either a string-keyed map or an `Any`-keyed map (`Map<String, Any>` / `Map<Any?, Any?>`, `map[string]any` / `map[any]any`, etc.), depending on the runtime msgpack library's choice given the key types seen on the wire. Receivers MUST tolerate both via a runtime cast that does not depend on the outer map's static key type. A silent type-assertion failure on the "wrong" variant produces a no-log no-error drop indistinguishable on the receive side from "the message never arrived."
- **Relay routing.** If the target message arrived over a Link whose validated §6.7.6 LINKIDENTIFY peer differs from the LXMF body's `source_hash`, the reaction-carrying LXMF MUST egress to the link peer's destination hash, not `source_hash`; otherwise the reaction bypasses the relay's fanout and is delivered direct to the original sender instead of the relay group.
#### 5.9.9 `fields[0x30]` + optional `fields[0x31]`: reply-to threading (app extension)
> ⚠️ **UNVERIFIED:** as with §5.9.8, these keys are outside the upstream LXMF allocation range. The shape below is the convergent app-extension across the same three FOSS clients. Columba's earlier `release/v0.10.x` builds used a legacy single-key shape (`fields[16] = {"reply_to": <hex>}`); Columba `v2.0.0-beta` (released 2026-05-23) and later emit the dual-key shape per `torlando-tech/columba#926`. The legacy-tolerance branch in the inbound rule below has a finite lifetime — drop it once `< v2.0.0-beta` Columba is no longer in the field.
```
fields[0x30] = <raw 32-byte LXMF message_id (NOT hex-encoded)>
fields[0x31] = <UTF-8 quoted content> # optional
```
- `fields[0x30]` IS the canonical LXMF `message_id` from §5.7 — the raw 32 bytes on the wire, not a hex-encoded string. The `message_id` MUST be computed over raw wire payload bytes per §5.5; a decode-re-encode diverges for reply messages (their `fields` map is non-trivial) and breaks any relay rewrite-cache keyed on the value.
- `fields[0x31]` (optional) carries the quoted text for offline-receiver fallback so the recipient can render a quote preview even when the original message hasn't arrived locally. Useful on intermittent links; SHOULD be omitted when the sender doesn't want to pay the airtime.
- **Legacy-tolerance (inbound only).** Receivers SHOULD also accept Columba's earlier `fields[16] = {"reply_to": <64-char hex>}` shape for interop with already-deployed `release/v0.10.x` builds. Senders MUST NOT emit it — emit only `[0x30]`/`[0x31]`. The same inner-map dual-shape tolerance described in §5.9.8 applies to this legacy parse branch (the value is a msgpack map).
- **Bandwidth rationale.** The dual-key shape is roughly 2× smaller than the legacy single-key shape for the same information — 32 raw bytes + 1-byte key vs. a msgpack-encoded dict carrying a hex-string hash. This is the [Zen of Reticulum](https://reticulum.network/manual/zen.html) alignment cited by the MeshChatX maintainer in `Quad4-Software/MeshChatX#14` and accepted by the Columba maintainer in `torlando-tech/columba#926`.
- The raw-bytes branches (`fields[0x30]`, `fields[0x31]`) are not maps and do not have the §5.9.8 inner-map risk, but receivers MUST still tolerate both `bytes` and `str` carriers per §5.6 in case an encoder ships the hash hex-encoded by mistake.
- **Relay routing.** Same as §5.9.8 — if the target message arrived over a Link whose §6.7.6 LINKIDENTIFY peer differs from `source_hash`, the reply-carrying LXMF MUST egress to the link peer's destination hash.
Reference implementations: `Quad4-Software/MeshChatX` (`meshchatx/src/backend/lxmf_utils.py`; `meshchat.py::send_reaction`); `torlando-tech/columba` v2.0.0-beta and later (`python/reticulum_wrapper.py`); `thatSFguy/reticulum-mobile-app` (`shared/.../engine/ReticulumEngine.kt::extractField16`, `sendReaction`, `tryDeliverOverLink::replyFields`).
### 5.10 Source
`LXMF/LXMessage.py` for pack/unpack; `LXMF/LXMF.py` for the app_data extraction helpers and the field/audio/renderer constants enumerated in §5.9; `LXMF/LXStamper.py` for stamps; `LXMF/LXMRouter.py` for receive-side stamp/ticket dispatch and propagation handlers; `LXMF/LXMPeer.py` for the propagation peer-to-peer state machine.
@ -1229,7 +1338,7 @@ implicit body = signature(64)
Where:
- `packet_hash = Identity.full_hash(original_packet.get_hashable_part())` — the full SHA-256 (32 bytes, **not** truncated to 16) of the prove-target packet's hashable part. `get_hashable_part` is the same recipe used for `link_id` derivation in §6.3, so the proof binds to the version of the packet that survived any HEADER_1↔HEADER_2 conversion in transit (the high nibble of flags, hops byte, and any HEADER_2 transport_id are stripped before hashing).
- `signature` is the destination's (or link's) Ed25519 signature **over `packet_hash`**, NOT over the proof body itself. The signing key is the destination's long-term Ed25519 private key for an opportunistic DATA proof, or the link-derived signing key for a Link DATA proof.
- `signature` is the destination's (or link's) Ed25519 signature **over `packet_hash`**, NOT over the proof body itself. The signing key is the destination's long-term Ed25519 private key for an opportunistic DATA proof, or **the link's Ed25519 signing key** (`Link.sig_prv`, `RNS/Link.py:279` / `:286` in RNS 1.2.9) for a Link DATA proof — the owner identity's long-term key on the responder side, the link's ephemeral Ed25519 keypair on the initiator side. (The HKDF-derived `signing_key` from §6.4.1 is a separate **symmetric HMAC key** for the link Token form §3.1 — it cannot produce an Ed25519 signature and is not used here.)
The two forms are distinguished **purely by length** at the receiver. `PacketReceipt.validate_proof` (`RNS/Packet.py:497-548`) dispatches on `len(proof) == 96` (explicit) vs `len(proof) == 64` (implicit); lengths matching neither are rejected outright. There is no flag bit or context byte that signals which form is being used — wire length is the only signal.
@ -1433,7 +1542,7 @@ def send_keepalive(self):
keepalive_packet.send()
```
Body is a single byte `0xFF` — the "ping" sentinel. The packet is Token-encrypted with the link's session key per §3.1 link-derived form, so the wire body is `iv(16) || ciphertext(...) || hmac(32)`; the decrypted plaintext is just `b'\xff'`.
Body is a single byte `0xFF` — the "ping" sentinel. **KEEPALIVE is NOT Token-encrypted**`RNS/Packet.py` `pack()` puts `KEEPALIVE` in its not-encrypted branch alongside `RESOURCE`, `RESOURCE_PRF`, link `PROOF`, and `CACHE_REQUEST` (`self.ciphertext = self.data`, `Packet.py:206-209` in RNS 1.2.9). The wire body is the single sentinel byte in the clear — no IV, no ciphertext expansion, no HMAC.
The **responder** receives this in `Link.receive` at `RNS/Link.py:1149-1153` and answers with the "pong" sentinel (in 1.2.4 the body is `bytes([0xFE])`):
@ -1449,7 +1558,7 @@ So:
- **Pong** = responder → initiator, body `0xFE`.
- Only the initiator originates KEEPALIVE traffic. The responder never spontaneously pings.
Both sentinel bytes are arbitrary; what actually matters for keep-alive purposes is that *any* inbound traffic on the link refreshes `last_inbound` (the watchdog's anchor for staleness decisions). KEEPALIVE packets, like all link DATA, also generate the mandatory PROOF receipt per §6.5, which is itself inbound traffic on the return path. So a successful ping/pong exchange resets the staleness clock on **both** sides via three round-trip artifacts: ping → pong → pong-proof.
Both sentinel bytes are arbitrary; what actually matters for keep-alive purposes is that *any* inbound traffic on the link refreshes `last_inbound` (the watchdog's anchor for staleness decisions). KEEPALIVE packets do **not** generate a PROOF receipt — the link-DATA proof path in `RNS/Link.py` `receive()` is gated on `packet.context == RNS.Packet.NONE` (`Link.py:988` in RNS 1.2.9), and KEEPALIVE takes its own branch at line 1149-1153. So a successful ping/pong exchange resets the staleness clock on **both** sides via the two-packet exchange itself: ping → pong (no pong-proof).
A clean-room responder MUST emit the pong on inbound `0xFF`; without it the initiator's watchdog will declare the link stale on the next cycle.
@ -1530,6 +1639,59 @@ For a clean-room implementation that wants links to survive idle periods longer
5. On every inbound `LINKCLOSE`, decrypt, verify body equals `link_id`, transition to `CLOSED`.
6. On `CLOSED`, zero the session keys and cancel any in-progress Resources.
#### 6.7.6 LINKIDENTIFY (`context = 0xFB`)
A separate Link control DATA packet — not part of the keepalive /
teardown cycle, but documented here alongside the other context-dispatched
Link control packets. Used by the initiator to prove which long-term
identity is making the request without re-running the Link handshake;
§11.6 covers the calling context on the NomadNet REQUEST path.
`Link.identify(identity)` (`RNS/Link.py:459-475` in RNS 1.2.9):
```python
def identify(self, identity):
if self.initiator and self.status == Link.ACTIVE:
signed_data = self.link_id + identity.get_public_key()
signature = identity.sign(signed_data)
proof_data = identity.get_public_key() + signature
proof = RNS.Packet(self, proof_data, RNS.Packet.DATA,
context = RNS.Packet.LINKIDENTIFY)
proof.send()
```
Wire body (128 bytes):
```
public_key(64) || signature(64)
```
- `public_key` is the initiator's **full 64-byte Identity public key**
— the same `get_public_key()` that LINKREQUEST and announces use,
the concatenation of the Ed25519 verification key and the X25519
encryption key per §3.
- `signature` is `identity.sign(link_id(16) || public_key(64))` — the
signature is over the link_id concatenated with the public_key,
NOT over `link_id` alone. The responder verifies with `public_key`
over that same concatenation.
The packet is a `DATA` packet on the active Link, so it IS
link-encrypted per §3.1 link-derived form like ordinary link DATA —
`context = 0xFB` is NOT in `RNS/Packet.py` `pack()`'s not-encrypted
set, unlike KEEPALIVE (§6.7.1) or link `PROOF` (§6.5).
The responder's `receive()` (`RNS/Link.py:1010-1029`) decrypts the
body, splits off the 64-byte public_key prefix, reconstructs
`signed_data = link_id || public_key`, verifies the signature, and on
success sets `self.remote_identity` so subsequent REQUEST handlers can
check the caller against per-page allowlists (§11.6).
A clean-room implementation MUST build the payload as
`public_key || signature` (NOT just the signature) and sign the
concatenation `link_id || public_key` (NOT `link_id` alone). Either
mistake makes every `ALLOW_LIST`-protected page return
`DEFAULT_NOTALLOWED`.
### 6.8 Channel mode (`CHANNEL = 0x0E`)
A Channel is a **continuous, bi-directional, message-typed stream** on top of an established Link. Distinct from §11 REQUEST/RESPONSE (single-shot, client-server) and §10 Resources (large unidirectional transfers): Channel messages are short, can flow in either direction at any time, and carry an application-defined type byte the receiver dispatches on. NomadNet uses it for its "channel" API (live chat over a Link), and any application can register custom message types via `RNS.Channel.Channel.register_message_type`.
@ -2439,6 +2601,34 @@ The segment_index is `part_index // HASHMAP_MAX_LEN`. The receiver applies this
If the part_index doesn't land on a `HASHMAP_MAX_LEN` boundary, the sender treats it as a sequencing error and cancels the resource (`Resource.py:1043-1046`).
> **An exhausted RESOURCE_REQ MAY still carry parts — a conformant
> sender fulfils them *and* sends the RESOURCE_HMU.** When the
> `hashmap_exhausted_flag` is `0xFF`, the REQ body may still end with a
> non-empty `requested_map_hashes` trailer (§10.5). The sender MUST emit
> the requested part packets **and** the hashmap continuation; the two
> are independent. Serving the HMU is not a substitute for fulfilling
> the bundled part requests.
>
> In the RNS reference (`Resource.py:982-1071`, `request()` — verified
> against RNS 1.2.9, the current release), the part-fulfilment loop runs
> for every REQ regardless of the flag, and the `if wants_more_hashmap:`
> HMU branch runs afterward, in addition. The reference receiver
> (`request_next`, `Resource.py:931-981`) routinely produces this
> packet shape: as its window scan reaches the end of the known
> hashmap, it has already accumulated the still-outstanding part-hashes
> from the known region into `requested_hashes`, then sets the exhausted
> flag and stops — emitting
> `0xFF || last_map_hash(4) || resource_hash(32) || requested_map_hashes`.
>
> A receiver MAY keep part requests and hashmap pulls in separate REQ
> packets — emitting a **part-less** exhausted REQ
> (body `0xFF || last_map_hash(4) || resource_hash(32)`, no trailing
> map_hashes) purely to pull the continuation. This interoperates with
> every conformant sender. But it is a receiver-side simplification
> only: a sender MUST NOT assume peers do this, and MUST NOT skip part
> fulfilment for an exhausted REQ. A sender that does drops every
> bundled part silently — see `playbook.md` §7 (2026-05-19).
### 10.8 RESOURCE_PRF — final proof
When the receiver has assembled the full resource (`received_count == total_parts`), it runs `assemble()` (`Resource.py:672-726`):
@ -2811,7 +3001,7 @@ Pages are registered with one of three allow modes (`Destination.py:35-40`):
- `ALLOW_LIST` — caller's identity hash must appear in the page's `.allowed` file. Server checks `remote_identity.hash` against the list at request time (`Node.py:152-154`).
- `ALLOW_NONE` — registered handlers that exist but reject all requests (rare; debug only).
For `ALLOW_LIST` the client MUST call `link.identify(identity)` immediately after the link transitions to ACTIVE and BEFORE issuing the REQUEST. This sends a `LINKIDENTIFY (context = 0xFB)` packet whose payload carries a signature over `link_id` proving the long-term identity hash. Without it, `remote_identity` is `None` server-side and every `ALLOW_LIST` page returns `DEFAULT_NOTALLOWED`. See `Browser.py:1245-1250` for the upstream call site:
For `ALLOW_LIST` the client MUST call `link.identify(identity)` immediately after the link transitions to ACTIVE and BEFORE issuing the REQUEST. This sends a `LINKIDENTIFY (context = 0xFB)` packet whose 128-byte payload is `public_key(64) || signature(64)`, with the signature computed over `link_id || public_key` — proving the long-term identity hash to the responder. The wire format is specified in §6.7.6. Without it, `remote_identity` is `None` server-side and every `ALLOW_LIST` page returns `DEFAULT_NOTALLOWED`. See `Browser.py:1245-1250` for the upstream call site:
```python
def link_established(self, link):
@ -3546,6 +3736,24 @@ If you're a category-3 developer building from scratch, you need everything. Ver
If you're not sure which category you're in: `grep -r "import RNS" your_codebase` is a quick check. Any hit means cat 1 (or cat 2 if it's behind an FFI wall). No hits means cat 3.
### 17.5 Application protocols layered over Reticulum
Reticulum is a transport substrate; user-facing features are *application protocols* built on top. This spec documents **LXMF** (§5) in depth because its wire format is published nowhere else. Other application protocols carry their own authoritative specs and are **out of scope here** — but a clean-room author building a client for one still needs the RNS-layer sections below.
**Reticulum Relay Chat (RRC)** — a live, IRC-style chat protocol. Authoritative spec: <https://rrc.kc1awv.net/> (its CBOR envelope, message types, and state machines live there, not here). RRC is a category-3-style consumer of RNS: it defines its own wire format but relies entirely on Reticulum for transport. An RRC client built clean-room needs:
| RRC depends on | Sections in this spec |
|---|---|
| Hub destination (`rrc.hub` aspect) + client identity | §1.1, §1.2, §9.8 |
| Identity hash as the canonical sender id (RRC envelope key 4, "opaque, do not re-encode") | §1.1; §9.1 — the identity-hash-vs-destination-hash pitfall RRC's rule guards against |
| All traffic over a single Reticulum Link | §6.1§6.4, §6.7 |
| Single-packet CBOR frames sized to the link MTU | §2.1§2.2, §2.4, §6.6 (MTU is *negotiated* — see note below) |
| Ordered/reliable delivery, *if* RRC layers over Channel | §6.5, §6.8 |
RRC does **not** use opportunistic packets, Resource transfer (§10), REQUEST/RESPONSE (§11), or LXMF (§5) — an RRC-only client can skip those entirely.
> Two things RRC's spec leaves implicit that this spec makes explicit: link MTU is **negotiated** (§6.6), not a fixed 500 bytes, so frames sized to the default can overflow a smaller link; and a bare Link does not itself guarantee ordered/reliable delivery — that comes from Channel (§6.8) or packet receipts (§6.5).
---
## 18. Test vectors

View file

@ -55,12 +55,35 @@ PRs must include the verifier scripts. Don't commit a "verified" claim without t
Agents working on this repo should have access to:
- A working Python 3 install with `rns` and `lxmf` packages: `pip install rns lxmf`
- A working Python 3 install with `rns` and `lxmf` packages (install per "Staying current" below — not a bare `pip install rns lxmf`).
- The `RNS/` and `LXMF/` source trees (typically at `~/AppData/Roaming/Python/Python3xx/site-packages/RNS/` on Windows or `~/.local/lib/python3.x/site-packages/RNS/` on Linux/macOS).
- Optional but very useful: a packet-trace tool. `tcpdump -i lo -A -X port 4242` works for TCPServerInterface; for BLE you need ADB + an RNode-aware capture tool.
Hardware (RNode, RatDeck, etc.) is NOT required for most verification — most byte-level claims can be checked entirely in Python RNS without any radio.
### Staying current with upstream — and verifying what you install
This spec is only as good as the upstream version it was checked against. **Before any verification pass, confirm you are looking at the latest RNS:**
1. **Check the latest upstream version.** Upstream is mid-migration *off* GitHub: the RNS 1.2.4 release notes announced 1.2.4 would "probably be the last release also published to GitHub," and GitHub releases do stop around **1.2.5** (1.2.6/1.2.7 are PyPI-only). Reticulum is moving to self-hosting — `rngit` (git served over Reticulum) and `rnpkg` (package distribution over Reticulum). Per Mark Qvist, "updates to pip will continue at least until `rnpkg` is complete." Check, in order:
- PyPI: `pip index versions rns` / `pip index versions lxmf` — most current while it lasts, but PyPI carries **no `.rsg`**, so PyPI alone is unverifiable.
- GitHub releases: <https://github.com/markqvist/Reticulum/releases> — signed (`.rsg`) but frozen near 1.2.5.
- `rngit` / `rnpkg` over Reticulum — the eventual canonical source of signed artifacts.
PyPI runs *ahead* of the newest signed release (1.2.7 on PyPI vs 1.2.5 signed, 2026-05-17). Re-check every session — do not assume a frozen version.
2. **Verify the package signature before installing.** Since RNS 1.2.x, signed releases ship a detached `.rsg` signature beside each artifact (on the GitHub release page while that lasts; via `rngit`/`rnpkg` afterward). Get the wheel *and* its `.rsg`, then:
```
rnid -i bc7291552be7a58f361522990465165c -V rns-<version>-py3-none-any.whl
```
`bc7291552be7a58f361522990465165c` is Mark Qvist's release-signing Reticulum identity — confirm that hash itself against a trusted channel (a release announcement) before trusting it. `rnid` works fully offline. Install the **exact wheel you verified** (`pip install --upgrade ./rns-<version>-py3-none-any.whl`), not a fresh `pip install rns` that re-downloads from PyPI.
3. **Prefer the latest *signed* version.** If PyPI is ahead of the newest release carrying a `.rsg`, install the signed one and note the gap — do not silently install an unverifiable newer version.
4. **After a version bump**, re-run every `tools/verify_*.py`, re-check the source-cited line numbers in any section you touch, and only advance SPEC.md's `**Last verified against:**` line once the whole document has been re-checked against the new version.
---
## 4. Marking convention

View file

@ -191,6 +191,13 @@ Spec-only repos with a "the source is the source of truth" attitude die slowly b
Each entry: date, one-line symptom, spec section that governs it, one-line fix, one-sentence lesson. Append-only. New entries go at the top.
### 2026-05-19 — fwdsvc dropped parts bundled into an exhausted RESOURCE_REQ
- **Symptom:** Images relayed mobile→mobile through the Fwd service never arrive (whole LXMF message lost); mobile→Sideband through the same service works. Recipient logs hundreds of `RESOURCE chunk did not match any known hashmap slot`. Only triggers for resources large enough to need RESOURCE_HMU (>`HASHMAP_MAX_LEN` ≈ 74 parts).
- **Spec section:** §10.7. An `exhausted == 0xFF` RESOURCE_REQ MAY still carry a `requested_map_hashes` trailer, and a conformant sender serves those parts **and** the RESOURCE_HMU. The fwdsvc Go sender did `if req.Exhausted { serveHmu(req); continue }`, skipping `fulfillRequest` entirely — its own comment claimed this "mirrors upstream `Resource.request()`", but upstream (`Resource.py:982-1071`, checked against RNS 1.2.9, the current release) runs part fulfilment unconditionally and *then* sends the HMU. The mobile receiver flags `exhausted` on the first REQ of each hashmap window and bundles ~74 part-hashes with it — which a reference RNS sender honours — so fwdsvc served HMUs and dropped every bundled part across all 19 windows.
- **Fix:** `resource_sender.go` Run loop now runs `fulfillRequest` for every REQ, then `serveHmu` when `req.Exhausted`. It never skips part fulfilment. (`reticulum-forwarding-service`.)
- **Lesson:** mobile→Sideband "working" was a false green — a reference RNS receiver drains each segment before it flags exhausted, so on a clean link it sends part-less exhausted REQs and never exercised the bug. A lenient/conventional peer masks a divergence as effectively as a self-round-trip does (§5.1); the fault is receiver-dependent in a hop whose sender is constant. A `// mirrors upstream` comment proves nothing without the §2.4 / §6.2 check behind it.
### 2026-05-10 — LRPROOF signed_data signalling asymmetry
- **Symptom:** Mobile-app's Kotlin engine fails LRPROOF signature verification against fwdsvc on every attempt. Falls back to opportunistic; link delivery never works.