Compare commits
10 commits
1b955d19a9
...
ae5738ea2f
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ae5738ea2f | ||
|
|
41d2fd61ee | ||
|
|
7a85385bb4 | ||
|
|
dee108e787 | ||
|
|
375521c963 |
||
|
|
4c1d3de421 | ||
|
|
a9a5857122 | ||
|
|
fd2951be82 | ||
|
|
ef3d98963b | ||
|
|
9c64f9b805 |
3 changed files with 245 additions and 7 deletions
220
SPEC.md
220
SPEC.md
|
|
@ -34,6 +34,7 @@ Source citations refer to the standard `pip install rns lxmf` install layout (`R
|
|||
- [4.3 `app_data` format for LXMF delivery destinations](#43-app_data-format-for-lxmf-delivery-destinations)
|
||||
- [4.4 Announce filtering by `name_hash`](#44-announce-filtering-by-name_hash)
|
||||
- [4.5 Announce validation rules (receive side)](#45-announce-validation-rules-receive-side)
|
||||
- [4.6 `rrc.hub` announce app_data (Reticulum Relay Chat)](#46-rrchub-announce-app_data-reticulum-relay-chat)
|
||||
- [5. LXMF wire format](#5-lxmf-wire-format)
|
||||
- [5.1 Opportunistic delivery (single Reticulum DATA packet)](#51-opportunistic-delivery-single-reticulum-data-packet)
|
||||
- [5.2 Direct delivery (over an established Reticulum Link)](#52-direct-delivery-over-an-established-reticulum-link)
|
||||
|
|
@ -150,6 +151,7 @@ Source citations refer to the standard `pip install rns lxmf` install layout (`R
|
|||
- [17.2 Section relevance by category](#172-section-relevance-by-category)
|
||||
- [17.3 Worked example: §2.3 originator HEADER_1→HEADER_2 conversion](#173-worked-example-23-originator-header_1header_2-conversion)
|
||||
- [17.4 Pragmatic implication](#174-pragmatic-implication)
|
||||
- [17.5 Application protocols layered over Reticulum](#175-application-protocols-layered-over-reticulum)
|
||||
- [18. Test vectors](#18-test-vectors)
|
||||
- [19. Source map](#19-source-map)
|
||||
|
||||
|
|
@ -624,6 +626,42 @@ These are not wire-spec MUST rules but most working clients implement them; with
|
|||
| `RNS/Interfaces/Interface.py:60-200` | ingress-limit constants, `should_ingress_limit`, `hold_announce`, `process_held_announces` |
|
||||
| `RNS/Packet.py:83` | `PATH_RESPONSE = 0x0B` context constant |
|
||||
|
||||
### 4.6 `rrc.hub` announce app_data (Reticulum Relay Chat)
|
||||
|
||||
Reticulum Relay Chat hubs announce a destination on the `rrc.hub`
|
||||
aspect — `name_hash = SHA256("rrc.hub")[:10] = ac9fd3a81e4036f86e1d`.
|
||||
Unlike §4.3 (LXMF delivery), the `app_data` is **not** a msgpack
|
||||
`[name, cost]` array, and the two hub implementations disagree on
|
||||
its shape:
|
||||
|
||||
- **`rrcd`** — the Python reference hub. `app_data` is a **CBOR**
|
||||
(RFC 8949) map: `{"proto": "rrc", "v": 1, "hub": <hub_name>}` —
|
||||
CBOR, *not* msgpack, because RRC's wire codec (`rrcd/codec.py`,
|
||||
Python `cbor2`) is CBOR throughout. The human hub name is the
|
||||
`"hub"` key's value (a text string); `"proto"` is always `"rrc"`
|
||||
and `"v"` is the app_data schema version (`1`). Source: `rrcd`
|
||||
`service.py` — `app_data = encode({"proto": "rrc", "v": 1, "hub":
|
||||
self.config.hub_name})`, where `encode` is the CBOR encoder.
|
||||
- **`reticulum-relay-chat`** — the Go hub. `app_data` is the hub name
|
||||
as **plain UTF-8 bytes**, unwrapped. Source:
|
||||
`internal/service/service.go` — `BuildAnnounce(id, "rrc.hub",
|
||||
[]byte(s.cfg.Hub.Name), ...)`.
|
||||
|
||||
> ⚠️ **CBOR-vs-msgpack gotcha.** A CBOR 3-entry map begins with byte
|
||||
> `0xa3`. In msgpack `0xa3` is `fixstr` of length 3 — so a client that
|
||||
> blindly msgpack-decodes the `rrcd` app_data reads the next three
|
||||
> bytes (`0x65 0x70 0x72`, the CBOR text-string header of `"proto"`
|
||||
> plus its first two characters) as the 3-character string `"epr"`.
|
||||
> Decode `rrc.hub` app_data with a CBOR decoder, keyed on the
|
||||
> `rrc.hub` name_hash — do not feed it to the LXMF (msgpack) app_data
|
||||
> parser.
|
||||
|
||||
A client listing RRC hubs should resolve the name as: the `"hub"`
|
||||
value when `app_data` CBOR-decodes to a map; else a bare UTF-8 string
|
||||
(the Go hub's shape); else a generic "RRC hub" label. The same name
|
||||
is also delivered authoritatively in the RRC `WELCOME` body key
|
||||
`B_WELCOME_HUB` once a session is established.
|
||||
|
||||
---
|
||||
|
||||
## 5. LXMF wire format
|
||||
|
|
@ -674,6 +712,8 @@ signature = Ed25519_sign(signed_data, sender_identity.Ed25519_priv)
|
|||
|
||||
For opportunistic delivery, `destination_hash` is the recipient's destination hash (from the outer packet header, not from the LXMF body).
|
||||
|
||||
The `msgpack_payload` MUST be the exact bytes received on the wire — not a decode-then-re-encode — when computing `message_hash`. The same value is reused as the LXMF `message_id` (workblock input for stamps in §5.7, target identifier for reactions in §5.9.8 and replies in §5.9.9). A receiver that re-encodes will compute a divergent identifier for any message whose `fields` map does not round-trip byte-identically — most commonly reply messages — which breaks both the §5.6 path-1 raw-signature check and any relay rewrite-cache keyed on the value. For a stamped message (§5.7.1) the raw payload on the wire is a 5-element array; the hash is over the first 4 elements, which requires a byte-canonical re-pack per §5.6.1.
|
||||
|
||||
### 5.6 Signature verification — msgpack variant tolerance
|
||||
|
||||
Different msgpack encoders produce subtly different byte sequences for the same logical value (e.g. integer encoding choice, string vs bin selection). The signer signed over THEIR encoder's output. A receiver should try verifying against:
|
||||
|
|
@ -962,7 +1002,7 @@ Sender and receiver agree on these keys; each value's structure is field-specifi
|
|||
| `0x02` | `FIELD_TELEMETRY` | A single telemetry snapshot (Sideband telemetry — see Sideband for the inner format; LXMF is opaque to the contents). |
|
||||
| `0x03` | `FIELD_TELEMETRY_STREAM` | A list of telemetry snapshots (history flush). |
|
||||
| `0x04` | `FIELD_ICON_APPEARANCE` | Sender-supplied avatar / appearance hint. |
|
||||
| `0x05` | `FIELD_FILE_ATTACHMENTS` | A list of attached files (multiple attachments per message). |
|
||||
| `0x05` | `FIELD_FILE_ATTACHMENTS` | A list of attached files (multiple attachments per message). See §5.9.7 for the wire shape. |
|
||||
| `0x06` | `FIELD_IMAGE` | Single embedded image — `[extension_string, image_bytes]`. See §5.9.2 for the wire shape. |
|
||||
| `0x07` | `FIELD_AUDIO` | Single embedded audio clip — `[mode_byte, audio_bytes]`. Mode byte chooses the codec; see §5.9.3. |
|
||||
| `0x08` | `FIELD_THREAD` | Conversation thread ID (links related messages). |
|
||||
|
|
@ -979,7 +1019,7 @@ Sender and receiver agree on these keys; each value's structure is field-specifi
|
|||
| `0xFE` | `FIELD_NON_SPECIFIC` | Development / unstructured payload — not for production. |
|
||||
| `0xFF` | `FIELD_DEBUG` | Debug payload — not for production. |
|
||||
|
||||
> ⚠️ **UNVERIFIED:** the byte-level shape of `FIELD_EMBEDDED_LXMS`, `FIELD_TELEMETRY*`, `FIELD_FILE_ATTACHMENTS`, `FIELD_COMMANDS`, `FIELD_RESULTS`, `FIELD_GROUP`, `FIELD_EVENT`, and `FIELD_RNR_REFS` is not described here because no test vectors have been captured against upstream Sideband emissions for these. The constants are verified (see `tools/verify_lxmf_fields.py`) but the value structures are application-defined and not pinned by LXMF itself. Future PRs should add per-field byte layouts as test vectors arrive.
|
||||
> ⚠️ **UNVERIFIED:** the byte-level shape of `FIELD_EMBEDDED_LXMS`, `FIELD_TELEMETRY*`, `FIELD_COMMANDS`, `FIELD_RESULTS`, `FIELD_GROUP`, `FIELD_EVENT`, and `FIELD_RNR_REFS` is not described here because no test vectors have been captured against upstream Sideband emissions for these. The constants are verified (see `tools/verify_lxmf_fields.py`) but the value structures are application-defined and not pinned by LXMF itself. Future PRs should add per-field byte layouts as test vectors arrive. (`FIELD_FILE_ATTACHMENTS` was on this list until 2026-05-18 — its shape is now documented in §5.9.7 from upstream Sideband source.)
|
||||
|
||||
#### 5.9.2 `FIELD_IMAGE` (`0x06`) value shape
|
||||
|
||||
|
|
@ -1061,6 +1101,75 @@ For announce-level capability negotiation:
|
|||
|---|---|---|
|
||||
| `0x00` | `SF_COMPRESSION` | Sender supports compressed message bodies (see §10.12) |
|
||||
|
||||
#### 5.9.7 `FIELD_FILE_ATTACHMENTS` (`0x05`) value shape
|
||||
|
||||
```
|
||||
fields[0x05] = [ [filename(str-or-bytes), file_bytes(bytes)], ... ]
|
||||
```
|
||||
|
||||
A list of attachments — one LXMF message may carry more than one
|
||||
file. Each attachment is itself a 2-element list: element `[0]` is the
|
||||
file name, element `[1]` is the raw file content. As with `FIELD_IMAGE`
|
||||
(§5.9.2) the file name may arrive as msgpack `str` or `bin` depending
|
||||
on the encoder — receivers must tolerate both (see §9.3).
|
||||
|
||||
The file name is **sender-controlled and untrusted**. Upstream Sideband
|
||||
strips `../` from it on receive (`sbapp/ui/messages.py`:
|
||||
`filename = str(attachment[0]).replace("../", "")`); receivers MUST
|
||||
sanitise it — reject or strip path separators, `..` segments and
|
||||
control characters — before display or save, and never let it
|
||||
influence a write path. The extension is likewise untrusted: do not
|
||||
auto-open or auto-execute an attachment based on its claimed type.
|
||||
|
||||
Files larger than a single Reticulum DATA packet are delivered as a
|
||||
Resource over a Link, identically to large `FIELD_IMAGE` payloads
|
||||
(§10).
|
||||
|
||||
Source: `markqvist/Sideband` `sbapp/sideband/core.py`
|
||||
(`fields[LXMF.FIELD_FILE_ATTACHMENTS] = [attachment]`, where each
|
||||
`attachment = [filename, filedata]`) and `sbapp/ui/messages.py`
|
||||
(receive side indexes `attachment[0]` filename / `attachment[1]`
|
||||
bytes). Confirmed from upstream source 2026-05-18; a captured wire
|
||||
test vector would further pin the msgpack `str`-vs-`bin` choice.
|
||||
|
||||
#### 5.9.8 `fields[16]` (`0x10`): tap-back reactions (app extension)
|
||||
|
||||
> ⚠️ **UNVERIFIED:** this key is outside the upstream LXMF allocation range (§5.9.1) — there is no `FIELD_REACTION` constant in `LXMF/LXMF.py`. The shape below is a convergent app-extension across three shipping FOSS clients (`reticulum-mobile-app`, `Quad4-Software/MeshChatX`, `torlando-tech/columba` `release/v0.10.x` and later). Documented here because the convergence is stable enough for new implementations to interop. Replace with the upstream allocation if/when `markqvist/LXMF` blesses one.
|
||||
|
||||
```
|
||||
fields[16] = {
|
||||
"reaction_to": <target LXMF message_id, 64-char hex string>,
|
||||
"emoji": <unicode>,
|
||||
"sender": <reactor's identity hash, 32-char hex string>,
|
||||
}
|
||||
```
|
||||
|
||||
- The carrying LXMF has empty `content` and empty `title`; `fields[16]` IS the entire payload.
|
||||
- `reaction_to` is the canonical LXMF `message_id` (= `message_hash` of §5.5; the same value used as the workblock input for stamps in §5.7). It MUST be derived from the recipient-side raw wire payload bytes per the §5.5 normative rule — a decode-re-encode diverges for any non-trivial `fields` map (including reply-carrying messages, §5.9.9) and the reaction misses the relay rewrite-cache for that target.
|
||||
- `sender` is the reactor's **identity hash** (`SHA256(identity_public_key)[:16]`), NOT the lxmf.delivery `source_hash`. Both are 16-byte SHA-256 truncations and trivially easy to conflate, but receivers key reaction aggregation by `(emoji, sender)`, so emitting the destination hash mis-buckets against every (identity-hash-emitting) peer. All three convergent clients emit the identity hash (`python/reticulum_wrapper.py::send_reaction` in Columba `release/v0.10.x` and `v2.0.0-beta+`; `meshchat.py::send_reaction` in MeshChatX; `ReticulumEngine.kt::sendReaction` in `reticulum-mobile-app`).
|
||||
- Receivers MUST aggregate by `(emoji, sender)` with dedup and SHOULD NOT render the reaction-carrying LXMF as a separate bubble.
|
||||
- String-vs-bytes tolerance: msgpack values may arrive as `str` or `bin` depending on encoder. Receivers MUST accept both — same precedent as §5.6 for the LXMF signature.
|
||||
- **Inner-map decode tolerance.** The `fields[16]` value is a msgpack map. Strongly-typed decoders may surface it as either a string-keyed map or an `Any`-keyed map (`Map<String, Any>` / `Map<Any?, Any?>`, `map[string]any` / `map[any]any`, etc.), depending on the runtime msgpack library's choice given the key types seen on the wire. Receivers MUST tolerate both via a runtime cast that does not depend on the outer map's static key type. A silent type-assertion failure on the "wrong" variant produces a no-log no-error drop indistinguishable on the receive side from "the message never arrived."
|
||||
- **Relay routing.** If the target message arrived over a Link whose validated §6.7.6 LINKIDENTIFY peer differs from the LXMF body's `source_hash`, the reaction-carrying LXMF MUST egress to the link peer's destination hash, not `source_hash`; otherwise the reaction bypasses the relay's fanout and is delivered direct to the original sender instead of the relay group.
|
||||
|
||||
#### 5.9.9 `fields[0x30]` + optional `fields[0x31]`: reply-to threading (app extension)
|
||||
|
||||
> ⚠️ **UNVERIFIED:** as with §5.9.8, these keys are outside the upstream LXMF allocation range. The shape below is the convergent app-extension across the same three FOSS clients. Columba's earlier `release/v0.10.x` builds used a legacy single-key shape (`fields[16] = {"reply_to": <hex>}`); Columba `v2.0.0-beta` (released 2026-05-23) and later emit the dual-key shape per `torlando-tech/columba#926`. The legacy-tolerance branch in the inbound rule below has a finite lifetime — drop it once `< v2.0.0-beta` Columba is no longer in the field.
|
||||
|
||||
```
|
||||
fields[0x30] = <raw 32-byte LXMF message_id (NOT hex-encoded)>
|
||||
fields[0x31] = <UTF-8 quoted content> # optional
|
||||
```
|
||||
|
||||
- `fields[0x30]` IS the canonical LXMF `message_id` from §5.7 — the raw 32 bytes on the wire, not a hex-encoded string. The `message_id` MUST be computed over raw wire payload bytes per §5.5; a decode-re-encode diverges for reply messages (their `fields` map is non-trivial) and breaks any relay rewrite-cache keyed on the value.
|
||||
- `fields[0x31]` (optional) carries the quoted text for offline-receiver fallback so the recipient can render a quote preview even when the original message hasn't arrived locally. Useful on intermittent links; SHOULD be omitted when the sender doesn't want to pay the airtime.
|
||||
- **Legacy-tolerance (inbound only).** Receivers SHOULD also accept Columba's earlier `fields[16] = {"reply_to": <64-char hex>}` shape for interop with already-deployed `release/v0.10.x` builds. Senders MUST NOT emit it — emit only `[0x30]`/`[0x31]`. The same inner-map dual-shape tolerance described in §5.9.8 applies to this legacy parse branch (the value is a msgpack map).
|
||||
- **Bandwidth rationale.** The dual-key shape is roughly 2× smaller than the legacy single-key shape for the same information — 32 raw bytes + 1-byte key vs. a msgpack-encoded dict carrying a hex-string hash. This is the [Zen of Reticulum](https://reticulum.network/manual/zen.html) alignment cited by the MeshChatX maintainer in `Quad4-Software/MeshChatX#14` and accepted by the Columba maintainer in `torlando-tech/columba#926`.
|
||||
- The raw-bytes branches (`fields[0x30]`, `fields[0x31]`) are not maps and do not have the §5.9.8 inner-map risk, but receivers MUST still tolerate both `bytes` and `str` carriers per §5.6 in case an encoder ships the hash hex-encoded by mistake.
|
||||
- **Relay routing.** Same as §5.9.8 — if the target message arrived over a Link whose §6.7.6 LINKIDENTIFY peer differs from `source_hash`, the reply-carrying LXMF MUST egress to the link peer's destination hash.
|
||||
|
||||
Reference implementations: `Quad4-Software/MeshChatX` (`meshchatx/src/backend/lxmf_utils.py`; `meshchat.py::send_reaction`); `torlando-tech/columba` v2.0.0-beta and later (`python/reticulum_wrapper.py`); `thatSFguy/reticulum-mobile-app` (`shared/.../engine/ReticulumEngine.kt::extractField16`, `sendReaction`, `tryDeliverOverLink::replyFields`).
|
||||
|
||||
### 5.10 Source
|
||||
|
||||
`LXMF/LXMessage.py` for pack/unpack; `LXMF/LXMF.py` for the app_data extraction helpers and the field/audio/renderer constants enumerated in §5.9; `LXMF/LXStamper.py` for stamps; `LXMF/LXMRouter.py` for receive-side stamp/ticket dispatch and propagation handlers; `LXMF/LXMPeer.py` for the propagation peer-to-peer state machine.
|
||||
|
|
@ -1229,7 +1338,7 @@ implicit body = signature(64)
|
|||
Where:
|
||||
|
||||
- `packet_hash = Identity.full_hash(original_packet.get_hashable_part())` — the full SHA-256 (32 bytes, **not** truncated to 16) of the prove-target packet's hashable part. `get_hashable_part` is the same recipe used for `link_id` derivation in §6.3, so the proof binds to the version of the packet that survived any HEADER_1↔HEADER_2 conversion in transit (the high nibble of flags, hops byte, and any HEADER_2 transport_id are stripped before hashing).
|
||||
- `signature` is the destination's (or link's) Ed25519 signature **over `packet_hash`**, NOT over the proof body itself. The signing key is the destination's long-term Ed25519 private key for an opportunistic DATA proof, or the link-derived signing key for a Link DATA proof.
|
||||
- `signature` is the destination's (or link's) Ed25519 signature **over `packet_hash`**, NOT over the proof body itself. The signing key is the destination's long-term Ed25519 private key for an opportunistic DATA proof, or **the link's Ed25519 signing key** (`Link.sig_prv`, `RNS/Link.py:279` / `:286` in RNS 1.2.9) for a Link DATA proof — the owner identity's long-term key on the responder side, the link's ephemeral Ed25519 keypair on the initiator side. (The HKDF-derived `signing_key` from §6.4.1 is a separate **symmetric HMAC key** for the link Token form §3.1 — it cannot produce an Ed25519 signature and is not used here.)
|
||||
|
||||
The two forms are distinguished **purely by length** at the receiver. `PacketReceipt.validate_proof` (`RNS/Packet.py:497-548`) dispatches on `len(proof) == 96` (explicit) vs `len(proof) == 64` (implicit); lengths matching neither are rejected outright. There is no flag bit or context byte that signals which form is being used — wire length is the only signal.
|
||||
|
||||
|
|
@ -1433,7 +1542,7 @@ def send_keepalive(self):
|
|||
keepalive_packet.send()
|
||||
```
|
||||
|
||||
Body is a single byte `0xFF` — the "ping" sentinel. The packet is Token-encrypted with the link's session key per §3.1 link-derived form, so the wire body is `iv(16) || ciphertext(...) || hmac(32)`; the decrypted plaintext is just `b'\xff'`.
|
||||
Body is a single byte `0xFF` — the "ping" sentinel. **KEEPALIVE is NOT Token-encrypted** — `RNS/Packet.py` `pack()` puts `KEEPALIVE` in its not-encrypted branch alongside `RESOURCE`, `RESOURCE_PRF`, link `PROOF`, and `CACHE_REQUEST` (`self.ciphertext = self.data`, `Packet.py:206-209` in RNS 1.2.9). The wire body is the single sentinel byte in the clear — no IV, no ciphertext expansion, no HMAC.
|
||||
|
||||
The **responder** receives this in `Link.receive` at `RNS/Link.py:1149-1153` and answers with the "pong" sentinel (in 1.2.4 the body is `bytes([0xFE])`):
|
||||
|
||||
|
|
@ -1449,7 +1558,7 @@ So:
|
|||
- **Pong** = responder → initiator, body `0xFE`.
|
||||
- Only the initiator originates KEEPALIVE traffic. The responder never spontaneously pings.
|
||||
|
||||
Both sentinel bytes are arbitrary; what actually matters for keep-alive purposes is that *any* inbound traffic on the link refreshes `last_inbound` (the watchdog's anchor for staleness decisions). KEEPALIVE packets, like all link DATA, also generate the mandatory PROOF receipt per §6.5, which is itself inbound traffic on the return path. So a successful ping/pong exchange resets the staleness clock on **both** sides via three round-trip artifacts: ping → pong → pong-proof.
|
||||
Both sentinel bytes are arbitrary; what actually matters for keep-alive purposes is that *any* inbound traffic on the link refreshes `last_inbound` (the watchdog's anchor for staleness decisions). KEEPALIVE packets do **not** generate a PROOF receipt — the link-DATA proof path in `RNS/Link.py` `receive()` is gated on `packet.context == RNS.Packet.NONE` (`Link.py:988` in RNS 1.2.9), and KEEPALIVE takes its own branch at line 1149-1153. So a successful ping/pong exchange resets the staleness clock on **both** sides via the two-packet exchange itself: ping → pong (no pong-proof).
|
||||
|
||||
A clean-room responder MUST emit the pong on inbound `0xFF`; without it the initiator's watchdog will declare the link stale on the next cycle.
|
||||
|
||||
|
|
@ -1530,6 +1639,59 @@ For a clean-room implementation that wants links to survive idle periods longer
|
|||
5. On every inbound `LINKCLOSE`, decrypt, verify body equals `link_id`, transition to `CLOSED`.
|
||||
6. On `CLOSED`, zero the session keys and cancel any in-progress Resources.
|
||||
|
||||
#### 6.7.6 LINKIDENTIFY (`context = 0xFB`)
|
||||
|
||||
A separate Link control DATA packet — not part of the keepalive /
|
||||
teardown cycle, but documented here alongside the other context-dispatched
|
||||
Link control packets. Used by the initiator to prove which long-term
|
||||
identity is making the request without re-running the Link handshake;
|
||||
§11.6 covers the calling context on the NomadNet REQUEST path.
|
||||
|
||||
`Link.identify(identity)` (`RNS/Link.py:459-475` in RNS 1.2.9):
|
||||
|
||||
```python
|
||||
def identify(self, identity):
|
||||
if self.initiator and self.status == Link.ACTIVE:
|
||||
signed_data = self.link_id + identity.get_public_key()
|
||||
signature = identity.sign(signed_data)
|
||||
proof_data = identity.get_public_key() + signature
|
||||
proof = RNS.Packet(self, proof_data, RNS.Packet.DATA,
|
||||
context = RNS.Packet.LINKIDENTIFY)
|
||||
proof.send()
|
||||
```
|
||||
|
||||
Wire body (128 bytes):
|
||||
|
||||
```
|
||||
public_key(64) || signature(64)
|
||||
```
|
||||
|
||||
- `public_key` is the initiator's **full 64-byte Identity public key**
|
||||
— the same `get_public_key()` that LINKREQUEST and announces use,
|
||||
the concatenation of the Ed25519 verification key and the X25519
|
||||
encryption key per §3.
|
||||
- `signature` is `identity.sign(link_id(16) || public_key(64))` — the
|
||||
signature is over the link_id concatenated with the public_key,
|
||||
NOT over `link_id` alone. The responder verifies with `public_key`
|
||||
over that same concatenation.
|
||||
|
||||
The packet is a `DATA` packet on the active Link, so it IS
|
||||
link-encrypted per §3.1 link-derived form like ordinary link DATA —
|
||||
`context = 0xFB` is NOT in `RNS/Packet.py` `pack()`'s not-encrypted
|
||||
set, unlike KEEPALIVE (§6.7.1) or link `PROOF` (§6.5).
|
||||
|
||||
The responder's `receive()` (`RNS/Link.py:1010-1029`) decrypts the
|
||||
body, splits off the 64-byte public_key prefix, reconstructs
|
||||
`signed_data = link_id || public_key`, verifies the signature, and on
|
||||
success sets `self.remote_identity` so subsequent REQUEST handlers can
|
||||
check the caller against per-page allowlists (§11.6).
|
||||
|
||||
A clean-room implementation MUST build the payload as
|
||||
`public_key || signature` (NOT just the signature) and sign the
|
||||
concatenation `link_id || public_key` (NOT `link_id` alone). Either
|
||||
mistake makes every `ALLOW_LIST`-protected page return
|
||||
`DEFAULT_NOTALLOWED`.
|
||||
|
||||
### 6.8 Channel mode (`CHANNEL = 0x0E`)
|
||||
|
||||
A Channel is a **continuous, bi-directional, message-typed stream** on top of an established Link. Distinct from §11 REQUEST/RESPONSE (single-shot, client-server) and §10 Resources (large unidirectional transfers): Channel messages are short, can flow in either direction at any time, and carry an application-defined type byte the receiver dispatches on. NomadNet uses it for its "channel" API (live chat over a Link), and any application can register custom message types via `RNS.Channel.Channel.register_message_type`.
|
||||
|
|
@ -2439,6 +2601,34 @@ The segment_index is `part_index // HASHMAP_MAX_LEN`. The receiver applies this
|
|||
|
||||
If the part_index doesn't land on a `HASHMAP_MAX_LEN` boundary, the sender treats it as a sequencing error and cancels the resource (`Resource.py:1043-1046`).
|
||||
|
||||
> **An exhausted RESOURCE_REQ MAY still carry parts — a conformant
|
||||
> sender fulfils them *and* sends the RESOURCE_HMU.** When the
|
||||
> `hashmap_exhausted_flag` is `0xFF`, the REQ body may still end with a
|
||||
> non-empty `requested_map_hashes` trailer (§10.5). The sender MUST emit
|
||||
> the requested part packets **and** the hashmap continuation; the two
|
||||
> are independent. Serving the HMU is not a substitute for fulfilling
|
||||
> the bundled part requests.
|
||||
>
|
||||
> In the RNS reference (`Resource.py:982-1071`, `request()` — verified
|
||||
> against RNS 1.2.9, the current release), the part-fulfilment loop runs
|
||||
> for every REQ regardless of the flag, and the `if wants_more_hashmap:`
|
||||
> HMU branch runs afterward, in addition. The reference receiver
|
||||
> (`request_next`, `Resource.py:931-981`) routinely produces this
|
||||
> packet shape: as its window scan reaches the end of the known
|
||||
> hashmap, it has already accumulated the still-outstanding part-hashes
|
||||
> from the known region into `requested_hashes`, then sets the exhausted
|
||||
> flag and stops — emitting
|
||||
> `0xFF || last_map_hash(4) || resource_hash(32) || requested_map_hashes`.
|
||||
>
|
||||
> A receiver MAY keep part requests and hashmap pulls in separate REQ
|
||||
> packets — emitting a **part-less** exhausted REQ
|
||||
> (body `0xFF || last_map_hash(4) || resource_hash(32)`, no trailing
|
||||
> map_hashes) purely to pull the continuation. This interoperates with
|
||||
> every conformant sender. But it is a receiver-side simplification
|
||||
> only: a sender MUST NOT assume peers do this, and MUST NOT skip part
|
||||
> fulfilment for an exhausted REQ. A sender that does drops every
|
||||
> bundled part silently — see `playbook.md` §7 (2026-05-19).
|
||||
|
||||
### 10.8 RESOURCE_PRF — final proof
|
||||
|
||||
When the receiver has assembled the full resource (`received_count == total_parts`), it runs `assemble()` (`Resource.py:672-726`):
|
||||
|
|
@ -2811,7 +3001,7 @@ Pages are registered with one of three allow modes (`Destination.py:35-40`):
|
|||
- `ALLOW_LIST` — caller's identity hash must appear in the page's `.allowed` file. Server checks `remote_identity.hash` against the list at request time (`Node.py:152-154`).
|
||||
- `ALLOW_NONE` — registered handlers that exist but reject all requests (rare; debug only).
|
||||
|
||||
For `ALLOW_LIST` the client MUST call `link.identify(identity)` immediately after the link transitions to ACTIVE and BEFORE issuing the REQUEST. This sends a `LINKIDENTIFY (context = 0xFB)` packet whose payload carries a signature over `link_id` proving the long-term identity hash. Without it, `remote_identity` is `None` server-side and every `ALLOW_LIST` page returns `DEFAULT_NOTALLOWED`. See `Browser.py:1245-1250` for the upstream call site:
|
||||
For `ALLOW_LIST` the client MUST call `link.identify(identity)` immediately after the link transitions to ACTIVE and BEFORE issuing the REQUEST. This sends a `LINKIDENTIFY (context = 0xFB)` packet whose 128-byte payload is `public_key(64) || signature(64)`, with the signature computed over `link_id || public_key` — proving the long-term identity hash to the responder. The wire format is specified in §6.7.6. Without it, `remote_identity` is `None` server-side and every `ALLOW_LIST` page returns `DEFAULT_NOTALLOWED`. See `Browser.py:1245-1250` for the upstream call site:
|
||||
|
||||
```python
|
||||
def link_established(self, link):
|
||||
|
|
@ -3546,6 +3736,24 @@ If you're a category-3 developer building from scratch, you need everything. Ver
|
|||
|
||||
If you're not sure which category you're in: `grep -r "import RNS" your_codebase` is a quick check. Any hit means cat 1 (or cat 2 if it's behind an FFI wall). No hits means cat 3.
|
||||
|
||||
### 17.5 Application protocols layered over Reticulum
|
||||
|
||||
Reticulum is a transport substrate; user-facing features are *application protocols* built on top. This spec documents **LXMF** (§5) in depth because its wire format is published nowhere else. Other application protocols carry their own authoritative specs and are **out of scope here** — but a clean-room author building a client for one still needs the RNS-layer sections below.
|
||||
|
||||
**Reticulum Relay Chat (RRC)** — a live, IRC-style chat protocol. Authoritative spec: <https://rrc.kc1awv.net/> (its CBOR envelope, message types, and state machines live there, not here). RRC is a category-3-style consumer of RNS: it defines its own wire format but relies entirely on Reticulum for transport. An RRC client built clean-room needs:
|
||||
|
||||
| RRC depends on | Sections in this spec |
|
||||
|---|---|
|
||||
| Hub destination (`rrc.hub` aspect) + client identity | §1.1, §1.2, §9.8 |
|
||||
| Identity hash as the canonical sender id (RRC envelope key 4, "opaque, do not re-encode") | §1.1; §9.1 — the identity-hash-vs-destination-hash pitfall RRC's rule guards against |
|
||||
| All traffic over a single Reticulum Link | §6.1–§6.4, §6.7 |
|
||||
| Single-packet CBOR frames sized to the link MTU | §2.1–§2.2, §2.4, §6.6 (MTU is *negotiated* — see note below) |
|
||||
| Ordered/reliable delivery, *if* RRC layers over Channel | §6.5, §6.8 |
|
||||
|
||||
RRC does **not** use opportunistic packets, Resource transfer (§10), REQUEST/RESPONSE (§11), or LXMF (§5) — an RRC-only client can skip those entirely.
|
||||
|
||||
> Two things RRC's spec leaves implicit that this spec makes explicit: link MTU is **negotiated** (§6.6), not a fixed 500 bytes, so frames sized to the default can overflow a smaller link; and a bare Link does not itself guarantee ordered/reliable delivery — that comes from Channel (§6.8) or packet receipts (§6.5).
|
||||
|
||||
---
|
||||
|
||||
## 18. Test vectors
|
||||
|
|
|
|||
25
agent.md
25
agent.md
|
|
@ -55,12 +55,35 @@ PRs must include the verifier scripts. Don't commit a "verified" claim without t
|
|||
|
||||
Agents working on this repo should have access to:
|
||||
|
||||
- A working Python 3 install with `rns` and `lxmf` packages: `pip install rns lxmf`
|
||||
- A working Python 3 install with `rns` and `lxmf` packages (install per "Staying current" below — not a bare `pip install rns lxmf`).
|
||||
- The `RNS/` and `LXMF/` source trees (typically at `~/AppData/Roaming/Python/Python3xx/site-packages/RNS/` on Windows or `~/.local/lib/python3.x/site-packages/RNS/` on Linux/macOS).
|
||||
- Optional but very useful: a packet-trace tool. `tcpdump -i lo -A -X port 4242` works for TCPServerInterface; for BLE you need ADB + an RNode-aware capture tool.
|
||||
|
||||
Hardware (RNode, RatDeck, etc.) is NOT required for most verification — most byte-level claims can be checked entirely in Python RNS without any radio.
|
||||
|
||||
### Staying current with upstream — and verifying what you install
|
||||
|
||||
This spec is only as good as the upstream version it was checked against. **Before any verification pass, confirm you are looking at the latest RNS:**
|
||||
|
||||
1. **Check the latest upstream version.** Upstream is mid-migration *off* GitHub: the RNS 1.2.4 release notes announced 1.2.4 would "probably be the last release also published to GitHub," and GitHub releases do stop around **1.2.5** (1.2.6/1.2.7 are PyPI-only). Reticulum is moving to self-hosting — `rngit` (git served over Reticulum) and `rnpkg` (package distribution over Reticulum). Per Mark Qvist, "updates to pip will continue at least until `rnpkg` is complete." Check, in order:
|
||||
- PyPI: `pip index versions rns` / `pip index versions lxmf` — most current while it lasts, but PyPI carries **no `.rsg`**, so PyPI alone is unverifiable.
|
||||
- GitHub releases: <https://github.com/markqvist/Reticulum/releases> — signed (`.rsg`) but frozen near 1.2.5.
|
||||
- `rngit` / `rnpkg` over Reticulum — the eventual canonical source of signed artifacts.
|
||||
|
||||
PyPI runs *ahead* of the newest signed release (1.2.7 on PyPI vs 1.2.5 signed, 2026-05-17). Re-check every session — do not assume a frozen version.
|
||||
|
||||
2. **Verify the package signature before installing.** Since RNS 1.2.x, signed releases ship a detached `.rsg` signature beside each artifact (on the GitHub release page while that lasts; via `rngit`/`rnpkg` afterward). Get the wheel *and* its `.rsg`, then:
|
||||
|
||||
```
|
||||
rnid -i bc7291552be7a58f361522990465165c -V rns-<version>-py3-none-any.whl
|
||||
```
|
||||
|
||||
`bc7291552be7a58f361522990465165c` is Mark Qvist's release-signing Reticulum identity — confirm that hash itself against a trusted channel (a release announcement) before trusting it. `rnid` works fully offline. Install the **exact wheel you verified** (`pip install --upgrade ./rns-<version>-py3-none-any.whl`), not a fresh `pip install rns` that re-downloads from PyPI.
|
||||
|
||||
3. **Prefer the latest *signed* version.** If PyPI is ahead of the newest release carrying a `.rsg`, install the signed one and note the gap — do not silently install an unverifiable newer version.
|
||||
|
||||
4. **After a version bump**, re-run every `tools/verify_*.py`, re-check the source-cited line numbers in any section you touch, and only advance SPEC.md's `**Last verified against:**` line once the whole document has been re-checked against the new version.
|
||||
|
||||
---
|
||||
|
||||
## 4. Marking convention
|
||||
|
|
|
|||
|
|
@ -191,6 +191,13 @@ Spec-only repos with a "the source is the source of truth" attitude die slowly b
|
|||
|
||||
Each entry: date, one-line symptom, spec section that governs it, one-line fix, one-sentence lesson. Append-only. New entries go at the top.
|
||||
|
||||
### 2026-05-19 — fwdsvc dropped parts bundled into an exhausted RESOURCE_REQ
|
||||
|
||||
- **Symptom:** Images relayed mobile→mobile through the Fwd service never arrive (whole LXMF message lost); mobile→Sideband through the same service works. Recipient logs hundreds of `RESOURCE chunk did not match any known hashmap slot`. Only triggers for resources large enough to need RESOURCE_HMU (>`HASHMAP_MAX_LEN` ≈ 74 parts).
|
||||
- **Spec section:** §10.7. An `exhausted == 0xFF` RESOURCE_REQ MAY still carry a `requested_map_hashes` trailer, and a conformant sender serves those parts **and** the RESOURCE_HMU. The fwdsvc Go sender did `if req.Exhausted { serveHmu(req); continue }`, skipping `fulfillRequest` entirely — its own comment claimed this "mirrors upstream `Resource.request()`", but upstream (`Resource.py:982-1071`, checked against RNS 1.2.9, the current release) runs part fulfilment unconditionally and *then* sends the HMU. The mobile receiver flags `exhausted` on the first REQ of each hashmap window and bundles ~74 part-hashes with it — which a reference RNS sender honours — so fwdsvc served HMUs and dropped every bundled part across all 19 windows.
|
||||
- **Fix:** `resource_sender.go` Run loop now runs `fulfillRequest` for every REQ, then `serveHmu` when `req.Exhausted`. It never skips part fulfilment. (`reticulum-forwarding-service`.)
|
||||
- **Lesson:** mobile→Sideband "working" was a false green — a reference RNS receiver drains each segment before it flags exhausted, so on a clean link it sends part-less exhausted REQs and never exercised the bug. A lenient/conventional peer masks a divergence as effectively as a self-round-trip does (§5.1); the fault is receiver-dependent in a hop whose sender is constant. A `// mirrors upstream` comment proves nothing without the §2.4 / §6.2 check behind it.
|
||||
|
||||
### 2026-05-10 — LRPROOF signed_data signalling asymmetry
|
||||
|
||||
- **Symptom:** Mobile-app's Kotlin engine fails LRPROOF signature verification against fwdsvc on every attempt. Falls back to opportunistic; link delivery never works.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue