jlpoole/reticiulum-specification
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift

Browse source 
Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last
release that is also published to GitHub" — pip continues until rnpkg
is complete and RNS is self-hosting. All 13 verifiers pass against
1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed
between 1.2.0 and 1.2.4, so the changes here are purely currency:

- Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the
  verifier stays reproducible if upstream stops mirroring to PyPI
  before the migration is ready.
- Add an "Upstream distribution shift" watch-list to todo.md (local
  Reticulum node, repo destination hash, rnpkg install/upgrade
  commands, rsg signature verification, mirroring source citations).
- Bump SPEC.md frontmatter and re-anchor ~50 line citations across
  Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py,
  Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13
  to +31 lines; Transport.py was variable). Fix one numeric
  (MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU
  clamp citation pointed at the wrong location — corrected to point
  at the transit-relay clamp at Transport.py:1539-1556).
- Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap
  adoption, with citations to Resource.py:686-691 and Buffer.py:95-97
  plus a "do not use one-shot bz2.decompress()" warning.
- Re-anchor 11 flows/ files (version pins + ~30 line citations).
- Bump version labels in tools/README.md, test-vectors/README.md, and
  4 verifier docstrings + 2 hardcoded print strings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rob 2026-05-08 07:42:25 -04:00
commit cfd0d8249b
23 changed files with 255 additions and 190 deletions
Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -19,7 +19,7 @@ Each finding is grounded in upstream source citations (file + line) so it can be
- [`SPEC.md`](SPEC.md) — the single combined spec document, organized by protocol layer
- [`flows/`](flows/) — chronological end-to-end narratives (e.g. "send a message"), cross-referencing SPEC.md sections
- [`tools/`](tools/) — self-contained Python verifier scripts that test SPEC.md claims against upstream RNS / LXMF
- [`tools/`](tools/) — self-contained Python verifier scripts that test SPEC.md claims against upstream RNS / LXMF. Pinned via [`tools/requirements.txt`](tools/requirements.txt) to the upstream versions the scripts were last re-verified against
- [`test-vectors/`](test-vectors/) — known-good byte sequences each implementation should be able to round-trip (intent: grow into a compliance suite)
As content grows, `SPEC.md` will be split into per-layer files (packet header, identity, announce, token-crypto, LXMF, link, resource, transport).

224
SPEC.md
View file

@ -2,7 +2,7 @@
A byte-level reference for implementing Reticulum-compatible clients. This document focuses on what implementations need to interop with the canonical Python implementation ([`markqvist/Reticulum`](https://github.com/markqvist/Reticulum) and [`markqvist/LXMF`](https://github.com/markqvist/LXMF)) plus the existing client ecosystem (Sideband, Nomadnet, MeshChat, the various firmware projects).
**Last verified against:** `RNS 1.2.0` / `LXMF 0.9.6` / `RNode_Firmware` (master at the spec's last revision date). Each section's source citations were re-checked against these versions; runtime verifiers in [`tools/`](tools/) lock the wire-format claims in against actually-running upstream code. When you upgrade past these, re-run every `tools/verify_*.py` and look for `FAIL`s.
**Last verified against:** `RNS 1.2.4` / `LXMF 0.9.7` / `RNode_Firmware` (master at the spec's last revision date). Each section's source citations were re-checked against these versions; runtime verifiers in [`tools/`](tools/) lock the wire-format claims in against actually-running upstream code. When you upgrade past these, re-run every `tools/verify_*.py` and look for `FAIL`s.
Source citations refer to the standard `pip install rns lxmf` install layout (`RNS/`, `LXMF/`).
@ -208,7 +208,7 @@ Common pre-computed `name_hash` values:
prv_bytes_blob = X25519_priv(32) || Ed25519_priv(32) // 64 bytes total
```
`Identity.get_private_key()` at `RNS/Identity.py:694-698` returns this exact concatenation:
`Identity.get_private_key()` at `RNS/Identity.py:723-728` returns this exact concatenation:
```python
def get_private_key(self):
@ -243,7 +243,7 @@ The reason: `X25519PrivateKey.from_private_bytes` and `Ed25519PrivateKey.from_pr
#### Cross-implementation portability
The format is portable across implementations because there's nothing in it but the raw bytes. A 64-byte file written by Python RNS is byte-identical to one written by any clean-room implementation that follows this section, and both produce the same `identity_hash` and `lxmf.delivery` `destination_hash` when fed back through §1.1 and §1.2 — test vectors at [`test-vectors/identities.json`](test-vectors/identities.json) demonstrate the round-trip against RNS 1.2.0.
The format is portable across implementations because there's nothing in it but the raw bytes. A 64-byte file written by Python RNS is byte-identical to one written by any clean-room implementation that follows this section, and both produce the same `identity_hash` and `lxmf.delivery` `destination_hash` when fed back through §1.1 and §1.2 — test vectors at [`test-vectors/identities.json`](test-vectors/identities.json) demonstrate the round-trip against RNS 1.2.4.
> ⚠️ **Spec correction:** Earlier revisions of this section described the on-disk order as Ed25519 first, X25519 second ("opposite of the public_key concatenation"). That was wrong — verified by re-running `Identity.to_file` and reading back the bytes against the test vector at `test-vectors/identities.json`, the actual order is X25519 first, Ed25519 second, identical to the public_key order. Implementations following the prior spec wording would have corrupted identity files when interoperating with upstream Python RNS.
@ -324,7 +324,7 @@ bit 3-2 : destination_type (0 = SINGLE, 1 = GROUP, 2 = PLAIN, 3 = LINK)
bit 1-0 : packet_type (0 = DATA, 1 = ANNOUNCE, 2 = LINKREQUEST, 3 = PROOF)
```
Each subfield is **1 bit** (or 2 for `destination_type` / `packet_type`). Upstream's parser extracts them with these masks (`RNS/Packet.py:246-250` in RNS 1.2.0):
Each subfield is **1 bit** (or 2 for `destination_type` / `packet_type`). Upstream's parser extracts them with these masks (`RNS/Packet.py:246-250` in RNS 1.2.4):
```python
self.header_type = (self.flags & 0b01000000) >> 6 # bit 6 only
@ -334,7 +334,7 @@ self.destination_type = (self.flags & 0b00001100) >> 2
self.packet_type = (self.flags & 0b00000011)
```
Bit 7 (`ifac_flag`) is set by `Transport.transmit` immediately before transmission when the egress interface has an IFAC identity attached (`RNS/Transport.py:993-1024`). The setter is unambiguous:
Bit 7 (`ifac_flag`) is set by `Transport.transmit` immediately before transmission when the egress interface has an IFAC identity attached (`RNS/Transport.py:994-1024`). The setter is unambiguous:
```python
# Set IFAC flag
@ -343,7 +343,7 @@ new_header = bytes([raw[0] | 0x80, raw[1]]) # 0x80 = bit 7
new_raw = new_header + ifac + raw[2:] # IFAC field is inserted between header and addresses
```
When `ifac_flag = 1`, an `ifac_size`-byte IFAC field appears immediately after byte 2 of the header — i.e. between the `hops` byte and the start of `ADDRESSES`. `ifac_size` is interface-configured and ranges from `IFAC_MIN_SIZE = 1` byte (`RNS/Reticulum.py:151-154`) up to 64 bytes (full Ed25519 signature). The receiving side strips the IFAC after verification and rebuilds the un-IFACed packet for upstream processing (`RNS/Transport.py:1342-1369`).
When `ifac_flag = 1`, an `ifac_size`-byte IFAC field appears immediately after byte 2 of the header — i.e. between the `hops` byte and the start of `ADDRESSES`. `ifac_size` is interface-configured and ranges from `IFAC_MIN_SIZE = 1` byte (`RNS/Reticulum.py:148-152`) up to 64 bytes (full Ed25519 signature). The receiving side strips the IFAC after verification and rebuilds the un-IFACed packet for upstream processing (`RNS/Transport.py:1343-1369`).
> ⚠️ **Spec correction.** Earlier revisions of this section (through commit [`8c4d550`](https://github.com/thatSFguy/reticulum-specifications/commit/8c4d550)) treated `header_type` as a 2-bit field occupying bits 7-6, with bit 7 reserved. That was wrong: bit 7 has always been the IFAC flag (`Transport.transmit` line 1003), and `header_type` is 1 bit at position 6. The official manual §4.6.3 documents the correct layout. Implementations that followed the prior wording will mis-parse IFAC-protected packets as `header_type = 2 or 3` and reject them. Surfaced by the reporter on [issue #4](https://github.com/thatSFguy/reticulum-specifications/issues/4) item #1.
@ -358,7 +358,7 @@ HEADER_2: flags(1) hops(1) transport_id(16) dest_hash(16) context(1) data(...)
### 2.3 Originator HEADER_1 → HEADER_2 conversion
This is non-obvious and matters: when an **originator** (not a relay) sends a packet to a destination known to be more than 1 hop away, the originator MUST also do the HEADER_2 conversion. From `RNS/Transport.py::outbound` (lines 1074-1083 in RNS 1.2.0; verified by `tools/verify_packet_header.py`):
This is non-obvious and matters: when an **originator** (not a relay) sends a packet to a destination known to be more than 1 hop away, the originator MUST also do the HEADER_2 conversion. From `RNS/Transport.py::outbound` (lines 1077-1108 in RNS 1.2.4; verified by `tools/verify_packet_header.py`):
```python
if path_entry[IDX_PT_HOPS] > 1:
@ -370,7 +370,7 @@ if path_entry[IDX_PT_HOPS] > 1:
new_raw += packet.raw[2:] # original dest_hash + context + payload
```
For destinations 0 or 1 hops away, the originator may stay HEADER_1 — the receiving rnsd auto-fills the transport_id when the destination matches a local client (`for_local_client` branch at `RNS/Transport.py:1451` in RNS 1.2.0). Implementations that always emit HEADER_1 will silently fail to deliver to multi-hop destinations even with a known path.
For destinations 0 or 1 hops away, the originator may stay HEADER_1 — the receiving rnsd auto-fills the transport_id when the destination matches a local client (`for_local_client` branch at `RNS/Transport.py:1454` in RNS 1.2.4). Implementations that always emit HEADER_1 will silently fail to deliver to multi-hop destinations even with a known path.
### 2.4 Hop count
@ -380,7 +380,7 @@ Byte 1 is `hops`, an 8-bit counter that each transit relay increments by 1. `0`
Single byte after the destination hash (offset 18 for HEADER_1, offset 34 for HEADER_2). Common values:
Full context inventory from `RNS/Packet.py:72-92` (RNS 1.2.0):
Full context inventory from `RNS/Packet.py:74-92` (RNS 1.2.4):
| Hex | Name | Used for |
|---|---|---|
@ -402,7 +402,7 @@ Full context inventory from `RNS/Packet.py:72-92` (RNS 1.2.0):
| `0xFA` | KEEPALIVE | Link keepalive (sent periodically while a Link is idle) |
| `0xFB` | LINKIDENTIFY | Backchannel-identify proof on an established Link (§5 backchannel) |
| `0xFC` | LINKCLOSE | Link teardown notification |
| `0xFD` | LINKPROOF | Defined but **not actually emitted** by upstream RNS 1.2.0 in this revision. Both `Identity.prove` and `Link.prove_packet` build their proof packets with `context = NONE (0x00)` — the proof-ness is conveyed by `packet_type = PROOF (3)`, not by this context byte. Reserved for a future revision; see §6.5 |
| `0xFD` | LINKPROOF | Defined but **not actually emitted** by upstream RNS 1.2.4 in this revision. Both `Identity.prove` and `Link.prove_packet` build their proof packets with `context = NONE (0x00)` — the proof-ness is conveyed by `packet_type = PROOF (3)`, not by this context byte. Reserved for a future revision; see §6.5 |
| `0xFE` | LRRTT | Link RTT measurement reply |
| `0xFF` | LRPROOF | Link request proof (§6.2) |
@ -466,7 +466,7 @@ The 64-byte `public_key` is the X25519 || Ed25519 concat described in section 1.
random_hash = RNS.Identity.get_random_hash()[0:5] + int(time.time()).to_bytes(5, "big")
```
Transit relays read the timestamp portion via `Transport.timebase_from_random_blob(random_blob) = int.from_bytes(random_blob[5:10], "big")` (`RNS/Transport.py:3100-3101`) to make ordering decisions when an inbound announce carries a higher hop count than the cached path: only newer-emitted announces can refresh the path table (see §4.5). 5 bytes of seconds covers ~34,000 years, so wraparound is not a near-term concern. Implementations MUST emit this exact format, including a clock value that's monotonically non-decreasing across announces from the same destination — clockless sender devices (per §9.6) may end up locked out of long-range path table updates.
Transit relays read the timestamp portion via `Transport.timebase_from_random_blob(random_blob) = int.from_bytes(random_blob[5:10], "big")` (`RNS/Transport.py:3106-3107`) to make ordering decisions when an inbound announce carries a higher hop count than the cached path: only newer-emitted announces can refresh the path table (see §4.5). 5 bytes of seconds covers ~34,000 years, so wraparound is not a near-term concern. Implementations MUST emit this exact format, including a clock value that's monotonically non-decreasing across announces from the same destination — clockless sender devices (per §9.6) may end up locked out of long-range path table updates.
> ⚠️ **UNVERIFIED — Known deviation:** `attermann/microReticulum/src/Destination.cpp:270-272` (and therefore every project that uses microReticulum unmodified, including [`thatSFguy/reticulum-lora-repeater`](https://github.com/thatSFguy/reticulum-lora-repeater) and the Faketec sibling project) currently emits 10 fully-random bytes for `random_hash` — the timestamp half is a TODO that never landed:
>
@ -476,7 +476,7 @@ Transit relays read the timestamp portion via `Transport.timebase_from_random_bl
> Bytes random_hash = Cryptography::random(Type::Identity::RANDOM_HASH_LENGTH/8);
> ```
>
> Python RNS receivers interpret `random_hash[5:10]` as a big-endian uint40 unix_seconds. A uniformly-random uint40 has median value ~5.5×10¹¹ ≈ year 19403 AD, so a microReticulum announce will (with overwhelming probability) appear "far-future" to a Python receiver. Effect: once one such announce populates `path_table[dest][IDX_PT_RANDBLOBS]`, the equal-or-greater-hop branch at `RNS/Transport.py:1721-1745` will reject any real-timestamped announce as "stale" until the path TTL expires. First-contact path-table population is unaffected; the bug only surfaces on path replacement under §4.5 step 6.3. The microReticulum receive side does NOT consult the timestamp half so microReticulum-to-microReticulum traffic is unaffected. The repeater repo's `pre_build.py` patches several microReticulum protocol bugs but not this one (as of [`thatSFguy/reticulum-lora-repeater@95823ad`-vintage upstream](https://github.com/thatSFguy/reticulum-lora-repeater)). Verifying by capture-and-decode against an actual mixed-vendor mesh is the work that would let this callout be removed.
> Python RNS receivers interpret `random_hash[5:10]` as a big-endian uint40 unix_seconds. A uniformly-random uint40 has median value ~5.5×10¹¹ ≈ year 19403 AD, so a microReticulum announce will (with overwhelming probability) appear "far-future" to a Python receiver. Effect: once one such announce populates `path_table[dest][IDX_PT_RANDBLOBS]`, the equal-or-greater-hop branch at `RNS/Transport.py:1723-1755` will reject any real-timestamped announce as "stale" until the path TTL expires. First-contact path-table population is unaffected; the bug only surfaces on path replacement under §4.5 step 6.3. The microReticulum receive side does NOT consult the timestamp half so microReticulum-to-microReticulum traffic is unaffected. The repeater repo's `pre_build.py` patches several microReticulum protocol bugs but not this one (as of [`thatSFguy/reticulum-lora-repeater@95823ad`-vintage upstream](https://github.com/thatSFguy/reticulum-lora-repeater)). Verifying by capture-and-decode against an actual mixed-vendor mesh is the work that would let this callout be removed.
The optional 32-byte `ratchet_pub` (an X25519 public key) is present iff the packet header's `context_flag` bit is 1. Indexing through this layout accordingly is mandatory; see `RNS/Identity.py::validate_announce` for the canonical parser.
@ -491,10 +491,10 @@ Note that `dest_hash` is INCLUDED in the signed data even though it's not in the
### 4.3 `app_data` format for LXMF delivery destinations
Upstream `LXMF/LXMRouter.py::get_announce_app_data` produces a 2-element msgpack array (verified against LXMF 0.9.6 by `tools/verify_announce_app_data.py`):
Upstream `LXMF/LXMRouter.py::get_announce_app_data` produces a 2-element msgpack array (verified against LXMF 0.9.7 by `tools/verify_announce_app_data.py`):
```python
# LXMF/LXMRouter.py:986-1002 in LXMF 0.9.6
# LXMF/LXMRouter.py:985-1002 in LXMF 0.9.7
peer_data = [display_name, stamp_cost] # stamp_cost = None unless 1 ≤ N ≤ 254
return msgpack.packb(peer_data)
```
@ -510,7 +510,7 @@ c0 # nil (stamp_cost)
Encoding the display name as msgpack `bin` (`0xc4 NN`) is required for upstream interop — see section 9.3 below. The stamp_cost field can be `int 0` (`0x00`) or `nil` (`0xc0`); upstream's `stamp_cost_from_app_data` doesn't strict-type-check.
**A third optional `[capability_flags]` element** (e.g. `[SF_COMPRESSION]`, the only flag currently defined at `LXMF/LXMF.py:108`) is **read by the parser** (`compression_support_from_app_data` at `LXMF/LXMF.py:154-167`) but is **not emitted by the LXMF 0.9.6 producer** — `LXMRouter.py:999` computes `supported_functionality = [SF_COMPRESSION]` but never appends it to `peer_data`. Implementations should accept the 3-element form on inbound (a future LXMF version may re-enable it; older deployments may emit it) but should not rely on receiving it.
**A third optional `[capability_flags]` element** (e.g. `[SF_COMPRESSION]`, the only flag currently defined at `LXMF/LXMF.py:108`) is **read by the parser** (`compression_support_from_app_data` at `LXMF/LXMF.py:154-167`) but is **not emitted by the LXMF 0.9.7 producer** — `LXMRouter.py:998` computes `supported_functionality = [SF_COMPRESSION]` but never appends it to `peer_data`. Implementations should accept the 3-element form on inbound (a future LXMF version may re-enable it; older deployments may emit it) but should not rely on receiving it.
The parser also tolerates a 1-element msgpack array (just the name) and a raw UTF-8 string ("original announce format" branch at `LXMF/LXMF.py:138-139`) — see `LXMF/LXMF.py::display_name_from_app_data` for all four accepted shapes.
@ -528,7 +528,7 @@ Treating every announce as a contact (the naive default) populates the UI with h
### 4.5 Announce validation rules (receive side)
These are the MUST rules a receiver applies to every inbound announce before considering the announced destination "known". The canonical implementation is `RNS/Identity.py::validate_announce` (line 496-598 in RNS 1.2.0); the dispatch site that calls it is `RNS/Transport.py::inbound` line 1623-1650.
These are the MUST rules a receiver applies to every inbound announce before considering the announced destination "known". The canonical implementation is `RNS/Identity.py::validate_announce` (line 509-612 in RNS 1.2.4); the dispatch site that calls it is `RNS/Transport.py::inbound` line 1631-1655.
#### 1. Body parse — branch on `context_flag`
@ -574,33 +574,33 @@ identity_hash = SHA256(public_key)[:16]
expected_hash = SHA256(name_hash || identity_hash)[:16]
```
Reject the announce iff `expected_hash != packet.destination_hash` (the value from the outer header). This catches both random hash collisions and active spoofing attempts that pair a valid signature with an unrelated dest_hash. (`RNS/Identity.py:548-551`).
Reject the announce iff `expected_hash != packet.destination_hash` (the value from the outer header). This catches both random hash collisions and active spoofing attempts that pair a valid signature with an unrelated dest_hash. (`RNS/Identity.py:562-565`).
#### 4. Public-key collision rejection
If the receiver already has a different public_key cached for this `destination_hash` (from a prior announce), the new announce MUST be rejected with a critical-severity log even if the signature is otherwise valid. Per the upstream comment: "In reality, this should never occur, but in the odd case that someone manages a hash collision, we reject the announce" (`RNS/Identity.py:554-560`).
If the receiver already has a different public_key cached for this `destination_hash` (from a prior announce), the new announce MUST be rejected with a critical-severity log even if the signature is otherwise valid. Per the upstream comment: "In reality, this should never occur, but in the odd case that someone manages a hash collision, we reject the announce" (`RNS/Identity.py:569-575`).
This rule means: **first-announcer-wins for any given destination_hash** within a receiver's lifetime. A peer who loses their identity material and regenerates with the same display name + app_name will produce a different identity_hash → different destination_hash → no collision. A peer who tries to *replace* their announced public key under the same destination_hash, however, gets rejected — the real defense against this class of attack.
#### 5. Blackhole list check
Before everything else, check `RNS.Transport.blackholed_identities`. An identity_hash on the blackhole list is dropped silently regardless of signature validity (`RNS/Identity.py:538-541`). This is operator-controlled state, not a wire feature.
Before everything else, check `RNS.Transport.blackholed_identities`. An identity_hash on the blackhole list is dropped silently regardless of signature validity (`RNS/Identity.py:551-554`). This is operator-controlled state, not a wire feature.
#### 6. Caching the announce contents
On a fully validated announce, the receiver MUST update its caches in this order:
1. **`known_destinations[destination_hash]`** ← `[recv_time, packet_hash, public_key, app_data, last_used]` — populates the table that `RNS.Identity.recall(dest_hash)` reads when constructing outbound destinations (`RNS/Identity.py::remember`, line 100-112). Without this, every subsequent outbound message to this peer fails because no public key is available for Token encryption.
1. **`known_destinations[destination_hash]`** ← `[recv_time, packet_hash, public_key, app_data, last_used]` — populates the table that `RNS.Identity.recall(dest_hash)` reads when constructing outbound destinations (`RNS/Identity.py::remember`, line 101-113). Without this, every subsequent outbound message to this peer fails because no public key is available for Token encryption.
2. **`known_ratchets[destination_hash]`** ← `ratchet_pub` (only if `context_flag == 1` and `ratchet_pub != b""`) — `Identity._remember_ratchet`, line 395-428. The ratchet is also persisted to disk under `{storagepath}/ratchets/{hexhash}` for use across restarts.
3. **`path_table`** entry update or insertion (see §4.6 — TBD when the relay rebroadcast spec lands), gated by:
- `random_blob` (= `random_hash`) not in the cached `random_blobs` history for this destination — cheap replay defence (`RNS/Transport.py:1707, 1732, 1745`).
- Hop count comparison against any existing entry: equal-or-fewer hops always win; more hops win only if the cached path has expired or the new announce's emission timestamp (from `random_hash[5:10]`) is more recent than every cached blob's timestamp (`RNS/Transport.py:1700-1745`).
- `random_blob` (= `random_hash`) not in the cached `random_blobs` history for this destination — cheap replay defence (`RNS/Transport.py:1710, 1735, 1748`).
- Hop count comparison against any existing entry: equal-or-fewer hops always win; more hops win only if the cached path has expired or the new announce's emission timestamp (from `random_hash[5:10]`) is more recent than every cached blob's timestamp (`RNS/Transport.py:1700-1755`).
#### 7. `PATH_RESPONSE` distinction
An announce whose outer packet `context == PATH_RESPONSE (0x0B)` is the responder's reply to a recent `path?` request, not a periodic re-announce. Validation is identical (rules 1-6 above), but listener dispatch differs:
- The default behavior of `Transport.announce_handlers` registered via `RNS.Transport.register_announce_handler` is to **skip** path-response announces unless the handler sets `receive_path_responses = True` on itself (`RNS/Transport.py:1989-1991`).
- The default behavior of `Transport.announce_handlers` registered via `RNS.Transport.register_announce_handler` is to **skip** path-response announces unless the handler sets `receive_path_responses = True` on itself (`RNS/Transport.py:1991-1995`).
- The path table population path is the same either way — both regular and path-response announces refresh the path entry — so a leaf client that ignores PATH_RESPONSE entirely at the application layer still benefits from the path-table side effect.
#### 8. Implementation-private behavior (SHOULD)
@ -608,18 +608,18 @@ An announce whose outer packet `context == PATH_RESPONSE (0x0B)` is the responde
These are not wire-spec MUST rules but most working clients implement them; without them the implementation will misbehave in busy meshes:
- **Per-interface ingress rate limiting.** When the inbound announce rate on an interface exceeds `IC_BURST_FREQ_NEW = 6 Hz` (interfaces less than 2 hours old) or `IC_BURST_FREQ = 35 Hz` (older), and the announced destination is **not** in `path_table` and **not** in `path_requests`, the announce is held in the interface's `held_announces` dict for later release rather than processed immediately. Released later in lowest-hop-count-first order. (`RNS/Interfaces/Interface.py:60-200`.) Without this, a flood of unknown-destination announces can drown out everything else.
- **`random_blob` history cap.** The cached `random_blobs` list per destination is bounded by `Transport.MAX_RANDOM_BLOBS` to keep the path table from growing without bound under a long-lived destination's announce stream (`RNS/Transport.py:1820`).
- **`random_blob` history cap.** The cached `random_blobs` list per destination is bounded by `Transport.MAX_RANDOM_BLOBS` (= 64 in RNS 1.2.4 at `RNS/Transport.py:97`) to keep the path table from growing without bound under a long-lived destination's announce stream (`RNS/Transport.py:1823`).
- **Self-announce filter.** §9.5 — drop announces where `destination_hash` matches one of the receiver's own destinations to avoid populating its own contact list with itself.
#### 9. Source map for §4.5
| File | What it pins down |
|---|---|
| `RNS/Identity.py:496-598` | `validate_announce` — body parse, signed_data, sig verify, dest_hash recompute, collision check |
| `RNS/Identity.py:100-112` | `Identity.remember` — `known_destinations` update |
| `RNS/Identity.py:395-428` | `_remember_ratchet` — ratchet persistence |
| `RNS/Transport.py:1623-2024` | inbound dispatch for `packet_type == ANNOUNCE`: quick sig check, ingress limiting, path table population, handler dispatch |
| `RNS/Transport.py:3100-3117` | `timebase_from_random_blob`, `announce_emitted` |
| `RNS/Identity.py:509-612` | `validate_announce` — body parse, signed_data, sig verify, dest_hash recompute, collision check |
| `RNS/Identity.py:101-113` | `Identity.remember` — `known_destinations` update |
| `RNS/Identity.py:408-441` | `_remember_ratchet` — ratchet persistence |
| `RNS/Transport.py:1631-2030` | inbound dispatch for `packet_type == ANNOUNCE`: quick sig check, ingress limiting, path table population, handler dispatch |
| `RNS/Transport.py:3106-3122` | `timebase_from_random_blob`, `announce_emitted` |
| `RNS/Interfaces/Interface.py:60-200` | ingress-limit constants, `should_ingress_limit`, `hold_announce`, `process_held_announces` |
| `RNS/Packet.py:83` | `PATH_RESPONSE = 0x0B` context constant |
@ -938,10 +938,10 @@ Receivers parse this via `pn_announce_data_is_valid` (`LXMF/LXMF.py:191-206`), w
| File | What |
|---|---|
| `LXMF/LXMRouter.py:173` | propagation_destination construction |
| `LXMF/LXMRouter.py:307-319` | propagation announce app_data shape |
| `LXMF/LXMRouter.py:651-655` | `/offer` and `/get` handler registration |
| `LXMF/LXMRouter.py:1427-1500` | `message_get_request` handler (client `/get`) |
| `LXMF/LXMRouter.py:2142-2192` | `offer_request` handler (peer `/offer`) |
| `LXMF/LXMRouter.py:306-322` | propagation announce app_data shape |
| `LXMF/LXMRouter.py:650-651` | `/offer` and `/get` handler registration |
| `LXMF/LXMRouter.py:1426-1500` | `message_get_request` handler (client `/get`) |
| `LXMF/LXMRouter.py:2145-2200` | `offer_request` handler (peer `/offer`) |
| `LXMF/LXMPeer.py:14-50` | path constants and error-response constants |
| `LXMF/LXMPeer.py:370-486` | initiator-side `/offer` flow |
| `LXMF/LXStamper.py::validate_peering_key` | peering-key PoW validation |
@ -971,13 +971,13 @@ Both initiator-side keys are **fresh ephemeral keys** (not the initiator's long-
A `packet_type = PROOF (3)` with `context = 0xff`, addressed to the link itself — i.e. `dest_hash` in the packet header is the 16-byte `link_id` (`RNS/Packet.py:182-184`: when context is `LRPROOF`, `header += destination.link_id` and the body is appended unencrypted).
Body (`proof_data` at `RNS/Link.py:376`):
Body (`proof_data` at `RNS/Link.py:371`):
```
signature(64) || responder_X25519_pub(32) || [signalling(3)]
```
Only the responder's X25519 is fresh-ephemeral; the responder signs with its **long-term** Ed25519 private key (asymmetric with the initiator). The responder's long-term Ed25519 public key is **not** sent on the wire — both sides already know it from the responder's prior announce, and it is included implicitly in the signature input. Signature input (`RNS/Link.py:373` for the signer, `:417` for the validator):
Only the responder's X25519 is fresh-ephemeral; the responder signs with its **long-term** Ed25519 private key (asymmetric with the initiator). The responder's long-term Ed25519 public key is **not** sent on the wire — both sides already know it from the responder's prior announce, and it is included implicitly in the signature input. Signature input (`RNS/Link.py:373-374` for the signer, `:417` for the validator):
```
signed_data = link_id || responder_X25519_pub || responder_long_term_Ed25519_pub || [signalling]
@ -1003,7 +1003,7 @@ hashable_part = byte(flags & 0x0F) || raw[N:]
The "hashable part" deliberately strips `header_type`, `context_flag`, `transport_type` (top 4 bits of flags — modifiable by transit relays), the `hops` byte (modified by every relay), and (for HEADER_2) the `transport_id` (added by the originator and re-written by each relay). What remains in both cases is the low nibble of flags + dest_hash + context + body, so the resulting `link_id` is the same whether the LINKREQUEST is hashed at the initiator (HEADER_1) or at the responder after one or more transport relays (HEADER_2). Both sides agree on the 16-byte ID.
For LINKREQUEST packets specifically, the trailing signalling bytes (if present, indicated by `len(packet.data) > Link.ECPUBSIZE` in `link_id_from_lr_packet` at `RNS/Link.py:340-347`) are stripped from the END of `hashable_part` before hashing, so the link_id is invariant under MTU-discovery signalling.
For LINKREQUEST packets specifically, the trailing signalling bytes (if present, indicated by `len(packet.data) > Link.ECPUBSIZE` in `link_id_from_lr_packet` at `RNS/Link.py:341-348`) are stripped from the END of `hashable_part` before hashing, so the link_id is invariant under MTU-discovery signalling.
### 6.4 Session key derivation
@ -1050,7 +1050,7 @@ The two forms are distinguished **purely by length** at the receiver. `PacketRec
Sender side, two distinct policies:
**Opportunistic DATA addressed to a SINGLE destination** — `RNS.Identity.prove(packet, destination)` at `RNS/Identity.py:912-923`:
**Opportunistic DATA addressed to a SINGLE destination** — `RNS.Identity.prove(packet, destination)` at `RNS/Identity.py:943-954`:
```python
def prove(self, packet, destination=None):
@ -1064,7 +1064,7 @@ def prove(self, packet, destination=None):
proof.send()
```
The default upstream value is `Reticulum.__use_implicit_proof = True` (`RNS/Reticulum.py:259`), so **upstream emits the 64-byte implicit form by default**. The 96-byte explicit form is only emitted when the operator's `[reticulum]` config sets `use_implicit_proof = No`. A clean-room implementation that hardcodes either single form will fail to interop with peers running the other one — receiver-side validators handle both, but a hardcoded sender writing the wrong length to the wire is not negotiable.
The default upstream value is `Reticulum.__use_implicit_proof = True` (`RNS/Reticulum.py:256`), so **upstream emits the 64-byte implicit form by default**. The 96-byte explicit form is only emitted when the operator's `[reticulum]` config sets `use_implicit_proof = No`. A clean-room implementation that hardcodes either single form will fail to interop with peers running the other one — receiver-side validators handle both, but a hardcoded sender writing the wrong length to the wire is not negotiable.
**DATA on an active Link** — `RNS.Link.prove_packet(packet)` at `RNS/Link.py:383-394`:
@ -1076,7 +1076,7 @@ def prove_packet(self, packet):
proof.send()
```
with the upstream comment `# TODO: Hardcoded as explicit proof for now`. Link DATA proofs are **always** the 96-byte explicit form in RNS 1.2.0 regardless of the `use_implicit_proof` setting, and the matching `validate_link_proof` at `RNS/Packet.py:449-494` has the implicit-form branch commented out with the same note. Today, Link DATA proofs are explicit-only on both ends; an implementation may match this behavior with a single hardcoded length on the link path, but should be ready to revisit if upstream re-enables the implicit branch (no fixed timeline).
with the upstream comment `# TODO: Hardcoded as explicit proof for now`. Link DATA proofs are **always** the 96-byte explicit form in RNS 1.2.4 regardless of the `use_implicit_proof` setting, and the matching `validate_link_proof` at `RNS/Packet.py:449-494` has the implicit-form branch commented out with the same note. Today, Link DATA proofs are explicit-only on both ends; an implementation may match this behavior with a single hardcoded length on the link path, but should be ready to revisit if upstream re-enables the implicit branch (no fixed timeline).
#### 6.5.3 Where the proof packet is addressed
@ -1098,7 +1098,7 @@ implicit form (64 bytes total body):
[ 64B Ed25519_signature(SHA256(get_hashable_part(original_packet))) ]
```
Note `context = 0x00 (NONE)` in both cases — the proof-ness is conveyed by `packet_type = PROOF (3)` in the flag byte, not by a context. This is in contrast to LRPROOF (which uses `context = 0xFF`) and RESOURCE_PRF (which uses `context = 0x05`). The `LINKPROOF (0xFD)` context constant defined at `RNS/Packet.py:90` is reserved but not actually used by either prove path in RNS 1.2.0.
Note `context = 0x00 (NONE)` in both cases — the proof-ness is conveyed by `packet_type = PROOF (3)` in the flag byte, not by a context. This is in contrast to LRPROOF (which uses `context = 0xFF`) and RESOURCE_PRF (which uses `context = 0x05`). The `LINKPROOF (0xFD)` context constant defined at `RNS/Packet.py:90` is reserved but not actually used by either prove path in RNS 1.2.4.
#### 6.5.5 Receiver tolerance
@ -1124,7 +1124,7 @@ byte 1 : m m m m m m m m — mtu[15..8]
byte 2 : m m m m m m m m — mtu[7..0]
```
Encoded by `RNS/Link.py:147-151`:
Encoded by `RNS/Link.py:148-152`:
```python
@staticmethod
@ -1148,9 +1148,9 @@ The mtu decode trick: the full 24-bit value of all three bytes is masked with th
#### 6.6.2 Mode field
3-bit value (0..7) at the top of byte 0. Defined values, with `RNS/Link.py:125-142`:
3-bit value (0..7) at the top of byte 0. Defined values, with `RNS/Link.py:126-145`:
| Mode | Name | Status in RNS 1.2.0 | Derived key length |
| Mode | Name | Status in RNS 1.2.4 | Derived key length |
|---|---|---|---|
| `0x00` | `MODE_AES128_CBC` | Defined, NOT enabled (sender-side will raise `TypeError`) | 32 bytes |
| `0x01` | `MODE_AES256_CBC` | Default; the only enabled mode (`ENABLED_MODES = [0x01]`) | 64 bytes |
@ -1158,7 +1158,7 @@ The mtu decode trick: the full 24-bit value of all three bytes is masked with th
| `0x03` | `MODE_OTP_RESERVED` | Reserved, not enabled | — |
| `0x04`–`0x07` | `MODE_PQ_RESERVED_*` | Reserved for the post-quantum migration; not enabled | — |
The `derived_key_length` at `RNS/Link.py:358-360` is what the HKDF in §6.4 produces, split as `signing_key(32) || encrypt_key(32)` for the AES-256 path or `signing_key(16) || encrypt_key(16)` for the AES-128 path.
The `derived_key_length` at `RNS/Link.py:359-361` is what the HKDF in §6.4 produces, split as `signing_key(32) || encrypt_key(32)` for the AES-256 path or `signing_key(16) || encrypt_key(16)` for the AES-128 path.
A receiver MUST tolerate seeing any 3-bit value in the mode field on inbound traffic — `mode_from_lr_packet` returns the raw integer without validating it against `ENABLED_MODES`. The mode is enforced at handshake time (`Link.handshake` at line 353-368): unknown / disabled modes raise `TypeError` and the link transitions to `CLOSED` rather than `ACTIVE`. Senders MUST NOT emit any mode value not in `ENABLED_MODES` — `signalling_bytes()` raises if you try.
@ -1178,7 +1178,7 @@ else:
signalling_bytes = Link.signalling_bytes(RNS.Reticulum.MTU, self.mode)
```
When the responder emits an LRPROOF with signalling, the encoded `mtu` is the **min** of its own next-hop view and what arrived in the LINKREQUEST, computed during validation (`RNS/Transport.py:2042-2051`):
When the responder emits an LRPROOF with signalling, the encoded `mtu` is the **min** of its own next-hop view and what arrived in the LINKREQUEST. In RNS 1.2.4 the clamp is performed by transit relays in the DATA forwarding path (`RNS/Transport.py:1539-1556`), which rewrite the LINKREQUEST's signalling bytes in place before forwarding so the responder's `Link.validate_request` (`RNS/Link.py:186-200`) sees the already-clamped value:
```python
path_mtu = Link.mtu_from_lr_packet(packet) or Reticulum.MTU
@ -1214,7 +1214,7 @@ Per §6.2, the LRPROOF's signed_data when signalling is present is:
signed_data = link_id || responder_X25519_pub || responder_long_term_Ed25519_pub || signalling
```
A clean-room implementation that omits the signalling bytes when present (or includes them when absent) computes a different signed_data than the responder did, fails signature validation, and the link never establishes. This is the most common interop break in this area; cross-check against `RNS/Link.py:373` (signer) and `:417` (validator).
A clean-room implementation that omits the signalling bytes when present (or includes them when absent) computes a different signed_data than the responder did, fails signature validation, and the link never establishes. This is the most common interop break in this area; cross-check against `RNS/Link.py:373-374` (signer) and `:417` (validator).
#### 6.6.6 Disabling MTU discovery
@ -1228,7 +1228,7 @@ A Link goes through five states (`RNS/Link.py:110-114`): `PENDING → HANDSHAKE
#### 6.7.1 KEEPALIVE (`context = 0xFA`)
Cadence (`RNS/Link.py:844-846`):
Cadence (`RNS/Link.py:845-847`):
```python
def __update_keepalive(self):
@ -1248,7 +1248,7 @@ def send_keepalive(self):
Body is a single byte `0xFF` — the "ping" sentinel. The packet is Token-encrypted with the link's session key per §3.1 link-derived form, so the wire body is `iv(16) || ciphertext(...) || hmac(32)`; the decrypted plaintext is just `b'\xff'`.
The **responder** receives this in `Link.receive` at `RNS/Link.py:1149-1153` and answers with the "pong" sentinel:
The **responder** receives this in `Link.receive` at `RNS/Link.py:1149-1153` and answers with the "pong" sentinel (in 1.2.4 the body is `bytes([0xFE])`):
```python
elif packet.context == RNS.Packet.KEEPALIVE:
@ -1281,7 +1281,7 @@ elif self.status == Link.STALE:
`teardown_reason` is set to `Link.TIMEOUT` (constant value `0x01`) so the application's `link_closed_callback` can distinguish "the peer went dark" from "the peer cleanly closed".
There is also an explicit-cleanup path: after a STALE-induced teardown the watchdog adds a final grace period of `RTT × KEEPALIVE_TIMEOUT_FACTOR + STALE_GRACE` (= `RTT × 4 + 5s`) at line 797 to allow a delayed reply to bring the link back into ACTIVE before final teardown — but in upstream RNS 1.2.0 the `STALE → CLOSED` transition runs immediately on the next watchdog pass without consulting that grace period. The grace constant lives in case a future revision restores the soft-stale window.
There is also an explicit-cleanup path: after a STALE-induced teardown the watchdog adds a final grace period of `RTT × KEEPALIVE_TIMEOUT_FACTOR + STALE_GRACE` (= `RTT × 4 + 5s`) at line 797 to allow a delayed reply to bring the link back into ACTIVE before final teardown — but in upstream RNS 1.2.4 the `STALE → CLOSED` transition runs immediately on the next watchdog pass without consulting that grace period. The grace constant lives in case a future revision restores the soft-stale window.
#### 6.7.3 LINKCLOSE (`context = 0xFC`)
@ -1297,7 +1297,7 @@ Wire form:
- `packet_type = DATA (0)`, `context = 0xFC`, `dest_hash = link_id`.
- Body is the **16-byte link_id**, Token-encrypted by the link's session key.
The peer's receiver path at `RNS/Link.py:1061-1063` calls `teardown_packet(packet)` (line 710-722):
The peer's receiver path at `RNS/Link.py:1061-1063` calls `teardown_packet(packet)` (line 710-728):
```python
def teardown_packet(self, packet):
@ -1426,7 +1426,7 @@ A clean-room client that only implements opportunistic LXMF can ignore Channel e
### 7.1 Path requests: peers send `path?` before opportunistic LXMF when no path is known
The path-request preamble in upstream LXMF is **conditional, not unconditional** (verified by `tools/verify_path_request.py` against LXMF 0.9.6):
The path-request preamble in upstream LXMF is **conditional, not unconditional** (verified by `tools/verify_path_request.py` against LXMF 0.9.7):
```python
# LXMF/LXMRouter.py::handle_outbound, ~line 1672
@ -1436,7 +1436,7 @@ if not RNS.Transport.has_path(destination_hash) and lxmessage.method == LXMessag
lxmessage.next_delivery_attempt = time.time() + LXMRouter.PATH_REQUEST_WAIT
```
In other words: a `path?` is sent before the LXM **only when no entry exists in `Transport.path_table`** for the target — `has_path()` is just a key-presence check (`RNS/Transport.py:2570-2576`). Existing-but-stale path entries are NOT replaced by this preamble; LXMF instead leans on the periodic `Transport.jobs` cycle to evict expired path entries (`stale_paths` accumulator at `RNS/Transport.py:747+`), after which the next outbound LXM rediscovers the unknown-path branch and triggers the `request_path`. A second `request_path` is issued from the retry path (`LXMRouter.py:2571+`) once `lxmessage.delivery_attempts >= MAX_PATHLESS_TRIES`, so on a flaky path peers can see multiple `path?` retransmits without intervening DATA — that matches BLE-trace observations.
In other words: a `path?` is sent before the LXM **only when no entry exists in `Transport.path_table`** for the target — `has_path()` is just a key-presence check via `LXMRouter.handle_outbound`. Existing-but-stale path entries are NOT replaced by this preamble; LXMF instead leans on the periodic `Transport.jobs` cycle to evict expired path entries (`stale_paths` accumulator at `RNS/Transport.py:750+`), after which the next outbound LXM rediscovers the unknown-path branch and triggers the `request_path`. A second `request_path` is issued from the retry path (`LXMRouter.py:2568+`) once `lxmessage.delivery_attempts >= MAX_PATHLESS_TRIES`, so on a flaky path peers can see multiple `path?` retransmits without intervening DATA — that matches BLE-trace observations.
A `path?` request itself is a regular DATA packet (verified by `tools/verify_path_request.py`):
@ -1452,7 +1452,7 @@ A `path?` request itself is a regular DATA packet (verified by `tools/verify_pat
#### 7.2.1 Path-request packet parse rules
The path-request handler at `RNS/Transport.py:2800-2843` parses inbound packets addressed to `path_request_destination` (the dest_hash in §7.1). The handler is registered as the destination's `packet_callback` at `Transport.py:237-240`, so any DATA packet to that dest_hash flows through it.
The path-request handler at `RNS/Transport.py:2806-2850` parses inbound packets addressed to `path_request_destination` (the dest_hash in §7.1). The handler is registered as the destination's `packet_callback` at `Transport.py:241`, so any DATA packet to that dest_hash flows through it.
The path-request destination is a **PLAIN destination** with no identity attached, which is why its `dest_hash` derives only from the name: `dest_hash = SHA256(SHA256("rnstransport.path.request")[:10])[:16]` per the PLAIN/GROUP recipe in §1.4.3 (the `identity == None` branch of `Destination.hash` at `RNS/Destination.py:121-130`). The result is a constant — `6b9f66014d9853faab220fba47d02761` — that every node on the mesh resolves identically without needing to discover a per-peer identity first.
@ -1481,7 +1481,7 @@ Three observations that matter for interop:
#### 7.2.2 Tag-based deduplication
The handler builds `unique_tag = destination_hash || tag_bytes` and consults `Transport.discovery_pr_tags` (`Transport.py:2829-2839`):
The handler builds `unique_tag = destination_hash || tag_bytes` and consults `Transport.discovery_pr_tags` (`Transport.py:2835-2845`):
```python
unique_tag = destination_hash + tag_bytes
@ -1498,13 +1498,13 @@ with Transport.discovery_pr_tags_lock:
# ignore duplicate path request
```
`discovery_pr_tags` is bounded at `Transport.max_pr_tags = 32000` entries (`Transport.py:126`); older entries are aged out by the periodic `Transport.jobs` cycle. **Every node — leaf or transport — that wants to respond to path requests MUST maintain this dedup table** or it will respond to every retransmit, and a transport-enabled node will additionally re-forward to all other interfaces, generating a broadcast storm.
`discovery_pr_tags` is bounded at `Transport.max_pr_tags = 32000` entries (`Transport.py:127`); older entries are aged out by the periodic `Transport.jobs` cycle. **Every node — leaf or transport — that wants to respond to path requests MUST maintain this dedup table** or it will respond to every retransmit, and a transport-enabled node will additionally re-forward to all other interfaces, generating a broadcast storm.
The `unique_tag = dest_hash || tag` format means the same tag bytes against different destination_hashes are distinct — so two different requesters racing for the same target with happenstance-identical random tags don't suppress each other. Senders MUST use a fresh random tag per fresh request (the upstream emitter calls `Identity.get_random_hash()`); reusing tags across requests for the same destination_hash makes the second request appear to be a duplicate.
#### 7.2.3 The five-way dispatch in `Transport.path_request`
`RNS/Transport.py:2846-2973`. After dedup, the handler calls into `path_request()` which decides how to respond. Five mutually-exclusive branches in priority order:
`RNS/Transport.py:2852-2980`. After dedup, the handler calls into `path_request()` which decides how to respond. Five mutually-exclusive branches in priority order:
1. **`destination_hash` is local** (i.e. it's one of our own registered destinations, line 2873-2875):
```python
@ -1538,11 +1538,11 @@ A `tag` argument hands a previously-built path-response announce body back uncha
#### 7.2.5 Timing: `PATH_REQUEST_GRACE` and roaming
When branch 2 fires (transit relay answering on behalf of a remote destination), the rebroadcast is delayed by `PATH_REQUEST_GRACE = 0.4s` (`Transport.py:80, 2917`) — extra grace to let directly-reachable peers respond first if they're in earshot. On `MODE_ROAMING` interfaces an additional `PATH_REQUEST_RG = 1.5s` is added on top (`Transport.py:81, 2922-2923`) so well-connected fixed nodes get a chance to answer before mobile ones.
When branch 2 fires (transit relay answering on behalf of a remote destination), the rebroadcast is delayed by `PATH_REQUEST_GRACE = 0.4s` (`Transport.py:80`) — extra grace to let directly-reachable peers respond first if they're in earshot. On `MODE_ROAMING` interfaces an additional `PATH_REQUEST_RG = 1.5s` is added on top (`Transport.py:81`) so well-connected fixed nodes get a chance to answer before mobile ones.
Branch 1 (local destination answers) fires immediately with no grace, since the leaf is the authoritative source for its own destination — there's no point waiting for someone else to potentially answer faster.
Local-client originators also bypass the grace period (`Transport.py:2909-2910`): a relay answering for a destination that lives on a local-client interface can send back the cached announce instantly because the answer doesn't need to compete with peer-mesh announces.
Local-client originators also bypass the grace period (`Transport.py:2915-2920`): a relay answering for a destination that lives on a local-client interface can send back the cached announce instantly because the answer doesn't need to compete with peer-mesh announces.
#### 7.2.6 Minimum responsibility for a leaf
@ -1565,7 +1565,7 @@ For a chronological walk-through of the full request → response → path-table
The 32-byte `ratchet_pub` field in announces is meant to rotate periodically. The **purpose** is forward secrecy: rotating the ECDH key on a regular cadence limits the plaintext window an adversary can decrypt if a single ratchet privkey leaks. It is **not** what makes your announces visible to the mesh.
The actual replay-and-loop defence in upstream is keyed on **`random_hash`**, not on `ratchet_pub` — see §4.5 step 6.3 (path-table replacement check `not random_blob in random_blobs` at `RNS/Transport.py:1707, 1732, 1745`). Verified by `tools/verify_ratchet_dedup.py`: two announces sharing a `ratchet_pub` but differing in `random_hash[:5]` are both accepted by upstream's replay machinery.
The actual replay-and-loop defence in upstream is keyed on **`random_hash`**, not on `ratchet_pub` — see §4.5 step 6.3 (path-table replacement check `not random_blob in random_blobs` at `RNS/Transport.py:1710, 1735, 1748`). Verified by `tools/verify_ratchet_dedup.py`: two announces sharing a `ratchet_pub` but differing in `random_hash[:5]` are both accepted by upstream's replay machinery.
> ⚠️ **Spec correction:** Earlier revisions of this section claimed transit nodes dedup announces on `(destination_hash, ratchet_pub)` tuples and that a non-rotating client becomes invisible to the mesh after one announce. That was wrong on the mechanism: upstream's `RATCHET_INTERVAL = 30 min` × `ANNOUNCE_INTERVAL = 5–15 min` means most upstream announces share a ratchet across 2–6 emissions, so if relays really dropped on `ratchet_pub` equality, upstream wouldn't function. The actual win observed in the bootstrap test (per `agent.md` §5) was incidental — the fix that rotated ratchets per announce also rotated `random_hash`, and it was the latter that mattered.
@ -1611,7 +1611,7 @@ Emitting `context_flag = 0` (no `ratchet_pub` field, body layout per §4.5 step
Senders cache the most recent ratchet they've seen for each destination. If you rotate your ratchet faster than relays propagate the announce, in-flight messages may arrive encrypted to your *previous* ratchet. To decrypt these, keep a ring of recent ratchet privkeys and try each in order during decrypt. The fallback to the long-term identity privkey is the ultimate safety net.
Upstream's default ring size is **`Destination.RATCHET_COUNT = 512`** (`RNS/Destination.py:85` in RNS 1.2.0), with a minimum rotation interval of `RATCHET_INTERVAL = 30*60` seconds (line 90) and per-ratchet `RATCHET_EXPIRY = 60*60*24*30` seconds (`RNS/Identity.py:69`). A new ratchet is generated on each `rotate_ratchets()` call and prepended to the in-memory list; `_clean_ratchets` truncates back to `RATCHET_COUNT`. The 512 figure is generous and not a hard interop requirement — it's an in-memory bound on the inbound-decrypt try-list.
Upstream's default ring size is **`Destination.RATCHET_COUNT = 512`** (`RNS/Destination.py:85` in RNS 1.2.4), with a minimum rotation interval of `RATCHET_INTERVAL = 30*60` seconds (line 90) and per-ratchet `RATCHET_EXPIRY = 60*60*24*30` seconds (`RNS/Identity.py:69`). A new ratchet is generated on each `rotate_ratchets()` call and prepended to the in-memory list; `_clean_ratchets` truncates back to `RATCHET_COUNT`. The 512 figure is generous and not a hard interop requirement — it's an in-memory bound on the inbound-decrypt try-list.
A minimal client may keep just the current ratchet privkey, accepting that the brief window between rotation and announce-propagation will lose some messages. Mention the trade-off in your implementation notes.
@ -1956,7 +1956,7 @@ Effective HW_MTU is `1196` bytes (`AutoInterface.HW_MTU`, line 44) — chosen to
#### 8.6.6 IFAC integration
If the AutoInterface is configured with an `ifac_identity` (out-of-band-shared key), every Reticulum packet on the data port is IFAC-sealed and unmasked using the standard Transport-level IFAC mechanism (`RNS/Transport.py:1338-1387`). Peers with mismatched IFAC keys can see each other's discovery announces but can't decode each other's data — a pragmatic privacy boundary on a shared LAN.
If the AutoInterface is configured with an `ifac_identity` (out-of-band-shared key), every Reticulum packet on the data port is IFAC-sealed and unmasked using the standard Transport-level IFAC mechanism (`RNS/Transport.py:1338-1390`). Peers with mismatched IFAC keys can see each other's discovery announces but can't decode each other's data — a pragmatic privacy boundary on a shared LAN.
#### 8.6.7 Source map
@ -2020,7 +2020,7 @@ LoRa devices without an RTC will populate the LXMF `timestamp` field with second
Even after a successful initial announce, paths in the mesh expire within minutes. Without a 5–15 minute re-announce loop, the second message any peer tries to send you will fail because the relay's path table has aged out. (See also §7.5.)
There is **no upstream-mandated default** — `RNS/Reticulum.py:745` uses `6*60*60` (6 h) for interface-level *discovery* announces and `RNS/Transport.py:162` uses `2*60*60` (2 h) for transport-management announces, but those are not the cadence end-user destinations announce at. Sideband emits roughly every 30 minutes; the upstream manual recommends 30–60 minutes for a desktop client. Practical guidance for application destinations:
There is **no upstream-mandated default** — `RNS/Reticulum.py:764` uses `6*60*60` (6 h) for interface-level *discovery* announces and `RNS/Transport.py:192` uses `2*60*60` (2 h) for transport-management announces, but those are not the cadence end-user destinations announce at. Sideband emits roughly every 30 minutes; the upstream manual recommends 30–60 minutes for a desktop client. Practical guidance for application destinations:
| Deployment | RECOMMENDED cadence |
|---|---|
@ -2048,7 +2048,7 @@ logged before any filtering converts hours of "messages aren't arriving" debuggi
Real interop bug to plan around: `attermann/microReticulum`'s `Destination::announce` emits 10 fully-random bytes for the announce `random_hash` field rather than the upstream Python form of `5 random bytes || big-endian uint40 unix_seconds` (see §4.1). The Python form is preserved as a comment in the C++ source with a `TODO add in time to random hash` next to it; the timestamp half was never implemented.
Effect on a mixed-vendor mesh: a Python RNS receiver parses `random_hash[5:10]` of a microReticulum announce as a far-future timestamp (median ~year 19403 AD because the random uint40 is uniformly distributed across `0..2^40-1`). The path-table replacement rule at `RNS/Transport.py:1721-1745` rejects subsequent real-timestamped announces from Python sources as "stale" until the path TTL expires.
Effect on a mixed-vendor mesh: a Python RNS receiver parses `random_hash[5:10]` of a microReticulum announce as a far-future timestamp (median ~year 19403 AD because the random uint40 is uniformly distributed across `0..2^40-1`). The path-table replacement rule at `RNS/Transport.py:1723-1755` rejects subsequent real-timestamped announces from Python sources as "stale" until the path TTL expires.
Symptom: a microReticulum repeater works fine when it's the only path; in a mesh that also has Python relays, paths "stick" to the microReticulum side even when shorter / fresher Python paths come up, until natural TTL expiry. First-contact path-table population is unaffected — the bug only surfaces on path replacement.
@ -2064,7 +2064,7 @@ The repeater repo's `pre_build.py` patches several other microReticulum protocol
A **Resource** transfers a payload that exceeds the per-packet content limit of an established Reticulum Link. It is the only way to carry an LXMF body, NomadNet page, or file larger than ~360 bytes (`LINK_PACKET_MAX_CONTENT`) over a Link. Resource is built **on top of** an active Link — it relies on the Link's session key for encryption (§3.1 link-derived form) and on the Link's bidirectional DATA channel for control traffic.
The complete reference is `RNS/Resource.py` (1383 lines in RNS 1.2.0); `RNS/Packet.py:72-78` defines the context constants. This section describes the wire-level invariants a clean-room implementation must respect; many implementation choices (window sizing heuristics, watchdog timers, EIFR computation) are private and listed only when their absence would cause an interop break.
The complete reference is `RNS/Resource.py` (1380 lines in RNS 1.2.4); `RNS/Packet.py:74-79` defines the context constants. This section describes the wire-level invariants a clean-room implementation must respect; many implementation choices (window sizing heuristics, watchdog timers, EIFR computation) are private and listed only when their absence would cause an interop break.
### 10.1 When Resource runs
@ -2163,6 +2163,20 @@ The advertisement is sent once on `Resource.advertise()`; if no part requests ar
> implementing `delivery_resource_advertised(resource)` returning
> `False` (§5.8.3 / §16.9) is the upstream-blessed way to refuse
> oversized advertisements.
>
> Upstream RNS adopted this cap in 1.1.9 after a CVE-class report:
> `Resource.assemble` uses `bz2.BZ2Decompressor.decompress(data,
> max_length=self.max_decompressed_size)` and rejects the resource
> if `decompressor.eof` is False after the bounded read
> (`RNS/Resource.py:686-691`). The Channel-mode counterpart is
> `Buffer.RawChannelReader` which caps each chunk at
> `RawChannelWriter.MAX_CHUNK_LEN` via the same `max_length` mechanism
> (`RNS/Buffer.py:95-97`). Clean-room implementations should mirror
> this — a `bz2.BZ2Decompressor.decompress(data, max_length=N)` plus
> `eof` check is the minimum. **Do not use the one-shot
> `bz2.decompress()` API for resource bodies** — it has no output
> bound and will allocate as much memory as the input legitimately
> expands to.
### 10.5 RESOURCE_REQ — receiver requests parts
@ -2497,7 +2511,7 @@ Registered via `Destination.register_request_handler(path, response_generator, a
### 11.5 RequestReceipt — initiator-side state machine
`RNS/Link.py:1348-1448`. When `Link.request()` returns a `RequestReceipt`, the initiator can attach:
`RNS/Link.py:1335-1530`. When `Link.request()` returns a `RequestReceipt`, the initiator can attach:
- `response_callback(receipt)` — fires when the response has fully arrived (single packet OR resource concluded).
- `failed_callback(receipt)` — fires on timeout or link teardown.
@ -2654,7 +2668,7 @@ None of these are wire-spec — they're caller conventions layered on top of §1
| `RNS/Link.py:478-527` | `Link.request()` — initiator-side packing and dispatch by size |
| `RNS/Link.py:853-904` | `Link.handle_request()` — server-side path lookup + auth + response dispatch |
| `RNS/Link.py:906-925` | `Link.handle_response()` — initiator-side response correlation |
| `RNS/Link.py:1348-1448` | `RequestReceipt` — callback machinery |
| `RNS/Link.py:1335-1530` | `RequestReceipt` — callback machinery |
| `RNS/Destination.py::register_request_handler` | Server-side handler registration |
| `RNS/Destination.py:35-40` | `ALLOW_NONE/ALLOW_LIST/ALLOW_ALL` constants |
| `RNS/Packet.py:81-82` | `REQUEST = 0x09`, `RESPONSE = 0x0A` context constants |
@ -2685,7 +2699,7 @@ For an inbound DATA packet (`packet_type == DATA`, `destination_type` not LINK)
- `packet.transport_id == Transport.identity.hash` (the originator picked us as the next hop), AND
- `packet.destination_hash` is in `Transport.path_table`,
the relay rewrites the wire bytes according to `path_table[dest][HOPS]` and re-transmits on `path_table[dest][RVCD_IF]`. From `RNS/Transport.py:1497-1573`, three cases by `remaining_hops`:
the relay rewrites the wire bytes according to `path_table[dest][HOPS]` and re-transmits on `path_table[dest][RVCD_IF]`. From `RNS/Transport.py:1500-1580`, three cases by `remaining_hops`:
#### 12.2.1 `remaining_hops > 1` — forward as HEADER_2
@ -2719,7 +2733,7 @@ The destination is registered on the relay itself (it's both our path-table next
#### 12.2.4 LINKREQUEST forwarding extras
When the forwarded packet is a `LINKREQUEST`, the relay also writes a `link_table` entry keyed by the link_id (computed via §6.3's `link_id_from_lr_packet`). Entry contents (`Transport.py:1553-1561`):
When the forwarded packet is a `LINKREQUEST`, the relay also writes a `link_table` entry keyed by the link_id (computed via §6.3's `link_id_from_lr_packet`). Entry contents (`Transport.py:1556-1565`):
```
[ now, # 0 IDX_LT_TIMESTAMP
@ -2739,7 +2753,7 @@ The relay also performs the §6.6 MTU clamp at this point: if the LINKREQUEST ca
#### 12.2.5 Non-LINKREQUEST DATA — reverse_table entry
For any other forwarded DATA (the much-more-common opportunistic LXMF case), the relay writes a `reverse_table` entry keyed by `packet.getTruncatedHash()` (`Transport.py:1567-1571`):
For any other forwarded DATA (the much-more-common opportunistic LXMF case), the relay writes a `reverse_table` entry keyed by `packet.getTruncatedHash()` (`Transport.py:1570-1574`):
```
[ packet.receiving_interface, # 0 IDX_RT_RCVD_IF — interface to send PROOF back through
@ -2751,7 +2765,7 @@ The reverse_table is what lets the eventual PROOF receipt (§6.5) trace its way
### 12.3 ANNOUNCE rebroadcasting
When an inbound ANNOUNCE validates (per §4.5) AND the destination is non-local AND `transport_enabled OR is_from_local_client`, the relay queues a rebroadcast. From `Transport.py:1810-1890`:
When an inbound ANNOUNCE validates (per §4.5) AND the destination is non-local AND `transport_enabled OR is_from_local_client`, the relay queues a rebroadcast. From `Transport.py:1810-1895`:
```python
if (Reticulum.transport_enabled() or is_from_local_client) and packet.context != PATH_RESPONSE:
@ -2781,7 +2795,7 @@ The cap is per-interface, not global — a relay with multiple interfaces budget
### 12.4 Path table management
`Transport.path_table[destination_hash]` entry shape (`Transport.py:3439-3446`):
`Transport.path_table[destination_hash]` entry shape (`Transport.py:3457-3464`):
```
[ timestamp, # 0 IDX_PT_TIMESTAMP — when last refreshed
@ -2807,7 +2821,7 @@ The wide spread of defaults reflects expected churn rates: AP-mode interfaces ha
#### 12.4.2 Eviction
`Transport.jobs` runs a `stale_paths` accumulator that walks `path_table` and pops entries whose `expires` timestamp has passed (`Transport.py:747-769`). Eviction is silent — no notification to the application; the next outbound message to the destination just re-discovers it via `request_path` per §7.1.
`Transport.jobs` runs a `stale_paths` accumulator that walks `path_table` and pops entries whose `expires` timestamp has passed (`Transport.py:750-770`). Eviction is silent — no notification to the application; the next outbound message to the destination just re-discovers it via `request_path` per §7.1.
A relay also evicts path entries whose underlying interface has been removed (`receiving_interface not in Transport.interfaces`). This handles the case where a TCP client disconnects.
@ -2821,7 +2835,7 @@ Once a Link's LINKREQUEST has been forwarded by a relay (§12.2.4 wrote the `lin
#### 12.5.1 LRPROOF forwarding
When an LRPROOF arrives whose `dest_hash` (= link_id) is in the relay's `link_table` AND the proof arrives on the next-hop interface (`packet.receiving_interface == link_entry[IDX_LT_NH_IF]`), the relay validates the signature against the destination's known long-term public key (recalled via `Identity.recall(link_entry[DSTHASH])`) and forwards on the receive interface (`Transport.py:2110-2138`):
When an LRPROOF arrives whose `dest_hash` (= link_id) is in the relay's `link_table` AND the proof arrives on the next-hop interface (`packet.receiving_interface == link_entry[IDX_LT_NH_IF]`), the relay validates the signature against the destination's known long-term public key (recalled via `Identity.recall(link_entry[DSTHASH])`) and forwards on the receive interface (`Transport.py:2110-2145`):
```python
new_raw = packet.raw[0:1] + struct.pack("!B", packet.hops) + packet.raw[2:]
@ -2837,7 +2851,7 @@ For a `DATA` packet with `destination_type == LINK` whose `dest_hash` is in `lin
#### 12.5.3 PROOF receipt forwarding via `reverse_table`
`Transport.py:2196-2205`. When a PROOF arrives whose `dest_hash` is in `reverse_table` (i.e. an opportunistic-DATA proof being routed back to its originator), the relay pops the entry, checks the proof arrived on the correct outbound interface (`receiving_interface == reverse_entry[IDX_RT_OUTB_IF]`), and forwards on the originally-receiving interface:
`Transport.py:2199-2208`. When a PROOF arrives whose `dest_hash` is in `reverse_table` (i.e. an opportunistic-DATA proof being routed back to its originator), the relay pops the entry, checks the proof arrived on the correct outbound interface (`receiving_interface == reverse_entry[IDX_RT_OUTB_IF]`), and forwards on the originally-receiving interface:
```python
new_raw = packet.raw[0:1] + struct.pack("!B", packet.hops) + packet.raw[2:]
@ -2852,7 +2866,7 @@ Two related state mechanisms a transport node maintains:
#### 12.6.1 `discovery_path_requests`
When a transport-enabled relay receives a path? for a destination it doesn't know AND doesn't have a local client to forward to, it records a `discovery_path_requests[dest_hash]` entry (`Transport.py:2949-2963`):
When a transport-enabled relay receives a path? for a destination it doesn't know AND doesn't have a local client to forward to, it records a `discovery_path_requests[dest_hash]` entry (`Transport.py:2955-2970`):
```python
pr_entry = {
@ -2867,7 +2881,7 @@ Then forwards the path? to every other interface preserving the original tag. Th
#### 12.6.2 `tunnels`
A tunnel is an interface-level path mechanism for handling temporarily-disconnected interfaces (e.g. a mobile peer that comes and goes). The `tunnels[interface_tunnel_id]` state lets the relay reconstruct paths through the interface when it reconnects, without requiring all paths to be re-discovered from scratch. The shape (`Transport.py:783-832`):
A tunnel is an interface-level path mechanism for handling temporarily-disconnected interfaces (e.g. a mobile peer that comes and goes). The `tunnels[interface_tunnel_id]` state lets the relay reconstruct paths through the interface when it reconnects, without requiring all paths to be re-discovered from scratch. The shape (`Transport.py:792-832`):
```
[ now, # 0 IDX_TT_TIMESTAMP
@ -2884,8 +2898,8 @@ Each path inside the tunnel's `paths_dict` mirrors a `path_table` entry. When th
When multiple processes on one host share a single Reticulum stack (via `share_instance = Yes` in the rnsd config), one process owns `Transport` and the others connect to it as **local clients** via a small TCP loopback interface. The shared instance treats local-client traffic specially:
- `from_local_client` and `for_local_client` are computed on every inbound packet (`Transport.py:1450-1454`).
- Path-table entries with `IDX_PT_HOPS == 0` mean "destination is a local client" — the §2.3 originator-side HEADER_1 conversion applies for hops==1 too, so the shared instance gets a transport_id-tagged packet (`Transport.py:1094-1105`).
- `from_local_client` and `for_local_client` are computed on every inbound packet (`Transport.py:1453-1456`).
- Path-table entries with `IDX_PT_HOPS == 0` mean "destination is a local client" — the §2.3 originator-side HEADER_1 conversion applies for hops==1 too, so the shared instance gets a transport_id-tagged packet (`Transport.py:1097-1108`).
- Local-client originated path? requests are forwarded to every external interface, fanning out the search across the shared mesh (§7.2 dispatch branch 3).
The wire protocol for shared-instance loopback is just the same Reticulum packets over a TCP loopback interface — no special framing or commands. What's "shared" is the path_table and announce dispatch, not the wire format.
@ -2894,15 +2908,15 @@ The wire protocol for shared-instance loopback is just the same Reticulum packet
| File | What |
|---|---|
| `RNS/Transport.py:1497-1573` | DATA forwarding (HEADER_1↔HEADER_2 conversion for relay) |
| `RNS/Transport.py:1553-1561` | `link_table` entry shape |
| `RNS/Transport.py:1567-1571` | `reverse_table` entry shape |
| `RNS/Transport.py:1810-1969` | ANNOUNCE rebroadcast queue and per-interface dispatch |
| `RNS/Transport.py:2110-2138` | LRPROOF forwarding via `link_table` |
| `RNS/Transport.py:2196-2205` | PROOF receipt forwarding via `reverse_table` |
| `RNS/Transport.py:3439-3446` | `path_table` entry shape (IDX_PT_*) |
| `RNS/Transport.py:747-832` | stale-path / stale-tunnel eviction |
| `RNS/Transport.py:2949-2963` | `discovery_path_requests` recursive search |
| `RNS/Transport.py:1500-1580` | DATA forwarding (HEADER_1↔HEADER_2 conversion for relay) |
| `RNS/Transport.py:1556-1565` | `link_table` entry shape |
| `RNS/Transport.py:1570-1574` | `reverse_table` entry shape |
| `RNS/Transport.py:1810-1975` | ANNOUNCE rebroadcast queue and per-interface dispatch |
| `RNS/Transport.py:2110-2145` | LRPROOF forwarding via `link_table` |
| `RNS/Transport.py:2199-2208` | PROOF receipt forwarding via `reverse_table` |
| `RNS/Transport.py:3457-3464` | `path_table` entry shape (IDX_PT_*) |
| `RNS/Transport.py:750-832` | stale-path / stale-tunnel eviction |
| `RNS/Transport.py:2955-2970` | `discovery_path_requests` recursive search |
| `RNS/Interfaces/Interface.py:232-272` | per-interface `announce_queue` and `ANNOUNCE_CAP` enforcement |
---
@ -2917,14 +2931,14 @@ Upstream RNS spawns the following persistent daemon threads at `Transport.start(
| Thread | Source | Cadence | Purpose |
|---|---|---|---|
| **`Transport.jobloop`** | `RNS/Transport.py:280, 483-486` | every `job_interval = 0.250s` | Runs `Transport.jobs()` — the catch-all maintenance pass: link state checks, announce-queue drain, stale-path eviction, hashlist cleanup, reverse-table cleanup, tunnels housekeeping. |
| **`Transport.count_traffic_loop`** | `RNS/Transport.py:281, 449-480` | every 1s | Snapshots per-interface RX/TX byte counters into rolling-window deques for bandwidth/airtime accounting. |
| **`Link.__watchdog_job`** | `RNS/Link.py:746-821` | per-link, RTT-driven | One per active Link. Drives keepalive emission (initiator side), STALE→CLOSED transitions, and link-establishment timeouts. Sleeps `min(WATCHDOG_MAX_SLEEP=5s, RTT-derived)` between iterations. |
| **`Resource.__watchdog_job`** | `RNS/Resource.py:564-642` | per-resource | One per in-progress Resource. Detects retransmit timeouts, advertisement retries, and PRF-wait timeouts. |
| **`AnnounceHandler` callbacks** | `RNS/Transport.py:1995-2016` | per inbound announce | Each accepted announce fires its registered handler **on a fresh daemon thread** — the dispatcher does not serialize. Two announces from the same destination back-to-back run two handler threads concurrently. |
| **`Transport.jobloop`** | `RNS/Transport.py:283, 486-489` | every `job_interval = 0.250s` | Runs `Transport.jobs()` — the catch-all maintenance pass: link state checks, announce-queue drain, stale-path eviction, hashlist cleanup, reverse-table cleanup, tunnels housekeeping. |
| **`Transport.count_traffic_loop`** | `RNS/Transport.py:284, 452-483` | every 1s | Snapshots per-interface RX/TX byte counters into rolling-window deques for bandwidth/airtime accounting. |
| **`Link.__watchdog_job`** | `RNS/Link.py:751-828` | per-link, RTT-driven | One per active Link. Drives keepalive emission (initiator side), STALE→CLOSED transitions, and link-establishment timeouts. Sleeps `min(WATCHDOG_MAX_SLEEP=5s, RTT-derived)` between iterations. |
| **`Resource.__watchdog_job`** | `RNS/Resource.py:564-670` | per-resource | One per in-progress Resource. Detects retransmit timeouts, advertisement retries, and PRF-wait timeouts. |
| **`AnnounceHandler` callbacks** | `RNS/Transport.py:1995-2025` | per inbound announce | Each accepted announce fires its registered handler **on a fresh daemon thread** — the dispatcher does not serialize. Two announces from the same destination back-to-back run two handler threads concurrently. |
| **Per-interface RX threads** | `RNS/Interfaces/*Interface.py` | always | Each interface (TCP, KISS, RNode, AutoInterface) has its own blocking-read RX thread that calls `Transport.inbound(raw, self)` on each complete frame. |
| **`process_announce_queue`** | `RNS/Interfaces/Interface.py:266-267` | one-shot timer per drain | Per-interface `announce_queue` drain uses `threading.Timer` to schedule the next emission at the airtime-cap-derived wait time. Not a long-running thread but a chain of one-shots. |
| **`Resource.__advertise_job`** | `RNS/Resource.py:520-541` | per-resource | One-shot daemon thread that performs the resource hashmap construction (which can take seconds on a large body) so the calling thread doesn't block. |
| **`Resource.__advertise_job`** | `RNS/Resource.py:520-542` | per-resource | One-shot daemon thread that performs the resource hashmap construction (which can take seconds on a large body) so the calling thread doesn't block. |
A clean-room implementation with cooperative scheduling (e.g. asyncio, embedded RTOS task model) needs to provide equivalent behavior for each row. The key invariants — not the exact thread inventory — are what matter for interop:
@ -2967,7 +2981,7 @@ What upstream **guarantees** to application-level callbacks:
- **`Link.set_link_established_callback`** — fires once when a link transitions PENDING → ACTIVE. On the receive thread.
- **`Link.set_link_closed_callback`** — fires once when a link transitions to CLOSED, regardless of cause (timeout, peer close, local teardown). On the watchdog thread or the receive thread depending on which path triggered the close.
- **`PacketReceipt.set_delivery_callback`** — fires once when a PROOF arrives matching this receipt. On the receive thread.
- **`AnnounceHandler.received_announce`** — fires once per accepted announce, **on a fresh daemon thread**. This is the only callback that's NOT on the receive thread (`Transport.py:1995-2016`).
- **`AnnounceHandler.received_announce`** — fires once per accepted announce, **on a fresh daemon thread**. This is the only callback that's NOT on the receive thread (`Transport.py:1995-2025`).
- **`Resource.callback`** — fires once on resource conclude, on the assembly thread.
Implications for a clean-room implementation:
@ -2997,15 +3011,15 @@ A client running on a constrained device (less RAM, slower CPU) can scale all of
| File | What |
|---|---|
| `RNS/Transport.py:280-281` | top-level thread spawn at startup |
| `RNS/Transport.py:128-148` | the lock inventory (Transport-side) |
| `RNS/Transport.py:172, 175, 186` | `job_interval`, `links_check_interval`, `tables_last_culled` |
| `RNS/Transport.py:483-486` | `jobloop` — the periodic driver |
| `RNS/Transport.py:489+` | `jobs()` body (held under `jobs_lock`) |
| `RNS/Transport.py:1995-2016` | announce-handler dispatch (fresh thread per callback) |
| `RNS/Link.py:746-821` | per-link `__watchdog_job` |
| `RNS/Resource.py:564-642` | per-resource `__watchdog_job` |
| `RNS/Resource.py:520-541` | one-shot `__advertise_job` |
| `RNS/Transport.py:283-284` | top-level thread spawn at startup |
| `RNS/Transport.py:132-148` | the lock inventory (Transport-side) |
| `RNS/Transport.py:174, 176, 187` | `job_interval`, `links_check_interval`, `tables_last_culled` |
| `RNS/Transport.py:486-489` | `jobloop` — the periodic driver |
| `RNS/Transport.py:492+` | `jobs()` body (held under `jobs_lock`) |
| `RNS/Transport.py:1995-2025` | announce-handler dispatch (fresh thread per callback) |
| `RNS/Link.py:751-828` | per-link `__watchdog_job` |
| `RNS/Resource.py:564-670` | per-resource `__watchdog_job` |
| `RNS/Resource.py:520-542` | one-shot `__advertise_job` |
| `RNS/Interfaces/*Interface.py` | per-interface RX thread |
---
@ -3209,7 +3223,7 @@ Embedded clean-room implementations need to know up front which data structures
| Structure | Cap | Where | Notes |
|---|---|---|---|
| `Transport.path_table` | (unbounded — count grows with mesh size) | §12.4 | Grows with the number of distinct destinations the node has heard about. Bounded effectively by TTL eviction (§12.4.2): AP_PATH_TIME (1h), ROAMING_PATH_TIME (4h), PATHFINDER_E (30d). On a tiny LoRa mesh this is dozens of entries; on a global Reticulum mesh routed through a TCP backbone it can be thousands. |
| `path_table[dest][IDX_PT_RANDBLOBS]` (per-destination random_blob history) | `Transport.MAX_RANDOM_BLOBS = 32` | §4.5 step 6.3, §12.3.2 | Sliding window. Caps memory growth from one destination's announce stream. |
| `path_table[dest][IDX_PT_RANDBLOBS]` (per-destination random_blob history) | `Transport.MAX_RANDOM_BLOBS = 64` (RNS 1.2.4) | §4.5 step 6.3, §12.3.2 | Sliding window. Caps memory growth from one destination's announce stream. |
| `Transport.announce_table` | (unbounded — populated only for in-flight announces awaiting rebroadcast) | §12.3 | Drains via `Transport.jobs` retransmit timer, capped at `PATHFINDER_R = 4` retries each. Effective cap: number of announces seen × time. |
| `Transport.discovery_pr_tags` | `Transport.max_pr_tags = 32000` | §7.2.2 | Path-request dedup table. Older entries aged out by `Transport.jobs`. |
| `Transport.path_requests` | (unbounded — one entry per recently-issued path? request) | §7.1 | Aged out at `Transport.PATH_REQUEST_GATE_TIMEOUT = 120s`. |
@ -3327,7 +3341,7 @@ A reader hitting §2.3 might wonder "do I need this?" Three different answers:
- **Cat 1 (e.g. MeshChat, Sideband):** No — `RNS.Transport.outbound` at lines 1074-1083 does the conversion automatically when you call `Packet.send()` to a destination with `path_table[dest][HOPS] > 1`. Your app just calls `LXMessage.send()` or `Packet.send()` and §2.3 happens invisibly. You can read §2.3 to understand WHY some captures show HEADER_2 with a transport_id, but you have no code to write.
- **Cat 2 (wrappers):** Same as Cat 1 — the wrapped Python RNS does the conversion. Your wrapper is just relaying API calls.
- **Cat 3 (clean-room):** **Yes, you implement §2.3 yourself.** Failure to do so means your packets aren't forwarded by transit relays — they're processed and dropped silently per `Transport.py:1497` (only HEADER_2 packets with `transport_id == relay.identity.hash` enter the forwarding branch). The symptom is "messages I send through a relay never arrive, but direct-link messages do." Sideband works in shared-instance and direct-TCP modes both because **upstream** does the conversion; a clean-room app working only via shared-instance is masking the missing §2.3.
- **Cat 3 (clean-room):** **Yes, you implement §2.3 yourself.** Failure to do so means your packets aren't forwarded by transit relays — they're processed and dropped silently per `Transport.py:1500` (only HEADER_2 packets with `transport_id == relay.identity.hash` enter the forwarding branch). The symptom is "messages I send through a relay never arrive, but direct-link messages do." Sideband works in shared-instance and direct-TCP modes both because **upstream** does the conversion; a clean-room app working only via shared-instance is masking the missing §2.3.
### 17.4 Pragmatic implication

2
agent.md
View file

@ -100,7 +100,7 @@ Initial confidence assessment (subjective, not authoritative — re-do this audi
| §7.6 `TCPServerInterface.OUT` override | Source-cited; matches behavior observed in the mobile-app's local-transport experiments. |
| §8 KISS / HDLC framing | High — both work in production on the reference clients |
| §9.1–§9.8 Implementation gotchas | Each was a real bug that bit a real implementation. High confidence each is real; some lack formal test scripts. |
| §10 Resource fragmentation | Source-cited from `RNS/Resource.py` against RNS 1.2.0; not yet runtime-verified in this repo's `tools/`. |
| §10 Resource fragmentation | Source-cited from `RNS/Resource.py` against RNS 1.2.4; not yet runtime-verified in this repo's `tools/`. |
| §11 Test vectors | The vectors themselves are verified; the test-vectors/ directory needs to be populated in this repo (currently partially populated). |
| §12 Source map | High |

6
flows/forward-announce.md
View file

@ -1,6 +1,6 @@
# Flow: forward an announce (transport-node rebroadcast)
What a transport-mode node does when it receives an inbound announce destined for a non-local destination. This is the flow that makes the mesh actually mesh — without it, announces never propagate beyond direct radio range. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §4.5 (validation), §12.3 (rebroadcast rules), §12.4 (path table).
What a transport-mode node does when it receives an inbound announce destined for a non-local destination. This is the flow that makes the mesh actually mesh — without it, announces never propagate beyond direct radio range. Pinned against **RNS 1.2.4**; cross-references [`../SPEC.md`](../SPEC.md) §4.5 (validation), §12.3 (rebroadcast rules), §12.4 (path table).
This flow only runs on a node with `enable_transport = Yes` per §12.1. Leaf clients can ignore it entirely.
@ -14,7 +14,7 @@ The receive-announce flow ([`receive-announce.md`](receive-announce.md)) runs fi
### 2. Eligibility checks
`RNS/Transport.py:1822`. Three conditions all must hold:
`RNS/Transport.py:1825`. Three conditions all must hold:
```python
if (Reticulum.transport_enabled() or is_from_local_client) \
@ -72,7 +72,7 @@ The actual code is in `Transport.py:1196-1300, 1810-1969`; the structure above i
### 5. Per-interface `announce_queue` drain
Each interface independently throttles its outbound announces against `interface.announce_cap` (default `Reticulum.ANNOUNCE_CAP = 2.0` = 2% airtime). `Interface.process_announce_queue` (`RNS/Interfaces/Interface.py:232-272`) drains the queue at a rate the cap permits, **picking the lowest-hop-count entry first** so closer destinations propagate before further ones:
Each interface independently throttles its outbound announces against `interface.announce_cap` (default `Reticulum.ANNOUNCE_CAP = 2.0` = 2% airtime). `Interface.process_announce_queue` (`RNS/Interfaces/Interface.py:237-272`) drains the queue at a rate the cap permits, **picking the lowest-hop-count entry first** so closer destinations propagate before further ones:
```python
min_hops = min(e["hops"] for e in self.announce_queue)

6
flows/path-discovery.md
View file

@ -2,7 +2,7 @@
What happens chronologically when one node wants to reach another whose path is missing from `Transport.path_table`. This flow drives the well-known `rnstransport.path.request` destination (dest_hash `6b9f66014d9853faab220fba47d02761`, see [`../SPEC.md`](../SPEC.md) §1.2 and §7.1) and the path-response announce that returns along the reverse path.
Pinned against **RNS 1.2.0**.
Pinned against **RNS 1.2.4**.
Out of scope: the LXMF outbound retry that gates this flow (`flows/send-opportunistic-lxmf.md` step 4); the periodic `Transport.jobs` cycle that ages out stale paths; transport-mode rebroadcast of a known path on behalf of a remote requester (Tier 3 spec gap, see `../todo.md`).
@ -20,7 +20,7 @@ Out of scope: the LXMF outbound retry that gates this flow (`flows/send-opportun
### 1. Initiator: `RNS.Transport.request_path(dest_hash)`
`RNS/Transport.py:2707-2745`. Triggered by:
`RNS/Transport.py:2713-2750`. Triggered by:
- `LXMRouter.handle_outbound` when `not has_path(dest) and method == OPPORTUNISTIC` (see `send-opportunistic-lxmf.md` step 4).
- `LXMRouter.process_outbound` retry path after `MAX_PATHLESS_TRIES`.
@ -45,7 +45,7 @@ Transport.path_requests[destination_hash] = time.time()
The well-known dest hash `6b9f66014d9853faab220fba47d02761` is computed as `SHA256(SHA256("rnstransport.path.request")[:10])[:16]` per [`../SPEC.md`](../SPEC.md) §1.2 (the `identity=None` branch of `Destination.hash`). It is the same on every node — no per-node uniqueness — because the destination is `PLAIN` with no identity attached.
The packet is `DATA` (not ANNOUNCE), `BROADCAST` transport, `HEADER_1`, and `context = NONE`. The body is **unencrypted** because the outer destination is `PLAIN` (`RNS/Packet.py:192-194` skips encryption for PLAIN-type destinations).
The packet is `DATA` (not ANNOUNCE), `BROADCAST` transport, `HEADER_1`, and `context = NONE`. The body is **unencrypted** because the outer destination is `PLAIN` — `Packet.pack` falls through to `self.destination.encrypt(self.data)` at `RNS/Packet.py:215`, and `Destination.encrypt` returns `plaintext` unchanged for `Destination.PLAIN` (`RNS/Destination.py:592-593`).
Initiator records `Transport.path_requests[destination_hash] = time.time()` for `PATH_REQUEST_GATE_TIMEOUT = 120s` rate-limiting (prevents the same node from repeating identical path? requests faster than `PATH_REQUEST_MI = 20s`).

24
flows/receive-announce.md
View file

@ -2,7 +2,7 @@
What happens chronologically on a node when an `ANNOUNCE` packet arrives on one of its interfaces. Without this flow working, nothing else does — no peer is ever known, every outbound message fails at `Identity.recall(dest_hash)`, and no path table entry exists for routing.
Pinned against **RNS 1.2.0**. Line numbers below are from that version.
Pinned against **RNS 1.2.4**. Line numbers below are from that version.
This flow covers the **receive + ingest** path that every Reticulum node runs (leaf clients and transport nodes alike). The **rebroadcast** logic that transport nodes additionally run is covered separately in `forward-announce.md` (TODO — Tier 3 in `../todo.md`).
@ -22,12 +22,12 @@ This flow covers the **receive + ingest** path that every Reticulum node runs (l
Steps 1-6 of [`receive-opportunistic-lxmf.md`](receive-opportunistic-lxmf.md) apply unchanged:
1. KISS / HDLC deframer (and RNode 2-frame split-packet reassembly per [`../SPEC.md`](../SPEC.md) §8.3 if applicable) hands the raw Reticulum packet bytes to `RNS.Transport.inbound(raw, interface)` (`RNS/Transport.py:1327`).
1. KISS / HDLC deframer (and RNode 2-frame split-packet reassembly per [`../SPEC.md`](../SPEC.md) §8.3 if applicable) hands the raw Reticulum packet bytes to `RNS.Transport.inbound(raw, interface)` (`RNS/Transport.py:1330`).
2. IFAC unmask if the interface has `ifac_identity` configured.
3. `packet = RNS.Packet(None, raw); packet.unpack(); packet.hops += 1`.
4. RSSI / SNR / Q stats are attached to the packet.
5. `Transport.packet_filter(packet)` dedup check; the packet's hash is added to `Transport.packet_hashlist` unless it belongs to a link or is an LRPROOF.
6. Dispatch by `packet.packet_type`. For an announce: `packet_type == ANNOUNCE (1)`, `destination_type == SINGLE (0)`, `transport_type == BROADCAST (0)`. Control reaches `RNS/Transport.py:1628`:
6. Dispatch by `packet.packet_type`. For an announce: `packet_type == ANNOUNCE (1)`, `destination_type == SINGLE (0)`, `transport_type == BROADCAST (0)`. Control reaches `RNS/Transport.py:1631`:
```python
if packet.packet_type == RNS.Packet.ANNOUNCE:
@ -36,7 +36,7 @@ if packet.packet_type == RNS.Packet.ANNOUNCE:
### 2. Quick signature-only validation (cheap pre-filter)
`RNS/Transport.py:1629`:
`RNS/Transport.py:1632`:
```python
if interface != None and RNS.Identity.validate_announce(packet, only_validate_signature=True):
@ -52,7 +52,7 @@ If the signature fails, the announce is silently dropped here without the deque
### 3. Ingress rate limiting (defer-and-hold)
`RNS/Transport.py:1632-1643`:
`RNS/Transport.py:1635-1646`:
```python
if not packet.destination_hash in Transport.path_table:
@ -74,7 +74,7 @@ Held announces are released later by `Interface.process_held_announces()` (`RNS/
### 4. Local-destination short-circuit
`RNS/Transport.py:1645-1648`:
`RNS/Transport.py:1648-1651`:
```python
local_destination = None
@ -87,7 +87,7 @@ If the announce's `destination_hash` matches one of our own registered destinati
### 5. Full announce validation
`RNS/Transport.py:1650`:
`RNS/Transport.py:1653`:
```python
if local_destination == None and RNS.Identity.validate_announce(packet):
@ -108,7 +108,7 @@ If any check fails, the function returns `False` and the wider dispatch skips ev
### 6. `random_blob` extraction and `received_from` resolution
`RNS/Transport.py:1651-1677`. The 10-byte `random_hash` is re-extracted as `random_blob` (same bytes, different name in the routing-table code):
`RNS/Transport.py:1654-1680`. The 10-byte `random_hash` is re-extracted as `random_blob` (same bytes, different name in the routing-table code):
```python
random_blob = packet.data[64+10 : 64+10+10] # bytes 74..84 of the announce body
@ -120,7 +120,7 @@ The trailing 5 bytes of `random_blob` are the emission timestamp (SPEC.md §4.1)
### 7. Path table population
`RNS/Transport.py:1679-1969`. The full decision tree for whether and how to update `path_table[destination_hash]` is roughly:
`RNS/Transport.py:1682-1975`. The full decision tree for whether and how to update `path_table[destination_hash]` is roughly:
```
local_and_hops_condition := (packet.hops < PATHFINDER_M+1) # default 128
@ -160,7 +160,7 @@ A leaf client that **doesn't** maintain a path table can still send opportunisti
### 8. Announce handler dispatch
`RNS/Transport.py:1970-2024`. With path-table updated, the receiver fans the announce out to any application-level listeners that registered via `RNS.Transport.register_announce_handler`:
`RNS/Transport.py:1976-2030`. With path-table updated, the receiver fans the announce out to any application-level listeners that registered via `RNS.Transport.register_announce_handler`:
```python
for handler in Transport.announce_handlers:
@ -183,7 +183,7 @@ Three details that bite implementers:
1. **`aspect_filter` is a name string, not a name_hash.** A handler that wants only `lxmf.delivery` announces sets `self.aspect_filter = "lxmf.delivery"` and Reticulum recomputes the expected dest_hash from `(name, announced_identity)` for each candidate. So a single handler can match across all peers' `lxmf.delivery` destinations without enumerating them.
2. **`PATH_RESPONSE` is filtered OUT by default.** A handler must opt in via `handler.receive_path_responses = True` to see path-response announces; otherwise it sees only periodic re-announces. The path-table update in step 7 still happens regardless.
3. **The handler is called in a fresh daemon thread.** A slow handler doesn't block subsequent inbound packets, but it also can't assume serialised execution — two announces from the same destination arriving back-to-back will run two handler threads concurrently. Handlers that mutate shared state need their own locks. (`RNS/Transport.py:1995-2016`.)
3. **The handler is called in a fresh daemon thread.** A slow handler doesn't block subsequent inbound packets, but it also can't assume serialised execution — two announces from the same destination arriving back-to-back will run two handler threads concurrently. Handlers that mutate shared state need their own locks. (`RNS/Transport.py:1995-2025`.)
LXMF registers two handlers at `LXMF/LXMRouter.py:207-208`:
@ -198,7 +198,7 @@ so any client built on the LXMF router gets contacts/propagation-node discovery
If `RNS.Reticulum.transport_enabled() == True`, the same `Transport.inbound` dispatch then walks an additional code path that constructs a rebroadcast announce, queues it on each suitable interface (`announce_queue`), and emits it later subject to the per-interface `announce_cap` (default 2% of airtime). This entire path is skipped on a leaf client.
The rebroadcast logic is the bulk of `RNS/Transport.py:1810-1969` and is documented separately in `forward-announce.md` (TODO).
The rebroadcast logic is the bulk of `RNS/Transport.py:1810-1975` and is documented separately in `forward-announce.md` (TODO).
---

12
flows/receive-link-lxmf.md
View file

@ -1,6 +1,6 @@
# Flow: receive an LXMF message over a Reticulum Link
The inverse of [`send-link-lxmf.md`](send-link-lxmf.md), covering both halves of the responder side: accepting the inbound LINKREQUEST, sending the LRPROOF, then handling LXMF DATA on the established link. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §6 (Link), §6.5 (PROOF), §6.6 (signalling), §6.7 (KEEPALIVE/teardown), §10 (Resource).
The inverse of [`send-link-lxmf.md`](send-link-lxmf.md), covering both halves of the responder side: accepting the inbound LINKREQUEST, sending the LRPROOF, then handling LXMF DATA on the established link. Pinned against **RNS 1.2.4 / LXMF 0.9.7**; cross-references [`../SPEC.md`](../SPEC.md) §6 (Link), §6.5 (PROOF), §6.6 (signalling), §6.7 (KEEPALIVE/teardown), §10 (Resource).
---
@ -14,7 +14,7 @@ A Reticulum DATA packet with `packet_type = LINKREQUEST (2)`, addressed to the r
initiator_X25519_pub(32) || initiator_Ed25519_pub(32) || [signalling(3)]
```
`Transport.inbound` (`RNS/Transport.py:2027-2057`) recognizes `packet_type == LINKREQUEST + destination_type == SINGLE`, looks up the destination in `destinations_map`, and calls `Destination.receive(packet)` which routes to `Destination.incoming_link_request(data, packet)` per `RNS/Destination.py:403-406`:
`Transport.inbound` (`RNS/Transport.py:2030-2060`) recognizes `packet_type == LINKREQUEST + destination_type == SINGLE`, looks up the destination in `destinations_map`, and calls `Destination.receive(packet)` which routes to `Destination.incoming_link_request(data, packet)` per `RNS/Destination.py:403-450` (`receive` at 403, dispatches to `incoming_link_request` at 420):
```python
def receive(self, packet):
@ -24,7 +24,7 @@ def receive(self, packet):
### 2. Responder builds Link state via `Link.validate_request`
`RNS/Link.py:186-227`. Length-checks the body (`ECPUBSIZE` or `ECPUBSIZE + LINK_MTU_SIZE`), rejects otherwise. On success:
`RNS/Link.py:186-230`. Length-checks the body (`ECPUBSIZE` or `ECPUBSIZE + LINK_MTU_SIZE`), rejects otherwise. On success:
1. Build a `Link` object with `peer_pub_bytes = data[:32]` and `peer_sig_pub_bytes = data[32:64]`.
2. `set_link_id(packet)` per §6.3 — the link_id derives from `Packet.get_hashable_part`, invariant under HEADER_1↔HEADER_2 conversion.
@ -47,7 +47,7 @@ where `signature = sign(link_id || responder_X25519_pub || responder_long_term_E
After the initiator validates the LRPROOF and sends `Link.LRRTT (0xFE)` carrying its measured RTT, the responder receives it at `Link.receive` line 1056-1059 and calls `rtt_packet`. The responder's RTT cache updates and `Link.STATUS = ACTIVE` triggers the `link_established_callback` registered via `Destination.set_link_established_callback`.
For LXMF, that callback is `LXMRouter.delivery_link_established` (`LXMF/LXMRouter.py:1849-1856`):
For LXMF, that callback is `LXMRouter.delivery_link_established` (`LXMF/LXMRouter.py:1852-1858`):
```python
link.track_phy_stats(True)
@ -63,7 +63,7 @@ This is what makes inbound DATA on this link route into LXMF processing.
### 5. Inbound LXMF DATA — single-packet (PACKET representation)
A regular DATA packet on the link (`context = NONE`, Token-encrypted with link session key per §3.1). `Link.receive` decrypts and passes the plaintext to `delivery_packet(data, packet)` (`LXMF/LXMRouter.py:1819-1847`) — the same handler used by opportunistic delivery. Differences:
A regular DATA packet on the link (`context = NONE`, Token-encrypted with link session key per §3.1). `Link.receive` decrypts and passes the plaintext to `delivery_packet(data, packet)` (`LXMF/LXMRouter.py:1822-1850`) — the same handler used by opportunistic delivery. Differences:
- `packet.destination_type == LINK` so `method = DIRECT`.
- The LXMF body arrives **with** the recipient's dest_hash (§5.2), so no re-prepend like the opportunistic path does at step 9 of `receive-opportunistic-lxmf.md`.
@ -72,7 +72,7 @@ The handler calls `packet.prove()` immediately (mandatory PROOF receipt per §6.
### 6. Inbound LXMF DATA — Resource representation
A larger LXMF body arrives as a Resource transfer per `flows/receive-resource.md`. The Link's `resource_strategy = ACCEPT_APP` triggers `delivery_resource_advertised(resource)` (`LXMF/LXMRouter.py:1864-1871`):
A larger LXMF body arrives as a Resource transfer per `flows/receive-resource.md`. The Link's `resource_strategy = ACCEPT_APP` triggers `delivery_resource_advertised(resource)` (`LXMF/LXMRouter.py:1867-1874`):
```python
def delivery_resource_advertised(self, resource):

34
flows/receive-opportunistic-lxmf.md
View file

@ -2,7 +2,7 @@
The inverse of [`send-opportunistic-lxmf.md`](send-opportunistic-lxmf.md). What happens chronologically on the recipient when wire bytes for an opportunistic LXMF DATA packet arrive at one of its interfaces.
Pinned against **RNS 1.2.0 / LXMF 0.9.6**. Line numbers below are from those versions.
Pinned against **RNS 1.2.4 / LXMF 0.9.7**. Line numbers below are from those versions.
Out of scope: receiving a packet over an established Reticulum Link (DIRECT method), receiving propagated messages from a propagation node, and receiving an announce / path-request / link-request. Each gets its own flow document.
@ -10,7 +10,7 @@ Out of scope: receiving a packet over an established Reticulum Link (DIRECT meth
## Preconditions
- Recipient has an `RNS.Identity` with the X25519 + Ed25519 private keys, plus a `lxmf.delivery` `RNS.Destination` registered with `LXMRouter.register_delivery_identity` — that registration calls `delivery_destination.set_packet_callback(self.delivery_packet)` at `LXMF/LXMRouter.py:341`, which is the hand-off point in step 7 below.
- Recipient has an `RNS.Identity` with the X25519 + Ed25519 private keys, plus a `lxmf.delivery` `RNS.Destination` registered with `LXMRouter.register_delivery_identity` — that registration calls `delivery_destination.set_packet_callback(self.delivery_packet)` at `LXMF/LXMRouter.py:340`, which is the hand-off point in step 7 below.
- Recipient has, at some point, been the target of one or more announces from the sender, so `RNS.Identity.known_destinations` knows the sender's full `public_key` (X25519 || Ed25519) under their `dest_hash`. Without this, signature validation in step 11 will fail with `unverified_reason = SOURCE_UNKNOWN`.
---
@ -25,17 +25,17 @@ For RNode KISS specifically, `CMD_STAT_RSSI = 0x23` and `CMD_STAT_SNR = 0x24` si
### 2. `Transport.inbound(raw, interface)` entry point
`RNS/Transport.py:1327`. The single entry point for any inbound packet on any interface. The function is gated by `Transport.ready` — packets arriving before transport startup are dropped with a warning.
`RNS/Transport.py:1330`. The single entry point for any inbound packet on any interface. The function is gated by `Transport.ready` — packets arriving before transport startup are dropped with a warning.
### 3. IFAC unmask (Interface Authentication Codes)
`RNS/Transport.py:1338-1387`. If the interface has an `ifac_identity` configured, the high bit of `raw[0]` must be set; the IFAC bytes at `raw[2:2+ifac_size]` are then used to derive an HKDF mask, the rest of the packet is unmasked in place, and the IFAC is verified against `ifac_identity.sign(unmasked_raw)[-ifac_size:]`. Mismatch drops the packet silently.
`RNS/Transport.py:1338-1390`. If the interface has an `ifac_identity` configured, the high bit of `raw[0]` must be set; the IFAC bytes at `raw[2:2+ifac_size]` are then used to derive an HKDF mask, the rest of the packet is unmasked in place, and the IFAC is verified against `ifac_identity.sign(unmasked_raw)[-ifac_size:]`. Mismatch drops the packet silently.
If the interface has no IFAC and the high bit IS set, the packet is dropped (an unexpected IFAC).
### 4. Packet parse and physical-layer stats
`RNS/Transport.py:1391-1395`:
`RNS/Transport.py:1394-1398`:
```python
packet = RNS.Packet(None, raw)
@ -46,20 +46,20 @@ packet.hops += 1
`packet.unpack` reads the header byte fields per SPEC.md §2.1, sets `packet.header_type`, `packet.packet_type`, `packet.destination_type`, `packet.destination_hash`, `packet.context`, and slices `packet.data` from the remainder. Importantly, **hops is incremented by 1 here**, so even on a leaf-endpoint receive the local `packet.hops` is one more than what flew on the wire — flow logic that treats `packet.hops == 0` as "originator on this interface" must use the wire byte before this increment.
RSSI / SNR / Q link-quality stats are attached to `packet` if the interface exposed them (`RNS/Transport.py:1397-1417`).
RSSI / SNR / Q link-quality stats are attached to `packet` if the interface exposed them (`RNS/Transport.py:1400-1420`).
### 5. Hop fix-up for shared-instance and local-client interfaces
`RNS/Transport.py:1419-1422`. If the receiving interface is to a local shared instance or a local-client TCP socket, the +1 increment from step 4 is undone — the shared-instance path doesn't count as a real network hop.
`RNS/Transport.py:1422-1425`. If the receiving interface is to a local shared instance or a local-client TCP socket, the +1 increment from step 4 is undone — the shared-instance path doesn't count as a real network hop.
### 6. Dedup, then dispatch by packet_type / destination_type
`RNS/Transport.py:1424-1444`. `Transport.packet_filter` checks `packet.packet_hash` against `Transport.packet_hashlist` to drop replays. Hashes are added to the dedup list except for two cases that must be deferred:
`RNS/Transport.py:1427-1447`. `Transport.packet_filter` checks `packet.packet_hash` against `Transport.packet_hashlist` to drop replays. Hashes are added to the dedup list except for two cases that must be deferred:
- packet whose `destination_hash` is in `Transport.link_table` — the dedup decision is left to the link itself,
- LRPROOF packets — these may legitimately arrive on multiple interfaces during routing-fork chaos and the dedup list is updated only after the LRPROOF is validated.
Then the function fans out by `(packet_type, destination_type)`. For an opportunistic LXMF DATA packet — `packet_type == DATA`, `destination_type == SINGLE` — control reaches `RNS/Transport.py:2087-2103`:
Then the function fans out by `(packet_type, destination_type)`. For an opportunistic LXMF DATA packet — `packet_type == DATA`, `destination_type == SINGLE` — control reaches `RNS/Transport.py:2090-2106`:
```python
destination = Transport.destinations_map.get(packet.destination_hash)
@ -78,7 +78,7 @@ If `destination_hash` does not match any locally-registered destination, this br
### 7. `Destination.receive(packet)` — decrypt and run packet callback
`RNS/Destination.py:403-418`:
`RNS/Destination.py:403-450`:
```python
def receive(self, packet):
@ -93,11 +93,11 @@ def receive(self, packet):
return True
```
For `lxmf.delivery` destinations, `self.callbacks.packet` was set at `LXMF/LXMRouter.py:341` to the router's `delivery_packet` — see step 9.
For `lxmf.delivery` destinations, `self.callbacks.packet` was set at `LXMF/LXMRouter.py:340` to the router's `delivery_packet` — see step 9.
### 8. `Destination.decrypt` → `Identity.decrypt` — Token decode with ratchet ring
`RNS/Destination.py:611-643` → `RNS/Identity.py:818-872`. The packet body is the Token form from SPEC.md §3.1: `ephemeral_pub(32) || iv(16) || aes_ciphertext || hmac_sha256(32)`.
`RNS/Destination.py:611-645` → `RNS/Identity.py:849-905`. The packet body is the Token form from SPEC.md §3.1: `ephemeral_pub(32) || iv(16) || aes_ciphertext || hmac_sha256(32)`.
```python
peer_pub_bytes = ciphertext_token[:32] # sender's ephemeral X25519 pub
@ -123,7 +123,7 @@ If the destination has ratchets enabled but on-disk state has gone stale, `Desti
### 9. `LXMRouter.delivery_packet(data, packet)` — proof first, then async parse
`LXMF/LXMRouter.py:1819-1847`:
`LXMF/LXMRouter.py:1822-1850`:
```python
def delivery_packet(self, data, packet):
@ -144,7 +144,7 @@ The opportunistic-form re-prepends `packet.destination.hash` because step 6 of t
### 10. `LXMessage.unpack_from_bytes(lxmf_data)` — body parse and signature validation
`LXMF/LXMessage.py:736-807`. Field slicing:
`LXMF/LXMessage.py:736-810`. Field slicing:
```
destination_hash = lxmf_data[ 0:16]
@ -163,11 +163,11 @@ signed_part = hashed_part || message_hash
Signature validation calls `source.identity.validate(signature, signed_part)`. The sender's identity is recalled from `RNS.Identity.known_destinations` via `RNS.Identity.recall(source_hash)` at line 765 — keyed by the sender's **destination hash**, not their identity hash, per SPEC.md §5.4 and §9.1. If the sender is unknown locally (no announce ever received), `unverified_reason = SOURCE_UNKNOWN` and `signature_validated = False`; the message is still surfaced to the app callback in step 12, but downstream UI should mark it untrusted.
Note: `unpack_from_bytes` does the stamp-strip-and-reencode variant but does **not** also try the as-received `packed_payload` if validation fails. SPEC.md §5.6 documents both raw and stripped-reencoded as valid receiver behavior; upstream LXMF 0.9.6 only does the stripped form (or the raw form when no stamp was present). The spec's stronger receiver tolerance is a recommendation for non-upstream implementers, not a description of upstream.
Note: `unpack_from_bytes` does the stamp-strip-and-reencode variant but does **not** also try the as-received `packed_payload` if validation fails. SPEC.md §5.6 documents both raw and stripped-reencoded as valid receiver behavior; upstream LXMF 0.9.7 only does the stripped form (or the raw form when no stamp was present). The spec's stronger receiver tolerance is a recommendation for non-upstream implementers, not a description of upstream.
### 11. Stamp / ticket / dedup checks
`LXMF/LXMRouter.py:1741-1803`:
`LXMF/LXMRouter.py:1741-1810`:
- **Ticket** (`message.fields[FIELD_TICKET]`): if present and non-expired, cached for outbound use.
- **Stamp**: if the local destination has a `stamp_cost`, `message.validate_stamp(required_cost, tickets=...)` runs; a missing/invalid stamp causes the message to be dropped when `_enforce_stamps` is true.
@ -177,7 +177,7 @@ Note: `unpack_from_bytes` does the stamp-strip-and-reencode variant but does **n
### 12. `__delivery_callback` fires — message reaches the app
`LXMF/LXMRouter.py:1805-1812`. The router's caller (Sideband, NomadNet, MeshChat, …) sets `__delivery_callback` via `register_delivery_callback(...)`. It receives the validated `LXMessage` object and decides what to show in the inbox.
`LXMF/LXMRouter.py:1812-1820`. The router's caller (Sideband, NomadNet, MeshChat, …) sets `__delivery_callback` via `register_delivery_callback(...)`. It receives the validated `LXMessage` object and decides what to show in the inbox.
Important: the recipient's app should apply the SPEC.md §9.6 clockless-sender heuristic at this point — `if message.timestamp < 1577836800: message.timestamp = local_now()` — to keep clockless LoRa devices out of January 1970 in the inbox. Upstream `LXMessage.unpack_from_bytes` does **not** do this fix-up.

10
flows/receive-propagated-lxmf.md
View file

@ -1,6 +1,6 @@
# Flow: receive a propagated LXMF message (recipient pulls via `/get`)
The closing half of [`send-propagated-lxmf.md`](send-propagated-lxmf.md): how a recipient client retrieves messages that were store-and-forwarded for it by a propagation node. Pinned against **RNS 1.2.0 / LXMF 0.9.6**; cross-references [`../SPEC.md`](../SPEC.md) §5.8 (propagation protocol), §11 (REQUEST/RESPONSE).
The closing half of [`send-propagated-lxmf.md`](send-propagated-lxmf.md): how a recipient client retrieves messages that were store-and-forwarded for it by a propagation node. Pinned against **RNS 1.2.4 / LXMF 0.9.7**; cross-references [`../SPEC.md`](../SPEC.md) §5.8 (propagation protocol), §11 (REQUEST/RESPONSE).
This is the inverse-side flow that turns "the message was queued at a propagation node" (`send-propagated-lxmf.md` step 9) into "the message arrives in the recipient's inbox".
@ -17,7 +17,7 @@ This is the inverse-side flow that turns "the message was queued at a propagatio
### 1. Recipient initiates retrieval
`LXMRouter.request_messages_from_propagation_node(identity, max_messages)` (`LXMF/LXMRouter.py:485+`). Triggered by:
`LXMRouter.request_messages_from_propagation_node(identity, max_messages)` (`LXMF/LXMRouter.py:484+`). Triggered by:
- Manual user action (Sideband "Refresh inbox" button).
- Periodic background poll (every few minutes by default in long-running clients).
@ -34,7 +34,7 @@ self.outbound_propagation_link = RNS.Link(
)
```
(`LXMF/LXMRouter.py:514`). Standard Link establishment per `flows/send-link-lxmf.md` steps 3-4.
(`LXMF/LXMRouter.py:513`). Standard Link establishment per `flows/send-link-lxmf.md` steps 3-4.
### 3. Identify on the link
@ -47,7 +47,7 @@ data = [None, None] # [wanted, have
link.request("/get", data, response_callback=on_message_list)
```
The propagation node's `message_get_request` handler at `LXMF/LXMRouter.py:1427-1450` walks `propagation_entries` for messages keyed to the requester's destination_hash and returns:
The propagation node's `message_get_request` handler at `LXMF/LXMRouter.py:1426-1450` walks `propagation_entries` for messages keyed to the requester's destination_hash and returns:
```python
[ [transient_id_1(16), size_1(int)],
@ -86,7 +86,7 @@ Returns this as a §11 RESPONSE. If the bundle fits in `link.mdu` it's a single
### 7. Recipient unpacks the bundle and processes each message
The recipient's `propagation_resource_concluded` handler (or its single-packet equivalent) at `LXMF/LXMRouter.py:2194+` walks the bundle:
The recipient's `propagation_resource_concluded` handler (or its single-packet equivalent) at `LXMF/LXMRouter.py:2200+` walks the bundle:
```python
data = msgpack.unpackb(resource.data.read())

4
flows/receive-resource.md
View file

@ -1,6 +1,6 @@
# Flow: receive a Resource (large body) over a Link
The inverse of [`send-resource.md`](send-resource.md). What happens chronologically on the receiver when an inbound Resource transfer arrives. Pinned against **RNS 1.2.0**; see [`../SPEC.md`](../SPEC.md) §10 for the wire bytes.
The inverse of [`send-resource.md`](send-resource.md). What happens chronologically on the receiver when an inbound Resource transfer arrives. Pinned against **RNS 1.2.4**; see [`../SPEC.md`](../SPEC.md) §10 for the wire bytes.
---
@ -29,7 +29,7 @@ Branch by `resource_strategy`:
### 3. Receiver issues the first RESOURCE_REQ
`Resource.request_next()` (`RNS/Resource.py:934-983`) builds the request body per §10.5:
`Resource.request_next()` (`RNS/Resource.py:931-981`) builds the request body per §10.5:
```
exhausted_flag(1) [|| last_map_hash(4)] || resource_hash(32) || requested_map_hashes(N × 4)

6
flows/send-announce.md
View file

@ -1,6 +1,6 @@
# Flow: send an announce
What happens chronologically when a node emits an announce — the periodic broadcast that lets the rest of the mesh discover or refresh a path to this destination. Pinned against **RNS 1.2.0**.
What happens chronologically when a node emits an announce — the periodic broadcast that lets the rest of the mesh discover or refresh a path to this destination. Pinned against **RNS 1.2.4**.
Out of scope: the relay-side rebroadcast (`forward-announce.md` — see [`../SPEC.md`](../SPEC.md) §12.3) and path-response announces (already covered in [`path-discovery.md`](path-discovery.md)).
@ -85,11 +85,11 @@ Wire form per §4.1:
- `context = NONE (0x00)` for periodic re-announces, `PATH_RESPONSE (0x0B)` for path-response announces
- `context_flag = 1` if ratchet present (signals the optional ratchet_pub slot in the body)
Announce packets are NOT encrypted — `Packet.pack` (`RNS/Packet.py:189-191`) special-cases ANNOUNCE to skip encryption. The body is signed but plaintext, so anyone in earshot can validate the signature and decode the public key.
Announce packets are NOT encrypted — `Packet.pack` (`RNS/Packet.py:188-191`) special-cases ANNOUNCE to skip encryption. The body is signed but plaintext, so anyone in earshot can validate the signature and decode the public key.
### 8. `Transport.outbound` broadcasts on every OUT interface
Same broadcast branch as a path? request (`flows/path-discovery.md` step 2) — the dest_hash isn't in `path_table` (it's our own destination, not a remote one), so the broadcast branch at `RNS/Transport.py:1119+` fires, emitting on every interface where `interface.OUT == True`. Per §7.5 the announce is rate-limited by `ANNOUNCE_CAP = 2.0` (2% airtime) on each interface.
Same broadcast branch as a path? request (`flows/path-discovery.md` step 2) — the dest_hash isn't in `path_table` (it's our own destination, not a remote one), so the broadcast branch at `RNS/Transport.py:1122+` fires, emitting on every interface where `interface.OUT == True`. Per §7.5 the announce is rate-limited by `ANNOUNCE_CAP = 2.0` (2% airtime) on each interface.
### 9. Periodic re-announce loop

22
flows/send-link-lxmf.md
View file

@ -2,7 +2,7 @@
What happens chronologically when an app calls `LXMRouter.handle_outbound(lxm)` for an `LXMessage` whose `desired_method == DIRECT` (or whose payload exceeds the opportunistic single-packet content limit and is downgraded from `OPPORTUNISTIC` to `DIRECT` at pack time). The `DIRECT` method runs the LXMF body over an established Reticulum Link rather than a single Reticulum DATA packet.
Pinned against **RNS 1.2.0 / LXMF 0.9.6**. Line numbers below are from those versions.
Pinned against **RNS 1.2.4 / LXMF 0.9.7**. Line numbers below are from those versions.
Out of scope: opportunistic delivery (see [`send-opportunistic-lxmf.md`](send-opportunistic-lxmf.md)), propagation-node delivery (`PROPAGATED`), and paper messages (`PAPER`).
@ -30,7 +30,7 @@ Same as steps 1-2 of `send-opportunistic-lxmf.md`. `LXMessage.pack()` builds the
### 2. `process_outbound` enters the DIRECT branch
`LXMF/LXMRouter.py:2599`. Logic for an `LXMessage` whose `method == DIRECT`:
`LXMF/LXMRouter.py:2531-2545`. Logic for an `LXMessage` whose `method == DIRECT`:
```python
delivery_destination_hash = lxmessage.get_destination().hash
@ -63,13 +63,13 @@ Setting `process_outbound` itself as the link's established-callback is the tric
### 3. `RNS.Link(destination)` builds and sends a LINKREQUEST
`RNS/Link.py:233-327`. The `Link` constructor on the **initiator** side generates a fresh ephemeral X25519 + Ed25519 keypair (`pub_bytes` and `sig_pub_bytes`) and constructs the LINKREQUEST body (`RNS/Link.py:308-324`):
`RNS/Link.py:233-330`. The `Link` constructor on the **initiator** side generates a fresh ephemeral X25519 + Ed25519 keypair (`pub_bytes` and `sig_pub_bytes`) and constructs the LINKREQUEST body (`RNS/Link.py:308-324`):
```
request_data = initiator_X25519_pub(32) || initiator_Ed25519_pub(32) || [signalling(3)]
```
Both initiator-side keys are **fresh-ephemeral**, used only for this link. The optional 3-byte `signalling` field, present iff `Reticulum.link_mtu_discovery()` returns true and the next-hop interface advertises an HW MTU, encodes the path-MTU and link-mode hints (SPEC.md §6.1; encode/decode helpers at `RNS/Link.py:148`).
Both initiator-side keys are **fresh-ephemeral**, used only for this link. The optional 3-byte `signalling` field, present iff `Reticulum.link_mtu_discovery()` returns true and the next-hop interface advertises an HW MTU, encodes the path-MTU and link-mode hints (SPEC.md §6.1; encode/decode helpers at `RNS/Link.py:148-152`).
```python
self.packet = RNS.Packet(destination, self.request_data, packet_type=RNS.Packet.LINKREQUEST)
@ -88,12 +88,12 @@ The LINKREQUEST then goes through the same `Transport.outbound` path as any othe
### 4. Wait for LRPROOF; verify; complete handshake
The initiator's link sits in `Link.PENDING` while it waits for the responder's LRPROOF. The responder's side of this exchange (entering `Destination.receive` with `packet_type == LINKREQUEST` → `incoming_link_request` → `Link.validate_request` → `Link.handshake` → `Link.prove`, all at `RNS/Link.py:186-227, 353-381`) is its own flow document; this flow describes only what the initiator sees.
The initiator's link sits in `Link.PENDING` while it waits for the responder's LRPROOF. The responder's side of this exchange (entering `Destination.receive` with `packet_type == LINKREQUEST` → `incoming_link_request` → `Link.validate_request` → `Link.handshake` → `Link.prove`, all at `RNS/Link.py:186-230, 353-381`) is its own flow document; this flow describes only what the initiator sees.
The LRPROOF arrives back at `Transport.inbound` with `packet_type = PROOF`, `context = LRPROOF (0xff)`, `dest_hash = link_id`. `Transport.inbound` looks up the link by its `link_id` and dispatches to `Link.validate_proof` (`RNS/Link.py:396`). For an initiator with link still in `PENDING`:
```
proof_data layout (RNS/Link.py:376):
proof_data layout (RNS/Link.py:371):
signature(64) || responder_X25519_pub(32) || [signalling(3)]
```
@ -103,21 +103,21 @@ Validation (`RNS/Link.py:410-422`):
2. Read the responder's long-term Ed25519 public key from the `Destination.identity` we already had cached (from a prior announce — line 412: `self.destination.identity.get_public_key()[ECPUBSIZE//2:ECPUBSIZE]`). The Ed25519 pub is **not** sent in the LRPROOF body; it must be known locally already.
3. Derive `signed_data = link_id || responder_X25519_pub || responder_long_term_Ed25519_pub || [signalling]` (line 417).
4. Verify the Ed25519 signature against `signed_data` using the responder's long-term Ed25519 pub.
5. On success: `link.handshake()` runs (`RNS/Link.py:353-368`) — ECDH with the responder's fresh X25519 pub, then HKDF over the shared secret with `salt = link_id`, derives `signing_key(32) || encrypt_key(32)` per SPEC.md §6.4. Link state transitions to `Link.ACTIVE`.
5. On success: `link.handshake()` runs (`RNS/Link.py:353-370`) — ECDH with the responder's fresh X25519 pub, then HKDF over the shared secret with `salt = link_id`, derives `signing_key(32) || encrypt_key(32)` per SPEC.md §6.4. Link state transitions to `Link.ACTIVE`.
6. The initiator's `established_callback` registered at step 2 — `LXMRouter.process_outbound` — fires, re-entering step 2 with the link now ACTIVE.
A confirmed-MTU hint in the LRPROOF (length is `64+32+3 = 99` instead of `64+32 = 96`) updates `link.mtu` to the smaller of the responder's hint and the initiator's view (`RNS/Link.py:404-408`). An RTT report message follows automatically (`RNS/Link.py:441`, packet with `context = LRRTT`).
### 5. Transfer the LXM body over the active link
Back in `process_outbound` (`LXMF/LXMRouter.py:2618-2625`), with `direct_link.status == ACTIVE`:
Back in `process_outbound` (`LXMF/LXMRouter.py:2620-2630`), with `direct_link.status == ACTIVE`:
```python
lxmessage.set_delivery_destination(direct_link) # __delivery_destination = link
lxmessage.send() # LXMessage.send branches on representation
```
`LXMessage.send` for DIRECT/PACKET (`LXMF/LXMessage.py:471-484`) calls `self.__as_packet()` which constructs:
`LXMessage.send` for DIRECT/PACKET (`LXMF/LXMessage.py:471-490`) calls `self.__as_packet()` which constructs:
```python
RNS.Packet(self.__delivery_destination, self.packed) # full LXMF body, dest_hash included
@ -161,10 +161,10 @@ When the resource concludes successfully, the same `__mark_delivered` path runs
### 9. Backchannel identification (optional, after first successful DIRECT delivery)
`LXMF/LXMRouter.py:2532-2545`. After a DIRECT delivery completes (`lxmessage.state == DELIVERED`), if the link doesn't yet have a backchannel identity associated, the initiator's `LXMRouter` calls `direct_link.identify(backchannel_identity)`:
`LXMF/LXMRouter.py:2531-2545`. After a DIRECT delivery completes (`lxmessage.state == DELIVERED`), if the link doesn't yet have a backchannel identity associated, the initiator's `LXMRouter` calls `direct_link.identify(backchannel_identity)`:
- `direct_link.identify` sends an IDENTIFY packet on the link bound to one of the **sender's** local `lxmf.delivery` destinations.
- The receiving side's `delivery_link_established` callback (set at `LXMRouter.py:1849-1856`) installs `delivery_packet` on the link's packet callback, so subsequent inbound DATA on this link reaches the LXMF parser.
- The receiving side's `delivery_link_established` callback (set at `LXMRouter.py:1852-1858`) installs `delivery_packet` on the link's packet callback, so subsequent inbound DATA on this link reaches the LXMF parser.
- The link is now usable in **both directions** without each side having to open its own Link to the other. The receiver can now reply over this same link.
This enables an interactive conversation over a single link rather than each message opening a new link.

8
flows/send-opportunistic-lxmf.md
View file

@ -2,7 +2,7 @@
What happens chronologically when an app calls `LXMRouter.handle_outbound(lxm)` for an `LXMessage` whose `desired_method == OPPORTUNISTIC` and whose payload fits in a single Reticulum packet.
Pinned against **RNS 1.2.0 / LXMF 0.9.6**. Line numbers below are from those versions.
Pinned against **RNS 1.2.4 / LXMF 0.9.7**. Line numbers below are from those versions.
Out of scope: messages that need a Reticulum Link (`DIRECT` method, larger payloads), propagation-node delivery (`PROPAGATED`), and paper messages (`PAPER`). Each gets its own flow document.
@ -66,7 +66,7 @@ if not RNS.Transport.has_path(destination_hash) and lxmessage.method == LXMessag
lxmessage.next_delivery_attempt = time.time() + LXMRouter.PATH_REQUEST_WAIT
```
Only fires when **no entry exists** in `Transport.path_table`. Stale-but-present entries are not refreshed by this preamble — they are evicted by the periodic `stale_paths` accumulator in `RNS/Transport.py:747+`, after which the next outbound attempt rediscovers the unknown-path branch and triggers `request_path`. The path-request packet itself (well-known dest hash `6b9f66014d9853faab220fba47d02761`, payload `target_dest_hash || [transport_id ||] tag`) is described in SPEC.md §7.1.
Only fires when **no entry exists** in `Transport.path_table`. Stale-but-present entries are not refreshed by this preamble — they are evicted by the periodic `stale_paths` accumulator in `RNS/Transport.py:750+`, after which the next outbound attempt rediscovers the unknown-path branch and triggers `request_path`. The path-request packet itself (well-known dest hash `6b9f66014d9853faab220fba47d02761`, payload `target_dest_hash || [transport_id ||] tag`) is described in SPEC.md §7.1.
When this preamble fires the message is queued, not sent — control returns; sending resumes at step 5 after `PATH_REQUEST_WAIT` elapses or an announce response populates the path table.
@ -113,7 +113,7 @@ ephemeral_pub(32) || iv(16) || aes_ciphertext(...) || hmac_sha256(32)
### 8. `Transport.outbound(packet)` — path-table-aware framing
`RNS/Transport.py:1031-1113`. Verified by [`../tools/verify_packet_header.py`](../tools/verify_packet_header.py).
`RNS/Transport.py:1034-1116`. Verified by [`../tools/verify_packet_header.py`](../tools/verify_packet_header.py).
- If `path_table[dest][HOPS] > 1`: convert HEADER_1 → HEADER_2 (SPEC.md §2.3). The originator inserts `path_table[dest][NEXT_HOP]` (16-byte transport_id) at offset 2 and flips the flag bits to `HEADER_2 | TRANSPORT | (orig_low_nibble)`. Resulting wire packet is 35 + ciphertext bytes.
- If `path_table[dest][HOPS] == 1` AND the local node is connected to a shared instance: same conversion applies (lines 1094-1105).
@ -148,7 +148,7 @@ The receive flow is its own document; see `receive-opportunistic-lxmf.md` (TODO)
### 12. PROOF receipt returns
`RNS/Transport.py:1031-1054`. Because the packet is `DATA` for a non-PLAIN, non-LINK destination with `create_receipt == True`, `Transport.outbound` registered a `PacketReceipt` on the sender side. When the recipient calls `Packet.prove`, a PROOF packet flies back containing `SHA256(packet.hashable_part)`; the sender's `PacketReceipt` matches it, fires the delivery callback registered at step 5, and the LXMessage state advances `SENT → DELIVERED`.
`RNS/Transport.py:1034-1057`. Because the packet is `DATA` for a non-PLAIN, non-LINK destination with `create_receipt == True`, `Transport.outbound` registered a `PacketReceipt` on the sender side. When the recipient calls `Packet.prove`, a PROOF packet flies back containing `SHA256(packet.hashable_part)`; the sender's `PacketReceipt` matches it, fires the delivery callback registered at step 5, and the LXMessage state advances `SENT → DELIVERED`.
If no proof arrives within the receipt timeout, `__link_packet_timed_out` runs on the receipt's timeout callback and the LXMessage state can drive a retry (see `LXMRouter.py::process_outbound` retry logic at `:2571+`, which may itself trigger a fresh `request_path` after `MAX_PATHLESS_TRIES`).

6
flows/send-propagated-lxmf.md
View file

@ -1,6 +1,6 @@
# Flow: send a PROPAGATED LXMF message via a propagation node
What happens when an LXMF client submits a message to a propagation node for store-and-forward delivery — the path used when the recipient is offline, intermittent, or simply somewhere the sender can't reach directly. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §5.8 (propagation protocol), §6 (Link), §10 (Resource), §11 (REQUEST/RESPONSE).
What happens when an LXMF client submits a message to a propagation node for store-and-forward delivery — the path used when the recipient is offline, intermittent, or simply somewhere the sender can't reach directly. Pinned against **RNS 1.2.4 / LXMF 0.9.7**; cross-references [`../SPEC.md`](../SPEC.md) §5.8 (propagation protocol), §6 (Link), §10 (Resource), §11 (REQUEST/RESPONSE).
---
@ -15,7 +15,7 @@ What happens when an LXMF client submits a message to a propagation node for sto
### 1. App constructs an LXMessage with `desired_method = PROPAGATED`
Same as `send-opportunistic-lxmf.md` step 1 except the desired_method differs. The router's `handle_outbound` (`LXMF/LXMRouter.py:1639+`) will eventually route this to the propagation pipeline.
Same as `send-opportunistic-lxmf.md` step 1 except the desired_method differs. The router's `handle_outbound` (`LXMF/LXMRouter.py:1644+`) will eventually route this to the propagation pipeline.
### 2. `LXMessage.pack()` — propagation-specific encryption
@ -30,7 +30,7 @@ Same as `send-opportunistic-lxmf.md` step 1 except the desired_method differs. T
### 3. `LXMRouter.process_outbound` for PROPAGATED method
`LXMF/LXMRouter.py:2547-...` (the `PROPAGATED` branch is structurally similar to the `DIRECT` branch in `send-link-lxmf.md`). High-level state:
`LXMF/LXMRouter.py:2544+` (the `PROPAGATED` branch is structurally similar to the `DIRECT` branch in `send-link-lxmf.md`). High-level state:
- If a Link to the propagation node already exists and is `ACTIVE`: reuse it.
- Else if the path is known: open a fresh `RNS.Link(propagation_node_destination)` with `LXMRouter.process_outbound` registered as the `link_established_callback` so the LXM is sent as soon as the link establishes.

6
flows/send-resource.md
View file

@ -2,7 +2,7 @@
What happens chronologically when an LXMF DIRECT message, NomadNet page, or `rncp` file transfer too big to fit in one Link DATA packet is sent as an `RNS.Resource`. Builds on top of an established Reticulum Link — a Link must already be `ACTIVE` before this flow starts (see [`send-link-lxmf.md`](send-link-lxmf.md) steps 3-4 for how the Link gets there).
Pinned against **RNS 1.2.0**. Wire-level details are in [`../SPEC.md`](../SPEC.md) §10; this document covers chronology and step ordering.
Pinned against **RNS 1.2.4**. Wire-level details are in [`../SPEC.md`](../SPEC.md) §10; this document covers chronology and step ordering.
Out of scope: the receive side (`receive-resource.md` — TODO), Resource cancellation paths beyond a brief mention in step 9, and the watchdog / RTT estimation machinery (implementation-private).
@ -125,7 +125,7 @@ body = resource_hash(32) || full_proof(32)
as `RNS.Packet(link, proof_data, packet_type=PROOF, context=RESOURCE_PRF)` — a PROOF-type packet, not DATA.
The initiator's `validate_proof(proof_data)` (`RNS/Resource.py:785-829`):
The initiator's `validate_proof(proof_data)` (`RNS/Resource.py:782-826`):
1. Checks `len(proof_data) == 64` and `proof_data[32:] == self.expected_proof`.
2. On match, transitions `status = COMPLETE` and fires the resource callback.
@ -144,7 +144,7 @@ Both transition `status = FAILED` and notify `link.resource_concluded(self)` so
### 10. Watchdog and recovery
`RNS/Resource.py:564-642`. The Resource owns a watchdog thread that runs through the lifecycle and adjusts timeouts based on observed link RTT. Key points for interop:
`RNS/Resource.py:564-670`. The Resource owns a watchdog thread that runs through the lifecycle and adjusts timeouts based on observed link RTT. Key points for interop:
- **Per-part timeout:** `PART_TIMEOUT_FACTOR = 4` × (link RTT) before any part has arrived; drops to `PART_TIMEOUT_FACTOR_AFTER_RTT = 2` once RTT is calibrated.
- **Proof timeout:** `PROOF_TIMEOUT_FACTOR = 3` × link RTT after all parts have been sent.

2
test-vectors/README.md
View file

@ -4,7 +4,7 @@ Known-good byte sequences that any Reticulum-compatible implementation should be
## Status
Populated against RNS 1.2.0 / LXMF 0.9.6:
Populated against RNS 1.2.4 / LXMF 0.9.7:
- ✅ `identities.json` — Alice + Bob identity vectors (regenerator: `../tools/regen_identities.py`, verifier: `../tools/verify_destination_hash.py`).
- ✅ `announces.json` — two announce vectors (no-ratchet + with-ratchet) signed by Alice (regenerator: `../tools/regen_announces.py`, verifier: `../tools/verify_announce_roundtrip.py`).

36
todo.md
View file

@ -396,6 +396,42 @@ order: top three save the most debugging hours.
implicit in §4.5 / §7.x / §10 / §12 today; a single appendix
table would be a quick reference card.
## Upstream distribution shift
RNS 1.2.4 (2026-05-07) is *"probably the last release that is also
published to GitHub, since everything can now run over Reticulum
itself."* Pip continues *"at least until `rnpkg` is complete, and RNS
is completely self-hosting."* Watch-items so the verifier doesn't
strand when GitHub / PyPI stop being authoritative:
- [ ] **Stand up a local Reticulum node with internet reach.** Doesn't
need to be 24/7. Needed so `rngit` and (later) `rnpkg` can fetch
from upstream once the GitHub mirror is gone. Capture the node's
identity / config in a private spot (not this repo).
- [ ] **Capture the upstream Reticulum repo node's destination hash
once published** — markqvist will publish a `rngit` address
for the official source repo. When that lands, record it
somewhere durable (suggest `tools/sources.md`, new file) so
anyone bringing up the verifier knows where to fetch from
when GitHub goes dark.
- [ ] **Watch `rnpkg` for install/upgrade commands.** Currently a stub
in 1.2.4 (only `--config` / `--exampleconfig` / `--version`
flags). When `rnpkg install` / `rnpkg upgrade` ship, swap
`pip install -r tools/requirements.txt` instructions to the
`rnpkg` equivalent (or document both during the transition).
- [ ] **`rsg` signature verification.** RNS 1.2.4 introduced a new
`rsg` file signature format for release artifacts. Once releases
stop being GitHub-signed, we'll need to verify `rsg` signatures
on whatever we pull through `rnpkg`/`rngit` to know we got
authentic upstream code. Likely a small helper script in
`tools/`.
- [ ] **Mirror upstream source citations into `references/`.** SPEC.md
cites upstream Python by file + line throughout. Once upstream
moves off GitHub, those citations get harder to follow without a
checkout. Consider extracting the cited functions/lines into a
`references/` tree keyed by RNS version, so the spec stays
navigable even when upstream is Reticulum-only.
## Spec polishing (lower priority)
- [x] **Navigation polish for `SPEC.md`** — at ~3300 lines, splitting

2
tools/README.md
View file

@ -19,7 +19,7 @@ The scripts read `RNS.__version__` at startup and print it in their output so a
## Status
Populated against RNS 1.2.0 / LXMF 0.9.6:
Populated against RNS 1.2.4 / LXMF 0.9.7:
| Script | Verifies SPEC.md section | Status |
|---|---|---|

15
tools/requirements.txt Normal file
View file

@ -0,0 +1,15 @@
# Pinned versions the verifier scripts have been run against.
#
# Update only after the corresponding verifier scripts have been
# re-run cleanly against the new upstream version. The README
# "Spec corrections" section exists to cover the case where the
# spec ships text that turns out to be wrong against a new pin.
#
# Why pin: upstream is migrating distribution off GitHub onto
# Reticulum-native channels (rngit / rnpkg). PyPI continues
# "at least until rnpkg is complete" per the RNS 1.2.4 release
# notes (2026-05-07). A pin keeps the verifier reproducible if
# upstream stops mirroring to PyPI before our migration is ready.
rns==1.2.4
lxmf==0.9.7

8
tools/verify_announce_app_data.py
View file

@ -4,8 +4,8 @@ destinations).
Exercises:
- Upstream LXMF.LXMRouter.get_announce_app_data emits a 2-element msgpack
array [display_name_bytes, stamp_cost] in LXMF 0.9.6. The dead-code
supported_functionality line at LXMF/LXMRouter.py:999 is computed but
array [display_name_bytes, stamp_cost] in LXMF 0.9.7. The dead-code
supported_functionality line at LXMF/LXMRouter.py:998 is computed but
never appended.
- The wire-byte form for display_name="Reticulum5", stamp_cost=None matches
the hex documented in SPEC.md S4.3:
@ -57,7 +57,7 @@ def verify_two_element_wire_bytes():
def verify_producer_is_two_element_in_this_lxmf():
# Read the LXMRouter source and confirm it appends 2 elements (not 3) in
# the LXMF 0.9.6 producer. We do this by inspecting the function source so
# the LXMF 0.9.7 producer. We do this by inspecting the function source so
# the verifier breaks loudly if upstream restores the 3-element variant.
import inspect
from LXMF.LXMRouter import LXMRouter
@ -72,7 +72,7 @@ def verify_producer_is_two_element_in_this_lxmf():
fail("S4.3 producer NOW appends supported_functionality. "
"Update SPEC.md S4.3 to describe the 3-element variant as live.")
print(f"PASS S4.3 LXMF {LXMF.__version__} producer emits 2-element form only "
"(supported_functionality is dead code at LXMF/LXMRouter.py:999)")
"(supported_functionality is dead code at LXMF/LXMRouter.py:998)")
def verify_parser_tolerance():

4
tools/verify_packet_header.py
View file

@ -12,7 +12,7 @@ Verifies:
and HEADER_2 layout flags(1) hops(1) transport_id(16) dest_hash(16)
context(1) data.
- §2.3: originator HEADER_1 → HEADER_2 conversion when path_table reports
hops > 1. The conversion logic at RNS/Transport.py:1074-1083 is
hops > 1. The conversion logic at RNS/Transport.py:1077-1108 is
exercised by stubbing Transport.transmit and seeding the path_table
with a synthetic multi-hop entry. The wire bytes captured at
transmit-time are compared to the expected HEADER_2 form.
@ -209,7 +209,7 @@ def verify_header_conversion(rns_instance):
fail("§2.3 conversion: trailing bytes (orig dest_hash + ctx + payload) mismatch")
print("PASS S2.3 HEADER_1 -> HEADER_2 conversion at originator "
"(matches RNS/Transport.py:1074-1083)")
"(matches RNS/Transport.py:1077-1108)")
finally:
Transport.transmit = real_transmit
with Transport.path_table_lock:

4
tools/verify_proof_packet.py
View file

@ -1,11 +1,11 @@
"""
Verifier for SPEC.md S6.5 (regular PROOF packet wire form).
Three scenarios against upstream RNS 1.2.0:
Three scenarios against upstream RNS 1.2.4:
1. Implicit-mode opportunistic DATA proof: when Reticulum is
configured with use_implicit_proof = True (the upstream default
per Reticulum.py:259), Identity.prove emits a 64-byte body
per Reticulum.py:256), Identity.prove emits a 64-byte body
containing only the Ed25519 signature over packet.packet_hash.
2. Explicit-mode opportunistic DATA proof: when use_implicit_proof

2
tools/verify_token_crypto.py
View file

@ -2,7 +2,7 @@
Verifier for SPEC.md S3 (Token cryptography).
Exercises the modified-Fernet Token construction in two directions
against upstream RNS 1.2.0:
against upstream RNS 1.2.4:
1. Identity-style encrypt (with ephemeral X25519 prefix) per S3.1
opportunistic form. Round-trips a known plaintext through