jlpoole/reticiulum-specification
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add five companion flow docs

Browse source 
- flows/receive-resource.md: inverse of send-resource. ADV
    ingestion, accept/reject decision, request_next loop,
    receive_part insertion, assemble + decrypt + hash-validate,
    RESOURCE_PRF emission, multi-segment continuation.

  - flows/receive-link-lxmf.md: responder side of the link
    handshake plus inbound LXMF DATA handling. validate_request
    -> handshake -> prove (LRPROOF emission) -> link_established
    callback wires delivery_packet. PACKET-form inbound runs
    delivery_packet directly; RESOURCE-form inbound runs through
    delivery_resource_advertised + delivery_resource_concluded
    pipeline.

  - flows/send-announce.md: random_hash construction (5B random +
    5B BE-uint40 timestamp), optional ratchet rotation, signed_data
    assembly, sign + pack, the broadcast emission. Notes that
    ANNOUNCE packets are NOT encrypted (Packet.pack special-cases
    line 189-191) and the periodic re-announce loop drives 5-15min
    cadence.

  - flows/forward-announce.md: relay-side rebroadcast for
    transport-mode nodes. Eligibility checks (transport_enabled,
    not PATH_RESPONSE, not rate_blocked), announce_table queue,
    Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s,
    per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime
    enforcement, lowest-hop-count-first emission order, hops byte
    increment, local-rebroadcast counter for loop break.

  - flows/send-propagated-lxmf.md: PROPAGATED method end to end.
    LXMessage.pack with body encrypted to recipient (propagation
    node never decrypts), Link establishment to the propagation
    node, optional propagation stamp (1000 PoW rounds vs 3000 for
    regular stamps), submission via Link DATA or Resource,
    state goes to SENT (not DELIVERED — recipient pulls via /get
    later per §5.8.3).

flows/README.md status table updated; receive-propagated-lxmf.md
added as the only remaining ⏳ flow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rob 2026-05-03 12:21:05 -04:00
commit 282d5d59eb
6 changed files with 567 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

11
flows/README.md
View file

@ -14,11 +14,12 @@ The two views are complementary: SPEC.md tells you what each piece looks like; t
| [`receive-announce.md`](receive-announce.md) | ✅ |
| [`send-resource.md`](send-resource.md) (Resource fragmentation over a Link) | ✅ |
| [`path-discovery.md`](path-discovery.md) (path? request, path-response wire detail, path-table population) | ✅ |
| `receive-resource.md` (inverse of send-resource: ADV ingestion, part assembly, proof emission) | ⏳ |
| `receive-link-lxmf.md` (inverse of send-link-lxmf, including responder side of the handshake) | ⏳ |
| `send-propagated-lxmf.md` (PROPAGATED method, via a propagation node) | ⏳ |
| `send-announce.md` (build, sign, transmit, ratchet rotation, periodic re-announce) | ⏳ |
| `forward-announce.md` (transport-node rebroadcast logic, announce_cap, queue) | ⏳ |
| [`receive-resource.md`](receive-resource.md) (inverse of send-resource: ADV ingestion, part assembly, proof emission) | ✅ |
| [`receive-link-lxmf.md`](receive-link-lxmf.md) (inverse of send-link-lxmf, including responder side of the handshake) | ✅ |
| [`send-announce.md`](send-announce.md) (build, sign, transmit, ratchet rotation, periodic re-announce) | ✅ |
| [`forward-announce.md`](forward-announce.md) (transport-node rebroadcast logic, announce_cap, queue) | ✅ |
| [`send-propagated-lxmf.md`](send-propagated-lxmf.md) (PROPAGATED method, via a propagation node) | ✅ |
| `receive-propagated-lxmf.md` (recipient pulling messages via `/get`) | ⏳ |
## Conventions

139
flows/forward-announce.md Normal file
View file

@ -0,0 +1,139 @@
# Flow: forward an announce (transport-node rebroadcast)
What a transport-mode node does when it receives an inbound announce destined for a non-local destination. This is the flow that makes the mesh actually mesh — without it, announces never propagate beyond direct radio range. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §4.5 (validation), §12.3 (rebroadcast rules), §12.4 (path table).
This flow only runs on a node with `enable_transport = Yes` per §12.1. Leaf clients can ignore it entirely.
---
## Sequence
### 1. Inbound announce passes validation
The receive-announce flow ([`receive-announce.md`](receive-announce.md)) runs first — signature check, dest_hash recompute, public-key collision check, ingress rate limit, path table population. By the time the rebroadcast logic runs, the announce is known-valid and the path_table entry is already updated.
### 2. Eligibility checks
`RNS/Transport.py:1822`. Three conditions all must hold:
```python
if (Reticulum.transport_enabled() or is_from_local_client) \
and packet.context != PATH_RESPONSE \
and not rate_blocked:
# rebroadcast
```
- `transport_enabled OR is_from_local_client` — leaf clients only forward announces that came from a local-client interface (the rare "client behind shared rnsd" case).
- `packet.context != PATH_RESPONSE` — path-response announces are NOT rebroadcast (they go on a single specific interface back to the requester per [`path-discovery.md`](path-discovery.md) step 7).
- `not rate_blocked` — per-interface announce-rate limits aren't tripped for this destination.
### 3. Insert into `announce_table`
`Transport.py:1833-1844`. The relay adds an entry:
```python
Transport.announce_table[packet.destination_hash] = [
now, # 0 IDX_AT_TIMESTAMP
retransmit_timeout, # 1 IDX_AT_RTRNS_TMO — when to actually emit
retries, # 2 IDX_AT_RETRIES — PATHFINDER_R, default 4
received_from, # 3 IDX_AT_RCVD_IF — interface NOT to rebroadcast on
announce_hops, # 4 IDX_AT_HOPS
packet, # 5 IDX_AT_PACKET — full Packet object
local_rebroadcasts, # 6 IDX_AT_LCL_RBRD — count of times peers retransmitted
block_rebroadcasts, # 7 IDX_AT_BLCK_RBRD — true if a peer beat us to it
attached_interface, # 8 IDX_AT_ATTCHD_IF
]
```
`retransmit_timeout` for non-local-client originated announces is `now + PATH_REQUEST_GRACE = 0.4s` (giving directly-reachable peers time to rebroadcast first; if they do, `block_rebroadcasts = True` is set and we suppress our own emission). Local-client originated announces fire `now` with no grace.
### 4. Periodic `Transport.jobs` walk drains the table
The relay's per-second-or-so housekeeping loop walks `announce_table` and for entries whose `retransmit_timeout <= now`, queues them for emission on each suitable interface:
```python
# Pseudocode of the relevant Transport.jobs branch
for dest_hash, entry in announce_table.items():
if entry[BLCK_RBRD]:
continue # peer already rebroadcast — drop
if now < entry[RTRNS_TMO]:
continue # not yet time
if entry[RETRIES] <= 0:
announce_table.pop(dest_hash) # exhausted
continue
for interface in interfaces:
if interface == entry[RCVD_IF]:
continue # don't re-emit on receive interface
interface.announce_queue.append({"raw": entry[PACKET].raw, ...})
entry[RETRIES] -= 1
```
The actual code is in `Transport.py:1196-1300, 1810-1969`; the structure above is a simplification for the spec.
### 5. Per-interface `announce_queue` drain
Each interface independently throttles its outbound announces against `interface.announce_cap` (default `Reticulum.ANNOUNCE_CAP = 2.0` = 2% airtime). `Interface.process_announce_queue` (`RNS/Interfaces/Interface.py:232-272`) drains the queue at a rate the cap permits, **picking the lowest-hop-count entry first** so closer destinations propagate before further ones:
```python
min_hops = min(e["hops"] for e in self.announce_queue)
selected = sorted([e for e in self.announce_queue if e["hops"] == min_hops],
key=lambda e: e["time"])[0]
tx_time = (len(selected["raw"]) * 8) / self.bitrate
wait_time = tx_time / self.announce_cap
self.announce_allowed_at = now + wait_time
self.process_outgoing(selected["raw"])
```
The `wait_time` is what enforces the cap: on a 5kbps LoRa channel, a 200-byte announce takes 320ms airtime; with cap=2%, the next announce isn't allowed for `320ms / 0.02 = 16s`. This is what makes Reticulum's mesh well-behaved on slow shared channels.
### 6. Rebroadcast emission with hop increment
When the queue actually emits, the wire bytes are the original announce's `packet.raw` with the `hops` byte already incremented (`Transport.inbound` did this at line 1395 on receive). No re-signing — the signature in the announce body covers the original hop=0 emission, and signature validation ignores the outer hops byte (it's not in `signed_data`). What's wire-visible is the same body, the same dest_hash, the same random_hash, and a hops byte that's now (hops_received + 1).
### 7. Local-rebroadcast counter cleanup
If the relay later **hears its own rebroadcast** (a peer further along in the chain re-emitted it), `Transport.inbound` at line 1660-1668 increments `entry[IDX_AT_LCL_RBRD]`. Once `local_rebroadcasts >= LOCAL_REBROADCASTS_MAX`, the entry is removed from `announce_table`:
```python
if announce_entry[IDX_AT_LCL_RBRD] >= LOCAL_REBROADCASTS_MAX:
announce_table.pop(packet.destination_hash)
```
This is what stops the rebroadcast loop: once enough downstream peers have echoed the announce back, the relay stops trying.
### 8. `random_blob` replay defence
§4.5 step 6.3 / §12.3.2 already covers this — the relay won't even queue a rebroadcast if the inbound announce's `random_hash` is already in the cached `random_blobs` for this destination. The receive-announce flow drops it at the path-table-update stage.
---
## Wire-byte summary
The forwarded announce is wire-identical to the original, except:
- The `hops` byte is one higher.
- If the relay is between a HEADER_1-emitting originator and a destination > 1 hop away, the relay does NOT do the §2.3 conversion for ANNOUNCE packets — announces always travel HEADER_1+BROADCAST. The HEADER_2 conversion is only for DATA packets.
```
Before relay (received): After relay (emitted):
[ flags: HEADER_1+BROADCAST+SINGLE+ANNOUNCE ] [ same flags ]
[ hops = N ] [ hops = N+1 ]
[ 16B dest_hash ] [ same dest_hash ]
[ 1B context = 0x00 ] [ same context ]
[ ... announce body identical ... ] [ ... bytes unchanged ... ]
```
---
## Source map
| Step | File | Function / line |
|---|---|---|
| 1 | (this flow follows `receive-announce.md` step 7) | |
| 2 | `RNS/Transport.py` | rebroadcast eligibility, line 1822 |
| 3 | `RNS/Transport.py` | announce_table insert, line 1833-1844 |
| 4 | `RNS/Transport.py` | jobs / queue drain, line 1196+ |
| 5 | `RNS/Interfaces/Interface.py` | `process_announce_queue`, line 232 |
| 6 | `RNS/Transport.py` | hops increment in `inbound`, line 1395 |
| 7 | `RNS/Transport.py` | local-rebroadcast counter, line 1660-1668 |
| 8 | `RNS/Transport.py` | random_blob replay check, line 1707-1745 |

111
flows/receive-link-lxmf.md Normal file
View file

@ -0,0 +1,111 @@
# Flow: receive an LXMF message over a Reticulum Link
The inverse of [`send-link-lxmf.md`](send-link-lxmf.md), covering both halves of the responder side: accepting the inbound LINKREQUEST, sending the LRPROOF, then handling LXMF DATA on the established link. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §6 (Link), §6.5 (PROOF), §6.6 (signalling), §6.7 (KEEPALIVE/teardown), §10 (Resource).
---
## Sequence
### 1. LINKREQUEST arrives
A Reticulum DATA packet with `packet_type = LINKREQUEST (2)`, addressed to the responder's `lxmf.delivery` dest_hash. Body (§6.1):
```
initiator_X25519_pub(32) || initiator_Ed25519_pub(32) || [signalling(3)]
```
`Transport.inbound` (`RNS/Transport.py:2027-2057`) recognizes `packet_type == LINKREQUEST + destination_type == SINGLE`, looks up the destination in `destinations_map`, and calls `Destination.receive(packet)` which routes to `Destination.incoming_link_request(data, packet)` per `RNS/Destination.py:403-406`:
```python
def receive(self, packet):
if packet.packet_type == LINKREQUEST:
self.incoming_link_request(packet.data, packet)
```
### 2. Responder builds Link state via `Link.validate_request`
`RNS/Link.py:186-227`. Length-checks the body (`ECPUBSIZE` or `ECPUBSIZE + LINK_MTU_SIZE`), rejects otherwise. On success:
1. Build a `Link` object with `peer_pub_bytes = data[:32]` and `peer_sig_pub_bytes = data[32:64]`.
2. `set_link_id(packet)` per §6.3 — the link_id derives from `Packet.get_hashable_part`, invariant under HEADER_1↔HEADER_2 conversion.
3. If signalling present, parse MTU and mode per §6.6.
4. Call `link.handshake()` — ECDH with the initiator's X25519 ephemeral pub, HKDF over the shared secret with `salt=link_id`, derives `signing_key || encrypt_key`. Status `PENDING → HANDSHAKE`.
5. Call `link.prove()` to emit the LRPROOF.
6. Register the link in `Transport.active_links` and append to the destination's `links` list.
### 3. Responder emits LRPROOF
`RNS/Link.py:371-381`. Body per §6.2:
```
proof_data = signature(64) || responder_X25519_pub(32) || [signalling(3)]
```
where `signature = sign(link_id || responder_X25519_pub || responder_long_term_Ed25519_pub || [signalling])`. Wire packet: `packet_type = PROOF (3)`, `context = LRPROOF (0xFF)`, `dest_hash` field carries the link_id.
### 4. RTT measurement (LRRTT round trip)
After the initiator validates the LRPROOF and sends `Link.LRRTT (0xFE)` carrying its measured RTT, the responder receives it at `Link.receive` line 1056-1059 and calls `rtt_packet`. The responder's RTT cache updates and `Link.STATUS = ACTIVE` triggers the `link_established_callback` registered via `Destination.set_link_established_callback`.
For LXMF, that callback is `LXMRouter.delivery_link_established` (`LXMF/LXMRouter.py:1849-1856`):
```python
link.track_phy_stats(True)
link.set_packet_callback(self.delivery_packet) # ← inbound LXMF flows here
link.set_resource_strategy(RNS.Link.ACCEPT_APP)
link.set_resource_callback(self.delivery_resource_advertised)
link.set_resource_started_callback(self.resource_transfer_began)
link.set_resource_concluded_callback(self.delivery_resource_concluded)
link.set_remote_identified_callback(self.delivery_remote_identified)
```
This is what makes inbound DATA on this link route into LXMF processing.
### 5. Inbound LXMF DATA — single-packet (PACKET representation)
A regular DATA packet on the link (`context = NONE`, Token-encrypted with link session key per §3.1). `Link.receive` decrypts and passes the plaintext to `delivery_packet(data, packet)` (`LXMF/LXMRouter.py:1819-1847`) — the same handler used by opportunistic delivery. Differences:
- `packet.destination_type == LINK` so `method = DIRECT`.
- The LXMF body arrives **with** the recipient's dest_hash (§5.2), so no re-prepend like the opportunistic path does at step 9 of `receive-opportunistic-lxmf.md`.
The handler calls `packet.prove()` immediately (mandatory PROOF receipt per §6.5), then dispatches the body to `LXMessage.unpack_from_bytes` and `lxmf_delivery` exactly like the opportunistic flow's steps 10-12.
### 6. Inbound LXMF DATA — Resource representation
A larger LXMF body arrives as a Resource transfer per `flows/receive-resource.md`. The Link's `resource_strategy = ACCEPT_APP` triggers `delivery_resource_advertised(resource)` (`LXMF/LXMRouter.py:1864-1871`):
```python
def delivery_resource_advertised(self, resource):
size = resource.get_data_size()
if self.delivery_per_transfer_limit and size > self.delivery_per_transfer_limit*1000:
return False # reject — over limit
return True # accept
```
If accepted, `Resource.accept` runs and the receiver state machine in `flows/receive-resource.md` takes over. On completion, `delivery_resource_concluded(resource)` fires, reads the assembled file, and feeds it through `lxmf_delivery` exactly like the single-packet path.
### 7. KEEPALIVE / teardown
Standard §6.7 protocol. The responder reflects every `0xFF` ping with a `0xFE` pong, emits its own KEEPALIVE-driven STALE→CLOSED transition if `last_inbound + 2*keepalive` elapses, and accepts inbound LINKCLOSE packets (validating that `decrypt(body) == link_id`).
### 8. Backchannel-identify (post-first-delivery)
After a successful inbound LXMF delivery, the LXMRouter on the **initiator** side may emit a `LINKIDENTIFY (context = 0xFB)` proof so the responder can record the initiator's long-term identity for backchannel use (`flows/send-link-lxmf.md` step 9). On the responder side, this triggers `delivery_remote_identified` which lets the responder send LXMF replies back over the same link without opening its own.
---
## Source map
| Step | File | Function / line |
|---|---|---|
| 1 | `RNS/Transport.py` | LINKREQUEST dispatch, line 2027 |
| 1 | `RNS/Destination.py` | `receive` LINKREQUEST branch, line 403 |
| 2 | `RNS/Link.py` | `validate_request`, line 186 |
| 2 | `RNS/Link.py` | `handshake`, line 353 |
| 3 | `RNS/Link.py` | `prove` (LRPROOF emission), line 371 |
| 4 | `RNS/Link.py` | `rtt_packet`, line 534 |
| 4 | `LXMF/LXMRouter.py` | `delivery_link_established`, line 1849 |
| 5 | `LXMF/LXMRouter.py` | `delivery_packet`, line 1819 |
| 6 | `LXMF/LXMRouter.py` | `delivery_resource_advertised` / `_concluded`, lines 1864-1900 |
| 7 | `RNS/Link.py` | KEEPALIVE handling, line 1149 |
| 8 | `LXMF/LXMRouter.py` | `delivery_remote_identified`, line 1849+ |

97
flows/receive-resource.md Normal file
View file

@ -0,0 +1,97 @@
# Flow: receive a Resource (large body) over a Link
The inverse of [`send-resource.md`](send-resource.md). What happens chronologically on the receiver when an inbound Resource transfer arrives. Pinned against **RNS 1.2.0**; see [`../SPEC.md`](../SPEC.md) §10 for the wire bytes.
---
## Preconditions
- Link is `ACTIVE` (§6).
- Receiver registered a `resource_strategy` on the Link via `set_resource_strategy(...)` — `ACCEPT_NONE`, `ACCEPT_APP`, or `ACCEPT_ALL`. Default `ACCEPT_NONE` rejects every Resource on the link; LXMF and NomadNet flip this to `ACCEPT_APP` so an app callback can decide.
---
## Sequence
### 1. RESOURCE_ADV arrives
Inbound Link DATA packet with `context = RESOURCE_ADV (0x02)`. `Link.receive` at `RNS/Link.py:1065-1098` decrypts it and runs `RNS.ResourceAdvertisement.unpack` against the plaintext to extract the msgpack dict (§10.4).
### 2. Resource accept / reject decision
Branch by `resource_strategy`:
- **`ACCEPT_NONE`** → call `RNS.Resource.reject(adv_packet)` which sends `RESOURCE_RCL` back; resource is dropped.
- **`ACCEPT_APP`** → run the application callback `link.callbacks.resource(adv)`. If it returns truthy, `RNS.Resource.accept(...)`; otherwise reject.
- **`ACCEPT_ALL`** → unconditional `RNS.Resource.accept(...)`.
`Resource.accept` (`RNS/Resource.py:167-244`) constructs a receiver-side Resource object, copies fields from the advertisement, sets `status = TRANSFERRING`, and queues the first request.
### 3. Receiver issues the first RESOURCE_REQ
`Resource.request_next()` (`RNS/Resource.py:934-983`) builds the request body per §10.5:
```
exhausted_flag(1) [|| last_map_hash(4)] || resource_hash(32) || requested_map_hashes(N × 4)
```
`N = link.window` initially (default 4). Sent as Link DATA with `context = RESOURCE_REQ (0x03)`.
### 4. Sender fulfills with RESOURCE part packets
For each requested map_hash, the sender (per `flows/send-resource.md` step 5) emits one Link DATA packet with `context = RESOURCE (0x01)`, body = pre-encrypted part bytes. The receiver matches each arriving part to the hashmap by recomputing its 4-byte map_hash (`Resource.receive_part` line 831-932).
Successful match: `parts[i] = part_data`, `consecutive_completed_height` advances. The window grows by 1 each successful round (capped at `window_max`, with rate-detection upgrades to FAST or VERY_SLOW per §10.10).
### 5. Repeat steps 3-4 until `received_count == total_parts`
When the receiver has consumed every map_hash in the current segment, it issues another RESOURCE_REQ. If the hashmap is exhausted (`exhausted_flag = 0xFF`), the sender responds with a RESOURCE_HMU carrying the next hashmap segment (§10.7), and the loop continues.
### 6. `Resource.assemble()` reassembles, validates, decrypts
`Resource.py:672-726`:
1. `stream = b"".join(self.parts)` — concatenate every part.
2. `data = link.decrypt(stream)` — single Link Token decrypt of the whole blob (§10.12: encryption was applied to the whole concatenated body before splitting).
3. Strip the 4-byte `random_hash` prefix.
4. If `compressed`: bz2-decompress.
5. `calculated_hash = SHA256(data || random_hash)`. Compare to `self.hash` (= advertisement's `h` field). On match: `status = COMPLETE`. On mismatch: `status = CORRUPT`; cancel.
6. If `has_metadata`: peel off the 3-byte length-prefixed msgpack metadata blob, write to `meta_storagepath`.
7. Write the data to `storagepath` (file-backed) or hold in `self.data` (memory-backed).
8. Call the application callback (the one passed to `Resource.accept`).
### 7. RESOURCE_PRF emission
`Resource.prove()` (line 755-766) sends back:
```
proof_data = resource_hash(32) || full_proof(32)
where full_proof = SHA256(data_with_random || resource_hash)
```
as a PROOF-type packet with `context = RESOURCE_PRF (0x05)`. The sender's `validate_proof` matches `proof_data[32:]` against its precomputed `expected_proof` and transitions to `COMPLETE` (§10.8).
### 8. Multi-segment continuation
If `segment_index < total_segments`, the sender prepares and sends the next RESOURCE_ADV after receiving this segment's PRF. The receiver loops back to step 1 for the next segment. Each segment is a fully independent Resource transfer; the only thing that ties them together is the `original_hash` field in the advertisement.
### 9. Cancellation paths
`RESOURCE_ICL` (sender cancel) → receiver pops the matching incoming Resource and discards accumulated parts (`Link.py:1131-1138`).
`RESOURCE_RCL` (sender hears the receiver rejected) → already handled receiver-side at step 2.
---
## Source map
| Step | File | Function / line |
|---|---|---|
| 1 | `RNS/Link.py` | `Link.receive` RESOURCE_ADV branch, line 1065 |
| 2 | `RNS/Resource.py` | `Resource.accept`, `Resource.reject`, lines 155-244 |
| 3 | `RNS/Resource.py` | `request_next`, line 934 |
| 4 | `RNS/Resource.py` | `receive_part`, line 831 |
| 5 | `RNS/Link.py` | RESOURCE_HMU branch, line 1122 |
| 6 | `RNS/Resource.py` | `assemble`, line 672 |
| 7 | `RNS/Resource.py` | `prove`, line 755 |
| 8 | `RNS/Resource.py` | `__prepare_next_segment`, line 768 |
| 9 | `RNS/Link.py` | RESOURCE_ICL branch, line 1131 |

130
flows/send-announce.md Normal file
View file

@ -0,0 +1,130 @@
# Flow: send an announce
What happens chronologically when a node emits an announce — the periodic broadcast that lets the rest of the mesh discover or refresh a path to this destination. Pinned against **RNS 1.2.0**.
Out of scope: the relay-side rebroadcast (`forward-announce.md` — see [`../SPEC.md`](../SPEC.md) §12.3) and path-response announces (already covered in [`path-discovery.md`](path-discovery.md)).
---
## Sequence
### 1. Caller invokes `Destination.announce(app_data=...)`
`RNS/Destination.py:243-318`. Triggers:
- Periodic re-announce loop (every 5-15 minutes per §7.5; LXMF runs this from `LXMRouter.jobs`).
- Application-explicit announce (e.g. user clicks "announce now" in Sideband).
- Path-response branch — see [`path-discovery.md`](path-discovery.md) step 6.
### 2. Build `random_hash`
```python
random_hash = RNS.Identity.get_random_hash()[:5] + int(time.time()).to_bytes(5, "big")
```
5 random bytes + 5 bytes big-endian uint40 unix-seconds timestamp per [`../SPEC.md`](../SPEC.md) §4.1. The timestamp is used by transit relays for path-table replay-ordering decisions per §4.5 step 6.3.
### 3. Optional ratchet rotation
If the destination has ratchets enabled (`destination.ratchets != None`), `rotate_ratchets()` runs (`Destination.py:227-235`):
```python
if now > self.latest_ratchet_time + self.ratchet_interval:
new_ratchet = Identity._generate_ratchet() # X25519 keypair, 32B priv
self.ratchets.insert(0, new_ratchet) # most-recent-first
self.latest_ratchet_time = now
self._clean_ratchets() # cap at RATCHET_COUNT = 512
self._persist_ratchets() # storagepath/ratchets/<hex>
```
The new ratchet's public key is what gets included in this announce. `RATCHET_INTERVAL = 30*60s` so a ratchet rotates at most every 30 minutes — back-to-back announces within that window reuse the current ratchet (per §7.3 the relays would dedup them on `(dest_hash, ratchet_pub)` if even the random_hash collided, but the 10 random bytes prevent that).
### 4. Build signed_data
Per §4.2:
```python
signed_data = self.hash + self.identity.get_public_key() + self.name_hash + random_hash + ratchet
if app_data is not None:
signed_data += app_data
```
`self.hash` is the destination_hash (which appears in the outer Reticulum header on the wire, but is also signed). `ratchet` is `b""` when no ratchet is included.
### 5. Sign and pack the announce body
```python
signature = self.identity.sign(signed_data) # Ed25519 long-term key
announce_data = self.identity.get_public_key() + self.name_hash + random_hash + ratchet + signature
if app_data is not None:
announce_data += app_data
```
The `dest_hash` is **not** in `announce_data` even though it's in `signed_data` — the receiver gets `dest_hash` from the outer packet header per §4.1.
### 6. Cache the body for path-response replay
`Destination.py:303-309`: store `self.path_responses[tag] = [time.time(), announce_data]` if a `tag` was supplied (path-response branch). The cache TTL is `PR_TAG_WINDOW = 30s` per §7.2.4 — same wire bytes served to multiple racing relays for dedup convergence.
### 7. Construct and emit the Reticulum ANNOUNCE packet
```python
context_flag = FLAG_SET if ratchet else FLAG_UNSET
announce_context = PATH_RESPONSE if path_response else NONE
announce_packet = RNS.Packet(self, announce_data,
RNS.Packet.ANNOUNCE,
context = announce_context,
attached_interface = attached_interface,
context_flag = context_flag)
announce_packet.send()
```
Wire form per §4.1:
- `packet_type = ANNOUNCE (1)`, `transport_type = BROADCAST (0)`, `destination_type = SINGLE (0)`
- `header_type = HEADER_1` (the ratchet rotation announce is always broadcast — no transport_id at this stage)
- `context = NONE (0x00)` for periodic re-announces, `PATH_RESPONSE (0x0B)` for path-response announces
- `context_flag = 1` if ratchet present (signals the optional ratchet_pub slot in the body)
Announce packets are NOT encrypted — `Packet.pack` (`RNS/Packet.py:189-191`) special-cases ANNOUNCE to skip encryption. The body is signed but plaintext, so anyone in earshot can validate the signature and decode the public key.
### 8. `Transport.outbound` broadcasts on every OUT interface
Same broadcast branch as a path? request (`flows/path-discovery.md` step 2) — the dest_hash isn't in `path_table` (it's our own destination, not a remote one), so the broadcast branch at `RNS/Transport.py:1119+` fires, emitting on every interface where `interface.OUT == True`. Per §7.5 the announce is rate-limited by `ANNOUNCE_CAP = 2.0` (2% airtime) on each interface.
### 9. Periodic re-announce loop
LXMF runs this via `LXMRouter.jobs` calling `LXMRouter.announce_propagation_node` and `LXMRouter.announce_destination` (for delivery destinations) at the configured cadence. Default re-announce interval is 5-15 minutes per §7.5. Without it, transit-relay path tables age out within minutes and peers can't message you.
---
## Wire-byte summary
```
[ 1B flags: HEADER_1 | context_flag | BROADCAST | SINGLE | ANNOUNCE ]
[ 1B hops=0 ]
[ 16B dest_hash ]
[ 1B context: 0x00 normal / 0x0B PATH_RESPONSE ]
[ 64B public_key (X25519 || Ed25519) ]
[ 10B name_hash ]
[ 10B random_hash (5B random || 5B big-endian uint40 unix_seconds) ]
[ 32B ratchet_pub ] ← present iff context_flag bit set
[ 64B Ed25519 signature ]
[ N B app_data ] ← may be empty
```
---
## Source map
| Step | File | Function / line |
|---|---|---|
| 1 | `RNS/Destination.py` | `announce`, line 243 |
| 2 | `RNS/Destination.py` | random_hash construction, line 282 |
| 3 | `RNS/Destination.py` | `rotate_ratchets`, line 227 |
| 4 | `RNS/Destination.py` | signed_data assembly, line 297 |
| 5 | `RNS/Destination.py` | sign + pack, line 300-303 |
| 6 | `RNS/Destination.py` | path_responses cache, line 305 |
| 7 | `RNS/Destination.py` | Packet construction, line 313 |
| 7 | `RNS/Packet.py` | ANNOUNCE-skips-encryption, line 189-191 |
| 8 | `RNS/Transport.py` | outbound broadcast branch, line 1119 |
| 9 | `LXMF/LXMRouter.py` | jobs / re-announce cadence |

84
flows/send-propagated-lxmf.md Normal file
View file

@ -0,0 +1,84 @@
# Flow: send a PROPAGATED LXMF message via a propagation node
What happens when an LXMF client submits a message to a propagation node for store-and-forward delivery — the path used when the recipient is offline, intermittent, or simply somewhere the sender can't reach directly. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §5.8 (propagation protocol), §6 (Link), §10 (Resource), §11 (REQUEST/RESPONSE).
---
## Preconditions
- Sender has discovered at least one propagation node via its `lxmf.propagation` announce (§4.4 / §5.8.5). `LXMRouter.set_outbound_propagation_node(propagation_node_dest_hash)` records which one to use.
- Sender has a path to the propagation node in `Transport.path_table`.
---
## Sequence
### 1. App constructs an LXMessage with `desired_method = PROPAGATED`
Same as `send-opportunistic-lxmf.md` step 1 except the desired_method differs. The router's `handle_outbound` (`LXMF/LXMRouter.py:1639+`) will eventually route this to the propagation pipeline.
### 2. `LXMessage.pack()` — propagation-specific encryption
`LXMF/LXMessage.py:423-441`. Differences from the opportunistic and direct paths:
- The body is encrypted **to the recipient's public key** (Token form per §3.1 with eph_pub prefix), the same as opportunistic — the propagation node never decrypts.
- The encrypted bytes form `pn_encrypted_data`; the wire body delivered to the propagation node is `dest_hash || pn_encrypted_data` (the recipient's destination_hash is preserved so the propagation node can route to the right recipient on retrieval).
- `transient_id = SHA256(lxmf_data)` (full hash of the encrypted body) — the propagation node's storage key.
- The whole thing is then wrapped: `propagation_packed = msgpack.packb([time.time(), [lxmf_data]])`.
`representation` is set to `PACKET` if `propagation_packed` fits in `link.mdu`, else `RESOURCE`.
### 3. `LXMRouter.process_outbound` for PROPAGATED method
`LXMF/LXMRouter.py:2547-...` (the `PROPAGATED` branch is structurally similar to the `DIRECT` branch in `send-link-lxmf.md`). High-level state:
- If a Link to the propagation node already exists and is `ACTIVE`: reuse it.
- Else if the path is known: open a fresh `RNS.Link(propagation_node_destination)` with `LXMRouter.process_outbound` registered as the `link_established_callback` so the LXM is sent as soon as the link establishes.
- Else: `Transport.request_path(propagation_node_dest_hash)` and defer for `LXMRouter.PATH_REQUEST_WAIT`.
### 4. Link establishes (per `send-link-lxmf.md` steps 3-4)
LINKREQUEST → LRPROOF → ACTIVE. The propagation node's `delivery_link_established` analogue for the propagation destination wires `LXMRouter.propagation_packet` as the link's packet callback.
### 5. (Optional) Stamp generation for propagation cost
If the propagation node's announce app_data declared a `stamp_cost` (element [5][0] per §5.8.5), the sender computes a propagation stamp via `LXMessage.get_propagation_stamp(target_cost)` (`LXMessage.py:326-350`). Algorithm same as §5.7 stamps but with `WORKBLOCK_EXPAND_ROUNDS_PN = 1000` rounds (cheaper than the regular `3000`-round stamp), and computed over the `transient_id` rather than `message_id`.
The stamp is appended to the wire body: `lxmf_data += propagation_stamp` (the propagation node validates the stamp before storing).
### 6. Submit via Link DATA (PACKET representation)
If `representation == PACKET`, the sender emits a Link DATA packet (`context = NONE`) carrying `propagation_packed`. The propagation node's `propagation_packet` callback (`LXMRouter.py:2080+`) decodes the msgpack outer, extracts the LXMF bodies, validates the propagation stamp if required, and stores each one in `propagation_entries[transient_id]`.
The Link DATA packet gets the standard mandatory PROOF receipt per §6.5; the receipt resolves the sender's `PacketReceipt` and `LXMessage.state` advances to `SENT`. **The state goes to `SENT`, not `DELIVERED`** — propagated messages are "delivered to the propagation node" but not yet "delivered to the recipient". The recipient pulls them later via `/get` (§5.8.3).
### 7. Submit via Resource (RESOURCE representation)
If `representation == RESOURCE`, the sender emits the `propagation_packed` blob as a Resource transfer per `flows/send-resource.md`. The propagation node accepts via `delivery_resource_advertised`, and on completion runs `propagation_resource_concluded` (`LXMRouter.py:2194+`) which decodes the `[time, [lxmf_data, ...]]` outer and stores each contained LXMF body.
### 8. The link can stay open or be torn down
After a successful propagation submission, the sender either tears down the link (`link.teardown()` per §6.7) or keeps it for another submission. The propagation node doesn't care — it has the messages and will offer them to peers via §5.8.2 sync independently.
### 9. Eventual delivery — recipient pulls via `/get`
When the recipient comes online (or just periodically), they open a Link to the propagation node, run `link.identify(my_identity)` so the propagation node knows whose mail to deliver, then issue a `/get` REQUEST per §5.8.3. The propagation node returns the stored LXMF bodies, and the recipient processes each through the same `lxmf_delivery` path that handles opportunistic / direct deliveries (`receive-opportunistic-lxmf.md` from step 10 onwards — the LXMF body bytes are identical regardless of how they arrived).
This step is the recipient's flow (`receive-propagated-lxmf.md` — TODO), not the sender's, but it's worth noting here so the full lifecycle is visible.
---
## Source map
| Step | File | Function / line |
|---|---|---|
| 1 | (app code) | `LXMessage(..., desired_method=PROPAGATED)` |
| 2 | `LXMF/LXMessage.py` | `pack` PROPAGATED branch, line 423-441 |
| 3 | `LXMF/LXMRouter.py` | `process_outbound` PROPAGATED branch, line 2547+ |
| 4 | (see `send-link-lxmf.md` steps 3-4) | |
| 5 | `LXMF/LXMessage.py` | `get_propagation_stamp`, line 326-350 |
| 5 | `LXMF/LXStamper.py` | `WORKBLOCK_EXPAND_ROUNDS_PN = 1000` |
| 6 | `LXMF/LXMRouter.py` | `propagation_packet`, line 2080+ |
| 7 | `LXMF/LXMRouter.py` | `propagation_resource_concluded`, line 2194 |
| 9 | `LXMF/LXMRouter.py` | `request_messages_from_propagation_node`, line 485 |
| 9 | `LXMF/LXMPeer.py` | client-side `/get` flow |