diff --git a/flows/README.md b/flows/README.md index 81312a1..f98b854 100644 --- a/flows/README.md +++ b/flows/README.md @@ -14,11 +14,12 @@ The two views are complementary: SPEC.md tells you what each piece looks like; t | [`receive-announce.md`](receive-announce.md) | ✅ | | [`send-resource.md`](send-resource.md) (Resource fragmentation over a Link) | ✅ | | [`path-discovery.md`](path-discovery.md) (path? request, path-response wire detail, path-table population) | ✅ | -| `receive-resource.md` (inverse of send-resource: ADV ingestion, part assembly, proof emission) | ⏳ | -| `receive-link-lxmf.md` (inverse of send-link-lxmf, including responder side of the handshake) | ⏳ | -| `send-propagated-lxmf.md` (PROPAGATED method, via a propagation node) | ⏳ | -| `send-announce.md` (build, sign, transmit, ratchet rotation, periodic re-announce) | ⏳ | -| `forward-announce.md` (transport-node rebroadcast logic, announce_cap, queue) | ⏳ | +| [`receive-resource.md`](receive-resource.md) (inverse of send-resource: ADV ingestion, part assembly, proof emission) | ✅ | +| [`receive-link-lxmf.md`](receive-link-lxmf.md) (inverse of send-link-lxmf, including responder side of the handshake) | ✅ | +| [`send-announce.md`](send-announce.md) (build, sign, transmit, ratchet rotation, periodic re-announce) | ✅ | +| [`forward-announce.md`](forward-announce.md) (transport-node rebroadcast logic, announce_cap, queue) | ✅ | +| [`send-propagated-lxmf.md`](send-propagated-lxmf.md) (PROPAGATED method, via a propagation node) | ✅ | +| `receive-propagated-lxmf.md` (recipient pulling messages via `/get`) | ⏳ | ## Conventions diff --git a/flows/forward-announce.md b/flows/forward-announce.md new file mode 100644 index 0000000..7046ec1 --- /dev/null +++ b/flows/forward-announce.md @@ -0,0 +1,139 @@ +# Flow: forward an announce (transport-node rebroadcast) + +What a transport-mode node does when it receives an inbound announce destined for a non-local destination. This is the flow that makes the mesh actually mesh — without it, announces never propagate beyond direct radio range. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §4.5 (validation), §12.3 (rebroadcast rules), §12.4 (path table). + +This flow only runs on a node with `enable_transport = Yes` per §12.1. Leaf clients can ignore it entirely. + +--- + +## Sequence + +### 1. Inbound announce passes validation + +The receive-announce flow ([`receive-announce.md`](receive-announce.md)) runs first — signature check, dest_hash recompute, public-key collision check, ingress rate limit, path table population. By the time the rebroadcast logic runs, the announce is known-valid and the path_table entry is already updated. + +### 2. Eligibility checks + +`RNS/Transport.py:1822`. Three conditions all must hold: + +```python +if (Reticulum.transport_enabled() or is_from_local_client) \ + and packet.context != PATH_RESPONSE \ + and not rate_blocked: + # rebroadcast +``` + +- `transport_enabled OR is_from_local_client` — leaf clients only forward announces that came from a local-client interface (the rare "client behind shared rnsd" case). +- `packet.context != PATH_RESPONSE` — path-response announces are NOT rebroadcast (they go on a single specific interface back to the requester per [`path-discovery.md`](path-discovery.md) step 7). +- `not rate_blocked` — per-interface announce-rate limits aren't tripped for this destination. + +### 3. Insert into `announce_table` + +`Transport.py:1833-1844`. The relay adds an entry: + +```python +Transport.announce_table[packet.destination_hash] = [ + now, # 0 IDX_AT_TIMESTAMP + retransmit_timeout, # 1 IDX_AT_RTRNS_TMO — when to actually emit + retries, # 2 IDX_AT_RETRIES — PATHFINDER_R, default 4 + received_from, # 3 IDX_AT_RCVD_IF — interface NOT to rebroadcast on + announce_hops, # 4 IDX_AT_HOPS + packet, # 5 IDX_AT_PACKET — full Packet object + local_rebroadcasts, # 6 IDX_AT_LCL_RBRD — count of times peers retransmitted + block_rebroadcasts, # 7 IDX_AT_BLCK_RBRD — true if a peer beat us to it + attached_interface, # 8 IDX_AT_ATTCHD_IF +] +``` + +`retransmit_timeout` for non-local-client originated announces is `now + PATH_REQUEST_GRACE = 0.4s` (giving directly-reachable peers time to rebroadcast first; if they do, `block_rebroadcasts = True` is set and we suppress our own emission). Local-client originated announces fire `now` with no grace. + +### 4. Periodic `Transport.jobs` walk drains the table + +The relay's per-second-or-so housekeeping loop walks `announce_table` and for entries whose `retransmit_timeout <= now`, queues them for emission on each suitable interface: + +```python +# Pseudocode of the relevant Transport.jobs branch +for dest_hash, entry in announce_table.items(): + if entry[BLCK_RBRD]: + continue # peer already rebroadcast — drop + if now < entry[RTRNS_TMO]: + continue # not yet time + if entry[RETRIES] <= 0: + announce_table.pop(dest_hash) # exhausted + continue + for interface in interfaces: + if interface == entry[RCVD_IF]: + continue # don't re-emit on receive interface + interface.announce_queue.append({"raw": entry[PACKET].raw, ...}) + entry[RETRIES] -= 1 +``` + +The actual code is in `Transport.py:1196-1300, 1810-1969`; the structure above is a simplification for the spec. + +### 5. Per-interface `announce_queue` drain + +Each interface independently throttles its outbound announces against `interface.announce_cap` (default `Reticulum.ANNOUNCE_CAP = 2.0` = 2% airtime). `Interface.process_announce_queue` (`RNS/Interfaces/Interface.py:232-272`) drains the queue at a rate the cap permits, **picking the lowest-hop-count entry first** so closer destinations propagate before further ones: + +```python +min_hops = min(e["hops"] for e in self.announce_queue) +selected = sorted([e for e in self.announce_queue if e["hops"] == min_hops], + key=lambda e: e["time"])[0] +tx_time = (len(selected["raw"]) * 8) / self.bitrate +wait_time = tx_time / self.announce_cap +self.announce_allowed_at = now + wait_time +self.process_outgoing(selected["raw"]) +``` + +The `wait_time` is what enforces the cap: on a 5kbps LoRa channel, a 200-byte announce takes 320ms airtime; with cap=2%, the next announce isn't allowed for `320ms / 0.02 = 16s`. This is what makes Reticulum's mesh well-behaved on slow shared channels. + +### 6. Rebroadcast emission with hop increment + +When the queue actually emits, the wire bytes are the original announce's `packet.raw` with the `hops` byte already incremented (`Transport.inbound` did this at line 1395 on receive). No re-signing — the signature in the announce body covers the original hop=0 emission, and signature validation ignores the outer hops byte (it's not in `signed_data`). What's wire-visible is the same body, the same dest_hash, the same random_hash, and a hops byte that's now (hops_received + 1). + +### 7. Local-rebroadcast counter cleanup + +If the relay later **hears its own rebroadcast** (a peer further along in the chain re-emitted it), `Transport.inbound` at line 1660-1668 increments `entry[IDX_AT_LCL_RBRD]`. Once `local_rebroadcasts >= LOCAL_REBROADCASTS_MAX`, the entry is removed from `announce_table`: + +```python +if announce_entry[IDX_AT_LCL_RBRD] >= LOCAL_REBROADCASTS_MAX: + announce_table.pop(packet.destination_hash) +``` + +This is what stops the rebroadcast loop: once enough downstream peers have echoed the announce back, the relay stops trying. + +### 8. `random_blob` replay defence + +§4.5 step 6.3 / §12.3.2 already covers this — the relay won't even queue a rebroadcast if the inbound announce's `random_hash` is already in the cached `random_blobs` for this destination. The receive-announce flow drops it at the path-table-update stage. + +--- + +## Wire-byte summary + +The forwarded announce is wire-identical to the original, except: + +- The `hops` byte is one higher. +- If the relay is between a HEADER_1-emitting originator and a destination > 1 hop away, the relay does NOT do the §2.3 conversion for ANNOUNCE packets — announces always travel HEADER_1+BROADCAST. The HEADER_2 conversion is only for DATA packets. + +``` +Before relay (received): After relay (emitted): +[ flags: HEADER_1+BROADCAST+SINGLE+ANNOUNCE ] [ same flags ] +[ hops = N ] [ hops = N+1 ] +[ 16B dest_hash ] [ same dest_hash ] +[ 1B context = 0x00 ] [ same context ] +[ ... announce body identical ... ] [ ... bytes unchanged ... ] +``` + +--- + +## Source map + +| Step | File | Function / line | +|---|---|---| +| 1 | (this flow follows `receive-announce.md` step 7) | | +| 2 | `RNS/Transport.py` | rebroadcast eligibility, line 1822 | +| 3 | `RNS/Transport.py` | announce_table insert, line 1833-1844 | +| 4 | `RNS/Transport.py` | jobs / queue drain, line 1196+ | +| 5 | `RNS/Interfaces/Interface.py` | `process_announce_queue`, line 232 | +| 6 | `RNS/Transport.py` | hops increment in `inbound`, line 1395 | +| 7 | `RNS/Transport.py` | local-rebroadcast counter, line 1660-1668 | +| 8 | `RNS/Transport.py` | random_blob replay check, line 1707-1745 | diff --git a/flows/receive-link-lxmf.md b/flows/receive-link-lxmf.md new file mode 100644 index 0000000..a47c87b --- /dev/null +++ b/flows/receive-link-lxmf.md @@ -0,0 +1,111 @@ +# Flow: receive an LXMF message over a Reticulum Link + +The inverse of [`send-link-lxmf.md`](send-link-lxmf.md), covering both halves of the responder side: accepting the inbound LINKREQUEST, sending the LRPROOF, then handling LXMF DATA on the established link. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §6 (Link), §6.5 (PROOF), §6.6 (signalling), §6.7 (KEEPALIVE/teardown), §10 (Resource). + +--- + +## Sequence + +### 1. LINKREQUEST arrives + +A Reticulum DATA packet with `packet_type = LINKREQUEST (2)`, addressed to the responder's `lxmf.delivery` dest_hash. Body (§6.1): + +``` +initiator_X25519_pub(32) || initiator_Ed25519_pub(32) || [signalling(3)] +``` + +`Transport.inbound` (`RNS/Transport.py:2027-2057`) recognizes `packet_type == LINKREQUEST + destination_type == SINGLE`, looks up the destination in `destinations_map`, and calls `Destination.receive(packet)` which routes to `Destination.incoming_link_request(data, packet)` per `RNS/Destination.py:403-406`: + +```python +def receive(self, packet): + if packet.packet_type == LINKREQUEST: + self.incoming_link_request(packet.data, packet) +``` + +### 2. Responder builds Link state via `Link.validate_request` + +`RNS/Link.py:186-227`. Length-checks the body (`ECPUBSIZE` or `ECPUBSIZE + LINK_MTU_SIZE`), rejects otherwise. On success: + +1. Build a `Link` object with `peer_pub_bytes = data[:32]` and `peer_sig_pub_bytes = data[32:64]`. +2. `set_link_id(packet)` per §6.3 — the link_id derives from `Packet.get_hashable_part`, invariant under HEADER_1↔HEADER_2 conversion. +3. If signalling present, parse MTU and mode per §6.6. +4. Call `link.handshake()` — ECDH with the initiator's X25519 ephemeral pub, HKDF over the shared secret with `salt=link_id`, derives `signing_key || encrypt_key`. Status `PENDING → HANDSHAKE`. +5. Call `link.prove()` to emit the LRPROOF. +6. Register the link in `Transport.active_links` and append to the destination's `links` list. + +### 3. Responder emits LRPROOF + +`RNS/Link.py:371-381`. Body per §6.2: + +``` +proof_data = signature(64) || responder_X25519_pub(32) || [signalling(3)] +``` + +where `signature = sign(link_id || responder_X25519_pub || responder_long_term_Ed25519_pub || [signalling])`. Wire packet: `packet_type = PROOF (3)`, `context = LRPROOF (0xFF)`, `dest_hash` field carries the link_id. + +### 4. RTT measurement (LRRTT round trip) + +After the initiator validates the LRPROOF and sends `Link.LRRTT (0xFE)` carrying its measured RTT, the responder receives it at `Link.receive` line 1056-1059 and calls `rtt_packet`. The responder's RTT cache updates and `Link.STATUS = ACTIVE` triggers the `link_established_callback` registered via `Destination.set_link_established_callback`. + +For LXMF, that callback is `LXMRouter.delivery_link_established` (`LXMF/LXMRouter.py:1849-1856`): + +```python +link.track_phy_stats(True) +link.set_packet_callback(self.delivery_packet) # ← inbound LXMF flows here +link.set_resource_strategy(RNS.Link.ACCEPT_APP) +link.set_resource_callback(self.delivery_resource_advertised) +link.set_resource_started_callback(self.resource_transfer_began) +link.set_resource_concluded_callback(self.delivery_resource_concluded) +link.set_remote_identified_callback(self.delivery_remote_identified) +``` + +This is what makes inbound DATA on this link route into LXMF processing. + +### 5. Inbound LXMF DATA — single-packet (PACKET representation) + +A regular DATA packet on the link (`context = NONE`, Token-encrypted with link session key per §3.1). `Link.receive` decrypts and passes the plaintext to `delivery_packet(data, packet)` (`LXMF/LXMRouter.py:1819-1847`) — the same handler used by opportunistic delivery. Differences: + +- `packet.destination_type == LINK` so `method = DIRECT`. +- The LXMF body arrives **with** the recipient's dest_hash (§5.2), so no re-prepend like the opportunistic path does at step 9 of `receive-opportunistic-lxmf.md`. + +The handler calls `packet.prove()` immediately (mandatory PROOF receipt per §6.5), then dispatches the body to `LXMessage.unpack_from_bytes` and `lxmf_delivery` exactly like the opportunistic flow's steps 10-12. + +### 6. Inbound LXMF DATA — Resource representation + +A larger LXMF body arrives as a Resource transfer per `flows/receive-resource.md`. The Link's `resource_strategy = ACCEPT_APP` triggers `delivery_resource_advertised(resource)` (`LXMF/LXMRouter.py:1864-1871`): + +```python +def delivery_resource_advertised(self, resource): + size = resource.get_data_size() + if self.delivery_per_transfer_limit and size > self.delivery_per_transfer_limit*1000: + return False # reject — over limit + return True # accept +``` + +If accepted, `Resource.accept` runs and the receiver state machine in `flows/receive-resource.md` takes over. On completion, `delivery_resource_concluded(resource)` fires, reads the assembled file, and feeds it through `lxmf_delivery` exactly like the single-packet path. + +### 7. KEEPALIVE / teardown + +Standard §6.7 protocol. The responder reflects every `0xFF` ping with a `0xFE` pong, emits its own KEEPALIVE-driven STALE→CLOSED transition if `last_inbound + 2*keepalive` elapses, and accepts inbound LINKCLOSE packets (validating that `decrypt(body) == link_id`). + +### 8. Backchannel-identify (post-first-delivery) + +After a successful inbound LXMF delivery, the LXMRouter on the **initiator** side may emit a `LINKIDENTIFY (context = 0xFB)` proof so the responder can record the initiator's long-term identity for backchannel use (`flows/send-link-lxmf.md` step 9). On the responder side, this triggers `delivery_remote_identified` which lets the responder send LXMF replies back over the same link without opening its own. + +--- + +## Source map + +| Step | File | Function / line | +|---|---|---| +| 1 | `RNS/Transport.py` | LINKREQUEST dispatch, line 2027 | +| 1 | `RNS/Destination.py` | `receive` LINKREQUEST branch, line 403 | +| 2 | `RNS/Link.py` | `validate_request`, line 186 | +| 2 | `RNS/Link.py` | `handshake`, line 353 | +| 3 | `RNS/Link.py` | `prove` (LRPROOF emission), line 371 | +| 4 | `RNS/Link.py` | `rtt_packet`, line 534 | +| 4 | `LXMF/LXMRouter.py` | `delivery_link_established`, line 1849 | +| 5 | `LXMF/LXMRouter.py` | `delivery_packet`, line 1819 | +| 6 | `LXMF/LXMRouter.py` | `delivery_resource_advertised` / `_concluded`, lines 1864-1900 | +| 7 | `RNS/Link.py` | KEEPALIVE handling, line 1149 | +| 8 | `LXMF/LXMRouter.py` | `delivery_remote_identified`, line 1849+ | diff --git a/flows/receive-resource.md b/flows/receive-resource.md new file mode 100644 index 0000000..ccfcb0b --- /dev/null +++ b/flows/receive-resource.md @@ -0,0 +1,97 @@ +# Flow: receive a Resource (large body) over a Link + +The inverse of [`send-resource.md`](send-resource.md). What happens chronologically on the receiver when an inbound Resource transfer arrives. Pinned against **RNS 1.2.0**; see [`../SPEC.md`](../SPEC.md) §10 for the wire bytes. + +--- + +## Preconditions + +- Link is `ACTIVE` (§6). +- Receiver registered a `resource_strategy` on the Link via `set_resource_strategy(...)` — `ACCEPT_NONE`, `ACCEPT_APP`, or `ACCEPT_ALL`. Default `ACCEPT_NONE` rejects every Resource on the link; LXMF and NomadNet flip this to `ACCEPT_APP` so an app callback can decide. + +--- + +## Sequence + +### 1. RESOURCE_ADV arrives + +Inbound Link DATA packet with `context = RESOURCE_ADV (0x02)`. `Link.receive` at `RNS/Link.py:1065-1098` decrypts it and runs `RNS.ResourceAdvertisement.unpack` against the plaintext to extract the msgpack dict (§10.4). + +### 2. Resource accept / reject decision + +Branch by `resource_strategy`: + +- **`ACCEPT_NONE`** → call `RNS.Resource.reject(adv_packet)` which sends `RESOURCE_RCL` back; resource is dropped. +- **`ACCEPT_APP`** → run the application callback `link.callbacks.resource(adv)`. If it returns truthy, `RNS.Resource.accept(...)`; otherwise reject. +- **`ACCEPT_ALL`** → unconditional `RNS.Resource.accept(...)`. + +`Resource.accept` (`RNS/Resource.py:167-244`) constructs a receiver-side Resource object, copies fields from the advertisement, sets `status = TRANSFERRING`, and queues the first request. + +### 3. Receiver issues the first RESOURCE_REQ + +`Resource.request_next()` (`RNS/Resource.py:934-983`) builds the request body per §10.5: + +``` +exhausted_flag(1) [|| last_map_hash(4)] || resource_hash(32) || requested_map_hashes(N × 4) +``` + +`N = link.window` initially (default 4). Sent as Link DATA with `context = RESOURCE_REQ (0x03)`. + +### 4. Sender fulfills with RESOURCE part packets + +For each requested map_hash, the sender (per `flows/send-resource.md` step 5) emits one Link DATA packet with `context = RESOURCE (0x01)`, body = pre-encrypted part bytes. The receiver matches each arriving part to the hashmap by recomputing its 4-byte map_hash (`Resource.receive_part` line 831-932). + +Successful match: `parts[i] = part_data`, `consecutive_completed_height` advances. The window grows by 1 each successful round (capped at `window_max`, with rate-detection upgrades to FAST or VERY_SLOW per §10.10). + +### 5. Repeat steps 3-4 until `received_count == total_parts` + +When the receiver has consumed every map_hash in the current segment, it issues another RESOURCE_REQ. If the hashmap is exhausted (`exhausted_flag = 0xFF`), the sender responds with a RESOURCE_HMU carrying the next hashmap segment (§10.7), and the loop continues. + +### 6. `Resource.assemble()` reassembles, validates, decrypts + +`Resource.py:672-726`: + +1. `stream = b"".join(self.parts)` — concatenate every part. +2. `data = link.decrypt(stream)` — single Link Token decrypt of the whole blob (§10.12: encryption was applied to the whole concatenated body before splitting). +3. Strip the 4-byte `random_hash` prefix. +4. If `compressed`: bz2-decompress. +5. `calculated_hash = SHA256(data || random_hash)`. Compare to `self.hash` (= advertisement's `h` field). On match: `status = COMPLETE`. On mismatch: `status = CORRUPT`; cancel. +6. If `has_metadata`: peel off the 3-byte length-prefixed msgpack metadata blob, write to `meta_storagepath`. +7. Write the data to `storagepath` (file-backed) or hold in `self.data` (memory-backed). +8. Call the application callback (the one passed to `Resource.accept`). + +### 7. RESOURCE_PRF emission + +`Resource.prove()` (line 755-766) sends back: + +``` +proof_data = resource_hash(32) || full_proof(32) +where full_proof = SHA256(data_with_random || resource_hash) +``` + +as a PROOF-type packet with `context = RESOURCE_PRF (0x05)`. The sender's `validate_proof` matches `proof_data[32:]` against its precomputed `expected_proof` and transitions to `COMPLETE` (§10.8). + +### 8. Multi-segment continuation + +If `segment_index < total_segments`, the sender prepares and sends the next RESOURCE_ADV after receiving this segment's PRF. The receiver loops back to step 1 for the next segment. Each segment is a fully independent Resource transfer; the only thing that ties them together is the `original_hash` field in the advertisement. + +### 9. Cancellation paths + +`RESOURCE_ICL` (sender cancel) → receiver pops the matching incoming Resource and discards accumulated parts (`Link.py:1131-1138`). +`RESOURCE_RCL` (sender hears the receiver rejected) → already handled receiver-side at step 2. + +--- + +## Source map + +| Step | File | Function / line | +|---|---|---| +| 1 | `RNS/Link.py` | `Link.receive` RESOURCE_ADV branch, line 1065 | +| 2 | `RNS/Resource.py` | `Resource.accept`, `Resource.reject`, lines 155-244 | +| 3 | `RNS/Resource.py` | `request_next`, line 934 | +| 4 | `RNS/Resource.py` | `receive_part`, line 831 | +| 5 | `RNS/Link.py` | RESOURCE_HMU branch, line 1122 | +| 6 | `RNS/Resource.py` | `assemble`, line 672 | +| 7 | `RNS/Resource.py` | `prove`, line 755 | +| 8 | `RNS/Resource.py` | `__prepare_next_segment`, line 768 | +| 9 | `RNS/Link.py` | RESOURCE_ICL branch, line 1131 | diff --git a/flows/send-announce.md b/flows/send-announce.md new file mode 100644 index 0000000..afb8a9d --- /dev/null +++ b/flows/send-announce.md @@ -0,0 +1,130 @@ +# Flow: send an announce + +What happens chronologically when a node emits an announce — the periodic broadcast that lets the rest of the mesh discover or refresh a path to this destination. Pinned against **RNS 1.2.0**. + +Out of scope: the relay-side rebroadcast (`forward-announce.md` — see [`../SPEC.md`](../SPEC.md) §12.3) and path-response announces (already covered in [`path-discovery.md`](path-discovery.md)). + +--- + +## Sequence + +### 1. Caller invokes `Destination.announce(app_data=...)` + +`RNS/Destination.py:243-318`. Triggers: + +- Periodic re-announce loop (every 5-15 minutes per §7.5; LXMF runs this from `LXMRouter.jobs`). +- Application-explicit announce (e.g. user clicks "announce now" in Sideband). +- Path-response branch — see [`path-discovery.md`](path-discovery.md) step 6. + +### 2. Build `random_hash` + +```python +random_hash = RNS.Identity.get_random_hash()[:5] + int(time.time()).to_bytes(5, "big") +``` + +5 random bytes + 5 bytes big-endian uint40 unix-seconds timestamp per [`../SPEC.md`](../SPEC.md) §4.1. The timestamp is used by transit relays for path-table replay-ordering decisions per §4.5 step 6.3. + +### 3. Optional ratchet rotation + +If the destination has ratchets enabled (`destination.ratchets != None`), `rotate_ratchets()` runs (`Destination.py:227-235`): + +```python +if now > self.latest_ratchet_time + self.ratchet_interval: + new_ratchet = Identity._generate_ratchet() # X25519 keypair, 32B priv + self.ratchets.insert(0, new_ratchet) # most-recent-first + self.latest_ratchet_time = now + self._clean_ratchets() # cap at RATCHET_COUNT = 512 + self._persist_ratchets() # storagepath/ratchets/ +``` + +The new ratchet's public key is what gets included in this announce. `RATCHET_INTERVAL = 30*60s` so a ratchet rotates at most every 30 minutes — back-to-back announces within that window reuse the current ratchet (per §7.3 the relays would dedup them on `(dest_hash, ratchet_pub)` if even the random_hash collided, but the 10 random bytes prevent that). + +### 4. Build signed_data + +Per §4.2: + +```python +signed_data = self.hash + self.identity.get_public_key() + self.name_hash + random_hash + ratchet +if app_data is not None: + signed_data += app_data +``` + +`self.hash` is the destination_hash (which appears in the outer Reticulum header on the wire, but is also signed). `ratchet` is `b""` when no ratchet is included. + +### 5. Sign and pack the announce body + +```python +signature = self.identity.sign(signed_data) # Ed25519 long-term key +announce_data = self.identity.get_public_key() + self.name_hash + random_hash + ratchet + signature +if app_data is not None: + announce_data += app_data +``` + +The `dest_hash` is **not** in `announce_data` even though it's in `signed_data` — the receiver gets `dest_hash` from the outer packet header per §4.1. + +### 6. Cache the body for path-response replay + +`Destination.py:303-309`: store `self.path_responses[tag] = [time.time(), announce_data]` if a `tag` was supplied (path-response branch). The cache TTL is `PR_TAG_WINDOW = 30s` per §7.2.4 — same wire bytes served to multiple racing relays for dedup convergence. + +### 7. Construct and emit the Reticulum ANNOUNCE packet + +```python +context_flag = FLAG_SET if ratchet else FLAG_UNSET +announce_context = PATH_RESPONSE if path_response else NONE +announce_packet = RNS.Packet(self, announce_data, + RNS.Packet.ANNOUNCE, + context = announce_context, + attached_interface = attached_interface, + context_flag = context_flag) +announce_packet.send() +``` + +Wire form per §4.1: +- `packet_type = ANNOUNCE (1)`, `transport_type = BROADCAST (0)`, `destination_type = SINGLE (0)` +- `header_type = HEADER_1` (the ratchet rotation announce is always broadcast — no transport_id at this stage) +- `context = NONE (0x00)` for periodic re-announces, `PATH_RESPONSE (0x0B)` for path-response announces +- `context_flag = 1` if ratchet present (signals the optional ratchet_pub slot in the body) + +Announce packets are NOT encrypted — `Packet.pack` (`RNS/Packet.py:189-191`) special-cases ANNOUNCE to skip encryption. The body is signed but plaintext, so anyone in earshot can validate the signature and decode the public key. + +### 8. `Transport.outbound` broadcasts on every OUT interface + +Same broadcast branch as a path? request (`flows/path-discovery.md` step 2) — the dest_hash isn't in `path_table` (it's our own destination, not a remote one), so the broadcast branch at `RNS/Transport.py:1119+` fires, emitting on every interface where `interface.OUT == True`. Per §7.5 the announce is rate-limited by `ANNOUNCE_CAP = 2.0` (2% airtime) on each interface. + +### 9. Periodic re-announce loop + +LXMF runs this via `LXMRouter.jobs` calling `LXMRouter.announce_propagation_node` and `LXMRouter.announce_destination` (for delivery destinations) at the configured cadence. Default re-announce interval is 5-15 minutes per §7.5. Without it, transit-relay path tables age out within minutes and peers can't message you. + +--- + +## Wire-byte summary + +``` +[ 1B flags: HEADER_1 | context_flag | BROADCAST | SINGLE | ANNOUNCE ] +[ 1B hops=0 ] +[ 16B dest_hash ] +[ 1B context: 0x00 normal / 0x0B PATH_RESPONSE ] +[ 64B public_key (X25519 || Ed25519) ] +[ 10B name_hash ] +[ 10B random_hash (5B random || 5B big-endian uint40 unix_seconds) ] +[ 32B ratchet_pub ] ← present iff context_flag bit set +[ 64B Ed25519 signature ] +[ N B app_data ] ← may be empty +``` + +--- + +## Source map + +| Step | File | Function / line | +|---|---|---| +| 1 | `RNS/Destination.py` | `announce`, line 243 | +| 2 | `RNS/Destination.py` | random_hash construction, line 282 | +| 3 | `RNS/Destination.py` | `rotate_ratchets`, line 227 | +| 4 | `RNS/Destination.py` | signed_data assembly, line 297 | +| 5 | `RNS/Destination.py` | sign + pack, line 300-303 | +| 6 | `RNS/Destination.py` | path_responses cache, line 305 | +| 7 | `RNS/Destination.py` | Packet construction, line 313 | +| 7 | `RNS/Packet.py` | ANNOUNCE-skips-encryption, line 189-191 | +| 8 | `RNS/Transport.py` | outbound broadcast branch, line 1119 | +| 9 | `LXMF/LXMRouter.py` | jobs / re-announce cadence | diff --git a/flows/send-propagated-lxmf.md b/flows/send-propagated-lxmf.md new file mode 100644 index 0000000..64a5305 --- /dev/null +++ b/flows/send-propagated-lxmf.md @@ -0,0 +1,84 @@ +# Flow: send a PROPAGATED LXMF message via a propagation node + +What happens when an LXMF client submits a message to a propagation node for store-and-forward delivery — the path used when the recipient is offline, intermittent, or simply somewhere the sender can't reach directly. Pinned against **RNS 1.2.0**; cross-references [`../SPEC.md`](../SPEC.md) §5.8 (propagation protocol), §6 (Link), §10 (Resource), §11 (REQUEST/RESPONSE). + +--- + +## Preconditions + +- Sender has discovered at least one propagation node via its `lxmf.propagation` announce (§4.4 / §5.8.5). `LXMRouter.set_outbound_propagation_node(propagation_node_dest_hash)` records which one to use. +- Sender has a path to the propagation node in `Transport.path_table`. + +--- + +## Sequence + +### 1. App constructs an LXMessage with `desired_method = PROPAGATED` + +Same as `send-opportunistic-lxmf.md` step 1 except the desired_method differs. The router's `handle_outbound` (`LXMF/LXMRouter.py:1639+`) will eventually route this to the propagation pipeline. + +### 2. `LXMessage.pack()` — propagation-specific encryption + +`LXMF/LXMessage.py:423-441`. Differences from the opportunistic and direct paths: + +- The body is encrypted **to the recipient's public key** (Token form per §3.1 with eph_pub prefix), the same as opportunistic — the propagation node never decrypts. +- The encrypted bytes form `pn_encrypted_data`; the wire body delivered to the propagation node is `dest_hash || pn_encrypted_data` (the recipient's destination_hash is preserved so the propagation node can route to the right recipient on retrieval). +- `transient_id = SHA256(lxmf_data)` (full hash of the encrypted body) — the propagation node's storage key. +- The whole thing is then wrapped: `propagation_packed = msgpack.packb([time.time(), [lxmf_data]])`. + +`representation` is set to `PACKET` if `propagation_packed` fits in `link.mdu`, else `RESOURCE`. + +### 3. `LXMRouter.process_outbound` for PROPAGATED method + +`LXMF/LXMRouter.py:2547-...` (the `PROPAGATED` branch is structurally similar to the `DIRECT` branch in `send-link-lxmf.md`). High-level state: + +- If a Link to the propagation node already exists and is `ACTIVE`: reuse it. +- Else if the path is known: open a fresh `RNS.Link(propagation_node_destination)` with `LXMRouter.process_outbound` registered as the `link_established_callback` so the LXM is sent as soon as the link establishes. +- Else: `Transport.request_path(propagation_node_dest_hash)` and defer for `LXMRouter.PATH_REQUEST_WAIT`. + +### 4. Link establishes (per `send-link-lxmf.md` steps 3-4) + +LINKREQUEST → LRPROOF → ACTIVE. The propagation node's `delivery_link_established` analogue for the propagation destination wires `LXMRouter.propagation_packet` as the link's packet callback. + +### 5. (Optional) Stamp generation for propagation cost + +If the propagation node's announce app_data declared a `stamp_cost` (element [5][0] per §5.8.5), the sender computes a propagation stamp via `LXMessage.get_propagation_stamp(target_cost)` (`LXMessage.py:326-350`). Algorithm same as §5.7 stamps but with `WORKBLOCK_EXPAND_ROUNDS_PN = 1000` rounds (cheaper than the regular `3000`-round stamp), and computed over the `transient_id` rather than `message_id`. + +The stamp is appended to the wire body: `lxmf_data += propagation_stamp` (the propagation node validates the stamp before storing). + +### 6. Submit via Link DATA (PACKET representation) + +If `representation == PACKET`, the sender emits a Link DATA packet (`context = NONE`) carrying `propagation_packed`. The propagation node's `propagation_packet` callback (`LXMRouter.py:2080+`) decodes the msgpack outer, extracts the LXMF bodies, validates the propagation stamp if required, and stores each one in `propagation_entries[transient_id]`. + +The Link DATA packet gets the standard mandatory PROOF receipt per §6.5; the receipt resolves the sender's `PacketReceipt` and `LXMessage.state` advances to `SENT`. **The state goes to `SENT`, not `DELIVERED`** — propagated messages are "delivered to the propagation node" but not yet "delivered to the recipient". The recipient pulls them later via `/get` (§5.8.3). + +### 7. Submit via Resource (RESOURCE representation) + +If `representation == RESOURCE`, the sender emits the `propagation_packed` blob as a Resource transfer per `flows/send-resource.md`. The propagation node accepts via `delivery_resource_advertised`, and on completion runs `propagation_resource_concluded` (`LXMRouter.py:2194+`) which decodes the `[time, [lxmf_data, ...]]` outer and stores each contained LXMF body. + +### 8. The link can stay open or be torn down + +After a successful propagation submission, the sender either tears down the link (`link.teardown()` per §6.7) or keeps it for another submission. The propagation node doesn't care — it has the messages and will offer them to peers via §5.8.2 sync independently. + +### 9. Eventual delivery — recipient pulls via `/get` + +When the recipient comes online (or just periodically), they open a Link to the propagation node, run `link.identify(my_identity)` so the propagation node knows whose mail to deliver, then issue a `/get` REQUEST per §5.8.3. The propagation node returns the stored LXMF bodies, and the recipient processes each through the same `lxmf_delivery` path that handles opportunistic / direct deliveries (`receive-opportunistic-lxmf.md` from step 10 onwards — the LXMF body bytes are identical regardless of how they arrived). + +This step is the recipient's flow (`receive-propagated-lxmf.md` — TODO), not the sender's, but it's worth noting here so the full lifecycle is visible. + +--- + +## Source map + +| Step | File | Function / line | +|---|---|---| +| 1 | (app code) | `LXMessage(..., desired_method=PROPAGATED)` | +| 2 | `LXMF/LXMessage.py` | `pack` PROPAGATED branch, line 423-441 | +| 3 | `LXMF/LXMRouter.py` | `process_outbound` PROPAGATED branch, line 2547+ | +| 4 | (see `send-link-lxmf.md` steps 3-4) | | +| 5 | `LXMF/LXMessage.py` | `get_propagation_stamp`, line 326-350 | +| 5 | `LXMF/LXStamper.py` | `WORKBLOCK_EXPAND_ROUNDS_PN = 1000` | +| 6 | `LXMF/LXMRouter.py` | `propagation_packet`, line 2080+ | +| 7 | `LXMF/LXMRouter.py` | `propagation_resource_concluded`, line 2194 | +| 9 | `LXMF/LXMRouter.py` | `request_messages_from_propagation_node`, line 485 | +| 9 | `LXMF/LXMPeer.py` | client-side `/get` flow |