reticiulum-specification/flows/forward-announce.md

139 lines
7.5 KiB
Markdown
Raw Normal View History

Add five companion flow docs - flows/receive-resource.md: inverse of send-resource. ADV ingestion, accept/reject decision, request_next loop, receive_part insertion, assemble + decrypt + hash-validate, RESOURCE_PRF emission, multi-segment continuation. - flows/receive-link-lxmf.md: responder side of the link handshake plus inbound LXMF DATA handling. validate_request -> handshake -> prove (LRPROOF emission) -> link_established callback wires delivery_packet. PACKET-form inbound runs delivery_packet directly; RESOURCE-form inbound runs through delivery_resource_advertised + delivery_resource_concluded pipeline. - flows/send-announce.md: random_hash construction (5B random + 5B BE-uint40 timestamp), optional ratchet rotation, signed_data assembly, sign + pack, the broadcast emission. Notes that ANNOUNCE packets are NOT encrypted (Packet.pack special-cases line 189-191) and the periodic re-announce loop drives 5-15min cadence. - flows/forward-announce.md: relay-side rebroadcast for transport-mode nodes. Eligibility checks (transport_enabled, not PATH_RESPONSE, not rate_blocked), announce_table queue, Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s, per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime enforcement, lowest-hop-count-first emission order, hops byte increment, local-rebroadcast counter for loop break. - flows/send-propagated-lxmf.md: PROPAGATED method end to end. LXMessage.pack with body encrypted to recipient (propagation node never decrypts), Link establishment to the propagation node, optional propagation stamp (1000 PoW rounds vs 3000 for regular stamps), submission via Link DATA or Resource, state goes to SENT (not DELIVERED — recipient pulls via /get later per §5.8.3). flows/README.md status table updated; receive-propagated-lxmf.md added as the only remaining ⏳ flow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:21:05 -04:00
# Flow: forward an announce (transport-node rebroadcast)
Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last release that is also published to GitHub" — pip continues until rnpkg is complete and RNS is self-hosting. All 13 verifiers pass against 1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed between 1.2.0 and 1.2.4, so the changes here are purely currency: - Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the verifier stays reproducible if upstream stops mirroring to PyPI before the migration is ready. - Add an "Upstream distribution shift" watch-list to todo.md (local Reticulum node, repo destination hash, rnpkg install/upgrade commands, rsg signature verification, mirroring source citations). - Bump SPEC.md frontmatter and re-anchor ~50 line citations across Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py, Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13 to +31 lines; Transport.py was variable). Fix one numeric (MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU clamp citation pointed at the wrong location — corrected to point at the transit-relay clamp at Transport.py:1539-1556). - Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap adoption, with citations to Resource.py:686-691 and Buffer.py:95-97 plus a "do not use one-shot bz2.decompress()" warning. - Re-anchor 11 flows/ files (version pins + ~30 line citations). - Bump version labels in tools/README.md, test-vectors/README.md, and 4 verifier docstrings + 2 hardcoded print strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:42:25 -04:00
What a transport-mode node does when it receives an inbound announce destined for a non-local destination. This is the flow that makes the mesh actually mesh — without it, announces never propagate beyond direct radio range. Pinned against **RNS 1.2.4**; cross-references [`../SPEC.md`](../SPEC.md) §4.5 (validation), §12.3 (rebroadcast rules), §12.4 (path table).
Add five companion flow docs - flows/receive-resource.md: inverse of send-resource. ADV ingestion, accept/reject decision, request_next loop, receive_part insertion, assemble + decrypt + hash-validate, RESOURCE_PRF emission, multi-segment continuation. - flows/receive-link-lxmf.md: responder side of the link handshake plus inbound LXMF DATA handling. validate_request -> handshake -> prove (LRPROOF emission) -> link_established callback wires delivery_packet. PACKET-form inbound runs delivery_packet directly; RESOURCE-form inbound runs through delivery_resource_advertised + delivery_resource_concluded pipeline. - flows/send-announce.md: random_hash construction (5B random + 5B BE-uint40 timestamp), optional ratchet rotation, signed_data assembly, sign + pack, the broadcast emission. Notes that ANNOUNCE packets are NOT encrypted (Packet.pack special-cases line 189-191) and the periodic re-announce loop drives 5-15min cadence. - flows/forward-announce.md: relay-side rebroadcast for transport-mode nodes. Eligibility checks (transport_enabled, not PATH_RESPONSE, not rate_blocked), announce_table queue, Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s, per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime enforcement, lowest-hop-count-first emission order, hops byte increment, local-rebroadcast counter for loop break. - flows/send-propagated-lxmf.md: PROPAGATED method end to end. LXMessage.pack with body encrypted to recipient (propagation node never decrypts), Link establishment to the propagation node, optional propagation stamp (1000 PoW rounds vs 3000 for regular stamps), submission via Link DATA or Resource, state goes to SENT (not DELIVERED — recipient pulls via /get later per §5.8.3). flows/README.md status table updated; receive-propagated-lxmf.md added as the only remaining ⏳ flow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:21:05 -04:00
This flow only runs on a node with `enable_transport = Yes` per §12.1. Leaf clients can ignore it entirely.
---
## Sequence
### 1. Inbound announce passes validation
The receive-announce flow ([`receive-announce.md`](receive-announce.md)) runs first — signature check, dest_hash recompute, public-key collision check, ingress rate limit, path table population. By the time the rebroadcast logic runs, the announce is known-valid and the path_table entry is already updated.
### 2. Eligibility checks
Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last release that is also published to GitHub" — pip continues until rnpkg is complete and RNS is self-hosting. All 13 verifiers pass against 1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed between 1.2.0 and 1.2.4, so the changes here are purely currency: - Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the verifier stays reproducible if upstream stops mirroring to PyPI before the migration is ready. - Add an "Upstream distribution shift" watch-list to todo.md (local Reticulum node, repo destination hash, rnpkg install/upgrade commands, rsg signature verification, mirroring source citations). - Bump SPEC.md frontmatter and re-anchor ~50 line citations across Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py, Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13 to +31 lines; Transport.py was variable). Fix one numeric (MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU clamp citation pointed at the wrong location — corrected to point at the transit-relay clamp at Transport.py:1539-1556). - Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap adoption, with citations to Resource.py:686-691 and Buffer.py:95-97 plus a "do not use one-shot bz2.decompress()" warning. - Re-anchor 11 flows/ files (version pins + ~30 line citations). - Bump version labels in tools/README.md, test-vectors/README.md, and 4 verifier docstrings + 2 hardcoded print strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:42:25 -04:00
`RNS/Transport.py:1825`. Three conditions all must hold:
Add five companion flow docs - flows/receive-resource.md: inverse of send-resource. ADV ingestion, accept/reject decision, request_next loop, receive_part insertion, assemble + decrypt + hash-validate, RESOURCE_PRF emission, multi-segment continuation. - flows/receive-link-lxmf.md: responder side of the link handshake plus inbound LXMF DATA handling. validate_request -> handshake -> prove (LRPROOF emission) -> link_established callback wires delivery_packet. PACKET-form inbound runs delivery_packet directly; RESOURCE-form inbound runs through delivery_resource_advertised + delivery_resource_concluded pipeline. - flows/send-announce.md: random_hash construction (5B random + 5B BE-uint40 timestamp), optional ratchet rotation, signed_data assembly, sign + pack, the broadcast emission. Notes that ANNOUNCE packets are NOT encrypted (Packet.pack special-cases line 189-191) and the periodic re-announce loop drives 5-15min cadence. - flows/forward-announce.md: relay-side rebroadcast for transport-mode nodes. Eligibility checks (transport_enabled, not PATH_RESPONSE, not rate_blocked), announce_table queue, Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s, per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime enforcement, lowest-hop-count-first emission order, hops byte increment, local-rebroadcast counter for loop break. - flows/send-propagated-lxmf.md: PROPAGATED method end to end. LXMessage.pack with body encrypted to recipient (propagation node never decrypts), Link establishment to the propagation node, optional propagation stamp (1000 PoW rounds vs 3000 for regular stamps), submission via Link DATA or Resource, state goes to SENT (not DELIVERED — recipient pulls via /get later per §5.8.3). flows/README.md status table updated; receive-propagated-lxmf.md added as the only remaining ⏳ flow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:21:05 -04:00
```python
if (Reticulum.transport_enabled() or is_from_local_client) \
and packet.context != PATH_RESPONSE \
and not rate_blocked:
# rebroadcast
```
- `transport_enabled OR is_from_local_client` — leaf clients only forward announces that came from a local-client interface (the rare "client behind shared rnsd" case).
- `packet.context != PATH_RESPONSE` — path-response announces are NOT rebroadcast (they go on a single specific interface back to the requester per [`path-discovery.md`](path-discovery.md) step 7).
- `not rate_blocked` — per-interface announce-rate limits aren't tripped for this destination.
### 3. Insert into `announce_table`
`Transport.py:1833-1844`. The relay adds an entry:
```python
Transport.announce_table[packet.destination_hash] = [
now, # 0 IDX_AT_TIMESTAMP
retransmit_timeout, # 1 IDX_AT_RTRNS_TMO — when to actually emit
retries, # 2 IDX_AT_RETRIES — PATHFINDER_R, default 4
received_from, # 3 IDX_AT_RCVD_IF — interface NOT to rebroadcast on
announce_hops, # 4 IDX_AT_HOPS
packet, # 5 IDX_AT_PACKET — full Packet object
local_rebroadcasts, # 6 IDX_AT_LCL_RBRD — count of times peers retransmitted
block_rebroadcasts, # 7 IDX_AT_BLCK_RBRD — true if a peer beat us to it
attached_interface, # 8 IDX_AT_ATTCHD_IF
]
```
`retransmit_timeout` for non-local-client originated announces is `now + PATH_REQUEST_GRACE = 0.4s` (giving directly-reachable peers time to rebroadcast first; if they do, `block_rebroadcasts = True` is set and we suppress our own emission). Local-client originated announces fire `now` with no grace.
### 4. Periodic `Transport.jobs` walk drains the table
The relay's per-second-or-so housekeeping loop walks `announce_table` and for entries whose `retransmit_timeout <= now`, queues them for emission on each suitable interface:
```python
# Pseudocode of the relevant Transport.jobs branch
for dest_hash, entry in announce_table.items():
if entry[BLCK_RBRD]:
continue # peer already rebroadcast — drop
if now < entry[RTRNS_TMO]:
continue # not yet time
if entry[RETRIES] <= 0:
announce_table.pop(dest_hash) # exhausted
continue
for interface in interfaces:
if interface == entry[RCVD_IF]:
continue # don't re-emit on receive interface
interface.announce_queue.append({"raw": entry[PACKET].raw, ...})
entry[RETRIES] -= 1
```
The actual code is in `Transport.py:1196-1300, 1810-1969`; the structure above is a simplification for the spec.
### 5. Per-interface `announce_queue` drain
Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last release that is also published to GitHub" — pip continues until rnpkg is complete and RNS is self-hosting. All 13 verifiers pass against 1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed between 1.2.0 and 1.2.4, so the changes here are purely currency: - Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the verifier stays reproducible if upstream stops mirroring to PyPI before the migration is ready. - Add an "Upstream distribution shift" watch-list to todo.md (local Reticulum node, repo destination hash, rnpkg install/upgrade commands, rsg signature verification, mirroring source citations). - Bump SPEC.md frontmatter and re-anchor ~50 line citations across Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py, Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13 to +31 lines; Transport.py was variable). Fix one numeric (MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU clamp citation pointed at the wrong location — corrected to point at the transit-relay clamp at Transport.py:1539-1556). - Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap adoption, with citations to Resource.py:686-691 and Buffer.py:95-97 plus a "do not use one-shot bz2.decompress()" warning. - Re-anchor 11 flows/ files (version pins + ~30 line citations). - Bump version labels in tools/README.md, test-vectors/README.md, and 4 verifier docstrings + 2 hardcoded print strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:42:25 -04:00
Each interface independently throttles its outbound announces against `interface.announce_cap` (default `Reticulum.ANNOUNCE_CAP = 2.0` = 2% airtime). `Interface.process_announce_queue` (`RNS/Interfaces/Interface.py:237-272`) drains the queue at a rate the cap permits, **picking the lowest-hop-count entry first** so closer destinations propagate before further ones:
Add five companion flow docs - flows/receive-resource.md: inverse of send-resource. ADV ingestion, accept/reject decision, request_next loop, receive_part insertion, assemble + decrypt + hash-validate, RESOURCE_PRF emission, multi-segment continuation. - flows/receive-link-lxmf.md: responder side of the link handshake plus inbound LXMF DATA handling. validate_request -> handshake -> prove (LRPROOF emission) -> link_established callback wires delivery_packet. PACKET-form inbound runs delivery_packet directly; RESOURCE-form inbound runs through delivery_resource_advertised + delivery_resource_concluded pipeline. - flows/send-announce.md: random_hash construction (5B random + 5B BE-uint40 timestamp), optional ratchet rotation, signed_data assembly, sign + pack, the broadcast emission. Notes that ANNOUNCE packets are NOT encrypted (Packet.pack special-cases line 189-191) and the periodic re-announce loop drives 5-15min cadence. - flows/forward-announce.md: relay-side rebroadcast for transport-mode nodes. Eligibility checks (transport_enabled, not PATH_RESPONSE, not rate_blocked), announce_table queue, Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s, per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime enforcement, lowest-hop-count-first emission order, hops byte increment, local-rebroadcast counter for loop break. - flows/send-propagated-lxmf.md: PROPAGATED method end to end. LXMessage.pack with body encrypted to recipient (propagation node never decrypts), Link establishment to the propagation node, optional propagation stamp (1000 PoW rounds vs 3000 for regular stamps), submission via Link DATA or Resource, state goes to SENT (not DELIVERED — recipient pulls via /get later per §5.8.3). flows/README.md status table updated; receive-propagated-lxmf.md added as the only remaining ⏳ flow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:21:05 -04:00
```python
min_hops = min(e["hops"] for e in self.announce_queue)
selected = sorted([e for e in self.announce_queue if e["hops"] == min_hops],
key=lambda e: e["time"])[0]
tx_time = (len(selected["raw"]) * 8) / self.bitrate
wait_time = tx_time / self.announce_cap
self.announce_allowed_at = now + wait_time
self.process_outgoing(selected["raw"])
```
The `wait_time` is what enforces the cap: on a 5kbps LoRa channel, a 200-byte announce takes 320ms airtime; with cap=2%, the next announce isn't allowed for `320ms / 0.02 = 16s`. This is what makes Reticulum's mesh well-behaved on slow shared channels.
### 6. Rebroadcast emission with hop increment
When the queue actually emits, the wire bytes are the original announce's `packet.raw` with the `hops` byte already incremented (`Transport.inbound` did this at line 1395 on receive). No re-signing — the signature in the announce body covers the original hop=0 emission, and signature validation ignores the outer hops byte (it's not in `signed_data`). What's wire-visible is the same body, the same dest_hash, the same random_hash, and a hops byte that's now (hops_received + 1).
### 7. Local-rebroadcast counter cleanup
If the relay later **hears its own rebroadcast** (a peer further along in the chain re-emitted it), `Transport.inbound` at line 1660-1668 increments `entry[IDX_AT_LCL_RBRD]`. Once `local_rebroadcasts >= LOCAL_REBROADCASTS_MAX`, the entry is removed from `announce_table`:
```python
if announce_entry[IDX_AT_LCL_RBRD] >= LOCAL_REBROADCASTS_MAX:
announce_table.pop(packet.destination_hash)
```
This is what stops the rebroadcast loop: once enough downstream peers have echoed the announce back, the relay stops trying.
### 8. `random_blob` replay defence
§4.5 step 6.3 / §12.3.2 already covers this — the relay won't even queue a rebroadcast if the inbound announce's `random_hash` is already in the cached `random_blobs` for this destination. The receive-announce flow drops it at the path-table-update stage.
---
## Wire-byte summary
The forwarded announce is wire-identical to the original, except:
- The `hops` byte is one higher.
- If the relay is between a HEADER_1-emitting originator and a destination > 1 hop away, the relay does NOT do the §2.3 conversion for ANNOUNCE packets — announces always travel HEADER_1+BROADCAST. The HEADER_2 conversion is only for DATA packets.
```
Before relay (received): After relay (emitted):
[ flags: HEADER_1+BROADCAST+SINGLE+ANNOUNCE ] [ same flags ]
[ hops = N ] [ hops = N+1 ]
[ 16B dest_hash ] [ same dest_hash ]
[ 1B context = 0x00 ] [ same context ]
[ ... announce body identical ... ] [ ... bytes unchanged ... ]
```
---
## Source map
| Step | File | Function / line |
|---|---|---|
| 1 | (this flow follows `receive-announce.md` step 7) | |
| 2 | `RNS/Transport.py` | rebroadcast eligibility, line 1822 |
| 3 | `RNS/Transport.py` | announce_table insert, line 1833-1844 |
| 4 | `RNS/Transport.py` | jobs / queue drain, line 1196+ |
| 5 | `RNS/Interfaces/Interface.py` | `process_announce_queue`, line 232 |
| 6 | `RNS/Transport.py` | hops increment in `inbound`, line 1395 |
| 7 | `RNS/Transport.py` | local-rebroadcast counter, line 1660-1668 |
| 8 | `RNS/Transport.py` | random_blob replay check, line 1707-1745 |