Worked for 4m 50s

Completed the destination-routed DATA and reverse-PROOF three-tier unit.

Key findings:

Intermediate relays retarget HEADER_2; the final relay delivers HEADER_1.
Each relay records independent reverse-table state.
PROOF destination hashes remain invariant across transport transformations.
A wrong-interface PROOF consumes the reverse entry before being dropped.
REVERSE_TIMEOUT is 480 seconds, not 30 seconds.
Added Tier 1 audit, two-relay flow, deterministic vectors, regenerator, and runtime verifier. Corrected affected specification, flows, playbook, and status documentation.

Verification:

Deterministic regeneration: identical SHA-256
Full pinned suite: 22 passed, 0 failed
git diff --check: passed
No commit created.
This commit is contained in:
John Poole 2026-06-08 18:32:26 -07:00
commit 9c3b628c6a
14 changed files with 578 additions and 16 deletions

View file

@ -57,6 +57,9 @@ As content grows, `SPEC.md` will be split into per-layer files (packet header, i
Errata that may invalidate code built against an earlier revision of `SPEC.md`. Newest first. Feature additions and ordinary edits live in `git log` — this section is reserved for cases where the spec said one thing, that turned out to be wrong, and an implementer who pulled the bad version needs to fix their code.
- **2026-06-08 — §12.5 ordinary DATA reverse routing and timeout.**
Earlier prose stated that reverse-table entries expire after 30 seconds and the opportunistic receive flow showed HEADER_2 arriving at a relayed endpoint. RNS 1.2.4 defines `REVERSE_TIMEOUT = 8*60`, and the final relay strips HEADER_2 before endpoint delivery. The audit also found that a wrong-interface PROOF consumes the one-shot reverse entry before being dropped. Corrected and runtime-locked by `tools/verify_transport_data.py`.
- **2026-06-08 — §12.5 transport-relayed Link forwarding gates and proof routing.**
Earlier prose implied that established-Link forwarding begins only after LRPROOF validation, that relay lookup requires `destination_type == LINK`, and that Link proofs use `reverse_table`. Upstream RNS 1.2.4 forwards matching link-id traffic based on interface and hop count without checking `IDX_LT_VALIDATED` or destination type; endpoints still require LINK destination type for active-Link DATA dispatch. Link-addressed proofs use `link_table`, while `reverse_table` is for ordinary destination-routed DATA proofs. Corrected and runtime-locked by `tools/verify_transport_link.py`.

View file

@ -3252,7 +3252,7 @@ For any other forwarded DATA (the much-more-common opportunistic LXMF case), the
time.time() ] # 2 IDX_RT_TIMESTAMP
```
The reverse_table is what lets the eventual PROOF receipt (§6.5) trace its way back to the originator without consulting the path_table again — see §12.5.
The reverse_table is what lets the eventual PROOF receipt (§6.5) trace its way back to the originator without consulting the path_table again — see §12.5. The key is invariant across HEADER_1/HEADER_2 conversion, transport-id replacement, and hop increments because those mutable fields are excluded from `Packet.get_hashable_part()`.
### 12.3 ANNOUNCE rebroadcasting
@ -3355,7 +3355,7 @@ new_raw = packet.raw[0:1] + struct.pack("!B", packet.hops) + packet.raw[2:]
Transport.transmit(reverse_entry[IDX_RT_RCVD_IF], new_raw)
```
Reverse_table entries are popped on use (one-shot routing) and aged out by `Transport.jobs` after `Transport.REVERSE_TIMEOUT` (default `30s`). This bounds the relay's memory regardless of whether the proof ever arrives.
The pop occurs **before** the interface check. A PROOF arriving on the wrong interface is dropped but still consumes the reverse route, so a later valid copy cannot be forwarded. Reverse_table entries are otherwise aged out by `Transport.jobs` after `Transport.REVERSE_TIMEOUT = 8*60` (480 seconds in RNS 1.2.4), or when either recorded interface disappears.
### 12.6 Tunnels and shared-instance protocol
@ -3589,6 +3589,7 @@ A client running on a constrained device (less RAM, slower CPU) can scale all of
| RNode receives correctly but TX is silent | §8.4.2 — KISS configuration handshake incomplete. CMD_RADIO_STATE = 0x01 must be the LAST step | §8.4.2 |
| Received RSSI/SNR values are garbage | §8.4.5 — wrong sidecar decode. `RSSI = byte - 157`, `SNR = signed Q6.2 / 4`. Sidecar frames precede each `CMD_DATA` frame | §8.4.5 |
| Multi-hop packets arrive but local-destination packets don't | §2.3 — originator HEADER_1→HEADER_2 conversion not applied for hops > 1. Originators must do this conversion themselves when path table reports `hops > 1` | `tools/verify_packet_header.py` |
| DATA reaches destination and a PROOF is observed, but originator never receives it | §12.5.3 — inspect every reverse-table entry and PROOF ingress interface. A wrong-interface PROOF is dropped after consuming the one-shot reverse route | `tools/verify_transport_data.py` |
| Sending to multi-hop peers fails silently after path table populated | §7.6 — `TCPServerInterface.OUT` is True by default in practice (constructor's `False` is overridden at runtime). Don't waste time chasing a stuck OUT flag | §7.6 |
### LXMF specifics
@ -3730,7 +3731,7 @@ Embedded clean-room implementations need to know up front which data structures
| `Transport.path_requests` | (unbounded — one entry per recently-issued path? request) | §7.1 | Aged out at `Transport.PATH_REQUEST_GATE_TIMEOUT = 120s`. |
| `Transport.discovery_path_requests` | (unbounded) | §7.2.3, §12.6.1 | Aged out at `Transport.PATH_REQUEST_TIMEOUT = 15s`. |
| `Transport.link_table` (transit-relay link state) | (unbounded) | §12.2.4, §12.5 | One per Link the relay is forwarding for; cleared on link teardown or stale aging. |
| `Transport.reverse_table` | (unbounded) | §12.5.3 | One entry per in-flight DATA→PROOF round-trip; popped on use, aged at `Transport.REVERSE_TIMEOUT = 30s`. |
| `Transport.reverse_table` | (unbounded) | §12.5.3 | One entry per in-flight DATA→PROOF round-trip; popped before proof-interface validation, aged at `Transport.REVERSE_TIMEOUT = 480s`, or removed when a recorded interface disappears. |
| `Transport.tunnels` | (unbounded) | §12.6.2 | One per tunnel-able interface; aged at `Transport.TUNNEL_TIMEOUT`. |
| `Transport.packet_hashlist` (dedup ring) | `Transport.hashlist_maxsize = 1,000,000` | §13.4 | Half is purged on next `Transport.jobs` after the cap is hit. |
| `Transport.active_links` | (unbounded — one per active Link the node owns or relays) | §6 | |

View file

@ -170,7 +170,7 @@ Initial confidence assessment (subjective, not authoritative — re-do this audi
| §10 Resource fragmentation | Source-audited against RNS 1.2.4 and runtime-verified by `tools/verify_resource.py`, including deterministic vectors, receiver assembly/proof, multi-segment sizing, and negative cases. |
| §11 REQUEST/RESPONSE | Source-audited against RNS 1.2.4 and runtime-verified by `tools/verify_request_response.py`, including packet/Resource forms, request-ID domains, correlation, and authorization constants. |
| §18 Test vectors | Populated with identities, announces, opportunistic LXMF, Link establishment, link-delivered LXMF, Resource, and REQUEST/RESPONSE. Future work should add broader negative vectors. |
| §12 Transport | Source-audited against RNS 1.2.4. Transport-relayed Link establishment, bidirectional traffic, and failure cases are runtime-verified by `tools/verify_transport_link.py`; broader announce/path/tunnel behavior remains source-cited. |
| §12 Transport | Source-audited against RNS 1.2.4. Relayed Link behavior is runtime-verified by `tools/verify_transport_link.py`; ordinary DATA and reverse-PROOF behavior by `tools/verify_transport_data.py`. Broader announce/path/tunnel behavior remains source-cited. |
**Historical bootstrap tasks from the initial audit, now mostly complete:**

View file

@ -0,0 +1,66 @@
# Tier 1 Audit: Destination-Routed DATA and PROOF Transport
Question: How does RNS 1.2.4 forward ordinary destination-routed DATA across
multiple transport relays and route the returning PROOF?
Evidence baseline:
- RNS package: `rns==1.2.4`
- Primary sources: `RNS/Transport.py`, `RNS/Packet.py`
- Audit date: 2026-06-08
Tier 2 evidence is `tools/verify_transport_data.py` and
`test-vectors/transport-data.json`. Confirmed findings are promoted into
`SPEC.md` and `flows/transport-data.md`.
## Confirmed Model
1. An intermediate relay receiving HEADER_2 DATA addressed to its transport
identity keeps HEADER_2, increments hops, and replaces `transport_id` with
the next relay's identity hash.
2. The final relay, whose path entry reports one remaining hop, strips the
transport slot and delivers HEADER_1 toward the destination. Ordinary
routed DATA does not arrive at the endpoint as HEADER_2.
3. Every relay that forwards non-LINKREQUEST DATA writes a `reverse_table`
entry keyed by `packet.getTruncatedHash()`. This proof-destination hash is
invariant across HEADER_1/HEADER_2 conversion, transport-id replacement,
and hop increments.
4. A returning ordinary PROOF is HEADER_1 and addressed to that truncated
packet hash. Each relay pops its reverse entry and forwards the proof on
the entry's recorded receive-side interface.
5. The reverse entry is popped **before** the relay verifies that PROOF arrived
on the recorded outbound interface. A wrong-interface PROOF is dropped but
consumes the route; a later correct PROOF cannot be forwarded.
6. DATA addressed to another transport identity is rejected by
`packet_filter`. DATA addressed to the local relay but lacking a path is
dropped without creating reverse state.
7. `Transport.REVERSE_TIMEOUT` is 480 seconds in RNS 1.2.4, not 30 seconds.
Entries are also culled when either recorded interface disappears.
## Corrections to Bootstrap Prose
- The opportunistic receive flow incorrectly showed HEADER_2 arriving at the
endpoint after relay transport. The last relay strips the transport header.
- §12.5.3 and §16.1 stated `REVERSE_TIMEOUT = 30s`; upstream defines
`8*60`.
- §12.5.3 did not warn that a wrong-interface PROOF consumes reverse state.
## Tier 2 Scope
`tools/verify_transport_data.py` drives upstream `Transport.inbound` and
verifies:
1. Intermediate HEADER_2 retargeting and final HEADER_1 delivery.
2. Stable proof-destination hash across all DATA wire forms.
3. Per-relay reverse-table entry shape.
4. Two-relay PROOF return and one-shot entry consumption.
5. Wrong-interface PROOF drop plus reverse-entry consumption.
6. Unknown-path and wrong-transport-identity DATA drops.
7. The RNS 1.2.4 reverse timeout constant.

View file

@ -22,6 +22,8 @@ The two views are complementary: SPEC.md tells you what each piece looks like; t
| [`receive-propagated-lxmf.md`](receive-propagated-lxmf.md) (recipient pulling messages via `/get`) | ✅ |
| [`propagation-peer-sync.md`](propagation-peer-sync.md) (propagation-node announce, `/offer`, and peer Resource sync) | ✅ |
| [`lxmf-outbound-retry.md`](lxmf-outbound-retry.md) (process_outbound retry loop, per-message state machine, fail_message) | ✅ |
| [`transport-link.md`](transport-link.md) (relayed Link establishment and established-Link traffic) | ✅ |
| [`transport-data.md`](transport-data.md) (ordinary DATA and returning PROOF through relays) | ✅ |
## Conventions

View file

@ -195,20 +195,18 @@ SPEC.md §9.5: if the operator runs both an originator and a transport node on t
## Wire-byte summary (mirror of the send-flow summary)
What arrives at the recipient before deframing — assumes a 0-hop direct send (HEADER_1) or a >1-hop relayed packet that arrives as HEADER_2 with a transport_id at offset 2:
What arrives at the recipient before deframing is HEADER_1 whether sent directly or delivered by a last-hop transport relay. Intermediate relays use HEADER_2, but the final relay strips the transport-id slot before transmission:
```
HEADER_1:
[ 1B flags ][ 1B hops ][ 16B dest_hash ][ 1B context=0x00 ]
[ 32B sender_ephemeral_X25519_pub ][ 16B iv ]
[ N×16B aes_ciphertext ][ 32B hmac_sha256 ]
HEADER_2 (after relay):
[ 1B flags ][ 1B hops ][ 16B transport_id ][ 16B dest_hash ][ 1B context=0x00 ]
[ 32B sender_ephemeral_X25519_pub ][ 16B iv ]
[ N×16B aes_ciphertext ][ 32B hmac_sha256 ]
```
See [`transport-data.md`](transport-data.md) for the intermediate HEADER_2
forms and reverse-PROOF path.
After AES-CBC decryption with the matching ratchet (or long-term) key, the plaintext is the opportunistic LXMF body **without** the recipient's dest_hash (SPEC.md §5.1):
```

View file

@ -142,7 +142,7 @@ Strictly speaking, the flow above ends at step 10. Steps 11-13 are about **what
### 11. Recipient processes the inbound DATA packet
Inverse of steps 7-9, in the order: deframe → optional HEADER_2 strip / hop-table lookup → packet enters `Transport.inbound` → handed to the destination → `RNS.Destination.decrypt` reverses the Token (HMAC verified **before** AES decrypt per SPEC.md §3.3) → LXMF body parsed → Ed25519 signature verified, with the dual-msgpack-variant tolerance described in SPEC.md §5.6 → message surfaced to the recipient's app.
Inverse of steps 7-9, in the order: intermediate relays process HEADER_2 via their path tables → the last relay strips the transport slot and emits HEADER_1 → recipient deframes → packet enters `Transport.inbound` → handed to the destination → `RNS.Destination.decrypt` reverses the Token (HMAC verified **before** AES decrypt per SPEC.md §3.3) → LXMF body parsed → Ed25519 signature verified, with the dual-msgpack-variant tolerance described in SPEC.md §5.6 → message surfaced to the recipient's app. See `transport-data.md` for the relay sequence.
The receive flow is its own document; see `receive-opportunistic-lxmf.md` (TODO) for the detailed step list.

46
flows/transport-data.md Normal file
View file

@ -0,0 +1,46 @@
# Flow: Destination-Routed DATA Through Transport Relays
This flow follows ordinary DATA and its returning PROOF across two transport
relays, pinned to RNS 1.2.4. Opportunistic LXMF is the common example.
## Forward DATA
1. The originator emits HEADER_2 because its path has more than one hop. The
`transport_id` names relay 1; the final destination hash remains in the
second address slot.
2. Relay 1 verifies that `transport_id` names itself and finds the final
destination in `path_table`.
3. With more than one remaining hop, relay 1 increments hops, replaces
`transport_id` with relay 2's identity hash, and preserves HEADER_2.
4. Relay 1 records `reverse_table[packet.getTruncatedHash()]` with the ingress
and egress interfaces.
5. Relay 2 receives the retargeted HEADER_2 packet. With one remaining hop, it
increments hops, strips the transport slot, and emits HEADER_1 toward the
destination.
6. Relay 2 records its own reverse entry under the same truncated packet hash.
The hash is stable because `Packet.get_hashable_part()` excludes hops,
transport type/header type, and the HEADER_2 transport-id slot.
## Returning PROOF
1. The destination emits a HEADER_1 PROOF addressed to the original DATA
packet's truncated hash.
2. Relay 2 pops that hash from `reverse_table`. If the PROOF arrived on the
recorded outbound interface, it increments hops and sends toward relay 1.
3. Relay 1 repeats the operation and sends toward the originator.
4. Each entry is one-shot. A duplicate PROOF has no route.
## Failure Diagnostics
| Observation | Meaning |
|---|---|
| First relay receives DATA but does not forward it | Check HEADER_2 `transport_id` and relay `path_table`. |
| Intermediate relay forwards HEADER_2 but final endpoint parser expects HEADER_2 | Endpoint receives HEADER_1; the last relay strips the transport slot. |
| DATA reaches endpoint but no PROOF returns | Inspect every relay's reverse entry and recorded interfaces. |
| PROOF appears first on the wrong interface | Upstream drops it after popping the reverse entry; a later correct copy cannot return. |
| Reverse entry remains indefinitely in a clean-room relay | RNS 1.2.4 expires it after 480 seconds or when a recorded interface disappears. |
Executable evidence: `tools/verify_transport_data.py` and
`test-vectors/transport-data.json`.

View file

@ -247,6 +247,13 @@ Each entry: date, one-line symptom, spec section that governs it, one-line fix,
- **Fix:** `useHeader2 = dest.hopCount > 1 && dest.nextHop != null`. Build the packet with `headerType = HEADER_2` and `transportId = dest.nextHop`.
- **Lesson:** "It works at one hop" is not "it works." Test multi-hop early.
### 2026-06 — Wrong-interface PROOF consumes the reverse route
- **Symptom:** DATA reaches the destination, a PROOF is observed, but the originator never receives it; later valid PROOF copies also fail.
- **Spec section:** §12.5.3. Upstream pops `reverse_table[proof_dest_hash]` before checking the PROOF's receiving interface.
- **Fix:** Prevent wrong-interface proof injection and log reverse-table creation, ingress, pop, and forwarding as one diagnostic sequence.
- **Lesson:** A dropped PROOF can mutate relay state; observing a later correct packet is not enough.
(Older entries: see `agent.md` §5 audit table and `reticulum-mobile-app/CLAUDE.md` "Key bugs we found" for additional history.)
---

View file

@ -16,8 +16,9 @@ Populated against RNS 1.2.4 / LXMF 0.9.7:
- ✅ `propagated-lxmf.json` — PROPAGATED LXMF submission packet/Resource boundary, full transient IDs, and `/get` framing (regenerator: `../tools/regen_propagated_lxmf.py`, verifier: `../tools/verify_propagated_lxmf.py`).
- ✅ `propagation-peer.json` — propagation-node announce, directional peering key, `/offer`, and peer-sync Resource plaintext (regenerator: `../tools/regen_propagation_peer.py`, verifier: `../tools/verify_propagation_peer.py`).
- ✅ `transport-link.json` — one-relay LINKREQUEST, LRPROOF, established-Link DATA, and invalid HEADER_2 fixtures (regenerator: `../tools/regen_transport_link.py`, verifier: `../tools/verify_transport_link.py`).
- ✅ `transport-data.json` — two-relay ordinary DATA and returning reverse-table PROOF fixtures (regenerator: `../tools/regen_transport_data.py`, verifier: `../tools/verify_transport_data.py`).
All ten files are byte-deterministic across runs: regenerators pin every random source (ephemeral keys, IVs, `random_hash` values, timestamps) so the output is reproducible against a fixed upstream RNS / LXMF version.
All eleven files are byte-deterministic across runs: regenerators pin every random source (ephemeral keys, IVs, `random_hash` values, timestamps) so the output is reproducible against a fixed upstream RNS / LXMF version.
See [`../agent.md`](../agent.md) §3 and [`../todo.md`](../todo.md) for the evidence model and remaining task list.
@ -35,6 +36,7 @@ Each vector lives in a per-domain JSON file, e.g.:
- `propagated-lxmf.json` — PROPAGATED submission and `/get` response forms
- `propagation-peer.json` — propagation-node announce and peer-sync forms
- `transport-link.json` — transport-relayed Link establishment and traffic forms
- `transport-data.json` — destination-routed DATA and reverse-PROOF forms
Each entry should include:
@ -65,5 +67,6 @@ For the spec to claim "an implementation that passes all test vectors interopera
9. **Propagated LXMF** — recipient-encrypted submission, full transient-ID derivation, packet/Resource selection, and `/get` framing.
10. **Propagation peer sync** — node announce, directional peering key, `/offer` selection, and stamped Resource transfer.
11. **Transport-relayed Link** — LINKREQUEST/LRPROOF relay state, bidirectional established-Link forwarding, and invalid-header failure behavior.
12. **Transport-relayed DATA** — multi-relay HEADER_2/HEADER_1 transitions, reverse-table state, and returning proofs.
A separate vector set for FAILURE cases is also useful: malformed announces, expired ratchets, mismatched signatures. An implementation should reject those as a regression-prevention measure.

View file

@ -0,0 +1,30 @@
{
"_about": "Two-relay ordinary DATA and reverse-table PROOF fixtures derived from the deterministic opportunistic LXMF vector.",
"inputs": {
"lxmf_vector_label": "alice_to_bob_simple",
"relay_one_identity_label": "alice",
"relay_one_identity_hash_hex": "28d43a11abc1094301a59ed3b44f127b",
"relay_two_identity_label": "bob",
"relay_two_identity_hash_hex": "c090410e5b5bf8956194c1872dccec3b"
},
"expected": {
"destination_hash_hex": "9695d17f22fa6e45d2b0cd3439a7ca7e",
"proof_destination_hash_hex": "15042202b08d6529fcb2c222e9dff137",
"origin_header1_data_raw_hex": "00009695d17f22fa6e45d2b0cd3439a7ca7e0021c3332b61be6a7b6ab8461e155651b17501b6e07532ecf9ab6661bd5a2ca57511223344556677889900aabbccddeeffefa9d24b76e1adf393cfd588214e236e219743697af96912b8eae84f3a1f28a3d68abd62f3e42c6944015c3d00e5e7aa8af732123d079ab10353597669c8cd3ba57cfae3a28ea1a99a44e0b492ba5deedd23232d2edab78fa037967757808c8578496aee7b21c70ce2476c54540d96d928e8ddf35c6bfb5d76261c07f1bb48af9d7bec8261cd30f3b03986614ba93173",
"relay_one_inbound_header2_data_raw_hex": "500028d43a11abc1094301a59ed3b44f127b9695d17f22fa6e45d2b0cd3439a7ca7e0021c3332b61be6a7b6ab8461e155651b17501b6e07532ecf9ab6661bd5a2ca57511223344556677889900aabbccddeeffefa9d24b76e1adf393cfd588214e236e219743697af96912b8eae84f3a1f28a3d68abd62f3e42c6944015c3d00e5e7aa8af732123d079ab10353597669c8cd3ba57cfae3a28ea1a99a44e0b492ba5deedd23232d2edab78fa037967757808c8578496aee7b21c70ce2476c54540d96d928e8ddf35c6bfb5d76261c07f1bb48af9d7bec8261cd30f3b03986614ba93173",
"relay_one_forwarded_header2_data_raw_hex": "5001c090410e5b5bf8956194c1872dccec3b9695d17f22fa6e45d2b0cd3439a7ca7e0021c3332b61be6a7b6ab8461e155651b17501b6e07532ecf9ab6661bd5a2ca57511223344556677889900aabbccddeeffefa9d24b76e1adf393cfd588214e236e219743697af96912b8eae84f3a1f28a3d68abd62f3e42c6944015c3d00e5e7aa8af732123d079ab10353597669c8cd3ba57cfae3a28ea1a99a44e0b492ba5deedd23232d2edab78fa037967757808c8578496aee7b21c70ce2476c54540d96d928e8ddf35c6bfb5d76261c07f1bb48af9d7bec8261cd30f3b03986614ba93173",
"relay_two_delivered_header1_data_raw_hex": "00029695d17f22fa6e45d2b0cd3439a7ca7e0021c3332b61be6a7b6ab8461e155651b17501b6e07532ecf9ab6661bd5a2ca57511223344556677889900aabbccddeeffefa9d24b76e1adf393cfd588214e236e219743697af96912b8eae84f3a1f28a3d68abd62f3e42c6944015c3d00e5e7aa8af732123d079ab10353597669c8cd3ba57cfae3a28ea1a99a44e0b492ba5deedd23232d2edab78fa037967757808c8578496aee7b21c70ce2476c54540d96d928e8ddf35c6bfb5d76261c07f1bb48af9d7bec8261cd30f3b03986614ba93173",
"destination_proof_raw_hex": "030015042202b08d6529fcb2c222e9dff13700abababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababab",
"relay_two_forwarded_proof_raw_hex": "030115042202b08d6529fcb2c222e9dff13700abababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababab",
"relay_one_forwarded_proof_raw_hex": "030215042202b08d6529fcb2c222e9dff13700abababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababab"
},
"rns_version_at_generation": "1.2.4",
"lxmf_version_at_generation": "0.9.7",
"generator_script": "tools/regen_transport_data.py",
"verifies_spec_sections": [
"2.3",
"6.5",
"12.2",
"12.5.3"
]
}

15
todo.md
View file

@ -80,6 +80,13 @@ Outstanding work for the spec repo.
admission, and throttle timing, and adds
`flows/propagation-peer-sync.md`.
- [x] **Transport DATA and reverse-PROOF three-tier work unit.** Tier 1 is
recorded in `audits/transport-data-tier1-rns-1.2.4.md`; Tier 2 adds
`test-vectors/transport-data.json`, `tools/regen_transport_data.py`, and
`tools/verify_transport_data.py`; Tier 3 corrects final-hop HEADER_1
delivery, the 480-second reverse timeout, and wrong-interface PROOF
route consumption, and adds `flows/transport-data.md`.
## Open `⚠️ UNVERIFIED` items in SPEC.md
These need either a runtime test or a stronger upstream source citation
@ -316,10 +323,10 @@ discovery_path_requests):
that survives interface flap, and the shared-instance wire
protocol (just regular Reticulum packets over a TCP loopback;
what's "shared" is the Transport state, not the wire format).
- [x] **Reverse-table link transport** — §12.5 covers LRPROOF
forwarding via link_table, Link DATA forwarding in both
directions once the link_table entry is validated, and PROOF
receipt forwarding via reverse_table (one-shot pop on use).
- [x] **Link-table and reverse-table transport** — §12.5 covers LRPROOF
forwarding and established-Link traffic via `link_table`, plus ordinary
DATA proof forwarding via `reverse_table`. Focused runtime verifiers are
`tools/verify_transport_link.py` and `tools/verify_transport_data.py`.
## Developer-experience gaps (would save real implementers real time)

View file

@ -0,0 +1,111 @@
"""
Regenerator for test-vectors/transport-data.json.
Builds a deterministic two-relay opportunistic DATA and returning PROOF path
from the existing LXMF and identity vectors.
"""
from __future__ import annotations
import json
import os
import RNS
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
LXMF_PATH = os.path.join(REPO_ROOT, "test-vectors", "lxmf.json")
IDS_PATH = os.path.join(REPO_ROOT, "test-vectors", "identities.json")
OUT_PATH = os.path.join(REPO_ROOT, "test-vectors", "transport-data.json")
def load_json(path: str):
with open(path, "r", encoding="utf-8") as input_file:
return json.load(input_file)
def header_two(raw: bytes, transport_id: bytes) -> bytes:
flags = (
(RNS.Packet.HEADER_2 << 6)
| (RNS.Transport.TRANSPORT << 4)
| (raw[0] & 0x0F)
)
return bytes([flags, raw[1]]) + transport_id + raw[2:]
def packet(raw: bytes) -> RNS.Packet:
parsed = RNS.Packet(None, raw)
if not parsed.unpack():
raise RuntimeError("generated invalid packet")
return parsed
def main() -> None:
lxmf = load_json(LXMF_PATH)["vectors"][0]
identities = load_json(IDS_PATH)["vectors"]
alice = next(item for item in identities if item["label"] == "alice")
bob = next(item for item in identities if item["label"] == "bob")
relay_one = bytes.fromhex(alice["expected"]["identity_hash_hex"])
relay_two = bytes.fromhex(bob["expected"]["identity_hash_hex"])
destination_hash = bytes.fromhex(
lxmf["expected"]["fields_layout"]["destination_hash_hex"]
)
ciphertext = bytes.fromhex(lxmf["expected"]["token_ciphertext_hex"])
header_one_data = (
bytes([(RNS.Destination.SINGLE << 2) | RNS.Packet.DATA, 0])
+ destination_hash
+ bytes([RNS.Packet.NONE])
+ ciphertext
)
inbound_relay_one = header_two(header_one_data, relay_one)
forwarded_relay_one = (
inbound_relay_one[:1] + b"\x01" + relay_two + inbound_relay_one[18:]
)
delivered_header_one = header_one_data[:1] + b"\x02" + header_one_data[2:]
proof_destination = packet(header_one_data).getTruncatedHash()
implicit_proof_body = bytes.fromhex("ab" * 64)
proof = (
bytes([(RNS.Destination.SINGLE << 2) | RNS.Packet.PROOF, 0])
+ proof_destination
+ bytes([RNS.Packet.NONE])
+ implicit_proof_body
)
vector = {
"_about": (
"Two-relay ordinary DATA and reverse-table PROOF fixtures derived "
"from the deterministic opportunistic LXMF vector."
),
"inputs": {
"lxmf_vector_label": lxmf["label"],
"relay_one_identity_label": "alice",
"relay_one_identity_hash_hex": relay_one.hex(),
"relay_two_identity_label": "bob",
"relay_two_identity_hash_hex": relay_two.hex(),
},
"expected": {
"destination_hash_hex": destination_hash.hex(),
"proof_destination_hash_hex": proof_destination.hex(),
"origin_header1_data_raw_hex": header_one_data.hex(),
"relay_one_inbound_header2_data_raw_hex": inbound_relay_one.hex(),
"relay_one_forwarded_header2_data_raw_hex": forwarded_relay_one.hex(),
"relay_two_delivered_header1_data_raw_hex": delivered_header_one.hex(),
"destination_proof_raw_hex": proof.hex(),
"relay_two_forwarded_proof_raw_hex": (proof[:1] + b"\x01" + proof[2:]).hex(),
"relay_one_forwarded_proof_raw_hex": (proof[:1] + b"\x02" + proof[2:]).hex(),
},
"rns_version_at_generation": RNS.__version__,
"lxmf_version_at_generation": "0.9.7",
"generator_script": "tools/regen_transport_data.py",
"verifies_spec_sections": ["2.3", "6.5", "12.2", "12.5.3"],
}
with open(OUT_PATH, "w", encoding="utf-8") as output_file:
json.dump(vector, output_file, indent=2)
output_file.write("\n")
print(f"Wrote {OUT_PATH}")
if __name__ == "__main__":
main()

View file

@ -0,0 +1,288 @@
"""
Verifier for ordinary DATA and PROOF routing through transport relays.
Exercises a deterministic two-relay path against upstream RNS 1.2.4
Transport.inbound. This distinguishes path_table forwarding from link_table
forwarding and verifies the one-shot reverse_table return path.
"""
from __future__ import annotations
import json
import os
import sys
import tempfile
import time
import RNS
from RNS import Transport
from RNS.Transport import IDX_RT_OUTB_IF, IDX_RT_RCVD_IF, IDX_RT_TIMESTAMP
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
VECTORS_PATH = os.path.join(REPO_ROOT, "test-vectors", "transport-data.json")
IDS_PATH = os.path.join(REPO_ROOT, "test-vectors", "identities.json")
def fail(message: str) -> None:
print(f"FAIL: {message}")
sys.exit(1)
def load_json(path: str):
with open(path, "r", encoding="utf-8") as input_file:
return json.load(input_file)
def init_minimal_rns():
config_dir = tempfile.mkdtemp(prefix="rns-verify-transport-data-")
config_path = os.path.join(config_dir, "config")
with open(config_path, "w", encoding="utf-8") as config:
config.write("[reticulum]\nenable_transport = No\nshare_instance = No\n")
return RNS.Reticulum(configdir=config_dir, loglevel=0)
class FakeInterface:
OUT = True
IN = True
HW_MTU = RNS.Reticulum.MTU
AUTOCONFIGURE_MTU = True
FIXED_MTU = True
bitrate = 1_000_000
mode = RNS.Interfaces.Interface.Interface.MODE_FULL
def __init__(self, name: str):
self.name = name
def __str__(self):
return self.name
def clear_state() -> None:
Transport.packet_hashlist = set()
Transport.packet_hashlist_prev = set()
Transport.path_table.clear()
Transport.reverse_table.clear()
Transport.link_table.clear()
Transport.destinations_map.clear()
Transport.receipts.clear()
Transport.local_client_interfaces.clear()
def seed_path(destination_hash: bytes, next_hop: bytes, hops: int, outbound_if) -> None:
Transport.path_table[destination_hash] = [
time.time(), next_hop, hops, time.time() + 60, [], outbound_if, None,
]
def expect_forward(captured: list, interface, raw: bytes, label: str) -> None:
if len(captured) != 1:
fail(f"{label}: expected one forwarded packet, got {len(captured)}")
if captured[0][0] is not interface:
fail(f"{label}: wrong outbound interface")
if captured[0][1] != raw:
fail(f"{label}: forwarded bytes mismatch")
def packet(raw: bytes) -> RNS.Packet:
parsed = RNS.Packet(None, raw)
if not parsed.unpack():
fail("could not unpack fixture")
return parsed
def check_reverse_entry(entry: list, ingress, egress, label: str) -> None:
if entry[IDX_RT_RCVD_IF] is not ingress or entry[IDX_RT_OUTB_IF] is not egress:
fail(f"{label}: reverse-table interfaces mismatch")
if entry[IDX_RT_TIMESTAMP] > time.time():
fail(f"{label}: reverse-table timestamp is in the future")
def verify_data_path(vector: dict, relay_one_identity, relay_two_identity,
origin_if, between_if, destination_if, captured: list) -> tuple[list, list]:
expected = vector["expected"]
destination_hash = bytes.fromhex(expected["destination_hash_hex"])
proof_hash = bytes.fromhex(expected["proof_destination_hash_hex"])
Transport.identity = relay_one_identity
seed_path(destination_hash, relay_two_identity.hash, 2, between_if)
Transport.inbound(
bytes.fromhex(expected["relay_one_inbound_header2_data_raw_hex"]), origin_if
)
expect_forward(
captured, between_if,
bytes.fromhex(expected["relay_one_forwarded_header2_data_raw_hex"]),
"intermediate relay DATA",
)
if proof_hash not in Transport.reverse_table:
fail("intermediate relay did not create reverse-table entry")
relay_one_reverse = Transport.reverse_table[proof_hash]
check_reverse_entry(relay_one_reverse, origin_if, between_if, "intermediate relay")
captured.clear()
Transport.packet_hashlist.clear()
Transport.packet_hashlist_prev.clear()
Transport.path_table.clear()
Transport.reverse_table.clear()
Transport.identity = relay_two_identity
seed_path(destination_hash, destination_hash, 1, destination_if)
Transport.inbound(
bytes.fromhex(expected["relay_one_forwarded_header2_data_raw_hex"]), between_if
)
expect_forward(
captured, destination_if,
bytes.fromhex(expected["relay_two_delivered_header1_data_raw_hex"]),
"last-hop relay DATA",
)
if proof_hash not in Transport.reverse_table:
fail("last-hop relay did not create reverse-table entry")
relay_two_reverse = Transport.reverse_table[proof_hash]
check_reverse_entry(relay_two_reverse, between_if, destination_if, "last-hop relay")
forms = [
bytes.fromhex(expected["origin_header1_data_raw_hex"]),
bytes.fromhex(expected["relay_one_inbound_header2_data_raw_hex"]),
bytes.fromhex(expected["relay_one_forwarded_header2_data_raw_hex"]),
bytes.fromhex(expected["relay_two_delivered_header1_data_raw_hex"]),
]
hashes = {packet(raw).getTruncatedHash() for raw in forms}
if hashes != {proof_hash}:
fail("DATA proof destination changed across header/hop transformations")
print("PASS S12.2 two-relay DATA path retargets HEADER_2 then delivers HEADER_1")
print("PASS S6.5/S12.2.5 proof destination is invariant and each relay records reverse state")
return relay_one_reverse, relay_two_reverse
def verify_proof_path(vector: dict, relay_one_identity, relay_two_identity,
origin_if, between_if, destination_if, captured: list,
relay_one_reverse: list, relay_two_reverse: list) -> None:
expected = vector["expected"]
proof_hash = bytes.fromhex(expected["proof_destination_hash_hex"])
proof = bytes.fromhex(expected["destination_proof_raw_hex"])
captured.clear()
Transport.packet_hashlist.clear()
Transport.packet_hashlist_prev.clear()
Transport.reverse_table = {proof_hash: relay_two_reverse}
Transport.identity = relay_two_identity
Transport.inbound(proof, destination_if)
expect_forward(
captured, between_if,
bytes.fromhex(expected["relay_two_forwarded_proof_raw_hex"]),
"last-hop relay PROOF",
)
if proof_hash in Transport.reverse_table:
fail("last-hop relay did not consume reverse-table entry")
captured.clear()
Transport.packet_hashlist.clear()
Transport.packet_hashlist_prev.clear()
Transport.reverse_table = {proof_hash: relay_one_reverse}
Transport.identity = relay_one_identity
Transport.inbound(
bytes.fromhex(expected["relay_two_forwarded_proof_raw_hex"]), between_if
)
expect_forward(
captured, origin_if,
bytes.fromhex(expected["relay_one_forwarded_proof_raw_hex"]),
"intermediate relay PROOF",
)
if proof_hash in Transport.reverse_table:
fail("intermediate relay did not consume reverse-table entry")
wrong_if = FakeInterface("wrong-proof-interface")
captured.clear()
Transport.packet_hashlist.clear()
Transport.packet_hashlist_prev.clear()
Transport.reverse_table = {proof_hash: relay_two_reverse}
Transport.identity = relay_two_identity
Transport.inbound(proof, wrong_if)
if captured:
fail("wrong-interface PROOF was forwarded")
if proof_hash in Transport.reverse_table:
fail("wrong-interface PROOF did not consume reverse-table entry")
captured.clear()
Transport.packet_hashlist.clear()
Transport.packet_hashlist_prev.clear()
Transport.inbound(proof, destination_if)
if captured:
fail("PROOF routed after wrong-interface packet consumed reverse state")
print("PASS S12.5.3 PROOF returns through both relays and consumes one-shot reverse entries")
print("PASS S12.5.3 wrong-interface PROOF is dropped after consuming its reverse entry")
def verify_guards(vector: dict, relay_one_identity, origin_if, between_if,
captured: list) -> None:
expected = vector["expected"]
destination_hash = bytes.fromhex(expected["destination_hash_hex"])
incoming = bytes.fromhex(expected["relay_one_inbound_header2_data_raw_hex"])
captured.clear()
clear_state()
Transport.identity = relay_one_identity
Transport.inbound(incoming, origin_if)
if captured or Transport.reverse_table:
fail("DATA with no path was forwarded or recorded")
captured.clear()
clear_state()
Transport.identity = RNS.Identity()
seed_path(destination_hash, bytes(16), 1, between_if)
Transport.inbound(incoming, origin_if)
if captured or Transport.reverse_table:
fail("DATA addressed to another transport identity was forwarded or recorded")
if Transport.REVERSE_TIMEOUT != 8 * 60:
fail(f"REVERSE_TIMEOUT is {Transport.REVERSE_TIMEOUT}, expected 480 seconds")
print("PASS S12.2 unknown-path/wrong-transport DATA drops without reverse state")
print("PASS S12.5.3 RNS 1.2.4 REVERSE_TIMEOUT is 480 seconds")
def main() -> None:
print(f"verify_transport_data.py against RNS {RNS.__version__}")
init_minimal_rns()
vector = load_json(VECTORS_PATH)
identities = load_json(IDS_PATH)["vectors"]
alice = next(item for item in identities if item["label"] == "alice")
bob = next(item for item in identities if item["label"] == "bob")
relay_one_identity = RNS.Identity.from_bytes(bytes.fromhex(alice["inputs"]["private_key_hex"]))
relay_two_identity = RNS.Identity.from_bytes(bytes.fromhex(bob["inputs"]["private_key_hex"]))
origin_if = FakeInterface("origin-side")
between_if = FakeInterface("between-relays")
destination_if = FakeInterface("destination-side")
captured: list[tuple[object, bytes]] = []
original_transmit = Transport.transmit
original_transport_enabled = RNS.Reticulum.transport_enabled
original_identity = Transport.identity
try:
clear_state()
RNS.Reticulum.transport_enabled = staticmethod(lambda: True)
Transport.transmit = staticmethod(lambda interface, raw: captured.append((interface, raw)))
relay_one_reverse, relay_two_reverse = verify_data_path(
vector, relay_one_identity, relay_two_identity,
origin_if, between_if, destination_if, captured,
)
verify_proof_path(
vector, relay_one_identity, relay_two_identity,
origin_if, between_if, destination_if, captured,
relay_one_reverse, relay_two_reverse,
)
verify_guards(vector, relay_one_identity, origin_if, between_if, captured)
print("ALL PASS")
finally:
Transport.transmit = original_transmit
RNS.Reticulum.transport_enabled = original_transport_enabled
Transport.identity = original_identity
clear_state()
try:
RNS.Reticulum.exit_handler()
except Exception:
pass
if __name__ == "__main__":
main()