reticiulum-specification/todo.md

TODO

Outstanding work for the spec repo.

Outreach

• File a community-documentation issue on markqvist/Reticulum. Link this repo as a community-maintained byte-level spec. Ask whether the maintainer would like to bless / link from the official Reticulum manual. Frame it as a complement to (not a replacement for) the existing operator-focused docs.

Test infrastructure

• Bootstrap test-vectors/identities.json — Alice + Bob identities populated against RNS 1.2.0. Regenerator at tools/regen_identities.py.

• Bootstrap remaining test-vectors files (announces.json, lxmf.json, links.json) with the existing vectors from reticulum-mobile-app/reference/test-vectors.json. Convert to the proposed JSON format documented in test-vectors/README.md, adding the regenerator scripts so future contributors can verify vectors against newer upstream RNS releases.

• Write the priority verifier scripts listed in tools/README.md, in this order (highest interop value first): 1. [x] verify_destination_hash.py — pure-function check, no RNS state needed 2. [x] verify_packet_header.py — bit layout + HEADER_1/HEADER_2 round-trip + originator HEADER_1→HEADER_2 conversion 3. [ ] verify_announce_roundtrip.py — closes the SPEC.md §4 gap (partial coverage in verify_announce_app_data.py) 4. [ ] verify_token_crypto.py — closes SPEC.md §3 gap 5. [ ] verify_lxmf_opportunistic.py — closes SPEC.md §5 gap 6. [ ] verify_link_handshake.py — closes SPEC.md §6 gap 7. [x] verify_path_request.py — closes SPEC.md §7.1, §7.2 gaps 8. [ ] verify_msgpack_quirk.py — closes SPEC.md §9.3 gap

  Each verifier should remove its corresponding `⚠️ UNVERIFIED` /
  `🔮 SPECULATION` callout in `SPEC.md` (per `agent.md` §1).
  

Open ⚠️ UNVERIFIED items in SPEC.md

These need either a runtime test or a stronger upstream source citation to remove their markers:

• §2.3 Originator HEADER_1 → HEADER_2 conversion. Verified against RNS 1.2.0 by tools/verify_packet_header.py, which seeds Transport.path_table with a multi-hop entry and confirms the converted wire bytes via stubbed Transport.transmit. Citation updated to RNS/Transport.py:1074-1083.

• §4.3 The 3-element [name, stamp_cost, [capabilities]] app_data variant. Verified against LXMF 0.9.6 by tools/verify_announce_app_data.py. Finding: in this LXMF version the producer emits a 2-element form only (the supported_functionality line at LXMF/LXMRouter.py:999 is dead code); the parser is prepared for a 3-element form via compression_support_from_app_data. SPEC.md §4.3 updated to describe the actual current behavior.

• §7.1 path? always precedes LXMF DATA. Verified against LXMF 0.9.6 by tools/verify_path_request.py. Finding: the preamble fires only when not has_path() AND method is OPPORTUNISTIC; the retry path can fire a second request_path after MAX_PATHLESS_TRIES (LXMRouter.py:2571+). SPEC.md §7.1 rewritten accordingly. Also fixed a documentation bug in §1.2 (path-request name_hash column).

• §7.4 Ratchet ring count default = 8. False — actual upstream default is Destination.RATCHET_COUNT = 512 at RNS/Destination.py:85 in RNS 1.2.0, with RATCHET_INTERVAL = 30*60 (line 90) and RATCHET_EXPIRY = 60*60*24*30 (RNS/Identity.py:69). SPEC.md §7.4 corrected.

Open ⚠️ items needing a runtime verifier

• tools/verify_proof_packet.py to lock in §6.5. Run two side-by-side scenarios against upstream RNS: opportunistic DATA with use_implicit_proof = True (default) and with = False, capture the resulting PROOF packet's body length, and assert it's 64 / 96 respectively with the matching content layout. Also exercise a Link DATA proof and confirm it's always 96B regardless of the config setting. Lock in the §6.5 wire shapes.

• tools/verify_rnode_split.py to lock in §8.3. The RNode air-frame split-packet protocol is now documented in SPEC.md §8.3 against direct citations in markqvist/RNode_Firmware/Framing.h, Config.h, Utilities.h, and RNode_Firmware.ino, plus the clean-room reimplementation in thatSFguy/reticulum-lora-repeater/src/Radio.cpp. A runtime verifier would: build a 300-byte synthetic Reticulum packet, run it through a Python implementation of the TX-side header rules, and confirm the byte-level frames match what RNode_Firmware.ino:716-742 would emit (header byte high nibble random + low-nibble FLAG_SPLIT bit, both frames sharing the same header, split point at 255 bytes total per LoRa frame). RX-side verifier should drive the state-table at SPEC.md §8.3 and confirm the four reassembly cases.

• Lock in the §6.2 / §6.3 corrections with verify_link_handshake.py. The wire-byte order of the LRPROOF body (signature || responder_X25519_pub || signalling, not link_id || responder_X25519_pub || signature || signalling) and the link_id derivation offsets (N=2 for HEADER_1, N=18 for HEADER_2, not 18/34) were corrected against direct upstream source citations (RNS/Link.py:376, RNS/Packet.py:354-361) in SPEC.md §6.2/§6.3 while writing flows/send-link-lxmf.md. They are source-cited but not yet exercised by a runtime verifier. Add tools/verify_link_handshake.py that drives an upstream LINKREQUEST → LRPROOF → ACTIVE handshake and asserts byte-level layouts + link_id invariance under HEADER_1↔HEADER_2.

Spec gaps for a functional client (priority-ordered)

The items below are missing pieces that prevent a client built only from this spec (plus the existing flows/) from interoperating with upstream. Tier 1 = required to talk at all to the mesh as a leaf LXMF client. Tier 2 = required for a client that's actually useful (chat that works in the wild). Tier 3 = required to act as a transport node / relay.

Where I've already done the source reading, I've left the file/line citations inline so whoever picks the item up can start without re-research.

Tier 1 — required for a barebones leaf LXMF client to interop

• flows/receive-announce.md + SPEC.md §4.5 announce validation rules. Done. SPEC.md §4.5 covers the MUST validation rules (body parse with context_flag branch, signed_data reconstruction, signature verification, dest_hash recomputation, public-key collision rejection, blackhole list, cache update order, PATH_RESPONSE handling). flows/receive-announce.md walks the chronology end-to-end. Side fixes: SPEC.md §4.1 corrected (random_hash is 5 random bytes + 5 bytes big-endian uint40 unix_seconds, not 10 random bytes); SPEC.md §2.5 contexts table now lists 0x0B PATH_RESPONSE.
• SPEC.md §10 / flows/send-resource.md: Reticulum Resource fragmentation. Done. SPEC.md §10 covers the wire-level MUST rules: 13 sub-sections from "when Resource runs" through wire contexts (ADV / REQ / RESOURCE / HMU / PRF / ICL / RCL), hashmap collision-guard, sliding window, multi-segment cutover at MAX_EFFICIENT_SIZE = 1 MiB - 1, and the encryption-then-split layering. flows/send-resource.md walks the chronology in 10 steps with a wire-byte ladder diagram. Side fixes during the drafting: SPEC.md §2.5 contexts table now lists ALL upstream contexts (was missing all RESOURCE_*, REQUEST/RESPONSE, COMMAND, CHANNEL, LINKIDENTIFY, LINKCLOSE, LRRTT entries) and corrects KEEPALIVE from 0xFD (which is actually LINKPROOF) to 0xFA per RNS/Packet.py:87. SPEC.md §6.5 wording updated to use the correct LINKPROOF context name. The previously-existing §10 "Test vectors" and §11 "Source map" were renumbered to §11 and §12 to put §10 in the protocol-stack flow.
• SPEC.md §6.5 expansion: regular (non-LRPROOF) PROOF body. Done. SPEC.md §6.5 now has six sub-sections covering explicit (96B packet_hash || signature) vs implicit (64B signature-only) forms, the upstream default (Reticulum.__use_implicit_proof = True per RNS/Reticulum.py:259 — opportunistic DATA proofs default to the implicit form on the wire), the Link DATA proof exception (always explicit per RNS/Link.py:383-394), the length-dispatch receiver-side, where the proof packet is addressed (packet_hash[:16] as a synthetic ProofDestination vs link.link_id for Link proofs), wire-byte ladders for both forms. The previously-misleading SPEC §2.5 entry for LINKPROOF (0xFD) is corrected — it's a defined-but-unused constant in RNS 1.2.0; the actual proof packets carry context = NONE (0x00). todo for tools/verify_proof_packet.py moves to "needs a runtime verifier" section.
• SPEC.md §6 sub-section: 3-byte MTU/mode signalling field. Present on LINKREQUEST and LRPROOF iff Reticulum.link_mtu_discovery() == True and the next-hop interface advertises an HW MTU. Encode/decode helpers at RNS/Link.py::signalling_bytes line 148; consumers at mtu_from_lr_packet / mode_from_lr_packet / mtu_from_lp_packet / mode_from_lp_packet. Spec currently shows this slot as "[signalling(3)]" with no byte definition — a client that emits a wrong format gets wrong MTU on the link.
• SPEC.md §7.2 expansion + new flow flows/path-discovery.md: path-response announce vs periodic announce. When a node fulfills a path? request it emits an announce with path_response=True, which sets context = PATH_RESPONSE = 0x0B on the announce packet (RNS/Packet.py:83). Receivers distinguish via packet.context == RNS.Packet.PATH_RESPONSE (RNS/Transport.py:1989-1991); announce handlers default to ignoring path-responses unless they set receive_path_responses = True on themselves. Spec mentions §7.2 "respond by re-announcing" but doesn't name the wire context.
• SPEC.md §1.3 expansion: identity on-disk format. §1.3 names the byte order (Ed25519 first, X25519 second, opposite of the public-key concat) but not the file structure. RNS/Identity.py::to_file is the reference. Without this, identities can't be exported / imported across implementations.

Tier 2 — required for a client to be useful in the wild

• SPEC.md: Propagation node protocol. Offline message retrieval via store-and-forward propagation nodes. Without this, every message requires both peers online simultaneously. Authoritative source: LXMF/LXMRouter.py::process_propagated, the lxmf.propagation peering exchange (peer() / sync() between nodes — LXMRouter.py:1892+, 2118+). The propagated method is already in LXMessage.py but the wire protocol between propagation nodes is undocumented. Cross-flow: flows/send-propagated-lxmf.md (already a ⏳ entry in flows/README.md).
• SPEC.md §6 expansion: KEEPALIVE / link teardown protocol. CTX_KEEPALIVE = 0xfd packets — exact wire body, exact cadence (Link.KEEPALIVE constant), exact teardown packet (Link.PROOF context). Real clients drop links incorrectly without this.
• SPEC.md §5.x (new): LXMF stamps + tickets for spam control. LXMF.Stamp (proof-of-work field in the optional 5th element of the msgpack payload), FIELD_TICKET lookup. Modern Sideband 1.x treats missing-stamp messages as spam in the UI. Spec currently doesn't mention stamps at all. Authoritative source: LXMF/LXMessage.py::validate_stamp, LXMF/LXMRouter.py:1741-1774 (the stamp-check branch in lxmf_delivery).
• SPEC.md §13 (new): NomadNet page protocol. Distinct from LXMF — pages fetched over a Link with context = CTX_REQUEST (0x09) / CTX_RESPONSE (0x0a) (already in §2.5 contexts table). Request body is a path string + field map; response is a body bytes blob. Without this, a client can do LXMF chat but can't render NomadNet content (nodes serving content, telemetry, micron pages).
• SPEC.md §1.4 (new): GROUP destinations. RNS.Destination.GROUP type uses symmetric AES-256-CBC with a pre-shared key; different encrypt/decrypt paths in RNS/Destination.py:601+ (prv is a symmetric-key wrapper, not an X25519 priv). Almost no clients implement this but the protocol allows it.
• SPEC.md §8.4 (new): CSMA / airtime tracking. LoRa-only — carrier-sense + random backoff that prevents transmitter collisions on shared channel. The clean-room repeater explicitly flags "no CSMA" as a phase-2 simplification. A serious LoRa client needs RNS.Reticulum.ANNOUNCE_CAP-aware backoff and the airtime_bins accounting from RNode_Firmware.ino:683-712.
• SPEC.md §8.5 (new): RNode KISS configuration handshake. Beyond §8.3 (split-packet protocol), a client opening an RNode drives CMD_DETECT / CMD_FREQUENCY / CMD_BANDWIDTH / CMD_SF / CMD_CR / CMD_TXPOWER / CMD_RADIO_STATE over KISS to bring up the radio. All defined in RNode_Firmware/Framing.h:24-95. Spec just says "send Reticulum packets via CMD_DATA" — that's not enough.
• SPEC.md §6.5 second sub-bullet: implicit vs explicit proof mode. RNS.Reticulum.should_use_implicit_proof() mode trims the proof body to just the signature (no packet_hash prefix), saving 32 bytes. RNS/Link.py:386-389 has the explicit form hard-coded with the implicit branch commented out, but at least one upstream branch toggles it — a client that hard-codes the explicit form will eventually meet a peer in implicit mode.

Tier 3 — required to act as a transport node / relay

• SPEC.md §7.7 (new): DATA forwarding rules. Forwarding non- local DATA per path_table[dest][NEXT_HOP], with hop increment, MTU-fit check, blackhole avoidance, and IFAC re-signing. Currently mentioned only obliquely in §2.3 / §7.6. The full forwarding logic is the bulk of RNS/Transport.py::inbound's ~800-line dispatch table at lines 1499-1620. The repeater repo patches microReticulum to enable this — see commit Add DATA and PROOF forwarding patches for transport repeating.
• SPEC.md §4.6 (new): ANNOUNCE rebroadcasting. Including the announce-cap (RNS.Reticulum.ANNOUNCE_CAP, default 2% airtime), the announce queue, the path_responses cache, and the random_blob history that lets a relay drop replays. Most of RNS/Transport.py:1196-1300, 1810-1969.
• SPEC.md §7.8 (new): path table management. TTL-based expiry (Transport.AP_PATH_TIME, ROAMING_PATH_TIME, DESTINATION_TIMEOUT), eviction on stale-link, persistence-across-reboot file format. Hooks: RNS/Transport.py:747-769 (stale_paths accumulator) and the paths file under storagepath.
• SPEC.md §7.9 (new): tunnels and shared-instance protocol. tunnels, discovery_path_requests, RNS/Reticulum.py::is_connected_to_shared_instance — how a process talks to a co-resident rnsd. Spec's §7.6 covers one symptom (TCP OUT default) but not the actual shared-instance wire format.
• SPEC.md §6.x (new): reverse table + link transport. When a Link's path crosses a relay, the relay must forward both directions of every Link DATA + PROOF using Transport.reverse_table (RNS/Transport.py:2087-2204). Distinct from path-table forwarding — different lookup, different lifecycle.

Spec polishing (lower priority)

• Split SPEC.md into per-layer files as the document grows past ~1500 lines. Suggested layout per README.md: 00-overview.md, 01-packet-header.md, 02-identity.md, 03-announce.md, 04-token-crypto.md, 05-lxmf.md, 06-link.md, 07-resource.md, 08-transport.md, 09-paths-and-discovery.md, 10-implementation-gotchas.md.

• Add a "last-verified-against-rns" line to SPEC.md frontmatter (per agent.md §7) so readers know which RNS version the spec was tested against.