Upstream RNS enforces two requirements in code that SPEC.md left implicit;
both caused silent message loss in a clean-room Go LXMF service against
upstream Python rns 1.2.4 / lxmf 0.9.7.
§6.4.2 LRRTT — initiator's link-activation packet
- HEADER_1, DATA, dest_type=LINK (0x03), ctx=0xfe; body is
`umsgpack.packb(rtt_seconds)` encrypted with the link's session keys.
- The responder transitions HANDSHAKE→ACTIVE only on LRRTT receipt
(Link.py:534-553), which is also what fires the link_established
callback. LXMF's set_resource_strategy(ACCEPT_APP) is installed
from that callback; without it, every RESOURCE_ADV the initiator
sends hits the silent ACCEPT_NONE branch at Link.py:1087.
§6.4.3 Header type for post-handshake DATA and Resource
- Link-addressed packets are routed via link_table, which forwards
header bytes verbatim (Transport.py:1587-1622). HEADER_2 with a
relay's transport_id therefore arrives at the destination intact
and is dropped by packet_filter (Transport.py:1283-1285) as
"for another transport instance".
- Mandates HEADER_1 with no transport_id for all post-handshake
link DATA / Resource / control packets regardless of hop count.
- Asymmetry with LINKREQUEST (which IS path_table-routed and so
HEADER_2-eligible) is spelled out.
Companion changes:
- §6.4 renamed to "Session keys and link activation"; existing
HKDF content moved into §6.4.1.
- §2.5 LRRTT context-byte entry points at §6.4.2.
- §12.5.2 (Link DATA forwarding) cross-references §6.4.3.
- §14 failure-modes table: two new entries for the silent-drop
chains documented above.
- flows/send-link-lxmf.md step 4 strengthened (LRRTT is mandatory,
not informational); step 6 corrected (Transport.outbound does NOT
apply HEADER_1→HEADER_2 for link DATA — that conversion is
path_table-keyed, link DATA is link_table-keyed).
- test-vectors/links.json extended with an LRRTT entry: pinned
rtt_seconds=0.05 + pinned 16-byte IV produces deterministic
wire bytes for the encrypted body.
- tools/regen_links.py drives the LRRTT generation with an
os.urandom patch for the Token IV.
- tools/verify_link_lrrtt.py (new) locks the wire claims:
HEADER_1, ctx=0xfe, dest=link_id, body decrypts under
derived_key to msgpack float64 matching rtt_seconds.
Citations all verified against installed RNS 1.2.4 / LXMF 0.9.7.
All 14 verifiers PASS.
Documents the outbound retry layer that wraps the existing per-method
send-* flows. Pinned to LXMF 0.9.7 / RNS 1.2.4 with literal-quoted
upstream source for every claim:
- 4-second tick cadence (PROCESSING_INTERVAL × JOB_OUTBOUND_INTERVAL)
- All seven retry constants (MAX_DELIVERY_ATTEMPTS, DELIVERY_RETRY_WAIT,
PATH_REQUEST_WAIT, MAX_PATHLESS_TRIES, MESSAGE_EXPIRY,
LINK_MAX_INACTIVITY, P_LINK_MAX_INACTIVITY) at LXMRouter.py:30-38
- Eight-state machine (GENERATING/OUTBOUND/SENDING/SENT/DELIVERED/
REJECTED/CANCELLED/FAILED) at LXMessage.py:13-22
- The four terminal-state branches at top of process_outbound (lines
2517-2558) and the three per-method retry branches (OPPORTUNISTIC
2566-2592, DIRECT 2596-2673, PROPAGATED 2677-2730)
- fail_message semantics at LXMRouter.py:2395-2402
Includes a "what does NOT happen" section calling out common
misconceptions: no automatic DIRECT→PROPAGATED fallback, no
exponential backoff, no in-router persistence of pending_outbound,
MESSAGE_EXPIRY governs the propagation-node store not per-sender
retries, SENT is the terminal success state for PROPAGATED (not
DELIVERED).
No verifier needed per agent.md §1 — all claims are direct upstream
source citations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last
release that is also published to GitHub" — pip continues until rnpkg
is complete and RNS is self-hosting. All 13 verifiers pass against
1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed
between 1.2.0 and 1.2.4, so the changes here are purely currency:
- Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the
verifier stays reproducible if upstream stops mirroring to PyPI
before the migration is ready.
- Add an "Upstream distribution shift" watch-list to todo.md (local
Reticulum node, repo destination hash, rnpkg install/upgrade
commands, rsg signature verification, mirroring source citations).
- Bump SPEC.md frontmatter and re-anchor ~50 line citations across
Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py,
Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13
to +31 lines; Transport.py was variable). Fix one numeric
(MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU
clamp citation pointed at the wrong location — corrected to point
at the transit-relay clamp at Transport.py:1539-1556).
- Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap
adoption, with citations to Resource.py:686-691 and Buffer.py:95-97
plus a "do not use one-shot bz2.decompress()" warning.
- Re-anchor 11 flows/ files (version pins + ~30 line citations).
- Bump version labels in tools/README.md, test-vectors/README.md, and
4 verifier docstrings + 2 hardcoded print strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifiers:
tools/verify_proof_packet.py — locks in §6.5. Toggles
Reticulum.__use_implicit_proof to test both modes; confirms
Identity.prove emits 64B (implicit) or 96B (explicit) proof
body; PacketReceipt.validate_proof accepts both lengths and
rejects an 80B body.
tools/verify_link_handshake.py — locks in §6.1, §6.2, §6.3, §6.6.
Most importantly verifies the previously-corrected §6.2 LRPROOF
body order (signature(64) || responder_X25519_pub(32) ||
[signalling]) and §6.3 link_id offsets (N=2 for HEADER_1) by
actually building a Link initiator-side, capturing the
LINKREQUEST raw bytes, computing link_id by the spec recipe,
running validate_request inline (since the upstream wrapper
swallows exceptions), and confirming the responder's LRPROOF
bytes match the spec layout. This was the single most
interop-critical correction we made.
tools/verify_rnode_split.py — locks in §8.3. Pure-function
re-implementation of the canonical TX and RX state machines
from RNode_Firmware.ino:359-446 + 716-742; tests header-byte
layout, single-frame TX, split-frame TX (300B → 254+46 with
shared header byte), all four RX state-machine cases (a/b/c/d
from the spec table), and end-to-end TX/RX round-trip at
sizes 50, 254, 255, 300, 508.
tools/verify_msgpack_quirk.py — locks in §9.3. Confirms umsgpack
distinguishes str (fixstr/0xa5) from bytes (bin8/0xc4); confirms
LXMF.display_name_from_app_data parses bytes-encoded display
names correctly and silently returns None (not crash) on
str-encoded ones, matching the bug-tolerance documented in §9.3.
All 11 verifiers pass against RNS 1.2.0 / LXMF 0.9.6.
Plus:
- SPEC.md frontmatter: 'Last verified against' line per agent.md §7.
- flows/receive-propagated-lxmf.md: closing half of the propagated
LXMF lifecycle. /get listing query, fetch query, ack-and-purge
via the have_ids slot, message-bundle unpack and dispatch
through lxmf_delivery.
- tools/README.md status table refreshed; flows/README.md flips
receive-propagated-lxmf.md to ✅.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- flows/receive-resource.md: inverse of send-resource. ADV
ingestion, accept/reject decision, request_next loop,
receive_part insertion, assemble + decrypt + hash-validate,
RESOURCE_PRF emission, multi-segment continuation.
- flows/receive-link-lxmf.md: responder side of the link
handshake plus inbound LXMF DATA handling. validate_request
-> handshake -> prove (LRPROOF emission) -> link_established
callback wires delivery_packet. PACKET-form inbound runs
delivery_packet directly; RESOURCE-form inbound runs through
delivery_resource_advertised + delivery_resource_concluded
pipeline.
- flows/send-announce.md: random_hash construction (5B random +
5B BE-uint40 timestamp), optional ratchet rotation, signed_data
assembly, sign + pack, the broadcast emission. Notes that
ANNOUNCE packets are NOT encrypted (Packet.pack special-cases
line 189-191) and the periodic re-announce loop drives 5-15min
cadence.
- flows/forward-announce.md: relay-side rebroadcast for
transport-mode nodes. Eligibility checks (transport_enabled,
not PATH_RESPONSE, not rate_blocked), announce_table queue,
Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s,
per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime
enforcement, lowest-hop-count-first emission order, hops byte
increment, local-rebroadcast counter for loop break.
- flows/send-propagated-lxmf.md: PROPAGATED method end to end.
LXMessage.pack with body encrypted to recipient (propagation
node never decrypts), Link establishment to the propagation
node, optional propagation stamp (1000 PoW rounds vs 3000 for
regular stamps), submission via Link DATA or Resource,
state goes to SENT (not DELIVERED — recipient pulls via /get
later per §5.8.3).
flows/README.md status table updated; receive-propagated-lxmf.md
added as the only remaining ⏳ flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes Tier 1 #5. The previous §7.2 was four bullet points naming the
"answer with an announce" rule but missing every wire detail —
implementation-time the SF mobile client got steps 4 (dedup) and 5
(local-destination check) wrong on its first cut and the bug only
surfaced as "I can message my own destination but no one else can
reply".
§7.2 is now six sub-sections:
§7.2.1 Path-request packet parse rules. The handler's slice
recipe with branching on payload length (32B = leaf form
target||tag; 48B+ = transport form target||transport_id||
tag); tag cap at 16B; tagless-request rejection.
§7.2.2 Tag-based dedup via Transport.discovery_pr_tags. The
unique_tag = dest_hash || tag construction, the 32000-
entry cap, why missing this turns a leaf into a broadcast-
storm amplifier on retransmits.
§7.2.3 The five-way dispatch in Transport.path_request:
local-destination / transit-knows-path / local-client-
forward / discovery-recursive / drop. Branches 1 and 5
are the only ones a leaf needs.
§7.2.4 Path-response announce wire format. Body byte-identical
to a regular announce (§4.1); only the outer packet
context byte differs (NONE → PATH_RESPONSE 0x0B).
PR_TAG_WINDOW=30s body-cache that serves identical wire
bytes to racing relays so transit dedup converges.
§7.2.5 Timing constants: PATH_REQUEST_GRACE = 0.4s, +
PATH_REQUEST_RG = 1.5s for roaming-mode interfaces.
Local-destination and local-client originator branches
bypass the grace.
§7.2.6 Minimum responsibility for a non-transport leaf — the
six-step protocol-level recipe.
flows/path-discovery.md: 9-step chronology covering both
single-hop leaf-owns-target and two-hop transit-relay-knows-path
cases. Wire-byte ladder diagrams for both. Notes the ingress-limit
bypass for path-responses (Transport.py:1632-1639), the
receive_path_responses opt-in for handler dispatch
(Transport.py:1989-1991), and the timeout/escalation path through
LXMRouter.process_outbound's MAX_PATHLESS_TRIES retry counter.
flows/README.md status table updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes Tier 1 #2. Without this, a client can't send any LXMF body
larger than LINK_PACKET_MAX_CONTENT ≈ 360 B, can't receive a NomadNet
page that doesn't fit in one MTU, and can't transfer files via rncp.
SPEC.md §10 (new): full Resource fragmentation protocol with citations
to RNS/Resource.py. 13 sub-sections covering preparation pipeline
(metadata prefix → optional bz2 → random_hash prefix → SHA-256 over
data||random_hash → link.encrypt of the WHOLE blob → part-split into
SDU-sized chunks → 4-byte map_hash hashmap with collision guard within
COLLISION_GUARD_SIZE = 2*WINDOW_MAX + HASHMAP_MAX_LEN), wire context
inventory (RESOURCE_ADV / RESOURCE / RESOURCE_REQ / RESOURCE_HMU /
RESOURCE_PRF / RESOURCE_ICL / RESOURCE_RCL), the msgpack dict for the
advertisement (t/d/n/h/r/o/i/l/q/f/m), the request payload format with
the hashmap_exhausted sentinel, the lazy-hashmap RESOURCE_HMU
continuation that lets large hashmaps avoid breaking small-MTU links,
the proof body
resource_hash(32) || full_proof = SHA256(data||hash) (32)
returned in a PROOF-type packet, the sliding window dynamics
(WINDOW=4 → WINDOW_MAX_FAST=75 / WINDOW_MAX_VERY_SLOW=4 with rate
detection), multi-segment cutover at MAX_EFFICIENT_SIZE = 1 MiB - 1
with the lazy `__prepare_next_segment` pattern, and the
encryption-before-split layering that means a missing part can't be
decrypted in isolation.
flows/send-resource.md: 10-step chronology from RNS.Resource()
construction through advertise → req/parts loop → HMU continuation →
final RESOURCE_PRF → multi-segment fan-out, with a wire-byte ladder
diagram and a per-step source map.
Side fixes found while drafting:
- SPEC.md §2.5 contexts table was wildly incomplete and had a real
bug: KEEPALIVE was listed as 0xFD; upstream is 0xFA per
RNS/Packet.py:87. 0xFD is actually LINKPROOF (the regular
DATA-receipt context, §6.5). Replaced with the full upstream
context inventory: NONE, RESOURCE_*, CACHE_REQUEST, REQUEST,
RESPONSE, PATH_RESPONSE, COMMAND, COMMAND_STATUS, CHANNEL,
KEEPALIVE, LINKIDENTIFY, LINKCLOSE, LINKPROOF, LRRTT, LRPROOF.
- SPEC.md §6.5 reworded: "send back a PROOF packet (no context
byte specifics)" → "send back a PROOF-type packet with
context = LINKPROOF (0xFD)" for clarity.
- The previously-numbered §10 "Test vectors" and §11 "Source map"
are renumbered to §11 / §12 so the new Resource section lands in
its correct protocol-stack position. agent.md §5 audit table
updated accordingly.
flows/README.md status table updated; receive-resource.md added as
the next pending flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the highest-priority Tier 1 gap. Without this, a from-scratch
client can't learn any peers exist; known_destinations stays empty and
every outbound message fails at recall(dest_hash).
SPEC.md §4.5 (new): announce validation rules with full citations to
RNS/Identity.py::validate_announce (line 496) and the dispatch path in
RNS/Transport.py:1623-2024. Covers the body parse with context_flag
branch, signed_data reconstruction (including the empty-bytes-not-absent
ratchet rule), Ed25519 signature verification, dest_hash recomputation,
public-key collision rejection, blackhole list, cache update order
(known_destinations -> known_ratchets -> path_table), PATH_RESPONSE
distinction, and the implementation-private SHOULD rules around
ingress rate limiting, random_blob history caps, and self-announce
filtering.
flows/receive-announce.md: chronological walk through 9 steps from
deframing to handler dispatch, with the cheap-pre-filter design
(signature-checked-then-counted) called out, the burst-active ingress
limiter explained against IC_BURST_FREQ_NEW=6Hz / IC_BURST_FREQ=35Hz,
the path-table decision tree, and the announce_handlers fan-out with
aspect_filter and PATH_RESPONSE filtering. Ends with a wire-byte
diagram and a per-step source map.
Two side fixes found while drafting:
- SPEC.md §4.1 had random_hash described as "10 random bytes". It's
actually random_hash = get_random_hash()[0:5] + int(time.time()).to_bytes(5, "big")
per RNS/Destination.py:282. Transit relays parse the trailing 5
bytes via timebase_from_random_blob (RNS/Transport.py:3100) for
replay-ordering decisions.
- SPEC.md §2.5 contexts table was missing PATH_RESPONSE = 0x0B
(RNS/Packet.py:83).
flows/README.md status table updated; the priority-ordered todo list
also gets a few new entries spun off from the work
(send-announce, forward-announce, send-resource, path-discovery flows).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
receive-opportunistic-lxmf.md mirrors the send flow on the recipient
side: KISS/HDLC deframe -> Transport.inbound -> packet_filter dedup ->
DATA/SINGLE branch -> Destination.receive -> Identity.decrypt with the
ratchet ring + long-term-key fallback -> LXMRouter.delivery_packet (which
fires the PROOF receipt before parsing) -> LXMessage.unpack_from_bytes
with msgpack stamp-strip -> ticket/stamp/dedup checks -> __delivery_callback
to the app. Notes upstream's narrower variant tolerance vs SPEC.md §5.6
and the missing clockless-sender fix-up vs §9.6.
send-link-lxmf.md walks the DIRECT method end-to-end: process_outbound
DIRECT branch decides reuse-vs-establish, RNS.Link.__init__ builds the
unencrypted LINKREQUEST body (initiator_X25519_pub || initiator_Ed25519_pub
|| optional signalling), link_id derived from get_hashable_part, LRPROOF
arrives back and validate_proof verifies signature against the responder's
long-term Ed25519 pub recalled from a prior announce, handshake() does
ECDH+HKDF over the shared secret with salt=link_id, lxmessage.send sends
the full LXMF body (with dest_hash, per §5.2) over the link with Token
encryption that omits the eph_pub prefix per §3.1, mandatory PROOF
receipts per §6.5 resolve the PacketReceipt. Sketches the RESOURCE
representation for oversize bodies and the backchannel-identify trick
that makes the link bidirectional.
flows/README.md status table updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
flows/ documents end-to-end chronological narratives for common Reticulum
operations, complementing SPEC.md (which is organized by protocol layer).
Each step cross-references the SPEC.md section that defines the wire
bytes, so the directory introduces no new normative claims.
First flow: send-opportunistic-lxmf.md walks the 13-step sequence from
LXMRouter.handle_outbound through LXMessage.pack, the path-request
preamble, Token encryption, Transport.outbound HEADER_1→HEADER_2
conversion, and per-interface KISS/HDLC framing. Pinned against RNS 1.2.0
/ LXMF 0.9.6 with file+line citations for each step.
README.md updated to advertise flows/ and tools/ alongside SPEC.md and
test-vectors/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
