Find a file
John Poole 5aa3920b76 Completed the propagation-node announce and peer-sync three-tier unit.
Added:
Tier 1 audit
Peer-sync flow
Deterministic vectors
Regenerator
Verifier
Corrected §5.8 regarding:
Directional peering-key identity ordering.
Public versus control destination handlers.
Permissive announce parser behavior.
Autopeer rules.
Peer Resource framing and admission.
PN_STAMP_THROTTLE = 180 seconds.
Two documented LXMF 0.9.7 hazards.
Verification: deterministic regeneration passed; full pinned suite passed 20/20; git diff --check passed. No commit created.
2026-06-08 17:32:55 -07:00
.claude/skills/rns-update Add /rns-update skill and gitignore for .claude scratch / per-user config 2026-05-08 07:58:44 -04:00
audits Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
flows Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
templates docs: add playbook.md + AGENTS.md template for new Reticulum impls 2026-05-10 20:05:22 -04:00
test-vectors Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
tools Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
.gitignore Implemented the portable verification baseline and completed the first Resource three-tier pass. 2026-06-08 13:22:22 -07:00
agent.md Completed the REQUEST/RESPONSE three-tier work unit. 2026-06-08 14:03:50 -07:00
Codex_prompt_1.md After 2nd prompt 2026-06-08 12:47:32 -07:00
Codex_response_1.md After 2nd prompt 2026-06-08 12:47:32 -07:00
LICENSE Initial bootstrap: README, LICENSE, SPEC.md, agent.md, scaffolding 2026-05-03 09:38:46 -04:00
playbook.md Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
README.md Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
SPEC.md Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00
todo.md Completed the propagation-node announce and peer-sync three-tier unit. 2026-06-08 17:32:55 -07:00

Reticulum Specifications

Byte-level interoperability specifications for the Reticulum Network Stack and LXMF — the parts that aren't in the upstream manuals but are needed to build a working client from scratch.

Upstream Reticulum has excellent operator-facing documentation (config, deployment, design philosophy). What's missing — and what every alternative implementation has had to reverse-engineer from the Python source — is an authoritative wire-level spec: header bit layouts, msgpack field types, signature input formats, the exact behavior of Transport.outbound, and the long list of "would never guess from reading the manual" gotchas that cost hours of debugging each.

This repo collects those findings in one place. The hope is that future client authors (Kotlin, Swift, Rust, Go, embedded C — pick your stack) can read this instead of re-deriving everything from RNS/Transport.py.

Status

Early days, contributions welcome. Current content was bootstrapped from the working notes of two reverse-engineering efforts:

Each finding is grounded in upstream source citations (file + line) so it can be re-verified as RNS evolves.

Method

This fork proceeds with a three-tier evidence model that preserves the existing Claude-assisted work while promoting claims only after verification:

  1. Source analysis report — answer one narrow protocol question from pinned upstream source, with exact citations and unresolved cases.
  2. Executable verification — add runnable evidence under tools/ and deterministic vectors under test-vectors/ where bytes are involved.
  3. Specification promotion — update SPEC.md as normative prose only after the verifier/vector exists; add correction or incident notes when earlier text would mislead implementers.

See agent.md §3 for the detailed rules.

Verify a Clone

The verifier suite has no dependency on a particular user's virtual environment. From a fresh clone, create any isolated Python environment and install the repository pins:

python3 -m venv .venv
.venv/bin/python -m pip install -r tools/requirements.txt
.venv/bin/python tools/verify_all.py

On Windows, use .venv\Scripts\python.exe in place of .venv/bin/python. Activation is optional; verify_all.py uses the interpreter that launched it and refuses to run when installed RNS/LXMF versions do not match the pins.

What's here

  • SPEC.md — the single combined spec document, organized by protocol layer
  • playbook.md — how to troubleshoot interop bugs, design tests that don't lie to you, and navigate the protocol's code-as-spec parts. Read this if you're starting any Reticulum implementation work, not just contributing to this repo. Includes an incident registry of past wire-format bugs and their fixes.
  • agent.md — verification rules for adding to this repo (markers, tools/, test-vectors)
  • templates/ — drop-in AGENTS.md for new Reticulum implementation projects in any language. Copy into your project root, edit the marked sections, and the next agent or contributor lands on the right docs automatically.
  • flows/ — chronological end-to-end narratives (e.g. "send a message"), cross-referencing SPEC.md sections
  • tools/ — self-contained Python verifier scripts that test SPEC.md claims against upstream RNS / LXMF. Pinned via tools/requirements.txt to the upstream versions the scripts were last re-verified against
  • test-vectors/ — known-good byte sequences each implementation should be able to round-trip (intent: grow into a compliance suite)

As content grows, SPEC.md will be split into per-layer files (packet header, identity, announce, token-crypto, LXMF, link, resource, transport).

Spec corrections

Errata that may invalidate code built against an earlier revision of SPEC.md. Newest first. Feature additions and ordinary edits live in git log — this section is reserved for cases where the spec said one thing, that turned out to be wrong, and an implementer who pulled the bad version needs to fix their code.

  • 2026-06-08 — §5.8 propagated-LXMF transient IDs, /get framing, and error constants. Earlier §5.8 text described transient IDs as 16-byte truncated hashes; upstream LXMF 0.9.7 uses the full 32-byte SHA256(lxmf_data). It also incorrectly described /get responses as propagation bundles shaped [time, [messages]]; the /get handler actually returns a plain message list carried by the generic [request_id, response] Link RESPONSE. Accepted submission and peer-transfer entries always append a required 32-byte propagation stamp, including at cost zero. The section also incorrectly placed operator handlers on the public propagation destination, described the announce parser as exact/strict, and stated a 30-minute peer throttle; operator handlers use lxmf.propagation.control, the parser is deliberately permissive, and PN_STAMP_THROTTLE is 180 seconds. Finally, ERROR_THROTTLED is 0xf6, ERROR_NOT_FOUND is 0xfd, and 0xf5 is ERROR_INVALID_STAMP. Corrected and runtime-locked by tools/verify_propagated_lxmf.py and tools/verify_propagation_peer.py.

  • 2026-06-08 — §11.4 REQUEST authorization constants were reversed. Earlier §11.4 text assigned ALLOW_LIST = 0x01 and ALLOW_ALL = 0x02. Upstream RNS 1.2.4 defines ALLOW_NONE = 0x00, ALLOW_ALL = 0x01, and ALLOW_LIST = 0x02 in RNS/Destination.py. An implementation following the prior table would expose list-restricted handlers publicly and incorrectly restrict public handlers. §11.4 is corrected and runtime-locked by tools/verify_request_response.py. The same audit corrected §11.2 file metadata wording: Resource advertisement field m is always the hashmap; file metadata is carried inside Resource plaintext and signaled by flag x.

  • 2026-05-17 — §10.2 Resource integrity hash: the 4-byte prefix is NOT r, and is NOT in the hash input. Bad text introduced in 95823ad; on master from 2026-05-03 to 2026-05-17. §10.2 step 3 wrongly equated the random-hash prefix prepended to the Resource body with the advertisement's r field, and step 5 wrongly fed that prefix into hash/expected_proof (claiming hash = SHA256(random_hash || body || random_hash)). Upstream RNS/Resource.py (1.2.4) uses two distinct get_random_hash()[:4] values: a throwaway prefix the receiver strips and discards (:405/412, :682), and self.random_hash — the advertisement's r field (:440, :1285). The integrity hash is SHA256(uncompressed_plaintext || r) over the prefix-stripped, decompressed body (:441, :694) — exactly as §10.8 already stated. An implementer who trusted §10.2 step 5 computes a hash no spec-compliant peer accepts; every Resource is rejected as CORRUPT. §10.2 corrected to agree with §10.8; §10.12's wire-layering block fixed to match. Surfaced by issue #9.

  • 2026-05-06 — §2.1 flag byte: bit 7 is the IFAC flag, not part of header_type. Bad text introduced in 8c4d550, corrected in 0c2021e; on master from 2026-05-04 to 2026-05-06. The corrected layout is ifac_flag(bit 7) | header_type(bit 6) | context_flag(5) | transport_type(4) | destination_type(3-2) | packet_type(1-0), matching the official manual §4.6.3 and upstream RNS/Packet.py:246 (parse mask 0b01000000 >> 6) / RNS/Transport.py:1003 (IFAC setter raw[0] | 0x80). Implementers who consumed the bad version will mis-parse every IFAC-protected packet as header_type ∈ {2, 3} and drop it. Surfaced by issue #4 item #1.

Scope

In scope:

  • Wire formats: byte layouts, field encodings, framing
  • Signing inputs and what's hashed where
  • Cross-cutting behaviors required for interop (path requests, ratchet rotation, retransmit semantics)
  • "Gotchas" — things upstream code does that aren't obvious from the manual or RFC-style sketches
  • Test vectors that any implementation must be able to round-trip

Out of scope:

  • Operator/user documentation — see the official manual
  • API design choices for any specific implementation
  • Networking layer config (interfaces, transport modes) — already well documented

Source citations

Where a finding cites upstream Python code, the path is relative to a standard pip install rns lxmf installation, e.g. RNS/Transport.py, LXMF/LXMF.py. Where the bundled umsgpack is referenced, the path is RNS/vendor/umsgpack.py.

When upstream code changes such that a citation no longer matches, file an issue or PR — the goal is to track the de-facto wire spec as it actually behaves, not as it was at any single snapshot.

Contributing

If you've debugged a Reticulum interop problem and the answer wasn't in the upstream docs, please add it. Format:

### N.M Short description of the finding

**Symptom:** what you observed that prompted the investigation.

**What's happening:** the actual mechanism, ideally with upstream source citation (file + line).

**Implication / fix:** what an implementation must do to interop.

**Source:** upstream file paths and approximate line numbers.

Add a worked test vector to test-vectors/ if the finding is byte-level.

Provenance

This project is a fork (June 8, 2026) of https://github.com/thatSFguy/reticulum-specifications. thatSFguy's approach was to use Claude, I use Codex and have applied the approach I used for the LXMF-specification project here.

License

CC BY 4.0 — use freely, attribution appreciated.