Commit graph

65 commits

Author SHA1 Message Date
7ffbb0ef5e Completed the full link-delivered LXMF unit:
Tier 1 audit: `link-lxmf-tier1-rns-1.2.4-lxmf-0.9.7.md`
Tier 2 vectors/verifier: link-lxmf.json, regen_link_lxmf.py, and verify_link_lxmf.py
Tier 3 promotion: updated SPEC.md, flows, status, and documentation
Key correction: the 319/320 boundary uses upstream’s computed LXMF content_size, not simply raw message content length.
Also corrected stale flow descriptions for KEEPALIVE (0xFA) and encrypted LINKCLOSE teardown (0xFC).
Verification:
Deterministic vector regeneration: identical SHA-256
Portable-path and formatting checks: pass
Full pinned suite: 17 passed, 0 failed
2026-06-08 13:54:27 -07:00
7433063bfb Completed the Resource three-tier work unit.
Added deterministic `resources.json` and `regen_resources.py`.
Extended `verify_resource.py` with receiver assembly/proof and requested negative cases.
Updated specification, audit, status, and tool documentation.
Fixed an unrelated nondeterministic wrong-ticket test in verify_stamps.py.
Confirmed vector regeneration is byte-identical.
Confirmed no tracked reliance on specenv or user-specific paths.
git diff --check: pass.
Complete pinned suite: 16 passed, 0 failed.
2026-06-08 13:38:24 -07:00
3659ba0324 Added provenence. 2026-06-08 13:28:04 -07:00
1fb8b8ec10 Meant to include these in the prior commit, adding here & now. 2026-06-08 13:23:43 -07:00
2c9ac94d7c Implemented the portable verification baseline and completed the first Resource three-tier pass.
Clone Portability

Added fresh-clone setup instructions using repository-local .venv in README.md (line 28) and tools/README.md (line 12).
Documented that any virtual-environment path works and activation is optional.
Added .venv/ and venv/ to .gitignore (line 17).
Confirmed no tracked project files reference your specenv or rnsenv.
Verification Infrastructure

Added verify_all.py (line 1), which:
Enforces versions from tools/requirements.txt.
Runs every verifier independently.
Summarizes all failures.
Confirmed it rejects the older RNS 1.1.3/LXMF 0.9.3 environment.
Resource Audit

Added Tier 1 report: resource-tier1-rns-1.2.4.md (line 1).
Added verify_resource.py (line 1).
Corrected §10 and stale flow documentation:
Direct LXMF Resource threshold is 319 bytes.
Advertisement d is total logical-resource size.
Resource packets contain slices of one encrypted stream.
Exhausted requests can also request parts.
RESOURCE_RCL rejects advertisements; ordinary receiver cancellation is local-only.
Validation:

Passed: 16
Failed: 0
ALL VERIFIERS PASS
Remaining Resource work is deterministic resources.json vectors and negative/rejection cases.
2026-06-08 13:22:22 -07:00
e15e8fd62e After 2nd prompt 2026-06-08 12:47:32 -07:00
Rob
ae5738ea2f spec: §5.9.8 reactions + §5.9.9 reply-to (app extensions) + §5.5 raw-wire-bytes rule
Resolves #8. Documents the convergent fields[16] / fields[0x30]+[0x31]
shapes shared by reticulum-mobile-app, MeshChatX, and Columba.

§5.5 gains the normative line that msgpack_payload MUST be hashed from
raw wire bytes — without it, reactions/replies miss the relay rewrite
cache for any message whose fields don't round-trip byte-identically.

§5.9.8 (tap-back reactions on fields[16]) and §5.9.9 (reply-to on
fields[0x30] + optional fields[0x31]) carry > UNVERIFIED markers since
these keys are outside the upstream LXMF allocation range. Columba's
legacy fields[16] = {reply_to} shape is recorded as an inbound-only
tolerance branch with a finite lifetime — torlando-tech/columba#926
shipped the dual-key shape in v2.0.0-beta on 2026-05-23.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 08:36:39 -04:00
Rob
41d2fd61ee spec: §6.7.6 LINKIDENTIFY wire format + §11.6 prose
§6 specified the wire format of every Link control packet — LRRTT
(§6.4.2), KEEPALIVE (§6.7.1), LINKCLOSE (§6.7.3) — but had no
subsection for LINKIDENTIFY (`context = 0xFB`). §11.6 described it
only in prose, and called the payload "a signature over `link_id`" —
imprecise on two counts.

Per RNS 1.2.9 `RNS/Link.py:459-475` `Link.identify()`:
  - Payload is `public_key(64) || signature(64)` (128 bytes total),
    NOT just the signature.
  - `signature` is over `link_id || public_key`, NOT over `link_id`
    alone.

LINKIDENTIFY is a DATA packet on the active link, so it IS
link-encrypted like ordinary link DATA (unlike KEEPALIVE / link
PROOF, §6.7.1 / §6.5). Added explicit §6.7.6 subsection with the
wire layout, the responder-side parse path, and the two clean-room
pitfalls; corrected the §11.6 prose to match and point at the new
subsection.

Closes #12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:34:32 -04:00
Rob
7a85385bb4 spec: §6.5.1 — Link DATA proof is signed with an Ed25519 key, not the link-derived signing key
The §6.5.1 bullet said the Link DATA proof's signing key is "the
link-derived signing key" — misleading. The link's HKDF output
(§6.4.1) splits into a symmetric HMAC `signing_key` (for the link
Token form, §3.1) and an `encryption_key`; that signing_key cannot
produce an Ed25519 signature.

Per RNS 1.2.9, the Link DATA proof is signed with the link's Ed25519
`sig_prv` (`RNS/Link.py:1212-1213` `sign()` uses `self.sig_prv`):
  - responder side: `owner.identity.sig_prv` (long-term identity
    Ed25519 private key, `Link.py:279`)
  - initiator side: a fresh ephemeral Ed25519 keypair generated at
    link creation (`Ed25519PrivateKey.generate()`, `Link.py:286`)

§6.5.1 now states this explicitly and distinguishes it from the
symmetric §6.4.1 signing_key.

Closes #11.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:34:32 -04:00
Rob
dee108e787 spec: §6.7.1 — KEEPALIVE is not Token-encrypted and does not generate a PROOF receipt
The §6.7.1 prose said KEEPALIVE is Token-encrypted (wire body
`iv(16) || ciphertext || hmac(32)`) and generates a mandatory PROOF
receipt like other link DATA. Both claims contradict the reference.

Verified against RNS 1.2.9:
- `RNS/Packet.py:206-209` `pack()` puts `KEEPALIVE` in its
  not-encrypted branch — the wire body is the sentinel byte in the
  clear, alongside RESOURCE / RESOURCE_PRF / link PROOF / CACHE_REQUEST.
- `RNS/Link.py:988` gates the link-DATA proof path on
  `context == NONE`; KEEPALIVE takes its own branch at line 1149.

A clean-room receiver that expected `iv(16) || ciphertext || hmac(32)`
per the old text would fail to parse a real KEEPALIVE.

Closes #10.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:34:32 -04:00
Rob
375521c963
spec: §10.7 — an exhausted RESOURCE_REQ may carry parts
A conformant sender fulfils any bundled `requested_map_hashes` AND
sends the RESOURCE_HMU. Verified against RNS 1.2.9 (`Resource.py:982-1071`):
part fulfilment runs unconditionally for every REQ, and the HMU branch
runs in addition. The reference receiver (`request_next`) routinely
bundles parts into an exhausted REQ. §10.7 now states the correct
rule; part-less exhausted REQs are an allowed receiver-side
simplification. `playbook.md` §7 records the matching fwdsvc
conformance bug (since fixed in `reticulum-forwarding-service` PR #10).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:19:56 -04:00
Rob
4c1d3de421 spec: §4.6 — rrc.hub app_data is CBOR, not msgpack
Correction. RRC's wire codec is CBOR throughout (rrcd/codec.py uses
Python cbor2), so the rrcd announce app_data {"proto","v","hub"} is
a CBOR map, not a msgpack map as §4.6 first stated.

Added the concrete gotcha that motivated the correction: a CBOR
3-entry map starts 0xa3, which msgpack reads as fixstr-3 — so a naive
msgpack decode of the app_data yields the bogus string "epr" (the
CBOR text-string header of "proto" + its first two chars). Clients
must CBOR-decode rrc.hub app_data, keyed on the name_hash.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 14:47:14 -04:00
Rob
a9a5857122 spec: §4.6 — rrc.hub announce app_data shape
Reticulum Relay Chat hubs announce on the `rrc.hub` aspect, and the
app_data is NOT an LXMF [name, cost] array. The two hubs disagree:
rrcd (Python) emits a msgpack map {"proto","v","hub"} with the human
name under "hub"; the Go hub emits the name as plain UTF-8 bytes.
Documents both, the resolve order a client should use, and that the
authoritative name also arrives in the RRC WELCOME (B_WELCOME_HUB).

Confirmed from upstream source — rrcd service.py and
reticulum-relay-chat internal/service/service.go.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 13:47:05 -04:00
Rob
fd2951be82 spec: pin FIELD_FILE_ATTACHMENTS (0x05) value shape
§5.9.7 — FIELD_FILE_ATTACHMENTS is a list of [filename, file_bytes]
pairs (multiple attachments per message). Confirmed from upstream
Sideband source: core.py builds `fields[FIELD_FILE_ATTACHMENTS] =
[attachment]` with `attachment = [filename, filedata]`; ui/messages.py
reads `attachment[0]`/`attachment[1]` on receive and strips `../`
from the filename.

Removed FIELD_FILE_ATTACHMENTS from the §5.9 UNVERIFIED list and
added a §5.9.7 pointer to the field table. Documents the
sender-controlled-filename sanitisation requirement.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 12:40:05 -04:00
Rob
ef3d98963b docs(agent): add rule to track latest RNS and verify packages with rnid
§3 now requires confirming the latest upstream version each session and
verifying package signatures before install. Records that upstream is
migrating off GitHub (1.2.5 ≈ last GitHub release) toward rngit/rnpkg
self-hosting over Reticulum, and that signed wheels must be rnid-checked
against Mark Qvist's release identity rather than trusting a bare
`pip install` from PyPI (PyPI carries no .rsg).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 10:34:22 -04:00
Rob
9c64f9b805 docs(spec): add §17.5 — application protocols layered over Reticulum
Adds a reader-navigation subsection pointing RRC (Reticulum Relay Chat)
implementers at the RNS-layer sections they need. RRC carries its own
authoritative spec (rrc.kc1awv.net); this only maps its dependencies
onto §§ here and notes the skip-list (§5/§10/§11 unused). No RRC wire
format is restated and no normative claim is added.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 10:34:21 -04:00
Rob
1b955d19a9 docs(spec): fix §10.2 Resource integrity hash — prefix is not r, not hashed
§10.2 step 3 wrongly equated the random-hash prefix prepended to the
Resource body with the advertisement's `r` field, and step 5 fed that
prefix into the hash/expected_proof input. Upstream RNS uses two
distinct get_random_hash()[:4] values: a throwaway prefix the receiver
strips and discards, and self.random_hash (the adv `r` field). The
integrity hash is SHA256(uncompressed_plaintext || r) over the
prefix-stripped, decompressed body — exactly as §10.8 already stated.

- §10.2 steps 3 & 5 corrected to agree with §10.8
- §10.8: renamed misleading plaintext_with_random / data_with_random
- §10.12: wire-layering block rewritten to match
- README: errata entry under Spec corrections

Verified against RNS 1.2.5 (Resource.py:332,405,412,440-443,682-694,755).

Resolves #9.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 10:28:20 -04:00
Rob
3eea25977a docs(lxmf): enumerate FIELD_*/AM_*/RENDERER_*/PN_META_*/SF_* constants in §5.9
Adds a new §5.9 "LXMF field constants and helper specifiers" that
documents every numeric allocation in upstream LXMF/LXMF.py — the
20 top-level fields dict keys (0x01..0x0F + 0xFB..0xFF), the 20
FIELD_AUDIO mode bytes (Codec2 + Opus families), the 4 renderer
bytes, the 7 propagation-node metadata keys, and the
SF_COMPRESSION capability flag. Each entry is cross-referenced
against the section that describes its byte-level shape where one
exists (e.g. §5.9.2 for FIELD_IMAGE, §5.9.4 for FIELD_RENDERER).

Verifier in tools/verify_lxmf_fields.py loads upstream LXMF (0.9.7
in this run) and confirms every constant numerically — and fails if
upstream adds a new FIELD_/AM_/RENDERER_/PN_META_/SF_ constant that
isn't yet enumerated in the spec. Per agent.md §1, this promotes
the inventory from "implicit in code" to "verified against upstream."

The byte-level value shapes for fields whose contents are
application-defined (FIELD_EMBEDDED_LXMS, FIELD_TELEMETRY*,
FIELD_FILE_ATTACHMENTS, FIELD_COMMANDS, FIELD_RESULTS, FIELD_GROUP,
FIELD_EVENT, FIELD_RNR_REFS) are marked UNVERIFIED — future PRs
should pin them with captured test vectors.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 13:12:41 -04:00
Rob
c3795cb24c docs: add playbook.md + AGENTS.md template for new Reticulum impls
Two new files plus pointers from README.md and agent.md.

playbook.md — companion to agent.md. Where agent.md governs what
evidence is admissible as you add to the spec, playbook.md covers
how to navigate the work itself: triage checklist for wire-format
bugs, common debugging anti-patterns (the stale-sibling-binary trap,
trusting LLM training data on Reticulum specifics, chasing
intermittent symptoms with retries), the three layers of test
trustworthiness, and how to work productively in a code-as-spec
domain. Includes an incident registry seeded with the §6.2/§6.6
signed_data signalling bug surfaced in mobile-app today plus older
HEADER_2, REQUEST path_hash, DEST_LINK, and stale-binary incidents.
Append-only — every future interop fix gets a registry entry per §8.

templates/AGENTS.md — drop-in boilerplate for new Reticulum
implementation projects in any language. Uppercase plural matches
the emerging AGENTS.md convention (Claude Code, Codex, Cursor,
Copilot Workspace). Sections: read-these-first reading list,
cardinal rules summary, project-specific FILL-IN placeholders,
contributing-findings-back obligation, attribution. Project-specific
bits use HTML comments so they're obvious to edit. §5 attribution
points back to this repo and is mandatory per CC BY 4.0.

templates/README.md — names the template, says where to put it,
restates the attribution expectation.

agent.md and README.md updated with pointers to the new files so
anyone reading the front door of the repo finds them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:05:22 -04:00
Rob
073203abae Resolve issue #6 — LRRTT and HEADER_1 for link-addressed DATA (§6.4.2, §6.4.3)
Upstream RNS enforces two requirements in code that SPEC.md left implicit;
both caused silent message loss in a clean-room Go LXMF service against
upstream Python rns 1.2.4 / lxmf 0.9.7.

§6.4.2 LRRTT — initiator's link-activation packet
  - HEADER_1, DATA, dest_type=LINK (0x03), ctx=0xfe; body is
    `umsgpack.packb(rtt_seconds)` encrypted with the link's session keys.
  - The responder transitions HANDSHAKE→ACTIVE only on LRRTT receipt
    (Link.py:534-553), which is also what fires the link_established
    callback. LXMF's set_resource_strategy(ACCEPT_APP) is installed
    from that callback; without it, every RESOURCE_ADV the initiator
    sends hits the silent ACCEPT_NONE branch at Link.py:1087.

§6.4.3 Header type for post-handshake DATA and Resource
  - Link-addressed packets are routed via link_table, which forwards
    header bytes verbatim (Transport.py:1587-1622). HEADER_2 with a
    relay's transport_id therefore arrives at the destination intact
    and is dropped by packet_filter (Transport.py:1283-1285) as
    "for another transport instance".
  - Mandates HEADER_1 with no transport_id for all post-handshake
    link DATA / Resource / control packets regardless of hop count.
  - Asymmetry with LINKREQUEST (which IS path_table-routed and so
    HEADER_2-eligible) is spelled out.

Companion changes:
  - §6.4 renamed to "Session keys and link activation"; existing
    HKDF content moved into §6.4.1.
  - §2.5 LRRTT context-byte entry points at §6.4.2.
  - §12.5.2 (Link DATA forwarding) cross-references §6.4.3.
  - §14 failure-modes table: two new entries for the silent-drop
    chains documented above.
  - flows/send-link-lxmf.md step 4 strengthened (LRRTT is mandatory,
    not informational); step 6 corrected (Transport.outbound does NOT
    apply HEADER_1→HEADER_2 for link DATA — that conversion is
    path_table-keyed, link DATA is link_table-keyed).
  - test-vectors/links.json extended with an LRRTT entry: pinned
    rtt_seconds=0.05 + pinned 16-byte IV produces deterministic
    wire bytes for the encrypted body.
  - tools/regen_links.py drives the LRRTT generation with an
    os.urandom patch for the Token IV.
  - tools/verify_link_lrrtt.py (new) locks the wire claims:
    HEADER_1, ctx=0xfe, dest=link_id, body decrypts under
    derived_key to msgpack float64 matching rtt_seconds.

Citations all verified against installed RNS 1.2.4 / LXMF 0.9.7.
All 14 verifiers PASS.
2026-05-10 14:35:56 -04:00
Rob
5574d3bed3 flows: add lxmf-outbound-retry — process_outbound retry loop + state machine
Documents the outbound retry layer that wraps the existing per-method
send-* flows. Pinned to LXMF 0.9.7 / RNS 1.2.4 with literal-quoted
upstream source for every claim:

- 4-second tick cadence (PROCESSING_INTERVAL × JOB_OUTBOUND_INTERVAL)
- All seven retry constants (MAX_DELIVERY_ATTEMPTS, DELIVERY_RETRY_WAIT,
  PATH_REQUEST_WAIT, MAX_PATHLESS_TRIES, MESSAGE_EXPIRY,
  LINK_MAX_INACTIVITY, P_LINK_MAX_INACTIVITY) at LXMRouter.py:30-38
- Eight-state machine (GENERATING/OUTBOUND/SENDING/SENT/DELIVERED/
  REJECTED/CANCELLED/FAILED) at LXMessage.py:13-22
- The four terminal-state branches at top of process_outbound (lines
  2517-2558) and the three per-method retry branches (OPPORTUNISTIC
  2566-2592, DIRECT 2596-2673, PROPAGATED 2677-2730)
- fail_message semantics at LXMRouter.py:2395-2402

Includes a "what does NOT happen" section calling out common
misconceptions: no automatic DIRECT→PROPAGATED fallback, no
exponential backoff, no in-router persistence of pending_outbound,
MESSAGE_EXPIRY governs the propagation-node store not per-sender
retries, SENT is the terminal success state for PROPAGATED (not
DELIVERED).

No verifier needed per agent.md §1 — all claims are direct upstream
source citations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 10:09:02 -04:00
Rob
784343a33f Add /rns-update skill and gitignore for .claude scratch / per-user config
The /rns-update skill checks PyPI for new RNS / LXMF releases, runs the
verifier suite against the upgrade, samples five anchor citations for
drift, and proposes a pin-bump diff (without committing). It treats
"PyPI cold for >60 days" as a signal that upstream may have moved to
the Reticulum-network-only distribution promised in the 1.2.4 release
notes, and walks through the rngit / rnpkg fallback in step 9 — most
of which is to-be-built per todo.md "Upstream distribution shift".

.gitignore excludes per-user settings.local.json and the scratch
copies of upstream source (microReticulum, RNode_Firmware, repeater)
that get curl'd in during sessions. Those live under their upstream
licenses and don't belong in this repo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:58:44 -04:00
Rob
cfd0d8249b Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift
Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last
release that is also published to GitHub" — pip continues until rnpkg
is complete and RNS is self-hosting. All 13 verifiers pass against
1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed
between 1.2.0 and 1.2.4, so the changes here are purely currency:

- Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the
  verifier stays reproducible if upstream stops mirroring to PyPI
  before the migration is ready.
- Add an "Upstream distribution shift" watch-list to todo.md (local
  Reticulum node, repo destination hash, rnpkg install/upgrade
  commands, rsg signature verification, mirroring source citations).
- Bump SPEC.md frontmatter and re-anchor ~50 line citations across
  Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py,
  Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13
  to +31 lines; Transport.py was variable). Fix one numeric
  (MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU
  clamp citation pointed at the wrong location — corrected to point
  at the transit-relay clamp at Transport.py:1539-1556).
- Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap
  adoption, with citations to Resource.py:686-691 and Buffer.py:95-97
  plus a "do not use one-shot bz2.decompress()" warning.
- Re-anchor 11 flows/ files (version pins + ~30 line citations).
- Bump version labels in tools/README.md, test-vectors/README.md, and
  4 verifier docstrings + 2 hardcoded print strings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:42:25 -04:00
Rob
c51d914f92 README: add 'Spec corrections' section with the §2.1 IFAC bit-7 erratum
A focused errata section, distinct from a general changelog. Reserved
for cases where the spec said something wrong and an implementer who
pulled the bad version needs to fix their code. Today's bit-7 IFAC
correction is the first entry; covers the affected commit range
(8c4d550..0c2021e) and points at the upstream sources that pin down
the correct layout.

Feature additions and ordinary edits stay in git log; this section
exists so a returning reader sees the breakage at the README level
rather than depending on noticing an in-place callout buried inside
SPEC.md §2.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 21:51:44 -04:00
Rob
0c2021e757 Correct §2.1 flag byte: bit 7 is IFAC, not part of header_type
Reverts the wrong normative claim added in 8c4d550. The official
manual §4.6.3 documents byte 1 as ifac_flag(7) | header_type(6) |
context_flag(5) | propagation_type(4) | dest_type(3-2) | packet_type(1-0).
Upstream confirms:

- RNS/Packet.py:246 — `(self.flags & 0b01000000) >> 6` parses
  header_type as a 1-bit field at position 6.
- RNS/Transport.py:1003 — `bytes([raw[0] | 0x80, raw[1]])` sets the
  IFAC flag at bit 7 in Transport.transmit when ifac_identity is
  attached.

The reporter on issue #4 was correct: bit 7 has always been the IFAC
indicator. The 8c4d550 paragraph telling implementations "MUST NOT
treat bit 7 as a separate flag" is removed and replaced with the
correct layout, the upstream parse masks, and the IFAC sealing snippet
showing where the bit gets set on the wire.

A spec-correction callout in §2.1 documents the prior-version mistake
so anyone who consumed the bad guidance can identify the breakage.

verify_packet_header.py gains verify_ifac_bit_position() which locks
in the bit-7-is-IFAC invariant against future regression: it asserts
header_type's parse mask covers bit 6 only, never bit 7, and that the
IFAC mask 0x80 is disjoint from the header_type mask. The existing
flag-layout cases were always correct (header_type << 6 puts it at
bit 6); only the docstring described the wrong layout.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 21:36:23 -04:00
Rob
8c4d5506ed Address feedback issue #4 — clarify header_type/IFAC, canonical msgpack, ratchet-less interop, re-announce cadence
Per issue #4 (interop feedback from JS webclient implementer):

- §2.1: header_type is a 2-bit field, values 2/3 reserved, bit 7 not
  shared with IFAC. IFAC is a trailing field, not a flag bit.
- §5.6.1 (new): name umsgpack as canonical encoder for signing inputs;
  table of per-type encoding rules (str/bin/int width/float64).
- §7.3.5 (new): ratchet-less announces (context_flag=0) are accepted
  by every RNS 1.x receiver; trade-off is forward-secrecy only.
- §9.7: concrete RECOMMENDED re-announce ranges by deployment type;
  AVOID <60s and >30min thresholds with rationale.

Items #3 (§8.3 truncated) and #4 (§10.1 truncated) in the issue are
based on a stale fetch — both sections are fully present.
Items #2 (re-announce interval) and #7 (clockless senders) were already
covered (§7.5/§9.7 and §9.6 respectively); reply on the issue points
the reporter at them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 21:05:07 -04:00
Rob
0576f04c36 todo: mark microReticulum random_hash issue filed (#48)
Filed as attermann/microReticulum#48 on 2026-05-04.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:16:40 -04:00
Rob
68afe192e5 SPEC.md: collapsible ToC + collapse §11.6 NomadNet specifics
Adds a per-section table of contents at the top of the doc, wrapped
in <details> so it's collapsed by default (the spec body is visible
as soon as the file opens; click "Contents" to navigate). Every H2
and H3 heading is linked, including the unnumbered §14 failure-mode
categories.

Also wraps §11.6 (NomadNet specifics) in <details> — it's already
flagged "informational, not normative" and is the longest H3 sub-tree
in the document. Readers implementing only the §11 wire layer can
skim past it; readers implementing a NomadNet client one click away.

tools/_gen_toc.py regenerates the ToC by re-extracting headings.
Run it after adding/removing/renaming any H2 or H3.

Picked this over the per-layer-file split previously listed in todo
because the split would have broken ~37 cross-references (flow docs,
verifier docstrings, agent.md, README) for marginal reader benefit
at the document's current size (~3300 lines).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 22:05:17 -04:00
Rob
038e39401f Bootstrap test-vectors/{announces,lxmf,links}.json + regenerators
Three deterministic vector files complete the test-vectors/ bootstrap.
Each regenerator pins every random source so output is byte-identical
across runs against a fixed upstream RNS / LXMF version.

- announces.json: two vectors (no-ratchet + with-ratchet) signed by
  Alice. Determinism via patched Identity.get_random_hash + module-
  local time.time shim inside RNS.Destination.

- lxmf.json: two opportunistic-LXMF vectors Alice -> Bob, captures
  full plaintext (S5.2 layout) plus Token-encrypted ciphertext (S3).
  Determinism via fixed LXMessage.timestamp, ephemeral X25519 priv,
  and Token CBC IV.

- links.json: full Link handshake — LINKREQUEST + LRPROOF wire bytes,
  derived link_id, ECDH shared secret, and HKDF-derived session key
  that both initiator and responder MUST agree on. Determinism via
  three queued ephemeral priv-key blobs (initiator X25519, initiator
  Ed25519, responder X25519) consumed in source-call order at
  RNS/Link.py:285, :286, :278.

Status table in test-vectors/README.md and tools/README.md updated to
reflect the completed bootstrap. todo.md cleaned up to reflect actual
state (the previous "Open ⚠️ items needing a runtime verifier" section
was stale — all three verifiers were completed earlier).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 21:56:44 -04:00
Rob
1e375e52ea
§10/§11 NomadNet + Resource pipeline corrections (supersedes #2) (#3)
* §11/§10: NomadNet conventions + REQUEST/RESPONSE security clarifications

Expands the NomadNet-specific conventions documented in the spec
based on bytes-on-the-wire findings from a clean-room Kotlin port.
Three motivating bug classes from upstream interop:

1. Element [2] of the REQUEST envelope. Pre-clarification text said
   "application-defined bytes (often msgpack itself, or None)" which
   reads as "you can pre-msgpack-encode a dict and pass the bytes."
   Doing so produces a wire envelope where decode yields a `bytes`
   object for [2], not the dict the upstream `Node.py:109` /
   `LXMRouter.__get_handler` `isinstance(data, dict)` /
   `isinstance(data, list)` checks expect. Result: silent no-op
   on every form submission and every propagation /get round.
   §11.1 now spells out that the whole envelope is msgpacked once
   with `data` as a native msgpack value, with a worked example.

2. RESPONSE element [0] verification. The spec already documented
   request_id correlation but didn't flag it as a MUST for security.
   Without the check, a misbehaving / compromised transit relay can
   replay a stale RESPONSE from a prior request and the initiator
   accepts it as the answer to whatever's currently pending. Latent
   today on implementations that drive one in-flight request per
   link, but a real footgun the moment they add link reuse,
   partials, or pipelining. §11.2 now calls this out as a security
   requirement.

3. Resource size cap (§10.4). Today implementations pre-allocate
   buffers from `t` / `d` and have no cap on bz2 decompression
   output. A small (~tens of KB) compressed payload can legitimately
   expand to gigabytes. The HASHMAP_MAX_LEN chunk-count limit
   bounds raw on-wire chunks but does NOT bound post-decompression.
   §10.4 now recommends a per-application cap and notes that
   decompressors MUST also abort if the running output total
   exceeds the cap (defense in depth — a sender that lies about
   `d` would otherwise bypass the parse-time check).

Substantially expands §11.6 NomadNet specifics from a 4-bullet
informational paragraph to eight sub-sections covering:

- §11.6.1 Paths and the `nomadnetwork.node` aspect.
- §11.6.2 Form-data dict shape: `field_<name>` (widget values) and
  `var_<name>` (URL-query-style link parameters), both mapped to
  env vars by `Node.py:109-111`. Includes checkbox semantics
  (omit-unchecked, comma-join multi-select).
- §11.6.3 Link target syntax: same-node `/path`, cross-node
  `<32hex>:/path`, bare-hash default, `nnn@`/`lxmf@` shorthands
  with the `expand_shorthands` table. Notes hash-hex case
  normalization and rejection of separator-laden variants.
- §11.6.4 Page-level header conventions: `#!c=N` cache-TTL,
  `#!bg=` / `#!fg=` colors.
- §11.6.5 File downloads via `/file/...` returning
  `(file_handle, metadata_dict)`.
- §11.6.6 ALLOW_ALL vs ALLOW_LIST + the LINKIDENTIFY (0xFB)
  precondition for ALLOW_LIST pages, plus a privacy note that
  identify on every link pins long-term identity to the page
  operator.
- §11.6.7 Partial pages (server-side includes via `` `{path} ``).
- §11.6.8 Source map of NomadNet ↔ wire concept references.

All citations are to upstream `markqvist/NomadNet` master fetched
2026-05-04; the spec text is informational (not normative) since
the wire layer is §11 itself. The expansion is for clean-room
implementers who'd otherwise need to read several thousand lines
of `Browser.py` + `Node.py` to know what wire shape a NomadNet
server expects.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* §10/§11.1/§11.2: Resource pipeline + request_id corrections

Found by sideloading a clean-room Kotlin port (`thatSFguy/reticulum-mobile-app`)
on a phone and watching multi-packet NomadNet pages fail to load.
Each correction below was the difference between "spec implementer
got it wrong silently" and "page actually loads."

§11.1 — request_id formula precision:
  Prior text "16-byte truncated hash of `packed_request`" reads
  ambiguously. Several implementations (including the v0.1.54
  build of the Kotlin port) hashed the inner plaintext bytes —
  formula matches nothing the server sent, every RESPONSE drops.
  Upstream `Link.handle_request:1286` is `packet.getTruncatedHash()`
  i.e. SHA-256 of the on-the-wire packet hashable_part:
  `(raw[0] & 0x0F) || raw[2:]` (HEADER_1) / `... || raw[18:]` (HEADER_2).
  For Resource REQUESTs, the request_id IS plaintext-derived
  (carried in adv.q, set by initiator in `Resource.__init__:478`)
  because there's no single packet to hash. Updated text spells
  out both forms explicitly.

§11.2 — security note matched to corrected formula:
  Same fix in the implementer-gotcha box. Was telling clean-room
  ports to compute the wrong thing.

§10.6 — chunks are not individually encrypted:
  Per §10.2 step 4 the entire `random_hash || data` blob is link-
  encrypted ONCE, then sliced at step 6. Each wire chunk is just
  `outerToken[i*sdu : (i+1)*sdu]` with no per-chunk Token header.
  Receivers MUST hand chunks to the hashmap match without per-
  chunk decrypt. Spec text "parts are link-encrypted" reads
  ambiguously enough that decrypting per-chunk feels reasonable —
  added an explicit callout with the upstream slice loop and a
  warning that per-chunk `link.decrypt(chunk)` will HMAC-fail on
  every packet.

§10.8 — random_hash prefix is stripped, NOT compared to adv.r:
  Sender at `Resource.py:567` uses
  `RNS.Identity.get_random_hash()[:4]` for the prefix — a fresh
  random call, deliberately distinct from `self.random_hash`
  (the value `r` carries). A receiver that does
  `assert prefix == adv.r` rejects every legitimate Resource
  as corrupt. Step 3 of the assemble flow now says "strip and
  discard"; integrity is proven exclusively by step 5's
  `SHA256(data || r) == h`.

§11.6 (NomadNet specifics) and §10.4 (Resource size cap) carried
over from the closed PR #2 unchanged — those parts of #2 were
correct.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 20:35:31 -04:00
Rob
61bfc03413 Resolve issue #1 — five §7.2/§7.3 gaps from clean-room JS implementation
Reporter implemented §7.2.6 minimum-leaf path-request responder + §7.3
ratchet rotation in thatSFguy/reticulum-lora-webclient and surfaced
five small gaps. Each is fixed below; the first is a real spec
correction backed by a new runtime verifier.

#### 1. §7.3 dedup-mechanism claim was wrong (verified)

Earlier §7.3 claimed transit nodes dedup on '(destination_hash,
ratchet_pub)' tuples. Reporter pointed out this can't be right:
upstream's RATCHET_INTERVAL = 30 min × ANNOUNCE_INTERVAL = 5-15 min
means most upstream announces share a ratchet across 2-6 emissions.
If relays really dropped on ratchet_pub equality, upstream wouldn't
function.

Confirmed by new tools/verify_ratchet_dedup.py: builds two announces
with same ratchet_pub but distinct random_hash[:5], walks the
upstream replay-defence machinery (Transport.py:1707,1732,1745
'not random_blob in random_blobs' check) by hand. Both announces
ACCEPTED — dedup is keyed on random_blob, not on ratchet_pub.

§7.3 rewritten:
  - Drops the wrong dedup claim with an explicit ⚠️ Spec correction
    callout naming the bug.
  - Reframes ratchet rotation as forward-secrecy hygiene, not a
    mesh-visibility requirement.
  - Points at §4.5 step 6.3 / §4.1 for the actual replay-defence
    mechanism.
  - Documents upstream's at-most-every-30-min rotation cadence
    (rotate_ratchets is a no-op if RATCHET_INTERVAL hasn't elapsed).
  - Says clean-room MAY rotate per-announce or follow upstream's
    cadence — either is interop-correct.

#### 2. Path-response ratchet rotation guidance — §7.3.4 (new)

Added explicit guidance: path-response announces SHOULD reuse the
current ratchet rather than rotate. Burst-rotating on identical-target
path? requests would burn ratchet-ring slots without forward-secrecy
benefit. Upstream's no-op-if-recent gate enforces this implicitly.

#### 3. Leaf dedup-table size — §7.2.6 step 4

Added: 'A leaf-appropriate cap is 128–256 entries with FIFO eviction;
the upstream max_pr_tags = 32000 is sized for a transit node.'

#### 4. PR_TAG_WINDOW body cache for leaves — §7.2.6 trailing

Added: 'Leaves may skip the §7.2.5 PR_TAG_WINDOW body cache' with
explanation that step 4's dedup table already collapses identical-tag
retransmits and a leaf isn't fanning to multiple downstream relays.

#### 5. PLAIN destination recipe link — §7.2.1

Added: 'The path-request destination is a PLAIN destination ... per
the PLAIN/GROUP recipe in §1.4.3 (the identity == None branch).'
Surfaces the connection that's currently buried in §1.4 titled 'GROUP
destinations' but actually covers PLAIN too.

agent.md §5 audit table updated — §7.3 entry corrected to note the
prior 'verified' claim was actually mis-attributed; the test result
came from incidental random_hash rotation, not ratchet rotation.

13 of 13 verifiers in tools/ now pass.

Closes #1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 20:38:01 -04:00
Rob
366825c7a0 Add §17 implementation taxonomy: who needs which sections
Surfaced from real interop testing: a clean-room implementation
hit the §2.3 HEADER_1->HEADER_2 conversion bug, while category-1
clients (Sideband, MeshChat, NomadNet) couldn't have hit it
because they inherit upstream Python RNS's Transport.outbound
which does the conversion automatically.

The spec previously left this implicit. Without explicit guidance,
a reader could over-engineer (re-implementing things their
category-1 platform already handles) or under-engineer (assuming
'Sideband works, so my clean-room implementation works the same
way' when in fact Sideband works *because* upstream RNS handles
§2.3).

Four sub-sections:

  §17.1  The three categories of Reticulum app:
           Cat 1: Upstream-RNS-based (import RNS) — Sideband,
                  NomadNet, MeshChat, rncp, rnsh, rnstatus
           Cat 2: Wrappers / language bindings via FFI or
                  subprocess to upstream
           Cat 3: Clean-room implementations — microReticulum,
                  the Faketec repeater, anything from-scratch in
                  C++/Rust/JS/Kotlin/Swift

  §17.2  Section-relevance table by category. Wire formats are
         reference for cat 1/2, must-implement for cat 3.
         Behavioural guidance (§7, §12, §13, §14, §15, §16) is
         critical for cat 3, mostly informational for cat 1/2.

  §17.3  §2.3 worked example. Cat 1/2: don't write §2.3 code —
         upstream's Transport.outbound at line 1074-1083 does
         it automatically. Cat 3: implement it yourself or your
         packets get silently dropped by transit relays per
         line 1497 (which only forwards HEADER_2 with matching
         transport_id).

  §17.4  Pragmatic implication: a quick .claude/settings.local.json:      "Bash(python -c \"import RNS, LXMF; print\\('RNS:', RNS.__version__\\); print\\('LXMF:', LXMF.__version__\\)\")",
SPEC.md:| **1: Upstream-RNS-based** | Python application that does `import RNS` and uses upstream's `Reticulum` / `Transport` / `Identity` / `Destination` / `Packet` / `Link` directly. Inherits all wire-level behavior from upstream. | Sideband (Mark Qvist's flagship), NomadNet, [`liamcottle/reticulum-meshchat`](https://github.com/liamcottle/reticulum-meshchat), `rncp`, `rnsh`, `rnstatus`, anything in `pip show rns` example code |
SPEC.md:If you're not sure which category you're in: `grep -r "import RNS" your_codebase` is a quick check. Any hit means cat 1 (or cat 2 if it's behind an FFI wall). No hits means cat 3.
tools/regen_identities.py:import RNS
tools/verify_announce_app_data.py:import RNS
tools/verify_announce_app_data.py:import RNS.vendor.umsgpack as umsgpack
tools/verify_announce_roundtrip.py:import RNS
tools/verify_destination_hash.py:import RNS
tools/verify_link_handshake.py:import RNS
tools/verify_lxmf_opportunistic.py:import RNS
tools/verify_msgpack_quirk.py:import RNS
tools/verify_msgpack_quirk.py:import RNS.vendor.umsgpack as umsgpack
tools/verify_packet_header.py:import RNS
tools/verify_path_request.py:import RNS
tools/verify_proof_packet.py:import RNS
tools/verify_stamps.py:import RNS
tools/verify_token_crypto.py:import RNS
         tells you which category you're in. cat 1/2 readers can
         skip the implementation-depth sections; cat 3 readers
         need everything plus the verifiers as a regression suite.

Test vectors moves to §18, Source map to §19.

The provisional understanding is that this categorisation is
correct; if real-world testing reveals a category boundary that
doesn't hold the way described, the section gets revised or
removed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 16:02:59 -04:00
Rob
512ae058e7 Add tools/verify_stamps.py — runtime-lock §5.7
12th verifier in the suite. Locks in the LXMF stamps and tickets
spec (§5.7) by exercising LXMF.LXMStamper directly:

  - Workblock construction: deterministic for a given material;
    confirms exactly 768 KiB at the WORKBLOCK_EXPAND_ROUNDS = 3000
    default (matches spec §5.7.2 documented size).
  - PoW search-and-validate: brute-force search at target_cost = 4
    bits (fast — usually 1-16 iterations). Confirms stamp_valid and
    stamp_value round-trip; tampers a byte and confirms rejection.
  - LXMessage.validate_stamp end-to-end: accepts a valid PoW stamp
    on a synthesized LXMessage; rejects a tampered one.
  - Ticket shortcut: builds stamp = SHA256(ticket || message_id)
    by hand, confirms validate_stamp(target_cost, tickets=[...])
    accepts with the matching ticket and rejects with a wrong one.

target_cost = 4 keeps the test fast; real interop uses 8-16 bits.
The verifier doesn't claim to be a stamp-cost benchmark — that's a
separate use case.

12 of 12 verifiers in tools/ now pass against RNS 1.2.0 / LXMF 0.9.6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:13:59 -04:00
Rob
d6a83a525f Add §16 bounded-state inventory (dev-experience #6)
Single-table reference for every memory-bounded structure across
§1-§15, organised by scope: per-node, per-interface, per-destination,
per-Link, per-Resource, identity caches, LXMF, Channel.

Eight sub-tables cover ~30 named structures with their caps and
pointers to the explanatory section. Notable entries:

  - path_table, link_table, reverse_table, tunnels — unbounded;
    drained by TTL eviction in Transport.jobs
  - MAX_RANDOM_BLOBS = 32 (per-destination replay defence)
  - max_pr_tags = 32000 (path-request dedup)
  - hashlist_maxsize = 1,000,000 (packet dedup ring; half-purged
    on overflow)
  - MAX_HELD_ANNOUNCES = 256 per interface
  - RATCHET_COUNT = 512 per destination
  - WINDOW_MAX_FAST = 75 per Resource
  - known_destinations — UNBOUNDED in upstream; the main growth
    vector for embedded clients to manage explicitly

Closes with §16.9 'What this means for embedded targets' — explicit
guidance for ~64KB-RAM class clients (Faketec, RAK4631 stock) on
what to bound (known_destinations to 50-200 entries), what to
reject (Resource ADVs whose advertised n exceeds memory budget),
what to skip (transport-mode operation entirely), and what to
constrain (Resource WINDOW_MAX to SLOW=10 not FAST=75). Notes
that desktop rnsd typically settles around 50-200 MB.

Source map renumbered to §18; Test vectors stays at §17.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:12:23 -04:00
Rob
cf30d9b7b3 Add §8.6 AutoInterface multicast discovery (dev-experience #5)
LAN auto-detect protocol — drop a Reticulum node on any IPv6-capable
network, configure AutoInterface, peers find each other with zero
manual config.

Seven sub-sections:

  §8.6.1  IPv6 multicast group derivation: address built from
          SHA256(group_id) with multicast_address_type (4 bits,
          permanent/temporary) and discovery_scope (4 bits, link/
          admin/site/org/global). Default group_id = 'reticulum'.

  §8.6.2  UDP ports: 29716 discovery (multicast announces),
          29717 unicast probe (interface disambiguation),
          42671 data (Reticulum packets after peering).

  §8.6.3  Cadence: ANNOUNCE_INTERVAL = 1.6s, PEERING_TIMEOUT = 22s,
          PEER_JOB_INTERVAL = 4s, MCAST_ECHO_TIMEOUT = 6.5s.

  §8.6.4  Discovery announce body — msgpack with group_hash +
          MTU + optional IFAC seal. Peers from different groups
          on the same link don't accidentally peer.

  §8.6.5  Post-discovery: plain UDP datagrams on the data port,
          one full Reticulum packet per datagram, HW_MTU = 1196
          (Ethernet-MTU-friendly).

  §8.6.6  IFAC integration — peers with mismatched IFAC keys see
          each other's discovery but can't decode each other's
          data.

  §8.6.7  Source map.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:09:09 -04:00
Rob
7703cee748 Add §6.8 Channel mode (dev-experience #4)
Channel mode (CHANNEL = 0x0E context) is a continuous, bi-directional,
message-typed stream on top of an established Link. Distinct from
§11 REQUEST/RESPONSE (single-shot, client-server) and §10 Resources
(large unidirectional). NomadNet uses it for the live-channel API
beyond simple page fetches.

Six sub-sections:

  §6.8.1  Wire form: 6-byte big-endian fixed-prefix header
          (msgtype(2) || sequence(2) || length(2)) followed by
          payload, Token-encrypted by the link session key.

  §6.8.2  Reserved system types: SMT_STREAM_DATA = 0xff00.
          Application-defined types stay in 0x0000..0xfeff.

  §6.8.3  MSGTYPE registration: both endpoints must register
          matching message classes via register_message_type
          before sending/receiving that type.

  §6.8.4  Reliable delivery via §6.5 PROOF + sliding window with
          the same window-growth dynamics as §10 Resources.

  §6.8.5  Decision matrix: REQUEST/RESPONSE for one-shot RPC,
          Resources for one-shot large transfers, Channel for
          continuous bi-directional streams.

  §6.8.6  Source map across Channel.py.

Old §6.8 Source moved to §6.9.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:07:24 -04:00
Rob
f13985846c Add §15 time/clock requirements roundup (dev-experience #3)
Consolidates time-sensitive behaviour scattered across §4.1, §5.3,
§5.7.3, §5.8.5, §6.7.1, §7.1, §7.2, §7.5, §9.6, §10, §13.4 into
one reference for what kind of clock you need where.

Seven sub-sections:

  §15.1  Three clock kinds (wall time, boot-relative monotonic,
         hi-res monotonic). Embedded clean-rooms must be careful
         which call site needs which.

  §15.2  Required: monotonic seconds. Seven specific use sites
         that break a single-clock implementation if missing.
         All can be satisfied by boot-relative seconds.

  §15.3  random_hash timestamp encoding strategy for no-RTC
         devices: emit boot-relative seconds (look stale, lose
         path-replace ties — that's correct). Don't emit fully-
         random bytes (the §9.10 microReticulum bug — locks you
         in as 'latest' forever).

  §15.4  Wall-time-required (LXMF body timestamp, ticket expiry,
         propagation timebase). Tickets can't substitute —
         no-RTC devices must use PoW stamps instead.

  §15.5  Optional hi-res monotonic for diagnostics.

  §15.6  Explicit fails-vs-works inventory for a no-RTC,
         no-NTP-sync device. Net: opportunistic LXMF, propagated
         LXMF retrieval, and Links all work; only ticket-based
         shortcuts fail. A one-time clock sync flips most of the
         ⚠️ items to .

  §15.7  Source map across all sections that touch time.

Test vectors stays at §16; Source map renumbered to §17.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:05:47 -04:00
Rob
2be4830231 Add §14 failure-mode cheatsheet (dev-experience #2)
Symptom-keyed inverse index of §9. Eight tables organised by
problem domain — Identity/announce, Token crypto / opportunistic
LXMF, Link establishment / proofs, Resource transfers, Path
discovery, Transport / framing, LXMF specifics, Concurrency —
each mapping observable symptoms to root-cause sections and
relevant tools/verify_*.py scripts.

Closes Tier 2 #16 of the dev-experience todo. Section now serves
as the fault-finding entry point for new implementers: 'I see
symptom X' -> table row -> direct link to §N.M with full
explanation -> verifier that locks it in.

Worked-example entries for the high-cost interop bugs we caught
during the spec sweep (§1.3 byte order, §6.2 LRPROOF body,
§9.10 microReticulum random_hash, §6.5 implicit/explicit proof
length-dispatch, etc) so future readers get the diagnosis instantly.

Test vectors and Source map renumbered to §15 and §16.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:03:05 -04:00
Rob
70a24060b5 Add §13 threading/concurrency model (dev-experience #1)
The wire spec is silent on threading, but a clean-room client built
single-threaded mostly works for opportunistic LXMF and starts
breaking on Resource transfers and Link keepalives. This is the #1
cause of 'my client compiles and almost works but is flaky'.

Five sub-sections:

  §13.1  Long-running threads — Transport.jobloop (every 250ms,
         runs all maintenance), count_traffic_loop (every 1s
         bandwidth snapshots), per-Link Link.__watchdog_job
         (RTT-driven keepalive emission and STALE→CLOSED
         transitions), per-Resource Resource.__watchdog_job
         (retransmit timeouts), announce-handler callbacks fire
         on FRESH daemon threads per inbound announce, per-interface
         RX thread, process_announce_queue chained one-shot timers.

  §13.2  Lock inventory — 18 named Transport / Identity / Link /
         Resource / Destination locks. jobs_lock is the most
         aggressive: held for the entire jobs() body so parallel
         job invocations can't pile up.

  §13.3  Callback-thread guarantees: packet/link/receipt callbacks
         all run synchronously on the receive thread; only
         announce-handler callbacks run on fresh threads. Critical
         design implications:
           - Don't block the receive thread (queue-and-return).
           - Announce handlers race; lock shared state.
           - link_closed can fire from two paths (watchdog OR peer
             LINKCLOSE); make idempotent.

  §13.4  Implementation-private timing constants —
         job_interval = 250ms, links_check_interval = 1s,
         tables_cull_interval = 5s, hashlist_maxsize = 1M,
         WATCHDOG_MAX_SLEEP, PROCESSING_GRACE, SENDER_GRACE_TIME,
         etc. Don't scale below 100ms job_interval.

  §13.5  Source map.

Test vectors and Source map renumbered to §14 and §15. Other
section numbers unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 14:59:24 -04:00
Rob
e575da7d17 todo: add developer-experience gaps for follow-on work
Six items not strictly wire-format but high-value for clean-room
implementers, in priority order:

  Top three (most debugging-hour savings):
    §15 Threading / concurrency model — which loops run when,
        which callbacks fire on which thread, lock inventory
    §16 Failure-mode -> root-cause cheatsheet — symptom-keyed
        inverse index of §9 with worked examples
    §17 Time / clock requirements roundup — consolidates random_hash
        timestamps, LXMF clockless senders, ticket expiry, keepalive
        RTT, re-announce cadence into one no-RTC reference

  Medium:
    §6.x Channel mode (CHANNEL = 0x0E) — multiplexed app data over
        Link, used by NomadNet beyond page fetches
    §8.x AutoInterface multicast discovery — UDP magic for LAN
        peer auto-detect

  Appendix:
    Bounded-state inventory — single table of every memory-bounded
        structure for embedded implementers

Plus:
  - Marked the 'last-verified-against-rns' polish item done
    (already added to SPEC.md frontmatter in commit abf66b9).
  - Added a tools/verify_stamps.py todo to runtime-lock §5.7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 13:35:05 -04:00
Rob
abf66b9cef Add four more verifiers + receive-propagated flow + frontmatter version
Verifiers:
  tools/verify_proof_packet.py — locks in §6.5. Toggles
    Reticulum.__use_implicit_proof to test both modes; confirms
    Identity.prove emits 64B (implicit) or 96B (explicit) proof
    body; PacketReceipt.validate_proof accepts both lengths and
    rejects an 80B body.
  tools/verify_link_handshake.py — locks in §6.1, §6.2, §6.3, §6.6.
    Most importantly verifies the previously-corrected §6.2 LRPROOF
    body order (signature(64) || responder_X25519_pub(32) ||
    [signalling]) and §6.3 link_id offsets (N=2 for HEADER_1) by
    actually building a Link initiator-side, capturing the
    LINKREQUEST raw bytes, computing link_id by the spec recipe,
    running validate_request inline (since the upstream wrapper
    swallows exceptions), and confirming the responder's LRPROOF
    bytes match the spec layout. This was the single most
    interop-critical correction we made.
  tools/verify_rnode_split.py — locks in §8.3. Pure-function
    re-implementation of the canonical TX and RX state machines
    from RNode_Firmware.ino:359-446 + 716-742; tests header-byte
    layout, single-frame TX, split-frame TX (300B → 254+46 with
    shared header byte), all four RX state-machine cases (a/b/c/d
    from the spec table), and end-to-end TX/RX round-trip at
    sizes 50, 254, 255, 300, 508.
  tools/verify_msgpack_quirk.py — locks in §9.3. Confirms umsgpack
    distinguishes str (fixstr/0xa5) from bytes (bin8/0xc4); confirms
    LXMF.display_name_from_app_data parses bytes-encoded display
    names correctly and silently returns None (not crash) on
    str-encoded ones, matching the bug-tolerance documented in §9.3.

All 11 verifiers pass against RNS 1.2.0 / LXMF 0.9.6.

Plus:
  - SPEC.md frontmatter: 'Last verified against' line per agent.md §7.
  - flows/receive-propagated-lxmf.md: closing half of the propagated
    LXMF lifecycle. /get listing query, fetch query, ack-and-purge
    via the have_ids slot, message-bundle unpack and dispatch
    through lxmf_delivery.
  - tools/README.md status table refreshed; flows/README.md flips
    receive-propagated-lxmf.md to .

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:54:34 -04:00
Rob
75169b0631 Add three high-value verifiers: token crypto, announce, LXMF opportunistic
tools/verify_token_crypto.py — locks in §3:
    - Opportunistic Token encrypt/decrypt round-trip with full
      ephemeral_pub(32) || iv(16) || aes(...) || hmac(32) layout check.
    - HKDF salt = recipient.identity_hash verified by re-deriving
      the key by hand and confirming decrypt succeeds.
    - Link-derived Token form (no eph_pub prefix) round-trip.
    - HMAC-then-AES order proven by tampering each region: HMAC
      mismatch raises before AES decrypt.
    - PKCS#7 padding boundaries (1B and 16B plaintexts).

  tools/verify_announce_roundtrip.py — locks in §4 + §4.5:
    - Build via upstream Destination.announce(send=False).
    - Body layout walk with context_flag branching for the optional
      ratchet slot.
    - signed_data reconstruction per §4.2 with empty-bytes-not-absent
      ratchet rule.
    - dest_hash recompute per §1.2.
    - random_hash[5:10] is a recent unix_seconds timestamp per §4.1
      (corrected — confirms upstream emits the timestamp half).
    - Upstream validate_announce accepts.
    - Tamper detection: bit-flips in signature, public_key, name_hash,
      random_hash, app_data are all rejected.

  tools/verify_lxmf_opportunistic.py — locks in §5.1, §5.2, §5.5, §5.6
  plus §3 layered correctly:
    - Two identities (Alice, Bob) with mutual discovery.
    - LXMessage build with title, content, fields.
    - Body layout: dest(16) || src(16) || sig(64) || msgpack.
    - Opportunistic-form strip of leading dest_hash before encryption.
    - Encrypt to Bob via Token, decrypt as Bob, byte-identical
      round-trip.
    - Re-prepend dest_hash and run unpack_from_bytes; confirms
      signature_validated=True and title/content/fields preserved.

All three pass against RNS 1.2.0 / LXMF 0.9.6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:41:20 -04:00
Rob
282d5d59eb Add five companion flow docs
- flows/receive-resource.md: inverse of send-resource. ADV
    ingestion, accept/reject decision, request_next loop,
    receive_part insertion, assemble + decrypt + hash-validate,
    RESOURCE_PRF emission, multi-segment continuation.

  - flows/receive-link-lxmf.md: responder side of the link
    handshake plus inbound LXMF DATA handling. validate_request
    -> handshake -> prove (LRPROOF emission) -> link_established
    callback wires delivery_packet. PACKET-form inbound runs
    delivery_packet directly; RESOURCE-form inbound runs through
    delivery_resource_advertised + delivery_resource_concluded
    pipeline.

  - flows/send-announce.md: random_hash construction (5B random +
    5B BE-uint40 timestamp), optional ratchet rotation, signed_data
    assembly, sign + pack, the broadcast emission. Notes that
    ANNOUNCE packets are NOT encrypted (Packet.pack special-cases
    line 189-191) and the periodic re-announce loop drives 5-15min
    cadence.

  - flows/forward-announce.md: relay-side rebroadcast for
    transport-mode nodes. Eligibility checks (transport_enabled,
    not PATH_RESPONSE, not rate_blocked), announce_table queue,
    Transport.jobs drain with PATH_REQUEST_GRACE = 0.4s,
    per-interface announce_queue with ANNOUNCE_CAP = 2.0% airtime
    enforcement, lowest-hop-count-first emission order, hops byte
    increment, local-rebroadcast counter for loop break.

  - flows/send-propagated-lxmf.md: PROPAGATED method end to end.
    LXMessage.pack with body encrypted to recipient (propagation
    node never decrypts), Link establishment to the propagation
    node, optional propagation stamp (1000 PoW rounds vs 3000 for
    regular stamps), submission via Link DATA or Resource,
    state goes to SENT (not DELIVERED — recipient pulls via /get
    later per §5.8.3).

flows/README.md status table updated; receive-propagated-lxmf.md
added as the only remaining  flow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:21:05 -04:00
Rob
ee5ba48802 Add §12 transport-relay behaviour (Tier 3 — TIER 3 COMPLETE)
Closes Tier 3 in a single consolidated section because all five items
share state (path_table, announce_table, link_table, reverse_table,
tunnels) and are emergent behaviours of the same Transport.inbound
dispatch logic.

Seven sub-sections:

  §12.1  transport_enabled toggle — leaf clients populate path_table
         only for destinations they personally need; transport-mode
         nodes populate it for everything they hear about.

  §12.2  DATA forwarding rules — three-case branch on remaining_hops
         (>1 forward as HEADER_2 with new transport_id; ==1 strip
         transport_id and forward as HEADER_1 broadcast; ==0 local).
         LINKREQUEST forwarding extras (link_table entry + §6.6 MTU
         clamp). Non-LINKREQUEST gets a reverse_table entry.

  §12.3  ANNOUNCE rebroadcasting — announce_table retransmit queue,
         per-interface ANNOUNCE_CAP airtime budget, announce_queue
         drain order (lowest-hop-count first), random_blob replay
         defence with MAX_RANDOM_BLOBS sliding window, and the
         PATH_RESPONSE short-circuit (path-responses go on a
         specific interface, not broadcast).

  §12.4  Path table management — entry shape (IDX_PT_* indexes),
         three TTLs by interface mode (AP_PATH_TIME 1h, ROAMING_PATH_TIME
         4h, PATHFINDER_E 30 days), stale-paths eviction, persistence
         to storagepath/paths.

  §12.5  Reverse-table link transport — LRPROOF forwarding via
         link_table validation against the destination's known
         long-term Ed25519 pub, Link DATA forwarding once
         link_table[IDX_LT_VALIDATED] is set, PROOF receipt
         forwarding via reverse_table (one-shot pop on use,
         REVERSE_TIMEOUT bound for memory).

  §12.6  Tunnels and shared-instance protocol — discovery_path_requests
         recursive search (15s timeout), tunnels[] persistence across
         interface flap, shared-instance protocol (regular Reticulum
         packets over TCP loopback; the 'sharing' is Transport state,
         not wire format).

  §12.7  Source map.

Old §12 Test vectors -> §13; old §13 Source map -> §14. Section
order preserves protocol content before appendices.

TIER 3 COMPLETE. All Tier 1, 2, and 3 spec gaps closed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:16:26 -04:00
Rob
5f67277b62 Add §5.8 propagation node protocol (Tier 2 #8 — TIER 2 COMPLETE)
Closes Tier 2. Six sub-sections covering store-and-forward LXMF:

  §5.8.1  The lxmf.propagation destination, well-known name_hash
          e03a09b77ac21b22258e, four registered request handlers
          (/offer, /get, /stats, /sync) all reached via §11
          REQUEST/RESPONSE protocol on an active Link.

  §5.8.2  Peer-to-peer sync via /offer:
            data = [peering_key(32), [transient_id_1, ...]]
          Three response shapes: False (peer has all), True (peer
          wants all), [list] (peer wants subset). Wanted messages
          are bundled into a Resource carrying the full encrypted
          LXMF bodies — propagation nodes never decrypt.

  §5.8.3  Client retrieval via /get:
            data = [wanted_ids, have_ids, optional_limit_kb]
          Listing query (both None), fetch query (wanted_ids set),
          purge query (have_ids set). The propagation node only
          returns messages keyed to the requester's
          destination_hash — structural defense against
          mis-routing.

  §5.8.4  Peering keys: PoW with 25 rounds of workblock expansion
          (~6 KiB), amortized once per peering relationship.
          peering_id = self_identity_hash || remote_identity_hash.

  §5.8.5  Propagation node announce app_data: distinct 7-element
          msgpack array (vs §4.3's 2-element form for lxmf.delivery).
          Element [5] is a 3-list of [stamp_cost,
          stamp_cost_flexibility, peering_cost] — most common
          interop break is misparsing as a single integer.

  §5.8.6  Source map across LXMRouter, LXMPeer, LXStamper, LXMF.

Old §5.8 'Source' renamed to §5.9.

Tier 2 complete: 8 of 8 done. Moving to Tier 3 (transport-relay
specs).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:12:21 -04:00
Rob
c5fe9c13de Add §11 REQUEST/RESPONSE protocol (Tier 2 #7)
NomadNet pages aren't a separate wire format — they ride on a generic
Reticulum REQUEST/RESPONSE protocol that's also used by LXMF
propagation /get and any custom RPC. Spec covers:

  §11.1  REQUEST wire form. Single Link DATA packet (ctx=0x09)
         carrying msgpack [timestamp, path_hash(16), data] when
         it fits in link.mdu, or a Resource transfer with
         is_response=False otherwise.

  §11.2  RESPONSE wire form. Single Link DATA packet (ctx=0x0A)
         carrying msgpack [request_id(16), response] when it fits,
         or a Resource transfer with is_response=True. File-handle
         responses ride through the §10 Resource pipeline with
         optional metadata.

  §11.3  Path hash collision avoidance — paths hashed to 16 bytes
         (2^128 collision space, negligible in practice). The path
         string itself is not on the wire.

  §11.4  Authorization modes: ALLOW_NONE / ALLOW_LIST / ALLOW_ALL.
         ALLOW_LIST requires the requester to have called
         link.identify() first (LINKIDENTIFY ctx=0xFB).

  §11.5  RequestReceipt callback machinery on the initiator side.

  §11.6  NomadNet conventions (informational): paths like
         /page/foo.mu, msgpack form-field request data, file-handle
         responses for downloads. None of this is wire-spec.

Old §11 Test vectors -> §12; old §12 Source map -> §13. Sections
renumbered to keep protocol content before the appendix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:08:30 -04:00
Rob
e47e32cf8c Add §5.7 LXMF stamps + tickets (Tier 2 #6)
Five sub-sections covering the proof-of-work-or-ticket spam control
mechanism. Wire form (32B optional 5th element of msgpack body),
stamp generation algorithm (3000 rounds of HKDF expand → 768 KiB
workblock → SHA256 search for target_cost leading zeros), tickets
(16B pre-shared shortcut: stamp = SHA256(ticket || message_id)[:32]),
the FIELD_TICKET = 0x0C exchange format ([expires, ticket_bytes]),
stamp_cost field in announce app_data (§4.3) and the receiver-side
_enforce_stamps drop policy.

Minimum interop: implement PoW for outbound (so peers with
stamp_cost set will accept your messages), tolerate-but-not-validate
inbound (your peers won't refuse to talk to you for not enforcing
their own anti-spam). Full ticket support is a Tier-3 nice-to-have.

Old §5.7 'Source' moved to §5.8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:05:25 -04:00
Rob
d27f01946e Add §1.4 GROUP destinations (Tier 2 #4)
GROUP destinations use a pre-shared 64-byte symmetric key (32B
signing + 32B encryption split) with the same Token wire format as
Link-derived encryption — iv(16) || aes_ciphertext || hmac(32), no
eph_pub prefix. dest_hash recipe matches SINGLE with identity
optional. Spec covers key gen via Token.generate_key, wire form,
dest_hash variants, on-disk format (raw bytes, no header), and a
why-rarely-used note (no forward secrecy, key distribution
unsolved at the protocol layer).

Most LXMF interop clients can ignore GROUPs entirely but should
still recognize the 0x01 type byte to gracefully reject inbound
packets they can't decrypt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:03:31 -04:00
Rob
7931eb1d8c Add §8.4 RNode KISS handshake + §8.5 airtime caps (Tier 2 #3+#5)
§8.4: full bring-up sequence for a host driving an RNode over KISS.
Five sub-sections — command-byte inventory (28 commands cited from
RNode_Firmware/Framing.h:24-95), 12-step bring-up recipe, the
CMD_DETECT/DETECT_REQ(0x73)/DETECT_RESP(0x46) handshake with
asynchronous CMD_FW_VERSION/CMD_PLATFORM/CMD_MCU replies,
4-byte big-endian numerics for FREQUENCY/BANDWIDTH (subject to KISS
escape), and the RX sidecar format (RSSI = byte − 157,
SNR = signed Q6.2 / 4 dB).

§8.5: airtime caps via CMD_ST_ALOCK (0x0B) and CMD_LT_ALOCK (0x0C),
encoded as 2-byte big-endian uint16 of (limit_percent × 100).
Reticulum.ANNOUNCE_CAP = 2.0 is the upstream default. Pre-TX
carrier-sense is firmware-private — host-side clients driving an
RNode don't implement their own LBT, but native LoRa clients need
the airtime_bins algorithm from RNode_Firmware.ino:683-712.

Closes Tier 2 #3 and #5 (CSMA/airtime). Tier 2: 4 of 8 done.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:01:52 -04:00
Rob
22ee7636ef Add §6.7 KEEPALIVE / link teardown (Tier 2 #1+#2)
Documents the link control plane that's required for any client
that wants links to survive idle periods. Five sub-sections:

  §6.7.1  KEEPALIVE wire form: context = 0xFA, initiator-originated
          0xFF ping body → responder 0xFE pong reply body, both
          Token-encrypted by the link session key. Cadence formula
          RTT × (KEEPALIVE_MAX/KEEPALIVE_MAX_RTT) = RTT × 205.7,
          clamped to [5s, 360s]. Initial value is 360s before RTT
          is measured by validate_proof.

  §6.7.2  STALE → CLOSED transition. Watchdog moves link to STALE
          when last_inbound + 2*keepalive elapses, then on next
          watchdog pass emits LINKCLOSE and goes to CLOSED.
          teardown_reason = TIMEOUT.

  §6.7.3  LINKCLOSE wire form: context = 0xFC, body = 16-byte
          link_id Token-encrypted. Receiver MUST verify
          plaintext == link_id before accepting the close. After
          accepting, link.shared_key/derived_key zeroed for forward
          secrecy.

  §6.7.4  Teardown reason codes: TIMEOUT(0x01), INITIATOR_CLOSED
          (0x02), DESTINATION_CLOSED(0x03). Local-state values, not
          on the wire.

  §6.7.5  Six-step minimum-receiver-responsibility recipe.

Also marks Tier 2 implicit/explicit proof item done — already
covered as part of §6.5's Tier 1 #3 expansion.

Old §6.7 "Source" renumbered to §6.8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 11:59:13 -04:00