2026-05-03 09:38:46 -04:00
|
|
|
# Verifier scripts
|
|
|
|
|
|
|
|
|
|
Self-contained Python scripts that test claims in [`../SPEC.md`](../SPEC.md) against the upstream RNS / LXMF Python stack.
|
|
|
|
|
|
|
|
|
|
## Conventions
|
|
|
|
|
|
|
|
|
|
- Each script verifies one claim or one related cluster of claims.
|
|
|
|
|
- Exit code 0 on PASS, non-zero on FAIL.
|
|
|
|
|
- Print a one-line PASS/FAIL summary plus a unified diff or hex dump on mismatch.
|
|
|
|
|
- Reference the SPEC.md section the script verifies in a docstring at the top.
|
|
|
|
|
|
|
|
|
|
## Required environment
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
pip install rns lxmf
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The scripts read `RNS.__version__` at startup and print it in their output so a future reader can tell which RNS version a verification ran against.
|
|
|
|
|
|
|
|
|
|
## Status
|
|
|
|
|
|
Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift
Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last
release that is also published to GitHub" — pip continues until rnpkg
is complete and RNS is self-hosting. All 13 verifiers pass against
1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed
between 1.2.0 and 1.2.4, so the changes here are purely currency:
- Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the
verifier stays reproducible if upstream stops mirroring to PyPI
before the migration is ready.
- Add an "Upstream distribution shift" watch-list to todo.md (local
Reticulum node, repo destination hash, rnpkg install/upgrade
commands, rsg signature verification, mirroring source citations).
- Bump SPEC.md frontmatter and re-anchor ~50 line citations across
Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py,
Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13
to +31 lines; Transport.py was variable). Fix one numeric
(MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU
clamp citation pointed at the wrong location — corrected to point
at the transit-relay clamp at Transport.py:1539-1556).
- Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap
adoption, with citations to Resource.py:686-691 and Buffer.py:95-97
plus a "do not use one-shot bz2.decompress()" warning.
- Re-anchor 11 flows/ files (version pins + ~30 line citations).
- Bump version labels in tools/README.md, test-vectors/README.md, and
4 verifier docstrings + 2 hardcoded print strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:42:25 -04:00
|
|
|
Populated against RNS 1.2.4 / LXMF 0.9.7:
|
2026-05-03 10:14:51 -04:00
|
|
|
|
|
|
|
|
| Script | Verifies SPEC.md section | Status |
|
|
|
|
|
|---|---|---|
|
Fix and expand §1.3 — on-disk identity format (real spec bug!)
Closes Tier 1 #6 and the entire Tier 1 sweep. Previous §1.3 said the
on-disk byte order was Ed25519_priv(32) || X25519_priv(32) ("opposite
of the public_key concatenation"). That was WRONG.
Verified empirically against RNS 1.2.0 by round-tripping the existing
test vectors through Identity.to_file and reading the bytes back:
disk = X25519_priv(32) || Ed25519_priv(32) # same as public_key
This matches Identity.get_private_key() at RNS/Identity.py:694-698:
return self.prv_bytes + self.sig_prv_bytes
where prv_bytes is X25519 (line 679) and sig_prv_bytes is Ed25519
(line 682). It also matches load_private_key at line 706-717.
Implementations following the prior spec wording would have written
identity files that fail to load on upstream RNS — a real interop
break that would have been very hard to debug because the failure is
in keypair-loading, before any signature operation runs.
§1.3 rewritten and expanded:
- Correct byte order with citation to upstream code.
- 64-byte raw-blob format with explicit "no header / no version /
no checksum / no encryption".
- File-system facts: no chmod, expected to live in OS-protected
storage, filename is caller-controlled.
- from_bytes HAZARD note: feeding raw random bytes skips the
`cryptography` library's keypair-generation invariants
(X25519 RFC 7748 §5 scalar clamping etc).
- Cross-implementation portability follows automatically because
there's nothing in the file but the bytes.
- ⚠️ Spec correction callout warning future readers about the
previous wording so the bug history is on record.
tools/verify_destination_hash.py extended with a §1.3 to_file /
from_file round-trip section. For each test vector it now:
- writes the identity via to_file
- asserts the on-disk file is exactly 64 bytes
- asserts disk[:32] hex == expected x25519_priv_hex
- asserts disk[32:64] hex == expected ed25519_priv_hex
- reloads via from_file and asserts identity_hash invariance
This is what would have caught the bug if it had been there from the
start. tools/README.md updated to reflect §1.3 coverage.
Cumulative Tier 1 status: 6 of 6 done. A from-scratch client built
from §1-§9 + §10 + §11 + flows/ can now interop with upstream
Reticulum / LXMF / RNode for identity, announce, opportunistic LXMF
DATA, Resource fragmentation, regular PROOF receipts, link
handshakes with MTU/mode signalling, path-? discovery, and
KISS/HDLC/RNode-air-frame framing. Tiers 2 and 3 remain open in the
todo for follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 11:54:54 -04:00
|
|
|
| `verify_destination_hash.py` | §1.1, §1.2, §1.3 — identity composition, `dest_hash = SHA256(name_hash \|\| identity_hash)[:16]`, on-disk private-key round-trip via `to_file`/`from_file` | ✅ |
|
2026-05-03 10:14:51 -04:00
|
|
|
| `verify_packet_header.py` | §2.1, §2.2, §2.3 — flag byte layout, HEADER_1/HEADER_2 form, originator HEADER_1→HEADER_2 conversion via upstream `Transport.outbound` | ✅ |
|
Add four more verifiers + receive-propagated flow + frontmatter version
Verifiers:
tools/verify_proof_packet.py — locks in §6.5. Toggles
Reticulum.__use_implicit_proof to test both modes; confirms
Identity.prove emits 64B (implicit) or 96B (explicit) proof
body; PacketReceipt.validate_proof accepts both lengths and
rejects an 80B body.
tools/verify_link_handshake.py — locks in §6.1, §6.2, §6.3, §6.6.
Most importantly verifies the previously-corrected §6.2 LRPROOF
body order (signature(64) || responder_X25519_pub(32) ||
[signalling]) and §6.3 link_id offsets (N=2 for HEADER_1) by
actually building a Link initiator-side, capturing the
LINKREQUEST raw bytes, computing link_id by the spec recipe,
running validate_request inline (since the upstream wrapper
swallows exceptions), and confirming the responder's LRPROOF
bytes match the spec layout. This was the single most
interop-critical correction we made.
tools/verify_rnode_split.py — locks in §8.3. Pure-function
re-implementation of the canonical TX and RX state machines
from RNode_Firmware.ino:359-446 + 716-742; tests header-byte
layout, single-frame TX, split-frame TX (300B → 254+46 with
shared header byte), all four RX state-machine cases (a/b/c/d
from the spec table), and end-to-end TX/RX round-trip at
sizes 50, 254, 255, 300, 508.
tools/verify_msgpack_quirk.py — locks in §9.3. Confirms umsgpack
distinguishes str (fixstr/0xa5) from bytes (bin8/0xc4); confirms
LXMF.display_name_from_app_data parses bytes-encoded display
names correctly and silently returns None (not crash) on
str-encoded ones, matching the bug-tolerance documented in §9.3.
All 11 verifiers pass against RNS 1.2.0 / LXMF 0.9.6.
Plus:
- SPEC.md frontmatter: 'Last verified against' line per agent.md §7.
- flows/receive-propagated-lxmf.md: closing half of the propagated
LXMF lifecycle. /get listing query, fetch query, ack-and-purge
via the have_ids slot, message-bundle unpack and dispatch
through lxmf_delivery.
- tools/README.md status table refreshed; flows/README.md flips
receive-propagated-lxmf.md to ✅.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:54:34 -04:00
|
|
|
| `verify_token_crypto.py` | §3 — Token encrypt/decrypt, HKDF salt = identity_hash, HMAC-then-AES order, PKCS#7 padding | ✅ |
|
2026-05-03 10:14:51 -04:00
|
|
|
| `verify_announce_app_data.py` | §4.3 — LXMF announce app_data 2-element form, parser tolerance | ✅ |
|
Add four more verifiers + receive-propagated flow + frontmatter version
Verifiers:
tools/verify_proof_packet.py — locks in §6.5. Toggles
Reticulum.__use_implicit_proof to test both modes; confirms
Identity.prove emits 64B (implicit) or 96B (explicit) proof
body; PacketReceipt.validate_proof accepts both lengths and
rejects an 80B body.
tools/verify_link_handshake.py — locks in §6.1, §6.2, §6.3, §6.6.
Most importantly verifies the previously-corrected §6.2 LRPROOF
body order (signature(64) || responder_X25519_pub(32) ||
[signalling]) and §6.3 link_id offsets (N=2 for HEADER_1) by
actually building a Link initiator-side, capturing the
LINKREQUEST raw bytes, computing link_id by the spec recipe,
running validate_request inline (since the upstream wrapper
swallows exceptions), and confirming the responder's LRPROOF
bytes match the spec layout. This was the single most
interop-critical correction we made.
tools/verify_rnode_split.py — locks in §8.3. Pure-function
re-implementation of the canonical TX and RX state machines
from RNode_Firmware.ino:359-446 + 716-742; tests header-byte
layout, single-frame TX, split-frame TX (300B → 254+46 with
shared header byte), all four RX state-machine cases (a/b/c/d
from the spec table), and end-to-end TX/RX round-trip at
sizes 50, 254, 255, 300, 508.
tools/verify_msgpack_quirk.py — locks in §9.3. Confirms umsgpack
distinguishes str (fixstr/0xa5) from bytes (bin8/0xc4); confirms
LXMF.display_name_from_app_data parses bytes-encoded display
names correctly and silently returns None (not crash) on
str-encoded ones, matching the bug-tolerance documented in §9.3.
All 11 verifiers pass against RNS 1.2.0 / LXMF 0.9.6.
Plus:
- SPEC.md frontmatter: 'Last verified against' line per agent.md §7.
- flows/receive-propagated-lxmf.md: closing half of the propagated
LXMF lifecycle. /get listing query, fetch query, ack-and-purge
via the have_ids slot, message-bundle unpack and dispatch
through lxmf_delivery.
- tools/README.md status table refreshed; flows/README.md flips
receive-propagated-lxmf.md to ✅.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:54:34 -04:00
|
|
|
| `verify_announce_roundtrip.py` | §4.1, §4.2, §4.5 — announce body layout, signature, dest_hash recompute, tamper rejection | ✅ |
|
|
|
|
|
| `verify_lxmf_opportunistic.py` | §5.1, §5.2, §5.5, §5.6 — full identity → encrypt → decrypt → parse round-trip | ✅ |
|
|
|
|
|
| `verify_proof_packet.py` | §6.5 — implicit (64B) and explicit (96B) proof body forms, validator length-dispatch | ✅ |
|
|
|
|
|
| `verify_link_handshake.py` | §6.1, §6.2, §6.3, §6.6 — LINKREQUEST/LRPROOF body order, link_id derivation, signalling | ✅ |
|
Resolve issue #6 — LRRTT and HEADER_1 for link-addressed DATA (§6.4.2, §6.4.3)
Upstream RNS enforces two requirements in code that SPEC.md left implicit;
both caused silent message loss in a clean-room Go LXMF service against
upstream Python rns 1.2.4 / lxmf 0.9.7.
§6.4.2 LRRTT — initiator's link-activation packet
- HEADER_1, DATA, dest_type=LINK (0x03), ctx=0xfe; body is
`umsgpack.packb(rtt_seconds)` encrypted with the link's session keys.
- The responder transitions HANDSHAKE→ACTIVE only on LRRTT receipt
(Link.py:534-553), which is also what fires the link_established
callback. LXMF's set_resource_strategy(ACCEPT_APP) is installed
from that callback; without it, every RESOURCE_ADV the initiator
sends hits the silent ACCEPT_NONE branch at Link.py:1087.
§6.4.3 Header type for post-handshake DATA and Resource
- Link-addressed packets are routed via link_table, which forwards
header bytes verbatim (Transport.py:1587-1622). HEADER_2 with a
relay's transport_id therefore arrives at the destination intact
and is dropped by packet_filter (Transport.py:1283-1285) as
"for another transport instance".
- Mandates HEADER_1 with no transport_id for all post-handshake
link DATA / Resource / control packets regardless of hop count.
- Asymmetry with LINKREQUEST (which IS path_table-routed and so
HEADER_2-eligible) is spelled out.
Companion changes:
- §6.4 renamed to "Session keys and link activation"; existing
HKDF content moved into §6.4.1.
- §2.5 LRRTT context-byte entry points at §6.4.2.
- §12.5.2 (Link DATA forwarding) cross-references §6.4.3.
- §14 failure-modes table: two new entries for the silent-drop
chains documented above.
- flows/send-link-lxmf.md step 4 strengthened (LRRTT is mandatory,
not informational); step 6 corrected (Transport.outbound does NOT
apply HEADER_1→HEADER_2 for link DATA — that conversion is
path_table-keyed, link DATA is link_table-keyed).
- test-vectors/links.json extended with an LRRTT entry: pinned
rtt_seconds=0.05 + pinned 16-byte IV produces deterministic
wire bytes for the encrypted body.
- tools/regen_links.py drives the LRRTT generation with an
os.urandom patch for the Token IV.
- tools/verify_link_lrrtt.py (new) locks the wire claims:
HEADER_1, ctx=0xfe, dest=link_id, body decrypts under
derived_key to msgpack float64 matching rtt_seconds.
Citations all verified against installed RNS 1.2.4 / LXMF 0.9.7.
All 14 verifiers PASS.
2026-05-10 14:35:56 -04:00
|
|
|
| `verify_link_lrrtt.py` | §6.4.2, §6.4.3 — LRRTT wire form, HEADER_1 header, dest_type=LINK, ctx=0xfe, link-form Token body, msgpack float64 plaintext | ✅ |
|
2026-05-03 10:14:51 -04:00
|
|
|
| `verify_path_request.py` | §1.2 well-known hashes, §7.1 LXMF path-preamble gating | ✅ |
|
Add four more verifiers + receive-propagated flow + frontmatter version
Verifiers:
tools/verify_proof_packet.py — locks in §6.5. Toggles
Reticulum.__use_implicit_proof to test both modes; confirms
Identity.prove emits 64B (implicit) or 96B (explicit) proof
body; PacketReceipt.validate_proof accepts both lengths and
rejects an 80B body.
tools/verify_link_handshake.py — locks in §6.1, §6.2, §6.3, §6.6.
Most importantly verifies the previously-corrected §6.2 LRPROOF
body order (signature(64) || responder_X25519_pub(32) ||
[signalling]) and §6.3 link_id offsets (N=2 for HEADER_1) by
actually building a Link initiator-side, capturing the
LINKREQUEST raw bytes, computing link_id by the spec recipe,
running validate_request inline (since the upstream wrapper
swallows exceptions), and confirming the responder's LRPROOF
bytes match the spec layout. This was the single most
interop-critical correction we made.
tools/verify_rnode_split.py — locks in §8.3. Pure-function
re-implementation of the canonical TX and RX state machines
from RNode_Firmware.ino:359-446 + 716-742; tests header-byte
layout, single-frame TX, split-frame TX (300B → 254+46 with
shared header byte), all four RX state-machine cases (a/b/c/d
from the spec table), and end-to-end TX/RX round-trip at
sizes 50, 254, 255, 300, 508.
tools/verify_msgpack_quirk.py — locks in §9.3. Confirms umsgpack
distinguishes str (fixstr/0xa5) from bytes (bin8/0xc4); confirms
LXMF.display_name_from_app_data parses bytes-encoded display
names correctly and silently returns None (not crash) on
str-encoded ones, matching the bug-tolerance documented in §9.3.
All 11 verifiers pass against RNS 1.2.0 / LXMF 0.9.6.
Plus:
- SPEC.md frontmatter: 'Last verified against' line per agent.md §7.
- flows/receive-propagated-lxmf.md: closing half of the propagated
LXMF lifecycle. /get listing query, fetch query, ack-and-purge
via the have_ids slot, message-bundle unpack and dispatch
through lxmf_delivery.
- tools/README.md status table refreshed; flows/README.md flips
receive-propagated-lxmf.md to ✅.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 12:54:34 -04:00
|
|
|
| `verify_rnode_split.py` | §8.3 — RNode air-frame split-packet TX/RX state machines | ✅ |
|
|
|
|
|
| `verify_msgpack_quirk.py` | §9.3 — encoding name as bytes vs str affects upstream parsing | ✅ |
|
2026-05-03 15:13:59 -04:00
|
|
|
| `verify_stamps.py` | §5.7 — workblock determinism, PoW stamp search/validate, ticket shortcut | ✅ |
|
2026-05-03 20:38:01 -04:00
|
|
|
| `verify_ratchet_dedup.py` | §7.3 / §4.5 step 6.3 — confirms replay defence is keyed on `random_blob`, NOT on `(dest_hash, ratchet_pub)` | ✅ |
|
2026-05-03 10:14:51 -04:00
|
|
|
| `regen_identities.py` | regenerates `test-vectors/identities.json` | ✅ |
|
Bootstrap test-vectors/{announces,lxmf,links}.json + regenerators
Three deterministic vector files complete the test-vectors/ bootstrap.
Each regenerator pins every random source so output is byte-identical
across runs against a fixed upstream RNS / LXMF version.
- announces.json: two vectors (no-ratchet + with-ratchet) signed by
Alice. Determinism via patched Identity.get_random_hash + module-
local time.time shim inside RNS.Destination.
- lxmf.json: two opportunistic-LXMF vectors Alice -> Bob, captures
full plaintext (S5.2 layout) plus Token-encrypted ciphertext (S3).
Determinism via fixed LXMessage.timestamp, ephemeral X25519 priv,
and Token CBC IV.
- links.json: full Link handshake — LINKREQUEST + LRPROOF wire bytes,
derived link_id, ECDH shared secret, and HKDF-derived session key
that both initiator and responder MUST agree on. Determinism via
three queued ephemeral priv-key blobs (initiator X25519, initiator
Ed25519, responder X25519) consumed in source-call order at
RNS/Link.py:285, :286, :278.
Status table in test-vectors/README.md and tools/README.md updated to
reflect the completed bootstrap. todo.md cleaned up to reflect actual
state (the previous "Open ⚠️ items needing a runtime verifier" section
was stale — all three verifiers were completed earlier).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 21:56:44 -04:00
|
|
|
| `regen_announces.py` | regenerates `test-vectors/announces.json` (deterministic announce wire bytes, with and without ratchet) | ✅ |
|
|
|
|
|
| `regen_lxmf.py` | regenerates `test-vectors/lxmf.json` (deterministic opportunistic-LXMF plaintext + Token ciphertext) | ✅ |
|
|
|
|
|
| `regen_links.py` | regenerates `test-vectors/links.json` (deterministic LINKREQUEST + LRPROOF + derived session key) | ✅ |
|
2026-05-03 10:14:51 -04:00
|
|
|
|
2026-06-08 12:47:32 -07:00
|
|
|
See [`../agent.md`](../agent.md) §3 and [`../todo.md`](../todo.md) for the evidence model and remaining priority order.
|
