reticiulum-specification/test-vectors/README.md

52 lines
3.4 KiB
Markdown
Raw Normal View History

# Test vectors
Known-good byte sequences that any Reticulum-compatible implementation should be able to round-trip in both directions.
## Status
Re-anchor against RNS 1.2.4 / LXMF 0.9.7 + track upstream distribution shift Upstream RNS 1.2.4 (2026-05-07) announces it is "probably the last release that is also published to GitHub" — pip continues until rnpkg is complete and RNS is self-hosting. All 13 verifiers pass against 1.2.4 / 0.9.7; no wire-format, signing, or protocol behavior changed between 1.2.0 and 1.2.4, so the changes here are purely currency: - Pin tools/requirements.txt to rns==1.2.4 / lxmf==0.9.7 so the verifier stays reproducible if upstream stops mirroring to PyPI before the migration is ready. - Add an "Upstream distribution shift" watch-list to todo.md (local Reticulum node, repo destination hash, rnpkg install/upgrade commands, rsg signature verification, mirroring source citations). - Bump SPEC.md frontmatter and re-anchor ~50 line citations across Identity.py, Transport.py, Resource.py, Link.py, Reticulum.py, Packet.py, and LXMF/* (Identity.py drift was the heaviest at +13 to +31 lines; Transport.py was variable). Fix one numeric (MAX_RANDOM_BLOBS = 32 → 64) and one semantic (§6.6.3 LRPROOF MTU clamp citation pointed at the wrong location — corrected to point at the transit-relay clamp at Transport.py:1539-1556). - Update §10.4 decompression-bomb hazard to note upstream's 1.1.9 cap adoption, with citations to Resource.py:686-691 and Buffer.py:95-97 plus a "do not use one-shot bz2.decompress()" warning. - Re-anchor 11 flows/ files (version pins + ~30 line citations). - Bump version labels in tools/README.md, test-vectors/README.md, and 4 verifier docstrings + 2 hardcoded print strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 07:42:25 -04:00
Populated against RNS 1.2.4 / LXMF 0.9.7:
-`identities.json` — Alice + Bob identity vectors (regenerator: `../tools/regen_identities.py`, verifier: `../tools/verify_destination_hash.py`).
-`announces.json` — two announce vectors (no-ratchet + with-ratchet) signed by Alice (regenerator: `../tools/regen_announces.py`, verifier: `../tools/verify_announce_roundtrip.py`).
-`lxmf.json` — two opportunistic-LXMF vectors Alice → Bob (regenerator: `../tools/regen_lxmf.py`, verifier: `../tools/verify_lxmf_opportunistic.py`).
Resolve issue #6 — LRRTT and HEADER_1 for link-addressed DATA (§6.4.2, §6.4.3) Upstream RNS enforces two requirements in code that SPEC.md left implicit; both caused silent message loss in a clean-room Go LXMF service against upstream Python rns 1.2.4 / lxmf 0.9.7. §6.4.2 LRRTT — initiator's link-activation packet - HEADER_1, DATA, dest_type=LINK (0x03), ctx=0xfe; body is `umsgpack.packb(rtt_seconds)` encrypted with the link's session keys. - The responder transitions HANDSHAKE→ACTIVE only on LRRTT receipt (Link.py:534-553), which is also what fires the link_established callback. LXMF's set_resource_strategy(ACCEPT_APP) is installed from that callback; without it, every RESOURCE_ADV the initiator sends hits the silent ACCEPT_NONE branch at Link.py:1087. §6.4.3 Header type for post-handshake DATA and Resource - Link-addressed packets are routed via link_table, which forwards header bytes verbatim (Transport.py:1587-1622). HEADER_2 with a relay's transport_id therefore arrives at the destination intact and is dropped by packet_filter (Transport.py:1283-1285) as "for another transport instance". - Mandates HEADER_1 with no transport_id for all post-handshake link DATA / Resource / control packets regardless of hop count. - Asymmetry with LINKREQUEST (which IS path_table-routed and so HEADER_2-eligible) is spelled out. Companion changes: - §6.4 renamed to "Session keys and link activation"; existing HKDF content moved into §6.4.1. - §2.5 LRRTT context-byte entry points at §6.4.2. - §12.5.2 (Link DATA forwarding) cross-references §6.4.3. - §14 failure-modes table: two new entries for the silent-drop chains documented above. - flows/send-link-lxmf.md step 4 strengthened (LRRTT is mandatory, not informational); step 6 corrected (Transport.outbound does NOT apply HEADER_1→HEADER_2 for link DATA — that conversion is path_table-keyed, link DATA is link_table-keyed). - test-vectors/links.json extended with an LRRTT entry: pinned rtt_seconds=0.05 + pinned 16-byte IV produces deterministic wire bytes for the encrypted body. - tools/regen_links.py drives the LRRTT generation with an os.urandom patch for the Token IV. - tools/verify_link_lrrtt.py (new) locks the wire claims: HEADER_1, ctx=0xfe, dest=link_id, body decrypts under derived_key to msgpack float64 matching rtt_seconds. Citations all verified against installed RNS 1.2.4 / LXMF 0.9.7. All 14 verifiers PASS.
2026-05-10 14:35:56 -04:00
-`links.json` — full Link handshake vector (LINKREQUEST + LRPROOF + derived session key) Alice → Bob, plus an LRRTT packet (§6.4.2) emitted from the initiator with pinned IV and `rtt_seconds = 0.05` (regenerator: `../tools/regen_links.py`, verifiers: `../tools/verify_link_handshake.py`, `../tools/verify_link_lrrtt.py`).
All four files are byte-deterministic across runs: regenerators pin every random source (ephemeral keys, IVs, `random_hash` prefix + timestamp, LXMF timestamp) so the output is reproducible against a fixed upstream RNS / LXMF version.
See [`../agent.md`](../agent.md) §5 and [`../todo.md`](../todo.md) for the remaining bootstrap task list.
## Format (proposed)
Each vector lives in a per-domain JSON file, e.g.:
- `identities.json` — Alice + Bob with `encPriv`, `sigPriv`, `ratchetPriv` (hex), plus the derived `publicKey`, `identityHash`, `destinationHash` for `lxmf.delivery`
- `announces.json` — full hex of a signed announce packet, plus the inputs that produced it (display_name, ratchetPub, etc.)
- `lxmf.json` — sender + recipient identity, plaintext, expected ciphertext bytes
- `links.json` — LINKREQUEST + LRPROOF + derived session keys
Each entry should include:
```json
{
"description": "Alice's lxmf.delivery announce with ratchet, display_name='AliceTest'",
"inputs": { ... },
"expected_bytes_hex": "...",
"rns_version_at_generation": "1.2.0",
"generator_script": "tools/regen_announces.py"
}
```
The `generator_script` is the file in `../tools/` that, when run against upstream RNS, regenerates `expected_bytes_hex`. Keeping the generator alongside the vector lets a future contributor verify the vector still matches a newer upstream RNS.
## What needs to round-trip
For the spec to claim "an implementation that passes all test vectors interoperates with upstream", the vectors must cover:
1. **Identity construction** — given the same private-key inputs, derive the same public key, identity hash, destination hash.
2. **Announce build + parse** — build a signed announce; verify the same bytes come back through upstream's parser; verify upstream-built announces parse correctly.
3. **Token encrypt + decrypt** — bidirectional, with both ratchet and long-term keys.
4. **Opportunistic LXMF** — full plaintext → ciphertext → plaintext round-trip, signature valid both ways.
5. **Link handshake** — LINKREQUEST built by client A, LRPROOF computed by upstream as B, both arrive at the same `link_id` and session keys.
6. **Link-delivered LXMF** — body packed by client, decrypted + parsed by upstream.
A separate vector set for FAILURE cases is also useful: malformed announces, expired ratchets, mismatched signatures. An implementation should reject those as a regression-prevention measure.