reticiulum-specification/agent.md

# AGENT.md — Instructions for AI agents working on this repository

This file tells AI coding agents (Claude Code, Codex, Cursor, Copilot Workspace, etc.) how to maintain this repository responsibly. The goal of this repo is to be the **canonical, verifiable** byte-level reference for the Reticulum protocol. Speculation is acceptable as a starting point but must be clearly marked as such until tested against the upstream Python implementation.

If you are an AI agent reading this: follow these rules. If you are a human reviewing an agent's PR: enforce these rules.

**Working on a Reticulum implementation (not just this spec repo) — start with [`playbook.md`](playbook.md).** This file (`agent.md`) governs what evidence is admissible when adding to the spec; `playbook.md` covers how to troubleshoot interop bugs, design tests that don't lie to you, and navigate a protocol whose primary documentation is its source code. Both files are short; read both.

---

## 1. Verification is mandatory

Every claim in this repository falls into one of three states:

| State | Marker | Meaning |
|---|---|---|
| **Verified** | (no marker, or `[verified]`) | Claim has been tested against the upstream RNS Python stack with a runnable test, OR is a direct citation of upstream source code with file + line. |
| **Unverified** | `> ⚠️ **UNVERIFIED:**` callout | Plausible from source-reading or other-implementation behavior, but not directly tested against upstream Python. May be wrong. |
| **Speculation** | `> 🔮 **SPECULATION:**` callout | Hypothesis based on reasoning about how the protocol *probably* works, with no source citation OR runtime test. Must be resolved (verified or removed) before any release. |

Do not silently promote a claim from **unverified** to **verified** without doing the actual verification. The marker exists so a future reader can trust unmarked content.

### What counts as verification

In rough order of strength:

1. **Round-trip with upstream Python RNS.** A small script in `tools/` that loads the relevant RNS module, performs the operation in both directions, and compares bytes. Strongest evidence.
2. **Direct upstream source citation.** File path and line number in the standard `pip install rns lxmf` install layout (`RNS/`, `LXMF/`). Acceptable for behaviors that are hard to test (e.g. multi-hop forwarding rules).
3. **Wire capture with byte-level diff.** Capturing actual upstream emission (e.g. tcpdump on `rnsd`) and showing it matches the spec.

What does **NOT** count as verification:

- "It worked on my Sideband install" without a script anyone can re-run
- "The webclient does it this way" — webclient may have the same bug
- "Reading this code, I think it does X" — that's source-reading, which is acceptable as a citation but only if the citation is included
- "GPT/Claude/Gemini said so" — no

---

## 2. Workflow for adding a new claim

1. Draft the claim in the relevant section of `SPEC.md` (or the future per-layer file).
2. Mark it `> 🔮 **SPECULATION:**` if it's pure reasoning, or `> ⚠️ **UNVERIFIED:**` if there's a source citation but no runtime test.
3. Write a verifier in `tools/`:
   - For byte-level claims: a Python script that produces the expected upstream bytes and compares to a literal `expected = bytes.fromhex(...)`.
   - For behavioral claims (e.g. "originator inserts transport_id at offset 2 for >1-hop paths"): a script that exercises `RNS.Transport` with a known path table and dumps `process_outgoing` calls.
4. Run the verifier. If it confirms, remove the marker and add a `(verified by tools/<script>.py)` parenthetical to the claim.
5. If the verifier disproves, EITHER fix the claim AND the verifier, OR delete the claim entirely. Do not commit a verifier that hand-waves away a discrepancy.

PRs must include the verifier scripts. Don't commit a "verified" claim without the script that backs it.

---

## 3. Three-tier evidence model

This repository keeps the breadth of the existing Claude-assisted extraction, but promotes claims through the narrower three-tier process used for the LXMF extraction. Treat existing prose as useful candidate material until the relevant tier is satisfied.

### Tier 1 — Source analysis report

Use this tier to answer one narrow protocol question, not to rewrite a whole subsystem. The output is an analysis note, issue comment, or draft SPEC.md text that includes:

- The exact upstream RNS/LXMF version or commit inspected.
- The files, functions, constants, and line numbers that define the behavior.
- A byte layout or state-machine description precise enough to test.
- A clear list of unresolved cases and assumptions.

Tier 1 is enough to add an `⚠️ UNVERIFIED` claim with citations. It is not enough to remove a marker for a foundational wire-format claim.

### Tier 2 — Executable verification

Convert the Tier 1 claim into runnable evidence under `tools/` and, where byte-level data is involved, deterministic vectors under `test-vectors/`.

Prefer upstream-runtime verification: a Python script should exercise installed upstream RNS/LXMF and compare exact bytes, parsed fields, signatures, keys, or state transitions. For high-risk layouts, add an independent reconstruction or decoder as a cross-check. The LXMF extraction's Perl MessagePack verifier is a good example of this second form.

Tier 2 is enough to remove an `⚠️ UNVERIFIED` marker when the verifier is committed beside the claim and passes against the pinned upstream versions.

### Tier 3 — Specification promotion and maintenance

Only after Tier 2, update `SPEC.md` as normative prose:

- Add or correct the section text.
- Link the verifier or vector that backs the claim.
- Preserve version-specific behavior if upstream changed.
- Add a `README.md` spec-correction entry when earlier published text would mislead implementers.
- Add a `playbook.md` incident entry when the finding explains a non-obvious interoperability failure.

For foundational wire behavior, prefer two evidence forms before calling the result stable: upstream-runtime verification plus either an independent decoder/reconstruction or a live interop test. Flows and troubleshooting prose may be source-cited without this second form, but must not claim byte-level finality they do not have.

---

## 4. Required tools

Agents working on this repo should have access to:

- A working Python 3 install and an isolated virtual environment containing
  the versions in `tools/requirements.txt`. The environment path is
  contributor-local; `.venv` is the documented default, but no particular
  path or activation command is required. Run the suite with that
  environment's Python executable, for example
  `.venv/bin/python tools/verify_all.py`.
- The `RNS/` and `LXMF/` source trees (typically at `~/AppData/Roaming/Python/Python3xx/site-packages/RNS/` on Windows or `~/.local/lib/python3.x/site-packages/RNS/` on Linux/macOS).
- Optional but very useful: a packet-trace tool. `tcpdump -i lo -A -X port 4242` works for TCPServerInterface; for BLE you need ADB + an RNode-aware capture tool.

Hardware (RNode, RatDeck, etc.) is NOT required for most verification — most byte-level claims can be checked entirely in Python RNS without any radio.

### Staying current with upstream — and verifying what you install

This spec is only as good as the upstream version it was checked against. **Before any verification pass, confirm you are looking at the latest RNS:**

1. **Check the latest upstream version.** Upstream is mid-migration *off* GitHub: the RNS 1.2.4 release notes announced 1.2.4 would "probably be the last release also published to GitHub," and GitHub releases do stop around **1.2.5** (1.2.6/1.2.7 are PyPI-only). Reticulum is moving to self-hosting — `rngit` (git served over Reticulum) and `rnpkg` (package distribution over Reticulum). Per Mark Qvist, "updates to pip will continue at least until `rnpkg` is complete." Check, in order:
   - PyPI: `pip index versions rns` / `pip index versions lxmf` — most current while it lasts, but PyPI carries **no `.rsg`**, so PyPI alone is unverifiable.
   - GitHub releases: <https://github.com/markqvist/Reticulum/releases> — signed (`.rsg`) but frozen near 1.2.5.
   - `rngit` / `rnpkg` over Reticulum — the eventual canonical source of signed artifacts.

   PyPI runs *ahead* of the newest signed release (1.2.7 on PyPI vs 1.2.5 signed, 2026-05-17). Re-check every session — do not assume a frozen version.

2. **Verify the package signature before installing.** Since RNS 1.2.x, signed releases ship a detached `.rsg` signature beside each artifact (on the GitHub release page while that lasts; via `rngit`/`rnpkg` afterward). Get the wheel *and* its `.rsg`, then:

   ```
   rnid -i bc7291552be7a58f361522990465165c -V rns-<version>-py3-none-any.whl
   ```

   `bc7291552be7a58f361522990465165c` is Mark Qvist's release-signing Reticulum identity — confirm that hash itself against a trusted channel (a release announcement) before trusting it. `rnid` works fully offline. Install the **exact wheel you verified** (`pip install --upgrade ./rns-<version>-py3-none-any.whl`), not a fresh `pip install rns` that re-downloads from PyPI.

3. **Prefer the latest *signed* version.** If PyPI is ahead of the newest release carrying a `.rsg`, install the signed one and note the gap — do not silently install an unverifiable newer version.

4. **After a version bump**, re-run every `tools/verify_*.py`, re-check the source-cited line numbers in any section you touch, and only advance SPEC.md's `**Last verified against:**` line once the whole document has been re-checked against the new version.

---

## 5. Marking convention

Use these GitHub-flavored Markdown blockquote forms so they render distinctly:

```markdown
> ⚠️ **UNVERIFIED:** This is plausible from reading `RNS/Transport.py:1485` but I have not run a test that demonstrates the behavior end-to-end. Specifically need to confirm that a HEADER_1 packet from a TCP client to a sibling TCP client is forwarded after the rnsd auto-fills `transport_id`.

> 🔮 **SPECULATION:** The path-request payload may include the requester's own transport_id when issued from a transport-enabled originator. The `RNS/Transport.py::request_path` source suggests this but I have not decoded a captured upstream emission to confirm.
```

When you verify a previously-marked claim, **delete the entire blockquote** and (if helpful) append `(verified by tools/<script>.py)` to the claim's prose. Do not just remove the emoji — the structure should disappear so a future reader doesn't have to wonder.

---

## 6. Audit pass — initial state of `SPEC.md`

The bootstrap `SPEC.md` was assembled from the working notes of two reverse-engineering efforts ([webclient](https://github.com/thatSFguy/reticulum-lora-webclient) and [mobile-app](https://github.com/thatSFguy/reticulum-mobile-app)). Some sections are already strongly verified by working code in those repos; some are source-cited but not directly tested in this spec repo's tools; some are observational claims that need formal verification.

Before any v1.0 release, every section needs an explicit verification status. The first task for the next agent is to walk `SPEC.md` section by section and mark each claim per §1 above.

Initial confidence assessment (subjective, not authoritative — re-do this audit independently):

| SPEC.md section | Confidence | Notes |
|---|---|---|
| §1 Identity & destination hashes | High | Round-tripped against upstream in both reverse-engineering repos |
| §2.1, §2.2 Packet header bit layout | High | Matches upstream `RNS/Packet.py`; fixed test vectors round-trip |
| §2.3 Originator HEADER_1 → HEADER_2 conversion | **Source-cited but not test-verified in this repo.** Mark as such until a tools/ script confirms by exercising upstream `Transport.outbound` with a known multi-hop path table. |
| §3 Token crypto | High | Test vectors pass; both reference repos interop with upstream Sideband |
| §4 Announce wire format | High | Test vectors round-trip; signature verification works against upstream emissions |
| §4.3 `app_data` format | Mostly high. The `[name_bytes, stamp_cost, [capability_flags]]` 3-element variant is observed but not formally verified for this repo. |
| §5 LXMF wire format | High |
| §5.6 Dual msgpack-variant signature verification | High — fixed an interop bug in the webclient when added |
| §6 Reticulum Link protocol | High | Both initiator and responder are working in the reference repos |
| §7.1, §7.2 Path requests | **Recently surfaced bug-fix.** §7.2 (responding to inbound path requests) is verified end-to-end on BLE in the mobile-app. §7.1's claim that path requests *always* precede LXMF DATA needs verification — may only happen on stale paths. |
| §7.3 Ratchet rotation | **Spec corrected.** Earlier audit treated this as "verified end-to-end" — but the test result that prompted the verification was attributed to the wrong mechanism (ratchet rotation), when the actual win was the incidental `random_hash` rotation that came along for the ride. `tools/verify_ratchet_dedup.py` (RNS 1.2.0) confirms upstream replay defence is keyed on `random_blob`, not `(dest_hash, ratchet_pub)`. §7.3 reframed as forward-secrecy guidance; §4.5 step 6.3 documents the actual dedup mechanism. |
| §7.4 Ratchet ring (inbound decrypt tolerance) | **UNVERIFIED in current implementations.** The reference repos discard old ratchet privkeys on rotation. Upstream's "8 ratchets" default needs source citation. |
| §7.6 `TCPServerInterface.OUT` override | Source-cited; matches behavior observed in the mobile-app's local-transport experiments. |
| §8 KISS / HDLC framing | High — both work in production on the reference clients |
| §9.1–§9.8 Implementation gotchas | Each was a real bug that bit a real implementation. High confidence each is real; some lack formal test scripts. |
| §10 Resource fragmentation | Source-audited against RNS 1.2.4 and runtime-verified by `tools/verify_resource.py`, including deterministic vectors, receiver assembly/proof, multi-segment sizing, and negative cases. |
| §11 Test vectors | Historical bootstrap item. `test-vectors/` now covers identities, announces, opportunistic LXMF, Link establishment, link-delivered LXMF, and Resource. Future work should add broader negative vectors. |
| §12 Source map | High |

**Historical bootstrap tasks from the initial audit, now mostly complete:**

1. Re-evaluate the table above with your own reasoning, not by copying it.
2. Keep completed vectors current: identities, signed announces, opportunistic LXMF, and Link handshake.
3. Maintain the verifier suite listed in `tools/README.md`; add focused scripts for new claims instead of broad catch-all verifiers.
4. For each new `> ⚠️` or `> 🔮` callout, track the Tier 2 verifier or issue needed to resolve it.
5. Open issues for claims that cannot be verified yet and tag them `needs-verification`.

---

## 7. PR rules

For any PR touching `SPEC.md`:

- Every new claim has a verification marker OR is unmarked because it has a `(verified by tools/...)` reference.
- Every removed marker has a corresponding new tools/ script in the same PR.
- No "I tested this manually and it worked" — capture the test as a runnable script.
- Reviewers should reject PRs that quietly remove markers without including the verifier.

For PRs adding to `test-vectors/`:

- Include the script that generated the vector (so it can be regenerated against future upstream RNS).
- Include the upstream RNS version the vector was generated against (`pip show rns` output is fine).
- The test vector should round-trip in both directions (build+sign AND parse+verify) when run through upstream RNS.

---

## 8. What to do when upstream changes

RNS evolves. When a future RNS version changes the wire format (or the in-source behavior cited in this spec):

1. **Don't silently update the spec.** The old behavior may still be in production deployments and matter for interop.
2. Add a versioned note: `As of RNS x.y.z this changed to ...`. Keep the prior behavior described too.
3. Update `tools/` verifiers to test against both the old and new behavior if possible.
4. Bump a `last-verified-against-rns: x.y.z` line in `SPEC.md`'s frontmatter.

The goal is for this spec to be useful even when run against an RNS version a year out of date — that's the worst-case in heterogeneous mesh deployments.

---

## 9. Don't

- Don't paste large blocks of upstream code into this repo (license & churn). Cite by file + line + small inline snippet only.
- Don't add claims based purely on what some other client does. Other clients have bugs too.
- Don't remove a `⚠️` or `🔮` marker without doing the verification work.
- Don't commit a verifier that swallows discrepancies (`if expected != actual: print("close enough")`). Either it matches or it doesn't.
- Don't trust your own training data on this — Reticulum-specific protocol details are sparse on the public web and most LLM knowledge is wrong or out of date. Verify everything.

---

## 10. Repo layout

```
reticulum-specifications/
├── README.md              Project intro
├── LICENSE                CC BY 4.0
├── agent.md               This file
├── SPEC.md                Combined spec (will be split into per-layer files as it grows)
├── tools/                 Verifier scripts (Python, callable against upstream RNS)
│   ├── verify_announce_roundtrip.py
│   ├── verify_packet_header.py
│   ├── verify_lxmf_opportunistic.py
│   ├── verify_link_handshake.py
│   └── ...
└── test-vectors/          Known-good byte sequences
    ├── README.md
    ├── identities.json
    ├── announces.json
    ├── lxmf.json
    └── links.json
```

Tools are Python because Python RNS is the reference. Verifiers should be self-contained scripts that print PASS/FAIL plus a diagnostic on mismatch. Exit code 0 on PASS, non-zero on FAIL.