Reticulum: A Precision Trace Among 7 Nodes

What actually happens inside a Reticulum mesh running over LoRa when two endpoints cannot communicate directly?

I built a seven-node testbed using LilyGo T-Beam SUPREME units named AMY, BOB, CY, DAN, ED, FLO, and GUY. AMY and GUY exchange one encrypted “Hi” message per minute, while software-defined radio blocks force their traffic through the other five nodes as transports.

All seven units discipline their clocks from GNSS satellites. In a roughly 17.5-minute synchronization run, the span among their median end-to-end observed clock offsets was about 0.166 ms, or 166 microseconds, and the minute-by-minute spread remained below about 0.633 ms. That common time base makes it possible to correlate events across seven independent logs with useful precision.

Background

I have seven LilyGo T-Beam SUPREME units named AMY, BOB, CY, DAN, ED, FLO & GUY.  The T-Beam is a small electrical assembly of components including a very powerful ESP32-S microprocessor which features wifi, bluetooth. There are also modules GPS¹ and LoRa radio.  Reticulum’s reference implementation is written in Python. For these constrained ESP32-based devices, I chose Chad Attermann’s C++ microReticulum implementation. ²

The T-Beams running microReticulum can provide a mesh network which essentially means some units can act as forwarders or middlemen between two radio units that are out of range from each other. I like to analogize a mesh network as akin to a bucket brigade.

What I have Accomplished as of July 4, 2026

I have modified microreticulum by riddling it with debug statements so I can understand what is going on at every step.  Since my 7 units are all in proximity to one another, I have used blocks written into the software to simulate a field test where all units are dispersed and no more than two units are within range of each other.

Here is a diagram of the 7 units showing the possible paths of radio communication when they are all in the same room.  What I have done is create artificial “blocks” in the software among various pairs so that Reticulum packets between AMY and GUY are forced to go through the other five units which act as transports.  Since each unit has a custom block list, each unit has to be individually compiled.  During my development I found that I could compile a 7 different firmwares on my high end server and save turn-around time by doing in 2 minutes what would take 10 minutes.

A major hurdle I had to overcome was disciplining the clocks on each unit so everyone had the same time.  That way when they produce logs, all the logs are using the same time and small differences such as one thousandth of a second, or more, can be accepted as real and not an aberration of different clocks.  Learning to synchronize time to satellites took several weeks, and yesterday, to my surprise, I found Internet celebrity Engineer Jeff Geerling wrestling with the same issue with his pal Sean Hodgins in their collaboration and whimsical: The Most Accurate Watch That Doesn’t Tell Time   In a roughly 17.5-minute run, the span among the seven units’ median end-to-end observed clock offsets was about 0.166 ms (166 µs — yes, that is 166 millionths of a second), with the minute-by-minute spread remaining below about 0.633 ms.  The clock-synchronization analysis covered roughly 17.5 minutes. The detailed chronology and CAD plots below are from a separate 10-minute message run.

My test run consists of the following:

• Only AMY and GUY talk to each, i.e. send “Hi” messages back and forth
• BOB, CY, DAN, ED & FLO are active on the network, but they are not sending “Hi” messages, though they will forward packets on when requested.
• AMY and GUY are programmed to send a “Hi” message once a minute
• All units have the blocking rules illustrated above, so the only path for a message between AMY & GUY is via the five other units.
• All units must discipline/synchronize their internal clocks to 4 or more satellites, so the experiment must be conducted where they can readily see satellites.
• A successful run of 6 minutes should show in their logs 5 or more receptions of “Hi” messages from the other.

Upshot of all of this is a high precision chronology table that serves as a time line.  During the 10-minute run, AMY and GUY each logged eight successfully decrypted “Hi” messages from the other endpoint. BOB, CY, DAN, ED, and FLO participated as transport nodes, forwarding traffic along the forced path.

jlpoole@jp ~/logs/20260704_1041 $ grep -n "Hi" *.log |nl
     1	AMY_raw_20260704_104100.log:323:20260704_104307.670 RNSLINKRX: ms=126733 board=library role=unknown event=decrypt_ok link_id=3ac84ff362419439d51b3606d57b431d plaintext_len=25 plaintext_crc32=658D6C47 text_encoding=percent text=GUY%20says%20Hi%20to%20AMY%20iter%3D0 link_obj={Link:3ac84ff362419439d51b3606d57b431d}
     ...
     9	GUY_raw_20260704_104106.log:309:20260704_104309.001 RNSLINKRX: ms=122100 board=library role=unknown event=decrypt_ok link_id=3ac84ff362419439d51b3606d57b431d plaintext_len=25 plaintext_crc32=B936D34C text_encoding=percent text=AMY%20says%20Hi%20to%20GUY%20iter%3D0 link_obj={Link:3ac84ff362419439d51b3606d57b431d}
     ...
    16	GUY_raw_20260704_104106.log:780:20260704_105009.033 RNSLINKRX: ms=542119 board=library role=unknown event=decrypt_ok link_id=1ecfef05be2b3d986e649549090bd38e plaintext_len=25 plaintext_crc32=275246EF text_encoding=percent text=AMY%20says%20Hi%20to%20GUY%20iter%3D7 link_obj={Link:1ecfef05be2b3d986e649549090bd38e}
jlpoole@jp ~/logs/20260704_1041 $ 

The raw logs can be difficult to read, so I developed a database and parse all values of all seven logs.  From the database, I can then generate an HTML table:

The full table, with its 800+ entries for the 10 minute run is available here: microReticulum LINK Chronological Event Table

One of the issues I had not given much thought about is the sharing of the radio wave.  All units use the same LoRa RF channel, so they must share airtime much like callers sharing an old telephone party line.  Before transmitting, a node performs Channel Activity Detection (aka “CAD”). If the channel is busy, it waits and retries rather than transmitting immediately. With all the activity of forwarding packets to and from, there were contentions.  Using a radio wave waterfall tool, OpenWebrx, I saw mild usage, but in several experiments (a run-away forwarding chain with 255 hops) I encountered saturation which I fixed later by placing guards within the software

I therefore plotted out the times the units were having to delay their transmissions because the airwave was currently being used.

Below are screenshots of the Plot.

In the plot, you can zoom in and out and pan about.  Here is a zoomed-in portion for the first cluster.

The interactive plot is available at: Channel Activity Detection Delay

All log files and the SQLite database are available here. Access to my Forgejo source code is available on request, I had my Forgejo instance publicly accessible, but soon bots starts hitting my sight bout 10 times a second, so I have restricted Forgejo access to authenticated accounts.

I could not have possibly accomplished this alone and I relied heavily upon ChatGPT and Codex using my $20/month subscription.  I’ve used ChatGPT as an architect collaborator and Codex as the software engineer.  I chose not to pursue the management path at my former employer (24 years), so never had to “manage.”  My wife quips now I am playing that role between the two AIs.

1. The LK76 or u-Blox versions are available.
2. There is a Rust based version of Reticulum being developed by Lew Palm.