microReticulumTbeam/exercises/306_microReticulum_ble_file_transfer_oled

Exercise 306: microReticulum BLE file transfer with OLED display

This exercise builds on Exercise 305's equal-peer BLE file transfer and adds the shared lib/tbeam_display OLED service. Both boards run the same dual-role BLE interface, form a Reticulum Link, and then send the selected text file across that Link at the same time.

The OLED display is intentionally plain: incoming FTD file data is rendered as scrolling text only. File begin/end metadata, checksums, link state, and debug records remain on the serial monitor but are not shown on the OLED.

The file transfer protocol is intentionally small and visible on the serial console:

FTB -> file begin, with file name, byte count, chunk count, and checksum
FTD -> numbered file data chunk
FTE -> file end, with verification metadata repeated

Each receiver checks byte count, chunk count, and FNV-1a checksum. After a sender completes a transfer, it rests for 10 seconds and then starts the same selected file again.

Sample Set

Exercise 306 uses the same payload files as the Pi Zero BLE Reticulum tests:

texts/If.txt                 195 bytes
texts/If_full.txt           1583 bytes
texts/US_Constitution.txt  44225 bytes
texts/little_boy_blue.txt    942 bytes

The selected file is compiled into the firmware. The transfer code does not care which file is selected; platformio.ini chooses the source text through custom_text_source, and scripts/embed_text.py generates SelectedText.h in the build directory before compilation.

Transfer Profiles

The transfer pressure is selected in platformio.ini with build flags:

-D FILE_TRANSFER_CHUNK_SIZE=32
-D FILE_TRANSFER_CHUNK_INTERVAL_MS=500

FILE_TRANSFER_CHUNK_SIZE is the number of text bytes placed in each application-level FTD message before microReticulum wraps and encrypts it as a Link packet. Larger chunks reduce the number of packets needed for a file, but each encrypted packet becomes larger. If it grows beyond what the ESP32 BLE transport can reliably carry under simultaneous two-way traffic, Reticulum may log Link decrypt/HMAC failures because ciphertext arrived damaged or incomplete.

FILE_TRANSFER_CHUNK_INTERVAL_MS is the delay between application-level file chunks. Smaller intervals increase throughput, but also increase BLE write/notify pressure. With both nodes transmitting at the same time, too short a cadence can overflow buffers or expose ordering/loss issues in the current ESP32 BLE transport.

The conservative bring-up profile uses:

32 byte chunks, 500 ms between chunks

The Pi-Zero-comparison profile uses:

300 byte chunks, 100 ms between chunks

That is the apples-to-apples starting point for the previous Zero-to-Zero tests. Those commands requested --message-chunk-size 900, but the Python sender intentionally applied an internal board/Link-budget cap before sending. The run17 report for the Constitution transfer shows effective chunk data around 300 to 316 bytes, not 900 bytes, with roughly 100 ms sender pacing.

VERIFY_FAIL means the file protocol received an incomplete or corrupted transfer. A Reticulum Link HMAC/decryption error means corruption happened earlier, before the file protocol could parse the packet.

Priority

See Exercise 304_microReticulum_ble_dual_role_ping_pong README.md for explanation of "deterministic tie-breaker" of the role of client and server based on the ESP32 MAC.

Environments

Conservative ESP32 bring-up environments:

tbeam_if
tbeam_if_full
tbeam_constitution

Pi-Zero-comparison environments:

tbeam_if_pi_zero_profile
tbeam_if_full_pi_zero_profile
tbeam_constitution_pi_zero_profile

Host-native environment:

jp_native
jp_native_peripheral
jp_native_dual
pi_zero_1_native
pi_zero_1_peripheral
pi_zero_1_dual
pi_zero_2_native
pi_zero_2_peripheral
pi_zero_2_dual

jp_native builds a Linux console program instead of ESP32 firmware. It uses the host Bluetooth adapter through BlueZ D-Bus, skips the OLED path, and prints received text to stdout. The current jp payload is texts/little_boy_blue.txt.

jp_native_peripheral builds a Linux BLE peripheral/server. It registers the Exercise 306 GATT service through BlueZ, advertises the Reticulum service UUID, accepts T-Beam central connections, receives writes on RX, and notifies outgoing fragments on TX.

jp_native_dual registers both host interfaces in one process: the Linux central/client path and the Linux peripheral/server path.

pi_zero_1_dual and pi_zero_2_dual do the same for Raspberry Pi native builds, but use Pi-specific node labels and selected text files.

Building

ESP32

Build the ESP32/T-Beam firmware with one of the tbeam_* environments. Each selected text environment produces one firmware image. Build it once, then upload that same image to both boards.

Build the short If sample:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_if

Build the Pi-Zero-profile Constitution sample:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_constitution_pi_zero_profile

After the build succeeds, upload the same environment to both boards. These commands may be run one after the other:

pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_if -t upload --upload-port /dev/ttytDAN
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_if -t upload --upload-port /dev/ttytBOB

Use the same -e value in upload commands that you used for the build.

For strict parallel uploads, use esptool.py directly against the already-built artifacts. This avoids two concurrent pio run processes touching the same .pio build directory:

cd /usr/local/src/microreticulum/microReticulumTbeam/exercises/306_microReticulum_ble_file_transfer_oled
esptool.py --chip esp32s3 --port /dev/ttytDAN --baud 460800 write_flash -z \
  0x0000 .pio/build/tbeam_if/bootloader.bin \
  0x8000 .pio/build/tbeam_if/partitions.bin \
  0xe000 /home/jlpoole/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/tbeam_if/firmware.bin &
esptool.py --chip esp32s3 --port /dev/ttytBOB --baud 460800 write_flash -z \
  0x0000 .pio/build/tbeam_if/bootloader.bin \
  0x8000 .pio/build/tbeam_if/partitions.bin \
  0xe000 /home/jlpoole/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/tbeam_if/firmware.bin &
wait

For another environment, replace each .pio/build/tbeam_if/ path with that environment's build directory.

Monitor:

pio device monitor -p /dev/ttytDAN -b 115200
pio device monitor -p /dev/ttytBOB -b 115200

Intel x86_64

Build the jp Linux host binary with the jp_native environment:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native

The resulting executable is:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program

Run it from the repository root:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program

The host binary is a BlueZ BLE central. It expects the T-Beam to advertise the Exercise 306 service, then receives file-transfer text on the console. Because it uses system D-Bus and the Bluetooth adapter, sandboxed runs may require approval.

Build the jp peripheral capability-check binary with:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native_peripheral

The resulting executable is:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_peripheral/program

Run it from the repository root:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_peripheral/program

This build is expected to register the GATT server, advertise the Exercise 306 service, and exchange Reticulum file-transfer traffic when a T-Beam connects as the BLE central.

Build the jp dual-role test binary with:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native_dual

The resulting executable is:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_dual/program

Run it from the repository root:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_dual/program

For a central-mode host test, start jp_native or jp_native_dual first and wait for BLE linux-central: scanning for Reticulum service, then RESET the T-Beam.

For a peripheral-mode host test, start jp_native_peripheral first and wait for BLE linux-peripheral: advertising Reticulum service; waiting for central, then RESET the T-Beam. In this order, the T-Beam can connect as the BLE central and jp receives an inbound Reticulum link.

AMD64

AMD64 is the same 64-bit x86 Linux target class as Intel x86_64 for this PlatformIO native build. On eos, build the same jp_native environment on that machine:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native

The output path is the same relative path:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program

Do not copy the jp-built executable to ARM machines. Rebuild on the target architecture unless a cross-compile environment is added.

ARM64

For a 64-bit Raspberry Pi OS on a Pi Zero 2W or Pi 4B, use the same jp_native environment and build directly on the Pi:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native

PlatformIO native produces a binary for the machine doing the build, so an ARM64 Pi build produces an ARM64 executable at:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program

The Pi must have BlueZ, GLib/GIO development headers, and a BLE adapter that supports LE central mode. If the Pi is running a 32-bit OS, the result is a 32-bit ARM binary, not ARM64.

Two Pi Zero 2Ws

The preferred Pi-to-Pi experiment is now the dual-role pair. Each Pi starts both BlueZ roles in one process: central/client scanning and peripheral/server advertising. The two builds differ only by node label and compiled-in text payload:

pi_zero_1_dual  Node-PIZERO1-DUAL  little_boy_blue.txt
pi_zero_2_dual  Node-PIZERO2-DUAL  children.txt

Build on a Pi with the same OS architecture and compatible libraries as the target Pi Zero 2Ws:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e pi_zero_1_dual -e pi_zero_2_dual

The output binaries are:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_dual/program
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_2_dual/program

For web/download publication, copy or rename those artifacts with names that include the platform, role mode, and poem:

microreticulum_306_rpi_arm64_dual_little_boy_blue
microreticulum_306_rpi_arm64_dual_children

On the Pi build machine, make the download copies and strip them:

cp exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_dual/program microreticulum_306_rpi_arm64_dual_little_boy_blue
cp exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_2_dual/program microreticulum_306_rpi_arm64_dual_children
strip microreticulum_306_rpi_arm64_dual_little_boy_blue microreticulum_306_rpi_arm64_dual_children

Run one on each Pi. There should be no required start order for the dual-role test, although starting one Pi a few seconds before the other makes the log easier to read.

./microreticulum_306_rpi_arm64_dual_little_boy_blue
./microreticulum_306_rpi_arm64_dual_children

Dual builds default to --ble-dual-policy=first-path-wins. Both BlueZ roles start, but once either central or peripheral establishes the first peer path, the opposite role is stopped. This avoids exposing two simultaneous Reticulum interfaces to the same peer.

Temporary policy overrides are available for debugging:

./microreticulum_306_rpi_arm64_dual_little_boy_blue --ble-dual-policy=both
./microreticulum_306_rpi_arm64_dual_little_boy_blue --ble-dual-policy=first-path-wins
./microreticulum_306_rpi_arm64_dual_little_boy_blue --ble-dual-policy=central-only
./microreticulum_306_rpi_arm64_dual_little_boy_blue --ble-dual-policy=peripheral-only

both preserves the earlier Linux dual behavior and may produce Reticulum Link-associated packet received on unexpected interface errors if two BLE paths form at the same time. The central-only and peripheral-only policies are useful when isolating BlueZ adapter behavior without rebuilding split-role binaries.

You can copy those binaries to another Pi Zero 2W and rename them for clarity if both Pis use the same CPU architecture, OS bitness, and compatible runtime libraries. Check with:

file exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_dual/program
ldd exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_dual/program

If one Pi is 32-bit and the other is 64-bit, or their GLib/BlueZ runtime libraries differ significantly, build directly on each target instead of copying binaries.

The older split-role environments are still available for controlled tests where you want to force one Pi to advertise and the other Pi to scan:

For two Pi Zero 2Ws, build one peripheral/server binary and one central/client binary. These environments use distinct node labels and text payloads:

pi_zero_1_peripheral  Node-PIZERO1-PERIPHERAL  little_boy_blue.txt
pi_zero_2_native      Node-PIZERO2-CLIENT      children.txt

Build on a Pi with the same OS architecture and compatible libraries as the target Pi Zero 2Ws:

source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e pi_zero_1_peripheral -e pi_zero_2_native

The output binaries are:

exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_peripheral/program
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_2_native/program

You can copy those binaries to another Pi Zero 2W and rename them for clarity if both Pis use the same CPU architecture, OS bitness, and compatible runtime libraries. Check with:

file exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_peripheral/program
ldd exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_peripheral/program

If one Pi is 32-bit and the other is 64-bit, or their GLib/BlueZ runtime libraries differ significantly, build directly on each target instead of copying binaries.

Run peripheral first on the first Pi:

./pi_zero_1_peripheral

Wait for:

BLE linux-peripheral: advertising Reticulum service; waiting for central

Then run central on the second Pi:

./pi_zero_2_native

The reverse role binaries are also available if you want to swap which Pi advertises:

pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e pi_zero_2_peripheral -e pi_zero_1_native

Expected Output

Once the Link is active, both nodes start sending:

Selected file=If.txt bytes=195 chunk=32 interval_ms=500 repeat_rest_ms=10000
TX FILE BEGIN: round=1 file=If.txt bytes=195 chunks=7 crc=...
TX FILE DATA: round=1 seq=1/7 bytes=32 preview="If you can keep your head..."
TX FILE END: round=1 file=If.txt bytes=195 chunks=7 crc=... next_round_in_ms=10000

The receiver verifies the transfer:

RX FILE BEGIN: from=Node-... file=If.txt bytes=195 chunks=7 crc=...
RX FILE DATA: from=Node-... seq=1/7 bytes=32 preview="If you can keep your head..."
RX FILE END: from=Node-... file=If.txt received=195/195 chunks=7/7 crc=... status=OK

Ten seconds after TX FILE END, the same selected file starts again. This rest interval is measured after transfer completion, so large files get the same 10-second pause before the next round.

Debug Lines

Exercise 305 includes machine-parseable debug records for the role-dependent BLE/Link failure investigation. Each record is one line of key=value fields.

RNSLINK   Link/announce events, peer hashes, Link ids, and Link object ids.
RNSTX     Application plaintext sends and encrypted Reticulum packets handed to BLE.
RNSRX     Reassembled BLE packets immediately before Reticulum receives them.
RNSDEC    Link encrypt/decrypt attempts and failures from microReticulum Link.cpp.
RNSBLE    BLE connect, identity, fragment TX/RX, and packet assembly events.
RNSQUEUE  BLE RX queue depth, pushes, pops, drops, and high-water marks.
RNSMEM    Heap, largest block, PSRAM, and current task stack high-water mark.
RNSERR    Classified adapter errors: reassembly gaps, short fragments, queue overflow, allocation failure.

Normal packet logging is rate-limited: first 25 packets, then every 25th packet, then all packets for two seconds after the first failure. Define RNS_DEBUG_VERBOSE=1 in platformio.ini to print every packet fingerprint.

Debug hooks are inserted at these points:

src/TBeamSupremeBleInterface.cpp  BLE callback boundary, fragment TX/RX, reassembly, queue, packet handoff.
src/main.cpp                     board-name mapping, announce/link events, application Link send.
/usr/local/src/microreticulum/microReticulum/src/Link.cpp
                                 Link encrypt/decrypt token fingerprints and classified decrypt failures.

Use the CRC fields to split the failure:

Same RNSTX token crc and RNSRX/RNSDEC token crc, but HMAC_INVALID -> likely wrong key/session/Link context.
Different RNSTX token crc and RNSRX/RNSDEC token crc -> BLE fragmentation, reassembly, queue, or buffer corruption.
RNSERR rx_reassembly_gap/timeout or queue_overflow -> mechanical adapter failure before Reticulum decrypt.
RNSMEM largest_block or min_heap collapse before failures -> heap pressure or fragmentation.