microReticulumTbeam/exercises/306_microReticulum_ble_file_transfer_oled/README.md

373 lines
15 KiB
Markdown
Raw Normal View History

# Exercise 306: microReticulum BLE file transfer with OLED display
This exercise builds on Exercise 305's equal-peer BLE file transfer and adds the shared `lib/tbeam_display` OLED service. Both boards run the same dual-role BLE interface, form a Reticulum Link, and then send the selected text file across that Link at the same time.
The OLED display is intentionally plain: incoming `FTD` file data is rendered as scrolling text only. File begin/end metadata, checksums, link state, and debug records remain on the serial monitor but are not shown on the OLED.
The file transfer protocol is intentionally small and visible on the serial console:
```text
FTB -> file begin, with file name, byte count, chunk count, and checksum
FTD -> numbered file data chunk
FTE -> file end, with verification metadata repeated
```
Each receiver checks byte count, chunk count, and FNV-1a checksum. After a sender completes a transfer, it rests for 10 seconds and then starts the same selected file again.
## Sample Set
Exercise 306 uses the same payload files as the Pi Zero BLE Reticulum tests:
```text
texts/If.txt 195 bytes
texts/If_full.txt 1583 bytes
texts/US_Constitution.txt 44225 bytes
texts/little_boy_blue.txt 942 bytes
```
The selected file is compiled into the firmware. The transfer code does not care which file is selected; `platformio.ini` chooses the source text through `custom_text_source`, and `scripts/embed_text.py` generates `SelectedText.h` in the build directory before compilation.
## Transfer Profiles
The transfer pressure is selected in `platformio.ini` with build flags:
```ini
-D FILE_TRANSFER_CHUNK_SIZE=32
-D FILE_TRANSFER_CHUNK_INTERVAL_MS=500
```
`FILE_TRANSFER_CHUNK_SIZE` is the number of text bytes placed in each application-level `FTD` message before microReticulum wraps and encrypts it as a Link packet. Larger chunks reduce the number of packets needed for a file, but each encrypted packet becomes larger. If it grows beyond what the ESP32 BLE transport can reliably carry under simultaneous two-way traffic, Reticulum may log Link decrypt/HMAC failures because ciphertext arrived damaged or incomplete.
`FILE_TRANSFER_CHUNK_INTERVAL_MS` is the delay between application-level file chunks. Smaller intervals increase throughput, but also increase BLE write/notify pressure. With both nodes transmitting at the same time, too short a cadence can overflow buffers or expose ordering/loss issues in the current ESP32 BLE transport.
The conservative bring-up profile uses:
```text
32 byte chunks, 500 ms between chunks
```
The Pi-Zero-comparison profile uses:
```text
300 byte chunks, 100 ms between chunks
```
That is the apples-to-apples starting point for the previous Zero-to-Zero tests. Those commands requested `--message-chunk-size 900`, but the Python sender intentionally applied an internal board/Link-budget cap before sending. The run17 report for the Constitution transfer shows effective chunk data around 300 to 316 bytes, not 900 bytes, with roughly 100 ms sender pacing.
`VERIFY_FAIL` means the file protocol received an incomplete or corrupted transfer. A Reticulum Link HMAC/decryption error means corruption happened earlier, before the file protocol could parse the packet.
## Priority
See Exercise 304_microReticulum_ble_dual_role_ping_pong README.md for explanation of "deterministic tie-breaker" of the role of client and server based on the ESP32 MAC.
## Environments
Conservative ESP32 bring-up environments:
```text
tbeam_if
tbeam_if_full
tbeam_constitution
```
Pi-Zero-comparison environments:
```text
tbeam_if_pi_zero_profile
tbeam_if_full_pi_zero_profile
tbeam_constitution_pi_zero_profile
```
Host-native environment:
```text
jp_native
jp_native_peripheral
jp_native_dual
pi_zero_1_native
pi_zero_1_peripheral
pi_zero_2_native
pi_zero_2_peripheral
```
`jp_native` builds a Linux console program instead of ESP32 firmware. It uses the host Bluetooth adapter through BlueZ D-Bus, skips the OLED path, and prints received text to stdout. The current jp payload is `texts/little_boy_blue.txt`.
`jp_native_peripheral` builds a Linux BLE peripheral/server. It registers the Exercise 306 GATT service through BlueZ, advertises the Reticulum service UUID, accepts T-Beam central connections, receives writes on RX, and notifies outgoing fragments on TX.
`jp_native_dual` registers both host interfaces in one process: the Linux central/client path and the Linux peripheral/server path.
## Building
### ESP32
Build the ESP32/T-Beam firmware with one of the `tbeam_*` environments. Each selected text environment produces one firmware image. Build it once, then upload that same image to both boards.
Build the short If sample:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_if
```
Build the Pi-Zero-profile Constitution sample:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_constitution_pi_zero_profile
```
After the build succeeds, upload the same environment to both boards. These commands may be run one after the other:
```bash
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_if -t upload --upload-port /dev/ttytDAN
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e tbeam_if -t upload --upload-port /dev/ttytBOB
```
Use the same `-e` value in upload commands that you used for the build.
For strict parallel uploads, use `esptool.py` directly against the already-built artifacts. This avoids two concurrent `pio run` processes touching the same `.pio` build directory:
```bash
cd /usr/local/src/microreticulum/microReticulumTbeam/exercises/306_microReticulum_ble_file_transfer_oled
esptool.py --chip esp32s3 --port /dev/ttytDAN --baud 460800 write_flash -z \
0x0000 .pio/build/tbeam_if/bootloader.bin \
0x8000 .pio/build/tbeam_if/partitions.bin \
0xe000 /home/jlpoole/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
0x10000 .pio/build/tbeam_if/firmware.bin &
esptool.py --chip esp32s3 --port /dev/ttytBOB --baud 460800 write_flash -z \
0x0000 .pio/build/tbeam_if/bootloader.bin \
0x8000 .pio/build/tbeam_if/partitions.bin \
0xe000 /home/jlpoole/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
0x10000 .pio/build/tbeam_if/firmware.bin &
wait
```
For another environment, replace each `.pio/build/tbeam_if/` path with that environment's build directory.
Monitor:
```bash
pio device monitor -p /dev/ttytDAN -b 115200
pio device monitor -p /dev/ttytBOB -b 115200
```
### Intel x86_64
Build the jp Linux host binary with the `jp_native` environment:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native
```
The resulting executable is:
```text
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program
```
Run it from the repository root:
```bash
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program
```
The host binary is a BlueZ BLE central. It expects the T-Beam to advertise the Exercise 306 service, then receives file-transfer text on the console. Because it uses system D-Bus and the Bluetooth adapter, sandboxed runs may require approval.
Build the jp peripheral capability-check binary with:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native_peripheral
```
The resulting executable is:
```text
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_peripheral/program
```
Run it from the repository root:
```bash
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_peripheral/program
```
This build is expected to register the GATT server, advertise the Exercise 306 service, and exchange Reticulum file-transfer traffic when a T-Beam connects as the BLE central.
Build the jp dual-role test binary with:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native_dual
```
The resulting executable is:
```text
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_dual/program
```
Run it from the repository root:
```bash
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native_dual/program
```
For a central-mode host test, start `jp_native` or `jp_native_dual` first and wait for `BLE linux-central: scanning for Reticulum service`, then RESET the T-Beam.
For a peripheral-mode host test, start `jp_native_peripheral` first and wait for `BLE linux-peripheral: advertising Reticulum service; waiting for central`, then RESET the T-Beam. In this order, the T-Beam can connect as the BLE central and jp receives an inbound Reticulum link.
### AMD64
AMD64 is the same 64-bit x86 Linux target class as Intel x86_64 for this PlatformIO `native` build. On eos, build the same `jp_native` environment on that machine:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native
```
The output path is the same relative path:
```text
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program
```
Do not copy the jp-built executable to ARM machines. Rebuild on the target architecture unless a cross-compile environment is added.
### ARM64
For a 64-bit Raspberry Pi OS on a Pi Zero 2W or Pi 4B, use the same `jp_native` environment and build directly on the Pi:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e jp_native
```
PlatformIO `native` produces a binary for the machine doing the build, so an ARM64 Pi build produces an ARM64 executable at:
```text
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/jp_native/program
```
The Pi must have BlueZ, GLib/GIO development headers, and a BLE adapter that supports LE central mode. If the Pi is running a 32-bit OS, the result is a 32-bit ARM binary, not ARM64.
### Two Pi Zero 2Ws
For two Pi Zero 2Ws, build one peripheral/server binary and one central/client binary. These environments use distinct node labels and text payloads:
```text
pi_zero_1_peripheral Node-PIZERO1-PERIPHERAL little_boy_blue.txt
pi_zero_2_native Node-PIZERO2-CLIENT children.txt
```
Build on a Pi with the same OS architecture and compatible libraries as the target Pi Zero 2Ws:
```bash
source /home/jlpoole/rnsenv/bin/activate
cd /usr/local/src/microreticulum/microReticulumTbeam
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e pi_zero_1_peripheral -e pi_zero_2_native
```
The output binaries are:
```text
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_peripheral/program
exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_2_native/program
```
You can copy those binaries to another Pi Zero 2W and rename them for clarity if both Pis use the same CPU architecture, OS bitness, and compatible runtime libraries. Check with:
```bash
file exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_peripheral/program
ldd exercises/306_microReticulum_ble_file_transfer_oled/.pio/build/pi_zero_1_peripheral/program
```
If one Pi is 32-bit and the other is 64-bit, or their GLib/BlueZ runtime libraries differ significantly, build directly on each target instead of copying binaries.
Run peripheral first on the first Pi:
```bash
./pi_zero_1_peripheral
```
Wait for:
```text
BLE linux-peripheral: advertising Reticulum service; waiting for central
```
Then run central on the second Pi:
```bash
./pi_zero_2_native
```
The reverse role binaries are also available if you want to swap which Pi advertises:
```bash
pio run -d exercises/306_microReticulum_ble_file_transfer_oled -e pi_zero_2_peripheral -e pi_zero_1_native
```
## Expected Output
Once the Link is active, both nodes start sending:
```text
Selected file=If.txt bytes=195 chunk=32 interval_ms=500 repeat_rest_ms=10000
TX FILE BEGIN: round=1 file=If.txt bytes=195 chunks=7 crc=...
TX FILE DATA: round=1 seq=1/7 bytes=32 preview="If you can keep your head..."
TX FILE END: round=1 file=If.txt bytes=195 chunks=7 crc=... next_round_in_ms=10000
```
The receiver verifies the transfer:
```text
RX FILE BEGIN: from=Node-... file=If.txt bytes=195 chunks=7 crc=...
RX FILE DATA: from=Node-... seq=1/7 bytes=32 preview="If you can keep your head..."
RX FILE END: from=Node-... file=If.txt received=195/195 chunks=7/7 crc=... status=OK
```
Ten seconds after `TX FILE END`, the same selected file starts again. This rest interval is measured after transfer completion, so large files get the same 10-second pause before the next round.
## Debug Lines
Exercise 305 includes machine-parseable debug records for the role-dependent BLE/Link failure investigation. Each record is one line of `key=value` fields.
```text
RNSLINK Link/announce events, peer hashes, Link ids, and Link object ids.
RNSTX Application plaintext sends and encrypted Reticulum packets handed to BLE.
RNSRX Reassembled BLE packets immediately before Reticulum receives them.
RNSDEC Link encrypt/decrypt attempts and failures from microReticulum Link.cpp.
RNSBLE BLE connect, identity, fragment TX/RX, and packet assembly events.
RNSQUEUE BLE RX queue depth, pushes, pops, drops, and high-water marks.
RNSMEM Heap, largest block, PSRAM, and current task stack high-water mark.
RNSERR Classified adapter errors: reassembly gaps, short fragments, queue overflow, allocation failure.
```
Normal packet logging is rate-limited: first 25 packets, then every 25th packet, then all packets for two seconds after the first failure. Define `RNS_DEBUG_VERBOSE=1` in `platformio.ini` to print every packet fingerprint.
Debug hooks are inserted at these points:
```text
src/TBeamSupremeBleInterface.cpp BLE callback boundary, fragment TX/RX, reassembly, queue, packet handoff.
src/main.cpp board-name mapping, announce/link events, application Link send.
/usr/local/src/microreticulum/microReticulum/src/Link.cpp
Link encrypt/decrypt token fingerprints and classified decrypt failures.
```
Use the CRC fields to split the failure:
```text
Same RNSTX token crc and RNSRX/RNSDEC token crc, but HMAC_INVALID -> likely wrong key/session/Link context.
Different RNSTX token crc and RNSRX/RNSDEC token crc -> BLE fragmentation, reassembly, queue, or buffer corruption.
RNSERR rx_reassembly_gap/timeout or queue_overflow -> mechanical adapter failure before Reticulum decrypt.
RNSMEM largest_block or min_heap collapse before failures -> heap pressure or fragmentation.
```