microReticulumTbeam/exercises/26_Bluetooth_discover/README.md

# Exercise 26: BLE Discovery

Plain BLE neighborhood discovery for LilyGO T-Beam SUPREME ESP32-S3 units. This exercise is not Reticulum and does not use LoRa.

Each unit displays `Take me outside` at startup, waits for GPS UTC, GPS coordinates, PPS-backed RTC discipline, and then starts BLE advertising/scanning. AMY may run without SD logging because its card reader is defective; other units should mount SD and write logs.

## Build

```sh
cd /usr/local/src/microreticulum/microReticulumTbeam/exercises/26_Bluetooth_discover
pio run -e amy
pio run -e bob
pio run -e cy
pio run -e dan
pio run -e ed
pio run -e flo
pio run -e guy
```

Upload example:

```sh
pio run -e bob -t upload --upload-port /dev/ttytBOB
pio device monitor -b 115200 -p /dev/ttytBOB
```

## Behavior

- Advertises manufacturer data in this format: `B2|NODE|seq|tx_payload_epoch_ms`.
- Accepts current `B2` advertisements and legacy `TBMSND|1|NODE|seq|uptime` advertisements from known nodes in `AMY, BOB, CY, DAN, ED, FLO, GUY`.
- Displays heard nodes sorted by rolling RSSI average, strongest first.
- Drops stale entries after 20 seconds.
- Logs valid accepted advertisements to SD as `/logs/YYYYMMDD_HHMISS_NODE_ble_search.log`.
- Refreshes the receiver GPS position at 1 Hz and logs the last known receiver coordinates plus GPS fix age. Observation rows are written while the holdover clock is valid; stale GPS is recorded with `gps_valid=0` instead of suppressing the row.
- Writes periodic `# STATUS` heartbeat lines so a quiet BLE interval can be distinguished from a reset, hang, SD fault, or power interruption.
- Starts a WiFi AP and HTTP web service during boot. This does not wait for GPS.

Default web addresses:

- AMY: SSID `TBEAM-AMY`, `http://192.168.23.1/`
- BOB: SSID `TBEAM-BOB`, `http://192.168.24.1/`
- CY: SSID `TBEAM-CY`, `http://192.168.25.1/`
- DAN: SSID `TBEAM-DAN`, `http://192.168.26.1/`
- ED: SSID `TBEAM-ED`, `http://192.168.27.1/`
- FLO: SSID `TBEAM-FLO`, `http://192.168.28.1/`
- GUY: SSID `TBEAM-GUY`, `http://192.168.29.1/`

The T-Beam hosts the WiFi access point and web page itself. No service needs to run on the workstation. To use the page, connect the workstation WiFi interface to the unit SSID, for example `TBEAM-ED`, then browse to that unit address, for example `http://192.168.27.1/`. A Panda USB WiFi adapter is useful only as the workstation WiFi interface used to join the T-Beam AP. The root page is a lightweight status page; click `Files` when you want the SD directory listing.

## Field Procedure

1. Flash all units with this BLE Discovery image, each with its own environment.
2. Start all units where they can see sky and wait until each passes GPS/RTC startup discipline.
3. Place one unit near the intended origin, preferably central to a star topology.
4. Carry another unit and watch the OLED.
5. Find a position where it hears only the intended neighbor with weak or moderate RSSI.
6. Place it there.
7. Repeat outward to create a chain.
8. Record the topology manually before switching to the Reticulum BLE-interface firmware.

## Limitations

- RSSI is not a precise distance measurement.
- Body position, antenna orientation, terrain, buildings, and foliage matter.
- BLE advertisements are not guaranteed delivery.
- Scan and advertise duty-cycle choices affect detection latency and power draw.
- WiFi log download can affect BLE timing; use it for retrieving logs, not while making fine placement decisions.

## Bench Acceptance Test

- With AMY and BOB on the bench, AMY displays BOB with RSSI and age, and BOB displays AMY.
- A third unrelated Bluetooth device nearby does not appear on OLED or accepted serial records.
- If BOB is powered off, AMY ages BOB out after the stale timeout.
- Serial output can be captured with:

```sh
pio device monitor -b 115200 -p /dev/ttytAMY | tee logs/ble_discovery_AMY_YYYYMMDD_HHMMSS.log
```
## Logs
Here are two logs from ED and FLO which were activated in the field.
```bash
jlpoole@jp ~/work/tbeam/logs $ ls -lath ed/*_16*
-rw-r--r-- 1 jlpoole jlpoole 1.1M May 25 09:53 ed/20260525_162217_ED_ble_search.log
jlpoole@jp ~/work/tbeam/logs $ ls -lath flo/*_16*
-rw-r--r-- 1 jlpoole jlpoole 1.1M May 25 09:53 flo/20260525_162213_FLO_ble_search.log
jlpoole@jp ~/work/tbeam/logs $ 
```

Here are start and end samples from an earlier trial. These show the previous 12-column schema; current firmware uses the 19-column schema described below.
```bash
jlpoole@jp ~/work/tbeam/logs $ cat -n ed/20260525_162217_ED_ble_search.log|head -n 3
     1  human_time,epoch_ms,receiver,lat,lon,heard,rssi,avg_rssi,age_s,count,seq,payload
     2  2026-05-25 16:22:18,1779726138897,ED,44.9364577,-123.0218702,FLO,-56,-56,0,1,1,TBMSND|1|FLO|0001|0775
     3  2026-05-25 16:22:18,1779726138938,ED,44.9364577,-123.0218702,FLO,-51,-54,0,2,1,TBMSND|1|FLO|0001|0775
jlpoole@jp ~/work/tbeam/logs $ cat -n ed/20260525_162217_ED_ble_search.log|tail -n 1
 10277  2026-05-25 16:47:42,1779727662217,ED,44.9364577,-123.0218702,FLO,-42,-42,0,10276,611,TBMSND|1|FLO|0611|2300
jlpoole@jp ~/work/tbeam/logs $ cat -n flo/20260525_162213_FLO_ble_search.log|head -n 3
     1  human_time,epoch_ms,receiver,lat,lon,heard,rssi,avg_rssi,age_s,count,seq,payload
     2  2026-05-25 16:22:16,1779726136737,FLO,44.9365132,-123.0218183,ED,-52,-52,0,1,0,TBMSND|1|ED|0000|0805
     3  2026-05-25 16:22:16,1779726136829,FLO,44.9365132,-123.0218183,ED,-51,-52,0,2,0,TBMSND|1|ED|0000|0805
jlpoole@jp ~/work/tbeam/logs $ cat -n flo/20260525_162213_FLO_ble_search.log|tail -n 1
 10121  2026-05-25 16:47:37,1779727657219,FLO,44.9365132,-123.0218183,ED,-41,-40,0,10120,608,TBMSND|1|ED|0608|2325
jlpoole@jp ~/work/tbeam/logs $ 
``` 
The header represents:

| Column | Header | Explanation |
| ---: | --- | --- |
| 1 | `human_time` | Receiver timestamp in human-readable UTC form. |
| 2 | `rx_epoch_ms` | Receiver timestamp as Unix epoch milliseconds. |
| 3 | `receiver` | Unit that wrote the log row. |
| 4 | `rx_lat` | Receiver GPS latitude from the latest valid local GPS fix. |
| 5 | `rx_lon` | Receiver GPS longitude from the latest valid local GPS fix. |
| 6 | `gps_fix_age_ms` | Age of the receiver GPS fix in milliseconds when the row was written. If GPS becomes unavailable, the last known coordinates remain and this age grows. |
| 7 | `clock_valid` | `1` when the startup-disciplined clock is still inside the configured holdover window. |
| 8 | `gps_valid` | `1` when the receiver GPS fix age is inside the configured freshness window. |
| 9 | `discipline_age_ms` | Milliseconds since the startup GPS/PPS/RTC discipline event. |
| 10 | `last_discipline_epoch_ms` | Epoch milliseconds assigned to the PPS edge used for startup discipline. |
| 11 | `heard` | Remote unit heard in the BLE advertisement. |
| 12 | `rssi` | RSSI measured by the receiver for this advertisement. |
| 13 | `avg_rssi` | Receiver-calculated arithmetic mean of the most recent RSSI measurements for this heard unit, using up to the last 5 accepted advertisements and rounded to the nearest integer. The window size is `kRssiWindow = 5` in `main.cpp`. |
| 14 | `age_s` | Age in seconds of the displayed/heard entry. |
| 15 | `count` | Number of accepted advertisements from that heard unit. |
| 16 | `seq` | Sequence number advertised by the heard unit. |
| 17 | `tx_payload_epoch_ms` | Sender payload generation timestamp from the BLE payload as Unix epoch milliseconds, when available. Legacy v1 payloads report `0` here. |
| 18 | `payload` | Raw BLE manufacturer-data string. See Payload Definition below. |
| 19 | `vbat_mv` | Battery voltage reported by the PMU, in millivolts. `-1` means the PMU value was unavailable. |

## Payload definition

`payload` is the exact BLE manufacturer-data string received from the other unit. Current firmware advertises a compact v2 text payload:

```text
B2|NODE|SEQ|TX_PAYLOAD_EPOCH_MS
```

Example:

```text
B2|FLO|0611|1779727662217
```

Meaning:

| Part | Example | Meaning |
| --- | --- | --- |
| `B2` | `B2` | Compact Exercise 26 payload prefix and version. |
| `NODE` | `FLO` | Sending unit name. Receiver accepts only known units and ignores itself. |
| `SEQ` | `0611` | Sender’s advertisement sequence number, zero-padded, wraps every 10,000 advertisements. |
| `TX_PAYLOAD_EPOCH_MS` | `1779727662217` | Sender payload generation timestamp as Unix epoch milliseconds, derived from the disciplined local clock. This is not a per-advertisement RF transmit timestamp. |

The receiver also accepts the legacy v1 payload used by earlier Exercise 26 firmware:

```text
TBMSND|1|NODE|SEQ|UPTIME
```

Constraints currently enforced by the receiver:
- Payload must fit in the receive buffer, currently less than 48 bytes.
- It must be pipe-delimited.
- Prefix must be current `B2` or legacy `TBMSND` version `1`.
- Node must be one of `AMY, BOB, CY, DAN, ED, FLO, GUY`.
- Node must not be the receiver’s own name.
- Current `B2` payloads provide `SEQ` and `TX_PAYLOAD_EPOCH_MS`; legacy v1 payloads provide `SEQ` only and log `tx_payload_epoch_ms` as `0`.

## Clock Discipline and Timestamp Semantics

BLE survey advertising starts only after startup GPS UTC, GPS coordinates, and PPS-backed RTC discipline have succeeded. Before that point the OLED remains on a blocking GPS/clock status page and the unit does not enter active BLE survey mode.

Observation logging requires `clock_valid=1`. The clock uses a holdover model: startup discipline sets `last_discipline_epoch_ms`, and `clock_valid` remains true until `discipline_age_ms` exceeds the configured holdover limit. Temporary GPS loss does not immediately invalidate the clock, and it does not suppress observation rows. GPS validity is tracked separately with `gps_fix_age_ms` and `gps_valid`, so stale-coordinate rows can be filtered later.

`rx_epoch_ms` is the receiver's local disciplined-clock time at the moment it writes the observation row. `tx_payload_epoch_ms` is the sender's local disciplined-clock time when it generated and installed that BLE advertising payload. A single payload may be observed many times before the sender replaces it, so `tx_payload_epoch_ms` is useful for correlating logs but is not an RF transmit timestamp.

`rx_epoch_ms - tx_payload_epoch_ms` is diagnostic for clock alignment, scheduling, payload age, and repeated-observation behavior. It is not BLE propagation delay. The status fields that prove a row was written under valid survey conditions are `clock_valid`, `gps_valid`, `gps_fix_age_ms`, `discipline_age_ms`, and `last_discipline_epoch_ms`.

## Event Lines

Comment-style event lines begin with `# EVENT` and are not normal CSV observation rows. They preserve diagnostic transitions without complicating the ordinary CSV row format. Current event names are:

- `boot`
- `log_opened`
- `log_closed`
- `gps_stale`
- `gps_fresh`
- `clock_stale`
- `sd_removed`
- `sd_remounted`
- `sd_write_short`
- `sd_write_ok`

## Status Lines

Comment-style status lines begin with `# STATUS` and are written about every 10 seconds while the log is open. They are not normal CSV observation rows. They report the unit's clock/GPS state, SD/log state, rows written, heard-node counts, and `vbat_mv`. These heartbeat lines are intended to show whether the firmware was still alive during periods with no BLE observations.