Prior to GPS code changes

2026-05-25 11:59:09 -07:00 · 2026-05-25 11:59:09 -07:00 · b407554210
commit b407554210
parent d38c646553
5 changed files with 232 additions and 49 deletions
--- a/exercises/26_Bluetooth_discover/README.md
+++ b/exercises/26_Bluetooth_discover/README.md
@ -1,8 +1,8 @@
-# Exercise 26: Bluetooth Discover
+# Exercise 26: BLE Discovery

-Plain BLE neighborhood sounder for LilyGO T-Beam SUPREME ESP32-S3 units. This exercise is not Reticulum and does not use LoRa.
+Plain BLE neighborhood discovery for LilyGO T-Beam SUPREME ESP32-S3 units. This exercise is not Reticulum and does not use LoRa.

-Each unit waits for GPS UTC, GPS coordinates, PPS-backed RTC discipline, and then starts BLE advertising/scanning. AMY may run without SD logging because its card reader is defective; other units should mount SD and write logs.
+Each unit displays `Take me outside` at startup, waits for GPS UTC, GPS coordinates, PPS-backed RTC discipline, and then starts BLE advertising/scanning. AMY may run without SD logging because its card reader is defective; other units should mount SD and write logs.

 ## Build

@ -20,7 +20,7 @@ pio run -e guy
 Upload example:

 ```sh
-pio run -e bob -t upload
+pio run -e bob -t upload --upload-port /dev/ttytBOB
 pio device monitor -b 115200 -p /dev/ttytBOB
 ```

@ -35,17 +35,19 @@ pio device monitor -b 115200 -p /dev/ttytBOB

 Default web addresses:

- AMY: `http://192.168.23.1/`
- BOB: `http://192.168.24.1/`
- CY: `http://192.168.25.1/`
- DAN: `http://192.168.26.1/`
- ED: `http://192.168.27.1/`
- FLO: `http://192.168.28.1/`
- GUY: `http://192.168.29.1/`
+- AMY: SSID `TBEAM-AMY`, `http://192.168.23.1/`
+- BOB: SSID `TBEAM-BOB`, `http://192.168.24.1/`
+- CY: SSID `TBEAM-CY`, `http://192.168.25.1/`
+- DAN: SSID `TBEAM-DAN`, `http://192.168.26.1/`
+- ED: SSID `TBEAM-ED`, `http://192.168.27.1/`
+- FLO: SSID `TBEAM-FLO`, `http://192.168.28.1/`
+- GUY: SSID `TBEAM-GUY`, `http://192.168.29.1/`
+
+The T-Beam hosts the WiFi access point and web page itself. No service needs to run on the workstation. To use the page, connect the workstation WiFi interface to the unit SSID, for example `TBEAM-ED`, then browse to that unit address, for example `http://192.168.27.1/`. A Panda USB WiFi adapter is useful only as the workstation WiFi interface used to join the T-Beam AP. The root page is a lightweight status page; click `Files` when you want the SD directory listing.

 ## Field Procedure

-1. Flash all units with this sounder image, each with its own environment.
+1. Flash all units with this BLE Discovery image, each with its own environment.
 2. Start all units where they can see sky and wait until each passes GPS/RTC startup discipline.
 3. Place one unit near the intended origin, preferably central to a star topology.
 4. Carry another unit and watch the OLED.
@ -70,5 +72,93 @@ Default web addresses:
 - Serial output can be captured with:

 ```sh
-pio device monitor -b 115200 -p /dev/ttytAMY | tee logs/ble_sounder_AMY_YYYYMMDD_HHMMSS.log
+pio device monitor -b 115200 -p /dev/ttytAMY | tee logs/ble_discovery_AMY_YYYYMMDD_HHMMSS.log
 ```
+## Logs
+Here are two logs from ED and FLO which were activated in the field.
+```bash
+jlpoole@jp ~/work/tbeam/logs $ ls -lath ed/*_16*
+-rw-r--r-- 1 jlpoole jlpoole 1.1M May 25 09:53 ed/20260525_162217_ED_ble_seach.log
+jlpoole@jp ~/work/tbeam/logs $ ls -lath flo/*_16*
+-rw-r--r-- 1 jlpoole jlpoole 1.1M May 25 09:53 flo/20260525_162213_FLO_ble_seach.log
+jlpoole@jp ~/work/tbeam/logs $ 
+```
+
+Here are start and end samples from both:
+```bash
+jlpoole@jp ~/work/tbeam/logs $ cat -n ed/20260525_162217_ED_ble_seach.log|head -n 3
+     1  human_time,epoch_ms,receiver,lat,lon,heard,rssi,avg_rssi,age_s,count,seq,payload
+     2  2026-05-25 16:22:18,1779726138897,ED,44.9364577,-123.0218702,FLO,-56,-56,0,1,1,TBMSND|1|FLO|0001|0775
+     3  2026-05-25 16:22:18,1779726138938,ED,44.9364577,-123.0218702,FLO,-51,-54,0,2,1,TBMSND|1|FLO|0001|0775
+jlpoole@jp ~/work/tbeam/logs $ cat -n ed/20260525_162217_ED_ble_seach.log|tail -n 1
+ 10277  2026-05-25 16:47:42,1779727662217,ED,44.9364577,-123.0218702,FLO,-42,-42,0,10276,611,TBMSND|1|FLO|0611|2300
+jlpoole@jp ~/work/tbeam/logs $ cat -n flo/20260525_162213_FLO_ble_seach.log|head -n 3
+     1  human_time,epoch_ms,receiver,lat,lon,heard,rssi,avg_rssi,age_s,count,seq,payload
+     2  2026-05-25 16:22:16,1779726136737,FLO,44.9365132,-123.0218183,ED,-52,-52,0,1,0,TBMSND|1|ED|0000|0805
+     3  2026-05-25 16:22:16,1779726136829,FLO,44.9365132,-123.0218183,ED,-51,-52,0,2,0,TBMSND|1|ED|0000|0805
+jlpoole@jp ~/work/tbeam/logs $ cat -n flo/20260525_162213_FLO_ble_seach.log|tail -n 1
+ 10121  2026-05-25 16:47:37,1779727657219,FLO,44.9365132,-123.0218183,ED,-41,-40,0,10120,608,TBMSND|1|ED|0608|2325
+jlpoole@jp ~/work/tbeam/logs $ 
+``` 
+The header represents:
+
+| Column | Header | Explanation |
+| ---: | --- | --- |
+| 1 | `human_time` | Receiver timestamp in human-readable UTC form based on Greenwich Mean Time ("G<T") (default for T-Beam units). |
+| 2 | `epoch_ms` | Receiver timestamp as Unix epoch milliseconds GMT. |
+| 3 | `receiver` | Unit that wrote the log row. |
+| 4 | `lat` | Receiver GPS latitude at the time of the row. |
+| 5 | `lon` | Receiver GPS longitude at the time of the row. |
+| 6 | `heard` | Remote unit heard in the BLE advertisement. |
+| 7 | `rssi` | RSSI measured by the receiver for this advertisement. |
+| 8 | `avg_rssi` | Receiver-calculated arithmetic mean of the most recent RSSI measurements for this heard unit, using up to the last 5 accepted advertisements and rounded to the nearest integer.Defined in main.cpp at:
+static constexpr uint8_t kRssiWindow = 5;
+|
+| 9 | `age_s` | Age in seconds of the displayed/heard entry. |
+| 10 | `count` | Number of accepted advertisements from that heard unit. |
+| 11 | `seq` | Sequence number advertised by the heard unit. |
+| 12 | `payload` | (See section Payload Definition below.)|
+
+## Payload definition
+
+`payload` is the exact BLE manufacturer-data string received from the other unit. It is currently defined in [main.cpp](/usr/local/src/microreticulum/microReticulumTbeam/exercises/26_Bluetooth_discover/src/main.cpp:389) as:
+
+```cpp
+"%s|%u|%s|%04lu|%04lu"
+```
+
+So the format is:
+
+```text
+TBMSND|1|NODE|SEQ|UPTIME
+```
+
+Example:
+
+```text
+TBMSND|1|FLO|0611|2300
+```
+
+Meaning:
+
+| Part | Example | Meaning |
+| --- | --- | --- |
+| `TBMSND` | `TBMSND` | Exercise/project BLE prefix. Receiver ignores payloads without this prefix. |
+| `1` | `1` | Payload protocol version. Receiver accepts only version `1`. |
+| `NODE` | `FLO` | Sending unit name. Receiver accepts only known units and ignores itself. |
+| `SEQ` | `0611` | Sender’s advertisement sequence number, zero-padded, wraps every 10,000 advertisements. |
+| `UPTIME` | `2300` | Sender uptime in seconds, zero-padded, modulo 10,000 seconds. |
+
+The numbers you are seeing are the last two fields:
+- `0611`: the sender’s advertisement sequence counter.
+- `2300`: the sender’s uptime seconds modulo 10,000.
+
+Constraints currently enforced by the receiver:
+- Payload must fit in the receive buffer, currently less than 48 bytes.
+- It must be pipe-delimited.
+- Prefix must equal `TBMSND`.
+- Version must equal `1`.
+- Node must be one of `AMY, BOB, CY, DAN, ED, FLO, GUY`.
+- Node must not be the receiver’s own name.
+- The receiver parses `SEQ`, but currently does not parse or use `UPTIME`; it just preserves the full raw payload in the log.
+- 
--- a/exercises/26_Bluetooth_discover/platformio.ini
+++ b/exercises/26_Bluetooth_discover/platformio.ini
@ -1,4 +1,4 @@
-; Exercise 26: plain Bluetooth discovery sounder for LilyGO T-Beam SUPREME
+; Exercise 26: plain Bluetooth discovery for LilyGO T-Beam SUPREME

 [platformio]
 default_envs = cy
--- a/exercises/26_Bluetooth_discover/src/main.cpp
+++ b/exercises/26_Bluetooth_discover/src/main.cpp
@ -1,5 +1,5 @@
 // 2026-05-24 ChatGPT/Codex generated
-// Exercise 26: Bluetooth Discover
+// Exercise 26: BLE Discovery
 // $Id$
 // $HeadURL$
 //
@ -62,6 +62,7 @@ using field_qa::ClockDateTime;
 using field_qa::ClockDiscipline;
 using field_qa::GnssSample;

+static constexpr const char* kAppTitle = "BLE Discovery";
 static constexpr const char* kProjectPrefix = "TBMSND";
 static constexpr uint8_t kProtocolVersion = 1;
 static constexpr uint32_t kStaleMs = 20000;
@ -108,7 +109,8 @@ NodeState g_nodes[] = {

 bool g_disciplined = false;
 bool g_bleStarted = false;
-bool g_storageReady = false;
+bool g_sdReady = false;
+bool g_logReady = false;
 bool g_webReady = false;
 bool g_buttonWasPressed = false;
 uint8_t g_displayMode = 0;
@ -217,6 +219,32 @@ void showLines(const char* l1,
  g_display.showLines(l1, l2, l3, l4, l5, l6);
 }

+void makeTitle(char* out, size_t outSize) {
+  snprintf(out, outSize, "EX 26 %s %s", NODE_NAME, kAppTitle);
+}
+
+void showBootScreen(const char* phase) {
+  char title[32];
+  makeTitle(title, sizeof(title));
+  showLines(title, "Take me outside", "Obtain GPS signal", phase, "BLE disabled");
+}
+
+void showGpsGateScreen(const GnssSample& sample, uint32_t attemptCount) {
+  char title[32];
+  char gpsLine[32];
+  char fixLine[32];
+  makeTitle(title, sizeof(title));
+  snprintf(gpsLine,
+           sizeof(gpsLine),
+           "time:%c loc:%c fix:%c",
+           sample.validTime ? 'Y' : 'N',
+           sample.validLocation ? 'Y' : 'N',
+           sample.validFix ? 'Y' : 'N');
+  snprintf(fixLine, sizeof(fixLine), "sats:%d hdop:%.1f", sample.satsUsed, sample.hdop);
+  showLines(title, "Take me outside", "Obtain GPS signal", gpsLine, fixLine, "BLE disabled");
+  (void)attemptCount;
+}
+
 bool waitForPps(void*, uint32_t timeoutMs) {
  const uint32_t startEdges = g_ppsEdgeCount;
  const uint32_t startMs = millis();
@ -233,7 +261,7 @@ bool waitForPps(void*, uint32_t timeoutMs) {
 void printBootBanner() {
  Serial.println();
  Serial.println("==================================================");
-  Serial.println("Exercise 26: Bluetooth Discover");
+  Serial.println("Exercise 26: BLE Discovery");
  Serial.println("==================================================");
  Serial.printf("node=%s\n", NODE_NAME);
  Serial.printf("prefix=%s version=%u\n", kProjectPrefix, (unsigned)kProtocolVersion);
@ -252,8 +280,10 @@ void markSelfNode() {
  }
 }

+void pollStorageWeb();
+
 bool disciplineStartupClock() {
-  uint32_t lastStatusMs = 0;
+  uint32_t lastStatusMs = millis() - kStartupStatusPeriodMs;
  uint32_t attemptCount = 0;
  while (true) {
    g_gnss.poll();
@ -263,11 +293,7 @@ bool disciplineStartupClock() {
    const uint32_t now = millis();
    if ((uint32_t)(now - lastStatusMs) >= kStartupStatusPeriodMs) {
      lastStatusMs = now;
-      char l2[32];
-      char l3[32];
-      snprintf(l2, sizeof(l2), "gps time:%c loc:%c fix:%c", sample.validTime ? 'Y' : 'N', sample.validLocation ? 'Y' : 'N', sample.validFix ? 'Y' : 'N');
-      snprintf(l3, sizeof(l3), "sats:%d hdop:%.1f", sample.satsUsed, sample.hdop);
-      showLines(NODE_NAME " BLE SOUNDER", "WAIT GPS/RTC", l2, l3, "BLE disabled");
+      showGpsGateScreen(sample, attemptCount);
      Serial.printf("startup_gate node=%s gps_time=%u gps_loc=%u gps_fix=%u sats=%d hdop=%.1f attempts=%lu\n",
                    NODE_NAME,
                    sample.validTime ? 1U : 0U,
@ -301,12 +327,13 @@ bool disciplineStartupClock() {
      }
      Serial.println("clock discipline attempt failed; waiting for next valid GPS sample");
    }
+    pollStorageWeb();
    delay(20);
  }
 }

 bool openDatedLog() {
-  if (!g_storage.ready()) {
+  if (!g_disciplined || !g_storage.ready()) {
    return false;
  }
  char stamp[24];
@ -323,6 +350,7 @@ bool openDatedLog() {
  g_storage.println("human_time,epoch_ms,receiver,lat,lon,heard,rssi,avg_rssi,age_s,count,seq,payload");
  g_storage.flush();
  Serial.printf("sd_log_open path=%s\n", g_logPath);
+  g_logReady = true;
  return true;
 }

@ -333,18 +361,16 @@ void startStorageAndWeb() {
  storageConfig.recoveryRailOffMs = 400;
  storageConfig.recoveryRailOnSettleMs = 1200;

-  g_storageReady = g_storage.begin(storageConfig);
-  if (!g_storageReady) {
+  g_sdReady = g_storage.begin(storageConfig);
+  if (!g_sdReady) {
    Serial.printf("sd_unavailable node=%s err=%s\n", NODE_NAME, g_storage.lastError());
    if (!isSelfName("AMY")) {
      Serial.println("WARNING: SD unavailable; BLE will run, but non-AMY units should be fixed before field logging");
    }
-    return;
  }

-  g_storageReady = openDatedLog();
  tbeam::WebConfig webConfig;
-  webConfig.ssidPrefix = "TBMSND";
+  webConfig.ssidPrefix = "TBEAM";
  webConfig.boardId = NODE_NAME;
  webConfig.password = nullptr;
  webConfig.ipOctet = LOG_AP_IP_OCTET;
@ -354,6 +380,10 @@ void startStorageAndWeb() {
                g_webReady ? 1U : 0U,
                g_web.ssid(),
                g_web.ip().toString().c_str());
+
+  if (g_disciplined && g_sdReady) {
+    g_logReady = openDatedLog();
+  }
 }

 void updateAdvertisement() {
@ -433,7 +463,7 @@ void logAcceptedAdvertisement(const NodeState& node, const char* payload) {
                (unsigned long)node.lastSeq,
                payload);

-  if (g_storageReady && g_storage.isLogOpen()) {
+  if (g_logReady && g_storage.isLogOpen()) {
    char line[224];
    snprintf(line,
             sizeof(line),
@ -473,7 +503,7 @@ void acceptAdvertisement(const char* name, int rssi, uint32_t seq, const char* p
  logAcceptedAdvertisement(node, payload);
 }

-class SounderAdvertisedCallbacks : public BLEAdvertisedDeviceCallbacks {
+class DiscoveryAdvertisedCallbacks : public BLEAdvertisedDeviceCallbacks {
 public:
  void onResult(BLEAdvertisedDevice advertisedDevice) override {
    if (!advertisedDevice.haveManufacturerData()) {
@ -493,7 +523,7 @@ void startBle() {
  BLEDevice::init(std::string("TBMSND-") + NODE_NAME);
  updateAdvertisement();
  g_scan = BLEDevice::getScan();
-  g_scan->setAdvertisedDeviceCallbacks(new SounderAdvertisedCallbacks(), true);
+  g_scan->setAdvertisedDeviceCallbacks(new DiscoveryAdvertisedCallbacks(), true);
  g_scan->setInterval(1349);
  g_scan->setWindow(449);
  g_scan->setActiveScan(false);
@ -531,7 +561,7 @@ void renderDisplay() {
  g_lastDisplayMs = now;

  char title[32];
-  snprintf(title, sizeof(title), "%s BLE SOUNDER", NODE_NAME);
+  makeTitle(title, sizeof(title));

  if (g_displayMode == 1) {
    char rows[5][32] = {};
@ -567,7 +597,7 @@ void renderDisplay() {
    char l3[32];
    char l4[32];
    snprintf(l2, sizeof(l2), "fresh:%lu total:%lu", (unsigned long)freshTotal, (unsigned long)heardTotal);
-    snprintf(l3, sizeof(l3), "sd:%s web:%s", g_storageReady ? "Y" : "N", g_webReady ? "Y" : "N");
+    snprintf(l3, sizeof(l3), "sd:%s web:%s log:%s", g_sdReady ? "Y" : "N", g_webReady ? "Y" : "N", g_logReady ? "Y" : "N");
    snprintf(l4, sizeof(l4), "log:%s", g_logPath[0] ? g_logPath + 6 : "none");
    showLines(title, "DIAG", l2, l3, l4);
    return;
@ -604,17 +634,23 @@ void renderDisplay() {
 void pollStorageWeb() {
  g_storage.update();
  if (g_storage.consumeRemovedEvent()) {
-    g_storageReady = false;
+    g_sdReady = false;
+    g_logReady = false;
    g_logPath[0] = '\0';
    Serial.println("sd_removed logging_disabled");
  }
-  if (g_storage.consumeMountedEvent() && !g_storageReady) {
-    g_storageReady = openDatedLog();
+  if (g_storage.consumeMountedEvent()) {
+    g_sdReady = true;
+    Serial.println("sd_mounted");
+  }
+  if (g_disciplined && g_storage.ready() && !g_logReady) {
+    g_sdReady = true;
+    g_logReady = openDatedLog();
  }
  if (g_webReady) {
    g_web.update();
  }
-  if (g_storageReady && (uint32_t)(millis() - g_lastFlushMs) >= kLogFlushPeriodMs) {
+  if (g_logReady && (uint32_t)(millis() - g_lastFlushMs) >= kLogFlushPeriodMs) {
    g_lastFlushMs = millis();
    g_storage.flush();
  }
@ -624,24 +660,30 @@ void pollStorageWeb() {

 void setup() {
  Serial.begin(115200);
-  delay(3000);
-  printBootBanner();
-  markSelfNode();
-  pinMode(BUTTON_PIN, INPUT_PULLUP);
-
  if (!tbeam_supreme::initPmuForPeripherals(g_pmu, &Serial)) {
    Serial.println("WARNING: PMU init failed");
  }
  g_display.begin();
-  showLines(NODE_NAME " BLE SOUNDER", "booting", "PMU/OLED/GPS");
+  showBootScreen("PMU/OLED ready");
+  printBootBanner();
+  markSelfNode();
+  pinMode(BUTTON_PIN, INPUT_PULLUP);
+
+  showBootScreen("Starting Web");
+  startStorageAndWeb();
+  showBootScreen("Starting GPS");
  g_gnss.begin();
+  showBootScreen("Probing GPS");
  (void)g_gnss.probeAtStartup(Serial);
 #ifdef GPS_1PPS_PIN
  attachInterrupt(digitalPinToInterrupt(GPS_1PPS_PIN), onPpsEdge, RISING);
 #endif

+  showBootScreen("BLE disabled");
  g_disciplined = disciplineStartupClock();
-  startStorageAndWeb();
+  showBootScreen("GPS disciplined");
+  pollStorageWeb();
+  showBootScreen("Starting BLE");
  startBle();
 }

--- a/lib/tbeam_web/src/TBeamWeb.cpp
+++ b/lib/tbeam_web/src/TBeamWeb.cpp
@ -96,18 +96,31 @@ void TBeamWeb::handleNotFoundThunk() {
 }

 void TBeamWeb::handleRoot() {
+  bool countTruncated = false;
+  const bool sdReady = storage_ && storage_->ready();
+  const size_t logsCount = sdReady ? countDirectoryEntries("/logs", 250, &countTruncated) : 0;
+
  String body;
  body.reserve(2048);
  body += F("<!doctype html><html><head><meta charset='utf-8'>");
  body += F("<meta name='viewport' content='width=device-width,initial-scale=1'>");
-  body += F("<title>T-Beam Files</title></head><body>");
-  body += F("<h1>T-Beam Files</h1>");
+  body += F("<title>T-Beam Status</title></head><body>");
+  body += F("<h1>T-Beam Status</h1>");
  body += F("<p>SSID: ");
  body += htmlEscape(ssid_);
  body += F("</p><p>IP: ");
  body += htmlEscape(ip_.toString());
  body += F("</p><p>SD: ");
-  body += (storage_ && storage_->ready()) ? F("mounted") : F("not mounted");
+  body += sdReady ? F("mounted") : F("not mounted");
+  body += F("</p><p>Log entries: ");
+  if (sdReady) {
+    body += String((unsigned long)logsCount);
+    if (countTruncated) {
+      body += F("+");
+    }
+  } else {
+    body += F("unavailable");
+  }
  body += F("</p><p>Stations: ");
  body += String(stationCount());
  body += F("</p><p><a href='/files?path=/logs'>Files</a> ");
@ -154,7 +167,7 @@ void TBeamWeb::handleFiles() {
  body += F("<h1>SD Files</h1><p>Path: ");
  body += htmlEscape(path);
  body += F("</p><p><a href='/'>Home</a> <a href='/files?path=/'>Root</a> <a href='/files?path=/logs'>Logs</a></p><ul>");
-  listDirectoryHtml(body, path, 4);
+  listDirectoryHtml(body, path, 0);
  body += F("</ul></body></html>");
  server_.send(200, "text/html", body);
 }
@ -289,6 +302,43 @@ void TBeamWeb::listDirectoryHtml(String& body, const char* path, uint8_t depth)
  dir.close();
 }

+size_t TBeamWeb::countDirectoryEntries(const char* path, size_t maxEntries, bool* truncated) {
+  if (truncated) {
+    *truncated = false;
+  }
+  if (maxEntries == 0) {
+    return 0;
+  }
+  File dir = SD.open(path, FILE_READ);
+  if (!dir) {
+    return 0;
+  }
+  if (!dir.isDirectory()) {
+    dir.close();
+    return 1;
+  }
+
+  size_t count = 0;
+  File entry = dir.openNextFile();
+  while (entry) {
+    entry.close();
+    ++count;
+    if (count >= maxEntries) {
+      File extra = dir.openNextFile();
+      if (extra) {
+        if (truncated) {
+          *truncated = true;
+        }
+        extra.close();
+      }
+      break;
+    }
+    entry = dir.openNextFile();
+  }
+  dir.close();
+  return count;
+}
+
 bool TBeamWeb::normalizePath(const String& input, char* out, size_t outSize) const {
  if (!out || outSize < 2 || input.length() == 0 || input.indexOf("..") >= 0) {
    return false;
--- a/lib/tbeam_web/src/TBeamWeb.h
+++ b/lib/tbeam_web/src/TBeamWeb.h
@ -48,6 +48,7 @@ class TBeamWeb {
  void handleNotFound();

  void listDirectoryHtml(String& body, const char* path, uint8_t depth);
+  size_t countDirectoryEntries(const char* path, size_t maxEntries, bool* truncated);
  bool normalizePath(const String& input, char* out, size_t outSize) const;
  String htmlEscape(const String& in) const;
  String urlEncode(const String& in) const;