Commit graph

73 commits

Author SHA1 Message Date
torlando-tech
f3cafedb60 fix(ble): Resolve connection role and startup errors
This commit addresses two critical issues that prevented the BLE
interface from functioning correctly after the driver abstraction
refactor.

1.  **Fix `exec()` Startup Error:**
    The interface failed to load via `rnsd` due to a `KeyError: '__name__'`
    caused by using relative imports (`from . import ...`). The `exec()`
    environment used by Reticulum does not preserve package context,
    breaking these imports. This is fixed by reverting to absolute
    imports (`from bluetooth_driver import ...`) which work correctly
    with the existing `sys.path` manipulation logic.

2.  **Fix Connection Role Logic:**
    Connections were failing because the interface would always attempt
    to read the peer's identity, even when acting as the peripheral.
    This caused a `Can only read characteristics in central mode` error.

    The fix introduces role-aware logic into the connection callback:
    - A `get_peer_role()` method was added to the driver interface.
    - `BLEInterface` now checks the role on connection.
    - If central, it reads the identity characteristic.
    - If peripheral, it waits for the identity handshake packet,
      preventing the invalid operation.
2025-11-03 23:43:30 -05:00
torlando-tech
38ebd17008 fix import 2025-11-03 23:25:49 -05:00
torlando-tech
bbbe3afd21 Merge: Resolve conflicts after moving driver files to src/RNS/Interfaces
Resolved import conflicts by keeping the updated relative import paths.
Driver files are now correctly located in src/RNS/Interfaces/ instead of
project root.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 23:19:35 -05:00
torlando-tech
63064ccf3a Refactor BLEInterface to driver-based architecture
Major architectural refactoring to separate high-level Reticulum protocol
logic from platform-specific Bluetooth operations. This enables code sharing
between pure Python and Android (Columba) implementations, improves
testability, and creates a clean boundary for future platform support.

ARCHITECTURE CHANGES:

1. **Driver Abstraction Layer**
   - Created BLEDriverInterface (bluetooth_driver.py) defining the contract
     for all platform-specific BLE drivers
   - Abstraction includes 18 methods + 6 callbacks for complete BLE lifecycle
   - Enhanced BLEDevice dataclass with service_uuids and manufacturer_data
   - Added on_mtu_negotiated callback for delayed MTU reporting
   - Added on_error callback for consistent platform error reporting

2. **Linux Driver Implementation**
   - Created LinuxBluetoothDriver (linux_bluetooth_driver.py, 1534 lines)
   - Moved ALL bleak/bluezero/D-Bus code from BLEInterface
   - Preserves 5 critical platform workarounds:
     * BlueZ ServicesResolved race condition patch
     * D-Bus LE-only connection (ConnectDevice)
     * BLE Agent registration for Just Works pairing
     * MTU negotiation with 3-method fallback
     * Service discovery delay for bluezero timing
   - Role-aware send() automatically chooses GATT write vs notification
   - Dedicated asyncio event loop management in separate thread
   - Configuration via constructor (no Reticulum dependencies)

3. **Refactored BLEInterface**
   - Removed 801 lines (32.3% reduction: 2479 → 1678 lines)
   - Removed all platform-specific imports (bleak, bluezero, dbus_fast)
   - Removed 9 async methods (moved to driver)
   - Driver dependency injection via constructor
   - Implemented 6 driver callbacks for event handling
   - PRESERVED high-level logic:
     * Peer scoring algorithm (RSSI + history + recency)
     * Connection blacklist with exponential backoff
     * MAC-based connection direction (prevents dual connections)
     * Fragmentation/reassembly orchestration (identity-based keying)
     * Interface spawning per peer

4. **Simplified BLEPeerInterface**
   - Removed connection_type, client, mtu parameters
   - Deleted _send_via_central() and _send_via_peripheral() methods
   - Single send path via driver.send() (driver handles role routing)
   - 77 lines removed from peer interface class

5. **Mock Driver for Testing**
   - Created MockBLEDriver (tests/mock_ble_driver.py)
   - Complete BLEDriverInterface implementation without hardware
   - Bidirectional communication via link_drivers()
   - Enables unit testing of BLEInterface logic (fragmentation, reassembly,
     peer lifecycle, blacklist management)

CRITICAL FIXES:

1. **Restored Periodic Cleanup Task** (CRITICAL: prevents memory leaks)
   - Converted from async (driver-owned loop) to threading.Timer
   - Runs every 30 seconds to clean stale reassembly buffers
   - Essential for long-running instances (Pi Zero with 512MB RAM)
   - Properly cancelled in detach() for clean shutdown

2. **Fixed Naming Consistency**
   - Renamed processOutgoing → process_outgoing (snake_case)

FILES MODIFIED:
- src/RNS/Interfaces/BLEInterface.py (refactored, -801 lines)

FILES ADDED:
- bluetooth_driver.py (driver abstraction interface)
- linux_bluetooth_driver.py (Linux/BlueZ implementation, 1534 lines)
- tests/mock_ble_driver.py (mock driver for unit tests)
- REFACTORING_GUIDE.md (comprehensive refactoring documentation)
- BLE_PROTOCOL_v2.2.md (protocol specification)
- tests/test_refactor_suite.py (initial test suite)

BENEFITS:

1. **Testability** - Mock driver enables hardware-free unit testing
2. **Portability** - Easy to create Android/Windows/macOS drivers
3. **Maintainability** - Platform quirks isolated in single driver file
4. **Code Sharing** - High-level logic shared across all platforms
5. **Clean Architecture** - Clear separation of concerns

TESTING REQUIRED:

- Tier 1 (Unit): Test with MockBLEDriver (fragmentation, reassembly, lifecycle)
- Tier 2 (Integration): Test on Raspberry Pi hardware (scanning, connecting,
  dual mode, MTU negotiation, identity exchange)
- Tier 3 (Regression): Full Reticulum stack (announces, LXMF, multi-hop)
- Tier 4 (Edge Cases): MAC rotation, identity handshake, reconnection,
  reassembly timeout, discovery cache pruning

BACKWARD COMPATIBILITY:

- Configuration: Fully backward compatible (same config parameters)
- Protocol: No changes to BLE wire protocol (v2.2)
- Interface API: Unchanged for Reticulum Transport integration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 23:15:22 -05:00
torlando-tech
8fea6c810d Refactor BLEInterface to driver-based architecture
Major architectural refactoring to separate high-level Reticulum protocol
logic from platform-specific Bluetooth operations. This enables code sharing
between pure Python and Android (Columba) implementations, improves
testability, and creates a clean boundary for future platform support.

ARCHITECTURE CHANGES:

1. **Driver Abstraction Layer**
   - Created BLEDriverInterface (bluetooth_driver.py) defining the contract
     for all platform-specific BLE drivers
   - Abstraction includes 18 methods + 6 callbacks for complete BLE lifecycle
   - Enhanced BLEDevice dataclass with service_uuids and manufacturer_data
   - Added on_mtu_negotiated callback for delayed MTU reporting
   - Added on_error callback for consistent platform error reporting

2. **Linux Driver Implementation**
   - Created LinuxBluetoothDriver (linux_bluetooth_driver.py, 1534 lines)
   - Moved ALL bleak/bluezero/D-Bus code from BLEInterface
   - Preserves 5 critical platform workarounds:
     * BlueZ ServicesResolved race condition patch
     * D-Bus LE-only connection (ConnectDevice)
     * BLE Agent registration for Just Works pairing
     * MTU negotiation with 3-method fallback
     * Service discovery delay for bluezero timing
   - Role-aware send() automatically chooses GATT write vs notification
   - Dedicated asyncio event loop management in separate thread
   - Configuration via constructor (no Reticulum dependencies)

3. **Refactored BLEInterface**
   - Removed 801 lines (32.3% reduction: 2479 → 1678 lines)
   - Removed all platform-specific imports (bleak, bluezero, dbus_fast)
   - Removed 9 async methods (moved to driver)
   - Driver dependency injection via constructor
   - Implemented 6 driver callbacks for event handling
   - PRESERVED high-level logic:
     * Peer scoring algorithm (RSSI + history + recency)
     * Connection blacklist with exponential backoff
     * MAC-based connection direction (prevents dual connections)
     * Fragmentation/reassembly orchestration (identity-based keying)
     * Interface spawning per peer

4. **Simplified BLEPeerInterface**
   - Removed connection_type, client, mtu parameters
   - Deleted _send_via_central() and _send_via_peripheral() methods
   - Single send path via driver.send() (driver handles role routing)
   - 77 lines removed from peer interface class

5. **Mock Driver for Testing**
   - Created MockBLEDriver (tests/mock_ble_driver.py)
   - Complete BLEDriverInterface implementation without hardware
   - Bidirectional communication via link_drivers()
   - Enables unit testing of BLEInterface logic (fragmentation, reassembly,
     peer lifecycle, blacklist management)

CRITICAL FIXES:

1. **Restored Periodic Cleanup Task** (CRITICAL: prevents memory leaks)
   - Converted from async (driver-owned loop) to threading.Timer
   - Runs every 30 seconds to clean stale reassembly buffers
   - Essential for long-running instances (Pi Zero with 512MB RAM)
   - Properly cancelled in detach() for clean shutdown

2. **Fixed Naming Consistency**
   - Renamed processOutgoing → process_outgoing (snake_case)

FILES MODIFIED:
- src/RNS/Interfaces/BLEInterface.py (refactored, -801 lines)

FILES ADDED:
- bluetooth_driver.py (driver abstraction interface)
- linux_bluetooth_driver.py (Linux/BlueZ implementation, 1534 lines)
- tests/mock_ble_driver.py (mock driver for unit tests)
- REFACTORING_GUIDE.md (comprehensive refactoring documentation)
- BLE_PROTOCOL_v2.2.md (protocol specification)
- tests/test_refactor_suite.py (initial test suite)

BENEFITS:

1. **Testability** - Mock driver enables hardware-free unit testing
2. **Portability** - Easy to create Android/Windows/macOS drivers
3. **Maintainability** - Platform quirks isolated in single driver file
4. **Code Sharing** - High-level logic shared across all platforms
5. **Clean Architecture** - Clear separation of concerns

TESTING REQUIRED:

- Tier 1 (Unit): Test with MockBLEDriver (fragmentation, reassembly, lifecycle)
- Tier 2 (Integration): Test on Raspberry Pi hardware (scanning, connecting,
  dual mode, MTU negotiation, identity exchange)
- Tier 3 (Regression): Full Reticulum stack (announces, LXMF, multi-hop)
- Tier 4 (Edge Cases): MAC rotation, identity handshake, reconnection,
  reassembly timeout, discovery cache pruning

BACKWARD COMPATIBILITY:

- Configuration: Fully backward compatible (same config parameters)
- Protocol: No changes to BLE wire protocol (v2.2)
- Interface API: Unchanged for Reticulum Transport integration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-03 23:03:54 -05:00
torlando-tech
4383b1ec65 try fix mtu negotiation 2025-11-02 13:46:48 -05:00
torlando-tech
e73d7cdcc2 use id based reassembler lookup on receiving notifications 2025-11-02 12:56:46 -05:00
torlando-tech
7017c3d53a restore id handshake 2025-11-02 12:38:34 -05:00
torlando-tech
da551cb627 add diag logs, early return guard, type protection 2025-11-02 01:32:52 -04:00
torlando-tech
2ad3f3f46d diag logs 2025-11-01 22:55:29 -04:00
torlando-tech
cb8dd19279 refactor: Simplify BLE protocol implementation and remove scope creep
Major cleanup of BLE interface implementation to focus on core identity-based
tracking goal while removing unnecessary complexity added during troubleshooting.

Key changes:
- Remove unified dual-connection architecture (single-direction connections)
- Remove Protocol v1 MAC-based compatibility fallbacks (~200 lines)
- Simplify connection handshake (handle_peripheral_data: 173→54 lines)
- Extract _compute_identity_hash() helper (DRY: 11 duplicates removed)
- Add 60s timeout to identity wait loop (prevent hung threads)
- Remove GATT characteristic descriptors (UUID 2901)
- Remove DIAGNOSTIC logging statements (~15 occurrences)
- Revert TX characteristic to 'notify' flag (better throughput)

Net reduction: 249 lines removed (473 deletions, 224 additions)

Maintains core functionality:
- Identity characteristic for stable tracking (MAC rotation immunity)
- Identity-based device naming (Protocol v2.1)
- MAC sorting for connection direction (Protocol v2.2)
- Identity-keyed fragmenters/reassemblers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-01 22:01:21 -04:00
torlando-tech
cdd642a70f feat: Add identity-based device naming for BLE discovery (Protocol v2.1)
Implement automatic device name generation from Transport.identity hash
to enable reliable peer discovery when bluezero service_uuid exposure
is unreliable.

Changes:
- Auto-generate device_name as RNS-{32-hex-identity} if not configured
- Parse peer identity from device name pattern (RNS-[0-9a-f]{32})
- Update GATT server device_name before advertising
- Store parsed identities in address_to_identity mapping

Limitations discovered:
- bluezero Peripheral uses system hostname for BLE local_name, not
  the device_name parameter we set
- BlueZ D-Bus cache issues cause service_uuid exposure to be unreliable
- Reboot + cache clear (/var/lib/bluetooth/*/cache) temporarily fixes
  service_uuid visibility

Current status:
- Bidirectional discovery works via service_uuid after fresh reboot
- Identity parsing infrastructure ready for future manufacturer_data approach
- Fallback to Protocol v1 address-based tracking remains functional

Tested on Raspberry Pi 4 with BlueZ 5.76, bluezero 0.9.1, bleak 1.1.1

Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-01 14:33:12 -04:00
torlando-tech
07e1f2e426 feat: Add peripheral fragmenter creation for bidirectional BLE data flow
Enable BLE peripheral connections to send data by creating fragmenters
in handle_peripheral_data() after identity handshake. Previously,
fragmenters were only created for central connections (_connect_to_peer),
which caused "No fragmenter for peer" warnings when peripheral-only
connections attempted to transmit data.

This fix ensures bidirectional data flow works correctly regardless of
which device initiates the BLE connection, completing the unified
interface architecture.

Impact: Fixes announce rebroadcasting from peripheral-only connections
and enables full mesh networking over BLE.

Tested on Raspberry Pi 4 with BlueZ 5.76.

Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 23:15:02 -04:00
torlando-tech
e79cd5320d fix: Use identity-based keying for fragmenters/reassemblers (MAC rotation immunity)
Critical fix for message delivery and Android MAC rotation support.

**Problem:**
- Fragmenters keyed by MAC address
- Failed with "dev:" prefix mismatch
- Would break on Android MAC rotation

**Solution:**
Use identity_hash for fragmenter/reassembler keys (with Protocol v1 MAC fallback).

**Changes:**
1. Added _get_fragmenter_key() helper - returns identity_hash or normalized MAC
2. Updated _connect_to_peer() - creates fragmenters with identity keys
3. Updated BLEPeerInterface.processOutgoing() - looks up fragmenters with identity keys

**Benefits:**
-  Fixes immediate "No fragmenter" bug
-  Survives Android MAC address rotation
-  Consistent with unified interface architecture
-  One fragmenter per peer identity (not per ephemeral MAC)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 21:06:58 -04:00
torlando-tech
c5f8ff153f feat: Add identity exchange in connection handshake for true unified interfaces
Enhances BLE Protocol v2 handshake to include the central's identity (16 bytes)
instead of empty bytes. This enables the peripheral side to create identity-based
unified interfaces even without discovering the central via scanning.

**Problem Solved:**
- Peripheral couldn't create identity-based interface without scanning the central
- Resulted in separate "legacy" and identity-based interfaces for same peer
- Prevented true interface unification in asymmetric discovery scenarios

**Solution:**
1. Central sends its own identity (16 bytes) in handshake write
2. Peripheral detects identity handshake (16 bytes, first write)
3. Peripheral extracts identity and migrates interface from legacy to identity-based
4. Both sides now have identity-based interfaces that can unify!

**Changes:**

**_connect_to_peer() (line 1487):**
```python
# OLD: await client.write_gatt_char(RX_UUID, b'', response=True)
# NEW: Send our own identity in handshake
our_identity = self.gatt_server.identity_value if self.gatt_server else b'\x00' * 16
await client.write_gatt_char(RX_UUID, our_identity, response=True)
```

**handle_peripheral_data() (line 1792):**
```python
# Detect identity handshake (16 bytes, first write)
if len(data) == 16 and sender_address not in self.address_to_identity:
    central_identity = bytes(data)
    central_identity_hash = RNS.Identity.full_hash(central_identity)[:16].hex()[:16]

    # Store identity mapping
    self.address_to_identity[sender_address] = central_identity
    self.identity_to_address[central_identity_hash] = sender_address

    # Migrate interface from legacy to identity-based tracking
    legacy_conn_id = f"{sender_address}-peripheral"
    if legacy_conn_id in self.spawned_interfaces:
        legacy_if = self.spawned_interfaces[legacy_conn_id]
        del self.spawned_interfaces[legacy_conn_id]
        legacy_if.peer_identity = central_identity
        self.spawned_interfaces[central_identity_hash] = legacy_if

    return  # Don't process handshake as fragment data
```

**Flow:**
1. Pi1 connects to Pi2 as central
2. Pi1 reads Pi2's identity → creates identity-based interface
3. Pi1 sends handshake WITH Pi1's identity
4. Pi2 receives handshake, extracts Pi1's identity
5. Pi2 migrates interface to identity-based tracking
6. When Pi2 later discovers Pi1, adds central connection to SAME interface
7. Result: Both Pis have unified "central+peripheral" interfaces!

**Benefits:**
-  Works with asymmetric discovery (only one side scans)
-  Enables true unified interfaces in all scenarios
-  Solves Android backgrounding (peripheral gets central's identity immediately)
-  Faster interface unification (don't wait for bidirectional discovery)

**Backward Compatibility:**
- Protocol v1 devices send/receive empty handshake, work as before
- Handshake size detection (0 vs 16 bytes) determines protocol version

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 20:47:42 -04:00
torlando-tech
af06243939 feat: Implement unified BLE interface architecture with dual-connection support
Major architectural improvement enabling one BLEPeerInterface to handle BOTH
central and peripheral connections for a given peer identity, eliminating
duplicate interfaces and fixing ACK routing issues.

**Key Changes:**

1. **BLEPeerInterface Dual-Connection Support:**
   - Added has_central_connection/has_peripheral_connection flags
   - Added add_central_connection() and add_peripheral_connection() methods
   - Intelligent routing in processOutgoing() - prefers central, falls back to peripheral
   - Graceful degradation when only one connection type exists

2. **Identity-Based Interface Tracking:**
   - Changed spawned_interfaces key from address-based to identity_hash
   - Added address_to_identity and identity_to_address mapping dicts
   - Enables stable peer tracking despite MAC address rotation

3. **Unified Spawning Method:**
   - Created _spawn_or_update_peer_interface() to replace old _spawn_peer_interface()
   - Checks if interface exists, adds new connection type if so
   - Creates new interface with first connection type otherwise

4. **Updated Connection Handlers:**
   - handle_central_connected(): Uses unified interface spawning for peripheral connections
   - handle_central_disconnected(): Removes peripheral connection, only detaches if no connections remain
   - Disconnect callback in _connect_to_peer(): Removes central connection with graceful cleanup

5. **Updated Data Routing:**
   - _handle_ble_data(): Routes by identity_hash instead of address-based conn_id
   - handle_peripheral_data(): Routes by identity_hash with Protocol v1 fallback

**Benefits:**
-  Fixes ACK routing issue (only 1 interface per peer instead of 2-4)
-  Identity-based tracking immune to MAC rotation
-  Path redundancy - can use both connections if available
-  Android backgrounding ready - peripheral path survives when app can't scan
-  Backward compatible with Protocol v1 devices

**Testing:**
- Pi-to-Pi bidirectional discovery
- Round-trip LXMF messaging with ACK verification
- Connection dynamics (loss/recovery)

Fixes ACK routing issue discovered in testing session 2025-10-31.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 19:48:07 -04:00
torlando-tech
cd723e08c3 feat: Add connection handshake to trigger peripheral callbacks
Sends empty WRITE to RX characteristic immediately after connection
to guarantee remote side's on_central_connected callback fires.

Problem: Peripheral callback triggered by WRITE events, not connections.
When central connects and only READs (Identity characteristic), the
peripheral's on_central_connected never fires, preventing peer interface
spawning on the peripheral side.

Solution: After reading Identity, write empty bytes to RX characteristic.
This triggers the WRITE callback which calls _handle_central_connected(),
ensuring bidirectional peer interface spawning.

Benefits:
- Works for Pi-to-Pi (ensures both sides spawn interfaces)
- Works for Android-to-Pi (Pi spawns interface when Android connects)
- Minimal overhead (single empty GATT write)
- Backwards compatible (empty write is harmless)

Implementation:
- Added after Identity read in _connect_to_peer()
- Uses write_gatt_char() with response=True for reliability
- Non-critical failure (logged as warning, doesn't block connection)
- TODO comment for future handshake protocol enhancements

This solves the asymmetric peer spawning issue seen in testing where
only the central side had a peer interface.

Tested: Enables bidirectional data flow for single-direction discoveries.
2025-10-31 17:43:10 -04:00
torlando-tech
27d7ea91a3 fix: Remove non-existent tunnel() method calls
Removed calls to self.owner.tunnel(peer_if) which caused AttributeError.

Root cause: Transport class doesn't have a tunnel() method. The tunnel()
method was incorrectly assumed based on other interface patterns, but
direct peer interfaces (like I2PInterface) only use:
  RNS.Transport.interfaces.append(peer_if)

No tunnel registration is needed for direct peer connections.

Changes:
- Removed tunnel() call from central connection spawn (~line 1607)
- Removed tunnel() call from peripheral connection spawn (~line 1778)
- Added explanatory comment about I2PInterface pattern

This fixes the AttributeError seen in Pi logs:
  "failed to connect: AttributeError: type object 'Transport'
   has no attribute 'tunnel'"

Peer interfaces still register correctly via RNS.Transport.interfaces[].

Tested: Interface spawning works, AttributeError eliminated.
2025-10-31 15:18:43 -04:00
torlando-tech
fae7a8c954 fix: Add debug logging and accept RSSI -127 from BlueZ
Fixes critical discovery issues caused by BlueZ/Bleak limitations.

Root cause analysis (via nRF Connect + debug logging):
1. Bleak doesn't parse service UUIDs from advertisement data (service_uuids=[])
   despite UUIDs being present (verified with nRF Connect showing correct UUID)
2. Name-based fallback works but RSSI -127 caused rejection
3. BlueZ hides connected/known devices from scan results

Changes:
- Added debug logging to detection_callback to diagnose Bleak data parsing
- Accept RSSI -127 as valid (BlueZ sentinel for "RSSI unknown")
- Confirmed name fallback pattern (RNS-*) works when service UUID fails

Test results:
- nRF Connect confirms correct UUID in advertisement: 37145b00-442d-4a94-917f-8f42c5da28e3
- Bleak sees device name "RNS-Pi1" but service_uuids=[]
- After bluetoothctl remove + RSSI fix: discovered via name pattern
- Asymmetric success: Pi 1→Pi 2 peer interface spawned, 72 bytes transmitted

Known issues:
- Bleak/BlueZ doesn't populate service_uuids from advertisement (Linux limitation)
- BlueZ auto-reconnects and hides devices from scans (requires bluetoothctl remove)
- Asymmetric discovery due to scan-hiding issue

Related: BLE_TEST_RESULTS_2025_10_31.md, BLE_DISCOVERY_TROUBLESHOOTING.md
2025-10-31 15:08:20 -04:00
torlando-tech
693cf185e4 fix: Replace placeholder BLE UUIDs with Reticulum standard UUIDs
Fixed discovery failure caused by GATT server advertising wrong service UUIDs.

Root cause: BLEGATTServer and BLEInterface were using placeholder/test UUIDs
(00000001-5824-4f48-9e1a-3b3e8f0c1234 etc.) instead of the Reticulum standard
UUID namespace (37145b00-442d-4a94-917f-8f42c5da28e*).

This caused Pis to advertise services that scanners couldn't recognize,
blocking all BLE discovery and connection attempts.

Changes:
- BLEGATTServer.py: Updated all 4 service/characteristic UUIDs
- BLEInterface.py: Updated all 4 service/characteristic UUIDs

Diagnosed using nRF Connect mobile app which showed wrong UUIDs being advertised.

Related: BLE_TEST_RESULTS_2025_10_31.md
2025-10-31 14:04:06 -04:00
torlando-tech
3c28070d3b Fix: Add missing tunnel registration for BLEPeerInterface
Problem:
- BLEPeerInterface was spawning but never calling owner.tunnel(self)
- Result: 0 tunnel table entries, no data transmission
- Peer interfaces showed as "reachable" but Transport couldn't route through them

Solution:
- Added owner.tunnel(peer_if) call after interface creation
- Applied to both spawn locations (central and peripheral connections)
- Pattern matches Android implementation in android_ble_interface.py

Changes:
- Line 1599-1601: Added tunnel registration for central connections
- Line 1770-1772: Added tunnel registration for peripheral connections

Testing:
- Peer interfaces now appear in rnstatus output
- BLEPeerInterface[RNS-Pi2/central] visible and marked as "reachable"
- AttributeError logged during tunnel() but interface still spawns
- Further investigation needed for data transmission

References:
- BLE_DATA_ROUTING_ISSUE.md - Root cause analysis
- BLE_SESSION_2025_10_31_PROGRESS.md - Detailed session notes
- android_ble_interface.py:403 - Reference implementation

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 11:42:08 -04:00
torlando-tech
905f9b684c refactor: use final_init() hook with infinite wait instead of timeout polling
Replace timeout-based polling with cleaner event-driven approach using Interface.final_init() lifecycle hook. Launches background thread that waits indefinitely for Transport.identity (which is guaranteed to load), then starts GATT server with valid 16-byte identity value.

Benefits:
- No arbitrary timeout (Transport.identity WILL load, just timing varies)
- Uses proper Interface lifecycle hook (final_init)
- Non-blocking background thread
- GATT server guaranteed to have valid identity when it starts
- Cleaner separation of concerns

Same polling mechanism as I2PInterface, but better integrated with Interface lifecycle.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 21:30:51 -04:00
torlando-tech
1a4de3b4ea fix: resolve deadlock between interface online and Transport.identity loading
Move self.online = True BEFORE waiting for Transport.identity to break circular dependency. Reticulum loads Transport.identity only after interfaces are online, so blocking before self.online = True creates infinite wait.

New sequence:
1. Set self.online = True (unblocks Reticulum startup)
2. Reticulum loads Transport.identity from storage
3. Wait completes successfully
4. Identity set on GATT server
5. GATT server starts with valid 16-byte identity

Reduced timeout from 30s to 10s since identity should load within 1s once interface is online.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 21:15:42 -04:00
torlando-tech
6ca2e85142 fix: ensure Transport.identity loaded before GATT server starts
Change from async deferred loading to synchronous wait before GATT server startup. This ensures the Identity characteristic is created with a valid 16-byte value instead of empty [], preventing BlueZ from rejecting or corrupting the advertisement which caused "0 matching service UUID" discovery failures.

The bug: Identity characteristic was being created with value=[] because the GATT server thread started before Transport.identity was loaded from storage (~1s timing window). BlueZ may silently reject advertisements when validating GATT databases with empty READ characteristics.

The fix: Block interface startup for up to 30s waiting for Transport.identity (typically available within 0.5-1s), then set it on GATT server BEFORE starting the server thread. Identity characteristic now always has valid 16-byte value when registered with BlueZ.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 20:40:31 -04:00
torlando-tech
76bfa019ca fix: defer Transport.identity loading to avoid startup timing issue
Move Transport.identity extraction from synchronous startup to async background task. The identity is loaded from storage AFTER interface initialization, causing "Transport.identity not available yet" warning. Now polls for identity every 1s for up to 30s and sets it when available.

Fixes Protocol v2 identity characteristic serving on GATT server.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 20:16:46 -04:00
torlando-tech
b05295ff67 feat: add Identity characteristic for stable peer tracking across MAC rotations
Implements BLE Protocol v2 with Transport identity GATT characteristic to solve Android MAC address rotation issues. Adds IDENTITY_CHAR_UUID (00000004-...) that serves the 16-byte RNS.Transport.identity.hash, enabling reliable bidirectional mesh connectivity with Android devices whose BLE MAC addresses rotate every ~15 minutes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 19:39:33 -04:00
torlando-tech
195886c429 Merge pull request #10 from torlando-tech/fix/issue-3-bluez-experimental-mode
feat: comprehensive installer improvements and ARM CI testing
2025-10-29 12:19:11 -04:00
torlando-tech
fb1cd50cd1 ci: make tests conditional based on changed files
Add intelligent test selection to save CI minutes:
- Added 'detect-changes' job using paths-filter to determine what changed
- Unit/integration tests only run when Python source files change
- Installer tests only run when install.sh or test_installer.sh changes

This prevents running unnecessary tests:
- Changing install.sh won't trigger Python unit/integration tests
- Changing Python source won't trigger all 5 installer tests
- Workflow changes trigger installer tests (to verify CI changes)

Saves approximately 3-4 minutes of CI time per push when only one
category of files is changed.
2025-10-29 12:01:20 -04:00
torlando-tech
9c0b656c40 ci: add path filters to skip tests on docs-only changes
Add path-based triggers to workflow to run tests only when relevant files
change. This saves CI resources on documentation-only or README changes
while maintaining broad coverage for code and configuration changes.

Tests now run when these files change:
- install.sh (installer script)
- src/** (all source code)
- tests/** (test files)
- .github/workflows/** (CI configuration)
- *.txt, *.toml, *.cfg (dependency/config files)
- *.py (Python files in root)

Tests skip when only these change:
- README.md, docs, examples
- Other markdown or documentation files

This is especially beneficial for ARM tests which take 5-10 minutes due
to QEMU emulation overhead.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 12:00:41 -04:00
torlando-tech
fc3f5a544e ci: re-enable unit and integration tests
All installer test failures have been resolved. Re-enabling the full test
suite now that the install.sh script works correctly in Docker containers.

Installer tests are now passing on all platforms:
- debian:12 ✓
- debian:trixie ✓
- ubuntu:22.04 ✓
- ubuntu:24.04 ✓
- archlinux:latest ✓
2025-10-29 11:56:05 -04:00
torlando-tech
5f1a8bf84d fix: install system dependencies before Reticulum to provide libffi-dev
Move system dependencies installation (Step 1) before Reticulum installation
(Step 2) to ensure libffi-dev is available when RNS installs its dependency
cryptography, which requires cffi compilation on armhf (32-bit ARM).

Root cause: RNS dependency chain (RNS -> cryptography -> cffi) triggered
cffi source compilation during pip install rns, before libffi-dev was
installed in the old Step 2.

New order:
1. Basic prerequisites (python3, pip, git)
2. System dependencies (including conditional libffi-dev for armhf)
3. Reticulum installation
4. Python dependencies (bleak, bluezero)

The libffi-dev package remains conditional for armhf only - x86_64 and
arm64 have pre-built cffi wheels and don't need compilation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 11:37:08 -04:00
torlando-tech
c6b62e9e30 debug: add architecture detection logging for armhf troubleshooting
Add debug output to show:
- Detected architecture value from dpkg
- Whether armhf condition is matched
- Final package list being installed

This will help diagnose why libffi-dev isn't being installed on armhf
systems in the CI environment.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 11:22:54 -04:00
torlando-tech
9aeee07e69 refactor: make libffi-dev conditional for armhf (32-bit ARM) only
Only install libffi-dev on armhf (32-bit ARM) systems where cffi needs
to compile from source. x86_64 and arm64 have pre-built cffi wheels
available, so they don't need the development headers.

Changes:
- install.sh: Detect architecture and conditionally add libffi-dev for armhf
- test_installer.sh: Show libffi-dev in output only for armhf systems
- test.yml: Update ARM CI summary to reflect conditional dependency

This reduces unnecessary dependencies on x86_64 and arm64 systems while
maintaining full compatibility with 32-bit Raspberry Pi devices.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 11:02:51 -04:00
torlando-tech
2a1ab3fe27 fix: add libffi-dev dependency for ARM cffi compilation
Add libffi-dev to system dependencies for Debian/Ubuntu/Raspberry Pi OS
to provide FFI headers needed when cffi compiles from source on ARM
platforms. This fixes ARM 32-bit and 64-bit installation failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:42:16 -04:00
torlando-tech
7ab5d53352 ci: add Raspberry Pi OS ARM testing (32-bit and 64-bit)
Add QEMU-based ARM testing for both armhf (32-bit) and arm64 (64-bit)
architectures to validate installer on Raspberry Pi OS Lite. Tests run
only on PRs to main branch to conserve CI resources while ensuring
compatibility with ARM platforms before merge.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:34:39 -04:00
torlando-tech
2f2b59c86e Merge pull request #9 from torlando-tech/fix/issue-3-bluez-experimental-mode
Fix/issue 3 bluez experimental mode
2025-10-29 10:32:45 -04:00
torlando-tech
3e574af64e fix: make systemctl commands non-fatal in container environments
The systemctl commands (daemon-reload, restart bluetooth) were causing CI
failures in Docker containers where systemd is not running as PID 1.

Changes:
- Added '|| true' and '2>/dev/null' to systemctl commands to make them non-fatal
- Updated verification logic to detect container environments (/.dockerenv)
- Added appropriate messages for container environments
- Systemd override file is still created, but service restart is skipped

This allows the installer to complete successfully in containers while still
properly configuring BlueZ experimental mode on real systems with systemd.
2025-10-29 10:27:16 -04:00
torlando-tech
5f47e995bc fix: save repo root path in test_installer.sh for later cd
The test script was failing because it changed directories during execution
(to /tmp/test-config/interfaces) and then tried to use a relative path
to navigate back to the repo root, which failed.

Fix: Save the absolute path to repo root at the beginning and reuse it
when needed instead of calculating it relative to the current directory.

Fixes the error: 'cd: tests/..: No such file or directory'
2025-10-29 10:25:01 -04:00
torlando-tech
5feb5a9af1 ci: temporarily disable unit and integration tests
Disabling unit and integration test jobs to save GitHub Actions minutes
while troubleshooting installer test failures in Docker containers.

This reduces CI runtime from ~4-5 minutes to ~1 minute per run.

To re-enable: remove the 'if: false' lines from unit-tests and
integration-tests jobs in .github/workflows/test.yml
2025-10-29 10:18:45 -04:00
torlando-tech
bc839fba93 fix: skip setcap when running as root to avoid container issues
The setcap command was causing "Operation not permitted" errors when trying
to run Python in Docker containers after capabilities were applied. This is
because setcap can cause security restrictions that are incompatible with
container environments.

Root users don't need capabilities anyway - they already have full permissions
to access Bluetooth hardware. This change:
- Detects when running as root (EUID == 0)
- Skips the entire setcap process for root users
- Adds informative messages explaining why it's being skipped
- Simplifies the code by removing nested root checks

This allows the installer tests to pass in CI while still properly granting
capabilities for non-root users on real systems.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:13:55 -04:00
torlando-tech
53a38652f0 fix: make bluetoothctl commands non-fatal in container environments
The bluetoothctl command was causing CI failures with abort signal (exit 134)
in Docker containers where D-Bus is not running or Bluetooth hardware is not
available. This is expected behavior in container/CI environments.

Changes:
- Added `|| true` to bluetoothctl power on command to prevent script failure
- Added `2>/dev/null` to bluetoothctl show commands to suppress D-Bus errors
- Added clarifying comment about container/CI environment behavior

This allows the installer to complete successfully in CI while still attempting
to configure Bluetooth on real systems where the hardware is available.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:07:40 -04:00
torlando-tech
eebdeb3907 fix: add root detection for setcap and rfkill commands in CI
The installer was failing in CI Docker containers with "sudo: command
not found" because Docker containers run as root and don't have sudo
installed. Added root detection (checking $EUID) before setcap and
rfkill commands, following the same pattern used elsewhere in the script.

This allows the installer to work correctly in both scenarios:
- User systems: Uses sudo when needed (non-root users)
- Docker/CI containers: Runs commands directly as root (no sudo required)

Fixes the CI failure at install.sh:458 where setcap was unconditionally
using sudo, and prevents future failures at the rfkill command.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:02:51 -04:00
torlando-tech
9410630561 fix spacing for copy paste 2025-10-29 09:53:30 -04:00
torlando-tech
d46ee840b0 fix: add automatic rfkill unblocking to installer
Automatically detect and unblock soft-blocked Bluetooth adapters during
installation, preventing "Failed to set power on" errors on Raspberry Pi
and other systems where rfkill may block the adapter by default.

Changes:
- Add rfkill soft-block detection in install.sh Step 5B
- Automatically run 'sudo rfkill unblock bluetooth' when needed
- Verify unblock succeeded before attempting power-on
- Provide clear feedback during unblock process

Fixes the root cause of power-on failures when adapter is soft-blocked,
which was discovered during testing on Raspberry Pi Zero 2 W.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 00:23:33 -04:00
torlando-tech
a0f361a3f4 feat: add Bluetooth adapter power state handling and auto-power-on
Addresses the common "Not Powered" error where Bluetooth adapters are
powered off, preventing BLE operations. This issue is particularly common
on Raspberry Pi after boot or system updates.

Changes:

1. README.md - Added comprehensive troubleshooting section
   - New section: "Bluetooth adapter not powered"
   - Documented symptoms, causes, and 4 solution methods
   - Instructions for automatic power-on at boot
   - Cross-referenced from other sections

2. BLEInterface.py - Enhanced error handling in _discover_peers()
   - Detect "Not Powered" errors specifically
   - Show clear, actionable error messages instead of stack traces
   - Provide direct solution commands
   - Link to troubleshooting documentation
   - Gracefully handle error without crashing

3. install.sh - Automatic power state checking
   - New "Step 5B: Bluetooth Adapter Power State" section
   - Check if adapter is powered using `bluetoothctl show`
   - Automatically power on adapter if needed
   - Verify operation succeeded
   - Provide troubleshooting steps if power-on fails
   - Check for rfkill blocks

4. examples/bluetooth-power-on.service - New systemd service
   - Ensures Bluetooth is powered on at boot
   - Optional but recommended for production
   - Includes installation instructions in README

5. examples/config_example.toml - Added troubleshooting entry #7
   - Documents power state issue in config comments
   - Cross-references systemd service example
   - Notes that installer (v1.x+) handles this automatically

Impact:
- Users get clear guidance instead of cryptic stack traces
- Installer automatically fixes the issue during setup
- Reduces support burden for common power state errors
- Enables automatic recovery at boot via systemd service

Fixes: "Not Powered" / "No powered Bluetooth adapters found" errors
Tested on: Raspberry Pi Zero 2 W with Raspberry Pi OS Lite 64-bit

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 00:10:57 -04:00
torlando-tech
d0845d0409 Merge pull request #8 from torlando-tech/fix/issue-3-bluez-experimental-mode
Enable BlueZ experimental mode by default to fix BLE connection issues (fixes #3)
2025-10-29 00:09:05 -04:00
torlando-tech
1687e6fc8f Merge pull request #7 from torlando-tech/fix/issue-4-missing-dependencies
Fix #4: Use system packages to avoid compilation
2025-10-29 00:08:42 -04:00
torlando-tech
e7ee42ddc5 refactor: remove box border from config example for easier copy-paste
The fancy box border (┌─┐│└┘) around the configuration example made it
difficult to copy-paste into the config file. Users had to manually remove
the vertical line characters from each line.

Changes:
- Removed box border characters (┌, ─, ┐, │, └, ┘)
- Kept the same indentation and formatting
- Added "(copy-paste ready)" note to clarify it can be directly pasted
- Maintains visual structure with proper spacing

The configuration example is now directly copy-pasteable into the Reticulum
config file without any manual editing required.

Before:
   ┌─────────────────────────────────────────┐
   │ [[BLE Interface]]                       │
   │   type = BLEInterface                   │
   ...

After:
   [[BLE Interface]]
     type = BLEInterface
     enabled = yes
   ...

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 00:02:11 -04:00
torlando-tech
21c8082dff feat: automatically add ~/.local/bin to PATH when RNS installed there
When pip installs RNS to ~/.local/bin (user-local installation), the rnsd
command is not immediately available because ~/.local/bin is not in PATH.
This required users to manually add it to their .bashrc.

Changes:
- Automatically add ~/.local/bin to PATH in .bashrc after RNS user installation
- Check if .local/bin is already in .bashrc before adding (avoid duplicates)
- Show clear message about reloading shell: "source ~/.bashrc"
- Update final "Next steps" to conditionally show PATH setup
- Only show manual PATH instructions if auto-add didn't happen

This makes installation fully automated - users can run rnsd immediately
after reloading their shell, without manual PATH configuration.

User experience:
✓ Reticulum installed successfully (user installation)
ℹ rnsd location: /home/user/.local/bin/rnsd
ℹ Adding ~/.local/bin to PATH in ~/.bashrc...
✓ Added ~/.local/bin to PATH in ~/.bashrc
⚠ Reload your shell to use rnsd command:
  source ~/.bashrc
  # Or open a new terminal

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-28 23:58:41 -04:00
torlando-tech
7cd0244e5a fix: resolve Python symlinks and make Bluetooth permissions automatic
The setcap command fails on symlinks with "Invalid file 'setcap' for capability operation".
On many systems (including Raspberry Pi OS), /usr/bin/python3 is a symlink to the versioned
binary (e.g., /usr/bin/python3.13). The interactive prompt also required manual intervention
during installation.

Changes:
- Automatically grant Bluetooth permissions (removed y/N prompt)
- Detect if python3 is a symlink and resolve to actual binary using readlink -f
- Verify resolved path is valid before applying setcap
- Use getcap to verify capabilities were actually set
- Added --skip-bt-permissions flag for users who want to skip this step
- Improved error messages with specific troubleshooting steps
- Show detected and resolved Python paths for transparency

This makes installation fully automated while handling the symlink issue that was
causing setcap failures on Raspberry Pi OS and other Debian-based systems.

Fixes: "Invalid file 'setcap' for capability operation" error
Tested on: Raspberry Pi Zero 2 W with Raspberry Pi OS Lite 64-bit (Debian Trixie)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-28 23:51:15 -04:00