jlpoole/ble-reticulum
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

fix: restore RNS.Interfaces.Interface import for base class

Browse source 
The sed replacement was too aggressive - it replaced the import for
the base Interface class from the Reticulum package itself.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
torlando-tech 2025-12-29 23:38:21 -05:00
commit ca88c6b4c9
9 changed files with 1212 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

10
.claude/commands/fix-ci.md Normal file
View file

@ -0,0 +1,10 @@
Check the GitHub Actions CI status and fix any failures:
1. Use `gh run list --limit 1` to get the latest run
2. Use `gh run view --log` to see what failed
3. Analyze the error logs
4. Fix the issues in the code
5. Run tests locally to verify
6. Commit and push the fix
7. Monitor the new CI run with `gh run watch`
8. If it fails again, iterate until it passes

8
.claude/commands/fix-issue.md Normal file
View file

@ -0,0 +1,8 @@
Please analyze and fix the GitHub issue: $ARGUMENTS. Follow these steps:
1. Use `gh issue view` to get the issue details
2. Understand the problem described in the issue
3. Search the codebase for relevant files
4. Implement the necessary changes to fix the issue
5. Run tests to verify the fix works
6. Create a PR with `gh pr create` with a clear description
7. Link the PR to the issue

148
COLUMBA_REFACTORING_GUIDE.md Normal file
View file

@ -0,0 +1,148 @@
# Refactoring Columba's BLE Layer to a Driver-Based Architecture
## 1. Goal
This guide outlines the process of refactoring the existing BLE implementation in the Columba Android project to align with the new driver-based architecture of the `ble-reticulum` project.
The goal is to:
- Reuse the battle-tested `BLEInterface.py` from `ble-reticulum` as the main Reticulum logic for BLE in Columba.
- Create a new Android-specific BLE driver in Python (`AndroidBLEDriver.py`) that implements the `BLEDriverInterface`.
- Bridge this new Python driver to a dedicated Kotlin class (`KotlinBLEBridge.kt`) that handles all native Android BLE operations.
- Isolate the Kotlin BLE logic from the rest of the Columba UI application, with the `KotlinBLEBridge` acting as the sole entry point for the Python layer.
This will result in a more modular, maintainable, and testable system, and will allow Columba to easily stay up-to-date with the latest improvements in the `ble-reticulum` project.
## 2. Current State Analysis
Based on the file structure of the Columba project, the current BLE implementation is a monolithic Kotlin implementation with a Python bridge.
- **Kotlin:** The core BLE logic is in the `com.lxmf.messenger.reticulum.ble` package, with classes like `BleConnectionManager`, `BleGattClient`, `BleGattServer`, `BleScanner`, and `BleAdvertiser`.
- **Python:** The `rn_ble_interface.py` script acts as the Chaquopy bridge, importing and using the Kotlin classes to create a Reticulum interface.
This architecture is tightly coupled, making it difficult to update and maintain. The new driver-based architecture will address these issues.
## 3. Proposed Architecture
The new architecture will consist of three main components:
1. **`BLEInterface.py`:** The high-level, platform-agnostic Reticulum interface logic from the `ble-reticulum` project.
2. **`AndroidBLEDriver.py`:** A new Python class that implements the `BLEDriverInterface` and acts as a bridge to the Kotlin layer.
3. **`KotlinBLEBridge.kt`:** A new, isolated Kotlin class that exposes a clean API for the `AndroidBLEDriver.py` to interact with the native Android BLE stack.
This architecture will allow us to reuse the `BLEInterface.py` and only implement the platform-specific BLE operations in the `AndroidBLEDriver.py` and `KotlinBLEBridge.kt`.
## 4. Step-by-Step Refactoring Guide
### Step 1: Create the `KotlinBLEBridge.kt`
Create a new Kotlin class, `KotlinBLEBridge.kt`, in the `com.lxmf.messenger.reticulum.ble.service` package. This class will be the single entry point for all BLE operations from the Python layer. It should be a singleton and should not have any dependencies on the Columba UI.
The `KotlinBLEBridge.kt` class should expose methods that correspond to the `BLEDriverInterface` in Python. For example:
```kotlin
class KotlinBLEBridge(private val context: Context) {
fun start(serviceUuid: String, rxCharUuid: String, txCharUuid: String, identityCharUuid: String) {
// Initialize the BLE stack
}
fun stop() {
// Stop all BLE activity
}
fun setIdentity(identityBytes: ByteArray) {
// Set the identity for the GATT server
}
fun startScanning() {
// Start scanning for devices
}
fun stopScanning() {
// Stop scanning
}
fun startAdvertising(deviceName: String) {
// Start advertising
}
fun stopAdvertising() {
// Stop advertising
}
fun connect(address: String) {
// Connect to a device
}
fun disconnect(address: String) {
// Disconnect from a device
}
fun send(address: String, data: ByteArray) {
// Send data to a device
}
// ... other methods as needed
}
```
This class will also be responsible for invoking the callbacks on the Python driver. You can use a listener interface to achieve this.
### Step 2: Create the `AndroidBLEDriver.py`
Create a new Python file, `AndroidBLEDriver.py`, in the `columba/app/src/main/python` directory. This class will implement the `BLEDriverInterface` and will use Chaquopy to call the methods of the `KotlinBLEBridge`.
```python
from com.chaquo.python import Python
from RNS.Interfaces.bluetooth_driver import BLEDriverInterface, BLEDevice, DriverState
class AndroidBLEDriver(BLEDriverInterface):
def __init__(self):
self.kotlin_ble_bridge = Python.getPlatform().getApplication().getKotlinBLEBridge()
# Set up callbacks from Kotlin to Python
# ...
def start(self, service_uuid, rx_char_uuid, tx_char_uuid, identity_char_uuid):
self.kotlin_ble_bridge.start(service_uuid, rx_char_uuid, tx_char_uuid, identity_char_uuid)
def stop(self):
self.kotlin_ble_bridge.stop()
# ... implement all other methods of the BLEDriverInterface
```
### Step 3: Refactor `rn_ble_interface.py`
Modify the existing `rn_ble_interface.py` to use the new `BLEInterface` and `AndroidBLEDriver`.
```python
# rn_ble_interface.py
from RNS.Interfaces.BLEInterface import BLEInterface
from AndroidBLEDriver import AndroidBLEDriver
# ... other imports
class RNBLEInterface(BLEInterface):
def __init__(self, owner, config):
driver = AndroidBLEDriver()
super().__init__(owner, config, driver=driver)
# ... rest of the file
```
### Step 4: Replace the old BLE implementation
Once the new driver-based architecture is in place, you can start removing the old BLE implementation in the Columba project. This includes classes like `BleConnectionManager`, `BleGattClient`, `BleGattServer`, etc. The new `KotlinBLEBridge` should encapsulate all the necessary BLE logic.
## 5. Testing
Thorough testing is crucial for this refactoring.
- **Unit Tests:** Write unit tests for the `KotlinBLEBridge` to ensure that it correctly interacts with the Android BLE stack.
- **Integration Tests:** Write integration tests that verify the communication between the `AndroidBLEDriver.py` and the `KotlinBLEBridge.kt`.
- **End-to-End Tests:** Run the full Columba application and test the BLE functionality to ensure that everything works as expected.
By following this guide, you can refactor the Columba BLE layer to a more modern, modular, and maintainable architecture, while at the same time reusing the battle-tested `BLEInterface.py` from the `ble-reticulum` project.

476
comprehensive_refactor.py Normal file
View file

@ -0,0 +1,476 @@
#!/usr/bin/env python3
"""
Comprehensive refactoring script for BLEInterface.py to use driver abstraction.
This script:
1. Removes platform-specific imports (bleak, bluezero, dbus_fast, monkey patch)
2. Adds driver abstraction imports
3. Refactors __init__ to create and configure driver
4. Removes async methods moved to driver
5. Adds driver callback implementations
6. Updates BLE operations to use driver calls
"""
import re
def read_file(path):
with open(path, 'r') as f:
return f.read()
def write_file(path, content):
with open(path, 'w') as f:
f.write(content)
def remove_imports_and_add_driver_imports(content):
"""Remove bleak/bluezero/monkey patch, add driver imports."""
# Find the section to replace (from "# Check for bleak" to end of monkey patch)
pattern = r'# Check for bleak dependency.*?(?=class DiscoveredPeer)'
replacement = '''# Import driver abstraction
try:
from bluetooth_driver import BLEDriverInterface, BLEDevice
except ImportError:
try:
from RNS.Interfaces.bluetooth_driver import BLEDriverInterface, BLEDevice
except ImportError:
# Fallback to root directory
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../")))
from bluetooth_driver import BLEDriverInterface, BLEDevice
# Import platform-specific driver
try:
from linux_bluetooth_driver import LinuxBluetoothDriver
except ImportError:
try:
from RNS.Interfaces.linux_bluetooth_driver import LinuxBluetoothDriver
except ImportError:
# Fallback to root directory
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../")))
from linux_bluetooth_driver import LinuxBluetoothDriver
HAS_DRIVER = True
'''
content = re.sub(pattern, replacement, content, flags=re.DOTALL)
return content
def remove_method(content, method_name):
"""Remove a method definition entirely."""
# Pattern to match method definition and its body
# Match from "def method_name" or "async def method_name" until the next method/class definition
pattern = rf'^( )(async )?def {method_name}\(.*?\n((?:(?!\1(?:def|async def|class)\b).*\n)*)'
content = re.sub(pattern, '', content, flags=re.MULTILINE)
return content
def refactor_init_method(content):
"""Refactor __init__ to use driver abstraction."""
# Replace HAS_BLEAK check with HAS_DRIVER
content = content.replace(
'if not HAS_BLEAK:\n raise ImportError(\n "BLEInterface requires the \'bleak\' library. "\n "Install with: pip install bleak==1.1.1"\n )',
'if not HAS_DRIVER:\n raise ImportError(\n "BLEInterface requires the driver abstraction. "\n "Ensure bluetooth_driver.py and linux_bluetooth_driver.py are available."\n )'
)
# Remove GATT server creation section (lines starting with "# GATT server for peripheral mode" until "# Fragmentation")
pattern = r' # GATT server for peripheral mode.*?(?= # Fragmentation)'
content = re.sub(pattern, '', content, flags=re.DOTALL)
# Remove async loop setup (lines starting with "# Async event loop" until "# Discovery state")
pattern = r' # Async event loop.*?(?= # Discovery state)'
content = re.sub(pattern, '', content, flags=re.DOTALL)
# Remove BlueZ version detection
content = content.replace(
' # BlueZ version and capabilities (for LE-specific connection support)\n self.bluez_version = self._detect_bluez_version()\n self.has_connect_device = False # Set to True if ConnectDevice() available\n',
''
)
# Add driver creation after fragmentation section
driver_init = '''
# Initialize BLE driver
self.driver = LinuxBluetoothDriver(
discovery_interval=self.discovery_interval,
connection_timeout=self.connection_timeout,
min_rssi=self.min_rssi,
service_discovery_delay=self.service_discovery_delay,
max_peers=self.max_peers,
adapter_index=0 # TODO: Make configurable
)
# Set driver callbacks
self.driver.on_device_discovered = self._device_discovered_callback
self.driver.on_device_connected = self._device_connected_callback
self.driver.on_mtu_negotiated = self._mtu_negotiated_callback
self.driver.on_data_received = self._data_received_callback
self.driver.on_device_disconnected = self._device_disconnected_callback
self.driver.on_error = self._error_callback
# Set driver power mode
self.driver.set_power_mode(self.power_mode)
'''
# Insert after "# Discovery state with prioritization" line
content = content.replace(
' # Discovery state with prioritization\n',
' # Discovery state with prioritization\n' + driver_init + '\n'
)
return content
def add_driver_callbacks(content):
"""Add driver callback implementations after _periodic_cleanup method."""
callbacks = '''
def _device_discovered_callback(self, device: BLEDevice):
"""
Driver callback: Handle discovered BLE device.
This callback is invoked by the driver when a device is discovered during scanning.
We use peer scoring and connection logic to decide whether to connect.
"""
# Update or create discovered peer entry
if device.address not in self.discovered_peers:
self.discovered_peers[device.address] = DiscoveredPeer(
address=device.address,
name=device.name,
rssi=device.rssi
)
else:
self.discovered_peers[device.address].update_rssi(device.rssi)
# Prune discovery cache if needed (HIGH #4)
if len(self.discovered_peers) > self.max_discovered_peers:
# Remove oldest entries by last_seen timestamp
sorted_peers = sorted(
self.discovered_peers.items(),
key=lambda x: x[1].last_seen
)
to_remove = sorted_peers[:-self.max_discovered_peers]
for addr, _ in to_remove:
del self.discovered_peers[addr]
# Decide whether to connect based on peer scoring
peers_to_connect = self._select_peers_to_connect()
if device.address in [p.address for p in peers_to_connect]:
# Initiate connection via driver
try:
self.driver.connect(device.address)
except Exception as e:
RNS.log(f"{self} failed to initiate connection to {device.name}: {e}", RNS.LOG_ERROR)
def _device_connected_callback(self, address: str):
"""
Driver callback: Handle successful device connection.
Called when driver has established a connection. We read the identity
characteristic and prepare to receive data.
"""
RNS.log(f"{self} connected to {address}, reading identity...", RNS.LOG_INFO)
# Read identity characteristic
try:
identity_bytes = self.driver.read_characteristic(
address,
BLEInterface.CHARACTERISTIC_IDENTITY_UUID
)
if identity_bytes and len(identity_bytes) == 16:
peer_identity = bytes(identity_bytes)
identity_hash = self._compute_identity_hash(peer_identity)
# Store identity mappings
self.address_to_identity[address] = peer_identity
self.identity_to_address[identity_hash] = address
RNS.log(f"{self} received peer identity from {address}: {identity_hash}", RNS.LOG_INFO)
# Record successful connection
self._record_connection_success(address)
else:
RNS.log(f"{self} invalid identity from {address}, disconnecting", RNS.LOG_WARNING)
self.driver.disconnect(address)
self._record_connection_failure(address)
except Exception as e:
RNS.log(f"{self} failed to read identity from {address}: {e}", RNS.LOG_ERROR)
self.driver.disconnect(address)
self._record_connection_failure(address)
def _mtu_negotiated_callback(self, address: str, mtu: int):
"""
Driver callback: Handle MTU negotiation completion.
Creates or updates the fragmenter for this peer with the negotiated MTU.
"""
RNS.log(f"{self} MTU negotiated with {address}: {mtu} bytes", RNS.LOG_INFO)
# Get peer identity
peer_identity = self.address_to_identity.get(address)
if not peer_identity:
RNS.log(f"{self} no identity for {address}, cannot create fragmenter", RNS.LOG_WARNING)
return
# Create or update fragmenter
frag_key = self._get_fragmenter_key(peer_identity, address)
with self.frag_lock:
# Create fragmenter with MTU
self.fragmenters[frag_key] = BLEFragmenter(mtu=mtu)
# Create reassembler if not exists
if frag_key not in self.reassemblers:
self.reassemblers[frag_key] = BLEReassembler()
# Spawn peer interface if not exists
identity_hash = self._compute_identity_hash(peer_identity)
if identity_hash not in self.spawned_interfaces:
# Get peer name from discovered peers
peer_name = None
if address in self.discovered_peers:
peer_name = self.discovered_peers[address].name
else:
peer_name = f"BLE-{address[-8:]}"
# Determine connection type based on MAC sorting
connection_type = "central"
if self.driver.get_local_address():
local_mac = self.driver.get_local_address().lower()
peer_mac = address.lower()
if local_mac > peer_mac:
connection_type = "peripheral"
self._spawn_peer_interface(
address=address,
name=peer_name,
peer_identity=peer_identity,
mtu=mtu,
connection_type=connection_type
)
def _data_received_callback(self, address: str, data: bytes):
"""
Driver callback: Handle received data from peer.
Passes data to reassembly and routing logic.
"""
self._handle_ble_data(address, data)
def _device_disconnected_callback(self, address: str):
"""
Driver callback: Handle device disconnection.
Cleans up peer state, interfaces, and fragmentation buffers.
"""
RNS.log(f"{self} disconnected from {address}", RNS.LOG_INFO)
# Clean up peer connection state
with self.peer_lock:
if address in self.peers:
del self.peers[address]
# Detach interface
peer_identity = self.address_to_identity.get(address)
if peer_identity:
identity_hash = self._compute_identity_hash(peer_identity)
if identity_hash in self.spawned_interfaces:
peer_if = self.spawned_interfaces[identity_hash]
peer_if.detach()
del self.spawned_interfaces[identity_hash]
RNS.log(f"{self} detached interface for {address}", RNS.LOG_DEBUG)
# Clean up fragmenter/reassembler
if peer_identity:
frag_key = self._get_fragmenter_key(peer_identity, address)
with self.frag_lock:
if frag_key in self.fragmenters:
del self.fragmenters[frag_key]
if frag_key in self.reassemblers:
del self.reassemblers[frag_key]
def _error_callback(self, severity: str, message: str, exc: Exception = None):
"""
Driver callback: Handle driver errors.
Logs errors with appropriate severity level.
"""
if severity == "critical":
log_level = RNS.LOG_CRITICAL
elif severity == "error":
log_level = RNS.LOG_ERROR
elif severity == "warning":
log_level = RNS.LOG_WARNING
else:
log_level = RNS.LOG_DEBUG
if exc:
RNS.log(f"{self} driver {severity}: {message} - {type(exc).__name__}: {exc}", log_level)
else:
RNS.log(f"{self} driver {severity}: {message}", log_level)
'''
# Insert callbacks after _periodic_cleanup method
# Find the end of _periodic_cleanup (next method definition)
pattern = r'( async def _periodic_cleanup\(self\):.*?(?=\n def ))'
match = re.search(pattern, content, re.DOTALL)
if match:
insert_pos = match.end()
content = content[:insert_pos] + '\n' + callbacks + content[insert_pos:]
return content
def refactor_start_method(content):
"""Refactor start() method to use driver."""
# Replace loop thread creation with driver start
old_start = r' # Create and start async event loop in separate thread\s+self\.loop_thread = threading\.Thread\(target=self\._run_async_loop, daemon=True\)\s+self\.loop_thread\.start\(\)\s+# Wait for loop to initialize.*?return'
new_start = ''' # Start the BLE driver
try:
self.driver.start(
service_uuid=self.service_uuid,
rx_char_uuid=BLEInterface.CHARACTERISTIC_RX_UUID,
tx_char_uuid=BLEInterface.CHARACTERISTIC_TX_UUID,
identity_char_uuid=BLEInterface.CHARACTERISTIC_IDENTITY_UUID
)
RNS.log(f"{self} driver started successfully", RNS.LOG_INFO)
except Exception as e:
RNS.log(f"{self} failed to start driver: {e}", RNS.LOG_ERROR)
return'''
content = re.sub(old_start, new_start, content, flags=re.DOTALL)
# Remove discovery and cleanup task scheduling
content = content.replace(
' # Schedule discovery to start (if central mode enabled)\n if self.enable_central:\n asyncio.run_coroutine_threadsafe(self._start_discovery(), self.loop)\n else:\n RNS.log(f"{self} central mode disabled, skipping peer discovery", RNS.LOG_INFO)\n\n # Start periodic cleanup task (CRITICAL #2: prevent unbounded reassembly buffer growth)\n asyncio.run_coroutine_threadsafe(self._periodic_cleanup(), self.loop)\n',
''
)
return content
def refactor_final_init(content):
"""Refactor final_init() to set identity on driver and start advertising."""
old_final_init = r' def final_init\(self\):.*?(?=\n def _start_gatt_when_identity_ready)'
new_final_init = ''' def final_init(self):
"""
Interface lifecycle hook called AFTER interface is added to Transport.interfaces
but BEFORE Transport.start() loads Transport.identity.
Use this to start a background thread that waits for Transport.identity to be
loaded, then sets it on the driver and starts advertising.
"""
if self.enable_peripheral:
RNS.log(f"{self} Launching driver advertising startup thread (will wait for Transport.identity)", RNS.LOG_DEBUG)
startup_thread = threading.Thread(target=self._start_advertising_when_identity_ready, daemon=True, name="BLE-Advertising-Startup")
startup_thread.start()
def _start_advertising_when_identity_ready(self):
"""
Background thread that waits for Transport.identity, sets it on driver,
then starts advertising. Times out after 60 seconds if identity doesn't load.
"""
import RNS.Transport as Transport
attempt = 0
start_time = time.time()
timeout = 60.0 # 60 second timeout
RNS.log(f"{self} Waiting for Transport.identity to be loaded...", RNS.LOG_DEBUG)
# Poll until Transport.identity is available (with 60s timeout)
while time.time() - start_time < timeout:
attempt += 1
try:
if hasattr(Transport, 'identity') and Transport.identity:
identity_hash = Transport.identity.hash
if identity_hash and len(identity_hash) == 16:
elapsed = time.time() - start_time
RNS.log(f"{self} Transport.identity available after {elapsed:.1f}s", RNS.LOG_INFO)
# Generate identity-based device name if not configured
if self.device_name is None:
identity_str = identity_hash.hex() # Full 16 bytes as 32 hex chars
self.device_name = f"RNS-{identity_str}"
RNS.log(f"{self} Auto-generated identity-based device name: {self.device_name}", RNS.LOG_INFO)
# Set identity on driver
self.driver.set_identity(identity_hash)
# Start advertising
try:
self.driver.start_advertising(self.device_name, identity_hash)
RNS.log(f"{self} Started advertising as {self.device_name}", RNS.LOG_INFO)
except Exception as e:
RNS.log(f"{self} Failed to start advertising: {e}", RNS.LOG_ERROR)
return
except Exception as e:
RNS.log(f"{self} Error waiting for identity: {e}", RNS.LOG_DEBUG)
time.sleep(0.5)
RNS.log(f"{self} Timeout waiting for Transport.identity after {timeout}s", RNS.LOG_ERROR)
'''
content = re.sub(old_final_init, new_final_init, content, flags=re.DOTALL)
return content
def main():
input_file = 'src/RNS/Interfaces/BLEInterface.py'
print("Reading file...")
content = read_file(input_file)
print("Step 1: Removing imports and adding driver imports...")
content = remove_imports_and_add_driver_imports(content)
print("Step 2: Removing async methods moved to driver...")
methods_to_remove = [
'_run_async_loop',
'_detect_bluez_version',
'_log_bluez_config',
'_connect_via_dbus_le',
'_get_local_adapter_address',
'_start_discovery',
'_start_server',
'_discover_peers',
'_connect_to_peer'
]
for method in methods_to_remove:
print(f" Removing {method}...")
content = remove_method(content, method)
print("Step 3: Refactoring __init__ method...")
content = refactor_init_method(content)
print("Step 4: Refactoring start() method...")
content = refactor_start_method(content)
print("Step 5: Refactoring final_init() method...")
content = refactor_final_init(content)
print("Step 6: Adding driver callbacks...")
content = add_driver_callbacks(content)
print("Writing refactored file...")
write_file(input_file, content)
print("Done! Refactoring complete.")
print("\nManual review needed for:")
print(" - BLEPeerInterface._send_via_central() and _send_via_peripheral()")
print(" - Any remaining bleak/bluezero references")
print(" - Local address retrieval (now driver.get_local_address())")
if __name__ == '__main__':
main()

109
perform_refactor.py Normal file
View file

@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""
Comprehensively refactor BLEInterface.py to use driver abstraction.
"""
def main():
input_file = '/home/tyler/repos/public/ble-reticulum/src/RNS/Interfaces/BLEInterface.py'
output_file = input_file # Overwrite
with open(input_file, 'r') as f:
lines = f.readlines()
new_lines = []
skip_until = -1 # Line number to skip until
in_method_to_remove = False
method_indent = 0
i = 0
while i < len(lines):
line = lines[i]
line_no = i + 1
# Skip lines we've marked for deletion
if i < skip_until:
i += 1
continue
# Remove bleak/bluezero imports (lines 99-172 approximately)
if line_no == 99 and '# Check for bleak dependency' in line:
# Skip until we find the end of monkey patch section (line 172)
while i < len(lines) and not (i > 172 or 'class DiscoveredPeer' in lines[i]):
i += 1
# Add driver imports instead
new_lines.append('# Import driver abstraction\n')
new_lines.append('try:\n')
new_lines.append(' from bluetooth_driver import BLEDriverInterface, BLEDevice\n')
new_lines.append('except ImportError:\n')
new_lines.append(' try:\n')
new_lines.append(' from RNS.Interfaces.bluetooth_driver import BLEDriverInterface, BLEDevice\n')
new_lines.append(' except ImportError:\n')
new_lines.append(' # Fallback to root directory\n')
new_lines.append(' import sys\n')
new_lines.append(' import os\n')
new_lines.append(' sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../")))\n')
new_lines.append(' from bluetooth_driver import BLEDriverInterface, BLEDevice\n')
new_lines.append('\n')
new_lines.append('# Import platform-specific driver\n')
new_lines.append('try:\n')
new_lines.append(' from linux_bluetooth_driver import LinuxBluetoothDriver\n')
new_lines.append('except ImportError:\n')
new_lines.append(' try:\n')
new_lines.append(' from RNS.Interfaces.linux_bluetooth_driver import LinuxBluetoothDriver\n')
new_lines.append(' except ImportError:\n')
new_lines.append(' # Fallback to root directory\n')
new_lines.append(' import sys\n')
new_lines.append(' import os\n')
new_lines.append(' sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../")))\n')
new_lines.append(' from linux_bluetooth_driver import LinuxBluetoothDriver\n')
new_lines.append('\n')
new_lines.append('HAS_DRIVER = True\n')
new_lines.append('\n')
continue
# Detect methods to remove
methods_to_remove = [
'_run_async_loop',
'_detect_bluez_version',
'_log_bluez_config',
'_connect_via_dbus_le',
'_get_local_adapter_address',
'_start_discovery',
'_start_server',
'_discover_peers',
'_connect_to_peer'
]
# Check if we're entering a method to remove
if any(f'def {method}' in line for method in methods_to_remove):
# Get the indent level of this method
method_indent = len(line) - len(line.lstrip())
in_method_to_remove = True
RNS.log(f"Removing method at line {line_no}: {line.strip()[:50]}")
i += 1
continue
# If we're in a method to remove, skip until we find the next method or class
if in_method_to_remove:
current_indent = len(line) - len(line.lstrip())
# If we find a line at the same or less indent (and it's not blank), we've exited the method
if line.strip() and current_indent <= method_indent:
in_method_to_remove = False
# Don't skip this line, process it normally
else:
i += 1
continue
# Add the line
new_lines.append(line)
i += 1
# Write output
with open(output_file, 'w') as f:
f.writelines(new_lines)
print(f"Refactored {len(lines)} lines to {len(new_lines)} lines")
print(f"Removed {len(lines) - len(new_lines)} lines")
if __name__ == '__main__':
main()

105
refactor_ble_interface.py Normal file
View file

@ -0,0 +1,105 @@
#!/usr/bin/env python3
"""
Script to refactor BLEInterface.py to use the driver abstraction.
This script performs automated transformations to remove platform-specific
code and replace it with driver abstraction calls.
"""
import re
def read_file(path):
with open(path, 'r') as f:
return f.read()
def write_file(path, content):
with open(path, 'w') as f:
f.write(content)
def refactor_imports(content):
"""Remove platform-specific imports and add driver imports."""
# Remove bleak imports
content = re.sub(r'# Check for bleak dependency.*?HAS_BLEAK = False\n',
'', content, flags=re.DOTALL)
# Remove monkey patch code (lines 107-172 approximately)
content = re.sub(r'# ={70,}\n# Monkey patch.*?RNS\.log\(f"Failed to apply.*?\n',
'', content, flags=re.DOTALL)
# Add driver imports after BLEFragmentation imports
driver_imports = '''
# Import driver abstraction
try:
from bluetooth_driver import BLEDriverInterface, BLEDevice
except ImportError:
try:
from RNS.Interfaces.bluetooth_driver import BLEDriverInterface, BLEDevice
except ImportError:
# Fallback to root directory
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../../')))
from bluetooth_driver import BLEDriverInterface, BLEDevice
# Import platform-specific driver
try:
from linux_bluetooth_driver import LinuxBluetoothDriver
except ImportError:
try:
from RNS.Interfaces.linux_bluetooth_driver import LinuxBluetoothDriver
except ImportError:
# Fallback to root directory
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../../')))
from linux_bluetooth_driver import LinuxBluetoothDriver
HAS_DRIVER = True
'''
# Find BLEGATTServer import section and add driver imports after
content = re.sub(
r'(except ImportError:\s+HAS_GATT_SERVER = False)\n',
r'\1\n' + driver_imports + '\n',
content
)
return content
def refactor_init(content):
"""Refactor __init__ method to use driver."""
# This is complex, will need manual editing
# For now, just remove the dependency check for bleak
content = re.sub(
r' # Check dependencies\s+if not HAS_BLEAK:.*?pip install bleak==1\.1\.1"\s+\)',
' # Check dependencies\n if not HAS_DRIVER:\n raise ImportError(\n "BLEInterface requires the driver abstraction. "\n "Ensure bluetooth_driver.py and linux_bluetooth_driver.py are available."\n )',
content,
flags=re.DOTALL
)
return content
def main():
input_path = '/home/tyler/repos/public/ble-reticulum/src/RNS/Interfaces/BLEInterface.py'
output_path = input_path # Overwrite in place
print(f"Reading {input_path}...")
content = read_file(input_path)
print("Refactoring imports...")
content = refactor_imports(content)
print("Refactoring __init__...")
content = refactor_init(content)
print(f"Writing {output_path}...")
write_file(output_path, content)
print("Done! Manual edits still required for:")
print(" - __init__ method (driver creation, callbacks)")
print(" - Remove async methods (_discover_peers, _connect_to_peer, etc.)")
print(" - Replace BLE operations with driver calls")
print(" - Add driver callback implementations")
if __name__ == '__main__':
main()

43
refactor_helper.py Normal file
View file

@ -0,0 +1,43 @@
#!/usr/bin/env python3
"""
Helper script to identify sections of BLEInterface.py that need refactoring.
"""
with open('/home/tyler/repos/public/ble-reticulum/src/RNS/Interfaces/BLEInterface.py', 'r') as f:
lines = f.readlines()
# Find methods to remove
methods_to_remove = [
'_run_async_loop',
'_detect_bluez_version',
'_log_bluez_config',
'_connect_via_dbus_le',
'_get_local_adapter_address',
'_start_discovery',
'_start_server',
'_discover_peers',
'_connect_to_peer'
]
print("Methods to remove:")
for method in methods_to_remove:
for i, line in enumerate(lines):
if f'def {method}' in line or f'async def {method}' in line:
print(f" Line {i+1}: {line.strip()}")
break
# Find key sections
print("\nKey sections:")
for i, line in enumerate(lines):
if 'class DiscoveredPeer' in line:
print(f" DiscoveredPeer class: line {i+1}")
elif 'class BLEInterface' in line:
print(f" BLEInterface class: line {i+1}")
elif 'class BLEPeerInterface' in line:
print(f" BLEPeerInterface class: line {i+1}")
elif line.strip().startswith('def __init__(self, owner, configuration)'):
print(f" BLEInterface.__init__: line {i+1}")
elif '_score_peer' in line and 'def' in line:
print(f" _score_peer: line {i+1}")
elif '_handle_ble_data' in line and 'def' in line:
print(f" _handle_ble_data: line {i+1}")

310
refactor_pass2.py Normal file
View file

@ -0,0 +1,310 @@
#!/usr/bin/env python3
"""
Second pass refactoring: Replace remaining BLE operations with driver calls.
"""
import re
def read_file(path):
with open(path, 'r') as f:
return f.read()
def write_file(path, content):
with open(path, 'w') as f:
f.write(content)
def refactor_detach_method(content):
"""Replace async operations in detach() with driver.stop()."""
old_detach = r''' def detach\(self\):
"""Detach and shutdown the interface\."""
RNS\.log\(f"\{self\} detaching interface", RNS\.LOG_INFO\)
self\.online = False
# MEDIUM #4: Graceful shutdown - wait for operations to complete before stopping event loop
# Stop GATT server gracefully
if self\.gatt_server:
try:
future = asyncio\.run_coroutine_threadsafe\(self\.gatt_server\.stop\(\), self\.loop\)
future\.result\(timeout=5\.0\) # Wait for graceful shutdown
RNS\.log\(f"\{self\} GATT server stopped", RNS\.LOG_DEBUG\)
except Exception as e:
RNS\.log\(f"\{self\} error stopping GATT server: \{e\}", RNS\.LOG_ERROR\)
# Disconnect all peers gracefully
disconnect_futures = \[\]
with self\.peer_lock:
for address, \(client, last_seen, mtu\) in list\(self\.peers\.items\(\)\):
try:
future = asyncio\.run_coroutine_threadsafe\(client\.disconnect\(\), self\.loop\)
disconnect_futures\.append\(\(address, future\)\)
except Exception as e:
RNS\.log\(f"\{self\} error scheduling disconnect for \{address\}: \{e\}", RNS\.LOG_ERROR\)
self\.peers\.clear\(\)
# Wait for all disconnections \(with timeout\)
for address, future in disconnect_futures:
try:
future\.result\(timeout=2\.0\)
RNS\.log\(f"\{self\} disconnected from \{address\}", RNS\.LOG_DEBUG\)
except Exception as e:
RNS\.log\(f"\{self\} disconnect timeout for \{address\}: \{e\}", RNS\.LOG_WARNING\)
# Detach spawned interfaces
for peer_if in list\(self\.spawned_interfaces\.values\(\)\):
peer_if\.detach\(\)
self\.spawned_interfaces\.clear\(\)
# Clear fragmentation state
with self\.frag_lock:
self\.fragmenters\.clear\(\)
self\.reassemblers\.clear\(\)
# NOW safe to stop event loop \(all operations completed\)
if self\.loop:
self\.loop\.call_soon_threadsafe\(self\.loop\.stop\)
# Give it a moment to actually stop
time\.sleep\(0\.1\)
RNS\.log\(f"\{self\} detached", RNS\.LOG_INFO\)'''
new_detach = ''' def detach(self):
"""Detach and shutdown the interface."""
RNS.log(f"{self} detaching interface", RNS.LOG_INFO)
self.online = False
# Detach spawned interfaces
for peer_if in list(self.spawned_interfaces.values()):
peer_if.detach()
self.spawned_interfaces.clear()
# Clear fragmentation state
with self.frag_lock:
self.fragmenters.clear()
self.reassemblers.clear()
# Stop the driver (handles graceful disconnection and cleanup)
try:
self.driver.stop()
RNS.log(f"{self} driver stopped", RNS.LOG_DEBUG)
except Exception as e:
RNS.log(f"{self} error stopping driver: {e}", RNS.LOG_ERROR)
RNS.log(f"{self} detached", RNS.LOG_INFO)'''
content = re.sub(old_detach, new_detach, content)
return content
def refactor_send_methods(content):
"""Replace asyncio operations in _send_via_central and _send_via_peripheral with driver.send()."""
# Replace _send_via_peripheral
old_peripheral = r''' def _send_via_peripheral\(self, fragments\):
"""
Send fragments via GATT server notifications\.
Args:
fragments: List of fragment bytes to send
Returns:
bool: True if all fragments sent successfully, False otherwise
"""
if not self\.parent_interface\.gatt_server:
RNS\.log\(f"No GATT server available for \{self\.peer_name\}", RNS\.LOG_ERROR\)
return False
for i, fragment in enumerate\(fragments\):
try:
# Schedule the async notification in the parent's event loop
future = asyncio\.run_coroutine_threadsafe\(
self\.parent_interface\.gatt_server\.send_notification\(fragment, self\.peer_address\),
self\.parent_interface\.loop
\)
# Wait for completion \(with timeout\)
future\.result\(timeout=2\.0\)
self\.txb \+= len\(fragment\)
self\.parent_interface\.txb \+= len\(fragment\)
except Exception as e:
RNS\.log\(f"Failed to send notification \{i\+1\}/\{len\(fragments\)\} to \{self\.peer_name\}: \{e\}", RNS\.LOG_ERROR\)
return False
return True'''
new_peripheral = ''' def _send_via_peripheral(self, fragments):
"""
Send fragments via driver (peripheral mode uses notifications).
Args:
fragments: List of fragment bytes to send
Returns:
bool: True if all fragments sent successfully, False otherwise
"""
for i, fragment in enumerate(fragments):
try:
# Driver automatically handles notification vs write based on connection type
self.parent_interface.driver.send(self.peer_address, fragment)
self.txb += len(fragment)
self.parent_interface.txb += len(fragment)
except Exception as e:
RNS.log(f"Failed to send fragment {i+1}/{len(fragments)} to {self.peer_name}: {e}", RNS.LOG_ERROR)
return False
return True'''
content = re.sub(old_peripheral, new_peripheral, content)
# Replace _send_via_central
old_central = r''' def _send_via_central\(self, fragments\):
"""
Send fragments via GATT characteristic write \(central mode\)\.
Args:
fragments: List of fragment bytes to send
Returns:
bool: True if all fragments sent successfully, False otherwise
"""
# Use stored central_client \(set at initialization for central connections\)
if not self\.central_client or not self\.central_client\.is_connected:
RNS\.log\(f"\{self\} peer \{self\.peer_name\} \(\{self\.peer_address\}\) not connected or disconnected", RNS\.LOG_WARNING\)
return False
client = self\.central_client
# Send each fragment via BLE characteristic write
for i, fragment in enumerate\(fragments\):
try:
# Schedule the async write in the parent's event loop
future = asyncio\.run_coroutine_threadsafe\(
client\.write_gatt_char\(BLEInterface\.CHARACTERISTIC_RX_UUID, fragment\),
self\.parent_interface\.loop
\)
# Wait for completion \(with timeout\)
future\.result\(timeout=2\.0\)
self\.txb \+= len\(fragment\)
self\.parent_interface\.txb \+= len\(fragment\)
except asyncio\.TimeoutError:
RNS\.log\(f"\{self\} timeout sending fragment \{i\+1\}/\{len\(fragments\)\} to \{self\.peer_name\}, "
f"packet lost \(Reticulum will retransmit\)", RNS\.LOG_WARNING\)
return False
# HIGH #3: Comprehensive asyncio exception handling
except \(asyncio\.CancelledError, RuntimeError\) as e:
RNS\.log\(f"\{self\} event loop error sending fragment \{i\+1\}/\{len\(fragments\)\}: "
f"\{type\(e\)\.__name__\}: \{e\}", RNS\.LOG_ERROR\)
# Mark interface as offline if event loop died
if isinstance\(e, RuntimeError\) and "closed" in str\(e\)\.lower\(\):
RNS\.log\(f"\{self\} event loop is closed, marking interface offline", RNS\.LOG_ERROR\)
self\.parent_interface\.online = False
return False
except ConnectionError as e:
RNS\.log\(f"\{self\} connection lost to \{self\.peer_name\} while sending fragment \{i\+1\}/\{len\(fragments\)\}: "
f"\{type\(e\)\.__name__\}: \{e\}, packet lost", RNS\.LOG_WARNING\)
return False
except Exception as e:
error_type = type\(e\)\.__name__
RNS\.log\(f"\{self\} unexpected exception sending fragment \{i\+1\}/\{len\(fragments\)\} to \{self\.peer_name\}: "
f"\{error_type\}: \{e\}, packet lost \(Reticulum will retransmit\)", RNS\.LOG_WARNING\)
# If one fragment fails, the whole packet is lost
# Reticulum's upper layers will handle retransmission
return False
return True'''
new_central = ''' def _send_via_central(self, fragments):
"""
Send fragments via driver (central mode uses GATT writes).
Args:
fragments: List of fragment bytes to send
Returns:
bool: True if all fragments sent successfully, False otherwise
"""
# Check if peer is still connected
if self.peer_address not in self.parent_interface.driver.connected_peers:
RNS.log(f"{self} peer {self.peer_name} ({self.peer_address}) not connected", RNS.LOG_WARNING)
return False
# Send each fragment via driver
for i, fragment in enumerate(fragments):
try:
# Driver automatically handles write vs notification based on connection type
self.parent_interface.driver.send(self.peer_address, fragment)
self.txb += len(fragment)
self.parent_interface.txb += len(fragment)
except ConnectionError as e:
RNS.log(f"{self} connection lost to {self.peer_name} while sending fragment {i+1}/{len(fragments)}: "
f"{type(e).__name__}: {e}, packet lost", RNS.LOG_WARNING)
return False
except Exception as e:
error_type = type(e).__name__
RNS.log(f"{self} unexpected exception sending fragment {i+1}/{len(fragments)} to {self.peer_name}: "
f"{error_type}: {e}, packet lost (Reticulum will retransmit)", RNS.LOG_WARNING)
return False
return True'''
content = re.sub(old_central, new_central, content)
return content
def remove_stale_references(content):
"""Remove or update stale references to self.loop, self.gatt_server, etc."""
# Remove _start_gatt_when_identity_ready method (replaced in pass 1)
pattern = r' def _start_gatt_when_identity_ready\(self\):.*?(?=\n def )'
content = re.sub(pattern, '', content, flags=re.DOTALL)
# Remove remaining asyncio imports that aren't needed
# (Keep asyncio since it might still be imported elsewhere, but comment about driver ownership)
# Update threading model docstring
content = content.replace(
' THREADING MODEL:\n - Main asyncio loop in separate thread (_run_async_loop)\n - LOCK ORDERING CONVENTION (to prevent deadlocks):\n 1. peer_lock - ALWAYS acquire first for peer state access\n 2. frag_lock - THEN acquire for fragmentation state\n NEVER acquire locks in reverse order! (HIGH #2: deadlock prevention)\n - Uses asyncio.run_coroutine_threadsafe for cross-thread calls',
' THREADING MODEL:\n - Driver owns async event loop in separate thread\n - LOCK ORDERING CONVENTION (to prevent deadlocks):\n 1. peer_lock - ALWAYS acquire first for peer state access\n 2. frag_lock - THEN acquire for fragmentation state\n NEVER acquire locks in reverse order! (HIGH #2: deadlock prevention)\n - Driver callbacks invoked from driver thread'
)
return content
def main():
input_file = 'src/RNS/Interfaces/BLEInterface.py'
print("Reading file...")
content = read_file(input_file)
print("Step 1: Refactoring detach() method...")
content = refactor_detach_method(content)
print("Step 2: Refactoring send methods...")
content = refactor_send_methods(content)
print("Step 3: Removing stale references...")
content = remove_stale_references(content)
print("Writing refactored file...")
write_file(input_file, content)
print("Done! Pass 2 complete.")
print("\nRemaining manual tasks:")
print(" - Verify all driver callbacks are correct")
print(" - Test the refactored interface")
print(" - Remove any remaining comments about bleak/bluezero")
if __name__ == '__main__':
main()

9
src/ble_reticulum/BLEInterface.py
View file

@ -67,17 +67,14 @@ except NameError:
if _interface_dir not in sys.path:
sys.path.insert(0, _interface_dir)
# Import base Interface class
# When integrated into Reticulum, this will be:
# from ble_reticulum.Interface import Interface
# For now, we'll need to handle the import path
# Import base Interface class from Reticulum
try:
from ble_reticulum.Interface import Interface
from RNS.Interfaces.Interface import Interface
except ImportError:
# Fallback for development
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../../../'))
from ble_reticulum.Interface import Interface
from RNS.Interfaces.Interface import Interface
# Import fragmentation module
# Note: When loaded as external interface, use absolute imports