Add SD Card & FAT Format

jlpoole 2026-02-14 15:24:02 -08:00
commit 49a518189b

228
SD-Card-%26-FAT-Format.md Normal file

@ -0,0 +1,228 @@
The SD card must be FAT formatted. Here are the reasons why:
### Prompt
Would you know if Linux file systems could be supported? Does Arduino just limit itself to FAT?
### Response
Excellent question. This gets into architecture, not just formatting.
Short answer:
**Yes — Arduinos SD library is effectively limited to FAT (FAT16/FAT32, sometimes exFAT).
No — Linux filesystems like ext4 cannot realistically be supported on an ESP32-class microcontroller.**
Now lets unpack why.
---
# 1. What Arduino Is Actually Using
When you call:
```cpp
SD.begin(...)
```
You are using:
* Arduino-ESP32 SD wrapper
* Which wraps **FatFS**
* Which implements **FAT16/FAT32**
* Sometimes exFAT depending on build configuration
FatFS is small, deterministic, and designed for microcontrollers.
It is not a Linux VFS stack.
---
# 2. Why FAT Is Used in Embedded Systems
FAT has advantages:
* Extremely simple on-disk structure
* No journal
* No complex metadata trees
* Low RAM requirement
* Small code footprint
* Deterministic behavior
* Universally readable by Windows / macOS / Linux
That last point matters enormously for field devices like yours.
---
# 3. Why ext4 (or other Linux filesystems) Are Not Practical
Lets examine what ext4 requires:
* Journaling
* Block group descriptors
* Extents
* Multi-level inode structures
* Dynamic allocation bitmaps
* HTree directory indexing
* Crash recovery journal replay
* Kernel-level VFS layer
* Substantial RAM buffers
An ESP32-S3 has:
* ~512KB internal RAM
* 8MB PSRAM (external, slower)
* No MMU
* No kernel
* No block device abstraction layer like Linux
You would need to:
* Port the entire ext4 driver
* Provide block caching
* Provide journaling layer
* Handle power-loss atomicity
It would consume massive RAM and flash and be very fragile.
In practice: **not realistic**.
---
# 4. What You *Could* Use Instead
## Option A — FAT (what you're using)
Best choice for removable SD cards.
## Option B — LittleFS (internal flash)
For internal storage (SPI flash partitions), many embedded systems use:
* LittleFS
* SPIFFS (older)
* Wear-leveling FS
These are:
* Designed for microcontrollers
* Power-loss resilient
* No journaling overhead
* Small RAM footprint
But they are **not suitable for large SD cards**.
---
# 5. Could exFAT Be Used?
Possibly.
ESP32s FatFS can be built with exFAT support (depending on SDK config).
exFAT supports:
* Large volumes
* Large files
* More modern structure
But still lightweight compared to ext4.
However, FAT32 is usually sufficient for logging systems like yours.
---
# 6. Important Architectural Insight
Your SD card is not just storage — it is a removable exchange medium.
If you format it ext4:
* Windows cannot read it.
* macOS cannot read it.
* You lose plug-and-read convenience.
* Field debugging becomes painful.
For devices like:
* LoRa nodes
* Data loggers
* Field sensors
FAT is the industry standard specifically because it is portable.
---
# 7. If You Truly Wanted Linux Filesystems
You would need:
* Embedded Linux
* A kernel
* ext4 driver
* Block device driver
* MMU
* Proper memory management
That means something like:
* BeagleBone
* Raspberry Pi
* Not ESP32
You crossed that boundary.
---
# 8. So Is Arduino “limiting” you?
Not really.
The limitation is:
* Microcontroller architecture
* RAM footprint
* Deterministic real-time constraints
Arduino simply wraps FatFS because it is appropriate for this class of device.
---
# 9. For Your Specific Project (microReticulum + T-Beam)
FAT32 is actually ideal because:
* You want easy card extraction
* You want to analyze logs on Linux workstation
* You want simple directory trees
* You do not need journaling
* You do not need Unix permissions
Your current design is correct for the platform.
---
# 10. Advanced Thought
If you ever wanted:
* Transaction-safe logging
* Crash-resistant metadata
* Append-only log guarantees
You would implement that at the **application layer**, not by switching to ext4.
For example:
* Write logs with length-prefixed records
* Use atomic rename patterns
* fsync after writes
* Maintain journal file in userland
Thats how embedded systems solve it.
---
If youre curious, I can also explain:
* Why FAT corruption risk is often overstated on embedded devices
* Or how to implement power-fail-safe logging patterns on FAT
Both are relevant to your field nodes.