Running a 28.9-Million-Parameter Language Model on an ESP32-S3 How a 28.9-Million-Parameter Language Model Runs on an ESP32-S3 Running a transformer language model directly on a microcontroller is normally constrained not only by computational throughput, but more fundamentally by the amount of fast memory available to hold model parameters, activations, temporary buffers and the attention state. A project published in July 2026 demonstrated a different approach: a 28.9-million-parameter language model running entirely on an ESP32-S3 , generating approximately 9.5 tokens/s end-to-end . The result is not based on fitting the complete model into SRAM. Instead, it relies on a deliberate decomposition of the model across the device's memory hierarchy, with a large Per-Layer Embedding (PLE) table retained in flash and only the computationally active portion of the model kept in fast memory. This result deserves a...
Lets learn something new
Random posts to achieve enlightenment