AI developer runs 28.9-million-parameter model on $10 ESP32-S3 microcontroller — uses Google's Per-Layer Embeddings technique, stores table on 16MB Flash memory

When we talk about running local AI these days, the conversation usually either revolves around mini-PCs like the RTX Spark or drifts into wistful thinking about home servers and ludicrously expensive professional GPUs. Well, I reckon the most impressive AI hardware trick in a good while just happened on a piece of silicon that costs less than a decent burger. Last week, a Ukrainian developer named Slava S, who simply goes by 'slvDev' on GitHub, dropped a project called ESP32-AI. It's exactly what you think: he got a 28.9-million-parameter language model running locally, entirely on-device, on an ESP32-S3 microcontroller. If you haven't read any of our previous coverage of this tiny chip, ESP32-S3 boards offer about the best bang for buck in the whole computing world. You can snag one online with a protective case for under $20 here in the States, and bare boards are readily available for under $10 around most of the world. As you'd expect from a chip so cheap, it's not powerful. On this variant, the S3, you get exactly 512KB of SRAM, 8MB of PSRAM, and 16MB of flash memory, which is not very much memory at all. So how exactly do you cram a nearly 30-million parameter model onto a chip with less primary storage than a single raw photo from your smartphone? Usually, to run an LLM, the entire model has to sit in your system's fast memory because the processor needs to constantly do math against every parameter to generate the next word. If you try to run a 29M parameter model normally on an ESP32, you run out of fast RAM instantly. The previous record for a chip like this was around 260,000 parameters by one Mr. Dave Bennett, as pointed out by Slava himself on X. Our clever hacker got around this bottleneck by borrowing a brilliant architectural trick from Google's Gemma called Per-Layer Embeddings. He quantized the model down to 4-bit (making the total file size just 14.9 MB) and changed where the data lives; instead of trying to stuff the whole thing into the tiny 512KB SRAM or the only slightly-less-tiny 8MB PSRAM, he dumped the 25-million-parameter embedding table into the relatively-slow 16MB Flash memory. Because this specific model architecture only needs to pull a few rows from this table per token, the inherent slowness of the Flash memory doesn't choke the processor, and so the 512KB of fast SRAM is kept clear for just the "thinking core", the actual reasoning weights. Now, let's pump the brakes for a second, because I know someone out there is already wondering if they can replace their server with an $8 chip. The model he used was trained on the TinyStories dataset, and it's really more of a Small Language Model (SLM), or honestly, a "micro LM." Due to the way it was created, it's only capable of writing short, simple, fictional stories. It will not answer questions, it will not follow instructions, it won't write your Python code, and it possesses exactly zero factual knowledge about the real world. Focusing on that limitation completel…