Doom has been ported to an earbud

Summary

Let's switch to a more readable font first. I'll put out an article / video diving deeper into this later, but here are a few bits of info: This project is made up of four parts: The DOOM port that runs on the earbuds The 'serial server' that acts as a bridge between the earbuds and the web server and also transcodes the MJPEG stream to twitch The web server that serves assets, manages the queue, forwards keypresses and displays the MJPEG stream The static webpage that tells your browser what to display on the screen and how to communicate with the web server The firmware pushes up against a few hardware limitations: Serial Connection Earbuds don't have displays, so the only way to transfer data to/from them is either via bluetooth, or the UART contact pads. Bluetooth is pretty slow, you'd be lucky to get a consistent 1mbps connection, UART is easily the better option. DOOM's framebuffer is (width * height) bytes, 320 * 200 = 96kB. (doom's internal framebuffer is 8-bit not 24-bit) The UART connection provides us with 2.4mbps of usable bandwidth. 2,400,000 / 8 / 96,000 gives us... 3 frames per second. Clearly we need to compress the video stream. Modern video codecs like h264 consume way too much CPU and RAM. The only feasible approach is sending the video as an MJPEG stream. MJPEG is a stream of JPEG images shown one after the other. I found an excellent JPEG encoder for embedded devices here, thanks Larry! A conservative estimate for the average HIGH quality JPEG frame is around 13.5KB, but most scenes (without enemies) are around 11kb. Theoretical maximum FPS: - Optimistic: `2,400,000 / (11,000 * 8)` = 27.3 FPS - Conservative: `2,400,000 / (13,500 * 8)` = 22.2 FPS CPU The stock open source firmware has the CPU set to 100mhz, so I cranked that up to 300mhz and disabled low power mode. The Cortex-M4F running at 300mhz is actually more than enough for DOOM, however it struggles with JPEG encoding. This is why it maxes out at ~18fps, I don't think there's much else I ...