We replaced H.264 streaming with JPEG screenshots (and it worked better)

Summary

Part 2 of our video streaming saga. Read Part 1: How we replaced WebRTC with WebSockets →Let me tell you about the time we spent three months building a gorgeous, hardware-accelerated, WebCodecs-powered, 60fps H.264 streaming pipeline over WebSockets......and then replaced it with grim | curl when the WiFi got a bit sketchy.I wish I was joking.We’re building Helix, an AI platform where autonomous coding agents work in cloud sandboxes. Users need to watch their AI assistants work. Think “screen share, but the thing being shared is a robot writing code.”Last week, we explained how we replaced WebRTC with a custom WebSocket streaming pipeline. This week: why that wasn’t enough.The constraint that ruined everything: It has to work on enterprise networks.You know what enterprise networks love? HTTP. HTTPS. Port 443. That’s it. That’s the list.You know what enterprise networks hate?UDP — Blocked. Deprioritized. Dropped. “Security risk.”WebRTC — Requires TURN servers, which requires UDP, which is blockedCustom ports — Firewall says noSTUN/ICE — NAT traversal? In my corporate network? Absolutely notLiterally anything fun — Denied by policyWe tried WebRTC first. Worked great in dev. Worked great in our cloud. Deployed to an enterprise customer.“The video doesn’t connect.”checks network — Outbound UDP blocked. TURN server unreachable. ICE negotiation failing.We could fight this. Set up TURN servers. Configure enterprise proxies. Work with IT departments.Or we could accept reality: Everything must go through HTTPS on port 443.So we built a pure WebSocket video pipeline:H.264 encoding via GStreamer + VA-API (hardware acceleration, baby)Binary frames over WebSocket (L7 only, works through any proxy)WebCodecs API for hardware decoding in the browser60fps at 40Mbps with sub-100ms latencyWe were so proud. We wrote Rust. We wrote TypeScript. We implemented our own binary protocol. We measured things in microseconds.Then someone tried to use it from a coffee shop.“The video is frozen...