I Love You, Redis, but I'm Leaving You for SolidQueue

Summary

Rails 8, the latest release of the popular web application framework based on Ruby, excised Redis from its standard technology stack. Redis is no longer required to queue jobs, cache partials and data, and send real-time messages. Instead, Rails’s new features—SolidQueue for job queuing, SolidCache for caching, and SolidCable for transiting ActionCable messages—run entirely on your application’s existing relational database service. For most Rails applications, Redis can be discarded. I know how that sounds. The Redis key-value store is fast, adept, and robust, and its reliability made it the preferred infrastructure for Rails job queueing and caching for more than a decade. Countless applications depend on Redis every day. However, Redis does add complexity. SolidQueue, SolidCache, and SolidCable sparked something of an epiphany for me: boring technology such as relational database tables can be just as capable as a specialized solution. Here, let’s examine the true cost of running Redis, discover how SolidQueue works and supplants a key-value store, and learn how to use SolidQueue to migrate an application’s job queues to vanilla PostgreSQL (or SQLite or MySQL). Web development is already too complicated—let’s simplify. The True Cost of Redis What does Redis cost beyond its monthly hosting bill? Setup and ongoing maintenance are not free. To use Redis you must: In addition to those taxes, there are other ongoing burdens to infrastructure and interoperability. You must also: Sustain network connectivity, including firewall rules, between Rails and Redis Authenticate your Redis clients Build and care for a high availability (HA) Redis cluster Orchestrate the lifecycles of Sidekiq processes across deployments Further, when something goes wrong with a job, you’re faced with debugging Redis and your RDBMS, two data stores with very different semantics, switching context between different query languages and tools. And then there’s the issue of two separate backup strat...