Message Queues: A Simple Guide with Analogies

Summary

I find stories and analogies very fascinating and — to explain message queues in a super approachable way, we will use some analogies: databases, warehouses and post offices. Stay with me … Databases are primarily used for data persistence — think Postgres or MongoDB. Like databases, message queues also perform some storage function. But why use message queues for data storage when there are databases? Think of databases and message queues in terms of warehouses and post offices. Databases are like warehouses - they are designed to hold a lot of different things, most times, over a long period of time. Message queues on the other hand are like post offices — Where letters and packages stop briefly on their way to being delivered. The packages don't stay there long; they're just sorted and sent off to where they need to go. Essentially, databases are primarily designed for scenarios where you need to store and manage some state over a long period of time. In contrast, you would want to use a message queue for data that you do not want to keep around for very long— A message queue holds information just long enough to send it to the next stop. If you look at message queues from this post office perspective, then you will begin to appreciate the fact that a message queue is simply a medium through which data flows from a source system to a destination system. Looking at message queues as medium of communication is just one perspective, but it’s sufficient to help you get started with message queues — Let’s double down on that perspective. A message queue is a technology that simply receives data, formally called messages in the message queueing world from a source system(s) (producer), lines up these messages in the order they arrive, then sends each message to some final destination, usually another system called the consumer. Note that both the producer and consumer could also just be modules in the same application. Now that we understand the core essence of message...