In this blog post we are going to address the problem of controlling the access to a particular resource in a distributed system. The technique for solving this problem is well know in computer science, it's called Semaphore and it was invented by Dijkstra in 1965 in his paper called "Cooperating Sequential Processes". We are going to see how to implement it using AMQP's building blocks, like consumers, producers and queues.
Different services in our architecture will require a certain amount of resources for operation, whether these resources are CPUs, RAM or disk space, we need to make sure we have enough of them. If we don't put limits on how many resources our servers are going to use, at some point we will be in trouble. This happens with your database if it runs out of file system space, your media storage if you fill it with images and never move them somewhere else, or your JVM if it runs out of RAM. Even your back up solution will be a problem if you don't have a policy for expiring/deleting old backups. Well, queues are no exception. We have to make sure that our application won't allow the queues to grow for ever. We need to have some strategy in place to delete/evict/migrate old messages.
With RabbitMQ 3.2.0 we introduced Consumer Priorities which not surprisingly allows us to set priorities for our consumers. This provides us with a bit of control over how RabbitMQ will deliver messages to consumers in order to obtain a different kind of scheduling that might be beneficial for our application.
When would you want to use Consumer Priorities in your code?
So we added support for federated queues in RabbitMQ 3.2.0. This blog post explains what they're for and how to use them.
RabbitMQ is a very extensible message broker, allowing users to extend the server's functionality by writing plugins. Many of the broker features are even shipped as plugins that come by default with the broker installation: the Management Plugin, or STOMP support, to name just a couple. While that's pretty cool, the fact that plugins must be written in Erlang is sometimes a challenge. I decided to see if it was possible to write plugins in another language that targeted the Erlang Virtual Machine (EVM), and in this post I'll share my progress.
So we've talked about how RabbitMQ 3.0 can break things, but that's not very positive. Let's have a look at some of the new features! Just some of them - quite a lot changed in 3.0, and we don't have all day...
RabbitMQ includes a bunch of cool new features. But in order to implement some of them we needed to change some things. So in this blog post I'm going to list some of those things in case you need to do anything about them.
I've written a plugin for RabbitMQ that adds support for the MQTT 3.1 protocol. MQ Telemetry Transport is a light-weight PUB/SUB protocol designed for resource-constrained devices and limited bandwidth situations, making it ideally suited to sensors and mobile devices. The implementation is a protocol adapter plugin, allowing MQTT clients to connect to a RabbitMQ broker simultaneously with clients implementing other protocols. We encourage projects that demand the combination of a low-overhead protocol on a robust, scalable broker with high reliability and enterprise features to consider this option.
Here at Rabbit HQ we've been enjoying "RabbitMQ in Action", the introduction to RabbitMQ and messaging. Part of the Manning series, the book is written by Jason Williams and Alvaro Videla, both well known for their many contributions to the Rabbit community.
Today we'd like to say thank-you to Jason and Alvaro. Thank-you Jason and Alvaro! You did an amazing job and infinite beers are on us.
But there's more... Manning have kindly offered a promotional discount of 37% to readers of this blog. All is revealed below, in a guest post by Jason Williams himself...