Monday, November 23, 2009

Drizzle Replication using RabbitMQ as a transport

Having spent a bit of time learning how the transaction log in drizzle works (and it is incredibly easy to work with), I got an idea to use RabbitMQ as a transport. RabbitMQ is an implementation of the AMQP standard in Erlang, so it must be awesome.

It works like this; on the master, a java app is simply tailing the transaction log, sending all new transactions, in raw format, to a rabbitmq server.

The slave(s) are connected to the messaging server and are guaranteed to get the raw messages. When the slave gets the message, it transforms it to a JDBC prepared statement and executes every statement as a batch operation. The reason I use prepared statements and batch operations is that I get a lot for free from the JDBC driver, for example correct escaping of strings etc, and I can also enable the rewrite batch handler feature to get a great performance boost.

Another great thing we get for free by using rabbitmq is the fail-safety, if an exception is thrown in the slave, the message is not acknowledged and it will be retried later.

Being able to write something like this in a few hours really shows how powerful the drizzle replication system is. It will be one of the killer features.

There is one issue i really need to fix, namely that the master does not keep track of which transactions it has sent over the wire, so it will resend all transactions in the log every time it is restarted. Of course, since this was written in a short time, there are probably lots of other issues as well. If you want a real replication solution, go check out Tungsten Replicator.

If anyone wants to contribute, the code is on launchpad: https://code.launchpad.net/~krummas/+junk/rabbitmq-replication

If you simply want to try it out, get the binaries here: http://marcus.no-ip.biz/rabbitrepl.zip To start it, you do java -jar replication.jar master.properties - just make sure you edit the .properties files before starting.

4 comments:

Robert Hodges said...

Hi Marcus! I had never heard of this but it sounds quite useful. Another alternative to move things into RabbitMQ is to write a Tungsten applier into RabbitMQ. We can discuss that after we release Drizzle support on Tungsten based on your patch. (Thanks!!) Robert

edward said...

Followed the instructions and it worked like a charm. Good work, Marcus.

clint-fewbar said...

Wow this is very cool. I believe the original plan for Drizzle's replication is to use Gearman, so it would make sense that any messaging system will work.

Really the "resending everything" issue isn't a problem. Slaves should keep track of the last transaction they successfully committed, and simply ignore everything up until that point.

mygamebest said...
This comment has been removed by a blog administrator.