Thursday, April 7, 2011

Visualizing data in real time using drizzle and rabbitmq, DEMO!

A couple of weeks ago, when Mozilla released firefox 4, i watched glow.mozilla.org - a site made to visualize the firefox downloads in real time. I figured this could be quite a neat use case for my old drizzle to web socket replicator.

So, the compontents involved:

The basic flow is this:
  1. 10 inserts per second are executed in the drizzle database using drizzle-jdbc, these inserts contain a random IP address
  2. Drizzle does its magic and packages up each insert as a google protobuf transaction message
  3. The rabbitmq plugin picks up the message and publishes it to a RabbitMQ server
  4. RabbitReplication consumes the message from RabbitMQ and maps it onto an annotated java object. This gives alot of flexibility to the rabbitreplication user since you can manipulate the information any way you want, in this example i extract a longitude and latitude value from the IP address using MaxMind GeoLite City. Have a look at row the "setSomething()" method in the ExampleRepl.java file. This file is something a rabbitreplication end user would implement and provide
  5. The java object is serialized to json
  6. The json string is published to all connected clients
  7. A javascript on the page takes the lat/lng-values and maps these onto a google map
  8. The same javascript appends some of the raw data to a table

So, I'm hoping the little demo i'm linking below shows some of the extreme flexibility of the Drizzle replication. This, for me, is the unique selling point for Drizzle. Also, if anyone has an actual production use case for this, i'd be extremely happy to help implement it!

http://demo.rabbitreplication.org


(Thanks to Rackspace for hosting!)


1 comment:

LinuxJedi said...

Fantastic work. Really impresses me what can be done when thinking outside the box :)