Saturday, March 21, 2009

Using EnumSet when communicating over the network

Many applications use bitmasks to send flags over the wire since it is a very efficient way of representing settings etc, problem is that the code to handle them is quite hard to read and maintain, on both the server and client side. To overcome this, EnumSet in Java can be used. If the flags that are masked in the value sent are represented as enums, the bitmask value can be seen as a set of enums, this concept is used in drizzle-jdbc.

For example, to represent server capabilities the following enum is used:

public enum ServerCapabilities {
LONG_PASSWORD((short)1), /* new more secure passwords */
FOUND_ROWS((short)2), /* Found instead of affected rows */
LONG_FLAG((short)4), /* Get all column flags */
CONNECT_WITH_DB((short)8), /* One can specify db on connect */
NO_SCHEMA((short)16), /* Don't allow database.table.column */
COMPRESS((short)32), /* Can use compression protocol */
ODBC((short)64), /* Odbc client */
LOCAL_FILES((short)128), /* Can use LOAD DATA LOCAL */
IGNORE_SPACE((short)256), /* Ignore spaces before '(' */
CLIENT_PROTOCOL_41((short)512), /* New 4.1 protocol */
INTERACTIVE((short)1024), /* This is an interactive client */
SSL((short)2048), /* Switch to SSL after handshake */
IGNORE_SIGPIPE((short)4096), /* IGNORE sigpipes */
TRANSACTIONS((short)8192), /* Client knows about transactions */
RESERVED((short)16384), /* Old flag for 4.1 protocol */
SECURE_CONNECTION((short)32768), /* New 4.1 authentication */
...
private final short bitmapFlag; //holds the value
ServerCapabilities(short i) {
this.bitmapFlag = i;
}

public static short fromSet(Set<ServerCapabilities> capabilities) {
short retVal = 0;
for(ServerCapabilities cap : capabilities) {
retVal = (short) (retVal | cap.getBitmapFlag());
}
return retVal;
}


public static Set<ServerCapabilities> getServerCapabilitiesSet(short i) {
Set<ServerCapabilities> statusSet = EnumSet.noneOf(ServerCapabilities.class);
for(ServerCapabilities value : ServerCapabilities.values())
if((i & value.getBitmapFlag()) == value.getBitmapFlag())
statusSet.add(value);
return statusSet;
}
}


This enum contains all capabilities of the drizzle server we are talking to. The value in the enum constructor is the bit position in the the value sent over the network. So, two convenience methods are implemented in the enum, one for taking the actual bit-masked value and creating an enum set, and one for taking an enum set and creating a bitmasked value. fromSet creates a short which contains the flags set, and getServerCapabilitiesSet creates an EnumSet which contains the enums represented by i.

Now it is possible to handle bitmasked values like this:

    short bitmask = readShortFromNetwork(); // yeah not really, but hey
Set<ServerCapabilities> serverCapabilities = ServerCapabilities.getServerCapabilitiesSet(bitmask);
if(serverCapabilities.containsAll(EnumSet.of(ServerCapabilities.TRANSACTIONS, ServerCapabilities.SSL)) {
doSomethingSecure();
}


Also, when sending something to the server:


Set<ServerCapabilities> capabilities = EnumSet.of(ServerCapabilities.TRANSACTIONS, ServerCapabilities.SSL);
writeShortToNetwork(ServerCapabilities.fromSet(capabilities));


So, what we've got is a type safe way of sending/receiving flags to/from a server. Internally, the enum set is stored in a long, so performance-wise it is close to equivalent of handling the bit-operations yourself.

Look at these files for usage examples:

DrizzleProtocol.java

ServerCapabilities.java

Thursday, March 19, 2009

Introducing Drizzle JDBC

The last couple of months I've been working on a JDBC driver for Drizzle, the driver is available here. The aim for the driver was to be lightweight and easy to develop. It is likely that Connector/J will support drizzle in the future, so the aim for this driver is to be a lightweight BSD-licensed alternative.

To build the driver, do the following (java 6 is required):

$ bzr branch lp:drizzle-jdbc
$ mvn compile

If you want to package the driver into something usable, you need to have a drizzle server on your localhost with a database called test_units_jdbc, then you can do:

$ mvn package

And you will get a .jar file in the target directory. There are also nightly builds available in hudson.

The only current dependency is slf4j, so you need to put the api-jar and one of the implementations on the classpath, check here for more help. The reason I use slf4j is that i don't want to impose a logging mechanism on the users, and I dont want to build something that scans the classpath for known logging implementations etc.

Below is an example of how to use it:

Connection connection = DriverManager.getConnection("jdbc:drizzle://localhost:4427/test_units_jdbc");
Statement statement = connection.createStatement();
statement.executeUpdate("INSERT INTO t1 (id) values (1)");

The connection string is on the form:
jdbc:drizzle://<username>:<password>@<host>:<port>/<database>

Notes about using this driver:
  • It is under heavy development and many features expected from a JDBC driver are not yet implemented. Of course, since it is very early in the development, there will be bugs!
  • Currently only Java6 is supported (JDBC4 was released with java6)
  • The internal protocols and API's will change when the new asynchronous server-API is available. This improve performance in the driver alot
  • Authentication is not supported, this is due to the fact that drizzle does not do this out-of-the-box and the way of doing the authentication is likely to change when the new protocol is in place
  • The driver reads back the full result set into memory and there is currently no way of not doing this.
  • Prepared statements are
  • It does work with hibernate and hibernate-jpa
  • It can be set up as a datasource in JBoss
  • It could, in theory, be used against MySQL as well
  • Bug reports / feature requests are needed! Also, if anyone is interrested in contributing, please do so, contact me on IRC for example, freenode, #drizzle, my nick is marcuse