Some time ago I blogged about the same idea Predictable Binary Representation (PBR)
, naively thinking that no one has done this before.

With the help of few friends, and some googling there turned out to be a zillion solutions that attempt to address that very problem; the etch has been really scratched by many, and the need for a compact, and predictable binary protocol is a must, no matter how fast the internet is or will become.

Parsing JSON, despite the fact that its among the most elegant human-readable text-protocols, would consume CPU cycles, and would be susceptible to errors more likely than not.

With a Binary protocol, you could …

• Know the total size of a message (aka packet) before hand
• Have its checksum that you can verify
• For every single data entry, you would exactly know what size in bytes to expect next and what the type of data is, as opposed to checking for special marking characters to tell when an end of that item is reached.
• No need to escape any special characters, should they occur plainly within the data. The problems JSON and XML have because of this deficiency are huge. Why all that hassle?! A question that puzzled me for years.
• Far more compact in size, even without compression. There are no xml tags repeating for every node. In hessian, a dictionary of entry-names is only defined once at the header, then corresponding id’s are used across the message.

From the technical perspective, when we talk about web services, we are talking about software-to-software communication. As such Human-readable protocols are mere overhead to say the least. We should focus more on optimizing software-to-software protocol than making it look good to the human eyes.

Forget about XML, JSON or anything similar; and consider the beauty, performance, and optimization of binary protocols.

Despite the competition from Thrift (by facebook), Protocol Buffers (by google), and ASN.1 (older standard), along many others, Hessian (by Caucho) stands out from the crowd …

Let me tell you why I quickly fell in love with Hessian,

• Dead-simple to use, both server-side and client-side.
• Can be equally used on Java, C/C++, .Net, Ruby, PHP, Flash and so forth.
• Saves tons of bandwidth, cpu cycles and increases reliability
• Runs via the firewall-friendly http-protocol

It was intriguing when reading the features and specifications. Being the hands-on person I am, I only believe what I can do …

So, lets roll up our sleeves and get in a quick practical session with Hessian ….

Hessian and Java

Preparations

Create few folders to host our samples

mkdir -p hessian/lib
cd hessian

I downloaded the latest hessian library jar file to my local lib folder.

wget 'http://hessian.caucho.com/download/hessian-3.2.0.jar'
mv hessian-3.2.0.jar lib

Using jetty 6.1.x or 7.x would do equally well,

wget 'http://dist.codehaus.org/jetty/jetty-6.1.12/jetty-6.1.12.zip'

Unpack

 unzip jetty-6.1.12.zip

we only need three jars, as we are going to use the compact embeddable jetty jars.

mv jetty-6.1.12/lib/jetty-* jetty-6.1.12/lib/servlet-api-2.5-6.1.12.jar lib/
rm -rf jetty-6.1.12

Make sure to define CLASSPATH environment variable that would properly point to all the jars in our lib folder. for that I use the following setenv.sh script that can be sourced every time needed.

setenv.sh

#!/bin/bash
CLASSPATH="."
for myjar in $(find $PWD/lib -iname '*.jar'); do CLASSPATH=$CLASSPATH:$myjar; done;
export CLASSPATH

Then you can source it like this

source setenv.sh

Define (create) our API interface using POJO

BasicAPI.java

public interface BasicAPI {
	public String hello ();
}

Create the service API implementation and expose via an embeddable servlet container

BasicService.java

import org.mortbay.jetty.servlet.Context;
import org.mortbay.jetty.Server;
import com.caucho.hessian.server.HessianServlet;
 
public class BasicService extends HessianServlet implements BasicAPI {
	private String _greeting = "Hello, world";
 
	public void setGreeting(String greeting) {
		_greeting = greeting;
	}
 
	public String hello() {
		return _greeting;
	}
 
	public static void main (String [] args) throws Exception {
		Server server = new Server(8080);  
		Context context = new Context(server, "/", Context.SESSIONS);
		context.addServlet(BasicService.class, "/basic");
		server.start();
	}
}

Compile the Java Server API and Interface and start our service

javac BasicService.java 
java BasicService

Create the Client app

Client.java

import com.caucho.hessian.client.*;
public class Client {
	public static void main (String [] args) throws Exception {
		String url = "http://localhost:8080/basic";
		HessianProxyFactory factory = new HessianProxyFactory();
		BasicAPI basic = (BasicAPI) factory.create(BasicAPI.class, url);
		System.out.println("basic.hello() said : " + basic.hello());
	}
}

Compile and run the client

javac Client.java
java Client

C’est tout! (Thats all). Your java-based client-server hessian binary protocol web service system is up and running. Wow!

Hessian in Ruby

After download and unpacking hessian library for ruby it only takes three lines of code to connect to our java web service

require "lib/hessian.rb"
client = Hessian::HessianClient.new('http://localhost:8080/basic')
print client.hello

It seems that you can also easily create hessian-based services via ruby, but i didn’t dig into it.

Hessian in PHP
Download, unpack and make accessible via apache the php library of hessian. There is a minor glitch in Hessian PHP in which the DateTime class conflicts with Php 5.2 pre-defined DateTime class. Simply re-touch the library code and rename that class.

php_hessian_client.php

<?
include_once 'HessianClient.php';
$proxy = &new HessianClient('http://localhost:8080/basic');
echo $proxy->hello(); 
?>

You can also write a simple hessian web service api provider …
php_hessian_service.php

<?
class Math{    
  function add($n1,$n2) {        
    return $n1+$n2;    
  }    
  function sub($n1,$n2) {        
    return $n1-$n2;    
  }    
  function mul($n1,$n2) {        
    return $n1*$n2;    
  }    
  function div($n1,$n2) {        
    return $n1/$n2;    
  }
}
include_once 'HessianService.php';
$wrapper = &new HessianService();
$wrapper->registerObject(new Math);
$wrapper->service();
?>

More!
Wait, there is more!

• Lanuages Support Hessian is supported on Grails via the remoting plugin, I didn’t try the rest of the languages supported by hessian yet, but it looks that they would be fine.
• Authentication and Http Proxy
  You can also make use of Http Authentication to add a security layer on top of your api’s, and proxy-by-pass in case there is a proxy in the way.
• Additional Services built on top of Hessian
  There are also two interesting technologies by Caucho built on top of hessian, viz BAM and comet .
• Performance benchmarks
  This is an interesting benchmark, demonstrating how hessian perform against classical text and binary protocols.

Gotcha’s
I can see a couple of minor gotcha’s in Hessian library that can be addressed …

A. The language-support projects seem not to be updated for a long time, some more than a year. Those projects only support hessian protocol version 1.0.x and don’t support the latest hessian 2 protocol.

B. The thing I really missed is a JavaScript library that connects me to a hessian service directly. That alone will immediately offer a great value to AJAX applications. I don’t see that far from possible, given the fact that there is already an ActionScript implementation for Flash. I hope someone will dedicate the effort to make that a reality.

Conclusion
The Internet is coming back to its senses , using Binary-based communication protocols might sound 30-year old concept, but it brings a lot on the table that current text-based human-readable protocols don’t.
I am all forward for making use of such protocols in SaaS / SOA setups. Simply Binary Protocols such as hessian, could provide an alternative communication option for today’s SOAP/XML-RPC/JSON ones.

But they all the good solutions remain sub-optimal when it comes to network bandwidth overhead and cpu usage needed to parse the blob every time.

This might have, and should have been considered long time ago. I didn’t find any open format to cover it however, hence I’m suggesting one…

Features

The following needs are not addressed by those data representation formats, but are by PBR :

1. Supporting Binary content. You need to revert to Base64 or the like to do this. but its not natively supported.
2. Completly Predictable (so no guess work in parsing). I don’t want to waste cpu cycles and logic to parse the content in an unpredicable manner. I want to be able to exactly tell what the next step a head and how much data I need to manage.
3. No escaping. I really don’t like escaping. I want to be able to put my content AS IS ™.
4. Reliable : Checksum. Thats a cool addition that will make me confident of my data.

Here I outline the details of PBR (pronoucde  Pe.be.r) along with reference implementation (TBD) :

The data is represented in a message.

Each message is made of envelope header and body.

The Data elements (Entries) are organized in Entry-Name/Entry-Value pairs.

The body is simple a list of Entries (aka root node).

Supported Data Types

Just like JSON PBR does support the basic and most used data types that cover most of today’s modern needs

1. Boolean
2. String (UTF-8)
   
3. Binary
4. Integer
5. Double
6. Date
7. Map (List is just a map with unnamed entries)
   

- Depth or number of entries is unlimited.
- Content-size limit it 2^31 (2GB).

PBR is not human readable, but using a binary editor or in the future a simple PBR reader would allow to easily read and manually manipulate the data.

Envelope Header

• PBR Version (2 Bytes, signed short)
• MD5 Checksum of the body (16 Bytes)
• Total Message Size exclusive of the envelope header  in Bytes(4 Bytes, signed long)
• Number of Level-One Entries (4 Bytes, signed long)

Total Envelop Header size is 26 bytes

The Message Body is a list of data entries. The number of those data entries (1st-level) is as defined in the header above.

Data Entry

• Entry Type (1 Byte, signed byte) : The type id of the entry, 1..7 as per the types list above.
• Entry Name Size (1 Byte, signed byte) : The Size of the Name of the entry.  can be set to 0.
• Entry Name Value (0..127 Bytes) : The entry name in UTF-8.
• Number of Children (4 Bytes, signed long)
• Value Size : (4 Bytes, signed long) : The size in bytes of the value of this entry.
• Value : ( 1 .. 2^31)

Any entry can have children, just like XML node.

However it can only either have children or a value (i.e. Number of Children ==0).

If Number of Children > 0, then the value contains an aggregation of a list of other entries that need to be read.

Now it is — and so for many years — I can see no reason why GNOME can’t do the switch.

This will be a major effort of course, and there will be two types of opponents in the GNOME community :
A. C-Lovers : Those who like C-language (not C++) as GTK and GNOME in general is more C-Lang than C++.
B. Mono-lovers : Those who are after mono, which is not a very successful imitation for Microsot .NET.

Lets put that a side for a minute and lets give it a hard look, GNOME development has been stalled due to many technology limitation. Even with the latest GUADEC and talk about tab-enabling (which i still think is a joke).

So, despite our love and hate, at the end of the day QT is THE most mature toolkit in the FLOSS world. It deserves all the blessings it can get … and very well deserves to be considered as an excellent candidate for the GNOME project, especially after what happened with KDE4 and their move into directions that are far from the users and community. The failure in KDE4 was a result of the people who assumed its leader ship in the last two years, and its not in any way related to QT.

Nevertheless, for the yours truly this does not have to be the case… here is my angle …

The need and demand to build content-oriented websites have been knocking my door for that last 10 years. I built several solutions, using ASP, PHP, Cocoon, Lenya, Ruby on Rails, Java Servlets … along few others. Each of them had its own shortcomings that made me not to choose again in my next project.

I spent good portion of my time just searching for a better alternative …

Ruby on Rails (RoR): My experience with Rails has started around two years ago. Rails was a cool technology. However it didn’t live up to the expectations. I think both Ruby and the Rails framework has fundamental implementation issues that should have been solved two years ago, but nothing was done about it. so, after the huge hype and promotion, its cooling down slowly. Now, as they say “Good things never go away” Ruby on Rails made every one think again about how to write web-frame works and how to make it easy and sensible. Java and PHP communities among many others made the move and benefited from the wonderful approach that Rails has offered to web-development.

That being said, RoR has become more like a reference to an original concept.

But what are the limitations of RoR? one may ask, the answer is simple : Performance and Scalability.

Should Ruby 2.0 have been released two years ago, RoR would probably have been the preferred choice. but any advancement in Ruby has been staggering for too too long.

I would never blame any one for that, things are what they are. I’m sure the ruby-developer-community have their reasons. but for me, it means that i have to find my way again in my search for a web-development framework. That was around the beginning of this year.

Lenya 2.0 :A well-architected option that simply has major problems in running up to its advertised features. its very hard to get things working, and if you come up with any specific need that wasn’t previously thought of it would also be super hard to implement. I know that my comments on Lenya might sound too subjective, but they are not; they are based on months of trying and digging and learning without any avail.

PHP (CakePHP, codeIginator, Symphony) : I never got into PHP really. I know that the computer language is but a tool, but I think PHP being made very easy by design made it too difficult for someone like myself.

Groovy and Grails : Although I heard about it couple of years, it never really caught my attention. “what would just-another-ruby-on-rails replica bring to the world!?” I asked by then.
Things have changed, though. For the last four weeks, I have got my hands dirty and my feet wet with Groovy and Grails… and I must say that I am amazed.

What I found so cool about it …

1. Compiled and not interpreted. So its not an interpreter on top a virtual machine.
2. Full Java interchangeability, even the syntax pretty much overlaps. to the degree that if you change the extension of a java file to groovy it would probably work with minimal changes. which means that you can inter-mingle java code and libraries with groovy ones.
3. Grails builds on top of industrial-strength components such as Hibernate and Spring. Hence you will find your most demanding needs covered.

The one shortcoming would be performance (but not scalability). That is grails is multi-threaded (unlike Rails that is single threaded, even on JRuby, rails remain to be single threaded).

But performance is improving, groovy 1.6 promises to bring it closer.

On the grails website, they say “The search is over” I started to realize and believe in that … hope my search will be over at least for the next few years.
more to come …