Accessing HTTP/JSON services with JVM-based languages

Lately, I have been working on a number of client APIs for a REST-like service Nu Echo offers for managing dynamic speech recognition grammars. (The APIs will soon be available on github.) This experience made me realize how difficult it is to provide an API in different programming languages using only the core language (i.e. without having to depend on third-party libraries).

To put you in context, my goal was to provide the same API for accessing an web-based, REST-like service in Java, JavaScript (ECMAScript), Python/Jython, Ruby/JRuby, and eventually Groovy and Clojure.

First problem: Base64

Since the web service uses Basic Authorization on most HTTP requests, the username:password string must be encoded using the Base64 algorithm before being added to the HTTP headers. Believe it or not, there is no standard public class in Java to encode/decode Base64 strings. Fortunately, most scripting languages provide one. Except JavaScript (rhino in my case). So I had to include an implementation of the Base64 algorithm in both the Java API and the JavaScript API.

Second problem: JSON

For simplicity, and the best integration possible with JavaScript, the web service can encode its responses in JSON format instead of XML. (The service was first intended to be used from VoiceXML applications, whose scripting language is ECMAScript.) I thought it would be relatively easy to encode/decode data structures in JSON in all the languages I wanted to support. WRONG!

Of course, Java does not have native JSON support. But I knew that from the start. So no surprise there. And JavaScript, through the eval function, supports JSON natively. Again, no surprise.

The first real surprise came from Python. There is no default JSON library that comes with Python 2.5/2.6 (I haven't tried Python 3). I had to install the simplejson library (which is very nice btw). Unfortunately, it cannot run on Jython 2.2, only on Jython 2.5. Since one of my goals was to run the APIs on Tropo, which only supports Jython 2.2, I had a difficult choice to make. I even tried to simply convert Python dictionaries to strings. But although the Python syntax for constant values is very close to JSON, it uses single quotes instead of double quotes for encoding dictionary keys. (The Python constant {'a': 1, 'b': 2} is written as {"a":1, "b": 2} in JSON.)

In the end, I decided to stick with simplejson for greater portability. (The Tropo guys will probably upgrade to Jython 2.2 one of these days.)

On the Ruby side, there is no standard JSON library. You have to install the 'json' Ruby gem. But it is really easy to install in both Ruby and JRuby. My main complaint is that it is not installed by default with [J]Ruby. And services like Tropo do not necessarily provide all the Ruby gems. (They do provide the 'json' gem, to my greatest surprise.)

Conclusion

When designing NuGram Hosted Server's web service, I thought it would really straightforward to provide APIs in most (scripting) languages running on the JVM. HTTP + Basic Authentication + JSON seemed so en vogue... But clearly, it was harder than expected and the code had to depend on classes/modules that don't come with the core language or the standard library.

I strongly believe that JSON should be more widely supported (natively) by all the major scripting languages, much as XML is. Their own syntax for constant data structures (maps, strings, arrays) is so close to JSON that they should encourage people to use JSON instead of XML. Or at least not discourage its use.

Disclaimer: I am fairly new to most JVM-based scripting languages: Python, Ruby, Groovy. I may have missed something trivial. If so, please let me know.