(The Scheme Way)

MSLUG meeting next week

2011-11-17T16:49:00.001-08:00

The MSLUG (Montreal Scheme/Lisp User Group) is rising from the ashes! Next week, Marc Feeley will talk about Gambit REPL, a version of Gambit-C for iOS and Android.

More details can be found here.

Session handling in NuGram Hosted Server

2011-05-25T06:57:00.000-07:00

Some time ago, I wrote a description of how session handling works in NuGram Hosted Server, which is implemented in Erlang. It's also briefly described on slide 15 of my talk at the Erlang Montreal meetup. Over on the Nu Echo blog, I wrote a short post explaining how the session timeout mechanism can be implemented using a plain Erlang/OTP gen_server behaviour. It's not a very profound post, but it's one of those areas where OTP shines (a very simple solution to a very common problem, and no need to reinvent the wheel).

Slides from my talk at the first Erlang Montreal meetup

2011-05-20T04:19:00.000-07:00

Over on our corporate blog, I've put the slides from the talk I gave at the first Erlang Montreal meetup. The talk was a summary of some lessons we've learned at Nu Echo building NuGram Hosted Server in Erlang.

First Erlang Montreal meeting tomorrow

2011-05-11T19:29:00.000-07:00

Tomorrow, May 12th, 2011, the first meeting of the Erlang Montreal user group will take place at the Notman House from 6:30PM to 8PM. I will have the pleasure to give the first talk, in which I will share some of the lessons we learned at Nu Echo on building real applications in Erlang.

Securing a couchdb database

2011-02-11T13:30:00.000-08:00

Last week, for a post on our corporate blog, I created and populated a hosted CouchDB database (on CouchOne, now CouchBase). Since I put the code on github and it contains references to my publicly available database, I had to secure it a bit. Here is what I did.

Note: this is not a tutorial on CouchDB security issues, only a list of steps I did to secure my database. For a comprehensive guide on CouchDB security, there are excellent online sources.

1. Creating the user/password

The first thing to do is to create a new user with an associated password on the instance.

% curl -X PUT 'https://dboucher.couchone.com/_config/admins/username' -d '"password"'

2. Configuring the user

The next step is to configure the user account by setting its type and roles. In my case, I wanted to add the editor role to my account so I could restrict the creation of new documents to this role (the validator below is kept in a document on the database itself, so it is readable by everyone and I don't want other people to know which user account to hack).

The user account configuration is done using another simple HTTP request:

% curl -X PUT 'https://username:password@dboucher.couchone.com/_users/org.couchdb.user:username' -d@userdata.json

where the file userdata.json contains the following text:

{
  "name": "newuser",
  "type": "user",
  "roles": ["editor"]
}

3. Validator

Finally, to prevent creation of new documents (and modifications or deletion as well) from unauthorized users, I created a design document for my database that only contains a validate_doc_update function. The validation function checks that the user that tries to create/modify/delete a document has the role editor. The code goes as follows:

{
  "_id": "_design/address",
  "validate_doc_update": "function (newdoc, olddoc, userCtx) { if (userCtx.roles.indexOf(\"editor\") == -1) throw({unauthorized: \"illegal access\"});}"
}

To create the design document, I simply entered the following at the shell prompt:

% curl -X PUT 'https://username:password@dboucher.couchone.com/streets/_design/address' -d@designdoc.json

That's it. I could certainly have done something fancier, but that worked and that was really easy to setup.

My resolutions for 2011

2011-01-06T19:09:00.000-08:00

First of all, I wish all of you a very happy new year 2011!

My regular readers may have noticed that I did not publish a lot of posts in the last months. I've been quite busy at work and wrote a number of articles on Nu Echo's blog instead. And most of all, I have not done as much Scheme or Erlang programming as I would have liked to do.

But I played a bit with a number of different technologies in the past months (Ruby, Groovy, CouchDB, NodeJS, websockets, etc.) and it be would worthwhile to write down my observations on these technologies. So my resolution for this new year (as far as this blog is concerned) is to write on a more regular basis about them. For example, I will try to explain:

what I think of Ruby and Groovy for writing DSLs from a Scheme perspective (you know I'm a big fan of macros and DSLs);
what makes Erlang a great language for writing voice applications;
my (very) personal take on NodeJS

and many other things.

Also, there seems to be a growing interest for SchemeScript in the last 6 months. The number of downloads on SourceForge has surged. For this reason, I will take some time to write tutorials and make some screencasts on how to use SchemeScript effectively.

Happy New Year!

ECMAScript - turning a binary function into a variadic one

2010-08-24T19:43:00.000-07:00

In my previous post, I mentioned that it's easy to take a binary JavaScript function and turn it into a variadic one. Here's the helper function to do that:

function variadic(fun, val0f, val1f) {
  return function() {
    var len = arguments.length;
    if (len == 0) {
       return val0f();
    } else if (len == 1) {
       if (val1f != undefined) {
         return val1f(arguments[0]);
       }
       else {
         return arguments[0];
       }
    } else {
      var tmp = fun(arguments[0], arguments[1]);
      for (index = 2; index < len; index++) {
        tmp = fun(tmp, arguments[index]);
      }
      return tmp;
    }
  };
}

The 'variadic' function takes three parameters:

the binary function;
a function that returns the value for the variadic version when called with zero arguments
a function that returns the value for the variadic version when called on a single. If not specified, the first argument to the variadic function is returned as is when called on a single argument.

For example, the summation operator is defined as

var sum = variadic(function(x,y) {return x+y;},
                   function() { return 0;},
                   function(x) { return x;});

while the equivalent of Scheme's / function is defined as

var div = variadic(function(x,y) {return x/y; },
                   function() { throw "not enough arguments to div"; },
                   function(x) { return 1/x;});

It is now possible to call

div.apply(this, [1,2,3,4,5])

to get 0.008333333333333333.

A small experiment with HTML5 canvas and Scheme to JavaScript conversion

2010-08-24T16:54:00.000-07:00

Lately, I've been playing around with some of the upcoming features of HTML5. Last week, I decided to try the HTML canvas. In order to make a realistic test, I tried to port the parse tree viewer in NuGram IDE, one of Nu Echo's products, to JavaScript and the HTML canvas.

To debug a speech recognition grammar with NuGram IDE, an Eclipse plugin, the developer simply enters a test sentence and get a parse tree in the Interpretation view:

NuGram IDE is written almost entirely in Kawa Scheme. The parse tree viewer does not use any graph layout toolkit, the layout algorithm has been written from scratch in Scheme (about 200 lines of commented code). It was a good candidate for a rewrite in JavaScript. Here is what I ended up with (in Chrome):

The conversion process

I translated the whole layout algorithm by hand. I did not use (or develop) any automatic translation tool. I wanted to see how different the JavaScript code would look like compared to the original Scheme code.

I knew that JavaScript/ECMAScript is one of Scheme's very close cousins, but it was amazingly easy to convert the Scheme code into very similarly looking JavaScript code. For instance, take this procedure definition:

(define (compute-nodes-size! gc tree-node level)
  (compute-node-size! gc (get-graph-node tree-node))
  (set-level! tree-node level)
  (for-each (lambda (child)
       (compute-nodes-size! gc child (+ level 1)))
            (get-node-children tree-node)))

It translates to the following JavaScript definition:

function computeNodesSize(ctx, treeNode, level) {
    computeNodeSize(ctx, treeNode.graphnode);
    treeNode.level = level;
    treeNode.children.forEach(function(child) { 
          computeNodesSize(ctx, child, level + 1); 
    });
}

Granted, it's a very simple example. There were some pieces of code that demanded a more complex rewrite. Uses of the Scheme apply procedure is an example. I had to translate the call

(apply max nodes-y)

var maxY = 0;
nodesY.forEach(function(y) { if (y > maxY) { maxY = y; } });

In JavaScript, the max function is not variadic function. It accepts exactly two arguments. It would have been possible to write a function that takes a function of two arguments and turns it into a variadic version of it. But it wasn't worth it, since there was only a single usage of apply and max in the whole algorithm.

Update: I was completely mistaken here, as pointed out by Jon-Carlos Rivera in another post. the max function is variadic. I could instead have written:

Math.max.apply(this, nodesY)

My mistake was to not use apply properly. The first argument to apply must be an object, and not the list of arguments. I should have RTFM before writing such nonsense.

Canvas Support

I tried my parse tree viewer on 3 different browsers (Chrome 5.0.375.29 beta, Firefox 3.6.8, Opera 10.61) on Ubuntu 9.04.

How portable is the resulting widget? Not as much as advertised. I know, HTML5 is not a standards yet, but most of the major browsers claim to support HTML5 canvases. There were two majors aspects that differed between browsers: rendering and user experience.

On the rendering side, Firefox is the worst. Here is what I get:

Some lines appear on top of the nodes (boxes), even though they were drawn before. Opera is not bad, but the texts are not always rendered properly for some font sizes.

On the user experience side, Opera seems the fastest of the three. Chrome comes close. In Firefox, however, the canvas is not redrawn while dragging the parse tree. The parse tree is redrawn only when the button is released.

Conclusion

In conclusion, I've been fairly pleased by the overall experience. The conversion from Scheme to JavaScript was painless, and the result was nice looking. However, and even though it's only based on anecdotal evidence or my lack of experience with the HTML5 canvas, portability is clearly not there yet. My guess is that we'll have to devote a lot of our development time fixing portability issues (we're already spending too much time doing exactly that for plain HTML, right?).

Update: I put the canvas demo online here. Try it and let me know if portability can be improved, if I did things the wrong way, etc.

A blog post on SchemeScript in Russian

2010-07-16T06:52:00.000-07:00

Pavel Samolisov wrote a very good blog post on how to get started with SchemeScript. It's in Russian, but Google can translate it to English (the translation is definitely not very good, but it is better than nothing).

Pavel noted that SchemeScript does not work well on Eclipse Helios. I just pushed a fix on the github repository (and the sourceforge git repository as well) that makes the interpreters work. If you find problems with Eclipse Helios and SchemeScript, please let me know.

2010 Workshop on Scheme and Functional Programming

2010-05-25T05:43:00.000-07:00

This year's Workshop on Scheme and Functional Programming will be held in Montreal on August 21-22. The call for papers is out and papers are due in three weeks (June 14th). Even for those of you not in the research community, it is possible to submit a 6-page paper on practical experience on using Scheme or functional programming.

My VoIP setup at home

2010-05-14T21:41:00.000-07:00

Recently, I had to work from home a bit more. Email and IM is great for communicating with my colleagues and friends, but sometimes talking to them is much more effective. So I decided to finalize my VoIP setup.

Skype

Of course, Skype is the obvious choice for calling friends and relatives. It works great for voice and video on both Windows and my Ubuntu 9.04 (yes, I'm still on Jaunty). My webcam (a cheap Microsoft LifeCam VX-100) is not that well supported on Ubuntu, but it's not so bad. And I use a FreeTalk Everyman USB headset (I got it free at ITExpo in January, thanks to Dan York!).

But Skype itself is not the ultimate answer, at least not for me. Not all my colleagues use Skype at work. So why use Skype Out and pay for local calls when I already have a land line? And I don't like holding a (traditional) phone for one hour when I'm on a conference call with customers or other people at the office. I want to be able to put both of my hands on my laptop keyboard to take notes.

SIP at home

The answer to this is SIP, and three components:

A VoIP adapter. Mine is a Linksys SPA 3102 ATA, a VoIP adapter and gateway from Cisco. This little box converts ordinary (land line) calls into VoIP (SIP) calls, and vice-versa.

Asterisk, a free open-source software PBX. A PBX is the kind of system that can play messages to callers, ask for input, route calls, hold conference calls, queue calls, etc. PBXs are usually costly pieces of hardware that you don't want to buy just for you personal needs.

A SIP softphone.

So my land line is connected directly to the VoIP gateway, which is itself connected to my wired network. When I receive a call on my land line, if I do not answer after a predetermined amount of time, the call is converted to a VoIP call and transferred to Asterisk on my Linux server. Asterisk tries to transfer the call to my SIP phone. If I ignore the call or when I'm not available, the call is simply sent to Asterisk's voice mail module, which will send me a notification email with the recorded message attached to it.

When I want to place a call, I simply enter the number on my softphone, press ENTER, et voilà!

Of course, this whole setup requires a bit of configuration and dialplan programming.

Receiving SIP calls

The only thing that I did not explain is how I can receive SIP calls from outside my home network. I sometimes need the ability to receive calls when I test outbound telephony applications on platforms like Tropo or Teleku.

In order to do this, I created an account on Ekiga.net, a free VoIP service provider. I then configured Asterisk to automatically register (login) to my Ekiga.net account and transfer all calls to my SIP phone when I'm available.

SIP phones

I mainly use two different SIP phones, depending on which OS I boot. On Windows, I use X-Lite free edition. It has a lot of features, has video, but it only supports one account. On Linux, I use Twinkle, which is also free. It does not support video, but it is more stable than the Ekiga softphone on my machine, supports two lines, and has some basic voicemail support.

RabbitMQ acquired by SpringSource

2010-04-13T06:12:00.000-07:00

Rabbit Technologies, a spin out of LShift Ltd. and Cohesive FT, has just been acquired by SpringSource, a division of VMWare. Rabbit Technologies is the company behind RabbitMQ, a high-performance open-source enterprise messaging system based on the AMPQ protocol.

For me, the most interesting thing about this acquisition is the fact that RabbitMQ is an Erlang-based product. This shows once again that Erlang is not an esoteric language anymore, that even large companies are not afraid of using it. This is another great success story for Erlang! And I wish the Rabbit Technologies team the best for the upcoming years.

SchemeScript on Github

2010-03-10T18:09:00.000-08:00

In addition to moving all the development of SchemeScript to git a few months ago, I just pushed my development repository to github. The repository is here:


  http://github.com/schemeway/SchemeScript

I did this essentially because github really rocks. It offers a lot of excellent features for tracking parallel development. I also happen to have a number of other projects hosted there.

SchemeScript used in the Scheme4edu project

2010-02-20T20:20:00.000-08:00

Here is a short screencast of Scheme4Edu, a version of Eclipse streamlined specifically for use by university and college students, featuring the SchemeScript plugin.

Best wishes

2009-12-25T15:56:00.001-08:00

I'd like to wish you all a Merry Christmas and a Happy New Year!

1-session-per-process - some further comments on the NuGram architecture

2009-11-18T16:47:00.000-08:00

In response to my previous post on the NuGram architecture, Ben Simon wrote a post in which he focused most exclusively on the idea of using a single process for handling all the requests for a given session. I wanted to add a few comments about this approach.

First, the idea is not new at all. In fact, I'd say that this is a pretty standard approach in Erlang and the language facilitates its implementation. For instance, the open-source ejabberd XMPP server uses this approach.

Also, it is true that it is completely transparent to the system whether the session's process runs in the same Erlang VM or on a remote machine. The syntax is exactly the same: Process ! Message. That's it. In the case of NuGram Server, the database (which replicates the session table on all nodes) holds references to Erlang processes. Once it has obtained the reference to the session's process, it simply sends a message encapsulating the request to that process (using the ! notation).

Of course, there are some complications if the node on which the process runs suddenly crashes or becomes unavailable. Session replication is less trivial to implement. In our case, the system maintains, together with the session table, a table holding all the relevant data to recreate a session if needed. This was fairly easy to do in our server given our requirements and the nature of the API. For more complex APIs, this could be much more challenging.

Although I'm a big fan of the 1-session-per-process approach (it is a very effective one for the implementation of comet-like servers), it has some limitations that the continuation-based approach do not suffer. The back-button/bookmarking problem immediately comes to mind. Serializable continuations can be put in a database for later retrieval. But the question remains whether this is a practical approach. For instance, for how long do you retain continuations in the database? What happens when the code changes?

Another benefit of using the continuation-based approach is the fact that the code handling the request is written in a more direct style. By this, I mean that the application flow is coded in a more sequential way: do this, then do that, etc. You don't have to code using state machines or callbacks.

But this can be achieved in Erlang as well using two processes per session. (Processes are so cheap in Erlang!) This approach can be used to implement dialog-based applications and providing the illusion of a synchronous API, à la Tropo). I'll talk more about this in another post very soon.

The architecture of NuGram Hosted Server

2009-11-11T16:31:00.000-08:00

In the comments section of one of my previous post, I mentioned that NuGram Hosted Server (www.grammarserver.com) is implemented in a mix of Erlang/Yaws, Java and Kawa Scheme. Alexandre Abreu then asked a few questions about some architectural aspects:

how do erlang and kawa scheme interop? http / json ? ffi ?
to follow up on that, why erlang / yaws and not scala / lift to have a better integration with the jvm ?
the server part if hosted on 1 machine ? how do you handle the load (both the connection load and the computation load since grammar and the services provided by NuGram seem pretty heavy computation-wise)? Maybe you don't need to do much given the amount of people that uses your service and I know you don't offer any guarantees with people using this service in a commercial setup but I am just wondering

These questions prompted for a long answer. So in the following sections I describe some aspects of the NuGram Hosted Server architecture at 10,000 feet high.

1. Context

NuGram Hosted Server is a grammar hosting server for use by communication applications (VoiceXML applications, IM bots, etc.). It provides various grammar-related services through a RESTful API, like dynamic grammar generation and semantic interpretation of text-based sentences (and it will soon provide robust parsing capabilities). It also provides a community-based HTML application to management and share grammars, consult application logs, etc., using standard AJAX techniques.

When we started the project, we had some ambitious requirements in mind:

High-performance HTTP server.
Scalability. (We needed the possibility of dynamically adding nodes to our cluster without interruption of service.)
Fault-tolerance.
Hot code swapping.
Distributed database.

Since I already had some experience with Erlang, I pushed the idea to the management team, after getting the buy-in from my development team. We decided to use this project to evaluate Erlang in the context of a real project.

To answer Alexandre's question more specifically, we did not consider Scala/lift mainly for the same reason we did not choose Java itself. Essentially, we could not use the usual session tracking mechanisms found in J2EE environments (jsessionid, cookies) for our RESTful API. Implementing a clustered solution would have required too much work. And using Erlang was a far better (and more elegant) approach.

2. The architecture

At a very high level, grammarserver.com is composed of two subsystems:

An Erlang/Yaws front-end.
This is the part handling the HTTP requests.
A number of Java/Kawa workers.

Let me describe each subsystem separately.

The Erlang/Yaws front-end

This subsystem handles all the HTTP requests. It currently consists of a single Erlang node, but it has been designed to support the clustering of many Erlang nodes. In this case, we would need a load-balancer in front of the Erlang cluster.

The Erlang system follows most OTP principles. It is a real Erlang application and provides a supervisor and a number of gen_servers. The application also starts an embedded yaws serving requests for several virtual hosts (the HTML websites at www.grammarserver.com, nugram.nuecho.com, with and without SSL support, and the RESTful API at www.grammarserver.com:8082).

Most requests to the RESTful API are done in the context of a session. Each session is associated with an Erlang process and the application keeps a mapping between the session ID and the process for the session in the Mnesia database (it is rather cool to store things like process IDs in a database!). So when the application receives a request, it extracts the session ID from the request URI, finds the corresponding process in the database, and simply forwards the request to that process.

When multiple Erlang nodes are running in a cluster, forwarding a request can involve sending the request to a different Erlang node. This is done completely transparently in Erlang.

Another interesting consequence of using processes to represent sessions is the fact that implementing session timeout becomes trivial. Each session process makes an explicit receive ... after. When the timeout is reached, the session is automatically terminated and removed from the database.

The application also uses Mnesia for other purposes:

It holds the user accounts, of course.
All the instantiated grammars are held in the database.
It holds the node IDs of the available Java/Kawa workers. Keeping that information in a persistent (disk-based) table of Mnesia makes it possible to shut down the Erlang application and reconnect it automatically to the Java nodes when we restart it. More on this below.

Java/Kawa workers

The Java/Kawa workers implement the basic NuGram services. They are written in a mix of Java and Kawa Scheme because most of the code is also shared with NuGram IDE, an Eclipse plugin.

The workers use jinterface to interface with Erlang. This has the advantage of exposing the workers as standard nodes to the Erlang application. In other words, the Erlang application does not even know that the workers are implemented in Java. This is completely transparent.

Many workers can be started, independently of the number of Erlang nodes. The first thing they do is try to find an Erlang node and register with it. If they cannot find an Erlang node, they wait for a specified amount of time, then try again. After a number of retries, they simply stop with an error.

Each grammar is assigned to a single worker. To distribute the load as evenly as possible across the workers, the Erlang system uses a round-robin strategy to assign workers to new grammars (if a session uses a grammar already loaded in a worker, requests are sent directly to that worker, of course).

3. Conclusion

Overall, our experience with Erlang has been excessively positive. (I have to confess that my team members already had some prior exposure to functional programming and Prolog, which helped a lot). Of course, we had to learn some things the hard way, we found some bugs in Yaws. But in the end the platform delivered on its promises. We have an architecture that can scale, we can hot swap code, dynamically change the database schema, add nodes dynamically, etc.

Since NuGram Hosted Server is a free service, we do not guarantee any quality of service, but the platform is really robust and fast and that is very important for communication applications (especially telephony applications where latency translates to dead-air during a conversation).

Vladimir Sedach on high-performance network servers in Lisp

2009-11-06T19:44:00.000-08:00

At our upcoming meeting of the Montreal Scheme/Lisp User Group (MSLUG) on Tuesday, November 17th, Vladimir Sedach will talk about developing high-performance network servers in Lisp. Here is the abstract:

This talk will cover techniques for developing high-performance network servers in Lisp, with examples and lessons from the TPD2, Antiweb, and the speaker's own soon-to-be-released
Common Lisp HTTP servers. Topics covered will include techniques for efficient input handling and output generation, vectored IO, thread pool design, and asynchronous IO management using continuations and state machines.

See you there!

Dynamically setting the Yaws log level

2009-10-25T18:28:00.000-07:00

Debugging a web-based application is more often than not done by first analyzing log files. Even when you can attach a REPL to a running application. They are an integral part of the developer's tool set.

But production applications usually do not enable tracing by default, because this could degrade performance and waste essential resources (memory, CPU time, disk, etc.). Would you dump the content of every HTTP request received by a web server on disk? Of course, when problems occur, trace level is simply raised, with the hope that the logs will contain some useful debugging info.

With the Yaws Erlang-based web server, the log level can be set programmatically like this:

-module(my_yaws_utils).
-export([set_yaws_trace_level/1]).

set_yaws_trace_level(Level) ->
  {ok, GC, Groups} = yaws_api:getconf(),
  yaws_config:hard_setconf(GC#gconf{trace=Level}, Groups).

with Level being either none, {true, http}, or {true, traffic}. The first value disables tracing, the second enables tracing of the HTTP requests in file trace.http, while the last enables tracing of the whole traffic (requests and responses).

It is certainly possible to attach to the Yaws server from another Erlang node to call this function. But if your application is administered by someone not really used to Erlang, it may be more appropriate to provide a command-line script for that purpose.

Basically, the script would connect to the Yaws node (named yaws_daemon@lelouch) and invoke the function with the appropriate log level:

#!/bin/sh
node=yaws_daemon@lelouch
# we should check the number of arguments to the script here...
case $1 in
  none) level=none;;
  http) level="{true, http}";;
  traffic) level="{true, traffic}"
esac

erl -sname trace_script -noinput \
    -setcookie ErlangCookie \
    -eval "rpc:call($node, my_yaws_utils, set_yaws_trace_level, [$level])." \
    -s init stop

The call to rpc:call does the actual RPC call to set the log level on the Yaws server.

The cool thing is the fact that this technique can be applied to a whole bunch of other utility tools, like scripts to stop the server, to reload the application configuration, etc. And this applies to all Erlang applications as well, this is in no way specific to Yaws.

And there are probably other ways to do the same thing. Let me know if you have a better solution.

SchemeScript 1.3.0.alpha9

2009-10-19T16:32:00.000-07:00

On the SchemeWay blog, I posted an entry describing the latest features I have added to SchemeScript. In particular, I have modified the indentation strategy for comments. SchemeScript now behaves more like the Emacs Scheme mode. I have also added some support for the Clojure maps syntax. I wrote some Clojure code lately and desperately needed that feature.

Accessing HTTP/JSON services with JVM-based languages

2009-09-11T17:34:00.000-07:00

Lately, I have been working on a number of client APIs for a REST-like service Nu Echo offers for managing dynamic speech recognition grammars. (The APIs will soon be available on github.) This experience made me realize how difficult it is to provide an API in different programming languages using only the core language (i.e. without having to depend on third-party libraries).

To put you in context, my goal was to provide the same API for accessing an web-based, REST-like service in Java, JavaScript (ECMAScript), Python/Jython, Ruby/JRuby, and eventually Groovy and Clojure.

First problem: Base64

Since the web service uses Basic Authorization on most HTTP requests, the username:password string must be encoded using the Base64 algorithm before being added to the HTTP headers. Believe it or not, there is no standard public class in Java to encode/decode Base64 strings. Fortunately, most scripting languages provide one. Except JavaScript (rhino in my case). So I had to include an implementation of the Base64 algorithm in both the Java API and the JavaScript API.

Second problem: JSON

For simplicity, and the best integration possible with JavaScript, the web service can encode its responses in JSON format instead of XML. (The service was first intended to be used from VoiceXML applications, whose scripting language is ECMAScript.) I thought it would be relatively easy to encode/decode data structures in JSON in all the languages I wanted to support. WRONG!

Of course, Java does not have native JSON support. But I knew that from the start. So no surprise there. And JavaScript, through the eval function, supports JSON natively. Again, no surprise.

The first real surprise came from Python. There is no default JSON library that comes with Python 2.5/2.6 (I haven't tried Python 3). I had to install the simplejson library (which is very nice btw). Unfortunately, it cannot run on Jython 2.2, only on Jython 2.5. Since one of my goals was to run the APIs on Tropo, which only supports Jython 2.2, I had a difficult choice to make. I even tried to simply convert Python dictionaries to strings. But although the Python syntax for constant values is very close to JSON, it uses single quotes instead of double quotes for encoding dictionary keys. (The Python constant {'a': 1, 'b': 2} is written as {"a":1, "b": 2} in JSON.)

In the end, I decided to stick with simplejson for greater portability. (The Tropo guys will probably upgrade to Jython 2.2 one of these days.)

On the Ruby side, there is no standard JSON library. You have to install the 'json' Ruby gem. But it is really easy to install in both Ruby and JRuby. My main complaint is that it is not installed by default with [J]Ruby. And services like Tropo do not necessarily provide all the Ruby gems. (They do provide the 'json' gem, to my greatest surprise.)

Conclusion

When designing NuGram Hosted Server's web service, I thought it would really straightforward to provide APIs in most (scripting) languages running on the JVM. HTTP + Basic Authentication + JSON seemed so en vogue... But clearly, it was harder than expected and the code had to depend on classes/modules that don't come with the core language or the standard library.

I strongly believe that JSON should be more widely supported (natively) by all the major scripting languages, much as XML is. Their own syntax for constant data structures (maps, strings, arrays) is so close to JSON that they should encourage people to use JSON instead of XML. Or at least not discourage its use.

Disclaimer: I am fairly new to most JVM-based scripting languages: Python, Ruby, Groovy. I may have missed something trivial. If so, please let me know.

Back to our regular program

2009-09-02T18:57:00.000-07:00

Wow! I'm back from vacation, summer is (almost) over, SpeechTEK 2009 is over, kids are back to school. Hope I'll finally have some spare time for small pet projects.

In particulary, I'll try to continue working on a small Erlang framework for developing communication applications that can be used across several channels: phone, IM, web. I used it for developing demos I gave at SpeechTEK.

The framework offers a simple synchronous API to write dialog-based applications, à la Tropo. In a matter of days (thanks to Erlang, Yaws, and exmpp), I have been able to make the framework

generate VoiceXML for the Voxeo platform,
interact with IMified and ejabberd,
do some outbound calling by interfacing to the Asterisk gateway interface, and
use NuGram Hosted Server for dynamic grammar generation and semantic interpretation of text-based interactions.

And a few more things. (I'll blog about this framework on our corporate blog in the upcoming weeks.)

I used the framework to develop a complete application for monitoring web sites. When a web site fails to answer, the application tries to reach me. It first tries to reach me on my preferred IM account. If I'm not available, it then tries to call me. If I don't answer, it sends me an SMS message. I also have a web interface that mimics an IM client.

This is, IMO, a very cool project. Unfortunately, most of the code I wrote has been completely obsoleted by some of the recent Voxeo announcements (here and here): their Prophecy platform and Tropo can now be used for text-based channels... Unless you want to run everything on your own server.

SchemeScript: git and wordpress

2009-07-07T04:37:00.000-07:00

In the last few days, I migrated the CVS repository of the SchemeScript plugin to git. We have started experimenting with git at work and I thought it would be a good idea to use it for my personal projects too.

Also, the web page for the project was old, ugly, and clearly not up-to-date. So I switched to Wordpress. This will ease the maintenance of the site. This is where I will post news announcements, describe features, try to create some buzz (!), etc.

I hope this is all for the best.

My first Tropo JavaScript application

2009-05-23T04:56:00.000-07:00

As promised in a previous post, I will show you the code of my first Tropo application, a simple voice application that gives me the status of the web sites I maintain and monitor.

Why such an application? Well, I have a Python script that pings my web sites every half hour (a simple HTTP GET request to the site's entry point). But sometimes the latency is so high that the script believes the site is down. So I receive an SMS on my cell phone. If I'm outside the office or home, I cannot check whether there is a real problem or not. So I decided to write a very simple voice application I could call from anywhere.

The VUI (voice user interface) of the application is simple: it asks me for a site name and, on a valid choice, makes an HTTP request to ping the site again. If the response code is 200, it tells me that the site is up and running, otherwise it tells me the site is down. The application loops and asks me for a site name again. At any time, I can ask for help by saying "what are my options" or "help me", or I can quit by either hanging up or say "quit".

The main application

The main part of the application begins with the initialization of a number of global variables:

var sitesUrl = "http://schemeway.dyndns.org/voiceapps/sites.json";
var sites = JSON.parse(http_request(sitesUrl, 'GET'));

var grammar = buildGrammar(sites);
var helpOption = sites.length;
var quitOption = sites.length + 1;

var helpPrompt = buildHelpPrompt(sites);

The sitesUrl variable holds the URL of a JSON document giving the names and URLs of the sites I can ping. The grammar variable holds a string representation of the grammar used for speech recognition (in a Tropo-specific syntax). Finally, helpPrompt holds a TTS (text-to-speech) string for the help message.

Once the variables are initialized, the application answers the call and play a welcome message (calls to the Tropo API are in bold):

answer();                                                                                                                                                                 
say("Hi! Welcome to the web site monitoring application.");

It then enters a loop that repeatedly asks for a site name and handles the outcome of the interaction:

var exit = false;
while (!exit) {
 ask("Choose a site.", {
   choices: grammar,
   maxTime: 10,
   onChoice: function(choice) {
     if (choice.value == helpOption) {
        say (helpPrompt);
        return;
     }
     else if (choice.value == quitOption)  {
        say("Thanks for calling!");
        exit = true;
        return;
     }

     var site = new Site(sites[choice.value]);
     say("The application " + site.name + " is ");

     var responseCode = site.test().status;
     if (responseCode == 200) {
       say("up and running.");
     }
     else {
       say("not reachable. The response code is " + responseCode + ".");
     }
   },

   onBadChoice: function() {
       say("Please try again.");
       say(helpPrompt);
   },

   onTimeout: function() {
       say(helpPrompt);
   }
 });
}

The ask function takes as its second argument a set of options, like the grammar, various timeouts, and callback functions. For example, when a valid choice has been made, the onChoice property is called with a single argument, the selected choice. The onBadChoice callback is called when the caller says something that is not covered by the grammar (a "no match"), while the onTimeout callback is called when the caller says nothing (what we usually call a "no input").

Some helper functions

The functions used to create the grammar and the help prompt are given here:

function buildGrammar(sites) {
  var grammar = "";
  var index = 0;
  for(index in sites) {
    var site = sites[index];
    grammar += (", " + index + "(" + index + ", " + site.name + ")");
  }
  grammar += (", " + (++index) + "(what are my options, help, help me)");
  grammar += (", " + (++index) + "(bye, quit)");
  grammar = grammar.substring(1);
  return grammar;
}

function buildHelpPrompt(sites) {
  var prompt = "Your options are: ";
  var index = 0;
  for(index in sites) {
    var site = sites[index];
    prompt += (site.name + ", ");
  }
  prompt += "help me, or quit.";
  return prompt;
}

Finally, the Site constructor function and the test property are given here:

function Site(obj) {
  this.name = obj.name;
  this.url = obj.url;
}

Site.prototype.test = function () {
  var connection = (new java.net.URL(this.url)).openConnection();
  connection.setConnectTimeout(3000);
  var responseCode = 500;
  try {
    responseCode = connection.getResponseCode();
    connection.disconnect();
  }
  catch (e) {
  }
  return {status: responseCode};
}

The whole source code can be accessed here. Feel free to comment, suggest enhancements, or even steal my code!

(2006-05-26 11AM EST) Update: My home internet access seems flacky today, so the above link may not work. Stay tuned, I am working on getting it back to normal.

(2006-05-26 1PM EST) Update: The link should be working now.

A new module system for Gambit-C

2009-05-12T06:12:00.000-07:00

Andrew Whaley commented on my post titled "Erlang or Gambit-C: A practitioner's perspective"

As a slight update to the situation, we now have the fantastic Black Hole module system for Gambit that includes both a continuations web server and a web client that addresses your point 8. Also I have recently released a pure Scheme mySQL interface for Gambit that addresses point 5.
So whilst it may not be Erlang yet - it's edging in the right direction.

Sounds promising!

Update: Black Hole is now on github: http://github.com/pereckerdal/blackhole/tree/master and the documentation can be found on the Gambit-C wiki.

VoicePHP and Tropo - first impressions

2009-05-11T22:00:00.000-07:00

In a previous post, I discussed a number of issues regarding the scripting approach taken by a number of new telephony platforms. But my opinions were somewhat uninformed, as they were based solely on the provided documentation and not on real experience with those platforms.

So I decided to try two of them: VoicePHP and Tropo. Here are my first impressions. (These are not necessarily profound thoughts, since I played with both platforms for a few hours each.)

VoicePHP

VoicePHP is a voice platform provided by TringMe, an Indian company. To use it you have to register for a 7 day beta period.

As its name implies, voice applications are written in PHP. You configure the script URL in your TringMe account and call the application. You have to host your scripts on your own web server.

To call the application, TringMe offers several choices:

Call a provided phone number in the US. (I reside in Canada, so it's a no go for me. I don't want to make long distance calls just to test small script changes.)
Call using a standard SIP phone. That's what I did. Works quite well.
Call the application using a Flash-based widget on a web page. That's really cool. Makes it very easy to develop a voice application and make it available to a large audience.

I tried to implement a small sample application making use of voice recognition. VoicePHP is supposed to support ABNF grammars. Unfortunately, all I got were "no match" events (or were they?). So I didn't get very far. Since there is no debugger or log files I can inspect, I quickly abandoned without knowing whether the fault was mine or not. The platform may be great for DTMF applications, but support for speech recognition did not convince me. And documentation is very light.

At least, their support team was quick at answering my questions. So I'm sure VoicePHP will get better over time. But for now, they did not convince me that the approach is superior to VoiceXML.

Tropo

Tropo is a voice platform in the cloud, provided by Voxeo, a Florida-based company. It's kind of a GoogleApp Engine for voice applications. It offers support for five scripting languages: PHP, Python, Ruby, JavaScript, and Groovy. Your applications can be hosted on their servers or you can serve them from your own web/application server.

You can add any number of applications to your account. Each application is assigned a separate Skype/SIP/INum phone number. As there is a local number for INum in Montreal, I can easily test my applications using either the PSTN or a SIP phone. Both for free.

To test the platform, I implemented a simple application that tells me the status of the web sites I maintain (I get SMS messages once in a while from a ping-like program telling me that one of my applications is down, but its usually due to unusual latency in the network. When this happens, I want a quick way to check if the web application is up and running.) The application asks me for the name of one of my web sites using speech recognition, and fetches the main page of the selected site. If the status code is 200, it tells me that the site is up and running, otherwise it tells me that the site is down. (I'll show you the code in my next post.)

I implemented the application in JavaScript, since it is the language among the supported ones I know best. Tropo uses Rhino, the Mozilla Java-based ECMAScript interpreter underneath, with support for E4X (ECMAScript for XML, an extension to ECMAScript that adds native support for XML).

In contrast to VoicePHP, Tropo provides a good web-based debugger and log viewer. That's an essential part of any development environment. It makes it easy to spot bugs and problems in your application. (I have not been able to stop the execution of an application, as advertised, though. But I have not contacted their support team yet to get help on this issue.)

Speech recognition works great, but my main complain is that the syntax for grammars is quite limited when you're used to developing grammars in ABNF or GrXML format. You cannot ask for dates or numbers, for example. Once they support those formats, sites like NuGram Hosted Server (shameless plug) will become usable to make great mashups with robust dynamic grammars. (I already have a working JavaScript API to generate dynamic grammars using NuGram Hosted Server that I have tested on Tropo. It's just that Tropo cannot use the generated grammars yet.)

Overall, developing a small application on Tropo was really fun!

Conclusion

Although both platforms are quite promising, I found Tropo to be much more mature than VoicePHP. Their website is well designed, effective, without too many bell and whistles. And they provide the right balance of features and tools to help test and debug applications.

So these were very first impressions on VoicePHP and Tropo. I know there are lot of issues I did not mention. I may address them in future posts.

So what is your experience with these platforms? Please share your thoughts in the comments below!

[Disclaimer: although Voxeo owns VoiceObjects, a Nu Echo partner, I based this very informal review solely on my experience using both platforms for a couple of hours each.]

Tropo on your local machine

2009-05-11T06:59:00.000-07:00

In my previous post (about scripting alternatives to VoiceXML), I wrote:

So I think the best of both worlds is to have a scripting language-based API that sits on top of a VoiceXML platform, with good support for speech recognition.

It seems that the Voxeo guys are already planning that with Tropo:

Yes, we fully plan on making the Tropo platform available for local installation. Unfortunately I do not have even a ball park guess to give you. There's a good bit of work for us to do in Tropo for a while to flesh it out properly.

VoiceXML vs the scripting approach

2009-05-05T07:47:00.000-07:00

[Disclaimer: I am partner at Nu Echo, but the opinions expressed here are mine only and do not necessarily reflect the opinions of my co-workers and partners.]

Fact of life: developing good IVR (interactive voice response) applications is hard. If it was easy, the GetHuman project would not have gotten so much attention in the past years. And more often than not, VoiceXML is blamed (not without some good reasons) for this. Just look at the number of new XML languages/scripting APIs out there that try to simplify the development of these applications: VoicePHP, Tropo, IfByPhone, Twilio, and many more.

But as Mark Headd pointed out, the problem is not so much with the language itself (VoiceXML), but rather with the fact that developing good interfaces (speech-enabled or visual) is difficult. Providing simpler interfaces will not automatically give us better applications. In fact, they make it more difficult to implement high performance voice user interfaces (VUIs) that take advantage of the speech-recognition technologies.

For example, most of these new APIs/languages, although they provide speech recognition, will not return multiple recognition results, with confidence scores and structured semantic results. That's a unfortunate because very clever dialog strategies can be implemented when you have such information:

confirming with the second hypothesis when the first has been rejected by the caller (unless its score is under a certain threshold, of course);
deciding whether the application accepts the answer, confirms it, or rejects it based on the confidence scores;
etc.

(But I may not get it, plain and simple. Are these new platforms targeting the large call centers or only the mass of web developers so they can experiment with telephony applications? And this is only the first generation of such APIs, they will certainly evolve over time and offer some more complex features.)

But although I think VoiceXML is here to stay in the call-center industry (too much investments to displace it anytime soon), I think a programmatic approach to IVR development is superior to a meta-language one:

The barrier to entry is lower in terms of development tools. One can reuse his preferred tools: editor, debugger, etc.
It is much easier to provide different implementations of the API, for unit testing purposes, integration with existing platforms.
The language's abstraction mechanisms (classes, higher-order functions, procedures, etc.) can be used to develop libraries of reusable dialog components.
There is a single language involved, instead of dealing with VoiceXML + ECMAScript + (PHP | JSP | ...).

So I think the best of both worlds is to have a scripting language-based API that sits on top of a VoiceXML platform, with good support for speech recognition. But that's a framework, right?

Or use a graphical VoiceXML service creation environment... VoiceObjects, OpenVXML, xMP Studio, Avaya Dialog Designer, Cisco Unified Call Studio, etc. etc. You'll certainly find one that meets your needs.

Gamerizon and QuantZ

2009-03-13T18:27:00.000-07:00

This morning, I paid a visit to the guys at Gamerizon (they have a number of jobs openings for seasoned Schemers). They gave me a demo of QuantZ, their flagship product. Wow! That is an amazing game! I'm not sure how much I can tell you, but just go to their website and look at the screencast. (In fact,

So if you are looking for a challenging Scheme job, send your CV. It's a strong, talented team (they have a lot of experience in game development), and they are highly passionate about their work and Scheme in general. I have a lot of respect for what they do. And it's nice to see places where Scheme is endorsed first and foremost by the founders/company executives.

Scheme job openings at Gamerizon

2009-03-11T12:46:00.000-07:00

Just seen on the MSLUG mailing list, a message from Robert Lizée about 2 openings at Gamerizon:

We are currently finishing the development of a game called "QuantZ" that has been mostly written in Scheme using Gambit-C. A video of the game can be seen at www.gamerizon.com. The current version of the game runs on Windows and Mac OS.

In the short term, we are looking for 2 developers: a game programmer with good optimization skills and a Scheme programmer with good understanding of compiling techniques to work on the iPhone version and smaller versions of the game in Flash, Java and BREW. For these versions, the objective would be to reuse/adapt parts of the current code base, in order to gradually implement a system on which we could develop games on many platforms at once using the same code base.

Tropo - a new platform for developing speech applications

2009-03-11T03:58:00.001-07:00

Last week, at eComm, Voxeo has launched Tropo, a platform for developing speech applications in a variety of dynamic programming languages. They provide a (synchronous) API to interact with the callers of your application. You can do things like playing prompts, asking questions (DTMF/speech recognition), transfering calls, and so on. All this in your language of choice, without having to mess with VoiceXML, CCXML, low level telephone APIS, and the like.

Tropo currently supports JavaScript, Groovy, Ruby, Python, and PHP. They expect to add support for new languages in the future. And if you follow my Twitter feed, you noticed that they are open to supporting Scheme/Lisp. I even volunteered to help them. What cool guys they are! (It seems I am not the only one who asked about Lisp...)

Now, which Lisp dialect would make most sense on the Tropo platform? Given the state of the Scheme community and the variety of implementations, the answer may not be as simple as you would expect. Here are a few possibilities, with their respective pros and cons.

JVM-based systems

First, there are a number of Lisp implementations on the Java platform: SISC, Kawa, and Clojure. With these implementations, their foreign interface to Java compensates for their lack of libraries.

SISC is a fully R5RS-compliant implementation of Scheme. It can interface to Java, optimizes all tail-calls, fully supports call/cc, etc. It is a mature implementation, but there has not been any new release since the end of 2007. Unfortunately, it does not implement optional and keyword arguments, and does not have a compact syntax for hash maps. I am not sure how elegant code will be in SISC compared to what we can do in JavaScript or the other scripting languages.

Kawa is a less compliant Scheme implementation than SISC and does not seem to have a lot of community support. It does have DSSSL-based optional and key arguments, which is nice, but it does not provide a clean, compact syntax for hash maps natively. But since the reader can be extended, this is not really a big deal.

Clojure, the newest Lisp on the block, may be a good choice. I am not really familiar with it, but from what I saw in the documentation, it supports objects, multi-methods, special syntax for hash maps, optional and keyword arguments, and much more. It does not optimizes tail-calls but, hey, it's a Lisp, not a Scheme. And there seems to have a strong community behind it.

Native implementations

Now, if we now turn to native implementations of Scheme (i.e. not running on the JVM), there are many more choices. I won't list them all, but if we consider the size of their respective community, the clear choices (from my point of view) are PLT-Scheme, Gambit-C, and Chicken. As you know, I have a preference for Gambit-C. It is a robust, fast implementation of Scheme (R5RS). It has very good debugging tools, supports DSSSL-based optional and keyword arguments. The readtable can be modified. Etc. Etc. Add Termite and you have a strong platform for developing amazing next-generation multi-modal voice-enabled applications. But on the down side, it cruelly lacks libraries.

Of course, PLT-Scheme and Chicken do have a lot of libraries, with a language support for using them. Chicken has most of the features of Gambit-C, except good debugging tools (and Termite, of course, which only runs on Gambit-C). PLT-Scheme, on the other hand, does not have runtime debugging tools as good as Gambit-C, but it comes with a great development environment, DrScheme.

And the winner is...

Well, I don't know. From a programmer point of view, I would hesitate between Clojure and Gambit-C. But there may be some other issues that I did not consider, like the architecture of the Tropo platform, the interpreter start time, memory footprint, etc. that would affect how the application could scale. Is there a new interpreter launched for each new call?

So tell me, what would be your ideal Scheme/Lisp implementation on the Tropo platform?

On the use of parser/lexer generators

2009-02-22T17:43:00.000-08:00

Today, I came across a blog post on Advogato about the use of parser and lexer generators. The author makes a case about writing hand-crafted lexers (in his case, a lexer for XML). As the maintainer of a LALR(1) parser generator for Scheme, I could not resist and wanted to share my thoughts on this subject.

First and foremost, let me say I have mixed feelings about the article. Although it addresses important issues, it concludes with a very simple lexing problem, not a challenging one. I think this alone greatly diminishes the strength of his argument.

Now let me give my own list of pros and cons of parser/lexer generators.

Pros

I am a big fan of DSLs. And parser/lexer generators obviously fall in this category of tools. They operate on a high level, declarative description of the language we wish to parse. This results in more maintainable software, with less bugs (most popular generators have been used extensively, so you can already be confident on the correctness of generated code).

At Nu Echo, I wrote two parsers for a language called ABNF (it's a language for defining grammars used in speech recognition applications), one using an LALR(1) parser generator and the other one being an hand-crafted recursive-descent parser. The latter is certainly faster and give better error diagnostics (I'll get back to this in a moment), but from a maintenance point of view, the former wins.

Another important aspect of parser generators is that they often support different parsing technologies/drivers. For instance, bison can generate LALR(1) or GLR parser (lalr-scm can, too). And ANTLR supports LL(1) but also LL(k) grammars. Without having to rewrite the source grammar. You can't do that with an hand-crafted parser. (Of course, you must stay in the same "family" of parsing technologies. A grammars written for an LR parser generator may not be appropriate for an LL parser generator, and vice versa.)

A corollary to this is that you can concentrate on offering the best interface to your users: the language itself. You don't have to adapt the language to a parsing technique. This is very important from a design point of view. Usability is (or should be?) one of the driving goals when designing a new language.

Cons

Unfortunately, they are more pragmatic issues that can preclude the use of parser/lexer generators. Here are a few:

My experience with parser generators showed me that they are nice for command-line tools, but they do not produce good parsers for use in interactive environments (like an editor). (I must confess that I have almost exclusively used LR parser generators a la yacc/bison, so this argument may be a bit stretched. And I know that there have been quite a lot of work on interactive programming environments and incremental parsing.) Error recovery mechanisms are often limited and tend to produce cryptic error messages.

Hand-crafted parsers are often much easier to debug. Ever tried to understand what a shift/reduce conflict is? And tried to figure out how to rewrite your grammar to resolve it? You have to understand the parsing technology. Talk about an abstraction leak!

Generated parsers are derived resources in a development project (i.e. resources obtained from other resources). This means they are usually not kept in the CVS/git/SVN/... repository. This can complicate the build process. This may not be a big deal, but in some projects this can be trickier.

In some languages other than C/C++/Java, like Scheme, this may not be a problem since the parser generator can be called at macro-expansion time.

Et voilà! I'm surely missing a lot of other equally important aspects, but it's getting late and I am tired. And you? What are you reasons for using or not using a parser generator/lexer generator?

Nils Holm's textbooks for free

2009-02-22T15:00:00.000-08:00

Nils Holm is offering most of his text books online free of charge. I have not read any of them yet, but judging from their title, I'm definitely missing something!

Reflection can be costly

2009-01-28T06:15:00.000-08:00

At least in Java.

A few days ago, I wanted to optimize a piece of code written in Kawa Scheme. You know how that works: you write something that seems to be good enough in most cases in terms of performance. Then you try it on a large input (a file several several megabytes long). And your code chokes. It does not crash, but takes way too much time to be usable in practice. But that's ok. Premature optimization is the root of all evil.

So I ran the profiler on my code to find out that it was using the Java reflection routines like hell. I was astonished, to say the least. My code was carefully written, with lots of type declarations (which are optional in Kawa) to prevent calls to the reflection API by the Kawa runtime. It seems that I was not careful enough. Or that I was expecting more from the Kawa compiler (I use a very old release, btw).

The problem was this. In a single static Kawa module, I was defining some classes and a few helper functions. Something along the lines of:


(module-name <mymodule>)
(module-static 'init-run)

(define-simple-class <ClassA> (<Object>)
  (some-field))

(define (helper-function (an-a :: <ClassA>))
  (do-something-with (field an-a 'some-field)))

Surprisingly, the reference to the some-field must be done through reflection! I'm not sure why, I don't master the inwards of the Kawa compiler.

Is there a solution to this problem? Yes. Put the class definitions in a module an the helper functions in a separate one. By doing this, the run time of my code dropped by 50% (a speedup of 2). There are many other possible improvements to speedup my code even further (data structures, algorithms, etc). But at least my code is now compiled the way I expect it to be.

Extended desktops and Compiz

2009-01-12T06:53:00.000-08:00

This morning, I realized that Compiz was not working anymore. I could not remember when it stopped working, but I remembered having a similar problem a month ago, when using my Wubi-based install of Ubuntu.

After a few minutes of googling around, I found that this probably happened after connecting a projector (or another monitor) on my laptop. If that happens again, I just need to issue the following command:

  % sudo dpkg-reconfigure -phigh xserver-xorg

and restart X (by just logging out and in).

A case for voice test cases

2009-01-07T06:40:00.000-08:00

To me, the future of voice service creation environments (SCEs) is in IDEs that will offer ways to test applications without having to resort on a VoiceXML platform, that will ease the creation of test cases. Testing voice applications manually by calling the application is costly (in both time and money) and the more you can do in your development environment, the better.

Think about how easy it is to run JUnit tests from Eclipse. In a fraction of a second, you can run hundreds of test cases. And you can use the power of Java to factor large portions of your tests and reuse lots of code. Doing the same for voice applications would be a killer app. (Of course, you can certainly do that when using a well-designed custom VoiceXML framework with a strong underlying model. But most SCEs have not been designed with this in mind.)

In a tough economy, environments that will help us deliver better and more robust applications in a more cost-effective way will clearly have a competitive advantage.

At least, some companies understand that.

How I switched to Ubuntu

2008-12-18T03:34:00.000-08:00

Yesterday, I decided to install Ubuntu on it own dedicated, ext3 partition, to get full performance (sometimes, on my Wubi-based install, a process called mount.ntfs takes 00% CPU and freezes most applications).

But prior to installing Ubuntu, I first had to shrink my Windows (ntfs) partition. Here is what I did, only using free software:

Removed unnecessary data to from Windows (I removed about 14 Gb of junk!)
Made a backup of my Windows partition. I first tried ghost but our corporate version is old and I did not succeed in detecting my Samba shares on my home server (CentOS). So I dowloaded and installed Macrium Reflect Free Edition. Great software. Run flawlessly.
I shrunk my Windows partition. To do this, I used EASUS Partition Manager. Again, this worked flawlessly.

At 10h30PM, I finally realized I didn't have any blank CD to write my Ubuntu installation CD! So I decided to use my USB key. To write the ISO file to my USB key, I used UNetbootin. It was very simple to use and I was able to boot the Ubuntu installed just a few minutes later.

That's it. Now I run Ubuntu natively. It's so fast, now!

A new fan of Ubuntu

2008-12-15T19:52:00.000-08:00

I installed Ubuntu (using the Wubi installer) on my Dell Latitude last week. It just rocks!

I was still under the impression that installing and configuring a Linux system was not an easy task (I have been a Linux user since the early '90s, I installed CentOS 5 just a few weeks ago on a server at home). I was just plain wrong. Installing Ubuntu 8.10 took just matter of minutes. Configuring the internet (wireless access + openvpn) is as intuitive as it can be.

Moreover, In the past couple of years, it took me hours to configure my HP printer on Windows XP. With Ubuntu, all I had to do was inserting the USB cable in my laptop et voilà!.

I'm sold to Ubuntu!

Now if I can take the time to ghost my machine and install Ubuntu on a native ext3 filesystem... I would unleash the full potential of my laptop (Wubi installs Ubuntu on a virtual drive, a file on an ntfs partition.).

VoiceObjects acquired by Voxeo

2008-12-10T09:41:00.000-08:00

Voxeo, a company I mentioned a few times in the past, announced yesterday the acquisition of VoiceObjects, a company located in Cologne, Germany, with which Nu Echo has developed a great relationship in the past eighteen months.

I'm very excited by this acquisition. This will certainly create new opportunities for VoiceObjects in the US, and Voxeo will extend its reach on the old continent.

But more importantly, I'm very happy for the all the people I know at VoiceObjects. I have a lot of respect for their work, their attitude, and their professionalism. Voxeo not only acquired a great technology and platform, they have joined their forces to a great team.

Jazz night at the MSLUG

2008-11-12T16:19:00.000-08:00

I just added a new meeting announcement to the next MSLUG website, to be held on November 27th. Here is an excerpt:

As some of you know, Guillaume Cartier has been working for the past 12 years on his Jazz Scheme system. It is a powerful and complete development environment for Scheme applications. You can read about it here: www.jazzscheme.org.

A couple of years ago, he decided to port the whole Jazz kernel to Gambit-C Scheme and to eventually release Jazz under an Open Source License. With the near completion of X11 support, the time to do a full public release is coming very close.

In order to help spot and iron out the remaining issues for a full public release, Guillaume will be doing a "Closed Beta" release to the participants of the MSLUG. The release will take place at the MSLUG meeting on November 27th. You are encouraged to bring your own laptop, as Guillaume will walk us through the installation of Jazz and the creation of a simple application. Note that Jazz should run on Windows *and* on systems featuring an X11 server.

Thanks to Marc Feeley, WiFi Internet access will be available at the meeting, which should help with the whole process.

Don't miss it!

Erlang impressions (in French)

2008-09-24T18:53:00.000-07:00

François Pinard, a colleague of mine, reveals his first impressions on Erlang. François has worked on Erlang stuff for the past 4 months and explored a lot of dark corners of both Erlang and Yaws. The only caveat: it's in French.

Erlang @ Yahoo

2008-09-17T16:23:00.000-07:00

Too bad I cannot make it... Mike Gerakines will talk about the use of Erlang at Yahoo at the BayFP. Hope the slides will be available afterward...

Hot code swapping - take 2

2008-09-12T06:16:00.000-07:00

This is a follow up on my previous article, in which I described a few problems with hot code swapping in Erlang.

In a comment, Philip Robinson wrote:

It sounds like you have another (lost?) process running code from that module, running version 1, that never receives the switch_code message.

And that was exactly the cause of my problem. We were having a process leakage. The switch_code message was only sent to processes that we were tracking, not all processes belonging to the loaded module. Unfortunately, we were not careful enough and some of these processes were lost in the process of switching code, preventing future code swapping.

So we have no need for an after clause anymore. And I think that if you need such a hack, then you may not be properly managing your processes. At least, send the switch_code message to all the processes for a given module:


  lists:foreach(fun(Pid) <- Pid ! switch_code end,
                [Pid || Pid <- processes(), erlang:check_process_code(Pid, Module)])

Hot code swapping pitfalls in Erlang

2008-09-11T17:36:00.001-07:00

Hot code swapping is a very nice feature of Erlang. (Please, don't bother commenting that this can be done in other languages too... I already know.) But there is little documentation about it, making everybody believe that it is trivial to support in every application. François and I had a hard time today trying to get hot code swapping run smoothly in one of our applications.

We basically had two problems. One we found quite easily (although this was not documented anywhere). The other one is still mystifying us, although we found a work-around that we don't completely understand. Let me discuss them in turn.

try-catch and tail calls

When you write a process loop function, it must be tail-recursive in order to avoid memory leaks. So you typically write something like:

server_loop(State) ->
 receive
   SomeMessage ->
     ...
     server_loop(NewState);
   ...
 end.

As you can see, the call to server_loop is called in tail-recursive position. This is equivalent to a GOTO with parameters. This call does not consume any stack space. To make this server support hot code swapping, we just need to add a clause for a special message, and fully qualify the recursive call with the module name:

server_loop(State) ->
 receive
   switch_code ->
     ?MODULE:server_loop(State);
   SomeMessage ->
     ...
     server_loop(NewState);
   ...
 end.

So you have to make sure that all the recursive calls to the server loop have to be in tail-call position. In our application, we had something like that:

     try
      NewState = doSomething(State),
      server_loop(NewState)
   catch
      Error ->
        server_loop(State)
   end

We assumed that both calls to server_loop were in tail-call position. We were wrong. Of course, the try-catch form installs an exception handler for the whole dynamic extent of its body statements. Which means that some stack space is consumed, and probably with references to the current code. Even if we add support for the switch_code message, this process will not switch to the new code and will eventually be killed.

But this was easy to fix, once we figured all this out.

Use of an after clause

The other problem we had, we simply fixed it without understanding why. (If you know, please leave me a comment!!) We have a very simple process loop, coded exactly as I explained above. Unfortunately, the process get killed when we reload the code 3 times. Even if we send the switch_code message.

The fix, suggested here but without a satisfactory explanation, consisted in adding a after clause to the receive statement:

server_loop(State) ->
 receive
   switch_code ->
     ?MODULE:server_loop(State);
   SomeMessage ->
     ...
     server_loop(NewState);
   ...
 after
    10 * 1000 ->
      ?MODULE:server_loop(State)
 end.

Now we can update our whole application without service interruption. But this was not as simple as it first seemed (and advertised ;-).

Update:I posted a follow-up explaining the problem we had.

Exceptional (and free) technical support

2008-09-01T18:07:00.000-07:00

For one of the demos I did at SpeechTEK two weeks ago, I had to use Voxeo Prophecy. Unfortunately, only a few days before I got to vacation, I found that my installation was broken, and I didn't know why. I reinstalled the software a few times with no more success.

I then contacted them on their technical support forums. What a great customer support experience it was! They were very responsive and pinpointed the problem in a matter of a few posts. But the most interesting thing is that it was completely free, for a free product. If at least we could get that level of support for most paying products... These guys are simpy amazing.

If you want to experiment with VoiceXML, VoIP, and voice applications, go here. (It's just sad they don't support Canadian French... ;-)

Our corporate blog

2008-09-01T17:08:00.000-07:00

At Nu Echo, we just launched our corporate blog. I will post articles there on a regular basis (in fact probably more often than here, unfortunately) as I will be one of the main contributors. I hope I will have the opportunity to describe (here) our use of some non-mainstream languages (I let you guess which ones ;-), our experience with interns learning and using them, and how they are perceived in the market place (as well as the show-stoppers, of course). I have learned a lot in the past months. Stay tuned!

Back to blogging!

2008-08-24T18:56:00.000-07:00

Wow! It's been 4 months since my last post. I've been really, really busy. But there's no excuse. I should have blogged a little, at least. It's just that now that I am a partner at Nu Echo, I have more things to do. Not only that, Nu Echo has released a new line of products during the SpeechTEK conference earlier this week, the NuGram platform. I worked like crazy in the last 3 months. (I even worked during my family vacations...)

I hope I'll be able to blog on a more regular basis in the upcoming weeks, months. Probably on subjects other than Scheme (Erlang, startups, etc.), but not that far from the original spirit of this blog. (What is it, btw?) But I will also contribute to our corporate blog as well. So stay tuned.

REST support in Voxeo Prophecy

2008-04-22T18:24:00.000-07:00

One of the nice additions of VoiceXML 2.1 to the 2.0 version of the standard is the ability to query XML data on other HTTP servers by means of the <data> element. The specification dictates support for the GET or POST HTTP methods. But it says nothing about the other methods (DELETE, PUT, etc.). This is rather limitating, IMO (and strange, given the fact that it was published last year).

Last night, I wanted to start playing with this feature on the free, hosted Voxeo Prophecy platform. And I found that the DELETE method was also supported. I did not try the PUT method, but I guess it is supported too. I really think they did a great move implementing support for all the HTTP methods and not limit the platform to the plain VoiceXML standard. This way, they enable RESTful applications to be accessed/used directly from VoiceXML applications.

lalr-scm on Google code

2008-04-13T18:10:00.000-07:00

I've just put lalr-scm, my LALR(1) parser generator for Scheme, on Google Code. The project can be found here. The documentation is in the associated wiki.

Arc support in SchemeScript - update

2008-02-05T06:41:00.001-08:00

I've just released a new version of SchemeScript. It's still an alpha release.

The main feature of this new version is an optional support for Arc. To activate it (this will be easier in a future release), unzip the dowloaded file in your Eclipse main directory. Then open $(eclipse_home)/plugins/org.schemeway.plugins.schemescript_1.3.0_alpha3/conf/user.scm and uncomment the last line (the one that loads "languages/arc.scm"). Restart Eclipse.

If you have any problem, let me know. Note that it may not support all Arc primitive forms and functions. But if you put all the Arc implementations files in your workspace, you'll have access to all of them both for code completion and content assist. I think this makes SchemeScript a pretty neat IDE for Arc.

Finally, note that I did not work a lot to optimize the indentation of all Arc special forms. Any help will be greatly appreciated!

Arc support in SchemeScript

2008-02-04T19:32:00.000-08:00

Although I do not agree with Arc's design goals (more on this in an upcoming post), I have decided to give it a try. As I usually say to my kids: you never know whether you don't like something unless you tried it first.

So I do downloaded it this evening and installed it.

Of course, since my main development environment for Scheme is my own Eclipse plugin, SchemeScript, my first step was to add support for Arc (like adding proper indentation rules, support for most special forms, support for code assist - proposals, completion, etc.):

If there is enough interest, I can package something and make it available. Just let me know.

Update: I just released a new version of SchemeScript that supports Arc.

Recruiting interns

2008-02-03T17:13:00.000-08:00

It's that time of the year. For the second year in a row, Nü Echo is recruiting interns for next summer. We mainly advertise our projects at École Polytechnique, an engineering school here in Montréal. Since I am deeply involved in the whole recruiting process, I came to a number rules that proved very beneficial.

First, let me set the context. Recruiting interns is not easy in Montreal. (I guess this is true in all major cities.) There are five times more internship projects than there are students competing for them. So competition is tough. Moreover, most co-op programs favor students in the recruiting process (this is not the case at École Polytechnique). Add to that the fact that Nü Echo is a small, not really well-known company among students (we are well-known in the call center and telecommunications industries, though)...

Despite these pretty adverse conditions, last year's recruiting went really well, with great results at the end of summer. We filled all our positions and all students exceeded our expectations. And this year promises to be even better. We will have to decline great students.

To achieve these results, I try to apply the following rules:

Offer exciting projects. Most companies consider interns as cheap resources and ask them to do the dirty, uninteresting stuff. Like coding test cases, running the tests, etc.

When elaborating an internship project, I always ask myself: would I apply for the job? To me, the best project is self-contained, with a right balance between analysis, design, coding, testing. It shall also involve leading-edge technologies and be a project that will matter to the company. Not something that will be put on the shelf in September.
Be quick! As soon as you receive the students' resumes, screen them and schedule interviews. And if the interview goes well, make an offer. At École Polytechnique, students have only 24h to accept/decline an offer. This can turn to your advantage. Bigger HR departments usually move more slowly, so a 2nd year student will accept a nice project early if they are not certain of being offered a better one later during the semester.
Supervise them adequately. This should go without saying, but it is not always the case. Summer is usually the time of the year when regular employees and managers take their vacations. It may be difficult to supervise your students. But believe me. If you are not gonna be able to supervise them adequately, don't offer internship projects.

Of course, we expect our students to be rather automonous, but don't forget that they are students, they are with us to learn. I've seen companies where students were left almost unsupervised, not really knowing what to do. They waste time and money.
Treat them like regular employees. You want them to come back next year. So make them feel that they are part of the team, that they bring value to the company. They are your future.
Hire both 2nd year and 3rd students. The 3rd year students (given they are on a 4 years program) can make good candidates for your future job openings.

The 2nd year students, even if they won't work for you during their 3rd year summer, will help cultivate the reputation of your company at school. They will talk about you. This is akin to viral marketing.

Internship projects, when done right, are a great way to advertise your company. Not to potential customers (although this could be), but to your future employees. Students will return to school at the end of the summer and will talk about your company if they appreciated their experience working with you. And good students usually attract other good students. This could definitely help your whole recruiting process.

Voxeo Prophecy

2008-02-02T06:21:00.001-08:00

Want to experiment with speech technologies, VoiceXML, VoIP? Give Prophecy a try. It's Voxeo's complete standards-based IVR solution. You can download a free, fully-functional, copy with decent speech recognition and text-to-speech (US english only), a SIP phone, a complete web-based application designer, and more. It supports VoiceXML 2.0, SRGS/SISR, CCXML, SSML, and MRCP. The free version supports 2 callers at once (2 ports in telephony jargon).

I downloaded it two days ago and it installed out of the box and worked right away. I was deeply impressed!

They also have a number of corporate blogs, and they are very responsive.

The only drawback is that they don't support Canadian French (unless you buy the Nuance speech recognizer... but it's not cheap). I would have used it at home otherwise.

Renaming a symbol globally

2008-01-04T19:24:00.000-08:00

A while ago, I implemented a feature in SchemeScript that proved quite useful today: an action that renames a symbol in all the Scheme source files in the workspace.

To do that, simply put the cursor on the symbol and press Alt-Shift-R G. The following dialog box will be shown:

Press Ok to accept the new symbol name. All the files containing the modified references will be automatically saved (and this cannot be undone, so be warned!).

This features is useful when you want to rename a function or constant globally, or (in my case) when you want to rename a module name and make sure all the importing modules are propertly updated. Of course, this is not a semantic-preserving transformation. It's only lexical. But nonetheless it proves useful once in a while.

Note that two other renaming actions exist:

rename symbol in file: this action will rename all the symbol references, but only in the same file (Alt-Shift-R F);
rename symbol locally: this action will rename the symbol and its references in the same top-level definition (Alt-Shift-R L).

This feature is only available in the alpha version that can be downloaded from the SourceForge download section.

Experiments with IM client and server

2008-01-02T16:32:00.001-08:00

Yesterday, I installed Openfire, a Java-based XMPP server from Ignite Realtime. I also installed their XMPP client, Spark.

My first impression is very good. Both softwares were installed and made operational in a matter of seconds (really!). The server can be easily configured using a web-based interface. And it ships with hsql for light installs not requiring high volume or scalability.

Ignite Realtime also provides a Java library for developing XMPP clients, Smack. And the documentation that comes with the API is very good.

Of course, the first thing I did once everything was up and running was to develop a small client application in Kawa Scheme. The client is a very simple-minded bot that can evaluate Scheme expressions. Here is a screen shot:

It took about 45 lines of code to achieve that. And most of the code is declaring Kawa namespaces for Java/Scheme integration, and processing a message (converting the body to a string, reading the expression, evaluating it, handling errors, etc.).

I don't know how long it would have taken me to write the same thing directly in Scheme, without the Smack library. Handling the XMPP stream is not trivial at all. One must be able to recover from many error situations and I'm not sure if the current XML libraries in Scheme (SSAX-SXML or the ones that come with PLT-Scheme) would be usable at all. Maybe. I don't know. If you have opinions on this, please share them with me.

Happy New Year

2008-01-02T16:24:00.000-08:00

I wish you all a very happy new year!

I've been quite busy since the start of November, explaining the lack of interesting posts on this blog. I'm working on an internal project at Nü Echo that takes all my time. I will blog about it eventually, since it is all done in Scheme. (No, it does not involve Gambit-C nor Termite, only Kawa. But it's worth it anyway ;-) I may also give demos of the product at local DemoCamp/BarCamp events, here in Montreal, in a near future. Stay tuned!

MSLUG meeting next week

2007-12-01T08:13:00.001-08:00

The next meeting of the MSLUG will be next Wednesday, December 5th. Four talks are scheduled and they will all cover the same subject, from different angles: the commercial use of Scheme/Lisp. I will talk about our experience using Scheme at Nü Echo, and in particular what pitfalls to avoid when introducing a non-mainstream language in the enterprise.

A new release of SchemeScript!

2007-11-16T09:53:00.001-08:00

SchemeScript 1.3.0 alpha 1 is out! This is the first release in more than a year now.

The new noteworthy features are:

Precise location of symbol definitions. All the symbol dictionary stuff has been reimplemented from scratch, and now mainly in Scheme. The outline now better reflects the structure of the code.
Support for hyperlinks to jump to a symbol definition.
Support for #! on the first line of a Scheme file (typically used for Scheme shell scripts).
Punctuation tokens are now colorized independently.
Support for Snow package* and R6RS library forms.
[] are treated like ().

Of course, a few bugs have been fixed.

Note: Since it is only an alpha version, a few things are lacking, especially regarding the documentation.

Video of Manuel Serrano's talk on HOP

2007-11-08T09:01:00.000-08:00

A video of Manuel Serrano's talk at the MSLUG is available on Google Video, thanks to Daniel Caune.

Update: the link above has been modified. The previous one was broken.

Domain-specific languages and GUIs

2007-11-06T17:59:00.000-08:00

Phil Windley has a nice post on DSLs (found via Patrick Logan). I share most of his thoughts on the subject, except regarding GUIs.

I think that when they are well designed, GUIs can be a good substitute for the DSL itself. GUIs can be just another way of representing the same program but in a different form. They can even provide insights or different views on the program that may not be easily visualized by looking at the program text itself. Of course, this assumes that the GUI can give access to the full power of the underlying language and not be a pale substitute.

That's why designing GUIs for DSLs can be really challenging. And fun!

A new beginning

2007-11-02T18:42:00.000-07:00

Today, I resigned as the main organizer of the MSLUG. Here is the message I sent to the mailing-list:

Guys,

the last few months have been very exciting. Our group has reached a critical mass and has now a lot of visibility. Many people have joined our group and we see more and more interest in Scheme/Lisp (as well as Erlang) in Montreal.

Unfortunately, I have to resign as the main organizer. This was a tough decision. But I am currently working on other projects that require all of my spare time. I hate not being able to give the best I can on something. And I clearly don't have enough time to organize as good meetings as I'd like. And our group definitely deserves the best meetings.

But I am please to announce that Guillaume Germain will, as of today, become the main organizer of the MSLUG. I am totally convinced that Guillaume will make a better job than I did. Thanks Guillaume!

Thank you everyone for making the MSLUG one of the best Scheme/Lisp user group on the planet. All the effort was really worth it! Thanks especially to Marc Feeley who helped me organize the meetings.

See you soon!

Dominique

P.S. Don't worry! I will still program (even professionally) in Scheme and attend the MSLUG meetings, although not necessarily as regularly as I used to.

What a meeting!

2007-10-25T05:48:00.000-07:00

All I can say is that our last meeting of the MSLUG was a true success! We had more than 60 attendees. Of course, since we held it during the OOPSLA conference, it was much easier to attract people that would not have come otherwise (like Mark Miller from Google, the Smalltalkers from Cincom, Erick Gallesio from Nice University, Shriram Krishnamurti from Brown University, Bill Clementson, and the list goes on). Nonetheless, some came only to attend the MSLUG meeting, like Xach Beane from Maine, USA, and Slava Pestov from Ottawa, Canada.

The meeting began at 7h45PM and lasted more than 2 hours. One third of the attendees then joined us at the pub.

Both speakers really entertained their audience and they got a lot of insightful questions and remarks. Manuel Serrano presented its HOP system, a programming language designed to help write interactive multimedia applications. Then Pascal Costanza presented ContextL, his extension to Common Lisp for context-oriented programming. Daniel Caune videotaped the first part and it should be available soon on Google Video.

François Magnan was responsible for finding a conference room at UQAM and did a very good job. It was a really nice place and not too far from the Convention center. François also wrote a nice writeup. Thank you François!

So thank you all for attending the meeting and making it a great event!

Montreal Guide for hackers

2007-10-19T18:04:00.000-07:00

If you come to Montreal next week for OOPSLA and the MSLUG (you can't miss that great event ;-), make sure to visit Yannick Gingras' Guide to Montréal for Hackers.

For those who won't be there

2007-10-19T18:00:00.000-07:00

For those of you who won't be able to make it, our next meeting of the MSLUG, next Monday, will be videotaped and I will make it available on one of the popular video sites. Stay tuned!

Why attend OOPSLA'2007

2007-10-08T07:02:00.000-07:00

[Message from Richard Gabriel, chair OOPSLA'2007]

"The brightest minds in computer science will be at ooPSLA 2007, convening in chic, sophisticated, metropolitan Montreal, Quebec. Peter Turchi, author of "Maps of the Imagination," will use examples from writing and cartography to explore the challenges of discovery, the challenges of presenting those discoveries, and how the presentation itself is often the key to discovery (think Impressionism). Jim Purbrick and Mark Lentczner, also known as Babbage Linden and Zero Linden, will step away from reality into the virtual world of Second Life. Two Turing Award winners, Fred Brooks and John McCarthy, will be giving talks. Brooks will talk about collaboration and telecollaboration in design.
McCarthy will be presenting Elephant 2000, a proposed programming language good for writing and verifying programs that interact with people (e.g., transaction processing) or interact with programs belonging to other organizations. Gregor Kiczales will deliver a talk on how different contexts affect developer perspectives on software. David Parnas examines the problem of documenting the behavior of systems and their components, and how precise documents can make validation and verification easier. Patti Maes, honored with the title "Global Leader for Tomorrow" by the World Economic Forum, will be speaking from her background in media arts and sciences. Also planned is a special reprise of Gabriel and Steele's keynote "50 in 50" from HOPL III. OOPSLA 2007 runs October 21 through 25"

But, I would highlight this:

Fred Brooks, Turing Award winner, Mythical Man Month, No Silver Bullet. He doesn't agree often to speak.

John McCarthy, Turing Award winner, Lisp, AI. He doesn't speak often, and he's getting old.

David Parnas, very early guy in software engineering, modularity, information hiding.
Ornery. Doesn't speak much.

Jim Purbrick & Mark Lentczner, these are the Second Life guys.

Gregor Kiczales, Aspects, metaobjects, CLOS. Very good speaker. One of our most important thinkers.

Pattie Maes, reflection, ambient intelligence.

Guy Steele, Scheme, Fortress.

Brian Foote, Big Ball of Mud, one of the most entertaining speakers in computer science.

Richard Gabriel, Worse is Better

special surprise keynote:

Kathy Sierra, of creating passionate users fame.

Most conferences would consider themselves lucky to see any one of these people at their conference (except for me, of course). To have all of them is extraordinary. This is probably the best lineup of speakers at any CS conference in history.

October meeting of the MSLUG - the details

2007-10-04T17:26:00.000-07:00

The MSLUG is a group developers of the Greater Montreal interested in the Scheme/Lisp family of programming languages and in functional programming in general.

The October meeting will take place during the OOPSLA 2007 conference. For this special event, the MSLUG invited two well-known speakers: Manuel Serrano and Pascal Costanza. Don't miss this exceptional event!

Where: UQAM, 100 Sherbrooke West, Room SU-1550
When: October 22nd, 2007 at 7h30PM

Manuel Serrano, INRIA Sophia-Antipolis

Manuel Serrano holds a Ph.D. degree from the Université Pierre et Marie Curie (Paris 6), France, and works at INRIA Sophia-Antipolis as a research director. Most of his research activities are based, or use, the functional programming language Scheme. He is well-known for his highly-optimizing Scheme compiler, Bigloo. Manuel Serrano served on several international conferences: ICFP, JFLA, Scheme workshops, and more.

Title: Programming Multimedia Applications on the Web with HOP

Hop is a language dedicated to programming reactive and dynamic applications for the web, such as web agendas, web galleries, web mail clients, etc. In this presentation, we highlight the linguistic novelties introduced by Hop and its execution environment by describing Hop's user libraries, its extensions to the HTML-based standards, and its execution platform, the Hop web broker. The presentation will focus on presenting the development on multimedia applications with Hop. There will be several live demonstrations during the presentation.

Pascal Costanza, Vrije Universiteit Brussel.

Pascal Costanza has a Ph.D. degree from the University of Bonn, Germany, and works as a research assistant at the Programming Technology Lab of the Vrije Universiteit Brussel, Belgium. His past involvements include specification and implementation of the languages Gilgul and Lava, and the design and application of the JMangler framework for load-time transformation of Java class files. He has also implemented ContextL, the first programming language extension for Context-oriented Programming based on CLOS, and aspect-oriented extensions for CLOS. He has also co-organized numerous workshops on Unanticipated Software Evolution, Aspect-Oriented Programming, Object Technology for Ambient Intelligence, Lisp, and redefinition of computing.

Title: Context-oriented Programming in ContextL

There is an increased need for context-aware applications that can dynamically adjust their behavior to the context of their use. Two years ago, we have introduced ContextL, our first programming language extension that explicitly supports Context-oriented Programming (COP), based on the Common Lisp Object System (CLOS). In COP, programs consist of partial class and method definitions that can be selected and combined at runtime as needed. Employing runtime adaptations to class and method definitions, COP does not only allow expressing context-aware behavior, but encourages continually adjusting behavior
of programs according to their context.

In my presentation, I will show and discuss the following.

The basic language constructs of ContextL.
The development of non-trivial examples.
The current state of the art in COP: language extensions, example applications and support for the various stages in software development.
Future work: promising next steps that several researchers in our own groups and elsewhere are currently undertaking.

October meeting of the MSLUG - Update

2007-09-12T17:07:00.000-07:00

The October meeting of the Montreal Scheme/Lisp User Group, initially scheduled for Thurdsay, October 25th, is moved to Monday, October 22nd. This is the same day as the Dynamic Languages Symposium that will take place during the OOPSLA conference.

I will post more details soon (time, location, etc.).

October meeting of the MSLUG

2007-09-10T18:57:00.000-07:00

Since this year's OOPSLA conference (as well as the Dynamic Languages Symposium) will be held in Montreal in October, the Montreal Scheme/Lisp User Group will hold its monthly meeting on the last day of the conference, i.e. on Thursday, October 25th at 6PM.

For this very special occasion, we will have two talks instead of just one. One by a well-known Schemer, Manuel Serrano, on Hop, and another one by a well-known Lisper, Pascal Costanza, on Context-Oriented Programming.

So don't miss this opportunity to join one of the most vibrant Scheme/Lisp Community!

Note: I make this early announcement only to let people schedule their flight accordingly. We plan to hold a regular meeting in September too.

Now on Facebook

2007-09-02T19:50:00.000-07:00

I don't know if it's a coincidence or not (maybe some old friends from the Collège de Montréal are reading my blog), but I received an invitation today to join a group on Facebook! So I now have an account on Facebook. I'll have to keep it up-to-date, like my account on LinkedIn. Hope this won't take too much of my time...

Embarrassing, isn't it?

2007-09-02T06:39:00.001-07:00

Get a First Life is a parody of Second Life, which tries to put things in perspective. (Personally, I don't get this whole virtual reality thing myself. Maybe I'm getting too old - I don't even have a Facebook profile, can you imagine? At least, I have a blog...).

But look at the Google ads at the lower right corner of the front page. Most ads are for Second Life itself. Seems to defeat the purpose!

Scheme job at AMARC

2007-08-31T12:33:00.000-07:00

The AMARC (the World Association of Community Radio Broadcasters) is seeking a Scheme/Web developer. The job description is in French.

A great MSLUG meeting

2007-08-30T19:14:00.000-07:00

Yesterday's MSLUG meeting was a true success. About 25 people attended the meeting, and 15 of us went for a beer after the talks. And again, there were a few new faces. Maybe that's because it was our first meeting of the season, or because of the new format (that I called a SchemeCamp). Or both. But I know that many people liked the idea of having short presentations. A variety of subjects helps gather more people, I think.

Let me briefly summarize the 4 presentations:

Christopher Diggins talked about Cat, a language he is currently developing. Cat is a stack-based language, like Factor, but it is functional and statically-typed. The goal of the talk was to show how Scheme can be compiled to Cat, but Christopher went short of time. I'm sure he will give another presentation in a near future.

The slides from his talk can be found here.
Étienne Bergeron presented a number of results from his Ph.D. thesis work. Étienne is working on a JIT synthesizer, i.e. a JIT that generates FPGA configurations from Scheme code. He mentioned that, in the course of his work, he had to reverse engineer the Xilinx protocol using innovative techniques, but which seems to cause a number of IP problems. The JIT synthesizer is currently limited to small expressions, but the results are interesting and promising.
Adrien Piérard presented part of his Master's thesis work, namely a portable interpreter of Scheme, with an emphasis on serialization/deserialization issues for mobile code. The ultimate goal of his work is to provide a portable and interoperable implementation of Termite.
Finally, David St-Hilaire and Étienne Laurin described the implementation of the Tetris game in Scheme. Their implementation targets Tcl/Tk through PS/Tk, but they are now working on a SDL port. It can be obtained from Snow. Note that David and Étienne are undergraduate students and they worked on this project in their spare time.

In short, we had a great time! Thanks to all who contributed to this success.

Another Scheme job in Montreal

2007-08-28T08:15:00.001-07:00

There is another opening for a Scheme programmer in Montreal:

This person would work with Gambit to help in the development of the LORNET project. In particular, the initial project would be continue the development of a workflow/dataflow engine (based on Termite). This work is part of the TELOS system which is a Semantic Web based OS. Taking place between research and applications, the work is very interesting and provides great challenges.

Next meeting of the Montreal Scheme/Lisp User Group

2007-08-26T17:15:00.001-07:00

Our next meeting will be this Wednesday, August 29th. The details of the meeting are not fully settled, but at least Christopher Diggins will talk about its translator from Scheme to a statically typed functional stacked-based language. The details will soon be posted to the MSLUG web site.

By the way, we will be experimenting with another meeting format this fall, inspired by the BarCamp and DemoCamp type of events. At each meeting, three or four people will give short talks (about 15 minutes) about one of their favorite projects (academic, commercial, open-source, or hobby). Many people have already showed an interest in giving such a talk. (I will certainly give one myself, about the grammar IDE we are currently developing at Nü Echo.)

Gambit-C 4.0 released

2007-08-26T04:55:00.000-07:00

Gambit-C 4.0.0 has finally been released. Its website has also been revamped, and is powered by MediaWiki.

This release is a very good step forward, in terms of community support. At last, Marc got it right this time. Welcome to the open-source world, Marc! What we really need now is a whole set of useful libraries to help the Gambit-C community grow and not reinvent the wheel for each new application.

As users of open-source software, we should all give back to the community.

Musical composition program in Scheme

2007-08-23T10:15:00.000-07:00

Kjetil Svalastog Matheussen just announced on the SISC mailing list that he has almost completed the implementation of a nice musical composition program in Scheme (it's an applet that requires Java 1.5):

The applet/program is a special kind of composition program which is going
to be used in music education in Norway, age 9 and up or something
like that. It might look a little bit like a toy, but its not, and its
also possible to make high quality music with it.

Almost all of the graphics and gui is made directly in SISC, while the
signal processing is programmed using Java. The source is GPL, placed
here.

I did not play a lot with it, but it seems really well done and demonstrates that even interpreted code can be fast enough for most GUI applications.

Disruptive change

2007-08-17T07:27:00.000-07:00

In my previous post, I claimed that R6RS will break with the Scheme tradition. But don't get me wrong. I'm not against change. It's rather the opposite.

But I see R6RS as a disruptive change w.r.t. the long-standing tradition. As I wrote, I think it will kill innovation in the Scheme community. Where will there be room for innovation? Development tools, maybe, libraries, of course, but there is not a lot of room left for new language features.

And if an implementations doesn't implement it and is only R5RS/R4RS-compatible, I fear that it will eventually be regarded as an old implementation. Is this an intended side-effect of the Steering Committee? I doubt it. And I hope not.

To me, the only reasonable thing to do with this standard is to call it something else. It is a successor of Scheme, clearly. Probably a good one. But it's not Scheme anymore. Let's call it CommonScheme. Or Scheme 2.0.

R6RS preliminary voting results

2007-08-16T18:41:00.000-07:00

They're out. The preliminary results of the ratification vote. Seems that the draft will be ratified. Congratulations to the R6RS committee! They did a really good PR job.

When I look at the people who voted NO, I see a lot of big names: Jonathan Reese, Will Clinger, Marc Feeley, Felix Winkelmann, Jeffrey Mark Siskind, Aubrey Jaffer, Chris Hanson, Shiro Kawai. People who I have a lot of respect for. (I'm surprised Manuel Serrano did not vote at all.)

What does this mean? Well, I think R6RS will break with tradition. The question is to know whether it's a good thing or not. I think it's not. R6RS will kill innovation, like Jeff Siskind pointed out. Every Scheme implementation, to be of practical interest, is itself a new language. But you could (almost always) run R4RS/R5RS compatible code on them. The previous standards allow really different (and divergent) systems to be developed. Simply consider how Gambit-C, Bigloo, Stalin, Chicken, and PLT-Scheme are different in their implementations strategies. I'm not sure R6RS will still allow this. This should not call this new language "Scheme" anymore.

And the committee was so irrespective of the SRFI process, that's just disgusting. Many brilliant people contributed to this process.

Finally, I don't understand why Matthias Felleisen qualifies the draft as "perfect". He probably means that it's good enough for ratification, but it's certainly not perfect.

The only thing I'm sure is that R6RS will create a schism...

Bye bye vacations

2007-08-14T04:07:00.001-07:00

I'm back from vacations. I was in Gaspésie, a wonderful region of the Québec province in Canada, with my family. We had a really great time. But it's over. Now back to work!

I noticed that Erlang is getting more and more attention. Even Sam Ruby is betting on Erlang on the long term (5-10 years) for the implementation of web architecture. That's good.

Also, I was unable to vote against the R6RS ratification. I didn't have access to the internet during my vacations, and arrived too late Sunday to send it.

The recruiting secret weapon

2007-07-16T18:31:00.000-07:00

At Nü Echo, we've been trying to fulfill several positions for a few months now. Our industry is growing at an incredible rate and there are lots of opportunities for people wanting to work on leading-edge technologies. We even hired a few head hunters to help us. But we've been unsuccessful. Until recently...

Last week, I published a post for one of our job openings, and we received several résumés. And from people who would not have considered working for us, if it was not for the word Scheme in the job description. People who have stable jobs in much larger companies.

It is well-known that great programmers are difficult to recruit. And that traditional ways of recruiting are broken. It is especially true when there is a shortage in supply of programmers, as we are beginning to experience right now. (New ways of recruiting are needed, and a new Montreal-based startup is innovating in this area, Standout Jobs. Their approach is really interesting, both conceptually and operationally.)

Great programmers are not looking for jobs. They already have one. And they don't want to switch from one Java job to another, unless they are dissatisfied with other aspects of their job. But a fraction of them would easily consider another job if it involved Scheme, Lisp or Erlang programming (or other non-mainstream languages like OCaml, Prolog, Haskell, etc.).

So I claim that it is easier to recruit Scheme, Lisp, or Erlang programmers, even in Montreal, than Java/C# programmers. Given that you use the right channels (Standout Jobs, or by advertising on group specific mailing lists/blogs). These languages are not only secret weapons for implementing new applications, but they are also secret weapons for finding new employees. I strongly advise new startups to consider these languages if they want to have an edge when looking for new employees. In Montreal, I am convinced that there is now a critical mass of Scheme/Lisp/Erlang developers to mitigate all the risks usually associated with the use of non-mainstream languages.

(I would even add that Erlang is getting more and more attention from the programming community in Montreal, and that it would not be that difficult to find really good programmers for new Erlang projects.)

Scheme/Erlang interoperability - the sequel

2007-07-16T07:36:00.000-07:00

James Long announced today on the Gambit-C mailing list the availability of an Erlang/Gambit interface, based on an idea I had a few months ago.

The motivation behind this interface is to provide the ability to implement Erlang nodes directly in Gambit-C (James thinks that Termite is a little too immature to base any production code on, a view that I share). This way, you have the best of both worlds:

Code in Scheme (for those who prefer its syntax over everything else, like me)
Use the production-quality Erlang platform and OTP for the deployment of robust, scalable and fault-tolerant distributed applications,
Access the Mnesia database from Scheme (this would need a more complex proxy node in Erlang, but this is certainly doable).

Once I have a few minutes of spare time, I will certainly try to add a few functionalities to the MSLUG web site (which is written in Erlang) using this interface. This will provide a great proof of concept and a great migration path from Erlang to Termite.

John McCarthy in Montreal

2007-07-12T12:21:00.000-07:00

I just received an email from Dick Gabriel, the chair of the OOPSLA conference, telling me that John McCarthy (the father of Lisp) will be in Montreal for a keynote at OOPSLA 2007 on Wednesday at 13:30, Oct 24. Don't miss that event!

Work-life balance

2007-07-12T04:00:00.000-07:00

Ben Yoskovitz has a great post on work-life balance. And he knows what he's talking about. He is starting a new company (Standout Jobs) and at the same time his wife gave birth to their second child.

I think one of the great things of having a good work-life balance, especially when you have a family, is that both aspects of your like can benefit from each other. Great lessons from work can be applied to your family life and vice-versa. Here are two that I find particularly important:

Your family life needs projects. Short term projects (your next vacations, for example), as well as longer-term projects (buying a new house, planning renovations, etc.). This helps keep focus, create a family spirit (as opposed to a team spirit). And also to establish priorities, manage money, etc.
Managing people is very similar to raising children. Every single book on management will tell you that. So becoming a better boss will certainly help you become a better parent (and vice-versa, of course). So take time with your kids, especially in their first fewweeks of life. This creates tremendous emotional links that will last forever.

I have four kids myself and a very demanding job. But I still find time for my personal projects (organize meetings, blog, etc). Of course, I have a good employer and that's a very important term in the equation. And my wife is very supporting.

Scheme developer wanted (alive)

2007-07-11T12:46:00.001-07:00

Nü Echo is seeking a talented Scheme/Java developer (with a strong emphasis on Scheme) for working on a sophisticated speech-recognition grammar development environment based on Eclipse. Candidates should be able to work (at least) in French.

If you are interested, contact me (you can find my email address on the MSLUG website).

Article in Montreal Tech Watch

2007-07-11T06:22:00.000-07:00

Montreal Tech Watch has published an article I wrote about Scheme/Lisp in Montreal. This is an attempt to create more awareness about our Montreal Scheme/Lisp User Group and Scheme/Lisp in general. Our user base has grown over the years, but new faces are always welcome, especially those with a startup mindset.

I'm on the map!

2007-07-05T20:06:00.000-07:00

Montreal Tech Watch has published a map (a visual representation) of the web in Montreal, i.e. of people and projects known to MTW. Pretty cool! The map mimics the metro system here, in Montreal. I don't know on which side of the Rivières des Prairies I sit, though... (For those who don't know, Montreal is an island.)

Busy, busy, busy

2007-07-03T07:06:00.001-07:00

I have been quiet lately due to an upcoming talk I will give at SpeechTEK in late August, and (as usual) I have to develop most of the material at the last minute. Even the software I will showcase is nowhere near finished, and I take 3 weeks of vacation soon. Arrrghhh.......!!

Also, I'm preparing a post for Montreal Tech Watch about Scheme/Lisp (thanks to heri for giving me this opportunity!).

So stay tuned and don't despair, I will soon write the sequel to my series on the implementation of tuple-spaces in Termite/Scheme. I have so nice things to say and a few examples to show. BTW, I translated most of the code to Erlang to make some speed comparisons, so I will write about that too.

SourceForge.net 2007 Community Choice Award

2007-06-20T18:15:00.000-07:00

It's time to vote for your 2007 Community Choice Award at SourceForge. If you like SchemeScript, go to the project page and click on the "Nominate this project" link! (You must have a sourceforge account to do so...)

Tuple space implementation - Implementing the client API

2007-06-13T17:54:00.000-07:00

In a previous post, I designed the client API for an implementation of tuple spaces in Gambit Scheme/Termite. In this post, we will see how to implement the API.

Low-level API

As described in the previous post, the client API is composed of three macros:

(tspace-put! (tag expr-1 expr-2 ... expr-n))
(tspace-read (tag pat-1 pat-2 ... pat-n)
  expr-1 ... expr-n)
(tspace-get! (tag pat-1 pat-2 ... pat-n)
  expr-1 ... expr-n)

The approach taken here is to expand calls to these macros to calls to a lower-level API that sends requests/messages to the tuple space server (which could be on a different node or even a different machine on the network) using the basic Termite procedures. This low-level API is not meant to be called directly by a client application. It's a private API.

This low-level API is composed of three procedures, one for each of the macros:

(tspace:put! tag fields)
(tspace:read tag template)
(tspace:get! tag template)

We will see what a template can be below.

Implementing this low-level API is straightforward. If we assume that the server process is registered globally under tuple-space (with the register procedure), the last two procedures can be implemented like this:

(define (tspace:read tag template)
  (!? (resolve 'tuple-space) `(read ,tag ,template)))

(define (tspace:get! tag template)
  (!? (resolve 'tuple-space) `(get ,tag ,template)))

Note the use of the !? procedure calls. They are used instead of the ! procedure because we want to send a message to the server and block until we get a response. !? means "send and wait for a response".

The call to resolve simply tries to get the process object bound to the globally defined name given. (We won't handle exceptions and errors right now.)

The first procedure looks a little more complicated, but not that much:

(define (tspace:put! tag fields)
  (! (resolve 'tuple-space) (list (self) (make-tag) `(put ,tag ,fields))))

In this procedure, we explicitly pass (self) and a tag, a unique identifier. This is only to make the put message have the same structure as the read and get! messages. The !? procedure sends them automatically for us.

Implementing tspace-put!

Implementing the tspace-put! macro is trivial. In fact, we implement it as a macro simply to be more consistent with the syntax of the tspace-read and tspace-get! macros. Here is the code:

(define-macro (tspace-put! object)
  (match object
    ((tag . fields)
      (where (list? fields))
     `(tspace:put! ,tag (list ,@fields)))
    (else
     (error "invalid object in tspace-put! - " object))))

(I won't use R5RS syntax-rules macros here. They don't mix well with the rest of the Gambit system.) The match expression ensures that object is a list of at least one element.

"Don't care" variables

In calls to the tspace-get! and tspace-read macros, the patterns can be variable names (symbols) or constants. A special symbol, (_, the underscore) can also be used to indicate that the corresponding field can hold any value, and that we are not interested in that value. That's what we call a "don't care" variable.

From the perspective of the low-level API, all the pattern variables are "don't care" variables. In other words, it does not make sense to send the variable names to the server, as they are only used to name some of the values in the matched tuple. This means the tspace-read and tspace-get! macros will first have to replace all the variable patterns with "don't care" variables.

Implementing get! and read

Both macros differ only in the name of the low-level API procedure that must ultimately be called. We will thus abstract a little bit and defer the hard part of both macros to an auxiliary procedure (not a macro):

(define-macro (tspace-read pattern . body)
  (tspace-retrieve pattern body 'tspace:read))

(define-macro (tspace-get! pattern . body)
  (tspace-retrieve pattern body 'tspace:get!))

(In addition to saving us from some copy-paste, relying on an auxiliary procedure has another advantage: the procedure can be easily tested since it's not a macro.)

Here is the definition of the tspace-retrieve procedure:

(define (tspace-retrieve pattern body-list retriever-name)
  (match pattern
    ((tag . fields)
     (where (list? fields))
     (let ((variables (filter pair?
                              (map (lambda (object index)
                                     (if (and (symbol? object) (not (eq? '_ object)))
                                         (cons object index)
                                         #f))
                                   fields
                                   (iota (length fields)))))
           (fields    (map (lambda (object)
                             (if (symbol? object)
                                 '(quote _)
                                 object))
                           fields))
           (tuple-var (gensym)))
       `(let ((,tuple-var (,retriever-name ,tag (list ,@fields))))
          (let ,(map (lambda (var/index)
                      `(,(car var/index) (list-ref ,tuple-var ,(cdr var/index))))
                    variables)
            ,@body-list))))
    (else
     (error "invalid tspace pattern: " pattern))))

The first let extracts the variables from the patterns together with the corresponding index in the template, builds the template that will be send to the server, and creates a new symbol to ensure some hygiene. The body of the let builds the resulting code.

Here is an example of using this procedure. Calling

(tspace-retrieve '('tag x _ 1) '((+ x 1)) 'tspace:read)

yields

(let ((#:g0 (tspace:read 'tag (list '_ '_ 1))))
  (let ((x (list-ref #:g0 0)))
    (+ x 1)))

Note that this makes the assumption that the value returned by tspace:read is a list (the list of the tuple fields). That's an implementation choice. We could have used vectors instead. But as long as this representation does not leak outside of the low-level API and the generated code, that's ok.

That's it for today. Next time, we'll see how to implement the server. This will involve a lot of interesting design decisions.

SISC debugging with SchemeScript

2007-06-12T06:08:00.000-07:00

On the SISC users list, Daniel Sadilek explains his solution to debugging SISC code with SchemeScript:

I found a workaround:
(define-syntax debug
 (syntax-rules ()
   ((_ var ...)
    (begin
      (print "Debugging. Available lexical bindings: " '(var) ...)
      (let ((child-env (make-child-environment (interaction-environment))))
        (putprop 'var child-env var) ...
        (with-environment child-env (lambda () (repl)))
        (print "Debugging ended."))))))
This special syntactic form can be used like this:
(define (f x)
 (let ((sq (* x x)))
   (debug x sq)
   (+ sq sq)))
Btw, I use the SchemeScript IDE and really like the editing style that is made possible with the above debug-form. You can put (debug some-vars) inside a procedure and execute it (or some other procedure that calls it). Then, with access to the local variables, you can write your code inside the procedure -- right in place -- and evaluate by pressing Ctrl+Enter. If finished, you exit the debug REPL.

Gambit-C has (IMO) a much better debugger than any other free Scheme implementation. But the editing/debugging style described above is exactly what I had in mind when developing SchemeScript and what makes it comparable to Emacs in terms of integration with the Scheme system.

Yahoo Pipe for Gambit/Termite

2007-06-05T13:20:00.001-07:00

I have created a Yahoo Pipe for Gambit/Termite related articles. (The resulting feed is displayed at the right of this page.)

What? Another packaging system for Scheme?

2007-06-01T18:20:00.001-07:00

Manuel Serrano has just release Bigloo 3.0a. Its most important new feature is a new packaging system, ScmPkg. Yes, you read correctly: a new packaging system!

It is not related in any way to the Scheme Now! effort, even if it shares its initial goal and philosophy: an implementation-neutral package distribution system. (Even if, in fact, Manuel was one of the early Snow designers.) Were we really in need of another one?

Instead of uniting the Scheme community, these two competing efforts will help balkanize it more. That's my feeling. Who will take time to write packages for both Snow and ScmPkg? And R6RS, and Egg, and Planet, and ...

But you know what? I DON'T MIND! Me, as a Scheme developer, I (usually) don't want to port my code to a dozen implementations. Even two implementations. I use Gambit, that's all. Or Kawa. It depends on the project, the target platform. But once I have chosen an implementation, I stick to it. Porting a library to different implementations has a cost. A huge cost that I'm not willing to pay.

So we should stop worrying about compatibility. We should go write useful applications in our Scheme dialect of our choice. Create success stories in Scheme. (Where are they, btw, the Scheme success stories? I know of Erlang success stories, Common Lisp success stories. But Scheme ones? Do you know one?)

In the long run, it's not the core language that will make a difference. It's its use in the implementation of a truly unique and innovative platform or framework for developing amazing applications. Ruby would not be as successfull as it is without Rails. Erlang has Mnesia and Yaws. Even Java would not be so popular if it weren't for the J2EE specifiation and tools like Eclipse.

In my case, I bet on Gambit-C, JazzScheme, and Termite. This trio has the potential to have a strong influence on the Scheme community. It has unique and compelling features for the development of innovative products.

And Snow? I see it as a repository of Gambit packages/libraries. Nothing more.

Parallel map with Termite

2007-06-01T18:14:00.000-07:00

In Parallel Map Beer Song in Erlang, Bill Clementson gives an implementation of pmap (a parallel map) in Erlang.

Here is an implementatin of pmap in Termite Scheme, as given by Marc Feeley on the Gambit mailing list:

(define (pmap f lst)
   (let ((parent (self)))
     (map (lambda (pid)
            (recv ((,pid reply) reply)))
          (map (lambda (x)
                 (spawn (lambda () (! parent (list (self) (f x))))))
               lst))))

Scheme/Erlang interoperability - an idea

2007-05-31T18:22:00.000-07:00

Would it be cool to be able to send and receive messages to an Erlang node from Scheme (or vice versa)?

For Java-based Scheme systems (Kawa, SISC), one could build something on top of jinterface. It is a high-level Java API for doing exactly that. With some macrology, it would not be too hard to come with something useful. For C based systems, there is erl_interface, though it's a much lower level API. But no real showstopper.

Of course, it would not be possible to send/receive objects other than lists, tuples, atom, numbers, i.e. the basic Erlang objects, but that could be a good starting point.

What do you think? Would that be useful?

Portable Termite

2007-05-30T18:23:00.001-07:00

I heard, from trusted sources, that a portable implementation of Termite is underway. I won't give names, to protect the innocents, but trust me, we'll hear more about that this summer (or at the end of it).

This implementation will also support signed message passing. This means that a system will only be able to receive messages from trusted sources. THAT is interesting.

You'll be able to connect together different Scheme applications, developed using different Scheme systems. Will this be a way to re-unite the Scheme community?

What do you think? Is it a better approach than providing a common packaging infrastructure? I tend to believe it.

Tuple space implementation - The API

2007-05-29T22:22:00.000-07:00

This article is the first in a series that will describe my implementation of a simple-minded tuple space in Termite Scheme and show how Scheme is well-suited (or not, we'll see) for this task.

In this article, I will present the main concepts of a tuple space, as well as the high-level API that will be used to interact with the tuple space.

Concepts

A tuple space is an associative, shared memory. There are basically three operations on tuple spaces: put, read, and get. The put operation, as its name implies, writes a new object (a tuple) in the tuple space. The read operation tries to retrieve a tuple from the tuple space that matches a given pattern (blocking if none exists), but leaves the object in the tuple space. The get operation is like the read operation, except that it removes the tuple from the tuple space.

One important point to note is that all tuples put in (or read from) the tuple space are identified by a tag. That's why the tuple space is an associative memory.

High-level API

My approach in designing a new library is usually to start from the perspective of the user of the library. In other words, I first try to write some test code that will use the library (akin to what we do in TDD). So what would those three primitives (put, read, get) would look like in Scheme?

The easiest one is put, of course. To put a tuple in the tuple space, it will be sufficient to have procedure tspace-put!:

(tspace-put! 'tag expr-1 expr-2 ... expr-n)

In the case of read and get, things are a little more complicated. First, most tuple space implementations (like Linda) allow variables to appear in the pattern. It would be nice to replicate that. In Linda (C), read would look like:

rd("tag", ?var1, ?var2);

This statement reads a tuple and also defines two variables, var1 and var2. In the subsequent expressions/statements, the variables var1 and var2 can be used. But in Scheme, even a macro cannot introduce a variable that would be accessible from subsequent expressions (unless it expands in set! expressions mutating already defined variables).
The Scheme way of doing things is to add extra expressions to the read/get form:

(tspace-read (tag var1 var2) 
  expr-1 
  ...
  expr-n)

But then, the tspace-put! seems a little odd. It would be best to write:

(tspace-put! (tag expr-1 expr-2 ... expr-n))

This would be more consistent with the syntax of tspace-read and tspace-get.

Note that since tspace-put!, tspace-get, and tspace-read make no assumption at all on the representation of tuples. That's a good thing. And the sign of a good API.

The API is thus composed of three syntax forms:

(tspace-put! (tag expr-1 expr-2 ... expr-n))
(tspace-read (tag pat-1 pat-2 ... pat-n)
  expr-1 ... expr-n)
(tspace-get (tag pat-1 pat-2 ... pat-n)
  expr-1 ... expr-n)

where tag is a symbol, and pat-1 to pat-n are pattern elements, i.e. either a variable (symbol), or a constant (number, string, character, or quoted Scheme constant).

The other functions of the API are:

(tspace-connect node)
(tspace-disconnect)

to connect to a tuple space on node node and to disconnect from a tuple space.

What do you think? How can this API be improved?

Next steps

That's it for today. In the next article, I will show the implementation of the client API. This will involve some macrology and show the true power of Scheme in this respect.

Erlang Style Message Passing for SISC

2007-05-28T04:06:00.000-07:00

Ben Simon has implemented Erlang Style Message Passing for SISC. The system has a number of limitations, of course. But that's a nice starting point.

Startup culture in Montreal

2007-05-26T07:11:00.000-07:00

A startup culture is developing rapidly in Montreal. I've been following some startup-related blogs for a few months now, and my conclusion is that something is cooking. Here are a few highlights:

Many past and upcoming events;
Local entrepreneurs meet regularly at the Montreal Tech Entrepreneurs Breakfast;
Some well-known VC firms now have offices in Montreal (like Garage, Guy Kawasaki's capital fund);
A new wiki on startups has been launched.

Combine this with the growing interest in dynamic, non-mainstream languages, as witnessed by the following facts:

The Scheme User Group attracts about 20 people at each meeting, but I estimate to 35-40 the number of people who come to our meetings at least once a year;
At our previous meeting (on Erlang), a lot of new faces show up at the MSLUG, and some of them mentioned they're working on new startup ideas;
There is a good concentration of good Scheme developers who work on great, innovative tools: Gambit-C, Termite, JazzScheme, Snow.

Will Montreal become a small Silicon Valley?

Last night meeting

2007-05-25T07:42:00.000-07:00

Here are a few pictures from the MSLUG's last night meeting.

Marc Feeley talks about Snow:

Part of the audience (we were about 20 Scheme enthusiasts):

We finished the night at the pub:
(I am the guy with the red shirt, at the right of the picture)

Here we see Marc Feeley, Danny Dubé (author of SILex), and Étienne Bergeron.

Scheme programmer wanted at GrammaTech

2007-05-23T06:36:00.001-07:00

Here! Here!