Client-side Bumps Charts

History

For a while I've been producing bumps charts of various gliding competitions. See the page for the 2005 Booker Regionals for some background information. In addition, Melissa Jenkins has been using a slightly cut-down version of the scripts in her OnGlide system to provide similar charts for competitions which use that system.

My general workflow (automated with various scripts, of course) is to download the web page(s) containing the competition scores, extract the required information using an XSLT script tailored to the particular scoring system (or even the particular competition) storing the results in an XML file like:

<?xml version="1.0" encoding="utf-8"?>
<bmp:results xmlns:bmp="tag:edavies.nildram.co.uk,2006-05-23:bumps"
             comp="Booker Regionals 2010">
   <bmp:day day-no="1" date="Sun May 30th"/>
   <bmp:day day-no="2" date="Wed June 2nd"/>
   ...
   <bmp:glider ident="T6" pilot="Jon Gatfield / Tim Scott"/>
   <bmp:glider-day ident="T6" day-no="1" score="831"/>
   <bmp:glider-day ident="T6" day-no="2" score="1000"/>
   ...
   <bmp:glider ident="J1M" pilot="Jim White"/>
   <bmp:glider-day ident="J1M" day-no="1" score="920"/>
   ...
   <bmp:glider ident="A9" pilot="Shack Roberts / Glen Alison"/>
   <bmp:glider-day ident="A9" day-no="1" score="717" hc="true"/>
   ...
</bmp:results>

This is then processed by another XSLT script (which is generic to all competitions) to produce an SVG version of the final bumps chart.

In an ideal world I'd just serve that SVG directly and all the browsers would display it. Unfortunately, the real world contains Microsoft who have, until very recently, messed things up by not supporting SVG in Internet Explorer. To work around this I convert the charts to PNG format to serve.

Melissa's OnGlide system works similarly except that it goes straight from her database into the generic XSLT script (actually skipping some early steps of that as her database already contains the required information and it was stressing her server to recalculate it). A neat aspect of her system is that she has much more control of the server than I have with my simple static web hosting so she can decide what to serve depending on the browser. She therefore sends PNG to Internet Explorer and SVG to the proper browsers.

This all works but it's really not very satisfactory. There are too many moving parts with too many connections between them and the resulting images show a great glob of information but don't really help the user explore and understand what's happening.

Client-side bumps

I've always thought of the setup described above as a prototype with the idea of doing something a bit more interesting and interactive in the browser. The biggest stumbling blocks are the differences between browsers (well, mostly between Internet Explorer and the rest).

Luckily a rather bright chap by the name of Dmitry Baranovskiy has come up with a tidy solution: the Raphaël library which gives a straight-forward JavaScript interface to create SVG on most browsers and Microsoft's equivalent, VML, on Internet Explorer (video). It is, of course, limited to the intersection of the functions of the two graphics languages but that's plenty for most applications.

During the 2010 Booker Regionals I wrote some JavaScript code to implement bumps charts pretty much in the same form as the ones I'd produced previously in SVG. This test page shows off the basic functions and gives the opportunity for a bit of view sourcery to get an idea of the programming interface.

The BumpsChart object is created and associated with a Raphaël canvas. It is then passed two JavaScript data structures: the data containing information on the actual competition and the options containing information on how it is to be displayed. In the case of the test page there is some JavaScript code to set the options so that the chart displays with colours which match the surrounding page. The defaults are simpler: a grey background with black text.

As well as passing data and options when the chart is created it is also possible to add to and update these data-sets later using add/setData/Options calls.

To demonstrate different ways of displaying the chart the test page contains some javascript: links which call a chart method to update the options. Even with the current limited implementation it's possible to be a bit more creative than that. For example, while displaying that test page you can type the following in the browser address bar:

to set the baseline to William Parker (LC)'s score.

One thing which might not be immediately obvious about the test page is the way the data is loaded. I converted the results.xml (as described above) into JSON using a little Python script producing a file like mixed.json.

The Python script wraps the JSON in a JavaScript function call (to setCompData). When the test page is loaded or when a different set of data is selected (using the 2009 and 2010 links) a script element is dynamically created and set to load the appropriate JSON file. This load happens asynchronously so after a short delay the script completes loading and runs calling the setCompData function which sets the data on the chart.

There's a bit of extra code to cache the data for competitions using the original (relative) URI reference.

Client-side data extraction

The system described above is a bit of an improvement on the previous server-side bumps in that it allows quite a lot more flexibility in the display of the charts and somewhat reduces the server-side processing required. However, it would be much better to further reduce or eliminate the server-side processing.

Ideally, the client-side code could fetch the competition scores page(s) and derive the information required directly from them. Once the page is available in memory in a DOM tree this should be easy - certainly no harder than the existing XSLT data extraction with, of course, the same requirement to tailor the code to, at least, the particular competition scoring system.

However, there's a big problem: for basically good security reasons JavaScript code is not allowed to access pages from another server ('same-origin policy'). Unfortunately, these rules are rather draconian; they don't allow access even to public pages which contain no private information and which do not "leak" information on any cookies stored for their domains, or anything like that. I believe HTML5 I supposed to help with this but the few fragments of information on the subject that I've seen do not inspire me with confidence that it'll be a well thought through solution.

Anyway, downloading scores pages into HTMLIFrame objects and fishing around in the ensuing DOM is the next game to play but I don't think I'll have much time to do that for a while.

Display improvements

The test code produced so far follows, more or less, the form of the original server-side bumps charts with the addition of a few options to tweak the display a bit. There are plenty of opportunities to do more interesting things but, again, they'll have to wait for now.