Dave Herman explores the strange interactions between eval() and variable scoping in Javascript.
John Resig, the author of the amazing jQuery library, has published his interactive presentation on advanced Javascript topics like first class functions, closures and prototype based OOP. The talk also covers his Javascript port of the Processing language and offers a new demo that compares it with direct Javascript + Canvas programming. More in his blog.
Javascript interpreters and their performance are receiveing a lot of attention lately. There is a very healthy competition going between Mozilla TraceMonkey, Google V8 and now Apple is back with SquirrelFish Extreme, an improved version of the original SquirrelFish. The new version features extensive JIT optizations and a structure cache. They even went as far as including a JIT compiled regexp engine which as far as I know is completely unprecedented:
You can write the kind of text processing code you’d want to do in Perl or Python or Ruby, and do it in JavaScript instead. In fact we believe that in many cases our regular expression engine will beat the highly tuned regexp processing in those other languages.
Google's new Javascript JIT compiler V8 gets benchmarked and compared to both current and upcoming Javascript engines.
The latest issue of a A List Apart is out and features an article on advanced rollovers with Javascript and CSS. It's also a nice introduction to jQuery.
John Resig blogs about the new Firebug release:
The Script panel (the JavaScript debugger), the Net panel (network monitoring), and Console panel have all seen considerable updates. They're all much more performant and have a huge number of bug fixes.
Firebug changes the way you develop for the web. You won't be able to go back to alert() and background-color: red after you spend some time with it.
Ars Technica reports on tracing, a new optimization added to SpiderMonkey:
They are "getting ready to take JavaScript performance into the next tier" with a radically innovative optimization tactic called tracing that has already produced performance improvements ranging between 20 and 40 times faster in some cases.
Andreas Gal is the researcher behind the new optimization and writes about it:
Traces represent a single iteration through a loop, and can span multiple methods and program modules. If a function is invoked from inside a loop, we follow the function call and inline the instructions executed inside the called method. Function calls themselves are never actually recorded. We merely verify at runtime that the same conditions that caused that function to be activated still hold.
Implementing script tag long polling for Comet applications
In my previous post I showed how to implement a simple PubSub server with Twisted for asynchronous Comet updates to a web application. For the Javascript client side of the application I've choosen the <script> tag long polling method.
Choosing a connection method
There are many different ways to achieve the "instant update" feel in a Comet application. They mostly differ in their latency and the security restrictions imposed by the browsers. They are divided in two big groups:
- Streaming: a connection is kept open between the server and the browser. Regular updates are pushed through it and parsed by the client as they arrive. Offers the lowest latency.
- Long polling: a connection is kept open between the server and the browser. When an update arrives the connection is closed and a new one is opened.
Streaming connections offer the best user experience, but are more complicated to implement and are less tolerant of proxies and firewalls.
Why use <script> tag long polling
I decided to build my solution over the polling <script> tag method. It is a very simple idea: a <script> element is dynamically created and added to the <head> of the document. The src of this tag points to the subscription channel of the PubSub server. When the server wants to notify the client of new data it sends a JSONP string that invokes an existing method in the client. After processing the data this method starts a new connection to the server.
This method allows full cross domain requests and it is very broadly supported. After all it is based on the same concepts that enable online advertising, which depend on the browser allowing to load script sources from domains that are different from the originating one. I needed this feature since my hosting setup is very simple and I cannot unify different servers behind a load balancer like HAproxy.
Why not use <script> tag long polling
The biggest problem of long polling a script load is the lack of control over the connection. There is no feedback on the status of the load by the browser. onload callbacks are only supported by Firefox and Opera, Internet Explorer supports onreadystatechange, and Safari does not report anything. The callback call of the JSONP string makes it possible to detect a successful load. But a timed out or cancelled connection just fails silently.
Solving the timeout issue
The solution is to not wait for a timeout condition. In Peak Notes the client code deliberately "forgets" everything about the <script> load after 45 seconds, and starts a new one from scratch. An unique sequence number is sent along the request so when a stale script loads and invokes the JSONP callback can decide if the pushed data must be honored or just ignored. This means that for browsers that keep the script load active even when the <script> tag has been removed from the DOM (like Firefox) the application just discards anything that was sent along with it.
The server always sends a response and closes the connection after 60 to 65 seconds, even if there is no data to send to the client. This means that at most there are one stale and one valid connection concurrently. The stale connections get sent either no data all (on the server side timeout at 60 seconds) or redudant data that will be accepted on the connection with the valid sequence number (on a real push notification). The valid connections will always reinitiate the <script> load with a different, newer sequence number.
Implementation
ModelManager.startComet is the model method that starts the Comet connection. Its is called once during startup and then again in the Comet finalization callback.
ModelManager.prototype = {
...
startComet: function() {
getCometJSON('http://pubsub.example.com:8080/subscriptions/channel/'
+ globalModelManager.userModel.subscriptionChannel);
},
...
};
cometSerial is incremented by one on every new connection, cometRunID is a random number calculated during load and cometValidStamp contains the last valid sequence ID of the last initiated connection.
window.cometSerial = 0;
window.cometRunID = Math.floor((Math.random())*1000000);
window.cometValidSeq = null;
These variables are used for timing the connections and for detecting concurrent getCometJSON calls.
window.cometStartTime = 0;
window.cometEndTime = 0;
window.cometLastInterval = 0;
window.cometLastReason = 'juststarted';
cometFinalizeCB contains a closure with the local state of the last initiated connection.
window.cometFinalizeCB = null;
getCometJSON sets up a new <script> tag and a new closure for removing the tag and starting a new one. It can be called by the internal 45 seconds timeout or after a valid JSONP callback, whatever arrives first.
function getCometJSON(url) {
if (window.cometStartTime > 0)
return;
var nowd = new Date();
var now = nowd.getTime();
// stats
window.cometStartTime = now;
window.cometEndTime = 0;
window.cometSerial ++;
// add the new script tag
var head = document.getElementsByTagName("head")[0];
var script = document.createElement("script");
// calculate the new sequence ID
window.cometValidSeq = 'W' + window.cometRunID + '_' + window.cometSerial + '_' + now;
script.src = url + '?seq=' + window.cometValidStamp;
var timerObj = null;
// prepare the closure. all the manipulations occur in the lexical scope
// of the getCometJSON invocation
var loadCB = function() {
// whatever happens, delete the tag
head.removeChild(script);
script.src = '';
script = null;
// remove the timer obj
if (timerObj != null) {
clearTimeout(timerObj);
timerObj = null;
}
if (window.cometEndTime == 0) {
// the callback was never called
var nowd = new Date();
window.cometEndTime = nowd.getTime();
window.cometLastInterval = window.cometEndTime - window.cometStartTime;
window.cometStartTime = 0;
window.cometLastReason = 'closure';
}
// we are ready for a new connection
globalModelManager.startComet();
};
window.cometFinalizeCB = loadCB;
timerObj = setTimeout(loadCB, 45000);
head.appendChild(script);
}
remoteJSONReady is the JSONP callback.
function remoteJSONReady(data) {
if (data != null)
if ("seq" in data)
if (data.seq != window.cometValidSeq) {
// data.seq contains the sequence ID this script load was called with.
// if we are here it means this callback was done from a stale connection
return;
}
// stats
var nowd = new Date();
window.cometEndTime = nowd.getTime();
window.cometLastInterval = window.cometEndTime - window.cometStartTime;
window.cometStartTime = 0;
window.cometLastReason = 'callback';
// if we have valid data pass it to the data model.
// subject/observer pairs will do the rest
if (data != null)
if ("status" in data)
if (data.status == "changed") {
globalModelManager.newPushData(data);
}
// call the current closure
window.cometFinalizeCB();
}
Future improvements
The polling method works best when it is invoked from a XMLHTTPRequest connection. This interface has full connection status reporting and it is possible to detect a connection timeout or even cancel the connection from the client code, and it has been present in all mayor browsers for years now. For security reasons it requires connections to be made to the same domain as the calling document. An efficient front end server like nginx proxying to the application and the PubSub server, or a load balancer like HAproxy is required to effectively multiplex a single domain between multiple servers.
The WHATWG has posted an status update for the week, including links to the announcement of the new Web Workers specification and its discussion.