C. Enrique Ortiz reports on the recent public request for comments call from the MSA 2 working group. MSA 2 handsets and applications won't be anywhere near Android in terms of capabilities and above all platform integration, but it is a step in the right direction.
Gmail for mobile 2.0 is out. I've been an avid user of the 1.x version for years and here is my quick review:
The Good:
- Multiple account support, including Gmail for Domains support.
- Limited offline mode.
The Bad:
- The "Refresh" softkey is gone, replaced by a "Hide" one. "Refresh" is now a menu option and it apparently has no key shortcut. Thanks but no thanks. Most multitasking phones nowadays feature a dedicated "Home" or "Menu" key of some sort that hides the application. It would be fine to hide "Refresh" away in the menu if the application had push synchronization but that's not the case (not that it could anyway, long-lived push on J2ME is impossible on almost every phone. That's why "Refresh" should be in a easy, visible shortcut and not in the menu.)
I'm keeping this version for the multiple account support but I hope Google brings back the "Refresh" softkey or at least maps a key shortcut to it. Right now "Hide" is completely useless on my Nokia E61.
Scripting languages for J2ME
One of the many limitations of J2ME is the lack of dynamic code addition on runtime. It is impossible to add new classes, be it by class files or by reflection. The required APIs are just not included in CLDC. Given this limitation I find it surprising there are not many J2ME hosted script interpreters, specially now that medium and high end handsets have megabytes of available heap memory and fast JIT compilers. Here are the ones I can find.
Imperative languages
FScript, a very simple scripting language, has a J2ME port called FScriptME.
Hecl is a full blown scripting environment for J2ME. It has specific support for J2ME APIs and includes easy to use wrappers for many of them.
The Simple Forth Interpreter has a J2ME port. Forth is a very good fit for the limited resources you find in J2ME handsets but its reverse polish notation and stack-based, hum, everything, are usually very off-putting to programmers.
Lisp dialects
An extremely simple J2ME Lisp interpreter as an OKI software programming exercise. I'm not kidding, run it and see it for yourself. It lacks proper lambda expressions so it's not useful for anything other than as a demonstration, but it could be used as the base of a more complete interpreter (which is what part of the exercise appears to be).
JScheme is small enough to be ported to J2ME, specifically the old 1.x versions which have less bootstrapping implemented in pure Scheme (which allows for a smaller JAR file.) In fact somebody as already ported it to J2ME.
After reading the article on JSON for J2ME by Enrique Ortiz I remembered the J2ME port of Stringtree JSON that I made for Peak Notes. I've added a new section to my site with the source code for it and another small project.
While checking out the recently released JSR 226 benchmark, JBenchmark SVG, I noticed that the Samsung Omnia was the (so far) top performer in the database. Windows Mobile handsets are a total hit-or-miss kind of a deal when it comes to J2ME support. There is half a dozen different JVM for WM and some of them are absolutely terrible.
Esmertec Jbed has been for the past year the top performer and most estable of them and has been showing up on more and more devices. Aplix JBlend is the other manufacturer favorite. It's good to see competition in this area: the wide JSR coverage that JBlend features can put pressure on Jbed to improve its support for more JSRs. The Omnia is probably the first WM handset with Mobile Service Architecture support, as evidenced by its JBenchmark profile (the full report is behind a pay wall.)
The PSP port of the KVM/PhoneME project has come a long way. Called PSPVKM it's now perfectly usable and a very valid platform for J2ME testing. It supports advanced JSRs like the File/PIM Connection API and has working network access and MIDI sound.
For developers it offers full access control to the platform security warnings, so you can customize and test your midlet for different levels of trust without code signing.
Turning midlet life cycle states and callbacks into an event-driven execution model
One of the most frustrating areas of J2ME is the callback based, magic-thread-spawning approach that the API adopts for delivery of events and life cycle notifications. Basically a midlet can, and will, be constantly interrupted by VM-spawned threads. If you are not extremely careful with your programming the internal state of your midlet will be corrupted, by your own code.
A frequent mistake newbie J2ME programmers make is ignoring how event delivery and callbacks work, or just not bothering to understand them. This is of course a consequence of the grater problem of not understating how concurrency works in Java and how to work with synchronization primitives. Most of the time you can get away with it too, since most mobile VM support only a single flow of execution and just preempt away the current thread.
When the problems hit usually one of two things happen: either the programmers start adding a ton of useless, line-by-line boolean checks that sometimes catch the race condition. Or they dust off their books on Java concurrency and put a mutex check on every object in their code. Enjoy your deadlocks!
The Event-based model
There is a superior alternative. It has been around since the dawn of UI programming and it is jarring that the designers of J2ME choose to ignore it. It is a very simple idea: instead of callbacks and VM threads, you just have a synchronized queue of events. Your main execution loop consumes events from this queue, and the callbacks from the VM do absolutely nothing but add events to this queue.
The main idea is that callback and life cycle code never touches the internal state of your application. It doesn't set booleans. It doesn't add to a counter. It doesn't call a model method. It only, and only, creates an event object and inserts it into a synchronized queue.
How it works in Peak Notes
Here is a simplified version of the Event class in Peak Notes. It is a very simple class that doesn't reference any other object and it is meant to store the data of the callback that created it.
public class Event {
// add more types when needed
public static final int POINTER_PRESSED = 1;
public static final int KEY_PRESSED = 2;
public int type;
public int key;
public int x;
public int y;
}
Callback methods just create a new object of this class, fill it with the appropriate event data, and queue it in the controller. A pair of example callback methods in the Canvas-derived class:
public void pointerPressed(int x, int y) {
Event ev = new Event();
ev.type = Event.POINTER_PRESSED;
ev.x = x;
ev.y = y;
controller.queueEvent(ev);
}
public void keyPressed(int keyCode) {
Event ev = new Event();
ev.type = Event.KEY_PRESSED;
ev.key = keyCode;
controller.queueEvent(ev);
}
The methods in the controller class allow access to the event queue in a controlled, synchronized way. Since you are not doing model and logic access from the asynchronous callbacks this reduces the mutex usage to just the event queue.
// created in the controller constructor
private Vector eventQueue;
public void queueEvent(Event e) {
synchronized (eventQueue) {
eventQueue.addElement(e);
eventQueue.notify();
}
}
private Event[] lockAndWaitForEvents() {
Event[] postEvents = null;
synchronized (eventQueue) {
// lock only when there are no existing events
if (eventQueue.size() == 0) {
try { eventQueue.wait(); }
catch (InterruptedException e) { }
}
}
postEvents = new Event[eventQueue.size()];
eventQueue.copyInto(postEvents);
eventQueue.removeAllElements();
}
return postEvents;
}
The method lockAndWaitForEvents is what you call from your run loop to consume and process new events. If there are no events to process it will wait and sleep, freeing the CPU. Here is a simplified run loop:
public void run() {
// init code
...
// run loop
boolean quit = false;
while (!quit) {
Event[] events = lockAndWaitForEvents();
quit = processEvents(events);
}
midletInstance.destroyApp(true);
}
processEvents is the method that will call your logic depending on the events received in the queue. Peak Notes uses a complex custom UI system, but on simple games it could be something like:
private boolean processEvents(Event[] events) {
// this will be modified by the event logic if needed
boolean doQuit = false;
for (int i = 0; i < events.length; i++) {
switch (events[i].type) {
case Event.POINTER_PRESSED:
// resolve the event into high level call(s)
...
break;
case Event.KEY_PRESSED:
// resolve the event into high level call(s)
...
break;
}
}
return doQuit;
}
The pain of paint
Given the ridiculously limited and ugly array of UI elements the standard J2ME API provides virtually every midlet uses custom controls, or a fully custom UI. This leads to a hard problem: the only way to paint on screen is by implementing a callback. Which, like all the other callbacks in J2ME, is effectively asynchronous and should be treated like it.
For the Canvas.paint() callback we cannot use an event system since it is only in the execution flow granted by the callback that the midlet is allowed to paint on screen. It is possible to decouple application state into an Event-like class and "double buffer" it, having always a state dedicated to the application model code and another, "dormant" state awaiting for the next time the paint callback is called. This pattern is often used in game programming for more powerful platforms like video consoles. But on J2ME memory is limited and not always possible. It is also complex to implement and can be a real challenge to convince other programmers to follow it.
For Peak Notes I decided to layer a mutex over the logic of the application, just for the paint callback. Fortunately, since all the code that can modifiy the application state is isolated away in a single call, it is extremely easy to implement:
public void run() {
// init code
...
// run loop
boolean quit = false;
while (!quit) {
Event[] events = lockAndWaitForEvents();
synchronized (paintMutex) {
quit = processEvents(events);
}
}
midletInstance.destroyApp(true);
}
And similarly in the paint callback in the Canvas subclass:
public void paint(Graphics g) {
synchronized (paintMutex) {
// paint code goes here. it can access the state
// of the application with no restrictions or locks
}
}
Conclusion
Multithreading is real even on handheld, single core systems. If J2ME forces you to deal with it, learn how to use it and how it changes your preconceptions on the API.
Today I discovered that Antenna has out of the box support for mpowerplayer. Just set up the wtk.home variable to point to the mpowerplayer SDK root and you are all set. It will run the custom preverifier if you are on Mac OS X.