wabasoft - dealing with performance

Products
Overview
Downloads
Fun and Games
More VMs
Dev Tools

Development
FAQ
Support
Specifications
How-to Papers

Company
Overview
Contact Us
Home Page

"In the author's opinion, entirely too much programmers' time has been spent in writing such simulators and entirely too much computer time has been wasted in using them."

- Donald Knuth,
"The Art of Computer Programming"

Dealing with Performance

So, you've whipped up a small Waba program. It runs fine on your PC under a JVM and you package it up with warp and exegen to run it on a PalmPilot or Windows CE device.

You put it on your PalmPilot, run it, and notice it runs quite a bit slower than the same program on your PC. On a Windows CE device, it's slower but the performance difference is not as noticeable.

Why is this?

Fundamentals

Well, the main reason the PalmPilot version runs slowly is because the PalmPilot doesn't run programs as fast as your PC.

As far as a processor goes, the PalmPilot contains a Motorola 68328 "Dragonball" CPU. For a technical brief on this CPU, you can click this link. The 68328 first shipped in the summer of 1995. It has a 16 (or so) Mhz clock and runs about 2.7 MIPS (millions of instructions per second).

And for Windows CE, this page contains a list of the various MIPS processors Windows CE devices may use. This page contains the specification for the R4300i microprocessor. Windows CE devices may also use other processor types.

If your interested in comparing technical data sheets, a data sheet for performance of the Pentium II processor is available by clicking on this link.

However, these technical numbers probably don't give you a feel for how fast a program you write on the PC will run on a PalmPilot or Windows CE device.

Comparing PalmPilot and PC Performance

Instead of starting with Waba, let's start with C. Let's say we write the following C program and run it on a PC. How long do you think it will take to run?

  {
  int i, j, value;

  value = 0;
  for (j = 0; j < 100000; j++)
    {
    i = j / 1000;
    value += j / 7;
    if (value == 0)
      value += i / 3;
    value = value % 100;
    }
  printf("%d\n", value);
  }

If this program is compiled with debugging on (using Visual C++ 5.0) and executed on a Pentium 233Mhz (by current standards, an ancient machine), it comes right back with a value. We could time it in milliseconds, but instead, we'll just say that this program runs "fast" on a Pentium PC.

Now, we'll take the same program, change the "int" to a "long", compile it with the Metrowerks compiler for PalmOS (optimizations on), and run it on a PalmPilot.

  long i, j, value;

Why change the "int" to a "long"? We need to change it so the variables are 32 bits like they are on the PC. A C int on the PalmPilot defaults to 16 bits and if we don't change it to a long, the loop will never terminate.

So, how long do you think it will it take to run? On a PalmPilot Pro, it takes about 10 seconds.

So, it's pretty obvious there is a difference here. But since the program ran so fast under the Pentium, we don't know how big the difference is.

How Big Is The Difference?

A program using primarily 32 bit arithmetic can run 300 times slower on a PalmPilot. This is the speed difference of a C program running on a Pentium 233 vs. the same C program on a PalmPilot Pro.

So, a C program that takes a second to execute on a Pentium PC can take 5 minutes to execute on a PalmPilot.

This is, of course, a rough number. The real value will depend on your PC, the program written and various other factors. But it should give you a feel for the difference.

So, what does this mean for Waba programs? Waba programs will run a lot slower on your PalmPilot than on your PC. Probably 300 times slower just like C programs.

How about Windows CE?

A program using primarily 32 bit arithmetic can run about 10 times slower on a MIPS R4000 based Windows CE device versus a Pentium 233 Mhz. We tested using a Casio Cassiopeia E-10 Windows CE Palm-sized PC.

Of course, the real difference will depend on your Windows CE device, your PC, the program and other factors but that might give you a rough idea of the performance difference.

How fast is Waba compared to C?

So, we know that small devices are probably going to run programs slower than a Pentium PC. In some cases, quite a bit slower.

But let's say we were choosing between writing a program in C and writing one in Waba. What kind of performance can we expect from Waba relative to C?

Well, the WabaVM 1.0 does not use a JIT (Just In Time compiler), it interprets the bytecode it needs to execute.

To do this it needs to do the following for each bytecode operation:

load the operation

decode it (figure out what to do)

do something

go back to step 1.

So, for every 1 instruction that C might compile down into, you have more than one instruction that needs to be executed by a WabaVM.

So, we can roughly expect a Waba program running under the WabaVM 1.0 to run maybe 3 to 8 times slower than a similarly written C program. In our tests, it runs instructions about 5 times slower.

Any VM which is interpreting bytecode will have similar performance if a JIT is not used. If you are interpreting code instead of executing it directly, you just need to do more stuff.

The Big Picture

So, what's the big picture? If you are using a JDK to do development on your PC and you write a program and run it on the PalmPilot, you can expect it to run a lot slower on the PalmPilot than on your PC.

If you are using a Pentium based computer to develop your program and a JDK that does not have a JIT, you can expect your program to run about 300 times slower on a PalmPilot with a WabaVM than it does on the Pentium with a JVM.

And the same program will probably run 10 times slower on a standard MIPS based Windows CE Palm-sized PC relative to a Pentium 233 Mhz desktop PC.

If the JDK you are using on your PC has a JIT, you can expect the difference to be even greater.

We are continuing to do performance analysis of the WabaVM to make it faster but the big picture is that programs written in any language will run a lot slower on a 68328-based PalmPilot than they do on a Pentium-based PC.

So, what does this all mean? It means that when you write programs for small devices, you should design them to be as small and simple as possible. You should avoid looping too much and try to minimize, simplify or avoid compute intensive tasks.

The programs that ship with PalmPilot and Windows CE devices have "good performance" from the users perspective. You can write Waba programs that have the same feeling of "good performance" if you design them knowing the limitations of the device.