Monday, April 7, 2008

Benchmarking Javascript variables and function scopes


The other day, I was doing a quick templating plugin for jQuery. It would split long strings into arrays of tokens, and then parse these, based on a map(associative array) of handlers for the different "commands" in the tokens.

The Background

The thing is, that the tokenization was done on the first call, and only the actual parsing was executed each time the template was applied.
Surprisingly, the parsing function of the plugin was always taking around 2.5 seconds!
This was a recursive function, called around 10 times. It had a short loop of 3-4 iterations per call.
This shouldn't be SUCH a heavy task, certainly not enough to take that long.
I was really confused so I started experimenting.

The Situation

There were 3 types of tokens in the template:
  • Tokens that open/start a block, f.e: <?if $foo?>.
  • Tokens that close/end a block, f.e: </if?>.
  • Tokens that are their own block, f.e: <?elseif $bar/>.
I called this, the mode of the token. So I had 3 symbolic constants declared. All the code was wrapped with a self-executing function, to create a local context. It looked like this:
(function( $ ){
     OPEN = 1,
     CLOSE = 2,
     CLOSED = 3;
 /* the rest of the code */
})( jQuery );
These constants, were compared to a 'mode' attribute of the token. This caused 1 reference to each of them, on every loop in the parse function.

A Small Change

I tried tried any possible optimization, but Firebug's stopwatch was stuck at 2500ms.
I was nearly desperate, so I started changing things at random.
One of those changes, was modifying the way I was saving the mode of a token. Instead of storing the integer in a 'mode' attribute of the token, I used 'open','close' and 'closed' as boolean attributes.

There's no way to describe my happiness, when Firebug's stopwatch showed ~230ms instead of ~2500ms.
I must say I was really surprised and confused, this was such a small change, and one that I'd never had guessed.
After this, I decided to clear my doubts with a consistent benchmark.

The Benchmark

The first thing I did, was to make my "own Firebug". I created a script with a stopwatch, a logging method and an easy way to benchmark many sequential calls to a function, with many attempts, generating an average and finding the lowest.
After that, I set up the environment. Declared a global variable, and a function similar to the one showed above. Then nested another equal function inside it, and nested another one inside the latter.
In the end, it looked like this:
var global = 1;

   var local1 = 1;

      var local2 = 1;  

         var local3 = 1;

         /* here I executed the benchmark */  
This created 4 variables, 1 global, and 3 local. All of them could be accessed from where the benchmark was being ran, but they belonged to different scope chains. I benchmarked 4 functions, each of them accessed one of these variables 400,000 times with a loop.
Each benchmark was tested 5 times, in order to find an average and the minimum. The latter should be the one that really matters. I tested on all 4 major browsers, on my Windows XP.

The Results

The numbers were surprising, but not in all browsers. Opera 9 seemed to be really indifferent to the scope chain, all took the same time. Firefox 2 was really the slowest. Safari 3 and IE6 were in the middle.
These are the numbers, I'll only mention the lowest from the 5 attempts.
Firefox 2
* global: 969ms * local1: 672ms * local2: 359ms * local3: 78ms
IE 6(=7)
* global: 187ms * local1: 171ms * local2: 125ms * local3: 94ms
Safari 3
* global: 125ms * local1: 93ms * local2: 78ms * local3: 62ms
Firefox 3 beta 5
* global: 41ms * local1: 48ms * local2: 48ms * local3: 37ms
Opera 9
* global: 31ms * local1: 31ms * local2: 31ms * local3: 31ms
Opera is the clear champion, for some reason, the scope doesn't affect the timing at all.
It does affect the rest of the browsers. It will be specially noticeable in Firefox. Note that Firebug was turned off while benchmarking Firefox, it did affect the perfomance but not THAT much.
UPDATE:Added IE7 and Firefox 3 beta, congratz Mozilla, much better!

The Conclusions

Referencing might take time.
Using variables that were declared on different scope chains, takes longer than those that are local to the executing function.
If you are going to use one very often inside a deeply nested function, you should save it in a local variable first.
One common situation is when using the global variables window and document.
It does matter, but it's not always crucial.
Although it does make a difference, you shouldn't become paranoid about this.
These numbers were produced by 400,000 iterations. You probably don't access a variable that many times. Optimizing won't yield a noticeable improvement most of the time.
There are some situations where you should be careful though
  • Within loops, if you have a long loop, caching the variable could improve the perfomance greatly.
  • Inside frequently called functions. This is similar to loops.
  • Inside functions, that are deeply nested inside others. If these reference a distant variable, many times, it could hit on perfomance too.
Benchmarking is fun!
I always love running benchmarks, feel free to try this yourself. You'll find the link below.
The tool used for the benchmark (benchmark.js) can be taken for your own experiments.
If you try other browsers/OS, feel free to post the results, I'll include them so others can see.

I tried to find links about this, but none showed up. I'll update if I get to find any. Same for you, reader. If you can find articles about this, I'll add the links to this post.

Thanks for reading.




Richard D. Worth said...

Great article. Thanks for presenting it so well. You going to test FF3 and IE8 betas?

Scott González said...

Firefox BLOWS! Where are the Enigma results?

Ariel Flesler said...

Thanks man, I don't have any of those installed (they are betas).
I'll update if I get to install any. You can add your own results if you want.

I didn't know Enigma until now. I had to google it.
It'll probably have the same results as IE6.
Actually, I used Maxthon to test IE. My IE6 throws the svchost.exe to 99% CPU so it's useless.

Karl said...

Hey Ariel,
Excellent job with this article! (finally got a chance to read it.) Very interesting results. Having seen a couple other benchmarks recently, I'll bet Firefox 3 will be more in line with Safari 3 when it's released.

James I. Armes said...

Just ran the Benchmarks in FF3 Beta 5 and IE8 Beta 1. As you can see, FF3 is outperformed only by Opera and IE8 does not perform much better than IE6.

global: 40ms
local1: 51ms
local2: 49ms
local3: 37ms

global: 172ms
local1: 125ms
local2: 109ms
local3: 78ms

Ariel Flesler said...

Nice James, thanks.

Did the rest of the results went similar than mine ?
You could be running the test on a faster or slower machine than mine.

Scott González said...

Ariel, the comment about Enigma was a joke:

Ariel Flesler said...

Added Firefox 3 beta and IE 7.

claudiohs said...

I've just ran the tests in Safari 3.2 and Chrome 1.0.

global: 111ms
local1: 74ms
local2: 71ms
local3: 56ms

Chrome 1.0
global: 6ms
local1: 3ms
local2: 3ms
local3: 2ms

Ariel Flesler said...

Way to go for Chrome! :)

georgesjeandenis said...

Thanks for bluring the jQuery.ScrollTo api concept.

Why not just post the actual script clearly like all other jquery script?

Ariel Flesler said...

What? I'm not sure I understood the problem, what does this have to do with jquery.scrollTo?