Benchmarking Javascript variables and function scopes

Introduction

The other day, I was doing a quick templating plugin for jQuery. It would split long strings into arrays of tokens, and then parse these, based on a map(associative array) of handlers for the different "commands" in the tokens.

The Background

The thing is, that the tokenization was done on the first call, and only the actual parsing was executed each time the template was applied.
Surprisingly, the parsing function of the plugin was always taking around 2.5 seconds!
This was a recursive function, called around 10 times. It had a short loop of 3-4 iterations per call.
This shouldn't be SUCH a heavy task, certainly not enough to take that long.
I was really confused so I started experimenting.

The Situation

There were 3 types of tokens in the template:

• Tokens that open/start a block, f.e: <?if $foo?>.
• Tokens that close/end a block, f.e: </if?>.
• Tokens that are their own block, f.e: <?elseif $bar/>.

I called this, the mode of the token. So I had 3 symbolic constants declared. All the code was wrapped with a self-executing function, to create a local context. It looked like this:

(function( $ ){
 var
     OPEN = 1,
     CLOSE = 2,
     CLOSED = 3;
 
 /* the rest of the code */
 
})( jQuery );

These constants, were compared to a 'mode' attribute of the token. This caused 1 reference to each of them, on every loop in the parse function.

A Small Change

I tried tried any possible optimization, but Firebug's stopwatch was stuck at 2500ms.
I was nearly desperate, so I started changing things at random.
One of those changes, was modifying the way I was saving the mode of a token. Instead of storing the integer in a 'mode' attribute of the token, I used 'open','close' and 'closed' as boolean attributes.

There's no way to describe my happiness, when Firebug's stopwatch showed ~230ms instead of ~2500ms.
I must say I was really surprised and confused, this was such a small change, and one that I'd never had guessed.
After this, I decided to clear my doubts with a consistent benchmark.

The Benchmark

The first thing I did, was to make my "own Firebug". I created a script with a stopwatch, a logging method and an easy way to benchmark many sequential calls to a function, with many attempts, generating an average and finding the lowest.
After that, I set up the environment. Declared a global variable, and a function similar to the one showed above. Then nested another equal function inside it, and nested another one inside the latter.
In the end, it looked like this:

var global = 1;

(function(){
   var local1 = 1;

   (function(){
      var local2 = 1;  

      (function(){
         var local3 = 1;

         /* here I executed the benchmark */  
  
      })();  
   })(); 
})();

This created 4 variables, 1 global, and 3 local. All of them could be accessed from where the benchmark was being ran, but they belonged to different scope chains. I benchmarked 4 functions, each of them accessed one of these variables 400,000 times with a loop.
Each benchmark was tested 5 times, in order to find an average and the minimum. The latter should be the one that really matters. I tested on all 4 major browsers, on my Windows XP.

The Results

The numbers were surprising, but not in all browsers. Opera 9 seemed to be really indifferent to the scope chain, all took the same time. Firefox 2 was really the slowest. Safari 3 and IE6 were in the middle.
These are the numbers, I'll only mention the lowest from the 5 attempts.

Firefox 2
* global: 969ms
* local1: 672ms
* local2: 359ms
* local3: 78ms

IE 6(=7)
* global: 187ms
* local1: 171ms
* local2: 125ms
* local3: 94ms

Safari 3
* global: 125ms
* local1: 93ms
* local2: 78ms
* local3: 62ms

Firefox 3 beta 5
* global: 41ms
* local1: 48ms
* local2: 48ms
* local3: 37ms

Opera 9
* global: 31ms
* local1: 31ms
* local2: 31ms
* local3: 31ms

Opera is the clear champion, for some reason, the scope doesn't affect the timing at all.
It does affect the rest of the browsers. It will be specially noticeable in Firefox. Note that Firebug was turned off while benchmarking Firefox, it did affect the perfomance but not THAT much.
UPDATE:Added IE7 and Firefox 3 beta, congratz Mozilla, much better!

The Conclusions

Referencing might take time.

Using variables that were declared on different scope chains, takes longer than those that are local to the executing function.
If you are going to use one very often inside a deeply nested function, you should save it in a local variable first.
One common situation is when using the global variables window and document.

It does matter, but it's not always crucial.

Although it does make a difference, you shouldn't become paranoid about this.
These numbers were produced by 400,000 iterations. You probably don't access a variable that many times. Optimizing won't yield a noticeable improvement most of the time.
There are some situations where you should be careful though

• Within loops, if you have a long loop, caching the variable could improve the perfomance greatly.
• Inside frequently called functions. This is similar to loops.
• Inside functions, that are deeply nested inside others. If these reference a distant variable, many times, it could hit on perfomance too.

Benchmarking is fun!

I always love running benchmarks, feel free to try this yourself. You'll find the link below.
The tool used for the benchmark (benchmark.js) can be taken for your own experiments.
If you try other browsers/OS, feel free to post the results, I'll include them so others can see.

I tried to find links about this, but none showed up. I'll update if I get to find any. Same for you, reader. If you can find articles about this, I'll add the links to this post.

Thanks for reading.

Links

• Benchmark

Downloads

• Benchmark files and demo