Mozilla's New JavaScript Benchmark, the Kraken, Released

Just last week Mozilla was showing off Firefox 4's improvements in JavaScript performance, while at the same time noting the limitations of synthetic benchmarks.

Turns out that post may have been a little more self-serving that it first seemed, Mozilla has now released the Kraken, a new browser benchmark which it says should be more relevant for real-world performance.

"We’re pleased to announce the first version of Kraken, a new browser benchmark. More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications," Mozilla's Rob Sayer announced.

"We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward," he explained

"These are the things that people are saying are too slow to do with open web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available," he added.

Mozilla says that the aim with its new JavaScript benchmark was to focus on real-world applications. Of course, that's what all of the benchmark makers are saying.

Still, Mozilla believes that its approach is the most relevant as it tests for real use-cases, though it doesn't detail what these are.

However, if the benchmark manages to do that, it should be a lot more relevant to the users than SunSpider or V8 for example.

Mozilla says that developer spend too much time optimizing their JavaScript engines to perform well in synthetic benchmarks and that these improvements rarely translate into a significant boost for the user.

Mozilla plans to evolve the new Kraken benchmark quickly by diversifying the type of tasks it performs. Since this is just the first release, we should be seeing some big improvements in a relatively short period of time.

Kraken is of course completely open source so anyone can inspect and modify it. In fact, it uses some code from SunSpider itself.