14 DAY TRIAL // Google has announced that its Octane benchmarking suite is getting a big update and a brand new version number to boot. Octane 2.0 comes with several new tests and a new focus on latency. The changes aim to make Octane a better surrogate for real world conditions, and to better highlight real issues that browsers are still struggling with.
"Traditional benchmarks simply measure how quickly JavaScript can be executed. Octane 2.0 additionally measures latency, which is another important aspect of performance that is concerned with the smoothness of execution," Google wrote.
"In modern JavaScript engines like V8, latency comes from two main sources: compiling JavaScript to machine instructions so it can run faster, and garbage collecting memory that is no longer used. These tasks are computationally intensive and if they run for too long can be visible to users as small hiccups and freezes of JavaScript programs," the company explained.
The increasingly noticeable latency is a direct result of JavaScript engines becoming orders of magnitude faster than their predecessors. A few years ago, JavaScript engines were so slow that any temporary performance drop went unnoticed. Modern engines don't have that luxury anymore.
So Google has modified two existing Octane benchmarks, the Mandreel and Splay tests, to now measure latency as well.
However, the search giant has also added a couple of brand new test cases which are focused purely on performance. Both are complex tests that put a lot of strain on even the fastest engines. One is based on the Microsoft TypeScript compiler, the app that converts TypeScript code into plain JavaScript.
The second is the zlib benchmark from Mozilla's Emscripten suite. This benchmark tests asm.js performance and its inclusion into Octane is a bit surprising and encouraging.
Asm.js is a subset of the JavaScript language developed by Mozilla for applications to run as near to native speeds as possible. In particular, the asm.js subset is used by the Emscripten converter, which translates C/C++ code into JavaScript. By narrowing the translation to just the few instructions asm.js supports, the converted code can run significantly faster than if it were just plain JavaScript.
But asm.js is Mozilla's baby and, in fact, came as a direct response to Chrome's Native Client technology. When the technology was first announced, Google did say that it was looking into adding support for asm.js, or rather some optimizations that would allow asm.js code to be significantly faster than regular code. Including a test for it in Octane is a clear sign that the Chrome team wants to be on par with Firefox when it comes to asm.js, which is great news for developers.