14 DAY TRIAL // Microsoft will be publishing a whitepaper containing a set of guidelines designed to describe accurately the benchmarking procedure of Windows Vista. The Redmond Company revealed that the whitepaper will comprise Microsoft's own in-house methodology, and Matt Ayers, a Program Manager from the Client Performance Team delivered a preview of the procedure.
According to Ayers, Microsoft's approach to benchmarking Windows Vista involves configuring the system, preparing the system and preparing the workload. "We recommend turning off animations (of windows, menus, dialog boxes, etc) that introduce artificial delays. I love watching a menu slide down into place instead of popping up onto the screen but, in a benchmarking scenario, these animations could impact accurate measurement. This is especially true if you are timing a sequence of events. Although they improve user experience, animations may erroneously impact your responsiveness measurements," stated Ayers.
As User Account Control prompts can intervene into a workload that constantly requires elevation, they can also interfere with the Windows Vista benchmark. This is the reason why Microsoft's recommendation is to disable the UAC prompts via the security policy manager and automating the elevation process.
"Windows Vista has the ability to roll files back to previous versions which has saved my clumsy bacon on more than one occasion. Although the impact on standard users is minimal, the work that system restore does in the background may affect the repeatability of benchmarks (this really depends on the amount of disk activity)," added Ayers advising to turn off system restore.