14 DAY TRIAL // Microsoft is currently rolling out a range of improvements to its Windows Live platform of services and applications. And the optimizations are by no means limited to the surface. The Redmond company began introducing the new release of Live Search family of services and solutions last week. According to Nathan Buggia, Live Search Webmaster Center, the enhancements spanning across various items in the Windows Live catalog also affect the crawler associated with Live Search.
"A few months ago we announced two new features to MSNBot to reduce the burden of crawling on your website. These were part of a series of improvements we're making to our crawler during the Spring to increase the freshness and breadth of content in our index. As part of these latest improvements, you may notice an increase in the amount of traffic from MSNBot starting over the next couple weeks," Buggia said.
In terms of search, Microsoft continues to be the bronze winner in the three horse race against Yahoo and Google. However, the Redmond company is not yet ready to throw down the towel. And while it is waiting for Yahoo to accept its $44.6 billion dollar unsolicited acquisition proposal, Microsoft continues to hammer away at its own product. The introduction of features such as HTTP Conditional GET, HTTP Compression and now Crawl-delay are illustrative examples of the efforts poured by Microsoft into Live Search and the crawler associated with the search engine.
"Setting the crawl delay reduces the load on your servers, but it also increases the amount of time it will take MSNBot to index your website (proportional to the length of the delay), and possibly make it more difficult for your customers to find your site on Live Search," Buggia added.