14 DAY TRIAL // Back in November 2009, Microsoft announced that plans to kill MSNBot 1.1, as it was unveiling the new crawler (bot), MSNBot 2.0b. Since then, the company has been hard at work catalyzing the evolution of the Bing web crawler. At this point in time, the bot for the Bing search/decision engine, continues to remain in Beta development stage. However, this will change by the end of this year. Microsoft will introduce the final version of the Bing crawler by the end of 2010, and will also change the name to reflect the association with the new search engine.
“We want to announce in advance our plans to change the name of our crawler (aka user agent),” noted Dean Wierman and Fabrice Canel, Bing Crawl and Index teams. “On October 1st, 2010, we will drop the beta designation from the Bing crawler and change the name of the crawler to reflect Microsoft's new brand for search. Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be: Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm).”
Of course, there are additional changes that webmasters will be bound to notice. According to Microsoft, the new HTTP header From field will be modified to: “From: bingbot(at)microsoft.com” instead of the current “From: msnbot(at)microsoft.com.” One important aspect that needs to be underlined is the fact that the new Bingbot will still respect all of the robots.txt directives that have been set for msnbot. In this regard, webmasters will not have to “reconfigure” the robots.txt file(s).
“We always welcome feedback from the webmaster community. Over the past few months, during the recent development phase, we had different issues reported to us. We want to give our heartfelt thanks to the webmasters who contacted us, and offer our sincere apologies for any issues they may have encountered as a result of the crawler traffic. If you have any questions regarding BingBot traffic and your site, please contact us at bingbot at microsoft.com,” Wierman added.