New defect classifiers will help refine the relevance of answers

Aug 21, 2012 12:38 GMT  ·  By

Experts at Redmond-based software giant Microsoft detail in a new blog post how their search engine, Bing, minimizes the amount of so-called defective results it provides to users' search queries.

One of the most important components needed to ensure an accurate return – that also reflects the user's intention – is the defect classifier, a part of the search engine's internal indexes that ensure the nature of the returns reflects what the user was interested in in the first place.

A good example is the word elephant, the recent blog post indicates. People may be looking for either images of the animal, the Wikipedia page dedicated to elephants, or a video showcasing the majestic creature in action.

However, Bing also indexes product-related results. These are also trying to get as high up in the result chart as possible. What a defect classifier does is determine when instances of this are about to occur, and prevent them from doing so.

In other words, it stops the commercial result from being displayed at the top of the return page. This problem indicates a systemic problem that is endemic to web search results based solely on the percentage of clicks that a certain webpage receives.

If enough people click on a link, it may show up high in the result chart even if it is not the most accurate response to what people are looking for. This is why Bing uses other factors (in addition to click rater) in order to determine the most relevant answers.

“Our defect classifier uses multiple other signals besides click rate, including how people have historically engaged with a specific query category, and determines that this answer is not relevant for this query relative to other results and blocks it from the page,” the new blog post explains.

“By marking the answer as a defect, we are able to ensure that we’re matching the results more closely to the original query intent,” it goes on to say.

Microsoft engineers are currently working on building even more specialized defect classifiers. Though this involves a lot more effort, it beats having to rely exclusively on the answer algorithm.

“Bing is committed to delivering high quality results as quickly as possible. To that end, minimizing answer defects speeds up the page and removes irrelevant content. In the end our goal is to help you spend less time searching and more time doin,” says Bing principal development manager, Dr. Kieran McDonald.