Making it hard to come up with relevant results for all queries

Jun 5, 2012 12:50 GMT  ·  By

In the early days Twitter search was mostly useless. Today, it's not that different. For a site where search would be so important, it's hard to understand why it just isn't.

Well, it turns out the problem is much bigger and more difficult than it would seem and that Google has conditioned us to think.

Google's take is a hard one, scour the web and pop up the most relevant information, especially in the fast moving web of today. Even so, the vast majority of websites don't change that much or that often.

What's more, while Google sees new queries every day, by volume, the most popular ones are largely the same (and quite depressing like "facebook.com" or even "google").

Twitter queries change by the hour though, in fact, almost one in five top popular queries changes. Of the top 1,000 queries, 170 won't be there in the next hour.

The same is true on a day-to-day basis, 13 percent of top 1,000 queries one day will be gone in the next. And this is for a regular day, when major events happen, things change dramatically fast.

Of course, the relevant results for any of those queries change just as fast that you can understand why Twitter has a hard time. But it is working on it.

"When news breaks, Twitter users flock to the service to find out what's happening. Our goal is to instantly connect people everywhere to what's most meaningful to them; the speed at which our content (and the relevance signals stemming from it) evolves make this more technically challenging, and we are hard at work continuously refining our relevance algorithms to address this," Twitter explained.

The team has created a paper on visual representation of large search data sets, which helps it better understand queries as well as when and why people search on Twitter.