14 DAY TRIAL // Twitter updated its mobile apps, again, with a few changes aimed at streamlining and simplifying the experience. In that vein, there's also a new search which now includes tweets, photos, news, people all mixed together.
It may seem like just a visual change, but Twitter had to re-engineer the search engine for this and did a rather thorough job of that.
The two big challenges to doing this unified search were how to rank the results and what to mix in for different queries. Twitter solved both these problems with an ingenious solution.
When a query is received, lists of results for all types of data are compiled, one for tweets, one for news, one for people and so on.
Same type results are ranked according to their relevance. They are then given a universal score which is then used to rank different result types against each other. So if one piece of news has a high score it gets to go first.
The search engine also penalizes changes in data types between consecutive results, so the second result may still be a news item, even if its uniscore is lower than that of the first tweet result for example. The penalization is in place to prevent results from being too mixed and confusing to users.
The new ranking algorithm also deals with a sudden surge of interest in one data type or piece of info. That's a particular challenge for Twitter as the nature of what's relevant on the site changes from hour to hour, much faster than for a regular search engine.
"Certain types of content may not have many relevant items to show for a particular input query, in which case we may choose not to include this type of content in search results," Twitter explained.
"In other cases, for instance if query volume or matched item counts have an unusual spike (what we call a 'burst'), we show this type and may also boost it to appear at a higher place in the results," it added.