Twitter will allow users to export tweets, hopefully by the end of the year

Nov 27, 2012 09:50 GMT  ·  By

Most websites fight to get as much data on you as possible, more data means better advertising. What's more, the more data you can keep from your competitors the better.

On top of that, keeping users from moving their data to another service ensures they'll be forced to stick around even if they didn't want to.

This is Facebook's philosophy, the only big exception to the rule is Google with its comprehensive Data Liberation tools.

Twitter is trying to do right by its users, even as it continues to annoy developers, and now CEO Dick Costolo is promising, again, that users will be able to download everything they ever posted on Twitter if they're so inclined.

The feature isn't available yet, but it should be live by the end of the year, provided it can be completed in time. Costolo is confident that it can, but he says his engineers aren't so convinced.

The problem is that, while it may not look like it, retrieving all the tweets someone posted is a big undertaking. Twitter is not designed to access old tweets, the entire platform is geared towards real-time access of the freshest info.

To be able to allow users to export their entire archive without slowing down Twitter as a whole, its engineers had to create a new platform altogether to handle the management of older tweets.

Considering Twitter's scale, you can imagine that accessing billions and billions of tweets in an efficient manner can be hard.

One billion new tweets are created every two and a half days and the rate is increasing. It took more than three years for the first billion tweets to be published.

It's a complex project so it may not be ready by the promised deadline. This isn't the first time Costolo has talked about exporting tweets.