Yahoo Open Sources Internal 'Traffic Server'

Yahoo has been sending mixed messages and seems to be lagging at the corporate level, but, from a technology point of view, it's a very nimble and surprisingly open company. It's involved in several high-profile open source projects, most notably Hadoop, and now it's opening up one of its most trusted pieces of technology, the so-called Traffic Server, which powers many of its services.

The Traffic Server is used to speed up the load times of its pages by using several caching techniques. The HTTP web proxy cache server is deployed extensively and it now serves over 30 billion web objects and 400 terabytes of data per day and this with only 100 to 150 servers dedicated to the task.

Yahoo claims that it can handle 30,000 requests per second per server. The company uses it for a variety of tasks, but it evolved from its original purpose as a proxy cache server and, today, it's also used for load balancing and traffic management.

The technology was originally acquired by Yahoo when it bought Inktomi, which would later be used to power Yahoo Search. The Traffic Server is a robust piece of software with over 200,000 lines of C++ code and a very flexible API. Yahoo has already presented the Apache Foundation with the code and the Traffic Server will live on as an open source project. The code is now available through Apache's Incubator program.

Yahoo has also announced an update of the internal Hadoop version, which it also released as an open source project in June. Hadoop is an open source cloud computing platform used for large-scale distributed projects. Yahoo, one of the biggest contributors to the project, has been using a modified version internally to power many of its services. The company is now announcing that it will update the Hadoop code already released with new features and a number of bug fixes.