CloudFlare's Railgun Speeds Up the Web by Caching the Uncachable

The technology uses "delta compression" to cache dynamic resources

By on February 26th, 2013 14:42 GMT

CloudFlare is only a couple of years old, but it's creating quite a stir in the web hosting space. Now, it's showing off its latest technology, dubbed Railgun, designed to speed up connections between web servers and the CloudFlare data centers.

CloudFlare is a content distribution network in essence, it caches website assets at its various data centers around the world to speed up website loading.

Images, static resources such as CSS files and JavaScript don't change that often, so they can be easily duplicated in the CloudFlare data centers.

But not all of a website's resources are cacheable, CloudFlare's estimates that, on average, 34 percent of the data served by a website is dynamic so it has to be loaded from the origin server.

Railgun tackles this problem by optimizing the transaction of dynamic content between the origin server and the CloudFlare data centers.

Railgun uses a combination of methods and compression to cache the uncacheable. It does this by analyzing resources at the byte level and only requesting a diff file from the origin file, rather than the entire resources.

That is, it only needs to know what's changed since the last time it pulled a copy of a page. Even dynamic pages don't change all that much and that often, so the actual differences are small, resulting in a significant reduction in the amount of data that needs to be pulled from the origin server.

This results in huge "compression" rates, though perhaps the term compression doesn't really apply here. If a page is accessed often and thus updated often, the changes are so small that they even fit inside a single TCP packet.

Railgun keeps an encrypted TCP connection to the origin servers active at all times, eliminating the overhead of initiating the connection for each new request.

Obviously, Railgun software needs to be installed at the origin server for any of this to work, which is why CloudFlare has partnered with many of the biggest web hosting companies in the world.

1 Comment