A TCP stack design vulnerability could put Internet services everywhere at major DoS risk

Oct 2, 2008 14:02 GMT  ·  By

Researchers from the Swedish security company Outpost24 have discovered a major flaw in the design of TCP stacks which could put TCP-enabled devices at risk. According to their findings, which they kept secret for three years, performing DoS attacks through this vulnerability would require little bandwidth and the attacked devices would need rebooting in order for proper functionality to be restored.

The researchers discovered this vulnerability back in 2005 and they claim it affects most, if not all, TCP stack implementations out there. With every TCP-enabled device being at risk, this affects everyone on the Internet, ranging from billions of home users to ISPs, hosting and other companies. Due to the deep implications of this vulnerability, the researchers decided to keep it secret and try to find applicable solutions. Because they did not succeed in coming up with an answer for mitigation, they decided to go public and search for input and feedback from the security community.

They did not disclose full details to the general public, but they teamed up with the Finnish CERT and started providing information to various vendors world-wide, which include major OS developers and router/firewall manufacturers. Along with information regarding several different attack types, the researchers provided the vendors with a stress test kit which they developed and named Sockstress. “They're still trying to do triage and understand the individual attack types that we've identified for them. We're still trying to get them to back up a step. It's a class of attack, not necessarily individual things that the vendors need to be focusing on," said Robert E. Lee, Chief Security Officer at Outpost24.

The researchers tested their attacks on 15 different TCP stacks and all proved vulnerable. "We haven't found anybody who has a TCP stack that runs TCP based services that isn't vulnerable. If they make a TCP stack then it's probably still going to be vulnerable to one or all of these attacks because this is something fundamental in how TCP works," commented Mr. Lee for The Register.

Even if, at this point, there is some skepticism coming from the security community regarding the seriousness of the vulnerability, this is explainable through the lack of detailed technical information or confirmation from other parties. “My immediate reaction is skepticism: things like this tend to be hype. However, after listening to their audio interview, I believe they are probably right. They have been working deep within TCP stacks. If such problems exist, then they would have certainly come across them,” wrote on his blog Robert Graham, the CEO of Errata Security.

A new DoS attack technique

From what little information is available in an interview with Robert E. Lee, Chief Security Officer at Outpost24, the exploitation of this flaw consists of essentially tricking the TCP stack into not closing the TCP connections. By opening numerous such connections that are never dropped, the max limit of simultaneous connections that the stack can handle is eventually reached. This uses up all the available resources and causes a denial of service situation.

This actually works backwards compared to the classic distributed DoS (DDoS) attacks where, by using the combined bandwidth of multiple compromised machines (botnets), the attacker opens numerous connections and sends numerous packets in order to force the device into using up all its resources by attempting to process them. With such classic DoS attacks, there is a direct proportionality between the resources available to the attacked device and the bandwidth required to crash it successfully. However, in one of the new attacks described by the Mr. Lee, the concept is to trick the device into thinking that your connection is getting slower. In fact, the slower, the better.

A TCP stack will attempt to determine the maximum speed of a connection and adjust the speed at which it sends the packets in order to prevent packets from being dropped. So, if successfully tricked into thinking that your speed is getting so slow that it will take, for example years for the transfer to complete, the stack will keep the connection open for a very long period of time. Then, it's just a matter of opening enough such connections up to the point where the stack can't handle any more and becomes unresponsive. But, while a device can recover on its own within minutes from a classic DDoS attack, with this new technique, the only solution is to reboot the device. "It basically self thrashes, and the only recovery after about two to four minutes worth of attack flow, even after the attack stops, is to reboot the machine," explains Mr. Lee. Another Internet architecture design flaw

This TCP resource leak vulnerability is the second major Internet architecture design flaw made public in 2008. A few months ago, security researcher Dan Kaminsky discovered a critical flaw in the design of DNS (domain name system). That vulnerability allowed for an attacker to poison the cache of DNS servers with fake entries. Just like this flaw is supposed to affect all TCP stack implementations, the DNS one affected all DNS server platforms. That posed a great mitigation problem, but Mr. Kaminsky silently worked with several major affected vendors and companies that offer Internet-based services and eventually a patch was developed.

Facing pressure from the security community, Dan Kaminsky decided to make the technical details public after the patch was deployed on most of the DNS servers world-wide. He revealed at the time that the flaw also affects services like e-mail. Not long after, Russian physicist Evgeniy Polyakov presented a proof of concept exploit demonstrating that the patch only increases the time required to compromise a DNS server and that it doesn't really block attacks.

Miscreants like phishers and malware developers have already adopted the DNS vulnerability and complex attacks have been carried out. While a permanent solution is still being devised, the U.S. government announced plans to implement DNSSEC, an encryption-based DNS service, on their network by December 2009. All this puts into perspective what could happen if the TCP stack flaw issue is not properly addressed.