The massive 300Gbit-a-second DDoS attack against anti-spam non-profit Spamhaus this week didn’t actually break the internet’s backbone, contrary to many early reports.
The largest distributed denial-of-service (DDoS) assault in history began on 18 March, and initially hit the Spamhaus website and CloudFlare, the networking biz hired by the spammer-tracking outfit to keep its systems online, at 90Gbps. After failing to knock the organisation offline, the attackers targeted CloudFlare’s upstream ISPs as well as portions of the networks at internet traffic exchanges in London and Amsterdam.
The volume of this second-wave attack, which began on on 22 March, hit 300Gbps, an unnamed tier-1 service provider apparently told CloudFlare.
By far the largest source of attack traffic against Spamhaus came from DNS reflection, which exploits well-meaning, public-facing DNS servers to flood a selected target with network traffic – this is opposed to the usual tactic of using a huge botnet army of compromised computers.
DNS reflection attacks involve sending a request for a large DNS zone file to a DNS server; the request is crafted to appear as though it originated from the IP addresses of the victim. The server then responds to the request but sends the wad of data to the victim. The attackers’ requests are only a fraction of the size of the responses, meaning the attacker can effectively amplify his or her attack by a factor of 100 from the volume of bandwidth they control.
CloudFlare reckons there were 30,000 DNS servers involved in the attack against Spamhaus, which might have been launched from only a small botnet or cluster of virtual servers. The attack against Spamhaus and CloudFlare proved there is a serious design flaw in the underpinnings of the internet, one that security experts such as Team Cymru and others have been warning about for years – although the use of DNS servers in DDoS attacks is rare, Rob Horton from NCC Group told El Reg.
The open DNS server problem is both a huge and under-reported issue involving 21.7 MILLION DNS resolvers that can be abused to launch equally ferocious attacks in future.
But the good news is that fixing the problem only requires small changes in configuration files that take only minutes. Everybody El Reg has spoken to agrees there’s a problem with open DNS servers with some even suggesting the easily abused resource may replace botnets as a launchpad for DDoS attacks.
Joakim Sundberg, security solutions architect at security appliance maker F5, commented:
The Spamhaus attack is a demonstration of the kind of DDoS attack I have been expecting for some time: DNS Reflection. DNS Reflection attacks will play a more prominent role in DDoS attacks in the future.The major driver for this kind of attack is the decreasing number of bots available for rent, with the authorities more effectively cracking down on major botnets. With a lower number of bots now available, hacktivists and other cyber criminals are finding new ways in which to amplify their attacks.
However there’s deep disagreement about to what extent, if any, the DNS reflection attack thrown against Spamhaus and CloudFlare affected the internet more generally.
CloudFlare’s take of The DDoS That Almost Broke the Internet can be found in a blog post that the states the attacks against it and Spamhus eventually spilled over to knacker internet connections across Europe:
Over the last few days, as these attacks have increased, we’ve seen congestion across several major Tier 1s, primarily in Europe where most of the attacks were concentrated, that would have affected hundreds of millions of people even as they surfed sites unrelated to Spamhaus or CloudFlare. If the internet felt a bit more sluggish for you over the last few days in Europe, this may be part of the reason why.
Even the websites of large corporations or hosting providers would be swept away by an attack of this intensity, judging by CloudFlare’s rhetoric. However, this 300Gbps of traffic amounts to heavy congestion on a slip road that didn’t hold up the main flow of traffic across the interwebs.
We understand a massive dip in a graph of traffic flowing through the London Internet Exchange (LINX) on 23 March, a graphic included in CloudFlare’s blog post, is due to a data-plotting glitch and NOT due to the effects of the attack.
Spamhaus compiles lists of IP addresses of servers and other computers accused of distributing spam or promoted using junk mail. These blacklists are used by ISPs, businesses and spam-filtering firms to block the worst sources of unsolicited marketing mail before applying more computational intensive filtering techniques, such as analysing the actual content of messages.
Junk-mail distributors and the like regularly threaten, sue or DDoS Spamhaus. Some businesses also object to Spamhaus’s alleged vigilante approach to tackling spam.
Spamhaus’s blocklists are distributed via DNS and are widely mirrored in order to ensure the overall system is resilient to attacks. The blacklists were never affected and were even updated, with none of its core infrastructure going titsup, according to Spamhaus.
“Only the website and our email server were affected,” Steve Linford, chief executive for Spamhaus, told the El Reg. “All Spamhaus DNSBL [DNS Block List] services continued to run unaffected throughout the attack. In fact Spamhaus DNSBLs have never once been down since we started them in 2001.”
Linford praised the support of engineers at CloudFlare and Amazon, which supplied load balancing of DNS services, for ensuring its service remained available during the packet carpet bombing. He claimed the attack caused Netflix to slow down and caused congestion elsewhere on the web.
However internet traffic exchanges in both London and Amsterdam – two of the top three peering hubs in Europe, the arteries of the internet – both played down the impact of the attack beyond CloudFlare and its customers.
Malcolm Hutty, head of public affairs at LINX, the London Internet Exchange, said: “Apart from CloudFlare we saw a minor amount of collateral congestion in a small portion of our network which may, or may not have, have affected some members. This would have been accommodated through their normal procedures.”
Ordinary internet users would not have been affected because the DNS flood “only have affected CloudFlare and its customers”, he added.
CloudFlare uses Anycast technology which spreads the load of a distributed attack across all 23 of its data centres. Even so it was left reeling from the weight of the assault, which prompted it to suspend its peering in London.
Overblown reports that the internet slowed down or ground to halt appear to be well wide of the mark. This is not to dismiss the significance of the attack, or take anything away from CloudFlare for helping Spamhaus to weather the storm. The simple fact is the attack amounted to nothing more severe than minor congestion, an assessment backed up by AMX-IX, the Amsterdam internet exchange as well as its counterpart in London.
“We have not experienced any disruptions related to our platform,” a spokeswoman for AMX-IX told El Reg. “When we look at the amount of traffic some of our members and customers exchange we see some increases here and there, but they could easily manage it.”
The New York Times claimed that the attacks against Spamhaus appear to be tied to a dispute with CyberBunker, a website hosting provider in the Netherlands. CyberBunker is accused by Spamhaus of being the world’s most toxic haven of phishing and malware.
CyberBunker is quite open in running a bullet-proof anonymous hosting facility out of a Cold War bunker in the Netherlands where anything goes except child-abuse material and terror-related websites. “Customers are allowed to host any content they like, except child porn and anything related to terrorism,” its online policy states.
The hosting provider told El Reg it denies any involvement in spamming. It declined to respond directly to the accusation in the NYT article that CyberBunker was retaliating against Spamhaus for “abusing its influence” and using vigilante tactics in the fight against spam:
The only thing we would like to say is that we (including our clients) did not, and never have been, sent any spam. We have no further comment. Thank you.