Tuesday, February 14, 2012

Xmission - DNS (Restricting Recursive Lookups)


Today Xmission restricted recursive lookups for non Xmission network users. Essentially this broke DNS resolution for many individuals that I know.  The solution is to simply change the DNS servers from 198.60.22.2, 198.60.22.22 to the DNS servers supplied by your ISP or one of the following; 
  • Google (8.8.8.8 and 8.8.4.4)
  • OpenDNS (208.67.222.222, 208.67.220.220)
  • DNSAdvantage (156.154.70.1, 156.154.71.1)

After contacting Xmission multiple times, it became apparent that they weren't interested in explaining what their rationale was for this change. They only indicated that it was in response to a "back end issue" that they were trying to resolve.  This opens up speculation as to what may have been the cause, primarily a DDOS or Cache poisoning attack.

Technical Q&A
What is DNS?
DNS stands for Domain Name System. DNS servers are a critical part of the network infrastructure and the Internet at large. These servers contain information pertaining to every host on the Internet, and are the mechanism that allows information on the Internet to be available when you enter a URL in your Web browser.  An example of recursive DNS is when someone who subscribes to an ISP (e.g. Comcast) configures their computer to use the Xmission DNS servers rather than their ISPs DNS serves to access the Internet.
What is DNS Recursion?
DNS recursion is when the DNS server does not know the IP address of an Internet name but queries other DNS servers to look up the name. 
What security risks are involved in Recursive DNS?
  • DDoS attacks. Name servers can be used as distributed denial of service (DDoS) attack amplifiers (the attacker sends a small spoofed UDP name service query to an open name server, forging the victim's IP address; the open name server then returns a large "answer" to the forged IP address even though the victim didn't actually make the DNS query in the first place). If this is done on an ongoing basis with a large number of open name servers, it can flood the victim's IP address with responses from thousands (or tens of thousands) of name servers, thereby exhausting the victim's available network bandwidth).  Attacks of this sort can result in multi-Gbps flow volumes.
  • Cache poisoning attacks. Attackers can generate spoofed traffic to open recursive DNS servers that can result in so-called "cache poisoning" attacks, whereby vulnerable caching name servers can be made to return bogus results for a user's name service queries.
In a nutshell: The attacker "primes" the caching name server to respond to queries with an IP address of his/her choice, rather than the real/normal IP address for that site. The innocent victim asks the caching name server for the IP address of a site of interest, such as the IP address of their bank's Website. If the domain name of that site happens to be one that the attacker has poisoned, the victim is automatically and transparently misdirected to a Website of the attacker's choice rather than to their bank's real Web page, and confidential data can then be stolen (some refer to this type of attack as "pharming").
A variant of this attack uses cache poisoning to redirect queries for popular sites (such as google.com or hotmail.com) to a site that contains a virus or other malware. If your caching name server has been poisoned, when you try to visit one of these popular sites you can unknowingly be redirected to another site that stealthily tries to infect your PC with malware.

1 comment:

Todd said...

2/24/2012 - Update from Xmission

DNS Policy Change
-----------------

On Tuesday, February 14th at approximately 1:00 p.m., XMission began
limiting DNS server access so only our customers could reach them. This
effectively blocks DNS service to the outside world, as is the industry
norm due to abuse by outside attackers.

This change affected a small number of people, including former customers
who didn't update DNS settings for their network when they moved to a
different Internet provider and some existing customers who purchase other
services from XMission but not connectivity.

NOTE: if you have not noticed any issues connecting to web sites then you
can disregard this announcement.

Technical Details
-----------------
While it is always best to point to name servers run by your Internet
provider, XMission continued to keep our name servers open to the world as
a courtesy. In recent years though, attackers regularly use publicly open
name servers to perform DDoS (Distributed Denial of Service) DNS
amplification attacks by spoofing DNS lookups. Over time, we made changes
to make our name servers more redundant and robust but last year setup
rules to limit the number of requests per second to our DNS servers using
fail2ban. This largely worked for a time but last week we decided that it
was best to finally restrict DNS server access to connectivity customers
to ensure that all of our services and customers who rely on XMission's
name service would receive it reliably.

Conclusion
----------
XMission has always had a philosophy of contributing to the community and
we have done that in many ways over the years. Open name service has been
something we have been providing since 1993. In other cases, a customer
might purchase some products from us but perhaps not connectivity, so
using our DNS service could be convenient. In general, you get name
service from whomever you purchase your Internet connection through and
you configure your computers to resolve domain names into IP addresses.

We apologize to anyone who was affected by this unannounced change. We
simply had no choice and needed to restore 100% reliability and
performance to our DNS service since so many of our services rely on it.

If you have experienced any DNS issues since February 14th, and are not
directly connected to XMission's network, we recommend that you check to
ensure you are instead pointing to your own connectivity provider's name
servers or one of the remaining open DNS servers still available.

------------------------------------------------------------------------------
This has been an XMission Announcement. Past announcements available at:
WWW - http://www.xmission.com/about/announcements
Home - http://www.xmission.com
Portal - http://home.xmission.com
Status - http://stats.xmission.com/netstatus