Thursday, August 7, 2008

DNS Vulnerability Disclosure - PowerDNS - Lack of Response Considered Harmful

As an aside to my previous discussion about my own investigation into DNS, I had also been curious if there was any "lack of response" issues impacting today's DNS servers. Afterall DNS spoofing attacks generally involve a race, if you can initiate a race and leave the competor at the starting blocks, you'll always win. In my invesitgation, I started submitting malicious requests with non-standard and/or binary data in the queries. Pretty quickly I discovered that some servers were not responding to my requests even ones as simple as those including a leading space in the name for example.

While this is a seemingly benign flaw on face value, the implications given the Kaminsky style attack is that it allows even a less sophisticated attack attempt a very long window to spoof domains that are hosted on vulnerable servers. Note that at first I made an incorrect assumption that this particular flaw affected multiple DNS solutions, but after further investigation it turns out this one was in a particular implementation, namely PowerDNS. While PowerDNS is the only server I am aware of that has this flaw, I would be concerned about other malformed queries affecting other DNS implementations in similar ways. Feel free to let me know if you become aware of any other DNS servers with similar issues, I would be curious to know.

In case you are using PowerDNS the flaw I had discovered has just been recently patched (aka CVE-2008-3337), so please deploy this patch, to remove this extended time window advantage from attackers.
Thanks to Bert Hubert of PowerDNS for responding to my notification quickly, immediately seeing the importance and turning around this patch in such a short time frame given the heightened climate of attention on DNS.

Additional References:

The Critical DNS Vulnerability

Unless you've been sheltered for the last month, you are by now well aware of the important news about the critical DNS vulnerability. You have patched or otherwise protected your networks, right?

One thing that was immediately apparent about this vulnerability was the unique way that it was disclosed simultaneously by a number of vendors and their claim that the patch itself did not disclose the vulnerability (that alone may be a topic for future discussion). The fact that so many vendors were able to collaborate in secret on this vulnerability for more than 6 months before the patch was released was a clear sign to me of the importance of the discovery. It frankly surprises me that many were so skeptical at first that something "new" was found. In part I think their skepticism and unfounded early criticism of Dan was an attempt to try to lure the details out prior to the 30-day quiet period Dan had hoped for to enable patch deployment and other protections necessary as a result of this attack.

I took a different point of view. Instead of being skeptical, I actually chose to make my own attempts at deciphering the core problem that the patch was an attempt to remediate. I think I was fascinated by the fact that they released a patch with full confidence that reverse engineering the patch alone would not give insight into the real problem they had protected against. I don't think I even bothered diff'ing the patches at first because I knew what I was going to find was exactly what was said -- merely fixes to enable source port randomization.

To be honest, it didn't take me long to start deciphering the code. As I mentioned before, I've been around a few years, and I actually recall the turmoil back in 1993 and 1997 when CA-1997-22 came out (SecureWorks has a good Back History on this.). Back then it was a simple matter of predicting the next transaction ID and beating the real server to the answer. So in looking at this issue, I had that bit of understanding to go on, and below is a recount of my few days (about one and a half days on and off really) of investigation that resulted in my findings of what I believed was the crux of the issue. In the end, I found something that allowed me to spoof any domain I wanted, up to and including the root servers.

Now I'm not going to say I found the exact issue, right away, it did take a bit of investigation and review of DNS as I had not looked at this in a few years. Initially I was looking at this solely from the point of view what response would I have to successfully send to gain ownership of a domain at the level I believed that this issue clearly was. I knew that the responses had to follow the in-baliwick rules, and if you could successfully spoof one of these responses you could use the glue records to overwrite whatever you chose, including the existing A records for the name servers handling the domain. Immediately I realized how big this was, if I could overwrite the name server records in this way, I could take over .com, .net, or even the proverbial root domain, yes '.' itself. That's how DNS works afterall.

Now, as I mentioned, initially I was looking at what I'd had to spoof, and I did proof of concept spoofs by utilizing packet inspection. Obviously this was an easier attack method (if you control the network path you really don't need to spoof anyways), but I figured if I could do it that way, the remote spoof of blasting various txid's was just a coding exercise. Once I made this realization I then started looking at the birthday attacks from previous research. After getting caught up on this aspect of DNS exploitation, I started looking at crafting my own proof of concept exploit tools.

After getting past the above More awareness of the Birthday attacks made it clear to me what the attack was. The birthday attack previously was an effort to increase the chance of collision by causing a number of races at the same time with differing txids, increasing the odds for collision. But it was always done with the same query. What if you started a number of attacks at the same time against different queries? No longer would you have to worry about winning one race, you'd have the ability to start as many races as you liked. With that I began my attempts at the blind spoofing attacks by using the random in-baliwick name lookup method. Amazingly, I got this exploit to work within a few hours of coding (spoofing DNS packets is not something I ever felt the need to do before).

Now, mind you, I started working on this in my spare time Thursday and Friday after the initial announcement. I had working exploit code by the end of the week, and I couldn't imagine I was the only one to do this. In hindsight, Dan has made it clear that other researchers were able to figure this out as well.

Now, one would argue that if the good guys were able to do this so quickly, the bad guys were as well. I certainly won't disagree with that. But based on the information that has been released publicly, it is not clear if anyone actually exploited this publicly in a large way prior to the metasploit modules being released. Since they have been released, we've obviously seen some people uptake that code, or at least the concept to leverage for their own means.

I would like to say that I am glad that there are more responsible people out there who realized the scale of impact of this issue and chose to keep the information to themselves for as long as was possible. In the end, more people have patched, and we are better protected from this issue than had it been fully disclosed without this "gift of time."

Thursday, July 31, 2008

So what is all this hype about the DNS being broken?

As you have no doubt heard by now, there is a security problem with DNS or the Domain Name System that has gotten a lot of attention lately. You may be wondering exactly why all the hype, there are security problems everyday right? Well, unfortunately, this problem is at the core of the Internet -- DNS is a foundation infrastructure that allows us to use cool names like and that are easy to remember for the services that we use. You're computer uses DNS to turn the names into numbers or IP Addresses that it needs to connect with these services. The most recent flaw is unprecedented due to how easy it is to exploit, and how severe the consequences can be. Please bear with me for a minute as I attempt to explain the vulnerability without too much of the gory technical details.

The flaw works because of the distributed nature of DNS. The Internet is just far too big for a few computers to have all the addresses for every other computer or service that is connected on-line. So what happens is when your computer needs an address it asks someone else who in turn generally asks "around" for the answer for you. First your computer talks to your company's or ISP's domain name servers for, that server in-turn will ask other ".com" servers for who answers for "" and then it'll ask google's servers for the address of Because of the distributed nature just mentioned, if someone can respond to your local name server faster than the real servers do then their fake response will "win" and can provide the answer to the question "Where is" This is called DNS spoofing or cache poisoning, pointing to the fact that once your ISP's server has gotten the wrong answer, it will hang onto that information for a period of time (at the hackers choosing) and will kindly give that information to any other customers of that ISP.

Now DNS spoofing is certainly not new, it's been known about for well over 18 years with a number of improvements to the attacks discovered in recent years. However up until now, everyone thought the exposure was limited because it was a race condition that if it failed, and frequently it did, you'd have to wait a long time to try again with the same low odds of success. What makes this new flaw so critical is that the attacker holds all the cards. If they have the ability to perform a query or name lookup against a recursive server (by using a user's web browser for example), then they can continuously perform the attack until it is successful. On top of this, Dan Kaminsky made the connection that if the attacker can run this race as many times as he wants, then he can use a feature built into the heart of ALL DNS implementations that would allow him to overwrite information these caches thought they already knew about. It has been shown that this form of attack can be executed within about 10-15 seconds under the right conditions. So this is certainly something that is of great concern, and in the short time that the details have come out we have already seen a number of exploits in the wild.

Now, onto more of what I originally wanted to point out about the effects of this flaw.

Why is this such a big deal? Because everything we do on-line depends in some way on DNS. Not only is it how your browser finds the websites you read, but it is also used for so many other services. DNS was never intended to be a security platform in it's current form, however the explosive growth of the Web and connected nature of life on-line has placed unanswered expectations on DNS to be the trustworthy source for directory information. If someone can manipulate your DNS lookups and redirect your traffic to their own systems, they could perform a number of nefarious tasks including, but not limited to:

  • Steal your Usernames and Passwords to online services, bank accounts, etc
  • Inserting their own Ads or use your computer to perform Ad Clicking scams for their own profit.
  • Redirect email for entire domains to be filtered through their own servers
  • Redirect any or all web traffic for any domain through a proxy under their control (wpad auto proxy discovery is considered evil).
  • Watch your IM conversations with friends and family (if they're not encrypted).
  • Infect your computer with malware by getting it to download unsafe "fake updates" for various programs that don't verify the executables they download.
  • Other forms of Drive-by downloads that install malware - Now become a whole lot easier...
  • Breaking the trust relationships you have with certain web sites, SAAS sites, intranet resources etc.
  • Causing disruption or manipulation of other services such as NTP, FTP.
  • Potentially stealing credentials from employees of companies that allow their users to connect to webmail and SSL VPNs by first connecting to non-SSL based websites. (Majority of users will not notice they don't get magically redirected to Note that this same flaw will affect the numerous financial institutions that hide SSL from the user until they click submit on an insecure HTTP based login page. If you want it to be secure, you should only allow secure connections and educate your users about how to securely connect to you!
  • This attack is also unique because it can affect places you might not even be thinking about yet, it is a truly cross-platform attack. For example, what DNS server does your Windows Mobile device use? Your BlackBerry or iPhone? What automated connections/activities might they be doing?
Now you might be asking, why am I providing all these possibilities, won't the hackers like these ideas? They may, but the truth is most of them are already well known, I just want to raise awareness by discussing the concerns in one place. I fear that people in charge of name servers at many companies are not putting enough emphasis on getting these patches deployed, in fact we have seen some major ISPs have been slow to respond and have already been affected. The bigger they are, the more customers they have, the larger the target in these early stages of awareness. Granted, there have been performance and other deployment issues related to the patches, unfortunately there is little time left for testing, add some more resources to the problem and get it deployed!

For everyone else in corporate America, while you may think you're less of a target, or think you're protected by your firewall -- unfortunately, you are not. I suspect we will see the attacks escalate to using other web-based or mutli-part zombie attacks to compromise larger numbers of organization's name servers and in turn would increase the risk of more computers being infected with malware. So think about the resources you'd need to put behind that cleanup and get the patch deployed now.

Now, it is clear that this patch alone is not the ultimate answer, but it does give us a little more time. There are some other attacks that can still take place even with these fixes. It makes it harder, but nobody would claim impossible for other attacks to still take place. The jury is still out if the final answer is DNSSEC or something else that will have to come along to solve this problem in the long-term -- we've leveed far to much security burden on DNS over the years and it time for an overhaul.

I should also mention that the industry does need to look beyond DNS security to enable more end-to-end security as well, these DNS attacks simply make the "monkey in the middle" that much easier to perform, there are other more complicated attacks we need to prevent as well.

Please patch responsibly...

Tuesday, July 29, 2008

Security In Perspective

Welcome to my new blog -- yes, yet another blog about security. You may be asking yourself, who the heck is this guy, and is he really that audacious that he thinks we need another mono-blog about information security and that he actually does have something of interest to add to topic? Well, my answer to that is obviously a yes, as I've taken the effort to stake a claim on this tiny part of the web to express my own ideas, criticisms and possibly innovations in information technology and in particular security issues that are impacting us all.

So who am I? Well, I am an Information Security Professional who has worked in various large Enterprise and Consulting roles through my many years of experience. I guess you could say that I have been around the block a bit. I have many times considered getting more vocal about my observations and perspectives on security, but until recently, I just didn't have the inclination to do so. With some recent issue disclosures, it has become painfully clear that even the smallest subtle oversight can quickly become an issue that affects large populations of people (more on this later). I believe that like it or not, information security impacts all of us in this data-centric age, and we all should raise our awareness to the risks.

Information Security is a mindset, it is more about a way of thinking, than it is about the concept of making things safe and secure. I like to think that I am good at what I do because I creatively think about the ways that things can go wrong, be broken or bypassed. In many ways, I consider myself an inventor, and in regard to security I like to invent new ways to break things.

Through this blog I hope to share with you just how things break, and in the end hopefully I can convey the security mindset that I feel everyone must try to gain some insight into if we are to survive in this wild and crazy information based world we have built for ourselves and our children.

Hopefully I can, in my own small way, put security in perspective for my readers.

Thanks for reading...