twisted dns doesn't work

I'd like to know why the following doesn't work.

from twisted internet import defer, reactor
from twisted.python.failure import Failure
import twisted.names.client

def do_lookup(do_lookup):
    d = twisted.names.client.getHostByName(domain)
    d.addBoth(lookup_done)

def lookup_done(result):
    print 'result:', result
    reactor.stop()

domain = 'twistedmatrix.com'    
reactor.callLater(0, do_lookup, domain) 
reactor.run()

Results in:

result: [Failure instance: Tra开发者_开发技巧ceback
(failure with no frames): <class
'twisted.names.error.ResolverError'>:
Stuck at response without answers or
delegation ]

As of this writing, this fails on Windows, since it uses an invalid path for the windows host file (in twisted.names.client.createResolver. It uses 'c:\windows\hosts'. This was fine for windows versions 98 and Me (reference here), but would fail with a version as "modern" as XP.

Today, it should probably use something like:

hosts = os.path.join(
                     os.environ.get('systemroot','C:\\Windows'),
                     r'system32\drivers\etc\hosts'
                    )

I think this only partly resolves the issue though (or maybe this is a red herring).

This will only work now for names actually specified in this hosts file. What it likely needs to do is some sort of registry query for the DNS server, then query that for the actual DNS name lookup.

This recipe looks promising for getting the actual DNS server.

Correcting your example to the following, so that it is syntactically valid:

from twisted.internet import reactor
import twisted.names.client

def do_lookup(domain):
    d = twisted.names.client.getHostByName(domain)
    d.addBoth(lookup_done)

def lookup_done(result):
    print 'result:', result
    reactor.stop()

domain = 'twistedmatrix.com'
reactor.callLater(0, do_lookup, domain)
reactor.run()

I get:

$ python so-example.py 
result: 66.35.39.65

So, to answer your question: your local DNS environment is broken, not twisted.names. Or maybe there's a bug. You'll need to track it down further.

I did some digging why an explicit client.createResolver(servers) wasn't working on our corporate windows machines. In the guts of Twisted's createResolver is a os-dependant branch:

def createResolver(servers=None, resolvconf=None, hosts=None):
    ...
    from twisted.names import resolve, cache, root, hosts as hostsModule
    if platform.getType() == 'posix':
        if resolvconf is None:
            resolvconf = '/etc/resolv.conf'
        if hosts is None:
            hosts = '/etc/hosts'
        theResolver = Resolver(resolvconf, servers)
        hostResolver = hostsModule.Resolver(hosts)
    else:
        if hosts is None:
            hosts = r'c:\windows\hosts'
        from twisted.internet import reactor
        bootstrap = _ThreadedResolverImpl(reactor)
        hostResolver = hostsModule.Resolver(hosts)
        theResolver = root.bootstrap(bootstrap)

    L = [hostResolver, cache.CacheResolver(), theResolver]
    return resolve.ResolverChain(L)

The first warning sign for Windows is that argument servers is not used, so custom-DNS servers are ignored. Next is that the _ThreadedResolverImpl(), which uses platform specific code, is wrapped in a root.bootstrap() before being added to the resolver chain.

The purpose of root.bootstrap is to use the platform resolver to lookup a.root-servers.net, b.root-servers.net, etc. using the synchronous Windows platform resolver (which works - and returns IPs), and then do direct DNS queries against the root servers. The UDP packets fired off to root servers are then blocked by our corporate firewall - I see them in Wireshark.

The default getResolver() call used by names.client.getHostByName() invokes createResolver() directly, which may again result in this broken setup bootstrapped by a working DNS setup. For most environments I think adding the Resolver (with root or explicit servers) into the ResolverChain along with _ThreadedResolverImpl as a fallback would work - except that the platform resolver is a different interface.

Here's an example of detecting the local DNS servers asynchronously (by parsing the output of ipconfig) then installing a custom resolver to use them.

import sys

from twisted.python import log
from twisted.names import root, hosts, resolve, cache, client
from twisted.python.runtime import platform

from twisted.internet import reactor
from twisted.internet import utils
import re

def parseIpconfigDNSServers(output):
    servers = []
    found = False
    for line in output.split('\n'):
        if 'DNS Servers . . . . . . . . . . . :' in line or (found and not '. . . .' in line):
            servers.append(line[38:].strip())
            found = True
        else:
            found = False
    log.msg( 'Windows: Detected DNS servers %s' % (str(servers)))
    return servers

if platform.getType() != 'posix':
    d = utils.getProcessOutput(os.path.join(os.environ['WINDIR'], 'system32', 'ipconfig.exe'), ["/all"])
    d.addCallback(parseIpconfigDNSServers)
    d.addCallback(lambda r: client.Resolver(servers=[(h, 53) for h in r]))
    d.addErrback(log.msg)
    theResolver = root.DeferredResolver(d)
    client.theResolver = resolve.ResolverChain([cache.CacheResolver(), theResolver])

if __name__ == '__main__':
    log.startLogging(sys.stdout)
    def do_lookup(domain):
        d = client.getHostByName(domain)
        d.addBoth(log.msg)

    from twisted.internet import reactor
    reactor.callLater(0, do_lookup, 'example.com')
    reactor.run()

I've tidied this up and posted it here https://gist.github.com/shuckc/af7490e1c4a2652ca740