I have written some code that queries adns. Problem with this code is that it gets stuck, how? Let me explain it:

• Say my dnslist is ["8.8.4.4", "8.8.8.8", "208.67.220.220", "208.67.222.222", "192.168.50.1"]
• It would pop a dns from the list and query againt it, now that means that DNS will be queried in reverse order
• No matter what i do, It never shows results from the dns it picked up first (in our case 192.168.50.1)
• I was not sure if that dns ever replied so
  • First i changed DNS list to contain just that last DNS Server and code executes fine
  • Second i used the old list with 5 DNS servers except that the last one was managed by my so i could track if code even queries it or not, and to my surprise the query does take place.
  • So query is made, we get result but that result is never inserted into resolved_hosts for some reason, and because that results is not inserted, its length will remain less than the length of dnslist, causing a infinite loop.

What do you think could be causing this problem and how to solve it?

Code Execution Results

Inside class's init'
Data
host www.yahoo.com
dnslist length 5
intensity 1
Inside resolve()
inside finished_resolving()
Resolved : 0/5
Inside 'while not finished_resolving'
Queue: 0/1
Launching Querying for www.yahoo.com/1 on 192.168.50.1
Queue: 1/1
Launching Querying for www.yahoo.com/1 on 208.67.222.222
inside collect_results()
inside finished_resolving()
Resolved : 0/5
Inside 'while not finished_resolving'
------------------------ CLIPPED ----------------
Inside 'while not finished_resolving'
inside collect_results()
Inside collect_results's for query in self.adns.completed()
DNS used was208.67.222.222
Answered : (0, 'any-fp.wa1.b.yahoo.com', 1286169807, ('69.147.125.65', '67.195.160.76'))
Resolved www.yahoo.com to 69.147.125.65 using 208.67.222.222
Resolved hosts count1
And they are: 
{'208.67.222.222': '69.147.125.65'}


inside finished_resolving()
Resolved : 1/5
Inside 'while not finished_resolving'
Queue: 1/1
Launching Querying for www.yahoo.com/1 on 208.67.220.220
inside collect_results()
inside finished_resolving()
Resolved : 1/5
Inside 'while not finished_resolving'
-------------------------- CLIPPED --------------------
inside collect_results()
Inside collect_results's for query in self.adns.completed()
DNS used was208.67.220.220
Answered : (0, 'any-fp.wa1.b.yahoo.com', 1286169790, ('67.195.160.76', '69.147.125.65'))
Resolved www.yahoo.com to 67.195.160.76 using 208.67.220.220
Resolved hosts count2
And they are: 
{'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76'}


inside finished_resolving()
Resolved : 2/5
Inside 'while not finished_resolving'
Queue: 1/1
Launching Querying for www.yahoo.com/1 on 8.8.8.8
inside collect_results()
inside finished_resolving()
Resolved : 2/5
Inside 'while not finished_resolving'
-------------------------- CLIPPED --------------------
inside collect_results()
Inside collect_results's for query in self.adns.completed()
DNS used was8.8.8.8
Answered : (0, 'eu-fp.wa1.b.yahoo.com', 1286169758, ('87.248.122.122',))
Resolved www.yahoo.com to 87.248.122.122 using 8.8.8.8
Resolved hosts count3
And they are: 
{'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76', '8.8.8.8': '87.248.122.122'}


inside finished_resolving()
Resolved : 3/5
Inside 'while not finished_resolving'
Queue: 1/1
Launching Querying for www.yahoo.com/1 on 8.8.4.4
inside collect_results()
inside finished_resolving()
Resolved : 3/5
Inside 'while not finished_resolving'
-------------------- CLIPPED -------------------------------------
inside collect_results()
Inside collect_results's for query in self.adns.completed()
DNS used was8.8.4.4
Answered : (0, 'eu-fp.wa1.b.yahoo.com', 1286169757, ('87.248.122.122',))
Resolved www.yahoo.com to 87.248.122.122 using 8.8.4.4
Resolved hosts count4
And they are: 
{'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76', '8.8.8.8': '87.248.122.122', '8.8.4.4': '87.248.122.122'}


inside finished_resolving()
Resolved : 4/5
Inside 'while not finished_resolving'
inside collect_results()
inside finished_resolving()
Resolved : 4/5
---------------- CLIPPED -------------------------------

(last block keeps repeating until system starts to hang up, load goes upto 24)

Code

test.py

import adns
from time import time
from async_dns import AsyncResolver



dnslist2 = ["8.8.4.4", "8.8.8.8", "208.67.2开发者_如何学JAVA20.220", "208.67.222.222", "192.168.50.1"]  #192.168.50.1  is a dns server i manage
host = "www.yahoo.com"
record = adns.rr.A
intensity = len(dnslist2)/5+1

ar = AsyncResolver(dnslist2, host, record, intensity)
start = time()
resolved = ar.resolve()
end = time()

print "\n\n"

for dns, ip in resolved.items():
  if ip is None:
    print "%s could not resolv %s." % (dns, host)
  else:
    print "%s resolved it to %s : %s" % (dns, host, ip)

print "\n\n----------------------------------------------------"
print "It took %.2f seconds to query %d dns." % (end-start, len(dnslist))
print "----------------------------------------------------"

async_dns.py

#!/usr/bin/python
#

import sys
import adns
from time import time

class AsyncResolver(object):
    def __init__(self, dnslist, host, record, intensity=10):

        """
        dnslist: a list of dns used to resolve
        host : hostname to resolve
        record: recordtype to resolve
        intensity: how many hosts to resolve at once
        """
        print "Inside class's init'"
        self.dnslist = dnslist
        self.host = host
        self.record = record


        if intensity >= len(dnslist) :
            self.intensity = len(dnslist)/5+1
        else:
            self.intensity = intensity

        print "Data"
        print "host " + host
        print "dnslist length " + str(len(dnslist))
        print "intensity " + str(intensity)




    def resolve(self):
        """ Resolves hosts and returns a dictionary of { 'dns': 'ip' }. """
        print "Inside resolve()"

        host = self.host
        record = self.record
        resolved_hosts = {}
        active_queries = {}
        dns_queue = self.dnslist[:]

        def collect_results():
            print "inside collect_results()"
            for query in self.adns.completed():
                print "Inside collect_results's for query in self.adns.completed()"
                answer = query.check()
                dns = active_queries[query]
                print "DNS used was" + dns
                print "Answered : " + str(answer)
                del active_queries[query]
                if answer[0] == 0:
                    #print "Inside answer[0] == 0 , ip:" + answer[3][0]
                    ip = answer[3][0]
                    resolved_hosts[dns] = ip
                    print "Resolved %s to %s using %s" % (host, ip, dns)
                    print "Resolved hosts count" + str(len(resolved_hosts))
                    print "And they are: "
                    print str(resolved_hosts)
                    print "\n"

                elif answer[0] == 101 and not record == adns.rr.CNAME: # CNAME if CNAME wasn't required'
                    print "ooopppps, i got a CNAME, gotta find its A"
                    print "\n"
                    query = self.adns.submit(answer[1], adns.rr.A)
                    active_queries[query] = dns
                else:
                    resolved_hosts[dns] = None
                    print "THIS COULD NOT BE RESOLVED"

        def finished_resolving():
            print "inside finished_resolving()"
            print "Resolved : " + str(len(resolved_hosts)) + "/" + str(len(self.dnslist))
            return len(resolved_hosts) == len(self.dnslist)

        while not finished_resolving():
            print "Inside 'while not finished_resolving'"
            while dns_queue and len(active_queries) <= self.intensity:
                print "Queue: " + str(len(active_queries)) + "/" + str(self.intensity)
                dns = dns_queue.pop()
                self.adns = adns.init(adns.iflags.noautosys,sys.stderr,"nameserver "+dns)
                query = self.adns.submit(host, record)
                print "Launching Querying for " + host + "/" + str(record) + " on " + dns
                active_queries[query] = dns
            collect_results()
        return resolved_hosts

I have just attempted to use adns python binding and have experienced a similar problem - the completed queries array never gets completely filled, causing the check loop to run forever. In my case, it appeared that domains that could not be resolved (due to NXDOMAIN etc) were never added to the completed queries array (with empty/Null value)

I had a look at the C binding but could not see a separate data structure for holding failed results, so it does appear that these should be added to the 'completed' queries somehow. I suspect there is a bug in the binding or adns library preventing this from happening.

Your while not finished_resolving() loop will obviously continue as long as finished_resolving() returns True. That function simply compares len(resolved_hosts) and len(self.dnslist). However, as far as I can see you never change the length of either of those variables inside the loop, so it will continue forever.

Edit after comment There's nothing in either of those two blocks that affects the length of either of the two variables. Please explain how you think they are being changed.