Safely interrupt a program with threaded downloading

I have a (commandline/terminal) program that scrapes a website with worker threads that do the downloading from a queue an开发者_开发知识库d a main thread that downloads the index files (50 entries per page). How can I make the program check for an interrupt (either CTRL+C or my own defined one), and when it catches such interrupt it will first clean up (download the remaining queue) and then terminate.

Wrap the main function that's waiting for the threads to complete with an exception handler like this:

try:
    main()
except KeyboardInterrupt:
    stop()

def stop():
    for t in threads:
        t.my_stop_function()
    # wait for threads to stop here...

class MyThread(threading.Thread):
    def __init__(self, *args, **kwargs):
        threading.Thread.__init__(self, *args, **kwargs)
        self.stop = False

    def my_stop_function(self):
        self.stop = True

    def run(self):
        while not self.stop:
            scrape()

In the main loop, you want to catch KeyboardInterrupt exceptions (raised when user press CTRL-C). For handling cleanup, you can use atexit module to run some a global cleanup function or use threading.Event/threading.Condition to notify the worker threads to cleanup themselves and exit.

import atexit
atexit.register(cleanup_function)