What cool hacks can be done using sys.settrace?

I love being able to modify the arguments the get sent to a function, using settra开发者_JAVA技巧ce, like :

import sys

def trace_func(frame,event,arg):
    value = frame.f_locals["a"]
    if value % 2 == 0:
        value += 1
        frame.f_locals["a"] = value

def f(a):
    print a

if __name__ == "__main__":
    sys.settrace(trace_func)
    for i in range(0,5):
        f(i)

And this will print:

1
1
3
3
5

What other cool stuff can you do using settrace?

I would strongly recommend against abusing settrace. I'm assuming you understand this stuff, but others coming along later may not. There are a few reasons:

1. Settrace is a very blunt tool. The OP's example is a simple one, but there's practically no way to extend it for use in a real system.

2. It's mysterious. Anyone coming to look at your code would be completely stumped why it was doing what it was doing.

3. It's slow. Invoking a Python function for every line of Python executed is going to slow down your program by many multiples.

4. It's usually unnecessary. The original example here could have been accomplished in a few other ways (modify the function, wrap the function in a decorator, call it via another function, etc), any of which would have been better than settrace.

5. It's hard to get right. In the original example, if you had not called f directly, but instead called g which called f, your trace function wouldn't have done its job, because you returned None from the trace function, so it's only invoked once and then forgotten.

6. It will keep other tools from working. This program will not be debuggable (because debuggers use settrace), it will not be traceable, it will not be possible to measure its code coverage, etc. Part of this is due to lack of foresight on the part of the Python implementors: they gave us settrace but no gettrace, so it's difficult to have two trace functions that work together.

Trace functions make for cool hacks. It's fun to be able to abuse it, but please don't use it for real stuff. If I sound hectoring, I apologize, but this has been done in real code, and it's a pain. For example, DecoratorTools uses a trace function to perform the magic feat of making this syntax work in Python 2.3:

# Method decorator example
from peak.util.decorators import decorate

class Demo1(object):
    decorate(classmethod)   # equivalent to @classmethod
    def example(cls):
        print "hello from", cls

A neat hack, but unfortunately, it meant that any code that used DecoratorTools wouldn't work with coverage.py (or debuggers, I guess). Not a good tradeoff if you ask me. I changed coverage.py to provide a mode that lets it work with DecoratorTools, but I wish I hadn't had to.

Even code in the standard library sometimes gets this stuff wrong. Pyexpat decided to be different than every other extension module, and invoke the trace function as if it were Python code. Too bad they did a bad job of it.

</rant>

I made a module called pycallgraph which generates call graphs using sys.settrace().

Of course, code coverage is accomplished with the trace function. One cool thing we haven't had before is branch coverage measurement, and that's coming along nicely, about to be released in an alpha version of coverage.py.

So for example, consider this function:

def foo(x):
    if x:
        y = 10
    return y

if you test it with this call:

assert foo(1) == 10

then statement coverage will tell you that all the lines of the function were executed. But of course, there's a simple problem in that function: calling it with 0 raises a UnboundLocalError.

Branch measurement would tell you that there's a branch in the code that isn't fully exercised, because only one leg of the branch is ever taken.

For example, get the memory consumption of Python code line-by-line: http://pypi.python.org/pypi/memory_profiler

One latest project that uses settrace heavily is PySnooper

It helps new programmers to trace/log/monitor their program output. Cheers!

I don't have an exhaustively comprehensive answer but one thing I did with it, with the help of another user on SO, was create a program that generates the trace tables of other Python programs.

The python debugger Pdb uses sys.settrace to analyse lines to debug.

Here's an c optimization/extension for pdb that also uses sys.settrace

https://bitbucket.org/jagguli/cpdb