Clarifying this output from the pstats module

I ran the cProfile on my code and this is the output:

% stats 10
     646493 function calls (524209 primitive calls) in 3.606 CPU seconds

Ordered by: cumula开发者_C百科tive time
List reduced from 260 to 10 due to restriction <10>

ncalls   tottime  percall  cumtime  percall filename:lineno(function)
     1     0.000    0.000    3.606    3.606 <string>:1(<module>)
     1     0.007    0.007    3.606    3.606 {execfile}
     1     0.068    0.068    3.599    3.599 example.py:7(<module>)
     3     0.000    0.000    3.266    1.089 tree.py:1058(parseString)
6698/3     0.068    0.000    3.244    1.081 tree.py:2406(do_parse3)
104813/3   1.084    0.000    3.244    1.081 tree.py:926(_nocache)
2615/3     0.016    0.000    3.243    1.081 tree.py:2679(internal_parse)
3602/14    0.712    0.000    3.239    0.231 tree.py:2531(do_parse2)
  13/8     0.000    0.000    3.229    0.404 tree.py:2876(do_parse)
2546/20    0.024    0.000    3.218    0.161 tree.py:1003(parse)

From the docs,

We define primitive to mean that the call was not induced via recursion

Can I then safely conclude that the reasons why my code is slow are:

1. due to the 122284 recursive calls.
2. The biggest recursive methods are do_parse3 & _nocache.
3. Primitive calls are insignificant & cannot be optimized further.

1. I think you cannot see whether time is spent because of the method calls, or due to the work done inside those method.

2. I agree. That would be the place to start micro-optimizing python code. It may bring you some speedup but there is usually a better way to speed up some given task.

3. Not really. First of all, if you have a real application this shows you the call that you might want to get rid of. And I see in your profile that it's called three times; maybe it's doing the same computation three times, and (sub)results could be cached. Maybe the original (primitive) call could do something to reduce the amount of work to be done by the children.

May I suggest to look at a call graph instead? I use Gprof2Dot for this:

gprof2dot.py -f pstats tmp.pstats | dot -Tpng -o tmp.png

http://code.google.com/p/jrfonseca/wiki/Gprof2Dot
http://maxy.homeip.net/misc/gprof2dot_example.png