How to find total amount of memory used by python process/object in windows

I have some script that loads a lot of data to memory. I want to know how efficient the data stored in memory. So, I want to be able to know how many memory was used by python before I loaded data, and after I loaded data. Also I wondering, if it is some way to check memory usage of complex object. Let say i have nested dictionary with different types of data inside. How can i know how many memory used by all data in this diction开发者_高级运维ary. Thanks, Alex

As far as I know there is no easy way to see what the memory consumption of a certain object is. It would be a non-trivial thing to do because references could be shared among objects.

Here are my two favourite workarounds:

1. Use the process manager. Have the program pause for before allocation. Write down the memory used before allocation. Allocate. Write down memory after allocation. It's a low-tech method but it works.
2. Alternatively you can use pickle.dump to serialize your data structure. The resulting pickle will be comparable (not identical!) in size to the space needed to store the data structure in memory. For better results, use the binary pickle protocol.

In order to analyze how much memory an object uses, you could use Pympler:

>>> from pympler import asizeof
>>> obj = dict(nested=dict(trash=[1,2,3]))
>>> asizeof.asizeof(obj)
800
>>> asizeof.asizeof(obj['nested'])
480
>>> asizeof.asizeof(obj['nested']['trash'])
160
>>> asizeof.asizeof(obj['nested']['trash'][0])
24

You can take a look at the guppy package, which can give you informations about memory used by every loaded object. Unfortunately, it doesn't seem to work under python>=2.6, but it's good if you are using at most python 2.5. Its usage is really simple, just put these lines in your code, where you want to collect memory informations:

from guppy import hpy
hp = hpy()
print hp.heap()

Which will give you an output like this:

Partition of a set of 25961 objects. Total size = 1894868 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  11901  46   775408  41    775408  41 str
     1   6040  23   219964  12    995372  53 tuple
     2   1718   7   116824   6   1112196  59 types.CodeType
     3     73   0   113608   6   1225804  65 dict of module
     4    348   1   107232   6   1333036  70 dict (no owner)
     5    196   1   100192   5   1433228  76 dict of type
     6   1643   6    92008   5   1525236  80 function
     7    209   1    90572   5   1615808  85 type
     8    144   1    76800   4   1692608  89 dict of class
     9    984   4    35424   2   1728032  91 __builtin__.wrapper_descriptor

An alternative is that you could use windows's performance counters through pywin32