Python - Using cPickle to load a previously saved pickle uses too much memory?

Python - Using cPickle to load a previously saved pickle uses too much memory?

My pickle file is about 340MB but takes up 29% of 6gb of memory when loaded. This seems a bit too much. The pickle file is a dictionary of di开发者_C百科ctionaries. Is this appropriate?

Code used:

import cPickle as pickle

file = pickle.load( file_handle )

Thanks

I always had memory problems with big pickels and sub dicts. So i ended up writing my objects via pprint into files and later i import that files via a a custom module loader to get the data back in the process scope. Works fine and doesn't waste memory.

About 1.7GB seems a bit much, but not impossible. How much memory did the data take before it was pickled?

After unpickling the data should take about the same amount of memory as it took before it was pickled, how big it is in it's on-disk format is not really that significant.