Excessive Garbage Collection ?
Application details: pyQt application, Qt 4.4 and python 2.5
Issue: Over the time ( using application for long time w/o closing ) , the application gets slow sometimes. I tried most of the profiling application, but did not find anything significant to point out any issue.
My application creates multiple tables in different tabs using QTabWidget. So to speed up the tab switching I only construct the table once and keep the QTableWidgetItem in memory. The count of QTableWidgetItem ranges in ~5000 - 10000. There are more of such objects (# ~ 600 - 800) that are constructed and needs to be keep in memory until anything changes.
I was thinking could GC be slowing down my application sometimes. Because the default GC thresholds are 700,10,10 . In my case, 700 seems to be very less. And these QTableWidgetItem stays in memory until any data has changed. So could GC be kicking in frequently in phase 1, phase 2 and phase 3 ? Once the data is changed I just de-reference it by doing mytable.setRowCount(0). In this case, I do not need GC to reclaim memory. Am I right ?
Can I tell GC to not track some of these objects?
Please suggest if my understanding is wrong ?
Also, please let me know if question is unclear.
def fetchData (self, win=None):
"""Fetches data from the data sources.
"""
start = time.clock()
changed = self.adjustDateRange()
if not changed and self.chartEdition == self.fetchEdition: return
badIngredients = []
ingredientList = self.getIngredientList()
parentWin = win
if parentWin is None:
parentWin = qApp.activeWindow()
B = 0
N = len(ingredientList)
if N == 0: return
progress = QProgressDialog(parentWin)
progress.setWindowModality(Qt.WindowModal)
progress.setWindowTitle ("FetchData Progress")
progress.setRange (0, N)
#progress.forceShow()
self.lock()
try:
for i in xrange(len(ingredientList)):
progress.setValue(i)
if B > 0:
progress.setLabelText("Fetching time series %s of %s\n"\
"Number of missing series: %s" % \
(i+1,N,B))
else:
progress.setLabelText("Fetching time series %s of %s" % \
(i+1,N))
print self, "Fetching time series ", i+1, " of ", N
qApp.processEvents()
ingredient = ingredientList[i]
if progress.wasCanceled():
for ingredient in ingredientList[i:len(ingredientList)]:
ingredient.error = 1
badIngredients.append(ingredient)
break
try:
ingredient.getTimeSeries (win)
ingredient.error = 0
except:
#Handle Exception here
# Create a badIngredient list here
for ingredient in badIngredients:
self.deleteIngredient(ingredient)
finally:
progress.setValue(N)
progress.close()
self.callWhenFree (self, self.chartChanged, remove=True)
self.unlock()
self.emit (SIGNAL("dataChanged"))
self.fetchEdition = self.chartEdition
end = time.clock()
print "Data Fetched in %s sec" %(end-start)
EDIT: While executing this loop, over the time the progress Dialog keeps getting slower and slower. But there are lot of other things happening like creating huge table for these data and more Qt objects. Also, the chart drawing of QGraphicsItem gets slower. Currently, I have commented the qApp.processEvent() for testing purpose, in bugs list of Qt it says that QProgressDialog.setValue internally calls processEvents so I do not need to call explicitly. Also there is a freezing problem where application just freezes at this progress dialog and needs to kill it.
I used objgraph http://mg.pov.lt/objgraph/ to get the top most 50 objects in memory This is what I get when I open the application for first time
tuple 16243
dict 6325
function 4220
instance 3814
QTreeWidgetItem 2849
wrapper_descriptor 2642
weakref 1447
getset_descriptor 1387
list 934
method_descriptor 815
builtin_function_or_method 805
wrappertype 660
type 652
classobj 267
module 235
member_descriptor 181
QAction 154
instancemethod 47
property 36
frame 开发者_StackOverflow 34
QWidget 33
QColor 33
QVBoxLayout 27
_Condition 20
QLabel 18
QMenu 17
QToolButton 14
QTableWidgetItem 13
QGridLayout 13
SplitResult 13
QTimer 11
TypeInfo 10
classmethod_descriptor 10
QComboBox 9
QLineEdit 9
QCheckBox 9
QSpacerItem 8
QGroupBox 7
PenStyle 6
set 6
ChartViewAction 6
QHBoxLayout 6
MouseButton 6
QRadioButton 5
QActionGroup 5
QtMsgType 5
Thread 4
QPushButton 4
QToolBar 4
staticmethod 4
After running the code that might be leaking, the top most 30 objects in memory.
tuple 19948
QTableWidgetItem 9298
dict 7562
function 4220
instance 4157
QTreeWidgetItem 2849
wrapper_descriptor 2788
QGraphicsRectItem 2246
weakref 1527
getset_descriptor 1392
list 1198
QGraphicsLineItem 825
method_descriptor 816
QGraphicsTextItem 812
builtin_function_or_method 811
QColor 780
DQEllipse 748
wrappertype 660
type 652
QAction 280
classobj 267
module 235
member_descriptor 189
instancemethod 110
dqComboBoxTableItem 66
property 36
frame 35
QWidget 33
QVBoxLayout 27
GlobalColor 24
The QTableWidgetItems must be in memory. Currently, I am testing this application with disabling gc. Most of the objects that I see, are created and keep in memory like QGraphicsLineItem, QGraphicsRectItem, etc. When I create new objects, these objects are over-ridden with new objects thus making the old one de-referenced. What are the other means of testing memory leak?
You could try turning off garbage collection to test your theory with :
gc.disable()
or better still:
gc.set_debug(gc.DEBUG_LEAK)
See the relevant page in the python reference for more.
精彩评论