ibatis: improve insert performance
I am using ibatis for my sql insert stmt. In my code I am parsing files line by line from a folder. Each line that matches criteria, need to be inserted into database. Total number of insert in a single run of program can be any where along 200k.
SqlSession sess = null;
this.sess = sf.openSession(ExecutorType.BATCH, false);
for (each file) {
for( each line matching criteria ){
this.sess.insert("com.logs.util.insertFileInfo", fileData);
insertcount++;
if(insert cou开发者_如何学运维nt == 10)
this.sess.commit();
}
}
if(insert count > 0){
this.sess.commit();
}
}
This style slowly takes up lot of memory and after some times throws OutOfMemory exception. How can I improve performance here?
Is it your intention to commit after every 10 inserts? It looks like you only do so after the first 10 inserts. I guess that you would need something like
if ((insertCount % 10) == 0) {
this.sess.commit();
}
Those uncommitted changes have to be stored somewhere. I'm not familiar with Ibatis but if the uncommitted changes are being stored in a buffer allocated by Ibatis then you will eventually run out of memory if you don't commit the changes.
精彩评论