开发者

ibatis: improve insert performance

I am using ibatis for my sql insert stmt. In my code I am parsing files line by line from a folder. Each line that matches criteria, need to be inserted into database. Total number of insert in a single run of program can be any where along 200k.

    SqlSession sess = null;    
    this.sess = sf.openSession(ExecutorType.BATCH, false);
    for (each file) {
     for( each line matching criteria ){
         this.sess.insert("com.logs.util.insertFileInfo", fileData);
         insertcount++;
         if(insert cou开发者_如何学运维nt == 10)
              this.sess.commit();
         }    
      }
      if(insert count > 0){
           this.sess.commit();
      }   
    }

This style slowly takes up lot of memory and after some times throws OutOfMemory exception. How can I improve performance here?


Is it your intention to commit after every 10 inserts? It looks like you only do so after the first 10 inserts. I guess that you would need something like

if ((insertCount % 10) == 0) {
   this.sess.commit();
}

Those uncommitted changes have to be stored somewhere. I'm not familiar with Ibatis but if the uncommitted changes are being stored in a buffer allocated by Ibatis then you will eventually run out of memory if you don't commit the changes.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜