Java OutOfMemory in app with Hibernate, Quartz and and JavaMail

I have a web application running on Tomcat 6, that I created with Netbeans 6.9.1 and its wizzards for Entity classes and persistence. It uses Hibernate, MySQL as database, Quartz for running a task every 15 minutes, and JavaMail.

The hibernate configuration is as follows:

<?xml version="1.0" encoding="UTF-8" ?> 
 <persistence version="1.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">
 <persistence-unit name="GVPU" transaction-type="RESOURCE_LOCAL">
  <provider>org.hibernate.ejb.HibernatePersistence</provider> 
  <class>co开发者_开发技巧m.test.MyfirstEntity</class> 
  <class>com.test.MySecondEntity</class> 
  <exclude-unlisted-classes>true</exclude-unlisted-classes> 
 <properties>
  <property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect" /> 
  <property name="hibernate.connection.username" value="root" /> 
  <property name="hibernate.connection.driver_class" value="com.mysql.jdbc.Driver" /> 
  <property name="hibernate.connection.password" value="password" /> 
  <property name="hibernate.connection.url" value="jdbc:mysql://localhost:3306/db" /> 
  <property name="hibernate.cache.provider_class" value="org.hibernate.cache.NoCacheProvider" /> 
  <property name="hibernate.hbm2ddl.auto" value="update" /> 
  <property name="hibernate.connection.provider_class" value="org.hibernate.connection.C3P0ConnectionProvider" /> 
  <property name="hibernate.c3p0.acquire_increment" value="1" /> 
  <property name="hibernate.c3p0.idle_test_period" value="300" /> 
  <property name="hibernate.c3p0.timeout" value="100" /> 
  <property name="hibernate.c3p0.max_size" value="100" /> 
  <property name="hibernate.c3p0.min_size" value="0" /> 
  <property name="hibernate.c3p0.max_statements" value="0" /> 
  </properties>
  </persistence-unit>
  </persistence>

The task created with Quartz, that runs every 15 minutes, checks a GMail account using JavaMail (with IMAP), retrieves all the messages, process them (looks for some info) and store objects with that information into the DB using Hibernate. Then it mark all mails as deleted, so they are not process on the next run.

Everytime this task runs, it consumes 10, 20, 50MBytes depending on how many mails it process. I have looked at MySQL Administrator, and the DB connections are closed when not needed. But if I look at the process memory (in the linux machine it runs) the memory grows until it reach a limit where the CPU usage is 100%, and Tomcat does not responds any more.

I have changed several times -Xms -Xmx and PermGem Size in order to try different solutions, but it always reaches a point where the system stops working and the memory is not been released.

Any idea where I can start looking for this problem? I have assigned more that 1.5GBytes to the Java VM, and it also hangs even though it last more than a day.

When Tomcat and all the application stops working, there is no stacktrace.

Thanks

Attach with jvisualvm in the JDK to get a memory profiling.

Note that to my knowledge JavaMail does not handle large attachments efficiently, and the resulting mails get even larger. Hence you must make certain that the mail is cleared from memory before the next is being processed.

I would suggest that you set up a local SMTP server - this is very easy with most Linux distributions - and let your Java application talk to that instead. This will make your task finish much faster and then release the memory.

If you keep getting OOM exceptions then consider running the mail sending in a separate JVM so it does not influence the rest of your application.

General approach: get memory dump and look for most numerous objects or biggest objects and see if they should not exist, but still are. Then you can see what objects preventing this garbage from being collected (via holding references to the garbage).

How do you manage Hibernate sessions between the process runs? Are you sure you are not keeping all cached objects from all previous runs of your process in the Hibernate session cache or in your own cache?

You might not be closing the database connections properly. This may cause the objects to remain in the memory and not get garbage collected. Check for abandoned connections and log them (in your c3p0 configurations). I once ran into a similar issue and it boiled downed to rogue database connections.