Basics - Troubleshooting Hibernate / JDBC Connection Pool Issue

What is Hibernate's responsibility in regards to database connections it gets from an underlying connection pool. Does it test to see if a connection is closed before it uses it? and if so get another connection from the pool?

I've included error and confirmation info below. Any ideas of where I can start to troubleshoot this would be very helpful. And any advice on the SQL Server driver settings we are using.

from the Catalina log:

04-Nov-2010 21:54:52.691 WARNING org.apache.tomcat.jdbc.pool.ConnectionPool.abandon Connection has been abandoned PooledConnection[ConnectionID开发者_运维百科:8]:java.lang.Exception
    at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump(ConnectionPool.java:926)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:681)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:545)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:166)
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:106)

from our application log:

2010-11-04 21:54:52,705 [tomcat-http--18] WARN  util.JDBCExceptionReporter  - SQL Error: 0, SQLState: 08S01
2010-11-04 21:54:52,707 [tomcat-http--18] ERROR util.JDBCExceptionReporter  - Socket closed
2010-11-04 21:54:52,708 [tomcat-http--18] ERROR transaction.JDBCTransaction  - JDBC rollback failed
java.sql.SQLException: Connection has already been closed.
    at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:112)
    at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:94)
    at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:71)
    at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:94)
    at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState.invoke(ConnectionState.java:132)
    at $Proxy38.rollback(Unknown Source)
    at org.hibernate.transaction.JDBCTransaction.rollbackAndResetAutoCommit(JDBCTransaction.java:217)
    at org.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction.java:196)
    at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:676)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:845)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:822)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:412)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:111)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:625)

The configuration:

<Resource defaultAutoCommit="false" defaultReadOnly="false"
        defaultTransactionIsolation="SERIALIZABLE"
        driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
        factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
        fairQueue="false" initialSize="10"
        jdbcInterceptors="ConnectionState;StatementFinalizer"
        jmxEnabled="true" logAbandoned="true" maxActive="100"
        maxIdle="10" maxWait="30000"
        minEvictableIdleTimeMillis="10000" minIdle="10"
        name="com.ourcompany.ap.shoppingcart/datasource"
        password="somePassword" removeAbandoned="true"
        removeAbandonedTimeout="60" testOnBorrow="true"
        testOnReturn="false" testWhileIdle="false"
        timeBetweenEvictionRunsMillis="5000"
        type="javax.sql.DataSource"
        url="jdbc:sqlserver://approd\approd;databaseName=prod"
        useEquals="false" username="AccessPointNet"
        validationInterval="30000" validationQuery="SELECT 1"/>`

I had a similar problem which was solved by increasing the removeAbandonedTimeout value to a higher number. The problem we faced was due to the query which took longer time that the above mentioned timeout.

What is Hibernate's responsibility in regards to database connections it gets from an underlying connection pool.

Not much, releasing it when the Session gets closed.

Does it test to see if a connection is closed before it uses it? and if so get another connection from the pool?

No, Hibernate doesn't, checking the validity of connection(s) is the responsibility of a connection pool if you want to.

I've included error and confirmation info below. Any ideas of where I can start to troubleshoot this would be very helpful.

What kind of process are you running exactly? A long transaction? Does it timeout? What does the Caused by: say? About the trace:

2010-11-04 21:54:52,705 [tomcat-http--18] WARN util.JDBCExceptionReporter - SQL Error: 0, SQLState: 08S01 
2010-11-04 21:54:52,707 [tomcat-http--18] ERROR util.JDBCExceptionReporter - Socket closed
2010-11-04 21:54:52,708 [tomcat-http--18] ERROR transaction.JDBCTransaction - JDBC rollback failed java.sql.SQLException: Connection has already been closed.

Can you reproduce it in a deterministic way? Any networking problem?

And any advice on the SQL Server driver settings we are using.

I've added a great resource about Tomcat and connection pool configuration below. Not specific to SQL Server though.

Resources

• Configuring jdbc-pool for high-concurrency

We usually work around this by using dbcp, and providing a validationQuery when definining our data source. Then, dbcp will verify the usability of pooled connections by issuing that query (and transparently recreate the connection should it no longer work), prior to returning them to the application.

Check out http://tomcat.apache.org/tomcat-6.0-doc/jndi-datasource-examples-howto.html for more details.

I am currently using liquibase(v1.9) in my project, and when the changeSets run against a blank schema it always takes longer than 60 seconds which results in the thread being marked abandoned I'm not thrilled with increasing the removeAbandonedTimeout value, but this is the only solution I've been able to find to prevent this issue; however, after the initial schema population is complete this is seldom a problem so I set the value back to 60 seconds.

I worked on an issue in the past where we weren't returning connections back to the pool correctly. So, when a connection was used and not returned, making a database call when it was timing out would throw an exception.

We were able to reproduce the issue by making a call to the database, waited 8 hours (postgres' default time out) and tried to make a call to the database again. It throw the same exception every time. Our solution was to rethink (or better yet, add) a connection management strategy.

So, to sum up, are you actually returning your connections to the pool by closing the Session?

I got the solution for the above exception. Just close the instance of session factory as well while closing the session .

Look at the below Code:

public class HibernateUtil {
    private static final SessionFactory sessionFactory = buildSessionFactory();

    private static SessionFactory buildSessionFactory() {
        try {
            // Create the SessionFactory from hibernate.cfg.xml
            return new Configuration().configure("hibernate.cfg.xml").buildSessionFactory();
        }
        catch (Throwable ex) {
            ex.printStackTrace();
            // Make sure you log the exception, as it might be swallowed
            System.err.println("Initial SessionFactory creation failed." + ex);
            throw new ExceptionInInitializerError(ex);
        }
    }

    public static SessionFactory getSessionfactory() {
        return sessionFactory;
    }

    public static Session getSession() {
        Session session=sessionFactory.openSession();
        session.getTransaction().begin();
        return session;
    }
    public static void closeSession(Session session) {
        if(session!=null )
        {
            if(session.getTransaction().isActive())
            {
                session.getTransaction().commit();
            }
                session.close();
                getSessionfactory().close();
        }
    }
}

just call the method HibernateUtil.closeSession(). This will solve the problem.