MySQL Connection Timeout Issue - Grails Application on Tomcat using Hibernate and ORM

I have a small grails application running on Tomcat in Ubuntu on a VPS. I use MySql as my datastore and everything works fine unless I leave the application for more than half a day (8 hours?). I did some searching and apparently this is the default wait_timeout in mysql.cnf so after 8 hours the connection will die but Tomcat won't know so when the next user tries to view the site they will see the connection failure error. Refreshing the page will fix this but I want to get rid of the error altogether. For my version of MySql (5.0.75) I have only my.cnf and it doesn't contain such a parameter, In any case changing this parameter doesn't solve the problem.

This Blog Post seems to be reporting a similar error but I still don't fully understand what I need to configure to get this fixed and also I am hoping that there is a simpler solution than another third party library. The machine I'm running on has 256MB ram and I'm trying to keep the number of programs/services running to a minimum.

Is there something I can configure in Grails / Tomcat / MySql to get this to go away?

Thanks in advance,

Gav

From my Catalina.out;

2010-04-29 21:26:25,946 [http-8080-2] ERROR util.JDBCExceptionReporter  - The last packet successfully received from the server was 102,906,722 milliseconds$
2010-04-29 21:26:25,994 [http-8080-2] ERROR errors.GrailsExceptionResolver  - Broken pipe
java.net.SocketException: Broken pipe
        at java.net.SocketOutputStream.socketWrite0(Native Method)
         ...
2010-04-29 21:26:26,016 [http-8080-2] ERROR util.JDBCExceptionReporter  - Already closed.
2010-04-29 21:26:26,016 [http-8080-2] ERROR util.JDBCExceptionReporter  - Already closed.
2010-04-29 21:26:26,017 [http-8080-2] ERROR servlet.GrailsDispatcherServlet  - HandlerInterceptor.afterCompletion threw exception
org.hibernate.exception.GenericJDBCException: Cannot release connection
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.sql.SQLException: Already closed.
        at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:84)
        at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.开发者_如何学JAVAclose(PoolingDataSource.java:181)
        ... 1 more

Referring to this article, you have stale connections in your DBCP connections pool that are silently dropped by OS or firewall.

The solution is to define a validation query and do a sanity check of the connection before you actually use it in your application. In grails this is actually done by modifying the grails-app/conf/spring/Resource.groovy file and add the following:

beans = {
  dataSource(BasicDataSource) {
    //run the evictor every 30 minutes and evict any connections older than 30 minutes.
    minEvictableIdleTimeMillis=1800000
    timeBetweenEvictionRunsMillis=1800000
    numTestsPerEvictionRun=3
    //test the connection while its idle, before borrow and return it
    testOnBorrow=true
    testWhileIdle=true
    testOnReturn=true
    validationQuery="SELECT 1"
  }
} 

In grails 1.3.X, you can modify the evictor values in the DataSource.groovy file to make sure pooled connections are used during idle. This will make sure the mysql server will not time out the connection.

production {
  dataSource {
    pooled = true
    // Other database parameters..
    properties {
       maxActive = 50
       maxIdle = 25
       minIdle = 5
       initialSize = 5
       minEvictableIdleTimeMillis = 1800000
       timeBetweenEvictionRunsMillis = 1800000
       maxWait = 10000
    }
}

A quick way to verify this works is to modify the MySQL my.cnf configuration file [mysql] element and add wait_time parameter with a low value.

Try increasing the number of open MySQL connections by putting the following in your DataSources.groovy:

dataSource {
     driverClassName = "com.mysql.jdbc.Driver"
     pooled=true
     maxActive=10
     initialSize=5
     // Remaining connection params
}

If you want to go the whole hog, try implementing a connection pool; here is a useful link on this.

For grails 1.3.X, I had to add the following code to Bootstrap.groovy :

  def init = {servletContext ->
  def ctx=servletContext.getAttribute(ApplicationAttributes.APPLICATION_CONTEXT)

    //implement test on borrow
    def dataSource = ctx.dataSource
    dataSource.targetDataSource.setMinEvictableIdleTimeMillis(1000 * 60 * 30)
    dataSource.targetDataSource.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30)
    dataSource.targetDataSource.setNumTestsPerEvictionRun(3)
    dataSource.targetDataSource.setTestOnBorrow(true)
    dataSource.targetDataSource.setTestWhileIdle(true)
    dataSource.targetDataSource.setTestOnReturn(false)
    dataSource.targetDataSource.setValidationQuery("SELECT 1")

  }

I also had to import org.codehaus.groovy.grails.commons.ApplicationAttributes

Add these parameters to dataSource

        testOnBorrow = true
        testWhileIdle = true
        testOnReturn = true

See this article for more information http://sacharya.com/grails-dbcp-stale-connections/

Starting from grails 2.3.6 default configuration already has options for preventing closing connection by timeout

These are the new defaults.

    properties {
       // See http://grails.org/doc/latest/guide/conf.html#dataSource for documentation
       ....
       minIdle = 5
       maxIdle = 25
       maxWait = 10000
       maxAge = 10 * 60000
       timeBetweenEvictionRunsMillis = 5000
       minEvictableIdleTimeMillis = 60000
       validationQuery = "SELECT 1"
       validationQueryTimeout = 3
       validationInterval = 15000
       testOnBorrow = true
       testWhileIdle = true
       testOnReturn = false
       jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
       defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
    }

What does your JDBC connection string look like? You can set an autoReconneect param in your data source config, e.g.

jdbc:mysql://hostname/mydb?autoReconnect=true