StaleObjectStateException on high frequency updates

We're using Hibernate 3.6.3.Final and MySQL 5.5.8 for a web application. The backend is running on a JBoss 6.0.0 Final server. Most of the time things work really well but occasionally we're getting a StaleObjectStateException. After a while of experimenting we figured out that it can be reproduced by sending requests to the backend with a high frequency (ie. clicking a button which triggers the request as fast as possible).

As far as I know the exception means that a domain object got fetched from the database and when Hibernate tried to persist it again it noticed that another transaction changed it in the meantime.

However as far as I understand databases the conflicting transactions should be isolated to an extent which prevents exactly this behavior. I explicitly changed the isolation level to SERIALIZABLE which guarantees repeatable reads and I disabled Hibernate caching. This should prevent the situation where one transaction sees different versions of the same domain object.

The full stack trace is:

    2011-04-28 20:46:17,865 WARN  [com.arjuna.ats.arjuna] (WorkerThread#2[127.0.0.1:57772]) ARJUNA-12125 TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff7f000001:126a:4db9c7b0:74d, org.hibernate.transaction.synchronization.HibernateSynchronizationImpl@481efbaf >: javax.persistence.OptimisticLockException: org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [xxx.modules.domain.entity.User#118]
        at org.hibernate.ejb.AbstractEntityManagerImpl.wrapStaleStateException(AbstractEntityManagerImpl.java:1243) [:3.6.0.Final]
        at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1166) [:3.6.0.Final]
        at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1147) [:3.6.0.Final]
        at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1153) [:3.6.0.Final]
        at org.hibernate.ejb.AbstractEntityManagerImpl$3.mapManagedFlushFailure(AbstractEntityManagerImpl.java:1067) [:3.6.0.Final]
        at org.hibernate.transaction.synchronization.CallbackCoordinator.beforeCompletion(CallbackCoordinator.java:122) [:3.6.0.Final]
        at org.hibernate.transaction.synchronization.HibernateSynchronizationImpl.beforeCompletion(HibernateSynchronizationImpl.java:51) [:3.6.0.Final]
        at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:97) [:6.0.0.Final]
        at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:274) [:6.0.0.Final]
        at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:94) [:6.0.0.Final]
        at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:159) [:6.0.0.Final]
        at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1158) [:6.0.0.Final]
        at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:119) [:6.0.0.Final]
        at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) [:6.0.0.Final]
        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:82) [:0.0.1]
        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:255) [:0.0.1]
        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.required(CMTTxInterceptor.java:349) [:0.0.1]
        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.invoke(CMTTxInterceptor.java:209) [:0.0.1]
        at org.jboss.ejb3.tx2.aop.CMTTxInterceptorWrapper.invoke(CMTTxInterceptorWrapper.java:52) [:0.0.1]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:76) [:1.0.0.GA]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42) [:1.0.3]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:182) [:1.7.17]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:41) [:1.7.17]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:67) [:1.7.17]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.core.context.CurrentInvocationContextInterceptor.invoke(CurrentInvocationContextInterceptor.java:47) [:1.7.17]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67) [:1.0.1]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.interceptor.EJB3TCCLInterceptor.invoke(EJB3TCCLInterceptor.java:86) [:1.7.17]
        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke(StatelessContainer.java:392) [:1.7.17]
        at org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53) [:1.7.17]
        at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91) [jboss-aop.jar:2.2.1.GA]
        at org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82) [:1.0.1.GA]
        at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:898) [:6.0.0.Final]
        at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:791) [:6.0.0.Final]
        at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:744) [:6.0.0.Final]
        at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:586) [:6.0.0.Final]
        at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234) [:6.0.0.Final]
Caused by: org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [xxx.modules.domain.entity.User#118]
        at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1932) [:3.6.0.Final]
        at org.hibernate.persister开发者_高级运维.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2576) [:3.6.0.Final]
        at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2476) [:3.6.0.Final]
        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2803) [:3.6.0.Final]
        at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:113) [:3.6.0.Final]
        at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273) [:3.6.0.Final]
        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265) [:3.6.0.Final]
        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:185) [:3.6.0.Final]
        at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) [:3.6.0.Final]
        at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51) [:3.6.0.Final]
        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216) [:3.6.0.Final]
        at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:383) [:3.6.0.Final]
        at org.hibernate.transaction.synchronization.CallbackCoordinator.beforeCompletion(CallbackCoordinator.java:117) [:3.6.0.Final]
        ... 39 more

Any help is appreciated!

Thanks in advance Michael

It seems like you have configured Hibernate to use optimistic concurrency control. That means that your User table has a version field that Hibernate increments on every row update.

Most likely your transaction starts at the beginning of the HTTP request and ends a the end of the HTTP response. This means that the process of editing a user consists of two transactions: one transaction to populate the web form and one transaction to save the changes.

In that case, changing the isolation level of the database will not do any good. Most likely all you will get is worse performance and scalability.

It is not a bad thing to have StaleObjectExceptions. It reflects the real world - people do actually work on the same thing once in a while and conflicts may occur. The question is, when a conflict has been detected, how do you resolve it in a manner that is satisfying to the end users? Can it be resolved without the assistance of the user?

Possible strategies could be to

• overwrite the previous user's changes (often not what you want - hence the need for concurrency control),

• show an error message asking the user to refresh and perform his changes again,

• automatically merge the changes without overwriting the previous user's changes (sometimes possible)

• inform the user about the staleness of his data and offer him a way to manually merge his changes

It all depends on the context.

Are you very, very sure that you've set the TX to serializable? Because that should never happen on a serializable transaction.

If two TX read and modify the same row in a serializable transaction, then oracle throws an ORA-08177.

Please check that hibernate is actually setting the TX as serializable.

Edit

You can do what Jonas suggested or you can also check it from you application by getting the underlying connection and invoking Connection.getIsolationLevel(). For example

Connection c = session.connection()
int level = c.getIsolationLevel()

Edit 2

Ok, since you confirmed that the isolation level on the Connection is SERIALILIZABLE, could you check:

• That the tables are using the innoDB engine.
• As Jonas suggested, run SELECT @@tx_isolation; from your code while it is in the transaction. It should return SERIALIZABLE. This is to check that the Connection is actually propagating the isolation level. It's a bit paranoid, but what to do...
• Check that your code only opens one transaction, and runs everything in that TX. I just tested the SERIALIZABLE isolation level manually and it works as expected (it blocks any TX trying to read the same row).
• Last resort: check that the isolation level SERIALIZABLE works on your MySQL installation.

NOTE: As I mentioned before, MySQL will block any queries trying to read from the same row. That means that if you have some "common tables" such as country, company, user, etc. that many TXs read concurrently, it might make your app run almost sequencially rather than parallelly.