Spring Roo project as batch job without transactions

I have a Roo project that works "fine" with transactions, but each .merge() or .persist() takes longer and longer time, so that what should've taken 10ms takes 5000ms towards the end of the transaction. Luckily, my changes are individually idempotent, so I don't really need a transaction.

But when I throw out transaction handling I run into the classic "The context has been closed" when I do myObject.merge()

The job I'm running is from the command line as a batch, so here is what I usually do:

public static void main(final String[] args) {
    context = new ClassPathXmlApplicationContext("META-INF/spring/applicationContext.xml");
    JpaTransactionManager txMgr = (JpaTransactionManager) context.getBean("transactionManager");
    TransactionTemplate txTemplate = new Transactio开发者_开发知识库nTemplate(txMgr);
    txTemplate.execute(new TransactionCallback() { @SuppressWarnings("finally")
    public Object doInTransaction(TransactionStatus txStatus) {
    try {
        ImportUnitFromDisk importer = new ImportUnitFromDisk();
        int status = importer.run(args[0]);
        System.out.println("Import data complete status: " + status);
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        return null;
    }
    }});

    System.out.println("All done!");
    System.exit(0);
}

But what I really want to do is something like this:

public static void main(final String[] args) {
    ImportUnitFromDisk importer = new ImportUnitFromDisk();
    int status = importer.run(args[0]);
    System.out.println("Import data complete status: " + status);
    System.out.println("All done!");
    System.exit(0);
}

What can I do to allow me to persist() and merge() without using transactions, given that the entities are generated with Spring Roo (using OpenJPA and MySQL)?

Cheers

Nik

Even if your changes are idempotent, you still will need transaction.

As far as performance is concerned.

1. How tightly coupled is your entity objects. (For instance if all table fk refernces are migrated to entity relationship, then its pretty tightly coupled)?

2. May be you should remove some unwanted bidirectional relationships.

3. Identify master tables and remove entities mapping to master records.

4. What is your cascade options? Check if you have cascade all options everywhere.

For me it looks the Entity map is far too tightly coupled .(Everyone knows someone who has ...) and the cascade options kick off merging the whole object graph. (log your jpa sql, that can validate my assumption)

I have experienced exactly the same performance problem with a Spring / Hibernate batch process. Note that this has nothing to do with Spring Roo or even Spring - it is due to the workings of Hibernate / JPA.

The basic problem is that Hibernate maintains a session cache of all the Java entities that are part of the transaction, and for new entities (for which bytecode instrumentation has not been done) Hibernate must scan the entities on each flush to see if there were updates. This is at least O(n) for n = # of new entities in the session. If the batch process is primarily adding new entities, then this turns into O(n^2) behavior for the overall batch.

One solution if you want to maintain the whole process in one transaction is to periodically flush (to do inserts/updates) and then evict entities that you no longer need to keep in the session. Another solution is to split the batch process into multiple transactions.

See http://www.basilv.com/psd/blog/2010/avoiding-caching-to-improve-hibernate-performance for more details.