How to deal with a search task which takes more time than usual in Spring 3.0

I am looking for ideas on how to deal with a search related task which takes more than usual time (in human terms more than 3 seconds)

I have to query multiple sources, sift through information for the first time and then cache it in the DB for later quick return.

The context of the project is J2EE, Spring and Hibernate (on top of SpringROO)

The possible solutions I could think of -On the webpage let the user know that task is running in background, if possible give them a queue number or waiting time. Refresh the page via a controller which basically checks if the task is done, then when its done (ie the search result is prepared and stored in DB) then just forward to a new controller and fetch the result from the DB

-The background tasks could be done with Spring Task executor. I am not sure if it is easy to give a measure of how long it would take. It would probably be a bad idea to let all the search terms run concurrently, so some sort of pooling will be a good idea.

-Another option to use background tasks is to use JMS. This is perhaps a solution with more control (retries etc)

-Spring batch also comes to mind

Please suggest how you would do it. I would greatly appreciate a semi-detailed+ description. The sources of info c开发者_JAVA百科an be man and can be sequential in nature so it can take upto 4-5 minutes for the results to form. It is also possible that such tasks run automatically in the background without user intervention (ie to update from the sources)

From a user perspective, I use AJAX. The default web page contains some kind of "Busy" indicator. When the AJAX request completes, the busy indicator is replaced with the result.

In the background, request handlers are already multi-threaded. So you can simply format the default result, close&flush the output, and do the processing in the current thread. You should put something in the session or DB to make sure that no one can start the same heavy process a second time.

Running task pools in a web container is possible but there are some caveats, especially how to synchronize startup/shutdown: Do you want your web server to "hang" during shutdown while some thread is busy collecting your results? Also the additional load should be considered. It might be better to use JMS and offload the strain to a second server dedicated to build the search results.

Such a system will scale much better if your searches start to become a burden. It also makes it trivial to automate the process by writing a small program which posts searches in the JMS queue.

I've solved this problem in the past doing something like this:

• When the user initiates a long running task, I open a popup window that displays the task status. The task status includes a name and estimated time to complete
• This task is also stored in my "app" (this can be stored in the DB, session, or application context), so the user can continue doing other things on my web app while having an easy way to navigate back to the running task.
• I stored my tasks in a DB, so I could manage what happens on startup and shutdown of the web app. This requires storing the progress of the task in the DB.
• The tricky part is display results to the user. If you use the method I've described, you'll need to store results in either the DB, session, or application contexts.

This system I've described is pretty heavyweight, and may be overkill for your application.

In response to the comment

so what do you use to do the background computing. I have asked this before

I use java.util.concurrent. A lot of this depends on the nature of your application. Is the task (or steps in the task) idempotent? How critical is it that it run to completion? If you have a non-idempotent task that must run to completion, I would say you generally must record every piece of work you do, and you must do that piece of work within a transaction. For example, if one of your tasks is to email a list of people (this is definitely not idempotent) you would do the emailing in a "transaction" (I'm using the term lightly here) and store your progress after each transaction is complete.