Intermittent Tomcat ClassNotFoundException for some jsp pages

We have two tomcat servers load-balanced behind apache. Several times now we'e gotten reports of a 500 error on a page. Checking shows the following error, but only on one of the tomcats:

java.lang.ClassNotFoundException: org.apache.jsp.jsps.userLogin_jsp
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at 开发者_高级运维java.security.AccessController.doPrivileged(Native Method)
    etc...

I've tried clearing the work directory and editing the jsp page, with no luck. The only thing that solves it is a tomcat restart, but obviously we need to avoid this kind of solution in production.

Has anyone solved this problem before? My searching has only found questions without answers, including a tomcat bug that was resolved as 'works for me': https://issues.apache.org/bugzilla/show_bug.cgi?id=30450

I'd really appreciate any insight you have. Here is some other info about the project:

• apache-tomcat-6.0.28
• java 1.6
• maven
• Spring 2.5 (heavy use of MVC with jstl)

Thanks!

As noted in the comments above, I shouldn't be clearing the Tomcat's work directory while it is running. Thanks guys!

There's another instance I discovered, that may cause similar situations. I have two apps deployed in tomcat, and when I deploy the third one, I'm starting to observe this behavior.

In this particular case I've found out that the tomcat with these three apps opens a lot of files, and it requires in this instance sometimes about 1600 files opened to work (system libs, jars, temps etc.). When it exceeds system default 1024 max opened files in some requests, it looks that some files cannot be opened and I have randomly strange errors (including frequently ClassNotFoundException in jsp-s). All this depends on how quickly I click in these apps in browser windows, the files are opened and closed very quickly.

This helped for me and I no longer see problems with this instance:

ulimit -n 2048

This might be used to observe during the server runtime, what is the amount of opened files (if the tomcat is run as "tomcat" user):

while [ true ]; do lsof -u tomcat|wc -l; sleep 1s; done