java.lang.LinkageError on jBoss while running app with log4j custom layout class

I am trying to configure a custom layout class to Log4J as described in my previous post. The class uses java.util.regex.Matcher to identify potential credit card numbers in log messages. It works perfectly in unit tests, also in a minimal web app containing a single servlet. However when I try to deploy it with our app in JBoss, I get the following error:

--- MBEANS THAT ARE THE ROOT CAUSE OF THE PROBLEM ---
ObjectName: jboss.web.deployment:war=MyWebApp-2010_02-SNAPSHOT.war,id=476602902
  State: FAILED
  Reason: java.lang.LinkageError: java/util/regex/Matcher

I couldn't even find any info on this form of the error - typically LinkageError seems to show up with a "loader constrain violation" message, like in here.

Technical details: we use JBoss 4.2, Java 5, Log4J 1.2.12. We deploy our app in an .ear, which contains (among others) the above mentioned .war file, and the custom layout class in a separate jar file (let's call it Commons). We override the default settings in jboss-log4j.xml with our own log4j.properties located in a different folder, which is added to the classpath at startup, and is provided via Sapient's Carbon framework.

Update to @skaffman's answer:

The reason we have a separate log4j.properties file is the scheme propagated by Sapient Carbon. This basically decouples the configuration and data files from the application server environment, so that they are accessible via Carbon's lookup functionality and they can be stored in a directory external to the app server. We inherited this setup, and we hate it because it causes us lots of trouble with deployment, classpath issues etc. since it does not adhere to the JEE conventions. We aim to get rid of it in the long run, but it's gonna take time :-(

Even though the separate log4j.properties file is not best practice, it certainly works. It has been functioning in our app for years, and I could also make it work with a minimalist web app containing a single servlet (not using Sapient Carbon). If log4j.properties is put into the classpath, Log4J reads it properly when the web app is launched, and reconfigures logging accordingly.

Update#2: An interesting finding is that Matcher is not even used in MyWebApp, only in the Commons module (and another module, in a separate jar). In Commons, it has been used before, in a class called StringHelper, which is used indirectly by MyWebApp, via other modules.

I guess this rules out the possibility of two different Matcher class versions loaded by different classloaders. So my only remaining guess is that Matcher is loaded by two different classloaders when it is used from the jar and the war, and then attempted to pass from one to the other. This is explained by Frank Kieviet's excellent article. However, I believe that such a setup would cause a "loader constraint violation" rather than this form of the error.

Update#3: If I add this appender (example 3.8) to jboss-log4j.xml, the error disappears, and the server runs perfectly :-o This obviously has to do something with loading log4j.jar, because this setup requires the jar to be present in the server lib directory. It works also if I change the appender type to org.jboss.logging.appender.FileAppender, and set log level to WARN, which results in an empty ucl.log file. This may suit as a temporary workaround, but I am still eager to fully understand what's going on here.

What does this error message mean, and how can I fix it properly?

Epilogue

After a long wait, I finally got to eliminate Carbon from the logging process and migrate our logging config into server/conf/jboss-log4j.xml. This required that I publish our custom log filter class in a separate jar in the server/lib directory. After this, the class loading works again, without the workaround 开发者_开发知识库described in Update#3 above :-)

My first reaction is that in JBoss it's not possible to override the log4j configuration like that. JBoss isn't allowing log4j to locate its own configuration, as it normally would, the location of conf/jboss-log4j.xml is specified in conf/jboss-service.xml.

To my knowledge, all log4j configuration in a given JBoss server must be centralised in to a single file, usually conf/jboss-log4j.xml.

Have you tried, as a test, moving the contents of your log4j.properties into the existing conf/jboss-log4j.xml file? If that works fine, then the problem is almost certainly caused by your attempt to override log4j. Having said that, I'd be surprised if jboss/log4j is that fragile, but perhaps in certain cases, it rejects this.

you either have two classes of different signatures but the same path in your environment or you compiled against another signature of j.u.r.Matcher. Since this is standard Java API, I think you should check your source and compilation targets and the JVM runtime version of your JBoss installation.

Edit:

After that is ruled out, I'm sure, the classloader (the server's) that manages the appenders and tries to load the appender that's using your custom layout can't see the custom layout class instance. So you have two options:

1. Deploy your custom layout JAR to the server's lib-directory along with log4j.
2. Deploy log4j along with your application and isolate the application with your own classloader (jboss-app.xml):

<jboss-app> <loader-repository> com.myapplication:loader=MyClassLoader <loader-repository-config>java2ParentDelegation=false</loader-repository-config </loader-repository> </jboss-app>

I hope, the problem will go away then.