End to end profiling of web/J2EE and client applications

Is there a good tool to do end to end profiling of a system that uses J2EE and has clients?

I realize that you can easily profile the server and t开发者_运维问答he client seperately. However, I do not know of any tools that'll give you information about the effect of the network connection between the two and how it affect's their performance. [An end-to-end profiler, would take in consideration of when the application is stuck on communicating and when it is processing]

I am not aware of any profiling agent that provides an out-of-the-box feature to profile applications end-to-end. At least, not at the time of writing this.

However, you could consider using DTrace to collect profiling information (not necessarily all the info provided by a typical profiler) from both the client and the server at the same time. An example on how to use this in a Java web-application (with Firefox as the client, and Tomcat as the server) is available in this article. The trick lies in having DTrace instrumentation built into the JVM (running the server) and the client, and in writing a DTrace script that writes the collected information from the traces into a parseable output. Since, the question isn't clear about whether the client is in Java I'll assume that the client is also in Java; if not, the application/executable must support DTrace instrumentation.

A few caveats are to be mentioned though:

• DTrace is not available in all operating systems.
  • It is currently available in Solaris 10 and Mac OS Leopard.
  • I'm not aware of Dtrace support in Linux distributions, where SystemTap is the recommended alternative. FreeBSD has an experimental implementation of DTrace. It should be noted that SystemTap is not an implementation of DTrace for Linux, but a wholly different entity in itself.
  • On MSFT Windows, you're left to fend for yourself, atleast for the moment.
• Assumptions about the JVM and the client (since I know that the server runs on a JVM)
  • If your client is a JVM then you must use a JVM that has DTrace instrumentation or equivalent (I'm referring to SystemTap here) built into it, or your DTrace/SystemTap scripts will not work. This means that unless the authors of the JRE/JDK for your distribution, have added code to support DTrace/SystemTap, you are constrained to use a JVM that has such a support built in. If it is of importance in your context, OpenJDK 1.6.0 appears have instrumentation support for SystemTap, and the Oracle/Sun Java 6 distributions for Solaris have support for DTrace. On Java 1.4 and 5 on Solaris, you'll need to install a JVMTI agent.
  • If your client is a web-browser or a thick-client written in a different language, then you must ensure that the process it is running under can support DTrace. For instance, DTrace instrumented Firefox is available on Solaris 10, and presumably on Solaris Express 11 (I have verified DTrace support only on OpenSolaris 2009.06). SystemTap probes haven't been added in Firefox for a lot of Linux distributions, so you'll often need to build Firefox from source, with the appropriate flags in the Firefox build script to add the probes; my previous experience with this process leads me to believe that this is doable, but isn't easy. If you are running a different application (neither Firefox, nor a JVM) as the client, then you'll need to verify DTrace/SystemTap support for it.
• Writing good DTrace/SystemTap scripts to collect profiling information is not easy. You'll need to learn another language, and you'll need to ensure that the scripts do not add their own overhead (and validating the Heisenberg Uncertainty Principle in the context of profiling).

You can certainly write DTrace/SystemTap scripts that operate over remote clients and servers, but it is not advisable to write the collected instrumentation data to a socket (to avoid the afore-mentioned overhead).

Since version 8.0, JProfiler has the capability to follow RMI, web services and remote EJB calls between two profiled JVMs.

In the JVM that makes the call, call sites have hyperlinks that take you to the execution site in the remote JVM:

On the remote side, execution sites are recorded separately for each call sites, so you can inspect the call in isolation.

Disclaimer: My company develops JProfiler.