Is my execution plan trying to trick me?

I am trying to speed up a long running query that I have (takes about 10 minutes to run...). In order to track down what part of the query is costing me the most time I included the Actual Execution Plan when I ran it and found a particular section that was taking up 55% (screen shot below)

alt text http://img109.imageshack.us/img109/9571/53218794.png

This didn't quite seem right to me so I ad开发者_StackOverflowded Print '1' and Print '2' before and after this trouble section. When I run the query for a mere 17 seconds and then cancel it the 1 and 2 print out which I'm assuming means it's getting through that section in the first 17 seconds.

alt text http://img297.imageshack.us/img297/4739/66797633.png

Am I doing something wrong here or is my Execution plan misleading me?

Metrics from perfmon will also help figure out what's going wrong... you could be running into some serious IO issues with the drive your tempDB is residing on. Additionally, run a trace and look at the CPU & IO of the actual run.

Good perfmon metrics to look at are disk queue length (avg & writes).

If you don't have access to perfmon or don't want to trace things, use "SET STATISTICS IO ON" at the beginning of your query and allow it to complete...don't stop it. Just because an execution plan says it's taking over have the cost doesn't mean it will run for half of the query time...it could be much more (or less).

It says Query 10: Query cost (relative to the batch): 55%. Are 100% positive that it is the 10th statement in the batch that you surounded with Print statements? Could the INSERT ... INTO #mpProgramSet2 execute multiple times, some times in under 17 seconds other time for 5 minutes, depending on how much data was selected/inserted?

As a side note you should run with SET STATISTICS TIME ON rather that prints, this will give you exact compile/time and execution time of each statement in the batch.

I wouldn't trust that printing the '1' and '2' will prove anything about what has executed and what has not. I do the same thing, but I just wouldn't rely on it as proof. You could print the @@rowcount from that first insert query - that would indicate for sure that the insert has occurred.

Although the plan says that query may take 55% of the cost, it may not be 55% of the execution time, especially if the query results are cached.

Another advantage of printing the @@rowcount is to compare the actual number of rows to the estimated rows (51K). If they differ by a lot then you might investigate the statistics for your indexes.

We would need the full query to understand what's going on; but I would probably start with setting MAXDOP to 1 in order to limit the number of processors it's running on.

Note that sometimes queries need to be limited to only 1 processor due to locks etc.

Further you might try adding NOLOCKs to any of your selects which can get away with dirty reads.