Limit number of rows - TSQL - Merge - SQL Server 2008

Hi all i have the following merge sql script which works fine for a relatively small number of rows (up to about 20,000 i've found). However sometimes the data i have in Table B can be up to 100,000 rows and trying to merge this with Table A (which is currently at 60 million rows). This takes quite a while to process, which is understandable as it has to merge 100,000 with 60 million existing records!

I was just wondering if there was a better way to do this. Or is it possible to have some sort of count, so merge 20,000 rows from Table B to Table A. Then delete those merged rows from table B. Then do the next 20,000 rows and so on, until Table B has no rows left?

Script:

MERGE
    Table A AS [target]
开发者_运维技巧USING
    Table B AS [source]
ON
    ([target].recordID = [source].recordID)
WHEN NOT MATCHED BY TARGET
    THEN
        INSERT([recordID],[Field 1]),[Field 2],[Field 3],[Field 4],[Field 5])
        VALUES([source].[recordID],[source].[Field 1],[source].[Field 2],[source].[Field 3],[source].[Field 4],[source].[Field 5]
    );

MERGE is overkill for this since all you want is to INSERT missing values.

Try:

INSERT INTO Table_A
([recordID],[Field 1]),[Field 2],[Field 3],[Field 4],[Field 5])
SELECT  B.[recordID],
        B.[Field 1],B.[Field 2],B.[Field 3],B.[Field 4],B.[Field 5]
FROM Table_B as B
WHERE NOT EXISTS (SELECT 1 FROM Table_A A
                  WHERE A.RecordID = B.RecordID)

In my experience MERGE can perform worse for simple operations like this. I try to reserve it for when you need varying operations depending on conditions, like an UPSERT.

You can definitely do (SELECT TOP 20000 * FROM B ORDER BY [some_column]) as [source] in USING and then delete these records after MERGE. So you pseudo-code will look like :

1. Merge top 20000
2. Delete 20000 records from source table
3. Check @@ROWCOUNT. If it's 0, exit; otherwise goto step 1

I'm not sure if it runs any faster than merging all the records at the same time. Also, are you sure you need MERGE? From what I see in your code INSERT INTO ... SELECT should also work for you.