Shredding XML from DB using SSIS
I am looking for a way to pull XML from a SQL database and shred the XML via SSIS in bulk. I currently have a package that can pull XML from the database and pass the XML to a stored procedure, via variable, for shredding but this only works 1 record at a time. When processing 100,000 records, this can become quite time consuming.
I would like to shred multiple XML values at once using SSIS. Is this开发者_如何学JAVA possible with SSIS? Perhaps something in a Data Flow Task where all the XML values are selected from a source then passed to a parser of some sort?
So after a lot of searching and thinking about the problem, I redesigned the stored procedures that did the shredding. Instead of shredding a variable being passed into the stored procedure, I just shredded from the XML column itself. This allowed me to shred many XML values at once instead of looping and passing them to a stored procedure one at a time. This gave me the performance boost I was looking for. So instead of something like this...
SET @xmlData = CAST(@xmlMessageData AS XML)
SELECT
, row.value('xmlNode1[1]', 'BIT' ) AS Column1
, row.value('xmlNode2[1]', 'NVARCHAR(255)') AS Column2
, row.value('xmlNode3[1]', 'BIT' ) AS Column3
, row.value('xmlNode4[1]/Name[1]', 'NVARCHAR(255)' ) AS Column4
FROM @xmlData.nodes('xmlRootNode') AS T1(row)
I now do this...
SELECT
, row.value('xmlNode1[1]', 'BIT' ) AS Column1
, row.value('xmlNode2[1]', 'NVARCHAR(255)') AS Column2
, row.value('xmlNode3[1]', 'BIT' ) AS Column3
, row.value('xmlNode4[1]/Name[1]', 'NVARCHAR(255)' ) AS Column4
FROM [ESBMessagesData] D
CROSS APPLY
[DataItem].nodes('xmlRootNode') AS T1(row)
精彩评论