Efficent database access in terms of record insertion
I apologize in advance for the "rookie" question, but I am still fundamentally a novice in most aspects of SQL. My question stems from part of Chapter Two in The Art of SQL. In the passage titled, "Stable Database Connections", the author mentions several ways to insert a large number of rows into a database. Here is the corresponding list in the format
Test; Results Connect/Disconnect for each line in turn; 7.4 lines loaded per second Connect Once, all candidate lines individually inserted; 1,681 lines loaded per second Connect Once, all candidate lines inserted in arrays of 10 lines; 5,914 lines inserted per second Connect Once, all candidate lines inserted in arrays of of 100 lines; 9,190 lines inserted per second
In addition, the author mentions "direct-loading techniques that are even faster."
Unfortun开发者_运维技巧ately, I do not completely understand the idea of insertion of arrays (if at all). Could anyone elaborate on the techniques the author referring to, in terms of inserting arrays and the other "direct loading techniques" that he referenced?
The author is showing that
INSERT INTO Table (Column)
VALUES ('myValue')
is only going to be so fast.
It will be significantly faster to do
INSERT INTO Table (Column)
VALUES ('myFirstValue'),
('mySecondValue'), ...
The author is saying that having 10 VALUES
items inserts 5,914 lines per second, but if there 100 items in the single SQL statement, then the load rate goes up to 9,190 lines per second.
Finally, the "direct-loading techniques" are referring to bulk insert operations. You can google msSQL bulk insert to find numerous tutorials for uploading a file and then processing it into a table.
The fastest way to insert large amounts of data into a SQLServer table is with a process know as "Bulk Insert". There are several ways to do this. There is a command line app that comes with SQL Server called bcp, there is a TSQL command BULK INSERT, and there is a .NET sql client object, System.Data.SqlClient.SqlBulkCopy.
精彩评论