Performance question about Mongo database
today I have tested the Mongo database, but I got a performance issue. After I insert 1.800.00, I tried to make a sum of all values but it too 57s. Then I tried the same thing in MSSQL and took 0s!!
Can you give any tips what I'm doing wrong? Is this a Mango limitation?
static void Main(string[] args)
{
//Create a default mongo object. This handles our connections to the database.
//By default, this will connect to localhost, port 27017 which we already have running from earlier.
var connStr = new Mo开发者_如何学运维ngoConnectionStringBuilder();
connStr.ConnectTimeout = new TimeSpan(1, 0, 0);
connStr.SocketTimeout = new TimeSpan(1, 0, 0);
connStr.Server = new MongoServerAddress("localhost");
var mongo = MongoServer.Create(connStr);
//Get the blog database. If it doesn't exist, that's ok because MongoDB will create it
//for us when we first use it. Awesome!!!
var db = mongo.GetDatabase("blog");
var sw = new Stopwatch();
sw.Start();
//Get the Post collection. By default, we'll use the name of the class as the collection name. Again,
//if it doesn't exist, MongoDB will create it when we first use it.
var collection = db.GetCollection<Post>("Post");
Console.WriteLine(collection.Count());
sw.Stop();
Console.WriteLine("Time: " + sw.Elapsed.TotalSeconds);
sw.Reset();
sw.Start();
var starting = collection.Count();
var batch = new List<Post>();
for (int i = starting; i < starting + 200000; i++)
{
var post = new Post
{
Body = i.ToString(),
Title = "title " + i.ToString(),
CharCount = i.ToString().Length,
CreatedBy = "user",
ModifiedBy = "user",
ModifiedOn = DateTime.Now,
CreatedOn = DateTime.Now
};
//collection.Insert<Post>(post);
batch.Add(post);
}
collection.InsertBatch(batch);
Console.WriteLine(collection.Count());
sw.Stop();
Console.WriteLine("Time to insert 100.000 records: " + sw.Elapsed.TotalSeconds);
//var q = collection.Find(Query.LT("Body", "30000")).ToList();
//Console.WriteLine(q.Count());
sw.Reset();
sw.Start();
var q2 = collection.AsQueryable<Post>();
var sum = q2.Sum(p => p.CharCount);
Console.WriteLine(sum);
sw.Stop();
Console.WriteLine("Time to sum '" + q2.Count() + "' Post records: " + sw.Elapsed.TotalSeconds); //PROBLEM: take 57 to SUM 1.000.000 records
} }
Performance issue in the following row:
var q2 = collection.AsQueryable<Post>();
In row above you loading all posts from the posts collection into memory, because of driver does not support linq. In MSSQL it's taking less than second because of linq and calculating will go through the database. Here i guess almost all 57 second need to load data into memory.
In mongodb to achieve best performance you need to create extra fields (de normalize data) and calculate any sums,counters, etc whenever it possible. If it not possible you need to use map/reduce or available aggregate functions, like group (good fit for your example of sum calculation).
精彩评论