开发者

How to make record insert operation faster in django with postgres backend

I build a django based geo开发者_StackOverflowlocation service which determines user's location by their IP address. The first thing I need to do is to insert ip data into my database.

I used following code(simplified) to insert records into my db:

 for ipLoc in ipSeeker.ipLocationList:
     placeName =ipLoc.country + ipLoc.area
     IPLog.objects.create(
                startIP = int_to_dqn(ipLoc.startIP), 
                endIP = int_to_dqn(ipLoc.endIP), 
                place = placeName
     ).save()

The ipLocationList has approx 400k ip records. And my script only insert 20k records in 20 minutes. It is tooooooo slow and could not be accepted.

So my question is: where is the bottleneck and how could I make it faster?

Thanks in advance!


Use raw sql and transactions for django-side: https://docs.djangoproject.com/en/dev/topics/db/sql/ along with the Copy command from Postgres!

Whats the fastest way to do a bulk insert into Postgres?


I found a complete tutorial about how to load GEOIP database into postgresSQL in the siafoo. It is well written and covers most detail. The link is http://www.siafoo.net/article/53

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜