开发者

How to I read several lines in a file faster using python?

As of now I use the following python code:

file = open(filePath, "r")
lines=file.readlines()开发者_运维知识库
file.close()

Say my file has several lines (10,000 or more), then my program becomes slow if I do this for more than one file. Is there a way to speed this up in Python? Reading various links I understand that readlines stores the lines of file in memory thats why the code gets slow.

I have tried the following code as well and the time gain I got is 17%.

lines=[line for line in open(filePath,"r")]

Is there any other module in python2.4 (which I might have missed). Thanks, Sandhya


for line in file:

This gives you an iterator that reads the file object one line at a time and then discards the previous line from memory.

A file object is its own iterator, for example iter(f) returns f (unless f is closed). When a file is used as an iterator, typically in a for loop (for example, for line in f: print line), the next() method is called repeatedly. This method returns the next input line, or raises StopIteration when EOF is hit. In order to make a for loop the most efficient way of looping over the lines of a file (a very common operation), the next() method uses a hidden read-ahead buffer. As a consequence of using a read-ahead buffer, combining next() with other file methods (like readline()) does not work right. However, using seek() to reposition the file to an absolute position will flush the read-ahead buffer. New in version 2.3.

Short answer: don't assign the lines to a variable, just perform whatever operations you need inside the loop.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜