开发者

alternative faster method to fscanf in c++?

I store my data in a text file using fprintf("%f\t%f\t%f\t%f\t%f\t%f\n",p1,p2,p3,p4,p5,p6);

There are 100,000 entries, and when i read them using

while(!EOF)
  fscanf("%f %f %f %f %f %f "&p1,&p2,&p3,&p4,&p5,&am开发者_如何学Pythonp;p6);

its taking a long time.

Any suggestion to read at a faster speed? Thanks in advance.


Read them into a string buffer then parse the string - if you insist on using 'c' see fgets() and sscanf()


Try to use boost.spirit.


You should use fstream. Create an fstream in object, like fin and then input the data info your file the same way you would use a cin. You can use the iomanip library for better formatting.


The reason for fscanf to be so slow is mainly because fetching memory from the hard-drive is slow. With the code you posted, one line of text is fetched from the file, then parsed with the formatting you gave. This results in a large amount of fetching small chunks of the file at a time. This is inefficient.

The solution, like with any other type of memory fetching operation, is to fetch larger blocks at a give time, and then operate on the block that was retrieved. In other words, you should dump the entire content of the file to RAM and then process it. To achieve that, you can use a stringstream. Basically, reading the entire file into a stringstream will result in much fewer retrieval of memory from the hard-drive (in theory, it will just transfer the entire file in one operation). Then, stringstream can be used just like any other input stream.

In reality, it will probably be more sound to find a way to avoid having to do any file I/O within any sort of time-critical or CPU-intensive process. Either do it before or after, try not to do it in the middle of a loop. For example, if you load all the data into an array/vector, then process the array/vector, then it no longer matters how much time it takes to load the data from the file because you do it before starting any time-critical processing of the data.

PS: Since you use fscanf(), I'm guessing you might be looking for a C-style solution instead of C++ standard libraries like fstream and stringstream. I'm sorry, I can't give you any hint about that, wanting to stick to C functions is a problem that I have overcome a long time ago, and hopefully you did too or will do so soon.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜