Way to deal with large data files in Wolfram Mathematica

I wonder if there exists way to work with large files in Mathematica ? Currently I have a file about 500Mb with table data.

Import["data.txt","Table"];

What is 开发者_如何学Pythonalternate way?

Use OpenRead["file"] which gives you an InputStream object on which you can use Read[stream]. Depending on the formatting of the data file you may need to set custom option values in Read[] for RecordSeparators.

Example:

In[1]:= str = OpenRead["ExampleData/USConstitution.txt"]    
Out[1]= InputStream["ExampleData/USConstitution.txt", 24]   

In[2]:= Read[str, Word]    
Out[2]= "We"    
In[3]:= Read[str, Word]
Out[3]= "the"    
In[4]:= Read[str, Record]
Out[4]= "People of the United States, in Order to form a more perfect Union,"

You could also load your data into a database (for example MySQL) and access it from Mathematica using DatabaseLink

The function DumpSave can also be helpful for saving large datasets. It saves data in Mathematica's internal format, so it's more efficient in both time and file size.