Merge two large CSV files with PHP

I want to merge two 开发者_JAVA百科large CSV files with PHP. This files are too big to even put into memory all at once. In pseudocode, I can think of something like this:

for i in file1
  file3.write(file1.line(i) + ',' + file2.line(i))
end

But when I'm looping through a file using fgetcsv, it's not really clear how I would grab line n from a certain file without loading the whole thing into memory first.

Any ideas?

Edit: I forgot to mention that each of the two files has the same number of lines and they have a one-to-one relationship. That is, line 62,324 in file1 goes with line 62,324 in file2.

Not sure what operating system you're on, but if you're using Linux, using the paste command is probably a lot easier than trying to do this in PHP.

If this is a viable solution and you don't absolutely need to do it in PHP, you could try the following:

paste -d ',' file1 file2 > combined_file

Take a look at the fgets function. You could read a single line of each file, process them, and write them to your new file, then move on to the next line until you've reached the end of your file.

PHP: fgets

Specifically look at the example titled Example #1 Reading a file line by line in the PHP manual. It's also important to note the return value of the the fgets functions.

Returns a string of up to length - 1 bytes read from the file pointed to by handle. If there is no more data to read in the file pointer, then FALSE is returned.

So, if it doesn't return FALSE you know you still have more lines to process.

You can use fgets().

$file1 = fopen('file1.txt', 'r');
$file2 = fopen('file2.txt', 'r');
$merged = fopen('merged.txt', 'w');

while (
    ($line1 = fgets($file1)) !== false
    && ($line2 = fgets($file2)) !== false) {

    fwrite($merged, $line1 . ',' . $line2);
}

fgets() reads one line from a file. As you can see, this code uses it on both files at the same time, writing the merged lines to a third file. The manual here:

http://php.net/fgets

http://php.net/fopen

http://php.net/fwrite

Try using fgets() to read one line from each file at a time.

I think the solution for this is to map first line begins for each line ( and some kind of key if you need ) and then make a new csv using fread and fwrite ( we know beginning and ending of each line now , so we need just seek and read )

Another way is to put it into MySQL ( if it is possible ) and then back to new CSV