开发者

How do I make two perl files communicate?

So I have something like this:

fork.pl

for $str (@files)  
{        
    my($command) = "perl command.pl ".$str;
    exec( $command );
}

command.pl

$file=$ARGV[0].".csv";
#code t开发者_JS百科hat counts rows here
print $rowcount;

So as the end result I have 10 files launched which count how many rows are in each csv file.

I do not need help editting this code, it works (this is just a compressed version). I need help figuring out how to take the output ($rowcount) of ten files and combine it into one for further processing.


I keep some utility code around for just this purpose... this is tweaked slightly to your question and including a synchronized global counting method.

#!/usr/bin/perl
use threads;
use Thread::Queue;

my @workers;
my $num_threads = 10;
my $queue = new Thread::Queue;
my $total_ines = 0;

for (0..$num_threads-1) {
        $workers[$_] = new threads(\&worker);
}

while ($_ = shift @ARGV) {
        $queue->enqueue($_);
}

sub worker() {
        while ($file = $queue->dequeue) {
            #line counting code here
            global_counter($lines_counted);
        }
}

sub global_counter() :locked {
    #add to the number of lines counted
    $total_lines += shift
}

for (0..$num_threads-1) { $queue->enqueue(undef); }
for (0..$num_threads-1) { $workers[$_]->join; }

print $total_lines;


This kind of communication is solved using pipes (let me write a simple example):

# -- fork.pl -------------------------
for (1..3)  {        
   open my $PIPE, "perl command.pl |";
   print "catch: $_\n" while(<$PIPE>);
   close $PIPE;
}
# -- command.pl ----------------------
print rand(1);

It prints (random numbers):

catch: 0.58929443359375
catch: 0.1290283203125
catch: 0.907012939453125


You need to look either at threads or Interprocess communication with e.g. sockets or shared memory when using fork.


Compressed but won't work. I'm assuming that in fork.pl, you fork before exec'ing? Backticks capture the output of the called process, namely your prints: fork.pl

for $str (@files)  
{        
    my($command) = "perl command.pl ".$str;
    print `$command`;
}

But rather than forking and launching processes, wouldn't it be smarter to turn the second file into a module?

package MyCommand;
use Exporter;

our @EXPORT = qw( command );
sub command {
   my $file = $_[0] . '.csv';

   ...
   return $rowcount;
}

1;

fork.pl:

use MyCommand;

...
my @rowcounts;
for my $str (@files) {        
    push @rowcounts, command($str);
}

A bit of self-promotion, but I just posted this in your other thread, which seems relevant enough: How to run in parallel two child command from a parent one?


Accumulate pipes from children:

#!/usr/bin/perl -w

use strict;

my $files = qw/one.csv two.csv three.csv/;
my $command = "perl command.pl";

my @pipes;
foreach (@files) {
    my $fd;
    open $fd, "-|", "$command $_" and push @pipes, $fd;
};

my $sum = 0;
foreach my $pp (@pipes) {
    $sum += $_ if defined ($_=<$pp>);
};

print $sum;

Then you can just read them one by one (as in example), or use IO::Select to read data as it appears in each pipe.

A hash table in addition to array is also good if you want to know which data comes from which source.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜