Using maximum threads on multicore cpu

at first, my perlskills are limited so please keep that in mind. ;)

I have written a perlscript which indexes a directory and does something for each file in it.

Processing a file takes some time. Between 1 - 5 mins each at a cpu load nearly 100% (one core). My idea is, cause i have a quadcore cpu, to process more then one file at once, which leads me to perl threads.

So here a开发者_JAVA百科re my questions.

1) Is my assumption right, that perl threads are allocated automatically to multiple cores?

2) I found this code example which does what i need, i think but i cannot figure out, how to keep always only 8 threads active. The example starts a static count of threads and is done when they are processed. However, in my case i have, lets say, 50 files to process but only 8 threads should be active at the same time.

So it should be like this: Read the directory, start 8 threads for the 8 first files and keep 8 threads working until all files are processed.

#!/usr/local/roadm/bin/perl
# This is compiled with threading support

use strict;
use warnings;
use threads;
use threads::shared;

print "Starting main program\n";

my @threads;
for ( my $count = 1; $count <= 10; $count++) {
        my $t = threads->new(\&sub1, $count);
        push(@threads,$t);
}   
foreach (@threads) {
        my $num = $_->join;
        print "done with $num\n";
}   
print "End of main program\n";

sub sub1 {
        my $num = shift;
        print "started thread $num\n";
        sleep $num;
        print "done with thread $num\n";
        return $num;
}   

Source: https://wiki.bc.net/atl-conf/pages/viewpage.action?pageId=20548191

I searched some hours now but did not find an example, how to do this. Would be nice when someone could give me a hint how to start.

Thank you.

Threads in perl are heavyweight; they take time and cpu to start and time, cpu, and memory to share data between them (and require that you carefully check what modules you are using for thread safety).

You are often much better off forking; Parallel::ForkManager makes this easy.

Good solution to your problem using Thread::Queue and number of workers can be found in this answer.

#!/usr/bin/env perl -w
use strict;
use threads;

my $cpu ||= 5;
my @threads;
my @all_target = qw /1 2 3 4 5 6 7 8 9/;

while(my($t,$job) = each @all_target){
    $t = threads -> create(\&do,$job);
    push @threads,$t;

    if(@threads == $cpu or $t == $#all_target){
        for(@threads){
            $_ -> join();
        }
        @threads = ();
    }
}`enter code here`

sub do{
    my $be_do = shift;
    print"$be_do\n";
    sleep 2;
}

#!/usr/bin/env perl -w
use strict;
use threads;

my $cpu ||= 5;
my @threads;
my @all_target = qw /1 2 3 4 5 6 7 8 9/;

while(my($t,$job) = each @all_target){

    my $thread = threads -> create(\&do,$job);
    push @threads,$thread;

    if(@threads == $cpu or $t == $#all_target){
        for(@threads){
            $_ -> join();
            print "$_ join\n";
        }
        @threads = ();
    }
}

sub do{
    my $be_do = shift;

    print"$be_do\n";
    sleep 2;
#   $cmd = "do what";
#   system("$cmd");
}