Ordering Output in MPI

in a simple MPI program I have used a column wise division of a large matrix. How can I order the output so that each matrix appears next to the other ordered ? I hav开发者_开发百科e tried this simple code the effect is quite different from the wanted:

for(int i=0;i<10;i++)
{
    for(int k=0;k<numprocs;k++)
    {
        if (my_id==k){
            for(int j=1;j<10;j++)
                printf("%d",data[i][j]);
        }
        MPI_Barrier(com);
    }
    if(my_id==0)
        printf("\n");
}

Seems that each process has his own stdout and so is impossible to have ordered lines output without sending all the data to one master which will print out. Is my guess true ? Or what I'm doing wrong ?

You guessed right. The MPI standard does not specify how stdout from different nodes should be collected for printing at the originating process. It is often the case that when multiple processes are doing prints the output will get merged in an unspecified way. fflush doesn't help.

If you want the output ordered in a certain way, the most portable method would be to send the data to the master process for printing.

For example, in pseudocode:

if (rank == 0) {
    print_col(0);
    for (i = 1; i < comm_size; i++) {
        MPI_Recv(buffer, .... i, ...);
        print_col(i);
    }
} else {
    MPI_Send(data, ..., 0, ...);
}

Another method which can sometimes work would be to use barries to lock step processes so that each process prints in turn. This of course depends on the MPI Implementation and how it handles stdout.

for(i = 0; i < comm_size; i++) {
    MPI_Barrier(MPI_COMM_WORLD);
    if (i == rank) {
         printf(...);
    }
}

Of course, in production code where the data is too large to print sensibly anyway, data is eventually combine by having each process writing to a separate file and merged separately, or using MPI I/O (defined in the MPI2 standards) to coordinate parallel writes.

I produced ordered output to a file before using the exact same method. You could try printing to a temporary file, printing the contents of said file and then deleting it.

Have the root processor do all of the printing. Use MPI_Send/MPI_Recv or MPI_Gather (or whatever) to send the data in turn from each processor to the root.

To solve this problem you can use short sleep. I use and then it works in 99%

printf("text nr 1\n");
MPI_Barrier(MPI_COMM_WORLD);
usleep(100);
printf("text nr 2\n");

It's not very elegant but works.