Linux - force single-core execution and debug multi-threading with pthread

I'm debugging a multi-threaded problem with C, pthread and Linux. On my MacOS 10.5.8, C2D, is runs fine, on my Linux computers (2-4 cores) it produces undesired outputs.

I'm not experienced, therefore I attached my code. It's rather simple: each new thread creates two more threads until a maximum is reached. So no big deal... as I thought until a couple of days ago. Can I force single-core execution to prevent my bugs from occuring?

I profiled the programm execution, instrumenting with Valgrind:

valgrind --tool=drd --read-var-info=yes --trace-mutex=no ./threads

I get a couple of conflicts in the BSS segment - which are caused by my global structs and thread counter variales. However I could mitigate these conflicts with forced signle-core execution because I think the concurrent sheduling of my 2-4 core test-systems are responsible for my errors.

#include <pthread.h>
#include <stdio.h>
#include开发者_运维百科 <stdlib.h>
#include <unistd.h>

#define MAX_THR      12
#define NEW_THR      2

int wait_time = 0;  // log global wait time
int num_threads = 0;    // how many threads there are
pthread_t threads[MAX_THR]; // global array to collect threads
pthread_mutex_t mut = PTHREAD_MUTEX_INITIALIZER; // sync

struct thread_data 
{
    int nr;             // nr of thread, serves as id
    int time;           // wait time from rand()
};
struct thread_data thread_data_array[MAX_THR+1];


void 
*PrintHello(void *threadarg)
{

  if(num_threads < MAX_THR){
    // using the argument

    pthread_mutex_lock(&mut);
    struct thread_data *my_data;
    my_data = (struct thread_data *) threadarg;

    // updates
    my_data->nr = num_threads;
    my_data->time= rand() % 10 + 1;

    printf("Hello World! It's me, thread #%d and sleep time is %d!\n", 
                my_data->nr, 
                my_data->time); 

     pthread_mutex_unlock(&mut);

   // counter
   long t = 0;

   for(t = 0; t < NEW_THR; t++){
        pthread_mutex_lock(&mut);
            num_threads++;
            wait_time += my_data->time;
           pthread_mutex_unlock(&mut);
        pthread_create(&threads[num_threads], NULL, PrintHello, &thread_data_array[num_threads]);
      sleep(1);
   }
    printf("Bye from %d thread\n", my_data->nr);

   pthread_exit(NULL);
   }
   return 0;
}

int 
main (int argc, char *argv[])
{

    long t = 0;
    // srand(time(NULL));
    if(num_threads < MAX_THR){
       for(t = 0; t < NEW_THR; t++){
          // -> 2 threads entry point
          pthread_mutex_lock(&mut);
                 // rand time
                 thread_data_array[num_threads].time  = rand() % 10 + 1;
           // update global wait time variable
              wait_time += thread_data_array[num_threads].time;
              num_threads++;
        pthread_mutex_unlock(&mut);
        pthread_create(&threads[num_threads], NULL, PrintHello, &thread_data_array[num_threads]);
         pthread_mutex_lock(&mut);
        printf("In main: creating initial thread #%ld\n", t);
        pthread_mutex_unlock(&mut);

      }
    }

   for(t = 0; t < MAX_THR; t++){
        pthread_join(threads[t], NULL);
    }

    printf("Bye from program, wait was %d\n", wait_time);
    pthread_exit(NULL);
}

I hope that code isn't too bad. I didn't do too much C for a rather long time. :) The problem is:

printf("Bye from %d thread\n", my_data->nr);

my_data->nr sometimes resolves high integer values:

In main: creating initial thread #0
Hello World! It's me, thread #2 and sleep time is 8!
In main: creating initial thread #1
[...]
Hello World! It's me, thread #11 and sleep time is 8!
Bye from 9 thread
Bye from 5 thread
Bye from -1376900240 thread
[...] 

I don't now more ways to profile and debug this. If I debug this, it works - sometimes. Sometimes it doesn't :(

Thanks for reading this long question. :) I hope I didn't share too much of my currently unresolveable confusion.

Since this program seems to be just an exercise in using threads, with no actual goal, it is difficult to suggest how treat your problem rather than treat the symptom. I believe can actually pin a process or thread to a processor in Linux, but doing so for all threads removes most of the benefit of using threads, and I don't actually remember how to do it. Instead I'm going to talk about some things wrong with your program.

C compilers often make a lot of assumptions when they are doing optimizations. One of the assumptions is that unless the current code being examined looks like it might change some variable that variable does not change (this is a very rough approximation to this, and a more accurate explanation would take a very long time).

In this program you have variables which are shared and changed by different threads. If a variable is only read by threads (either const or effectively const after threads that look at it are created) then you don't have much to worry about (and in "read by threads" I'm including the main original thread) because since the variable doesn't change if the compiler only generates code to read that variable once (remembering it in a local temporary variable) or if it generates code to read it over and over the value is always the same so that calculations based on it always come out the same.

To force the compiler not do this you can use the volatile keyword. It is affixed to variable declarations just like the const keyword, and tells the compiler that the value of that variable can change at any instant, so reread it every time its value is needed, and rewrite it every time a new value for it is assigned.

NOTE that for pthread_mutex_t (and similar) variables you do not need volatile. It if were needed on the type(s) that make up pthread_mutex_t on your system volatile would have been used within the definition of pthread_mutex_t. Additionally the functions that access this type take the address of it and are specially written to do the right thing.

I'm sure now you are thinking that you know how to fix your program, but it is not that simple. You are doing math on a shared variable. Doing math on a variable using code like:

x = x + 1;

requires that you know the old value to generate the new value. If x is global then you have to conceptually load x into a register, add 1 to that register, and then store that value back into x. On a RISC processor you actually have to do all 3 of those instructions, and being 3 instructions I'm sure you can see how another thread accessing the same variable at nearly the same time could end up storing a new value in x just after we have read our value -- making our value old, so our calculation and the value we store will be wrong.

If you know any x86 assembly then you probably know that it has instructions that can do math on values in RAM (both getting from and storing the result in the same location in RAM all in one instruction). You might think that this instruction could be used for this operation on x86 systems, and you would almost be right. The problem is that this instruction is still executed in the steps that the RISC instruction would be executed in, and there are several opportunities for another processor to change this variable at the same time as we are doing our math on it. To get around this on x86 there is a lock prefix that may be applied to some x86 instructions, and I believe that glibc header files include atomic macro functions to do this on architectures that can support it, but this can't be done on all architectures.

To work right on all architectures you are going to need to:

 int local_thread_count;
 int create_a_thread;

 pthread_mutex_lock(&count_lock);
 local_thread_count = num_threads;
 if (local_thread_count < MAX_THR) {
     num_threads = local_thread_count + 1;
     pthread_mutex_unlock(&count_lock);

     thread_data_array[local_thread_count].nr = local_thread_count;
                                           /* moved this into the creator
                                            * since getting it in the
                                            * child will likely get the
                                            * wrong value. */

     pthread_create(&threads[local_thread_count], NULL, PrintHello,
                                       &thread_data_array[local_thread_count]);

 } else {
     pthread_mutex_unlock(&count_lock);
 }

Now, since you would have changed the num_threads to volatile you can atomically test and increment the thread count in all threads. At the end of this local_thread_count should be usable as an index into the array of threads. Note that I did not create but 1 thread in this code, while yours was supposed to create several. I did this to make the example more clear, but it should not be too difficult to change it to go ahead and add NEW_THR to num_threads, but if NEW_THR is 2 and MAX_THR - num_threads is 1 (somehow) then you have to handle that correctly somehow.

Now, all of that being said, there may be another way to accomplish similar things by using semaphores. Semaphores are like mutexes, but they have a count associated with them. You would not get a value to use as the index into the array of threads (the function to read a semaphore count won't really give you this), but I thought that it deserved to be mentioned since it is very similar.

man 3 semaphore.h

will tell you a little bit about it.

num_threads should at least be marked volatile, and preferably marked atomic too (although I believe that the int is practically fine), so that at least there is a higher chance that the different threads are seeing the same values. You might want to view the assembler output to see when the writes of num_thread to memory are actually supposedly taking place.

https://computing.llnl.gov/tutorials/pthreads/#PassingArguments

that seems to be the problem. you need to malloc the thread_data struct.