Problem with pipeline implementation

I have created an application which uses the pipeline pattern to do some processing. However, I noticed that when the pipeline is run multiple times in a row it tends to get slower and slower.

This is also the case when no actual processing is done in the pipeline stages - so I am curious if maybe my pipeline implementation has a problem.

This is a simple test program which repoduces the effect:

#include <iostream>
#include <boost/thread.hpp>

class Pipeline {

    void processStage(int i) {

        return;
    }


public:

    void run() {


        boost::thread_group threads;

        for (int i=0; i< 8; ++i) {

            threads.add_thread(new boost::thread(&Pipeline::processStage, this, i));
        }

        threads.join_all();
    }
};



int main() {


    Pipeline pipeline;

    int n=2000;
    for (int i=0;i<n; ++i) {

            pipeline.run();

          开发者_如何学Python  if (((i+1)*100)/n > (i*100)/n) 
                std::cout << "\r" << ((i+1)*100)/n << " %";
    }

}

In my understanding the threads are created in run() and at the end of run() they are terminated. So the state of the program at the beginning of the outer loop in the main program should always be the same...

But what I observe is an increating slowdown when processing this loop.

I know that it would be more efficient to keep the pipeline threads alive thoughout the whole program - but I need to know if there is a problem with my pipeline implementation.

Thanks! Constantin

I do not know the exact reason for the slowdown in run(), but when I use the code obove and insert a little sleep (500ms) at the end of the loop in main() then the slowdown of run() is gone. So the system seems to need some "recover time" until it is able to create new threads.

Since you do new boost::thread() did you try to clean them up? If you run on windows, see the Task Manager about the number of threads opened by the process and if required close the thread handles. I suspect, The number of threads created by the system is keep increasing..