Problem with foreground extraction using opencv c++ and MS VS2010

I have a problem in extracting foreground from the video frame. It extracts few background objects too.I took a snap shot from the sample video to be used as the background image.Suggest me what should b done.Whether a separate image to be taken for the background image instead of taking snapshot from video(assuming,snapshot is of low resolution than image) or some other better code.The code used for extraction is

int _tmain(int argc, _TCHAR* argv[])
{   

    IplImage  *frame = NULL;
    IplImage  *img1 = NULL;

    Ipl开发者_开发百科Image  *grey  = NULL;
    IplImage  *edges = NULL;
        int delay = 0, key=0, i=0;
    CvCapture *video = NULL;
    CvCapture *video1 = NULL;

        cvNamedWindow("window_name");

        video = cvCreateFileCapture("sample.avi");
        video1 = cvCreateFileCapture("sample.avi");


    frame = cvQueryFrame(video);
    img1 = cvQueryFrame(video1);
       grey  = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1);
       edges  = cvCreateImage(cvGetSize(img1), IPL_DEPTH_8U, 1);

  cvShowImage("backgrnd", img1);

//get height and width using OpenCV functions
 const int &rows = img1->width;
 const int &cols =frame->height;
  BYTE *Pixel2=0;
  cvGetRawData(img1,&Pixel2,0,0);
         while (frame) {


 BYTE *Pixel1=0;
//extract pixels using the OpenCV function cvGetRawData

 cvGetRawData(frame,&Pixel1,0,0);

//register int to increase the speed
register  int r,ri,c;

 //to find diffrence of 2 images by pixel to pixel comparision
for(r = 0, ri = 0; r < rows*3; r++, ri += cols)
 {
     for(c = 0; c < cols; c++)
         {
             //get the difference in pixels
             Pixel1[ri + c] = Pixel1[ri + c] - Pixel2[ri + c];

             //set threshold value as 100 for comparision, it can be changed to values between 50 and 200, for getting binary image
             if(Pixel1[ri + c] < 150)
                 {
                     Pixel1[ri + c]=0;
                 }
             else   
                    Pixel1[ri + c]=255;   

         }//for c

 }//for r, ri   
Return 0;
}

Foreground/Background segmentation is a deep, difficult problem. The first and most important thing to ask yourself is "what is the definition of foreground as opposed to background?" Sometimes the background changes appearance drastically over time, making Foreground/Background segmentation especially difficult. Depending on your answer, there are various methods that work quite well. Some are statistical in nature, dealing with probability distributions for the color at a given pixel, or dealing with features extracted from the image. Some attempt to track features in a video and segment the image based on the dynamics of the various moving features. Still others are more algebraic in nature, based on a variant of Principal Component Analysis. There are a huge number of good algorithms out there, and nobody is using frame differencing these days.

If you're using OpenCV, I highly recommend getting a hold of OpenCV2.2, which is fresh of the presses. In the video module of OpenCV2.2, there are several nice tools for foreground/background segmentation. The documentation in the code even points you to references and papers that describe the theory behind the various algorithms.

Best of luck to you!