Using OpenGL glutDisplayFunc within class

I've created a C++ class (myPixmap) to encapsulate the work performed by the OpenGL GLUT toolkit. The display() member function of the class contains most of the code required to set up GLUT.

void myPixmap::display()
{
    // open an OpenGL window if it hasn't already been opened
    if (!openedWindow)
    {
        // command-line arguments to appease glut
        char *argv[] = {"myPixmap"};
        int argc = 1;
        glutInit(&argc, argv);
        glutInitDisplayMode(GLUT_SINGLE | GLUT_RGB);
        glutInitWindowSize(640, 480);
        glutInitWindowPosition(30, 30);
        glutCreateWindow("Experiment");
        glutDisplayFunc(draw);
        glClearColor(0.9f, 0.9f, 0.9f, 0.0);
        glClear(GL_COLOR_BUFFER_BIT);
        glutMainLoop();

        openedWindow = true;
    }  
}

The display function passed to glutDisplayFunc() is another member function of the class:

void myPixmap::draw(void)
{
    glDrawPixels( m,n,GL_RGB,GL_UNSIGNED_BYTE, pixel );
}

However, gcc 4.2.1 on Mac OS X 10.6.4 refuses to compile this code, claiming that:

argument of type 'void (myPixmap::)()' does not match 'void (*)()'

Is there a way to use the GLUT toolkit inside a member function of a class, or must I call all GLUT functions within the main function of the program?

The problem is that a pointer to an instance bound member function has to include the this pointer. OpenGL is a C API, and knows nothing about this pointers. You'll have to use a static member function (which doesn't require an instance, and thus no this), and set some static data members (to access the instance) in order to use glutDisplayFunc.

class myPixmap
{
private:
  static myPixmap* currentInstance;

  static void drawCallback()
  {
    currentInstance->draw();
  }

  void setupDrawCallback()
  {
    currentInstance = this;
    ::glutDisplayFunc(myPixmap::drawCallback);
  }
};

You may also have problems with C linkage vs C++ linkage, in which case you'll have to play around with extern "C". If so, you might have to use a global function, rather than a static member function as your callback, and have that call myPixmap::draw. Something like:

class myPixmap
{
public:
  void draw();

private:
  void setupDrawCallback();
};

myPixmap* g_CurrentInstance;

extern "C"
void drawCallback()
{
  g_CurrentInstance->draw();
}

void myPixmap::setupDrawCallback();
{
  ::g_CurrentInstance = this;
  ::glutDisplayFunc(::drawCallback);
}

With all of this, try to make as few changes as possible, since this is really kind of a kludge to deal w/ OpenGL being a C API.

If you want multiple instances (I don't think most people using GLUT make multiple instances, but maybe you are), you'll have to figure out a solution using a std::map to retrieve the instance:

static std::map<int, myPixmap> instanceMap;

Where you'd get the int to resolve which instance, I am not sure :)

FYI, you should define functions that take no parameters this way:

void some_function() { }

not

void some_function(void) { }

Please check this tutorial, I think this is what you are looking for: http://paulsolt.com/2010/08/glut-object-oriented-framework-on-github/

Merlyn solution didnt work for me, so I made a little modification by bringing currentInstance outside of class and it worked:

Class myPixmap;
static myPixmap* currentInstance;
Class myPixmap {
...
}

I had trouble in understanding the correct answer from Merlyn, so here it is the simplified version:

In myPixmap.h, change display function to static member function. Should work.

class myPixmap
{
  //...
  static void display();
  //...
};