Using masks in OpenGL ES 2.0 for iOS apps

I have an app where I want user to draw in one certain area of the screen. For 开发者_StackOverflow中文版this purpose I use a picture of mask which is black in drawable area and transparent in non-drawable area. So user can draw only on the area of the screen inside the mask and inside the black area of the mask.

I've tried to implement it via stencil buffer and modified some code from GLPaint sample project: http://pastebin.com/94MBr1Su

However I still don't get the idea of stencil buffers usage. Can anyone please help me with code examples of stencil buffers for my issue? Also, is there any way to implement this without stencil buffers?

Because your mask is a texture, stencil buffer is not a good idea.

• during mask rendering, you must use "discard;" for transparent pixels in your fragment shader
• say welcome to antialiasing problems

For your curiosity, here some code to configure a mask with stencil buffer:

const bool invert_mask = false; // allow to draw inside or outside mask
unsigned mask_id = 1; // you can use this code multiple time without clearing stencil, just increment mask_id

glEnable(GL_STENCIL_TEST);
// write on stencil_mask
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
glStencilFunc(GL_ALWAYS, mask_id, 0);

// remove depth test and color writing
glDepthMask(GL_FALSE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);


// TODO: draw geometry of mask here. (if you use a texture, dont forget to use discard in the shader 


// enabled depth & color writing
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glDepthMask(GL_TRUE);

// no stencil write
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
// test stencil value
glStencilFunc(invert_mask ? GL_NOTEQUAL : GL_EQUAL, mask_id, 0xff);


// TODO: draw  "clipped" geometry here


// finally, remove stencil test
glDisable(GL_STENCIL_TEST);

The simplest way is to NOT USE STENCIL AT ALL. Create a grayscale screen-size texture, write your mask inside. Then bind it in your fragment shader:

uniform LOW_P sampler2D u_diffuse_sampler;
uniform LOW_P sampler2D u_mask_sampler;
varying mediump vec2 v_texcoord;

void main(void) {
    gl_FragColor = texture2D(u_diffuse_sampler, v_texcoord) * texture2D(u_mask_sampler, v_texcoord).r;
}