AVCapture appendSampleBuffer
I am going insane with this one - have looked everywhere and tried anything and everything I can thinks of.
Am making an iPhone app that uses AVFoundation - specifically AVCapture to capture video using the iPhone camera.
I need to have a custom image that is overlayed on the video feed included in the recording.
So far I have the AVCapture session set up, can display the feed, access the frame, save it as a UIImage and marge the overlay Image onto it. Then convert this new UIImage into a CVPixelBufferRef. annnd to double check that the bufferRef is working I converted it back to a UIImage and it displays the image fine still.
The trouble starts when I try to convert the CVPixelBufferRef into a CMSampleBufferRef to append to the AVCaptureSessions assetWriterInput. The CMSampleBufferRef always returning NULL when I attempt to create it.
Here is the -(void)captureOutput function
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection
{
UIImage *botImage = [self imageFromSampleBuffer:sampleBuffer];
UIImage *wheel = [self imageFromView:wheelView];
UIImage *finalImage = [self overlaidImage:botImage :wheel];
//[previewImage setImage:finalImage]; <- works -- the image is being merged into one UIImage
CVPixelBufferRef pixelBuffer = NULL;
CGImageRef cgImage = CGImageCreateCopy(finalImage.CGImage);
CFDataRef image = CGDataProviderCopyData(CGImageGetDataProvider(cgImage));
int status = CVPixelBufferCreateWithBytes(NULL,
self.view.bounds.size.width,
self.view.bounds.size.height,
kCVPixelFormatType_32BGRA,
(void*)CFDataGetBytePtr(image),
CGIm开发者_运维百科ageGetBytesPerRow(cgImage),
NULL,
0,
NULL,
&pixelBuffer);
if(status == 0){
OSStatus result = 0;
CMVideoFormatDescriptionRef videoInfo = NULL;
result = CMVideoFormatDescriptionCreateForImageBuffer(NULL, pixelBuffer, &videoInfo);
NSParameterAssert(result == 0 && videoInfo != NULL);
CMSampleBufferRef myBuffer = NULL;
result = CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault,
pixelBuffer, true, NULL, NULL, videoInfo, NULL, &myBuffer);
NSParameterAssert(result == 0 && myBuffer != NULL);//always null :S
NSLog(@"Trying to append");
if (!CMSampleBufferDataIsReady(myBuffer)){
NSLog(@"sampleBuffer data is not ready");
return;
}
if (![assetWriterInput isReadyForMoreMediaData]){
NSLog(@"Not ready for data :(");
return;
}
if (![assetWriterInput appendSampleBuffer:myBuffer]){
NSLog(@"Failed to append pixel buffer");
}
}
}
Another solution I keep hearing about is using a AVAssetWriterInputPixelBufferAdaptor which eliminates the need to do the messy CMSampleBufferRef wrapping. However I have scoured stacked and apple developer forums and docs and can't find a clear description or example on how to set this up or how to use it. If anyone has a working example of it could you please show me or help me nut out the above issue - have been working on this non-stop for a week and am at wits end.
Let me know if you need any other info
Thanks in advance,
Michael
You need AVAssetWriterInputPixelBufferAdaptor, here is the code to create it :
// Create dictionary for pixel buffer adaptor
NSDictionary *bufferAttributes = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA], kCVPixelBufferPixelFormatTypeKey, nil];
// Create pixel buffer adaptor
m_pixelsBufferAdaptor = [[AVAssetWriterInputPixelBufferAdaptor alloc] initWithAssetWriterInput:assetWriterInput sourcePixelBufferAttributes:bufferAttributes];
And the code to use it :
// If ready to have more media data
if (m_pixelsBufferAdaptor.assetWriterInput.readyForMoreMediaData) {
// Create a pixel buffer
CVPixelBufferRef pixelsBuffer = NULL;
CVPixelBufferPoolCreatePixelBuffer(NULL, m_pixelsBufferAdaptor.pixelBufferPool, &pixelsBuffer);
// Lock pixel buffer address
CVPixelBufferLockBaseAddress(pixelsBuffer, 0);
// Create your function to set your pixels data in the buffer (in your case, fill with your finalImage data)
[self yourFunctionToPutDataInPixelBuffer:CVPixelBufferGetBaseAddress(pixelsBuffer)];
// Unlock pixel buffer address
CVPixelBufferUnlockBaseAddress(pixelsBuffer, 0);
// Append pixel buffer (calculate currentFrameTime with your needing, the most simplest way is to have a frame time starting at 0 and increment each time you write a frame with the time of a frame (inverse of your framerate))
[m_pixelsBufferAdaptor appendPixelBuffer:pixelsBuffer withPresentationTime:currentFrameTime];
// Release pixel buffer
CVPixelBufferRelease(pixelsBuffer);
}
And don't forget to release your pixelsBufferAdaptor.
I do it by using CMSampleBufferCreateForImageBuffer() .
OSStatus ret = 0;
CMSampleBufferRef sample = NULL;
CMVideoFormatDescriptionRef videoInfo = NULL;
CMSampleTimingInfo timingInfo = kCMTimingInfoInvalid;
timingInfo.presentationTimeStamp = pts;
timingInfo.duration = duration;
ret = CMVideoFormatDescriptionCreateForImageBuffer(NULL, pixel, &videoInfo);
if (ret != 0) {
NSLog(@"CMVideoFormatDescriptionCreateForImageBuffer failed! %d", (int)ret);
goto done;
}
ret = CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault, pixel, true, NULL, NULL,
videoInfo, &timingInfo, &sample);
if (ret != 0) {
NSLog(@"CMSampleBufferCreateForImageBuffer failed! %d", (int)ret);
goto done;
}
精彩评论