IOS4: AVFoundation: how to get raw footage from a movie

How can i access the raw footage from movies taken with my camera, so i can edit or transform the raw footage ( eg: make it black/white ).

I know that you can load a mov with AVAsset make a composition with different AVAsset开发者_Python百科's and then export it to a new movie, but how do i access so i can edit the movie.

You need to read video frames from an input asset, create a CGContextRef for each frame to do your drawing, then write the frames out to a new video file. The basic steps are below. I've left out all the filler code and error handling, so the main steps are easier to read.

// AVURLAsset to read input movie (i.e. mov recorded to local storage)
NSDictionary *inputOptions = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] forKey:AVURLAssetPreferPreciseDurationAndTimingKey];
AVURLAsset *inputAsset = [[AVURLAsset alloc] initWithURL:inputURL options:inputOptions];

// Load the input asset tracks information
[inputAsset loadValuesAsynchronouslyForKeys:[NSArray arrayWithObject:@"tracks"] completionHandler: ^{

    // Check status of "tracks", make sure they were loaded    
    AVKeyValueStatus tracksStatus = [inputAsset statusOfValueForKey:@"tracks" error:&error];
    if (!tracksStatus == AVKeyValueStatusLoaded)
        // failed to load
        return;

    // Fetch length of input video; might be handy
    NSTimeInterval videoDuration = CMTimeGetSeconds([inputAsset duration]);
    // Fetch dimensions of input video
    CGSize videoSize = [inputAsset naturalSize];


    /* Prepare output asset writer */
    self.assetWriter = [[[AVAssetWriter alloc] initWithURL:outputURL fileType:AVFileTypeQuickTimeMovie error:&error] autorelease];
    NSParameterAssert(assetWriter);
    assetWriter.shouldOptimizeForNetworkUse = NO;


    // Video output
    NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                       AVVideoCodecH264, AVVideoCodecKey,
                       [NSNumber numberWithInt:videoSize.width], AVVideoWidthKey,
                       [NSNumber numberWithInt:videoSize.height], AVVideoHeightKey,
                       nil];
    self.assetWriterVideoInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo
                            outputSettings:videoSettings];
    NSParameterAssert(assetWriterVideoInput);
    NSParameterAssert([assetWriter canAddInput:assetWriterVideoInput]);
    [assetWriter addInput:assetWriterVideoInput];


    // Start writing
    CMTime presentationTime = kCMTimeZero;

    [assetWriter startWriting];
    [assetWriter startSessionAtSourceTime:presentationTime];


    /* Read video samples from input asset video track */
    self.reader = [AVAssetReader assetReaderWithAsset:inputAsset error:&error];

    NSMutableDictionary *outputSettings = [NSMutableDictionary dictionary];
    [outputSettings setObject: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA]  forKey: (NSString*)kCVPixelBufferPixelFormatTypeKey];
    self.readerVideoTrackOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:[[inputAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0]
                        outputSettings:outputSettings];


    // Assign the tracks to the reader and start to read
    [reader addOutput:readerVideoTrackOutput];
    if ([reader startReading] == NO) {
        // Handle error
    }


    dispatch_queue_t dispatch_queue = dispatch_get_main_queue();

    [assetWriterVideoInput requestMediaDataWhenReadyOnQueue:dispatch_queue usingBlock:^{
        CMTime presentationTime = kCMTimeZero;

        while ([assetWriterVideoInput isReadyForMoreMediaData]) {
            CMSampleBufferRef sample = [readerVideoTrackOutput copyNextSampleBuffer];
            if (sample) {
                presentationTime = CMSampleBufferGetPresentationTimeStamp(sample);

                /* Composite over video frame */

                CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sample); 

                // Lock the image buffer
                CVPixelBufferLockBaseAddress(imageBuffer,0); 

                // Get information about the image
                uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer); 
                size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
                size_t width = CVPixelBufferGetWidth(imageBuffer); 
                size_t height = CVPixelBufferGetHeight(imageBuffer); 

                // Create a CGImageRef from the CVImageBufferRef
                CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 
                CGContextRef newContext = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);

                /*** Draw into context ref to draw over video frame ***/

                // We unlock the  image buffer
                CVPixelBufferUnlockBaseAddress(imageBuffer,0);

                // We release some components
                CGContextRelease(newContext); 
                CGColorSpaceRelease(colorSpace);

                /* End composite */

                [assetWriterVideoInput appendSampleBuffer:sample];
                CFRelease(sample);

            }
            else {
                [assetWriterVideoInput markAsFinished];

                /* Close output */

                [assetWriter endSessionAtSourceTime:presentationTime];
                if (![assetWriter finishWriting]) {
                    NSLog(@"[assetWriter finishWriting] failed, status=%@ error=%@", assetWriter.status, assetWriter.error);
                }

            }

        }
    }];

}];

I don't know the whole process, but I know some:

You might need to use AV Foundation Framework and perhaps Core Video Framework to process individual frames. You will probably use AVWriter:

AVAssetWriter *videoWriter = [[AVAssetWriter alloc] initWithURL:
                              [NSURL fileURLWithPath:path]
                                            fileType:AVFileTypeQuickTimeMovie
                                               error:&error];

You can maintain a pixel buffer using AVFoundation or CV, and then write it like (this example is for CV):

[pixelBufferAdaptor appendPixelBuffer:buffer withPresentationTime:kCMTimeZero];

To get frames, AVAssetStillImageGenerator isn't really sufficient.

Alternatively, There may be a filter or instruction that can be used with AVVideoMutableComposition, AVMutableComposition, or AVAssetExportSession.

If you have made progress since you asked in August, please post as I'm interested!