Using Task Parallel Library (C# .NET 4.0) with an external exe (ffmpeg) without conflicts

I've been trying to solve a problem for several days. I am a beginner in multithreading. My goal is to run several video encoding tasks simultaneously using ffmpeg.exe and to use all the power of the server.

I've got a C# wrapper which launches the ffmpeg.exe process and works without threading (or with ffmpeg internal threading only (not available for flv encoding)) which looks like this:

using (Process process = new Process())
{
    process.StartInfo.FileName = encoderPath + "ffmpeg.exe";

    process.StartInfo.UseShellExecute = false;
    process.StartInfo.RedirectStandardOutput = true;
    process.StartInfo.RedirectStandardError = true;
    process.StartInfo.CreateNoWindow = false;

    string arguments = "-y -i " + filenameInput + " -f " + 
        GetVideoFormatName(format) + " -vcodec " + GetVideoCodecName(codec);

    // (most argument setup has been omitted for brevity)
    arguments += " " + filenameOutput + " ";

    process.StartInfo.Arguments = arguments;
    process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
    process.Start();

    bool succes = LireSortie(process);
    process.WaitForExit();
    process.Close();
    return succes;
}

The code below calls the wrapper. The second parameter of each Encode method is the number of threads to use for internal ffmpeg threading. When I disable it, it doesn't work.

var fm = new FFMpegWrapper();

fm.FilenameInput = "test.mp4";
//VideoInfo videoinfo = fm.GetVideoInfo();
Task[] tasks = {
    Task.Factory.StartNew(
        new Action(()=>{ fm.Encodeto200p("test200p.mp4", 4); })),
    Task.Factory.StartNew(
        new Action(()=>{ fm.EncodetoFlash200p("test200p.flv"); })),
    // ... (calls to other Encode methods ommitted) ...
    Task.Factory.StartNew(
        new Action(()=>{ fm.Encodeto404p("test404p.mp4", 4); })),
    Task.Factory.StartNew(
        new Action(()=>{ fm.EncodetoFlash404p("test404p.flv"); })),
    Task.Factory.StartNew(
        new Action(()=>{ fm.Encodeto720p("test720p.mp4", 4); }))
};

Task.开发者_运维百科WaitAll(tasks, 5000);

You are probably wondering why I put a timeout of 5000 for WaitAll(). It's because the calling thread waits indefinitely because TPL doesn't detect an end to the tasks. ffmpeg.exe processes "stop" in the middle of the encoding and keep running with 0% of CPU.

I think TPL and Process are conflicting. When I get the status of each task with TPL, It remains "Running" all the time. I would like to capture the real events of ffmpeg processes with TPL (or some other mechanism) because, I want to prevent the application from crashing and want to manage successes and failures.

I doubt that the problem is with TPL or with any home-grown mechanism used to detect whether the process ends. I suspect the real reason is that the wait is not indefinite but a classic case of thrashing due to pressure on a particular resource.

Given your description of what is happening and that CPU appears to be at low levels, your pressure is likely on IO or memory. I suggest profiling your application using performance counters at the least (and using task manager as a quick look on the surface level) to find out what is going on.

As a commenter has suggested, it makes sense to eliminate the tasking and determine whether one, two or more serial calls to encode at least work. It may be that one particular call is holding everything up (e.g. an encoding bug), rather than a resource problem.