How to spawn several processes from the Windows shell and wait for them all to complete?

I want to do the following from a Windows batch script:

start proc1.exe

start proc2.exe

...

start procN.exe

<wait for all开发者_Python百科 N processes to complete> <-- What do I put here?

How do I wait for all spawned processes to complete?

This is ugly, but you could wrap each command in another batch file that runs the command and then signals that the command is complete. Your main batch file would call each of those batch files asynchronously and sit in a loop waiting for the signals.

For example:

main.bat

start cmd /c proc1.bat
start cmd /c proc2.bat
:wait
sleep 1
IF NOT EXIST proc1done GOTO wait
IF NOT EXIST proc2done GOTO wait
del proc1done
del proc2done
echo All processes are complete

proc1.bat

proc1.exe
echo Done > proc1done

The sleep command is available in Windows Server 2003 Resource Kit Tools. If you don't have that, you could use a ping on localhost just to slow down that tight loop.

I found a good solution in using an alternative shell called "Yori". You can read more about it and download it from http://www.malsmith.net/yori/. Yori is open source and its code repository is at https://github.com/malxau/yori.

Yori makes it easy to implement parallel process execution and wait for them to finish. In essence, what you need to do is to start the different sub-processes as so-called "jobs" and then use the internal command "wait" to wait for them to finish. You can start any process as a job by adding the special characters "&!" at the end of the command. Something like this:

ping localhost &!
ping someotherhost &!
wait

You can put your Yori-dependent commands in a separate batch-file (with the Yori-batch extension ".ys1") and invoke that from your Windows-batch for example with the following command:

echo Let Yori do the parallel execution of some lengthy operations...
start /wait /b /belownormal yori.exe -c "your-yori-script.ys1"
echo All the lengthy operations are finished, continue with the next steps...

You could do this if you write .NET code or Win32 C/C++ code to start the processes, but there is no way to do it in a batch file. Once you use start to make proc1.exe run asynchronously, there is no batch command that allows you to come back later and wait for it to complete.

But you can do this easily in one of the scripting languages designed for batch work, Python, WSH, etc.

For example, here's a simple script using Windows Script Host. WSH is included in all versions of Windows since Windows 98. So this script should run anywhere.

This script starts two programs and waits for them both to finish. Save it as as start2.wsf. Then just use it in your batch file:

start2.wsf "prog1.exe" "prog2.exe"


<package>
  <job id="Start 2 Procs">
    <runtime>
      <description>Start 2 programs and wait for them both to finish.
      </description>
      <unnamed
        name="program"
        helpstring="the program to run"
        many="false"
        required="2"
      />
      <example>
        Example: Start2.wsf "cl -c foo.obj" "cl -c bar.obj"
      </example>
    </runtime>

    <script language="JScript">

      if (WScript.Arguments.length < 2)
      {
        WScript.Arguments.ShowUsage();
        WScript.Quit();
      }

      var proc1 = WScript.Arguments.Unnamed.Item(0);
      var proc2 = WScript.Arguments.Unnamed.Item(1);

      var Shell = WScript.CreateObject("WScript.Shell");
      var oProc1 = Shell.Exec(proc1);
      var oProc2 = Shell.Exec(proc2);

      while (oProc1.Status == 0 && oProc2.Status == 0)
      {
        WScript.Sleep(1000);
      }
    </script>
  </job>
</package>

Typically you would do something like this using Python, but here it goes (inspired by this post and others):

calcFromList.bat < DesiredBatchFile > < TxtFile > [--async [SleepTimeBetweenSpawns]]

It will spawn one process for each line in TxtFile (the line is used to specify the individual parameters for the batch file).

calcFromList.bat:

echo off

pushd \
%~d0
cd "%~dp0"

set CALL_BAT="%~1"
set FILE_CONTAINING_LIST="%~2"

if %CALL_BAT%=="" (goto badval)
if %FILE_CONTAINING_LIST%=="" (goto badval2)

if "%3"=="--async" goto async

for /f "delims=¬" %%X in ('type %FILE_CONTAINING_LIST%') do (call %CALL_BAT% %%X)
goto end

:async
set /a AsyncCountNum=1

set SLEEP_VALUE_S=0

if not "%4"=="" (set SLEEP_VALUE_S=%4)
set AsyncParamBat=%CALL_BAT%

:: Find unique id even if called simultaneously (using simple collision detection)
:newid
timeout /t 1 /nobreak > NUL
set AsyncRand=%RANDOM:~-1%%RANDOM:~-2%%RANDOM:~-3%
set AsyncCountFilePrefix="%FILE_CONTAINING_LIST:"=%__%AsyncRand%"
set AsyncCollisionText="%~1;%~2;%~3;%~4;"
IF EXIST "%AsyncCountFilePrefix:"=%0" GOTO newid
echo %AsyncCollisionText% > "%AsyncCountFilePrefix:"=%_init_async_done.txt"
set /p AsyncCheckColision=<"%AsyncCountFilePrefix:"=%_init_async_done.txt"

if "%AsyncCollisionText:"=% "=="%AsyncCheckColision:"=%" GOTO idfound
timeout /t 1 /nobreak > NUL
GOTO newid
:idfound

echo Beginning spawning of processes (id = %AsyncRand%)...

for /f "delims=¬" %%X in ('type %FILE_CONTAINING_LIST%') do (set AsyncParamCall="%%~X" & start "Batch (from id %AsyncRand%)" /D "%~dp0" "cmd /c asyncExec.bat" & set /a AsyncCountNum+=1 & timeout /t %SLEEP_VALUE_S% /nobreak > NUL)

set /a AsyncCountNum-=1
echo All %AsyncCountNum% processes spawned, waiting for their completion...
:wait
timeout /t 2 /nobreak > NUL
for /l %%x in (1, 1, %AsyncCountNum%) do (IF NOT EXIST "%AsyncCountFilePrefix:"=%_%%x_tmp_async_done.txt" GOTO wait )

echo Finished, cleaning up...
for /l %%x in (1, 1, %AsyncCountNum%) do (del /F /Q  "%AsyncCountFilePrefix:"=%_%%x_tmp_async_done.txt")
del /F /Q "%AsyncCountFilePrefix:"=%_init_async_done.txt"
goto end

start "Calib for %~TARGET_FOLDER%" /D "%~dp0" "cmd /k calibCamsSequence.bat"

goto end
:badval
echo No bat file specified! (first parameter)

goto end
:badval2
echo No list file specified! (second parameter)

:end
popd

asyncExec.bat:

@call %AsyncParamBat% %AsyncParamCall%
@echo "Done at %DATE% %TIME%" > "%AsyncCountFilePrefix:"=%_%AsyncCountNum%_tmp_async_done.txt"