Msbuild: transforming one itemgroup into another itemgroup with a different structure

I may be asking the wrong question here, and I'm open to that, so I'll give a bit of background of what I'm trying to do. I invoke mstest via an msbuild project, after dynamically findi开发者_如何学JAVAng all test assemblies. I invoke mstest separately for every test assembly, so that the results can be imported into teamcity (my CI server) as soon as they're available, rather than waiting for all of them to be complete before showing any progress in TC.

The problem is that this runs a single test at a time, and combined with the slow overhead (even on an i7 quad, mstest takes 3-5 seconds overhead to open for each project) and many tests, the tests take a few minutes to run.

Using the msbuild task with BuildInParallel=true (and invoking with the /m parameter), it's possible to build several projects at once.

So what I'm trying to do is

• get a list of all *.Tests.dll

• Invoke the ExecMsTest target in the same project, in parallel, for each .dll

  <PropertyGroup>
          <MsTestExePath Condition="'$(MsTestExePath)'==''">C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe</MsTestExePath>
          <MsTestSettingsPath Condition="'$(MsTestSettingsPath)'==''">Project.testsettings</MsTestSettingsPath>
  </PropertyGroup>
  <ItemGroup>
          <TestAssemblies Include="**\bin\**\*.Tests.dll" />
  </ItemGroup>
  
  <Target Name="RunTests">
          <Message Text="Found test assemblies: @(TestAssemblies)" />
  
          <MakeDir Directories="TestResults" />
  
          <MsBuild Projects="@(ProjectsToBuild)" Targets="ExecMsTest" BuildInParallel="True" />
  </Target>
  
  <Target Name="ExecMsTest">
          <Message Text="Running tests in $(TestAssembly)" />
          <!-- show TC progress -->
          <Message Text="##teamcity[progressMessage 'Running tests in $(TestAssembly).dll']" Importance="High" />
  
          <PropertyGroup>
                  <MsTestCommand>"$(MsTestExePath)" /testcontainer:"$(TestAssembly)" /resultsfile:"TestResults\$(TestAssembly).trx" /testsettings:"$(MsTestSettingsPath)"</MsTestCommand>
          </PropertyGroup>
          <!-- Message Text="Exec: $(MsTestCommand)" / -->
          <Exec Command="$(MsTestCommand)" ContinueOnError="true" /> 
  
  <!-- import data to teamcity test results -->
          <Message Text="##teamcity[importData type='mstest' path='TestResults\$(TestAssembly).trx']" />
          <Message Text="Tests complete from $(TestAssembly)" />
  

However, this isn't quite right. You can see my itemgroup is called TestAssemblies, but I'm passing @(ProjectsToBuild) to mstest. This is because the msbuild task requires a differently-formatted item group, something like:

    <ItemGroup>
            <ProjectsToBuild Include="Project.mstest.proj">
                    <Properties>TestAssembly=Project.UI.Tests</Properties>
            </ProjectsToBuild>
            <ProjectsToBuild Include="Project.mstest.proj">
                    <Properties>TestAssembly=Project.Model.Tests</Properties>
            </ProjectsToBuild>
    </ItemGroup>

So this is the crux of my question, assuming I'm even asking the right thing: how do I transform the TestAssemblies itemgroup into something resembling the ProjectsToBuild itemgroup?

In case it's not obvious, the name of the items in TestAssemblies are the *.tests.Dll filenames, while I need that name to be a inside the item, and the name of the ProjectsToBuild item to all be the Project.mstest.proj file (since they're all invoking the same file).

Thanks to @Spider M9, this works:

    <ItemGroup>
            <TestAssemblies Include="**\bin\**\*.Tests.dll" />
    </ItemGroup>

    <Target Name="RunTests">
            <Message Text="Found test assemblies: @(TestAssemblies)" />
            <ItemGroup>
                    <TestAssembliesToBuild Include="Project.mstest.proj">
                            <Properties>TestAssembly=%(TestAssemblies.FileName);FullPath=%(TestAssemblies.FullPath)</Properties>
                    </TestAssembliesToBuild>
            </ItemGroup>

            <MakeDir Directories="TestResults" />

            <MsBuild Projects="@(TestAssembliesToBuild)" Targets="ExecMsTest" BuildInParallel="True" />
    </Target>

Running msbuild single-threaded, my entire build (which includes compiling, building application and database snapshots, deploying schema for a couple databases that get used in some of the unit tests, and then finally running mstest) took about 9m30s. After this change, it was taking ~7m.

However, prior to getting an answer to this question, I just tried running a single mstest instance, to see how much it would improve, and that takes about 4m50s (of which mstest takes slightly over 1 minute to run). The downside is I have to wait until all tests are complete before getting results, but considering the staggering improvement from 6m to 1m that's a perfectly acceptable trade-off.

To be clear, the only difference is that mstest is starting once, vs starting a dozen times, and presumably there is also some benefit from multitasking. I'm running this on a Core i7-860 (4 physical cores, 8 logical cores) and I suspect number of cores will highly influence the level of improvement this change makes.

Here is my new RunTests:

    <Target Name="RunTests">
            <Message Text="Found test assemblies: @(TestAssemblies)" />

            <MakeDir Directories="TestResults" />

            <!-- this executes mstest once, and runs all assemblies at the same time. Faster, but no output to TC until they're all completed -->
            <PropertyGroup>
                    <MsTestCommand>"$(MsTestExePath)" @(TestAssemblies->'/testcontainer:"%(FullPath)"', ' ') /resultsfile:"TestResults\Results.trx" /testsettings:"$(MsTestSettingsPath)"</MsTestCommand>
            </PropertyGroup>

            <Message Text="##teamcity[progressMessage 'Running tests']" Importance="High" />
            <Message Text="Exec: $(MsTestCommand)" />
            <Exec Command="$(MsTestCommand)" ContinueOnError="true" />
            <Message Text="##teamcity[importData type='mstest' path='TestResults\Results.trx']" />
    </Target>

also, you need a testsettings file with: <Execution parallelTestCount="0"> (0 means autodetect, default is 1) and need to invoke msbuild using the /m parameter and/or <Msbuild BulidInParallel="true">

Try this:

<ItemGroup>
   <TestAssemblies Include="**\bin\**\*.Tests.dll" />
   <TestAssembliesToBuild Include="Project.mstest.proj">
     <Properties>TestAssembly=%(TestAssemblies.FileName)</Properties>
   </TestAssembliesToBuild>
</ItemGroup>
<Message Text="'%(TestAssembliesToBuild.Properties)'" />