Powershell Receive-Job: Quota violation when invoking command as a job

I am trying to execute a cmdlet (test-connection) against multiple machines in parallel using the background jobs mechanism. The function below is fine until I pass more than 649 hostnames (at least on this machine) before an exception is thrown:

PS > .\test.ps1 .\650_hostnames.txt

Receive-Job : Quota violation
At H:\Development\Powershell\test.ps1:13 char:13
+ Receive-Job <<<< |
+ CategoryInfo " InvalidResult: (:) [Receive-Job], ManagementException
+ FullyQualifiedErrorId : JobStateFailed,Microsoft.PowerShell.Commands.ReceiveJobCommand

I would like to still execute the cmdlet as a background job for performance reasons, but control the number of jobs kicked off at a given time. I cannot control the input so I could get a list of many thousands to a few hundred. Any suggestions are appreciated!

test.ps1

$wkslist = $args[0]

function IsAlive {
  param(
      [Parameter(
      Mandatory = $true,
      Position = 0)]
      $computername
  )
  test-connection -count 1 -ComputerName $computername -TimeToLive 30 `
  -ThrottleLimit 10 -asJob |
  Wait-Job |
开发者_Go百科  Receive-Job |
  Where-Object { $_.StatusCode -eq 0 } |
  Select-Object -ExpandProperty Address
}

$ips = [System.IO.File]::ReadAllLines("$wkslist")
IsAlive -computername $ips

Looks like this has nothing to do with the fact that you're using jobs; it seems to be with the size of the input being passed to test-connection. I can reproduce the quota violation with this code:

$ra = @(0..1000) | %{ $env:computername };
test-connection -computername $ra;

Seems to be something internal to test-connection, which ends up creating Win32_Ping WMI objects methinks. I can also get the code to work by slicing out part of the input array:

$ra = @(0..1000) | %{ $env:computername };
test-connection -computername $ra[0..500];

Note that 650 does not appear to be a magic number - I can query up to about 800 hostnames without encountering the error.

Since the exception is a Management exception and not a PowerShell internal exception, I'd say you've found one of the many magical limits of WMI. Also sounds like something that test-connection should manage internally, but doesn't. I've logged an issue on Connect here, if you want to upvote it: test-connection throws quota exception when passed large list of computernames

I would try your job code as-is, but slice the list of hostnames into smaller segments:

$i = 100;
$j = 0;
while( $i -le $ra.length )
{ 
   test-connection -computername $ra[$j..$i] ... ;
   $j = $i + 1;
   $i += 100;
}

I'd look at restructuring the code slightly to look like this:

Put the following code into your root profile.ps1 c:\windows\system32\windowspowershell\v1.0\profile.ps1. If it doesn't exist, just create a text file and rename it. This profile.ps1 file will get loaded in by default when you start powershell.

Function Test-Connection()
{
param(
[Parameter(Mandatory = $true,Position = 0)]$Count,
[Parameter(Mandatory= $true,Position = 1)]$IP,
[Parameter(Mandatory= $true,Position = 2)]$TTL,
[Parameter(Mandatory= $true,Position = 3)]$ThrottleLimit
)
MyCommand $Count $IP $TTL $ThrottleLimit

}

Then use the following code to run the jobs, copying in my threshold routine from my blog http://www.poshpete.com/archives/27:

$ips = Get-Content $wkslist
ForEach($IP in $ips)
{

    Threshold -MaxConcurrent 10 -PauseTime 5
    Start-Job -ScriptBlock {test-connection} -ArgumentList $Count,$IP,$TTL,$ThrottleLimit -Name $IP
    $CompletedJobs = Get-Job -State Completed
    If(($CompletedJobs | Measure-Object).Count -gt 0)
    {
        ForEach($Job in $CompletedJobs)
        {
            $Job | Receive-Job | Where-Object { $_.StatusCode -eq 0 } | Select-Object -ExpandProperty Address

        }

    }
}

I'm using get-content to get the data from the text file, but you can use the .net object too.

Hopefully that makes sense!

After struggling with an attempt to raise the quota limit I realized that I needed to control the amount of input to the function anyway. Someone or something is going to pass more hosts than the quota allows, even if you raised it, or exceed the resources of your machine. The quotas are used to protect the local computer and/or the remote computer from excessive resource use, both accidental and malicious. This is a good thing (though I am still interested in finding out how to control the number).

Per the about_Remote_Troubleshooting help to resolve the error:

1. Change the remote command to comply with the quota. (Thanks Capt. Obvious)
2. Or, determine the source of the quota, and then increase the quota to allow the command to complete.

As for identifying the source of the quota it is still a mystery to me. I thought I was getting close when I found the configuration class for host provider processes _ProviderHostQuotaConfiguration. This class allows limits to be set on host process usage of system resources. The different properties available are:

• ThreadsPerHost: Defines the number of threads owned by any one host
• HandlesPerHost: Defines the number of kernel object handles each host may have
• ProcessLimitAllHosts: Defines the total number of host processes that can be executing concurrently
• MemoryPerHost: Defines the amount of private memory that can be held by each host
• MemoryAllHosts: Defines the combined amount of private memory (in bytes) that can be held by all hosts

However after increasing the values I still received the same violation. If someone knows what specifically to change to raise the number of hosts that can be used against Test-Connection I'm sure the group would appreciate it.