Chat with us, powered by LiveChat

Blog

Back

Limiting Multi-threading in Powershell

18 Jul 2012 by Dan Rose

I was recently looking for a way of running multiple instances of a single script within Powershell, but at the same time limiting the amount of threads as I didn’t really want to overload the server. Now there are many examples on the Internet of multithreading and multitasking using Powershell as well as many ways of achieving what you want, but, I typically had specific requirements which many of the examples I found didn’t quite meet.
So having taken in all that plethora of information I’ve decided to share this script with you. The specific requirement I had was that I wanted the script to be initiated through scheduled tasks each hour, I also wanted to limit the number of running tasks and finally I wanted the parent script to continue running until all the child scripts had finished.
The first thing I wanted was to start with an input, the maximum number of jobs that can be run concurrently. $JobLimit specifies the maximum job limit. Some of the VMs I was planning to run this on were pretty thin, so this parameter was an essential part to this multithreading script.
Param(
[Parameter(
Mandatory = $true,
ValueFromPipeline = $true)]
[int]$JobLimit
)

I then wanted to include a wait state between jobs. Obviously for some people this may not be necessary, but for my needs it was, so it was included
Function Sleep-Script
{
$global:i = 0
# sleep time between job instances
start-sleep -s 30
foreach ($job in (get-job -state "running")) {$global:i++}
write-host "$global:i running"
}

Next, was the bit to initiate the child script. This is where you specify the script that actually does the work you want. Using the Start-Job cmdlet this allows you to spawn multiple jobs. Once the new job runs, it will call the Sleep-Script function before kicking off a new instance. For further info on Start-Job and other related cmdlets try these guys, they really do know their stuff.
http://blogs.technet.com/b/heyscriptingguy/archive/2010/03/15/hey-scripting-guy-march-15-2010.aspx
 
Function Initiate-Scan
{
# child-script to be run
$ScriptBlock = {
C:\Powershell\myscript.ps1}
# initiate scriptblock
Start-Job $ScriptBlock | Out-Null
# initiate sleep state
Sleep-Script
}

and, of course the part that pins it all together. This is the logic that allows me to run the child script within the threshold I specify.
$global:i = 1
While (($global:i -le $JobLimit) -and ($global:i -ge 1)) {
Initiate-Scan
While ($global:i -ge $JobLimit) {
Sleep-Script
}
}
write-host "Completed"

The child-script that was initiated was one that would take variable length on time to run. So the idea here was that this parent script would run continuously until all jobs were complete, but essentially the number of threads is never exceeded.
 
Windows PowerShell
Copyright (C) 2009 Microsoft Corporation. All rights reserved.
PS C:\Powershell\> .\parent.ps1 -JobLimit 3
1 running
2 running
2 running
3 running
2 running
3 running
3 running
3 running
3 running
1 running
2 running
3 running
1 running
0 running
Completed
PS C:\Powershell\>