- Joined
- Nov 12, 2002
- Location
- Rootstown, OH
Building off my previous thread giving powershell event log examples, I had mentioned I needed to multithread it but I hadn't gotten around to actually doing it. I took a few minutes this morning, and it was really simple to do that and this is how.
This is the powershell script that pulls the events I want from remote machines, and it accepts a computer name as an argument. I saved it as Get-Event7001Failed.ps1
This is the powershell script that handles the multithreading (credit: http://www.get-blog.com/?p=189). I saved this script as Run-CommandMultiThreaded.ps1:
So to fire off my little event log script, I just do the following which runs the multithreading script, feeds in my event log script as a command, the target list of computers as an objectlist, and tells it to run 50 threads at a time:
.\Run-CommandMultiThreaded.ps1 -Command .\Get-Event7001Failed.ps1 -ObjectList (gc C:\targets.txt) -MaxThreads 50
This completes in a few minutes now running against 1300 machines, instead of taking a couple hours. Output is the same as before, except I get about 900 lines back instead of these few lines I'm posting as an example:
This is the powershell script that pulls the events I want from remote machines, and it accepts a computer name as an argument. I saved it as Get-Event7001Failed.ps1
Code:
$computer = $args[0]
$days = (Get-Date) - (New-TimeSpan -Day 90)
If(Test-Connection -ComputerName $computer -Quiet){
Get-WinEvent -FilterHashtable @{logname='Microsoft-Windows-GroupPolicy/Operational'; id=7001; StartTime=$days} -EA SilentlyContinue -cn $computer | where-object { $_.Message -like '*failed*' } | select @{label='TimeCreated';expression={$_.TimeCreated.ToString("yyyy-M-d HH:mm:ss")}},MachineName,@{n='Message';e={$_.Message -replace '\s+', " "}}}
This is the powershell script that handles the multithreading (credit: http://www.get-blog.com/?p=189). I saved this script as Run-CommandMultiThreaded.ps1:
Code:
#.Synopsis
# This is a quick and open-ended script multi-threader searcher
#
#.Description
# This script will allow any general, external script to be multithreaded by providing a single
# argument to that script and opening it in a seperate thread. It works as a filter in the
# pipeline, or as a standalone script. It will read the argument either from the pipeline
# or from a filename provided. It will send the results of the child script down the pipeline,
# so it is best to use a script that returns some sort of object.
#
# Authored by Ryan Witschger - http://www.Get-Blog.com
#
#.PARAMETER Command
# This is where you provide the PowerShell Cmdlet / Script file that you want to multithread.
# You can also choose a built in cmdlet. Keep in mind that your script. This script is read into
# a scriptblock, so any unforeseen errors are likely caused by the conversion to a script block.
#
#.PARAMETER ObjectList
# The objectlist represents the arguments that are provided to the child script. This is an open ended
# argument and can take a single object from the pipeline, an array, a collection, or a file name. The
# multithreading script does it's best to find out which you have provided and handle it as such.
# If you would like to provide a file, then the file is read with one object on each line and will
# be provided as is to the script you are running as a string. If this is not desired, then use an array.
#
#.PARAMETER InputParam
# This allows you to specify the parameter for which your input objects are to be evaluated. As an example,
# if you were to provide a computer name to the Get-Process cmdlet as just an argument, it would attempt to
# find all processes where the name was the provided computername and fail. You need to specify that the
# parameter that you are providing is the "ComputerName".
#
#.PARAMETER AddParam
# This allows you to specify additional parameters to the running command. For instance, if you are trying
# to find the status of the "BITS" service on all servers in your list, you will need to specify the "Name"
# parameter. This command takes a hash pair formatted as follows:
#
# @{"ParameterName" = "Value"}
# @{"ParameterName" = "Value" ; "ParameterTwo" = "Value2"}
#
#.PARAMETER AddSwitch
# This allows you to add additional switches to the command you are running. For instance, you may want
# to include "RequiredServices" to the "Get-Service" cmdlet. This parameter will take a single string, or
# an aray of strings as follows:
#
# "RequiredServices"
# @("RequiredServices", "DependentServices")
#
#.PARAMETER MaxThreads
# This is the maximum number of threads to run at any given time. If resources are too congested try lowering
# this number. The default value is 20.
#
#.PARAMETER SleepTimer
# This is the time between cycles of the child process detection cycle. The default value is 200ms. If CPU
# utilization is high then you can consider increasing this delay. If the child script takes a long time to
# run, then you might increase this value to around 1000 (or 1 second in the detection cycle).
#
#
#.EXAMPLE
# Both of these will execute the script named ServerInfo.ps1 and provide each of the server names in AllServers.txt
# while providing the results to the screen. The results will be the output of the child script.
#
# gc AllServers.txt | .\Run-CommandMultiThreaded.ps1 -Command .\ServerInfo.ps1
# .\Run-CommandMultiThreaded.ps1 -Command .\ServerInfo.ps1 -ObjectList (gc .\AllServers.txt)
#
#.EXAMPLE
# The following demonstrates the use of the AddParam statement
#
# $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -InputParam ComputerName -AddParam @{"Name" = "BITS"}
#
#.EXAMPLE
# The following demonstrates the use of the AddSwitch statement
#
# $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -AddSwitch @("RequiredServices", "DependentServices")
#
#.EXAMPLE
# The following demonstrates the use of the script in the pipeline
#
# $ObjectList | .\Run-CommandMultiThreaded.ps1 -Command "Get-Service" -InputParam ComputerName -AddParam @{"Name" = "BITS"} | Select Status, MachineName
#
Param($Command = $(Read-Host "Enter the script file"),
[Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]$ObjectList,
$InputParam = $Null,
$MaxThreads = 20,
$SleepTimer = 200,
$MaxResultTime = 120,
[HashTable]$AddParam = @{},
[Array]$AddSwitch = @()
)
Begin{
$ISS = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
$RunspacePool = [runspacefactory]::CreateRunspacePool(1, $MaxThreads, $ISS, $Host)
$RunspacePool.Open()
If ($(Get-Command | Select-Object Name) -match $Command){
$Code = $Null
}Else{
$OFS = "`r`n"
$Code = [ScriptBlock]::Create($(Get-Content $Command))
Remove-Variable OFS
}
$Jobs = @()
}
Process{
Write-Progress -Activity "Preloading threads" -Status "Starting Job $($jobs.count)"
ForEach ($Object in $ObjectList){
If ($Code -eq $Null){
$PowershellThread = [powershell]::Create().AddCommand($Command)
}Else{
$PowershellThread = [powershell]::Create().AddScript($Code)
}
If ($InputParam -ne $Null){
$PowershellThread.AddParameter($InputParam, $Object.ToString()) | out-null
}Else{
$PowershellThread.AddArgument($Object.ToString()) | out-null
}
ForEach($Key in $AddParam.Keys){
$PowershellThread.AddParameter($Key, $AddParam.$key) | out-null
}
ForEach($Switch in $AddSwitch){
$Switch
$PowershellThread.AddParameter($Switch) | out-null
}
$PowershellThread.RunspacePool = $RunspacePool
$Handle = $PowershellThread.BeginInvoke()
$Job = "" | Select-Object Handle, Thread, object
$Job.Handle = $Handle
$Job.Thread = $PowershellThread
$Job.Object = $Object.ToString()
$Jobs += $Job
}
}
End{
$ResultTimer = Get-Date
While (@($Jobs | Where-Object {$_.Handle -ne $Null}).count -gt 0) {
$Remaining = "$($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).object)"
If ($Remaining.Length -gt 60){
$Remaining = $Remaining.Substring(0,60) + "..."
}
Write-Progress `
-Activity "Waiting for Jobs - $($MaxThreads - $($RunspacePool.GetAvailableRunspaces())) of $MaxThreads threads running" `
-PercentComplete (($Jobs.count - $($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False}).count)) / $Jobs.Count * 100) `
-Status "$(@($($Jobs | Where-Object {$_.Handle.IsCompleted -eq $False})).count) remaining - $remaining"
ForEach ($Job in $($Jobs | Where-Object {$_.Handle.IsCompleted -eq $True})){
$Job.Thread.EndInvoke($Job.Handle)
$Job.Thread.Dispose()
$Job.Thread = $Null
$Job.Handle = $Null
$ResultTimer = Get-Date
}
If (($(Get-Date) - $ResultTimer).totalseconds -gt $MaxResultTime){
Write-Error "Child script appears to be frozen, try increasing MaxResultTime"
Exit
}
Start-Sleep -Milliseconds $SleepTimer
}
$RunspacePool.Close() | Out-Null
$RunspacePool.Dispose() | Out-Null
}
So to fire off my little event log script, I just do the following which runs the multithreading script, feeds in my event log script as a command, the target list of computers as an objectlist, and tells it to run 50 threads at a time:
.\Run-CommandMultiThreaded.ps1 -Command .\Get-Event7001Failed.ps1 -ObjectList (gc C:\targets.txt) -MaxThreads 50
This completes in a few minutes now running against 1300 machines, instead of taking a couple hours. Output is the same as before, except I get about 900 lines back instead of these few lines I'm posting as an example:
Code:
2016-1-18 09:53:15 computername User logon policy processing failed for domain\user in 1 seconds.
2016-1-18 09:47:04 computername2 User logon policy processing failed for domain\user2 in 1 seconds.
2016-1-18 09:45:26 computername3 User logon policy processing failed for domain\user3 in 1 seconds.
2016-1-18 09:43:28 computername4 User logon policy processing failed for domain\user4 in 1 seconds.
2016-1-18 09:42:04 computername5 User logon policy processing failed for domain\user5 in 1 seconds.