Exporting Hash Table Using Property Dictionary to CSV - regex

Question

I can't seem to figure out how to simply export formatted information to a CSV unless I iterate through each item in the object and write to the CSV line by line, which takes forever. I can export values instantly to the CSV, it's just when using the properties dictionary I run into issues.
The TestCSV file is formatted with a column that has IP addresses.
Here's what I have:
$CSV = "C:\TEMP\OutputFile.csv"
$RX = "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(?:\.|dot|\[dot\]|\[\.\])){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"
$TestCSV = "C:\TEMP\FileWithIPs.csv"
$spreadsheetDataobject = import-csv $TestCSV
$Finding = $spreadsheetDataObject | Select-String $RX
$Props = #{ #create a properties dictionary
LineNumber = $finding.LineNumber
Matches = $finding.Matches.Value
}
$OBJ = New-Object -TypeName psobject -Property $Props
$OBJ | Select-Object Matches,LineNumber | Export-Csv -Path $CSV -Append -NoTypeInformation

Answer 1

This isn't going to work as written. You are using Import-CSV which creates an array of objects with properties. The Select-String command expects strings as input, not objects. If you want to use Select-String you would want to simply specify the file name, or use Get-Content on the file, and pass that to Select-String. If what you want is the line number, and the IP I think this would probably work just as well if not better for you:
$CSV = "C:\TEMP\OutputFile.csv"
$RX = "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(?:\.|dot|\[dot\]|\[\.\])){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"
$TestCSV = "C:\TEMP\FileWithIPs.csv"
$spreadsheetDataobject = import-csv $TestCSV
$spreadsheetDataobject |
Where{$_.IP -match $RX} |
Select-Object #{l='Matches';e={$_.IP}},#{l='LineNumber';e={[array]::IndexOf($spreadsheetDataobject,$_)+1}} |
Export-Csv -Path $CSV -Append -NoTypeInformation
Edit: wOxxOm is quite right, this answer has considerably more overhead than parsing the text directly like he does. Though, for somebody who's new to PowerShell it's probably easier to understand.
In regards to $_.IP, since you use Import-CSV you create an array of objects. Each object has properties associated with it based on the header of the CSV file. IP was listed in the header as one of your columns, so each object has a property of IP, and the value of that property is whatever was in the IP column for that record.
Let me explain the Select line for you, and then you'll see that it's easy to add your source path as another column.
What I'm doing is defining properties with a hashtable. For my examples I will refer to the first one shown above. Since it is a hashtable it starts with #{ and ends with }. Inside there are two key/value pairs:
l='Matches'
e={$_.IP}
Essentially 'l' is short for Label, and 'e' is short for Expression. The label determines the name of the property being defined (which equates to the column header when you export). The expression defines the value assigned to the property. In this case I am really just renaming the IP column to Matches, since the value that I assign for each row is whatever is in the IP field. If you open the CSV in Excel, copy the entire IP column, paste it in at the end, and change the header to Matches, that is basically all I'm doing. So to add the file path as a column we can add one more hashtable to the Select line with this:
#{
l='FilePath'
e={$CSV}
}
That adds a third property, where the name is FilePath, and the value is whatever is stored in $CSV. That updated Select line would look like this:
Select-Object #{l='Matches';e={$_.IP}},#{l='LineNumber';e={[array]::IndexOf($spreadsheetDataobject,$_)+1}},#{l='FilePath'e={$CSV}} |

Answer 2

Any code based on the built-in CSV cmdlets is extremely slow because objects are created for each field on each line, and it's noticeable on large files (for example, code from the other answer takes 900 seconds to process a 9MB file with 100k lines).
If your input CSV file is simple, you can process it as text in less than a second for a 100k lines file:
$CSV = .......
$RX = .......
$TestCSV = .......
$line = 0 # header line doesn't count
$lastMatchPos = 0
$text = [IO.File]::ReadAllText($TestCSV) -replace '"http.+?",', ','
$out = New-Object Text.StringBuilder
ForEach ($m in ([regex]"(?<=,""?)$RX(?=""?,)").Matches($text)) {
$line += $m.index - $lastMatchPos -
$text.substring($lastMatchPos, $m.index-$lastMatchPos).Replace("`n",'').length
$lastMatchPos = $m.Index + $m.length
$out.AppendLine('' + $line + ',' + $m.value) >$null
}
if (!(Test-Path $CSV)) {
'LineNumber,IP' | Out-File $CSV -Encoding ascii
}
$out.ToString() | Out-File $CSV -Encoding ascii -Append
The code zaps quoted URLs fields just in the unlikely but possible case those contain a matching IP.