Regex from a line where I want specific things to be extracted. Example: In the line I have: This is a test 2016/01/23 This is test 04:05 AM this is a test Success Login at 01.33 sec. How can I extract only the date, time & success time. So output would be
(2016/01/23 04:05 AM Success 01.33 sec).
Appreciate your help. Cheers!
I suspect I'm going to answer, and then you're going to comment "my real data is completely different how do I match that?". Let's see.
# Here is your log line
$logLine = 'This is a test 2016/01/23 This is test 04:05 AM this is a test Success Login at 01.33 sec'
# Here is my regex which matches
# * date * time AMPM * Success Login time *
$myRegex = '.*(\d{4}/\d\d/\d\d).*(\d\d:\d\d \S\S).*Success Login at ([0-9.]+ \S+).*'
# Here is the test and output building
if ($logLine -match $myRegex) {
Write-Host "($($matches[1]) $($matches[2]) Success $($matches[3]))"
} else {
Write-Host "Log doesn't match"
}
Related
I have a pipeline (classic view) with the task "Visual Studio Test", with task version "2.*".
After the task completes I can see that it prints in the log the test results.
How can I save 'Total Tests' and 'Passed Tests' in variables to use with further tasks of the pipeline?
I tried extracting the .trx file but it gets deleted after the task completes.
Performing VsTest gives me this (Some tests fail, but it's OK):
Adding trx file C:\vsts-agent-win-x64-2.165.2\_work\6\s\TestResults\TestResults\----.trx to run attachments
**************** Completed test execution *********************
Result Attachments will be stored in LogStore
Publishing test results to test run '3748'.
TestResults To Publish 189, Test run id:3748
Test results publishing 189, remaining: 0. Test run id: ---
Published test case results: 189
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Received the command : Stop
TestExecutionHost.ProcessCommand. Stop Command handled
SliceFetch Aborted. Moving to the TestHostEnd phase
Please use this link to analyze the test run : https://---
Test run '---' is in 'Completed' state with 'Total Tests' : 202 and 'Passed Tests' : 19.
##[error]System.Exception: Some tests in the test run did not pass, failing the task.
##########################################################################
Finishing: VsTest - testPlan
When I try to cd into the TestResults:
+ cd C:\vsts-agent-win-x64-2.165.2\_work\6\s\TestResults\TestResults
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\vsts-agent-w...lts\TestResults:String) [Set-Location], ItemNotFoundE
xception
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.SetLocationCommand
##[error]PowerShell exited with code '1'.
You can change the default test result output folder by setting the Test results folder field. See below:
Folder to store test results. When this input is not specified, results are stored in $(Agent.TempDirectory)/TestResults by default, which is cleaned at the end of a pipeline run
In above example. The test result .trx file will be stored at $(System.DefaultWorkingDirectory)\TestResults folder which will not be cleaned up.
Then you can extracting the .trx file in the following tasks and save 'Total Tests' and 'Passed Tests' in variables.
See below screenshot from my test pipeline:
Vstest task log:
Powershell task to ls the contents:
So it seems VsTest deletes all its results after the task is complete.
I solved this with a REST API command.
Make sure you convert your Personal Access Token to Base64...
Here's how I did it:
$personalToken = [your token]
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($personalToken)"))
$header = #{authorization = "Basic $token"}
$params = #{
Uri = 'https://dev.azure.com/[organization]/[project]/_apis/test/runs?buildIds=[BuildId]&api-version=6.0'
Headers = $header
Method = 'GET'
}
$output = Invoke-RestMethod #params
$run = $output.value | Where-Object{$_.name -match [BuildId]}
Write-Host "Total Tests: $($run.totalTests)"
Write-Host "Passed Tests: $($run.passedTests)"
Write-Host "Failed Tests: $($run.unanalyzedTests)"
Write-Host "Skipped Tests: $($run.incompleteTests)"
I have an Azure DevOps pipeline running in YAML.
I'm using the VSTest#2 task to execute some unit tests. This is all working fine and I see the test results appear in the Stage overview UI itself, and in the 'Tests and Coverage' overview in the header.
My YAML pipeline also posts a message to a Slack channel with links to the build, success/failure status and other things. I'm keen to add Test results into the message too...just a simple 'Total tests X - Passed X - Failed X - Skipped X' display. This happens in a separate Stage at the end.
Is there a way to get the tests results from a previous stage in a later stage in the pipline (running on a different agent)?
Are the tests available as an artifact, and if so where are they and in what format?
Would I be right in thinking the only way to do this is via the Azure API? (I can't really be bothered to setup auth with that in the pipeline just for this feature, I don't interact with the API yet anywhere else)
The test results should be generated if you use VSTest#2 task to execute some tests. You can check the task log of VSTest task to check where the test result file is output to. Usually the default test result is trx file. You can change the output location by adding resultsFolder: 'output location' to the vstest task.
Once you get the test result file, you can write scripts to extract the test result summary by adding a script task.
For below example, use powershell script to extract the test summay from trx file and set it to env variable, which make it available in the following task.
- powershell: |
#get the path of the trx file from the output folder.
$path = Get-ChildItem -Path $(Agent.TempDirectory)\TestResults -Recurse -ErrorAction SilentlyContinue -Filter *.trx | Where-Object { $_.Extension -eq '.trx' }
$appConfigFile = $path.FullName #path to test result trx file
#$appConfigFile = '$(System.DefaultWorkingDirectory)\Result\****.trx' #path to test result trx file
$appConfig = New-Object XML
$appConfig.Load($appConfigFile)
$testsummary = $appConfig.DocumentElement.ResultSummary.Counters | select total, passed, failed, aborted
echo "##vso[task.setvariable variable=testSummary]$($testsummary)" #set the testsummary to environment variable
displayName: 'GetTestSummary'
condition: always()
In order to make the variable testSummary available in the following stage, Then you need to add dependency on this stage to the following stage. And use expression dependencies.<Previous stage name>.outputs['<name of the job which execute the task.setvariable >.TaskName.VariableName'] to pass the test summary to the variable in the following stages.
Please check below example
stages:
- stage: Test
displayName: 'Publish stage'
jobs:
- job: jobA
pool: Default
...
- powershell: |
#get the path of the trx file from the output folder.
$path = Get-ChildItem -Path $(Agent.TempDirectory)\TestResults -Recurse -ErrorAction SilentlyContinue -Filter *.trx | Where-Object { $_.Extension -eq '.trx' }
$appConfigFile = $path.FullName #path to test result trx file
#$appConfigFile = '$(System.DefaultWorkingDirectory)\Result\****.trx' #path to test result trx file
$appConfig = New-Object XML
$appConfig.Load($appConfigFile)
$testsummary = $appConfig.DocumentElement.ResultSummary.Counters | select total, passed, failed, aborted
echo "##vso[task.setvariable variable=testSummary]$($testsummary)" #set the testsummary to environment variable
displayName: 'GetTestSummary'
condition: always()
- stage: Release
dependsOn: Test
jobs:
- job: jobA
variables:
testInfo: $[dependencies.Test.outputs['jobA.GetTestSummary.testSummary']]
steps:
Then you can get the extracted test results info by referencing the variable $(testInfo).
Hope above helps!
I'm trying to parse some logs using split and regexes in powershell
Here's my code :
$string = "Starting ChromeDriver 78.0.3904.70Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code. Test 229: Passed Test 260: Failed. Error message: Status: Test case failed. Steps: Navigate to: PurchReqTableListPage (purchreqpreparedbyme) Use the Quick Filter to find records. For example, filter on the Purchase requisition fION()</StackTrace> </Error> Playback results: Tests: 2 Passed: 1 Failed: 1"
$string -Split '(Test (\d)+:)'
Result :
Starting ChromeDriver 78.0.3904.70Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
Test 229:
9
Passed
Test 260:
0
Failed. Error message: Status: Test case failed. Steps: Navigate to: PurchReqTableListPage (purchreqpreparedbyme) Use the Quick Filter to find records. For example, filter on the Purchase requisition fION()</StackTrace> </Error> Playback results: Tests: 2 Passed: 1 Failed: 1
Expected result:
Starting ChromeDriver 78.0.3904.70Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
Test 229:
Passed
Test 260:
Failed. Error message: Status: Test case failed. Steps: Navigate to: PurchReqTableListPage (purchreqpreparedbyme) Use the Quick Filter to find records. For example, filter on the Purchase requisition fION()</StackTrace> </Error> Playback results: Tests: 2 Passed: 1 Failed: 1
On this site : https://regexr.com/3c0lf I tried this regex and the groups captured were : Test 260: and Test 229: (which is exactly what I want)
I do not understand where the 0 and the 9 comes from.
Thanks a lot
Those are the last digits of the number. 0 from 26*0* and 9 from 22*9*.
You are seeing those because you've created an additional capturing group by putting parentheses around the digits. Just remove them like so:
$string -Split '(Test \d+:)
You probably don't even need those parentheses either, leaving just
$string -Split 'Test \d+:
What is the correct way to tokenize a string on double-forward-slash // in a Jenkinsfile?
The example below results in the string being tokenized on single-forward-slash / instead, which is not the desired behavior.
Jenkinsfile
An abbreviated, over-simplified example of the Jenkinsfile containing the relevant part is:
node {
// Clean workspace before doing anything
deleteDir()
try {
stage ('Clone') {
def theURL = "http://<ip-on-lan>:<port-num>/path/to/some.asset"
sh "echo 'theURL is: ${theURL}'"
def tokenizedURL = theURL.tokenize('//')
sh "echo 'tokenizedURL is: ${tokenizedURL}'"
}
} catch (err) {
currentBuild.result = 'FAILED'
throw err
}
}
The Logs:
The log output from the preceding is:
echo 'theURL is: http://<ip-on-lan>:<port-num>/path/to/some.asset'— Shell Script<1s
[ne_Branch-Name-M2X23QGNMETLDZWFK7IXVZQRCNSWYNTDFJZU54VP7DMIOD6Z4DGA] Running shell script
+ echo theURL is: http://<ip-on-lan>:<port-num>/path/to/some.asset
theURL is: http://<ip-on-lan>:<port-num>/path/to/some.asset
echo 'tokenizedURL is: [http:, <ip-on-lan>:<port-num>, path, to, some.asset]'— Shell Script<1s
[ne_Branch-Name-M2X23QGNMETLDZWFK7IXVZQRCNSWYNTDFJZU54VP7DMIOD6Z4DGA] Running shell script
+ echo tokenizedURL is: [http:, <ip-on-lan>:<port-num>, path, to, some.asset]
tokenizedURL is: [http:, <ip-on-lan>:<port-num>, path, to, some.asset]
Note that the logs show that the string is being tokeni on / instead of on //.
tokenize takes string as optional argument that may contain 1 or more characters as delimiters. It treats each character in string argument as separate delimiter so // is effectively same as /
To split on //, you may use split that supports regex:
theURL.split(/\/{2}/)
Code Demo
Has anyone ran into this issue? When the script gets to the banner text the script just hangs.
I am using Net::Appliance::Session
Here is the error I get in debug. The rest of the script inserts code perfectly. I did test what I read about adding a # to the banner for each line. Same result.
banner login +
[ 4.092880] tr nope, doesn't (yet) match (?-xism:[\/a-zA-Z0-9._\[\]-]+ ?(?:\(config[^)]*\))? ?[#>] ?$)
[ 4.093124] du SEEN:
banner login +
[ 4.093304] tr nope, doesn't (yet) match (?-xism:[\/a-zA-Z0-9._\[\]-]+ ?(?:\(config[^)]*\))? ?[#>] ?$)
[ 4.305872] du SEEN:
Enter TEXT message. End with the character '+'
[ 4.306121] tr nope, doesn't (yet) match (?-xism:[\/a-zA-Z0-9._\[\]-]+ ?(?:\(config[^)]*\))? ?[#>] ?$)
We had an issue when accessing the device : 10.49.216.74
The reported error was : read timed-out at /usr/lib/perl5/site_perl/5.10.0/Net/CLI/Interact/Transport/Wrapper/Net_Telnet.pm line 35
Here is a snip of code.
my $session_obj = Net::Appliance::Session->new(
host => $ios_device_ip,
transport => 'Telnet',
personality => 'ios',
timeout => 60,
);
#interace
$session_obj->set_global_log_at('debug');
eval {
# try to login to the ios device, ignoring host check
$session_obj->connect(
username => $ios_username,
password => $ios_password,
#SHKC => 0
);
# get our running config
$version_info = $session_obj->begin_privileged;
$session_obj->cmd('conf t');
$session_obj->cmd('line con 0');
$session_obj->cmd('exec-character-bits 8');
$session_obj->cmd('international');
$session_obj->cmd('line vty 0 4');
$session_obj->cmd('exec-character-bits 8');
$session_obj->cmd('international');
$session_obj->cmd('line vty 5 15');
$session_obj->cmd('exec-character-bits 8');
$session_obj->cmd('international');
$session_obj->cmd('exit');
$session_obj->cmd('no banner login');
$session_obj->cmd('banner login +');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('* test *');
$session_obj->cmd('* *');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('+');
$session_obj->cmd('no banner MOTD');
$session_obj->cmd('banner motd +');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('* test *');
$session_obj->cmd('* *');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('+');
$session_obj->cmd('exit');
$session_obj->cmd('write memory');
$session_obj->end_privileged;
# close down our session
$session_obj->close;
};
If you look at the regexp that matches the prompt before sending a new command you'll see that it requires a specific string that closely matches user, privileged or config mode of a router.
When you send the banner login + command you get the Enter TEXT message. End with the character '+' followed by blank line from a router (instead of Router(config)# that your script expects. After a while it just times out since there is no match for the regexp.
The easiest solution is to try to send the whole banner in one command. Try concatenating your banner with a \r in one string and sending it as a one command that looks like (note the double quotes):
$session_obj->cmd("banner login + line1 \r line2 \r line3\r +");
Took way too long to figure this out... spaces are not your friend.
$session_obj->cmd("banner login + \rline1\rline2\rline3\r+");
Example with my orginal problem:
$session_obj->cmd('*************************************************************************\r* test *\r* *\r*************************************************************************');