We're using Azure DevOps to run a nightly build that checks our tests results. Our tests are decorated with Owner attribute (MSTest) of the test's owner name.
What is the simplest way to auto send email to the owner of the failing tests as part of the nightly run?
Can this functionality be achieved by using some built-in feature of Azure DevOps? If not, which custom flow can be integrated to the nightly build to achieve this?
There is no such build-in feature in Azure DevOps for now.
Please take a look at similar question here: How to send a detailed email to specific developer from Azure DevOps on failure of unit tests from a Build pipeline?
As workaround, we could use a powershell script to get test results with REST API then send emails:
To get test results with RESI API Releases - Get Task Log:
GET https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/releases/{releaseId}/environments/{environmentId}/deployPhases/{releaseDeployPhaseId}/tasks/{taskId}/logs?api-version=5.0-preview.2
Then use powershell scripts to read the details of the test result file (.trx) and create a formatted email message to sent to the defined recipients:
You could check this thread Send Test Result Email After Running Automated Tests with Release Management for some details.
There is also an Email Report Extension could handle e-mail related things in pipeline.
Related
I've seen many discussions on-line about Sonar web-hooks to send scan results to Jenkins, but as a CodePipeline acolyte, I could use some basic help with the steps to supply Sonar scan results (e.g., quality-gate pass/fail status) to the pipeline.
Is the Sonar web-hook the right way to go, or is it possible to use Sonar's API to fetch the status of a scan for a given code-project?
Our code is in BitBucket. I'm working with the AWS admin who will create the CodePipeline that fires when code is attempted to be pushed into the repo. sonar-scanner will be run, and then we'd like the pipeline to stop if the quality does not pass the Quality Gate.
If I would use a Sonar web-hook, I imagine the value for host would be, what, the AWS instance running the CodeBuild?
Any pointers, references, examples welcome.
I created a powershell to use with Azure DevOps, that possible may be migrated to some shell script that runs in the code build activity
https://github.com/michaelcostabr/SonarQubeBuildBreaker
How do I run a specific set of appium java testng test cases in AWS Device farm? I could see that device farm ignores all the annotations for testNG including group and enabled. Is there any way out?
To only run a subset of tests the project will need to include the testng.xml file in the root of the *-tests.jar. Here is a GitHub post I've authored showing how to do that.
https://github.com/aws-samples/aws-device-farm-appium-tests-for-sample-app/pull/14
In the standard environment the tests are parse and executed individually. As a result some testng features like priority and nested groups don't get honored. Also tests are executed slower because the Appium server would be restarted between tests.
https://docs.aws.amazon.com/devicefarm/latest/developerguide/test-environments.html#test-environments-standard
If these feature are needed the project will need to use the custom environments in Device Farm.
https://docs.aws.amazon.com/devicefarm/latest/developerguide/custom-test-environments.html
This produces one set of logs and video of all the tests since the test package is not parsed.
Hth
-James
I set up a build in VSTS to run ANT migration tool to deploy to Salesforce Org. I would like to somehow publish the results of the unit tests to the VSTS, so I can leverage the VSTS test results overview. I can see the test results in the log output of the ANT task in the build job, but using the VSTS overview seems more convenient. Is there a way to do this?
If you can get the test results file, then you can use the Test: Publish Test Results task to publish the results to VSTS.
Besides, you can check if this article helps: Versioning and Deploying Salesforce Metadata using TFS/VSTS
I am working on a testing automation project and I am able to automate end to end test cases but in my scope I have to enable functionality to run test-case when a workflow is checked-in to repository. For this I need a trigger with information like what workflow and in which folder is checked it's checked in. Unfortunately I am unable to get anything till now and my admins are also not sure if this can be achieved. Any lead will be a great help.
Even if I can get the logs of workflow check-in, I can process them in real-time to get this information.
I have taken a look at alerts management for TFS 2012 after installing power tools, and I can see about four types of alert templates:
Work Item
Code Review
Check In
Build
I was wondering if there was a possibility of having a supported way to register alerts under “Publish” event-type, manageable directly via the Alerts explorer.
If not, I thought of some workarounds:
If this cannot be done and managed via the Alerts explorer, can I customize an alert to be triggered on Publish event via a Web Service? If so, does the TFS API support such customization?
I can also instead go with a continuous delivery approach and set an automated publish upon successful build of a solution, with an email alert on Build-Event Success (which would also mean that a solution has been published).
Which approach would be a supported way for setting "on publish"-event alerts for web solutions via TFS?
My suggestion would be not to use the Publish in Visual Studio but instead to use a Build to publish your solution (either triggered on check-in, or manually triggered). Then you can easily setup an alert on that build. Using a build instead of VS-Publish is also considered a better practice because it gives you more power and flexibility for the deployment process.