Show Salesforce tests results in VSTS build result - unit-testing

I set up a build in VSTS to run ANT migration tool to deploy to Salesforce Org. I would like to somehow publish the results of the unit tests to the VSTS, so I can leverage the VSTS test results overview. I can see the test results in the log output of the ANT task in the build job, but using the VSTS overview seems more convenient. Is there a way to do this?

If you can get the test results file, then you can use the Test: Publish Test Results task to publish the results to VSTS.
Besides, you can check if this article helps: Versioning and Deploying Salesforce Metadata using TFS/VSTS

Related

Build and run tests with different jobs

I have some cpp/header files in Github that I am currently building through 2 Jenkins jobs.
The first Jenkins job will build the tests and the second will run the tests. I could have combined both into one job, but I can't publish the test results of Google Tests with JUnit due to this issue Append multiple xml test reports of google test, so separating both jobs would make it clearer.
My solution was making the test job share the same workspace as the build job and triggering it once the build is stable.
I am new to Jenkins and don't know if this is a typical workflow. Is there any other idea on how to do this or is this reasonable?

How to automatically notify tests' owners on failing tests?

We're using Azure DevOps to run a nightly build that checks our tests results. Our tests are decorated with Owner attribute (MSTest) of the test's owner name.
What is the simplest way to auto send email to the owner of the failing tests as part of the nightly run?
Can this functionality be achieved by using some built-in feature of Azure DevOps? If not, which custom flow can be integrated to the nightly build to achieve this?
There is no such build-in feature in Azure DevOps for now.
Please take a look at similar question here: How to send a detailed email to specific developer from Azure DevOps on failure of unit tests from a Build pipeline?
As workaround, we could use a powershell script to get test results with REST API then send emails:
To get test results with RESI API Releases - Get Task Log:
GET https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/releases/{releaseId}/environments/{environmentId}/deployPhases/{releaseDeployPhaseId}/tasks/{taskId}/logs?api-version=5.0-preview.2
Then use powershell scripts to read the details of the test result file (.trx) and create a formatted email message to sent to the defined recipients:
You could check this thread Send Test Result Email After Running Automated Tests with Release Management for some details.
There is also an Email Report Extension could handle e-mail related things in pipeline.

Test Results and Documentation Site

Currently i am using these tools to run my tests,code coverage and documentation:
Unit testing:
jasmine
xUnit
Code Coverage:
Istanbul
dotCover
Documentation:
Typedoc
As i'm trying to do everything modular for both frontend and backend we have multiple bower components and nuget packages where of course each components runs different type of tests and documentation.
Now what i want to do is to have a dedicated site which grabs all test results and documentation and have a dedicated site where all developers etc. can use it as a point of reference.
Is there any plugin available that can help me achieve it?
if not do you have any idea from where can i start as i tried googling a bit but with no luck.
I'm using roughly the same technologies.
As a build server I use TeamCity.
In a nutshell: your build is composed by steps, e.g (simplified):
build .sln
gulp build
xUnit tests (*A: publishing coverage)
karma run
remap coverage from Javascript to Typescript (*B: publish coverage)
The only problem I had so far is with the coverage (*A + *B). The last data will overwrite the first one, (not average it all). So in that case I use custom reports page to display the istanbul generated html report and only use the xUnit coverage report.
You could have the coverage.json from istanbul as an artifact of your build, and a second build picks up and reports that coverage through teamcity. It would be simply a coverage report build (only 1 step, report code coverage). The trigger is a successful build generating the coverage.
For your generated documentation you can also use custom reports page.
About the unit tests execution (both jasmine (karma?) and xunit), both report its numbers and the final Test report will show them combined.

Creating chart out of ember-cli-blanket report in Jenkins

I have integrated ember-cli-blanket in my Jenkins job successfully. And the ember test will generate a output file in JSON format(eg: coverage-report.json).
Now for making it more user friendly or making it more understandable, I need to show a chart/graph which describes the result of test(content of coverage-report.json), after the build finishes successfully.
I have tried some Jenkins plugin like, cucumber plugin, cucumber reports plugin, cucumber JSON test reporting etc. But nothing works.
Is there any way to achieve this? Any help is appreciated.

use hudson to get remote tests results

I've a server running a proprietary language on which I'm able to run "unit tests" in this language. I cannot install a Hudson slave on this machine, but would like to have these tests results appearing in a job of hudson (to have at least a monitoring of the code quality for this server code).
I'm currently trying to use web services to get the results and store them in Hudson workspace, but I do fear it is not the right solution.
What solutions can you advice me ?
I finally have gotten through the web services path, although it was not easy.
There are some steps in this path
I created a maven mojo with groovy (see GMaven for more infos) which, using groovyws, called a web service that, from tests results, creates the junit report.
Armed with this mojo, I created a maven project that called the web service and stores the junit.xml file in an output folder
Finally, i created in hudson a maven job for this project and called it regularly. Thanks to junit reporting integration in maven builds, my tests results are visible as a graph in Hudson and user can drill down to failing tests.
Not sure if these are possible but...
Maybe one option is when the build job finished execute a second build target or script to scp the test results from the remote server to the local build server so they appear in hudson
Or if the platform allows
Map a directory on the remote machine to the local file system by using something like sshfs etc
karl
Yup, you can scp or whatever the results (in junit xml format) to the current workspace dir using a script task. Then have a "Publish JUnit test result report" post-build task & point it at the copied-in files.
Obviously if it's not in junit-compatible format you'll have to convert it.
Sounds like you're on the right path though