I have multiple test suites configured (as shown below) in my gradle build
task group1(type: Test) {
useTestNG() {
includeGroups 'group1'
}
// ... some excluded tests
ext.useDefaultListeners = true
ext.workingDirectory = 'build/'
// ... some system properties required by this test
}
task group2(type: Test, dependsOn: group1) {
useTestNG() {
includeGroups 'group2'
}
// ... some excluded tests
ext.useDefaultListeners = true
ext.workingDirectory = 'build/'
// ... some system properties required by this test
}
test.dependsOn group2
When I run the gradle clean build --debug, I could see after every test task (group1, group2 etc), gradle deletes the test report and regenerates it. Hence, after the build is done, whatever report I get does not have any information. All blank!
How can I get a combined report?
Thanks,
NN
You should use a separate task of type TestReport
Check out section '23.12.7. Test reporting' in the user guide
Related
I have a dart project that has several tests, when I try to run an isolated test I get this warning:
Warning: A tag was used that wasn't specified in dart_test.yaml.
"tagName" was used in the suite itself
how should i declare these tags correctly in dart_test.yaml?
Steps
Create a file dart_test.yaml at the root of your project
Add your tags one after another under a tags field
Add tags to your test or testWidget declaration
Run your tests with the -t flag followed by the wanted tag
Sample
Let's say I want to add the following tags: golden, atom, molecule, organism, mobile, desktop. My dart_test.yaml will look like this:
tags:
golden:
atom:
molecule:
organism:
mobile:
desktop:
And everything should be okay you can write your test:
void main() {
testWidgets(
'this is a test',
(tester) async {
// ...
},
tags: ['atom', 'mobile'],
);
}
You can run it with the following command:
$ flutter test -t mobile
source
I have a vNext build which runs a few NUnit tests and publishes the results. However, the output produced by the tests is not published, even though it is found in the respective XML file.
For example:
But, if I inspect the respective XML files - it is there:
Notice the <output> element.
I would very much like to see this output published along the test results. How can I do it?
P.S.
There is another problem with the publishing. The Run Duration is reported as 4h 59m in the top pane, but in the bottom details pane it is the correct 10:48.176 minutes. The 4h 59m looks very much as 5h, which is the time difference between EST and UTC. The tests were run on an Octopus server and were fetched by the vNext build. Maybe there is a time zone confusion somewhere.
EDIT 1
We have an on-premises TFS
EDIT 2
Examining the source code of the Publish Test Results task reveals that it uses Microsoft.TeamFoundation.TestClient.PublishTestResults assembly to parse the NUnit XML test result. Specifically the following C# code is used to parse the test-case element (NUnit3ResultsXmlReader.cs):
if (testCaseResultNode.Attributes["result"] != null)
{
testCaseResultData.TestCaseResult.Outcome = !string.Equals(testCaseResultNode.Attributes["result"].Value, "Passed", StringComparison.OrdinalIgnoreCase) ? (!string.Equals(testCaseResultNode.Attributes["result"].Value, "Failed", StringComparison.OrdinalIgnoreCase) ? (!string.Equals(testCaseResultNode.Attributes["result"].Value, "Skipped", StringComparison.OrdinalIgnoreCase) ? TestOutcome.Inconclusive.ToString() : TestOutcome.NotExecuted.ToString()) : TestOutcome.Failed.ToString()) : TestOutcome.Passed.ToString();
XmlNode xmlNode1 = testCaseResultNode.SelectSingleNode("failure");
if (xmlNode1 != null)
{
XmlNode xmlNode2 = xmlNode1.SelectSingleNode("message");
XmlNode xmlNode3 = xmlNode1.SelectSingleNode("stack-trace");
testCaseResultData.TestCaseResult.ErrorMessage = xmlNode2 != null ? xmlNode2.InnerText : (string) null;
testCaseResultData.TestCaseResult.StackTrace = xmlNode3 != null ? xmlNode3.InnerText : (string) null;
XmlNode xmlNode4 = testCaseResultNode.SelectSingleNode("output");
if (!string.IsNullOrWhiteSpace(xmlNode4 != null ? xmlNode4.InnerText : (string) null))
testCaseResultData.ConsoleLog = xmlNode4.InnerText;
}
}
From which it follows that the assembly authors think that test output is only useful when the test-case in question has failed. This is an unfortunate decision, because it is not up to them to decide when the output is useful. If it is in the test results XML, then it should be published.
Opened a new issue https://github.com/Microsoft/azure-pipelines-tasks/issues/8979
Opened an Azure DevOps feature request - https://developercommunity.visualstudio.com/idea/432166/the-publish-tests-azure-devops-plugin-should-publi.html
My solution to this unfortunate behavior force upon us is to mangle the XML test result before handing it off to the Publish Test Results task.
The following powershell code does the trick:
$Modified = $false
$xml = [xml](cat $OriginalTestResultsXmlFile -Raw)
$xml.SelectNodes('//test-case') |? { !$_.failure -and $_.output } |% {
$Modified = $true
$_.InnerXml = #"
<!-- Workaround the issue https://github.com/Microsoft/azure-pipelines-tasks/issues/8979. No real failure here -->
<failure/>
$($_.InnerXml)
"#
}
if ($Modified)
{
$xml.Save($TestResultsXmlFile)
}
else
{
move $OriginalTestResultsXmlFile $TestResultsXmlFile
$OriginalTestResultsXmlFile = $TestResultsXmlFile
}
We have to trick the task into executing the logic we need by creating a dummy failure element besides the output element.
Lo and behold:
In Grails 3.1.12, I want to unit test a service:
#Transactional
class PlanService {
List<Plan> getPlans(Map params) {
def currentUser = (User)springSecurityService.getCurrentUser()
return Plan.findAllByCompany(currentUser.employer, params)
}
}
Like this:
#TestFor(PlanService)
#Mock([Plan, User, Company])
class PlanServiceSpec extends Specification {
void "Retrieve plan from the current user"() {
setup:
// create and save entities here
when: "the plans are retrieved"
def params = null
def plans = service.getPlans(params)
then: "the result should only include plans associated to the current user's company"
plans.size() == 2
}
Running the test from the console:
grails> test-app my.PlanServiceSpec -unit
Fails with:
my.FundingPlanServiceSpec > Retrieve plan from the current user FAILED
java.lang.IllegalStateException at PlanServiceSpec.groovy:48
and in the test report (HTML):
java.lang.IllegalStateException: No transactionManager was specified.
Using #Transactional or #Rollback requires a valid configured transaction manager.
If you are running in a unit test ensure the test has been properly configured
and that you run the test suite not an individual test method.
Now if I comment out the #Transactional annotation in the service, the test passes, but that's not the intended implementation. I am able to work around the problem by mocking the transaction manager:
service.transactionManager = Mock(PlatformTransactionManager) {
getTransaction(_) >> Mock(TransactionStatus)
}
But this seems very awkward, if not wrong.
Is there some incantation I forgot to invoke?
EDIT: looks similar to an old bug, but it's been closed more than a year.
Have you tried what a comments says that fixes the problem? If not, try to annotate the test class with:
#TestMixin(DomainClassUnitTestMixin)
and then:
service.transactionManager = getTransactionManager()
Was getting the same error in grails 3.3.2 when trying to test transactional service.
adding DataTest interface solved the issue for me.
class HelloServiceSpec extends Specification implements ServiceUnitTest<HelloService>, DataTest {
}
I am unit testing a model class and I would like all Doctrine queries to be logged.
My settings.yml for the test environment contains
logging_enabled: true
and my script
$configuration = ProjectConfiguration::getApplicationConfiguration( 'frontend', 'test', true);
new sfDatabaseManager( $configuration ) ;
Still, I don't see any log in any log file.
So, I found a workaround by using an event listener on the Doctrine profiler.
$profiler = new Doctrine_Connection_Profiler();
$conn = Doctrine_Manager::connection();
$conn->setListener($profiler);
/* tests go here */
foreach ($profiler as $event) {
echo $event->getQuery() . "\n";
}
But the same query is printed out several times for some reason (I am sure it is executed only once). Plus it is not so convenient to have the query logs dissociated from the rest of the log messages.
I build my project with Gradle 1.0 and I use the EMMA plugin for code coverage info. I would like to exclude certain files from the coverage report.
How can I achieve that?
Are you including this Gradle script? I think you can exclude classes within your instrumentation definition (see example below). However, it doesn't look like you can set the exclude pattern by using a convention property.
ant.emma(enabled: 'true', verbosity:'info'){
instr(merge:"true", destdir: emmaInstDir.absolutePath, instrpathref:"run.classpath",
metadatafile: new File(emmaInstDir, '/metadata.emma').absolutePath) {
instrpath {
fileset(dir:sourceSets.main.output.classesDir.absolutePath, includes:"**/*.class", excludes:"**/Some*.class")
}
}
}
If I were you I'd try to fork the plugin, add a new field to EmmaPluginConvention that lets you set the exclude pattern and then use that variable in the instrpath definition. After changing the code and verifying that it works send a pull request to the author. I am sure he will incorporate your change.
This doesn't work with gradle 1.5. Emma takes a filter like so:
ant.emma(enabled: 'true', verbosity: $verbosityLevel) {
instr(merge: "true", destdir: emmaInstDir.absolutePath, instrpathref: "run.classpath",
metadatafile: new File(emmaInstDir, '/metadata.emma').absolutePath, filter: "-com.someclass.*" ) {
instrpath {
fileset(dir: sourceSets.main.output.classesDir.absolutePath, includes: "**/*.class" )
}
}
}
the filter follows the definition from this page:
http://emma.sourceforge.net/reference/ch02s06s02.html