I'm working on a Grails project. When I run unit tests with test coverage, it skips the enclosing brackets from code coverage. It's impossible to get 100% coverage. It happens for all methods.
I tried varying the coverage runner by using JaCoCo and Emma, but no success.
How can I fix this? Why do those brackets get added to total coverage anyway? It's super annoying.
EDIT: Actually it skips closing brackets right after return (or break) statements.
Related
I want to use Address Sanitizer to detect runtime errors while unit tests are running. However, Address Sanitizer terminates the app when it finds the first error so I can't see an information of all found runtime errors in one report. It would be possible if I could run google tests isolated from each other, and termination of one test woudn't influence other ones.
You can run specific test cases in Google Test, one by one.
From: https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#running-a-subset-of-the-tests
By default, a Google Test program runs all tests the user has defined.
Sometimes, you want to run only a subset of the tests (e.g. for
debugging or quickly verifying a change). If you set the GTEST_FILTER
environment variable or the --gtest_filter flag to a filter string,
Google Test will only run the tests whose full names (in the form of
TestCaseName.TestName) match the filter.
The format of a filter is a ':'-separated list of wildcard patterns
(called the positive patterns) optionally followed by a '-' and
another ':'-separated pattern list (called the negative patterns). A
test matches the filter if and only if it matches any of the positive
patterns but does not match any of the negative patterns.
A pattern may contain '' (matches any string) or '?' (matches any
single character). For convenience, the filter '-NegativePatterns'
can be also written as '-NegativePatterns'.
For example:
./foo_test Has no flag, and thus runs all its tests.
./foo_test --gtest_filter=* Also runs everything, due to the single
match-everything * value.
./foo_test --gtest_filter=FooTest.* Runs everything in test case
FooTest.
./foo_test --gtest_filter=Null:Constructor Runs any test whose
full name contains either "Null" or "Constructor".
./foo_test --gtest_filter=-DeathTest. Runs all non-death tests.
./foo_test --gtest_filter=FooTest.*-FooTest.Bar Runs everything in
test case FooTest except FooTest.Bar.
I followed the instructions in question
Ignore Issues in Blocks
but since it didn't work, I suspect it may be something related with the regex rather than the Sonar configurations.
I have some code that is generated by EMF. I changed the code generation templates to also add //end-generated tag at the end of block, which looks like this:
/*
* #generated
*/
public class MyGeneratedClass implements Enumerator {
/*
* #generated
*/
public void myGeneratedMethod() {
// some code that SonarQube doesn't like
} //end-generated
}
The idea is that the regex must only match method blocks. So the START-BLOCK must match text #generated as long it's not followed by 'class', 'enum' or 'interface' word after the comment termination.
I created this regex:
For start block:
#generated\h*\s[\s\*]*\/(?!\s*[^\s]*\s*(class|enum|interface)\s)
For end block:
\/\/end-generated
This works in my tests using a simple code with Java Pattern class, which returns true for class sample above:
public static void main(String[] args) throws IOException {
String regex = "#generated\\h*\\s[\\s\\*]*\\/(?!\\s*[^\\s]*\\s*(class|enum|interface)\\s)";
String text = new String(Files.readAllBytes(Paths.get("test.txt")));
Matcher matcher = Pattern.compile(regex).matcher(text);
System.out.println(matcher.find());
}
However when I add it to SonarQube configurations, it doesn't work, issues found in the generated method are still reported by SonarQube.
I added the regex in:
SonarQube » Settings » Exclusions » Issues » Ignore Issues in Blocks
I also tried with escaped version of regex, and result is the same:
#generated\\h*\\s[\\s\\*]*\\/(?!\\s*[^\\s]*\\s*(class|enum|interface)\\s)
Also adding these settings directly into Sonar properties didn't work, as reported already in the question I mentioned above:
sonar.issue.ignore.block=rule1
sonar.issue.ignore.block.rule1.beginBlockRegexp=#generated\\h*\\s[\\s\\*]*\\/(?!\\s*[^\\s]*\\s*(class|enum|interface)\\s)
sonar.issue.ignore.block.rule1.endBlockRegexp=\\/\\/end-generated
I'm using SonarQube server 5.1.2 and running the analysis from Sonar-ant-task 2.3.
I'm wondering if it may be something wrong with the regex that makes SonarQube not able to match it, or maybe some limitation in SonarQube regex processing?
EDIT: I changed the regex for something more simple to match just the '#generated' word, and it worked. But if I put '#generated[\n\r]' to force to only match if there's a new line after #generated, then it doesn't work anymore.
It seems SonarQube doesn't support basic things such as new line characters. Can someone confirm this?
I confirm that all patterns used to exclude issues are applied line per line. In the end it is always translated to "exclude issues from line X to line Y". I agree this is not perfect (especially now we have precise issue location) but this is an "historical" feature. We will probably not invest time in it, since our mood is to avoid complex configuration. The ideal solution for your use case would be to have https://jira.sonarsource.com/browse/SONARJAVA-71 implemented.
Since SonarQube applies regex per line, I came up with a different solution for this use-case:
I use the start-block regex per line: (?m)#generated$ and then exclude class|enum|interface by adding this pattern to the end-block regex, together with end-generated.
Final configuration looks like this:
sonar.issue.ignore.block=rule1
sonar.issue.ignore.block.rule1.beginBlockRegexp=(?m)#generated$
sonar.issue.ignore.block.rule1.endBlockRegexp=\/\/\h*end-generated|\sclass\s|\senum\s|\sinterface\s
All example I have found of file globbing only show character substitution, not alternation. I have the desire to glob a subset of files and want to do so by name. It seems it should be possible. Below are two examples of what I have tried.
['test/functional/**/(login|account|productManagement)-spec.*']
['test/functional/**/\b(login|account|productManagement)\b-spec.*']
Of course, the following does work as expected.
['test/functional/**/(login|account|productManagement)-spec.*']
This is javascript code running in node.js.
If it is not clear, I am expecting to find:
login-spec.*
account-spec.*
productManagement-spec.*
We have a large suite of protractor tests. Some fail randomly when running locally. Okay, that should be fixed, but it is an environmental thing and the obvious fix will slow down the whole test run. It is far simpler to run them, identify those that have failed, then re-run.
i'm trying to skip a specific test case using keyword, is there are any keyword to do that ? what i'm trying to do is to check if file name have "skip" word then i want to skip it. is there are any keyword like : Skip Test , Skip Execution If ...
#{regex}= Get Regexp Matches ${TEST NAME} Skip
${lenght}= Get Length ${regex}
Skip Execution If '${lenght}'>'0'
Nowadays, there are the following keywords added, in Robot Framework: Skip and Skip if
Ideally, tests that should not be run should be excluded from the run using tags or other means. Another option is to still run the tests, but simply check for your skip condition at the start of the test and pass the test without executing anything. There are two keywords, Builtin.Pass Execution and Builtin.Pass Execution If, useful for that.
http://robotframework.org/robotframework/latest/libraries/BuiltIn.html#Pass%20Execution%20If
In my Go package there are several benchmark files like map1_benchmark_test.go and map2_benchmark_test.go. In every *_benchmark_test.go file, there is more than one benchmark function like func BenchmarkMapTravel(b *testing.B) and func BenchmarkMapGet(b *testing.B).
Question is, how can I test just one benchmark function?
I attempted to read some manuals, and got nothing about benchmark by running go help test.
Description of testing flags
-test.bench pattern
Run benchmarks matching the regular expression.
By default, no benchmarks run.
-test.run pattern
Run only those tests and examples matching the regular
expression.
For convenience, each of these -test.X flags of the test binary is
also available as the flag -X in 'go test' itself.
For help,
$ go help testflag
For example,
go test -test.bench MapTravel
go test -test.bench MapGet
or
go test -bench MapTravel
go test -bench MapGet
To bypass test functions, include a -test.run pattern that filters out every single test. For example,
go test -test.bench MapTravel -test.run=thisexpressionwontmatchanytest
or
go test -bench MapTravel -run=^$
There is no flag you can provide, that will run only benchmarks (or only one benchmark). The only flags related to these are:
-bench regexp
Run benchmarks matching the regular expression.
By default, no benchmarks run. To run all benchmarks,
use '-bench .' or '-bench=.'.
-run regexp
Run only those tests and examples matching the regular
expression.
So if you want only to run one benchmark, you can do this:
go test -bench=nameOfYourBenchmark -run=^a
This will cause to run only tests that starts with a. And because each test should be named Test<something>, there will be no tests to run.
To run only benchmarks:
go test -bench=. -run=^a
Test only TestFuncOne
$>> go test -run TestFuncOne
stuff_to_test.go
TestFuncOne() {
}
TestFuncTwo() {
}