I want to use Address Sanitizer to detect runtime errors while unit tests are running. However, Address Sanitizer terminates the app when it finds the first error so I can't see an information of all found runtime errors in one report. It would be possible if I could run google tests isolated from each other, and termination of one test woudn't influence other ones.
You can run specific test cases in Google Test, one by one.
From: https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#running-a-subset-of-the-tests
By default, a Google Test program runs all tests the user has defined.
Sometimes, you want to run only a subset of the tests (e.g. for
debugging or quickly verifying a change). If you set the GTEST_FILTER
environment variable or the --gtest_filter flag to a filter string,
Google Test will only run the tests whose full names (in the form of
TestCaseName.TestName) match the filter.
The format of a filter is a ':'-separated list of wildcard patterns
(called the positive patterns) optionally followed by a '-' and
another ':'-separated pattern list (called the negative patterns). A
test matches the filter if and only if it matches any of the positive
patterns but does not match any of the negative patterns.
A pattern may contain '' (matches any string) or '?' (matches any
single character). For convenience, the filter '-NegativePatterns'
can be also written as '-NegativePatterns'.
For example:
./foo_test Has no flag, and thus runs all its tests.
./foo_test --gtest_filter=* Also runs everything, due to the single
match-everything * value.
./foo_test --gtest_filter=FooTest.* Runs everything in test case
FooTest.
./foo_test --gtest_filter=Null:Constructor Runs any test whose
full name contains either "Null" or "Constructor".
./foo_test --gtest_filter=-DeathTest. Runs all non-death tests.
./foo_test --gtest_filter=FooTest.*-FooTest.Bar Runs everything in
test case FooTest except FooTest.Bar.
Related
To reduce size of my simulation output files, I want to give variable name exceptions instead of a list of many certain variables to the simulationsOptions/outputFilter (cf. OpenModelica Users Guide / Output) of my model. I found the regexp operator "^" to fullfill my needs, but that didn't work as expected. So I think that something is wrong with the interpretation of connected character strings when negated.
Example:
When I have any derivatives der(...) in my model and use variableFilter=der.* the output file will contain all the filtered derivatives. Since there are no other varibles beginning with character d the same happens with variableFilter=d.*. For testing I also tried variableFilter=rde.* to confirm that every variable is filtered.
When I now try to except by variableFilter=^der.*, =^rde.* or =^d.*, I get exactly the same result as without using ^. So the operator seems to be ignored in this notation.
When I otherwise use variableFilter=[^der].*, =[^rde].* or even =[^d].*, all wanted derivation variables are filtered from the ouput, but there is no difference between those three expressions above. For me it seems that every character is interpretated standalone and not as as a connected string.
Did I understand and use the regexp usage right or could this be a code bug?
Side/follow-up question: Where can I officially report this for software revision?
_
OpenModelica v.1.19.2 (64-bit)
I followed the instructions in question
Ignore Issues in Blocks
but since it didn't work, I suspect it may be something related with the regex rather than the Sonar configurations.
I have some code that is generated by EMF. I changed the code generation templates to also add //end-generated tag at the end of block, which looks like this:
/*
* #generated
*/
public class MyGeneratedClass implements Enumerator {
/*
* #generated
*/
public void myGeneratedMethod() {
// some code that SonarQube doesn't like
} //end-generated
}
The idea is that the regex must only match method blocks. So the START-BLOCK must match text #generated as long it's not followed by 'class', 'enum' or 'interface' word after the comment termination.
I created this regex:
For start block:
#generated\h*\s[\s\*]*\/(?!\s*[^\s]*\s*(class|enum|interface)\s)
For end block:
\/\/end-generated
This works in my tests using a simple code with Java Pattern class, which returns true for class sample above:
public static void main(String[] args) throws IOException {
String regex = "#generated\\h*\\s[\\s\\*]*\\/(?!\\s*[^\\s]*\\s*(class|enum|interface)\\s)";
String text = new String(Files.readAllBytes(Paths.get("test.txt")));
Matcher matcher = Pattern.compile(regex).matcher(text);
System.out.println(matcher.find());
}
However when I add it to SonarQube configurations, it doesn't work, issues found in the generated method are still reported by SonarQube.
I added the regex in:
SonarQube » Settings » Exclusions » Issues » Ignore Issues in Blocks
I also tried with escaped version of regex, and result is the same:
#generated\\h*\\s[\\s\\*]*\\/(?!\\s*[^\\s]*\\s*(class|enum|interface)\\s)
Also adding these settings directly into Sonar properties didn't work, as reported already in the question I mentioned above:
sonar.issue.ignore.block=rule1
sonar.issue.ignore.block.rule1.beginBlockRegexp=#generated\\h*\\s[\\s\\*]*\\/(?!\\s*[^\\s]*\\s*(class|enum|interface)\\s)
sonar.issue.ignore.block.rule1.endBlockRegexp=\\/\\/end-generated
I'm using SonarQube server 5.1.2 and running the analysis from Sonar-ant-task 2.3.
I'm wondering if it may be something wrong with the regex that makes SonarQube not able to match it, or maybe some limitation in SonarQube regex processing?
EDIT: I changed the regex for something more simple to match just the '#generated' word, and it worked. But if I put '#generated[\n\r]' to force to only match if there's a new line after #generated, then it doesn't work anymore.
It seems SonarQube doesn't support basic things such as new line characters. Can someone confirm this?
I confirm that all patterns used to exclude issues are applied line per line. In the end it is always translated to "exclude issues from line X to line Y". I agree this is not perfect (especially now we have precise issue location) but this is an "historical" feature. We will probably not invest time in it, since our mood is to avoid complex configuration. The ideal solution for your use case would be to have https://jira.sonarsource.com/browse/SONARJAVA-71 implemented.
Since SonarQube applies regex per line, I came up with a different solution for this use-case:
I use the start-block regex per line: (?m)#generated$ and then exclude class|enum|interface by adding this pattern to the end-block regex, together with end-generated.
Final configuration looks like this:
sonar.issue.ignore.block=rule1
sonar.issue.ignore.block.rule1.beginBlockRegexp=(?m)#generated$
sonar.issue.ignore.block.rule1.endBlockRegexp=\/\/\h*end-generated|\sclass\s|\senum\s|\sinterface\s
All example I have found of file globbing only show character substitution, not alternation. I have the desire to glob a subset of files and want to do so by name. It seems it should be possible. Below are two examples of what I have tried.
['test/functional/**/(login|account|productManagement)-spec.*']
['test/functional/**/\b(login|account|productManagement)\b-spec.*']
Of course, the following does work as expected.
['test/functional/**/(login|account|productManagement)-spec.*']
This is javascript code running in node.js.
If it is not clear, I am expecting to find:
login-spec.*
account-spec.*
productManagement-spec.*
We have a large suite of protractor tests. Some fail randomly when running locally. Okay, that should be fixed, but it is an environmental thing and the obvious fix will slow down the whole test run. It is far simpler to run them, identify those that have failed, then re-run.
i'm trying to skip a specific test case using keyword, is there are any keyword to do that ? what i'm trying to do is to check if file name have "skip" word then i want to skip it. is there are any keyword like : Skip Test , Skip Execution If ...
#{regex}= Get Regexp Matches ${TEST NAME} Skip
${lenght}= Get Length ${regex}
Skip Execution If '${lenght}'>'0'
Nowadays, there are the following keywords added, in Robot Framework: Skip and Skip if
Ideally, tests that should not be run should be excluded from the run using tags or other means. Another option is to still run the tests, but simply check for your skip condition at the start of the test and pass the test without executing anything. There are two keywords, Builtin.Pass Execution and Builtin.Pass Execution If, useful for that.
http://robotframework.org/robotframework/latest/libraries/BuiltIn.html#Pass%20Execution%20If
I want to exclude all tests which include Abc in the namespace.
What is the correct pattern to use ?
!(FullyQualifiedName~.Abc.)
is not valid ?
I used this web-page as reference : http://blogs.msdn.com/b/visualstudioalm/archive/2012/09/26/running-selective-unit-tests-in-vs-2012-using-testcasefilter.aspx
There is no such pattern !(FullyQualifiedName~.Abc.) in build definition.
Use Test Case Filter in XAML build definition to select tests to run based on filter criteria. You can use the format Operator to construct your filter where Operator is one of =,!= or ~. You can also use the logical operator |, & to construct your filter and parenthesis() for grouping.
To achieve what you want, you can:
Give each of your Unit test except containing "Abc" a priority (maybe 1), and give unit tests containing "Abc" a priority (maybe2). In the test case filter, set “Proirity=1*”, then you can exclude the unit tests contain "Abc".
In another way, you can filter out unit tests not contain "Abc" by Test assembly file name. Identify unit tests not contain "Abc" based on a naming pattern Def.Tests.dll, then in the Test assembly file specification blank, input “**\ *Def.Tests.dll”.
Similar case: https://social.msdn.microsoft.com/Forums/vstudio/en-US/d415dbdd-74a0-484b-a5ae-e5af3e985e94/how-to-explicitly-exclude-assemblies-in-test-runner-during-tfs-build?forum=tfsgeneral