How to test only one benchmark function? - unit-testing

In my Go package there are several benchmark files like map1_benchmark_test.go and map2_benchmark_test.go. In every *_benchmark_test.go file, there is more than one benchmark function like func BenchmarkMapTravel(b *testing.B) and func BenchmarkMapGet(b *testing.B).
Question is, how can I test just one benchmark function?
I attempted to read some manuals, and got nothing about benchmark by running go help test.

Description of testing flags
-test.bench pattern
Run benchmarks matching the regular expression.
By default, no benchmarks run.
-test.run pattern
Run only those tests and examples matching the regular
expression.
For convenience, each of these -test.X flags of the test binary is
also available as the flag -X in 'go test' itself.
For help,
$ go help testflag
For example,
go test -test.bench MapTravel
go test -test.bench MapGet
or
go test -bench MapTravel
go test -bench MapGet
To bypass test functions, include a -test.run pattern that filters out every single test. For example,
go test -test.bench MapTravel -test.run=thisexpressionwontmatchanytest
or
go test -bench MapTravel -run=^$

There is no flag you can provide, that will run only benchmarks (or only one benchmark). The only flags related to these are:
-bench regexp
Run benchmarks matching the regular expression.
By default, no benchmarks run. To run all benchmarks,
use '-bench .' or '-bench=.'.
-run regexp
Run only those tests and examples matching the regular
expression.
So if you want only to run one benchmark, you can do this:
go test -bench=nameOfYourBenchmark -run=^a
This will cause to run only tests that starts with a. And because each test should be named Test<something>, there will be no tests to run.
To run only benchmarks:
go test -bench=. -run=^a

Test only TestFuncOne
$>> go test -run TestFuncOne
stuff_to_test.go
TestFuncOne() {
}
TestFuncTwo() {
}

Related

Intellij IDEA skipping closing bracket on test coverage

I'm working on a Grails project. When I run unit tests with test coverage, it skips the enclosing brackets from code coverage. It's impossible to get 100% coverage. It happens for all methods.
I tried varying the coverage runner by using JaCoCo and Emma, but no success.
How can I fix this? Why do those brackets get added to total coverage anyway? It's super annoying.
EDIT: Actually it skips closing brackets right after return (or break) statements.

How to run google tests completely isolated from each other?

I want to use Address Sanitizer to detect runtime errors while unit tests are running. However, Address Sanitizer terminates the app when it finds the first error so I can't see an information of all found runtime errors in one report. It would be possible if I could run google tests isolated from each other, and termination of one test woudn't influence other ones.
You can run specific test cases in Google Test, one by one.
From: https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#running-a-subset-of-the-tests
By default, a Google Test program runs all tests the user has defined.
Sometimes, you want to run only a subset of the tests (e.g. for
debugging or quickly verifying a change). If you set the GTEST_FILTER
environment variable or the --gtest_filter flag to a filter string,
Google Test will only run the tests whose full names (in the form of
TestCaseName.TestName) match the filter.
The format of a filter is a ':'-separated list of wildcard patterns
(called the positive patterns) optionally followed by a '-' and
another ':'-separated pattern list (called the negative patterns). A
test matches the filter if and only if it matches any of the positive
patterns but does not match any of the negative patterns.
A pattern may contain '' (matches any string) or '?' (matches any
single character). For convenience, the filter '-NegativePatterns'
can be also written as '-NegativePatterns'.
For example:
./foo_test Has no flag, and thus runs all its tests.
./foo_test --gtest_filter=* Also runs everything, due to the single
match-everything * value.
./foo_test --gtest_filter=FooTest.* Runs everything in test case
FooTest.
./foo_test --gtest_filter=Null:Constructor Runs any test whose
full name contains either "Null" or "Constructor".
./foo_test --gtest_filter=-DeathTest. Runs all non-death tests.
./foo_test --gtest_filter=FooTest.*-FooTest.Bar Runs everything in
test case FooTest except FooTest.Bar.

how do i glob for s specific set of files

All example I have found of file globbing only show character substitution, not alternation. I have the desire to glob a subset of files and want to do so by name. It seems it should be possible. Below are two examples of what I have tried.
['test/functional/**/(login|account|productManagement)-spec.*']
['test/functional/**/\b(login|account|productManagement)\b-spec.*']
Of course, the following does work as expected.
['test/functional/**/(login|account|productManagement)-spec.*']
This is javascript code running in node.js.
If it is not clear, I am expecting to find:
login-spec.*
account-spec.*
productManagement-spec.*
We have a large suite of protractor tests. Some fail randomly when running locally. Okay, that should be fixed, but it is an environmental thing and the obvious fix will slow down the whole test run. It is far simpler to run them, identify those that have failed, then re-run.

How to skip test case in Robot framework using keyword

i'm trying to skip a specific test case using keyword, is there are any keyword to do that ? what i'm trying to do is to check if file name have "skip" word then i want to skip it. is there are any keyword like : Skip Test , Skip Execution If ...
#{regex}= Get Regexp Matches ${TEST NAME} Skip
${lenght}= Get Length ${regex}
Skip Execution If '${lenght}'>'0'
Nowadays, there are the following keywords added, in Robot Framework: Skip and Skip if
Ideally, tests that should not be run should be excluded from the run using tags or other means. Another option is to still run the tests, but simply check for your skip condition at the start of the test and pass the test without executing anything. There are two keywords, Builtin.Pass Execution and Builtin.Pass Execution If, useful for that.
http://robotframework.org/robotframework/latest/libraries/BuiltIn.html#Pass%20Execution%20If

When and why did the output of qr() change?

The output of perl's qr has changed, apparently sometime between versions 5.10.1 and 5.14.2, and the change is not documented--at least not fully.
To demonstrate the change, execute the following one-liner on each version:
perl -e 'print qr(foo)is."\n"'
Output from perl 5.10.1-17squeeze6 (Debian squeeze):
(?-xism:foo)
Output from perl 5.14.2-21+deb7u1 (Debian wheezy):
(?^:foo)
The perl documentation (perldoc perlop) says:
$rex = qr/my.STRING/is;
print $rex; # prints (?si-xm:my.STRING)
s/$rex/foo/;
which appears to no longer be true:
$ perl -e 'print qr/my.STRING/is."\n"'
(?^si:my.STRING)
I would like to know when this change occurred (which version of Perl, or supporting library or whatever).
Some background, in case it's relevant:
This change has caused a bunch of unit tests to fail. I need to decide if I should simply update the unit tests to reflect the new format, or make the tests dynamic enough to support both formats, etc. To make an informed decision, I would like to understand why the change took place. Knowing when and where it took place seems like the best place to start in that investigation.
It's documented in perl5140delta:
Regular Expressions
(?^...) construct signifies default modifiers
[...] Stringification of regular expressions now uses this notation. [...]
This change is likely to break code that compares stringified regular expressions with fixed strings containing ?-xism.
The function regexp_pattern can be used to parse the modifiers for normalisation purposes.
Part of the reason this was added, was that regular expressions were getting quite a few new modifiers.
Your example would actually produce something like this if that change didn't happen:
(?d-xismpaul:foo)
That also doesn't really express the modifiers in place.
d/u/l can only be added to a regex, not subtracted like i.
They are also mutually exclusive.
a/aa There are actually two levels for this modifier.
While work went underway adding these modifiers it was determined that this will break quite a few tests on CPAN modules.
Seeing as the tests were going to break anyway, it was agreed upon that there should be a way of specifying just use the defaults ((?^:…)).
That way, the tests wouldn't have to updated every time a new modifier was added.
To receive the stringified form of a regexp you can use Regexp::Parser and its qr method. Using this module you can not only test the representation of a regexp, but also walk a tree.