I am interested in using the differential code coverage functionality in ifort. The documentation appears to address this thoroughly but I have failed to apply it to my reduced example. Heres what I have:
program test
integer :: userinput
print *, 'enter 1 or 0'
read *, userinput
if (userinput.eq.1) then
print *, 'You have entered ONE'
else
print *, 'You have not entered ONE'
end if
end program test
A simple program that can take one of two paths. If the user enters 1 then it goes into the if ... then statement, if the user enters 0 then it goes into the else... statement.
The goal of differential code coverage (as stated by intel docs) is as follows:
compare the profiles from two runs of an application: a reference run
and a new run identifying the code that is covered by the new run but
not covered by the reference run
So if we take a reference run where the user enters 0 and a new run where the user enters 1, the differential code coverage should be able to identify that the new run covers the if statement whereas the reference run does not (reference run goes into the else statement). I followed the docs as closely as possible. The source file is called test.f90. Here are the compile lines i'm using:
ifort test.f90 /Qcov-gen
Which generates PGOPTI.SPI, PGOPTI, test.exe and test.obj. I then run the executable and enter 0, I get the correct message "You have not entered ONE". This causes a .dyn file to be created (due to the Qcov-gen option). I then do the following:
profmerge
Which generates additional files pgopti.dpi, pgopti.dpi.lock. At this point I think I have enough material to generate my reference data. This I attempt using the following:
codecov -prj Project_Name -dpi pgopti.dpi -ref pgopti.dpi
Which generates html files similar to the ones displayed when code coverage is run in Visual Studio for Intel Fortran. I also get 100% code coverage which seems incorrect. The docs then show this command:
codecov -prj Project_Name -spi pgopti.spi -dpi pgopti.dpi
Which does not appear to provide an opportunity for a new run.
Could someone please explain how to do a simple differential code coverage on this particular example? I'm eventually trying to extrapolate this to a larger project but I'm trying to take baby steps to get there.
Related
I have a testbench to test my VHDL device (DUT), but part of the DUT debug output is an ASSERT/REPORT message to the console, which I would like to check for correctness but I can't change the DUT. The only way I can think of is to post-process the output log file.
Is there any way of capturing the console output in the testbench, so I can check the DUT output directly?
I do this as part of the testbench. However, rather than Assert, I use OSVVM alerts, log, and print. OSVVM is both at osvvm.org and github.
Rather than Assert, I use AffirmIf for self-checking/result checking. I use AlertIf for parameter checking.
Step 1 is get OSVVM. Once you have the code, compile it using the script. In either Mentor or Aldec, run the script by doing:
vlib osvvm
vmap osvvm osvvm
do $PATH_TO_OSVVM/osvvm.do $PATH_TO_OSVVM
Use VHDL-2008 and include all of OSVVM in your program by doing:
library osvvm;
context osvvm.OsvvmContext;
Then rather than:
assert Data /= expected report "..." severity error;
Do:
AffirmIf(Data = Expected, "...") ;
Both assert and AffirmIf/AlertIf print. However, the advantage to AffirmIf/AlertIf is that internally it keeps a count of the errors and you can get a pass fail at the end of your test by doing:
ReportAlerts;
The next advantage of OSVVM AffirmIf/AlertIf/Log/Print is that if you want the results in a file, you simply do:
TranscriptOpen("./results/Test1.txt");
If you want to both print to the screen and a file, also do:
SetTranscriptMirror(TRUE);
That ought get you started. I will leave the rest to the user guides. Start by looking at both the AlertLog package user guide and the transcript package user guide.
BACKGROUND
We have testers for our embedded GUI product and when a tester declares "test failed", sometimes it's hard for us developers to reproduce the exact problem because we don't have the exact trace of what happened.
We do currently have a logging framework but us devs have to manually input those logging statements in the code which is fine . . . except when a hard-to-reproduce bug occurs and we didn't have a log statement at the 'right' location and then when we re-build, re-run the test with the same steps, we get a different result.
ISSUE
We would like a solution wherein the compiler produces extra instrumentation code that allows us to see the exact sequence of events including, at the minimum:
function enter/exit (already provided by -finstrument-functions)
control-flow statement enter i.e. entering if/else, which case statement we jumped to
The log would look like this:
int main() entered
if-line 5 entered
else-line 10 entered
void EventLoop() entered
. . .
Some additional nice-to-haves are
Parameter values on function entry AND exit (for pass-by-reference types)
Function return value
QUESTION
Are there any gcc tools or even paid tools that can do this instrumentation automatically?
You can either use gdb for that, and you can automate that (I've got a tool for that in the works, you find it here or you can try to use gcov.
The gcov implementation is so, that it loads the latest gcov data when you start the program. But: you can manually dump and load the data. If you call __gcov_flush it will dump the current data and reset the current state. However: if you do that several times it will always merge the data, so you'd also need to rename the gcov data file then.
I'm having some trouble getting Matlab to run an executable file. Essentially, I have a C++ code that does some calculations and outputs these calculations to a text file; then, Matlab reads these text files and uses the calculations to makes plots and stuff.
I've been trying to get Matlab to run the C++ exe file so that, when it runs it, the output files are automatically generated and Matlab can start doing its stuff. This just allows the user to run the program quicker. I am using the system() command Like so:
system('MyCppProgram.exe');
However, when I run that, although everything compiles, nothing is outputted from CPP and I even get back something that says "ans = -1" and I have no idea what that means.
Any help would be greatly appreciated. Thank you!
Update: Result from test command.
[status, cmdout] = system('MyCppProgram.exe', '-echo');
status = -1
cmdout = ''
I am using MinGW (GCC) as a C++ compiler within my application. I have set it to redirect the output of its command line process to my app. Now, suppose I have the following simple C++ code:
int n = 5;
if (n == 6) cout << "YES";
else cout << "NO";
Is there a way to tell what line(s) of code were actually hit during execution of the application? Is there a command I can send to MinGW (GCC) process which, for the given example, would output 1 and 3, as those were the lines hit. And also, in case of a line inside a "for" loop, to tell how many times that statement was actually hit?
And, if not possible, what would be the best approach to having this information? Developing my own compiler or...? Thanks in advance
EDIT: Can someone provide a snippet of commands (in Windows) to be used in order to create a coverage-enabled GCC exe file?
"Is there a way to tell what line(s) of code were actually hit during execution of the application?"
Yes. It's an intrinsic GCC feature. You'll need to compile and link your code with the --coverage, -lgcov or -fprofile-arcs options set.
The gcov tool can be used to consolidate and interpret the actual informations gathered during program runs, that were instrumented with --coverage.
A very good tool to produce browsable consolidated and fairly visualized covearage information from gcov outputs is lcov.
Since you're using mingw you should be able to use gcov: https://gcc.gnu.org/onlinedocs/gcc/Gcov.html
The go test command has support for the -c flag, described as follows:
-c Compile the test binary to pkg.test but do not run it.
(Where pkg is the last element of the package's import path.)
As far as I understand, generating a binary like this is the way to run it interactively using GDB. However, since the test binary is created by combining the source and test files temporarily in some /tmp/ directory, this is what happens when I run list in gdb:
Loading Go Runtime support.
(gdb) list
42 github.com/<username>/<project>/_test/_testmain.go: No such file or directory.
This means I cannot happily inspect the Go source code in GDB like I'm used to. I know it is possible to force the temporary directory to stay by passing the -work flag to the go test command, but then it is still a huge hassle since the binary is not created in that directory and such. I was wondering if anyone found a clean solution to this problem.
Go 1.5 has been released, and there is still no officially sanctioned Go debugger. I haven't had much success using GDB for effectively debugging Go programs or test binaries. However, I have had success using Delve, a non-official debugger that is still undergoing development: https://github.com/derekparker/delve
To run your test code in the debugger, simply install delve:
go get -u github.com/derekparker/delve/cmd/dlv
... and then start the tests in the debugger from within your workspace:
dlv test
From the debugger prompt, you can single-step, set breakpoints, etc.
Give it a whirl!
Unfortunately, this appears to be a known issue that's not going to be fixed. See this discussion:
https://groups.google.com/forum/#!topic/golang-nuts/nIA09gp3eNU
I've seen two solutions to this problem.
1) create a .gdbinit file with a set substitute-path command to
redirect gdb to the actual location of the source. This file could be
generated by the go tool but you'd risk overwriting someone's custom
.gdbinit file and would tie the go tool to gdb which seems like a bad
idea.
2) Replace the source file paths in the executable (which are pointing
to /tmp/...) with the location they reside on disk. This is
straightforward if the real path is shorter then the /tmp/... path.
This would likely require additional support from the compiler /
linker to make this solution more generic.
It spawned this issue on the Go Google Code issue tracker, to which the decision ended up being:
https://code.google.com/p/go/issues/detail?id=2881
This is annoying, but it is the least of many annoying possibilities.
As a rule, the go tool should not be scribbling in the source
directories, which might not even be writable, and it shouldn't be
leaving files elsewhere after it exits. There is next to nothing
interesting in _testmain.go. People testing with gdb can break on
testing.Main instead.
Russ
Status: Unfortunate
So, in short, it sucks, and while you can work around it and GDB a test executable, the development team is unlikely to make it as easy as it could be for you.
I'm still new to the golang game but for what it's worth basic debugging seems to work.
The list command you're trying to work can be used so long as you're already at a breakpoint somewhere in your code. For example:
(gdb) b aws.go:54
Breakpoint 1 at 0x61841: file /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go, line 54.
(gdb) r
Starting program: /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.test
[snip: some arnings about BinaryCache]
Breakpoint 1, github.com/stellar/deliverator/aws.imageIsNewer (latest=0xc2081fe2d0, ami=0xc2081fe3c0, ~r2=false)
at /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go:54
54 layout := "2006-01-02T15:04:05.000Z"
(gdb) list
49 func imageIsNewer(latest *ec2.Image, ami *ec2.Image) bool {
50 if latest == nil {
51 return true
52 }
53
54 layout := "2006-01-02T15:04:05.000Z"
55
56 amiCreationTime, amiErr := time.Parse(layout, *ami.CreationDate)
57 if amiErr != nil {
58 panic(amiErr)
This is just after running the following in the aws subdir of my project:
go test -c
gdb aws.test
As an additional caveat, it does seem very selective about where breakpoints can be placed. Seems like it has to be an expression but that conclusion is only via experimentation.
If you're willing to use tools besides GDB, check out godebug. To use it, first install with:
go get github.com/mailgun/godebug
Next, insert a breakpoint somewhere by adding the following statement to your code:
_ = "breakpoint"
Now run your tests with the godebug test command.
godebug test
It supports many of the parameters from the go test command.
-test.bench string
regular expression per path component to select benchmarks to run
-test.benchmem
print memory allocations for benchmarks
-test.benchtime duration
approximate run time for each benchmark (default 1s)
-test.blockprofile string
write a goroutine blocking profile to the named file after execution
-test.blockprofilerate int
if >= 0, calls runtime.SetBlockProfileRate() (default 1)
-test.count n
run tests and benchmarks n times (default 1)
-test.coverprofile string
write a coverage profile to the named file after execution
-test.cpu string
comma-separated list of number of CPUs to use for each test
-test.cpuprofile string
write a cpu profile to the named file during execution
-test.memprofile string
write a memory profile to the named file after execution
-test.memprofilerate int
if >=0, sets runtime.MemProfileRate
-test.outputdir string
directory in which to write profiles
-test.parallel int
maximum test parallelism (default 4)
-test.run string
regular expression to select tests and examples to run
-test.short
run smaller test suite to save time
-test.timeout duration
if positive, sets an aggregate time limit for all tests
-test.trace string
write an execution trace to the named file after execution
-test.v
verbose: print additional output