I've been trying to verify the code coverage of an exe file. To achieve this, I'm trying to instrument the exe file (not dll) and then start/stop the coverage.
Every tutorial describes about the instrumentation of dll only. Is it possible to achieve it for exe? If yes, what are the special options I need to give.
Here are the commands, I've been trying for exe:
vsinstr -coverage Test.exe
Returns success message and creates another copy of exe
vsperfcmd -start:coverage -output:Result.coverage
vsperfcmd -shutdown
It creates the output file successfully. But when the file opened in visual studio shows the error message:
"Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings."
Please help
Your first command starts the profiler service and your second one shuts it down (and causes it to write its output). What you're missing is the -attach command.
vsperfcmd -attach:PID
where PID is your process's numerical ID.
Once you're attached to a process, it will start collecting coverage data. Do whatever you need to exercise your code, then run the shutdown command to get your output.
Related
How can I set the path to my external binaries during test discovery in visual studio's Test Explorer? After that how to make sure, it uses the correct paths?
I use windows 10 and VS 2019. I have a solution that builds some binaries and some tests into different folders. Also, I have some 3rd party dependencies, each in its own folder.
Something like:
solutionDir/
-ownBinaries/
-testBinaries/
-externalBinaries/
I'd like to use the Test Explorer to run my tests. For this purpose, I use a .runsettings file. I installed Google Test adapter via NuGet (later it will run on CI, so this is the only option). The automatic runsetting discovery is disabled, and this file is selected as the runsettings file. It overrides the workingDir to my ownBinaries folder, and extend the PATH enviroment variable with the externalBinaries. The relevant parts are:
<SolutionSettings>
<Settings>
<AdditionalTestExecutionParam>-testdirectory=$(SolutionDir)</AdditionalTestExecutionParam>
<WorkingDir>$(SolutionDir)ownBinaries</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
This is works fine, after my tests are discovered, but I have problems when it tries to discover my tests.
I use google test and c++, so the test discovery tries to run those tests with the --gtest-list-tests argument, then populate the view with the test name, case, etc. The binaries are just fine, builds without error, I can run them from the debugger, and they produce the output I want.
But the test explorer won't show them, because it doesn't set the externalBinaries path.
This is what lead me to this situation.
First I copied every binaries next to my test exe, namely into the testBinaries folder. Then, I could run it in the cmd with the --gtest-list-tests argument. Everything was fine, all my test names showed up. Started VS, and Test Explorer discovered all my tests, it was able to run them.
Then I done a clean build, so the external stuff deleted from the testBin folder. The Test Explorer cached the test names, so it was able to run them.
Restart VS. Test Explorer tries to discover my tests. but it fails whit this helping message: (removed date+time)
Google Test Adapter: Test discovery starting...
Failed to run test executable 'D:\MySolution\testBinaries\SBCUnitTest.exe': One or more errors occurred.
Check out Google Test Adapter's trouble shooting section at https://github.com/csoltenborn/GoogleTestAdapter#trouble_shooting
In particular: launch command prompt, change into directory '..\ownBinaries', and execute the following command to make sure your tests can be run in general.
D:\MySolution\testBinaries\SBCUnitTest.exe --gtest_list_tests -testdirectory=
Found 0 tests in executable D:\MySolution\testBinaries\SBCUnitTest.exe
Test discovery completed, overall duration: 00:00:00.3022924
Have you noticed that -testDirectory= is empty despite it is set in the runsettings file?
I'm completely lost how I can proceed with it. This workaround is quite heavy to copy all files, then delete all but the test binaries each time when I start VS.
Here is the link for the Troubleshooting section mentioned in the error message.
I've read through the readme file on github, also the runsetting docs on Microsoft's website.
Edit
I made progress with the VsTest.console.exe, I can successfully run all my tests with the proper arguments as below:
& "VSTest.console.exe" *_uTest.exe /Settings:..\MySolution.gta.runsettings /TestAdapterPath:"..\packages\GoogleTestAdapter.0.18.0\build\_common\"
I use the same *.runsettings and *.gta_settings_helper files. Those files are used to get absolute paths for the dependencies. I could run this from different folders, but then I had to adjust the arguments (test discovery pattern, relative path to runsettings, and relative path to GTA).
Great news, that it successfully runs on Azure (it uses vstest.console).
Edit 2
Tried to merge the workingDir and pathExtension nodes, so only one needed (the pathExtension). No success.
Tried to install Test adapter for google test in the VS installer, delete the runsetting file, and set the properties in VS->Tools->Options then Test adapter for google test. Even the example pathExtension didn't worked for me.
Found the extended logs under %AppData%/Local/Temp/TestAdapter/someNumber/*.txt and in that log I've found one line as the runsettings file. I paste here the formatted version of the log
<RunSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
<WorkingDir>$(SolutionDir)</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
</GoogleTestAdapterSettings>
</GoogleTestAdapterSettings>
</RunSettings>
Does anybody know why is there an empty google test adapter setting? Where does it comes from? I think this is overwrites my settings.
It turned out, before first run the relative paths are not known.
Trivial solution
Add the full path to the PATH Extension under Visual Studio -> Options -> Test Adapter for Google Test settings. Meanwhile the custom *.runsetting file is not selected.
Using this method all my tests are discoverable, but it is a manual setting for each repo cloned.
This is driving me crazy, I must admit. After finally being able to successfully compile two functions I need to process voice files, from C/C++ code that I downloaded from a trustworthy online repository (code that had been thoroughly tested in Linux), I am now struggling to launch those files from Matlab...
When I type the following command in cmd (dos)
Analysis b2.wav config_default
it works, no problem (see here Works).
Then, I build the exact same command into a string and feed it to the "system" Matlab function. Then the code crashes... (see here Fails) I've tried with full paths (c:\b2.wav, etc) but still does not work...
Any ideas as to why this might be happening?
Your image shows that the program Analysis stopped unexpectedly.
It might be a lot of reasons why, so let's go step by step:
1) Try executing Analysis from Terminal and passing wrong parameters (a file that doesn't exist, only one param (missing the config_defalut), no parameters at all, three parameters, etc...)
Can you make the program crash from terminal by passing wrong params?
2) Try creating the command first, checking that it's correct (\b is actually \b instead of a string modifier)
command_to_be_run = 'C:\Analysis C:\b2.wav C:\config_default'
disp(command_to_be_run) % is it showing exacly what you want?
system(command_to_be_run); % if so, run it.
3) Try creating a dummy executable dummy.exe in C that accepts two parameters and prints the received parameters (keep it super simple, just printing). Call it from Terminal. Does it work? Call it from Matlba. Does it Work?
With this 3 tests you can considerably narrow down where your error comes from.
By the way, is "config_default" a file or just a string that tells analysis how to behave? In some examples you treat it as a file, in others as a parameter without path.
Based on what's been tried so far and the outputs, here's my theory:
Premise: Analysis.exe came from code that's well tested in Linux. It works in Windows command line when run from the same directory where both it and the target file reside. But it stops working from Matlab console.
Assertion 1: Matlab console does not operate within the context of the directory where the binary is but rather within the Matlab directory. As such, Analysis.exe will try to find the target from the Matlab directory.
Validation for Assertion 1: Try putting the binary and the target wav in the Matlab directory. Then run system with the binary and target specified just by name (no path).
Assertion 2: If the file's full path is specified to address this issue, it still doesn't work. This may be because the code assumed a Linux file system where the delimiter is "/" rather than "\".
Validation for Assertion 2: Run with paths specified from the command line while in a diferent directory to see if it fails or not.
Possible Solution 1: Add the directory where both Analysis.exe and the target are into the Matlab path: (1) On the Home tab, in the Environment section, click Set Path. Add the path there. (2) addpath (folderName1,...,folderNameN) adds the specified folders to the top of the search path for the current MATLAB session. -> Then run the system command without the full paths.
Possible Solution 2: Add the directory where both Analysis.exe and the target are into the Windows environment path. Then run the system command without the full paths.
EDIT: Possible hackish solution - Create a batch file where: (1) you would cd to the directory where Analysis.exe and the target wav are; and (2) do a Matlab system call to the batch file.
EDIT 2: Possible experiment to validate assertion 2.
The IDE I'm using is VS2010 for writing C++
I want to execute the command cmd C:\utilities\unix\tail.exe -f -n15 $(ProjectDir)Log.txt every time the program I'm coding is run from within the IDE. (This command should open a console to track changes made to the file Log.txt)
There are ways to make a command run every time the program is built, but I can't find a way to make a command run whenever the program itself is run, even if it's already built. Where or how might I be able to set that kind of thing up?
I've tried putting $(TargetPath) & C:\utilities\unix\tail.exe -f -n15 $(ProjectDir)Log.txt into the project's Properties->Debugging->Command (TargetPath is the full name of the debug executable) but VS reads the entire thing as a filename and gets confused.
You can create a file run.cmd for example next to the vcxproj file, which would contain:
%1
C:\utilities\unix\tail.exe -f -n15 %2Log.txt
And then in Properties->Debugging->Command you write:
$(ProjectDir)\run.cmd
and in Command Arguments you write:
"$(TargetPath)" "$(ProjectDir)"
I may have misspelled the macros, but you get the idea: it executes first your program and then whatever you want.
Edit: Unfortunately it works only if you start without debugging (Ctrl+F5), because otherwise the debugger tries to attach to run.cmd and complains that the format is unsupported.
I have a process which calls CreateProcess. It appears that CreateProcess returns nonzero indicating success. However, the HANDLE to the process then gets immediately set, indicating the process has exited. When I call GetExitCodeProcess, STATUS_DLL_NOT_FOUND is then returned.
I understand that a DLL is missing. I even know exactly which one. However, what I don't understand is how to figure that out programmatically.
I noticed that Windows will present a dialog saying that the process failed to start because it couldn't find the specified DLL (screenshot: http://www.mediafire.com/view/?kd9ddq0e2dlvlb9 ). In the dialog, Windows specifies which DLL is missing. However, I find no way to get that information myself programmatically.
If a process fails to start and would return STATUS_DLL_NOT_FOUND, how do I programmatically retrieve the library name to which the target process was linked which couldn't be found? That way I can automatically record in an error report what DLL appears to be missing or corrupt in a given installation.
CreateProcess returns 0 indicating success.
CreateProcess() returns a BOOL, where 0 is FALSE, aka failure not success.
If a process fails to start and would return STATUS_DLL_NOT_FOUND, how do I programmatically retrieve the library name to which the target process was linked which couldn't be found?
Unfortunately, there is no API for that. Your only option would be to manually access and enumerate the executable's IMPORTS table to find out what DLLs it uses, and then recursively access and enumerate their IMPORTS tables, manually checking every DLL reference you find to see whether that DLL file exists on the OS's search path or not.
If the dll is statically linked you can walk the iat and see if the dll exists. If the dll is dynamically loaded then starting the process suspended and hooking LoadLibrary (or instead of hooking emulate a debugger) is the only way I see.
The best way is to use loader snaps. Basically you use gflags.exe (which is included with windbg) to enable loader snaps; then, run the process with the debugger attached. Loader snaps will enable the loader to print out dbg messages of the process and it will print the failures.
gflags.exe -i yourcode.exe +sls
windbg yourcode.exe
I know this is not a "programmatic" way to find out the problem, but what the loader does is complicated, and you don't really want to be redoing its logic to find the failure. That is why loader snaps were invented.
The very hard way would be: Parsing the .EXE and .DLL files and create the dependency tree of .DLL files.
I don't think there is a way to get a list of DLL files that are missing: When Windows finds one missing DLL file it stops loading so if one DLL file is missing you won't find out if more DLL files are missing.
Another problem you could have is that old DLL versions could have missing "exports" (functions). This is even harder to detect than the dependency tree.
Just since this is somehow the top stackoverflow result on Google for "STATUS_DLL_NOT_FOUND". How to trace and solve any random occurence:
Download SysInternals procmon64.exe (or just the entire set). After startup immediately hit the looking glass 'Stop capture' button (Ctrl+E). And 'Clear' (Ctrl+X).
Set filters for:
'Process name' is to whatever the mentioned process name was (for me it was 'build-script-build.exe') [Add]
'Result' is not 'SUCCESS' [Add]
'Path' ends with '.dll' [Add] [OK]
Start capture again (Ctrl+E).
Run the thing that had a problem again (for me: build cargo). Google for the last listed DLL file.
For me that was VCRUNTIME140.dll, so I installed the VC++ 2015 to 2019 redistributable.
ProcMon is kind of like unix strace.
I have a batch file which is calling CMake which also does some functionality
I want to call this batch file for Build.
If for some reason, CMake fails and throws error the same is not reported as failure in RTC. If my understanding is correct RTC is calling the Batch file and the Batch file calls CMake. The execution of batch file is successful and hence it is reported as success.
But I want the RTC to report CMake is failed which is called via Batch files
How can i achieve this?
I was looking at creating Ant tasks but don't have one proper example
thank you
Karthik
You will want to use the ANT exec task.
http://ant.apache.org/manual/Tasks/exec.html
There is an example in the documentation of calling a .bat file. You will also want to use the failonerror ="true" attribute to ensure you fail the RTC build if the bat file fails. Additionally you need to ensure that your bat file is indeed failing (returning a non-zero return code) if the CMake command fails.