how to convert address to lines and file names on windows with PDB files? - visual-studio-2017

I have PDB files and callstack like this
#1 - wseclient.dll+0x121170
#2 - wseclient.dll+0x120024
#3 - wseclient.dll+0x25fbf
I used to use add2line.exe to convert address to lines, but it doesn't work on vs2019, does anybody know is there any other way to convert address on callstack?

Setting up an example
Some sample code to be compiled with Visual Studio 2019:
#include <iostream>
class Test {
public:
__declspec(noinline) static int one()
{
throw std::exception("maximize debugging fun");
}
};
__declspec(noinline) int main()
{
return Test::one();
}
This will create the following call stack when being run in WinDbg:
0:000> k
# ChildEBP RetAddr
...
02 00affa6c 00661316 CallStackDecodingExample!Test::one+0x1d [C:\...\CallStackDecodingExample.cpp # 6]
...
And we can use some calculations to get the numbers back and forth:
0:000> ? CallStackDecodingExample!Test::one+0x1d
Evaluate expression: 6689037 = 0066110d
0:000> ? CallStackDecodingExample
Evaluate expression: 6684672 = 00660000
0:000> ? CallStackDecodingExample+0x110d
Evaluate expression: 6689037 = 0066110d
0:000> ln 0066110d
[C:\...\CallStackDecodingExample.cpp # 6]
(006610f0) CallStackDecodingExample!Test::one+0x1d |
(00661110) CallStackDecodingExample!main
Given the debugger was able to resolve the PDBs correctly, we now have the expected result. Let's try to get this without a debugging session, i.e. neither with live debugging nor with crash dump analysis, but by PDBs + text input.
Getting the address from a DLL + PDB
In WinDbg, use "Open dump file", although you don't have a crash dump file. Instead, open the DLL (wseclient.dll or CallStackDecodingExample.exe for this example).
Then use ln:
0:000> ln CallStackDecodingExample+0x110d
[C:\...\CallStackDecodingExample.cpp # 6]
(004010f0) CallStackDecodingExample!Test::one+0x1d
| (00401110) CallStackDecodingExample!main

Related

Why does '$' for funcname in git log -L cause an infinite search?

You can search for a filename and function name in git log with git log -L :funcname:filename.
I ran into an issue where we had been running this search programmatically and the funcname was set to '$', which caused an endless search. (e.g. git log -L :$:somefile.py)
'$' means end of string in regex, but why does this cause an endless search loop when other regex characters like '^' or '?' don't? What unique effects does the '$' character have?
I was able to reproduce this locally and step through the code using gdb.
This looks like a bug in the find_funcname_match_regexp function. It eventually ends up matching $ against the empty string, which matches successfully but causes no changes in the pointers used to mark the position in the file, resulting in an infinite loop.
Here's a walk through of the reproducer. In this example, we're running git log against the file main.go which has the following content:
package main
import "fmt"
func main() {
fmt.Println("example repository for demonstrating git log bug")
}
Start gdb and set a breakpoint at the beginning of the while loop in line-range.c. Arrange to print the value of start after each break:
(gdb) break line-range.c:140
Breakpoint 1 at 0x5a7a89: file line-range.c, line 140.
(gdb) commands
Type commands for breakpoint(s) 1, one per line.
End with a line saying just "end".
>p start
>end
(gdb)
Run git log -L :$:main.go under the control of gdb:
(gdb) run log -L :$:main.go
Starting program: /home/lars/src/git/git log -L :$:main.go
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[Detaching after fork from child process 2469232]
Breakpoint 1, find_funcname_matching_regexp (xecfg=0x0, start=0x8477c0 "package main\n\nimport \"fmt\"\n\nfunc main() {\n\tfmt.Println(\"example repository for demonstrating git log bug\")\n}\n", regexp=0x7fffffffc020) at line-range.c:140
140 reg_error = regexec(regexp, start, 1, match, 0);
$1 = 0x8477c0 "package main\n\nimport \"fmt\"\n\nfunc main() {\n\tfmt.Println(\"example repository for demonstrating git log bug\")\n}\n"
In this output, we can see that start is pointing at the beginning of the file:
$1 = 0x8477c0 "package main\n\nimport \"fmt\"\n\nfunc main() {\n\tfmt.Println(\"example repository for demonstrating git log bug\")\n}\n"
Skip a few iterations:
(gdb) c 7
Will ignore next 6 crossings of breakpoint 1. Continuing.
Breakpoint 1, find_funcname_matching_regexp (xecfg=0x0, start=0x84782b "}\n", regexp=0x7fffffffc020) at line-range.c:140
140 reg_error = regexec(regexp, start, 1, match, 0);
$2 = 0x84782b "}\n"
Here we see that start now points at the last line in the file.
Watch what happens if we iterate a few more times:
(gdb) c
Continuing.
Breakpoint 1, find_funcname_matching_regexp (xecfg=0x0, start=0x84782d "", regexp=0x7fffffffc020) at line-range.c:140
140 reg_error = regexec(regexp, start, 1, match, 0);
$3 = 0x84782d ""
(gdb) c
Continuing.
Breakpoint 1, find_funcname_matching_regexp (xecfg=0x0, start=0x84782d "", regexp=0x7fffffffc020) at line-range.c:140
140 reg_error = regexec(regexp, start, 1, match, 0);
$4 = 0x84782d ""
(gdb) c
Continuing.
Breakpoint 1, find_funcname_matching_regexp (xecfg=0x0, start=0x84782d "", regexp=0x7fffffffc020) at line-range.c:140
140 reg_error = regexec(regexp, start, 1, match, 0);
$5 = 0x84782d ""
After one more iteration of the loop, start now points at the empty string. It keeps this value in every subsequent iteration, and we never break out of the while loop.
I've submitted a patch to git that should correct this behavior.
You can follow the discussion there to see if they like my patch or if they decide there is a more appropriate way to resolve the problem.
With the patched version of the code, we see the following behavior instead:
$ git log -L :$:main.go
fatal: -L parameter '$' starting at line 1: no match

Python3 C++ Module - Exception - GIL not held

I have created a minimal reproduction of an issue I am having, trying to create a C++ based Python3 module. The build environment is CMake, Visual Studio Pro 2019, WinSDK 10.0.18362, Python 3.9.4. When executing:
python3 -c "import pymod"
I will get an exception in release mode with a NULL access. In debug Python, I get additional information.
My CMakeLists.txt
cmake_minimum_required(VERSION 3.15)
project(pymod_minimal_repo_example)
find_package(Python3 COMPONENTS Interpreter Development)
add_library(pymod SHARED)
target_sources(pymod PRIVATE pymodmodule.cpp)
set_target_properties(pymod PROPERTIES SUFFIX ".pyd")
target_compile_options(pymod PRIVATE /Zi)
target_link_options(pymod PRIVATE /DEBUG:FULL)
target_link_libraries(pymod PRIVATE ${Python3_LIBRARIES})
target_include_directories(pymod PRIVATE ${Python3_INCLUDE_DIRS})
Following this python reference: https://docs.python.org/3.9/extending/extending.html#a-simple-example I created the following:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
static struct PyModuleDef pymodmodule = {
PyModuleDef_HEAD_INIT, // m_base
"pymod", // m_name
NULL, // m_doc
-1, // m_size - submod not support must be static struct
NULL, // m_methods - no functions present
NULL, // m_slots - must be NULL
NULL, // m_traverse - not needed
NULL, // m_clear - not needed
NULL // m_free - not needed
};
PyMODINIT_FUNC PyInit_pymod(void) {
return PyModule_Create(&pymodmodule);
}
It is a functionally void module which does nothing, but should still run. Note that having m_methods defined, produces the same failure.
When the failure occurs, the following is output to the console:
E:\projects\pymod\build\Debug>python -c "import pymod"
Fatal Python error: _PyInterpreterState_GET: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: unknown
WinDbg jit debugger then catches the issue. A partial call stack shows my PyInit_pymod is called and when creating the python module, it cascades to a failure:
0:000> k
# Child-SP RetAddr Call Site
00 000000dc`abfed4a8 00007fff`71d51385 KERNELBASE!wil::details::DebugBreak+0x2
01 000000dc`abfed4b0 00007fff`71d511a8 python39_d!fatal_error_exit+0x15 [D:\a\1\s\Python\pylifecycle.c # 2201]
02 000000dc`abfed4e0 00007fff`71d4ea98 python39_d!fatal_error+0x1b8 [D:\a\1\s\Python\pylifecycle.c # 2286]
03 000000dc`abfed540 00007fff`71cbe730 python39_d!_Py_FatalErrorFunc+0x38 [D:\a\1\s\Python\pylifecycle.c # 2302]
04 000000dc`abfed580 00007fff`71aa8044 python39_d!_Py_FatalError_TstateNULL+0x10 [D:\a\1\s\Python\ceval.c # 251]
05 000000dc`abfed5b0 00007fff`71aa6cf2 python39_d!_PyInterpreterState_GET+0x34 [D:\a\1\s\Include\internal\pycore_pystate.h # 105]
06 000000dc`abfed5f0 00007fff`d0be13e7 python39_d!PyModule_Create2+0x12 [D:\a\1\s\Objects\moduleobject.c # 168]
>>>> 07 000000dc`abfed620 00007fff`751165dc pymod!PyInit_pymod+0x27 [E:\projects\pymod\pymodmodule.cpp # 19]
08 000000dc`abfed660 00007fff`751167ee python39!_PyImport_LoadDynamicModuleWithSpec+0x104 [C:\A\34\s\Python\importdl.c # 165]
09 000000dc`abfed6d0 00007fff`75116749 python39!_imp_create_dynamic_impl+0x86 [C:\A\34\s\Python\import.c # 2299]
0a 000000dc`abfed700 00007fff`750cb94b python39!_imp_create_dynamic+0x39 [C:\A\34\s\Python\clinic\import.c.h # 330]
0b 000000dc`abfed730 00007fff`750ad500 python39!cfunction_vectorcall_FASTCALL+0x9b [C:\A\34\s\Objects\methodobject.c # 426]
0c 000000dc`abfed7a0 00007fff`750ad2ef python39!PyVectorcall_Call+0x5c [C:\A\34\s\Objects\call.c # 248]
0d 000000dc`abfed800 00007fff`750ad418 python39!_PyObject_Call+0x4f [C:\A\34\s\Objects\call.c # 287]
0e (Inline Function) --------`-------- python39!PyObject_Call+0xc [C:\A\34\s\Objects\call.c # 293]
0f 000000dc`abfed830 00007fff`7508c65f python39!do_call_core+0xb8 [C:\A\34\s\Python\ceval.c # 5092]
10 000000dc`abfed880 00007fff`75083963 python39!_PyEval_EvalFrameDefault+0x5d6f [C:\A\34\s\Python\ceval.c # 3581]
11 (Inline Function) --------`-------- python39!_PyEval_EvalFrame+0x13 [C:\A\34\s\Include\internal\pycore_ceval.h # 40]
12 000000dc`abfedbb0 00007fff`750855a7 python39!_PyEval_EvalCode+0x2b3 [C:\A\34\s\Python\ceval.c # 4327]
13 000000dc`abfedc80 00007fff`7508823d python39!_PyFunction_Vectorcall+0x257 [C:\A\34\s\Objects\call.c # 396]
14 000000dc`abfedd80 00007fff`7508812f python39!_PyEval_EvalFrameDefault+0x194d [C:\A\34\s\Python\ceval.c # 3487]
15 000000dc`abfee0b0 00007fff`750888e5 python39!_PyEval_EvalFrameDefault+0x183f [C:\A\34\s\Python\ceval.c # 3504]
16 000000dc`abfee3e0 00007fff`750888e5 python39!_PyEval_EvalFrameDefault+0x1ff5 [C:\A\34\s\Python\ceval.c # 3518]
17 000000dc`abfee710 00007fff`750888e5 python39!_PyEval_EvalFrameDefault+0x1ff5 [C:\A\34\s\Python\ceval.c # 3518]
18 000000dc`abfeea40 00007fff`750854c4 python39!_PyEval_EvalFrameDefault+0x1ff5 [C:\A\34\s\Python\ceval.c # 3518]
I cannot find any information on the console message, nor the int3. I have tried full purge and reinstall/update of Python.
Can anyone offer some direction to help, or know the cause?
Edit: Modified that PyModule_pymod function to display Pre and Post msg to console. Debug build exceptions, Release appears not to:
PyMODINIT_FUNC PyInit_pymod(void) {
printf("Pre-PyModule_Create\n");
PyObject *obj = PyModule_Create(&pymodmodule);
printf("Post-PyModule_Create\n");
return obj;
}
Release:
E:\projects\pymod\build\Release>python -c "import pymod"
Pre-PyModule_Create
Post-PyModule_Create
Debug:
E:\projects\pymod\build\Debug>python -c "import pymod"
Pre-PyModule_Create
Fatal Python error: _PyInterpreterState_GET: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: unknown
I ran into the same issue as you, and while debugging TLS structures, I noticed that Visual Studio had loaded python310.dll instead of python310_d.dll.
To make a long story short, when you define the _DEBUG macro, Python.h will use python310_d.lib, so you need to run the debug version of Python:
python_d -c "import pymod"
I was confused further by the fact that I did not have this issue while using pybind11. As it turned out, pybind11 undefines the _DEBUG macro while including Python.h, so it is always using python310.lib instead of python310_d.lib, even for debugging builds.
As I do not see an option to use the python_d interpreter in Visual Studio, I ended up using the release library for debugging builds using the following code:
#if defined(_MSC_VER) && defined(_DEBUG)
#undef _DEBUG
#include <Python.h>
#define _DEBUG 1
#else
#include <Python.h>
#endif
As a sidenote, when you link against python310_d.dll, you will have to name your debug module pymod_d.pyd instead of pymod.pyd.

Access violation using different c++ runtime build VS 2017

I got an c0000005 memory exception when my program use a newer version of c++ runtime, specifically of msvcp140.dll :
14.16.27012.6 the program works
14.24.28127.4 the program crashes.
My application uses signalrclient library for connecting to our web service. When there’s a problem with the connection, the program crashes with an access violation.
I got the stack from a dump:
00 0b7ff34c 0b7ff370 BC32RECV!__ExceptionPtr::_RethrowException+0x82 [d:\agent\_work\3\s\src\vctools\crt\crtw32\eh\excptptr.cpp # 541]
WARNING: Frame IP not in any known module. Following frames may be wrong.
01 0b7ff350 052b1466 0xb7ff370
02 0b7ff358 052b1561 BC32RECV!std::exception_ptr::_RethrowException+0x6 [c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\exception # 282]
03 0b7ff370 052b395b BC32RECV!std::rethrow_exception+0x31 [c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\exception # 358]
04 0b7ff388 052b3e8f BC32RECV!Concurrency::details::_ExceptionHolder::_RethrowUserException+0x2b [c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\ppltasks.h # 789]
05 0b7ff3d8 052b9d09 BC32RECV!Concurrency::details::_Task_impl_base::_Wait+0x22f [c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\ppltasks.h # 1613]
06 (Inline) -------- BC32RECV!Concurrency::task<unsigned char>::wait+0xf [c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\ppltasks.h # 3297]
07 (Inline) -------- BC32RECV!Concurrency::task<void>::wait+0xf [c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\ppltasks.h # 4223]
08 0b7ffa60 052c75e1 BC32RECV!try_connection+0x479 [c:\my .net projects\builder\main\utility\bc32recv\bc32recv.cpp # 578]
The code of my function is :
try {
client->connection.start().wait(); -> access vio
connection_success = true;
}
catch (std::exception& ex)
{
...
}
It seems it had a different std::exception_ptr, changing version from 14.16.27012.6 to 14.24.28127.4 of msvcp140.dll.
I assumed compatibility was maintained in this case when only the build / minor version changes.
The exception info from the dump is:
0:014> .exr -1
ExceptionAddress: 052c7d2e (BC32RECV!__ExceptionPtr::_RethrowException+0x00000082)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000000
Parameter[1]: b0688e75
Attempt to read from address b0688e75
There's a attempt of reading b0688e75 address.
The first frame of the stack reports:
frame 0n0;dv /t /v
00 0b7ff34c 0b7ff370 BC32RECV!__ExceptionPtr::_RethrowException+0x82 [d:\agent\_work\3\s\src\vctools\crt\crtw32\eh\excptptr.cpp # 541]
#ebx              class __ExceptionPtr * this = 0x0a09b2bc
0b7ff2f8          struct _EXCEPTION_RECORD ThisException = struct _EXCEPTION_RECORD
#eax              struct _s_ThrowInfo * pThrow = 0xb0688e69
<unavailable>     struct _s_CatchableType * pType = <value unavailable>
<unavailable>     void * pExceptionBuffer = <value unavailable>
I dont know how to "read" theese infos. What it seems near to the address VAM is
#eax              struct _s_ThrowInfo * pThrow = 0xb0688e69
and it seems wrong:
((BC32RECV!_s_ThrowInfo *)0xffffffffb0688e69) : 0xffffffffb0688e69 [Type: _s_ThrowInfo *]
[+0x000] attributes : Unable to read memory at Address 0xffffffffb0688e69
[+0x004] pmfnUnwind : Unable to read memory at Address 0xffffffffb0688e6d
[+0x008] pForwardCompat : Unable to read memory at Address 0xffffffffb0688e71
[+0x00c] pCatchableTypeArray : Unable to read memory at Address 0xffffffffb0688e75

gdb step until source is available again

I'm running GNU grep under gdb on linux and single stepping it. After about 12 steps, control is transferred to setlocale.c, for which no source code is available.
Example session, after step 12 no source code information is available and the list command just shows the file.
Is there a way of getting gdb to keep stepping until a file with source code is available again. Alternatively, is there a way of telling gdb to keep stepping until control is transferred to a different file?
example session, showing source code initially available and then unavaiable for setlocale.c?
(gdb) start
Temporary breakpoint 1 at 0x402e50: file grep.c, line 2415.
Starting program: ~/ws/opt/grep/out/bin/grep --context=20 -r --line-number --byte-offset --include=\*.c int .
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Temporary breakpoint 1, main (argc=0x8, argv=0x7fffffffdaa8) at grep.c:2415
2415 {
(gdb) l
2410 return result;
2411 }
2412
2413 int
2414 main (int argc, char **argv)
2415 {
2416 char *keys = NULL;
2417 size_t keycc = 0, oldcc, keyalloc = 0;
2418 int matcher = -1;
2419 bool with_filenames = false;
(gdb) s 12
__GI_setlocale (category=category#entry=0x6, locale=locale#entry=0x420b7b "") at setlocale.c:220
220 setlocale.c: No such file or directory.
(gdb) l
215 in setlocale.c
You need gdb finish command. With this command you can go out of current stack frame which has no source code available. You can use it as many times as you want until you are again in stack frame with source code. See documentation.
I ended up writing a simple gdb script using the Python API to do this. It will keep stepping until control is transfered to the next file, regardless of whether that involves adding a new stack frame or leaving the current one.
The script can be loaded with source leave_this_file.py. It defines a command called leave_this_file that can be invoked with no arguments, or given a number of times to repeat.
The script is a little bit makeshift and ends up parsing the result of the gdb command frame 0 rather than using one of gdb's proper APIs for inspecting frames.
MAX_STEPS = 10000
def get_file_name():
"""extract the file name for the bottommost frame"""
# example string
#0 main (argc=0x7, argv=0x7fffffffdaa8) at grep.c:2415
# <source fragment>
where_str = gdb.execute("frame 0", from_tty=False, to_string=True)
# last word of first line is file:line
file_line = where_str.splitlines()[0].split()[-1]
filename, _, line = file_line.rpartition(":")
# confirm that line number is an int, raise otherwise
int(line)
return filename
def step_out_of_file_once():
orig_file_name = get_file_name()
current_file_name = orig_file_name
counter = 0
for x in range(MAX_STEPS):
gdb.execute("step", from_tty=False, to_string=True)
counter += 1
current_file_name = get_file_name()
if orig_file_name != current_file_name:
break
print("%s: %30s, %s: %s" % ("new", current_file_name, "steps", counter))
class LeaveThisFile(gdb.Command):
"""step out of the current file"""
def __init__(self):
gdb.Command.__init__(
self, "leave_this_file", gdb.COMMAND_DATA, gdb.COMPLETE_SYMBOL, True
)
def invoke(self, arg, from_tty):
# interpret the arg as a number of times to execute the command
# 1 by default
if arg:
arg = int(arg)
else:
arg = 1
for x in range(arg):
step_out_of_file_once()
LeaveThisFile()
Here's some example output when running GNU grep under gdb
2415 {
(gdb) startQuit
(gdb) source leave_this_file.py
(gdb) leave_this_file 15
new: setlocale.c, steps: 18
new: pthread_rwlock_wrlock.c, steps: 8
new: ../sysdeps/unix/sysv/linux/x86/hle.h, steps: 3
new: pthread_rwlock_wrlock.c, steps: 1
new: setlocale.c, steps: 7
new: ../sysdeps/x86_64/multiarch/../strcmp.S, steps: 1
new: setlocale.c, steps: 48
new: getenv.c, steps: 4
new: ../sysdeps/x86_64/strlen.S, steps: 2
new: getenv.c, steps: 16
new: ../sysdeps/x86_64/multiarch/../strcmp.S, steps: 64
new: getenv.c, steps: 53
new: setlocale.c, steps: 16
new: ../sysdeps/x86_64/multiarch/../strchr.S, steps: 5
new: setlocale.c, steps: 23

How can I make file association in Qt installer(1.5.0)?

How can I make file association in Qt installer framework(1.5.0) on Windows? I want to do this:
For example, when I double click myFile.x then my qt desktop application(Windows) will launch and open this file.
I want to correct this in installscript.qs :
component.addOperation("CreateShortcut", "#TargetDir#/A.exe", "#StartMenuDir#/A.lnk");
component.addOperation("RegisterFileType",
"fl",
"#TargetDir#\\A.exe" + "'%1'",
"myFiles",
"text/plain",
"#TargetDir#/A_icon.ico",
"ProgId=A.fl");
component.addOperation("CreateShortcut", "#TargetDir#/uninstall.exe", "#StartMenuDir#/Uninstall.lnk");
When I write this string, it gives parse error.
Output is :
Caught exception: Exception while loading component script: 'D:/Workspace/A/A_installer/A/packages/com.ge.mss/meta/installscript.qs
SyntaxError: Parse error
Backtrace:
()#D:/Workspace/A/A_installer/A/packages/com.ge.mss/meta/installscript.qs:102'
If you are using windows just follow this:
component.addOperation("RegisterFileType",
"bob",
"#TargetDir#\\BobiSoft.exe \" %1\"",
"BobiSoft Files",
"application/x-binary",
"#TargetDir#/bobi_file_icon.ico",
"ProgId=BobiSoft.bob");
I also had this problem, the trick was to replace the ' %1' as many examples say, with \" %1\".