I'm working on the crash dump of a standard C++ program.
I'm having a problem with following piece of code:
_variant_t* pvarTemp;
pvarTemp = <static_cast><_variant_t*> (pointer_array->GetAt(i));
if (!pvarTemp) continue;
pvarTemp->Clear();
As I presume something's going wrong in memory, I decided to have a look at the memory window (menu "Debug", "Windows", "Memory", "Memory 1"), where I entered the memory address of pvarTemp, but I only saw some question marks.
The crash dump itself is just a small crashdump, hereby PYKD's targetSystem().desc result (in Windgb):
0:049> !py
Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> print targetSystem().desc
User mini dump: C:\Temp_Folder\CRASH.DMP
My question is: what's the reason of the question marks?
The reason for having the question marks also is the reason for the application to crash.
As I'm working with a small crashdump, the memory addresses, mentioned in the dump, might not be trustworthy, so the question marks are normal behaviour.
Which one of the two is correct?
You're doing postmortem debugging. Generally, crash dumps don't capture all the memory in use by the process. If you try to inspect memory that wasn't included in the dump, you'll see question marks.
If you're doing live debugging, question marks mean the address isn't mapped into the process space.
Related
I've installed the Tensorflow package and complied it from source for IBM s390x architecture. The image recognition classify_image.py sample as described in the tutorial throws an error as shown below:
Run command:
python ./classify_image.py --model_dir=/data/shared/myprojects/tensorflow/models/models-master/tutorials/image/imagenet --image_file=/data/shared/myprojects/keras/images/claude_profile.jpg
Error message:
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 659, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Cannot reshape a tensor with 1041082757314414592 elements to shape [16777216,524288] (8796093022208 elements) for 'pool_3/_reshape' (op: 'Reshape') with input shapes: [1,22546423,22546423,2048], [2] and with input tensors computed as partial shapes: input[1] = [16777216,524288].
Version:
python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.VERSION
'1.3.1'
>>>
A possible error cause is an incompatibility of endianness as the trained model likely is stored in a little endian notation while the CPU works in a big endian mode. Is there an easy way to configure a byte swapping that changes the endianness of the input data? Other Tensorflow samples, without image processing routines execute OK.
1041082757314414592 sounds more like an overflow / underflow than endianness issue. If you don't try to load an example but try to run one from scratch do you also see issues?
This seems to be happening because the inception model being a pre-trained model on a Little Endian machine, will have issues while loading on Big Endian(s390x). Also any graph (eg. classify_image_graph_def.pb) will store values like size in one format, which gives unexpected results when read in the other.
As far as I know there is no tool available yet to convert any saved model to be compatible on big endian.
So for now, on big endian, we need to train our nodes from scratch.
I have a crash dump file from a customer which I have to analyze. I am new to the world of crash dump analysis. (The source code is C++).
Here is what I have tried:-
I opened the .dmp file with MS Visual Studio which indicated the following error - You cannot debug a 64-bit dump of a 32-bit process. So, I thought of giving WinDbg a try.
When I opened the file in WinDbg after setting the symbol search path, I started the getting the following - Debuggee not connected.
Can anyone point me out in the right direction? Should I be asking the customer to provide a 32-bit dump from his point or can this dump file be debugged.
Also, provide the necessary documentation to get started.
To some extent, you can debug a 64-bit dump of a 32-bit process with Windbg,
by use of the wow64exts. However if possible I think it’s best to have a 32 bits dump.
If the customer can provide a 32-bit dump , get it.
Here is a sample of the wow64exts:
0:008> k
Child-SP RetAddr Call Site
00000000`0291f128 00000000`779d263a wow64cpu!CpupSyscallStub+0x2
00000000`0291f130 00000000`7792c4f6 wow64cpu!WaitForMultipleObjects32+0x1d
00000000`0291f1e0 00000000`7792b8f5 wow64!RunCpuSimulation+0xa
00000000`0291f230 000007fe`e51fd6af wow64!Wow64LdrpInitialize+0x435
00000000`0291f770 000007fe`e519c1ae ntdll!_LdrpInitialize+0xde
00000000`0291f7e0 00000000`00000000 ntdll!LdrInitializeThunk+0xe
0:008> .load wow64exts
0:008> !sw
Switched to 32bit mode
0:008:x86> k
ChildEBP RetAddr
02a1f2dc 7783c752 ntdll_779e0000!NtWaitForMultipleObjects+0xc
02a1f460 75b956c0 KERNELBASE!WaitForMultipleObjectsEx+0x10b
02a1f4d4 75b9586a kernel32!WerpReportFaultInternal+0x1c4
02a1f4e8 75b67828 kernel32!WerpReportFault+0x6d
02a1f4f4 778c07c4 kernel32!BasepReportFault+0x19
The most useful tool for crash dump analysis is to load it into Windbg (File -> Open crash dump) and then use the
!analyze -v
command. This applies a number of heuristics to rewind slightly from the actual crash site to work out where the cause of the crash is likely to be, eg to where a null pointer dereference occurred. There's a good tutorial here. A really good site to bookmark is John Robbins' blog which has lots of great articles about Windbg.
I develop a win32 application using i686-w64-mingw32-gcc 4.7.2 crosscompiler under Ubuntu 12.10.
When I run my application compiled in the release mode and it crashes, I may receive an output like bellow.
I am wondering how to interpret it? Is there some useful information that I could get from that? For example would it be possible to get lines in my source code from the Stack dump?
wine: Unhandled page fault on read access to 0x00006c11 at address 0x401cb1 (thread 0009), starting debugger...
Application tried to create a window, but no driver could be loaded.
Make sure that your X server is running and that $DISPLAY is set correctly.
Unhandled exception: page fault on read access to 0x00006c11 in 32-bit code (0x00401cb1).
Register dump:
CS:0023 SS:002b DS:002b ES:002b FS:0063 GS:006b
EIP:00401cb1 ESP:007af750 EBP:007af9c8 EFLAGS:00010206( R- -- I - -P- )
EAX:00406a20 EBX:00000004 ECX:00006c11 EDX:00006c11
ESI:00000068 EDI:00110440
Stack dump:
0x007af750: 007af7df 00000030 00000043 00000004
0x007af760: 00142fc0 3ff54e5b 00406a20 3ff57208
0x007af770: 0207251c 3ff4cb7d 02072574 3ff51f7f
0x007af780: ea9e6eeb 3ff49b90 e09fe868 3ff4c562
0x007af790: 00006c11 00000045 0209c5b4 007afa0c
0x007af7a0: 00000004 019b0000 007af7f8 00110000
000c: sel=0067 base=00000000 limit=00000000 16-bit --x
Backtrace:
=>0 0x00401cb1 in analyser (+0x1cb1) (0x007af9c8)
1 0x00536147 in analyser (+0x136146) (0x007afd98)
2 0x004013fa __tmainCRTStartup+0x279() [/home/ruben/mingw-w64/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:313] in analyser (0x007afe70)
3 0x7b859ddc call_process_entry+0xb() in kernel32 (0x007afe88)
4 0x7b85b04f in kernel32 (+0x4b04e) (0x007afec8)
5 0x7bc71d90 call_thread_func_wrapper+0xb() in ntdll (0x007afed8)
6 0x7bc7486d call_thread_func+0x7c() in ntdll (0x007affa8)
7 0x7bc71d6e RtlRaiseException+0x21() in ntdll (0x007affc8)
8 0x7bc49f4e call_dll_entry_point+0x61d() in ntdll (0x007affe8)
0x00401cb1: movl 0x0(%edx),%eax
Modules:
Module Address Debug info Name (89 modules)
PE 240000- 273000 Deferred ssleay32
PE 280000- 37e000 Deferred libeay32
...
I am wondering how to interpret it?
It appears pretty clear already: your code, more precisely: instruction at address 0x401cb1, attempted to dereference memory at address 0x00006c11. That memory access was not valid, and triggered a page fault.
Is there some useful information that I could get from that?
Yes, as above.
For example would it be possible to get lines in my source code from the Stack dump?
Sure: when you build your application in release mode, add the -g flag. Then
addr2line -fe analyser 0x401cb1
should tell you which file/line the address 0x401cb1 corresponds to.
I have a Python module that is operating as a server for a wireless handheld computer. Every time the handheld sends a message to the server, the module determines what kind of message it is, and then assembles an appropriate response. Because the responses are often state-dependent, I am using global variables where needed to retain/share information between the individual functions that handle each type of message.
The problem I'm having is when the application is closed (for whatever reason), the global variable values are (of course) lost, so on re-launching the application it's out of synch with the handheld. I need a reliable way to store those values for recovery.
The direction I've gone so far (but have not gotten it to work yet) is to write the variable names and their values to a CSV file on the disk, every time they're updated -- and then (when the app is launched), look for that file and use it to assign the variables to their previous states. I have no trouble writing the file or reading it, but for some reason the values just aren't getting assigned.
I can post the code for comments/help, but before that I wanted to find out whether I'm just going an entirely wrong direction in the first place. Is there a better (or at least preferable) way to save and recover these values?
thanks,
JDM
====
Following up. It may be a touch klunky, but here's what I have and it's working. The only globals I care about are the ones that start with "CUR_". I had to use tempDict1 because the interpreter doesn't seem to like iterating directly over globals().
import pickle
CUR_GLO1 = 'valglo1'
CUR_GLO2 = 'valglo2'
CUR_GLO3 = 'valglo3'
def saveGlobs():
tempDict1 = globals().copy()
tempDict2 = {}
for key in tempDict1:
if (key[:4]=='CUR_'):tempDict2[key] = tempDict1[key]
pickle.dump(tempDict2,open('tempDict.p','wb'))
def retrieveGlobs():
tempDict = pickle.load(open('tempDict.p','rb'))
globals().update(tempDict)
writing it up as an answer..
What I think you want to do is a form of application checkpointing.
You can use the Pickle module for conveniently saving and loading Python variables. Here is a simple example of how to use it. This discussion on Stackoverflow and this note seem to agree, although part of me thinks that there must be a better way.
Incidentally, you don't need to put everything into a dictionary. As long as you dump and load variables in the right order, and make sure that you don't change that, insert data in the middle etc, you can just dump and load several variables. Using a dictionary like you did does remove the ordering dependency though.
% python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> foo=123
>>> bar="hello"
>>> d={'abc': 123, 'def': 456}
>>> f=open('p.pickle', 'wb')
>>> pickle.dump(foo, f)
>>> pickle.dump(bar, f)
>>> pickle.dump(d, f)
>>> ^D
% python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> f=open('p.pickle','rb')
>>> foo=pickle.load(f)
>>> foo
123
>>> bar=pickle.load(f)
>>> bar
'hello'
>>> d=pickle.load(f)
>>> d
{'abc': 123, 'def': 456}
>>>
Currently we call MiniDumpWriteDump with the MiniDumpNormal | MiniDumpWithIndirectlyReferencedMemory flags. That works just fine for internal builds in Debug configuration, but isn't giving as much information as we need in Release configuration.
In Release, the minidump data contains enough stack information for the debugger to work out where in the code the failure occurred, but no other data. I don't simply mean local variables are missing due to being optimised out, as you'd expect in a Release build - I mean, there is nothing useful except for the call stack and current code line. No registers, no locals, no globals, no objects pointed to by locals - nothing. We don't even get 'this' which would allow us to view the current object. That was the point of using MiniDumpWithIndirectlyReferencedMemory - it should have included memory referenced by locals and stack variables, but doesn't seem to.
What flags should we be using instead? We don't want to use MiniDumpWithFullMemory and start generating 600MB+ dumps, but would happily expand the dumps somewhat beyond the 90KB we currently get if it meant getting more useful data. Perhaps we should be using MiniDumpWithDataSegments (globals) or...?
WinDbg uses the following flags for a .dump /ma:
0:003> .dumpdebug
----- User Mini Dump Analysis
MINIDUMP_HEADER:
Version A793 (62F0)
NumberOfStreams 13
Flags 41826
0002 MiniDumpWithFullMemory
0004 MiniDumpWithHandleData
0020 MiniDumpWithUnloadedModules
0800 MiniDumpWithFullMemoryInfo
1000 MiniDumpWithThreadInfo
40000 MiniDumpWithTokenInformation
I suggest you replace MiniDumpWithFullMemory by MiniDumpWithIndirectlyReferencedMemory.