I have a .NET application that is crashing sometimes on exit. There's a bunch of COM and native stuff underneath the hood, too. It's a x86 application running on Windows 7 x64.
I've run through some WinDbg tutorials and I think I'm executing reasonable steps to get helpful information, but the stack trace itself isn't ringing any bells.
A few other tidbits:
I can reproduce this pretty consistently, say 75% of the time
If I clean up the threading (a lot of Thread.Abort()), it's reproducible maybe 20% of the time
Running the same procedure, I've seen a completely different stack trace than the one below, too
I'm using the 32-bit WinDbg. Here's the general process I've been using:
open the executable directly from WinDbg
set symbol path as: SRV*c:\sym*http://msdl.microsoft.com/download/symbols
type: .loadby sos clr
use the application, and get it to crash
Right after the crash, I get output:
(a38.1424): CLR exception - code e0434352 (first chance)
(a38.1424): CLR exception - code e0434352 (first chance)
(a38.1fd0): Unknown exception - code c000000d (first chance)
(a38.1fd0): Unknown exception - code c000000d (!!! second chance !!!)
eax=00000000 ebx=004dea1c ecx=7efdd000 edx=00000057 esi=7264d0c0 edi=07f2a248
eip=778715de esp=004dea08 ebp=004def50 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
ntdll!ZwRaiseException+0x12:
778715de 83c404 add esp,4
If I type in ~ I only get one thread:
. 0 Id: a38.1fd0 Suspend: 1 Teb: 7efdd000 Unfrozen
Now, if I type in !analyze -v I get a big stack trace:
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
[ a bunch of symbol stuff loading here ]
FAULTING_IP:
ntdll!TpReleaseCleanupGroupMembers+276
778e4f52 a1b4009577 mov eax,dword ptr [ntdll!TppLogpRoutine (779500b4)]
EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 778e4f52 (ntdll!TpReleaseCleanupGroupMembers+0x00000276)
ExceptionCode: c000000d
ExceptionFlags: 00000000
NumberParameters: 0
FAULTING_THREAD: 00001fd0
PROCESS_NAME: XXXXX.exe
ERROR_CODE: (NTSTATUS) 0xc000000d - An invalid parameter was passed to a service or function.
EXCEPTION_CODE: (NTSTATUS) 0xc000000d - An invalid parameter was passed to a service or function.
NTGLOBALFLAG: 70
APPLICATION_VERIFIER_FLAGS: 0
CONTEXT: 004dea6c -- (.cxr 0x4dea6c)
eax=004deee0 ebx=00000001 ecx=7efdd000 edx=00000057 esi=7264d0c0 edi=07f2a248
eip=778e4f52 esp=004deed0 ebp=004def50 iopl=0 nv up ei ng nz ac pe cy
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000297
ntdll!TpReleaseCleanupGroupMembers+0x276:
778e4f52 a1b4009577 mov eax,dword ptr [ntdll!TppLogpRoutine (779500b4)] ds:002b:779500b4=00000000
Resetting default scope
STACK_ADDR_RAW_STACK_SYMBOL: 4deb4c
ADDITIONAL_DEBUG_TEXT: Followup set based on attribute [Is_ChosenCrashFollowupThread] from Frame:[0] on thread:[ffffffff]
LAST_CONTROL_TRANSFER: from 00000000 to 77883c04
DEFAULT_BUCKET_ID: STATUS_INVALID_PARAMETER
PRIMARY_PROBLEM_CLASS: STATUS_INVALID_PARAMETER
BUGCHECK_STR: APPLICATION_FAULT_STATUS_INVALID_PARAMETER
STACK_TEXT:
778e4f52 ntdll!TpReleaseCleanupGroupMembers+0x276
72630d69 AUDIOSES!DllCanUnloadNow+0x42
7565b5f4 ole32!CClassCache::CDllPathEntry::CanUnload_rl+0x3b
7565b771 ole32!CClassCache::FreeUnused+0x83
7565b68f ole32!CoFreeUnusedLibrariesEx+0x36
756a0ccb ole32!CoFreeUnusedLibraries+0x9
15e2f549 GxMetadata+0xf549
15e45e3d GxMetadata!DllCanUnloadNow+0x1686d
77889950 ntdll!LdrpCallInitRoutine+0x14
7789d6b2 ntdll!LdrShutdownProcess+0x1aa
7789d554 ntdll!RtlExitUserProcess+0x74
754279f4 KERNEL32!ExitProcessStub+0x12
720642f0 mscoreei!RuntimeDesc::ShutdownAllActiveRuntimes+0x29c
72064321 mscoreei!CLRRuntimeHostInternalImpl::ShutdownAllRuntimesThenExit+0x15
5ea18580 clr!EEPolicy::ExitProcessViaShim+0x66
5ea1862f clr!SafeExitProcess+0x122
5e9638a9 clr!DisableRuntime+0x120
5e963905 clr!EEPolicy::HandleExitProcess+0x5c
5e9b8af8 clr!_CorExeMainInternal+0xdd
5e9b3a30 clr!_CorExeMain+0x4e
720555ab mscoreei!_CorExeMain+0x38
72f67f16 MSCOREE!ShellShim__CorExeMain+0x99
72f64de3 MSCOREE!_CorExeMain_Exported+0x8
7542339a KERNEL32!BaseThreadInitThunk+0xe
77889ef2 ntdll!__RtlUserThreadStart+0x70
77889ec5 ntdll!_RtlUserThreadStart+0x1b
FOLLOWUP_IP:
AUDIOSES!DllCanUnloadNow+42
72630d69 ff3514d06472 push dword ptr [AUDIOSES!_AudioClientThreadpoolCleanupGroup (7264d014)]
EDIT 1: (additional info)
!clrstack
OS Thread Id: 0x1fd0 (0)
Child SP IP Call Site
GetFrameContext failed: 1
!threads
ThreadCount: 7
UnstartedThread: 0
BackgroundThread: 4
PendingThread: 0
DeadThread: 3
Hosted Runtime: no
PreEmptive GC Alloc Lock
ID OSID ThreadOBJ State GC Context Domain Count APT Exception
0 1 1fd0 005afe88 16220 Enabled 03051294:03051e6c 00578550 0 STA
XXXX 2 e5c 005801d0 b220 Enabled 0305a22c:0305be6c 00578550 0 MTA (Finalizer)
XXXX 3 00641258 19820 Enabled 00000000:00000000 00578550 0 Ukn
XXXX 4 06e4b800 819820 Enabled 00000000:00000000 00578550 0 Ukn
XXXX 5 18a0 081be620 200b220 Enabled 00000000:00000000 00578550 1 MTA
XXXX 8 081d5e18 819820 Enabled 00000000:00000000 00578550 0 Ukn
XXXX 7 158 07ed78d8 220 Enabled 00000000:00000000 00578550 0 Ukn
Looks like the ntdll!TpReleaseCleanupGroupMembers (the same as kernel32!CloseThreadpoolCleanupGroupMembers - you can look it up on msdn) function (from top of the fault stack) does not like to be called when the process is being shut down - it throws the exception you're seeing (invalid parameter) in this case.
From the presence of two more libs on the stack (audioses and gxmetadata) I'd guess some objects are destroyed/released way too late. audioses.dll seems to the Core Audio API library, not sure about the gxmetadata.dll - can you explain the use of these?
Related
I am attempting to get a working ETW consumer of events from the Microsoft-Windows-Kernel-Process provider. I am mostly following Microsoft's "Using TdhGetProperty to Consume Event Data" example, found here.
The session starts, I can see it in the output of logman query -ets. My problem is when I get to call TdhGetEventInformation for the first time.
status = TdhGetEventInformation(pEvent, 0, NULL, pInfo, &BufferSize);
My consumer has an access violation at this point. This is what I see in Windbg.
(10e4.188c): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
mintdh!TdhGetEventInformation+0x7c:
00007ff9`4262224c 6641837cc0020b cmp word ptr [r8+rax*8+2],0Bh ds:00000000`00000002=????
0:004> r
rax=0000000000000000 rbx=0000000000000000 rcx=0000021ee256c320
rdx=00000048c6dff3c8 rsi=0000021ee256c320 rdi=00000048c6dff420
rip=00007ff94262224c rsp=00000048c6dff200 rbp=00000048c6dff4f0
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000010 r13=0000021ee2549a70
r14=0000000000000000 r15=000000000000000e
iopl=0 nv up ei pl zr na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
mintdh!TdhGetEventInformation+0x7c:
00007ff9`4262224c 6641837cc0020b cmp word ptr [r8+rax*8+2],0Bh ds:00000000`00000002=????
I am receiving an Event_Record at this point.
# Child-SP RetAddr Call Site
00 00000048`c6dff200 00007ff9`426710f6 mintdh!TdhGetEventInformation+0x7c
01 00000048`c6dff240 00007ff7`f1c22fbc tdh!TdhfGetEventInformation+0x56
02 00000048`c6dff290 00007ff7`f1c212fd ETWConsumer!GetEventInformation(
struct _EVENT_RECORD * ppEvent = 0x0000021e`e256c320,
I basically have a null pointer because r8 and rax are empty. I have no idea what this mintdh!TdhGetEventInformation function is expecting, or what it thinks is missing.
Has anyone had a similar problem?
I'm trying to set up a crashdump system to better be able to debug errors that I can't replicate easily on my own system
Here's my test program (Compiled with Release x64 Configuration with Visual Studio 2017):
#include "pch.h"
#include <iostream>
#include <Windows.h>
#include <DbgHelp.h>
#pragma comment(lib,"Dbghelp.lib")
BOOL MiniDumpWriteDump(
HANDLE hProcess,
DWORD ProcessId,
HANDLE hFile,
MINIDUMP_TYPE DumpType,
PMINIDUMP_EXCEPTION_INFORMATION ExceptionParam,
PMINIDUMP_USER_STREAM_INFORMATION UserStreamParam,
PMINIDUMP_CALLBACK_INFORMATION CallbackParam
);
LONG CreateMiniDump( struct _EXCEPTION_POINTERS *pExceptionInfo )//EXCEPTION_POINTERS* pep )
{
// Open the file
HANDLE hFile = CreateFile( "MiniDump.dmp", GENERIC_READ | GENERIC_WRITE,
0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL );
if( ( hFile != NULL ) && ( hFile != INVALID_HANDLE_VALUE ) )
{
// Create the minidump
MINIDUMP_EXCEPTION_INFORMATION mdei;
mdei.ThreadId = GetCurrentThreadId();
mdei.ExceptionPointers = pExceptionInfo;
mdei.ClientPointers = FALSE;
MINIDUMP_TYPE mdt = MiniDumpNormal;
BOOL rv = MiniDumpWriteDump( GetCurrentProcess(), GetCurrentProcessId(),
hFile, mdt, (pExceptionInfo != 0) ? &mdei : 0, 0, 0 );
if( !rv )
std::cout<<"MiniDumpWriteDump failed. Error: "<<GetLastError()<<std::endl;
else
std::cout<<"Minidump created."<<std::endl;
// Close the file
CloseHandle( hFile );
}
else
{
std::cout<<"CreateFile failed. Error: "<<GetLastError()<<std::endl;
}
return EXCEPTION_EXECUTE_HANDLER;
}
static void ftest3()
{
throw std::runtime_error("Error");
}
static void ftest2()
{
ftest3();
}
static void ftest1()
{
ftest2();
}
int main()
{
SetUnhandledExceptionFilter(&CreateMiniDump);
ftest1();
}
(Source: http://www.debuginfo.com/articles/effminidumps2.html)
If I run it on my primary system (on which I compiled the program), and then load the .dmp-file in Visual Studio or WinDbg, the full stack trace is displayed correctly and I can see where the error has occurred:
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 000000fd`3bbbf6f0 00007ff8`a72142cd KERNELBASE!RaiseException+0x68
01 000000fd`3bbbf7d0 00007ff7`7b4f130f VCRUNTIME140!_CxxThrowException+0xad [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp # 133]
02 000000fd`3bbbf840 00007ff7`7b4f1319 test_minidump!ftest3+0x1f [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 63]
03 000000fd`3bbbf890 00007ff7`7b4f1329 test_minidump!ftest2+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 68]
04 000000fd`3bbbf8c0 00007ff7`7b4f1346 test_minidump!ftest1+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 73]
05 000000fd`3bbbf8f0 00007ff7`7b4f1838 test_minidump!main+0x16 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 79]
06 (Inline Function) --------`-------- test_minidump!invoke_main+0x22 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl # 78]
07 000000fd`3bbbf920 00007ff8`b45d3034 test_minidump!__scrt_common_main_seh+0x10c [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl # 288]
08 000000fd`3bbbf960 00007ff8`b71a1461 kernel32!BaseThreadInitThunk+0x14
09 000000fd`3bbbf990 00000000`00000000 ntdll!RtlUserThreadStart+0x21
However, if I run the program on a different system, and then try to analyze the dump on my own system, all I get is this:
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 00000021`ccbef870 00007ffe`8cfd4722 KERNELBASE!RaiseException+0x68
01 00000021`ccbef950 00000021`ccbef9f0 VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp # 136]
02 00000021`ccbef958 00007ffe`00000000 0x00000021`ccbef9f0
03 00000021`ccbef960 000001ee`2e25ff30 0x00007ffe`00000000
04 00000021`ccbef968 00000000`00000000 0x000001ee`2e25ff30
Which isn't exactly helpful. I've loaded the dump-file in WinDbg and then used the following commands:
.symfix
.sympath+ C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
.srcpath C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
!analyze -v
.ecxr
k
Here's the full output from WinDbg:
Microsoft (R) Windows Debugger Version 10.0.17134.12 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Users\Florian\Desktop\tmp\MiniDump.dmp]
User Mini Dump File: Only registers, stack and portions of memory are available
Symbol search path is: srv*
Executable search path is:
Windows 10 Version 17134 MP (4 procs) Free x64
Product: WinNt, suite: SingleUserTS
17134.1.amd64fre.rs4_release.180410-1804
Machine Name:
Debug session time: Sun Sep 23 11:56:45.000 2018 (UTC + 2:00)
System Uptime: not available
Process Uptime: not available
..............
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(169c.1b78): C++ EH exception - code e06d7363 (first/second chance not available)
ntdll!NtGetContextThread+0x14:
00007ffe`9abfbc44 c3 ret
0:000> .symfix
0:000> .sympath+ C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
Symbol search path is: srv*;C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
Expanded Symbol search path is: cache*;SRV*https://msdl.microsoft.com/download/symbols;c:\users\florian\documents\projects\test_minidump\x64\release
************* Path validation summary **************
Response Time (ms) Location
Deferred srv*
OK C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
0:000> .srcpath C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
Source search path is: C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
************* Path validation summary **************
Response Time (ms) Location
OK C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
0:000> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
KEY_VALUES_STRING: 1
TIMELINE_ANALYSIS: 1
Timeline: !analyze.Start
Name: <blank>
Time: 2018-09-24T10:08:32.944Z
Diff: 87107944 mSec
Timeline: Dump.Current
Name: <blank>
Time: 2018-09-23T09:56:45.0Z
Diff: 0 mSec
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
CONTEXT: (.ecxr)
rax=0000000000000000 rbx=00007ffe973da570 rcx=0000000000000000
rdx=0000000000000030 rsi=00007ff798203cd0 rdi=000001ee2e25c750
rip=00007ffe97bba388 rsp=00000021ccbef870 rbp=00000021ccbef9b0
r8=0000000000000000 r9=0000000000000000 r10=000001ee2e0f0000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=00000021ccbef9f0 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
KERNELBASE!RaiseException+0x68:
00007ffe`97bba388 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h] ss:00000021`ccbef930=000052660787015b
Resetting default scope
FAULTING_IP:
KERNELBASE!RaiseException+68
00007ffe`97bba388 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h]
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 00007ffe97bba388 (KERNELBASE!RaiseException+0x0000000000000068)
ExceptionCode: e06d7363 (C++ EH exception)
ExceptionFlags: 00000001
NumberParameters: 4
Parameter[0]: 0000000019930520
Parameter[1]: 00000021ccbef9f0
Parameter[2]: 00007ff798203cd0
Parameter[3]: 00007ff798200000
BUGCHECK_STR: CPP_EXCEPTION
DEFAULT_BUCKET_ID: CPP_EXCEPTION
PROCESS_NAME: test_minidump.exe
ERROR_CODE: (NTSTATUS) 0xe06d7363 - <Unable to get error code text>
EXCEPTION_CODE: (NTSTATUS) 0xe06d7363 - <Unable to get error code text>
EXCEPTION_CODE_STR: e06d7363
EXCEPTION_PARAMETER1: 0000000019930520
EXCEPTION_PARAMETER2: 00000021ccbef9f0
EXCEPTION_PARAMETER3: 00007ff798203cd0
EXCEPTION_PARAMETER4: 7ff798200000
WATSON_BKT_PROCSTAMP: 5ba8b478
WATSON_BKT_MODULE: KERNELBASE.dll
WATSON_BKT_MODSTAMP: b0bb231d
WATSON_BKT_MODOFFSET: 3a388
WATSON_BKT_MODVER: 6.2.17134.165
MODULE_VER_PRODUCT: Microsoft® Windows® Operating System
BUILD_VERSION_STRING: 17134.1.amd64fre.rs4_release.180410-1804
MODLIST_WITH_TSCHKSUM_HASH: 636d437315fdc5447e163e64094b1f6a26123a83
MODLIST_SHA1_HASH: 74dec0c40d229cd1e109927f1f5e794fa53165ac
DUMP_FLAGS: 0
DUMP_TYPE: 2
ANALYSIS_SESSION_HOST: DESKTOP-VFOALG8
ANALYSIS_SESSION_TIME: 09-24-2018 12:08:32.0944
ANALYSIS_VERSION: 10.0.17134.12 amd64fre
THREAD_ATTRIBUTES:
PROBLEM_CLASSES:
ID: [0n317]
Type: [#APPLICATION_FAULT_STRING]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Omit
Data: Add
String: [CPP_EXCEPTION]
PID: [Unspecified]
TID: [Unspecified]
Frame: [0]
PRIMARY_PROBLEM_CLASS: CPP_EXCEPTION
LAST_CONTROL_TRANSFER: from 00007ffe8cfd4722 to 00007ffe97bba388
STACK_TEXT:
00000021`ccbef870 00007ffe`8cfd4722 : 00000021`ccbef9f0 00007ffe`00000000 000001ee`2e25ff30 00000000`00000000 : KERNELBASE!RaiseException+0x68
00000021`ccbef950 00000021`ccbef9f0 : 00007ffe`00000000 000001ee`2e25ff30 00000000`00000000 00000001`e06d7363 : VCRUNTIME140!_CxxThrowException+0xc2
00000021`ccbef958 00007ffe`00000000 : 000001ee`2e25ff30 00000000`00000000 00000001`e06d7363 00000000`00000000 : 0x00000021`ccbef9f0
00000021`ccbef960 000001ee`2e25ff30 : 00000000`00000000 00000001`e06d7363 00000000`00000000 00000000`00000000 : 0x00007ffe`00000000
00000021`ccbef968 00000000`00000000 : 00000001`e06d7363 00000000`00000000 00000000`00000000 00000000`00000004 : 0x000001ee`2e25ff30
THREAD_SHA1_HASH_MOD_FUNC: 74c55fd19222dcb5ad415427110811b213f4c078
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 7592380e0f749a34b2d654a7c3b829a033ee79fa
THREAD_SHA1_HASH_MOD: ad69e1f2190edbb9a7df71b4b6403437528645f2
FOLLOWUP_IP:
KERNELBASE!RaiseException+68
00007ffe`97bba388 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h]
FAULT_INSTR_CODE: 248c8b48
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: KERNELBASE!RaiseException+68
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: KERNELBASE
IMAGE_NAME: KERNELBASE.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 0
STACK_COMMAND: ~0s ; .ecxr ; kb
BUCKET_ID: CPP_EXCEPTION_KERNELBASE!RaiseException+68
FAILURE_EXCEPTION_CODE: e06d7363
FAILURE_IMAGE_NAME: KERNELBASE.dll
BUCKET_ID_IMAGE_STR: KERNELBASE.dll
FAILURE_MODULE_NAME: KERNELBASE
BUCKET_ID_MODULE_STR: KERNELBASE
FAILURE_FUNCTION_NAME: RaiseException
BUCKET_ID_FUNCTION_STR: RaiseException
BUCKET_ID_OFFSET: 68
BUCKET_ID_MODTIMEDATESTAMP: 0
BUCKET_ID_MODCHECKSUM: 27e598
BUCKET_ID_MODVER_STR: 6.2.17134.165
BUCKET_ID_PREFIX_STR: CPP_EXCEPTION_
FAILURE_PROBLEM_CLASS: CPP_EXCEPTION
FAILURE_SYMBOL_NAME: KERNELBASE.dll!RaiseException
FAILURE_BUCKET_ID: CPP_EXCEPTION_e06d7363_KERNELBASE.dll!RaiseException
TARGET_TIME: 2018-09-23T09:56:45.000Z
OSBUILD: 17134
OSSERVICEPACK: 1
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 256
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 WinNt SingleUserTS
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2020-08-28 06:38:41
BUILDDATESTAMP_STR: 180410-1804
BUILDLAB_STR: rs4_release
BUILDOSVER_STR: 10.0.17134.1.amd64fre.rs4_release.180410-1804
ANALYSIS_SESSION_ELAPSED_TIME: a63
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:cpp_exception_e06d7363_kernelbase.dll!raiseexception
FAILURE_ID_HASH: {1253aecc-520d-655b-58e3-5eb61e209188}
Followup: MachineOwner
---------
0:000> .ecxr
rax=0000000000000000 rbx=00007ffe973da570 rcx=0000000000000000
rdx=0000000000000030 rsi=00007ff798203cd0 rdi=000001ee2e25c750
rip=00007ffe97bba388 rsp=00000021ccbef870 rbp=00000021ccbef9b0
r8=0000000000000000 r9=0000000000000000 r10=000001ee2e0f0000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=00000021ccbef9f0 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
KERNELBASE!RaiseException+0x68:
00007ffe`97bba388 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h] ss:00000021`ccbef930=000052660787015b
0:000> k
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 00000021`ccbef870 00007ffe`8cfd4722 KERNELBASE!RaiseException+0x68
01 00000021`ccbef950 00000021`ccbef9f0 VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp # 136]
02 00000021`ccbef958 00007ffe`00000000 0x00000021`ccbef9f0
03 00000021`ccbef960 000001ee`2e25ff30 0x00007ffe`00000000
04 00000021`ccbef968 00000000`00000000 0x000001ee`2e25ff30
There are no warnings or errors from what I can tell, so why am I not getting the full stack trace information?
VCRUNTIME140!_CxxThrowException+0xad [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp # 133]
versus
VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp # 136]
is a bit worrying. It suggests that the minor version of the MSVC runtime doesn't match between the two machines (because otherwise, why would they invoke RaiseException from different source lines). If your WinDbg session doesn't have symbols for the remote machine's VCRUNTIME140 -- or, worse yet, if it's using the wrong PDB for it -- backtracing is going to become problematic at that point. WinDbg should automatically download and load symbols for whichever version is referenced from the minidump, but I suspect this isn't happening properly.
Use lm v to show the module listing for the minidump, and pay close attention to the checksum information. I suspect you'll see a different checksum for vcruntime140.dll between the two machines. If you do, you'll need to figure out why you're not getting the right symbols for the dll.
Thanks to the tips by users Sneftel and Sean Cline, I found the solution. Running the command !itoldyouso vcruntime140 gives me the following output:
sig MISMATCH: vcruntime140.amd64.pdb and VCRUNTIME140.dll
The problem was that my system which I use for debugging runs on an AMD CPU, while the system that generated the crashdump runs on an intel CPU. For some reason the vcruntime140.dll is system-specific and differs between the two. (Also, on AMD systems the pdb is called "vcruntime140.amd64.pdb", on intel it's "vcruntime140.i386.pdb".)
I've copied the vcruntime140.dll, vcruntime140.i386.pdb and msvcp140.dll from my intel system and added them to the symbol path for WinDbg. Then I ran the same commands as before, and now it outputs the full stacktrace.
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 00000021`ccbef870 00007ffe`8cfd4722 KERNELBASE!RaiseException+0x68
01 00000021`ccbef950 00007ff7`9820130f VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp # 136]
02 00000021`ccbef9d0 00007ff7`98201319 test_minidump!ftest3+0x1f [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 63]
03 00000021`ccbefa20 00007ff7`98201329 test_minidump!ftest2+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 68]
04 00000021`ccbefa50 00007ff7`98201346 test_minidump!ftest1+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 73]
05 00000021`ccbefa80 00007ff7`98201838 test_minidump!main+0x16 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp # 79]
06 (Inline Function) --------`-------- test_minidump!invoke_main+0x22 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl # 78]
07 00000021`ccbefab0 00007ffe`98983034 test_minidump!__scrt_common_main_seh+0x10c [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl # 288]
08 00000021`ccbefaf0 00007ffe`9abd1461 kernel32!BaseThreadInitThunk+0x14
09 00000021`ccbefb20 00000000`00000000 ntdll!RtlUserThreadStart+0x21
I use Cmake + mingw to compile my project. Since some unknown moment my program started to crash on startup and i figured out how to make windows make minidump for my crashing application. I would use GDB to directly debug my application, but when i use GDB the program doesn't crash.
Google breakpad contain tool to convert minidumps to coredumps, so i tryed to compile google breakpad but it seems breakpad is not windows solution.
I'm searching for solution how to inspect minidump and find why my program crashes on start.
How do you do it?
You don't need to analyze minidumps. Instead you can set your debugger as a postmortem debugger. I have searched on the Internet for "windows replace postmortem debugger with gdb". Look, there is Dr. Mingw: https://github.com/jrfonseca/drmingw . This is from their site:
Dr. Mingw is a Just-in-Time (JIT) debugger. When the application
throws an unhandled exception, Dr. Mingw attaches itself to the
application and collects information about the exception, using the
available debugging information.
I have writen a simple C++ test:
int f()
{
int *ptr = 0;
*ptr = *ptr +1;
return *ptr;
}
int main()
{
f();
return 0;
}
Build it:
g++ -g main.cpp
This is the output of DrMingw when my program crashed and there are source files on your disk. As you can see it is quite easy to locate a line with a problem:
.exe caused an Access Violation at location 0040139C in module a.exe Reading from location 00000000.
Registers:
eax=00000000 ebx=7ffdb000 ecx=00000001 edx=77c51ae8 esi=01cedca2 edi=2eafc26a
eip=0040139c esp=0022ff38 ebp=0022ff48 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
AddrPC Params
0040139C 0000001D 7FFDB000 0022FFA0 a.exe!f() [D:\src-c++\test.crasj/main.cpp # 4]
...
{
int *ptr = 0;
> *ptr = *ptr +1;
return *ptr;
}
...
004013BD 00000001 003D3DC0 003D2C78 a.exe!main [D:\src-c++\test.crasj/main.cpp # 11]
...
{
f();
> return 0;
}
...
004010B9 00000001 A9FF6D08 7C90E64E a.exe!__mingw_CRTStartup [C:\MinGW\msys\1.0\src\mingwrt/../mingw/crt1.c # 244]
00401284 2EAFC26A 01CEDCA2 7FFDB000 a.exe!WinMainCRTStartup [C:\MinGW\msys\1.0\src\mingwrt/../mingw/crt1.c # 274]
7C816D4F 0040126C 00000000 78746341 kernel32.dll!RegisterWaitForInputIdle
If the test is build with "-g" but there is no source files availbale then the output of Dr.Mingw looks like this and it is also clear on which line your program crashed:
a.exe caused an Access Violation at location 0040139C in module a.exe Reading from location 00000000.
Registers:
eax=00000000 ebx=7ffd7000 ecx=00000001 edx=77c51ae8 esi=01cedca8 edi=ef612c64
eip=0040139c esp=0022ff38 ebp=0022ff48 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
AddrPC Params
0040139C 0000001D 7FFD7000 0022FFA0 a.exe!f() [D:\src-c++\test.crasj/main.cpp # 4]
004013BD 00000001 003D3D98 003D2C50 a.exe!main [D:\src-c++\test.crasj/main.cpp # 11]
004010B9 00000001 F7114D08 7C90E64E a.exe!__mingw_CRTStartup [C:\MinGW\msys\1.0\src\mingwrt/../mingw/crt1.c # 244]
00401284 EF612C64 01CEDCA8 7FFD7000 a.exe!WinMainCRTStartup [C:\MinGW\msys\1.0\src\mingwrt/../mingw/crt1.c # 274]
7C816D4F 0040126C 00000000 78746341 kernel32.dll!RegisterWaitForInputIdle
If the test is build without "-g" then the output of Dr.Mingw looks like this:
a.exe caused an Access Violation at location 0040139C in module a.exe Reading from location 00000000.
Registers:
eax=00000000 ebx=7ffdf000 ecx=00000001 edx=77c51ae8 esi=01cedca4 edi=79abd658
eip=0040139c esp=0022ff38 ebp=0022ff48 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
AddrPC Params
0040139C 0000001D 7FFDF000 0022FFA0 a.exe
004013BD 00000001 003D3DC0 003D2C78 a.exe
004010B9 00000001 A9D73D08 7C90E64E a.exe!__mingw_CRTStartup [C:\MinGW\msys\1.0\src\mingwrt/../mingw/crt1.c # 244]
00401284 79ABD658 01CEDCA4 7FFDF000 a.exe!WinMainCRTStartup [C:\MinGW\msys\1.0\src\mingwrt/../mingw/crt1.c # 274]
7C816D4F 0040126C 00000000 78746341 kernel32.dll!RegisterWaitForInputIdle
Can you attach to a crashing process after you find its PID? I could do this for my app.
(gdb) attach 1337
Attaching to process 1337
...
(gdb)
Using GDB on Windows
I am using one Windows application which parses certain binary file. The app is crashing (Read access violation) every time at a certain location.
I am trying to find out the root cause of the crash.
(f74.fac): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=02b74141 ebx=00000000 ecx=02760000 edx=00414141 esi=00000000 edi=01426fe4
eip=7c91081e esp=0012eb64 ebp=0012eb8c iopl=0 nv up ei ng nz na pe cy
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00010287
ntdll!RtlpImageNtHeader+0x35:
7c91081e 813850450000 cmp dword ptr [eax],4550h ds:0023:02b74141=????????
At Crash Point:
0:000> u eip
ntdll!RtlpImageNtHeader+0x35:
7c91081e 813850450000 cmp dword ptr [eax],4550h
7c910824 0f858b830200 jne ntdll!RtlpImageNtHeader+0x3d (7c938bb5)
7c91082a 834dfcff or dword ptr [ebp-4],0FFFFFFFFh
7c91082e e8cfe5ffff call ntdll!_SEH_epilog (7c90ee02)
7c910833 c20400 ret 4
7c910836 90 nop
7c910837 90 nop
7c910838 ff ???
Stack Trace :
0:000> kb
ChildEBP RetAddr Args to Child
0012eb8c 7c91708f 02760000 00000216 0012f3d0 ntdll!RtlpImageNtHeader+0x35
0012ee40 7c916042 02734da8 0012eeb8 00000000 ntdll!LdrpCheckForLoadedDll+0x4cd
0012f0fc 7c9162da 00000000 02734da8 0012f3f0 ntdll!LdrpLoadDll+0x1ba
0012f3a4 7c801bb9 02734da8 0012f3f0 0012f3d0 ntdll!LdrLoadDll+0x230
0012f40c 7c801d6e 7ffdec00 00000000 00000001 kernel32!LoadLibraryExW+0x18e
*** WARNING: Unable to verify checksum for image00400000
*** ERROR: Module load completed but symbols could not be loaded for image00400000
0012f420 00407b8c 017f3ed8 00000000 00000001 kernel32!LoadLibraryExA+0x1f
WARNING: Stack unwind information not available. Following frames may be wrong.
0012f4a4 7c80c710 73eae590 0012f49c 0012f558 image00400000+0x7b8c
0012f4c8 73dd4381 017f3ed8 017f3db8 00000047 kernel32!lstrlenA+0x3b
0012f528 73dd2263 0012f628 00000000 0012f4f0 MFC42!CString::CString+0x47
0012f538 73dd2725 017ef0ac 0012f628 00407846 MFC42!CFixedAlloc::Free+0x28
0012f544 00407846 0012f628 00000000 017ef158 MFC42!CString::~CString+0x1c
00000000 00000000 00000000 00000000 00000000 image00400000+0x7846
Not sure,But I'm guessing its a problem related to heap as CString uses heap allocation.
So please suggest the possible cause for this crash.
Please let me know if more information is needed.
Thanks in Advance,
Parsing binary file does not have valid MZ/PE headers.
Binary base in ecx=02760000 is valid, in edx=00414141 must be an offset to the IMAGE_NT_HEADERS structure in bytes istead of 414141 'AAA'. You may see 00414141 using dd 02760000+3c L1 command. RtlpImageNtHeader adds 414141 to you Base and this is mast be PE signature. Show output from !address 02760000, !dh 02760000, !address 02b74141 I may assume that 02b74141 is not mapped at all.
Our Windows app is often hanging in memory and I'm trying to use windbg to track
down the problem. I'm very new to windbg and could use some advice (I
have started to read Advanced Windows Debugging though).
The app is a mix of C++ and COM objects written in VB. Occasionally when
you exit, the app appears to go away but task manager shows it hanging around
in memory, apparently idle.
!threads shows me this:
ThreadCount: 2
UnstartedThread: 0
BackgroundThread: 2
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
PreEmptive GC Alloc Lock
ID OSID ThreadOBJ State GC Context Domain Count APT Exception
0 1 175c 001aa040 4220 Enabled 09131b78:09131fe8 001a2b80 0 STA
6 2 143c 001b4b48 b220 Enabled 00000000:00000000 001a2b80 0 MTA (Finalizer)
To my untrained eye, it looks like it is being kept alive by the
finalize queue being blocked by a single-threaded apartment. Does this
seem reasonable?
~0kb yields:
ntdll!KiFastSystemCallRet
user32!NtUserGetMessage+0xc
mfc80!AfxInternalPumpMessage+0x18 [f:\sp\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp # 153]
mfc80!CWinThread::Run+0x54 [f:\sp\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp # 625]
mfc80!AfxWinMain+0x69 [f:\sp\vctools\vc7libs\ship\atlmfc\src\mfc\winmain.cpp # 47]
WARNING: Stack unwind information not available. Following frames may be wrong.
OurApp+0x7e8274
kernel32!BaseProcessStart+0x23
~6kb yields:
ntdll!KiFastSystemCallRet
ntdll!ZwWaitForMultipleObjects+0xc
kernel32!WaitForMultipleObjectsEx+0x12c
kernel32!WaitForMultipleObjects+0x18
mscorwks!WKS::WaitForFinalizerEvent+0x7a
mscorwks!WKS::GCHeap::FinalizerThreadWorker+0x75
mscorwks!Thread::UserResumeThread+0xfb
mscorwks!Thread::DoADCallBack+0x355
mscorwks!Thread::DoADCallBack+0x541
mscorwks!ManagedThreadBase_NoADTransition+0x32
mscorwks!ManagedThreadBase::FinalizerBase+0xb
mscorwks!WKS::GCHeap::FinalizerThreadStart+0xbb
mscorwks!Thread::intermediateThreadProc+0x49
kernel32!BaseThreadStart+0x37
I would appreciate a little course correction here. If my guess of a
blocked finalizer seems reasonable, please let me know. I would also be
very happy to get some advice on figuring out what exactly is blocking.
Edit:
Shane asked for the output from !analyze. This is actually from a different dump -- I have lots of them and they all look pretty much the same.
FAULTING_IP:
+18a952f00ebdf74
00000000 ?? ???
EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00000000
ExceptionCode: 80000007 (Wake debugger)
ExceptionFlags: 00000000
NumberParameters: 0
BUGCHECK_STR: 80000007
PROCESS_NAME: OurApp.exe
OVERLAPPED_MODULE: Address regions for 'OurApp' and 'Unknown_Module_00350062' overlap
ERROR_CODE: (NTSTATUS) 0x80000007 - {Kernel Debugger Awakened} the system debugger was awakened by an interrupt.
EXCEPTION_CODE: (HRESULT) 0x80000007 (2147483655) - Operation aborted
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
MANAGED_STACK: !dumpstack -EE
OS Thread Id: 0x4490 (0)
Current frame:
ChildEBP RetAddr Caller,Callee
DERIVED_WAIT_CHAIN:
Dl Eid Cid WaitType
-- --- ------- --------------------------
0 48c8.4490 Speculated (Triage) -->
5 48c8.4b74 Event
WAIT_CHAIN_COMMAND: ~0s;k;;~5s;k;;
BLOCKING_THREAD: 00004b74
DEFAULT_BUCKET_ID: APPLICATION_HANG_BlockedOn_EventHandle
PRIMARY_PROBLEM_CLASS: APPLICATION_HANG_BlockedOn_EventHandle
LAST_CONTROL_TRANSFER: from 7c90df4a to 7c90e514
FAULTING_THREAD: 00000005
STACK_TEXT:
0882fcd0 7c90df4a 7c809590 00000002 0882fcfc ntdll!KiFastSystemCallRet
0882fcd4 7c809590 00000002 0882fcfc 00000001 ntdll!ZwWaitForMultipleObjects+0xc
0882fd70 7c80a115 00000002 7a3b8d28 00000000 kernel32!WaitForMultipleObjectsEx+0x12c
0882fd8c 79f92c5b 00000002 7a3b8d28 00000000 kernel32!WaitForMultipleObjects+0x18
0882fdac 79f970b8 001b1ad8 0882feb0 001a0b18 mscorwks!WKS::WaitForFinalizerEvent+0x77
0882fdc0 79e984cf 0882feb0 00000000 00000000 mscorwks!WKS::GCHeap::FinalizerThreadWorker+0x49
0882fdd4 79e9846b 0882feb0 0882fe5c 79f7762b mscorwks!Thread::DoADCallBack+0x32a
0882fe68 79e98391 0882feb0 9f3f02e2 00000000 mscorwks!Thread::ShouldChangeAbortToUnload+0xe3
0882fea4 79eef74c 0882feb0 00000000 001a86c0 mscorwks!Thread::ShouldChangeAbortToUnload+0x30a
0882fecc 79eef75d 79f9706d 00000008 0882ff14 mscorwks!ManagedThreadBase_NoADTransition+0x32
0882fedc 79f3c6bc 79f9706d 9f3f0352 00000000 mscorwks!ManagedThreadBase::FinalizerBase+0xd
0882ff14 79f920a5 00000000 86fb6620 804fb078 mscorwks!WKS::GCHeap::FinalizerThreadStart+0xbb
0882ffb4 7c80b729 001a0b18 00730074 00610020 mscorwks!Thread::intermediateThreadProc+0x49
0882ffec 00000000 79f9205f 001a0b18 00000000 kernel32!BaseThreadStart+0x37
FOLLOWUP_IP:
mscorwks!WKS::WaitForFinalizerEvent+77
79f92c5b 85c0 test eax,eax
SYMBOL_STACK_INDEX: 4
SYMBOL_NAME: mscorwks!WKS::WaitForFinalizerEvent+77
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: mscorwks
IMAGE_NAME: mscorwks.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 492b82c1
STACK_COMMAND: ~5s ; kb
BUCKET_ID: 80000007_mscorwks!WKS::WaitForFinalizerEvent+77
FAILURE_BUCKET_ID: APPLICATION_HANG_BlockedOn_EventHandle_80000007_mscorwks.dll!WKS::WaitForFinalizerEvent
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/OurApp_exe/6_2_6_1/4a29a184/unknown/0_0_0_0/bbbbbbb4/80000007/00000000.htm?Retriage=1
Followup: MachineOwner
---------
0:000> !threads
ThreadCount: 2
UnstartedThread: 0
BackgroundThread: 2
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
PreEmptive GC Alloc Lock
ID OSID ThreadOBJ State GC Context Domain Count APT Exception
0 1 4490 0019de20 4220 Enabled 09003658:09003fe8 001a86c0 0 STA
5 2 4b74 001b1b08 b220 Enabled 00000000:00000000 001a86c0 0 MTA (Finalizer)
The finalizer thread is idle and is waiting for work -- its trace looks fine. Theread 0 also looks fine and is idle -- it waits for the next UI message.
Can you give some details on how you 'exit' the application? Given that the message loop is still running, it seems to me that something is wrong with your close-application logic.
I agree with J. Passing.
Since one thread is managed code, have you tried loading the SOS debug extension in windbg to get the managed stack trace. Also you could try windbg's "!analyze -v" command ans see what that says.