Unable to delete EXE after it crashes even though process not shown in Task Manager - c++

I have a program that I have written that crashes and I'm in the process of debugging it.
However, the issue is that when I attempt to create a new build, very frequently (but not always!) I get the message:
Cannot open file 'TheExecutable.exe'
I am then unable to delete, rename, move, or in any way modify the executable until the system is rebooted. Attempting to do so in Windows explorer gives
The action can't be completed because the file is open in TheExecutable.exe Close the file and try again.
This behavior isn't unique to the particular crash I'm dealing with right now, nor the particular program. Development is becoming a headache as every attempt to debug will now take several minutes to reboot and bring all my tooling back up.
What, if anything, can I do to prevent Windows from "locking" the executable in such a fashion?
No running process for that executable is visible in Task Manager
Full details of build system:
Windows 10
Intel Compiler, 19.1.0.166 Build 20191121
nmake
C++14

Your process is not being terminated all the way. Since it is not list in the task manager, you can use PSKILL to end it manually.
Open power shell or the console in administrator mode and run
pskill name_of_executable
and it should terminate it so you can re-run it.

Related

When are memory dump files exactly created?

I have configured my windows 7 to create mini dump files on crashes but when my application crashed, no dump file was created. The search for answer left me rather confused as to when are dump files created, when windows crashes or my application crashes?
In my case, I am looking for dump file when my application crashes. I receive a typical crash dialog that states:
TheApp Application has stopped working
Windows can check online for a solution to the problem
-> Check online for a solution and close the program
-> Close the program
-> Debug the program
So can I generate dump file for my application when it crashes? I can't produce this bug on development machine so I want to walk back from dump file. Is there any other option to trace the source of bug (to source code)?
First of all, there are different places to configure a "create a minidump on crash" setting, which are totally different.
You can configure Windows to create a kernel dump file when Windows crashes, i.e. when a Bluescreen of death (BSOD) occurs. This is done in the following screen on Windows 7:
You can configure Windows to create a user mode dump file when an application crashes, i.e. instead of the "Windows Error Reporting" dialog which would normally appear. To do so, and you know that in advance, then configure a Registry key called LocalDumps (MSDN). By default, dumps will be created below %LOCALAPPDATA%\CrashDumps and they will have the naming scheme app.exe.<PID>.dmp.
For the sake of completeness, there might be other triggers. The only sure way to tell is: when the method MiniDumpWriteDump (MSDN) is called.
I'm quite sure that you want option 2 of the above. If you have trouble with it, see whether all the conditions for LocalDump are fulfilled.
The answer given by #antlersoft does not work, for the reasons I have posted in my blog: at the time the dialog is shown, Windows has triggered a breakpoint to stop the application and it has injected a callstack of Windows Error Reporting. All in all, not a good starting point for debugging.
What would work is:
attach a debugger of your choice
press "Go" in the debugger
press the "Debug" button of the WER dialog
confirm the warning about the debugger which is already attached
click "No" when asked to start debugging using the selected debugger
Using Task Manager to create a crash dump is not recommended, since it will not consider the bitness of the application, which may cause trouble later. See ways to create good and useful crash dumps.
Minidump is created when Windows crashes. It's not intended to application crash.
If you want to debug crashes of your application, you may attach it to a debugger after it is started. Clicking on the "Debug" button when application crashes do the same. You can use the debugger of MS Visual Studio to do that, for example.
See this page for help on attaching a process to MS Visual Studio debugger:
https://msdn.microsoft.com/en-us/library/3s68z0b3.aspx
EDIT: following text removed, as this may not work as expected (comment from Thomas)
You can also create a dump file from task manager, however you will still need a debugger to analyze it and, actually I am not sure you will be able to get the dump file at the point application crashes. The best way, if you can, is to debug the process on the target machine by attaching it to debugger either after it is started or when crash occurs.
When you get the crash dialog, go to Task Manager, find the process, right click on the process, and select "Create dump file". The dump file is created in the AppData/Local/Temp folder for the user; it will be named %AppData%\Local\Temp\.DMP; if you create multiple it will be -1.DMP, etc. You can move the dump file to your development machine and open it within Visual Studio. Visual Studio will then act as if you had hit "Break all" at the point of the crash while running the process in the debugger.

VS Command Line "The Process cannot access the file...because it is being used by another process" build fails

I've seen a bunch of other questions that are similar to this one regarding VS, but none in regards to the command line.
Using VS Command line (i.e. BuildConsole "SOLUTIONNAME" /BUILD /cfg="CONFIG_SETTINGS" /prf="PROJECTNAME"), I get the highly reproducable (although not 100%) error shown in the question: "The process cannot access the file because it is being used by another process". However, I only get this error the first 1-5 times I run BuildConsole. It seems to be a timing-related thing. If I run BuildConsole once, then wait a while, then run again, it will work. If I run it 5 times consecutively, it will fail all 5 times or sometimes work on the 5th, depending on how fast the previous runs failed. Also, I can get it to work 100% of the time by opening the solution in VS IDE and then closing the IDE and re-running.
This build command is part of a tool that will report failure if the command fails (which it does, most of the time, although it shouldn't fail). So my question is this: how can I get the BuildConsole command to work the 1st time, 100% of the time? Obviously I don't expect it to work if the solution/project is actually being used by another process, but it should be able to take control from whatever is hanging onto the files.
I had the same problem on windows 7, have moved Visual Studio folder from Documents folder to the root of C:\, changed the location in Tools/Options/Help/Import and Export settings to point to the new location, same with Projects and Solutions all pointing to the new location.
No problems with building at all, unless there is an error with the code.
The problem was not running in administrator mode. When I set my C# program that is running BuildConsole to force admin rights, BuildConsole works every time.

How do I use RestartManager to restart explorer.exe with Windows Installer custom action?

I have an installer that prompts users to restart their computer after an install. I would rather not have the user restart their computer in this case, and have explorer.exe just restart itself using the RestartManager API provided with Windows Vista and up.
I've created a separate executable that gets copied to the local computer during install and runs after that. The separate executable registers explorer.exe, shuts it down, and restarts it based on this code: http://msdn.microsoft.com/en-us/library/aa373681%28v=VS.85%29.aspx. When the executable is run separately from the installer, it works as designed. But when it runs as a custom action as part of an MSI package created with InstalShield, it shuts down explorer.exe but does not restart it.
I always get a 160 error code for RmRestart when it runs with the installer. The docs say it's an error code meaning there were invalid arguments provided. (http://msdn.microsoft.com/en-us/library/aa373665%28v=vs.85%29.aspx). I'm fairly positive that my arguments are not invalid as they work when the executable runs separately from Windows Installer.
I'm stuck at this point and not sure what else to do to get this working. The only thing I'm uncertain of is if "0" can be a proper session handle returned from RmStartSession() with error code of 0 (Success). Assuming this was wrong, I set up my executable to also take in the RmSessionKey that's created by Windows Installer before InstallValidate. And I use that to call my executable as a deferred action. I get an error of 4c3 for RmShutdown in this case, which seems to be an invalid error code.
Cliffs: Have separate .exe that uses RestartManager API to shutdown, restart explorer.exe that works when not run with Windows Installer, but when combined, it breaks. Seeing error code of 160 for RmRestart(). Ran out of ideas to try to get this working. I can provide code snippets if people want...
Thanks for any suggestions/comments.
I ended up reaching a solution to this...
Rather than creating a separate executable that registers explorer.exe and shuts it down, create a MSI DLL Custom Action. All this DLL has to have is a single function that registers explorer.exe to be restarted and use the existing restart manager session provided by Windows Installer (by default). Then in your installer, add the MsiFilesInUse dialog and you'll be good to go.
Now when the installer runs, it starts the restart manager session, and calls your MSI DLL CA, and adds explorer.exe to the list. The list gets displayed and the user is given options to close or defer closing of the applications.
Using this method allows you to avoid having to distribute a pointless executable to the user, as well as simplifies the amount of code written greatly.

Application only runs if you run as administrator?

Edit: This problem only occurs on windows 7 and vista from what I've heard.
I have a very simple app developed with an external graphics library. If I install this app into a program files directory and run it, it will crash immediately but it works fine normally, with exactly the same files. I have realised it is because you need to run the application as administrator for it to work.
I appreciate if this is a problem directly related to the graphics engine I am using, but I don't really think so (but I'm clueless). Can anyone help me?
Edit for more detail:
The application executable and files that are needed to run it are installed into the default program directory - for me, C:\Program Files (x86). If you try and run with without clicking run as administrator, it will simple freeze and say "App has stopped working. Windows is checking for a solution to the problem..." My question is basically, how can I make it so run as administrator isn't necessary?
When a program cannot perform an operation, it (the operation) should fail gracefully. My guess is your application is attempting to do something that it cannot do as a normal user and then fails to check for a return code, and then subsequently crashes. You need to identify what it is your program is doing that it should not be able to do as a normal user. For example (off the top of my head):
Write a file to Program Files (x86)
Write to HKLM
(Without more details) The problem is most likely related to the fact that your program tries to write into the directory and then excepts the file creation/modification to actually have an effect. UAC prevents applications from writing the Program Files directories without administrator privilages. The solution is to redesign your application to not rely on such behavior or store the files in question in one of the intended locations (AppData, etc. folders).
If you right-click on the EXE and go to Properties -> Compatibility there are some options that might help. You could try running the app in compatibility mode for a previous Windows version or if that doesn't work at least mark the EXE to run as administrator by default.

Visual studio release build

I'm trying to generate a release build for a C++ application that I've written. The application runs fine (debug & release) when you run it from within VS2008; but when you run the executable it crashes nearly every single time.
Now, is there a hack so I can run this application as a standalone application without having to run through all of the code and finding the bug that is causing it?
Thanks in advance.
In short, no.
you will have to find the bug, if it works within VS, then I'd hazard a guess that it is a timing issue, possibly you're overwriting shared thread data, this would be less likely (though still possible to see) inside VS as its being run in a debug environment which slows it down a bit.
If you want help finding your bug, then tell us more. Otherwise, build your release with debug symbols (pdbs), install DrWatson as the system debugger and run it standalone. When it crashes DrWatson will create a minidump file, load this into WinDbg (my favourite) and you'll be able to see exactly where your bug is (it'll even tell you that the dump contains an exception and show you it by default. You need to add your source code path and path to your symbols in WinDbg to get it to do this correctly).
Then you will also know how to diagnose crashes when the app is run on-site too.
Are you loading external resources? If you are check that your relative paths are correct in the C++ program.
One possibility is that your program uses uninitialized heap data. Launching a program from the debugger enables the NT debug heap, which causes the heap allocator to fill new memory blocks with a fill pattern, and also enables some heap checking. Launching the same program from outside the debugger leaves the NT debug heap disabled, but if the program was linked against the debug version of the C runtime, then the CRT debug heap will still be enabled.
A much less likely possibility is that your program requires SeDebugPrivilege to be set in its process token. The debugger enables this privilege in its process token, which has the side effect that all programs launched from the debugger inherit this privilege. If your program tries to use OpenProcess()/ReadProcessMemory()/WriteProcessMemory() and doesn't handle errors correctly, it's conceivable that it could crash.
There are a few possibilities. Besides what has already been mentioned, running an app from Visual Studio will execute in the same security context as the Visual Studio instance. So if, for instance, you are working on Vista, you might be hitting an unhandled security violation if you're trying to access protected files, or the registry.
What if you build a debug version and run it standalone? Does it crash? If so, you can usually break into the debugger from there and get a call stack to see what the malfunction is.
From the details you've given, it sounds like there may be a library issue. Are you running the program on the same computer? If not then you'll also have to deploy the appropriate libraries for your application. If you are running on the same computer but outside of the dev environment, ensure that your application can see the appropriate libraries.
Best way i have found to debug in release is to create a crash dump when an crash happens and the dump then allows me to load debug symbols on my dev computer and find out whats going on. More info here: http://www.debuginfo.com/articles/effminidumps.html
You can also go to file => open in Visual Studio and open the .exe, so you are not starting it under the debugger per se. Not sure if it will help.
http://blogs.msdn.com/saraford/archive/2008/08/21/did-you-know-you-can-debug-an-executable-that-isn-t-a-part-of-a-visual-studio-project-without-using-tools-attach-to-process-296.aspx