How can I detect the subsystem used when my program was compiled? - c++

How can I detect whether my Win32 / Win64 program was compiled with /SUBSYSTEM:WINDOWS, or /SUBSYSTEM:CONSOLE?
A preprocessor method would be best if it already exists, but I'd also like to know how to detect it at runtime. It would be sufficient to just detect if there is a console associated with the app, as long as it returns true for console apps.
I'm doing this because I have code to make a console window appear for debugging my GUI apps, but I also build them in console mode (in which case I do not spawn a new console window), and don't really want to make/manage a new #define for this when it seems like there has to already be a way to determine this.

Since in your setting you desire a console in both modes the simplest solution is to call AllocConsole all the time.
A process can be associated with only one console, so the AllocConsole function fails if the calling process already has a console. A process can use the FreeConsole function to detach itself from its current console, then it can call AllocConsole to create a new console or AttachConsole to attach to another console.

Related

How can I output to the parent console window from a Win32 C++ application?

I have a WinAPI/Win32 application. If I try to use cin/cout/cerr when it is run from a command prompt, it doesn't work. I tried switching the project type from Windows Application to Console Application, but the problem is that a console window appears when I run it normally by double-clicking the executable.
So my question is: Is there any way I can use cin/cout/cerr with the parent (calling) console window in a Win32 application? (I only want this behaviour if parameter /c or /? were passed, so if it is called with no arguments then no matter what it should launch the GUI).
A GUI app does not have a console window attached to it by default.
When a GUI app is run from a console process, the GUI app can use AttachConsole() to attach itself to the console.
Or, if the GUI app is not run from a console process, but still wants to use a console window, it can create its own console window using AllocConsole().
Once the GUI app is attached to a console, it can use GetStdHandle() to get handles to the console's STDIN/STDOUT, and then redirect cin/cout to use them (how is dependent on your particular STL implementation).
Or, you can ignore cin/cout and just use ReadConsole() and WriteConsole() directly instead.

AllocConsole Questions

I read the MSDN documentation on AllocConsole, I do not understand one word referring to its purpose. In a very minor DLL tutorial he attaches a DLL to a simple console application and in the DLL, AllocConsole is called. When I remove it from the DLL code, nothing changes during runtime. So I'm curious as to its main purpose:
case DLL_PROCESS_ATTACH:
AllocConsole();
printf("\nInjected Successfully!");
msgBox(true);
break;
This has no difference during runtime as to when I remove AllocConsole.
The MSDN documentation for AllocConsole says:
A process can be associated with only one console, so the AllocConsole
function fails if the calling process already has a console. A process
can use the FreeConsole function to detach itself from its current
console, then it can call AllocConsole to create a new console or
AttachConsole to attach to another console.
So, call it if the process doesn't have a console but you want it to have one. A common example of where you might do this is in a Windows (GUI) application, which does not automatically create and display a console. (Unlike a console application, which does, making AllocConsole rather useless.)

Qt GUI application with console output - hide console on normal startup on Windows

If I open my application an empty console window appears, since I added CONFIG += console to my .pro file. I need the console, because I've implemented a CLI, where some stuff needs to get printed out on the console. On Linux and Mac OSX, I don't actually need the CONFIG += console there. It just works.
How can I prevent opening a windows console, if the .exe gets executed normally over a double click, but display some outputs if my .exe gets started via a console window?
Basically, I use qDebug() << "myText"; and then after that I exit the application with return 0;.
Unfortunately, Windows is somewhat deficient in this area. A console application will always open up a console, even if you don't want it. You can close it right away, but it still looks bad.
Your application must be a non-console application. On startup, check if you have access to a console, as you would when launched from cmd.exe. Then access cmd's console and inject your output into it.
See my question about this for details.
It is a GUI application? AFAIK it is not possible (or at least not trivial) writing a mixed Qt application that can act as both, a desktop (GUI) application and a console (CLI) application.
I am not sure what you intend to do. If you really need a console variant, try to build two different applications based on the same sources (one console build and one GUI build).
If you only need a GUI application that is able to print out some information, remove the console code and write the output into a file instead.
I think here is an answer for you question: https://stackoverflow.com/a/3370017/1091536
You need console application, than launch your GUI application and prints it's output.
The application for launch your GUI application you can find here https://github.com/gomons/AppDebugLauncher
It's worth noting that under some circumstances the console window won't briefly appear. For example if you run in gui mode via a shortcut with Run set to Minimized: the console window won't appear. Then, in your code, you can restore the size of the gui window. Its a bit of a nasty workaround but masks the behaviour from the user that bit more.
If you're program is going to be installed and usually started via shortcut, then its maybe its an option.

Win32 application with console output and without new window

I'd like to create a tool that can either act as command line (display some console output based on input parameters), or display a window, based on input parameters.
I'm using MSV2012 with C++, and it seems you have to 'choose' between console and window app.
I know the net is filled with samples which use AllocConsole() and redirect std::out, but it doesn't make it feel like a command line application : calling the exe from the windows console will open a new window with the console output...
Is there a way to have it use the current console window instead of allocating a new one?
If not possible, I'll make 2 applications instead, but that's a pity..
Someone else may have a more authoritative answer, but I don't believe it's supported.
The usual workaround is to create a Windows app, but have a command-line wrapper that launches it from the CLI (and provides a channel for communicating with the original console).
It's not technically supported but I found a good solution by getting a snapshot for the current process, finding the parent process, attaching to it's console if it's a console app or creating one with AllocConsole, redirecting the output, getting the thread of the parent process if it's cmd.exe and suspending it, resuming it just before I exit my app

Console output window in DLL

I am trying to redirect the output from my DLL to an external console window for easy debugging.
I have been told about AllocConsole but I am not able to reproduce it, i.e. the console window does not appear.
My current environment is Visual Studio 2005.
I tried the following example which is gotten off the Internet,
AllocConsole();
HANDLE han = GetStdHandle(STD_OUTPUT_HANDLE);
WriteConsole(han,"hello",6,new DWORD,0);
yet nothing happens. Can someone point me in the right direction if creating a console window via DLL is possible in the first place.
Thanks in advance!
The proper way to output debug strings is via OutputDebugString(), with an appropriate debugging tool listening for output strings.
Once loaded, there is nothing special about DLLs, so there is no way that allocating consoles would be any different for a DLL than for the EXE that originally loaded it.
Having said that, a process can be associated with only one console at a time, so if there is already a console attached to the process, then allocating a new one is not going to do anything (I assume you're checking the return value of AllocConsole? What does it return? What does GetLastError return?)
There are some other possibilities. For example, if your DLL is loaded into a service, then the service will (likely) be running under a different window station to the currently logged-in user so if you create a console window, you won't be able to see it.