I have a C++ library, which is used in both Linux and Windows.
I want to enable the user to control the debug level (0 - no debug, 1 - only critical errors ... 5 - informative debug information).
The debug log is printed to a text file.
In Windows, I can do it using a registry value (DWORD DebugLevel).
What can be a good replacement which works also for Linux?
(Without 3rd party tools, for example Linux "registry").
Thanks in advance!
Does your library have some sort of initialisation function? Make the level a parameter to that function. Ideally store the passed-in value in a context structure or class if it makes sense for your API (i.e. if you require clients to always operate via a "context") - but if not, a global might be reasonable.
If it's largely for development purposes (ie the "user" you refer to is a developer using your library, not the end user of that code), the quickest/easiest way is to use an environment variable.
If it's to be controlled by the end-user, you probably need to extend your API so that the app developer can set the debug level in code, after reading his configuration files -- you wouldn't normally have a separate config file for just one library used by a program.
You can use the log4cxx framework. This is configurable through a file. I haven't tried it yet, but it should work with Windows too.
You could use a configuration file in /etc/YOURAPP or ~/.YOURAPP or ~/.config/YOURAPP
Related
Question concerns only Windows for now - other OS's are not so relevant right now.
Just by quick googling - it's possible to load native .dll from RAM, there are for example following libraries:
https://www.joachim-bauch.de/tutorials/loading-a-dll-from-memory/
=>
https://github.com/fancycode/MemoryModule
https://forum.nim-lang.org/t/7943
But all of them requires:
in-depth knowledge of PE file format
mostly those approaches are not debugger friendly.
What I have checked - windows's LoadLibraryA / LoadLibraryW are directed to ntdll.dll / LdrLoadDll - and best picture of how things works can be found from here: https://github.com/hlldz/RefleXXion
And even thus I don't have windows source code - I've checked same functionality from Wine:
LdrLoadDll: https://source.winehq.org/source/dlls/ntdll/loader.c#3169
load_dll: https://source.winehq.org/source/dlls/ntdll/loader.c#3083
load_native_dll:
https://source.winehq.org/source/dlls/ntdll/loader.c#2564
NtMapViewOfSection: https://source.winehq.org/source/dlls/ntdll/unix/virtual.c#4469
find_dll_file: https://source.winehq.org/source/dlls/ntdll/loader.c#3021
open_dll_file: https://source.winehq.org/source/dlls/ntdll/loader.c#2467
Suspect loading dll happens via following function calls:
NtOpenFile, NtQueryAttributesFile, NtCreateSection/NtOpenSection, NtMapViewOfSection (*)
(More information could be found in
https://github.com/Hagrid29/PELoader
https://gist.github.com/bats3c/59932dfa1f5bb23dd36071119b91af0f
https://www.octawian.ro/fisiere/situri/asor/build/html/_downloads/122f95f9a032396603a837c53b125bb8/Russinovich_M_WinInternals_part1_7th_ed.pdf
)
I was also thinking if I could just override NtOpenFile and just redirect file open (in
https://github.com/SegaraRai/PathRedirector manner)
to different path - but main question what is the alternative location where to store file?
I was thinking if NtOpenFile can open even device, then maybe just replace file
with some sort of named pipe (https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipe-client) - but then in maps on how well this will work with NtMapViewOfSection.
Since I was not able to find any working example of such hook or operation (E.g. LoadLibary("\\.\pipe\mynamedpipe_as_dll")) - there is always a risk that such combination is not simply supported.
Is it possible to load native .dll purely from RAM:
Without using file system (not to store .dll e.g. in temporary folder)
Without involving custom drivers (like Dokan) ?
So loaded .dll would be still debugger friendly ?
Not tightly bound to PE file format structures (or use PE structures as less as possible)
If you miss bit more information, check also my own experiments with native dll loading (maybe can give some hints on solving the issue):
https://github.com/tapika/test_native_dll_loading
https://github.com/tapika/test_native_dll_loading/discussions/2
Distinguish between debug and release use cases. In debug, save the DLL in a temp file and load with LoadLibrary, which will enable debugging. In release, run from memory with no capability for debugging.
Here's another idea, from considering the linked Guthub issue. If the purpose is to let the users provide their own compression/decompression logic while building a ReadyToRun executable, let them provide that as a static library (object) as opposed to a DLL. The larger project is already about packaging stuff into a single executable, might do some linking while at it.
Yet another idea would be to let the users provide the codec in some kind of interpreted language and optionally plug in the interpreter that supports debugging. Windows comes with a built-in JavaScript interpreter, look up Active Scripting, and debugging those is a free bonus. The performance probably won't be on par with a native code implementation, though.
I think you could probably do something similar with Frida. Hook the functions LoadLibraryA / LoadLibraryW and reimplement them in Frida. but I don't believe this is something that would be stable for production.
For some reference
By analyzing existing approaches (like PE Loader https://github.com/Hagrid29/PELoader) and using minhook library - I've managed to load .dll from RAM.
I've created git repository with example code on github:
https://github.com/tapika/dllloader
Could you create a ramdisk to put your DLL there? What exactly is the use-case for this? There are a couple ways to spin up a file in RAM, C#'s MemoryMappedFile for example. I'm not sure if this would be debugger friendly.
I am making a suite of 64-bit plugin DLLs for a Windows host application using Visual Studio/C++, and from the current version onward, the setup.exe that they come in creates a single shared user-writable folder under ProgramData in which I cache all sorts of (non-user specific) data files. Older versions didn't have that folder yet.
However, the distribution of my plugin binaries is often out of my hands too. They are repackaged by a 3rd party bundle which can only do dumb file copies of the DLLs (so no real setup.exe functionality I need like creating folder + set permissions). And since my binary DLLs are all 100% self-contained, users also historically have a hand of just copying the DLLs around to other machines as they see fit, but that ofc also lacks the new folder setup phase.
I am looking into a workaround to have my DLLs create the folder at runtime if it is missing. I know I can't elevate the host process in-place whenever I want, but I thought of the following ways:
Have an extra "FixSetup" entry point in my DLL, and when the need arises, start an elevated RunDLL32.exe and let it use this entry point in my DLL.However, I see all sorts of people all over the place talking about RunDLL being as good as deprecated and advising against using it, but then again that was already since Windows XP and it's still with us. I also hear of RunDLL having it's own runtime context which can change with every Windows release (like switching to high-DPI aware when that came available), and that it thus is a 'hostile' environment to run in (read it on Raymond Chan's blog IIRC). Should I really be afraid of using it, or is my use case so simple it can barely break? (no GUI, just a wrapped CreateDirectory call)
Create a small "FixSetup.exe" which just does the folder creation, package it into my DLL's resources, and extract-to-temp + run-elevated it at runtime.While this would bloat my DLLs (depending on how small I can get the .exe), I feel like it's also a more fragile + convoluted solution than 1. above (with file extraction and all; prob. best to sign the utility exe too to keep HIPS / antivirus from acting funny etc?).
Alter my DLLs so that they're actually .exes in disguise which happen to export the host-expected DLL entry points, so that I can call them directly (elevated).I know there are some major caveats here (like conflicts between the C runtime being included in DLL or non-DLL mode, Visual Studio prob. not approving of these shenanigans, etc.), and honestly I already feel I need a shower just after talking about this one. So while theoretically maybe feasible, it is my last resort.
Does anyone have any advise on my uncertainties above? Or maybe an even better suggestion?
EDIT
I've already managed to get option 1. working, and while it works seamlessly there's one drawback I spotted: the UAC prompt (understandably) asks whether the user wants to run RunDLL32.exe, signed by Microsoft. This might confuse/scare people no end (that is: if they even read these prompts...). I'd rather have the UAC prompt asking about MyPluginSetup.exe signed by MyCompany, so now I'm more inclined to go with option 2. instead.
We have several Windows 8 Store C++ apps that need to maintain configuration and data files.
Files are written in subfolders of Windows::Storage::ApplicationData::Current->LocalFolder. Example:
C:\Users\<username>\AppData\Local\Packages\<packagename>\LocalState\SubFolder1\SubFolder2\data.txt
In Windows 8.1 we have received a few reports from users that say state isn't remembered between app invocations. Upon closer inspection the files are not created (the subfolders are indeed created, but there are no files inside them)
Notes:
Subfolders are created using CreateDirectory(), files are created using fopen()
Files are created/opened using absolute paths
This always worked under Windows 8.0 and the code has not been changed since. In fact, one of our user reports stated that the app saved files fine under Windows 8.0, but stopped saving after the user upgraded to Windows 8.1.
We have not been able to replicate the issue locally using Windows 8.1. We're not sure how common this failure is, but we estimate that most users are unaffected. Affected users do not appear to have any special hardware/software configuration.
If a user is affected, then files are consistently never saved, even after retrying or uninstalling and re-installing the app (i.e., it's not a case of intermittent failure)
It's hard to get error information given (i) the rarity of the issue (ii) the fact that the logs that would reveal this are by definition not saved, and (iii) the apps don't require internet connectivity so there is no alternative communication channel.
Can anyone think of any reason why this might fail under Windows 8.1?
Are there non-ascii characters in path to appdata? CreateDirectory has unicode version, but fopen takes const char* strings as argument.
If I were you, I'd try to abstract away from OS-specific calls using something like boost or Qt. That should work, because Qt uses unicode string for opening files and Boost should have something similar (unsure about this one).
Also on windows compiler _wfopen may be present. It is the same as fopen, but takes wchar_t strings as argument. It should work for you, but you'll need a few ifdefs here and there.
You could also try setting current directory with function that supports unicode and then calling fopen, but I wouldn't call it a "clean" solution.
Anyway, when you run into problem that is related to system calls, then on machine with a problem you can monitor calls using something like process monitor. You could instruct user with a problem to do that and send you a log.
Why don't you use the Windows.Storage classes to work with filesystem? WinRT is recommended way to work with IO not legacy C API. I believe it is more robust approach and you could get more info about the cause from WinRT exception rather than from an unknown failure of old API.
I have a newbie but really important question for me: I have a Mac Os X application that uses carbon api, but it is still a C++ application. I need to debug which functions are called at execution time and then make a C++ patch to replace one of those functions.
The real goal: I need to log all text printed into a chat window that the application has inside an unnacessible carbon view. I thought at first it was a cocoa application, but it's not, so fscript and imlib are no good to inject code.
Is it possible? Any clues? Thank you very much.
Cheers :)
You could look into using truss to figure out what system calls are being made but I'm not sure for user-calls. The LD_PRELOAD environment variable can allow you to inject methods into other apps, but C++ methods tend to have various dependencies regarding name mangling and calling method so it would probably be tricky to plug in your own.
Can you just have the app maintainer add actual hooks to allow for what you need?
I've encountered the following problem pattern frequently over the years:
I'm writing complex code for a package comprised of a standalone application and also a library version of the core that people can use from inside other apps.
Both our own app and presumably ones that users create with the core library are likely to be run both in batch mode (off-line, scripted, remote, and/or from command line), as well as interactively.
The library/app takes complex and large runtime input and there may be a variety of error-like outputs including severe error messages, input syntax warnings, status messages, and run statistics. Note that these are all incidental outputs, not the primary purpose of the application which would be displayed or saved elsewhere and using different methods.
Some of these (probably only the very severe ones) might require a dialog box if run interactively; but it needs to log without stalling for user input if run in batch mode; and if run as a library the client program obviously wants to intercept and/or examine the errors as they occur.
It all needs to be cross-platform: Linux, Windows, OSX. And we want the solution to not be weird on any platform. For example, output to stderr is fine for Linux, but won't work on Windows when linked to a GUI app.
Client programs of the library may create multiple instances of the main class, and it would be nice if the client app could distinguish a separate error stream with each instance.
Let's assume everybody agrees it's good enough for the library methods to log errors via a simple call (error code and/or severity, then printf-like arguments giving an error message). The contentious part is how this is recorded or retrieved by the client app.
I've done this many times over the years, and am never fully satisfied with the solution. Furthermore, it's the kind of subproblem that's actually not very important to users (they want to see the error log if something goes wrong, but they don't really care about our technique for implementing it), but the topic gets the programmers fired up and they invariably waste inordinate time on this detail and are never quite happy.
Anybody have any wisdom for how to integrate this functionality into a C++ API, or is there an accepted paradigm or a good open source solution (not GPL, please, I'd like a solution I can use in commercial closed apps as well as OSS projects)?
We use Apache's Log4cxx for logging which isn't perfect, but provides a lot of infrastructure and a consistent approach across projects. I believe it is cross-platform, though we only use it on Windows.
It provides for run time configuration via an ini file which allows you to control how the log file is output, and you could write your own appenders if you want specific behaviour (e.g. an error dialog under the UI).
If clients of your library also adopt it then it would integrate their logging output into the same log file(s).
Differentiation between instances of the main class could be supported using the nested diagnostic context (NDC) feature.
Log4Cxx should work for you. You need to implement a provider that allows the library user to catch the log output in callbacks. The library would export a function to install the callbacks. That function should, behind the scenes, reconfigure log4cxxx to get rid of all appenders and set up the "custom" appender.
Of course, the library user has an option to not install the callbacks and use log4cxx as is.